2024-05-17T21:31:55.337 INFO:root:teuthology version: 0.0.1.dev276+g2127e87 2024-05-17T21:31:55.337 DEBUG:teuthology.run:Teuthology command: teuthology --name pdonnell-2024-05-17_18:30:44-fs-wip-pdonnell-testing-20240517.012458-debug-distro-default-smithi --description fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} --archive /home/teuthworker/archive/pdonnell-2024-05-17_18:30:44-fs-wip-pdonnell-testing-20240517.012458-debug-distro-default-smithi/7711745 --verbose --owner scheduled_pdonnell@teuthology -- /home/teuthworker/archive/pdonnell-2024-05-17_18:30:44-fs-wip-pdonnell-testing-20240517.012458-debug-distro-default-smithi/7711745/orig.config.yaml 2024-05-17T21:31:55.390 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-17T21:31:55.484 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-05-17_18:30:44-fs-wip-pdonnell-testing-20240517.012458-debug-distro-default-smithi/7711745 branch: wip-pdonnell-testing-20240517.012458-debug description: fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} email: pdonnell@redhat.com first_in_suite: false job_id: '7711745' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi meta: - desc: 'setup ceph/quincy ' name: pdonnell-2024-05-17_18:30:44-fs-wip-pdonnell-testing-20240517.012458-debug-distro-default-smithi no_nested_subset: false nuke-on-error: true os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-pdonnell-testing-20240517.012458-debug ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: mon pg warn min per osd: 0 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - OSD_DOWN - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy sha1: 931809a9e5d295bd61984693cac44fe131ded4bc ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 931809a9e5d295bd61984693cac44fe131ded4bc kclient: syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20240517.012458-debug sha1: 931809a9e5d295bd61984693cac44fe131ded4bc owner: scheduled_pdonnell@teuthology priority: 75 repo: https://git.ceph.com/ceph-ci.git roles: - - host.a - client.0 - osd.0 - osd.1 - osd.2 - - host.b - client.1 - osd.3 - osd.4 - osd.5 seed: 1967 sha1: 931809a9e5d295bd61984693cac44fe131ded4bc sleep_before_teardown: 0 subset: 59/512 suite: fs suite_branch: wip-pdonnell-testing-20240517.012458-debug suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_931809a9e5d295bd61984693cac44fe131ded4bc/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 931809a9e5d295bd61984693cac44fe131ded4bc targets: smithi107.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOVhqHUkeu3/sHQtyZnwHZdpUVdm2yZQUf5nrfLplOhc/6KlznKMYju/zfiin0dn087PMxpU5UNDYGi53eltz7o= smithi138.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDQDUV+Fh0J59rIGzIc3/ViaBBlSxATYaJ0MwefATjYN1bmlfC7K7qh5ZbWfWxUYH70EtBKj1oWup8Z2BabnPMs= tasks: - install: branch: quincy exclude_packages: - ceph-volume - print: '**** done install task...' - cephadm: cephadm_branch: quincy cephadm_git_url: https://github.com/ceph/ceph conf: osd: osd_class_default_list: '*' osd_class_load_list: '*' image: quay.ceph.io/ceph-ci/ceph:quincy roleless: true - print: '**** done end installing quincy cephadm ...' - cephadm.shell: host.a: - ceph config set mgr mgr/cephadm/use_repo_digest true --force - print: '**** done cephadm.shell ceph config set mgr...' - cephadm.shell: host.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: host.a: - ceph fs volume create cephfs --placement=4 - ceph fs dump - cephadm.shell: host.a: - ceph fs set cephfs max_mds 1 - cephadm.shell: host.a: - ceph fs set cephfs allow_standby_replay true - cephadm.shell: host.a: - ceph fs set cephfs inline_data true --yes-i-really-really-mean-it - cephadm.shell: host.a: - ceph fs dump - ceph --format=json fs dump | jq -e ".filesystems | length == 1" - while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done - fs.pre_upgrade_save: null - ceph-fuse: null - print: '**** done client' - parallel: - upgrade-tasks - workload-tasks - cephadm.shell: host.a: - ceph fs dump - fs.post_upgrade_checks: null teuthology: fragments_dropped: - /cephfs/home/pdonnell/src/git.ceph.com_ceph-c_wip-pdonnell-testing-20240517.012458-debug/qa/suites/fs/upgrade/mds_upgrade_sequence/tasks/3-upgrade-mgr-staggered.yaml meta: {} postmerge: [] variables: fail_fs: false teuthology_branch: install-fix teuthology_sha1: 2127e87e3419439de550d2abbd7ed2aee2cdfb0e timestamp: 2024-05-17_18:30:44 tube: smithi upgrade-tasks: sequential: - cephadm.shell: env: - sha1 host.a: - ceph config set mgr mgr/orchestrator/fail_fs false || true - cephadm.shell: env: - sha1 host.a: - ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force - ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 - cephadm.shell: env: - sha1 host.a: - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done - ceph orch ps - ceph orch upgrade status - ceph health detail - ceph versions - echo "wait for servicemap items w/ changing names to refresh" - sleep 60 - ceph orch ps - ceph versions - ceph versions | jq -e '.overall | length == 1' - ceph versions | jq -e '.overall | keys' | grep $sha1 user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2698841 workload-tasks: sequential: - workunit: clients: all: - suites/fsstress.sh 2024-05-17T21:31:55.485 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_931809a9e5d295bd61984693cac44fe131ded4bc/qa; will attempt to use it 2024-05-17T21:31:55.486 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_931809a9e5d295bd61984693cac44fe131ded4bc/qa/tasks 2024-05-17T21:31:55.486 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-17T21:31:55.487 INFO:teuthology.task.internal:Checking packages... 2024-05-17T21:31:55.508 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '931809a9e5d295bd61984693cac44fe131ded4bc' 2024-05-17T21:31:55.508 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-17T21:31:55.508 INFO:teuthology.packaging:ref: None 2024-05-17T21:31:55.508 INFO:teuthology.packaging:tag: None 2024-05-17T21:31:55.508 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20240517.012458-debug 2024-05-17T21:31:55.508 INFO:teuthology.packaging:sha1: 931809a9e5d295bd61984693cac44fe131ded4bc 2024-05-17T21:31:55.509 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-pdonnell-testing-20240517.012458-debug 2024-05-17T21:31:55.650 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3794.g931809a9 2024-05-17T21:31:55.652 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-17T21:31:55.657 INFO:teuthology.task.internal:no buildpackages task found 2024-05-17T21:31:55.658 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-17T21:31:55.666 INFO:teuthology.task.internal:Saving configuration 2024-05-17T21:31:55.685 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-17T21:31:55.690 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-17T21:31:55.712 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi107.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-17_18:30:44-fs-wip-pdonnell-testing-20240517.012458-debug-distro-default-smithi/7711745', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-17 21:26:56.860779', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOVhqHUkeu3/sHQtyZnwHZdpUVdm2yZQUf5nrfLplOhc/6KlznKMYju/zfiin0dn087PMxpU5UNDYGi53eltz7o='} 2024-05-17T21:31:55.729 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi138.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-17_18:30:44-fs-wip-pdonnell-testing-20240517.012458-debug-distro-default-smithi/7711745', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-17 21:26:56.858719', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDQDUV+Fh0J59rIGzIc3/ViaBBlSxATYaJ0MwefATjYN1bmlfC7K7qh5ZbWfWxUYH70EtBKj1oWup8Z2BabnPMs='} 2024-05-17T21:31:55.729 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-17T21:31:55.738 INFO:teuthology.task.internal:roles: ubuntu@smithi107.front.sepia.ceph.com - ['host.a', 'client.0', 'osd.0', 'osd.1', 'osd.2'] 2024-05-17T21:31:55.738 INFO:teuthology.task.internal:roles: ubuntu@smithi138.front.sepia.ceph.com - ['host.b', 'client.1', 'osd.3', 'osd.4', 'osd.5'] 2024-05-17T21:31:55.738 INFO:teuthology.run_tasks:Running task console_log... 2024-05-17T21:31:55.871 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f6d850abac0>, signals=[15]) 2024-05-17T21:31:55.871 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-17T21:31:55.889 INFO:teuthology.task.internal:Opening connections... 2024-05-17T21:31:55.889 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi107.front.sepia.ceph.com 2024-05-17T21:31:55.890 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-17T21:31:55.969 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi138.front.sepia.ceph.com 2024-05-17T21:31:55.969 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-17T21:31:56.046 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-17T21:31:56.052 DEBUG:teuthology.orchestra.run.smithi107:> uname -m 2024-05-17T21:31:56.070 INFO:teuthology.orchestra.run.smithi107.stdout:x86_64 2024-05-17T21:31:56.070 DEBUG:teuthology.orchestra.run.smithi107:> cat /etc/os-release 2024-05-17T21:31:56.127 INFO:teuthology.orchestra.run.smithi107.stdout:NAME="CentOS Stream" 2024-05-17T21:31:56.128 INFO:teuthology.orchestra.run.smithi107.stdout:VERSION="9" 2024-05-17T21:31:56.128 INFO:teuthology.orchestra.run.smithi107.stdout:ID="centos" 2024-05-17T21:31:56.128 INFO:teuthology.orchestra.run.smithi107.stdout:ID_LIKE="rhel fedora" 2024-05-17T21:31:56.128 INFO:teuthology.orchestra.run.smithi107.stdout:VERSION_ID="9" 2024-05-17T21:31:56.128 INFO:teuthology.orchestra.run.smithi107.stdout:PLATFORM_ID="platform:el9" 2024-05-17T21:31:56.128 INFO:teuthology.orchestra.run.smithi107.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-17T21:31:56.128 INFO:teuthology.orchestra.run.smithi107.stdout:ANSI_COLOR="0;31" 2024-05-17T21:31:56.128 INFO:teuthology.orchestra.run.smithi107.stdout:LOGO="fedora-logo-icon" 2024-05-17T21:31:56.128 INFO:teuthology.orchestra.run.smithi107.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-17T21:31:56.128 INFO:teuthology.orchestra.run.smithi107.stdout:HOME_URL="https://centos.org/" 2024-05-17T21:31:56.128 INFO:teuthology.orchestra.run.smithi107.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-17T21:31:56.128 INFO:teuthology.orchestra.run.smithi107.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-17T21:31:56.128 INFO:teuthology.orchestra.run.smithi107.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-17T21:31:56.129 INFO:teuthology.lock.ops:Updating smithi107.front.sepia.ceph.com on lock server 2024-05-17T21:31:56.155 DEBUG:teuthology.orchestra.run.smithi138:> uname -m 2024-05-17T21:31:56.172 INFO:teuthology.orchestra.run.smithi138.stdout:x86_64 2024-05-17T21:31:56.172 DEBUG:teuthology.orchestra.run.smithi138:> cat /etc/os-release 2024-05-17T21:31:56.230 INFO:teuthology.orchestra.run.smithi138.stdout:NAME="CentOS Stream" 2024-05-17T21:31:56.230 INFO:teuthology.orchestra.run.smithi138.stdout:VERSION="9" 2024-05-17T21:31:56.231 INFO:teuthology.orchestra.run.smithi138.stdout:ID="centos" 2024-05-17T21:31:56.231 INFO:teuthology.orchestra.run.smithi138.stdout:ID_LIKE="rhel fedora" 2024-05-17T21:31:56.231 INFO:teuthology.orchestra.run.smithi138.stdout:VERSION_ID="9" 2024-05-17T21:31:56.231 INFO:teuthology.orchestra.run.smithi138.stdout:PLATFORM_ID="platform:el9" 2024-05-17T21:31:56.231 INFO:teuthology.orchestra.run.smithi138.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-17T21:31:56.231 INFO:teuthology.orchestra.run.smithi138.stdout:ANSI_COLOR="0;31" 2024-05-17T21:31:56.231 INFO:teuthology.orchestra.run.smithi138.stdout:LOGO="fedora-logo-icon" 2024-05-17T21:31:56.231 INFO:teuthology.orchestra.run.smithi138.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-17T21:31:56.231 INFO:teuthology.orchestra.run.smithi138.stdout:HOME_URL="https://centos.org/" 2024-05-17T21:31:56.231 INFO:teuthology.orchestra.run.smithi138.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-17T21:31:56.231 INFO:teuthology.orchestra.run.smithi138.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-17T21:31:56.232 INFO:teuthology.orchestra.run.smithi138.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-17T21:31:56.232 INFO:teuthology.lock.ops:Updating smithi138.front.sepia.ceph.com on lock server 2024-05-17T21:31:56.246 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-17T21:31:56.255 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-17T21:31:56.262 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-17T21:31:56.262 DEBUG:teuthology.orchestra.run.smithi107:> test '!' -e /home/ubuntu/cephtest 2024-05-17T21:31:56.264 DEBUG:teuthology.orchestra.run.smithi138:> test '!' -e /home/ubuntu/cephtest 2024-05-17T21:31:56.287 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-17T21:31:56.295 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-17T21:31:56.295 DEBUG:teuthology.orchestra.run.smithi107:> test -z $(ls -A /var/lib/ceph) 2024-05-17T21:31:56.323 DEBUG:teuthology.orchestra.run.smithi138:> test -z $(ls -A /var/lib/ceph) 2024-05-17T21:31:56.347 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-17T21:31:56.402 INFO:teuthology.run_tasks:Running task kernel... 2024-05-17T21:31:56.420 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-05-17T21:31:56.421 INFO:teuthology.task.kernel:config {'host.a': {'kdb': 1, 'sha1': 'distro'}, 'host.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-05-17T21:31:56.421 DEBUG:teuthology.orchestra.run.smithi107:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-17T21:31:56.422 DEBUG:teuthology.orchestra.run.smithi138:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-17T21:31:56.438 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-17T21:31:56.438 DEBUG:teuthology.orchestra.run.smithi138:> uname -r 2024-05-17T21:31:56.439 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-17T21:31:56.439 DEBUG:teuthology.orchestra.run.smithi107:> uname -r 2024-05-17T21:31:56.495 INFO:teuthology.orchestra.run.smithi138.stdout:5.14.0-437.el9.x86_64 2024-05-17T21:31:56.495 INFO:teuthology.task.kernel:Running kernel on smithi138: 5.14.0-437.el9.x86_64 2024-05-17T21:31:56.495 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum install -y kernel 2024-05-17T21:31:56.496 INFO:teuthology.orchestra.run.smithi107.stdout:5.14.0-437.el9.x86_64 2024-05-17T21:31:56.496 INFO:teuthology.task.kernel:Running kernel on smithi107: 5.14.0-437.el9.x86_64 2024-05-17T21:31:56.496 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum install -y kernel 2024-05-17T21:31:58.845 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.1 MB 00:00 2024-05-17T21:31:59.110 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - BaseOS 8.0 MB/s | 8.1 MB 00:01 2024-05-17T21:32:00.612 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - AppStream 28 MB/s | 19 MB 00:00 2024-05-17T21:32:03.461 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - AppStream 5.8 MB/s | 19 MB 00:03 2024-05-17T21:32:04.804 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.2 MB 00:00 2024-05-17T21:32:06.389 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - Extras packages 63 kB/s | 16 kB 00:00 2024-05-17T21:32:08.392 INFO:teuthology.orchestra.run.smithi107.stdout:Extra Packages for Enterprise Linux 11 MB/s | 21 MB 00:01 2024-05-17T21:32:09.186 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - CRB 2.9 MB/s | 6.2 MB 00:02 2024-05-17T21:32:10.779 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - Extras packages 67 kB/s | 16 kB 00:00 2024-05-17T21:32:12.658 INFO:teuthology.orchestra.run.smithi138.stdout:Extra Packages for Enterprise Linux 12 MB/s | 21 MB 00:01 2024-05-17T21:32:13.823 INFO:teuthology.orchestra.run.smithi107.stdout:lab-extras 5.6 kB/s | 1.7 kB 00:00 2024-05-17T21:32:15.786 INFO:teuthology.orchestra.run.smithi107.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-17T21:32:15.786 INFO:teuthology.orchestra.run.smithi107.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-17T21:32:15.877 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-05-17T21:32:15.885 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-05-17T21:32:15.885 INFO:teuthology.orchestra.run.smithi107.stdout: Package Architecture Version Repository Size 2024-05-17T21:32:15.885 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-05-17T21:32:15.886 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-05-17T21:32:15.886 INFO:teuthology.orchestra.run.smithi107.stdout: kernel x86_64 5.14.0-446.el9 baseos 550 k 2024-05-17T21:32:15.886 INFO:teuthology.orchestra.run.smithi107.stdout:Installing dependencies: 2024-05-17T21:32:15.886 INFO:teuthology.orchestra.run.smithi107.stdout: kernel-core x86_64 5.14.0-446.el9 baseos 15 M 2024-05-17T21:32:15.886 INFO:teuthology.orchestra.run.smithi107.stdout: kernel-modules x86_64 5.14.0-446.el9 baseos 34 M 2024-05-17T21:32:15.886 INFO:teuthology.orchestra.run.smithi107.stdout: kernel-modules-core x86_64 5.14.0-446.el9 baseos 28 M 2024-05-17T21:32:15.886 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:32:15.886 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-05-17T21:32:15.886 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-05-17T21:32:15.886 INFO:teuthology.orchestra.run.smithi107.stdout:Install 4 Packages 2024-05-17T21:32:15.886 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:32:15.887 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 78 M 2024-05-17T21:32:15.887 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 123 M 2024-05-17T21:32:15.887 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-05-17T21:32:16.140 INFO:teuthology.orchestra.run.smithi107.stdout:(1/4): kernel-5.14.0-446.el9.x86_64.rpm 2.8 MB/s | 550 kB 00:00 2024-05-17T21:32:17.515 INFO:teuthology.orchestra.run.smithi107.stdout:(2/4): kernel-core-5.14.0-446.el9.x86_64.rpm 9.9 MB/s | 15 MB 00:01 2024-05-17T21:32:17.830 INFO:teuthology.orchestra.run.smithi138.stdout:lab-extras 17 kB/s | 1.7 kB 00:00 2024-05-17T21:32:17.974 INFO:teuthology.orchestra.run.smithi107.stdout:(3/4): kernel-modules-core-5.14.0-446.el9.x86_6 15 MB/s | 28 MB 00:01 2024-05-17T21:32:18.766 INFO:teuthology.orchestra.run.smithi107.stdout:(4/4): kernel-modules-5.14.0-446.el9.x86_64.rpm 12 MB/s | 34 MB 00:02 2024-05-17T21:32:18.766 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-05-17T21:32:18.766 INFO:teuthology.orchestra.run.smithi107.stdout:Total 27 MB/s | 78 MB 00:02 2024-05-17T21:32:19.448 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-05-17T21:32:19.778 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-05-17T21:32:19.778 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-05-17T21:32:19.811 INFO:teuthology.orchestra.run.smithi138.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-17T21:32:19.811 INFO:teuthology.orchestra.run.smithi138.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-17T21:32:19.892 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-05-17T21:32:19.911 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-05-17T21:32:19.911 INFO:teuthology.orchestra.run.smithi138.stdout: Package Architecture Version Repository Size 2024-05-17T21:32:19.911 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-05-17T21:32:19.911 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-05-17T21:32:19.911 INFO:teuthology.orchestra.run.smithi138.stdout: kernel x86_64 5.14.0-446.el9 baseos 550 k 2024-05-17T21:32:19.911 INFO:teuthology.orchestra.run.smithi138.stdout:Installing dependencies: 2024-05-17T21:32:19.912 INFO:teuthology.orchestra.run.smithi138.stdout: kernel-core x86_64 5.14.0-446.el9 baseos 15 M 2024-05-17T21:32:19.912 INFO:teuthology.orchestra.run.smithi138.stdout: kernel-modules x86_64 5.14.0-446.el9 baseos 34 M 2024-05-17T21:32:19.912 INFO:teuthology.orchestra.run.smithi138.stdout: kernel-modules-core x86_64 5.14.0-446.el9 baseos 28 M 2024-05-17T21:32:19.912 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-17T21:32:19.912 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-05-17T21:32:19.912 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-05-17T21:32:19.912 INFO:teuthology.orchestra.run.smithi138.stdout:Install 4 Packages 2024-05-17T21:32:19.912 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-17T21:32:19.913 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 78 M 2024-05-17T21:32:19.913 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 123 M 2024-05-17T21:32:19.913 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-05-17T21:32:20.157 INFO:teuthology.orchestra.run.smithi138.stdout:(1/4): kernel-5.14.0-446.el9.x86_64.rpm 3.0 MB/s | 550 kB 00:00 2024-05-17T21:32:20.649 INFO:teuthology.orchestra.run.smithi138.stdout:(2/4): kernel-core-5.14.0-446.el9.x86_64.rpm 23 MB/s | 15 MB 00:00 2024-05-17T21:32:20.696 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-05-17T21:32:20.696 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-05-17T21:32:20.990 INFO:teuthology.orchestra.run.smithi138.stdout:(3/4): kernel-modules-core-5.14.0-446.el9.x86_6 34 MB/s | 28 MB 00:00 2024-05-17T21:32:21.391 INFO:teuthology.orchestra.run.smithi138.stdout:(4/4): kernel-modules-5.14.0-446.el9.x86_64.rpm 24 MB/s | 34 MB 00:01 2024-05-17T21:32:21.391 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-05-17T21:32:21.391 INFO:teuthology.orchestra.run.smithi138.stdout:Total 53 MB/s | 78 MB 00:01 2024-05-17T21:32:22.077 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-05-17T21:32:22.438 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-05-17T21:32:22.438 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-05-17T21:32:23.114 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-05-17T21:32:23.390 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-05-17T21:32:23.391 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-05-17T21:32:23.761 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : kernel-modules-core-5.14.0-446.el9.x86_64 1/4 2024-05-17T21:32:23.840 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : kernel-core-5.14.0-446.el9.x86_64 2/4 2024-05-17T21:32:25.744 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-05-17T21:32:26.426 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : kernel-modules-core-5.14.0-446.el9.x86_64 1/4 2024-05-17T21:32:26.508 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : kernel-core-5.14.0-446.el9.x86_64 2/4 2024-05-17T21:32:26.522 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: kernel-core-5.14.0-446.el9.x86_64 2/4 2024-05-17T21:32:26.652 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : kernel-modules-5.14.0-446.el9.x86_64 3/4 2024-05-17T21:32:29.209 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: kernel-core-5.14.0-446.el9.x86_64 2/4 2024-05-17T21:32:29.341 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : kernel-modules-5.14.0-446.el9.x86_64 3/4 2024-05-17T21:32:31.394 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: kernel-modules-5.14.0-446.el9.x86_64 3/4 2024-05-17T21:32:31.504 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : kernel-5.14.0-446.el9.x86_64 4/4 2024-05-17T21:32:34.067 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: kernel-modules-5.14.0-446.el9.x86_64 3/4 2024-05-17T21:32:34.185 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : kernel-5.14.0-446.el9.x86_64 4/4 2024-05-17T21:32:36.236 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: kernel-modules-core-5.14.0-446.el9.x86_64 4/4 2024-05-17T21:32:38.897 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: kernel-modules-core-5.14.0-446.el9.x86_64 4/4 2024-05-17T21:33:05.518 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: kernel-core-5.14.0-446.el9.x86_64 4/4 2024-05-17T21:33:05.557 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: kernel-modules-5.14.0-446.el9.x86_64 4/4 2024-05-17T21:33:05.862 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: kernel-5.14.0-446.el9.x86_64 4/4 2024-05-17T21:33:05.862 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : kernel-5.14.0-446.el9.x86_64 1/4 2024-05-17T21:33:05.862 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : kernel-core-5.14.0-446.el9.x86_64 2/4 2024-05-17T21:33:05.862 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : kernel-modules-5.14.0-446.el9.x86_64 3/4 2024-05-17T21:33:06.676 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : kernel-modules-core-5.14.0-446.el9.x86_64 4/4 2024-05-17T21:33:06.676 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:33:06.676 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-05-17T21:33:06.676 INFO:teuthology.orchestra.run.smithi107.stdout: kernel-5.14.0-446.el9.x86_64 2024-05-17T21:33:06.676 INFO:teuthology.orchestra.run.smithi107.stdout: kernel-core-5.14.0-446.el9.x86_64 2024-05-17T21:33:06.676 INFO:teuthology.orchestra.run.smithi107.stdout: kernel-modules-5.14.0-446.el9.x86_64 2024-05-17T21:33:06.677 INFO:teuthology.orchestra.run.smithi107.stdout: kernel-modules-core-5.14.0-446.el9.x86_64 2024-05-17T21:33:06.677 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:33:06.677 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-05-17T21:33:07.030 DEBUG:teuthology.orchestra.run.smithi107:> rpm -q kernel | sort -rV | head -n 1 2024-05-17T21:33:07.148 INFO:teuthology.orchestra.run.smithi107.stdout:kernel-5.14.0-446.el9.x86_64 2024-05-17T21:33:07.148 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-446.el9.x86_64 2024-05-17T21:33:07.148 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-446.el9.x86_64 2024-05-17T21:33:07.148 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-17T21:33:07.149 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-17T21:33:07.149 INFO:teuthology.task.kernel:Installing distro kernel on host.a... 2024-05-17T21:33:07.149 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi107.front.sepia.ceph.com, path=None, version=distro) 2024-05-17T21:33:07.149 DEBUG:teuthology.orchestra.run.smithi107:> rpm -q kernel | sort -rV | head -n 1 2024-05-17T21:33:07.199 INFO:teuthology.orchestra.run.smithi107.stdout:kernel-5.14.0-446.el9.x86_64 2024-05-17T21:33:07.199 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-446.el9.x86_64 2024-05-17T21:33:07.199 DEBUG:teuthology.orchestra.run.smithi107:> sudo rpm -qi grub2-tools 2024-05-17T21:33:07.229 INFO:teuthology.orchestra.run.smithi107.stdout:Name : grub2-tools 2024-05-17T21:33:07.229 INFO:teuthology.orchestra.run.smithi107.stdout:Epoch : 1 2024-05-17T21:33:07.229 INFO:teuthology.orchestra.run.smithi107.stdout:Version : 2.06 2024-05-17T21:33:07.229 INFO:teuthology.orchestra.run.smithi107.stdout:Release : 68.el9 2024-05-17T21:33:07.229 INFO:teuthology.orchestra.run.smithi107.stdout:Architecture: x86_64 2024-05-17T21:33:07.230 INFO:teuthology.orchestra.run.smithi107.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-17T21:33:07.230 INFO:teuthology.orchestra.run.smithi107.stdout:Group : Unspecified 2024-05-17T21:33:07.230 INFO:teuthology.orchestra.run.smithi107.stdout:Size : 8264254 2024-05-17T21:33:07.230 INFO:teuthology.orchestra.run.smithi107.stdout:License : GPLv3+ 2024-05-17T21:33:07.230 INFO:teuthology.orchestra.run.smithi107.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-17T21:33:07.230 INFO:teuthology.orchestra.run.smithi107.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-17T21:33:07.230 INFO:teuthology.orchestra.run.smithi107.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-17T21:33:07.230 INFO:teuthology.orchestra.run.smithi107.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-17T21:33:07.230 INFO:teuthology.orchestra.run.smithi107.stdout:Packager : builder@centos.org 2024-05-17T21:33:07.230 INFO:teuthology.orchestra.run.smithi107.stdout:Vendor : CentOS 2024-05-17T21:33:07.230 INFO:teuthology.orchestra.run.smithi107.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-17T21:33:07.230 INFO:teuthology.orchestra.run.smithi107.stdout:Summary : Support tools for GRUB. 2024-05-17T21:33:07.230 INFO:teuthology.orchestra.run.smithi107.stdout:Description : 2024-05-17T21:33:07.231 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:33:07.231 INFO:teuthology.orchestra.run.smithi107.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-17T21:33:07.231 INFO:teuthology.orchestra.run.smithi107.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-17T21:33:07.231 INFO:teuthology.orchestra.run.smithi107.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-17T21:33:07.231 INFO:teuthology.orchestra.run.smithi107.stdout:hardware devices. 2024-05-17T21:33:07.231 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:33:07.231 INFO:teuthology.orchestra.run.smithi107.stdout:This subpackage provides tools for support of all platforms. 2024-05-17T21:33:07.232 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-17T21:33:07.232 INFO:teuthology.task.kernel:Updating grub on smithi107 to boot 5.14.0-446.el9.x86_64 2024-05-17T21:33:07.232 DEBUG:teuthology.orchestra.run.smithi107:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-17T21:33:07.859 INFO:teuthology.orchestra.run.smithi107.stderr:Generating grub configuration file ... 2024-05-17T21:33:08.425 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: kernel-core-5.14.0-446.el9.x86_64 4/4 2024-05-17T21:33:08.463 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: kernel-modules-5.14.0-446.el9.x86_64 4/4 2024-05-17T21:33:08.746 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: kernel-5.14.0-446.el9.x86_64 4/4 2024-05-17T21:33:08.747 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : kernel-5.14.0-446.el9.x86_64 1/4 2024-05-17T21:33:08.747 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : kernel-core-5.14.0-446.el9.x86_64 2/4 2024-05-17T21:33:08.747 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : kernel-modules-5.14.0-446.el9.x86_64 3/4 2024-05-17T21:33:09.254 INFO:teuthology.orchestra.run.smithi107.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-17T21:33:09.285 INFO:teuthology.orchestra.run.smithi107.stderr:done 2024-05-17T21:33:09.287 DEBUG:teuthology.orchestra.run.smithi107:> mktemp 2024-05-17T21:33:09.303 INFO:teuthology.orchestra.run.smithi107.stdout:/tmp/tmp.on89y6aHf7 2024-05-17T21:33:09.303 DEBUG:teuthology.orchestra.run.smithi107:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.on89y6aHf7 2024-05-17T21:33:09.368 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod 0666 /tmp/tmp.on89y6aHf7 2024-05-17T21:33:09.504 DEBUG:teuthology.orchestra.remote:smithi107:/tmp/tmp.on89y6aHf7 is 6KB 2024-05-17T21:33:09.515 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : kernel-modules-core-5.14.0-446.el9.x86_64 4/4 2024-05-17T21:33:09.515 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-17T21:33:09.515 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-05-17T21:33:09.515 INFO:teuthology.orchestra.run.smithi138.stdout: kernel-5.14.0-446.el9.x86_64 2024-05-17T21:33:09.515 INFO:teuthology.orchestra.run.smithi138.stdout: kernel-core-5.14.0-446.el9.x86_64 2024-05-17T21:33:09.515 INFO:teuthology.orchestra.run.smithi138.stdout: kernel-modules-5.14.0-446.el9.x86_64 2024-05-17T21:33:09.515 INFO:teuthology.orchestra.run.smithi138.stdout: kernel-modules-core-5.14.0-446.el9.x86_64 2024-05-17T21:33:09.515 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-17T21:33:09.516 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-05-17T21:33:09.551 DEBUG:teuthology.orchestra.run.smithi107:> rm -fr /tmp/tmp.on89y6aHf7 2024-05-17T21:33:09.565 DEBUG:teuthology.orchestra.run.smithi107:> sudo /bin/ls /boot/loader/entries || true 2024-05-17T21:33:09.635 INFO:teuthology.orchestra.run.smithi107.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-17T21:33:09.636 INFO:teuthology.orchestra.run.smithi107.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-17T21:33:09.636 INFO:teuthology.orchestra.run.smithi107.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-17T21:33:09.636 INFO:teuthology.orchestra.run.smithi107.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-446.el9.x86_64.conf 2024-05-17T21:33:09.637 DEBUG:teuthology.orchestra.run.smithi107:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-446.el9.x86_64 2024-05-17T21:33:09.797 DEBUG:teuthology.orchestra.run.smithi107:> sudo shutdown -r now 2024-05-17T21:33:09.863 DEBUG:teuthology.orchestra.run.smithi138:> rpm -q kernel | sort -rV | head -n 1 2024-05-17T21:33:09.942 INFO:teuthology.orchestra.run.smithi138.stdout:kernel-5.14.0-446.el9.x86_64 2024-05-17T21:33:09.943 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-446.el9.x86_64 2024-05-17T21:33:09.943 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-446.el9.x86_64 2024-05-17T21:33:09.943 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-17T21:33:09.943 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-17T21:33:09.943 INFO:teuthology.task.kernel:Installing distro kernel on host.b... 2024-05-17T21:33:09.943 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi138.front.sepia.ceph.com, path=None, version=distro) 2024-05-17T21:33:09.943 DEBUG:teuthology.orchestra.run.smithi138:> rpm -q kernel | sort -rV | head -n 1 2024-05-17T21:33:09.992 INFO:teuthology.orchestra.run.smithi138.stdout:kernel-5.14.0-446.el9.x86_64 2024-05-17T21:33:09.993 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-446.el9.x86_64 2024-05-17T21:33:09.993 DEBUG:teuthology.orchestra.run.smithi138:> sudo rpm -qi grub2-tools 2024-05-17T21:33:10.027 INFO:teuthology.orchestra.run.smithi138.stdout:Name : grub2-tools 2024-05-17T21:33:10.028 INFO:teuthology.orchestra.run.smithi138.stdout:Epoch : 1 2024-05-17T21:33:10.028 INFO:teuthology.orchestra.run.smithi138.stdout:Version : 2.06 2024-05-17T21:33:10.028 INFO:teuthology.orchestra.run.smithi138.stdout:Release : 68.el9 2024-05-17T21:33:10.028 INFO:teuthology.orchestra.run.smithi138.stdout:Architecture: x86_64 2024-05-17T21:33:10.028 INFO:teuthology.orchestra.run.smithi138.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-17T21:33:10.028 INFO:teuthology.orchestra.run.smithi138.stdout:Group : Unspecified 2024-05-17T21:33:10.028 INFO:teuthology.orchestra.run.smithi138.stdout:Size : 8264254 2024-05-17T21:33:10.028 INFO:teuthology.orchestra.run.smithi138.stdout:License : GPLv3+ 2024-05-17T21:33:10.028 INFO:teuthology.orchestra.run.smithi138.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-17T21:33:10.028 INFO:teuthology.orchestra.run.smithi138.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-17T21:33:10.028 INFO:teuthology.orchestra.run.smithi138.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-17T21:33:10.028 INFO:teuthology.orchestra.run.smithi138.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-17T21:33:10.028 INFO:teuthology.orchestra.run.smithi138.stdout:Packager : builder@centos.org 2024-05-17T21:33:10.029 INFO:teuthology.orchestra.run.smithi138.stdout:Vendor : CentOS 2024-05-17T21:33:10.029 INFO:teuthology.orchestra.run.smithi138.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-17T21:33:10.029 INFO:teuthology.orchestra.run.smithi138.stdout:Summary : Support tools for GRUB. 2024-05-17T21:33:10.029 INFO:teuthology.orchestra.run.smithi138.stdout:Description : 2024-05-17T21:33:10.029 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-17T21:33:10.029 INFO:teuthology.orchestra.run.smithi138.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-17T21:33:10.029 INFO:teuthology.orchestra.run.smithi138.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-17T21:33:10.029 INFO:teuthology.orchestra.run.smithi138.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-17T21:33:10.029 INFO:teuthology.orchestra.run.smithi138.stdout:hardware devices. 2024-05-17T21:33:10.029 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-17T21:33:10.029 INFO:teuthology.orchestra.run.smithi138.stdout:This subpackage provides tools for support of all platforms. 2024-05-17T21:33:10.030 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-17T21:33:10.030 INFO:teuthology.task.kernel:Updating grub on smithi138 to boot 5.14.0-446.el9.x86_64 2024-05-17T21:33:10.030 DEBUG:teuthology.orchestra.run.smithi138:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-17T21:33:10.706 INFO:teuthology.orchestra.run.smithi138.stderr:Generating grub configuration file ... 2024-05-17T21:33:12.056 INFO:teuthology.orchestra.run.smithi138.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-17T21:33:12.082 INFO:teuthology.orchestra.run.smithi138.stderr:done 2024-05-17T21:33:12.083 DEBUG:teuthology.orchestra.run.smithi138:> mktemp 2024-05-17T21:33:12.099 INFO:teuthology.orchestra.run.smithi138.stdout:/tmp/tmp.wSwpTohE75 2024-05-17T21:33:12.099 DEBUG:teuthology.orchestra.run.smithi138:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.wSwpTohE75 2024-05-17T21:33:12.164 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 0666 /tmp/tmp.wSwpTohE75 2024-05-17T21:33:12.301 DEBUG:teuthology.orchestra.remote:smithi138:/tmp/tmp.wSwpTohE75 is 6KB 2024-05-17T21:33:12.320 DEBUG:teuthology.orchestra.run.smithi138:> rm -fr /tmp/tmp.wSwpTohE75 2024-05-17T21:33:12.333 DEBUG:teuthology.orchestra.run.smithi138:> sudo /bin/ls /boot/loader/entries || true 2024-05-17T21:33:12.404 INFO:teuthology.orchestra.run.smithi138.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-17T21:33:12.405 INFO:teuthology.orchestra.run.smithi138.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-17T21:33:12.405 INFO:teuthology.orchestra.run.smithi138.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-17T21:33:12.405 INFO:teuthology.orchestra.run.smithi138.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-446.el9.x86_64.conf 2024-05-17T21:33:12.406 DEBUG:teuthology.orchestra.run.smithi138:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-446.el9.x86_64 2024-05-17T21:33:12.590 DEBUG:teuthology.orchestra.run.smithi138:> sudo shutdown -r now 2024-05-17T21:33:39.823 INFO:teuthology.task.kernel:Checking client host.a for new kernel version... 2024-05-17T21:33:39.824 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi107.front.sepia.ceph.com' 2024-05-17T21:33:39.825 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-17T21:33:42.594 INFO:teuthology.task.kernel:Checking client host.b for new kernel version... 2024-05-17T21:33:42.594 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-05-17T21:33:42.595 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-17T21:33:58.179 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.107 2024-05-17T21:34:00.994 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.138 2024-05-17T21:34:07.186 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi107.front.sepia.ceph.com' 2024-05-17T21:34:07.187 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-17T21:34:09.998 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-05-17T21:34:09.999 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-17T21:34:10.238 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.107 2024-05-17T21:34:13.054 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.138 2024-05-17T21:34:22.248 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi107.front.sepia.ceph.com' 2024-05-17T21:34:22.249 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-17T21:34:25.058 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-05-17T21:34:25.059 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-17T21:34:40.670 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.107 2024-05-17T21:34:55.684 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi107.front.sepia.ceph.com' 2024-05-17T21:34:55.685 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-17T21:34:56.076 DEBUG:teuthology.orchestra.run.smithi107:> true 2024-05-17T21:34:56.485 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi107.front.sepia.ceph.com' 2024-05-17T21:34:56.485 INFO:teuthology.task.kernel:Checking kernel version of host.a, want "5.14.0-446.el9.x86_64"... 2024-05-17T21:34:56.486 DEBUG:teuthology.orchestra.run.smithi107:> uname -r 2024-05-17T21:34:56.500 INFO:teuthology.orchestra.run.smithi107.stdout:5.14.0-446.el9.x86_64 2024-05-17T21:34:56.500 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-446.el9.x86_64 vs 5.14.0-446.el9.x86_64 2024-05-17T21:34:56.500 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-17T21:34:56.500 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-17T21:34:56.815 DEBUG:teuthology.orchestra.run.smithi138:> true 2024-05-17T21:34:57.187 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-05-17T21:34:57.187 INFO:teuthology.task.kernel:Checking kernel version of host.b, want "5.14.0-446.el9.x86_64"... 2024-05-17T21:34:57.187 DEBUG:teuthology.orchestra.run.smithi138:> uname -r 2024-05-17T21:34:57.201 INFO:teuthology.orchestra.run.smithi138.stdout:5.14.0-446.el9.x86_64 2024-05-17T21:34:57.202 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-446.el9.x86_64 vs 5.14.0-446.el9.x86_64 2024-05-17T21:34:57.202 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-17T21:34:57.202 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-17T21:34:57.501 INFO:teuthology.task.kernel:Enabling kdb on host.a... 2024-05-17T21:34:57.501 DEBUG:teuthology.orchestra.run.smithi107:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-17T21:34:57.568 INFO:teuthology.orchestra.run.smithi107.stdout:ttyS1 2024-05-17T21:34:57.618 DEBUG:teuthology.parallel:result is None 2024-05-17T21:34:58.203 INFO:teuthology.task.kernel:Enabling kdb on host.b... 2024-05-17T21:34:58.203 DEBUG:teuthology.orchestra.run.smithi138:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-17T21:34:58.289 INFO:teuthology.orchestra.run.smithi138.stdout:ttyS1 2024-05-17T21:34:58.324 DEBUG:teuthology.parallel:result is None 2024-05-17T21:34:58.324 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-17T21:34:58.332 INFO:teuthology.task.internal:Creating test directory... 2024-05-17T21:34:58.332 DEBUG:teuthology.orchestra.run.smithi107:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-17T21:34:58.335 DEBUG:teuthology.orchestra.run.smithi138:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-17T21:34:58.351 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-17T21:34:58.357 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-17T21:34:58.365 INFO:teuthology.task.internal:Creating archive directory... 2024-05-17T21:34:58.365 DEBUG:teuthology.orchestra.run.smithi107:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-17T21:34:58.393 DEBUG:teuthology.orchestra.run.smithi138:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-17T21:34:58.413 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-17T21:34:58.420 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-17T21:34:58.420 DEBUG:teuthology.orchestra.run.smithi107:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-17T21:34:58.453 DEBUG:teuthology.orchestra.run.smithi138:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-17T21:34:58.483 INFO:teuthology.orchestra.run.smithi138.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-17T21:34:58.493 INFO:teuthology.orchestra.run.smithi138.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-17T21:34:58.532 INFO:teuthology.orchestra.run.smithi107.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-17T21:34:58.542 INFO:teuthology.orchestra.run.smithi107.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-17T21:34:58.543 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-17T21:34:58.551 INFO:teuthology.task.internal:Configuring sudo... 2024-05-17T21:34:58.551 DEBUG:teuthology.orchestra.run.smithi107:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-17T21:34:58.553 DEBUG:teuthology.orchestra.run.smithi138:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-17T21:34:58.577 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-17T21:34:58.586 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-17T21:34:58.586 DEBUG:teuthology.orchestra.run.smithi107:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-17T21:34:58.619 DEBUG:teuthology.orchestra.run.smithi138:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-17T21:34:58.634 DEBUG:teuthology.orchestra.run.smithi107:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-17T21:34:58.710 DEBUG:teuthology.orchestra.run.smithi107:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-17T21:34:58.760 DEBUG:teuthology.orchestra.run.smithi107:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-17T21:34:58.839 DEBUG:teuthology.orchestra.run.smithi107:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-17T21:34:58.882 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-05-17T21:34:58.883 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-17T21:34:58.947 DEBUG:teuthology.orchestra.run.smithi138:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-17T21:34:58.986 DEBUG:teuthology.orchestra.run.smithi138:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-17T21:34:59.041 DEBUG:teuthology.orchestra.run.smithi138:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-17T21:34:59.119 DEBUG:teuthology.orchestra.run.smithi138:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-17T21:34:59.163 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-17T21:34:59.164 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-17T21:34:59.227 DEBUG:teuthology.orchestra.run.smithi107:> sudo service rsyslog restart 2024-05-17T21:34:59.229 DEBUG:teuthology.orchestra.run.smithi138:> sudo service rsyslog restart 2024-05-17T21:34:59.276 INFO:teuthology.orchestra.run.smithi107.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-17T21:34:59.313 INFO:teuthology.orchestra.run.smithi138.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-17T21:34:59.783 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-17T21:34:59.792 INFO:teuthology.task.internal:Starting timer... 2024-05-17T21:34:59.792 INFO:teuthology.run_tasks:Running task pcp... 2024-05-17T21:34:59.801 INFO:teuthology.run_tasks:Running task selinux... 2024-05-17T21:34:59.810 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-05-17T21:34:59.810 DEBUG:teuthology.orchestra.run.smithi107:> sudo service auditd rotate 2024-05-17T21:34:59.883 INFO:teuthology.orchestra.run.smithi107.stdout:Rotating logs: 2024-05-17T21:34:59.885 DEBUG:teuthology.orchestra.run.smithi138:> sudo service auditd rotate 2024-05-17T21:34:59.987 INFO:teuthology.orchestra.run.smithi138.stdout:Rotating logs: 2024-05-17T21:34:59.991 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-17T21:34:59.991 DEBUG:teuthology.orchestra.run.smithi107:> /usr/sbin/getenforce 2024-05-17T21:35:00.021 INFO:teuthology.orchestra.run.smithi107.stdout:Permissive 2024-05-17T21:35:00.021 DEBUG:teuthology.orchestra.run.smithi138:> /usr/sbin/getenforce 2024-05-17T21:35:00.057 INFO:teuthology.orchestra.run.smithi138.stdout:Permissive 2024-05-17T21:35:00.057 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi107.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi138.front.sepia.ceph.com': 'permissive'} 2024-05-17T21:35:00.058 DEBUG:teuthology.orchestra.run.smithi107:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-17T21:35:00.086 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-17T21:35:00.086 DEBUG:teuthology.orchestra.run.smithi138:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-17T21:35:00.150 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-17T21:35:00.150 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-17T21:35:00.151 DEBUG:teuthology.orchestra.run.smithi107:> sudo /usr/sbin/setenforce permissive 2024-05-17T21:35:00.198 DEBUG:teuthology.orchestra.run.smithi138:> sudo /usr/sbin/setenforce permissive 2024-05-17T21:35:00.461 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-17T21:35:00.471 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-05-17T21:35:00.474 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-05-17T21:35:00.602 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-17T21:35:00.613 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-17T21:35:00.614 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi107.front.sepia.ceph.com,smithi138.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-17T21:39:14.765 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi107.front.sepia.ceph.com'), Remote(name='ubuntu@smithi138.front.sepia.ceph.com')] 2024-05-17T21:39:14.766 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi107.front.sepia.ceph.com' 2024-05-17T21:39:14.768 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-17T21:39:14.844 DEBUG:teuthology.orchestra.run.smithi107:> true 2024-05-17T21:39:14.922 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi107.front.sepia.ceph.com' 2024-05-17T21:39:14.923 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-05-17T21:39:14.923 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-17T21:39:14.999 DEBUG:teuthology.orchestra.run.smithi138:> true 2024-05-17T21:39:15.083 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-05-17T21:39:15.083 INFO:teuthology.run_tasks:Running task clock... 2024-05-17T21:39:15.093 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-17T21:39:15.094 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-17T21:39:15.094 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-17T21:39:15.097 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-17T21:39:15.097 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-17T21:39:15.125 INFO:teuthology.orchestra.run.smithi107.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-17T21:39:15.139 INFO:teuthology.orchestra.run.smithi107.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-17T21:39:15.154 INFO:teuthology.orchestra.run.smithi138.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-17T21:39:15.164 INFO:teuthology.orchestra.run.smithi107.stderr:sudo: ntpd: command not found 2024-05-17T21:39:15.169 INFO:teuthology.orchestra.run.smithi138.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-17T21:39:15.175 INFO:teuthology.orchestra.run.smithi107.stdout:506 Cannot talk to daemon 2024-05-17T21:39:15.189 INFO:teuthology.orchestra.run.smithi107.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-17T21:39:15.194 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: ntpd: command not found 2024-05-17T21:39:15.203 INFO:teuthology.orchestra.run.smithi107.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-17T21:39:15.205 INFO:teuthology.orchestra.run.smithi138.stdout:506 Cannot talk to daemon 2024-05-17T21:39:15.219 INFO:teuthology.orchestra.run.smithi138.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-17T21:39:15.233 INFO:teuthology.orchestra.run.smithi138.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-17T21:39:15.258 INFO:teuthology.orchestra.run.smithi107.stderr:bash: line 1: ntpq: command not found 2024-05-17T21:39:15.260 INFO:teuthology.orchestra.run.smithi107.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-17T21:39:15.260 INFO:teuthology.orchestra.run.smithi107.stdout:=============================================================================== 2024-05-17T21:39:15.260 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-17T21:39:15.260 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-17T21:39:15.260 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-17T21:39:15.260 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-17T21:39:15.298 INFO:teuthology.orchestra.run.smithi138.stderr:bash: line 1: ntpq: command not found 2024-05-17T21:39:15.302 INFO:teuthology.orchestra.run.smithi138.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-17T21:39:15.302 INFO:teuthology.orchestra.run.smithi138.stdout:=============================================================================== 2024-05-17T21:39:15.302 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-17T21:39:15.302 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-17T21:39:15.302 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-17T21:39:15.302 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-17T21:39:15.302 INFO:teuthology.run_tasks:Running task install... 2024-05-17T21:39:15.310 DEBUG:teuthology.task.install:project ceph 2024-05-17T21:39:15.311 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '931809a9e5d295bd61984693cac44fe131ded4bc'}} 2024-05-17T21:39:15.311 DEBUG:teuthology.task.install:config {'branch': 'quincy', 'exclude_packages': ['ceph-volume'], 'flavor': 'default', 'sha1': '931809a9e5d295bd61984693cac44fe131ded4bc'} 2024-05-17T21:39:15.311 INFO:teuthology.task.install:Using flavor: default 2024-05-17T21:39:15.316 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-17T21:39:15.316 INFO:teuthology.task.install:extra packages: [] 2024-05-17T21:39:15.317 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'quincy', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '931809a9e5d295bd61984693cac44fe131ded4bc', 'tag': None, 'wait_for_package': False} 2024-05-17T21:39:15.317 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-17T21:39:15.317 INFO:teuthology.packaging:ref: None 2024-05-17T21:39:15.317 INFO:teuthology.packaging:tag: None 2024-05-17T21:39:15.317 INFO:teuthology.packaging:branch: quincy 2024-05-17T21:39:15.317 INFO:teuthology.packaging:sha1: 931809a9e5d295bd61984693cac44fe131ded4bc 2024-05-17T21:39:15.317 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-05-17T21:39:15.319 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'quincy', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '931809a9e5d295bd61984693cac44fe131ded4bc', 'tag': None, 'wait_for_package': False} 2024-05-17T21:39:15.319 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-17T21:39:15.320 INFO:teuthology.packaging:ref: None 2024-05-17T21:39:15.320 INFO:teuthology.packaging:tag: None 2024-05-17T21:39:15.320 INFO:teuthology.packaging:branch: quincy 2024-05-17T21:39:15.320 INFO:teuthology.packaging:sha1: 931809a9e5d295bd61984693cac44fe131ded4bc 2024-05-17T21:39:15.320 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-05-17T21:39:15.551 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/quincy/9903f7767386a9d9efadb7c1effdcd3064d71804/centos/9/flavors/default/ 2024-05-17T21:39:15.551 INFO:teuthology.task.install.rpm:Package version is 17.2.7-952.g9903f776 2024-05-17T21:39:15.602 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/quincy/9903f7767386a9d9efadb7c1effdcd3064d71804/centos/9/flavors/default/ 2024-05-17T21:39:15.602 INFO:teuthology.task.install.rpm:Package version is 17.2.7-952.g9903f776 2024-05-17T21:39:15.731 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/quincy/9903f7767386a9d9efadb7c1effdcd3064d71804/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/quincy/9903f7767386a9d9efadb7c1effdcd3064d71804/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/quincy/9903f7767386a9d9efadb7c1effdcd3064d71804/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-17T21:39:15.731 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-05-17T21:39:15.732 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-17T21:39:15.758 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-05-17T21:39:15.758 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-17T21:39:15.758 INFO:teuthology.packaging:ref: None 2024-05-17T21:39:15.758 INFO:teuthology.packaging:tag: None 2024-05-17T21:39:15.758 INFO:teuthology.packaging:branch: quincy 2024-05-17T21:39:15.759 INFO:teuthology.packaging:sha1: 931809a9e5d295bd61984693cac44fe131ded4bc 2024-05-17T21:39:15.759 DEBUG:teuthology.orchestra.run.smithi107:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/quincy/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-17T21:39:15.793 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/quincy/9903f7767386a9d9efadb7c1effdcd3064d71804/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/quincy/9903f7767386a9d9efadb7c1effdcd3064d71804/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/quincy/9903f7767386a9d9efadb7c1effdcd3064d71804/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-17T21:39:15.793 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-17T21:39:15.793 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-17T21:39:15.821 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-05-17T21:39:15.821 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-17T21:39:15.821 INFO:teuthology.packaging:ref: None 2024-05-17T21:39:15.821 INFO:teuthology.packaging:tag: None 2024-05-17T21:39:15.821 INFO:teuthology.packaging:branch: quincy 2024-05-17T21:39:15.821 INFO:teuthology.packaging:sha1: 931809a9e5d295bd61984693cac44fe131ded4bc 2024-05-17T21:39:15.821 DEBUG:teuthology.orchestra.run.smithi138:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/quincy/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-17T21:39:15.825 DEBUG:teuthology.orchestra.run.smithi107:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-17T21:39:15.888 DEBUG:teuthology.orchestra.run.smithi138:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-17T21:39:15.900 DEBUG:teuthology.orchestra.run.smithi107:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-17T21:39:15.964 DEBUG:teuthology.orchestra.run.smithi138:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-17T21:39:15.964 INFO:teuthology.orchestra.run.smithi107.stdout:check_obsoletes = 1 2024-05-17T21:39:15.965 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum clean all 2024-05-17T21:39:16.028 INFO:teuthology.orchestra.run.smithi138.stdout:check_obsoletes = 1 2024-05-17T21:39:16.030 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum clean all 2024-05-17T21:39:16.201 INFO:teuthology.orchestra.run.smithi107.stdout:57 files removed 2024-05-17T21:39:16.228 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install ceph-radosgw 2024-05-17T21:39:16.277 INFO:teuthology.orchestra.run.smithi138.stdout:57 files removed 2024-05-17T21:39:16.307 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph-radosgw 2024-05-17T21:39:17.179 INFO:teuthology.orchestra.run.smithi138.stdout:ceph packages for x86_64 129 kB/s | 77 kB 00:00 2024-05-17T21:39:17.274 INFO:teuthology.orchestra.run.smithi107.stdout:ceph packages for x86_64 100 kB/s | 77 kB 00:00 2024-05-17T21:39:17.839 INFO:teuthology.orchestra.run.smithi138.stdout:ceph noarch packages 18 kB/s | 11 kB 00:00 2024-05-17T21:39:17.942 INFO:teuthology.orchestra.run.smithi107.stdout:ceph noarch packages 17 kB/s | 11 kB 00:00 2024-05-17T21:39:18.489 INFO:teuthology.orchestra.run.smithi138.stdout:ceph source packages 2.9 kB/s | 1.8 kB 00:00 2024-05-17T21:39:18.642 INFO:teuthology.orchestra.run.smithi107.stdout:ceph source packages 2.7 kB/s | 1.8 kB 00:00 2024-05-17T21:39:19.132 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - BaseOS 18 MB/s | 8.1 MB 00:00 2024-05-17T21:39:19.497 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - BaseOS 8.3 MB/s | 8.1 MB 00:00 2024-05-17T21:39:21.085 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - AppStream 23 MB/s | 19 MB 00:00 2024-05-17T21:39:21.407 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - AppStream 24 MB/s | 19 MB 00:00 2024-05-17T21:39:25.352 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - CRB 8.4 MB/s | 6.2 MB 00:00 2024-05-17T21:39:26.836 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - Extras packages 54 kB/s | 16 kB 00:00 2024-05-17T21:39:27.149 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - CRB 2.9 MB/s | 6.2 MB 00:02 2024-05-17T21:39:28.424 INFO:teuthology.orchestra.run.smithi107.stdout:Extra Packages for Enterprise Linux 15 MB/s | 21 MB 00:01 2024-05-17T21:39:28.792 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - Extras packages 35 kB/s | 16 kB 00:00 2024-05-17T21:39:30.127 INFO:teuthology.orchestra.run.smithi138.stdout:Extra Packages for Enterprise Linux 18 MB/s | 21 MB 00:01 2024-05-17T21:39:33.687 INFO:teuthology.orchestra.run.smithi107.stdout:lab-extras 16 kB/s | 1.7 kB 00:00 2024-05-17T21:39:35.318 INFO:teuthology.orchestra.run.smithi138.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-05-17T21:39:35.988 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-05-17T21:39:36.001 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-05-17T21:39:36.001 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repository Size 2024-05-17T21:39:36.001 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-05-17T21:39:36.001 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-05-17T21:39:36.001 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-radosgw x86_64 2:17.2.7-952.g9903f776.el9 ceph 14 M 2024-05-17T21:39:36.001 INFO:teuthology.orchestra.run.smithi107.stdout:Upgrading: 2024-05-17T21:39:36.001 INFO:teuthology.orchestra.run.smithi107.stdout: librados2 x86_64 2:17.2.7-952.g9903f776.el9 ceph 3.1 M 2024-05-17T21:39:36.001 INFO:teuthology.orchestra.run.smithi107.stdout: librbd1 x86_64 2:17.2.7-952.g9903f776.el9 ceph 2.9 M 2024-05-17T21:39:36.001 INFO:teuthology.orchestra.run.smithi107.stdout:Installing dependencies: 2024-05-17T21:39:36.001 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-base x86_64 2:17.2.7-952.g9903f776.el9 ceph 5.0 M 2024-05-17T21:39:36.001 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-common x86_64 2:17.2.7-952.g9903f776.el9 ceph 22 M 2024-05-17T21:39:36.001 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-selinux x86_64 2:17.2.7-952.g9903f776.el9 ceph 25 k 2024-05-17T21:39:36.002 INFO:teuthology.orchestra.run.smithi107.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-17T21:39:36.002 INFO:teuthology.orchestra.run.smithi107.stdout: libcephfs2 x86_64 2:17.2.7-952.g9903f776.el9 ceph 637 k 2024-05-17T21:39:36.002 INFO:teuthology.orchestra.run.smithi107.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-17T21:39:36.002 INFO:teuthology.orchestra.run.smithi107.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-17T21:39:36.002 INFO:teuthology.orchestra.run.smithi107.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-17T21:39:36.002 INFO:teuthology.orchestra.run.smithi107.stdout: libradosstriper1 x86_64 2:17.2.7-952.g9903f776.el9 ceph 457 k 2024-05-17T21:39:36.002 INFO:teuthology.orchestra.run.smithi107.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-17T21:39:36.002 INFO:teuthology.orchestra.run.smithi107.stdout: librgw2 x86_64 2:17.2.7-952.g9903f776.el9 ceph 4.9 M 2024-05-17T21:39:36.002 INFO:teuthology.orchestra.run.smithi107.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-17T21:39:36.002 INFO:teuthology.orchestra.run.smithi107.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-17T21:39:36.002 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ceph-argparse x86_64 2:17.2.7-952.g9903f776.el9 ceph 45 k 2024-05-17T21:39:36.002 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ceph-common x86_64 2:17.2.7-952.g9903f776.el9 ceph 117 k 2024-05-17T21:39:36.002 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cephfs x86_64 2:17.2.7-952.g9903f776.el9 ceph 156 k 2024-05-17T21:39:36.002 INFO:teuthology.orchestra.run.smithi107.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-17T21:39:36.003 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rados x86_64 2:17.2.7-952.g9903f776.el9 ceph 320 k 2024-05-17T21:39:36.003 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rbd x86_64 2:17.2.7-952.g9903f776.el9 ceph 293 k 2024-05-17T21:39:36.003 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rgw x86_64 2:17.2.7-952.g9903f776.el9 ceph 99 k 2024-05-17T21:39:36.003 INFO:teuthology.orchestra.run.smithi107.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-17T21:39:36.003 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:39:36.003 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-05-17T21:39:36.003 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-05-17T21:39:36.003 INFO:teuthology.orchestra.run.smithi107.stdout:Install 22 Packages 2024-05-17T21:39:36.003 INFO:teuthology.orchestra.run.smithi107.stdout:Upgrade 2 Packages 2024-05-17T21:39:36.003 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:39:36.004 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 58 M 2024-05-17T21:39:36.004 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-05-17T21:39:37.404 INFO:teuthology.orchestra.run.smithi107.stdout:(1/24): ceph-base-17.2.7-952.g9903f776.el9.x86_ 4.4 MB/s | 5.0 MB 00:01 2024-05-17T21:39:37.455 INFO:teuthology.orchestra.run.smithi107.stdout:(2/24): ceph-selinux-17.2.7-952.g9903f776.el9.x 504 kB/s | 25 kB 00:00 2024-05-17T21:39:37.597 INFO:teuthology.orchestra.run.smithi107.stdout:(3/24): libcephfs2-17.2.7-952.g9903f776.el9.x86 4.4 MB/s | 637 kB 00:00 2024-05-17T21:39:37.611 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-05-17T21:39:37.620 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-05-17T21:39:37.620 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repository Size 2024-05-17T21:39:37.620 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-05-17T21:39:37.620 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-05-17T21:39:37.621 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-radosgw x86_64 2:17.2.7-952.g9903f776.el9 ceph 14 M 2024-05-17T21:39:37.621 INFO:teuthology.orchestra.run.smithi138.stdout:Upgrading: 2024-05-17T21:39:37.621 INFO:teuthology.orchestra.run.smithi138.stdout: librados2 x86_64 2:17.2.7-952.g9903f776.el9 ceph 3.1 M 2024-05-17T21:39:37.621 INFO:teuthology.orchestra.run.smithi138.stdout: librbd1 x86_64 2:17.2.7-952.g9903f776.el9 ceph 2.9 M 2024-05-17T21:39:37.621 INFO:teuthology.orchestra.run.smithi138.stdout:Installing dependencies: 2024-05-17T21:39:37.621 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-base x86_64 2:17.2.7-952.g9903f776.el9 ceph 5.0 M 2024-05-17T21:39:37.621 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-common x86_64 2:17.2.7-952.g9903f776.el9 ceph 22 M 2024-05-17T21:39:37.621 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-selinux x86_64 2:17.2.7-952.g9903f776.el9 ceph 25 k 2024-05-17T21:39:37.621 INFO:teuthology.orchestra.run.smithi138.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-17T21:39:37.621 INFO:teuthology.orchestra.run.smithi138.stdout: libcephfs2 x86_64 2:17.2.7-952.g9903f776.el9 ceph 637 k 2024-05-17T21:39:37.621 INFO:teuthology.orchestra.run.smithi138.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-17T21:39:37.621 INFO:teuthology.orchestra.run.smithi138.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-17T21:39:37.621 INFO:teuthology.orchestra.run.smithi138.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-17T21:39:37.621 INFO:teuthology.orchestra.run.smithi138.stdout: libradosstriper1 x86_64 2:17.2.7-952.g9903f776.el9 ceph 457 k 2024-05-17T21:39:37.622 INFO:teuthology.orchestra.run.smithi138.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-17T21:39:37.622 INFO:teuthology.orchestra.run.smithi138.stdout: librgw2 x86_64 2:17.2.7-952.g9903f776.el9 ceph 4.9 M 2024-05-17T21:39:37.622 INFO:teuthology.orchestra.run.smithi138.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-17T21:39:37.622 INFO:teuthology.orchestra.run.smithi138.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-17T21:39:37.622 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ceph-argparse x86_64 2:17.2.7-952.g9903f776.el9 ceph 45 k 2024-05-17T21:39:37.622 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ceph-common x86_64 2:17.2.7-952.g9903f776.el9 ceph 117 k 2024-05-17T21:39:37.622 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cephfs x86_64 2:17.2.7-952.g9903f776.el9 ceph 156 k 2024-05-17T21:39:37.622 INFO:teuthology.orchestra.run.smithi138.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-17T21:39:37.622 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rados x86_64 2:17.2.7-952.g9903f776.el9 ceph 320 k 2024-05-17T21:39:37.622 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rbd x86_64 2:17.2.7-952.g9903f776.el9 ceph 293 k 2024-05-17T21:39:37.622 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rgw x86_64 2:17.2.7-952.g9903f776.el9 ceph 99 k 2024-05-17T21:39:37.622 INFO:teuthology.orchestra.run.smithi138.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-17T21:39:37.622 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-17T21:39:37.622 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-05-17T21:39:37.623 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-05-17T21:39:37.623 INFO:teuthology.orchestra.run.smithi138.stdout:Install 22 Packages 2024-05-17T21:39:37.623 INFO:teuthology.orchestra.run.smithi138.stdout:Upgrade 2 Packages 2024-05-17T21:39:37.623 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-17T21:39:37.623 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 58 M 2024-05-17T21:39:37.623 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-05-17T21:39:37.688 INFO:teuthology.orchestra.run.smithi107.stdout:(4/24): libradosstriper1-17.2.7-952.g9903f776.e 4.9 MB/s | 457 kB 00:00 2024-05-17T21:39:38.006 INFO:teuthology.orchestra.run.smithi107.stdout:(5/24): ceph-common-17.2.7-952.g9903f776.el9.x8 13 MB/s | 22 MB 00:01 2024-05-17T21:39:38.064 INFO:teuthology.orchestra.run.smithi107.stdout:(6/24): python3-ceph-argparse-17.2.7-952.g9903f 775 kB/s | 45 kB 00:00 2024-05-17T21:39:38.131 INFO:teuthology.orchestra.run.smithi107.stdout:(7/24): python3-ceph-common-17.2.7-952.g9903f77 1.7 MB/s | 117 kB 00:00 2024-05-17T21:39:38.215 INFO:teuthology.orchestra.run.smithi107.stdout:(8/24): librgw2-17.2.7-952.g9903f776.el9.x86_64 9.3 MB/s | 4.9 MB 00:00 2024-05-17T21:39:38.240 INFO:teuthology.orchestra.run.smithi107.stdout:(9/24): python3-cephfs-17.2.7-952.g9903f776.el9 1.4 MB/s | 156 kB 00:00 2024-05-17T21:39:38.265 INFO:teuthology.orchestra.run.smithi107.stdout:(10/24): python3-rados-17.2.7-952.g9903f776.el9 6.2 MB/s | 320 kB 00:00 2024-05-17T21:39:38.291 INFO:teuthology.orchestra.run.smithi107.stdout:(11/24): python3-rbd-17.2.7-952.g9903f776.el9.x 5.7 MB/s | 293 kB 00:00 2024-05-17T21:39:38.316 INFO:teuthology.orchestra.run.smithi107.stdout:(12/24): python3-rgw-17.2.7-952.g9903f776.el9.x 1.9 MB/s | 99 kB 00:00 2024-05-17T21:39:38.433 INFO:teuthology.orchestra.run.smithi107.stdout:(13/24): librabbitmq-0.11.0-7.el9.x86_64.rpm 388 kB/s | 45 kB 00:00 2024-05-17T21:39:38.474 INFO:teuthology.orchestra.run.smithi107.stdout:(14/24): libpmemobj-1.12.1-1.el9.x86_64.rpm 875 kB/s | 160 kB 00:00 2024-05-17T21:39:38.542 INFO:teuthology.orchestra.run.smithi107.stdout:(15/24): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.3 MB/s | 292 kB 00:00 2024-05-17T21:39:38.567 INFO:teuthology.orchestra.run.smithi107.stdout:(16/24): librdkafka-1.6.1-102.el9.x86_64.rpm 4.8 MB/s | 662 kB 00:00 2024-05-17T21:39:38.592 INFO:teuthology.orchestra.run.smithi107.stdout:(17/24): python3-prettytable-0.7.2-27.el9.noarc 838 kB/s | 42 kB 00:00 2024-05-17T21:39:38.984 INFO:teuthology.orchestra.run.smithi107.stdout:(18/24): liboath-2.6.7-2.el9.x86_64.rpm 124 kB/s | 49 kB 00:00 2024-05-17T21:39:39.278 INFO:teuthology.orchestra.run.smithi138.stdout:(1/24): ceph-radosgw-17.2.7-952.g9903f776.el9.x 12 MB/s | 14 MB 00:01 2024-05-17T21:39:39.284 INFO:teuthology.orchestra.run.smithi107.stdout:(19/24): libunwind-1.6.2-1.el9.x86_64.rpm 225 kB/s | 67 kB 00:00 2024-05-17T21:39:39.311 INFO:teuthology.orchestra.run.smithi138.stdout:(2/24): ceph-selinux-17.2.7-952.g9903f776.el9.x 760 kB/s | 25 kB 00:00 2024-05-17T21:39:39.387 INFO:teuthology.orchestra.run.smithi138.stdout:(3/24): libcephfs2-17.2.7-952.g9903f776.el9.x86 8.3 MB/s | 637 kB 00:00 2024-05-17T21:39:39.479 INFO:teuthology.orchestra.run.smithi138.stdout:(4/24): ceph-base-17.2.7-952.g9903f776.el9.x86_ 3.7 MB/s | 5.0 MB 00:01 2024-05-17T21:39:39.504 INFO:teuthology.orchestra.run.smithi138.stdout:(5/24): libradosstriper1-17.2.7-952.g9903f776.e 3.8 MB/s | 457 kB 00:00 2024-05-17T21:39:39.554 INFO:teuthology.orchestra.run.smithi138.stdout:(6/24): python3-ceph-argparse-17.2.7-952.g9903f 901 kB/s | 45 kB 00:00 2024-05-17T21:39:39.613 INFO:teuthology.orchestra.run.smithi138.stdout:(7/24): python3-ceph-common-17.2.7-952.g9903f77 2.0 MB/s | 117 kB 00:00 2024-05-17T21:39:39.680 INFO:teuthology.orchestra.run.smithi138.stdout:(8/24): python3-cephfs-17.2.7-952.g9903f776.el9 2.3 MB/s | 156 kB 00:00 2024-05-17T21:39:39.726 INFO:teuthology.orchestra.run.smithi107.stdout:(20/24): gperftools-libs-2.9.1-2.el9.x86_64.rpm 266 kB/s | 309 kB 00:01 2024-05-17T21:39:39.755 INFO:teuthology.orchestra.run.smithi138.stdout:(9/24): python3-rados-17.2.7-952.g9903f776.el9. 4.2 MB/s | 320 kB 00:00 2024-05-17T21:39:39.839 INFO:teuthology.orchestra.run.smithi138.stdout:(10/24): python3-rbd-17.2.7-952.g9903f776.el9.x 3.4 MB/s | 293 kB 00:00 2024-05-17T21:39:39.914 INFO:teuthology.orchestra.run.smithi138.stdout:(11/24): python3-rgw-17.2.7-952.g9903f776.el9.x 1.3 MB/s | 99 kB 00:00 2024-05-17T21:39:40.056 INFO:teuthology.orchestra.run.smithi138.stdout:(12/24): librgw2-17.2.7-952.g9903f776.el9.x86_6 8.5 MB/s | 4.9 MB 00:00 2024-05-17T21:39:40.123 INFO:teuthology.orchestra.run.smithi138.stdout:(13/24): libpmemobj-1.12.1-1.el9.x86_64.rpm 767 kB/s | 160 kB 00:00 2024-05-17T21:39:40.165 INFO:teuthology.orchestra.run.smithi138.stdout:(14/24): librabbitmq-0.11.0-7.el9.x86_64.rpm 417 kB/s | 45 kB 00:00 2024-05-17T21:39:40.240 INFO:teuthology.orchestra.run.smithi138.stdout:(15/24): librdkafka-1.6.1-102.el9.x86_64.rpm 5.5 MB/s | 662 kB 00:00 2024-05-17T21:39:40.260 INFO:teuthology.orchestra.run.smithi107.stdout:(21/24): librados2-17.2.7-952.g9903f776.el9.x86 5.8 MB/s | 3.1 MB 00:00 2024-05-17T21:39:40.274 INFO:teuthology.orchestra.run.smithi138.stdout:(16/24): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.6 MB/s | 292 kB 00:00 2024-05-17T21:39:40.299 INFO:teuthology.orchestra.run.smithi138.stdout:(17/24): python3-prettytable-0.7.2-27.el9.noarc 720 kB/s | 42 kB 00:00 2024-05-17T21:39:40.500 INFO:teuthology.orchestra.run.smithi138.stdout:(18/24): liboath-2.6.7-2.el9.x86_64.rpm 243 kB/s | 49 kB 00:00 2024-05-17T21:39:40.617 INFO:teuthology.orchestra.run.smithi138.stdout:(19/24): libunwind-1.6.2-1.el9.x86_64.rpm 577 kB/s | 67 kB 00:00 2024-05-17T21:39:40.635 INFO:teuthology.orchestra.run.smithi107.stdout:(22/24): ceph-radosgw-17.2.7-952.g9903f776.el9. 3.2 MB/s | 14 MB 00:04 2024-05-17T21:39:40.642 INFO:teuthology.orchestra.run.smithi138.stdout:(20/24): gperftools-libs-2.9.1-2.el9.x86_64.rpm 840 kB/s | 309 kB 00:00 2024-05-17T21:39:40.702 INFO:teuthology.orchestra.run.smithi107.stdout:(23/24): librbd1-17.2.7-952.g9903f776.el9.x86_6 6.7 MB/s | 2.9 MB 00:00 2024-05-17T21:39:40.884 INFO:teuthology.orchestra.run.smithi138.stdout:(21/24): ceph-common-17.2.7-952.g9903f776.el9.x 8.2 MB/s | 22 MB 00:02 2024-05-17T21:39:41.084 INFO:teuthology.orchestra.run.smithi138.stdout:(22/24): librados2-17.2.7-952.g9903f776.el9.x86 7.0 MB/s | 3.1 MB 00:00 2024-05-17T21:39:41.126 INFO:teuthology.orchestra.run.smithi138.stdout:(23/24): thrift-0.15.0-2.el9.x86_64.rpm 3.1 MB/s | 1.6 MB 00:00 2024-05-17T21:39:41.193 INFO:teuthology.orchestra.run.smithi138.stdout:(24/24): librbd1-17.2.7-952.g9903f776.el9.x86_6 9.5 MB/s | 2.9 MB 00:00 2024-05-17T21:39:41.195 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-05-17T21:39:41.195 INFO:teuthology.orchestra.run.smithi138.stdout:Total 16 MB/s | 58 MB 00:03 2024-05-17T21:39:41.255 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-05-17T21:39:41.287 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-05-17T21:39:41.287 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-05-17T21:39:41.875 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-05-17T21:39:41.875 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-05-17T21:39:43.054 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-05-17T21:39:43.293 INFO:teuthology.orchestra.run.smithi107.stdout:(24/24): thrift-0.15.0-2.el9.x86_64.rpm 404 kB/s | 1.6 MB 00:04 2024-05-17T21:39:43.296 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-05-17T21:39:43.296 INFO:teuthology.orchestra.run.smithi107.stdout:Total 7.9 MB/s | 58 MB 00:07 2024-05-17T21:39:43.360 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-05-17T21:39:43.393 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-05-17T21:39:43.393 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-05-17T21:39:43.415 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/26 2024-05-17T21:39:43.512 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : librados2-2:17.2.7-952.g9903f776.el9.x86_64 2/26 2024-05-17T21:39:43.604 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librados2-2:17.2.7-952.g9903f776.el9.x86_64 2/26 2024-05-17T21:39:43.683 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-rados-2:17.2.7-952.g9903f776.el9.x86_64 3/26 2024-05-17T21:39:43.705 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libcephfs2-2:17.2.7-952.g9903f776.el9.x86_64 4/26 2024-05-17T21:39:43.794 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: libcephfs2-2:17.2.7-952.g9903f776.el9.x86_64 4/26 2024-05-17T21:39:43.839 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : thrift-0.15.0-2.el9.x86_64 5/26 2024-05-17T21:39:43.868 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/26 2024-05-17T21:39:43.953 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-05-17T21:39:43.954 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-05-17T21:39:44.311 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/26 2024-05-17T21:39:44.342 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : librgw2-2:17.2.7-952.g9903f776.el9.x86_64 8/26 2024-05-17T21:39:44.405 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librgw2-2:17.2.7-952.g9903f776.el9.x86_64 8/26 2024-05-17T21:39:44.442 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : liboath-2.6.7-2.el9.x86_64 9/26 2024-05-17T21:39:44.485 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-ceph-argparse-2:17.2.7-952.g9903f776.el9.x 10/26 2024-05-17T21:39:44.522 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cephfs-2:17.2.7-952.g9903f776.el9.x86_64 11/26 2024-05-17T21:39:44.600 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-rgw-2:17.2.7-952.g9903f776.el9.x86_64 12/26 2024-05-17T21:39:44.628 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libradosstriper1-2:17.2.7-952.g9903f776.el9.x86_64 13/26 2024-05-17T21:39:44.696 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: libradosstriper1-2:17.2.7-952.g9903f776.el9.x86_64 13/26 2024-05-17T21:39:44.745 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/26 2024-05-17T21:39:44.778 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/26 2024-05-17T21:39:44.811 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 16/26 2024-05-17T21:39:45.107 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/26 2024-05-17T21:39:45.133 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-05-17T21:39:45.149 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : librbd1-2:17.2.7-952.g9903f776.el9.x86_64 18/26 2024-05-17T21:39:45.222 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librbd1-2:17.2.7-952.g9903f776.el9.x86_64 18/26 2024-05-17T21:39:45.295 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-rbd-2:17.2.7-952.g9903f776.el9.x86_64 19/26 2024-05-17T21:39:45.327 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-ceph-common-2:17.2.7-952.g9903f776.el9.x86 20/26 2024-05-17T21:39:45.523 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/26 2024-05-17T21:39:45.596 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : librados2-2:17.2.7-952.g9903f776.el9.x86_64 2/26 2024-05-17T21:39:45.670 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librados2-2:17.2.7-952.g9903f776.el9.x86_64 2/26 2024-05-17T21:39:45.757 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-rados-2:17.2.7-952.g9903f776.el9.x86_64 3/26 2024-05-17T21:39:45.788 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libcephfs2-2:17.2.7-952.g9903f776.el9.x86_64 4/26 2024-05-17T21:39:45.867 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: libcephfs2-2:17.2.7-952.g9903f776.el9.x86_64 4/26 2024-05-17T21:39:45.914 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : thrift-0.15.0-2.el9.x86_64 5/26 2024-05-17T21:39:45.953 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/26 2024-05-17T21:39:46.394 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/26 2024-05-17T21:39:46.425 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : librgw2-2:17.2.7-952.g9903f776.el9.x86_64 8/26 2024-05-17T21:39:46.488 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librgw2-2:17.2.7-952.g9903f776.el9.x86_64 8/26 2024-05-17T21:39:46.516 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : liboath-2.6.7-2.el9.x86_64 9/26 2024-05-17T21:39:46.559 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-ceph-argparse-2:17.2.7-952.g9903f776.el9.x 10/26 2024-05-17T21:39:46.596 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cephfs-2:17.2.7-952.g9903f776.el9.x86_64 11/26 2024-05-17T21:39:46.675 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-rgw-2:17.2.7-952.g9903f776.el9.x86_64 12/26 2024-05-17T21:39:46.703 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libradosstriper1-2:17.2.7-952.g9903f776.el9.x86_64 13/26 2024-05-17T21:39:46.783 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: libradosstriper1-2:17.2.7-952.g9903f776.el9.x86_64 13/26 2024-05-17T21:39:46.826 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/26 2024-05-17T21:39:46.861 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/26 2024-05-17T21:39:46.894 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 16/26 2024-05-17T21:39:47.187 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/26 2024-05-17T21:39:47.232 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : librbd1-2:17.2.7-952.g9903f776.el9.x86_64 18/26 2024-05-17T21:39:47.305 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librbd1-2:17.2.7-952.g9903f776.el9.x86_64 18/26 2024-05-17T21:39:47.378 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-rbd-2:17.2.7-952.g9903f776.el9.x86_64 19/26 2024-05-17T21:39:47.418 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-ceph-common-2:17.2.7-952.g9903f776.el9.x86 20/26 2024-05-17T21:39:47.779 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-common-2:17.2.7-952.g9903f776.el9.x86_64 21/26 2024-05-17T21:39:47.815 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-common-2:17.2.7-952.g9903f776.el9.x86_64 21/26 2024-05-17T21:39:48.348 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-common-2:17.2.7-952.g9903f776.el9.x86_64 21/26 2024-05-17T21:39:48.387 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-base-2:17.2.7-952.g9903f776.el9.x86_64 22/26 2024-05-17T21:39:48.464 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-base-2:17.2.7-952.g9903f776.el9.x86_64 22/26 2024-05-17T21:39:48.465 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-17T21:39:48.465 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-17T21:39:48.465 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-17T21:39:48.496 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-selinux-2:17.2.7-952.g9903f776.el9.x86_64 23/26 2024-05-17T21:39:49.850 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-common-2:17.2.7-952.g9903f776.el9.x86_64 21/26 2024-05-17T21:39:49.890 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-common-2:17.2.7-952.g9903f776.el9.x86_64 21/26 2024-05-17T21:39:50.404 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-common-2:17.2.7-952.g9903f776.el9.x86_64 21/26 2024-05-17T21:39:50.443 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-base-2:17.2.7-952.g9903f776.el9.x86_64 22/26 2024-05-17T21:39:50.534 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-base-2:17.2.7-952.g9903f776.el9.x86_64 22/26 2024-05-17T21:39:50.534 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-17T21:39:50.534 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-17T21:39:50.535 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:39:50.570 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-selinux-2:17.2.7-952.g9903f776.el9.x86_64 23/26 2024-05-17T21:39:57.961 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-selinux-2:17.2.7-952.g9903f776.el9.x86_64 23/26 2024-05-17T21:39:57.961 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /sys 2024-05-17T21:39:57.961 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /proc 2024-05-17T21:39:57.961 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /mnt 2024-05-17T21:39:57.961 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /var/tmp 2024-05-17T21:39:57.961 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /home 2024-05-17T21:39:57.961 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /root 2024-05-17T21:39:57.961 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /tmp 2024-05-17T21:39:57.962 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-17T21:39:59.276 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-radosgw-2:17.2.7-952.g9903f776.el9.x86_64 24/26 2024-05-17T21:39:59.354 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-952.g9903f776.el9.x86_64 24/26 2024-05-17T21:39:59.354 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-17T21:39:59.354 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-17T21:39:59.354 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-17T21:39:59.354 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-17T21:39:59.354 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-17T21:39:59.356 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-05-17T21:39:59.445 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-05-17T21:39:59.445 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 26/26 2024-05-17T21:39:59.831 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-selinux-2:17.2.7-952.g9903f776.el9.x86_64 23/26 2024-05-17T21:39:59.832 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /sys 2024-05-17T21:39:59.832 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /proc 2024-05-17T21:39:59.832 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /mnt 2024-05-17T21:39:59.832 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /var/tmp 2024-05-17T21:39:59.832 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /home 2024-05-17T21:39:59.832 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /root 2024-05-17T21:39:59.832 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /tmp 2024-05-17T21:39:59.832 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:40:00.682 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 26/26 2024-05-17T21:40:00.682 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-base-2:17.2.7-952.g9903f776.el9.x86_64 1/26 2024-05-17T21:40:00.682 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-common-2:17.2.7-952.g9903f776.el9.x86_64 2/26 2024-05-17T21:40:00.682 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-radosgw-2:17.2.7-952.g9903f776.el9.x86_64 3/26 2024-05-17T21:40:00.682 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-selinux-2:17.2.7-952.g9903f776.el9.x86_64 4/26 2024-05-17T21:40:00.682 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libcephfs2-2:17.2.7-952.g9903f776.el9.x86_64 5/26 2024-05-17T21:40:00.682 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libradosstriper1-2:17.2.7-952.g9903f776.el9.x86_64 6/26 2024-05-17T21:40:00.682 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librgw2-2:17.2.7-952.g9903f776.el9.x86_64 7/26 2024-05-17T21:40:00.683 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-ceph-argparse-2:17.2.7-952.g9903f776.el9.x 8/26 2024-05-17T21:40:00.683 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-ceph-common-2:17.2.7-952.g9903f776.el9.x86 9/26 2024-05-17T21:40:00.683 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cephfs-2:17.2.7-952.g9903f776.el9.x86_64 10/26 2024-05-17T21:40:00.683 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-rados-2:17.2.7-952.g9903f776.el9.x86_64 11/26 2024-05-17T21:40:00.683 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-rbd-2:17.2.7-952.g9903f776.el9.x86_64 12/26 2024-05-17T21:40:00.683 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-rgw-2:17.2.7-952.g9903f776.el9.x86_64 13/26 2024-05-17T21:40:00.683 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 14/26 2024-05-17T21:40:00.683 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 15/26 2024-05-17T21:40:00.683 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 16/26 2024-05-17T21:40:00.683 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 17/26 2024-05-17T21:40:00.683 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 18/26 2024-05-17T21:40:00.683 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 19/26 2024-05-17T21:40:00.683 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 20/26 2024-05-17T21:40:00.683 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 21/26 2024-05-17T21:40:00.683 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 22/26 2024-05-17T21:40:00.684 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librados2-2:17.2.7-952.g9903f776.el9.x86_64 23/26 2024-05-17T21:40:00.684 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 24/26 2024-05-17T21:40:00.684 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librbd1-2:17.2.7-952.g9903f776.el9.x86_64 25/26 2024-05-17T21:40:01.114 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-radosgw-2:17.2.7-952.g9903f776.el9.x86_64 24/26 2024-05-17T21:40:01.193 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-952.g9903f776.el9.x86_64 24/26 2024-05-17T21:40:01.193 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-17T21:40:01.193 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-17T21:40:01.193 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-17T21:40:01.194 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-17T21:40:01.194 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:40:01.195 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-05-17T21:40:01.275 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-05-17T21:40:01.275 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 26/26 2024-05-17T21:40:01.430 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 26/26 2024-05-17T21:40:01.430 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-17T21:40:01.430 INFO:teuthology.orchestra.run.smithi138.stdout:Upgraded: 2024-05-17T21:40:01.430 INFO:teuthology.orchestra.run.smithi138.stdout: librados2-2:17.2.7-952.g9903f776.el9.x86_64 2024-05-17T21:40:01.430 INFO:teuthology.orchestra.run.smithi138.stdout: librbd1-2:17.2.7-952.g9903f776.el9.x86_64 2024-05-17T21:40:01.430 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-05-17T21:40:01.430 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-base-2:17.2.7-952.g9903f776.el9.x86_64 2024-05-17T21:40:01.431 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-common-2:17.2.7-952.g9903f776.el9.x86_64 2024-05-17T21:40:01.431 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-radosgw-2:17.2.7-952.g9903f776.el9.x86_64 2024-05-17T21:40:01.431 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-selinux-2:17.2.7-952.g9903f776.el9.x86_64 2024-05-17T21:40:01.431 INFO:teuthology.orchestra.run.smithi138.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-17T21:40:01.431 INFO:teuthology.orchestra.run.smithi138.stdout: libcephfs2-2:17.2.7-952.g9903f776.el9.x86_64 2024-05-17T21:40:01.431 INFO:teuthology.orchestra.run.smithi138.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-17T21:40:01.431 INFO:teuthology.orchestra.run.smithi138.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-17T21:40:01.431 INFO:teuthology.orchestra.run.smithi138.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-17T21:40:01.431 INFO:teuthology.orchestra.run.smithi138.stdout: libradosstriper1-2:17.2.7-952.g9903f776.el9.x86_64 2024-05-17T21:40:01.431 INFO:teuthology.orchestra.run.smithi138.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-17T21:40:01.431 INFO:teuthology.orchestra.run.smithi138.stdout: librgw2-2:17.2.7-952.g9903f776.el9.x86_64 2024-05-17T21:40:01.431 INFO:teuthology.orchestra.run.smithi138.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-17T21:40:01.431 INFO:teuthology.orchestra.run.smithi138.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-17T21:40:01.431 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ceph-argparse-2:17.2.7-952.g9903f776.el9.x86_64 2024-05-17T21:40:01.431 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ceph-common-2:17.2.7-952.g9903f776.el9.x86_64 2024-05-17T21:40:01.432 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cephfs-2:17.2.7-952.g9903f776.el9.x86_64 2024-05-17T21:40:01.432 INFO:teuthology.orchestra.run.smithi138.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-17T21:40:01.432 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rados-2:17.2.7-952.g9903f776.el9.x86_64 2024-05-17T21:40:01.432 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rbd-2:17.2.7-952.g9903f776.el9.x86_64 2024-05-17T21:40:01.432 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rgw-2:17.2.7-952.g9903f776.el9.x86_64 2024-05-17T21:40:01.432 INFO:teuthology.orchestra.run.smithi138.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-17T21:40:01.432 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-17T21:40:01.432 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-05-17T21:40:01.813 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph-test 2024-05-17T21:40:02.392 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:00:27 ago on Fri 17 May 2024 09:39:35 PM UTC. 2024-05-17T21:40:02.483 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 26/26 2024-05-17T21:40:02.483 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-base-2:17.2.7-952.g9903f776.el9.x86_64 1/26 2024-05-17T21:40:02.483 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-common-2:17.2.7-952.g9903f776.el9.x86_64 2/26 2024-05-17T21:40:02.483 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-radosgw-2:17.2.7-952.g9903f776.el9.x86_64 3/26 2024-05-17T21:40:02.483 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-selinux-2:17.2.7-952.g9903f776.el9.x86_64 4/26 2024-05-17T21:40:02.483 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libcephfs2-2:17.2.7-952.g9903f776.el9.x86_64 5/26 2024-05-17T21:40:02.483 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libradosstriper1-2:17.2.7-952.g9903f776.el9.x86_64 6/26 2024-05-17T21:40:02.483 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librgw2-2:17.2.7-952.g9903f776.el9.x86_64 7/26 2024-05-17T21:40:02.483 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-ceph-argparse-2:17.2.7-952.g9903f776.el9.x 8/26 2024-05-17T21:40:02.483 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-ceph-common-2:17.2.7-952.g9903f776.el9.x86 9/26 2024-05-17T21:40:02.483 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cephfs-2:17.2.7-952.g9903f776.el9.x86_64 10/26 2024-05-17T21:40:02.484 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-rados-2:17.2.7-952.g9903f776.el9.x86_64 11/26 2024-05-17T21:40:02.484 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-rbd-2:17.2.7-952.g9903f776.el9.x86_64 12/26 2024-05-17T21:40:02.484 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-rgw-2:17.2.7-952.g9903f776.el9.x86_64 13/26 2024-05-17T21:40:02.484 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 14/26 2024-05-17T21:40:02.484 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 15/26 2024-05-17T21:40:02.484 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 16/26 2024-05-17T21:40:02.484 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 17/26 2024-05-17T21:40:02.484 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 18/26 2024-05-17T21:40:02.484 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 19/26 2024-05-17T21:40:02.484 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 20/26 2024-05-17T21:40:02.484 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 21/26 2024-05-17T21:40:02.484 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 22/26 2024-05-17T21:40:02.484 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librados2-2:17.2.7-952.g9903f776.el9.x86_64 23/26 2024-05-17T21:40:02.484 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 24/26 2024-05-17T21:40:02.485 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librbd1-2:17.2.7-952.g9903f776.el9.x86_64 25/26 2024-05-17T21:40:02.513 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-05-17T21:40:02.514 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-05-17T21:40:02.514 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repository Size 2024-05-17T21:40:02.514 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-05-17T21:40:02.514 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-05-17T21:40:02.515 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-test x86_64 2:17.2.7-952.g9903f776.el9 ceph 48 M 2024-05-17T21:40:02.515 INFO:teuthology.orchestra.run.smithi138.stdout:Installing dependencies: 2024-05-17T21:40:02.515 INFO:teuthology.orchestra.run.smithi138.stdout: libcephsqlite x86_64 2:17.2.7-952.g9903f776.el9 ceph 163 k 2024-05-17T21:40:02.515 INFO:teuthology.orchestra.run.smithi138.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-17T21:40:02.515 INFO:teuthology.orchestra.run.smithi138.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-17T21:40:02.515 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-17T21:40:02.515 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-05-17T21:40:02.515 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-05-17T21:40:02.515 INFO:teuthology.orchestra.run.smithi138.stdout:Install 4 Packages 2024-05-17T21:40:02.515 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-17T21:40:02.516 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 48 M 2024-05-17T21:40:02.516 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 192 M 2024-05-17T21:40:02.516 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-05-17T21:40:03.118 INFO:teuthology.orchestra.run.smithi138.stdout:(1/4): libcephsqlite-17.2.7-952.g9903f776.el9.x 367 kB/s | 163 kB 00:00 2024-05-17T21:40:03.232 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 26/26 2024-05-17T21:40:03.232 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:40:03.232 INFO:teuthology.orchestra.run.smithi107.stdout:Upgraded: 2024-05-17T21:40:03.232 INFO:teuthology.orchestra.run.smithi107.stdout: librados2-2:17.2.7-952.g9903f776.el9.x86_64 2024-05-17T21:40:03.232 INFO:teuthology.orchestra.run.smithi107.stdout: librbd1-2:17.2.7-952.g9903f776.el9.x86_64 2024-05-17T21:40:03.232 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-05-17T21:40:03.232 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-base-2:17.2.7-952.g9903f776.el9.x86_64 2024-05-17T21:40:03.232 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-common-2:17.2.7-952.g9903f776.el9.x86_64 2024-05-17T21:40:03.233 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-radosgw-2:17.2.7-952.g9903f776.el9.x86_64 2024-05-17T21:40:03.233 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-selinux-2:17.2.7-952.g9903f776.el9.x86_64 2024-05-17T21:40:03.233 INFO:teuthology.orchestra.run.smithi107.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-17T21:40:03.233 INFO:teuthology.orchestra.run.smithi107.stdout: libcephfs2-2:17.2.7-952.g9903f776.el9.x86_64 2024-05-17T21:40:03.233 INFO:teuthology.orchestra.run.smithi107.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-17T21:40:03.233 INFO:teuthology.orchestra.run.smithi107.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-17T21:40:03.233 INFO:teuthology.orchestra.run.smithi107.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-17T21:40:03.233 INFO:teuthology.orchestra.run.smithi107.stdout: libradosstriper1-2:17.2.7-952.g9903f776.el9.x86_64 2024-05-17T21:40:03.233 INFO:teuthology.orchestra.run.smithi107.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-17T21:40:03.233 INFO:teuthology.orchestra.run.smithi107.stdout: librgw2-2:17.2.7-952.g9903f776.el9.x86_64 2024-05-17T21:40:03.233 INFO:teuthology.orchestra.run.smithi107.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-17T21:40:03.233 INFO:teuthology.orchestra.run.smithi107.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-17T21:40:03.233 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ceph-argparse-2:17.2.7-952.g9903f776.el9.x86_64 2024-05-17T21:40:03.234 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ceph-common-2:17.2.7-952.g9903f776.el9.x86_64 2024-05-17T21:40:03.234 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cephfs-2:17.2.7-952.g9903f776.el9.x86_64 2024-05-17T21:40:03.234 INFO:teuthology.orchestra.run.smithi107.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-17T21:40:03.234 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rados-2:17.2.7-952.g9903f776.el9.x86_64 2024-05-17T21:40:03.234 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rbd-2:17.2.7-952.g9903f776.el9.x86_64 2024-05-17T21:40:03.234 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rgw-2:17.2.7-952.g9903f776.el9.x86_64 2024-05-17T21:40:03.234 INFO:teuthology.orchestra.run.smithi107.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-17T21:40:03.234 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:40:03.234 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-05-17T21:40:03.581 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install ceph-test 2024-05-17T21:40:03.743 INFO:teuthology.orchestra.run.smithi138.stdout:(2/4): socat-1.7.4.1-5.el9.x86_64.rpm 285 kB/s | 305 kB 00:01 2024-05-17T21:40:03.826 INFO:teuthology.orchestra.run.smithi138.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 90 kB/s | 64 kB 00:00 2024-05-17T21:40:04.152 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:00:31 ago on Fri 17 May 2024 09:39:33 PM UTC. 2024-05-17T21:40:04.272 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-05-17T21:40:04.272 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-05-17T21:40:04.272 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repository Size 2024-05-17T21:40:04.272 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-05-17T21:40:04.272 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-05-17T21:40:04.273 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-test x86_64 2:17.2.7-952.g9903f776.el9 ceph 48 M 2024-05-17T21:40:04.273 INFO:teuthology.orchestra.run.smithi107.stdout:Installing dependencies: 2024-05-17T21:40:04.273 INFO:teuthology.orchestra.run.smithi107.stdout: libcephsqlite x86_64 2:17.2.7-952.g9903f776.el9 ceph 163 k 2024-05-17T21:40:04.273 INFO:teuthology.orchestra.run.smithi107.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-17T21:40:04.273 INFO:teuthology.orchestra.run.smithi107.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-17T21:40:04.273 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:40:04.273 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-05-17T21:40:04.273 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-05-17T21:40:04.273 INFO:teuthology.orchestra.run.smithi107.stdout:Install 4 Packages 2024-05-17T21:40:04.273 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:40:04.274 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 48 M 2024-05-17T21:40:04.274 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 192 M 2024-05-17T21:40:04.274 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-05-17T21:40:04.663 INFO:teuthology.orchestra.run.smithi107.stdout:(1/4): socat-1.7.4.1-5.el9.x86_64.rpm 983 kB/s | 305 kB 00:00 2024-05-17T21:40:04.729 INFO:teuthology.orchestra.run.smithi107.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 958 kB/s | 64 kB 00:00 2024-05-17T21:40:05.088 INFO:teuthology.orchestra.run.smithi107.stdout:(3/4): libcephsqlite-17.2.7-952.g9903f776.el9.x 221 kB/s | 163 kB 00:00 2024-05-17T21:40:07.251 INFO:teuthology.orchestra.run.smithi138.stdout:(4/4): ceph-test-17.2.7-952.g9903f776.el9.x86_6 10 MB/s | 48 MB 00:04 2024-05-17T21:40:07.253 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-05-17T21:40:07.254 INFO:teuthology.orchestra.run.smithi138.stdout:Total 10 MB/s | 48 MB 00:04 2024-05-17T21:40:07.281 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-05-17T21:40:07.301 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-05-17T21:40:07.301 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-05-17T21:40:07.663 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-05-17T21:40:07.664 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-05-17T21:40:08.192 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-05-17T21:40:08.261 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-05-17T21:40:08.308 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/4 2024-05-17T21:40:08.336 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libcephsqlite-2:17.2.7-952.g9903f776.el9.x86_64 3/4 2024-05-17T21:40:09.554 INFO:teuthology.orchestra.run.smithi107.stdout:(4/4): ceph-test-17.2.7-952.g9903f776.el9.x86_6 9.2 MB/s | 48 MB 00:05 2024-05-17T21:40:09.556 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-05-17T21:40:09.557 INFO:teuthology.orchestra.run.smithi107.stdout:Total 9.2 MB/s | 48 MB 00:05 2024-05-17T21:40:09.584 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-05-17T21:40:09.603 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-05-17T21:40:09.603 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-05-17T21:40:09.962 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-05-17T21:40:09.963 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-05-17T21:40:10.499 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-05-17T21:40:10.572 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-05-17T21:40:10.619 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/4 2024-05-17T21:40:10.647 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libcephsqlite-2:17.2.7-952.g9903f776.el9.x86_64 3/4 2024-05-17T21:40:12.760 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: libcephsqlite-2:17.2.7-952.g9903f776.el9.x86_64 3/4 2024-05-17T21:40:12.842 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-test-2:17.2.7-952.g9903f776.el9.x86_64 4/4 2024-05-17T21:40:15.202 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-test-2:17.2.7-952.g9903f776.el9.x86_64 4/4 2024-05-17T21:40:15.202 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-test-2:17.2.7-952.g9903f776.el9.x86_64 1/4 2024-05-17T21:40:15.202 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libcephsqlite-2:17.2.7-952.g9903f776.el9.x86_64 2/4 2024-05-17T21:40:15.202 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 3/4 2024-05-17T21:40:15.308 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: libcephsqlite-2:17.2.7-952.g9903f776.el9.x86_64 3/4 2024-05-17T21:40:15.386 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-test-2:17.2.7-952.g9903f776.el9.x86_64 4/4 2024-05-17T21:40:15.496 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-05-17T21:40:15.497 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-17T21:40:15.497 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-05-17T21:40:15.497 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-test-2:17.2.7-952.g9903f776.el9.x86_64 2024-05-17T21:40:15.497 INFO:teuthology.orchestra.run.smithi138.stdout: libcephsqlite-2:17.2.7-952.g9903f776.el9.x86_64 2024-05-17T21:40:15.497 INFO:teuthology.orchestra.run.smithi138.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-17T21:40:15.497 INFO:teuthology.orchestra.run.smithi138.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-17T21:40:15.497 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-17T21:40:15.497 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-05-17T21:40:15.741 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph 2024-05-17T21:40:16.349 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:00:41 ago on Fri 17 May 2024 09:39:35 PM UTC. 2024-05-17T21:40:16.476 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-05-17T21:40:16.478 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-05-17T21:40:16.478 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repository Size 2024-05-17T21:40:16.478 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-05-17T21:40:16.478 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-05-17T21:40:16.478 INFO:teuthology.orchestra.run.smithi138.stdout: ceph x86_64 2:17.2.7-952.g9903f776.el9 ceph 6.5 k 2024-05-17T21:40:16.478 INFO:teuthology.orchestra.run.smithi138.stdout:Installing dependencies: 2024-05-17T21:40:16.478 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mds x86_64 2:17.2.7-952.g9903f776.el9 ceph 2.1 M 2024-05-17T21:40:16.479 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr x86_64 2:17.2.7-952.g9903f776.el9 ceph 1.4 M 2024-05-17T21:40:16.479 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-modules-core noarch 2:17.2.7-952.g9903f776.el9 ceph-noarch 240 k 2024-05-17T21:40:16.479 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mon x86_64 2:17.2.7-952.g9903f776.el9 ceph 3.9 M 2024-05-17T21:40:16.479 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-osd x86_64 2:17.2.7-952.g9903f776.el9 ceph 15 M 2024-05-17T21:40:16.479 INFO:teuthology.orchestra.run.smithi138.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-17T21:40:16.479 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-17T21:40:16.479 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-17T21:40:16.479 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-17T21:40:16.479 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-17T21:40:16.479 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-17T21:40:16.479 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-17T21:40:16.479 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-17T21:40:16.479 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-17T21:40:16.480 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-17T21:40:16.480 INFO:teuthology.orchestra.run.smithi138.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-17T21:40:16.480 INFO:teuthology.orchestra.run.smithi138.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-17T21:40:16.480 INFO:teuthology.orchestra.run.smithi138.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-17T21:40:16.480 INFO:teuthology.orchestra.run.smithi138.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-17T21:40:16.480 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-17T21:40:16.480 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-17T21:40:16.480 INFO:teuthology.orchestra.run.smithi138.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-17T21:40:16.480 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-17T21:40:16.480 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-17T21:40:16.480 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-17T21:40:16.480 INFO:teuthology.orchestra.run.smithi138.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-17T21:40:16.480 INFO:teuthology.orchestra.run.smithi138.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-17T21:40:16.480 INFO:teuthology.orchestra.run.smithi138.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-17T21:40:16.481 INFO:teuthology.orchestra.run.smithi138.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-17T21:40:16.481 INFO:teuthology.orchestra.run.smithi138.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-17T21:40:16.481 INFO:teuthology.orchestra.run.smithi138.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-17T21:40:16.481 INFO:teuthology.orchestra.run.smithi138.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-17T21:40:16.481 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-17T21:40:16.481 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-05-17T21:40:16.481 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-05-17T21:40:16.481 INFO:teuthology.orchestra.run.smithi138.stdout:Install 33 Packages 2024-05-17T21:40:16.481 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-17T21:40:16.482 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 27 M 2024-05-17T21:40:16.482 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 96 M 2024-05-17T21:40:16.482 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-05-17T21:40:17.405 INFO:teuthology.orchestra.run.smithi138.stdout:(1/33): ceph-17.2.7-952.g9903f776.el9.x86_64.rp 28 kB/s | 6.5 kB 00:00 2024-05-17T21:40:17.564 INFO:teuthology.orchestra.run.smithi138.stdout:(2/33): ceph-mds-17.2.7-952.g9903f776.el9.x86_6 5.3 MB/s | 2.1 MB 00:00 2024-05-17T21:40:17.691 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-test-2:17.2.7-952.g9903f776.el9.x86_64 4/4 2024-05-17T21:40:17.691 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-test-2:17.2.7-952.g9903f776.el9.x86_64 1/4 2024-05-17T21:40:17.691 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libcephsqlite-2:17.2.7-952.g9903f776.el9.x86_64 2/4 2024-05-17T21:40:17.691 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 3/4 2024-05-17T21:40:17.697 INFO:teuthology.orchestra.run.smithi138.stdout:(3/33): ceph-mgr-17.2.7-952.g9903f776.el9.x86_6 2.7 MB/s | 1.4 MB 00:00 2024-05-17T21:40:17.781 INFO:teuthology.orchestra.run.smithi138.stdout:(4/33): ceph-mgr-modules-core-17.2.7-952.g9903f 2.8 MB/s | 240 kB 00:00 2024-05-17T21:40:17.948 INFO:teuthology.orchestra.run.smithi138.stdout:(5/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.5 MB/s | 253 kB 00:00 2024-05-17T21:40:17.959 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-05-17T21:40:17.959 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:40:17.959 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-05-17T21:40:17.959 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-test-2:17.2.7-952.g9903f776.el9.x86_64 2024-05-17T21:40:17.960 INFO:teuthology.orchestra.run.smithi107.stdout: libcephsqlite-2:17.2.7-952.g9903f776.el9.x86_64 2024-05-17T21:40:17.960 INFO:teuthology.orchestra.run.smithi107.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-17T21:40:17.960 INFO:teuthology.orchestra.run.smithi107.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-17T21:40:17.960 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:40:17.960 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-05-17T21:40:18.040 INFO:teuthology.orchestra.run.smithi138.stdout:(6/33): ceph-mon-17.2.7-952.g9903f776.el9.x86_6 6.2 MB/s | 3.9 MB 00:00 2024-05-17T21:40:18.099 INFO:teuthology.orchestra.run.smithi138.stdout:(7/33): python3-cryptography-36.0.1-4.el9.x86_6 8.2 MB/s | 1.2 MB 00:00 2024-05-17T21:40:18.149 INFO:teuthology.orchestra.run.smithi138.stdout:(8/33): python3-pycparser-2.20-6.el9.noarch.rpm 2.7 MB/s | 135 kB 00:00 2024-05-17T21:40:18.174 INFO:teuthology.orchestra.run.smithi138.stdout:(9/33): python3-ply-3.11-14.el9.noarch.rpm 796 kB/s | 106 kB 00:00 2024-05-17T21:40:18.199 INFO:teuthology.orchestra.run.smithi138.stdout:(10/33): python3-requests-2.25.1-8.el9.noarch.r 2.4 MB/s | 125 kB 00:00 2024-05-17T21:40:18.245 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install ceph 2024-05-17T21:40:18.250 INFO:teuthology.orchestra.run.smithi138.stdout:(11/33): python3-urllib3-1.26.5-5.el9.noarch.rp 2.8 MB/s | 215 kB 00:00 2024-05-17T21:40:18.359 INFO:teuthology.orchestra.run.smithi138.stdout:(12/33): python3-markupsafe-1.1.1-12.el9.x86_64 320 kB/s | 35 kB 00:00 2024-05-17T21:40:18.384 INFO:teuthology.orchestra.run.smithi138.stdout:(13/33): python3-mako-1.1.4-6.el9.noarch.rpm 936 kB/s | 172 kB 00:00 2024-05-17T21:40:18.543 INFO:teuthology.orchestra.run.smithi138.stdout:(14/33): ceph-osd-17.2.7-952.g9903f776.el9.x86_ 16 MB/s | 15 MB 00:00 2024-05-17T21:40:18.568 INFO:teuthology.orchestra.run.smithi138.stdout:(15/33): python3-pytz-2021.1-5.el9.noarch.rpm 244 kB/s | 51 kB 00:00 2024-05-17T21:40:18.593 INFO:teuthology.orchestra.run.smithi138.stdout:(16/33): python3-toml-0.10.2-6.el9.noarch.rpm 199 kB/s | 42 kB 00:00 2024-05-17T21:40:18.777 INFO:teuthology.orchestra.run.smithi138.stdout:(17/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 186 kB/s | 43 kB 00:00 2024-05-17T21:40:18.823 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:00:45 ago on Fri 17 May 2024 09:39:33 PM UTC. 2024-05-17T21:40:18.827 INFO:teuthology.orchestra.run.smithi138.stdout:(18/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 214 kB/s | 11 kB 00:00 2024-05-17T21:40:18.886 INFO:teuthology.orchestra.run.smithi138.stdout:(19/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 541 kB/s | 172 kB 00:00 2024-05-17T21:40:18.911 INFO:teuthology.orchestra.run.smithi138.stdout:(20/33): python3-cherrypy-18.6.1-2.el9.noarch.r 1.3 MB/s | 358 kB 00:00 2024-05-17T21:40:18.936 INFO:teuthology.orchestra.run.smithi138.stdout:(21/33): python3-jaraco-classes-3.2.1-5.el9.noa 163 kB/s | 18 kB 00:00 2024-05-17T21:40:18.957 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-05-17T21:40:18.959 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-05-17T21:40:18.959 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repository Size 2024-05-17T21:40:18.959 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-05-17T21:40:18.959 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-05-17T21:40:18.959 INFO:teuthology.orchestra.run.smithi107.stdout: ceph x86_64 2:17.2.7-952.g9903f776.el9 ceph 6.5 k 2024-05-17T21:40:18.960 INFO:teuthology.orchestra.run.smithi107.stdout:Installing dependencies: 2024-05-17T21:40:18.960 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mds x86_64 2:17.2.7-952.g9903f776.el9 ceph 2.1 M 2024-05-17T21:40:18.960 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr x86_64 2:17.2.7-952.g9903f776.el9 ceph 1.4 M 2024-05-17T21:40:18.960 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-modules-core noarch 2:17.2.7-952.g9903f776.el9 ceph-noarch 240 k 2024-05-17T21:40:18.960 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mon x86_64 2:17.2.7-952.g9903f776.el9 ceph 3.9 M 2024-05-17T21:40:18.960 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-osd x86_64 2:17.2.7-952.g9903f776.el9 ceph 15 M 2024-05-17T21:40:18.960 INFO:teuthology.orchestra.run.smithi107.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-17T21:40:18.960 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-17T21:40:18.960 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-17T21:40:18.960 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-17T21:40:18.960 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-17T21:40:18.960 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-17T21:40:18.960 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-17T21:40:18.960 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-17T21:40:18.961 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-17T21:40:18.961 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-17T21:40:18.961 INFO:teuthology.orchestra.run.smithi107.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-17T21:40:18.961 INFO:teuthology.orchestra.run.smithi107.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-17T21:40:18.961 INFO:teuthology.orchestra.run.smithi107.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-17T21:40:18.961 INFO:teuthology.orchestra.run.smithi107.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-17T21:40:18.961 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-17T21:40:18.961 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-17T21:40:18.961 INFO:teuthology.orchestra.run.smithi107.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-17T21:40:18.961 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-17T21:40:18.961 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-17T21:40:18.961 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-17T21:40:18.962 INFO:teuthology.orchestra.run.smithi107.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-17T21:40:18.962 INFO:teuthology.orchestra.run.smithi107.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-17T21:40:18.962 INFO:teuthology.orchestra.run.smithi107.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-17T21:40:18.962 INFO:teuthology.orchestra.run.smithi107.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-17T21:40:18.962 INFO:teuthology.orchestra.run.smithi107.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-17T21:40:18.962 INFO:teuthology.orchestra.run.smithi107.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-17T21:40:18.962 INFO:teuthology.orchestra.run.smithi107.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-17T21:40:18.962 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:40:18.962 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-05-17T21:40:18.962 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-05-17T21:40:18.962 INFO:teuthology.orchestra.run.smithi107.stdout:Install 33 Packages 2024-05-17T21:40:18.962 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:40:18.963 INFO:teuthology.orchestra.run.smithi138.stdout:(22/33): python3-jaraco-collections-3.0.0-8.el9 309 kB/s | 23 kB 00:00 2024-05-17T21:40:18.963 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 27 M 2024-05-17T21:40:18.963 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 96 M 2024-05-17T21:40:18.963 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-05-17T21:40:19.003 INFO:teuthology.orchestra.run.smithi138.stdout:(23/33): python3-jaraco-functools-3.5.0-2.el9.n 290 kB/s | 19 kB 00:00 2024-05-17T21:40:19.028 INFO:teuthology.orchestra.run.smithi138.stdout:(24/33): python3-jaraco-text-3.2.0-6.el9.noarch 214 kB/s | 20 kB 00:00 2024-05-17T21:40:19.053 INFO:teuthology.orchestra.run.smithi138.stdout:(25/33): python3-logutils-0.3.5-21.el9.noarch.r 503 kB/s | 46 kB 00:00 2024-05-17T21:40:19.079 INFO:teuthology.orchestra.run.smithi138.stdout:(26/33): python3-more-itertools-8.12.0-2.el9.no 1.5 MB/s | 79 kB 00:00 2024-05-17T21:40:19.121 INFO:teuthology.orchestra.run.smithi138.stdout:(27/33): python3-pecan-1.4.2-3.el9.noarch.rpm 2.9 MB/s | 272 kB 00:00 2024-05-17T21:40:19.146 INFO:teuthology.orchestra.run.smithi138.stdout:(28/33): python3-portend-3.1.0-2.el9.noarch.rpm 178 kB/s | 16 kB 00:00 2024-05-17T21:40:19.171 INFO:teuthology.orchestra.run.smithi138.stdout:(29/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 979 kB/s | 90 kB 00:00 2024-05-17T21:40:19.197 INFO:teuthology.orchestra.run.smithi138.stdout:(30/33): python3-tempora-5.0.0-2.el9.noarch.rpm 475 kB/s | 36 kB 00:00 2024-05-17T21:40:19.222 INFO:teuthology.orchestra.run.smithi138.stdout:(31/33): python3-webob-1.8.7-6.el9.noarch.rpm 4.5 MB/s | 230 kB 00:00 2024-05-17T21:40:19.247 INFO:teuthology.orchestra.run.smithi138.stdout:(32/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 5.5 MB/s | 427 kB 00:00 2024-05-17T21:40:19.264 INFO:teuthology.orchestra.run.smithi138.stdout:(33/33): python3-zc-lockfile-2.0-10.el9.noarch. 302 kB/s | 20 kB 00:00 2024-05-17T21:40:19.269 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-05-17T21:40:19.269 INFO:teuthology.orchestra.run.smithi138.stdout:Total 9.7 MB/s | 27 MB 00:02 2024-05-17T21:40:19.384 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-05-17T21:40:19.420 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-05-17T21:40:19.420 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-05-17T21:40:19.422 INFO:teuthology.orchestra.run.smithi107.stdout:(1/33): ceph-17.2.7-952.g9903f776.el9.x86_64.rp 33 kB/s | 6.5 kB 00:00 2024-05-17T21:40:19.605 INFO:teuthology.orchestra.run.smithi107.stdout:(2/33): ceph-mgr-17.2.7-952.g9903f776.el9.x86_6 3.7 MB/s | 1.4 MB 00:00 2024-05-17T21:40:19.647 INFO:teuthology.orchestra.run.smithi107.stdout:(3/33): ceph-mds-17.2.7-952.g9903f776.el9.x86_6 4.9 MB/s | 2.1 MB 00:00 2024-05-17T21:40:19.723 INFO:teuthology.orchestra.run.smithi107.stdout:(4/33): ceph-mgr-modules-core-17.2.7-952.g9903f 3.1 MB/s | 240 kB 00:00 2024-05-17T21:40:19.777 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-05-17T21:40:19.778 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-05-17T21:40:19.856 INFO:teuthology.orchestra.run.smithi107.stdout:(5/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.9 MB/s | 253 kB 00:00 2024-05-17T21:40:19.923 INFO:teuthology.orchestra.run.smithi107.stdout:(6/33): ceph-mon-17.2.7-952.g9903f776.el9.x86_6 7.8 MB/s | 3.9 MB 00:00 2024-05-17T21:40:19.974 INFO:teuthology.orchestra.run.smithi107.stdout:(7/33): python3-cryptography-36.0.1-4.el9.x86_6 10 MB/s | 1.2 MB 00:00 2024-05-17T21:40:20.016 INFO:teuthology.orchestra.run.smithi107.stdout:(8/33): python3-pycparser-2.20-6.el9.noarch.rpm 3.2 MB/s | 135 kB 00:00 2024-05-17T21:40:20.041 INFO:teuthology.orchestra.run.smithi107.stdout:(9/33): python3-ply-3.11-14.el9.noarch.rpm 910 kB/s | 106 kB 00:00 2024-05-17T21:40:20.066 INFO:teuthology.orchestra.run.smithi107.stdout:(10/33): python3-requests-2.25.1-8.el9.noarch.r 2.4 MB/s | 125 kB 00:00 2024-05-17T21:40:20.108 INFO:teuthology.orchestra.run.smithi107.stdout:(11/33): python3-urllib3-1.26.5-5.el9.noarch.rp 3.1 MB/s | 215 kB 00:00 2024-05-17T21:40:20.183 INFO:teuthology.orchestra.run.smithi107.stdout:(12/33): python3-markupsafe-1.1.1-12.el9.x86_64 464 kB/s | 35 kB 00:00 2024-05-17T21:40:20.209 INFO:teuthology.orchestra.run.smithi107.stdout:(13/33): python3-mako-1.1.4-6.el9.noarch.rpm 1.2 MB/s | 172 kB 00:00 2024-05-17T21:40:20.242 INFO:teuthology.orchestra.run.smithi107.stdout:(14/33): python3-pytz-2021.1-5.el9.noarch.rpm 868 kB/s | 51 kB 00:00 2024-05-17T21:40:20.259 INFO:teuthology.orchestra.run.smithi107.stdout:(15/33): python3-toml-0.10.2-6.el9.noarch.rpm 829 kB/s | 42 kB 00:00 2024-05-17T21:40:20.360 INFO:teuthology.orchestra.run.smithi107.stdout:(16/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 372 kB/s | 43 kB 00:00 2024-05-17T21:40:20.426 INFO:teuthology.orchestra.run.smithi107.stdout:(17/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.0 MB/s | 172 kB 00:00 2024-05-17T21:40:20.610 INFO:teuthology.orchestra.run.smithi107.stdout:(18/33): ceph-osd-17.2.7-952.g9903f776.el9.x86_ 15 MB/s | 15 MB 00:01 2024-05-17T21:40:20.627 INFO:teuthology.orchestra.run.smithi107.stdout:(19/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 53 kB/s | 11 kB 00:00 2024-05-17T21:40:20.652 INFO:teuthology.orchestra.run.smithi107.stdout:(20/33): python3-cherrypy-18.6.1-2.el9.noarch.r 1.2 MB/s | 358 kB 00:00 2024-05-17T21:40:20.669 INFO:teuthology.orchestra.run.smithi107.stdout:(21/33): python3-jaraco-classes-3.2.1-5.el9.noa 302 kB/s | 18 kB 00:00 2024-05-17T21:40:20.686 INFO:teuthology.orchestra.run.smithi107.stdout:(22/33): python3-jaraco-functools-3.5.0-2.el9.n 577 kB/s | 19 kB 00:00 2024-05-17T21:40:20.703 INFO:teuthology.orchestra.run.smithi107.stdout:(23/33): python3-jaraco-text-3.2.0-6.el9.noarch 582 kB/s | 20 kB 00:00 2024-05-17T21:40:20.720 INFO:teuthology.orchestra.run.smithi107.stdout:(24/33): python3-jaraco-collections-3.0.0-8.el9 250 kB/s | 23 kB 00:00 2024-05-17T21:40:20.737 INFO:teuthology.orchestra.run.smithi107.stdout:(25/33): python3-logutils-0.3.5-21.el9.noarch.r 915 kB/s | 46 kB 00:00 2024-05-17T21:40:20.762 INFO:teuthology.orchestra.run.smithi107.stdout:(26/33): python3-more-itertools-8.12.0-2.el9.no 1.3 MB/s | 79 kB 00:00 2024-05-17T21:40:20.779 INFO:teuthology.orchestra.run.smithi107.stdout:(27/33): python3-portend-3.1.0-2.el9.noarch.rpm 391 kB/s | 16 kB 00:00 2024-05-17T21:40:20.813 INFO:teuthology.orchestra.run.smithi107.stdout:(28/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.7 MB/s | 90 kB 00:00 2024-05-17T21:40:20.830 INFO:teuthology.orchestra.run.smithi107.stdout:(29/33): python3-tempora-5.0.0-2.el9.noarch.rpm 714 kB/s | 36 kB 00:00 2024-05-17T21:40:20.863 INFO:teuthology.orchestra.run.smithi107.stdout:(30/33): python3-pecan-1.4.2-3.el9.noarch.rpm 1.9 MB/s | 272 kB 00:00 2024-05-17T21:40:20.905 INFO:teuthology.orchestra.run.smithi107.stdout:(31/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 5.5 MB/s | 427 kB 00:00 2024-05-17T21:40:20.918 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-05-17T21:40:20.930 INFO:teuthology.orchestra.run.smithi107.stdout:(32/33): python3-webob-1.8.7-6.el9.noarch.rpm 1.9 MB/s | 230 kB 00:00 2024-05-17T21:40:20.947 INFO:teuthology.orchestra.run.smithi107.stdout:(33/33): python3-zc-lockfile-2.0-10.el9.noarch. 239 kB/s | 20 kB 00:00 2024-05-17T21:40:20.948 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-05-17T21:40:20.949 INFO:teuthology.orchestra.run.smithi107.stdout:Total 14 MB/s | 27 MB 00:01 2024-05-17T21:40:20.975 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-05-17T21:40:21.007 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-05-17T21:40:21.049 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-05-17T21:40:21.070 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-05-17T21:40:21.107 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-05-17T21:40:21.108 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-05-17T21:40:21.141 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-05-17T21:40:21.236 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-05-17T21:40:21.277 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 6/33 2024-05-17T21:40:21.312 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-05-17T21:40:21.350 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/33 2024-05-17T21:40:21.384 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/33 2024-05-17T21:40:21.417 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/33 2024-05-17T21:40:21.466 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/33 2024-05-17T21:40:21.481 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-05-17T21:40:21.481 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-05-17T21:40:21.510 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/33 2024-05-17T21:40:21.544 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/33 2024-05-17T21:40:21.582 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/33 2024-05-17T21:40:21.655 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 15/33 2024-05-17T21:40:21.804 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 16/33 2024-05-17T21:40:21.892 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 17/33 2024-05-17T21:40:21.944 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 18/33 2024-05-17T21:40:21.983 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 19/33 2024-05-17T21:40:22.040 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-ply-3.11-14.el9.noarch 20/33 2024-05-17T21:40:22.100 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 21/33 2024-05-17T21:40:22.291 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 22/33 2024-05-17T21:40:22.337 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 23/33 2024-05-17T21:40:22.415 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 24/33 2024-05-17T21:40:22.504 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 25/33 2024-05-17T21:40:22.544 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 26/33 2024-05-17T21:40:22.668 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 27/33 2024-05-17T21:40:22.671 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-05-17T21:40:22.767 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-05-17T21:40:22.799 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-05-17T21:40:22.825 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-952.g9903f776.el9.n 28/33 2024-05-17T21:40:22.831 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-05-17T21:40:22.855 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-2:17.2.7-952.g9903f776.el9.x86_64 29/33 2024-05-17T21:40:22.881 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-2:17.2.7-952.g9903f776.el9.x86_64 29/33 2024-05-17T21:40:22.881 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-17T21:40:22.881 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-17T21:40:22.881 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-17T21:40:22.881 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-17T21:40:22.881 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-17T21:40:22.924 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-05-17T21:40:23.001 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-05-17T21:40:23.042 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 6/33 2024-05-17T21:40:23.086 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-05-17T21:40:23.116 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/33 2024-05-17T21:40:23.142 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/33 2024-05-17T21:40:23.175 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/33 2024-05-17T21:40:23.216 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/33 2024-05-17T21:40:23.251 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/33 2024-05-17T21:40:23.285 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/33 2024-05-17T21:40:23.324 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/33 2024-05-17T21:40:23.396 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 15/33 2024-05-17T21:40:23.535 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 16/33 2024-05-17T21:40:23.615 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 17/33 2024-05-17T21:40:23.668 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 18/33 2024-05-17T21:40:23.707 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 19/33 2024-05-17T21:40:23.756 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-ply-3.11-14.el9.noarch 20/33 2024-05-17T21:40:23.818 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 21/33 2024-05-17T21:40:24.007 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 22/33 2024-05-17T21:40:24.069 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 23/33 2024-05-17T21:40:24.139 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 24/33 2024-05-17T21:40:24.197 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-osd-2:17.2.7-952.g9903f776.el9.x86_64 30/33 2024-05-17T21:40:24.224 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-osd-2:17.2.7-952.g9903f776.el9.x86_64 30/33 2024-05-17T21:40:24.224 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-17T21:40:24.225 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-17T21:40:24.225 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-17T21:40:24.225 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-17T21:40:24.225 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-17T21:40:24.227 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 25/33 2024-05-17T21:40:24.268 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 26/33 2024-05-17T21:40:24.383 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 27/33 2024-05-17T21:40:24.531 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-952.g9903f776.el9.n 28/33 2024-05-17T21:40:24.553 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-2:17.2.7-952.g9903f776.el9.x86_64 29/33 2024-05-17T21:40:24.580 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-2:17.2.7-952.g9903f776.el9.x86_64 29/33 2024-05-17T21:40:24.580 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-17T21:40:24.580 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-17T21:40:24.580 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-17T21:40:24.580 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-17T21:40:24.580 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:40:24.589 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mon-2:17.2.7-952.g9903f776.el9.x86_64 31/33 2024-05-17T21:40:24.613 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mon-2:17.2.7-952.g9903f776.el9.x86_64 31/33 2024-05-17T21:40:24.613 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-17T21:40:24.613 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-17T21:40:24.613 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-17T21:40:24.614 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-17T21:40:24.614 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-17T21:40:24.824 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mds-2:17.2.7-952.g9903f776.el9.x86_64 32/33 2024-05-17T21:40:24.847 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mds-2:17.2.7-952.g9903f776.el9.x86_64 32/33 2024-05-17T21:40:24.848 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-17T21:40:24.848 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-17T21:40:24.848 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-17T21:40:24.848 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-17T21:40:24.848 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-17T21:40:24.911 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-2:17.2.7-952.g9903f776.el9.x86_64 33/33 2024-05-17T21:40:25.896 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-osd-2:17.2.7-952.g9903f776.el9.x86_64 30/33 2024-05-17T21:40:25.923 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-osd-2:17.2.7-952.g9903f776.el9.x86_64 30/33 2024-05-17T21:40:25.924 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-17T21:40:25.924 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-17T21:40:25.924 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-17T21:40:25.924 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-17T21:40:25.924 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:40:26.288 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mon-2:17.2.7-952.g9903f776.el9.x86_64 31/33 2024-05-17T21:40:26.312 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mon-2:17.2.7-952.g9903f776.el9.x86_64 31/33 2024-05-17T21:40:26.312 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-17T21:40:26.312 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-17T21:40:26.312 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-17T21:40:26.312 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-17T21:40:26.312 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:40:26.515 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mds-2:17.2.7-952.g9903f776.el9.x86_64 32/33 2024-05-17T21:40:26.538 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mds-2:17.2.7-952.g9903f776.el9.x86_64 32/33 2024-05-17T21:40:26.538 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-17T21:40:26.538 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-17T21:40:26.538 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-17T21:40:26.539 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-17T21:40:26.539 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:40:26.602 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-2:17.2.7-952.g9903f776.el9.x86_64 33/33 2024-05-17T21:40:26.791 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-2:17.2.7-952.g9903f776.el9.x86_64 33/33 2024-05-17T21:40:26.791 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-2:17.2.7-952.g9903f776.el9.x86_64 1/33 2024-05-17T21:40:26.791 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mds-2:17.2.7-952.g9903f776.el9.x86_64 2/33 2024-05-17T21:40:26.791 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-2:17.2.7-952.g9903f776.el9.x86_64 3/33 2024-05-17T21:40:26.791 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mon-2:17.2.7-952.g9903f776.el9.x86_64 4/33 2024-05-17T21:40:26.791 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-osd-2:17.2.7-952.g9903f776.el9.x86_64 5/33 2024-05-17T21:40:26.791 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-952.g9903f776.el9.n 6/33 2024-05-17T21:40:26.791 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/33 2024-05-17T21:40:26.791 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/33 2024-05-17T21:40:26.791 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/33 2024-05-17T21:40:26.792 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/33 2024-05-17T21:40:26.792 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/33 2024-05-17T21:40:26.792 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 12/33 2024-05-17T21:40:26.792 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/33 2024-05-17T21:40:26.792 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/33 2024-05-17T21:40:26.792 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 15/33 2024-05-17T21:40:26.792 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 16/33 2024-05-17T21:40:26.792 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-05-17T21:40:26.792 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-05-17T21:40:26.792 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-05-17T21:40:26.792 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-05-17T21:40:26.792 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-05-17T21:40:26.792 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-05-17T21:40:26.792 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-05-17T21:40:26.793 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-05-17T21:40:26.793 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-05-17T21:40:26.793 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-05-17T21:40:26.793 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-05-17T21:40:26.793 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-05-17T21:40:26.793 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-05-17T21:40:26.793 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-05-17T21:40:26.794 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 31/33 2024-05-17T21:40:26.794 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-05-17T21:40:27.711 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-05-17T21:40:27.711 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-17T21:40:27.711 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-05-17T21:40:27.711 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-2:17.2.7-952.g9903f776.el9.x86_64 2024-05-17T21:40:27.711 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mds-2:17.2.7-952.g9903f776.el9.x86_64 2024-05-17T21:40:27.711 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-2:17.2.7-952.g9903f776.el9.x86_64 2024-05-17T21:40:27.711 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-modules-core-2:17.2.7-952.g9903f776.el9.noarch 2024-05-17T21:40:27.711 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mon-2:17.2.7-952.g9903f776.el9.x86_64 2024-05-17T21:40:27.711 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-osd-2:17.2.7-952.g9903f776.el9.x86_64 2024-05-17T21:40:27.711 INFO:teuthology.orchestra.run.smithi138.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-17T21:40:27.711 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-17T21:40:27.712 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-17T21:40:27.712 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-17T21:40:27.712 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-17T21:40:27.712 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-17T21:40:27.712 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-17T21:40:27.712 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-17T21:40:27.712 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-17T21:40:27.712 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-17T21:40:27.712 INFO:teuthology.orchestra.run.smithi138.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-17T21:40:27.712 INFO:teuthology.orchestra.run.smithi138.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-17T21:40:27.712 INFO:teuthology.orchestra.run.smithi138.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-17T21:40:27.712 INFO:teuthology.orchestra.run.smithi138.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-17T21:40:27.712 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-17T21:40:27.712 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ply-3.11-14.el9.noarch 2024-05-17T21:40:27.712 INFO:teuthology.orchestra.run.smithi138.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-17T21:40:27.713 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-17T21:40:27.713 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-17T21:40:27.713 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-17T21:40:27.713 INFO:teuthology.orchestra.run.smithi138.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-17T21:40:27.713 INFO:teuthology.orchestra.run.smithi138.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-17T21:40:27.713 INFO:teuthology.orchestra.run.smithi138.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-17T21:40:27.713 INFO:teuthology.orchestra.run.smithi138.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-17T21:40:27.713 INFO:teuthology.orchestra.run.smithi138.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-17T21:40:27.713 INFO:teuthology.orchestra.run.smithi138.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-17T21:40:27.713 INFO:teuthology.orchestra.run.smithi138.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-17T21:40:27.713 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-17T21:40:27.713 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-05-17T21:40:28.063 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph-base 2024-05-17T21:40:28.257 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-2:17.2.7-952.g9903f776.el9.x86_64 33/33 2024-05-17T21:40:28.257 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-2:17.2.7-952.g9903f776.el9.x86_64 1/33 2024-05-17T21:40:28.257 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mds-2:17.2.7-952.g9903f776.el9.x86_64 2/33 2024-05-17T21:40:28.257 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-2:17.2.7-952.g9903f776.el9.x86_64 3/33 2024-05-17T21:40:28.257 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mon-2:17.2.7-952.g9903f776.el9.x86_64 4/33 2024-05-17T21:40:28.257 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-osd-2:17.2.7-952.g9903f776.el9.x86_64 5/33 2024-05-17T21:40:28.257 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-952.g9903f776.el9.n 6/33 2024-05-17T21:40:28.257 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/33 2024-05-17T21:40:28.257 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/33 2024-05-17T21:40:28.257 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/33 2024-05-17T21:40:28.258 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/33 2024-05-17T21:40:28.258 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/33 2024-05-17T21:40:28.258 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 12/33 2024-05-17T21:40:28.258 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/33 2024-05-17T21:40:28.258 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/33 2024-05-17T21:40:28.258 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 15/33 2024-05-17T21:40:28.258 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 16/33 2024-05-17T21:40:28.258 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-05-17T21:40:28.258 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-05-17T21:40:28.258 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-05-17T21:40:28.258 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-05-17T21:40:28.259 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-05-17T21:40:28.259 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-05-17T21:40:28.259 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-05-17T21:40:28.259 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-05-17T21:40:28.259 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-05-17T21:40:28.259 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-05-17T21:40:28.259 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-05-17T21:40:28.259 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-05-17T21:40:28.260 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-05-17T21:40:28.260 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-05-17T21:40:28.260 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 31/33 2024-05-17T21:40:28.260 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-05-17T21:40:28.635 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:00:53 ago on Fri 17 May 2024 09:39:35 PM UTC. 2024-05-17T21:40:28.714 INFO:teuthology.orchestra.run.smithi138.stdout:Package ceph-base-2:17.2.7-952.g9903f776.el9.x86_64 is already installed. 2024-05-17T21:40:28.755 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-05-17T21:40:28.756 INFO:teuthology.orchestra.run.smithi138.stdout:Nothing to do. 2024-05-17T21:40:28.756 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-05-17T21:40:28.836 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install cephadm 2024-05-17T21:40:29.193 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-05-17T21:40:29.193 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:40:29.194 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-05-17T21:40:29.194 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-2:17.2.7-952.g9903f776.el9.x86_64 2024-05-17T21:40:29.194 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mds-2:17.2.7-952.g9903f776.el9.x86_64 2024-05-17T21:40:29.194 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-2:17.2.7-952.g9903f776.el9.x86_64 2024-05-17T21:40:29.194 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-modules-core-2:17.2.7-952.g9903f776.el9.noarch 2024-05-17T21:40:29.194 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mon-2:17.2.7-952.g9903f776.el9.x86_64 2024-05-17T21:40:29.194 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-osd-2:17.2.7-952.g9903f776.el9.x86_64 2024-05-17T21:40:29.194 INFO:teuthology.orchestra.run.smithi107.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-17T21:40:29.194 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-17T21:40:29.194 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-17T21:40:29.194 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-17T21:40:29.194 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-17T21:40:29.194 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-17T21:40:29.195 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-17T21:40:29.195 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-17T21:40:29.195 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-17T21:40:29.195 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-17T21:40:29.195 INFO:teuthology.orchestra.run.smithi107.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-17T21:40:29.195 INFO:teuthology.orchestra.run.smithi107.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-17T21:40:29.195 INFO:teuthology.orchestra.run.smithi107.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-17T21:40:29.195 INFO:teuthology.orchestra.run.smithi107.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-17T21:40:29.195 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-17T21:40:29.195 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ply-3.11-14.el9.noarch 2024-05-17T21:40:29.195 INFO:teuthology.orchestra.run.smithi107.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-17T21:40:29.195 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-17T21:40:29.195 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-17T21:40:29.195 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-17T21:40:29.196 INFO:teuthology.orchestra.run.smithi107.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-17T21:40:29.196 INFO:teuthology.orchestra.run.smithi107.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-17T21:40:29.196 INFO:teuthology.orchestra.run.smithi107.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-17T21:40:29.196 INFO:teuthology.orchestra.run.smithi107.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-17T21:40:29.196 INFO:teuthology.orchestra.run.smithi107.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-17T21:40:29.196 INFO:teuthology.orchestra.run.smithi107.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-17T21:40:29.196 INFO:teuthology.orchestra.run.smithi107.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-17T21:40:29.196 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:40:29.196 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-05-17T21:40:29.406 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:00:54 ago on Fri 17 May 2024 09:39:35 PM UTC. 2024-05-17T21:40:29.528 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-05-17T21:40:29.529 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-05-17T21:40:29.529 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repository Size 2024-05-17T21:40:29.529 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-05-17T21:40:29.529 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-05-17T21:40:29.529 INFO:teuthology.orchestra.run.smithi138.stdout: cephadm noarch 2:17.2.7-952.g9903f776.el9 ceph-noarch 82 k 2024-05-17T21:40:29.529 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-17T21:40:29.529 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-05-17T21:40:29.529 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-05-17T21:40:29.529 INFO:teuthology.orchestra.run.smithi138.stdout:Install 1 Package 2024-05-17T21:40:29.529 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-17T21:40:29.529 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 82 k 2024-05-17T21:40:29.530 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 372 k 2024-05-17T21:40:29.530 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-05-17T21:40:29.568 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install ceph-base 2024-05-17T21:40:29.875 INFO:teuthology.orchestra.run.smithi138.stdout:cephadm-17.2.7-952.g9903f776.el9.noarch.rpm 238 kB/s | 82 kB 00:00 2024-05-17T21:40:29.876 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-05-17T21:40:29.876 INFO:teuthology.orchestra.run.smithi138.stdout:Total 237 kB/s | 82 kB 00:00 2024-05-17T21:40:29.876 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-05-17T21:40:29.881 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-05-17T21:40:29.881 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-05-17T21:40:29.883 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-05-17T21:40:29.883 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-05-17T21:40:29.952 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-05-17T21:40:30.147 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:00:57 ago on Fri 17 May 2024 09:39:33 PM UTC. 2024-05-17T21:40:30.226 INFO:teuthology.orchestra.run.smithi107.stdout:Package ceph-base-2:17.2.7-952.g9903f776.el9.x86_64 is already installed. 2024-05-17T21:40:30.268 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-05-17T21:40:30.269 INFO:teuthology.orchestra.run.smithi107.stdout:Nothing to do. 2024-05-17T21:40:30.269 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-05-17T21:40:30.337 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install cephadm 2024-05-17T21:40:30.392 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: cephadm-2:17.2.7-952.g9903f776.el9.noarch 1/1 2024-05-17T21:40:30.487 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : cephadm-2:17.2.7-952.g9903f776.el9.noarch 1/1 2024-05-17T21:40:30.818 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: cephadm-2:17.2.7-952.g9903f776.el9.noarch 1/1 2024-05-17T21:40:30.899 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:00:57 ago on Fri 17 May 2024 09:39:33 PM UTC. 2024-05-17T21:40:31.020 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-05-17T21:40:31.021 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-05-17T21:40:31.021 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repository Size 2024-05-17T21:40:31.021 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-05-17T21:40:31.021 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-05-17T21:40:31.021 INFO:teuthology.orchestra.run.smithi107.stdout: cephadm noarch 2:17.2.7-952.g9903f776.el9 ceph-noarch 82 k 2024-05-17T21:40:31.021 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:40:31.021 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-05-17T21:40:31.021 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-05-17T21:40:31.022 INFO:teuthology.orchestra.run.smithi107.stdout:Install 1 Package 2024-05-17T21:40:31.022 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:40:31.022 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 82 k 2024-05-17T21:40:31.022 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 372 k 2024-05-17T21:40:31.022 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-05-17T21:40:31.054 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : cephadm-2:17.2.7-952.g9903f776.el9.noarch 1/1 2024-05-17T21:40:31.054 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-17T21:40:31.054 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-05-17T21:40:31.054 INFO:teuthology.orchestra.run.smithi138.stdout: cephadm-2:17.2.7-952.g9903f776.el9.noarch 2024-05-17T21:40:31.054 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-17T21:40:31.054 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-05-17T21:40:31.263 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph-immutable-object-cache 2024-05-17T21:40:31.596 INFO:teuthology.orchestra.run.smithi107.stdout:cephadm-17.2.7-952.g9903f776.el9.noarch.rpm 143 kB/s | 82 kB 00:00 2024-05-17T21:40:31.596 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-05-17T21:40:31.597 INFO:teuthology.orchestra.run.smithi107.stdout:Total 142 kB/s | 82 kB 00:00 2024-05-17T21:40:31.597 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-05-17T21:40:31.602 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-05-17T21:40:31.602 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-05-17T21:40:31.604 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-05-17T21:40:31.605 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-05-17T21:40:31.673 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-05-17T21:40:31.845 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:00:56 ago on Fri 17 May 2024 09:39:35 PM UTC. 2024-05-17T21:40:31.973 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-05-17T21:40:31.973 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-05-17T21:40:31.973 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repo Size 2024-05-17T21:40:31.973 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-05-17T21:40:31.974 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-05-17T21:40:31.974 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-952.g9903f776.el9 ceph 137 k 2024-05-17T21:40:31.974 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-17T21:40:31.974 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-05-17T21:40:31.974 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-05-17T21:40:31.974 INFO:teuthology.orchestra.run.smithi138.stdout:Install 1 Package 2024-05-17T21:40:31.974 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-17T21:40:31.974 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 137 k 2024-05-17T21:40:31.975 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 379 k 2024-05-17T21:40:31.975 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-05-17T21:40:32.106 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: cephadm-2:17.2.7-952.g9903f776.el9.noarch 1/1 2024-05-17T21:40:32.201 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : cephadm-2:17.2.7-952.g9903f776.el9.noarch 1/1 2024-05-17T21:40:32.527 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: cephadm-2:17.2.7-952.g9903f776.el9.noarch 1/1 2024-05-17T21:40:32.572 INFO:teuthology.orchestra.run.smithi138.stdout:ceph-immutable-object-cache-17.2.7-952.g9903f77 230 kB/s | 137 kB 00:00 2024-05-17T21:40:32.573 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-05-17T21:40:32.573 INFO:teuthology.orchestra.run.smithi138.stdout:Total 229 kB/s | 137 kB 00:00 2024-05-17T21:40:32.574 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-05-17T21:40:32.581 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-05-17T21:40:32.581 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-05-17T21:40:32.639 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-05-17T21:40:32.639 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-05-17T21:40:32.774 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : cephadm-2:17.2.7-952.g9903f776.el9.noarch 1/1 2024-05-17T21:40:32.775 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:40:32.775 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-05-17T21:40:32.775 INFO:teuthology.orchestra.run.smithi107.stdout: cephadm-2:17.2.7-952.g9903f776.el9.noarch 2024-05-17T21:40:32.775 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:40:32.775 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-05-17T21:40:32.781 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-05-17T21:40:32.874 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-952.g9903f776.e 1/1 2024-05-17T21:40:32.900 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-952.g9903f776.e 1/1 2024-05-17T21:40:32.900 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-17T21:40:32.900 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-17T21:40:32.900 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-17T21:40:32.993 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install ceph-immutable-object-cache 2024-05-17T21:40:33.572 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:01:00 ago on Fri 17 May 2024 09:39:33 PM UTC. 2024-05-17T21:40:33.698 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-05-17T21:40:33.698 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-05-17T21:40:33.698 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repo Size 2024-05-17T21:40:33.699 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-05-17T21:40:33.699 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-05-17T21:40:33.699 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-952.g9903f776.el9 ceph 137 k 2024-05-17T21:40:33.699 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:40:33.699 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-05-17T21:40:33.699 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-05-17T21:40:33.699 INFO:teuthology.orchestra.run.smithi107.stdout:Install 1 Package 2024-05-17T21:40:33.699 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:40:33.700 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 137 k 2024-05-17T21:40:33.700 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 379 k 2024-05-17T21:40:33.700 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-05-17T21:40:33.804 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-952.g9903f776.e 1/1 2024-05-17T21:40:33.804 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-17T21:40:33.804 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-05-17T21:40:33.804 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-immutable-object-cache-2:17.2.7-952.g9903f776.el9.x86_64 2024-05-17T21:40:33.804 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-17T21:40:33.804 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-05-17T21:40:34.007 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph-mgr 2024-05-17T21:40:34.304 INFO:teuthology.orchestra.run.smithi107.stdout:ceph-immutable-object-cache-17.2.7-952.g9903f77 228 kB/s | 137 kB 00:00 2024-05-17T21:40:34.304 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-05-17T21:40:34.305 INFO:teuthology.orchestra.run.smithi107.stdout:Total 227 kB/s | 137 kB 00:00 2024-05-17T21:40:34.305 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-05-17T21:40:34.312 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-05-17T21:40:34.312 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-05-17T21:40:34.369 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-05-17T21:40:34.370 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-05-17T21:40:34.510 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-05-17T21:40:34.588 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-952.g9903f776.e 1/1 2024-05-17T21:40:34.591 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:00:59 ago on Fri 17 May 2024 09:39:35 PM UTC. 2024-05-17T21:40:34.613 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-952.g9903f776.e 1/1 2024-05-17T21:40:34.613 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-17T21:40:34.613 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-17T21:40:34.613 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:40:34.673 INFO:teuthology.orchestra.run.smithi138.stdout:Package ceph-mgr-2:17.2.7-952.g9903f776.el9.x86_64 is already installed. 2024-05-17T21:40:34.713 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-05-17T21:40:34.714 INFO:teuthology.orchestra.run.smithi138.stdout:Nothing to do. 2024-05-17T21:40:34.715 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-05-17T21:40:34.796 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph-mgr-dashboard 2024-05-17T21:40:35.361 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:00 ago on Fri 17 May 2024 09:39:35 PM UTC. 2024-05-17T21:40:35.485 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-05-17T21:40:35.486 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-05-17T21:40:35.486 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repository Size 2024-05-17T21:40:35.486 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-05-17T21:40:35.486 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-05-17T21:40:35.487 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-dashboard noarch 2:17.2.7-952.g9903f776.el9 ceph-noarch 3.4 M 2024-05-17T21:40:35.487 INFO:teuthology.orchestra.run.smithi138.stdout:Installing dependencies: 2024-05-17T21:40:35.487 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-grafana-dashboards noarch 2:17.2.7-952.g9903f776.el9 ceph-noarch 24 k 2024-05-17T21:40:35.487 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-prometheus-alerts noarch 2:17.2.7-952.g9903f776.el9 ceph-noarch 15 k 2024-05-17T21:40:35.487 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-05-17T21:40:35.487 INFO:teuthology.orchestra.run.smithi138.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-17T21:40:35.487 INFO:teuthology.orchestra.run.smithi138.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-17T21:40:35.487 INFO:teuthology.orchestra.run.smithi138.stdout:Installing weak dependencies: 2024-05-17T21:40:35.487 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-05-17T21:40:35.487 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-17T21:40:35.487 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-05-17T21:40:35.487 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-05-17T21:40:35.487 INFO:teuthology.orchestra.run.smithi138.stdout:Install 7 Packages 2024-05-17T21:40:35.488 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-17T21:40:35.488 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 3.7 M 2024-05-17T21:40:35.488 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 80 M 2024-05-17T21:40:35.488 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-05-17T21:40:35.556 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-952.g9903f776.e 1/1 2024-05-17T21:40:35.556 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:40:35.557 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-05-17T21:40:35.557 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-immutable-object-cache-2:17.2.7-952.g9903f776.el9.x86_64 2024-05-17T21:40:35.557 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:40:35.557 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-05-17T21:40:35.800 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install ceph-mgr 2024-05-17T21:40:36.027 INFO:teuthology.orchestra.run.smithi138.stdout:(1/7): ceph-prometheus-alerts-17.2.7-952.g9903f 39 kB/s | 15 kB 00:00 2024-05-17T21:40:36.110 INFO:teuthology.orchestra.run.smithi138.stdout:(2/7): ceph-grafana-dashboards-17.2.7-952.g9903 53 kB/s | 24 kB 00:00 2024-05-17T21:40:36.302 INFO:teuthology.orchestra.run.smithi138.stdout:(3/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 33 kB/s | 9.0 kB 00:00 2024-05-17T21:40:36.372 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:01:03 ago on Fri 17 May 2024 09:39:33 PM UTC. 2024-05-17T21:40:36.394 INFO:teuthology.orchestra.run.smithi138.stdout:(4/7): python3-jwt-2.4.0-1.el9.noarch.rpm 157 kB/s | 41 kB 00:00 2024-05-17T21:40:36.419 INFO:teuthology.orchestra.run.smithi138.stdout:(5/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 263 kB/s | 31 kB 00:00 2024-05-17T21:40:36.455 INFO:teuthology.orchestra.run.smithi107.stdout:Package ceph-mgr-2:17.2.7-952.g9903f776.el9.x86_64 is already installed. 2024-05-17T21:40:36.498 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-05-17T21:40:36.499 INFO:teuthology.orchestra.run.smithi107.stdout:Nothing to do. 2024-05-17T21:40:36.499 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-05-17T21:40:36.578 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install ceph-mgr-dashboard 2024-05-17T21:40:36.578 INFO:teuthology.orchestra.run.smithi138.stdout:(6/7): python3-routes-2.5.1-5.el9.noarch.rpm 1.0 MB/s | 188 kB 00:00 2024-05-17T21:40:37.144 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:01:04 ago on Fri 17 May 2024 09:39:33 PM UTC. 2024-05-17T21:40:37.267 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-05-17T21:40:37.268 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-05-17T21:40:37.268 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repository Size 2024-05-17T21:40:37.268 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-05-17T21:40:37.268 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-05-17T21:40:37.269 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-dashboard noarch 2:17.2.7-952.g9903f776.el9 ceph-noarch 3.4 M 2024-05-17T21:40:37.269 INFO:teuthology.orchestra.run.smithi107.stdout:Installing dependencies: 2024-05-17T21:40:37.269 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-grafana-dashboards noarch 2:17.2.7-952.g9903f776.el9 ceph-noarch 24 k 2024-05-17T21:40:37.269 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-prometheus-alerts noarch 2:17.2.7-952.g9903f776.el9 ceph-noarch 15 k 2024-05-17T21:40:37.269 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-05-17T21:40:37.269 INFO:teuthology.orchestra.run.smithi107.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-17T21:40:37.269 INFO:teuthology.orchestra.run.smithi107.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-17T21:40:37.269 INFO:teuthology.orchestra.run.smithi107.stdout:Installing weak dependencies: 2024-05-17T21:40:37.269 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-05-17T21:40:37.269 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:40:37.269 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-05-17T21:40:37.269 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-05-17T21:40:37.269 INFO:teuthology.orchestra.run.smithi107.stdout:Install 7 Packages 2024-05-17T21:40:37.269 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:40:37.270 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 3.7 M 2024-05-17T21:40:37.270 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 80 M 2024-05-17T21:40:37.270 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-05-17T21:40:37.849 INFO:teuthology.orchestra.run.smithi107.stdout:(1/7): ceph-grafana-dashboards-17.2.7-952.g9903 65 kB/s | 24 kB 00:00 2024-05-17T21:40:37.890 INFO:teuthology.orchestra.run.smithi107.stdout:(2/7): ceph-prometheus-alerts-17.2.7-952.g9903f 35 kB/s | 15 kB 00:00 2024-05-17T21:40:38.024 INFO:teuthology.orchestra.run.smithi107.stdout:(3/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 52 kB/s | 9.0 kB 00:00 2024-05-17T21:40:38.200 INFO:teuthology.orchestra.run.smithi107.stdout:(4/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 175 kB/s | 31 kB 00:00 2024-05-17T21:40:38.225 INFO:teuthology.orchestra.run.smithi107.stdout:(5/7): python3-jwt-2.4.0-1.el9.noarch.rpm 122 kB/s | 41 kB 00:00 2024-05-17T21:40:38.433 INFO:teuthology.orchestra.run.smithi107.stdout:(6/7): python3-routes-2.5.1-5.el9.noarch.rpm 808 kB/s | 188 kB 00:00 2024-05-17T21:40:39.386 INFO:teuthology.orchestra.run.smithi138.stdout:(7/7): ceph-mgr-dashboard-17.2.7-952.g9903f776. 919 kB/s | 3.4 MB 00:03 2024-05-17T21:40:39.390 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-05-17T21:40:39.390 INFO:teuthology.orchestra.run.smithi138.stdout:Total 958 kB/s | 3.7 MB 00:03 2024-05-17T21:40:39.391 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-05-17T21:40:39.400 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-05-17T21:40:39.400 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-05-17T21:40:39.468 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-05-17T21:40:39.468 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-05-17T21:40:39.764 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-05-17T21:40:39.821 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-05-17T21:40:39.854 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-05-17T21:40:39.901 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-05-17T21:40:39.933 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-05-17T21:40:39.962 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-952.g9903f776.el9.no 5/7 2024-05-17T21:40:40.925 INFO:teuthology.orchestra.run.smithi107.stdout:(7/7): ceph-mgr-dashboard-17.2.7-952.g9903f776. 995 kB/s | 3.4 MB 00:03 2024-05-17T21:40:40.927 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-05-17T21:40:40.927 INFO:teuthology.orchestra.run.smithi107.stdout:Total 1.0 MB/s | 3.7 MB 00:03 2024-05-17T21:40:40.927 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-05-17T21:40:40.937 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-05-17T21:40:40.937 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-05-17T21:40:40.963 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-952.g9903f776.el9.n 6/7 2024-05-17T21:40:41.003 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-05-17T21:40:41.004 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-05-17T21:40:41.007 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-952.g9903f776.el9.noarch 7/7 2024-05-17T21:40:41.307 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-05-17T21:40:41.369 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-05-17T21:40:41.401 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-05-17T21:40:41.448 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-05-17T21:40:41.488 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-05-17T21:40:41.517 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-952.g9903f776.el9.no 5/7 2024-05-17T21:40:42.061 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-952.g9903f776.el9.noarch 7/7 2024-05-17T21:40:42.061 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-952.g9903f776.el9.n 1/7 2024-05-17T21:40:42.062 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-952.g9903f776.el9.noarch 2/7 2024-05-17T21:40:42.062 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-952.g9903f776.el9.no 3/7 2024-05-17T21:40:42.062 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-05-17T21:40:42.062 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-05-17T21:40:42.062 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-05-17T21:40:42.328 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-05-17T21:40:42.328 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-17T21:40:42.328 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-05-17T21:40:42.328 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-grafana-dashboards-2:17.2.7-952.g9903f776.el9.noarch 2024-05-17T21:40:42.328 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-dashboard-2:17.2.7-952.g9903f776.el9.noarch 2024-05-17T21:40:42.329 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-prometheus-alerts-2:17.2.7-952.g9903f776.el9.noarch 2024-05-17T21:40:42.329 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-05-17T21:40:42.329 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-05-17T21:40:42.329 INFO:teuthology.orchestra.run.smithi138.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-17T21:40:42.329 INFO:teuthology.orchestra.run.smithi138.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-17T21:40:42.329 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-17T21:40:42.329 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-05-17T21:40:42.509 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-952.g9903f776.el9.n 6/7 2024-05-17T21:40:42.545 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-952.g9903f776.el9.noarch 7/7 2024-05-17T21:40:42.563 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph-mgr-diskprediction-local 2024-05-17T21:40:43.126 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:08 ago on Fri 17 May 2024 09:39:35 PM UTC. 2024-05-17T21:40:43.250 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-05-17T21:40:43.251 INFO:teuthology.orchestra.run.smithi138.stdout:====================================================================================== 2024-05-17T21:40:43.251 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repository Size 2024-05-17T21:40:43.251 INFO:teuthology.orchestra.run.smithi138.stdout:====================================================================================== 2024-05-17T21:40:43.251 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-05-17T21:40:43.251 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-952.g9903f776.el9 ceph-noarch 7.4 M 2024-05-17T21:40:43.251 INFO:teuthology.orchestra.run.smithi138.stdout:Installing dependencies: 2024-05-17T21:40:43.252 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-17T21:40:43.252 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-17T21:40:43.252 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-17T21:40:43.252 INFO:teuthology.orchestra.run.smithi138.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-17T21:40:43.252 INFO:teuthology.orchestra.run.smithi138.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-17T21:40:43.252 INFO:teuthology.orchestra.run.smithi138.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-17T21:40:43.252 INFO:teuthology.orchestra.run.smithi138.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-17T21:40:43.252 INFO:teuthology.orchestra.run.smithi138.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-17T21:40:43.252 INFO:teuthology.orchestra.run.smithi138.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-17T21:40:43.252 INFO:teuthology.orchestra.run.smithi138.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-17T21:40:43.252 INFO:teuthology.orchestra.run.smithi138.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-17T21:40:43.252 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-17T21:40:43.252 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-05-17T21:40:43.252 INFO:teuthology.orchestra.run.smithi138.stdout:====================================================================================== 2024-05-17T21:40:43.253 INFO:teuthology.orchestra.run.smithi138.stdout:Install 12 Packages 2024-05-17T21:40:43.253 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-17T21:40:43.253 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 38 M 2024-05-17T21:40:43.253 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 209 M 2024-05-17T21:40:43.253 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-05-17T21:40:43.565 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-952.g9903f776.el9.noarch 7/7 2024-05-17T21:40:43.566 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-952.g9903f776.el9.n 1/7 2024-05-17T21:40:43.566 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-952.g9903f776.el9.noarch 2/7 2024-05-17T21:40:43.566 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-952.g9903f776.el9.no 3/7 2024-05-17T21:40:43.566 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-05-17T21:40:43.566 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-05-17T21:40:43.566 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-05-17T21:40:43.821 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-05-17T21:40:43.821 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:40:43.821 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-05-17T21:40:43.821 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-grafana-dashboards-2:17.2.7-952.g9903f776.el9.noarch 2024-05-17T21:40:43.821 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-dashboard-2:17.2.7-952.g9903f776.el9.noarch 2024-05-17T21:40:43.821 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-prometheus-alerts-2:17.2.7-952.g9903f776.el9.noarch 2024-05-17T21:40:43.821 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-05-17T21:40:43.821 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-05-17T21:40:43.821 INFO:teuthology.orchestra.run.smithi107.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-17T21:40:43.822 INFO:teuthology.orchestra.run.smithi107.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-17T21:40:43.822 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:40:43.822 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-05-17T21:40:44.036 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install ceph-mgr-diskprediction-local 2024-05-17T21:40:44.044 INFO:teuthology.orchestra.run.smithi138.stdout:(1/12): ceph-mgr-diskprediction-local-17.2.7-95 12 MB/s | 7.4 MB 00:00 2024-05-17T21:40:44.169 INFO:teuthology.orchestra.run.smithi138.stdout:(2/12): flexiblas-3.0.4-8.el9.x86_64.rpm 262 kB/s | 33 kB 00:00 2024-05-17T21:40:44.336 INFO:teuthology.orchestra.run.smithi138.stdout:(3/12): libquadmath-11.4.1-3.el9.x86_64.rpm 216 kB/s | 195 kB 00:00 2024-05-17T21:40:44.470 INFO:teuthology.orchestra.run.smithi138.stdout:(4/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 10 MB/s | 3.0 MB 00:00 2024-05-17T21:40:44.495 INFO:teuthology.orchestra.run.smithi138.stdout:(5/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 115 kB/s | 18 kB 00:00 2024-05-17T21:40:44.529 INFO:teuthology.orchestra.run.smithi138.stdout:(6/12): openblas-0.3.21-2.el9.x86_64.rpm 603 kB/s | 35 kB 00:00 2024-05-17T21:40:44.601 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:01:11 ago on Fri 17 May 2024 09:39:33 PM UTC. 2024-05-17T21:40:44.671 INFO:teuthology.orchestra.run.smithi138.stdout:(7/12): libgfortran-11.4.1-3.el9.x86_64.rpm 648 kB/s | 803 kB 00:01 2024-05-17T21:40:44.712 INFO:teuthology.orchestra.run.smithi138.stdout:(8/12): python3-devel-3.9.18-3.el9.x86_64.rpm 1.3 MB/s | 244 kB 00:00 2024-05-17T21:40:44.724 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-05-17T21:40:44.725 INFO:teuthology.orchestra.run.smithi107.stdout:====================================================================================== 2024-05-17T21:40:44.725 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repository Size 2024-05-17T21:40:44.725 INFO:teuthology.orchestra.run.smithi107.stdout:====================================================================================== 2024-05-17T21:40:44.725 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-05-17T21:40:44.725 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-952.g9903f776.el9 ceph-noarch 7.4 M 2024-05-17T21:40:44.725 INFO:teuthology.orchestra.run.smithi107.stdout:Installing dependencies: 2024-05-17T21:40:44.725 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-17T21:40:44.725 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-17T21:40:44.726 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-17T21:40:44.726 INFO:teuthology.orchestra.run.smithi107.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-17T21:40:44.726 INFO:teuthology.orchestra.run.smithi107.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-17T21:40:44.726 INFO:teuthology.orchestra.run.smithi107.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-17T21:40:44.726 INFO:teuthology.orchestra.run.smithi107.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-17T21:40:44.726 INFO:teuthology.orchestra.run.smithi107.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-17T21:40:44.726 INFO:teuthology.orchestra.run.smithi107.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-17T21:40:44.726 INFO:teuthology.orchestra.run.smithi107.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-17T21:40:44.726 INFO:teuthology.orchestra.run.smithi107.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-17T21:40:44.727 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:40:44.727 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-05-17T21:40:44.727 INFO:teuthology.orchestra.run.smithi107.stdout:====================================================================================== 2024-05-17T21:40:44.727 INFO:teuthology.orchestra.run.smithi107.stdout:Install 12 Packages 2024-05-17T21:40:44.727 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:40:44.728 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 38 M 2024-05-17T21:40:44.728 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 209 M 2024-05-17T21:40:44.728 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-05-17T21:40:44.771 INFO:teuthology.orchestra.run.smithi138.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 4.2 MB/s | 250 kB 00:00 2024-05-17T21:40:44.938 INFO:teuthology.orchestra.run.smithi138.stdout:(10/12): openblas-openmp-0.3.21-2.el9.x86_64.rp 11 MB/s | 4.7 MB 00:00 2024-05-17T21:40:45.057 INFO:teuthology.orchestra.run.smithi107.stdout:(1/12): libquadmath-11.4.1-3.el9.x86_64.rpm 803 kB/s | 195 kB 00:00 2024-05-17T21:40:45.149 INFO:teuthology.orchestra.run.smithi107.stdout:(2/12): flexiblas-3.0.4-8.el9.x86_64.rpm 357 kB/s | 33 kB 00:00 2024-05-17T21:40:45.274 INFO:teuthology.orchestra.run.smithi107.stdout:(3/12): libgfortran-11.4.1-3.el9.x86_64.rpm 1.7 MB/s | 803 kB 00:00 2024-05-17T21:40:45.338 INFO:teuthology.orchestra.run.smithi138.stdout:(11/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 29 MB/s | 16 MB 00:00 2024-05-17T21:40:45.366 INFO:teuthology.orchestra.run.smithi107.stdout:(4/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 14 MB/s | 3.0 MB 00:00 2024-05-17T21:40:45.391 INFO:teuthology.orchestra.run.smithi107.stdout:(5/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 156 kB/s | 18 kB 00:00 2024-05-17T21:40:45.424 INFO:teuthology.orchestra.run.smithi107.stdout:(6/12): openblas-0.3.21-2.el9.x86_64.rpm 603 kB/s | 35 kB 00:00 2024-05-17T21:40:45.477 INFO:teuthology.orchestra.run.smithi107.stdout:(7/12): python3-devel-3.9.18-3.el9.x86_64.rpm 4.8 MB/s | 244 kB 00:00 2024-05-17T21:40:45.617 INFO:teuthology.orchestra.run.smithi107.stdout:(8/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 36 MB/s | 5.1 MB 00:00 2024-05-17T21:40:45.667 INFO:teuthology.orchestra.run.smithi107.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 4.9 MB/s | 250 kB 00:00 2024-05-17T21:40:45.876 INFO:teuthology.orchestra.run.smithi107.stdout:(10/12): ceph-mgr-diskprediction-local-17.2.7-9 7.0 MB/s | 7.4 MB 00:01 2024-05-17T21:40:45.996 INFO:teuthology.orchestra.run.smithi138.stdout:(12/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 3.8 MB/s | 5.1 MB 00:01 2024-05-17T21:40:45.997 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-05-17T21:40:45.998 INFO:teuthology.orchestra.run.smithi138.stdout:Total 14 MB/s | 38 MB 00:02 2024-05-17T21:40:46.092 INFO:teuthology.orchestra.run.smithi107.stdout:(11/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 38 MB/s | 16 MB 00:00 2024-05-17T21:40:46.353 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-05-17T21:40:46.367 INFO:teuthology.orchestra.run.smithi107.stdout:(12/12): openblas-openmp-0.3.21-2.el9.x86_64.rp 4.9 MB/s | 4.7 MB 00:00 2024-05-17T21:40:46.368 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-05-17T21:40:46.369 INFO:teuthology.orchestra.run.smithi107.stdout:Total 23 MB/s | 38 MB 00:01 2024-05-17T21:40:46.379 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-05-17T21:40:46.379 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-05-17T21:40:46.720 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-05-17T21:40:46.747 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-05-17T21:40:46.747 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-05-17T21:40:46.912 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-05-17T21:40:46.912 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-05-17T21:40:47.240 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-05-17T21:40:47.240 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-05-17T21:40:47.761 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-05-17T21:40:47.830 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-05-17T21:40:47.878 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 2/12 2024-05-17T21:40:47.976 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 3/12 2024-05-17T21:40:48.013 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 4/12 2024-05-17T21:40:48.061 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-05-17T21:40:48.137 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-05-17T21:40:48.174 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 2/12 2024-05-17T21:40:48.189 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-05-17T21:40:48.211 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-05-17T21:40:48.272 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 3/12 2024-05-17T21:40:48.289 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-05-17T21:40:48.316 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 4/12 2024-05-17T21:40:48.493 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-05-17T21:40:48.532 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-05-17T21:40:48.615 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-05-17T21:40:48.949 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-05-17T21:40:49.073 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-05-17T21:40:49.278 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-05-17T21:40:49.386 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-05-17T21:40:50.315 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-05-17T21:40:50.636 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-05-17T21:40:51.366 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-05-17T21:40:51.402 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-952.g9903f7 12/12 2024-05-17T21:40:51.673 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-05-17T21:40:51.697 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-952.g9903f7 12/12 2024-05-17T21:40:53.991 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-952.g9903f7 12/12 2024-05-17T21:40:53.991 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-952.g9903f7 1/12 2024-05-17T21:40:53.991 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-05-17T21:40:53.991 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 3/12 2024-05-17T21:40:53.991 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-05-17T21:40:53.991 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-05-17T21:40:53.991 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-05-17T21:40:53.991 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 7/12 2024-05-17T21:40:53.991 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 8/12 2024-05-17T21:40:53.991 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 9/12 2024-05-17T21:40:53.991 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-05-17T21:40:53.992 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-05-17T21:40:54.242 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-952.g9903f7 12/12 2024-05-17T21:40:54.242 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-952.g9903f7 1/12 2024-05-17T21:40:54.242 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-05-17T21:40:54.243 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 3/12 2024-05-17T21:40:54.243 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-05-17T21:40:54.243 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-05-17T21:40:54.243 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-05-17T21:40:54.243 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 7/12 2024-05-17T21:40:54.243 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 8/12 2024-05-17T21:40:54.243 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 9/12 2024-05-17T21:40:54.243 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-05-17T21:40:54.243 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-05-17T21:40:54.395 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-05-17T21:40:54.395 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-17T21:40:54.395 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-05-17T21:40:54.395 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-diskprediction-local-2:17.2.7-952.g9903f776.el9.noarch 2024-05-17T21:40:54.395 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-17T21:40:54.395 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-17T21:40:54.395 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-17T21:40:54.395 INFO:teuthology.orchestra.run.smithi138.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-17T21:40:54.395 INFO:teuthology.orchestra.run.smithi138.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-17T21:40:54.396 INFO:teuthology.orchestra.run.smithi138.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-17T21:40:54.396 INFO:teuthology.orchestra.run.smithi138.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-17T21:40:54.396 INFO:teuthology.orchestra.run.smithi138.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-17T21:40:54.396 INFO:teuthology.orchestra.run.smithi138.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-17T21:40:54.396 INFO:teuthology.orchestra.run.smithi138.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-17T21:40:54.396 INFO:teuthology.orchestra.run.smithi138.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-17T21:40:54.396 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-17T21:40:54.396 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-05-17T21:40:54.638 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-05-17T21:40:54.638 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:40:54.638 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-05-17T21:40:54.638 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-diskprediction-local-2:17.2.7-952.g9903f776.el9.noarch 2024-05-17T21:40:54.639 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-17T21:40:54.639 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-17T21:40:54.639 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-17T21:40:54.639 INFO:teuthology.orchestra.run.smithi107.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-17T21:40:54.639 INFO:teuthology.orchestra.run.smithi107.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-17T21:40:54.639 INFO:teuthology.orchestra.run.smithi107.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-17T21:40:54.639 INFO:teuthology.orchestra.run.smithi107.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-17T21:40:54.639 INFO:teuthology.orchestra.run.smithi107.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-17T21:40:54.639 INFO:teuthology.orchestra.run.smithi107.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-17T21:40:54.639 INFO:teuthology.orchestra.run.smithi107.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-17T21:40:54.639 INFO:teuthology.orchestra.run.smithi107.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-17T21:40:54.639 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:40:54.639 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-05-17T21:40:54.755 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph-mgr-rook 2024-05-17T21:40:55.017 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install ceph-mgr-rook 2024-05-17T21:40:55.343 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:20 ago on Fri 17 May 2024 09:39:35 PM UTC. 2024-05-17T21:40:55.468 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-05-17T21:40:55.469 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-05-17T21:40:55.469 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repository Size 2024-05-17T21:40:55.470 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-05-17T21:40:55.470 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-05-17T21:40:55.470 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-rook noarch 2:17.2.7-952.g9903f776.el9 ceph-noarch 47 k 2024-05-17T21:40:55.470 INFO:teuthology.orchestra.run.smithi138.stdout:Installing dependencies: 2024-05-17T21:40:55.470 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-17T21:40:55.470 INFO:teuthology.orchestra.run.smithi138.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-17T21:40:55.470 INFO:teuthology.orchestra.run.smithi138.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-17T21:40:55.470 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-17T21:40:55.470 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-17T21:40:55.470 INFO:teuthology.orchestra.run.smithi138.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-17T21:40:55.470 INFO:teuthology.orchestra.run.smithi138.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-17T21:40:55.470 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-17T21:40:55.471 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-17T21:40:55.471 INFO:teuthology.orchestra.run.smithi138.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-17T21:40:55.471 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-17T21:40:55.471 INFO:teuthology.orchestra.run.smithi138.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-17T21:40:55.471 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-17T21:40:55.471 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-05-17T21:40:55.471 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-05-17T21:40:55.471 INFO:teuthology.orchestra.run.smithi138.stdout:Install 13 Packages 2024-05-17T21:40:55.471 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-17T21:40:55.472 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 2.2 M 2024-05-17T21:40:55.472 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 26 M 2024-05-17T21:40:55.472 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-05-17T21:40:55.596 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:01:22 ago on Fri 17 May 2024 09:39:33 PM UTC. 2024-05-17T21:40:55.711 INFO:teuthology.orchestra.run.smithi138.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 297 kB/s | 26 kB 00:00 2024-05-17T21:40:55.717 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-05-17T21:40:55.719 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-05-17T21:40:55.719 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repository Size 2024-05-17T21:40:55.719 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-05-17T21:40:55.719 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-05-17T21:40:55.719 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-rook noarch 2:17.2.7-952.g9903f776.el9 ceph-noarch 47 k 2024-05-17T21:40:55.719 INFO:teuthology.orchestra.run.smithi107.stdout:Installing dependencies: 2024-05-17T21:40:55.719 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-17T21:40:55.719 INFO:teuthology.orchestra.run.smithi107.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-17T21:40:55.719 INFO:teuthology.orchestra.run.smithi107.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-17T21:40:55.719 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-17T21:40:55.719 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-17T21:40:55.719 INFO:teuthology.orchestra.run.smithi107.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-17T21:40:55.719 INFO:teuthology.orchestra.run.smithi107.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-17T21:40:55.720 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-17T21:40:55.720 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-17T21:40:55.720 INFO:teuthology.orchestra.run.smithi107.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-17T21:40:55.720 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-17T21:40:55.720 INFO:teuthology.orchestra.run.smithi107.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-17T21:40:55.720 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:40:55.720 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-05-17T21:40:55.720 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-05-17T21:40:55.720 INFO:teuthology.orchestra.run.smithi107.stdout:Install 13 Packages 2024-05-17T21:40:55.720 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:40:55.721 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 2.2 M 2024-05-17T21:40:55.721 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 26 M 2024-05-17T21:40:55.721 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-05-17T21:40:55.728 INFO:teuthology.orchestra.run.smithi138.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 186 kB/s | 19 kB 00:00 2024-05-17T21:40:55.837 INFO:teuthology.orchestra.run.smithi138.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.0 MB/s | 222 kB 00:00 2024-05-17T21:40:55.862 INFO:teuthology.orchestra.run.smithi138.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.2 MB/s | 159 kB 00:00 2024-05-17T21:40:55.896 INFO:teuthology.orchestra.run.smithi138.stdout:(5/13): ceph-mgr-rook-17.2.7-952.g9903f776.el9. 175 kB/s | 47 kB 00:00 2024-05-17T21:40:55.921 INFO:teuthology.orchestra.run.smithi138.stdout:(6/13): python3-pyasn1-modules-0.4.8-6.el9.noar 3.2 MB/s | 279 kB 00:00 2024-05-17T21:40:55.946 INFO:teuthology.orchestra.run.smithi138.stdout:(7/13): python3-requests-oauthlib-1.3.0-12.el9. 639 kB/s | 54 kB 00:00 2024-05-17T21:40:56.171 INFO:teuthology.orchestra.run.smithi138.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 117 kB/s | 32 kB 00:00 2024-05-17T21:40:56.196 INFO:teuthology.orchestra.run.smithi138.stdout:(9/13): python3-certifi-2021.10.8-2.el9.noarch. 54 kB/s | 15 kB 00:00 2024-05-17T21:40:56.347 INFO:teuthology.orchestra.run.smithi138.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 394 kB/s | 59 kB 00:00 2024-05-17T21:40:56.372 INFO:teuthology.orchestra.run.smithi138.stdout:(11/13): python3-google-auth-2.29.0-1.el9.noarc 483 kB/s | 206 kB 00:00 2024-05-17T21:40:56.388 INFO:teuthology.orchestra.run.smithi107.stdout:(1/13): ceph-mgr-rook-17.2.7-952.g9903f776.el9. 243 kB/s | 47 kB 00:00 2024-05-17T21:40:56.455 INFO:teuthology.orchestra.run.smithi107.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 73 kB/s | 19 kB 00:00 2024-05-17T21:40:56.464 INFO:teuthology.orchestra.run.smithi138.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 767 kB/s | 90 kB 00:00 2024-05-17T21:40:56.480 INFO:teuthology.orchestra.run.smithi107.stdout:(3/13): python3-jsonpatch-1.21-16.el9.noarch.rp 89 kB/s | 26 kB 00:00 2024-05-17T21:40:56.497 INFO:teuthology.orchestra.run.smithi138.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 3.1 MB/s | 1.0 MB 00:00 2024-05-17T21:40:56.500 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-05-17T21:40:56.500 INFO:teuthology.orchestra.run.smithi138.stdout:Total 2.1 MB/s | 2.2 MB 00:01 2024-05-17T21:40:56.565 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-05-17T21:40:56.580 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-05-17T21:40:56.580 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-05-17T21:40:56.666 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-05-17T21:40:56.666 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-05-17T21:40:56.797 INFO:teuthology.orchestra.run.smithi107.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 464 kB/s | 159 kB 00:00 2024-05-17T21:40:56.906 INFO:teuthology.orchestra.run.smithi107.stdout:(5/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 430 kB/s | 222 kB 00:00 2024-05-17T21:40:56.931 INFO:teuthology.orchestra.run.smithi107.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 401 kB/s | 54 kB 00:00 2024-05-17T21:40:56.957 INFO:teuthology.orchestra.run.smithi107.stdout:(7/13): python3-pyasn1-modules-0.4.8-6.el9.noar 586 kB/s | 279 kB 00:00 2024-05-17T21:40:57.057 INFO:teuthology.orchestra.run.smithi107.stdout:(8/13): python3-certifi-2021.10.8-2.el9.noarch. 120 kB/s | 15 kB 00:00 2024-05-17T21:40:57.082 INFO:teuthology.orchestra.run.smithi107.stdout:(9/13): python3-cachetools-4.2.4-1.el9.noarch.r 183 kB/s | 32 kB 00:00 2024-05-17T21:40:57.166 INFO:teuthology.orchestra.run.smithi107.stdout:(10/13): python3-google-auth-2.29.0-1.el9.noarc 985 kB/s | 206 kB 00:00 2024-05-17T21:40:57.181 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-05-17T21:40:57.191 INFO:teuthology.orchestra.run.smithi107.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 544 kB/s | 59 kB 00:00 2024-05-17T21:40:57.224 INFO:teuthology.orchestra.run.smithi107.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.5 MB/s | 90 kB 00:00 2024-05-17T21:40:57.308 INFO:teuthology.orchestra.run.smithi107.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 4.1 MB/s | 1.0 MB 00:00 2024-05-17T21:40:57.308 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-05-17T21:40:57.308 INFO:teuthology.orchestra.run.smithi107.stdout:Total 1.4 MB/s | 2.2 MB 00:01 2024-05-17T21:40:57.363 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-05-17T21:40:57.374 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-05-17T21:40:57.389 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-05-17T21:40:57.389 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-05-17T21:40:57.423 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-05-17T21:40:57.473 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-05-17T21:40:57.474 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-05-17T21:40:57.483 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-05-17T21:40:57.520 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-05-17T21:40:57.562 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-05-17T21:40:57.656 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-05-17T21:40:57.774 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 7/13 2024-05-17T21:40:57.823 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-05-17T21:40:58.010 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-05-17T21:40:58.181 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-05-17T21:40:58.242 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-05-17T21:40:58.306 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-05-17T21:40:58.343 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-05-17T21:40:58.384 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-05-17T21:40:58.478 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-05-17T21:40:58.546 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-05-17T21:40:58.596 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-05-17T21:40:58.597 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 7/13 2024-05-17T21:40:58.629 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-05-17T21:40:58.661 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-05-17T21:40:58.676 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-05-17T21:40:58.699 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-rook-2:17.2.7-952.g9903f776.el9.noarch 13/13 2024-05-17T21:40:59.380 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-05-17T21:40:59.433 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-05-17T21:40:59.467 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-05-17T21:40:59.521 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-05-17T21:40:59.531 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-952.g9903f776.el9.noarch 13/13 2024-05-17T21:40:59.531 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-rook-2:17.2.7-952.g9903f776.el9.noarch 1/13 2024-05-17T21:40:59.531 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-05-17T21:40:59.531 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-05-17T21:40:59.531 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-05-17T21:40:59.531 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-05-17T21:40:59.531 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-05-17T21:40:59.532 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-05-17T21:40:59.532 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-05-17T21:40:59.532 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-05-17T21:40:59.532 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 10/13 2024-05-17T21:40:59.532 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-05-17T21:40:59.532 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-05-17T21:40:59.545 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-rook-2:17.2.7-952.g9903f776.el9.noarch 13/13 2024-05-17T21:40:59.859 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-05-17T21:40:59.860 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-17T21:40:59.860 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-05-17T21:40:59.860 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-rook-2:17.2.7-952.g9903f776.el9.noarch 2024-05-17T21:40:59.860 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-17T21:40:59.860 INFO:teuthology.orchestra.run.smithi138.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-17T21:40:59.860 INFO:teuthology.orchestra.run.smithi138.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-17T21:40:59.860 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-17T21:40:59.860 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-17T21:40:59.860 INFO:teuthology.orchestra.run.smithi138.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-17T21:40:59.860 INFO:teuthology.orchestra.run.smithi138.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-17T21:40:59.860 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-17T21:40:59.860 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-17T21:40:59.860 INFO:teuthology.orchestra.run.smithi138.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-17T21:40:59.860 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-17T21:40:59.861 INFO:teuthology.orchestra.run.smithi138.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-17T21:40:59.861 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-17T21:40:59.861 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-05-17T21:41:00.115 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph-mgr-cephadm 2024-05-17T21:41:00.240 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-952.g9903f776.el9.noarch 13/13 2024-05-17T21:41:00.240 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-rook-2:17.2.7-952.g9903f776.el9.noarch 1/13 2024-05-17T21:41:00.240 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-05-17T21:41:00.240 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-05-17T21:41:00.241 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-05-17T21:41:00.241 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-05-17T21:41:00.241 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-05-17T21:41:00.241 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-05-17T21:41:00.241 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-05-17T21:41:00.241 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-05-17T21:41:00.241 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 10/13 2024-05-17T21:41:00.241 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-05-17T21:41:00.241 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-05-17T21:41:00.567 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-05-17T21:41:00.567 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:41:00.567 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-05-17T21:41:00.568 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-rook-2:17.2.7-952.g9903f776.el9.noarch 2024-05-17T21:41:00.568 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-17T21:41:00.568 INFO:teuthology.orchestra.run.smithi107.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-17T21:41:00.568 INFO:teuthology.orchestra.run.smithi107.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-17T21:41:00.568 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-17T21:41:00.568 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-17T21:41:00.568 INFO:teuthology.orchestra.run.smithi107.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-17T21:41:00.568 INFO:teuthology.orchestra.run.smithi107.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-17T21:41:00.568 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-17T21:41:00.568 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-17T21:41:00.568 INFO:teuthology.orchestra.run.smithi107.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-17T21:41:00.568 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-17T21:41:00.568 INFO:teuthology.orchestra.run.smithi107.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-17T21:41:00.568 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:41:00.569 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-05-17T21:41:00.691 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:25 ago on Fri 17 May 2024 09:39:35 PM UTC. 2024-05-17T21:41:00.813 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-05-17T21:41:00.814 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-05-17T21:41:00.814 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repository Size 2024-05-17T21:41:00.814 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-05-17T21:41:00.814 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-05-17T21:41:00.814 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-cephadm noarch 2:17.2.7-952.g9903f776.el9 ceph-noarch 123 k 2024-05-17T21:41:00.814 INFO:teuthology.orchestra.run.smithi138.stdout:Installing dependencies: 2024-05-17T21:41:00.814 INFO:teuthology.orchestra.run.smithi138.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-17T21:41:00.814 INFO:teuthology.orchestra.run.smithi138.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-17T21:41:00.814 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-17T21:41:00.814 INFO:teuthology.orchestra.run.smithi138.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-17T21:41:00.815 INFO:teuthology.orchestra.run.smithi138.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-17T21:41:00.815 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-17T21:41:00.815 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-05-17T21:41:00.815 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-05-17T21:41:00.815 INFO:teuthology.orchestra.run.smithi138.stdout:Install 6 Packages 2024-05-17T21:41:00.815 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-17T21:41:00.815 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 7.0 M 2024-05-17T21:41:00.816 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 33 M 2024-05-17T21:41:00.816 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-05-17T21:41:00.823 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install ceph-mgr-cephadm 2024-05-17T21:41:01.395 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:01:28 ago on Fri 17 May 2024 09:39:33 PM UTC. 2024-05-17T21:41:01.491 INFO:teuthology.orchestra.run.smithi138.stdout:(1/6): ceph-mgr-cephadm-17.2.7-952.g9903f776.el 594 kB/s | 123 kB 00:00 2024-05-17T21:41:01.516 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-05-17T21:41:01.516 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-05-17T21:41:01.517 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repository Size 2024-05-17T21:41:01.517 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-05-17T21:41:01.517 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-05-17T21:41:01.517 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-cephadm noarch 2:17.2.7-952.g9903f776.el9 ceph-noarch 123 k 2024-05-17T21:41:01.517 INFO:teuthology.orchestra.run.smithi107.stdout:Installing dependencies: 2024-05-17T21:41:01.517 INFO:teuthology.orchestra.run.smithi107.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-17T21:41:01.517 INFO:teuthology.orchestra.run.smithi107.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-17T21:41:01.517 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-17T21:41:01.517 INFO:teuthology.orchestra.run.smithi107.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-17T21:41:01.517 INFO:teuthology.orchestra.run.smithi107.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-17T21:41:01.517 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:41:01.517 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-05-17T21:41:01.518 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-05-17T21:41:01.518 INFO:teuthology.orchestra.run.smithi107.stdout:Install 6 Packages 2024-05-17T21:41:01.518 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:41:01.518 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 7.0 M 2024-05-17T21:41:01.518 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 33 M 2024-05-17T21:41:01.518 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-05-17T21:41:01.724 INFO:teuthology.orchestra.run.smithi138.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 2.3 MB/s | 548 kB 00:00 2024-05-17T21:41:01.766 INFO:teuthology.orchestra.run.smithi138.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 1.4 MB/s | 58 kB 00:00 2024-05-17T21:41:01.809 INFO:teuthology.orchestra.run.smithi138.stdout:(4/6): python3-typing-extensions-4.4.0-2.el9.no 1.2 MB/s | 51 kB 00:00 2024-05-17T21:41:01.993 INFO:teuthology.orchestra.run.smithi107.stdout:(1/6): ceph-mgr-cephadm-17.2.7-952.g9903f776.el 587 kB/s | 123 kB 00:00 2024-05-17T21:41:02.250 INFO:teuthology.orchestra.run.smithi138.stdout:(5/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 256 kB/s | 248 kB 00:00 2024-05-17T21:41:02.335 INFO:teuthology.orchestra.run.smithi107.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 1.6 MB/s | 548 kB 00:00 2024-05-17T21:41:02.393 INFO:teuthology.orchestra.run.smithi107.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 989 kB/s | 58 kB 00:00 2024-05-17T21:41:02.453 INFO:teuthology.orchestra.run.smithi107.stdout:(4/6): python3-typing-extensions-4.4.0-2.el9.no 872 kB/s | 51 kB 00:00 2024-05-17T21:41:02.628 INFO:teuthology.orchestra.run.smithi107.stdout:(5/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 294 kB/s | 248 kB 00:00 2024-05-17T21:41:03.142 INFO:teuthology.orchestra.run.smithi138.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 3.2 MB/s | 6.0 MB 00:01 2024-05-17T21:41:03.144 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-05-17T21:41:03.145 INFO:teuthology.orchestra.run.smithi138.stdout:Total 3.0 MB/s | 7.0 MB 00:02 2024-05-17T21:41:03.217 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-05-17T21:41:03.229 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-05-17T21:41:03.229 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-05-17T21:41:03.236 INFO:teuthology.orchestra.run.smithi107.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 4.1 MB/s | 6.0 MB 00:01 2024-05-17T21:41:03.238 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-05-17T21:41:03.238 INFO:teuthology.orchestra.run.smithi107.stdout:Total 4.0 MB/s | 7.0 MB 00:01 2024-05-17T21:41:03.309 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-05-17T21:41:03.320 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-05-17T21:41:03.321 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-05-17T21:41:03.325 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-05-17T21:41:03.325 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-05-17T21:41:03.421 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-05-17T21:41:03.422 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-05-17T21:41:03.636 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-05-17T21:41:03.738 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-05-17T21:41:03.820 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-05-17T21:41:03.861 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-05-17T21:41:03.908 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-05-17T21:41:03.955 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-05-17T21:41:04.408 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-05-17T21:41:04.471 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-05-17T21:41:04.489 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-05-17T21:41:04.545 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-05-17T21:41:04.557 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-05-17T21:41:04.574 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-952.g9903f776.el9.noarch 6/6 2024-05-17T21:41:04.613 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-05-17T21:41:04.642 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-952.g9903f776.el9.noarch 6/6 2024-05-17T21:41:05.391 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-952.g9903f776.el9.noarch 6/6 2024-05-17T21:41:05.391 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-952.g9903f776.el9.noarch 1/6 2024-05-17T21:41:05.391 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-05-17T21:41:05.391 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-05-17T21:41:05.391 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-05-17T21:41:05.391 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-05-17T21:41:05.434 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-952.g9903f776.el9.noarch 6/6 2024-05-17T21:41:05.434 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-952.g9903f776.el9.noarch 1/6 2024-05-17T21:41:05.435 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-05-17T21:41:05.435 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-05-17T21:41:05.435 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-05-17T21:41:05.435 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-05-17T21:41:05.640 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-05-17T21:41:05.640 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-17T21:41:05.640 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-05-17T21:41:05.640 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-cephadm-2:17.2.7-952.g9903f776.el9.noarch 2024-05-17T21:41:05.640 INFO:teuthology.orchestra.run.smithi138.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-17T21:41:05.640 INFO:teuthology.orchestra.run.smithi138.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-17T21:41:05.640 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-17T21:41:05.640 INFO:teuthology.orchestra.run.smithi138.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-17T21:41:05.640 INFO:teuthology.orchestra.run.smithi138.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-17T21:41:05.641 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-17T21:41:05.641 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-05-17T21:41:05.684 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-05-17T21:41:05.684 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:41:05.685 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-05-17T21:41:05.685 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-cephadm-2:17.2.7-952.g9903f776.el9.noarch 2024-05-17T21:41:05.685 INFO:teuthology.orchestra.run.smithi107.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-17T21:41:05.685 INFO:teuthology.orchestra.run.smithi107.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-17T21:41:05.685 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-17T21:41:05.685 INFO:teuthology.orchestra.run.smithi107.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-17T21:41:05.685 INFO:teuthology.orchestra.run.smithi107.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-17T21:41:05.685 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:41:05.685 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-05-17T21:41:05.918 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph-fuse 2024-05-17T21:41:05.958 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install ceph-fuse 2024-05-17T21:41:06.492 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:31 ago on Fri 17 May 2024 09:39:35 PM UTC. 2024-05-17T21:41:06.527 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:01:33 ago on Fri 17 May 2024 09:39:33 PM UTC. 2024-05-17T21:41:06.614 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-05-17T21:41:06.614 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-05-17T21:41:06.614 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repository Size 2024-05-17T21:41:06.614 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-05-17T21:41:06.614 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-05-17T21:41:06.614 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-fuse x86_64 2:17.2.7-952.g9903f776.el9 ceph 812 k 2024-05-17T21:41:06.615 INFO:teuthology.orchestra.run.smithi138.stdout:Installing dependencies: 2024-05-17T21:41:06.615 INFO:teuthology.orchestra.run.smithi138.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-17T21:41:06.615 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-17T21:41:06.615 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-05-17T21:41:06.615 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-05-17T21:41:06.615 INFO:teuthology.orchestra.run.smithi138.stdout:Install 2 Packages 2024-05-17T21:41:06.615 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-17T21:41:06.616 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 893 k 2024-05-17T21:41:06.616 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 2.6 M 2024-05-17T21:41:06.616 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-05-17T21:41:06.649 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-05-17T21:41:06.649 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-05-17T21:41:06.649 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repository Size 2024-05-17T21:41:06.649 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-05-17T21:41:06.649 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-05-17T21:41:06.650 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-fuse x86_64 2:17.2.7-952.g9903f776.el9 ceph 812 k 2024-05-17T21:41:06.650 INFO:teuthology.orchestra.run.smithi107.stdout:Installing dependencies: 2024-05-17T21:41:06.650 INFO:teuthology.orchestra.run.smithi107.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-17T21:41:06.650 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:41:06.650 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-05-17T21:41:06.650 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-05-17T21:41:06.650 INFO:teuthology.orchestra.run.smithi107.stdout:Install 2 Packages 2024-05-17T21:41:06.650 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:41:06.651 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 893 k 2024-05-17T21:41:06.651 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 2.6 M 2024-05-17T21:41:06.651 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-05-17T21:41:06.963 INFO:teuthology.orchestra.run.smithi138.stdout:(1/2): ceph-fuse-17.2.7-952.g9903f776.el9.x86_6 2.6 MB/s | 812 kB 00:00 2024-05-17T21:41:07.005 INFO:teuthology.orchestra.run.smithi138.stdout:(2/2): fuse-2.9.9-15.el9.x86_64.rpm 233 kB/s | 80 kB 00:00 2024-05-17T21:41:07.005 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-05-17T21:41:07.006 INFO:teuthology.orchestra.run.smithi138.stdout:Total 2.2 MB/s | 893 kB 00:00 2024-05-17T21:41:07.018 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-05-17T21:41:07.040 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-05-17T21:41:07.040 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-05-17T21:41:07.116 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-05-17T21:41:07.117 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-05-17T21:41:07.122 INFO:teuthology.orchestra.run.smithi107.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 228 kB/s | 80 kB 00:00 2024-05-17T21:41:07.197 INFO:teuthology.orchestra.run.smithi107.stdout:(2/2): ceph-fuse-17.2.7-952.g9903f776.el9.x86_6 1.9 MB/s | 812 kB 00:00 2024-05-17T21:41:07.198 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-05-17T21:41:07.198 INFO:teuthology.orchestra.run.smithi107.stdout:Total 1.6 MB/s | 893 kB 00:00 2024-05-17T21:41:07.210 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-05-17T21:41:07.233 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-05-17T21:41:07.233 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-05-17T21:41:07.301 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-05-17T21:41:07.308 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-05-17T21:41:07.309 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-05-17T21:41:07.435 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-05-17T21:41:07.505 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-fuse-2:17.2.7-952.g9903f776.el9.x86_64 2/2 2024-05-17T21:41:07.533 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-05-17T21:41:07.668 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-05-17T21:41:07.730 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-fuse-2:17.2.7-952.g9903f776.el9.x86_64 2/2 2024-05-17T21:41:08.150 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-fuse-2:17.2.7-952.g9903f776.el9.x86_64 2/2 2024-05-17T21:41:08.151 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-fuse-2:17.2.7-952.g9903f776.el9.x86_64 1/2 2024-05-17T21:41:08.423 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-05-17T21:41:08.424 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-17T21:41:08.424 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-05-17T21:41:08.424 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-fuse-2:17.2.7-952.g9903f776.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-05-17T21:41:08.424 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-17T21:41:08.424 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-05-17T21:41:08.425 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-fuse-2:17.2.7-952.g9903f776.el9.x86_64 2/2 2024-05-17T21:41:08.425 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-fuse-2:17.2.7-952.g9903f776.el9.x86_64 1/2 2024-05-17T21:41:08.676 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install librados-devel 2024-05-17T21:41:08.731 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-05-17T21:41:08.731 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:41:08.731 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-05-17T21:41:08.731 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-fuse-2:17.2.7-952.g9903f776.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-05-17T21:41:08.731 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:41:08.731 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-05-17T21:41:09.002 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install librados-devel 2024-05-17T21:41:09.262 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:34 ago on Fri 17 May 2024 09:39:35 PM UTC. 2024-05-17T21:41:09.389 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-05-17T21:41:09.390 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-05-17T21:41:09.390 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repo Size 2024-05-17T21:41:09.390 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-05-17T21:41:09.390 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-05-17T21:41:09.390 INFO:teuthology.orchestra.run.smithi138.stdout: librados-devel x86_64 2:17.2.7-952.g9903f776.el9 ceph 126 k 2024-05-17T21:41:09.390 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-17T21:41:09.390 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-05-17T21:41:09.390 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-05-17T21:41:09.390 INFO:teuthology.orchestra.run.smithi138.stdout:Install 1 Package 2024-05-17T21:41:09.390 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-17T21:41:09.391 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 126 k 2024-05-17T21:41:09.391 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 448 k 2024-05-17T21:41:09.391 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-05-17T21:41:09.587 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:01:36 ago on Fri 17 May 2024 09:39:33 PM UTC. 2024-05-17T21:41:09.697 INFO:teuthology.orchestra.run.smithi138.stdout:librados-devel-17.2.7-952.g9903f776.el9.x86_64. 413 kB/s | 126 kB 00:00 2024-05-17T21:41:09.697 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-05-17T21:41:09.697 INFO:teuthology.orchestra.run.smithi138.stdout:Total 411 kB/s | 126 kB 00:00 2024-05-17T21:41:09.698 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-05-17T21:41:09.703 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-05-17T21:41:09.703 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-05-17T21:41:09.713 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-05-17T21:41:09.713 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-05-17T21:41:09.714 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repo Size 2024-05-17T21:41:09.714 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-05-17T21:41:09.714 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-05-17T21:41:09.714 INFO:teuthology.orchestra.run.smithi107.stdout: librados-devel x86_64 2:17.2.7-952.g9903f776.el9 ceph 126 k 2024-05-17T21:41:09.714 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:41:09.714 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-05-17T21:41:09.714 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-05-17T21:41:09.714 INFO:teuthology.orchestra.run.smithi107.stdout:Install 1 Package 2024-05-17T21:41:09.714 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:41:09.715 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 126 k 2024-05-17T21:41:09.715 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 448 k 2024-05-17T21:41:09.715 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-05-17T21:41:09.769 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-05-17T21:41:09.769 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-05-17T21:41:09.922 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-05-17T21:41:10.027 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : librados-devel-2:17.2.7-952.g9903f776.el9.x86_64 1/1 2024-05-17T21:41:10.028 INFO:teuthology.orchestra.run.smithi107.stdout:librados-devel-17.2.7-952.g9903f776.el9.x86_64. 404 kB/s | 126 kB 00:00 2024-05-17T21:41:10.028 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-05-17T21:41:10.029 INFO:teuthology.orchestra.run.smithi107.stdout:Total 402 kB/s | 126 kB 00:00 2024-05-17T21:41:10.029 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-05-17T21:41:10.035 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-05-17T21:41:10.035 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-05-17T21:41:10.101 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-05-17T21:41:10.101 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-05-17T21:41:10.252 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-05-17T21:41:10.400 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : librados-devel-2:17.2.7-952.g9903f776.el9.x86_64 1/1 2024-05-17T21:41:10.452 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librados-devel-2:17.2.7-952.g9903f776.el9.x86_64 1/1 2024-05-17T21:41:10.707 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librados-devel-2:17.2.7-952.g9903f776.el9.x86_64 1/1 2024-05-17T21:41:10.707 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-17T21:41:10.707 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-05-17T21:41:10.707 INFO:teuthology.orchestra.run.smithi138.stdout: librados-devel-2:17.2.7-952.g9903f776.el9.x86_64 2024-05-17T21:41:10.707 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-17T21:41:10.707 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-05-17T21:41:10.796 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librados-devel-2:17.2.7-952.g9903f776.el9.x86_64 1/1 2024-05-17T21:41:10.950 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install libcephfs2 2024-05-17T21:41:11.049 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librados-devel-2:17.2.7-952.g9903f776.el9.x86_64 1/1 2024-05-17T21:41:11.049 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:41:11.049 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-05-17T21:41:11.049 INFO:teuthology.orchestra.run.smithi107.stdout: librados-devel-2:17.2.7-952.g9903f776.el9.x86_64 2024-05-17T21:41:11.049 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:41:11.049 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-05-17T21:41:11.275 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install libcephfs2 2024-05-17T21:41:11.533 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:36 ago on Fri 17 May 2024 09:39:35 PM UTC. 2024-05-17T21:41:11.615 INFO:teuthology.orchestra.run.smithi138.stdout:Package libcephfs2-2:17.2.7-952.g9903f776.el9.x86_64 is already installed. 2024-05-17T21:41:11.657 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-05-17T21:41:11.658 INFO:teuthology.orchestra.run.smithi138.stdout:Nothing to do. 2024-05-17T21:41:11.658 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-05-17T21:41:11.730 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install libcephfs-devel 2024-05-17T21:41:11.857 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:01:38 ago on Fri 17 May 2024 09:39:33 PM UTC. 2024-05-17T21:41:11.942 INFO:teuthology.orchestra.run.smithi107.stdout:Package libcephfs2-2:17.2.7-952.g9903f776.el9.x86_64 is already installed. 2024-05-17T21:41:11.983 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-05-17T21:41:11.984 INFO:teuthology.orchestra.run.smithi107.stdout:Nothing to do. 2024-05-17T21:41:11.984 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-05-17T21:41:12.057 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install libcephfs-devel 2024-05-17T21:41:12.306 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:37 ago on Fri 17 May 2024 09:39:35 PM UTC. 2024-05-17T21:41:12.426 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-05-17T21:41:12.429 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-05-17T21:41:12.429 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repo Size 2024-05-17T21:41:12.429 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-05-17T21:41:12.430 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-05-17T21:41:12.430 INFO:teuthology.orchestra.run.smithi138.stdout: libcephfs-devel x86_64 2:17.2.7-952.g9903f776.el9 ceph 25 k 2024-05-17T21:41:12.430 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-17T21:41:12.430 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-05-17T21:41:12.430 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-05-17T21:41:12.430 INFO:teuthology.orchestra.run.smithi138.stdout:Install 1 Package 2024-05-17T21:41:12.430 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-17T21:41:12.430 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 25 k 2024-05-17T21:41:12.430 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 110 k 2024-05-17T21:41:12.430 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-05-17T21:41:12.595 INFO:teuthology.orchestra.run.smithi138.stdout:libcephfs-devel-17.2.7-952.g9903f776.el9.x86_64 149 kB/s | 25 kB 00:00 2024-05-17T21:41:12.596 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-05-17T21:41:12.596 INFO:teuthology.orchestra.run.smithi138.stdout:Total 148 kB/s | 25 kB 00:00 2024-05-17T21:41:12.596 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-05-17T21:41:12.600 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-05-17T21:41:12.600 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-05-17T21:41:12.601 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-05-17T21:41:12.601 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-05-17T21:41:12.625 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:01:39 ago on Fri 17 May 2024 09:39:33 PM UTC. 2024-05-17T21:41:12.681 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-05-17T21:41:12.746 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-05-17T21:41:12.747 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-05-17T21:41:12.747 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repo Size 2024-05-17T21:41:12.747 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-05-17T21:41:12.747 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-05-17T21:41:12.747 INFO:teuthology.orchestra.run.smithi107.stdout: libcephfs-devel x86_64 2:17.2.7-952.g9903f776.el9 ceph 25 k 2024-05-17T21:41:12.747 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:41:12.747 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-05-17T21:41:12.747 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-05-17T21:41:12.748 INFO:teuthology.orchestra.run.smithi107.stdout:Install 1 Package 2024-05-17T21:41:12.748 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:41:12.748 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 25 k 2024-05-17T21:41:12.748 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 110 k 2024-05-17T21:41:12.748 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-05-17T21:41:12.792 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libcephfs-devel-2:17.2.7-952.g9903f776.el9.x86_64 1/1 2024-05-17T21:41:12.990 INFO:teuthology.orchestra.run.smithi107.stdout:libcephfs-devel-17.2.7-952.g9903f776.el9.x86_64 103 kB/s | 25 kB 00:00 2024-05-17T21:41:12.991 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-05-17T21:41:12.991 INFO:teuthology.orchestra.run.smithi107.stdout:Total 102 kB/s | 25 kB 00:00 2024-05-17T21:41:12.991 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-05-17T21:41:12.995 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-05-17T21:41:12.995 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-05-17T21:41:12.996 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-05-17T21:41:12.996 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-05-17T21:41:13.078 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-05-17T21:41:13.142 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: libcephfs-devel-2:17.2.7-952.g9903f776.el9.x86_64 1/1 2024-05-17T21:41:13.187 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libcephfs-devel-2:17.2.7-952.g9903f776.el9.x86_64 1/1 2024-05-17T21:41:13.372 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libcephfs-devel-2:17.2.7-952.g9903f776.el9.x86_64 1/1 2024-05-17T21:41:13.372 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-17T21:41:13.372 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-05-17T21:41:13.372 INFO:teuthology.orchestra.run.smithi138.stdout: libcephfs-devel-2:17.2.7-952.g9903f776.el9.x86_64 2024-05-17T21:41:13.372 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-17T21:41:13.372 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-05-17T21:41:13.538 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: libcephfs-devel-2:17.2.7-952.g9903f776.el9.x86_64 1/1 2024-05-17T21:41:13.573 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install librados2 2024-05-17T21:41:13.788 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libcephfs-devel-2:17.2.7-952.g9903f776.el9.x86_64 1/1 2024-05-17T21:41:13.789 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:41:13.789 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-05-17T21:41:13.789 INFO:teuthology.orchestra.run.smithi107.stdout: libcephfs-devel-2:17.2.7-952.g9903f776.el9.x86_64 2024-05-17T21:41:13.789 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:41:13.789 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-05-17T21:41:13.997 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install librados2 2024-05-17T21:41:14.141 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:39 ago on Fri 17 May 2024 09:39:35 PM UTC. 2024-05-17T21:41:14.222 INFO:teuthology.orchestra.run.smithi138.stdout:Package librados2-2:17.2.7-952.g9903f776.el9.x86_64 is already installed. 2024-05-17T21:41:14.263 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-05-17T21:41:14.265 INFO:teuthology.orchestra.run.smithi138.stdout:Nothing to do. 2024-05-17T21:41:14.265 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-05-17T21:41:14.349 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install librbd1 2024-05-17T21:41:14.563 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:01:41 ago on Fri 17 May 2024 09:39:33 PM UTC. 2024-05-17T21:41:14.643 INFO:teuthology.orchestra.run.smithi107.stdout:Package librados2-2:17.2.7-952.g9903f776.el9.x86_64 is already installed. 2024-05-17T21:41:14.684 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-05-17T21:41:14.685 INFO:teuthology.orchestra.run.smithi107.stdout:Nothing to do. 2024-05-17T21:41:14.685 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-05-17T21:41:14.761 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install librbd1 2024-05-17T21:41:14.929 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:39 ago on Fri 17 May 2024 09:39:35 PM UTC. 2024-05-17T21:41:15.011 INFO:teuthology.orchestra.run.smithi138.stdout:Package librbd1-2:17.2.7-952.g9903f776.el9.x86_64 is already installed. 2024-05-17T21:41:15.053 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-05-17T21:41:15.054 INFO:teuthology.orchestra.run.smithi138.stdout:Nothing to do. 2024-05-17T21:41:15.054 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-05-17T21:41:15.129 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install python3-rados 2024-05-17T21:41:15.327 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:01:42 ago on Fri 17 May 2024 09:39:33 PM UTC. 2024-05-17T21:41:15.407 INFO:teuthology.orchestra.run.smithi107.stdout:Package librbd1-2:17.2.7-952.g9903f776.el9.x86_64 is already installed. 2024-05-17T21:41:15.448 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-05-17T21:41:15.449 INFO:teuthology.orchestra.run.smithi107.stdout:Nothing to do. 2024-05-17T21:41:15.449 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-05-17T21:41:15.518 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install python3-rados 2024-05-17T21:41:15.705 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:40 ago on Fri 17 May 2024 09:39:35 PM UTC. 2024-05-17T21:41:15.788 INFO:teuthology.orchestra.run.smithi138.stdout:Package python3-rados-2:17.2.7-952.g9903f776.el9.x86_64 is already installed. 2024-05-17T21:41:15.830 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-05-17T21:41:15.831 INFO:teuthology.orchestra.run.smithi138.stdout:Nothing to do. 2024-05-17T21:41:15.831 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-05-17T21:41:15.903 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install python3-rgw 2024-05-17T21:41:16.089 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:01:43 ago on Fri 17 May 2024 09:39:33 PM UTC. 2024-05-17T21:41:16.169 INFO:teuthology.orchestra.run.smithi107.stdout:Package python3-rados-2:17.2.7-952.g9903f776.el9.x86_64 is already installed. 2024-05-17T21:41:16.209 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-05-17T21:41:16.211 INFO:teuthology.orchestra.run.smithi107.stdout:Nothing to do. 2024-05-17T21:41:16.211 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-05-17T21:41:16.289 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install python3-rgw 2024-05-17T21:41:16.478 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:41 ago on Fri 17 May 2024 09:39:35 PM UTC. 2024-05-17T21:41:16.559 INFO:teuthology.orchestra.run.smithi138.stdout:Package python3-rgw-2:17.2.7-952.g9903f776.el9.x86_64 is already installed. 2024-05-17T21:41:16.599 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-05-17T21:41:16.601 INFO:teuthology.orchestra.run.smithi138.stdout:Nothing to do. 2024-05-17T21:41:16.601 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-05-17T21:41:16.669 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install python3-cephfs 2024-05-17T21:41:16.862 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:01:43 ago on Fri 17 May 2024 09:39:33 PM UTC. 2024-05-17T21:41:16.943 INFO:teuthology.orchestra.run.smithi107.stdout:Package python3-rgw-2:17.2.7-952.g9903f776.el9.x86_64 is already installed. 2024-05-17T21:41:16.984 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-05-17T21:41:16.985 INFO:teuthology.orchestra.run.smithi107.stdout:Nothing to do. 2024-05-17T21:41:16.985 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-05-17T21:41:17.052 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install python3-cephfs 2024-05-17T21:41:17.246 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:42 ago on Fri 17 May 2024 09:39:35 PM UTC. 2024-05-17T21:41:17.326 INFO:teuthology.orchestra.run.smithi138.stdout:Package python3-cephfs-2:17.2.7-952.g9903f776.el9.x86_64 is already installed. 2024-05-17T21:41:17.368 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-05-17T21:41:17.369 INFO:teuthology.orchestra.run.smithi138.stdout:Nothing to do. 2024-05-17T21:41:17.369 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-05-17T21:41:17.448 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install python3-rbd 2024-05-17T21:41:17.622 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:01:44 ago on Fri 17 May 2024 09:39:33 PM UTC. 2024-05-17T21:41:17.701 INFO:teuthology.orchestra.run.smithi107.stdout:Package python3-cephfs-2:17.2.7-952.g9903f776.el9.x86_64 is already installed. 2024-05-17T21:41:17.742 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-05-17T21:41:17.743 INFO:teuthology.orchestra.run.smithi107.stdout:Nothing to do. 2024-05-17T21:41:17.743 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-05-17T21:41:17.813 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install python3-rbd 2024-05-17T21:41:18.024 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:43 ago on Fri 17 May 2024 09:39:35 PM UTC. 2024-05-17T21:41:18.106 INFO:teuthology.orchestra.run.smithi138.stdout:Package python3-rbd-2:17.2.7-952.g9903f776.el9.x86_64 is already installed. 2024-05-17T21:41:18.147 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-05-17T21:41:18.148 INFO:teuthology.orchestra.run.smithi138.stdout:Nothing to do. 2024-05-17T21:41:18.148 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-05-17T21:41:18.232 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install rbd-fuse 2024-05-17T21:41:18.385 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:01:45 ago on Fri 17 May 2024 09:39:33 PM UTC. 2024-05-17T21:41:18.465 INFO:teuthology.orchestra.run.smithi107.stdout:Package python3-rbd-2:17.2.7-952.g9903f776.el9.x86_64 is already installed. 2024-05-17T21:41:18.506 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-05-17T21:41:18.507 INFO:teuthology.orchestra.run.smithi107.stdout:Nothing to do. 2024-05-17T21:41:18.507 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-05-17T21:41:18.577 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install rbd-fuse 2024-05-17T21:41:18.802 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:43 ago on Fri 17 May 2024 09:39:35 PM UTC. 2024-05-17T21:41:18.924 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-05-17T21:41:18.924 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-05-17T21:41:18.924 INFO:teuthology.orchestra.run.smithi138.stdout: Package Architecture Version Repository Size 2024-05-17T21:41:18.924 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-05-17T21:41:18.924 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-05-17T21:41:18.924 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-fuse x86_64 2:17.2.7-952.g9903f776.el9 ceph 82 k 2024-05-17T21:41:18.924 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-17T21:41:18.925 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-05-17T21:41:18.925 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-05-17T21:41:18.925 INFO:teuthology.orchestra.run.smithi138.stdout:Install 1 Package 2024-05-17T21:41:18.925 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-17T21:41:18.925 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 82 k 2024-05-17T21:41:18.925 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 222 k 2024-05-17T21:41:18.925 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-05-17T21:41:19.137 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:01:46 ago on Fri 17 May 2024 09:39:33 PM UTC. 2024-05-17T21:41:19.242 INFO:teuthology.orchestra.run.smithi138.stdout:rbd-fuse-17.2.7-952.g9903f776.el9.x86_64.rpm 259 kB/s | 82 kB 00:00 2024-05-17T21:41:19.242 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-05-17T21:41:19.243 INFO:teuthology.orchestra.run.smithi138.stdout:Total 258 kB/s | 82 kB 00:00 2024-05-17T21:41:19.243 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-05-17T21:41:19.250 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-05-17T21:41:19.250 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-05-17T21:41:19.258 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-05-17T21:41:19.260 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-05-17T21:41:19.260 INFO:teuthology.orchestra.run.smithi107.stdout: Package Architecture Version Repository Size 2024-05-17T21:41:19.260 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-05-17T21:41:19.260 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-05-17T21:41:19.260 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-fuse x86_64 2:17.2.7-952.g9903f776.el9 ceph 82 k 2024-05-17T21:41:19.260 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:41:19.260 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-05-17T21:41:19.260 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-05-17T21:41:19.260 INFO:teuthology.orchestra.run.smithi107.stdout:Install 1 Package 2024-05-17T21:41:19.260 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:41:19.261 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 82 k 2024-05-17T21:41:19.261 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 222 k 2024-05-17T21:41:19.261 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-05-17T21:41:19.312 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-05-17T21:41:19.312 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-05-17T21:41:19.453 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-05-17T21:41:19.477 INFO:teuthology.orchestra.run.smithi107.stdout:rbd-fuse-17.2.7-952.g9903f776.el9.x86_64.rpm 377 kB/s | 82 kB 00:00 2024-05-17T21:41:19.477 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-05-17T21:41:19.478 INFO:teuthology.orchestra.run.smithi107.stdout:Total 374 kB/s | 82 kB 00:00 2024-05-17T21:41:19.478 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-05-17T21:41:19.485 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-05-17T21:41:19.485 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-05-17T21:41:19.547 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-05-17T21:41:19.547 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-05-17T21:41:19.554 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : rbd-fuse-2:17.2.7-952.g9903f776.el9.x86_64 1/1 2024-05-17T21:41:19.708 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-05-17T21:41:19.807 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : rbd-fuse-2:17.2.7-952.g9903f776.el9.x86_64 1/1 2024-05-17T21:41:19.942 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: rbd-fuse-2:17.2.7-952.g9903f776.el9.x86_64 1/1 2024-05-17T21:41:20.188 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : rbd-fuse-2:17.2.7-952.g9903f776.el9.x86_64 1/1 2024-05-17T21:41:20.188 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-17T21:41:20.188 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-05-17T21:41:20.188 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-fuse-2:17.2.7-952.g9903f776.el9.x86_64 2024-05-17T21:41:20.188 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-17T21:41:20.188 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-05-17T21:41:20.189 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: rbd-fuse-2:17.2.7-952.g9903f776.el9.x86_64 1/1 2024-05-17T21:41:20.409 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install rbd-mirror 2024-05-17T21:41:20.454 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : rbd-fuse-2:17.2.7-952.g9903f776.el9.x86_64 1/1 2024-05-17T21:41:20.454 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:41:20.454 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-05-17T21:41:20.454 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-fuse-2:17.2.7-952.g9903f776.el9.x86_64 2024-05-17T21:41:20.454 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:41:20.455 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-05-17T21:41:20.686 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install rbd-mirror 2024-05-17T21:41:20.992 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:45 ago on Fri 17 May 2024 09:39:35 PM UTC. 2024-05-17T21:41:21.117 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-05-17T21:41:21.117 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-05-17T21:41:21.117 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repo Size 2024-05-17T21:41:21.118 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-05-17T21:41:21.118 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-05-17T21:41:21.118 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-mirror x86_64 2:17.2.7-952.g9903f776.el9 ceph 2.9 M 2024-05-17T21:41:21.118 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-17T21:41:21.118 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-05-17T21:41:21.118 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-05-17T21:41:21.118 INFO:teuthology.orchestra.run.smithi138.stdout:Install 1 Package 2024-05-17T21:41:21.118 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-17T21:41:21.119 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 2.9 M 2024-05-17T21:41:21.119 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 12 M 2024-05-17T21:41:21.119 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-05-17T21:41:21.271 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:01:48 ago on Fri 17 May 2024 09:39:33 PM UTC. 2024-05-17T21:41:21.397 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-05-17T21:41:21.398 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-05-17T21:41:21.398 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repo Size 2024-05-17T21:41:21.399 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-05-17T21:41:21.399 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-05-17T21:41:21.399 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-mirror x86_64 2:17.2.7-952.g9903f776.el9 ceph 2.9 M 2024-05-17T21:41:21.399 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:41:21.399 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-05-17T21:41:21.399 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-05-17T21:41:21.399 INFO:teuthology.orchestra.run.smithi107.stdout:Install 1 Package 2024-05-17T21:41:21.399 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:41:21.399 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 2.9 M 2024-05-17T21:41:21.399 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 12 M 2024-05-17T21:41:21.400 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-05-17T21:41:21.523 INFO:teuthology.orchestra.run.smithi138.stdout:rbd-mirror-17.2.7-952.g9903f776.el9.x86_64.rpm 7.3 MB/s | 2.9 MB 00:00 2024-05-17T21:41:21.524 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-05-17T21:41:21.524 INFO:teuthology.orchestra.run.smithi138.stdout:Total 7.2 MB/s | 2.9 MB 00:00 2024-05-17T21:41:21.524 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-05-17T21:41:21.533 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-05-17T21:41:21.534 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-05-17T21:41:21.615 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-05-17T21:41:21.616 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-05-17T21:41:21.964 INFO:teuthology.orchestra.run.smithi107.stdout:rbd-mirror-17.2.7-952.g9903f776.el9.x86_64.rpm 5.2 MB/s | 2.9 MB 00:00 2024-05-17T21:41:21.965 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-05-17T21:41:21.965 INFO:teuthology.orchestra.run.smithi107.stdout:Total 5.2 MB/s | 2.9 MB 00:00 2024-05-17T21:41:21.965 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-05-17T21:41:21.974 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-05-17T21:41:21.974 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-05-17T21:41:22.044 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-05-17T21:41:22.053 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-05-17T21:41:22.053 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-05-17T21:41:22.116 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : rbd-mirror-2:17.2.7-952.g9903f776.el9.x86_64 1/1 2024-05-17T21:41:22.143 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: rbd-mirror-2:17.2.7-952.g9903f776.el9.x86_64 1/1 2024-05-17T21:41:22.143 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-17T21:41:22.143 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-17T21:41:22.143 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-17T21:41:22.143 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-17T21:41:22.143 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-17T21:41:22.465 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-05-17T21:41:22.540 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : rbd-mirror-2:17.2.7-952.g9903f776.el9.x86_64 1/1 2024-05-17T21:41:22.565 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: rbd-mirror-2:17.2.7-952.g9903f776.el9.x86_64 1/1 2024-05-17T21:41:22.565 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-17T21:41:22.565 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-17T21:41:22.565 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-17T21:41:22.566 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-17T21:41:22.566 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:41:23.255 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : rbd-mirror-2:17.2.7-952.g9903f776.el9.x86_64 1/1 2024-05-17T21:41:23.256 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-17T21:41:23.256 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-05-17T21:41:23.256 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-mirror-2:17.2.7-952.g9903f776.el9.x86_64 2024-05-17T21:41:23.256 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-17T21:41:23.256 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-05-17T21:41:23.538 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install rbd-nbd 2024-05-17T21:41:23.622 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : rbd-mirror-2:17.2.7-952.g9903f776.el9.x86_64 1/1 2024-05-17T21:41:23.622 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:41:23.622 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-05-17T21:41:23.622 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-mirror-2:17.2.7-952.g9903f776.el9.x86_64 2024-05-17T21:41:23.622 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:41:23.622 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-05-17T21:41:23.863 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install rbd-nbd 2024-05-17T21:41:24.122 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:49 ago on Fri 17 May 2024 09:39:35 PM UTC. 2024-05-17T21:41:24.244 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-05-17T21:41:24.245 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-05-17T21:41:24.245 INFO:teuthology.orchestra.run.smithi138.stdout: Package Architecture Version Repository Size 2024-05-17T21:41:24.245 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-05-17T21:41:24.245 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-05-17T21:41:24.245 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-nbd x86_64 2:17.2.7-952.g9903f776.el9 ceph 165 k 2024-05-17T21:41:24.245 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-17T21:41:24.245 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-05-17T21:41:24.245 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-05-17T21:41:24.245 INFO:teuthology.orchestra.run.smithi138.stdout:Install 1 Package 2024-05-17T21:41:24.245 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-17T21:41:24.246 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 165 k 2024-05-17T21:41:24.246 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 477 k 2024-05-17T21:41:24.246 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-05-17T21:41:24.442 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:01:51 ago on Fri 17 May 2024 09:39:33 PM UTC. 2024-05-17T21:41:24.572 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-05-17T21:41:24.573 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-05-17T21:41:24.573 INFO:teuthology.orchestra.run.smithi107.stdout: Package Architecture Version Repository Size 2024-05-17T21:41:24.573 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-05-17T21:41:24.573 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-05-17T21:41:24.574 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-nbd x86_64 2:17.2.7-952.g9903f776.el9 ceph 165 k 2024-05-17T21:41:24.574 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:41:24.574 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-05-17T21:41:24.574 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-05-17T21:41:24.574 INFO:teuthology.orchestra.run.smithi107.stdout:Install 1 Package 2024-05-17T21:41:24.574 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:41:24.574 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 165 k 2024-05-17T21:41:24.574 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 477 k 2024-05-17T21:41:24.574 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-05-17T21:41:24.605 INFO:teuthology.orchestra.run.smithi138.stdout:rbd-nbd-17.2.7-952.g9903f776.el9.x86_64.rpm 460 kB/s | 165 kB 00:00 2024-05-17T21:41:24.606 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-05-17T21:41:24.606 INFO:teuthology.orchestra.run.smithi138.stdout:Total 458 kB/s | 165 kB 00:00 2024-05-17T21:41:24.606 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-05-17T21:41:24.614 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-05-17T21:41:24.614 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-05-17T21:41:24.677 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-05-17T21:41:24.677 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-05-17T21:41:24.838 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-05-17T21:41:24.919 INFO:teuthology.orchestra.run.smithi107.stdout:rbd-nbd-17.2.7-952.g9903f776.el9.x86_64.rpm 479 kB/s | 165 kB 00:00 2024-05-17T21:41:24.920 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-05-17T21:41:24.920 INFO:teuthology.orchestra.run.smithi107.stdout:Total 477 kB/s | 165 kB 00:00 2024-05-17T21:41:24.920 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-05-17T21:41:24.929 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-05-17T21:41:24.929 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-05-17T21:41:24.935 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : rbd-nbd-2:17.2.7-952.g9903f776.el9.x86_64 1/1 2024-05-17T21:41:24.994 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-05-17T21:41:24.995 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-05-17T21:41:25.153 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-05-17T21:41:25.259 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : rbd-nbd-2:17.2.7-952.g9903f776.el9.x86_64 1/1 2024-05-17T21:41:25.400 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: rbd-nbd-2:17.2.7-952.g9903f776.el9.x86_64 1/1 2024-05-17T21:41:25.656 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: rbd-nbd-2:17.2.7-952.g9903f776.el9.x86_64 1/1 2024-05-17T21:41:25.663 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : rbd-nbd-2:17.2.7-952.g9903f776.el9.x86_64 1/1 2024-05-17T21:41:25.663 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-17T21:41:25.664 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-05-17T21:41:25.664 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-nbd-2:17.2.7-952.g9903f776.el9.x86_64 2024-05-17T21:41:25.664 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-17T21:41:25.664 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-05-17T21:41:25.888 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : rbd-nbd-2:17.2.7-952.g9903f776.el9.x86_64 1/1 2024-05-17T21:41:25.888 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:41:25.888 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-05-17T21:41:25.888 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-nbd-2:17.2.7-952.g9903f776.el9.x86_64 2024-05-17T21:41:25.888 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:41:25.888 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-05-17T21:41:25.951 DEBUG:teuthology.parallel:result is None 2024-05-17T21:41:26.133 DEBUG:teuthology.parallel:result is None 2024-05-17T21:41:26.133 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-17T21:41:26.133 INFO:teuthology.packaging:ref: None 2024-05-17T21:41:26.133 INFO:teuthology.packaging:tag: None 2024-05-17T21:41:26.133 INFO:teuthology.packaging:branch: quincy 2024-05-17T21:41:26.133 INFO:teuthology.packaging:sha1: 931809a9e5d295bd61984693cac44fe131ded4bc 2024-05-17T21:41:26.134 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-05-17T21:41:26.367 DEBUG:teuthology.orchestra.run.smithi107:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-17T21:41:26.391 INFO:teuthology.orchestra.run.smithi107.stdout:17.2.7-952.g9903f776.el9 2024-05-17T21:41:26.391 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-952.g9903f776.el9 2024-05-17T21:41:26.391 INFO:teuthology.task.install:The correct ceph version 17.2.7-952.g9903f776 is installed. 2024-05-17T21:41:26.393 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-17T21:41:26.393 INFO:teuthology.packaging:ref: None 2024-05-17T21:41:26.393 INFO:teuthology.packaging:tag: None 2024-05-17T21:41:26.393 INFO:teuthology.packaging:branch: quincy 2024-05-17T21:41:26.393 INFO:teuthology.packaging:sha1: 931809a9e5d295bd61984693cac44fe131ded4bc 2024-05-17T21:41:26.393 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-05-17T21:41:26.705 DEBUG:teuthology.orchestra.run.smithi138:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-17T21:41:26.727 INFO:teuthology.orchestra.run.smithi138.stdout:17.2.7-952.g9903f776.el9 2024-05-17T21:41:26.727 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-952.g9903f776.el9 2024-05-17T21:41:26.727 INFO:teuthology.task.install:The correct ceph version 17.2.7-952.g9903f776 is installed. 2024-05-17T21:41:26.729 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-17T21:41:26.729 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-05-17T21:41:26.729 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-17T21:41:26.760 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-17T21:41:26.760 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-17T21:41:26.793 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-17T21:41:26.793 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-05-17T21:41:26.793 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/usr/bin/daemon-helper 2024-05-17T21:41:26.828 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-17T21:41:26.896 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-17T21:41:26.896 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/usr/bin/daemon-helper 2024-05-17T21:41:26.921 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-17T21:41:26.985 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-17T21:41:26.985 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-05-17T21:41:26.985 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-17T21:41:27.012 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-17T21:41:27.078 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-17T21:41:27.078 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-17T21:41:27.103 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-17T21:41:27.167 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-17T21:41:27.167 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-05-17T21:41:27.167 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/usr/bin/stdin-killer 2024-05-17T21:41:27.193 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-17T21:41:27.257 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-17T21:41:27.257 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/usr/bin/stdin-killer 2024-05-17T21:41:27.282 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-17T21:41:27.346 INFO:teuthology.run_tasks:Running task print... 2024-05-17T21:41:27.358 INFO:teuthology.task.print:**** done install task... 2024-05-17T21:41:27.358 INFO:teuthology.run_tasks:Running task cephadm... 2024-05-17T21:41:27.442 INFO:tasks.cephadm:Config: {'cephadm_branch': 'quincy', 'cephadm_git_url': 'https://github.com/ceph/ceph', 'conf': {'osd': {'osd_class_default_list': '*', 'osd_class_load_list': '*', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'mon pg warn min per osd': 0}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}}, 'image': 'quay.ceph.io/ceph-ci/ceph:quincy', 'roleless': True, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'OSD_DOWN', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy'], 'sha1': '931809a9e5d295bd61984693cac44fe131ded4bc'} 2024-05-17T21:41:27.442 INFO:tasks.cephadm:Cluster image is quay.ceph.io/ceph-ci/ceph:quincy 2024-05-17T21:41:27.443 INFO:tasks.cephadm:Cluster fsid is 3722731a-1496-11ef-bc9a-c7b262605968 2024-05-17T21:41:27.443 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-05-17T21:41:27.443 INFO:tasks.cephadm:No mon roles; fabricating mons 2024-05-17T21:41:27.443 INFO:tasks.cephadm:Monitor IPs: {'mon.smithi107': '172.21.15.107', 'mon.smithi138': '172.21.15.138'} 2024-05-17T21:41:27.443 INFO:tasks.cephadm:Normalizing hostnames... 2024-05-17T21:41:27.443 DEBUG:teuthology.orchestra.run.smithi107:> sudo hostname $(hostname -s) 2024-05-17T21:41:27.469 DEBUG:teuthology.orchestra.run.smithi138:> sudo hostname $(hostname -s) 2024-05-17T21:41:27.493 INFO:tasks.cephadm:Downloading cephadm (repo https://github.com/ceph/ceph ref quincy)... 2024-05-17T21:41:27.494 DEBUG:teuthology.orchestra.run.smithi107:> curl --silent https://raw.githubusercontent.com/ceph/ceph/quincy/src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-17T21:41:27.723 INFO:teuthology.orchestra.run.smithi107.stdout:-rw-r--r--. 1 ubuntu ubuntu 375153 May 17 21:41 /home/ubuntu/cephtest/cephadm 2024-05-17T21:41:27.723 DEBUG:teuthology.orchestra.run.smithi138:> curl --silent https://raw.githubusercontent.com/ceph/ceph/quincy/src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-17T21:41:27.953 INFO:teuthology.orchestra.run.smithi138.stdout:-rw-r--r--. 1 ubuntu ubuntu 375153 May 17 21:41 /home/ubuntu/cephtest/cephadm 2024-05-17T21:41:27.953 DEBUG:teuthology.orchestra.run.smithi107:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-17T21:41:27.971 DEBUG:teuthology.orchestra.run.smithi138:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-17T21:41:28.002 INFO:tasks.cephadm:Pulling image quay.ceph.io/ceph-ci/ceph:quincy on all hosts... 2024-05-17T21:41:28.003 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy pull 2024-05-17T21:41:28.013 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy pull 2024-05-17T21:41:28.932 INFO:teuthology.orchestra.run.smithi107.stderr:Pulling container image quay.ceph.io/ceph-ci/ceph:quincy... 2024-05-17T21:41:28.973 INFO:teuthology.orchestra.run.smithi138.stderr:Pulling container image quay.ceph.io/ceph-ci/ceph:quincy... 2024-05-17T21:41:58.170 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-05-17T21:41:58.170 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_version": "ceph version 17.2.7-952-g9903f776 (9903f7767386a9d9efadb7c1effdcd3064d71804) quincy (stable)", 2024-05-17T21:41:58.170 INFO:teuthology.orchestra.run.smithi107.stdout: "image_id": "d15a3685379636f882a2c894ceec9b41568bbe95b633c123d31461e59b4021a5", 2024-05-17T21:41:58.170 INFO:teuthology.orchestra.run.smithi107.stdout: "repo_digests": [ 2024-05-17T21:41:58.170 INFO:teuthology.orchestra.run.smithi107.stdout: "quay.ceph.io/ceph-ci/ceph@sha256:c200327b094672e93680375e82d2de1a08f5881ef304978f791cce179f847d60" 2024-05-17T21:41:58.170 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-05-17T21:41:58.170 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-05-17T21:42:04.513 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-17T21:42:04.513 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_version": "ceph version 17.2.7-952-g9903f776 (9903f7767386a9d9efadb7c1effdcd3064d71804) quincy (stable)", 2024-05-17T21:42:04.514 INFO:teuthology.orchestra.run.smithi138.stdout: "image_id": "d15a3685379636f882a2c894ceec9b41568bbe95b633c123d31461e59b4021a5", 2024-05-17T21:42:04.514 INFO:teuthology.orchestra.run.smithi138.stdout: "repo_digests": [ 2024-05-17T21:42:04.514 INFO:teuthology.orchestra.run.smithi138.stdout: "quay.ceph.io/ceph-ci/ceph@sha256:c200327b094672e93680375e82d2de1a08f5881ef304978f791cce179f847d60" 2024-05-17T21:42:04.514 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-05-17T21:42:04.514 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-17T21:42:04.532 DEBUG:teuthology.orchestra.run.smithi107:> sudo mkdir -p /etc/ceph 2024-05-17T21:42:04.565 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /etc/ceph 2024-05-17T21:42:04.593 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod 777 /etc/ceph 2024-05-17T21:42:04.634 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 777 /etc/ceph 2024-05-17T21:42:04.658 INFO:tasks.cephadm:Writing seed config... 2024-05-17T21:42:04.659 INFO:tasks.cephadm: override: [osd] osd_class_default_list = * 2024-05-17T21:42:04.659 INFO:tasks.cephadm: override: [osd] osd_class_load_list = * 2024-05-17T21:42:04.659 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-05-17T21:42:04.659 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-05-17T21:42:04.659 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-05-17T21:42:04.659 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-05-17T21:42:04.659 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-05-17T21:42:04.659 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-05-17T21:42:04.660 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-05-17T21:42:04.660 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-05-17T21:42:04.660 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-05-17T21:42:04.660 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-05-17T21:42:04.660 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-05-17T21:42:04.660 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-05-17T21:42:04.660 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-05-17T21:42:04.660 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-05-17T21:42:04.660 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-05-17T21:42:04.660 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-05-17T21:42:04.660 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-05-17T21:42:04.660 INFO:tasks.cephadm: override: [client] debug client = 20 2024-05-17T21:42:04.660 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-05-17T21:42:04.660 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-05-17T21:42:04.660 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-05-17T21:42:04.661 INFO:tasks.cephadm: override: [global] mon pg warn min per osd = 0 2024-05-17T21:42:04.661 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-05-17T21:42:04.661 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-05-17T21:42:04.661 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-05-17T21:42:04.661 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-05-17T21:42:04.661 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-05-17T21:42:04.661 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-05-17T21:42:04.661 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-05-17T21:42:04.661 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-05-17T21:42:04.661 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-05-17T21:42:04.661 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-05-17T21:42:04.661 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-05-17T21:42:04.661 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-05-17T21:42:04.661 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-05-17T21:42:04.662 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-05-17T21:42:04.662 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-05-17T21:42:04.662 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-05-17T21:42:04.662 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-05-17T21:42:04.662 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-05-17T21:42:04.663 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-05-17T21:42:04.663 DEBUG:teuthology.orchestra.run.smithi107:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-05-17T21:42:04.691 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 3722731a-1496-11ef-bc9a-c7b262605968 mon pg warn min per osd = 0 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = true bdev debug aio = true osd sloppy crc = true osd_class_default_list = * osd_class_load_list = * bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-05-17T21:42:04.691 DEBUG:teuthology.orchestra.run.smithi107:mon.smithi107> sudo journalctl -f -n 0 -u ceph-3722731a-1496-11ef-bc9a-c7b262605968@mon.smithi107.service 2024-05-17T21:42:04.733 INFO:tasks.cephadm:Bootstrapping... 2024-05-17T21:42:04.733 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy -v bootstrap --fsid 3722731a-1496-11ef-bc9a-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-ip 172.21.15.107 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-05-17T21:42:04.939 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-05-17T21:42:04.939 INFO:teuthology.orchestra.run.smithi107.stdout:cephadm ['--image', 'quay.ceph.io/ceph-ci/ceph:quincy', '-v', 'bootstrap', '--fsid', '3722731a-1496-11ef-bc9a-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-ip', '172.21.15.107', '--skip-admin-label'] 2024-05-17T21:42:04.965 INFO:teuthology.orchestra.run.smithi107.stdout:/bin/podman: stdout 5.0.2 2024-05-17T21:42:04.965 INFO:teuthology.orchestra.run.smithi107.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-05-17T21:42:04.965 INFO:teuthology.orchestra.run.smithi107.stdout:Verifying podman|docker is present... 2024-05-17T21:42:04.989 INFO:teuthology.orchestra.run.smithi107.stdout:/bin/podman: stdout 5.0.2 2024-05-17T21:42:04.989 INFO:teuthology.orchestra.run.smithi107.stdout:Verifying lvm2 is present... 2024-05-17T21:42:04.989 INFO:teuthology.orchestra.run.smithi107.stdout:Verifying time synchronization is in place... 2024-05-17T21:42:04.997 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-17T21:42:04.997 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-17T21:42:05.003 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-17T21:42:05.003 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout inactive 2024-05-17T21:42:05.010 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout enabled 2024-05-17T21:42:05.017 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout active 2024-05-17T21:42:05.017 INFO:teuthology.orchestra.run.smithi107.stdout:Unit chronyd.service is enabled and running 2024-05-17T21:42:05.017 INFO:teuthology.orchestra.run.smithi107.stdout:Repeating the final host check... 2024-05-17T21:42:05.041 INFO:teuthology.orchestra.run.smithi107.stdout:/bin/podman: stdout 5.0.2 2024-05-17T21:42:05.041 INFO:teuthology.orchestra.run.smithi107.stdout:podman (/bin/podman) version 5.0.2 is present 2024-05-17T21:42:05.041 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl is present 2024-05-17T21:42:05.041 INFO:teuthology.orchestra.run.smithi107.stdout:lvcreate is present 2024-05-17T21:42:05.048 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-17T21:42:05.048 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-17T21:42:05.054 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-17T21:42:05.054 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout inactive 2024-05-17T21:42:05.061 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout enabled 2024-05-17T21:42:05.068 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout active 2024-05-17T21:42:05.068 INFO:teuthology.orchestra.run.smithi107.stdout:Unit chronyd.service is enabled and running 2024-05-17T21:42:05.068 INFO:teuthology.orchestra.run.smithi107.stdout:Host looks OK 2024-05-17T21:42:05.068 INFO:teuthology.orchestra.run.smithi107.stdout:Cluster fsid: 3722731a-1496-11ef-bc9a-c7b262605968 2024-05-17T21:42:05.068 INFO:teuthology.orchestra.run.smithi107.stdout:Acquiring lock 140673232400832 on /run/cephadm/3722731a-1496-11ef-bc9a-c7b262605968.lock 2024-05-17T21:42:05.068 INFO:teuthology.orchestra.run.smithi107.stdout:Lock 140673232400832 acquired on /run/cephadm/3722731a-1496-11ef-bc9a-c7b262605968.lock 2024-05-17T21:42:05.068 INFO:teuthology.orchestra.run.smithi107.stdout:Verifying IP 172.21.15.107 port 3300 ... 2024-05-17T21:42:05.069 INFO:teuthology.orchestra.run.smithi107.stdout:Verifying IP 172.21.15.107 port 6789 ... 2024-05-17T21:42:05.069 INFO:teuthology.orchestra.run.smithi107.stdout:Base mon IP(s) is [172.21.15.107:3300, 172.21.15.107:6789], mon addrv is [v2:172.21.15.107:3300,v1:172.21.15.107:6789] 2024-05-17T21:42:05.072 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.107 metric 100 2024-05-17T21:42:05.072 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.107 metric 100 2024-05-17T21:42:05.075 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-05-17T21:42:05.075 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-05-17T21:42:05.075 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1793sec hoplimit 64 pref medium 2024-05-17T21:42:05.078 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-05-17T21:42:05.078 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-05-17T21:42:05.078 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-17T21:42:05.078 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-05-17T21:42:05.078 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:47eb/64 scope link 2024-05-17T21:42:05.078 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-17T21:42:05.079 INFO:teuthology.orchestra.run.smithi107.stdout:Mon IP `172.21.15.107` is in CIDR network `172.21.0.0/20` 2024-05-17T21:42:05.079 INFO:teuthology.orchestra.run.smithi107.stdout:Mon IP `172.21.15.107` is in CIDR network `172.21.0.0/20` 2024-05-17T21:42:05.079 INFO:teuthology.orchestra.run.smithi107.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-05-17T21:42:05.079 INFO:teuthology.orchestra.run.smithi107.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-05-17T21:42:05.080 INFO:teuthology.orchestra.run.smithi107.stdout:Pulling container image quay.ceph.io/ceph-ci/ceph:quincy... 2024-05-17T21:42:05.592 INFO:teuthology.orchestra.run.smithi107.stdout:/bin/podman: stdout d15a3685379636f882a2c894ceec9b41568bbe95b633c123d31461e59b4021a5 2024-05-17T21:42:05.593 INFO:teuthology.orchestra.run.smithi107.stdout:/bin/podman: stderr Trying to pull quay.ceph.io/ceph-ci/ceph:quincy... 2024-05-17T21:42:05.593 INFO:teuthology.orchestra.run.smithi107.stdout:/bin/podman: stderr Getting image source signatures 2024-05-17T21:42:05.593 INFO:teuthology.orchestra.run.smithi107.stdout:/bin/podman: stderr Copying blob sha256:aeb7750101a386635a8f7f90151468b123e494b020b102c66f29e0ee60e36ab5 2024-05-17T21:42:05.593 INFO:teuthology.orchestra.run.smithi107.stdout:/bin/podman: stderr Copying blob sha256:d0e808cd455827ca3108db1b0125dcee9ffc148daa1d7dde963f550bc831d501 2024-05-17T21:42:05.593 INFO:teuthology.orchestra.run.smithi107.stdout:/bin/podman: stderr Copying config sha256:d15a3685379636f882a2c894ceec9b41568bbe95b633c123d31461e59b4021a5 2024-05-17T21:42:05.593 INFO:teuthology.orchestra.run.smithi107.stdout:/bin/podman: stderr Writing manifest to image destination 2024-05-17T21:42:06.515 INFO:teuthology.orchestra.run.smithi107.stdout:ceph: stdout ceph version 17.2.7-952-g9903f776 (9903f7767386a9d9efadb7c1effdcd3064d71804) quincy (stable) 2024-05-17T21:42:06.515 INFO:teuthology.orchestra.run.smithi107.stdout:Ceph version: ceph version 17.2.7-952-g9903f776 (9903f7767386a9d9efadb7c1effdcd3064d71804) quincy (stable) 2024-05-17T21:42:06.515 INFO:teuthology.orchestra.run.smithi107.stdout:Extracting ceph user uid/gid from container image... 2024-05-17T21:42:07.445 INFO:teuthology.orchestra.run.smithi107.stdout:stat: stdout 167 167 2024-05-17T21:42:07.445 INFO:teuthology.orchestra.run.smithi107.stdout:Creating initial keys... 2024-05-17T21:42:08.350 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph-authtool: stdout AQAvz0dmKk+eLxAAud637kJhDqiDNQEg2948DQ== 2024-05-17T21:42:09.298 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph-authtool: stdout AQAwz0dm9RoEKhAAFzMVphwwWOa8X5cBZhgjEA== 2024-05-17T21:42:10.171 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph-authtool: stdout AQAxz0dmBK7pIxAAJxM5pQORoUCyhhUHwI8l8w== 2024-05-17T21:42:10.172 INFO:teuthology.orchestra.run.smithi107.stdout:Creating initial monmap... 2024-05-17T21:42:11.111 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-17T21:42:11.111 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = octopus 2024-05-17T21:42:11.111 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 3722731a-1496-11ef-bc9a-c7b262605968 2024-05-17T21:42:11.111 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-17T21:42:11.112 INFO:teuthology.orchestra.run.smithi107.stdout:monmaptool for smithi107 [v2:172.21.15.107:3300,v1:172.21.15.107:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-17T21:42:11.112 INFO:teuthology.orchestra.run.smithi107.stdout:setting min_mon_release = octopus 2024-05-17T21:42:11.112 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/monmaptool: set fsid to 3722731a-1496-11ef-bc9a-c7b262605968 2024-05-17T21:42:11.112 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-17T21:42:11.112 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:42:11.112 INFO:teuthology.orchestra.run.smithi107.stdout:Creating mon... 2024-05-17T21:42:12.076 INFO:teuthology.orchestra.run.smithi107.stdout:create mon.smithi107 on 2024-05-17T21:42:12.280 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-05-17T21:42:12.416 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-05-17T21:42:12.558 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-3722731a-1496-11ef-bc9a-c7b262605968.target → /etc/systemd/system/ceph-3722731a-1496-11ef-bc9a-c7b262605968.target. 2024-05-17T21:42:12.558 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-3722731a-1496-11ef-bc9a-c7b262605968.target → /etc/systemd/system/ceph-3722731a-1496-11ef-bc9a-c7b262605968.target. 2024-05-17T21:42:12.722 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-3722731a-1496-11ef-bc9a-c7b262605968@mon.smithi107 2024-05-17T21:42:12.722 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Failed to reset failed state of unit ceph-3722731a-1496-11ef-bc9a-c7b262605968@mon.smithi107.service: Unit ceph-3722731a-1496-11ef-bc9a-c7b262605968@mon.smithi107.service not loaded. 2024-05-17T21:42:12.878 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-3722731a-1496-11ef-bc9a-c7b262605968.target.wants/ceph-3722731a-1496-11ef-bc9a-c7b262605968@mon.smithi107.service → /etc/systemd/system/ceph-3722731a-1496-11ef-bc9a-c7b262605968@.service. 2024-05-17T21:42:13.347 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:13 smithi107 podman[28021]: 2024-05-17 21:42:13.353813629 +0000 UTC m=+0.344145626 container create d84975310e6645171c6a04e267e9db323e2db4ed541a685748a38bdce9f6e88a (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-mon-smithi107, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=quincy-9903f77, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-17T21:42:15.354 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:15 smithi107 podman[28021]: 2024-05-17 21:42:15.0867453 +0000 UTC m=+2.077077284 container init d84975310e6645171c6a04e267e9db323e2db4ed541a685748a38bdce9f6e88a (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-mon-smithi107, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=quincy-9903f77, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-05-17T21:42:15.355 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:15 smithi107 podman[28021]: 2024-05-17 21:42:15.090552473 +0000 UTC m=+2.080884448 container start d84975310e6645171c6a04e267e9db323e2db4ed541a685748a38bdce9f6e88a (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-mon-smithi107, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=quincy-9903f77, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0) 2024-05-17T21:42:15.355 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:15 smithi107 ceph-mon[28052]: mkfs 3722731a-1496-11ef-bc9a-c7b262605968 2024-05-17T21:42:15.355 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:15 smithi107 ceph-mon[28052]: mon.smithi107 is new leader, mons smithi107 in quorum (ranks 0) 2024-05-17T21:42:16.029 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-17T21:42:16.029 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout disabled 2024-05-17T21:42:16.038 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-17T21:42:16.038 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout inactive 2024-05-17T21:42:16.038 INFO:teuthology.orchestra.run.smithi107.stdout:firewalld.service is not enabled 2024-05-17T21:42:16.038 INFO:teuthology.orchestra.run.smithi107.stdout:Not possible to enable service . firewalld.service is not available 2024-05-17T21:42:16.039 INFO:teuthology.orchestra.run.smithi107.stdout:Waiting for mon to start... 2024-05-17T21:42:16.039 INFO:teuthology.orchestra.run.smithi107.stdout:Waiting for mon... 2024-05-17T21:42:16.354 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:16 smithi107 bash[28021]: d84975310e6645171c6a04e267e9db323e2db4ed541a685748a38bdce9f6e88a 2024-05-17T21:42:16.354 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:16 smithi107 systemd[1]: Started Ceph mon.smithi107 for 3722731a-1496-11ef-bc9a-c7b262605968. 2024-05-17T21:42:16.355 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:16 smithi107 ceph-mon[28052]: mon.smithi107 is new leader, mons smithi107 in quorum (ranks 0) 2024-05-17T21:42:16.355 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:16 smithi107 ceph-mon[28052]: monmap e1: 1 mons at {smithi107=[v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-17T21:42:16.355 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:16 smithi107 ceph-mon[28052]: fsmap 2024-05-17T21:42:16.355 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:16 smithi107 ceph-mon[28052]: osdmap e1: 0 total, 0 up, 0 in 2024-05-17T21:42:16.355 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:16 smithi107 ceph-mon[28052]: mgrmap e1: no daemons active 2024-05-17T21:42:19.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:19 smithi107 ceph-mon[28052]: from='client.? 172.21.15.107:0/439657024' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-05-17T21:42:25.868 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout cluster: 2024-05-17T21:42:25.868 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout id: 3722731a-1496-11ef-bc9a-c7b262605968 2024-05-17T21:42:25.868 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-05-17T21:42:25.868 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-05-17T21:42:25.868 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout services: 2024-05-17T21:42:25.868 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum smithi107 (age 4s) 2024-05-17T21:42:25.868 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-05-17T21:42:25.868 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-05-17T21:42:25.868 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-05-17T21:42:25.868 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout data: 2024-05-17T21:42:25.869 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-05-17T21:42:25.869 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-05-17T21:42:25.869 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-05-17T21:42:25.869 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout pgs: 2024-05-17T21:42:25.869 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-05-17T21:42:25.869 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:19.391+0000 7fb9b85e3700 1 Processor -- start 2024-05-17T21:42:25.869 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:19.391+0000 7fb9b85e3700 1 -- start start 2024-05-17T21:42:25.869 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:19.391+0000 7fb9b85e3700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9b0102880 0x7fb9b0102c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:42:25.869 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:19.391+0000 7fb9b85e3700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b01031d0 con 0x7fb9b0102880 2024-05-17T21:42:25.869 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:19.392+0000 7fb9b637f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9b0102880 0x7fb9b0102c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:42:25.869 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:19.392+0000 7fb9b637f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9b0102880 0x7fb9b0102c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39222/0 (socket says 172.21.15.107:39222) 2024-05-17T21:42:25.869 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:19.392+0000 7fb9b637f700 1 -- 172.21.15.107:0/925389752 learned_addr learned my addr 172.21.15.107:0/925389752 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:42:25.869 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:19.392+0000 7fb9b637f700 1 -- 172.21.15.107:0/925389752 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9b0103310 con 0x7fb9b0102880 2024-05-17T21:42:25.870 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:19.392+0000 7fb9b637f700 1 --2- 172.21.15.107:0/925389752 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9b0102880 0x7fb9b0102c90 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac0098e0 tx=0x7fb9ac009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=699c705414b528db server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:42:25.870 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:19.393+0000 7fb9b537d700 1 -- 172.21.15.107:0/925389752 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb9ac00b860 con 0x7fb9b0102880 2024-05-17T21:42:25.870 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:19.393+0000 7fb9b537d700 1 -- 172.21.15.107:0/925389752 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9ac00b9c0 con 0x7fb9b0102880 2024-05-17T21:42:25.870 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:19.393+0000 7fb9b537d700 1 -- 172.21.15.107:0/925389752 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb9ac00bc90 con 0x7fb9b0102880 2024-05-17T21:42:25.870 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:19.393+0000 7fb9b85e3700 1 -- 172.21.15.107:0/925389752 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9b0102880 msgr2=0x7fb9b0102c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:42:25.870 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:19.393+0000 7fb9b85e3700 1 --2- 172.21.15.107:0/925389752 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9b0102880 0x7fb9b0102c90 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac0098e0 tx=0x7fb9ac009250 comp rx=0 tx=0).stop 2024-05-17T21:42:25.870 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:19.393+0000 7fb9b85e3700 1 -- 172.21.15.107:0/925389752 shutdown_connections 2024-05-17T21:42:25.870 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:19.393+0000 7fb9b85e3700 1 --2- 172.21.15.107:0/925389752 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9b0102880 0x7fb9b0102c90 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:42:25.870 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:19.393+0000 7fb9b85e3700 1 -- 172.21.15.107:0/925389752 >> 172.21.15.107:0/925389752 conn(0x7fb9b00fe060 msgr2=0x7fb9b0100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:42:25.870 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:19.393+0000 7fb9b85e3700 1 -- 172.21.15.107:0/925389752 shutdown_connections 2024-05-17T21:42:25.870 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:19.393+0000 7fb9b85e3700 1 -- 172.21.15.107:0/925389752 wait complete. 2024-05-17T21:42:25.870 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:19.394+0000 7fb9b85e3700 1 Processor -- start 2024-05-17T21:42:25.870 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:19.394+0000 7fb9b85e3700 1 -- start start 2024-05-17T21:42:25.871 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:19.394+0000 7fb9b85e3700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9b0102880 0x7fb9b0110e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:42:25.871 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:19.394+0000 7fb9b85e3700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b01113a0 con 0x7fb9b0102880 2024-05-17T21:42:25.871 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:19.394+0000 7fb9b637f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9b0102880 0x7fb9b0110e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:42:25.871 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:19.394+0000 7fb9b637f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9b0102880 0x7fb9b0110e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39230/0 (socket says 172.21.15.107:39230) 2024-05-17T21:42:25.871 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:19.394+0000 7fb9b637f700 1 -- 172.21.15.107:0/439657024 learned_addr learned my addr 172.21.15.107:0/439657024 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:42:25.871 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:19.394+0000 7fb9b637f700 1 -- 172.21.15.107:0/439657024 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9ac0094f0 con 0x7fb9b0102880 2024-05-17T21:42:25.871 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:19.395+0000 7fb9b637f700 1 --2- 172.21.15.107:0/439657024 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9b0102880 0x7fb9b0110e60 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac003080 tx=0x7fb9ac016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:42:25.871 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:19.395+0000 7fb9a6ffd700 1 -- 172.21.15.107:0/439657024 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb9ac01c5c0 con 0x7fb9b0102880 2024-05-17T21:42:25.871 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:19.395+0000 7fb9a6ffd700 1 -- 172.21.15.107:0/439657024 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9ac01c720 con 0x7fb9b0102880 2024-05-17T21:42:25.871 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:19.395+0000 7fb9a6ffd700 1 -- 172.21.15.107:0/439657024 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb9ac01ca30 con 0x7fb9b0102880 2024-05-17T21:42:25.871 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:19.395+0000 7fb9b85e3700 1 -- 172.21.15.107:0/439657024 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9b01115a0 con 0x7fb9b0102880 2024-05-17T21:42:25.871 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:19.395+0000 7fb9b85e3700 1 -- 172.21.15.107:0/439657024 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9b0111fd0 con 0x7fb9b0102880 2024-05-17T21:42:25.871 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:19.395+0000 7fb9a6ffd700 1 -- 172.21.15.107:0/439657024 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7fb9ac01cc60 con 0x7fb9b0102880 2024-05-17T21:42:25.871 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:19.395+0000 7fb9a6ffd700 1 -- 172.21.15.107:0/439657024 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fb9ac01ac80 con 0x7fb9b0102880 2024-05-17T21:42:25.872 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:19.396+0000 7fb9b85e3700 1 -- 172.21.15.107:0/439657024 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9b005f5c0 con 0x7fb9b0102880 2024-05-17T21:42:25.872 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:19.398+0000 7fb9a6ffd700 1 -- 172.21.15.107:0/439657024 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71547 (secure 0 0 0) 0x7fb9ac01a450 con 0x7fb9b0102880 2024-05-17T21:42:25.872 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:19.468+0000 7fb9b85e3700 1 -- 172.21.15.107:0/439657024 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7fb9b0112a80 con 0x7fb9b0102880 2024-05-17T21:42:25.872 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:19.469+0000 7fb9a6ffd700 1 -- 172.21.15.107:0/439657024 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+318 (secure 0 0 0) 0x7fb9ac025440 con 0x7fb9b0102880 2024-05-17T21:42:25.872 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:19.469+0000 7fb9b85e3700 1 -- 172.21.15.107:0/439657024 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9b0102880 msgr2=0x7fb9b0110e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:42:25.872 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:19.469+0000 7fb9b85e3700 1 --2- 172.21.15.107:0/439657024 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9b0102880 0x7fb9b0110e60 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac003080 tx=0x7fb9ac016000 comp rx=0 tx=0).stop 2024-05-17T21:42:25.872 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:19.470+0000 7fb9b85e3700 1 -- 172.21.15.107:0/439657024 shutdown_connections 2024-05-17T21:42:25.872 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:19.470+0000 7fb9b85e3700 1 --2- 172.21.15.107:0/439657024 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9b0102880 0x7fb9b0110e60 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:42:25.872 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:19.470+0000 7fb9b85e3700 1 -- 172.21.15.107:0/439657024 >> 172.21.15.107:0/439657024 conn(0x7fb9b00fe060 msgr2=0x7fb9b0105250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:42:25.872 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:19.470+0000 7fb9b85e3700 1 -- 172.21.15.107:0/439657024 shutdown_connections 2024-05-17T21:42:25.872 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:19.470+0000 7fb9b85e3700 1 -- 172.21.15.107:0/439657024 wait complete. 2024-05-17T21:42:25.872 INFO:teuthology.orchestra.run.smithi107.stdout:mon is available 2024-05-17T21:42:25.872 INFO:teuthology.orchestra.run.smithi107.stdout:Assimilating anything we can from ceph.conf... 2024-05-17T21:42:26.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:26 smithi107 ceph-mon[28052]: from='client.? 172.21.15.107:0/1680907502' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-17T21:42:26.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:26 smithi107 ceph-mon[28052]: from='client.? 172.21.15.107:0/1680907502' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-05-17T21:42:27.141 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-05-17T21:42:27.141 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout [global] 2024-05-17T21:42:27.141 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout fsid = 3722731a-1496-11ef-bc9a-c7b262605968 2024-05-17T21:42:27.141 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.107:3300,v1:172.21.15.107:6789] 2024-05-17T21:42:27.141 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-17T21:42:27.141 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-17T21:42:27.141 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-17T21:42:27.142 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-17T21:42:27.142 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-05-17T21:42:27.142 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-17T21:42:27.142 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-17T21:42:27.142 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-05-17T21:42:27.142 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout [osd] 2024-05-17T21:42:27.142 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-17T21:42:27.142 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-17T21:42:27.142 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-17T21:42:27.142 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-17T21:42:27.142 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-17T21:42:27.143 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:26.523+0000 7f892699e700 1 Processor -- start 2024-05-17T21:42:27.143 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:26.523+0000 7f892699e700 1 -- start start 2024-05-17T21:42:27.143 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:26.523+0000 7f892699e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8920076bf0 0x7f8920107860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:42:27.143 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:26.523+0000 7f892699e700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8920103250 con 0x7f8920076bf0 2024-05-17T21:42:27.143 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:26.524+0000 7f891ffff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8920076bf0 0x7f8920107860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:42:27.143 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:26.524+0000 7f891ffff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8920076bf0 0x7f8920107860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42498/0 (socket says 172.21.15.107:42498) 2024-05-17T21:42:27.143 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:26.524+0000 7f891ffff700 1 -- 172.21.15.107:0/3939349876 learned_addr learned my addr 172.21.15.107:0/3939349876 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:42:27.143 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:26.524+0000 7f891ffff700 1 -- 172.21.15.107:0/3939349876 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8920107da0 con 0x7f8920076bf0 2024-05-17T21:42:27.143 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:26.524+0000 7f891ffff700 1 --2- 172.21.15.107:0/3939349876 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8920076bf0 0x7f8920107860 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f89100098e0 tx=0x7f8910009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a756654494ce0c23 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:42:27.143 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:26.524+0000 7f891effd700 1 -- 172.21.15.107:0/3939349876 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f891000b860 con 0x7f8920076bf0 2024-05-17T21:42:27.143 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:26.524+0000 7f891effd700 1 -- 172.21.15.107:0/3939349876 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f891000b9c0 con 0x7f8920076bf0 2024-05-17T21:42:27.143 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:26.525+0000 7f891effd700 1 -- 172.21.15.107:0/3939349876 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f891000bc90 con 0x7f8920076bf0 2024-05-17T21:42:27.143 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:26.525+0000 7f892699e700 1 -- 172.21.15.107:0/3939349876 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8920076bf0 msgr2=0x7f8920107860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:42:27.143 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:26.525+0000 7f892699e700 1 --2- 172.21.15.107:0/3939349876 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8920076bf0 0x7f8920107860 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f89100098e0 tx=0x7f8910009250 comp rx=0 tx=0).stop 2024-05-17T21:42:27.144 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:26.525+0000 7f892699e700 1 -- 172.21.15.107:0/3939349876 shutdown_connections 2024-05-17T21:42:27.144 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:26.525+0000 7f892699e700 1 --2- 172.21.15.107:0/3939349876 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8920076bf0 0x7f8920107860 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:42:27.144 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:26.525+0000 7f892699e700 1 -- 172.21.15.107:0/3939349876 >> 172.21.15.107:0/3939349876 conn(0x7f89200fcef0 msgr2=0x7f89200ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:42:27.144 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:26.525+0000 7f892699e700 1 -- 172.21.15.107:0/3939349876 shutdown_connections 2024-05-17T21:42:27.144 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:26.525+0000 7f892699e700 1 -- 172.21.15.107:0/3939349876 wait complete. 2024-05-17T21:42:27.144 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:26.526+0000 7f892699e700 1 Processor -- start 2024-05-17T21:42:27.144 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:26.526+0000 7f892699e700 1 -- start start 2024-05-17T21:42:27.144 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:26.526+0000 7f892699e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8920076bf0 0x7f8920112fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:42:27.144 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:26.526+0000 7f892699e700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89201134f0 con 0x7f8920076bf0 2024-05-17T21:42:27.144 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:26.526+0000 7f891ffff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8920076bf0 0x7f8920112fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:42:27.144 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:26.526+0000 7f891ffff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8920076bf0 0x7f8920112fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42510/0 (socket says 172.21.15.107:42510) 2024-05-17T21:42:27.144 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:26.526+0000 7f891ffff700 1 -- 172.21.15.107:0/1680907502 learned_addr learned my addr 172.21.15.107:0/1680907502 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:42:27.144 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:26.526+0000 7f891ffff700 1 -- 172.21.15.107:0/1680907502 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89100094f0 con 0x7f8920076bf0 2024-05-17T21:42:27.145 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:26.527+0000 7f891ffff700 1 --2- 172.21.15.107:0/1680907502 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8920076bf0 0x7f8920112fb0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f8910009290 tx=0x7f8910016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:42:27.145 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:26.527+0000 7f891cff9700 1 -- 172.21.15.107:0/1680907502 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f891001c5c0 con 0x7f8920076bf0 2024-05-17T21:42:27.145 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:26.527+0000 7f891cff9700 1 -- 172.21.15.107:0/1680907502 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f891001c740 con 0x7f8920076bf0 2024-05-17T21:42:27.145 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:26.527+0000 7f892699e700 1 -- 172.21.15.107:0/1680907502 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89201136f0 con 0x7f8920076bf0 2024-05-17T21:42:27.145 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:26.527+0000 7f891cff9700 1 -- 172.21.15.107:0/1680907502 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f891001ca50 con 0x7f8920076bf0 2024-05-17T21:42:27.145 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:26.527+0000 7f892699e700 1 -- 172.21.15.107:0/1680907502 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8920114260 con 0x7f8920076bf0 2024-05-17T21:42:27.145 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:26.527+0000 7f891cff9700 1 -- 172.21.15.107:0/1680907502 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f891001cbb0 con 0x7f8920076bf0 2024-05-17T21:42:27.145 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:26.527+0000 7f891cff9700 1 -- 172.21.15.107:0/1680907502 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f891001ac30 con 0x7f8920076bf0 2024-05-17T21:42:27.145 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:26.528+0000 7f892699e700 1 -- 172.21.15.107:0/1680907502 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f892005f5c0 con 0x7f8920076bf0 2024-05-17T21:42:27.145 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:26.530+0000 7f891cff9700 1 -- 172.21.15.107:0/1680907502 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71547 (secure 0 0 0) 0x7f891002e440 con 0x7f8920076bf0 2024-05-17T21:42:27.145 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:26.596+0000 7f892699e700 1 -- 172.21.15.107:0/1680907502 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f8920108120 con 0x7f8920076bf0 2024-05-17T21:42:27.145 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:26.599+0000 7f891cff9700 1 -- 172.21.15.107:0/1680907502 <== mon.0 v2:172.21.15.107:3300/0 7 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f891001ae90 con 0x7f8920076bf0 2024-05-17T21:42:27.145 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:26.600+0000 7f891cff9700 1 -- 172.21.15.107:0/1680907502 <== mon.0 v2:172.21.15.107:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+431 (secure 0 0 0) 0x7f8920108120 con 0x7f8920076bf0 2024-05-17T21:42:27.146 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:26.601+0000 7f892699e700 1 -- 172.21.15.107:0/1680907502 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8920076bf0 msgr2=0x7f8920112fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:42:27.146 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:26.601+0000 7f892699e700 1 --2- 172.21.15.107:0/1680907502 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8920076bf0 0x7f8920112fb0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f8910009290 tx=0x7f8910016000 comp rx=0 tx=0).stop 2024-05-17T21:42:27.146 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:26.601+0000 7f892699e700 1 -- 172.21.15.107:0/1680907502 shutdown_connections 2024-05-17T21:42:27.146 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:26.601+0000 7f892699e700 1 --2- 172.21.15.107:0/1680907502 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8920076bf0 0x7f8920112fb0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:42:27.146 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:26.601+0000 7f892699e700 1 -- 172.21.15.107:0/1680907502 >> 172.21.15.107:0/1680907502 conn(0x7f89200fcef0 msgr2=0x7f89200fd2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:42:27.146 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:26.601+0000 7f892699e700 1 -- 172.21.15.107:0/1680907502 shutdown_connections 2024-05-17T21:42:27.146 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:26.601+0000 7f892699e700 1 -- 172.21.15.107:0/1680907502 wait complete. 2024-05-17T21:42:27.146 INFO:teuthology.orchestra.run.smithi107.stdout:Generating new minimal ceph.conf... 2024-05-17T21:42:27.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:27 smithi107 ceph-mon[28052]: from='client.? 172.21.15.107:0/3155819592' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:42:28.130 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:27.527+0000 7f6d32cf7700 1 Processor -- start 2024-05-17T21:42:28.130 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:27.528+0000 7f6d32cf7700 1 -- start start 2024-05-17T21:42:28.130 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:27.528+0000 7f6d32cf7700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d2c101d60 0x7f6d2c102170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:42:28.130 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:27.528+0000 7f6d32cf7700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d2c1026b0 con 0x7f6d2c101d60 2024-05-17T21:42:28.130 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:27.528+0000 7f6d30a93700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d2c101d60 0x7f6d2c102170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:42:28.130 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:27.528+0000 7f6d30a93700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d2c101d60 0x7f6d2c102170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42518/0 (socket says 172.21.15.107:42518) 2024-05-17T21:42:28.130 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:27.528+0000 7f6d30a93700 1 -- 172.21.15.107:0/4083210602 learned_addr learned my addr 172.21.15.107:0/4083210602 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:42:28.130 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:27.528+0000 7f6d30a93700 1 -- 172.21.15.107:0/4083210602 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d2c1027f0 con 0x7f6d2c101d60 2024-05-17T21:42:28.130 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:27.528+0000 7f6d30a93700 1 --2- 172.21.15.107:0/4083210602 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d2c101d60 0x7f6d2c102170 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f6d240098e0 tx=0x7f6d24009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77db751a725e2c59 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:42:28.130 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:27.528+0000 7f6d2b7fe700 1 -- 172.21.15.107:0/4083210602 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6d2400b860 con 0x7f6d2c101d60 2024-05-17T21:42:28.130 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:27.528+0000 7f6d2b7fe700 1 -- 172.21.15.107:0/4083210602 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f6d2400be50 con 0x7f6d2c101d60 2024-05-17T21:42:28.131 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:27.528+0000 7f6d2b7fe700 1 -- 172.21.15.107:0/4083210602 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6d24010380 con 0x7f6d2c101d60 2024-05-17T21:42:28.131 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:27.529+0000 7f6d32cf7700 1 -- 172.21.15.107:0/4083210602 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d2c101d60 msgr2=0x7f6d2c102170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:42:28.131 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:27.529+0000 7f6d32cf7700 1 --2- 172.21.15.107:0/4083210602 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d2c101d60 0x7f6d2c102170 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f6d240098e0 tx=0x7f6d24009250 comp rx=0 tx=0).stop 2024-05-17T21:42:28.131 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:27.529+0000 7f6d32cf7700 1 -- 172.21.15.107:0/4083210602 shutdown_connections 2024-05-17T21:42:28.131 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:27.529+0000 7f6d32cf7700 1 --2- 172.21.15.107:0/4083210602 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d2c101d60 0x7f6d2c102170 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:42:28.131 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:27.529+0000 7f6d32cf7700 1 -- 172.21.15.107:0/4083210602 >> 172.21.15.107:0/4083210602 conn(0x7f6d2c0fc6f0 msgr2=0x7f6d2c0feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:42:28.131 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:27.529+0000 7f6d32cf7700 1 -- 172.21.15.107:0/4083210602 shutdown_connections 2024-05-17T21:42:28.131 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:27.529+0000 7f6d32cf7700 1 -- 172.21.15.107:0/4083210602 wait complete. 2024-05-17T21:42:28.131 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:27.530+0000 7f6d32cf7700 1 Processor -- start 2024-05-17T21:42:28.132 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:27.530+0000 7f6d32cf7700 1 -- start start 2024-05-17T21:42:28.132 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:27.530+0000 7f6d32cf7700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d2c101d60 0x7f6d2c1015c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:42:28.132 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:27.530+0000 7f6d32cf7700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d2c114280 con 0x7f6d2c101d60 2024-05-17T21:42:28.132 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:27.530+0000 7f6d30a93700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d2c101d60 0x7f6d2c1015c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:42:28.132 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:27.530+0000 7f6d30a93700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d2c101d60 0x7f6d2c1015c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42524/0 (socket says 172.21.15.107:42524) 2024-05-17T21:42:28.132 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:27.530+0000 7f6d30a93700 1 -- 172.21.15.107:0/3155819592 learned_addr learned my addr 172.21.15.107:0/3155819592 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:42:28.133 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:27.530+0000 7f6d30a93700 1 -- 172.21.15.107:0/3155819592 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d240094f0 con 0x7f6d2c101d60 2024-05-17T21:42:28.133 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:27.530+0000 7f6d30a93700 1 --2- 172.21.15.107:0/3155819592 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d2c101d60 0x7f6d2c1015c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6d2400b440 tx=0x7f6d24009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:42:28.133 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:27.531+0000 7f6d297fa700 1 -- 172.21.15.107:0/3155819592 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6d24010760 con 0x7f6d2c101d60 2024-05-17T21:42:28.133 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:27.531+0000 7f6d297fa700 1 -- 172.21.15.107:0/3155819592 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f6d24010d50 con 0x7f6d2c101d60 2024-05-17T21:42:28.133 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:27.531+0000 7f6d297fa700 1 -- 172.21.15.107:0/3155819592 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6d2401ae20 con 0x7f6d2c101d60 2024-05-17T21:42:28.133 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:27.531+0000 7f6d32cf7700 1 -- 172.21.15.107:0/3155819592 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d2c1143c0 con 0x7f6d2c101d60 2024-05-17T21:42:28.134 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:27.531+0000 7f6d32cf7700 1 -- 172.21.15.107:0/3155819592 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d2c1148b0 con 0x7f6d2c101d60 2024-05-17T21:42:28.134 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:27.531+0000 7f6d297fa700 1 -- 172.21.15.107:0/3155819592 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f6d2402e440 con 0x7f6d2c101d60 2024-05-17T21:42:28.134 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:27.531+0000 7f6d297fa700 1 -- 172.21.15.107:0/3155819592 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f6d24024740 con 0x7f6d2c101d60 2024-05-17T21:42:28.134 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:27.532+0000 7f6d32cf7700 1 -- 172.21.15.107:0/3155819592 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d2c04bed0 con 0x7f6d2c101d60 2024-05-17T21:42:28.134 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:27.534+0000 7f6d297fa700 1 -- 172.21.15.107:0/3155819592 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71547 (secure 0 0 0) 0x7f6d24029070 con 0x7f6d2c101d60 2024-05-17T21:42:28.134 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:27.600+0000 7f6d32cf7700 1 -- 172.21.15.107:0/3155819592 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f6d2c103660 con 0x7f6d2c101d60 2024-05-17T21:42:28.134 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:27.600+0000 7f6d297fa700 1 -- 172.21.15.107:0/3155819592 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+177 (secure 0 0 0) 0x7f6d2401a6a0 con 0x7f6d2c101d60 2024-05-17T21:42:28.135 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:27.601+0000 7f6d32cf7700 1 -- 172.21.15.107:0/3155819592 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d2c101d60 msgr2=0x7f6d2c1015c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:42:28.135 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:27.601+0000 7f6d32cf7700 1 --2- 172.21.15.107:0/3155819592 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d2c101d60 0x7f6d2c1015c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6d2400b440 tx=0x7f6d24009250 comp rx=0 tx=0).stop 2024-05-17T21:42:28.135 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:27.601+0000 7f6d32cf7700 1 -- 172.21.15.107:0/3155819592 shutdown_connections 2024-05-17T21:42:28.135 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:27.601+0000 7f6d32cf7700 1 --2- 172.21.15.107:0/3155819592 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d2c101d60 0x7f6d2c1015c0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:42:28.135 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:27.601+0000 7f6d32cf7700 1 -- 172.21.15.107:0/3155819592 >> 172.21.15.107:0/3155819592 conn(0x7f6d2c0fc6f0 msgr2=0x7f6d2c0fd1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:42:28.135 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:27.601+0000 7f6d32cf7700 1 -- 172.21.15.107:0/3155819592 shutdown_connections 2024-05-17T21:42:28.135 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:27.601+0000 7f6d32cf7700 1 -- 172.21.15.107:0/3155819592 wait complete. 2024-05-17T21:42:28.136 INFO:teuthology.orchestra.run.smithi107.stdout:Restarting the monitor... 2024-05-17T21:42:28.377 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:28 smithi107 systemd[1]: Stopping Ceph mon.smithi107 for 3722731a-1496-11ef-bc9a-c7b262605968... 2024-05-17T21:42:28.631 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:28 smithi107 ceph-3722731a-1496-11ef-bc9a-c7b262605968-mon-smithi107[28041]: 2024-05-17T21:42:28.381+0000 7ff9ea8f6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi107 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-17T21:42:28.631 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:28 smithi107 ceph-3722731a-1496-11ef-bc9a-c7b262605968-mon-smithi107[28041]: 2024-05-17T21:42:28.381+0000 7ff9ea8f6700 -1 mon.smithi107@0(leader) e1 *** Got Signal Terminated *** 2024-05-17T21:42:28.884 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:28 smithi107 podman[28299]: 2024-05-17 21:42:28.637420451 +0000 UTC m=+0.337658086 container died d84975310e6645171c6a04e267e9db323e2db4ed541a685748a38bdce9f6e88a (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-mon-smithi107, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, GIT_CLEAN=True, RELEASE=quincy-9903f77, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-05-17T21:42:29.260 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:28 smithi107 podman[28299]: 2024-05-17 21:42:28.989761967 +0000 UTC m=+0.689999601 container cleanup d84975310e6645171c6a04e267e9db323e2db4ed541a685748a38bdce9f6e88a (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-mon-smithi107, GIT_CLEAN=True, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=quincy-9903f77, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-17T21:42:29.260 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:28 smithi107 bash[28299]: ceph-3722731a-1496-11ef-bc9a-c7b262605968-mon-smithi107 2024-05-17T21:42:29.585 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 podman[28312]: 2024-05-17 21:42:29.266042968 +0000 UTC m=+0.628186187 container remove d84975310e6645171c6a04e267e9db323e2db4ed541a685748a38bdce9f6e88a (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-mon-smithi107, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, RELEASE=quincy-9903f77, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-17T21:42:29.586 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 systemd[1]: ceph-3722731a-1496-11ef-bc9a-c7b262605968@mon.smithi107.service: Deactivated successfully. 2024-05-17T21:42:29.586 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 systemd[1]: Stopped Ceph mon.smithi107 for 3722731a-1496-11ef-bc9a-c7b262605968. 2024-05-17T21:42:29.586 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 systemd[1]: Starting Ceph mon.smithi107 for 3722731a-1496-11ef-bc9a-c7b262605968... 2024-05-17T21:42:29.855 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 podman[28415]: 2024-05-17 21:42:29.591878081 +0000 UTC m=+0.106198714 container create 7de9c23eee97e9e9fa855317247a2c2bcabb0b393bf44e87c3ae8c2e6ef63941 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-mon-smithi107, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=quincy-9903f77, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-05-17T21:42:29.855 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 podman[28415]: 2024-05-17 21:42:29.502950038 +0000 UTC m=+0.017270677 image pull d15a3685379636f882a2c894ceec9b41568bbe95b633c123d31461e59b4021a5 quay.ceph.io/ceph-ci/ceph:quincy 2024-05-17T21:42:29.855 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 podman[28415]: 2024-05-17 21:42:29.777437834 +0000 UTC m=+0.291758481 container init 7de9c23eee97e9e9fa855317247a2c2bcabb0b393bf44e87c3ae8c2e6ef63941 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-mon-smithi107, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=quincy-9903f77, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-05-17T21:42:29.855 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 podman[28415]: 2024-05-17 21:42:29.780443953 +0000 UTC m=+0.294764592 container start 7de9c23eee97e9e9fa855317247a2c2bcabb0b393bf44e87c3ae8c2e6ef63941 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-mon-smithi107, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, RELEASE=quincy-9903f77, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-17T21:42:29.855 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: set uid:gid to 167:167 (ceph:ceph) 2024-05-17T21:42:29.855 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: ceph version 17.2.7-952-g9903f776 (9903f7767386a9d9efadb7c1effdcd3064d71804) quincy (stable), process ceph-mon, pid 2 2024-05-17T21:42:29.855 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: pidfile_write: ignore empty --pid-file 2024-05-17T21:42:29.855 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: load: jerasure load: lrc 2024-05-17T21:42:29.855 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: RocksDB version: 6.15.5 2024-05-17T21:42:29.855 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-05-17T21:42:29.856 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Compile date May 17 2024 2024-05-17T21:42:29.856 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: DB SUMMARY 2024-05-17T21:42:29.856 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: DB Session ID: YZVHE2XHNLWD2NTQZG34 2024-05-17T21:42:29.856 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: CURRENT file: CURRENT 2024-05-17T21:42:29.856 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: IDENTITY file: IDENTITY 2024-05-17T21:42:29.856 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: MANIFEST file: MANIFEST-000009 size: 131 Bytes 2024-05-17T21:42:29.856 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi107/store.db dir, Total Num: 1, files: 000008.sst 2024-05-17T21:42:29.856 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi107/store.db: 000010.log size: 98490 ; 2024-05-17T21:42:29.856 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.error_if_exists: 0 2024-05-17T21:42:29.856 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.create_if_missing: 0 2024-05-17T21:42:29.856 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.paranoid_checks: 1 2024-05-17T21:42:29.857 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-17T21:42:29.857 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.env: 0x555b19ffbb00 2024-05-17T21:42:29.857 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.fs: Posix File System 2024-05-17T21:42:29.858 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.info_log: 0x555b1aa2f620 2024-05-17T21:42:29.858 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.max_file_opening_threads: 16 2024-05-17T21:42:29.858 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.statistics: (nil) 2024-05-17T21:42:29.858 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.use_fsync: 0 2024-05-17T21:42:29.858 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.max_log_file_size: 0 2024-05-17T21:42:29.858 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-17T21:42:29.858 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-17T21:42:29.858 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.keep_log_file_num: 1000 2024-05-17T21:42:29.858 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.recycle_log_file_num: 0 2024-05-17T21:42:29.858 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.allow_fallocate: 1 2024-05-17T21:42:29.858 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.allow_mmap_reads: 0 2024-05-17T21:42:29.858 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.allow_mmap_writes: 0 2024-05-17T21:42:29.859 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.use_direct_reads: 0 2024-05-17T21:42:29.859 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-17T21:42:29.859 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.create_missing_column_families: 0 2024-05-17T21:42:29.859 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.db_log_dir: 2024-05-17T21:42:29.859 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-smithi107/store.db 2024-05-17T21:42:29.859 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-17T21:42:29.859 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-17T21:42:29.859 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-17T21:42:29.859 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-17T21:42:29.859 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-17T21:42:29.859 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-17T21:42:29.859 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.advise_random_on_open: 1 2024-05-17T21:42:29.859 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.db_write_buffer_size: 0 2024-05-17T21:42:29.859 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.write_buffer_manager: 0x555b1b7229c0 2024-05-17T21:42:29.860 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-17T21:42:29.860 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-05-17T21:42:29.860 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-17T21:42:29.860 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-17T21:42:29.860 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.rate_limiter: (nil) 2024-05-17T21:42:29.860 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-17T21:42:29.860 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.wal_recovery_mode: 2 2024-05-17T21:42:29.860 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.enable_thread_tracking: 0 2024-05-17T21:42:29.860 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.enable_pipelined_write: 0 2024-05-17T21:42:29.860 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.unordered_write: 0 2024-05-17T21:42:29.860 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-17T21:42:29.860 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-17T21:42:29.860 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-17T21:42:29.861 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-17T21:42:29.861 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.row_cache: None 2024-05-17T21:42:29.861 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.wal_filter: None 2024-05-17T21:42:29.861 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-17T21:42:29.861 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.allow_ingest_behind: 0 2024-05-17T21:42:29.861 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.preserve_deletes: 0 2024-05-17T21:42:29.861 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.two_write_queues: 0 2024-05-17T21:42:29.861 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.manual_wal_flush: 0 2024-05-17T21:42:29.861 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.atomic_flush: 0 2024-05-17T21:42:29.861 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-17T21:42:29.861 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-17T21:42:29.861 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-17T21:42:29.861 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.log_readahead_size: 0 2024-05-17T21:42:29.861 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-17T21:42:29.862 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.best_efforts_recovery: 0 2024-05-17T21:42:29.862 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-17T21:42:29.862 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-17T21:42:29.862 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.allow_data_in_errors: 0 2024-05-17T21:42:29.862 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.db_host_id: __hostname__ 2024-05-17T21:42:29.862 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.max_background_jobs: 2 2024-05-17T21:42:29.862 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.max_background_compactions: -1 2024-05-17T21:42:29.862 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.max_subcompactions: 1 2024-05-17T21:42:29.862 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-17T21:42:29.862 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-17T21:42:29.862 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-17T21:42:29.862 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.max_total_wal_size: 0 2024-05-17T21:42:29.862 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-17T21:42:29.863 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-17T21:42:29.863 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-17T21:42:29.863 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-17T21:42:29.863 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.max_open_files: -1 2024-05-17T21:42:29.863 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.bytes_per_sync: 0 2024-05-17T21:42:29.863 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-17T21:42:29.863 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-17T21:42:29.863 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.compaction_readahead_size: 0 2024-05-17T21:42:29.863 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.max_background_flushes: -1 2024-05-17T21:42:29.863 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Compression algorithms supported: 2024-05-17T21:42:29.863 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-17T21:42:29.863 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: kZSTD supported: 0 2024-05-17T21:42:29.863 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: kXpressCompression supported: 0 2024-05-17T21:42:29.863 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: kLZ4HCCompression supported: 1 2024-05-17T21:42:29.864 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: kZlibCompression supported: 1 2024-05-17T21:42:29.864 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: kSnappyCompression supported: 1 2024-05-17T21:42:29.864 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: kLZ4Compression supported: 1 2024-05-17T21:42:29.864 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: kBZip2Compression supported: 0 2024-05-17T21:42:29.864 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-17T21:42:29.864 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: [db/version_set.cc:4724] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi107/store.db/MANIFEST-000009 2024-05-17T21:42:29.864 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: [db/column_family.cc:595] --------------- Options for column family [default]: 2024-05-17T21:42:29.864 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-17T21:42:29.864 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.merge_operator: 2024-05-17T21:42:29.864 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.compaction_filter: None 2024-05-17T21:42:29.864 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.compaction_filter_factory: None 2024-05-17T21:42:29.864 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.sst_partitioner_factory: None 2024-05-17T21:42:29.864 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-17T21:42:29.865 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-17T21:42:29.865 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x555b1aa02f58) 2024-05-17T21:42:29.865 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: cache_index_and_filter_blocks: 1 2024-05-17T21:42:29.865 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-17T21:42:29.865 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-17T21:42:29.865 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: pin_top_level_index_and_filter: 1 2024-05-17T21:42:29.865 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: index_type: 0 2024-05-17T21:42:29.865 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: data_block_index_type: 0 2024-05-17T21:42:29.865 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: index_shortening: 1 2024-05-17T21:42:29.865 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-17T21:42:29.865 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: hash_index_allow_collision: 1 2024-05-17T21:42:29.865 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: checksum: 1 2024-05-17T21:42:29.865 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: no_block_cache: 0 2024-05-17T21:42:29.865 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: block_cache: 0x555b1aa35090 2024-05-17T21:42:29.866 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: block_cache_name: BinnedLRUCache 2024-05-17T21:42:29.866 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: block_cache_options: 2024-05-17T21:42:29.866 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: capacity : 536870912 2024-05-17T21:42:29.866 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: num_shard_bits : 4 2024-05-17T21:42:29.866 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: strict_capacity_limit : 0 2024-05-17T21:42:29.866 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: high_pri_pool_ratio: 0.000 2024-05-17T21:42:29.866 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: block_cache_compressed: (nil) 2024-05-17T21:42:29.866 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: persistent_cache: (nil) 2024-05-17T21:42:29.866 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: block_size: 4096 2024-05-17T21:42:29.866 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: block_size_deviation: 10 2024-05-17T21:42:29.866 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: block_restart_interval: 16 2024-05-17T21:42:29.866 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: index_block_restart_interval: 1 2024-05-17T21:42:29.866 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: metadata_block_size: 4096 2024-05-17T21:42:29.867 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: partition_filters: 0 2024-05-17T21:42:29.867 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: use_delta_encoding: 1 2024-05-17T21:42:29.867 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: filter_policy: rocksdb.BuiltinBloomFilter 2024-05-17T21:42:29.867 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: whole_key_filtering: 1 2024-05-17T21:42:29.867 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: verify_compression: 0 2024-05-17T21:42:29.867 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: read_amp_bytes_per_bit: 0 2024-05-17T21:42:29.867 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: format_version: 4 2024-05-17T21:42:29.867 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: enable_index_compression: 1 2024-05-17T21:42:29.867 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: block_align: 0 2024-05-17T21:42:29.867 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.write_buffer_size: 33554432 2024-05-17T21:42:29.867 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.max_write_buffer_number: 2 2024-05-17T21:42:29.867 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.compression: NoCompression 2024-05-17T21:42:29.867 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.bottommost_compression: Disabled 2024-05-17T21:42:29.867 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.prefix_extractor: nullptr 2024-05-17T21:42:29.868 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-17T21:42:29.868 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.num_levels: 7 2024-05-17T21:42:29.868 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-17T21:42:29.868 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-17T21:42:29.868 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-17T21:42:29.868 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-17T21:42:29.868 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-17T21:42:29.868 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-17T21:42:29.868 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-17T21:42:29.868 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-17T21:42:29.868 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-17T21:42:29.868 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-17T21:42:29.868 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-17T21:42:29.868 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.compression_opts.level: 32767 2024-05-17T21:42:29.869 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.compression_opts.strategy: 0 2024-05-17T21:42:29.869 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-17T21:42:29.869 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-17T21:42:29.869 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-17T21:42:29.869 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.compression_opts.enabled: false 2024-05-17T21:42:29.869 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-17T21:42:29.869 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-17T21:42:29.869 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-17T21:42:29.869 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.target_file_size_base: 67108864 2024-05-17T21:42:29.869 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-17T21:42:29.869 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-17T21:42:29.869 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-17T21:42:29.869 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-17T21:42:29.869 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-17T21:42:29.870 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-17T21:42:29.870 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-17T21:42:29.870 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-17T21:42:29.870 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-17T21:42:29.870 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-17T21:42:29.870 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-17T21:42:29.870 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-17T21:42:29.870 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-17T21:42:29.870 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.arena_block_size: 4194304 2024-05-17T21:42:29.870 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-17T21:42:29.870 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-17T21:42:29.870 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-05-17T21:42:29.870 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.disable_auto_compactions: 0 2024-05-17T21:42:29.871 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-17T21:42:29.871 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-17T21:42:29.871 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-17T21:42:29.871 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-17T21:42:29.871 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-17T21:42:29.871 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-17T21:42:29.871 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-17T21:42:29.871 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-17T21:42:29.871 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-17T21:42:29.871 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-17T21:42:29.871 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.table_properties_collectors: 2024-05-17T21:42:29.871 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.inplace_update_support: 0 2024-05-17T21:42:29.871 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-17T21:42:29.871 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-17T21:42:29.872 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-17T21:42:29.872 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-17T21:42:29.872 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.bloom_locality: 0 2024-05-17T21:42:29.872 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.max_successive_merges: 0 2024-05-17T21:42:29.872 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-17T21:42:29.872 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.paranoid_file_checks: 0 2024-05-17T21:42:29.872 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.force_consistency_checks: 1 2024-05-17T21:42:29.872 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.report_bg_io_stats: 0 2024-05-17T21:42:29.872 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.ttl: 2592000 2024-05-17T21:42:29.872 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-17T21:42:29.872 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.enable_blob_files: false 2024-05-17T21:42:29.872 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.min_blob_size: 0 2024-05-17T21:42:29.872 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.blob_file_size: 268435456 2024-05-17T21:42:29.873 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-17T21:42:29.873 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-17T21:42:29.873 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-17T21:42:29.873 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: [db/version_set.cc:4764] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi107/store.db/MANIFEST-000009 succeeded,manifest_file_number is 9, next_file_number is 11, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-05-17T21:42:29.873 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: [db/version_set.cc:4779] Column family [default] (ID 0), log number is 5 2024-05-17T21:42:29.874 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: [db/version_set.cc:4082] Creating manifest 13 2024-05-17T21:42:29.874 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715982149807861, "job": 1, "event": "recovery_started", "wal_files": [10]} 2024-05-17T21:42:29.874 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: [db/db_impl/db_impl_open.cc:845] Recovering log #10 mode 2 2024-05-17T21:42:29.874 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: [table/block_based/filter_policy.cc:991] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-05-17T21:42:29.874 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715982149809047, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 14, "file_size": 93541, "file_checksum": "", "file_checksum_func_name": "Unknown", "table_properties": {"data_size": 91653, "index_size": 248, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13606, "raw_average_key_size": 49, "raw_value_size": 84191, "raw_average_value_size": 306, "num_data_blocks": 11, "num_entries": 275, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1715982149, "oldest_key_time": 0, "file_creation_time": 0, "db_id": "550fdaca-bfae-486e-8611-6ce40e96b2ee", "db_session_id": "YZVHE2XHNLWD2NTQZG34"}} 2024-05-17T21:42:29.874 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: [db/version_set.cc:4082] Creating manifest 15 2024-05-17T21:42:29.874 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715982149809469, "job": 1, "event": "recovery_finished"} 2024-05-17T21:42:29.874 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: [file/delete_scheduler.cc:69] Deleted file /var/lib/ceph/mon/ceph-smithi107/store.db/000010.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-17T21:42:29.874 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: [db/db_impl/db_impl_open.cc:1700] SstFileManager instance 0x555b1a9c9180 2024-05-17T21:42:29.874 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: DB pointer 0x555b1aa9e000 2024-05-17T21:42:29.874 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: [db/db_impl/db_impl.cc:901] ------- DUMPING STATS ------- 2024-05-17T21:42:29.874 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: rocksdb: [db/db_impl/db_impl.cc:903] 2024-05-17T21:42:29.874 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: ** DB Stats ** 2024-05-17T21:42:29.875 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-17T21:42:29.875 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-17T21:42:29.875 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-17T21:42:29.875 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-17T21:42:29.875 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-17T21:42:29.875 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-05-17T21:42:29.875 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-17T21:42:29.875 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: 2024-05-17T21:42:29.875 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: ** Compaction Stats [default] ** 2024-05-17T21:42:29.875 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-05-17T21:42:29.875 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-17T21:42:29.875 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: L0 2/0 93.15 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 115.7 0.00 0.00 1 0.001 0 0 2024-05-17T21:42:29.875 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Sum 2/0 93.15 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 115.7 0.00 0.00 1 0.001 0 0 2024-05-17T21:42:29.875 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 115.7 0.00 0.00 1 0.001 0 0 2024-05-17T21:42:29.876 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: 2024-05-17T21:42:29.876 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: ** Compaction Stats [default] ** 2024-05-17T21:42:29.876 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-05-17T21:42:29.876 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-17T21:42:29.876 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 115.7 0.00 0.00 1 0.001 0 0 2024-05-17T21:42:29.876 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-17T21:42:29.876 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-17T21:42:29.876 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-17T21:42:29.876 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-17T21:42:29.876 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-17T21:42:29.876 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-17T21:42:29.876 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Cumulative compaction: 0.00 GB write, 20.80 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-17T21:42:29.876 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Interval compaction: 0.00 GB write, 20.80 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-17T21:42:29.876 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-17T21:42:29.877 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: 2024-05-17T21:42:29.877 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-17T21:42:29.877 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: 2024-05-17T21:42:29.877 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: ** Compaction Stats [default] ** 2024-05-17T21:42:29.877 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-05-17T21:42:29.877 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-17T21:42:29.877 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: L0 2/0 93.15 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 115.7 0.00 0.00 1 0.001 0 0 2024-05-17T21:42:29.877 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Sum 2/0 93.15 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 115.7 0.00 0.00 1 0.001 0 0 2024-05-17T21:42:29.877 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-05-17T21:42:29.877 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: 2024-05-17T21:42:29.877 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: ** Compaction Stats [default] ** 2024-05-17T21:42:29.877 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-05-17T21:42:29.877 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-17T21:42:29.878 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 115.7 0.00 0.00 1 0.001 0 0 2024-05-17T21:42:29.878 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-17T21:42:29.878 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-17T21:42:29.878 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-17T21:42:29.878 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-17T21:42:29.878 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-17T21:42:29.878 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-17T21:42:29.878 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Cumulative compaction: 0.00 GB write, 20.64 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-17T21:42:29.878 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-17T21:42:29.878 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-17T21:42:29.878 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: 2024-05-17T21:42:29.878 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-17T21:42:29.878 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: starting mon.smithi107 rank 0 at public addrs [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] at bind addrs [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi107 fsid 3722731a-1496-11ef-bc9a-c7b262605968 2024-05-17T21:42:29.879 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: mon.smithi107@-1(???) e1 preinit fsid 3722731a-1496-11ef-bc9a-c7b262605968 2024-05-17T21:42:29.879 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: mon.smithi107@-1(???).mds e1 new map 2024-05-17T21:42:29.879 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: mon.smithi107@-1(???).mds e1 print_map 2024-05-17T21:42:29.879 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: e1 2024-05-17T21:42:29.879 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-17T21:42:29.879 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-17T21:42:29.879 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: legacy client fscid: -1 2024-05-17T21:42:29.879 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: 2024-05-17T21:42:29.879 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: No filesystems configured 2024-05-17T21:42:29.879 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: mon.smithi107@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-17T21:42:29.879 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: mon.smithi107@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-17T21:42:29.879 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: mon.smithi107@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-17T21:42:29.879 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: mon.smithi107@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-17T21:42:29.879 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: mon.smithi107@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-05-17T21:42:29.880 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 bash[28415]: 7de9c23eee97e9e9fa855317247a2c2bcabb0b393bf44e87c3ae8c2e6ef63941 2024-05-17T21:42:29.880 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 systemd[1]: Started Ceph mon.smithi107 for 3722731a-1496-11ef-bc9a-c7b262605968. 2024-05-17T21:42:29.880 INFO:teuthology.orchestra.run.smithi107.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-05-17T21:42:30.191 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: mon.smithi107 is new leader, mons smithi107 in quorum (ranks 0) 2024-05-17T21:42:30.191 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: monmap e1: 1 mons at {smithi107=[v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-17T21:42:30.191 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: fsmap 2024-05-17T21:42:30.191 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: osdmap e1: 0 total, 0 up, 0 in 2024-05-17T21:42:30.191 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:29 smithi107 ceph-mon[28434]: mgrmap e1: no daemons active 2024-05-17T21:42:30.938 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:30.298+0000 7f1a3fec8700 1 Processor -- start 2024-05-17T21:42:30.943 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:30.299+0000 7f1a3fec8700 1 -- start start 2024-05-17T21:42:30.943 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:30.299+0000 7f1a3fec8700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a38076b70 0x7f1a381054e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:42:30.943 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:30.299+0000 7f1a3fec8700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a38105a20 con 0x7f1a38076b70 2024-05-17T21:42:30.943 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:30.299+0000 7f1a3dc64700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a38076b70 0x7f1a381054e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:42:30.943 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:30.299+0000 7f1a3dc64700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a38076b70 0x7f1a381054e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42532/0 (socket says 172.21.15.107:42532) 2024-05-17T21:42:30.944 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:30.299+0000 7f1a3dc64700 1 -- 172.21.15.107:0/2654452067 learned_addr learned my addr 172.21.15.107:0/2654452067 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:42:30.944 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:30.300+0000 7f1a3dc64700 1 -- 172.21.15.107:0/2654452067 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a38105b60 con 0x7f1a38076b70 2024-05-17T21:42:30.944 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:30.300+0000 7f1a3dc64700 1 --2- 172.21.15.107:0/2654452067 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a38076b70 0x7f1a381054e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c0098e0 tx=0x7f1a2c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=519d17087a335c45 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:42:30.944 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:30.300+0000 7f1a3cc62700 1 -- 172.21.15.107:0/2654452067 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1a2c00b860 con 0x7f1a38076b70 2024-05-17T21:42:30.944 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:30.300+0000 7f1a3cc62700 1 -- 172.21.15.107:0/2654452067 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f1a2c00be50 con 0x7f1a38076b70 2024-05-17T21:42:30.944 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:30.300+0000 7f1a3cc62700 1 -- 172.21.15.107:0/2654452067 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1a2c010380 con 0x7f1a38076b70 2024-05-17T21:42:30.944 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:30.300+0000 7f1a3fec8700 1 -- 172.21.15.107:0/2654452067 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a38076b70 msgr2=0x7f1a381054e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:42:30.944 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:30.300+0000 7f1a3fec8700 1 --2- 172.21.15.107:0/2654452067 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a38076b70 0x7f1a381054e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c0098e0 tx=0x7f1a2c009250 comp rx=0 tx=0).stop 2024-05-17T21:42:30.944 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:30.300+0000 7f1a3fec8700 1 -- 172.21.15.107:0/2654452067 shutdown_connections 2024-05-17T21:42:30.944 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:30.300+0000 7f1a3fec8700 1 --2- 172.21.15.107:0/2654452067 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a38076b70 0x7f1a381054e0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:42:30.944 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:30.300+0000 7f1a3fec8700 1 -- 172.21.15.107:0/2654452067 >> 172.21.15.107:0/2654452067 conn(0x7f1a380fcef0 msgr2=0x7f1a380ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:42:30.944 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:30.301+0000 7f1a3fec8700 1 -- 172.21.15.107:0/2654452067 shutdown_connections 2024-05-17T21:42:30.944 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:30.301+0000 7f1a3fec8700 1 -- 172.21.15.107:0/2654452067 wait complete. 2024-05-17T21:42:30.945 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:30.301+0000 7f1a3fec8700 1 Processor -- start 2024-05-17T21:42:30.945 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:30.301+0000 7f1a3fec8700 1 -- start start 2024-05-17T21:42:30.945 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:30.301+0000 7f1a3fec8700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a38076b70 0x7f1a38102e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:42:30.945 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:30.301+0000 7f1a3fec8700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a38103370 con 0x7f1a38076b70 2024-05-17T21:42:30.945 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:30.301+0000 7f1a3dc64700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a38076b70 0x7f1a38102e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:42:30.945 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:30.301+0000 7f1a3dc64700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a38076b70 0x7f1a38102e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42548/0 (socket says 172.21.15.107:42548) 2024-05-17T21:42:30.945 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:30.301+0000 7f1a3dc64700 1 -- 172.21.15.107:0/2153340040 learned_addr learned my addr 172.21.15.107:0/2153340040 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:42:30.945 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:30.302+0000 7f1a3dc64700 1 -- 172.21.15.107:0/2153340040 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a2c0094f0 con 0x7f1a38076b70 2024-05-17T21:42:30.945 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:30.302+0000 7f1a3dc64700 1 --2- 172.21.15.107:0/2153340040 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a38076b70 0x7f1a38102e30 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c003050 tx=0x7f1a2c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:42:30.945 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:30.302+0000 7f1a2a7fc700 1 -- 172.21.15.107:0/2153340040 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1a2c010760 con 0x7f1a38076b70 2024-05-17T21:42:30.945 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:30.302+0000 7f1a2a7fc700 1 -- 172.21.15.107:0/2153340040 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f1a2c010d50 con 0x7f1a38076b70 2024-05-17T21:42:30.945 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:30.302+0000 7f1a3fec8700 1 -- 172.21.15.107:0/2153340040 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a381013e0 con 0x7f1a38076b70 2024-05-17T21:42:30.945 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:30.302+0000 7f1a2a7fc700 1 -- 172.21.15.107:0/2153340040 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1a2c01ae20 con 0x7f1a38076b70 2024-05-17T21:42:30.946 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:30.302+0000 7f1a3fec8700 1 -- 172.21.15.107:0/2153340040 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a381018d0 con 0x7f1a38076b70 2024-05-17T21:42:30.946 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:30.302+0000 7f1a2a7fc700 1 -- 172.21.15.107:0/2153340040 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f1a2c02e440 con 0x7f1a38076b70 2024-05-17T21:42:30.946 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:30.302+0000 7f1a2a7fc700 1 -- 172.21.15.107:0/2153340040 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f1a2c024740 con 0x7f1a38076b70 2024-05-17T21:42:30.946 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:30.303+0000 7f1a3fec8700 1 -- 172.21.15.107:0/2153340040 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a3804bed0 con 0x7f1a38076b70 2024-05-17T21:42:30.946 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:30.305+0000 7f1a2a7fc700 1 -- 172.21.15.107:0/2153340040 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71547 (secure 0 0 0) 0x7f1a2c029070 con 0x7f1a38076b70 2024-05-17T21:42:30.946 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:30.370+0000 7f1a3fec8700 1 -- 172.21.15.107:0/2153340040 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f1a38101b10 con 0x7f1a38076b70 2024-05-17T21:42:30.946 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:30.375+0000 7f1a2a7fc700 1 -- 172.21.15.107:0/2153340040 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f1a2c010ec0 con 0x7f1a38076b70 2024-05-17T21:42:30.946 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:30.376+0000 7f1a3fec8700 1 -- 172.21.15.107:0/2153340040 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a38076b70 msgr2=0x7f1a38102e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:42:30.946 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:30.376+0000 7f1a3fec8700 1 --2- 172.21.15.107:0/2153340040 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a38076b70 0x7f1a38102e30 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c003050 tx=0x7f1a2c009250 comp rx=0 tx=0).stop 2024-05-17T21:42:30.946 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:30.376+0000 7f1a3fec8700 1 -- 172.21.15.107:0/2153340040 shutdown_connections 2024-05-17T21:42:30.946 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:30.376+0000 7f1a3fec8700 1 --2- 172.21.15.107:0/2153340040 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a38076b70 0x7f1a38102e30 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:42:30.946 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:30.376+0000 7f1a3fec8700 1 -- 172.21.15.107:0/2153340040 >> 172.21.15.107:0/2153340040 conn(0x7f1a380fcef0 msgr2=0x7f1a380fd9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:42:30.947 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:30.376+0000 7f1a3fec8700 1 -- 172.21.15.107:0/2153340040 shutdown_connections 2024-05-17T21:42:30.947 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:30.376+0000 7f1a3fec8700 1 -- 172.21.15.107:0/2153340040 wait complete. 2024-05-17T21:42:30.947 INFO:teuthology.orchestra.run.smithi107.stdout:Wrote config to /etc/ceph/ceph.conf 2024-05-17T21:42:30.947 INFO:teuthology.orchestra.run.smithi107.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-05-17T21:42:30.947 INFO:teuthology.orchestra.run.smithi107.stdout:Creating mgr... 2024-05-17T21:42:30.947 INFO:teuthology.orchestra.run.smithi107.stdout:Verifying port 9283 ... 2024-05-17T21:42:31.115 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-3722731a-1496-11ef-bc9a-c7b262605968@mgr.smithi107.xhhyiw 2024-05-17T21:42:31.116 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Failed to reset failed state of unit ceph-3722731a-1496-11ef-bc9a-c7b262605968@mgr.smithi107.xhhyiw.service: Unit ceph-3722731a-1496-11ef-bc9a-c7b262605968@mgr.smithi107.xhhyiw.service not loaded. 2024-05-17T21:42:31.262 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-3722731a-1496-11ef-bc9a-c7b262605968.target.wants/ceph-3722731a-1496-11ef-bc9a-c7b262605968@mgr.smithi107.xhhyiw.service → /etc/systemd/system/ceph-3722731a-1496-11ef-bc9a-c7b262605968@.service. 2024-05-17T21:42:31.501 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:31 smithi107 ceph-mon[28434]: from='client.? 172.21.15.107:0/2153340040' entity='client.admin' 2024-05-17T21:42:32.778 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-17T21:42:32.778 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout disabled 2024-05-17T21:42:32.786 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-17T21:42:32.786 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout inactive 2024-05-17T21:42:32.786 INFO:teuthology.orchestra.run.smithi107.stdout:firewalld.service is not enabled 2024-05-17T21:42:32.786 INFO:teuthology.orchestra.run.smithi107.stdout:Not possible to enable service . firewalld.service is not available 2024-05-17T21:42:32.795 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-17T21:42:32.795 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout disabled 2024-05-17T21:42:32.804 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-17T21:42:32.804 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout inactive 2024-05-17T21:42:32.804 INFO:teuthology.orchestra.run.smithi107.stdout:firewalld.service is not enabled 2024-05-17T21:42:32.804 INFO:teuthology.orchestra.run.smithi107.stdout:Not possible to open ports <[9283]>. firewalld.service is not available 2024-05-17T21:42:32.804 INFO:teuthology.orchestra.run.smithi107.stdout:Waiting for mgr to start... 2024-05-17T21:42:32.804 INFO:teuthology.orchestra.run.smithi107.stdout:Waiting for mgr... 2024-05-17T21:42:33.445 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:33 smithi107 ceph-mon[28434]: from='client.? 172.21.15.107:0/1363797319' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-17T21:42:33.886 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-05-17T21:42:33.886 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout { 2024-05-17T21:42:33.886 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "fsid": "3722731a-1496-11ef-bc9a-c7b262605968", 2024-05-17T21:42:33.886 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "health": { 2024-05-17T21:42:33.886 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-17T21:42:33.886 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-17T21:42:33.886 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-17T21:42:33.886 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-05-17T21:42:33.887 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-17T21:42:33.887 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-17T21:42:33.887 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 0 2024-05-17T21:42:33.887 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ], 2024-05-17T21:42:33.887 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-17T21:42:33.887 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "smithi107" 2024-05-17T21:42:33.887 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ], 2024-05-17T21:42:33.887 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-05-17T21:42:33.887 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-17T21:42:33.887 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-17T21:42:33.887 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-05-17T21:42:33.887 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-17T21:42:33.887 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-05-17T21:42:33.888 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-17T21:42:33.888 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-17T21:42:33.888 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-17T21:42:33.888 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-17T21:42:33.888 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-17T21:42:33.888 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-17T21:42:33.888 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-17T21:42:33.888 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-17T21:42:33.888 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-05-17T21:42:33.888 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-17T21:42:33.888 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-17T21:42:33.888 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-17T21:42:33.888 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-17T21:42:33.888 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-17T21:42:33.888 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-17T21:42:33.889 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-17T21:42:33.889 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-17T21:42:33.889 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-17T21:42:33.889 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-05-17T21:42:33.889 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-17T21:42:33.889 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-17T21:42:33.889 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-17T21:42:33.889 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-17T21:42:33.889 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-05-17T21:42:33.889 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-17T21:42:33.889 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-17T21:42:33.895 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-17T21:42:33.895 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-17T21:42:33.895 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-17T21:42:33.895 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-17T21:42:33.895 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "restful" 2024-05-17T21:42:33.895 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ], 2024-05-17T21:42:33.895 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-17T21:42:33.896 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-05-17T21:42:33.896 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-17T21:42:33.896 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-17T21:42:33.896 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "modified": "2024-05-17T21:42:15.126617+0000", 2024-05-17T21:42:33.896 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-17T21:42:33.896 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-05-17T21:42:33.896 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-17T21:42:33.896 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout } 2024-05-17T21:42:33.896 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:33.212+0000 7f2697cd5700 1 Processor -- start 2024-05-17T21:42:33.896 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:33.212+0000 7f2697cd5700 1 -- start start 2024-05-17T21:42:33.896 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:33.213+0000 7f2697cd5700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2690101710 0x7f2690101b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:42:33.896 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:33.213+0000 7f2697cd5700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2690102060 con 0x7f2690101710 2024-05-17T21:42:33.896 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:33.213+0000 7f2695a71700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2690101710 0x7f2690101b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:42:33.896 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:33.213+0000 7f2695a71700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2690101710 0x7f2690101b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42584/0 (socket says 172.21.15.107:42584) 2024-05-17T21:42:33.896 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:33.213+0000 7f2695a71700 1 -- 172.21.15.107:0/366516659 learned_addr learned my addr 172.21.15.107:0/366516659 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:42:33.897 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:33.213+0000 7f2695a71700 1 -- 172.21.15.107:0/366516659 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26901021a0 con 0x7f2690101710 2024-05-17T21:42:33.897 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:33.213+0000 7f2695a71700 1 --2- 172.21.15.107:0/366516659 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2690101710 0x7f2690101b20 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f268c0098e0 tx=0x7f268c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3646a51dd71c8f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:42:33.897 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:33.213+0000 7f2694a6f700 1 -- 172.21.15.107:0/366516659 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f268c00b860 con 0x7f2690101710 2024-05-17T21:42:33.897 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:33.214+0000 7f2694a6f700 1 -- 172.21.15.107:0/366516659 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f268c00be50 con 0x7f2690101710 2024-05-17T21:42:33.897 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:33.214+0000 7f2694a6f700 1 -- 172.21.15.107:0/366516659 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f268c010380 con 0x7f2690101710 2024-05-17T21:42:33.897 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:33.214+0000 7f2697cd5700 1 -- 172.21.15.107:0/366516659 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2690101710 msgr2=0x7f2690101b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:42:33.897 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:33.214+0000 7f2697cd5700 1 --2- 172.21.15.107:0/366516659 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2690101710 0x7f2690101b20 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f268c0098e0 tx=0x7f268c009250 comp rx=0 tx=0).stop 2024-05-17T21:42:33.897 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:33.215+0000 7f2697cd5700 1 -- 172.21.15.107:0/366516659 shutdown_connections 2024-05-17T21:42:33.897 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:33.215+0000 7f2697cd5700 1 --2- 172.21.15.107:0/366516659 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2690101710 0x7f2690101b20 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:42:33.897 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:33.215+0000 7f2697cd5700 1 -- 172.21.15.107:0/366516659 >> 172.21.15.107:0/366516659 conn(0x7f26900fced0 msgr2=0x7f26900ff2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:42:33.897 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:33.215+0000 7f2697cd5700 1 -- 172.21.15.107:0/366516659 shutdown_connections 2024-05-17T21:42:33.897 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:33.215+0000 7f2697cd5700 1 -- 172.21.15.107:0/366516659 wait complete. 2024-05-17T21:42:33.897 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:33.215+0000 7f2697cd5700 1 Processor -- start 2024-05-17T21:42:33.897 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:33.215+0000 7f2697cd5700 1 -- start start 2024-05-17T21:42:33.898 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:33.215+0000 7f2697cd5700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2690101710 0x7f26901079b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:42:33.898 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:33.215+0000 7f2697cd5700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2690107ef0 con 0x7f2690101710 2024-05-17T21:42:33.898 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:33.215+0000 7f2695a71700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2690101710 0x7f26901079b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:42:33.898 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:33.215+0000 7f2695a71700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2690101710 0x7f26901079b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42598/0 (socket says 172.21.15.107:42598) 2024-05-17T21:42:33.898 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:33.216+0000 7f2695a71700 1 -- 172.21.15.107:0/1363797319 learned_addr learned my addr 172.21.15.107:0/1363797319 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:42:33.898 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:33.216+0000 7f2695a71700 1 -- 172.21.15.107:0/1363797319 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f268c0094f0 con 0x7f2690101710 2024-05-17T21:42:33.898 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:33.216+0000 7f2695a71700 1 --2- 172.21.15.107:0/1363797319 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2690101710 0x7f26901079b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f268c000c00 tx=0x7f268c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:42:33.898 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:33.216+0000 7f26867fc700 1 -- 172.21.15.107:0/1363797319 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f268c0107c0 con 0x7f2690101710 2024-05-17T21:42:33.898 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:33.216+0000 7f26867fc700 1 -- 172.21.15.107:0/1363797319 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f268c010db0 con 0x7f2690101710 2024-05-17T21:42:33.898 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:33.216+0000 7f2697cd5700 1 -- 172.21.15.107:0/1363797319 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2690105f60 con 0x7f2690101710 2024-05-17T21:42:33.898 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:33.216+0000 7f26867fc700 1 -- 172.21.15.107:0/1363797319 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f268c01ad00 con 0x7f2690101710 2024-05-17T21:42:33.898 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:33.216+0000 7f2697cd5700 1 -- 172.21.15.107:0/1363797319 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2690106450 con 0x7f2690101710 2024-05-17T21:42:33.898 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:33.216+0000 7f26867fc700 1 -- 172.21.15.107:0/1363797319 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f268c024440 con 0x7f2690101710 2024-05-17T21:42:33.898 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:33.216+0000 7f26867fc700 1 -- 172.21.15.107:0/1363797319 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f268c019760 con 0x7f2690101710 2024-05-17T21:42:33.899 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:33.217+0000 7f2697cd5700 1 -- 172.21.15.107:0/1363797319 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f269004bed0 con 0x7f2690101710 2024-05-17T21:42:33.899 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:33.219+0000 7f26867fc700 1 -- 172.21.15.107:0/1363797319 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71547 (secure 0 0 0) 0x7f268c042790 con 0x7f2690101710 2024-05-17T21:42:33.899 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:33.290+0000 7f2697cd5700 1 -- 172.21.15.107:0/1363797319 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f2690106690 con 0x7f2690101710 2024-05-17T21:42:33.899 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:33.291+0000 7f26867fc700 1 -- 172.21.15.107:0/1363797319 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1248 (secure 0 0 0) 0x7f268c031020 con 0x7f2690101710 2024-05-17T21:42:33.899 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:33.292+0000 7f2697cd5700 1 -- 172.21.15.107:0/1363797319 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2690101710 msgr2=0x7f26901079b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:42:33.899 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:33.292+0000 7f2697cd5700 1 --2- 172.21.15.107:0/1363797319 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2690101710 0x7f26901079b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f268c000c00 tx=0x7f268c009250 comp rx=0 tx=0).stop 2024-05-17T21:42:33.899 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:33.292+0000 7f2697cd5700 1 -- 172.21.15.107:0/1363797319 shutdown_connections 2024-05-17T21:42:33.899 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:33.292+0000 7f2697cd5700 1 --2- 172.21.15.107:0/1363797319 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2690101710 0x7f26901079b0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:42:33.899 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:33.292+0000 7f2697cd5700 1 -- 172.21.15.107:0/1363797319 >> 172.21.15.107:0/1363797319 conn(0x7f26900fced0 msgr2=0x7f26900fd910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:42:33.899 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:33.292+0000 7f2697cd5700 1 -- 172.21.15.107:0/1363797319 shutdown_connections 2024-05-17T21:42:33.899 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:33.292+0000 7f2697cd5700 1 -- 172.21.15.107:0/1363797319 wait complete. 2024-05-17T21:42:33.899 INFO:teuthology.orchestra.run.smithi107.stdout:mgr not available, waiting (1/15)... 2024-05-17T21:42:36.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:36 smithi107 ceph-mon[28434]: from='client.? 172.21.15.107:0/3384008830' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-17T21:42:37.242 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-05-17T21:42:37.242 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout { 2024-05-17T21:42:37.243 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "fsid": "3722731a-1496-11ef-bc9a-c7b262605968", 2024-05-17T21:42:37.243 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "health": { 2024-05-17T21:42:37.243 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-17T21:42:37.243 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-17T21:42:37.243 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-17T21:42:37.243 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-05-17T21:42:37.243 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-17T21:42:37.243 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-17T21:42:37.243 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 0 2024-05-17T21:42:37.243 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ], 2024-05-17T21:42:37.244 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-17T21:42:37.244 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "smithi107" 2024-05-17T21:42:37.244 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ], 2024-05-17T21:42:37.244 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2024-05-17T21:42:37.244 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-17T21:42:37.244 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-17T21:42:37.244 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-05-17T21:42:37.244 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-17T21:42:37.244 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-05-17T21:42:37.244 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-17T21:42:37.244 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-17T21:42:37.244 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-17T21:42:37.244 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-17T21:42:37.245 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-17T21:42:37.245 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-17T21:42:37.245 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-17T21:42:37.245 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-17T21:42:37.245 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-05-17T21:42:37.245 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-17T21:42:37.245 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-17T21:42:37.245 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-17T21:42:37.245 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-17T21:42:37.245 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-17T21:42:37.245 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-17T21:42:37.245 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-17T21:42:37.245 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-17T21:42:37.245 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-17T21:42:37.246 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-05-17T21:42:37.246 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-17T21:42:37.246 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-17T21:42:37.246 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-17T21:42:37.246 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-17T21:42:37.246 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-05-17T21:42:37.246 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-17T21:42:37.246 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-17T21:42:37.246 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-17T21:42:37.246 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-17T21:42:37.246 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-17T21:42:37.246 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-17T21:42:37.247 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "restful" 2024-05-17T21:42:37.247 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ], 2024-05-17T21:42:37.247 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-17T21:42:37.247 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-05-17T21:42:37.247 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-17T21:42:37.247 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-17T21:42:37.247 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "modified": "2024-05-17T21:42:15.126617+0000", 2024-05-17T21:42:37.247 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-17T21:42:37.247 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-05-17T21:42:37.247 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-17T21:42:37.247 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout } 2024-05-17T21:42:37.247 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:36.292+0000 7f38a1bf1700 1 Processor -- start 2024-05-17T21:42:37.247 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:36.292+0000 7f38a1bf1700 1 -- start start 2024-05-17T21:42:37.248 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:36.292+0000 7f38a1bf1700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f389c0712d0 0x7f389c107850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:42:37.248 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:36.292+0000 7f38a1bf1700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f389c103240 con 0x7f389c0712d0 2024-05-17T21:42:37.248 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:36.293+0000 7f389b7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f389c0712d0 0x7f389c107850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:42:37.248 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:36.293+0000 7f389b7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f389c0712d0 0x7f389c107850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34514/0 (socket says 172.21.15.107:34514) 2024-05-17T21:42:37.248 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:36.293+0000 7f389b7fe700 1 -- 172.21.15.107:0/4099582914 learned_addr learned my addr 172.21.15.107:0/4099582914 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:42:37.248 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:36.293+0000 7f389b7fe700 1 -- 172.21.15.107:0/4099582914 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f389c107d90 con 0x7f389c0712d0 2024-05-17T21:42:37.248 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:36.293+0000 7f389b7fe700 1 --2- 172.21.15.107:0/4099582914 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f389c0712d0 0x7f389c107850 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f388c0098e0 tx=0x7f388c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59cd292df161d402 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:42:37.248 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:36.293+0000 7f389a7fc700 1 -- 172.21.15.107:0/4099582914 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f388c00b860 con 0x7f389c0712d0 2024-05-17T21:42:37.248 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:36.293+0000 7f389a7fc700 1 -- 172.21.15.107:0/4099582914 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f388c00be50 con 0x7f389c0712d0 2024-05-17T21:42:37.248 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:36.293+0000 7f389a7fc700 1 -- 172.21.15.107:0/4099582914 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f388c010380 con 0x7f389c0712d0 2024-05-17T21:42:37.248 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:36.294+0000 7f38a1bf1700 1 -- 172.21.15.107:0/4099582914 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f389c0712d0 msgr2=0x7f389c107850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:42:37.248 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:36.294+0000 7f38a1bf1700 1 --2- 172.21.15.107:0/4099582914 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f389c0712d0 0x7f389c107850 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f388c0098e0 tx=0x7f388c009250 comp rx=0 tx=0).stop 2024-05-17T21:42:37.248 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:36.294+0000 7f38a1bf1700 1 -- 172.21.15.107:0/4099582914 shutdown_connections 2024-05-17T21:42:37.249 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:36.294+0000 7f38a1bf1700 1 --2- 172.21.15.107:0/4099582914 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f389c0712d0 0x7f389c107850 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:42:37.249 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:36.294+0000 7f38a1bf1700 1 -- 172.21.15.107:0/4099582914 >> 172.21.15.107:0/4099582914 conn(0x7f389c0fcec0 msgr2=0x7f389c0ff2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:42:37.249 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:36.294+0000 7f38a1bf1700 1 -- 172.21.15.107:0/4099582914 shutdown_connections 2024-05-17T21:42:37.249 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:36.294+0000 7f38a1bf1700 1 -- 172.21.15.107:0/4099582914 wait complete. 2024-05-17T21:42:37.249 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:36.295+0000 7f38a1bf1700 1 Processor -- start 2024-05-17T21:42:37.249 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:36.295+0000 7f38a1bf1700 1 -- start start 2024-05-17T21:42:37.249 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:36.295+0000 7f38a1bf1700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f389c0712d0 0x7f389c1137c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:42:37.249 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:36.295+0000 7f38a1bf1700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f389c113d00 con 0x7f389c0712d0 2024-05-17T21:42:37.249 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:36.295+0000 7f389b7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f389c0712d0 0x7f389c1137c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:42:37.249 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:36.295+0000 7f389b7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f389c0712d0 0x7f389c1137c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34530/0 (socket says 172.21.15.107:34530) 2024-05-17T21:42:37.249 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:36.295+0000 7f389b7fe700 1 -- 172.21.15.107:0/3384008830 learned_addr learned my addr 172.21.15.107:0/3384008830 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:42:37.249 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:36.295+0000 7f389b7fe700 1 -- 172.21.15.107:0/3384008830 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f388c0094f0 con 0x7f389c0712d0 2024-05-17T21:42:37.249 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:36.296+0000 7f389b7fe700 1 --2- 172.21.15.107:0/3384008830 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f389c0712d0 0x7f389c1137c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f388c006b00 tx=0x7f388c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:42:37.249 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:36.296+0000 7f3893fff700 1 -- 172.21.15.107:0/3384008830 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f388c0107c0 con 0x7f389c0712d0 2024-05-17T21:42:37.250 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:36.296+0000 7f3893fff700 1 -- 172.21.15.107:0/3384008830 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f388c010db0 con 0x7f389c0712d0 2024-05-17T21:42:37.250 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:36.296+0000 7f38a1bf1700 1 -- 172.21.15.107:0/3384008830 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f389c113f00 con 0x7f389c0712d0 2024-05-17T21:42:37.250 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:36.296+0000 7f3893fff700 1 -- 172.21.15.107:0/3384008830 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f388c01acf0 con 0x7f389c0712d0 2024-05-17T21:42:37.250 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:36.296+0000 7f38a1bf1700 1 -- 172.21.15.107:0/3384008830 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f389c114a70 con 0x7f389c0712d0 2024-05-17T21:42:37.250 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:36.296+0000 7f3893fff700 1 -- 172.21.15.107:0/3384008830 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f388c024480 con 0x7f389c0712d0 2024-05-17T21:42:37.250 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:36.296+0000 7f3893fff700 1 -- 172.21.15.107:0/3384008830 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f388c019710 con 0x7f389c0712d0 2024-05-17T21:42:37.250 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:36.297+0000 7f38a1bf1700 1 -- 172.21.15.107:0/3384008830 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f389c05f5c0 con 0x7f389c0712d0 2024-05-17T21:42:37.250 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:36.299+0000 7f3893fff700 1 -- 172.21.15.107:0/3384008830 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71547 (secure 0 0 0) 0x7f388c042790 con 0x7f389c0712d0 2024-05-17T21:42:37.250 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:36.370+0000 7f38a1bf1700 1 -- 172.21.15.107:0/3384008830 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f389c04bed0 con 0x7f389c0712d0 2024-05-17T21:42:37.250 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:36.371+0000 7f3893fff700 1 -- 172.21.15.107:0/3384008830 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1248 (secure 0 0 0) 0x7f389c04bed0 con 0x7f389c0712d0 2024-05-17T21:42:37.250 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:36.372+0000 7f38a1bf1700 1 -- 172.21.15.107:0/3384008830 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f389c0712d0 msgr2=0x7f389c1137c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:42:37.250 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:36.372+0000 7f38a1bf1700 1 --2- 172.21.15.107:0/3384008830 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f389c0712d0 0x7f389c1137c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f388c006b00 tx=0x7f388c009250 comp rx=0 tx=0).stop 2024-05-17T21:42:37.250 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:36.372+0000 7f38a1bf1700 1 -- 172.21.15.107:0/3384008830 shutdown_connections 2024-05-17T21:42:37.251 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:36.372+0000 7f38a1bf1700 1 --2- 172.21.15.107:0/3384008830 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f389c0712d0 0x7f389c1137c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:42:37.251 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:36.372+0000 7f38a1bf1700 1 -- 172.21.15.107:0/3384008830 >> 172.21.15.107:0/3384008830 conn(0x7f389c0fcec0 msgr2=0x7f389c0fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:42:37.251 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:36.372+0000 7f38a1bf1700 1 -- 172.21.15.107:0/3384008830 shutdown_connections 2024-05-17T21:42:37.251 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:36.372+0000 7f38a1bf1700 1 -- 172.21.15.107:0/3384008830 wait complete. 2024-05-17T21:42:37.251 INFO:teuthology.orchestra.run.smithi107.stdout:mgr not available, waiting (2/15)... 2024-05-17T21:42:39.339 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:39 smithi107 ceph-mon[28434]: Activating manager daemon smithi107.xhhyiw 2024-05-17T21:42:39.340 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:39 smithi107 ceph-mon[28434]: mgrmap e2: smithi107.xhhyiw(active, starting, since 0.00299459s) 2024-05-17T21:42:39.340 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:39 smithi107 ceph-mon[28434]: from='mgr.14100 172.21.15.107:0/339311390' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-17T21:42:39.340 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:39 smithi107 ceph-mon[28434]: from='mgr.14100 172.21.15.107:0/339311390' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-17T21:42:39.340 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:39 smithi107 ceph-mon[28434]: from='mgr.14100 172.21.15.107:0/339311390' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-17T21:42:39.340 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:39 smithi107 ceph-mon[28434]: from='mgr.14100 172.21.15.107:0/339311390' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mon metadata", "id": "smithi107"}]: dispatch 2024-05-17T21:42:39.340 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:39 smithi107 ceph-mon[28434]: from='mgr.14100 172.21.15.107:0/339311390' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mgr metadata", "who": "smithi107.xhhyiw", "id": "smithi107.xhhyiw"}]: dispatch 2024-05-17T21:42:39.340 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:39 smithi107 ceph-mon[28434]: Manager daemon smithi107.xhhyiw is now available 2024-05-17T21:42:39.340 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:39 smithi107 ceph-mon[28434]: from='mgr.14100 172.21.15.107:0/339311390' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi107.xhhyiw/mirror_snapshot_schedule"}]: dispatch 2024-05-17T21:42:39.340 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:39 smithi107 ceph-mon[28434]: from='mgr.14100 172.21.15.107:0/339311390' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi107.xhhyiw/trash_purge_schedule"}]: dispatch 2024-05-17T21:42:39.340 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:39 smithi107 ceph-mon[28434]: from='mgr.14100 172.21.15.107:0/339311390' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:42:39.340 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:39 smithi107 ceph-mon[28434]: from='mgr.14100 172.21.15.107:0/339311390' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:42:39.340 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:39 smithi107 ceph-mon[28434]: from='mgr.14100 172.21.15.107:0/339311390' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:42:40.366 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-05-17T21:42:40.366 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout { 2024-05-17T21:42:40.366 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "fsid": "3722731a-1496-11ef-bc9a-c7b262605968", 2024-05-17T21:42:40.366 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "health": { 2024-05-17T21:42:40.366 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-17T21:42:40.366 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-17T21:42:40.367 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-17T21:42:40.367 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-05-17T21:42:40.367 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-17T21:42:40.367 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-17T21:42:40.367 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 0 2024-05-17T21:42:40.367 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ], 2024-05-17T21:42:40.367 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-17T21:42:40.367 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "smithi107" 2024-05-17T21:42:40.367 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ], 2024-05-17T21:42:40.367 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-05-17T21:42:40.367 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-17T21:42:40.367 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-17T21:42:40.367 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-05-17T21:42:40.368 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-17T21:42:40.368 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-05-17T21:42:40.368 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-17T21:42:40.368 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-17T21:42:40.368 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-17T21:42:40.368 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-17T21:42:40.368 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-17T21:42:40.368 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-17T21:42:40.368 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-17T21:42:40.368 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-17T21:42:40.368 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-05-17T21:42:40.368 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-17T21:42:40.368 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-17T21:42:40.368 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-17T21:42:40.369 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-17T21:42:40.369 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-17T21:42:40.369 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-17T21:42:40.369 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-17T21:42:40.369 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-17T21:42:40.369 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-17T21:42:40.369 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-05-17T21:42:40.369 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-17T21:42:40.369 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-17T21:42:40.369 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-17T21:42:40.369 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-17T21:42:40.369 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-05-17T21:42:40.369 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-17T21:42:40.369 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-17T21:42:40.369 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-17T21:42:40.369 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-17T21:42:40.370 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-17T21:42:40.370 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-17T21:42:40.370 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "restful" 2024-05-17T21:42:40.370 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ], 2024-05-17T21:42:40.370 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-17T21:42:40.370 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-05-17T21:42:40.370 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-17T21:42:40.370 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-17T21:42:40.370 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "modified": "2024-05-17T21:42:15.126617+0000", 2024-05-17T21:42:40.370 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-17T21:42:40.370 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-05-17T21:42:40.370 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-17T21:42:40.370 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout } 2024-05-17T21:42:40.371 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:39.647+0000 7f37aed96700 1 Processor -- start 2024-05-17T21:42:40.371 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:39.647+0000 7f37aed96700 1 -- start start 2024-05-17T21:42:40.371 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:39.647+0000 7f37aed96700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f37a8102880 0x7f37a8102c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:42:40.371 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:39.647+0000 7f37aed96700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37a81031d0 con 0x7f37a8102880 2024-05-17T21:42:40.371 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:39.647+0000 7f37acb32700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f37a8102880 0x7f37a8102c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:42:40.371 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:39.647+0000 7f37acb32700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f37a8102880 0x7f37a8102c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34602/0 (socket says 172.21.15.107:34602) 2024-05-17T21:42:40.371 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:39.647+0000 7f37acb32700 1 -- 172.21.15.107:0/4065036723 learned_addr learned my addr 172.21.15.107:0/4065036723 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:42:40.371 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:39.648+0000 7f37acb32700 1 -- 172.21.15.107:0/4065036723 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37a8103310 con 0x7f37a8102880 2024-05-17T21:42:40.371 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:39.648+0000 7f37acb32700 1 --2- 172.21.15.107:0/4065036723 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f37a8102880 0x7f37a8102c90 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f379c0098e0 tx=0x7f379c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bb2d46144610fd0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:42:40.371 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:39.648+0000 7f37a77fe700 1 -- 172.21.15.107:0/4065036723 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f379c00b860 con 0x7f37a8102880 2024-05-17T21:42:40.371 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:39.648+0000 7f37a77fe700 1 -- 172.21.15.107:0/4065036723 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f379c00be50 con 0x7f37a8102880 2024-05-17T21:42:40.371 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:39.648+0000 7f37a77fe700 1 -- 172.21.15.107:0/4065036723 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f379c010380 con 0x7f37a8102880 2024-05-17T21:42:40.372 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:39.649+0000 7f37aed96700 1 -- 172.21.15.107:0/4065036723 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f37a8102880 msgr2=0x7f37a8102c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:42:40.372 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:39.649+0000 7f37aed96700 1 --2- 172.21.15.107:0/4065036723 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f37a8102880 0x7f37a8102c90 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f379c0098e0 tx=0x7f379c009250 comp rx=0 tx=0).stop 2024-05-17T21:42:40.372 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:39.649+0000 7f37aed96700 1 -- 172.21.15.107:0/4065036723 shutdown_connections 2024-05-17T21:42:40.372 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:39.649+0000 7f37aed96700 1 --2- 172.21.15.107:0/4065036723 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f37a8102880 0x7f37a8102c90 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:42:40.372 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:39.649+0000 7f37aed96700 1 -- 172.21.15.107:0/4065036723 >> 172.21.15.107:0/4065036723 conn(0x7f37a80fe060 msgr2=0x7f37a8100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:42:40.372 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:39.649+0000 7f37aed96700 1 -- 172.21.15.107:0/4065036723 shutdown_connections 2024-05-17T21:42:40.372 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:39.649+0000 7f37aed96700 1 -- 172.21.15.107:0/4065036723 wait complete. 2024-05-17T21:42:40.372 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:39.649+0000 7f37aed96700 1 Processor -- start 2024-05-17T21:42:40.372 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:39.649+0000 7f37aed96700 1 -- start start 2024-05-17T21:42:40.372 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:39.650+0000 7f37aed96700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f37a8102880 0x7f37a8078c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:42:40.372 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:39.650+0000 7f37aed96700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37a8079160 con 0x7f37a8102880 2024-05-17T21:42:40.372 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:39.650+0000 7f37acb32700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f37a8102880 0x7f37a8078c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:42:40.372 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:39.650+0000 7f37acb32700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f37a8102880 0x7f37a8078c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34610/0 (socket says 172.21.15.107:34610) 2024-05-17T21:42:40.373 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:39.650+0000 7f37acb32700 1 -- 172.21.15.107:0/1085335004 learned_addr learned my addr 172.21.15.107:0/1085335004 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:42:40.373 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:39.650+0000 7f37acb32700 1 -- 172.21.15.107:0/1085335004 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f379c0094f0 con 0x7f37a8102880 2024-05-17T21:42:40.373 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:39.650+0000 7f37acb32700 1 --2- 172.21.15.107:0/1085335004 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f37a8102880 0x7f37a8078c20 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f379c00b440 tx=0x7f379c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:42:40.373 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:39.650+0000 7f37a57fa700 1 -- 172.21.15.107:0/1085335004 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f379c010760 con 0x7f37a8102880 2024-05-17T21:42:40.373 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:39.650+0000 7f37aed96700 1 -- 172.21.15.107:0/1085335004 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37a80771d0 con 0x7f37a8102880 2024-05-17T21:42:40.373 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:39.650+0000 7f37a57fa700 1 -- 172.21.15.107:0/1085335004 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f379c010d50 con 0x7f37a8102880 2024-05-17T21:42:40.373 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:39.650+0000 7f37a57fa700 1 -- 172.21.15.107:0/1085335004 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f379c01ae20 con 0x7f37a8102880 2024-05-17T21:42:40.373 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:39.650+0000 7f37aed96700 1 -- 172.21.15.107:0/1085335004 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37a80776c0 con 0x7f37a8102880 2024-05-17T21:42:40.373 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:39.651+0000 7f37a57fa700 1 -- 172.21.15.107:0/1085335004 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 2) v1 ==== 43132+0+0 (secure 0 0 0) 0x7f379c02e440 con 0x7f37a8102880 2024-05-17T21:42:40.373 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:39.651+0000 7f37a57fa700 1 -- 172.21.15.107:0/1085335004 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f379c04b0e0 con 0x7f37a8102880 2024-05-17T21:42:40.373 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:39.651+0000 7f37aed96700 1 -- 172.21.15.107:0/1085335004 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37a804bed0 con 0x7f37a8102880 2024-05-17T21:42:40.374 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:39.654+0000 7f37a57fa700 1 -- 172.21.15.107:0/1085335004 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71547 (secure 0 0 0) 0x7f379c018940 con 0x7f37a8102880 2024-05-17T21:42:40.374 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:39.722+0000 7f37aed96700 1 -- 172.21.15.107:0/1085335004 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f37a8077b40 con 0x7f37a8102880 2024-05-17T21:42:40.374 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:39.722+0000 7f37a57fa700 1 -- 172.21.15.107:0/1085335004 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1248 (secure 0 0 0) 0x7f379c049090 con 0x7f37a8102880 2024-05-17T21:42:40.374 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:39.723+0000 7f37aed96700 1 -- 172.21.15.107:0/1085335004 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f37a8102880 msgr2=0x7f37a8078c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:42:40.374 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:39.723+0000 7f37aed96700 1 --2- 172.21.15.107:0/1085335004 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f37a8102880 0x7f37a8078c20 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f379c00b440 tx=0x7f379c009250 comp rx=0 tx=0).stop 2024-05-17T21:42:40.374 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:39.723+0000 7f37aed96700 1 -- 172.21.15.107:0/1085335004 shutdown_connections 2024-05-17T21:42:40.374 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:39.723+0000 7f37aed96700 1 --2- 172.21.15.107:0/1085335004 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f37a8102880 0x7f37a8078c20 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:42:40.374 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:39.723+0000 7f37aed96700 1 -- 172.21.15.107:0/1085335004 >> 172.21.15.107:0/1085335004 conn(0x7f37a80fe060 msgr2=0x7f37a80feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:42:40.374 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:39.723+0000 7f37aed96700 1 -- 172.21.15.107:0/1085335004 shutdown_connections 2024-05-17T21:42:40.374 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:39.723+0000 7f37aed96700 1 -- 172.21.15.107:0/1085335004 wait complete. 2024-05-17T21:42:40.374 INFO:teuthology.orchestra.run.smithi107.stdout:mgr not available, waiting (3/15)... 2024-05-17T21:42:40.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:40 smithi107 ceph-mon[28434]: from='client.? 172.21.15.107:0/1085335004' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-17T21:42:40.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:40 smithi107 ceph-mon[28434]: mgrmap e3: smithi107.xhhyiw(active, since 1.00613s) 2024-05-17T21:42:42.582 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:42 smithi107 ceph-mon[28434]: mgrmap e4: smithi107.xhhyiw(active, since 2s) 2024-05-17T21:42:43.354 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:43 smithi107 ceph-mon[28434]: from='client.? 172.21.15.107:0/2784663464' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-17T21:42:43.653 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-05-17T21:42:43.655 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout { 2024-05-17T21:42:43.655 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "fsid": "3722731a-1496-11ef-bc9a-c7b262605968", 2024-05-17T21:42:43.655 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "health": { 2024-05-17T21:42:43.655 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-17T21:42:43.655 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-17T21:42:43.655 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-17T21:42:43.656 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-05-17T21:42:43.656 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-17T21:42:43.656 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-17T21:42:43.656 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 0 2024-05-17T21:42:43.656 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ], 2024-05-17T21:42:43.656 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-17T21:42:43.656 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "smithi107" 2024-05-17T21:42:43.656 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ], 2024-05-17T21:42:43.656 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "quorum_age": 13, 2024-05-17T21:42:43.656 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-17T21:42:43.656 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-17T21:42:43.656 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-05-17T21:42:43.656 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-17T21:42:43.656 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-05-17T21:42:43.657 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-17T21:42:43.657 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-17T21:42:43.657 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-17T21:42:43.657 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-17T21:42:43.657 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-17T21:42:43.664 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-17T21:42:43.664 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-17T21:42:43.664 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-17T21:42:43.664 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-05-17T21:42:43.665 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-17T21:42:43.665 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-17T21:42:43.665 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-17T21:42:43.665 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-17T21:42:43.665 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-17T21:42:43.665 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-17T21:42:43.665 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-17T21:42:43.665 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-17T21:42:43.665 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-17T21:42:43.665 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-05-17T21:42:43.665 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-17T21:42:43.665 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-17T21:42:43.665 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-17T21:42:43.665 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-17T21:42:43.666 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-05-17T21:42:43.666 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-17T21:42:43.666 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-17T21:42:43.666 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-17T21:42:43.666 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-17T21:42:43.666 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-17T21:42:43.666 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-17T21:42:43.666 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "restful" 2024-05-17T21:42:43.666 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ], 2024-05-17T21:42:43.666 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-17T21:42:43.666 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-05-17T21:42:43.666 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-17T21:42:43.666 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-17T21:42:43.666 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "modified": "2024-05-17T21:42:15.126617+0000", 2024-05-17T21:42:43.667 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-17T21:42:43.667 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-05-17T21:42:43.667 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-17T21:42:43.667 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout } 2024-05-17T21:42:43.667 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:42.779+0000 7f8aeb025700 1 Processor -- start 2024-05-17T21:42:43.667 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:42.779+0000 7f8aeb025700 1 -- start start 2024-05-17T21:42:43.667 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:42.779+0000 7f8aeb025700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8ae4100c00 0x7f8ae4101010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:42:43.667 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:42.779+0000 7f8aeb025700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae4101550 con 0x7f8ae4100c00 2024-05-17T21:42:43.667 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:42.779+0000 7f8ae8dc1700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8ae4100c00 0x7f8ae4101010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:42:43.667 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:42.779+0000 7f8ae8dc1700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8ae4100c00 0x7f8ae4101010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34620/0 (socket says 172.21.15.107:34620) 2024-05-17T21:42:43.667 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:42.779+0000 7f8ae8dc1700 1 -- 172.21.15.107:0/2706879078 learned_addr learned my addr 172.21.15.107:0/2706879078 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:42:43.667 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:42.780+0000 7f8ae8dc1700 1 -- 172.21.15.107:0/2706879078 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ae4101690 con 0x7f8ae4100c00 2024-05-17T21:42:43.667 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:42.780+0000 7f8ae8dc1700 1 --2- 172.21.15.107:0/2706879078 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8ae4100c00 0x7f8ae4101010 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8acc0098e0 tx=0x7f8acc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91e11669d98b19d2 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:42:43.668 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:42.780+0000 7f8ae37fe700 1 -- 172.21.15.107:0/2706879078 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8acc00b860 con 0x7f8ae4100c00 2024-05-17T21:42:43.668 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:42.780+0000 7f8ae37fe700 1 -- 172.21.15.107:0/2706879078 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f8acc00be50 con 0x7f8ae4100c00 2024-05-17T21:42:43.668 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:42.780+0000 7f8ae37fe700 1 -- 172.21.15.107:0/2706879078 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8acc010380 con 0x7f8ae4100c00 2024-05-17T21:42:43.668 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:42.780+0000 7f8aeb025700 1 -- 172.21.15.107:0/2706879078 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8ae4100c00 msgr2=0x7f8ae4101010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:42:43.668 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:42.780+0000 7f8aeb025700 1 --2- 172.21.15.107:0/2706879078 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8ae4100c00 0x7f8ae4101010 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8acc0098e0 tx=0x7f8acc009250 comp rx=0 tx=0).stop 2024-05-17T21:42:43.668 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:42.780+0000 7f8aeb025700 1 -- 172.21.15.107:0/2706879078 shutdown_connections 2024-05-17T21:42:43.668 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:42.780+0000 7f8aeb025700 1 --2- 172.21.15.107:0/2706879078 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8ae4100c00 0x7f8ae4101010 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:42:43.668 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:42.781+0000 7f8aeb025700 1 -- 172.21.15.107:0/2706879078 >> 172.21.15.107:0/2706879078 conn(0x7f8ae40fc6f0 msgr2=0x7f8ae40feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:42:43.668 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:42.781+0000 7f8aeb025700 1 -- 172.21.15.107:0/2706879078 shutdown_connections 2024-05-17T21:42:43.668 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:42.781+0000 7f8aeb025700 1 -- 172.21.15.107:0/2706879078 wait complete. 2024-05-17T21:42:43.668 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:42.781+0000 7f8aeb025700 1 Processor -- start 2024-05-17T21:42:43.668 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:42.781+0000 7f8aeb025700 1 -- start start 2024-05-17T21:42:43.668 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:42.781+0000 7f8aeb025700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8ae4100c00 0x7f8ae4111590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:42:43.669 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:42.781+0000 7f8aeb025700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae4111ad0 con 0x7f8ae4100c00 2024-05-17T21:42:43.669 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:42.782+0000 7f8ae8dc1700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8ae4100c00 0x7f8ae4111590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:42:43.669 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:42.782+0000 7f8ae8dc1700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8ae4100c00 0x7f8ae4111590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34622/0 (socket says 172.21.15.107:34622) 2024-05-17T21:42:43.669 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:42.782+0000 7f8ae8dc1700 1 -- 172.21.15.107:0/2784663464 learned_addr learned my addr 172.21.15.107:0/2784663464 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:42:43.669 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:42.782+0000 7f8ae8dc1700 1 -- 172.21.15.107:0/2784663464 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8acc0094f0 con 0x7f8ae4100c00 2024-05-17T21:42:43.669 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:42.782+0000 7f8ae8dc1700 1 --2- 172.21.15.107:0/2784663464 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8ae4100c00 0x7f8ae4111590 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8acc00b440 tx=0x7f8acc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:42:43.669 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:42.782+0000 7f8ae17fa700 1 -- 172.21.15.107:0/2784663464 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8acc010760 con 0x7f8ae4100c00 2024-05-17T21:42:43.669 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:42.782+0000 7f8ae17fa700 1 -- 172.21.15.107:0/2784663464 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f8acc010d50 con 0x7f8ae4100c00 2024-05-17T21:42:43.669 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:42.782+0000 7f8ae17fa700 1 -- 172.21.15.107:0/2784663464 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8acc01ae20 con 0x7f8ae4100c00 2024-05-17T21:42:43.669 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:42.782+0000 7f8aeb025700 1 -- 172.21.15.107:0/2784663464 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ae4111cd0 con 0x7f8ae4100c00 2024-05-17T21:42:43.669 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:42.782+0000 7f8aeb025700 1 -- 172.21.15.107:0/2784663464 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ae4112840 con 0x7f8ae4100c00 2024-05-17T21:42:43.669 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:42.783+0000 7f8ae17fa700 1 -- 172.21.15.107:0/2784663464 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 4) v1 ==== 43365+0+0 (secure 0 0 0) 0x7f8acc02e440 con 0x7f8ae4100c00 2024-05-17T21:42:43.669 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:42.783+0000 7f8aeb025700 1 -- 172.21.15.107:0/2784663464 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ae405f5c0 con 0x7f8ae4100c00 2024-05-17T21:42:43.670 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:42.784+0000 7f8ae17fa700 1 --2- 172.21.15.107:0/2784663464 >> [v2:172.21.15.107:6800/1677232931,v1:172.21.15.107:6801/1677232931] conn(0x7f8ad00367e0 0x7f8ad0038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:42:43.670 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:42.784+0000 7f8ae17fa700 1 -- 172.21.15.107:0/2784663464 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f8acc04a750 con 0x7f8ae4100c00 2024-05-17T21:42:43.670 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:42.784+0000 7f8ae3fff700 1 --2- 172.21.15.107:0/2784663464 >> [v2:172.21.15.107:6800/1677232931,v1:172.21.15.107:6801/1677232931] conn(0x7f8ad00367e0 0x7f8ad0038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:42:43.670 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:42.784+0000 7f8ae3fff700 1 --2- 172.21.15.107:0/2784663464 >> [v2:172.21.15.107:6800/1677232931,v1:172.21.15.107:6801/1677232931] conn(0x7f8ad00367e0 0x7f8ad0038c90 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f8ad4009990 tx=0x7f8ad4006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:42:43.670 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:42.789+0000 7f8ae17fa700 1 -- 172.21.15.107:0/2784663464 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f8acc018940 con 0x7f8ae4100c00 2024-05-17T21:42:43.670 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:43.049+0000 7f8aeb025700 1 -- 172.21.15.107:0/2784663464 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f8ae404bed0 con 0x7f8ae4100c00 2024-05-17T21:42:43.670 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:43.049+0000 7f8ae17fa700 1 -- 172.21.15.107:0/2784663464 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1248 (secure 0 0 0) 0x7f8acc018350 con 0x7f8ae4100c00 2024-05-17T21:42:43.670 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:43.052+0000 7f8aeb025700 1 -- 172.21.15.107:0/2784663464 >> [v2:172.21.15.107:6800/1677232931,v1:172.21.15.107:6801/1677232931] conn(0x7f8ad00367e0 msgr2=0x7f8ad0038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:42:43.670 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:43.052+0000 7f8aeb025700 1 --2- 172.21.15.107:0/2784663464 >> [v2:172.21.15.107:6800/1677232931,v1:172.21.15.107:6801/1677232931] conn(0x7f8ad00367e0 0x7f8ad0038c90 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f8ad4009990 tx=0x7f8ad4006e40 comp rx=0 tx=0).stop 2024-05-17T21:42:43.670 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:43.052+0000 7f8aeb025700 1 -- 172.21.15.107:0/2784663464 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8ae4100c00 msgr2=0x7f8ae4111590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:42:43.670 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:43.052+0000 7f8aeb025700 1 --2- 172.21.15.107:0/2784663464 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8ae4100c00 0x7f8ae4111590 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8acc00b440 tx=0x7f8acc009250 comp rx=0 tx=0).stop 2024-05-17T21:42:43.670 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:43.052+0000 7f8aeb025700 1 -- 172.21.15.107:0/2784663464 shutdown_connections 2024-05-17T21:42:43.670 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:43.052+0000 7f8aeb025700 1 --2- 172.21.15.107:0/2784663464 >> [v2:172.21.15.107:6800/1677232931,v1:172.21.15.107:6801/1677232931] conn(0x7f8ad00367e0 0x7f8ad0038c90 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:42:43.671 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:43.052+0000 7f8aeb025700 1 --2- 172.21.15.107:0/2784663464 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8ae4100c00 0x7f8ae4111590 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:42:43.671 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:43.052+0000 7f8aeb025700 1 -- 172.21.15.107:0/2784663464 >> 172.21.15.107:0/2784663464 conn(0x7f8ae40fc6f0 msgr2=0x7f8ae40fd180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:42:43.671 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:43.052+0000 7f8aeb025700 1 -- 172.21.15.107:0/2784663464 shutdown_connections 2024-05-17T21:42:43.671 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:43.052+0000 7f8aeb025700 1 -- 172.21.15.107:0/2784663464 wait complete. 2024-05-17T21:42:43.671 INFO:teuthology.orchestra.run.smithi107.stdout:mgr is available 2024-05-17T21:42:44.673 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:44 smithi107 ceph-mon[28434]: from='client.? 172.21.15.107:0/4224080548' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-17T21:42:44.931 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-05-17T21:42:44.931 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout [global] 2024-05-17T21:42:44.931 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout fsid = 3722731a-1496-11ef-bc9a-c7b262605968 2024-05-17T21:42:44.931 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-17T21:42:44.932 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-17T21:42:44.932 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-17T21:42:44.932 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-17T21:42:44.932 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-05-17T21:42:44.932 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-17T21:42:44.932 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-17T21:42:44.932 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-05-17T21:42:44.932 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout [osd] 2024-05-17T21:42:44.932 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-17T21:42:44.932 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-17T21:42:44.932 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-17T21:42:44.932 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-17T21:42:44.932 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-17T21:42:44.932 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:44.124+0000 7f0856e49700 1 Processor -- start 2024-05-17T21:42:44.933 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:44.124+0000 7f0856e49700 1 -- start start 2024-05-17T21:42:44.933 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:44.124+0000 7f0856e49700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0850101740 0x7f0850101b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:42:44.933 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:44.124+0000 7f0856e49700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0850102090 con 0x7f0850101740 2024-05-17T21:42:44.933 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:44.125+0000 7f0854be5700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0850101740 0x7f0850101b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:42:44.933 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:44.125+0000 7f0854be5700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0850101740 0x7f0850101b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34632/0 (socket says 172.21.15.107:34632) 2024-05-17T21:42:44.933 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:44.125+0000 7f0854be5700 1 -- 172.21.15.107:0/2314306798 learned_addr learned my addr 172.21.15.107:0/2314306798 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:42:44.933 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:44.125+0000 7f0854be5700 1 -- 172.21.15.107:0/2314306798 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08501021d0 con 0x7f0850101740 2024-05-17T21:42:44.933 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:44.125+0000 7f0854be5700 1 --2- 172.21.15.107:0/2314306798 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0850101740 0x7f0850101b50 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f08440098e0 tx=0x7f0844009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51aab403f5776f86 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:42:44.933 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:44.125+0000 7f084f7fe700 1 -- 172.21.15.107:0/2314306798 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f084400b860 con 0x7f0850101740 2024-05-17T21:42:44.933 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:44.125+0000 7f084f7fe700 1 -- 172.21.15.107:0/2314306798 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f084400be50 con 0x7f0850101740 2024-05-17T21:42:44.933 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:44.125+0000 7f084f7fe700 1 -- 172.21.15.107:0/2314306798 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0844010380 con 0x7f0850101740 2024-05-17T21:42:44.933 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:44.126+0000 7f0856e49700 1 -- 172.21.15.107:0/2314306798 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0850101740 msgr2=0x7f0850101b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:42:44.933 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:44.126+0000 7f0856e49700 1 --2- 172.21.15.107:0/2314306798 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0850101740 0x7f0850101b50 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f08440098e0 tx=0x7f0844009250 comp rx=0 tx=0).stop 2024-05-17T21:42:44.934 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:44.126+0000 7f0856e49700 1 -- 172.21.15.107:0/2314306798 shutdown_connections 2024-05-17T21:42:44.934 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:44.126+0000 7f0856e49700 1 --2- 172.21.15.107:0/2314306798 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0850101740 0x7f0850101b50 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:42:44.934 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:44.126+0000 7f0856e49700 1 -- 172.21.15.107:0/2314306798 >> 172.21.15.107:0/2314306798 conn(0x7f08500fcec0 msgr2=0x7f08500ff2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:42:44.934 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:44.126+0000 7f0856e49700 1 -- 172.21.15.107:0/2314306798 shutdown_connections 2024-05-17T21:42:44.934 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:44.126+0000 7f0856e49700 1 -- 172.21.15.107:0/2314306798 wait complete. 2024-05-17T21:42:44.934 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:44.126+0000 7f0856e49700 1 Processor -- start 2024-05-17T21:42:44.934 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:44.127+0000 7f0856e49700 1 -- start start 2024-05-17T21:42:44.934 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:44.127+0000 7f0856e49700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0850101740 0x7f0850111590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:42:44.934 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:44.127+0000 7f0856e49700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0850111ad0 con 0x7f0850101740 2024-05-17T21:42:44.934 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:44.127+0000 7f0854be5700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0850101740 0x7f0850111590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:42:44.934 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:44.127+0000 7f0854be5700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0850101740 0x7f0850111590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34640/0 (socket says 172.21.15.107:34640) 2024-05-17T21:42:44.934 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:44.127+0000 7f0854be5700 1 -- 172.21.15.107:0/4224080548 learned_addr learned my addr 172.21.15.107:0/4224080548 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:42:44.934 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:44.127+0000 7f0854be5700 1 -- 172.21.15.107:0/4224080548 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08440094f0 con 0x7f0850101740 2024-05-17T21:42:44.935 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:44.127+0000 7f0854be5700 1 --2- 172.21.15.107:0/4224080548 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0850101740 0x7f0850111590 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0844000c00 tx=0x7f0844009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:42:44.935 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:44.128+0000 7f084d7fa700 1 -- 172.21.15.107:0/4224080548 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f08440107c0 con 0x7f0850101740 2024-05-17T21:42:44.935 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:44.128+0000 7f084d7fa700 1 -- 172.21.15.107:0/4224080548 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f0844010db0 con 0x7f0850101740 2024-05-17T21:42:44.935 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:44.128+0000 7f0856e49700 1 -- 172.21.15.107:0/4224080548 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0850111cd0 con 0x7f0850101740 2024-05-17T21:42:44.935 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:44.128+0000 7f084d7fa700 1 -- 172.21.15.107:0/4224080548 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f084401ad00 con 0x7f0850101740 2024-05-17T21:42:44.935 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:44.128+0000 7f0856e49700 1 -- 172.21.15.107:0/4224080548 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0850112840 con 0x7f0850101740 2024-05-17T21:42:44.935 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:44.128+0000 7f084d7fa700 1 -- 172.21.15.107:0/4224080548 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 4) v1 ==== 43365+0+0 (secure 0 0 0) 0x7f08440193c0 con 0x7f0850101740 2024-05-17T21:42:44.935 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:44.129+0000 7f0856e49700 1 -- 172.21.15.107:0/4224080548 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f085005f5c0 con 0x7f0850101740 2024-05-17T21:42:44.935 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:44.129+0000 7f084d7fa700 1 --2- 172.21.15.107:0/4224080548 >> [v2:172.21.15.107:6800/1677232931,v1:172.21.15.107:6801/1677232931] conn(0x7f0840036800 0x7f0840038cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:42:44.935 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:44.129+0000 7f084d7fa700 1 -- 172.21.15.107:0/4224080548 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f084404a660 con 0x7f0850101740 2024-05-17T21:42:44.935 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:44.129+0000 7f084ffff700 1 --2- 172.21.15.107:0/4224080548 >> [v2:172.21.15.107:6800/1677232931,v1:172.21.15.107:6801/1677232931] conn(0x7f0840036800 0x7f0840038cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:42:44.935 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:44.129+0000 7f084ffff700 1 --2- 172.21.15.107:0/4224080548 >> [v2:172.21.15.107:6800/1677232931,v1:172.21.15.107:6801/1677232931] conn(0x7f0840036800 0x7f0840038cb0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f083c009990 tx=0x7f083c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:42:44.935 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:44.133+0000 7f084d7fa700 1 -- 172.21.15.107:0/4224080548 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f0844019670 con 0x7f0850101740 2024-05-17T21:42:44.935 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:44.320+0000 7f0856e49700 1 -- 172.21.15.107:0/4224080548 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f085004bed0 con 0x7f0850101740 2024-05-17T21:42:44.935 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:44.321+0000 7f084d7fa700 1 -- 172.21.15.107:0/4224080548 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7f0844022340 con 0x7f0850101740 2024-05-17T21:42:44.936 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:44.323+0000 7f0856e49700 1 -- 172.21.15.107:0/4224080548 >> [v2:172.21.15.107:6800/1677232931,v1:172.21.15.107:6801/1677232931] conn(0x7f0840036800 msgr2=0x7f0840038cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:42:44.936 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:44.323+0000 7f0856e49700 1 --2- 172.21.15.107:0/4224080548 >> [v2:172.21.15.107:6800/1677232931,v1:172.21.15.107:6801/1677232931] conn(0x7f0840036800 0x7f0840038cb0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f083c009990 tx=0x7f083c006e40 comp rx=0 tx=0).stop 2024-05-17T21:42:44.936 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:44.323+0000 7f0856e49700 1 -- 172.21.15.107:0/4224080548 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0850101740 msgr2=0x7f0850111590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:42:44.936 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:44.323+0000 7f0856e49700 1 --2- 172.21.15.107:0/4224080548 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0850101740 0x7f0850111590 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0844000c00 tx=0x7f0844009250 comp rx=0 tx=0).stop 2024-05-17T21:42:44.936 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:44.323+0000 7f0856e49700 1 -- 172.21.15.107:0/4224080548 shutdown_connections 2024-05-17T21:42:44.936 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:44.323+0000 7f0856e49700 1 --2- 172.21.15.107:0/4224080548 >> [v2:172.21.15.107:6800/1677232931,v1:172.21.15.107:6801/1677232931] conn(0x7f0840036800 0x7f0840038cb0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:42:44.936 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:44.323+0000 7f0856e49700 1 --2- 172.21.15.107:0/4224080548 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0850101740 0x7f0850111590 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:42:44.936 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:44.323+0000 7f0856e49700 1 -- 172.21.15.107:0/4224080548 >> 172.21.15.107:0/4224080548 conn(0x7f08500fcec0 msgr2=0x7f08500fdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:42:44.936 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:44.323+0000 7f0856e49700 1 -- 172.21.15.107:0/4224080548 shutdown_connections 2024-05-17T21:42:44.936 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:44.323+0000 7f0856e49700 1 -- 172.21.15.107:0/4224080548 wait complete. 2024-05-17T21:42:44.936 INFO:teuthology.orchestra.run.smithi107.stdout:Enabling cephadm module... 2024-05-17T21:42:45.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:45 smithi107 ceph-mon[28434]: from='client.? 172.21.15.107:0/1242624698' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-05-17T21:42:46.161 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:45.336+0000 7f7179322700 1 Processor -- start 2024-05-17T21:42:46.161 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:45.336+0000 7f7179322700 1 -- start start 2024-05-17T21:42:46.161 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:45.336+0000 7f7179322700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7174102880 0x7f7174102c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:42:46.161 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:45.336+0000 7f7179322700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71741031d0 con 0x7f7174102880 2024-05-17T21:42:46.162 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:45.336+0000 7f7172ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7174102880 0x7f7174102c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:42:46.162 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:45.336+0000 7f7172ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7174102880 0x7f7174102c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34656/0 (socket says 172.21.15.107:34656) 2024-05-17T21:42:46.162 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:45.336+0000 7f7172ffd700 1 -- 172.21.15.107:0/3819659621 learned_addr learned my addr 172.21.15.107:0/3819659621 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:42:46.162 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:45.337+0000 7f7172ffd700 1 -- 172.21.15.107:0/3819659621 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7174103310 con 0x7f7174102880 2024-05-17T21:42:46.162 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:45.337+0000 7f7172ffd700 1 --2- 172.21.15.107:0/3819659621 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7174102880 0x7f7174102c90 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f71640098e0 tx=0x7f7164009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4874946d99a84112 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:42:46.162 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:45.337+0000 7f71727fc700 1 -- 172.21.15.107:0/3819659621 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f716400b860 con 0x7f7174102880 2024-05-17T21:42:46.162 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:45.337+0000 7f71727fc700 1 -- 172.21.15.107:0/3819659621 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f716400be50 con 0x7f7174102880 2024-05-17T21:42:46.162 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:45.337+0000 7f71727fc700 1 -- 172.21.15.107:0/3819659621 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7164010380 con 0x7f7174102880 2024-05-17T21:42:46.162 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:45.337+0000 7f7179322700 1 -- 172.21.15.107:0/3819659621 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7174102880 msgr2=0x7f7174102c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:42:46.162 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:45.337+0000 7f7179322700 1 --2- 172.21.15.107:0/3819659621 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7174102880 0x7f7174102c90 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f71640098e0 tx=0x7f7164009250 comp rx=0 tx=0).stop 2024-05-17T21:42:46.162 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:45.338+0000 7f7179322700 1 -- 172.21.15.107:0/3819659621 shutdown_connections 2024-05-17T21:42:46.162 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:45.338+0000 7f7179322700 1 --2- 172.21.15.107:0/3819659621 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7174102880 0x7f7174102c90 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:42:46.162 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:45.338+0000 7f7179322700 1 -- 172.21.15.107:0/3819659621 >> 172.21.15.107:0/3819659621 conn(0x7f71740fe060 msgr2=0x7f7174100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:42:46.163 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:45.338+0000 7f7179322700 1 -- 172.21.15.107:0/3819659621 shutdown_connections 2024-05-17T21:42:46.163 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:45.338+0000 7f7179322700 1 -- 172.21.15.107:0/3819659621 wait complete. 2024-05-17T21:42:46.163 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:45.338+0000 7f7179322700 1 Processor -- start 2024-05-17T21:42:46.163 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:45.338+0000 7f7179322700 1 -- start start 2024-05-17T21:42:46.163 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:45.338+0000 7f7179322700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7174102880 0x7f71741115f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:42:46.163 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:45.338+0000 7f7179322700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7174111b30 con 0x7f7174102880 2024-05-17T21:42:46.163 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:45.339+0000 7f7172ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7174102880 0x7f71741115f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:42:46.163 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:45.339+0000 7f7172ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7174102880 0x7f71741115f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34662/0 (socket says 172.21.15.107:34662) 2024-05-17T21:42:46.163 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:45.339+0000 7f7172ffd700 1 -- 172.21.15.107:0/1242624698 learned_addr learned my addr 172.21.15.107:0/1242624698 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:42:46.163 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:45.339+0000 7f7172ffd700 1 -- 172.21.15.107:0/1242624698 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71640094f0 con 0x7f7174102880 2024-05-17T21:42:46.163 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:45.339+0000 7f7172ffd700 1 --2- 172.21.15.107:0/1242624698 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7174102880 0x7f71741115f0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f716400b440 tx=0x7f7164009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:42:46.163 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:45.339+0000 7f716bfff700 1 -- 172.21.15.107:0/1242624698 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7164010760 con 0x7f7174102880 2024-05-17T21:42:46.163 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:45.339+0000 7f716bfff700 1 -- 172.21.15.107:0/1242624698 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f7164010d50 con 0x7f7174102880 2024-05-17T21:42:46.163 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:45.339+0000 7f7179322700 1 -- 172.21.15.107:0/1242624698 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7174111d30 con 0x7f7174102880 2024-05-17T21:42:46.164 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:45.339+0000 7f716bfff700 1 -- 172.21.15.107:0/1242624698 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f716401ae20 con 0x7f7174102880 2024-05-17T21:42:46.164 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:45.339+0000 7f7179322700 1 -- 172.21.15.107:0/1242624698 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71741128a0 con 0x7f7174102880 2024-05-17T21:42:46.164 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:45.340+0000 7f716bfff700 1 -- 172.21.15.107:0/1242624698 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 4) v1 ==== 43365+0+0 (secure 0 0 0) 0x7f716402e440 con 0x7f7174102880 2024-05-17T21:42:46.164 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:45.340+0000 7f716bfff700 1 --2- 172.21.15.107:0/1242624698 >> [v2:172.21.15.107:6800/1677232931,v1:172.21.15.107:6801/1677232931] conn(0x7f71600367e0 0x7f7160038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:42:46.164 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:45.340+0000 7f7179322700 1 -- 172.21.15.107:0/1242624698 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f717405f5c0 con 0x7f7174102880 2024-05-17T21:42:46.164 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:45.340+0000 7f716bfff700 1 -- 172.21.15.107:0/1242624698 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f716404a550 con 0x7f7174102880 2024-05-17T21:42:46.164 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:45.340+0000 7f716a7fc700 1 --2- 172.21.15.107:0/1242624698 >> [v2:172.21.15.107:6800/1677232931,v1:172.21.15.107:6801/1677232931] conn(0x7f71600367e0 0x7f7160038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:42:46.164 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:45.341+0000 7f716a7fc700 1 --2- 172.21.15.107:0/1242624698 >> [v2:172.21.15.107:6800/1677232931,v1:172.21.15.107:6801/1677232931] conn(0x7f71600367e0 0x7f7160038c90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f715c009990 tx=0x7f715c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:42:46.164 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:45.346+0000 7f716bfff700 1 -- 172.21.15.107:0/1242624698 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f7164018940 con 0x7f7174102880 2024-05-17T21:42:46.164 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:45.580+0000 7f7179322700 1 -- 172.21.15.107:0/1242624698 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f717404bed0 con 0x7f7174102880 2024-05-17T21:42:46.164 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:45.633+0000 7f716bfff700 1 -- 172.21.15.107:0/1242624698 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mgrmap(e 5) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f71640243f0 con 0x7f7174102880 2024-05-17T21:42:46.164 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:45.634+0000 7f716bfff700 1 -- 172.21.15.107:0/1242624698 <== mon.0 v2:172.21.15.107:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f716402d760 con 0x7f7174102880 2024-05-17T21:42:46.164 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:45.636+0000 7f7179322700 1 -- 172.21.15.107:0/1242624698 >> [v2:172.21.15.107:6800/1677232931,v1:172.21.15.107:6801/1677232931] conn(0x7f71600367e0 msgr2=0x7f7160038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:42:46.164 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:45.636+0000 7f7179322700 1 --2- 172.21.15.107:0/1242624698 >> [v2:172.21.15.107:6800/1677232931,v1:172.21.15.107:6801/1677232931] conn(0x7f71600367e0 0x7f7160038c90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f715c009990 tx=0x7f715c006e40 comp rx=0 tx=0).stop 2024-05-17T21:42:46.164 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:45.636+0000 7f7179322700 1 -- 172.21.15.107:0/1242624698 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7174102880 msgr2=0x7f71741115f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:42:46.165 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:45.636+0000 7f7179322700 1 --2- 172.21.15.107:0/1242624698 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7174102880 0x7f71741115f0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f716400b440 tx=0x7f7164009250 comp rx=0 tx=0).stop 2024-05-17T21:42:46.165 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:45.636+0000 7f7179322700 1 -- 172.21.15.107:0/1242624698 shutdown_connections 2024-05-17T21:42:46.165 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:45.636+0000 7f7179322700 1 --2- 172.21.15.107:0/1242624698 >> [v2:172.21.15.107:6800/1677232931,v1:172.21.15.107:6801/1677232931] conn(0x7f71600367e0 0x7f7160038c90 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:42:46.165 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:45.636+0000 7f7179322700 1 --2- 172.21.15.107:0/1242624698 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7174102880 0x7f71741115f0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:42:46.165 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:45.636+0000 7f7179322700 1 -- 172.21.15.107:0/1242624698 >> 172.21.15.107:0/1242624698 conn(0x7f71740fe060 msgr2=0x7f71740feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:42:46.165 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:45.636+0000 7f7179322700 1 -- 172.21.15.107:0/1242624698 shutdown_connections 2024-05-17T21:42:46.165 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:45.636+0000 7f7179322700 1 -- 172.21.15.107:0/1242624698 wait complete. 2024-05-17T21:42:46.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:46 smithi107 ceph-mon[28434]: from='client.? 172.21.15.107:0/1242624698' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-05-17T21:42:46.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:46 smithi107 ceph-mon[28434]: mgrmap e5: smithi107.xhhyiw(active, since 6s) 2024-05-17T21:42:47.600 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout { 2024-05-17T21:42:47.600 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-05-17T21:42:47.600 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-17T21:42:47.600 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "active_name": "smithi107.xhhyiw", 2024-05-17T21:42:47.600 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-17T21:42:47.600 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout } 2024-05-17T21:42:47.600 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:46.702+0000 7f9898505700 1 Processor -- start 2024-05-17T21:42:47.600 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:46.702+0000 7f9898505700 1 -- start start 2024-05-17T21:42:47.600 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:46.702+0000 7f9898505700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9890101d40 0x7f9890102150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:42:47.601 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:46.702+0000 7f9898505700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9890102690 con 0x7f9890101d40 2024-05-17T21:42:47.601 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:46.702+0000 7f98962a1700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9890101d40 0x7f9890102150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:42:47.601 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:46.702+0000 7f98962a1700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9890101d40 0x7f9890102150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:58288/0 (socket says 172.21.15.107:58288) 2024-05-17T21:42:47.601 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:46.703+0000 7f98962a1700 1 -- 172.21.15.107:0/1763001372 learned_addr learned my addr 172.21.15.107:0/1763001372 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:42:47.601 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:46.703+0000 7f98962a1700 1 -- 172.21.15.107:0/1763001372 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98901027d0 con 0x7f9890101d40 2024-05-17T21:42:47.601 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:46.703+0000 7f98962a1700 1 --2- 172.21.15.107:0/1763001372 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9890101d40 0x7f9890102150 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f98800098e0 tx=0x7f9880009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97b4fad576ae295f server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:42:47.601 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:46.703+0000 7f989529f700 1 -- 172.21.15.107:0/1763001372 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f988000b860 con 0x7f9890101d40 2024-05-17T21:42:47.601 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:46.703+0000 7f989529f700 1 -- 172.21.15.107:0/1763001372 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f988000be50 con 0x7f9890101d40 2024-05-17T21:42:47.601 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:46.703+0000 7f989529f700 1 -- 172.21.15.107:0/1763001372 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9880010380 con 0x7f9890101d40 2024-05-17T21:42:47.601 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:46.704+0000 7f9898505700 1 -- 172.21.15.107:0/1763001372 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9890101d40 msgr2=0x7f9890102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:42:47.601 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:46.704+0000 7f9898505700 1 --2- 172.21.15.107:0/1763001372 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9890101d40 0x7f9890102150 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f98800098e0 tx=0x7f9880009250 comp rx=0 tx=0).stop 2024-05-17T21:42:47.601 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:46.704+0000 7f9898505700 1 -- 172.21.15.107:0/1763001372 shutdown_connections 2024-05-17T21:42:47.601 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:46.704+0000 7f9898505700 1 --2- 172.21.15.107:0/1763001372 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9890101d40 0x7f9890102150 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:42:47.602 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:46.704+0000 7f9898505700 1 -- 172.21.15.107:0/1763001372 >> 172.21.15.107:0/1763001372 conn(0x7f98900fc6b0 msgr2=0x7f98900feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:42:47.602 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:46.704+0000 7f9898505700 1 -- 172.21.15.107:0/1763001372 shutdown_connections 2024-05-17T21:42:47.602 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:46.704+0000 7f9898505700 1 -- 172.21.15.107:0/1763001372 wait complete. 2024-05-17T21:42:47.602 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:46.704+0000 7f9898505700 1 Processor -- start 2024-05-17T21:42:47.602 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:46.705+0000 7f9898505700 1 -- start start 2024-05-17T21:42:47.602 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:46.705+0000 7f9898505700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9890101d40 0x7f9890101550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:42:47.602 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:46.705+0000 7f9898505700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9890114290 con 0x7f9890101d40 2024-05-17T21:42:47.602 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:46.705+0000 7f98962a1700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9890101d40 0x7f9890101550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:42:47.602 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:46.705+0000 7f98962a1700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9890101d40 0x7f9890101550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:58292/0 (socket says 172.21.15.107:58292) 2024-05-17T21:42:47.602 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:46.705+0000 7f98962a1700 1 -- 172.21.15.107:0/925798188 learned_addr learned my addr 172.21.15.107:0/925798188 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:42:47.602 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:46.705+0000 7f98962a1700 1 -- 172.21.15.107:0/925798188 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98800094f0 con 0x7f9890101d40 2024-05-17T21:42:47.602 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:46.705+0000 7f98962a1700 1 --2- 172.21.15.107:0/925798188 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9890101d40 0x7f9890101550 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f988000b440 tx=0x7f9880009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:42:47.603 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:46.705+0000 7f9886ffd700 1 -- 172.21.15.107:0/925798188 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9880010760 con 0x7f9890101d40 2024-05-17T21:42:47.603 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:46.705+0000 7f9886ffd700 1 -- 172.21.15.107:0/925798188 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f9880010d50 con 0x7f9890101d40 2024-05-17T21:42:47.603 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:46.705+0000 7f9898505700 1 -- 172.21.15.107:0/925798188 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98901143d0 con 0x7f9890101d40 2024-05-17T21:42:47.603 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:46.705+0000 7f9886ffd700 1 -- 172.21.15.107:0/925798188 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f988001ae20 con 0x7f9890101d40 2024-05-17T21:42:47.603 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:46.705+0000 7f9898505700 1 -- 172.21.15.107:0/925798188 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9890114860 con 0x7f9890101d40 2024-05-17T21:42:47.603 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:46.706+0000 7f9886ffd700 1 -- 172.21.15.107:0/925798188 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 5) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f988002e440 con 0x7f9890101d40 2024-05-17T21:42:47.603 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:46.706+0000 7f9898505700 1 -- 172.21.15.107:0/925798188 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f989004bed0 con 0x7f9890101d40 2024-05-17T21:42:47.603 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:46.706+0000 7f9886ffd700 1 --2- 172.21.15.107:0/925798188 >> [v2:172.21.15.107:6800/1677232931,v1:172.21.15.107:6801/1677232931] conn(0x7f987c036830 0x7f987c038ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:42:47.603 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:46.707+0000 7f9895aa0700 1 -- 172.21.15.107:0/925798188 >> [v2:172.21.15.107:6800/1677232931,v1:172.21.15.107:6801/1677232931] conn(0x7f987c036830 msgr2=0x7f987c038ce0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/1677232931 2024-05-17T21:42:47.603 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:46.707+0000 7f9886ffd700 1 -- 172.21.15.107:0/925798188 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f988004a730 con 0x7f9890101d40 2024-05-17T21:42:47.603 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:46.707+0000 7f9895aa0700 1 --2- 172.21.15.107:0/925798188 >> [v2:172.21.15.107:6800/1677232931,v1:172.21.15.107:6801/1677232931] conn(0x7f987c036830 0x7f987c038ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-17T21:42:47.603 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:46.711+0000 7f9886ffd700 1 -- 172.21.15.107:0/925798188 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f9880018940 con 0x7f9890101d40 2024-05-17T21:42:47.603 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:46.908+0000 7f9895aa0700 1 -- 172.21.15.107:0/925798188 >> [v2:172.21.15.107:6800/1677232931,v1:172.21.15.107:6801/1677232931] conn(0x7f987c036830 msgr2=0x7f987c038ce0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/1677232931 2024-05-17T21:42:47.604 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:46.908+0000 7f9895aa0700 1 --2- 172.21.15.107:0/925798188 >> [v2:172.21.15.107:6800/1677232931,v1:172.21.15.107:6801/1677232931] conn(0x7f987c036830 0x7f987c038ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-17T21:42:47.604 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:46.991+0000 7f9898505700 1 -- 172.21.15.107:0/925798188 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f9890114db0 con 0x7f9890101d40 2024-05-17T21:42:47.604 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:46.991+0000 7f9886ffd700 1 -- 172.21.15.107:0/925798188 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+103 (secure 0 0 0) 0x7f9880018350 con 0x7f9890101d40 2024-05-17T21:42:47.604 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:46.994+0000 7f9898505700 1 -- 172.21.15.107:0/925798188 >> [v2:172.21.15.107:6800/1677232931,v1:172.21.15.107:6801/1677232931] conn(0x7f987c036830 msgr2=0x7f987c038ce0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-17T21:42:47.604 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:46.994+0000 7f9898505700 1 --2- 172.21.15.107:0/925798188 >> [v2:172.21.15.107:6800/1677232931,v1:172.21.15.107:6801/1677232931] conn(0x7f987c036830 0x7f987c038ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:42:47.604 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:46.994+0000 7f9898505700 1 -- 172.21.15.107:0/925798188 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9890101d40 msgr2=0x7f9890101550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:42:47.604 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:46.994+0000 7f9898505700 1 --2- 172.21.15.107:0/925798188 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9890101d40 0x7f9890101550 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f988000b440 tx=0x7f9880009250 comp rx=0 tx=0).stop 2024-05-17T21:42:47.604 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:46.994+0000 7f9898505700 1 -- 172.21.15.107:0/925798188 shutdown_connections 2024-05-17T21:42:47.604 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:46.994+0000 7f9898505700 1 --2- 172.21.15.107:0/925798188 >> [v2:172.21.15.107:6800/1677232931,v1:172.21.15.107:6801/1677232931] conn(0x7f987c036830 0x7f987c038ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:42:47.604 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:46.994+0000 7f9898505700 1 --2- 172.21.15.107:0/925798188 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9890101d40 0x7f9890101550 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:42:47.604 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:46.994+0000 7f9898505700 1 -- 172.21.15.107:0/925798188 >> 172.21.15.107:0/925798188 conn(0x7f98900fc6b0 msgr2=0x7f98900fd140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:42:47.604 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:46.994+0000 7f9898505700 1 -- 172.21.15.107:0/925798188 shutdown_connections 2024-05-17T21:42:47.604 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:46.994+0000 7f9898505700 1 -- 172.21.15.107:0/925798188 wait complete. 2024-05-17T21:42:47.605 INFO:teuthology.orchestra.run.smithi107.stdout:Waiting for the mgr to restart... 2024-05-17T21:42:47.605 INFO:teuthology.orchestra.run.smithi107.stdout:Waiting for mgr epoch 5... 2024-05-17T21:42:47.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:47 smithi107 ceph-mon[28434]: from='client.? 172.21.15.107:0/925798188' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-17T21:42:52.354 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:52 smithi107 ceph-mon[28434]: Active manager daemon smithi107.xhhyiw restarted 2024-05-17T21:42:52.355 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:52 smithi107 ceph-mon[28434]: Activating manager daemon smithi107.xhhyiw 2024-05-17T21:42:52.355 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:52 smithi107 ceph-mon[28434]: osdmap e2: 0 total, 0 up, 0 in 2024-05-17T21:42:52.355 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:52 smithi107 ceph-mon[28434]: mgrmap e6: smithi107.xhhyiw(active, starting, since 0.00328076s) 2024-05-17T21:42:52.355 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:52 smithi107 ceph-mon[28434]: from='mgr.14120 172.21.15.107:0/2711624202' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mon metadata", "id": "smithi107"}]: dispatch 2024-05-17T21:42:52.355 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:52 smithi107 ceph-mon[28434]: from='mgr.14120 172.21.15.107:0/2711624202' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mgr metadata", "who": "smithi107.xhhyiw", "id": "smithi107.xhhyiw"}]: dispatch 2024-05-17T21:42:52.355 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:52 smithi107 ceph-mon[28434]: from='mgr.14120 172.21.15.107:0/2711624202' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-17T21:42:52.355 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:52 smithi107 ceph-mon[28434]: from='mgr.14120 172.21.15.107:0/2711624202' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-17T21:42:52.355 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:52 smithi107 ceph-mon[28434]: from='mgr.14120 172.21.15.107:0/2711624202' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-17T21:42:52.355 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:52 smithi107 ceph-mon[28434]: Manager daemon smithi107.xhhyiw is now available 2024-05-17T21:42:52.355 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:52 smithi107 ceph-mon[28434]: from='mgr.14120 172.21.15.107:0/2711624202' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:42:52.355 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:52 smithi107 ceph-mon[28434]: from='mgr.14120 172.21.15.107:0/2711624202' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:42:52.355 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:52 smithi107 ceph-mon[28434]: from='mgr.14120 172.21.15.107:0/2711624202' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:42:52.356 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:52 smithi107 ceph-mon[28434]: from='mgr.14120 172.21.15.107:0/2711624202' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:42:52.356 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:52 smithi107 ceph-mon[28434]: from='mgr.14120 172.21.15.107:0/2711624202' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:42:52.356 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:52 smithi107 ceph-mon[28434]: from='mgr.14120 172.21.15.107:0/2711624202' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi107.xhhyiw/mirror_snapshot_schedule"}]: dispatch 2024-05-17T21:42:52.356 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:52 smithi107 ceph-mon[28434]: from='mgr.14120 172.21.15.107:0/2711624202' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi107.xhhyiw/trash_purge_schedule"}]: dispatch 2024-05-17T21:42:53.354 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:53 smithi107 ceph-mon[28434]: Found migration_current of "None". Setting to last migration. 2024-05-17T21:42:53.354 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:53 smithi107 ceph-mon[28434]: mgrmap e7: smithi107.xhhyiw(active, since 1.0056s) 2024-05-17T21:42:53.642 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout { 2024-05-17T21:42:53.642 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-05-17T21:42:53.642 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-17T21:42:53.642 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout } 2024-05-17T21:42:53.643 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:48.051+0000 7fb30c405700 1 Processor -- start 2024-05-17T21:42:53.643 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:48.051+0000 7fb30c405700 1 -- start start 2024-05-17T21:42:53.643 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:48.051+0000 7fb30c405700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb304102810 0x7fb304102c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:42:53.643 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:48.051+0000 7fb30c405700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb304103160 con 0x7fb304102810 2024-05-17T21:42:53.643 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:48.051+0000 7fb30a1a1700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb304102810 0x7fb304102c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:42:53.643 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:48.051+0000 7fb30a1a1700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb304102810 0x7fb304102c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:58308/0 (socket says 172.21.15.107:58308) 2024-05-17T21:42:53.643 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:48.051+0000 7fb30a1a1700 1 -- 172.21.15.107:0/2011133828 learned_addr learned my addr 172.21.15.107:0/2011133828 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:42:53.643 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:48.052+0000 7fb30a1a1700 1 -- 172.21.15.107:0/2011133828 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3041032a0 con 0x7fb304102810 2024-05-17T21:42:53.643 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:48.052+0000 7fb30a1a1700 1 --2- 172.21.15.107:0/2011133828 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb304102810 0x7fb304102c20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb3000098e0 tx=0x7fb300009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99064c0374ba2c05 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:42:53.643 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:48.052+0000 7fb30919f700 1 -- 172.21.15.107:0/2011133828 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb30000b860 con 0x7fb304102810 2024-05-17T21:42:53.643 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:48.052+0000 7fb30919f700 1 -- 172.21.15.107:0/2011133828 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fb30000be50 con 0x7fb304102810 2024-05-17T21:42:53.643 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:48.052+0000 7fb30919f700 1 -- 172.21.15.107:0/2011133828 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb300010380 con 0x7fb304102810 2024-05-17T21:42:53.643 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:48.053+0000 7fb30c405700 1 -- 172.21.15.107:0/2011133828 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb304102810 msgr2=0x7fb304102c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:42:53.643 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:48.053+0000 7fb30c405700 1 --2- 172.21.15.107:0/2011133828 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb304102810 0x7fb304102c20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb3000098e0 tx=0x7fb300009250 comp rx=0 tx=0).stop 2024-05-17T21:42:53.644 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:48.053+0000 7fb30c405700 1 -- 172.21.15.107:0/2011133828 shutdown_connections 2024-05-17T21:42:53.644 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:48.053+0000 7fb30c405700 1 --2- 172.21.15.107:0/2011133828 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb304102810 0x7fb304102c20 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:42:53.644 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:48.053+0000 7fb30c405700 1 -- 172.21.15.107:0/2011133828 >> 172.21.15.107:0/2011133828 conn(0x7fb3040fdfd0 msgr2=0x7fb3041003e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:42:53.644 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:48.053+0000 7fb30c405700 1 -- 172.21.15.107:0/2011133828 shutdown_connections 2024-05-17T21:42:53.644 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:48.053+0000 7fb30c405700 1 -- 172.21.15.107:0/2011133828 wait complete. 2024-05-17T21:42:53.644 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:48.054+0000 7fb30c405700 1 Processor -- start 2024-05-17T21:42:53.644 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:48.054+0000 7fb30c405700 1 -- start start 2024-05-17T21:42:53.644 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:48.054+0000 7fb30c405700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb304102810 0x7fb3041116a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:42:53.644 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:48.054+0000 7fb30c405700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb304111be0 con 0x7fb304102810 2024-05-17T21:42:53.644 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:48.054+0000 7fb30a1a1700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb304102810 0x7fb3041116a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:42:53.644 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:48.054+0000 7fb30a1a1700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb304102810 0x7fb3041116a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:58324/0 (socket says 172.21.15.107:58324) 2024-05-17T21:42:53.644 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:48.054+0000 7fb30a1a1700 1 -- 172.21.15.107:0/3756358440 learned_addr learned my addr 172.21.15.107:0/3756358440 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:42:53.644 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:48.054+0000 7fb30a1a1700 1 -- 172.21.15.107:0/3756358440 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3000094f0 con 0x7fb304102810 2024-05-17T21:42:53.644 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:48.054+0000 7fb30a1a1700 1 --2- 172.21.15.107:0/3756358440 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb304102810 0x7fb3041116a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fb30000ba90 tx=0x7fb300009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:42:53.645 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:48.055+0000 7fb2faffd700 1 -- 172.21.15.107:0/3756358440 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb3000107c0 con 0x7fb304102810 2024-05-17T21:42:53.645 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:48.055+0000 7fb30c405700 1 -- 172.21.15.107:0/3756358440 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb304111de0 con 0x7fb304102810 2024-05-17T21:42:53.645 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:48.055+0000 7fb2faffd700 1 -- 172.21.15.107:0/3756358440 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fb300010db0 con 0x7fb304102810 2024-05-17T21:42:53.645 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:48.055+0000 7fb2faffd700 1 -- 172.21.15.107:0/3756358440 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb30001add0 con 0x7fb304102810 2024-05-17T21:42:53.645 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:48.055+0000 7fb30c405700 1 -- 172.21.15.107:0/3756358440 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb304112950 con 0x7fb304102810 2024-05-17T21:42:53.645 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:48.055+0000 7fb2faffd700 1 -- 172.21.15.107:0/3756358440 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 5) v1 ==== 43376+0+0 (secure 0 0 0) 0x7fb30002e440 con 0x7fb304102810 2024-05-17T21:42:53.645 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:48.056+0000 7fb2faffd700 1 --2- 172.21.15.107:0/3756358440 >> [v2:172.21.15.107:6800/1677232931,v1:172.21.15.107:6801/1677232931] conn(0x7fb2f00368a0 0x7fb2f0038d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:42:53.645 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:48.056+0000 7fb30c405700 1 -- 172.21.15.107:0/3756358440 --> [v2:172.21.15.107:6800/1677232931,v1:172.21.15.107:6801/1677232931] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb30405f5c0 con 0x7fb2f00368a0 2024-05-17T21:42:53.645 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:48.056+0000 7fb2faffd700 1 -- 172.21.15.107:0/3756358440 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fb30004a560 con 0x7fb304102810 2024-05-17T21:42:53.645 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:48.056+0000 7fb3099a0700 1 -- 172.21.15.107:0/3756358440 >> [v2:172.21.15.107:6800/1677232931,v1:172.21.15.107:6801/1677232931] conn(0x7fb2f00368a0 msgr2=0x7fb2f0038d50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/1677232931 2024-05-17T21:42:53.645 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:48.056+0000 7fb3099a0700 1 --2- 172.21.15.107:0/3756358440 >> [v2:172.21.15.107:6800/1677232931,v1:172.21.15.107:6801/1677232931] conn(0x7fb2f00368a0 0x7fb2f0038d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-17T21:42:53.645 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:48.257+0000 7fb3099a0700 1 -- 172.21.15.107:0/3756358440 >> [v2:172.21.15.107:6800/1677232931,v1:172.21.15.107:6801/1677232931] conn(0x7fb2f00368a0 msgr2=0x7fb2f0038d50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/1677232931 2024-05-17T21:42:53.645 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:48.257+0000 7fb3099a0700 1 --2- 172.21.15.107:0/3756358440 >> [v2:172.21.15.107:6800/1677232931,v1:172.21.15.107:6801/1677232931] conn(0x7fb2f00368a0 0x7fb2f0038d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-17T21:42:53.646 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:48.658+0000 7fb3099a0700 1 -- 172.21.15.107:0/3756358440 >> [v2:172.21.15.107:6800/1677232931,v1:172.21.15.107:6801/1677232931] conn(0x7fb2f00368a0 msgr2=0x7fb2f0038d50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/1677232931 2024-05-17T21:42:53.646 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:48.658+0000 7fb3099a0700 1 --2- 172.21.15.107:0/3756358440 >> [v2:172.21.15.107:6800/1677232931,v1:172.21.15.107:6801/1677232931] conn(0x7fb2f00368a0 0x7fb2f0038d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-17T21:42:53.646 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:49.459+0000 7fb3099a0700 1 -- 172.21.15.107:0/3756358440 >> [v2:172.21.15.107:6800/1677232931,v1:172.21.15.107:6801/1677232931] conn(0x7fb2f00368a0 msgr2=0x7fb2f0038d50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/1677232931 2024-05-17T21:42:53.646 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:49.459+0000 7fb3099a0700 1 --2- 172.21.15.107:0/3756358440 >> [v2:172.21.15.107:6800/1677232931,v1:172.21.15.107:6801/1677232931] conn(0x7fb2f00368a0 0x7fb2f0038d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-17T21:42:53.646 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:51.060+0000 7fb3099a0700 1 -- 172.21.15.107:0/3756358440 >> [v2:172.21.15.107:6800/1677232931,v1:172.21.15.107:6801/1677232931] conn(0x7fb2f00368a0 msgr2=0x7fb2f0038d50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/1677232931 2024-05-17T21:42:53.646 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:51.060+0000 7fb3099a0700 1 --2- 172.21.15.107:0/3756358440 >> [v2:172.21.15.107:6800/1677232931,v1:172.21.15.107:6801/1677232931] conn(0x7fb2f00368a0 0x7fb2f0038d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-17T21:42:53.646 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:52.101+0000 7fb2faffd700 1 -- 172.21.15.107:0/3756358440 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mgrmap(e 6) v1 ==== 43143+0+0 (secure 0 0 0) 0x7fb30002e740 con 0x7fb304102810 2024-05-17T21:42:53.646 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:52.101+0000 7fb2faffd700 1 -- 172.21.15.107:0/3756358440 >> [v2:172.21.15.107:6800/1677232931,v1:172.21.15.107:6801/1677232931] conn(0x7fb2f00368a0 msgr2=0x7fb2f0038d50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-17T21:42:53.646 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:52.101+0000 7fb2faffd700 1 --2- 172.21.15.107:0/3756358440 >> [v2:172.21.15.107:6800/1677232931,v1:172.21.15.107:6801/1677232931] conn(0x7fb2f00368a0 0x7fb2f0038d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:42:53.646 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:53.102+0000 7fb2faffd700 1 -- 172.21.15.107:0/3756358440 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mgrmap(e 7) v1 ==== 43270+0+0 (secure 0 0 0) 0x7fb30004bde0 con 0x7fb304102810 2024-05-17T21:42:53.646 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:53.102+0000 7fb2faffd700 1 --2- 172.21.15.107:0/3756358440 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7fb2f0039fc0 0x7fb2f003c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:42:53.646 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:53.102+0000 7fb2faffd700 1 -- 172.21.15.107:0/3756358440 --> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb30405f5c0 con 0x7fb2f0039fc0 2024-05-17T21:42:53.646 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:53.102+0000 7fb3099a0700 1 --2- 172.21.15.107:0/3756358440 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7fb2f0039fc0 0x7fb2f003c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:42:53.646 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:53.103+0000 7fb3099a0700 1 --2- 172.21.15.107:0/3756358440 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7fb2f0039fc0 0x7fb2f003c3a0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fb2f40039d0 tx=0x7fb2f4009350 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:42:53.647 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:53.104+0000 7fb2faffd700 1 -- 172.21.15.107:0/3756358440 <== mgr.14120 v2:172.21.15.107:6800/1718632476 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6549 (secure 0 0 0) 0x7fb30405f5c0 con 0x7fb2f0039fc0 2024-05-17T21:42:53.647 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:53.109+0000 7fb30c405700 1 -- 172.21.15.107:0/3756358440 --> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fb304112270 con 0x7fb2f0039fc0 2024-05-17T21:42:53.647 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:53.110+0000 7fb2faffd700 1 -- 172.21.15.107:0/3756358440 <== mgr.14120 v2:172.21.15.107:6800/1718632476 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fb304112270 con 0x7fb2f0039fc0 2024-05-17T21:42:53.647 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:53.110+0000 7fb30c405700 1 -- 172.21.15.107:0/3756358440 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7fb2f0039fc0 msgr2=0x7fb2f003c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:42:53.647 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:53.110+0000 7fb30c405700 1 --2- 172.21.15.107:0/3756358440 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7fb2f0039fc0 0x7fb2f003c3a0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fb2f40039d0 tx=0x7fb2f4009350 comp rx=0 tx=0).stop 2024-05-17T21:42:53.647 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:53.110+0000 7fb30c405700 1 -- 172.21.15.107:0/3756358440 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb304102810 msgr2=0x7fb3041116a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:42:53.647 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:53.110+0000 7fb30c405700 1 --2- 172.21.15.107:0/3756358440 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb304102810 0x7fb3041116a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fb30000ba90 tx=0x7fb300009250 comp rx=0 tx=0).stop 2024-05-17T21:42:53.647 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:53.110+0000 7fb30c405700 1 -- 172.21.15.107:0/3756358440 shutdown_connections 2024-05-17T21:42:53.647 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:53.110+0000 7fb30c405700 1 --2- 172.21.15.107:0/3756358440 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7fb2f0039fc0 0x7fb2f003c3a0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:42:53.647 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:53.110+0000 7fb30c405700 1 --2- 172.21.15.107:0/3756358440 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb304102810 0x7fb3041116a0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:42:53.647 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:53.110+0000 7fb30c405700 1 --2- 172.21.15.107:0/3756358440 >> [v2:172.21.15.107:6800/1677232931,v1:172.21.15.107:6801/1677232931] conn(0x7fb2f00368a0 0x7fb2f0038d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:42:53.647 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:53.110+0000 7fb30c405700 1 -- 172.21.15.107:0/3756358440 >> 172.21.15.107:0/3756358440 conn(0x7fb3040fdfd0 msgr2=0x7fb3040feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:42:53.647 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:53.110+0000 7fb30c405700 1 -- 172.21.15.107:0/3756358440 shutdown_connections 2024-05-17T21:42:53.648 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:53.110+0000 7fb30c405700 1 -- 172.21.15.107:0/3756358440 wait complete. 2024-05-17T21:42:53.648 INFO:teuthology.orchestra.run.smithi107.stdout:mgr epoch 5 is available 2024-05-17T21:42:53.648 INFO:teuthology.orchestra.run.smithi107.stdout:Setting orchestrator backend to cephadm... 2024-05-17T21:42:54.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:54 smithi107 ceph-mon[28434]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-17T21:42:54.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:54 smithi107 ceph-mon[28434]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-17T21:42:54.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:54 smithi107 ceph-mon[28434]: [17/May/2024:21:42:53] ENGINE Bus STARTING 2024-05-17T21:42:54.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:54 smithi107 ceph-mon[28434]: [17/May/2024:21:42:53] ENGINE Serving on https://172.21.15.107:7150 2024-05-17T21:42:54.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:54 smithi107 ceph-mon[28434]: [17/May/2024:21:42:53] ENGINE Bus STARTED 2024-05-17T21:42:54.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:54 smithi107 ceph-mon[28434]: from='mgr.14120 172.21.15.107:0/2711624202' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:42:54.605 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:54 smithi107 ceph-mon[28434]: from='mgr.14120 172.21.15.107:0/2711624202' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:42:54.605 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:54 smithi107 ceph-mon[28434]: from='mgr.14120 172.21.15.107:0/2711624202' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:42:54.605 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:54 smithi107 ceph-mon[28434]: from='mgr.14120 172.21.15.107:0/2711624202' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:42:54.812 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:54.013+0000 7fd4d73e8700 1 Processor -- start 2024-05-17T21:42:54.813 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:54.013+0000 7fd4d73e8700 1 -- start start 2024-05-17T21:42:54.813 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:54.013+0000 7fd4d73e8700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd4d0102880 0x7fd4d0102c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:42:54.813 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:54.013+0000 7fd4d73e8700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d01031d0 con 0x7fd4d0102880 2024-05-17T21:42:54.813 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:54.013+0000 7fd4d5184700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd4d0102880 0x7fd4d0102c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:42:54.813 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:54.014+0000 7fd4d5184700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd4d0102880 0x7fd4d0102c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:58394/0 (socket says 172.21.15.107:58394) 2024-05-17T21:42:54.813 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:54.014+0000 7fd4d5184700 1 -- 172.21.15.107:0/622039622 learned_addr learned my addr 172.21.15.107:0/622039622 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:42:54.813 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:54.014+0000 7fd4d5184700 1 -- 172.21.15.107:0/622039622 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4d0103310 con 0x7fd4d0102880 2024-05-17T21:42:54.813 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:54.014+0000 7fd4d5184700 1 --2- 172.21.15.107:0/622039622 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd4d0102880 0x7fd4d0102c90 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc0098e0 tx=0x7fd4cc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3565ba2e7de5fcc server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:42:54.813 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:54.014+0000 7fd4c7fff700 1 -- 172.21.15.107:0/622039622 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd4cc00b860 con 0x7fd4d0102880 2024-05-17T21:42:54.813 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:54.014+0000 7fd4c7fff700 1 -- 172.21.15.107:0/622039622 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fd4cc00be50 con 0x7fd4d0102880 2024-05-17T21:42:54.813 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:54.014+0000 7fd4c7fff700 1 -- 172.21.15.107:0/622039622 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd4cc010380 con 0x7fd4d0102880 2024-05-17T21:42:54.814 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:54.015+0000 7fd4d73e8700 1 -- 172.21.15.107:0/622039622 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd4d0102880 msgr2=0x7fd4d0102c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:42:54.814 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:54.015+0000 7fd4d73e8700 1 --2- 172.21.15.107:0/622039622 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd4d0102880 0x7fd4d0102c90 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc0098e0 tx=0x7fd4cc009250 comp rx=0 tx=0).stop 2024-05-17T21:42:54.814 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:54.015+0000 7fd4d73e8700 1 -- 172.21.15.107:0/622039622 shutdown_connections 2024-05-17T21:42:54.814 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:54.015+0000 7fd4d73e8700 1 --2- 172.21.15.107:0/622039622 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd4d0102880 0x7fd4d0102c90 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:42:54.814 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:54.015+0000 7fd4d73e8700 1 -- 172.21.15.107:0/622039622 >> 172.21.15.107:0/622039622 conn(0x7fd4d00fe060 msgr2=0x7fd4d0100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:42:54.814 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:54.015+0000 7fd4d73e8700 1 -- 172.21.15.107:0/622039622 shutdown_connections 2024-05-17T21:42:54.814 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:54.015+0000 7fd4d73e8700 1 -- 172.21.15.107:0/622039622 wait complete. 2024-05-17T21:42:54.814 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:54.015+0000 7fd4d73e8700 1 Processor -- start 2024-05-17T21:42:54.814 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:54.016+0000 7fd4d73e8700 1 -- start start 2024-05-17T21:42:54.814 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:54.016+0000 7fd4d73e8700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd4d0102880 0x7fd4d0078c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:42:54.814 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:54.016+0000 7fd4d73e8700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d0079160 con 0x7fd4d0102880 2024-05-17T21:42:54.814 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:54.016+0000 7fd4d5184700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd4d0102880 0x7fd4d0078c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:42:54.814 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:54.016+0000 7fd4d5184700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd4d0102880 0x7fd4d0078c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:58408/0 (socket says 172.21.15.107:58408) 2024-05-17T21:42:54.815 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:54.016+0000 7fd4d5184700 1 -- 172.21.15.107:0/1000200710 learned_addr learned my addr 172.21.15.107:0/1000200710 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:42:54.815 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:54.016+0000 7fd4d5184700 1 -- 172.21.15.107:0/1000200710 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4cc0094f0 con 0x7fd4d0102880 2024-05-17T21:42:54.815 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:54.016+0000 7fd4d5184700 1 --2- 172.21.15.107:0/1000200710 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd4d0102880 0x7fd4d0078c20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc00b440 tx=0x7fd4cc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:42:54.815 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:54.016+0000 7fd4c5ffb700 1 -- 172.21.15.107:0/1000200710 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd4cc010760 con 0x7fd4d0102880 2024-05-17T21:42:54.815 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:54.016+0000 7fd4d73e8700 1 -- 172.21.15.107:0/1000200710 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4d00771d0 con 0x7fd4d0102880 2024-05-17T21:42:54.815 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:54.016+0000 7fd4c5ffb700 1 -- 172.21.15.107:0/1000200710 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fd4cc010d50 con 0x7fd4d0102880 2024-05-17T21:42:54.815 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:54.016+0000 7fd4c5ffb700 1 -- 172.21.15.107:0/1000200710 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd4cc01ae20 con 0x7fd4d0102880 2024-05-17T21:42:54.815 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:54.017+0000 7fd4d73e8700 1 -- 172.21.15.107:0/1000200710 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4d00776c0 con 0x7fd4d0102880 2024-05-17T21:42:54.815 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:54.017+0000 7fd4c5ffb700 1 -- 172.21.15.107:0/1000200710 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 7) v1 ==== 43270+0+0 (secure 0 0 0) 0x7fd4cc02e440 con 0x7fd4d0102880 2024-05-17T21:42:54.815 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:54.017+0000 7fd4d73e8700 1 -- 172.21.15.107:0/1000200710 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4d004bed0 con 0x7fd4d0102880 2024-05-17T21:42:54.815 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:54.018+0000 7fd4c5ffb700 1 --2- 172.21.15.107:0/1000200710 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7fd4bc036720 0x7fd4bc038bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:42:54.815 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:54.018+0000 7fd4c5ffb700 1 -- 172.21.15.107:0/1000200710 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fd4cc04a650 con 0x7fd4d0102880 2024-05-17T21:42:54.815 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:54.018+0000 7fd4d4983700 1 --2- 172.21.15.107:0/1000200710 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7fd4bc036720 0x7fd4bc038bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:42:54.815 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:54.018+0000 7fd4d4983700 1 --2- 172.21.15.107:0/1000200710 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7fd4bc036720 0x7fd4bc038bd0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd4c0009990 tx=0x7fd4c0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:42:54.816 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:54.022+0000 7fd4c5ffb700 1 -- 172.21.15.107:0/1000200710 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fd4cc018940 con 0x7fd4d0102880 2024-05-17T21:42:54.816 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:54.219+0000 7fd4d73e8700 1 -- 172.21.15.107:0/1000200710 --> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7fd4d0077e80 con 0x7fd4bc036720 2024-05-17T21:42:54.816 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:54.228+0000 7fd4c5ffb700 1 -- 172.21.15.107:0/1000200710 <== mgr.14120 v2:172.21.15.107:6800/1718632476 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fd4d0077e80 con 0x7fd4bc036720 2024-05-17T21:42:54.816 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:54.231+0000 7fd4d73e8700 1 -- 172.21.15.107:0/1000200710 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7fd4bc036720 msgr2=0x7fd4bc038bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:42:54.816 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:54.231+0000 7fd4d73e8700 1 --2- 172.21.15.107:0/1000200710 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7fd4bc036720 0x7fd4bc038bd0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd4c0009990 tx=0x7fd4c0006e40 comp rx=0 tx=0).stop 2024-05-17T21:42:54.816 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:54.231+0000 7fd4d73e8700 1 -- 172.21.15.107:0/1000200710 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd4d0102880 msgr2=0x7fd4d0078c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:42:54.816 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:54.231+0000 7fd4d73e8700 1 --2- 172.21.15.107:0/1000200710 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd4d0102880 0x7fd4d0078c20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc00b440 tx=0x7fd4cc009250 comp rx=0 tx=0).stop 2024-05-17T21:42:54.816 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:54.231+0000 7fd4d73e8700 1 -- 172.21.15.107:0/1000200710 shutdown_connections 2024-05-17T21:42:54.816 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:54.231+0000 7fd4d73e8700 1 --2- 172.21.15.107:0/1000200710 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7fd4bc036720 0x7fd4bc038bd0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:42:54.816 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:54.231+0000 7fd4d73e8700 1 --2- 172.21.15.107:0/1000200710 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd4d0102880 0x7fd4d0078c20 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:42:54.816 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:54.231+0000 7fd4d73e8700 1 -- 172.21.15.107:0/1000200710 >> 172.21.15.107:0/1000200710 conn(0x7fd4d00fe060 msgr2=0x7fd4d00feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:42:54.816 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:54.231+0000 7fd4d73e8700 1 -- 172.21.15.107:0/1000200710 shutdown_connections 2024-05-17T21:42:54.816 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:54.231+0000 7fd4d73e8700 1 -- 172.21.15.107:0/1000200710 wait complete. 2024-05-17T21:42:55.433 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:55 smithi107 ceph-mon[28434]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:42:55.433 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:55 smithi107 ceph-mon[28434]: mgrmap e8: smithi107.xhhyiw(active, since 2s) 2024-05-17T21:42:56.027 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout value unchanged 2024-05-17T21:42:56.027 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:55.244+0000 7f792b341700 1 Processor -- start 2024-05-17T21:42:56.027 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:55.244+0000 7f792b341700 1 -- start start 2024-05-17T21:42:56.027 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:55.244+0000 7f792b341700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7924101750 0x7f7924101b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:42:56.027 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:55.244+0000 7f792b341700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79241020a0 con 0x7f7924101750 2024-05-17T21:42:56.027 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:55.244+0000 7f79290dd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7924101750 0x7f7924101b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:42:56.027 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:55.244+0000 7f79290dd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7924101750 0x7f7924101b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:58424/0 (socket says 172.21.15.107:58424) 2024-05-17T21:42:56.027 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:55.244+0000 7f79290dd700 1 -- 172.21.15.107:0/1653699570 learned_addr learned my addr 172.21.15.107:0/1653699570 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:42:56.028 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:55.244+0000 7f79290dd700 1 -- 172.21.15.107:0/1653699570 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79241021e0 con 0x7f7924101750 2024-05-17T21:42:56.028 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:55.245+0000 7f79290dd700 1 --2- 172.21.15.107:0/1653699570 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7924101750 0x7f7924101b60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f79200098e0 tx=0x7f7920009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4119eae514e93f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:42:56.028 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:55.245+0000 7f791bfff700 1 -- 172.21.15.107:0/1653699570 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f792000b860 con 0x7f7924101750 2024-05-17T21:42:56.028 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:55.245+0000 7f791bfff700 1 -- 172.21.15.107:0/1653699570 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f792000be50 con 0x7f7924101750 2024-05-17T21:42:56.028 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:55.245+0000 7f791bfff700 1 -- 172.21.15.107:0/1653699570 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7920010380 con 0x7f7924101750 2024-05-17T21:42:56.028 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:55.245+0000 7f792b341700 1 -- 172.21.15.107:0/1653699570 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7924101750 msgr2=0x7f7924101b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:42:56.028 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:55.245+0000 7f792b341700 1 --2- 172.21.15.107:0/1653699570 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7924101750 0x7f7924101b60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f79200098e0 tx=0x7f7920009250 comp rx=0 tx=0).stop 2024-05-17T21:42:56.028 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:55.245+0000 7f792b341700 1 -- 172.21.15.107:0/1653699570 shutdown_connections 2024-05-17T21:42:56.028 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:55.245+0000 7f792b341700 1 --2- 172.21.15.107:0/1653699570 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7924101750 0x7f7924101b60 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:42:56.028 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:55.245+0000 7f792b341700 1 -- 172.21.15.107:0/1653699570 >> 172.21.15.107:0/1653699570 conn(0x7f79240fcef0 msgr2=0x7f79240ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:42:56.028 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:55.245+0000 7f792b341700 1 -- 172.21.15.107:0/1653699570 shutdown_connections 2024-05-17T21:42:56.029 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:55.245+0000 7f792b341700 1 -- 172.21.15.107:0/1653699570 wait complete. 2024-05-17T21:42:56.029 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:55.246+0000 7f792b341700 1 Processor -- start 2024-05-17T21:42:56.029 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:55.246+0000 7f792b341700 1 -- start start 2024-05-17T21:42:56.029 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:55.246+0000 7f792b341700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7924101750 0x7f79241104d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:42:56.029 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:55.246+0000 7f792b341700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7924110a10 con 0x7f7924101750 2024-05-17T21:42:56.029 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:55.246+0000 7f79290dd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7924101750 0x7f79241104d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:42:56.029 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:55.246+0000 7f79290dd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7924101750 0x7f79241104d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:58426/0 (socket says 172.21.15.107:58426) 2024-05-17T21:42:56.029 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:55.246+0000 7f79290dd700 1 -- 172.21.15.107:0/758185604 learned_addr learned my addr 172.21.15.107:0/758185604 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:42:56.029 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:55.246+0000 7f79290dd700 1 -- 172.21.15.107:0/758185604 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79200094f0 con 0x7f7924101750 2024-05-17T21:42:56.029 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:55.247+0000 7f79290dd700 1 --2- 172.21.15.107:0/758185604 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7924101750 0x7f79241104d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7920000c00 tx=0x7f7920009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:42:56.029 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:55.247+0000 7f7919ffb700 1 -- 172.21.15.107:0/758185604 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f79200107c0 con 0x7f7924101750 2024-05-17T21:42:56.029 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:55.247+0000 7f7919ffb700 1 -- 172.21.15.107:0/758185604 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f7920010db0 con 0x7f7924101750 2024-05-17T21:42:56.029 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:55.247+0000 7f792b341700 1 -- 172.21.15.107:0/758185604 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7924110c10 con 0x7f7924101750 2024-05-17T21:42:56.029 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:55.247+0000 7f7919ffb700 1 -- 172.21.15.107:0/758185604 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f792001ad00 con 0x7f7924101750 2024-05-17T21:42:56.030 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:55.247+0000 7f792b341700 1 -- 172.21.15.107:0/758185604 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7924111780 con 0x7f7924101750 2024-05-17T21:42:56.030 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:55.248+0000 7f7919ffb700 1 -- 172.21.15.107:0/758185604 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f79200193c0 con 0x7f7924101750 2024-05-17T21:42:56.030 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:55.248+0000 7f792b341700 1 -- 172.21.15.107:0/758185604 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f792405f5c0 con 0x7f7924101750 2024-05-17T21:42:56.030 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:55.248+0000 7f7919ffb700 1 --2- 172.21.15.107:0/758185604 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7f79100368a0 0x7f7910038d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:42:56.030 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:55.248+0000 7f7919ffb700 1 -- 172.21.15.107:0/758185604 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f792004a600 con 0x7f7924101750 2024-05-17T21:42:56.030 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:55.248+0000 7f79288dc700 1 --2- 172.21.15.107:0/758185604 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7f79100368a0 0x7f7910038d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:42:56.030 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:55.249+0000 7f79288dc700 1 --2- 172.21.15.107:0/758185604 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7f79100368a0 0x7f7910038d50 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7914009990 tx=0x7f7914006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:42:56.030 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:55.253+0000 7f7919ffb700 1 -- 172.21.15.107:0/758185604 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f7920022930 con 0x7f7924101750 2024-05-17T21:42:56.030 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:55.439+0000 7f792b341700 1 -- 172.21.15.107:0/758185604 --> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f79240fe090 con 0x7f79100368a0 2024-05-17T21:42:56.030 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:55.440+0000 7f7919ffb700 1 -- 172.21.15.107:0/758185604 <== mgr.14120 v2:172.21.15.107:6800/1718632476 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f79240fe090 con 0x7f79100368a0 2024-05-17T21:42:56.030 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:55.442+0000 7f792b341700 1 -- 172.21.15.107:0/758185604 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7f79100368a0 msgr2=0x7f7910038d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:42:56.030 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:55.442+0000 7f792b341700 1 --2- 172.21.15.107:0/758185604 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7f79100368a0 0x7f7910038d50 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7914009990 tx=0x7f7914006e40 comp rx=0 tx=0).stop 2024-05-17T21:42:56.030 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:55.442+0000 7f792b341700 1 -- 172.21.15.107:0/758185604 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7924101750 msgr2=0x7f79241104d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:42:56.031 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:55.442+0000 7f792b341700 1 --2- 172.21.15.107:0/758185604 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7924101750 0x7f79241104d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7920000c00 tx=0x7f7920009250 comp rx=0 tx=0).stop 2024-05-17T21:42:56.031 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:55.442+0000 7f792b341700 1 -- 172.21.15.107:0/758185604 shutdown_connections 2024-05-17T21:42:56.031 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:55.442+0000 7f792b341700 1 --2- 172.21.15.107:0/758185604 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7f79100368a0 0x7f7910038d50 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:42:56.031 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:55.442+0000 7f792b341700 1 --2- 172.21.15.107:0/758185604 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7924101750 0x7f79241104d0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:42:56.031 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:55.442+0000 7f792b341700 1 -- 172.21.15.107:0/758185604 >> 172.21.15.107:0/758185604 conn(0x7f79240fcef0 msgr2=0x7f79240fd980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:42:56.031 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:55.442+0000 7f792b341700 1 -- 172.21.15.107:0/758185604 shutdown_connections 2024-05-17T21:42:56.031 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:55.442+0000 7f792b341700 1 -- 172.21.15.107:0/758185604 wait complete. 2024-05-17T21:42:56.031 INFO:teuthology.orchestra.run.smithi107.stdout:Generating ssh key... 2024-05-17T21:42:56.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:56 smithi107 ceph-mon[28434]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:42:57.417 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:57 smithi107 ceph-mon[28434]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:42:57.418 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:57 smithi107 ceph-mon[28434]: Generating ssh key... 2024-05-17T21:42:57.418 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:57 smithi107 ceph-mon[28434]: from='mgr.14120 172.21.15.107:0/2711624202' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:42:57.418 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:57 smithi107 ceph-mon[28434]: from='mgr.14120 172.21.15.107:0/2711624202' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:42:57.425 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:56.427+0000 7fd911a1a700 1 Processor -- start 2024-05-17T21:42:57.425 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:56.427+0000 7fd911a1a700 1 -- start start 2024-05-17T21:42:57.425 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:56.427+0000 7fd911a1a700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd90c101740 0x7fd90c101b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:42:57.425 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:56.427+0000 7fd911a1a700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd90c102090 con 0x7fd90c101740 2024-05-17T21:42:57.425 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:56.427+0000 7fd90affd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd90c101740 0x7fd90c101b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:42:57.425 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:56.428+0000 7fd90affd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd90c101740 0x7fd90c101b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54898/0 (socket says 172.21.15.107:54898) 2024-05-17T21:42:57.425 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:56.428+0000 7fd90affd700 1 -- 172.21.15.107:0/3578438620 learned_addr learned my addr 172.21.15.107:0/3578438620 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:42:57.425 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:56.428+0000 7fd90affd700 1 -- 172.21.15.107:0/3578438620 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd90c1021d0 con 0x7fd90c101740 2024-05-17T21:42:57.425 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:56.428+0000 7fd90affd700 1 --2- 172.21.15.107:0/3578438620 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd90c101740 0x7fd90c101b50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc00d0d0 tx=0x7fd8fc00ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6541c98eef2a2072 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:42:57.425 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:56.428+0000 7fd90a7fc700 1 -- 172.21.15.107:0/3578438620 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd8fc00ddf0 con 0x7fd90c101740 2024-05-17T21:42:57.426 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:56.428+0000 7fd90a7fc700 1 -- 172.21.15.107:0/3578438620 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fd8fc009610 con 0x7fd90c101740 2024-05-17T21:42:57.426 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:56.428+0000 7fd90a7fc700 1 -- 172.21.15.107:0/3578438620 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd8fc0145d0 con 0x7fd90c101740 2024-05-17T21:42:57.426 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:56.429+0000 7fd911a1a700 1 -- 172.21.15.107:0/3578438620 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd90c101740 msgr2=0x7fd90c101b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:42:57.426 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:56.429+0000 7fd911a1a700 1 --2- 172.21.15.107:0/3578438620 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd90c101740 0x7fd90c101b50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc00d0d0 tx=0x7fd8fc00ba50 comp rx=0 tx=0).stop 2024-05-17T21:42:57.426 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:56.429+0000 7fd911a1a700 1 -- 172.21.15.107:0/3578438620 shutdown_connections 2024-05-17T21:42:57.426 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:56.429+0000 7fd911a1a700 1 --2- 172.21.15.107:0/3578438620 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd90c101740 0x7fd90c101b50 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:42:57.426 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:56.429+0000 7fd911a1a700 1 -- 172.21.15.107:0/3578438620 >> 172.21.15.107:0/3578438620 conn(0x7fd90c0fcec0 msgr2=0x7fd90c0ff2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:42:57.426 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:56.429+0000 7fd911a1a700 1 -- 172.21.15.107:0/3578438620 shutdown_connections 2024-05-17T21:42:57.426 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:56.429+0000 7fd911a1a700 1 -- 172.21.15.107:0/3578438620 wait complete. 2024-05-17T21:42:57.426 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:56.429+0000 7fd911a1a700 1 Processor -- start 2024-05-17T21:42:57.426 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:56.429+0000 7fd911a1a700 1 -- start start 2024-05-17T21:42:57.426 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:56.430+0000 7fd911a1a700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd90c101740 0x7fd90c10f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:42:57.426 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:56.430+0000 7fd911a1a700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd90c10f8d0 con 0x7fd90c101740 2024-05-17T21:42:57.426 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:56.430+0000 7fd90affd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd90c101740 0x7fd90c10f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:42:57.427 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:56.430+0000 7fd90affd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd90c101740 0x7fd90c10f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54906/0 (socket says 172.21.15.107:54906) 2024-05-17T21:42:57.427 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:56.430+0000 7fd90affd700 1 -- 172.21.15.107:0/1792948349 learned_addr learned my addr 172.21.15.107:0/1792948349 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:42:57.427 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:56.430+0000 7fd90affd700 1 -- 172.21.15.107:0/1792948349 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8fc00bcf0 con 0x7fd90c101740 2024-05-17T21:42:57.427 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:56.430+0000 7fd90affd700 1 --2- 172.21.15.107:0/1792948349 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd90c101740 0x7fd90c10f390 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc00bcc0 tx=0x7fd8fc00ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:42:57.427 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:56.430+0000 7fd903fff700 1 -- 172.21.15.107:0/1792948349 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd8fc009a20 con 0x7fd90c101740 2024-05-17T21:42:57.427 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:56.430+0000 7fd911a1a700 1 -- 172.21.15.107:0/1792948349 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd90c10fad0 con 0x7fd90c101740 2024-05-17T21:42:57.427 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:56.430+0000 7fd903fff700 1 -- 172.21.15.107:0/1792948349 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fd8fc00b2e0 con 0x7fd90c101740 2024-05-17T21:42:57.427 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:56.430+0000 7fd903fff700 1 -- 172.21.15.107:0/1792948349 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd8fc01ebb0 con 0x7fd90c101740 2024-05-17T21:42:57.427 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:56.430+0000 7fd911a1a700 1 -- 172.21.15.107:0/1792948349 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd90c110640 con 0x7fd90c101740 2024-05-17T21:42:57.427 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:56.431+0000 7fd903fff700 1 -- 172.21.15.107:0/1792948349 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7fd8fc01ed10 con 0x7fd90c101740 2024-05-17T21:42:57.427 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:56.431+0000 7fd911a1a700 1 -- 172.21.15.107:0/1792948349 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd90c05f5c0 con 0x7fd90c101740 2024-05-17T21:42:57.427 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:56.431+0000 7fd903fff700 1 --2- 172.21.15.107:0/1792948349 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7fd8f80368a0 0x7fd8f8038d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:42:57.427 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:56.431+0000 7fd903fff700 1 -- 172.21.15.107:0/1792948349 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fd8fc04c230 con 0x7fd90c101740 2024-05-17T21:42:57.428 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:56.431+0000 7fd9027fc700 1 --2- 172.21.15.107:0/1792948349 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7fd8f80368a0 0x7fd8f8038d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:42:57.428 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:56.432+0000 7fd9027fc700 1 --2- 172.21.15.107:0/1792948349 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7fd8f80368a0 0x7fd8f8038d50 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd8f4009990 tx=0x7fd8f4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:42:57.428 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:56.436+0000 7fd903fff700 1 -- 172.21.15.107:0/1792948349 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fd8fc013dd0 con 0x7fd90c101740 2024-05-17T21:42:57.428 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:56.622+0000 7fd911a1a700 1 -- 172.21.15.107:0/1792948349 --> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fd90c110a50 con 0x7fd8f80368a0 2024-05-17T21:42:57.428 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:56.808+0000 7fd903fff700 1 -- 172.21.15.107:0/1792948349 <== mgr.14120 v2:172.21.15.107:6800/1718632476 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fd90c110a50 con 0x7fd8f80368a0 2024-05-17T21:42:57.428 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:56.810+0000 7fd911a1a700 1 -- 172.21.15.107:0/1792948349 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7fd8f80368a0 msgr2=0x7fd8f8038d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:42:57.428 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:56.810+0000 7fd911a1a700 1 --2- 172.21.15.107:0/1792948349 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7fd8f80368a0 0x7fd8f8038d50 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd8f4009990 tx=0x7fd8f4006e40 comp rx=0 tx=0).stop 2024-05-17T21:42:57.428 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:56.810+0000 7fd911a1a700 1 -- 172.21.15.107:0/1792948349 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd90c101740 msgr2=0x7fd90c10f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:42:57.428 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:56.810+0000 7fd911a1a700 1 --2- 172.21.15.107:0/1792948349 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd90c101740 0x7fd90c10f390 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc00bcc0 tx=0x7fd8fc00ba50 comp rx=0 tx=0).stop 2024-05-17T21:42:57.428 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:56.810+0000 7fd911a1a700 1 -- 172.21.15.107:0/1792948349 shutdown_connections 2024-05-17T21:42:57.428 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:56.810+0000 7fd911a1a700 1 --2- 172.21.15.107:0/1792948349 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7fd8f80368a0 0x7fd8f8038d50 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:42:57.428 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:56.810+0000 7fd911a1a700 1 --2- 172.21.15.107:0/1792948349 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd90c101740 0x7fd90c10f390 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:42:57.428 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:56.810+0000 7fd911a1a700 1 -- 172.21.15.107:0/1792948349 >> 172.21.15.107:0/1792948349 conn(0x7fd90c0fcec0 msgr2=0x7fd90c105b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:42:57.428 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:56.811+0000 7fd911a1a700 1 -- 172.21.15.107:0/1792948349 shutdown_connections 2024-05-17T21:42:57.428 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:56.811+0000 7fd911a1a700 1 -- 172.21.15.107:0/1792948349 wait complete. 2024-05-17T21:42:58.573 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDC82J4v0yngLUWHqgMnkz1BtuSxjzef52E8Eig8IVeZp7HkNJWFBdaHG++rWmW7COJuA4L7cjFG/t0n2Cv0ZDRoAWT1PN0+5I17aLtivFr77qLoi0rOxfNhp4gFrFF+9OosLjlTNKZjk1kkojWJC8yCfWY5jcl6LY+5y1YnAYfuquqK8cjLrzCjg9vvoihv82tBe+p20KD0HlyLzARqILlktDOUOHnXeCmgRVPuw5T+PZ/Kiytb28gHHsxGiGhttgs0D4ImCNO7+MOLBALzECk7OjmPPaVYsbv3rA+P2n9vbGMCLuVhpWCKEfvLwr4WUueKnBgApmXWW6nu0bP2XqNamn79HdL71tAH3rb0n8A2CUwWRqCGhkRWYH5iKEd0RyqSbLDv1K2wGuCD+SYLwCN1I8K1IGwnhhFR/9W4lXf3hJnhUOgLw6XLQTlo826uKq+Fs9/OUjVHossT1BFiZIkug5LDvO354kaIum6uo144Ub6kqG7giLTFssS+H3vy9U= ceph-3722731a-1496-11ef-bc9a-c7b262605968 2024-05-17T21:42:58.574 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:57.782+0000 7fcaa55b7700 1 Processor -- start 2024-05-17T21:42:58.574 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:57.782+0000 7fcaa55b7700 1 -- start start 2024-05-17T21:42:58.574 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:57.783+0000 7fcaa55b7700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcaa0100c70 0x7fcaa0101080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:42:58.574 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:57.783+0000 7fcaa55b7700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa01015c0 con 0x7fcaa0100c70 2024-05-17T21:42:58.574 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:57.783+0000 7fca9effd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcaa0100c70 0x7fcaa0101080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:42:58.574 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:57.783+0000 7fca9effd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcaa0100c70 0x7fcaa0101080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54920/0 (socket says 172.21.15.107:54920) 2024-05-17T21:42:58.574 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:57.783+0000 7fca9effd700 1 -- 172.21.15.107:0/3801114082 learned_addr learned my addr 172.21.15.107:0/3801114082 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:42:58.574 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:57.783+0000 7fca9effd700 1 -- 172.21.15.107:0/3801114082 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaa0101700 con 0x7fcaa0100c70 2024-05-17T21:42:58.574 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:57.783+0000 7fca9effd700 1 --2- 172.21.15.107:0/3801114082 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcaa0100c70 0x7fcaa0101080 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fca880098e0 tx=0x7fca88009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=871843467772dbff server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:42:58.574 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:57.783+0000 7fca9dffb700 1 -- 172.21.15.107:0/3801114082 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fca8800b860 con 0x7fcaa0100c70 2024-05-17T21:42:58.574 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:57.783+0000 7fca9dffb700 1 -- 172.21.15.107:0/3801114082 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fca8800be50 con 0x7fcaa0100c70 2024-05-17T21:42:58.575 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:57.783+0000 7fca9dffb700 1 -- 172.21.15.107:0/3801114082 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fca88010380 con 0x7fcaa0100c70 2024-05-17T21:42:58.575 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:57.784+0000 7fcaa55b7700 1 -- 172.21.15.107:0/3801114082 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcaa0100c70 msgr2=0x7fcaa0101080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:42:58.575 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:57.784+0000 7fcaa55b7700 1 --2- 172.21.15.107:0/3801114082 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcaa0100c70 0x7fcaa0101080 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fca880098e0 tx=0x7fca88009250 comp rx=0 tx=0).stop 2024-05-17T21:42:58.575 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:57.784+0000 7fcaa55b7700 1 -- 172.21.15.107:0/3801114082 shutdown_connections 2024-05-17T21:42:58.575 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:57.784+0000 7fcaa55b7700 1 --2- 172.21.15.107:0/3801114082 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcaa0100c70 0x7fcaa0101080 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:42:58.575 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:57.784+0000 7fcaa55b7700 1 -- 172.21.15.107:0/3801114082 >> 172.21.15.107:0/3801114082 conn(0x7fcaa00fc6f0 msgr2=0x7fcaa00feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:42:58.575 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:57.784+0000 7fcaa55b7700 1 -- 172.21.15.107:0/3801114082 shutdown_connections 2024-05-17T21:42:58.575 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:57.784+0000 7fcaa55b7700 1 -- 172.21.15.107:0/3801114082 wait complete. 2024-05-17T21:42:58.575 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:57.785+0000 7fcaa55b7700 1 Processor -- start 2024-05-17T21:42:58.575 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:57.785+0000 7fcaa55b7700 1 -- start start 2024-05-17T21:42:58.575 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:57.785+0000 7fcaa55b7700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcaa0100c70 0x7fcaa0111570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:42:58.575 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:57.785+0000 7fcaa55b7700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa0111ab0 con 0x7fcaa0100c70 2024-05-17T21:42:58.575 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:57.785+0000 7fca9effd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcaa0100c70 0x7fcaa0111570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:42:58.575 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:57.785+0000 7fca9effd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcaa0100c70 0x7fcaa0111570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54926/0 (socket says 172.21.15.107:54926) 2024-05-17T21:42:58.576 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:57.785+0000 7fca9effd700 1 -- 172.21.15.107:0/2777761002 learned_addr learned my addr 172.21.15.107:0/2777761002 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:42:58.576 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:57.785+0000 7fca9effd700 1 -- 172.21.15.107:0/2777761002 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca880094f0 con 0x7fcaa0100c70 2024-05-17T21:42:58.576 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:57.785+0000 7fca9effd700 1 --2- 172.21.15.107:0/2777761002 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcaa0100c70 0x7fcaa0111570 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fca8800b440 tx=0x7fca88009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:42:58.576 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:57.785+0000 7fca977fe700 1 -- 172.21.15.107:0/2777761002 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fca88010760 con 0x7fcaa0100c70 2024-05-17T21:42:58.576 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:57.786+0000 7fca977fe700 1 -- 172.21.15.107:0/2777761002 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fca88010d50 con 0x7fcaa0100c70 2024-05-17T21:42:58.576 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:57.786+0000 7fcaa55b7700 1 -- 172.21.15.107:0/2777761002 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcaa0111cb0 con 0x7fcaa0100c70 2024-05-17T21:42:58.576 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:57.786+0000 7fca977fe700 1 -- 172.21.15.107:0/2777761002 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fca8801ae20 con 0x7fcaa0100c70 2024-05-17T21:42:58.576 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:57.786+0000 7fcaa55b7700 1 -- 172.21.15.107:0/2777761002 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaa0112820 con 0x7fcaa0100c70 2024-05-17T21:42:58.576 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:57.786+0000 7fca977fe700 1 -- 172.21.15.107:0/2777761002 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7fca8802e440 con 0x7fcaa0100c70 2024-05-17T21:42:58.576 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:57.786+0000 7fcaa55b7700 1 -- 172.21.15.107:0/2777761002 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcaa005f5c0 con 0x7fcaa0100c70 2024-05-17T21:42:58.576 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:57.787+0000 7fca977fe700 1 --2- 172.21.15.107:0/2777761002 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7fca8c0368a0 0x7fca8c038d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:42:58.576 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:57.787+0000 7fca977fe700 1 -- 172.21.15.107:0/2777761002 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fca8804a5a0 con 0x7fcaa0100c70 2024-05-17T21:42:58.576 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:57.787+0000 7fca9e7fc700 1 --2- 172.21.15.107:0/2777761002 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7fca8c0368a0 0x7fca8c038d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:42:58.576 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:57.787+0000 7fca9e7fc700 1 --2- 172.21.15.107:0/2777761002 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7fca8c0368a0 0x7fca8c038d50 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fca90009990 tx=0x7fca90006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:42:58.576 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:57.791+0000 7fca977fe700 1 -- 172.21.15.107:0/2777761002 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fca88018940 con 0x7fcaa0100c70 2024-05-17T21:42:58.577 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:57.977+0000 7fcaa55b7700 1 -- 172.21.15.107:0/2777761002 --> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fcaa00fd8e0 con 0x7fca8c0368a0 2024-05-17T21:42:58.577 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:57.978+0000 7fca977fe700 1 -- 172.21.15.107:0/2777761002 <== mgr.14120 v2:172.21.15.107:6800/1718632476 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7fcaa00fd8e0 con 0x7fca8c0368a0 2024-05-17T21:42:58.577 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:57.980+0000 7fcaa55b7700 1 -- 172.21.15.107:0/2777761002 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7fca8c0368a0 msgr2=0x7fca8c038d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:42:58.577 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:57.980+0000 7fcaa55b7700 1 --2- 172.21.15.107:0/2777761002 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7fca8c0368a0 0x7fca8c038d50 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fca90009990 tx=0x7fca90006e40 comp rx=0 tx=0).stop 2024-05-17T21:42:58.577 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:57.980+0000 7fcaa55b7700 1 -- 172.21.15.107:0/2777761002 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcaa0100c70 msgr2=0x7fcaa0111570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:42:58.577 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:57.980+0000 7fcaa55b7700 1 --2- 172.21.15.107:0/2777761002 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcaa0100c70 0x7fcaa0111570 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fca8800b440 tx=0x7fca88009250 comp rx=0 tx=0).stop 2024-05-17T21:42:58.577 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:57.980+0000 7fcaa55b7700 1 -- 172.21.15.107:0/2777761002 shutdown_connections 2024-05-17T21:42:58.577 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:57.980+0000 7fcaa55b7700 1 --2- 172.21.15.107:0/2777761002 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7fca8c0368a0 0x7fca8c038d50 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:42:58.577 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:57.980+0000 7fcaa55b7700 1 --2- 172.21.15.107:0/2777761002 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcaa0100c70 0x7fcaa0111570 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:42:58.577 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:57.980+0000 7fcaa55b7700 1 -- 172.21.15.107:0/2777761002 >> 172.21.15.107:0/2777761002 conn(0x7fcaa00fc6f0 msgr2=0x7fcaa00fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:42:58.577 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:57.980+0000 7fcaa55b7700 1 -- 172.21.15.107:0/2777761002 shutdown_connections 2024-05-17T21:42:58.577 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:57.980+0000 7fcaa55b7700 1 -- 172.21.15.107:0/2777761002 wait complete. 2024-05-17T21:42:58.577 INFO:teuthology.orchestra.run.smithi107.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-05-17T21:42:58.577 INFO:teuthology.orchestra.run.smithi107.stdout:Adding key to root@localhost authorized_keys... 2024-05-17T21:42:58.578 INFO:teuthology.orchestra.run.smithi107.stdout:Adding host smithi107... 2024-05-17T21:42:58.822 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:58 smithi107 ceph-mon[28434]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:42:59.847 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:42:59 smithi107 ceph-mon[28434]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi107", "addr": "172.21.15.107", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:43:00.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:00 smithi107 ceph-mon[28434]: Deploying cephadm binary to smithi107 2024-05-17T21:43:01.714 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout Added host 'smithi107' with addr '172.21.15.107' 2024-05-17T21:43:01.714 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:59.026+0000 7fad022f2700 1 Processor -- start 2024-05-17T21:43:01.715 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:59.026+0000 7fad022f2700 1 -- start start 2024-05-17T21:43:01.715 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:59.026+0000 7fad022f2700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7facfc102880 0x7facfc102c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:43:01.715 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:59.026+0000 7fad022f2700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facfc1031d0 con 0x7facfc102880 2024-05-17T21:43:01.715 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:59.027+0000 7facfbfff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7facfc102880 0x7facfc102c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:43:01.715 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:59.027+0000 7facfbfff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7facfc102880 0x7facfc102c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54932/0 (socket says 172.21.15.107:54932) 2024-05-17T21:43:01.715 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:59.027+0000 7facfbfff700 1 -- 172.21.15.107:0/3706477781 learned_addr learned my addr 172.21.15.107:0/3706477781 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:43:01.715 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:59.027+0000 7facfbfff700 1 -- 172.21.15.107:0/3706477781 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facfc103310 con 0x7facfc102880 2024-05-17T21:43:01.715 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:59.027+0000 7facfbfff700 1 --2- 172.21.15.107:0/3706477781 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7facfc102880 0x7facfc102c90 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7facec0098e0 tx=0x7facec009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a4cb38e3a0fea73 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:43:01.715 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:59.027+0000 7facfaffd700 1 -- 172.21.15.107:0/3706477781 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7facec00b860 con 0x7facfc102880 2024-05-17T21:43:01.715 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:59.027+0000 7facfaffd700 1 -- 172.21.15.107:0/3706477781 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7facec00be50 con 0x7facfc102880 2024-05-17T21:43:01.715 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:59.027+0000 7facfaffd700 1 -- 172.21.15.107:0/3706477781 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7facec010380 con 0x7facfc102880 2024-05-17T21:43:01.715 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:59.028+0000 7fad022f2700 1 -- 172.21.15.107:0/3706477781 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7facfc102880 msgr2=0x7facfc102c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:43:01.715 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:59.028+0000 7fad022f2700 1 --2- 172.21.15.107:0/3706477781 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7facfc102880 0x7facfc102c90 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7facec0098e0 tx=0x7facec009250 comp rx=0 tx=0).stop 2024-05-17T21:43:01.715 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:59.028+0000 7fad022f2700 1 -- 172.21.15.107:0/3706477781 shutdown_connections 2024-05-17T21:43:01.716 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:59.028+0000 7fad022f2700 1 --2- 172.21.15.107:0/3706477781 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7facfc102880 0x7facfc102c90 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:43:01.716 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:59.028+0000 7fad022f2700 1 -- 172.21.15.107:0/3706477781 >> 172.21.15.107:0/3706477781 conn(0x7facfc0fe060 msgr2=0x7facfc100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:43:01.716 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:59.028+0000 7fad022f2700 1 -- 172.21.15.107:0/3706477781 shutdown_connections 2024-05-17T21:43:01.716 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:59.028+0000 7fad022f2700 1 -- 172.21.15.107:0/3706477781 wait complete. 2024-05-17T21:43:01.716 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:59.029+0000 7fad022f2700 1 Processor -- start 2024-05-17T21:43:01.716 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:59.029+0000 7fad022f2700 1 -- start start 2024-05-17T21:43:01.716 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:59.029+0000 7fad022f2700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7facfc102880 0x7facfc1115f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:43:01.716 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:59.029+0000 7fad022f2700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facfc111b30 con 0x7facfc102880 2024-05-17T21:43:01.716 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:59.029+0000 7facfbfff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7facfc102880 0x7facfc1115f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:43:01.716 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:59.029+0000 7facfbfff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7facfc102880 0x7facfc1115f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54940/0 (socket says 172.21.15.107:54940) 2024-05-17T21:43:01.716 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:59.029+0000 7facfbfff700 1 -- 172.21.15.107:0/1357208700 learned_addr learned my addr 172.21.15.107:0/1357208700 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:43:01.716 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:59.029+0000 7facfbfff700 1 -- 172.21.15.107:0/1357208700 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facec0094f0 con 0x7facfc102880 2024-05-17T21:43:01.716 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:59.029+0000 7facfbfff700 1 --2- 172.21.15.107:0/1357208700 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7facfc102880 0x7facfc1115f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7facec00b440 tx=0x7facec009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:43:01.716 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:59.030+0000 7facf8ff9700 1 -- 172.21.15.107:0/1357208700 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7facec010760 con 0x7facfc102880 2024-05-17T21:43:01.717 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:59.030+0000 7facf8ff9700 1 -- 172.21.15.107:0/1357208700 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7facec010d50 con 0x7facfc102880 2024-05-17T21:43:01.717 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:59.030+0000 7fad022f2700 1 -- 172.21.15.107:0/1357208700 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facfc111d30 con 0x7facfc102880 2024-05-17T21:43:01.717 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:59.030+0000 7facf8ff9700 1 -- 172.21.15.107:0/1357208700 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7facec01ae20 con 0x7facfc102880 2024-05-17T21:43:01.717 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:59.030+0000 7fad022f2700 1 -- 172.21.15.107:0/1357208700 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facfc1128a0 con 0x7facfc102880 2024-05-17T21:43:01.717 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:59.031+0000 7facf8ff9700 1 -- 172.21.15.107:0/1357208700 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7facec02e440 con 0x7facfc102880 2024-05-17T21:43:01.717 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:59.031+0000 7facf8ff9700 1 --2- 172.21.15.107:0/1357208700 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7face8036830 0x7face8038ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:43:01.717 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:59.031+0000 7fad022f2700 1 -- 172.21.15.107:0/1357208700 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facfc05f5c0 con 0x7facfc102880 2024-05-17T21:43:01.717 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:59.031+0000 7facf8ff9700 1 -- 172.21.15.107:0/1357208700 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7facec04a5d0 con 0x7facfc102880 2024-05-17T21:43:01.717 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:59.031+0000 7facfb7fe700 1 --2- 172.21.15.107:0/1357208700 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7face8036830 0x7face8038ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:43:01.717 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:59.031+0000 7facfb7fe700 1 --2- 172.21.15.107:0/1357208700 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7face8036830 0x7face8038ce0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7face4009990 tx=0x7face4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:43:01.717 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:59.036+0000 7facf8ff9700 1 -- 172.21.15.107:0/1357208700 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7facec018940 con 0x7facfc102880 2024-05-17T21:43:01.717 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:42:59.231+0000 7fad022f2700 1 -- 172.21.15.107:0/1357208700 --> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi107", "addr": "172.21.15.107", "target": ["mon-mgr", ""]}) v1 -- 0x7facfc0ff200 con 0x7face8036830 2024-05-17T21:43:01.717 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:01.112+0000 7facf8ff9700 1 -- 172.21.15.107:0/1357208700 <== mgr.14120 v2:172.21.15.107:6800/1718632476 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7facfc0ff200 con 0x7face8036830 2024-05-17T21:43:01.718 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:01.115+0000 7fad022f2700 1 -- 172.21.15.107:0/1357208700 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7face8036830 msgr2=0x7face8038ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:43:01.718 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:01.115+0000 7fad022f2700 1 --2- 172.21.15.107:0/1357208700 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7face8036830 0x7face8038ce0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7face4009990 tx=0x7face4006e40 comp rx=0 tx=0).stop 2024-05-17T21:43:01.718 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:01.115+0000 7fad022f2700 1 -- 172.21.15.107:0/1357208700 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7facfc102880 msgr2=0x7facfc1115f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:43:01.718 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:01.115+0000 7fad022f2700 1 --2- 172.21.15.107:0/1357208700 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7facfc102880 0x7facfc1115f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7facec00b440 tx=0x7facec009250 comp rx=0 tx=0).stop 2024-05-17T21:43:01.718 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:01.115+0000 7fad022f2700 1 -- 172.21.15.107:0/1357208700 shutdown_connections 2024-05-17T21:43:01.718 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:01.115+0000 7fad022f2700 1 --2- 172.21.15.107:0/1357208700 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7face8036830 0x7face8038ce0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:43:01.718 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:01.115+0000 7fad022f2700 1 --2- 172.21.15.107:0/1357208700 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7facfc102880 0x7facfc1115f0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:43:01.718 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:01.115+0000 7fad022f2700 1 -- 172.21.15.107:0/1357208700 >> 172.21.15.107:0/1357208700 conn(0x7facfc0fe060 msgr2=0x7facfc0feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:43:01.718 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:01.115+0000 7fad022f2700 1 -- 172.21.15.107:0/1357208700 shutdown_connections 2024-05-17T21:43:01.718 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:01.115+0000 7fad022f2700 1 -- 172.21.15.107:0/1357208700 wait complete. 2024-05-17T21:43:01.718 INFO:teuthology.orchestra.run.smithi107.stdout:Deploying mon service with default placement... 2024-05-17T21:43:02.343 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:02 smithi107 ceph-mon[28434]: from='mgr.14120 172.21.15.107:0/2711624202' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:43:02.343 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:02 smithi107 ceph-mon[28434]: Added host smithi107 2024-05-17T21:43:02.343 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:02 smithi107 ceph-mon[28434]: from='mgr.14120 172.21.15.107:0/2711624202' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:43:03.097 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-05-17T21:43:03.098 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:02.131+0000 7fa68dce8700 1 Processor -- start 2024-05-17T21:43:03.098 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:02.131+0000 7fa68dce8700 1 -- start start 2024-05-17T21:43:03.098 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:02.131+0000 7fa68dce8700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa688076c50 0x7fa688105420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:43:03.098 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:02.131+0000 7fa68dce8700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa688105960 con 0x7fa688076c50 2024-05-17T21:43:03.098 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:02.132+0000 7fa6877fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa688076c50 0x7fa688105420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:43:03.098 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:02.132+0000 7fa6877fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa688076c50 0x7fa688105420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54944/0 (socket says 172.21.15.107:54944) 2024-05-17T21:43:03.098 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:02.132+0000 7fa6877fe700 1 -- 172.21.15.107:0/2393792705 learned_addr learned my addr 172.21.15.107:0/2393792705 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:43:03.098 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:02.132+0000 7fa6877fe700 1 -- 172.21.15.107:0/2393792705 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa688105aa0 con 0x7fa688076c50 2024-05-17T21:43:03.098 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:02.132+0000 7fa6877fe700 1 --2- 172.21.15.107:0/2393792705 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa688076c50 0x7fa688105420 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa6780098e0 tx=0x7fa678009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7c29da634f08f50 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:43:03.098 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:02.132+0000 7fa686ffd700 1 -- 172.21.15.107:0/2393792705 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa67800b860 con 0x7fa688076c50 2024-05-17T21:43:03.098 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:02.132+0000 7fa686ffd700 1 -- 172.21.15.107:0/2393792705 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fa67800be50 con 0x7fa688076c50 2024-05-17T21:43:03.098 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:02.132+0000 7fa686ffd700 1 -- 172.21.15.107:0/2393792705 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa678010380 con 0x7fa688076c50 2024-05-17T21:43:03.098 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:02.133+0000 7fa68dce8700 1 -- 172.21.15.107:0/2393792705 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa688076c50 msgr2=0x7fa688105420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:43:03.099 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:02.133+0000 7fa68dce8700 1 --2- 172.21.15.107:0/2393792705 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa688076c50 0x7fa688105420 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa6780098e0 tx=0x7fa678009250 comp rx=0 tx=0).stop 2024-05-17T21:43:03.099 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:02.133+0000 7fa68dce8700 1 -- 172.21.15.107:0/2393792705 shutdown_connections 2024-05-17T21:43:03.099 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:02.133+0000 7fa68dce8700 1 --2- 172.21.15.107:0/2393792705 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa688076c50 0x7fa688105420 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:43:03.099 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:02.133+0000 7fa68dce8700 1 -- 172.21.15.107:0/2393792705 >> 172.21.15.107:0/2393792705 conn(0x7fa6880fcef0 msgr2=0x7fa6880ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:43:03.099 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:02.133+0000 7fa68dce8700 1 -- 172.21.15.107:0/2393792705 shutdown_connections 2024-05-17T21:43:03.099 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:02.133+0000 7fa68dce8700 1 -- 172.21.15.107:0/2393792705 wait complete. 2024-05-17T21:43:03.099 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:02.133+0000 7fa68dce8700 1 Processor -- start 2024-05-17T21:43:03.099 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:02.134+0000 7fa68dce8700 1 -- start start 2024-05-17T21:43:03.099 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:02.134+0000 7fa68dce8700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa688076c50 0x7fa688111600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:43:03.099 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:02.134+0000 7fa68dce8700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa688111b40 con 0x7fa688076c50 2024-05-17T21:43:03.099 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:02.134+0000 7fa6877fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa688076c50 0x7fa688111600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:43:03.099 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:02.134+0000 7fa6877fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa688076c50 0x7fa688111600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54950/0 (socket says 172.21.15.107:54950) 2024-05-17T21:43:03.099 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:02.134+0000 7fa6877fe700 1 -- 172.21.15.107:0/1978932891 learned_addr learned my addr 172.21.15.107:0/1978932891 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:43:03.100 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:02.134+0000 7fa6877fe700 1 -- 172.21.15.107:0/1978932891 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6780094f0 con 0x7fa688076c50 2024-05-17T21:43:03.100 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:02.134+0000 7fa6877fe700 1 --2- 172.21.15.107:0/1978932891 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa688076c50 0x7fa688111600 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa678006b00 tx=0x7fa678009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:43:03.100 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:02.134+0000 7fa684ff9700 1 -- 172.21.15.107:0/1978932891 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa6780107c0 con 0x7fa688076c50 2024-05-17T21:43:03.100 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:02.134+0000 7fa684ff9700 1 -- 172.21.15.107:0/1978932891 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fa678010db0 con 0x7fa688076c50 2024-05-17T21:43:03.100 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:02.134+0000 7fa68dce8700 1 -- 172.21.15.107:0/1978932891 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa688111d40 con 0x7fa688076c50 2024-05-17T21:43:03.100 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:02.134+0000 7fa684ff9700 1 -- 172.21.15.107:0/1978932891 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa67801acf0 con 0x7fa688076c50 2024-05-17T21:43:03.100 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:02.134+0000 7fa68dce8700 1 -- 172.21.15.107:0/1978932891 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6881128b0 con 0x7fa688076c50 2024-05-17T21:43:03.100 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:02.135+0000 7fa684ff9700 1 -- 172.21.15.107:0/1978932891 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7fa6780193c0 con 0x7fa688076c50 2024-05-17T21:43:03.100 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:02.135+0000 7fa684ff9700 1 --2- 172.21.15.107:0/1978932891 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7fa674036850 0x7fa674038d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:43:03.100 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:02.135+0000 7fa684ff9700 1 -- 172.21.15.107:0/1978932891 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fa67804a660 con 0x7fa688076c50 2024-05-17T21:43:03.100 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:02.135+0000 7fa68dce8700 1 -- 172.21.15.107:0/1978932891 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa68805f5c0 con 0x7fa688076c50 2024-05-17T21:43:03.100 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:02.136+0000 7fa67effd700 1 --2- 172.21.15.107:0/1978932891 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7fa674036850 0x7fa674038d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:43:03.101 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:02.136+0000 7fa67effd700 1 --2- 172.21.15.107:0/1978932891 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7fa674036850 0x7fa674038d00 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa670009990 tx=0x7fa670006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:43:03.101 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:02.140+0000 7fa684ff9700 1 -- 172.21.15.107:0/1978932891 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fa6780196c0 con 0x7fa688076c50 2024-05-17T21:43:03.101 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:02.349+0000 7fa68dce8700 1 -- 172.21.15.107:0/1978932891 --> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}) v1 -- 0x7fa6880fe100 con 0x7fa674036850 2024-05-17T21:43:03.101 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:02.356+0000 7fa684ff9700 1 -- 172.21.15.107:0/1978932891 <== mgr.14120 v2:172.21.15.107:6800/1718632476 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fa6880fe100 con 0x7fa674036850 2024-05-17T21:43:03.101 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:02.358+0000 7fa68dce8700 1 -- 172.21.15.107:0/1978932891 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7fa674036850 msgr2=0x7fa674038d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:43:03.101 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:02.358+0000 7fa68dce8700 1 --2- 172.21.15.107:0/1978932891 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7fa674036850 0x7fa674038d00 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa670009990 tx=0x7fa670006e40 comp rx=0 tx=0).stop 2024-05-17T21:43:03.101 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:02.358+0000 7fa68dce8700 1 -- 172.21.15.107:0/1978932891 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa688076c50 msgr2=0x7fa688111600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:43:03.101 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:02.358+0000 7fa68dce8700 1 --2- 172.21.15.107:0/1978932891 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa688076c50 0x7fa688111600 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa678006b00 tx=0x7fa678009250 comp rx=0 tx=0).stop 2024-05-17T21:43:03.101 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:02.358+0000 7fa68dce8700 1 -- 172.21.15.107:0/1978932891 shutdown_connections 2024-05-17T21:43:03.101 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:02.358+0000 7fa68dce8700 1 --2- 172.21.15.107:0/1978932891 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7fa674036850 0x7fa674038d00 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:43:03.101 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:02.358+0000 7fa68dce8700 1 --2- 172.21.15.107:0/1978932891 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa688076c50 0x7fa688111600 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:43:03.101 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:02.358+0000 7fa68dce8700 1 -- 172.21.15.107:0/1978932891 >> 172.21.15.107:0/1978932891 conn(0x7fa6880fcef0 msgr2=0x7fa6880fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:43:03.101 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:02.358+0000 7fa68dce8700 1 -- 172.21.15.107:0/1978932891 shutdown_connections 2024-05-17T21:43:03.101 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:02.358+0000 7fa68dce8700 1 -- 172.21.15.107:0/1978932891 wait complete. 2024-05-17T21:43:03.102 INFO:teuthology.orchestra.run.smithi107.stdout:Deploying mgr service with default placement... 2024-05-17T21:43:03.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:03 smithi107 ceph-mon[28434]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:43:03.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:03 smithi107 ceph-mon[28434]: Saving service mon spec with placement count:5 2024-05-17T21:43:03.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:03 smithi107 ceph-mon[28434]: from='mgr.14120 172.21.15.107:0/2711624202' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:43:05.011 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:04 smithi107 ceph-mon[28434]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:43:05.011 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:04 smithi107 ceph-mon[28434]: Saving service mgr spec with placement count:2 2024-05-17T21:43:05.011 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:04 smithi107 ceph-mon[28434]: from='mgr.14120 172.21.15.107:0/2711624202' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:43:05.012 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:04 smithi107 ceph-mon[28434]: from='mgr.14120 172.21.15.107:0/2711624202' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:43:05.018 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-05-17T21:43:05.018 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:03.710+0000 7ff6b7ec6700 1 Processor -- start 2024-05-17T21:43:05.018 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:03.710+0000 7ff6b7ec6700 1 -- start start 2024-05-17T21:43:05.019 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:03.710+0000 7ff6b7ec6700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6b0102810 0x7ff6b0102c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:43:05.019 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:03.710+0000 7ff6b7ec6700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6b0103160 con 0x7ff6b0102810 2024-05-17T21:43:05.019 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:03.710+0000 7ff6b5c62700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6b0102810 0x7ff6b0102c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:43:05.019 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:03.710+0000 7ff6b5c62700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6b0102810 0x7ff6b0102c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54952/0 (socket says 172.21.15.107:54952) 2024-05-17T21:43:05.019 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:03.710+0000 7ff6b5c62700 1 -- 172.21.15.107:0/2404116803 learned_addr learned my addr 172.21.15.107:0/2404116803 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:43:05.019 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:03.711+0000 7ff6b5c62700 1 -- 172.21.15.107:0/2404116803 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6b01032a0 con 0x7ff6b0102810 2024-05-17T21:43:05.019 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:03.711+0000 7ff6b5c62700 1 --2- 172.21.15.107:0/2404116803 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6b0102810 0x7ff6b0102c20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac0098e0 tx=0x7ff6ac009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77c89b0f47578418 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:43:05.019 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:03.711+0000 7ff6b4c60700 1 -- 172.21.15.107:0/2404116803 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff6ac00b860 con 0x7ff6b0102810 2024-05-17T21:43:05.019 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:03.711+0000 7ff6b4c60700 1 -- 172.21.15.107:0/2404116803 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7ff6ac00be50 con 0x7ff6b0102810 2024-05-17T21:43:05.019 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:03.711+0000 7ff6b4c60700 1 -- 172.21.15.107:0/2404116803 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff6ac010380 con 0x7ff6b0102810 2024-05-17T21:43:05.020 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:03.711+0000 7ff6b7ec6700 1 -- 172.21.15.107:0/2404116803 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6b0102810 msgr2=0x7ff6b0102c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:43:05.020 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:03.711+0000 7ff6b7ec6700 1 --2- 172.21.15.107:0/2404116803 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6b0102810 0x7ff6b0102c20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac0098e0 tx=0x7ff6ac009250 comp rx=0 tx=0).stop 2024-05-17T21:43:05.020 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:03.712+0000 7ff6b7ec6700 1 -- 172.21.15.107:0/2404116803 shutdown_connections 2024-05-17T21:43:05.020 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:03.712+0000 7ff6b7ec6700 1 --2- 172.21.15.107:0/2404116803 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6b0102810 0x7ff6b0102c20 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:43:05.020 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:03.712+0000 7ff6b7ec6700 1 -- 172.21.15.107:0/2404116803 >> 172.21.15.107:0/2404116803 conn(0x7ff6b00fdfd0 msgr2=0x7ff6b01003e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:43:05.020 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:03.712+0000 7ff6b7ec6700 1 -- 172.21.15.107:0/2404116803 shutdown_connections 2024-05-17T21:43:05.020 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:03.712+0000 7ff6b7ec6700 1 -- 172.21.15.107:0/2404116803 wait complete. 2024-05-17T21:43:05.020 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:03.712+0000 7ff6b7ec6700 1 Processor -- start 2024-05-17T21:43:05.020 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:03.712+0000 7ff6b7ec6700 1 -- start start 2024-05-17T21:43:05.020 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:03.712+0000 7ff6b7ec6700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6b0102810 0x7ff6b01116a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:43:05.020 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:03.712+0000 7ff6b7ec6700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6b0111be0 con 0x7ff6b0102810 2024-05-17T21:43:05.020 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:03.713+0000 7ff6b5c62700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6b0102810 0x7ff6b01116a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:43:05.020 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:03.713+0000 7ff6b5c62700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6b0102810 0x7ff6b01116a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54968/0 (socket says 172.21.15.107:54968) 2024-05-17T21:43:05.020 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:03.713+0000 7ff6b5c62700 1 -- 172.21.15.107:0/1041426037 learned_addr learned my addr 172.21.15.107:0/1041426037 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:43:05.021 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:03.713+0000 7ff6b5c62700 1 -- 172.21.15.107:0/1041426037 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6ac0094f0 con 0x7ff6b0102810 2024-05-17T21:43:05.021 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:03.713+0000 7ff6b5c62700 1 --2- 172.21.15.107:0/1041426037 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6b0102810 0x7ff6b01116a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac006b00 tx=0x7ff6ac009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:43:05.021 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:03.713+0000 7ff6a67fc700 1 -- 172.21.15.107:0/1041426037 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff6ac010760 con 0x7ff6b0102810 2024-05-17T21:43:05.021 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:03.713+0000 7ff6a67fc700 1 -- 172.21.15.107:0/1041426037 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7ff6ac010d50 con 0x7ff6b0102810 2024-05-17T21:43:05.021 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:03.713+0000 7ff6b7ec6700 1 -- 172.21.15.107:0/1041426037 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6b0111de0 con 0x7ff6b0102810 2024-05-17T21:43:05.021 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:03.713+0000 7ff6a67fc700 1 -- 172.21.15.107:0/1041426037 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff6ac01ad40 con 0x7ff6b0102810 2024-05-17T21:43:05.021 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:03.713+0000 7ff6b7ec6700 1 -- 172.21.15.107:0/1041426037 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6b0112950 con 0x7ff6b0102810 2024-05-17T21:43:05.021 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:03.714+0000 7ff6a67fc700 1 -- 172.21.15.107:0/1041426037 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7ff6ac0108c0 con 0x7ff6b0102810 2024-05-17T21:43:05.021 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:03.714+0000 7ff6b7ec6700 1 -- 172.21.15.107:0/1041426037 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6b005f5c0 con 0x7ff6b0102810 2024-05-17T21:43:05.021 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:03.714+0000 7ff6a67fc700 1 --2- 172.21.15.107:0/1041426037 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7ff69c036850 0x7ff69c038d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:43:05.021 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:03.714+0000 7ff6a67fc700 1 -- 172.21.15.107:0/1041426037 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7ff6ac04a540 con 0x7ff6b0102810 2024-05-17T21:43:05.021 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:03.714+0000 7ff6b5461700 1 --2- 172.21.15.107:0/1041426037 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7ff69c036850 0x7ff69c038d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:43:05.021 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:03.715+0000 7ff6b5461700 1 --2- 172.21.15.107:0/1041426037 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7ff69c036850 0x7ff69c038d00 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff6a0009990 tx=0x7ff6a0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:43:05.021 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:03.719+0000 7ff6a67fc700 1 -- 172.21.15.107:0/1041426037 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7ff6ac018b40 con 0x7ff6b0102810 2024-05-17T21:43:05.022 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:03.919+0000 7ff6b7ec6700 1 -- 172.21.15.107:0/1041426037 --> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}) v1 -- 0x7ff6b00ff200 con 0x7ff69c036850 2024-05-17T21:43:05.022 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:03.923+0000 7ff6a67fc700 1 -- 172.21.15.107:0/1041426037 <== mgr.14120 v2:172.21.15.107:6800/1718632476 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7ff6b00ff200 con 0x7ff69c036850 2024-05-17T21:43:05.022 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:03.925+0000 7ff6b7ec6700 1 -- 172.21.15.107:0/1041426037 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7ff69c036850 msgr2=0x7ff69c038d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:43:05.022 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:03.925+0000 7ff6b7ec6700 1 --2- 172.21.15.107:0/1041426037 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7ff69c036850 0x7ff69c038d00 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff6a0009990 tx=0x7ff6a0006e40 comp rx=0 tx=0).stop 2024-05-17T21:43:05.022 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:03.925+0000 7ff6b7ec6700 1 -- 172.21.15.107:0/1041426037 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6b0102810 msgr2=0x7ff6b01116a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:43:05.022 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:03.925+0000 7ff6b7ec6700 1 --2- 172.21.15.107:0/1041426037 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6b0102810 0x7ff6b01116a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac006b00 tx=0x7ff6ac009250 comp rx=0 tx=0).stop 2024-05-17T21:43:05.022 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:03.925+0000 7ff6b7ec6700 1 -- 172.21.15.107:0/1041426037 shutdown_connections 2024-05-17T21:43:05.022 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:03.925+0000 7ff6b7ec6700 1 --2- 172.21.15.107:0/1041426037 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7ff69c036850 0x7ff69c038d00 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:43:05.022 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:03.925+0000 7ff6b7ec6700 1 --2- 172.21.15.107:0/1041426037 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6b0102810 0x7ff6b01116a0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:43:05.022 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:03.926+0000 7ff6b7ec6700 1 -- 172.21.15.107:0/1041426037 >> 172.21.15.107:0/1041426037 conn(0x7ff6b00fdfd0 msgr2=0x7ff6b00feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:43:05.022 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:03.926+0000 7ff6b7ec6700 1 -- 172.21.15.107:0/1041426037 shutdown_connections 2024-05-17T21:43:05.022 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:03.926+0000 7ff6b7ec6700 1 -- 172.21.15.107:0/1041426037 wait complete. 2024-05-17T21:43:05.022 INFO:teuthology.orchestra.run.smithi107.stdout:Deploying crash service with default placement... 2024-05-17T21:43:06.233 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout Scheduled crash update... 2024-05-17T21:43:06.234 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:05.425+0000 7f28c0b40700 1 Processor -- start 2024-05-17T21:43:06.234 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:05.425+0000 7f28c0b40700 1 -- start start 2024-05-17T21:43:06.234 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:05.425+0000 7f28c0b40700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28bc102810 0x7f28bc102c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:43:06.234 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:05.425+0000 7f28c0b40700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28bc103160 con 0x7f28bc102810 2024-05-17T21:43:06.234 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:05.425+0000 7f28ba59c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28bc102810 0x7f28bc102c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:43:06.234 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:05.425+0000 7f28ba59c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28bc102810 0x7f28bc102c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54980/0 (socket says 172.21.15.107:54980) 2024-05-17T21:43:06.234 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:05.425+0000 7f28ba59c700 1 -- 172.21.15.107:0/3475391615 learned_addr learned my addr 172.21.15.107:0/3475391615 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:43:06.234 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:05.426+0000 7f28ba59c700 1 -- 172.21.15.107:0/3475391615 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28bc1032a0 con 0x7f28bc102810 2024-05-17T21:43:06.234 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:05.426+0000 7f28ba59c700 1 --2- 172.21.15.107:0/3475391615 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28bc102810 0x7f28bc102c20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f28ac0098e0 tx=0x7f28ac009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4ab66426ee363e server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:43:06.234 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:05.426+0000 7f28b9d9b700 1 -- 172.21.15.107:0/3475391615 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f28ac00b860 con 0x7f28bc102810 2024-05-17T21:43:06.234 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:05.426+0000 7f28b9d9b700 1 -- 172.21.15.107:0/3475391615 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f28ac00be90 con 0x7f28bc102810 2024-05-17T21:43:06.234 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:05.426+0000 7f28b9d9b700 1 -- 172.21.15.107:0/3475391615 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f28ac0103c0 con 0x7f28bc102810 2024-05-17T21:43:06.234 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:05.426+0000 7f28c0b40700 1 -- 172.21.15.107:0/3475391615 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28bc102810 msgr2=0x7f28bc102c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:43:06.235 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:05.426+0000 7f28c0b40700 1 --2- 172.21.15.107:0/3475391615 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28bc102810 0x7f28bc102c20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f28ac0098e0 tx=0x7f28ac009250 comp rx=0 tx=0).stop 2024-05-17T21:43:06.235 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:05.426+0000 7f28c0b40700 1 -- 172.21.15.107:0/3475391615 shutdown_connections 2024-05-17T21:43:06.235 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:05.426+0000 7f28c0b40700 1 --2- 172.21.15.107:0/3475391615 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28bc102810 0x7f28bc102c20 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:43:06.235 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:05.426+0000 7f28c0b40700 1 -- 172.21.15.107:0/3475391615 >> 172.21.15.107:0/3475391615 conn(0x7f28bc0fdfd0 msgr2=0x7f28bc1003e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:43:06.235 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:05.427+0000 7f28c0b40700 1 -- 172.21.15.107:0/3475391615 shutdown_connections 2024-05-17T21:43:06.235 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:05.427+0000 7f28c0b40700 1 -- 172.21.15.107:0/3475391615 wait complete. 2024-05-17T21:43:06.235 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:05.427+0000 7f28c0b40700 1 Processor -- start 2024-05-17T21:43:06.235 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:05.427+0000 7f28c0b40700 1 -- start start 2024-05-17T21:43:06.235 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:05.427+0000 7f28c0b40700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28bc102810 0x7f28bc1116c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:43:06.235 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:05.427+0000 7f28c0b40700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28bc111c00 con 0x7f28bc102810 2024-05-17T21:43:06.235 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:05.427+0000 7f28ba59c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28bc102810 0x7f28bc1116c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:43:06.235 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:05.427+0000 7f28ba59c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28bc102810 0x7f28bc1116c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54984/0 (socket says 172.21.15.107:54984) 2024-05-17T21:43:06.235 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:05.427+0000 7f28ba59c700 1 -- 172.21.15.107:0/3638553122 learned_addr learned my addr 172.21.15.107:0/3638553122 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:43:06.235 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:05.428+0000 7f28ba59c700 1 -- 172.21.15.107:0/3638553122 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28ac0094f0 con 0x7f28bc102810 2024-05-17T21:43:06.235 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:05.428+0000 7f28ba59c700 1 --2- 172.21.15.107:0/3638553122 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28bc102810 0x7f28bc1116c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f28ac011c10 tx=0x7f28ac009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:43:06.236 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:05.428+0000 7f28b37fe700 1 -- 172.21.15.107:0/3638553122 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f28ac010890 con 0x7f28bc102810 2024-05-17T21:43:06.236 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:05.428+0000 7f28c0b40700 1 -- 172.21.15.107:0/3638553122 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28bc111e00 con 0x7f28bc102810 2024-05-17T21:43:06.236 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:05.428+0000 7f28b37fe700 1 -- 172.21.15.107:0/3638553122 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f28ac010ec0 con 0x7f28bc102810 2024-05-17T21:43:06.236 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:05.428+0000 7f28b37fe700 1 -- 172.21.15.107:0/3638553122 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f28ac0243f0 con 0x7f28bc102810 2024-05-17T21:43:06.236 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:05.428+0000 7f28c0b40700 1 -- 172.21.15.107:0/3638553122 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28bc112970 con 0x7f28bc102810 2024-05-17T21:43:06.236 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:05.429+0000 7f28b37fe700 1 -- 172.21.15.107:0/3638553122 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f28ac02d3f0 con 0x7f28bc102810 2024-05-17T21:43:06.236 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:05.429+0000 7f28c0b40700 1 -- 172.21.15.107:0/3638553122 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28bc05f5c0 con 0x7f28bc102810 2024-05-17T21:43:06.236 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:05.429+0000 7f28b37fe700 1 --2- 172.21.15.107:0/3638553122 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7f28a80368a0 0x7f28a8038d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:43:06.236 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:05.429+0000 7f28b37fe700 1 -- 172.21.15.107:0/3638553122 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f28ac04a220 con 0x7f28bc102810 2024-05-17T21:43:06.236 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:05.429+0000 7f28b1d9b700 1 --2- 172.21.15.107:0/3638553122 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7f28a80368a0 0x7f28a8038d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:43:06.236 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:05.430+0000 7f28b1d9b700 1 --2- 172.21.15.107:0/3638553122 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7f28a80368a0 0x7f28a8038d50 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f28a4009990 tx=0x7f28a4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:43:06.236 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:05.434+0000 7f28b37fe700 1 -- 172.21.15.107:0/3638553122 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f28ac02b8e0 con 0x7f28bc102810 2024-05-17T21:43:06.236 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:05.636+0000 7f28c0b40700 1 -- 172.21.15.107:0/3638553122 --> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}) v1 -- 0x7f28bc0ff230 con 0x7f28a80368a0 2024-05-17T21:43:06.236 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:05.640+0000 7f28b37fe700 1 -- 172.21.15.107:0/3638553122 <== mgr.14120 v2:172.21.15.107:6800/1718632476 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7f28bc0ff230 con 0x7f28a80368a0 2024-05-17T21:43:06.237 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:05.642+0000 7f28c0b40700 1 -- 172.21.15.107:0/3638553122 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7f28a80368a0 msgr2=0x7f28a8038d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:43:06.237 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:05.642+0000 7f28c0b40700 1 --2- 172.21.15.107:0/3638553122 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7f28a80368a0 0x7f28a8038d50 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f28a4009990 tx=0x7f28a4006e40 comp rx=0 tx=0).stop 2024-05-17T21:43:06.237 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:05.642+0000 7f28c0b40700 1 -- 172.21.15.107:0/3638553122 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28bc102810 msgr2=0x7f28bc1116c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:43:06.237 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:05.642+0000 7f28c0b40700 1 --2- 172.21.15.107:0/3638553122 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28bc102810 0x7f28bc1116c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f28ac011c10 tx=0x7f28ac009250 comp rx=0 tx=0).stop 2024-05-17T21:43:06.237 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:05.642+0000 7f28c0b40700 1 -- 172.21.15.107:0/3638553122 shutdown_connections 2024-05-17T21:43:06.237 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:05.642+0000 7f28c0b40700 1 --2- 172.21.15.107:0/3638553122 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7f28a80368a0 0x7f28a8038d50 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:43:06.237 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:05.642+0000 7f28c0b40700 1 --2- 172.21.15.107:0/3638553122 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28bc102810 0x7f28bc1116c0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:43:06.237 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:05.642+0000 7f28c0b40700 1 -- 172.21.15.107:0/3638553122 >> 172.21.15.107:0/3638553122 conn(0x7f28bc0fdfd0 msgr2=0x7f28bc0feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:43:06.237 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:05.643+0000 7f28c0b40700 1 -- 172.21.15.107:0/3638553122 shutdown_connections 2024-05-17T21:43:06.237 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:05.643+0000 7f28c0b40700 1 -- 172.21.15.107:0/3638553122 wait complete. 2024-05-17T21:43:06.237 INFO:teuthology.orchestra.run.smithi107.stdout:Deploying prometheus service with default placement... 2024-05-17T21:43:06.843 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:06 smithi107 ceph-mon[28434]: from='mgr.14120 172.21.15.107:0/2711624202' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:43:06.844 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:06 smithi107 ceph-mon[28434]: from='client.14146 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:43:06.844 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:06 smithi107 ceph-mon[28434]: Saving service crash spec with placement * 2024-05-17T21:43:06.844 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:06 smithi107 ceph-mon[28434]: from='mgr.14120 172.21.15.107:0/2711624202' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:43:07.863 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:07 smithi107 ceph-mon[28434]: from='client.14148 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:43:07.863 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:07 smithi107 ceph-mon[28434]: Saving service prometheus spec with placement count:1 2024-05-17T21:43:07.863 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:07 smithi107 ceph-mon[28434]: from='mgr.14120 172.21.15.107:0/2711624202' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:43:07.863 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:07 smithi107 ceph-mon[28434]: from='mgr.14120 172.21.15.107:0/2711624202' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:43:07.868 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout Scheduled prometheus update... 2024-05-17T21:43:07.868 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:06.631+0000 7f54245b1700 1 Processor -- start 2024-05-17T21:43:07.868 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:06.631+0000 7f54245b1700 1 -- start start 2024-05-17T21:43:07.868 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:06.631+0000 7f54245b1700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f541c102810 0x7f541c102c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:43:07.868 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:06.631+0000 7f54245b1700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f541c103160 con 0x7f541c102810 2024-05-17T21:43:07.868 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:06.632+0000 7f542234d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f541c102810 0x7f541c102c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:43:07.868 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:06.632+0000 7f542234d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f541c102810 0x7f541c102c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54290/0 (socket says 172.21.15.107:54290) 2024-05-17T21:43:07.868 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:06.632+0000 7f542234d700 1 -- 172.21.15.107:0/2663519150 learned_addr learned my addr 172.21.15.107:0/2663519150 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:43:07.869 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:06.632+0000 7f542234d700 1 -- 172.21.15.107:0/2663519150 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f541c1032a0 con 0x7f541c102810 2024-05-17T21:43:07.869 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:06.632+0000 7f542234d700 1 --2- 172.21.15.107:0/2663519150 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f541c102810 0x7f541c102c20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5418009e50 tx=0x7f5418009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=480ac6f9d6f0811d server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:43:07.869 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:06.632+0000 7f542134b700 1 -- 172.21.15.107:0/2663519150 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f54180092e0 con 0x7f541c102810 2024-05-17T21:43:07.869 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:06.632+0000 7f542134b700 1 -- 172.21.15.107:0/2663519150 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5418012430 con 0x7f541c102810 2024-05-17T21:43:07.869 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:06.632+0000 7f542134b700 1 -- 172.21.15.107:0/2663519150 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f54180075a0 con 0x7f541c102810 2024-05-17T21:43:07.869 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:06.633+0000 7f54245b1700 1 -- 172.21.15.107:0/2663519150 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f541c102810 msgr2=0x7f541c102c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:43:07.869 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:06.633+0000 7f54245b1700 1 --2- 172.21.15.107:0/2663519150 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f541c102810 0x7f541c102c20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5418009e50 tx=0x7f5418009840 comp rx=0 tx=0).stop 2024-05-17T21:43:07.869 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:06.633+0000 7f54245b1700 1 -- 172.21.15.107:0/2663519150 shutdown_connections 2024-05-17T21:43:07.869 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:06.633+0000 7f54245b1700 1 --2- 172.21.15.107:0/2663519150 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f541c102810 0x7f541c102c20 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:43:07.869 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:06.633+0000 7f54245b1700 1 -- 172.21.15.107:0/2663519150 >> 172.21.15.107:0/2663519150 conn(0x7f541c0fdfd0 msgr2=0x7f541c1003e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:43:07.869 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:06.633+0000 7f54245b1700 1 -- 172.21.15.107:0/2663519150 shutdown_connections 2024-05-17T21:43:07.869 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:06.633+0000 7f54245b1700 1 -- 172.21.15.107:0/2663519150 wait complete. 2024-05-17T21:43:07.869 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:06.633+0000 7f54245b1700 1 Processor -- start 2024-05-17T21:43:07.870 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:06.633+0000 7f54245b1700 1 -- start start 2024-05-17T21:43:07.870 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:06.633+0000 7f54245b1700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f541c102810 0x7f541c1116e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:43:07.870 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:06.633+0000 7f54245b1700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f541c111c20 con 0x7f541c102810 2024-05-17T21:43:07.870 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:06.634+0000 7f542234d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f541c102810 0x7f541c1116e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:43:07.870 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:06.634+0000 7f542234d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f541c102810 0x7f541c1116e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54294/0 (socket says 172.21.15.107:54294) 2024-05-17T21:43:07.870 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:06.634+0000 7f542234d700 1 -- 172.21.15.107:0/2736698805 learned_addr learned my addr 172.21.15.107:0/2736698805 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:43:07.870 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:06.634+0000 7f542234d700 1 -- 172.21.15.107:0/2736698805 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5418009ae0 con 0x7f541c102810 2024-05-17T21:43:07.870 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:06.634+0000 7f542234d700 1 --2- 172.21.15.107:0/2736698805 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f541c102810 0x7f541c1116e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5418012de0 tx=0x7f5418009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:43:07.870 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:06.634+0000 7f5412ffd700 1 -- 172.21.15.107:0/2736698805 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f54180092e0 con 0x7f541c102810 2024-05-17T21:43:07.870 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:06.634+0000 7f5412ffd700 1 -- 172.21.15.107:0/2736698805 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5418007d30 con 0x7f541c102810 2024-05-17T21:43:07.870 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:06.634+0000 7f54245b1700 1 -- 172.21.15.107:0/2736698805 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f541c111e20 con 0x7f541c102810 2024-05-17T21:43:07.870 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:06.634+0000 7f5412ffd700 1 -- 172.21.15.107:0/2736698805 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5418025880 con 0x7f541c102810 2024-05-17T21:43:07.870 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:06.634+0000 7f54245b1700 1 -- 172.21.15.107:0/2736698805 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f541c112990 con 0x7f541c102810 2024-05-17T21:43:07.871 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:06.635+0000 7f5412ffd700 1 -- 172.21.15.107:0/2736698805 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f5418019050 con 0x7f541c102810 2024-05-17T21:43:07.871 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:06.635+0000 7f54245b1700 1 -- 172.21.15.107:0/2736698805 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f541c05f5c0 con 0x7f541c102810 2024-05-17T21:43:07.871 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:06.635+0000 7f5412ffd700 1 --2- 172.21.15.107:0/2736698805 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7f5408036850 0x7f5408038d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:43:07.871 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:06.635+0000 7f5412ffd700 1 -- 172.21.15.107:0/2736698805 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f541804a230 con 0x7f541c102810 2024-05-17T21:43:07.871 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:06.636+0000 7f5421b4c700 1 --2- 172.21.15.107:0/2736698805 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7f5408036850 0x7f5408038d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:43:07.871 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:06.636+0000 7f5421b4c700 1 --2- 172.21.15.107:0/2736698805 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7f5408036850 0x7f5408038d00 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f540c009990 tx=0x7f540c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:43:07.871 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:06.640+0000 7f5412ffd700 1 -- 172.21.15.107:0/2736698805 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f541802d350 con 0x7f541c102810 2024-05-17T21:43:07.871 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:06.848+0000 7f54245b1700 1 -- 172.21.15.107:0/2736698805 --> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}) v1 -- 0x7f541c0ff270 con 0x7f5408036850 2024-05-17T21:43:07.871 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:06.853+0000 7f5412ffd700 1 -- 172.21.15.107:0/2736698805 <== mgr.14120 v2:172.21.15.107:6800/1718632476 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+31 (secure 0 0 0) 0x7f541c0ff270 con 0x7f5408036850 2024-05-17T21:43:07.871 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:06.855+0000 7f54245b1700 1 -- 172.21.15.107:0/2736698805 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7f5408036850 msgr2=0x7f5408038d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:43:07.871 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:06.855+0000 7f54245b1700 1 --2- 172.21.15.107:0/2736698805 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7f5408036850 0x7f5408038d00 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f540c009990 tx=0x7f540c006e40 comp rx=0 tx=0).stop 2024-05-17T21:43:07.871 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:06.855+0000 7f54245b1700 1 -- 172.21.15.107:0/2736698805 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f541c102810 msgr2=0x7f541c1116e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:43:07.872 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:06.855+0000 7f54245b1700 1 --2- 172.21.15.107:0/2736698805 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f541c102810 0x7f541c1116e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5418012de0 tx=0x7f5418009840 comp rx=0 tx=0).stop 2024-05-17T21:43:07.872 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:06.855+0000 7f54245b1700 1 -- 172.21.15.107:0/2736698805 shutdown_connections 2024-05-17T21:43:07.872 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:06.855+0000 7f54245b1700 1 --2- 172.21.15.107:0/2736698805 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7f5408036850 0x7f5408038d00 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:43:07.872 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:06.855+0000 7f54245b1700 1 --2- 172.21.15.107:0/2736698805 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f541c102810 0x7f541c1116e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:43:07.872 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:06.855+0000 7f54245b1700 1 -- 172.21.15.107:0/2736698805 >> 172.21.15.107:0/2736698805 conn(0x7f541c0fdfd0 msgr2=0x7f541c0feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:43:07.872 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:06.855+0000 7f54245b1700 1 -- 172.21.15.107:0/2736698805 shutdown_connections 2024-05-17T21:43:07.872 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:06.855+0000 7f54245b1700 1 -- 172.21.15.107:0/2736698805 wait complete. 2024-05-17T21:43:07.872 INFO:teuthology.orchestra.run.smithi107.stdout:Deploying grafana service with default placement... 2024-05-17T21:43:09.015 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout Scheduled grafana update... 2024-05-17T21:43:09.015 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:08.251+0000 7f1e46747700 1 Processor -- start 2024-05-17T21:43:09.015 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:08.251+0000 7f1e46747700 1 -- start start 2024-05-17T21:43:09.016 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:08.251+0000 7f1e46747700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1e40102810 0x7f1e40102c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:43:09.016 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:08.251+0000 7f1e46747700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e40103160 con 0x7f1e40102810 2024-05-17T21:43:09.016 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:08.252+0000 7f1e3ffff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1e40102810 0x7f1e40102c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:43:09.016 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:08.252+0000 7f1e3ffff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1e40102810 0x7f1e40102c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54302/0 (socket says 172.21.15.107:54302) 2024-05-17T21:43:09.016 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:08.252+0000 7f1e3ffff700 1 -- 172.21.15.107:0/1262620363 learned_addr learned my addr 172.21.15.107:0/1262620363 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:43:09.016 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:08.252+0000 7f1e3ffff700 1 -- 172.21.15.107:0/1262620363 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e401032a0 con 0x7f1e40102810 2024-05-17T21:43:09.016 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:08.252+0000 7f1e3ffff700 1 --2- 172.21.15.107:0/1262620363 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1e40102810 0x7f1e40102c20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1e280098e0 tx=0x7f1e28009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35740402d1976bff server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:43:09.016 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:08.252+0000 7f1e3effd700 1 -- 172.21.15.107:0/1262620363 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1e2800b860 con 0x7f1e40102810 2024-05-17T21:43:09.016 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:08.252+0000 7f1e3effd700 1 -- 172.21.15.107:0/1262620363 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1e2800be90 con 0x7f1e40102810 2024-05-17T21:43:09.016 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:08.252+0000 7f1e3effd700 1 -- 172.21.15.107:0/1262620363 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1e280103c0 con 0x7f1e40102810 2024-05-17T21:43:09.016 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:08.253+0000 7f1e46747700 1 -- 172.21.15.107:0/1262620363 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1e40102810 msgr2=0x7f1e40102c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:43:09.016 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:08.253+0000 7f1e46747700 1 --2- 172.21.15.107:0/1262620363 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1e40102810 0x7f1e40102c20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1e280098e0 tx=0x7f1e28009250 comp rx=0 tx=0).stop 2024-05-17T21:43:09.016 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:08.253+0000 7f1e46747700 1 -- 172.21.15.107:0/1262620363 shutdown_connections 2024-05-17T21:43:09.016 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:08.253+0000 7f1e46747700 1 --2- 172.21.15.107:0/1262620363 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1e40102810 0x7f1e40102c20 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:43:09.017 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:08.253+0000 7f1e46747700 1 -- 172.21.15.107:0/1262620363 >> 172.21.15.107:0/1262620363 conn(0x7f1e400fdfd0 msgr2=0x7f1e401003e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:43:09.017 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:08.253+0000 7f1e46747700 1 -- 172.21.15.107:0/1262620363 shutdown_connections 2024-05-17T21:43:09.017 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:08.253+0000 7f1e46747700 1 -- 172.21.15.107:0/1262620363 wait complete. 2024-05-17T21:43:09.017 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:08.253+0000 7f1e46747700 1 Processor -- start 2024-05-17T21:43:09.017 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:08.254+0000 7f1e46747700 1 -- start start 2024-05-17T21:43:09.017 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:08.254+0000 7f1e46747700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1e40102810 0x7f1e401116c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:43:09.017 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:08.254+0000 7f1e46747700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e40111c00 con 0x7f1e40102810 2024-05-17T21:43:09.017 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:08.254+0000 7f1e3ffff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1e40102810 0x7f1e401116c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:43:09.017 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:08.254+0000 7f1e3ffff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1e40102810 0x7f1e401116c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54306/0 (socket says 172.21.15.107:54306) 2024-05-17T21:43:09.017 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:08.254+0000 7f1e3ffff700 1 -- 172.21.15.107:0/3566955056 learned_addr learned my addr 172.21.15.107:0/3566955056 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:43:09.017 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:08.254+0000 7f1e3ffff700 1 -- 172.21.15.107:0/3566955056 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e280094f0 con 0x7f1e40102810 2024-05-17T21:43:09.017 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:08.254+0000 7f1e3ffff700 1 --2- 172.21.15.107:0/3566955056 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1e40102810 0x7f1e401116c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1e28011c10 tx=0x7f1e28009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:43:09.017 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:08.254+0000 7f1e3cff9700 1 -- 172.21.15.107:0/3566955056 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1e28010890 con 0x7f1e40102810 2024-05-17T21:43:09.017 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:08.254+0000 7f1e46747700 1 -- 172.21.15.107:0/3566955056 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e40111e00 con 0x7f1e40102810 2024-05-17T21:43:09.018 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:08.254+0000 7f1e3cff9700 1 -- 172.21.15.107:0/3566955056 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1e28010ec0 con 0x7f1e40102810 2024-05-17T21:43:09.018 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:08.254+0000 7f1e3cff9700 1 -- 172.21.15.107:0/3566955056 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1e280243f0 con 0x7f1e40102810 2024-05-17T21:43:09.018 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:08.254+0000 7f1e46747700 1 -- 172.21.15.107:0/3566955056 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e40112970 con 0x7f1e40102810 2024-05-17T21:43:09.018 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:08.255+0000 7f1e3cff9700 1 -- 172.21.15.107:0/3566955056 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f1e2802d3f0 con 0x7f1e40102810 2024-05-17T21:43:09.018 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:08.255+0000 7f1e3cff9700 1 --2- 172.21.15.107:0/3566955056 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7f1e2c036850 0x7f1e2c038d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:43:09.018 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:08.255+0000 7f1e3cff9700 1 -- 172.21.15.107:0/3566955056 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f1e2804b400 con 0x7f1e40102810 2024-05-17T21:43:09.018 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:08.255+0000 7f1e3f7fe700 1 --2- 172.21.15.107:0/3566955056 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7f1e2c036850 0x7f1e2c038d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:43:09.018 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:08.256+0000 7f1e46747700 1 -- 172.21.15.107:0/3566955056 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e4005f5c0 con 0x7f1e40102810 2024-05-17T21:43:09.018 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:08.256+0000 7f1e3f7fe700 1 --2- 172.21.15.107:0/3566955056 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7f1e2c036850 0x7f1e2c038d00 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1e30009990 tx=0x7f1e30006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:43:09.018 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:08.260+0000 7f1e3cff9700 1 -- 172.21.15.107:0/3566955056 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f1e2802b8e0 con 0x7f1e40102810 2024-05-17T21:43:09.018 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:08.470+0000 7f1e46747700 1 -- 172.21.15.107:0/3566955056 --> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}) v1 -- 0x7f1e400ff230 con 0x7f1e2c036850 2024-05-17T21:43:09.018 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:08.478+0000 7f1e3cff9700 1 -- 172.21.15.107:0/3566955056 <== mgr.14120 v2:172.21.15.107:6800/1718632476 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f1e400ff230 con 0x7f1e2c036850 2024-05-17T21:43:09.018 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:08.480+0000 7f1e46747700 1 -- 172.21.15.107:0/3566955056 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7f1e2c036850 msgr2=0x7f1e2c038d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:43:09.018 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:08.480+0000 7f1e46747700 1 --2- 172.21.15.107:0/3566955056 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7f1e2c036850 0x7f1e2c038d00 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1e30009990 tx=0x7f1e30006e40 comp rx=0 tx=0).stop 2024-05-17T21:43:09.019 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:08.480+0000 7f1e46747700 1 -- 172.21.15.107:0/3566955056 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1e40102810 msgr2=0x7f1e401116c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:43:09.019 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:08.480+0000 7f1e46747700 1 --2- 172.21.15.107:0/3566955056 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1e40102810 0x7f1e401116c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1e28011c10 tx=0x7f1e28009250 comp rx=0 tx=0).stop 2024-05-17T21:43:09.019 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:08.480+0000 7f1e46747700 1 -- 172.21.15.107:0/3566955056 shutdown_connections 2024-05-17T21:43:09.019 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:08.480+0000 7f1e46747700 1 --2- 172.21.15.107:0/3566955056 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7f1e2c036850 0x7f1e2c038d00 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:43:09.019 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:08.480+0000 7f1e46747700 1 --2- 172.21.15.107:0/3566955056 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1e40102810 0x7f1e401116c0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:43:09.019 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:08.480+0000 7f1e46747700 1 -- 172.21.15.107:0/3566955056 >> 172.21.15.107:0/3566955056 conn(0x7f1e400fdfd0 msgr2=0x7f1e400feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:43:09.019 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:08.480+0000 7f1e46747700 1 -- 172.21.15.107:0/3566955056 shutdown_connections 2024-05-17T21:43:09.019 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:08.480+0000 7f1e46747700 1 -- 172.21.15.107:0/3566955056 wait complete. 2024-05-17T21:43:09.019 INFO:teuthology.orchestra.run.smithi107.stdout:Deploying node-exporter service with default placement... 2024-05-17T21:43:09.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:09 smithi107 ceph-mon[28434]: from='mgr.14120 172.21.15.107:0/2711624202' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:43:09.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:09 smithi107 ceph-mon[28434]: from='client.14150 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:43:09.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:09 smithi107 ceph-mon[28434]: Saving service grafana spec with placement count:1 2024-05-17T21:43:09.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:09 smithi107 ceph-mon[28434]: from='mgr.14120 172.21.15.107:0/2711624202' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:43:10.141 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout Scheduled node-exporter update... 2024-05-17T21:43:10.141 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:09.402+0000 7f967a11c700 1 Processor -- start 2024-05-17T21:43:10.141 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:09.402+0000 7f967a11c700 1 -- start start 2024-05-17T21:43:10.141 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:09.402+0000 7f967a11c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9674101750 0x7f9674101b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:43:10.141 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:09.402+0000 7f967a11c700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96741020a0 con 0x7f9674101750 2024-05-17T21:43:10.141 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:09.403+0000 7f96737fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9674101750 0x7f9674101b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:43:10.142 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:09.403+0000 7f96737fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9674101750 0x7f9674101b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54310/0 (socket says 172.21.15.107:54310) 2024-05-17T21:43:10.142 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:09.403+0000 7f96737fe700 1 -- 172.21.15.107:0/1998984745 learned_addr learned my addr 172.21.15.107:0/1998984745 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:43:10.142 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:09.403+0000 7f96737fe700 1 -- 172.21.15.107:0/1998984745 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96741021e0 con 0x7f9674101750 2024-05-17T21:43:10.142 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:09.403+0000 7f96737fe700 1 --2- 172.21.15.107:0/1998984745 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9674101750 0x7f9674101b60 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f96640098e0 tx=0x7f9664009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=800a2648348837b1 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:43:10.142 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:09.403+0000 7f96727fc700 1 -- 172.21.15.107:0/1998984745 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f966400b860 con 0x7f9674101750 2024-05-17T21:43:10.142 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:09.403+0000 7f96727fc700 1 -- 172.21.15.107:0/1998984745 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f966400be90 con 0x7f9674101750 2024-05-17T21:43:10.142 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:09.403+0000 7f96727fc700 1 -- 172.21.15.107:0/1998984745 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f96640103c0 con 0x7f9674101750 2024-05-17T21:43:10.142 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:09.404+0000 7f967a11c700 1 -- 172.21.15.107:0/1998984745 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9674101750 msgr2=0x7f9674101b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:43:10.142 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:09.404+0000 7f967a11c700 1 --2- 172.21.15.107:0/1998984745 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9674101750 0x7f9674101b60 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f96640098e0 tx=0x7f9664009250 comp rx=0 tx=0).stop 2024-05-17T21:43:10.142 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:09.404+0000 7f967a11c700 1 -- 172.21.15.107:0/1998984745 shutdown_connections 2024-05-17T21:43:10.142 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:09.404+0000 7f967a11c700 1 --2- 172.21.15.107:0/1998984745 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9674101750 0x7f9674101b60 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:43:10.142 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:09.404+0000 7f967a11c700 1 -- 172.21.15.107:0/1998984745 >> 172.21.15.107:0/1998984745 conn(0x7f96740fcef0 msgr2=0x7f96740ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:43:10.143 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:09.404+0000 7f967a11c700 1 -- 172.21.15.107:0/1998984745 shutdown_connections 2024-05-17T21:43:10.143 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:09.404+0000 7f967a11c700 1 -- 172.21.15.107:0/1998984745 wait complete. 2024-05-17T21:43:10.143 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:09.405+0000 7f967a11c700 1 Processor -- start 2024-05-17T21:43:10.143 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:09.405+0000 7f967a11c700 1 -- start start 2024-05-17T21:43:10.143 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:09.405+0000 7f967a11c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9674101750 0x7f9674113800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:43:10.143 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:09.405+0000 7f967a11c700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9674113d40 con 0x7f9674101750 2024-05-17T21:43:10.143 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:09.405+0000 7f96737fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9674101750 0x7f9674113800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:43:10.143 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:09.405+0000 7f96737fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9674101750 0x7f9674113800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54320/0 (socket says 172.21.15.107:54320) 2024-05-17T21:43:10.143 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:09.405+0000 7f96737fe700 1 -- 172.21.15.107:0/3763740230 learned_addr learned my addr 172.21.15.107:0/3763740230 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:43:10.143 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:09.406+0000 7f96737fe700 1 -- 172.21.15.107:0/3763740230 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96640094f0 con 0x7f9674101750 2024-05-17T21:43:10.143 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:09.406+0000 7f96737fe700 1 --2- 172.21.15.107:0/3763740230 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9674101750 0x7f9674113800 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9664011680 tx=0x7f9664009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:43:10.143 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:09.406+0000 7f965bfff700 1 -- 172.21.15.107:0/3763740230 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9664010890 con 0x7f9674101750 2024-05-17T21:43:10.143 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:09.406+0000 7f965bfff700 1 -- 172.21.15.107:0/3763740230 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9664010ec0 con 0x7f9674101750 2024-05-17T21:43:10.143 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:09.406+0000 7f967a11c700 1 -- 172.21.15.107:0/3763740230 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9674113f40 con 0x7f9674101750 2024-05-17T21:43:10.144 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:09.406+0000 7f965bfff700 1 -- 172.21.15.107:0/3763740230 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f96640243f0 con 0x7f9674101750 2024-05-17T21:43:10.144 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:09.406+0000 7f967a11c700 1 -- 172.21.15.107:0/3763740230 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9674114ab0 con 0x7f9674101750 2024-05-17T21:43:10.144 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:09.407+0000 7f965bfff700 1 -- 172.21.15.107:0/3763740230 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f966402d3f0 con 0x7f9674101750 2024-05-17T21:43:10.144 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:09.407+0000 7f967a11c700 1 -- 172.21.15.107:0/3763740230 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f967405f5c0 con 0x7f9674101750 2024-05-17T21:43:10.144 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:09.407+0000 7f965bfff700 1 --2- 172.21.15.107:0/3763740230 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7f965c036850 0x7f965c038d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:43:10.144 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:09.407+0000 7f965bfff700 1 -- 172.21.15.107:0/3763740230 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f96640109f0 con 0x7f9674101750 2024-05-17T21:43:10.144 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:09.407+0000 7f9672ffd700 1 --2- 172.21.15.107:0/3763740230 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7f965c036850 0x7f965c038d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:43:10.144 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:09.408+0000 7f9672ffd700 1 --2- 172.21.15.107:0/3763740230 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7f965c036850 0x7f965c038d00 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f9668009990 tx=0x7f9668006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:43:10.144 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:09.412+0000 7f965bfff700 1 -- 172.21.15.107:0/3763740230 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f966402b8e0 con 0x7f9674101750 2024-05-17T21:43:10.144 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:09.617+0000 7f967a11c700 1 -- 172.21.15.107:0/3763740230 --> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}) v1 -- 0x7f9674115630 con 0x7f965c036850 2024-05-17T21:43:10.144 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:09.624+0000 7f965bfff700 1 -- 172.21.15.107:0/3763740230 <== mgr.14120 v2:172.21.15.107:6800/1718632476 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+34 (secure 0 0 0) 0x7f9674115630 con 0x7f965c036850 2024-05-17T21:43:10.145 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:09.626+0000 7f967a11c700 1 -- 172.21.15.107:0/3763740230 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7f965c036850 msgr2=0x7f965c038d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:43:10.145 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:09.626+0000 7f967a11c700 1 --2- 172.21.15.107:0/3763740230 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7f965c036850 0x7f965c038d00 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f9668009990 tx=0x7f9668006e40 comp rx=0 tx=0).stop 2024-05-17T21:43:10.145 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:09.626+0000 7f967a11c700 1 -- 172.21.15.107:0/3763740230 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9674101750 msgr2=0x7f9674113800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:43:10.145 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:09.626+0000 7f967a11c700 1 --2- 172.21.15.107:0/3763740230 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9674101750 0x7f9674113800 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9664011680 tx=0x7f9664009250 comp rx=0 tx=0).stop 2024-05-17T21:43:10.145 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:09.626+0000 7f967a11c700 1 -- 172.21.15.107:0/3763740230 shutdown_connections 2024-05-17T21:43:10.145 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:09.626+0000 7f967a11c700 1 --2- 172.21.15.107:0/3763740230 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7f965c036850 0x7f965c038d00 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:43:10.145 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:09.626+0000 7f967a11c700 1 --2- 172.21.15.107:0/3763740230 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9674101750 0x7f9674113800 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:43:10.145 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:09.626+0000 7f967a11c700 1 -- 172.21.15.107:0/3763740230 >> 172.21.15.107:0/3763740230 conn(0x7f96740fcef0 msgr2=0x7f96740fdcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:43:10.145 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:09.626+0000 7f967a11c700 1 -- 172.21.15.107:0/3763740230 shutdown_connections 2024-05-17T21:43:10.145 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:09.626+0000 7f967a11c700 1 -- 172.21.15.107:0/3763740230 wait complete. 2024-05-17T21:43:10.145 INFO:teuthology.orchestra.run.smithi107.stdout:Deploying alertmanager service with default placement... 2024-05-17T21:43:10.711 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:10 smithi107 ceph-mon[28434]: from='client.14152 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:43:10.711 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:10 smithi107 ceph-mon[28434]: Saving service node-exporter spec with placement * 2024-05-17T21:43:10.712 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:10 smithi107 ceph-mon[28434]: from='mgr.14120 172.21.15.107:0/2711624202' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:43:11.771 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:11 smithi107 ceph-mon[28434]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:43:11.771 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:11 smithi107 ceph-mon[28434]: Saving service alertmanager spec with placement count:1 2024-05-17T21:43:11.771 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:11 smithi107 ceph-mon[28434]: from='mgr.14120 172.21.15.107:0/2711624202' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:43:11.776 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout Scheduled alertmanager update... 2024-05-17T21:43:11.776 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:10.537+0000 7f5df7d2e700 1 Processor -- start 2024-05-17T21:43:11.776 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:10.538+0000 7f5df7d2e700 1 -- start start 2024-05-17T21:43:11.776 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:10.538+0000 7f5df7d2e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5df00712d0 0x7f5df0105470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:43:11.776 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:10.538+0000 7f5df7d2e700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5df01059b0 con 0x7f5df00712d0 2024-05-17T21:43:11.776 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:10.538+0000 7f5df5aca700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5df00712d0 0x7f5df0105470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:43:11.776 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:10.538+0000 7f5df5aca700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5df00712d0 0x7f5df0105470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54326/0 (socket says 172.21.15.107:54326) 2024-05-17T21:43:11.776 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:10.538+0000 7f5df5aca700 1 -- 172.21.15.107:0/3183000217 learned_addr learned my addr 172.21.15.107:0/3183000217 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:43:11.776 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:10.538+0000 7f5df5aca700 1 -- 172.21.15.107:0/3183000217 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5df0105af0 con 0x7f5df00712d0 2024-05-17T21:43:11.777 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:10.538+0000 7f5df5aca700 1 --2- 172.21.15.107:0/3183000217 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5df00712d0 0x7f5df0105470 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5de40098e0 tx=0x7f5de4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f9ed9bd2d57add6 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:43:11.777 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:10.539+0000 7f5df4ac8700 1 -- 172.21.15.107:0/3183000217 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5de400b860 con 0x7f5df00712d0 2024-05-17T21:43:11.777 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:10.539+0000 7f5df4ac8700 1 -- 172.21.15.107:0/3183000217 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5de400be90 con 0x7f5df00712d0 2024-05-17T21:43:11.777 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:10.539+0000 7f5df4ac8700 1 -- 172.21.15.107:0/3183000217 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5de40103c0 con 0x7f5df00712d0 2024-05-17T21:43:11.777 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:10.539+0000 7f5df7d2e700 1 -- 172.21.15.107:0/3183000217 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5df00712d0 msgr2=0x7f5df0105470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:43:11.777 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:10.539+0000 7f5df7d2e700 1 --2- 172.21.15.107:0/3183000217 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5df00712d0 0x7f5df0105470 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5de40098e0 tx=0x7f5de4009250 comp rx=0 tx=0).stop 2024-05-17T21:43:11.777 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:10.539+0000 7f5df7d2e700 1 -- 172.21.15.107:0/3183000217 shutdown_connections 2024-05-17T21:43:11.777 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:10.539+0000 7f5df7d2e700 1 --2- 172.21.15.107:0/3183000217 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5df00712d0 0x7f5df0105470 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:43:11.777 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:10.539+0000 7f5df7d2e700 1 -- 172.21.15.107:0/3183000217 >> 172.21.15.107:0/3183000217 conn(0x7f5df00fcea0 msgr2=0x7f5df00ff2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:43:11.777 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:10.539+0000 7f5df7d2e700 1 -- 172.21.15.107:0/3183000217 shutdown_connections 2024-05-17T21:43:11.777 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:10.539+0000 7f5df7d2e700 1 -- 172.21.15.107:0/3183000217 wait complete. 2024-05-17T21:43:11.777 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:10.540+0000 7f5df7d2e700 1 Processor -- start 2024-05-17T21:43:11.777 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:10.540+0000 7f5df7d2e700 1 -- start start 2024-05-17T21:43:11.777 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:10.540+0000 7f5df7d2e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5df00712d0 0x7f5df0111670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:43:11.778 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:10.540+0000 7f5df7d2e700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5df0111bb0 con 0x7f5df00712d0 2024-05-17T21:43:11.778 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:10.540+0000 7f5df5aca700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5df00712d0 0x7f5df0111670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:43:11.778 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:10.540+0000 7f5df5aca700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5df00712d0 0x7f5df0111670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54332/0 (socket says 172.21.15.107:54332) 2024-05-17T21:43:11.778 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:10.540+0000 7f5df5aca700 1 -- 172.21.15.107:0/3056858199 learned_addr learned my addr 172.21.15.107:0/3056858199 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:43:11.778 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:10.540+0000 7f5df5aca700 1 -- 172.21.15.107:0/3056858199 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5de40094f0 con 0x7f5df00712d0 2024-05-17T21:43:11.778 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:10.540+0000 7f5df5aca700 1 --2- 172.21.15.107:0/3056858199 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5df00712d0 0x7f5df0111670 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5de40105b0 tx=0x7f5de4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:43:11.778 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:10.540+0000 7f5de27fc700 1 -- 172.21.15.107:0/3056858199 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5de4010840 con 0x7f5df00712d0 2024-05-17T21:43:11.778 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:10.541+0000 7f5de27fc700 1 -- 172.21.15.107:0/3056858199 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5de4010e70 con 0x7f5df00712d0 2024-05-17T21:43:11.778 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:10.541+0000 7f5df7d2e700 1 -- 172.21.15.107:0/3056858199 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5df0111db0 con 0x7f5df00712d0 2024-05-17T21:43:11.778 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:10.541+0000 7f5de27fc700 1 -- 172.21.15.107:0/3056858199 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5de40243f0 con 0x7f5df00712d0 2024-05-17T21:43:11.778 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:10.541+0000 7f5df7d2e700 1 -- 172.21.15.107:0/3056858199 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5df0112920 con 0x7f5df00712d0 2024-05-17T21:43:11.778 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:10.541+0000 7f5de27fc700 1 -- 172.21.15.107:0/3056858199 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f5de4024550 con 0x7f5df00712d0 2024-05-17T21:43:11.778 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:10.541+0000 7f5de27fc700 1 --2- 172.21.15.107:0/3056858199 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7f5ddc036850 0x7f5ddc038d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:43:11.778 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:10.542+0000 7f5de27fc700 1 -- 172.21.15.107:0/3056858199 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f5de404a590 con 0x7f5df00712d0 2024-05-17T21:43:11.779 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:10.542+0000 7f5df7d2e700 1 -- 172.21.15.107:0/3056858199 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5df005f5c0 con 0x7f5df00712d0 2024-05-17T21:43:11.779 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:10.542+0000 7f5df52c9700 1 --2- 172.21.15.107:0/3056858199 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7f5ddc036850 0x7f5ddc038d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:43:11.779 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:10.542+0000 7f5df52c9700 1 --2- 172.21.15.107:0/3056858199 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7f5ddc036850 0x7f5ddc038d00 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f5dec009990 tx=0x7f5dec006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:43:11.779 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:10.546+0000 7f5de27fc700 1 -- 172.21.15.107:0/3056858199 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f5de4018940 con 0x7f5df00712d0 2024-05-17T21:43:11.779 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:10.751+0000 7f5df7d2e700 1 -- 172.21.15.107:0/3056858199 --> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}) v1 -- 0x7f5df00fe120 con 0x7f5ddc036850 2024-05-17T21:43:11.779 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:10.757+0000 7f5de27fc700 1 -- 172.21.15.107:0/3056858199 <== mgr.14120 v2:172.21.15.107:6800/1718632476 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+33 (secure 0 0 0) 0x7f5df00fe120 con 0x7f5ddc036850 2024-05-17T21:43:11.779 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:10.760+0000 7f5df7d2e700 1 -- 172.21.15.107:0/3056858199 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7f5ddc036850 msgr2=0x7f5ddc038d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:43:11.779 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:10.760+0000 7f5df7d2e700 1 --2- 172.21.15.107:0/3056858199 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7f5ddc036850 0x7f5ddc038d00 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f5dec009990 tx=0x7f5dec006e40 comp rx=0 tx=0).stop 2024-05-17T21:43:11.779 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:10.760+0000 7f5df7d2e700 1 -- 172.21.15.107:0/3056858199 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5df00712d0 msgr2=0x7f5df0111670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:43:11.779 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:10.760+0000 7f5df7d2e700 1 --2- 172.21.15.107:0/3056858199 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5df00712d0 0x7f5df0111670 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5de40105b0 tx=0x7f5de4009250 comp rx=0 tx=0).stop 2024-05-17T21:43:11.779 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:10.760+0000 7f5df7d2e700 1 -- 172.21.15.107:0/3056858199 shutdown_connections 2024-05-17T21:43:11.779 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:10.760+0000 7f5df7d2e700 1 --2- 172.21.15.107:0/3056858199 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7f5ddc036850 0x7f5ddc038d00 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:43:11.779 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:10.760+0000 7f5df7d2e700 1 --2- 172.21.15.107:0/3056858199 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5df00712d0 0x7f5df0111670 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:43:11.780 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:10.760+0000 7f5df7d2e700 1 -- 172.21.15.107:0/3056858199 >> 172.21.15.107:0/3056858199 conn(0x7f5df00fcea0 msgr2=0x7f5df00fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:43:11.780 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:10.760+0000 7f5df7d2e700 1 -- 172.21.15.107:0/3056858199 shutdown_connections 2024-05-17T21:43:11.780 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:10.760+0000 7f5df7d2e700 1 -- 172.21.15.107:0/3056858199 wait complete. 2024-05-17T21:43:12.956 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:12.217+0000 7f81cdea0700 1 Processor -- start 2024-05-17T21:43:12.957 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:12.217+0000 7f81cdea0700 1 -- start start 2024-05-17T21:43:12.957 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:12.218+0000 7f81cdea0700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f81c81028b0 0x7f81c8102cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:43:12.957 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:12.218+0000 7f81cdea0700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81c8103200 con 0x7f81c81028b0 2024-05-17T21:43:12.957 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:12.218+0000 7f81c77fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f81c81028b0 0x7f81c8102cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:43:12.957 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:12.218+0000 7f81c77fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f81c81028b0 0x7f81c8102cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54334/0 (socket says 172.21.15.107:54334) 2024-05-17T21:43:12.957 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:12.218+0000 7f81c77fe700 1 -- 172.21.15.107:0/1342799734 learned_addr learned my addr 172.21.15.107:0/1342799734 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:43:12.957 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:12.218+0000 7f81c77fe700 1 -- 172.21.15.107:0/1342799734 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81c8103340 con 0x7f81c81028b0 2024-05-17T21:43:12.957 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:12.218+0000 7f81c77fe700 1 --2- 172.21.15.107:0/1342799734 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f81c81028b0 0x7f81c8102cc0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f81b80098e0 tx=0x7f81b8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e205a215b79594b1 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:43:12.957 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:12.218+0000 7f81c6ffd700 1 -- 172.21.15.107:0/1342799734 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f81b800b860 con 0x7f81c81028b0 2024-05-17T21:43:12.957 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:12.218+0000 7f81c6ffd700 1 -- 172.21.15.107:0/1342799734 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f81b800be90 con 0x7f81c81028b0 2024-05-17T21:43:12.957 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:12.218+0000 7f81c6ffd700 1 -- 172.21.15.107:0/1342799734 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f81b80103c0 con 0x7f81c81028b0 2024-05-17T21:43:12.957 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:12.219+0000 7f81cdea0700 1 -- 172.21.15.107:0/1342799734 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f81c81028b0 msgr2=0x7f81c8102cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:43:12.958 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:12.219+0000 7f81cdea0700 1 --2- 172.21.15.107:0/1342799734 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f81c81028b0 0x7f81c8102cc0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f81b80098e0 tx=0x7f81b8009250 comp rx=0 tx=0).stop 2024-05-17T21:43:12.958 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:12.219+0000 7f81cdea0700 1 -- 172.21.15.107:0/1342799734 shutdown_connections 2024-05-17T21:43:12.958 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:12.219+0000 7f81cdea0700 1 --2- 172.21.15.107:0/1342799734 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f81c81028b0 0x7f81c8102cc0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:43:12.958 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:12.219+0000 7f81cdea0700 1 -- 172.21.15.107:0/1342799734 >> 172.21.15.107:0/1342799734 conn(0x7f81c80fe070 msgr2=0x7f81c8100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:43:12.958 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:12.219+0000 7f81cdea0700 1 -- 172.21.15.107:0/1342799734 shutdown_connections 2024-05-17T21:43:12.958 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:12.219+0000 7f81cdea0700 1 -- 172.21.15.107:0/1342799734 wait complete. 2024-05-17T21:43:12.958 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:12.220+0000 7f81cdea0700 1 Processor -- start 2024-05-17T21:43:12.958 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:12.220+0000 7f81cdea0700 1 -- start start 2024-05-17T21:43:12.958 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:12.220+0000 7f81cdea0700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f81c81028b0 0x7f81c8078c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:43:12.958 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:12.220+0000 7f81cdea0700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81c8079160 con 0x7f81c81028b0 2024-05-17T21:43:12.958 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:12.220+0000 7f81c77fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f81c81028b0 0x7f81c8078c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:43:12.958 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:12.220+0000 7f81c77fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f81c81028b0 0x7f81c8078c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54350/0 (socket says 172.21.15.107:54350) 2024-05-17T21:43:12.958 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:12.220+0000 7f81c77fe700 1 -- 172.21.15.107:0/683748235 learned_addr learned my addr 172.21.15.107:0/683748235 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:43:12.958 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:12.220+0000 7f81c77fe700 1 -- 172.21.15.107:0/683748235 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81b80094f0 con 0x7f81c81028b0 2024-05-17T21:43:12.959 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:12.220+0000 7f81c77fe700 1 --2- 172.21.15.107:0/683748235 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f81c81028b0 0x7f81c8078c20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f81b80090b0 tx=0x7f81b8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:43:12.959 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:12.220+0000 7f81c4ff9700 1 -- 172.21.15.107:0/683748235 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f81b8010840 con 0x7f81c81028b0 2024-05-17T21:43:12.959 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:12.220+0000 7f81c4ff9700 1 -- 172.21.15.107:0/683748235 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f81b8010e70 con 0x7f81c81028b0 2024-05-17T21:43:12.959 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:12.220+0000 7f81cdea0700 1 -- 172.21.15.107:0/683748235 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81c80771d0 con 0x7f81c81028b0 2024-05-17T21:43:12.959 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:12.220+0000 7f81c4ff9700 1 -- 172.21.15.107:0/683748235 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f81b8024e90 con 0x7f81c81028b0 2024-05-17T21:43:12.959 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:12.221+0000 7f81cdea0700 1 -- 172.21.15.107:0/683748235 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81c80776c0 con 0x7f81c81028b0 2024-05-17T21:43:12.959 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:12.221+0000 7f81c4ff9700 1 -- 172.21.15.107:0/683748235 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f81b801a510 con 0x7f81c81028b0 2024-05-17T21:43:12.959 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:12.222+0000 7f81c4ff9700 1 --2- 172.21.15.107:0/683748235 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7f81b4036800 0x7f81b4038cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:43:12.959 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:12.222+0000 7f81cdea0700 1 -- 172.21.15.107:0/683748235 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81c804bed0 con 0x7f81c81028b0 2024-05-17T21:43:12.959 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:12.222+0000 7f81c4ff9700 1 -- 172.21.15.107:0/683748235 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f81b804a8f0 con 0x7f81c81028b0 2024-05-17T21:43:12.959 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:12.222+0000 7f81bffff700 1 --2- 172.21.15.107:0/683748235 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7f81b4036800 0x7f81b4038cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:43:12.959 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:12.222+0000 7f81bffff700 1 --2- 172.21.15.107:0/683748235 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7f81b4036800 0x7f81b4038cb0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f81b0009990 tx=0x7f81b0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:43:12.959 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:12.226+0000 7f81c4ff9700 1 -- 172.21.15.107:0/683748235 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f81b8018530 con 0x7f81c81028b0 2024-05-17T21:43:12.959 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:12.414+0000 7f81cdea0700 1 -- 172.21.15.107:0/683748235 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f81c8104650 con 0x7f81c81028b0 2024-05-17T21:43:12.960 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:12.418+0000 7f81c4ff9700 1 -- 172.21.15.107:0/683748235 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7f81b8018350 con 0x7f81c81028b0 2024-05-17T21:43:12.960 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:12.421+0000 7f81cdea0700 1 -- 172.21.15.107:0/683748235 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7f81b4036800 msgr2=0x7f81b4038cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:43:12.960 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:12.421+0000 7f81cdea0700 1 --2- 172.21.15.107:0/683748235 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7f81b4036800 0x7f81b4038cb0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f81b0009990 tx=0x7f81b0006e40 comp rx=0 tx=0).stop 2024-05-17T21:43:12.960 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:12.421+0000 7f81cdea0700 1 -- 172.21.15.107:0/683748235 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f81c81028b0 msgr2=0x7f81c8078c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:43:12.960 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:12.421+0000 7f81cdea0700 1 --2- 172.21.15.107:0/683748235 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f81c81028b0 0x7f81c8078c20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f81b80090b0 tx=0x7f81b8009250 comp rx=0 tx=0).stop 2024-05-17T21:43:12.960 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:12.421+0000 7f81cdea0700 1 -- 172.21.15.107:0/683748235 shutdown_connections 2024-05-17T21:43:12.960 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:12.421+0000 7f81cdea0700 1 --2- 172.21.15.107:0/683748235 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7f81b4036800 0x7f81b4038cb0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:43:12.960 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:12.421+0000 7f81cdea0700 1 --2- 172.21.15.107:0/683748235 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f81c81028b0 0x7f81c8078c20 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:43:12.960 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:12.421+0000 7f81cdea0700 1 -- 172.21.15.107:0/683748235 >> 172.21.15.107:0/683748235 conn(0x7f81c80fe070 msgr2=0x7f81c80feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:43:12.960 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:12.421+0000 7f81cdea0700 1 -- 172.21.15.107:0/683748235 shutdown_connections 2024-05-17T21:43:12.960 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:12.421+0000 7f81cdea0700 1 -- 172.21.15.107:0/683748235 wait complete. 2024-05-17T21:43:13.594 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:13 smithi107 ceph-mon[28434]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T21:43:13.594 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:13 smithi107 ceph-mon[28434]: from='client.? 172.21.15.107:0/683748235' entity='client.admin' 2024-05-17T21:43:14.196 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:13.406+0000 7f6cd5530700 1 Processor -- start 2024-05-17T21:43:14.196 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:13.406+0000 7f6cd5530700 1 -- start start 2024-05-17T21:43:14.196 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:13.407+0000 7f6cd5530700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6cd0102880 0x7f6cd0102c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:43:14.196 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:13.407+0000 7f6cd5530700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd01031d0 con 0x7f6cd0102880 2024-05-17T21:43:14.196 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:13.407+0000 7f6cceffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6cd0102880 0x7f6cd0102c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:43:14.196 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:13.407+0000 7f6cceffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6cd0102880 0x7f6cd0102c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54366/0 (socket says 172.21.15.107:54366) 2024-05-17T21:43:14.196 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:13.407+0000 7f6cceffd700 1 -- 172.21.15.107:0/1638684278 learned_addr learned my addr 172.21.15.107:0/1638684278 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:43:14.197 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:13.407+0000 7f6cceffd700 1 -- 172.21.15.107:0/1638684278 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cd0103310 con 0x7f6cd0102880 2024-05-17T21:43:14.197 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:13.407+0000 7f6cceffd700 1 --2- 172.21.15.107:0/1638684278 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6cd0102880 0x7f6cd0102c90 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6cc00098e0 tx=0x7f6cc0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e032c39348c47999 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:43:14.197 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:13.407+0000 7f6ccdffb700 1 -- 172.21.15.107:0/1638684278 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6cc000b860 con 0x7f6cd0102880 2024-05-17T21:43:14.197 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:13.407+0000 7f6ccdffb700 1 -- 172.21.15.107:0/1638684278 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6cc000be90 con 0x7f6cd0102880 2024-05-17T21:43:14.197 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:13.407+0000 7f6ccdffb700 1 -- 172.21.15.107:0/1638684278 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6cc00103c0 con 0x7f6cd0102880 2024-05-17T21:43:14.197 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:13.408+0000 7f6cd5530700 1 -- 172.21.15.107:0/1638684278 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6cd0102880 msgr2=0x7f6cd0102c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:43:14.197 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:13.408+0000 7f6cd5530700 1 --2- 172.21.15.107:0/1638684278 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6cd0102880 0x7f6cd0102c90 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6cc00098e0 tx=0x7f6cc0009250 comp rx=0 tx=0).stop 2024-05-17T21:43:14.197 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:13.408+0000 7f6cd5530700 1 -- 172.21.15.107:0/1638684278 shutdown_connections 2024-05-17T21:43:14.197 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:13.408+0000 7f6cd5530700 1 --2- 172.21.15.107:0/1638684278 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6cd0102880 0x7f6cd0102c90 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:43:14.197 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:13.408+0000 7f6cd5530700 1 -- 172.21.15.107:0/1638684278 >> 172.21.15.107:0/1638684278 conn(0x7f6cd00fe060 msgr2=0x7f6cd0100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:43:14.197 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:13.408+0000 7f6cd5530700 1 -- 172.21.15.107:0/1638684278 shutdown_connections 2024-05-17T21:43:14.197 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:13.408+0000 7f6cd5530700 1 -- 172.21.15.107:0/1638684278 wait complete. 2024-05-17T21:43:14.198 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:13.409+0000 7f6cd5530700 1 Processor -- start 2024-05-17T21:43:14.198 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:13.409+0000 7f6cd5530700 1 -- start start 2024-05-17T21:43:14.198 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:13.409+0000 7f6cd5530700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6cd0102880 0x7f6cd0111620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:43:14.198 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:13.409+0000 7f6cd5530700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd0111b60 con 0x7f6cd0102880 2024-05-17T21:43:14.198 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:13.409+0000 7f6cceffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6cd0102880 0x7f6cd0111620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:43:14.198 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:13.409+0000 7f6cceffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6cd0102880 0x7f6cd0111620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54370/0 (socket says 172.21.15.107:54370) 2024-05-17T21:43:14.198 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:13.409+0000 7f6cceffd700 1 -- 172.21.15.107:0/1261237408 learned_addr learned my addr 172.21.15.107:0/1261237408 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:43:14.198 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:13.409+0000 7f6cceffd700 1 -- 172.21.15.107:0/1261237408 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cc00094f0 con 0x7f6cd0102880 2024-05-17T21:43:14.198 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:13.409+0000 7f6cceffd700 1 --2- 172.21.15.107:0/1261237408 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6cd0102880 0x7f6cd0111620 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6cc0011680 tx=0x7f6cc0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:43:14.198 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:13.409+0000 7f6cb77fe700 1 -- 172.21.15.107:0/1261237408 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6cc0010890 con 0x7f6cd0102880 2024-05-17T21:43:14.198 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:13.410+0000 7f6cb77fe700 1 -- 172.21.15.107:0/1261237408 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6cc0010ec0 con 0x7f6cd0102880 2024-05-17T21:43:14.198 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:13.410+0000 7f6cb77fe700 1 -- 172.21.15.107:0/1261237408 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6cc00243f0 con 0x7f6cd0102880 2024-05-17T21:43:14.198 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:13.410+0000 7f6cd5530700 1 -- 172.21.15.107:0/1261237408 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cd0111d60 con 0x7f6cd0102880 2024-05-17T21:43:14.199 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:13.410+0000 7f6cd5530700 1 -- 172.21.15.107:0/1261237408 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cd01128d0 con 0x7f6cd0102880 2024-05-17T21:43:14.199 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:13.410+0000 7f6cb77fe700 1 -- 172.21.15.107:0/1261237408 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f6cc002d3f0 con 0x7f6cd0102880 2024-05-17T21:43:14.199 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:13.410+0000 7f6cd5530700 1 -- 172.21.15.107:0/1261237408 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cd005f5c0 con 0x7f6cd0102880 2024-05-17T21:43:14.199 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:13.411+0000 7f6cb77fe700 1 --2- 172.21.15.107:0/1261237408 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7f6cb8036850 0x7f6cb8038d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:43:14.202 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:13.411+0000 7f6cb77fe700 1 -- 172.21.15.107:0/1261237408 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f6cc004a3b0 con 0x7f6cd0102880 2024-05-17T21:43:14.202 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:13.411+0000 7f6cce7fc700 1 --2- 172.21.15.107:0/1261237408 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7f6cb8036850 0x7f6cb8038d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:43:14.202 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:13.411+0000 7f6cce7fc700 1 --2- 172.21.15.107:0/1261237408 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7f6cb8036850 0x7f6cb8038d00 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6cc4009990 tx=0x7f6cc4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:43:14.202 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:13.415+0000 7f6cb77fe700 1 -- 172.21.15.107:0/1261237408 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f6cc0018ce0 con 0x7f6cd0102880 2024-05-17T21:43:14.202 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:13.599+0000 7f6cd5530700 1 -- 172.21.15.107:0/1261237408 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f6cd004bed0 con 0x7f6cd0102880 2024-05-17T21:43:14.202 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:13.603+0000 7f6cb77fe700 1 -- 172.21.15.107:0/1261237408 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f6cc0006d70 con 0x7f6cd0102880 2024-05-17T21:43:14.202 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:13.605+0000 7f6cd5530700 1 -- 172.21.15.107:0/1261237408 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7f6cb8036850 msgr2=0x7f6cb8038d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:43:14.202 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:13.605+0000 7f6cd5530700 1 --2- 172.21.15.107:0/1261237408 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7f6cb8036850 0x7f6cb8038d00 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6cc4009990 tx=0x7f6cc4006e40 comp rx=0 tx=0).stop 2024-05-17T21:43:14.202 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:13.605+0000 7f6cd5530700 1 -- 172.21.15.107:0/1261237408 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6cd0102880 msgr2=0x7f6cd0111620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:43:14.202 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:13.605+0000 7f6cd5530700 1 --2- 172.21.15.107:0/1261237408 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6cd0102880 0x7f6cd0111620 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6cc0011680 tx=0x7f6cc0009250 comp rx=0 tx=0).stop 2024-05-17T21:43:14.202 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:13.605+0000 7f6cd5530700 1 -- 172.21.15.107:0/1261237408 shutdown_connections 2024-05-17T21:43:14.202 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:13.605+0000 7f6cd5530700 1 --2- 172.21.15.107:0/1261237408 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7f6cb8036850 0x7f6cb8038d00 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:43:14.203 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:13.605+0000 7f6cd5530700 1 --2- 172.21.15.107:0/1261237408 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6cd0102880 0x7f6cd0111620 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:43:14.203 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:13.605+0000 7f6cd5530700 1 -- 172.21.15.107:0/1261237408 >> 172.21.15.107:0/1261237408 conn(0x7f6cd00fe060 msgr2=0x7f6cd00feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:43:14.203 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:13.605+0000 7f6cd5530700 1 -- 172.21.15.107:0/1261237408 shutdown_connections 2024-05-17T21:43:14.203 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:13.605+0000 7f6cd5530700 1 -- 172.21.15.107:0/1261237408 wait complete. 2024-05-17T21:43:14.203 INFO:teuthology.orchestra.run.smithi107.stdout:Enabling the dashboard module... 2024-05-17T21:43:14.829 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:14 smithi107 ceph-mon[28434]: from='client.? 172.21.15.107:0/1261237408' entity='client.admin' 2024-05-17T21:43:15.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:15 smithi107 ceph-mon[28434]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T21:43:15.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:15 smithi107 ceph-mon[28434]: from='client.? 172.21.15.107:0/1203834316' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-05-17T21:43:16.227 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:14.585+0000 7f9347d6e700 1 Processor -- start 2024-05-17T21:43:16.227 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:14.585+0000 7f9347d6e700 1 -- start start 2024-05-17T21:43:16.227 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:14.586+0000 7f9347d6e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9340101730 0x7f9340101b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:43:16.227 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:14.586+0000 7f9347d6e700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9340102080 con 0x7f9340101730 2024-05-17T21:43:16.227 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:14.586+0000 7f9345b0a700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9340101730 0x7f9340101b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:43:16.227 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:14.586+0000 7f9345b0a700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9340101730 0x7f9340101b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54380/0 (socket says 172.21.15.107:54380) 2024-05-17T21:43:16.227 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:14.586+0000 7f9345b0a700 1 -- 172.21.15.107:0/2693020648 learned_addr learned my addr 172.21.15.107:0/2693020648 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:43:16.227 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:14.586+0000 7f9345b0a700 1 -- 172.21.15.107:0/2693020648 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93401021c0 con 0x7f9340101730 2024-05-17T21:43:16.227 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:14.586+0000 7f9345b0a700 1 --2- 172.21.15.107:0/2693020648 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9340101730 0x7f9340101b40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f933c0098e0 tx=0x7f933c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d451794c90a29fe3 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:43:16.227 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:14.586+0000 7f9344b08700 1 -- 172.21.15.107:0/2693020648 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f933c00b860 con 0x7f9340101730 2024-05-17T21:43:16.227 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:14.586+0000 7f9344b08700 1 -- 172.21.15.107:0/2693020648 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f933c00be90 con 0x7f9340101730 2024-05-17T21:43:16.228 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:14.586+0000 7f9344b08700 1 -- 172.21.15.107:0/2693020648 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f933c0103c0 con 0x7f9340101730 2024-05-17T21:43:16.228 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:14.587+0000 7f9347d6e700 1 -- 172.21.15.107:0/2693020648 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9340101730 msgr2=0x7f9340101b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:43:16.228 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:14.587+0000 7f9347d6e700 1 --2- 172.21.15.107:0/2693020648 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9340101730 0x7f9340101b40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f933c0098e0 tx=0x7f933c009250 comp rx=0 tx=0).stop 2024-05-17T21:43:16.228 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:14.587+0000 7f9347d6e700 1 -- 172.21.15.107:0/2693020648 shutdown_connections 2024-05-17T21:43:16.228 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:14.587+0000 7f9347d6e700 1 --2- 172.21.15.107:0/2693020648 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9340101730 0x7f9340101b40 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:43:16.228 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:14.587+0000 7f9347d6e700 1 -- 172.21.15.107:0/2693020648 >> 172.21.15.107:0/2693020648 conn(0x7f93400fcef0 msgr2=0x7f93400ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:43:16.228 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:14.587+0000 7f9347d6e700 1 -- 172.21.15.107:0/2693020648 shutdown_connections 2024-05-17T21:43:16.228 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:14.587+0000 7f9347d6e700 1 -- 172.21.15.107:0/2693020648 wait complete. 2024-05-17T21:43:16.228 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:14.587+0000 7f9347d6e700 1 Processor -- start 2024-05-17T21:43:16.228 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:14.587+0000 7f9347d6e700 1 -- start start 2024-05-17T21:43:16.228 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:14.588+0000 7f9347d6e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9340101730 0x7f9340110530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:43:16.228 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:14.588+0000 7f9347d6e700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9340110a70 con 0x7f9340101730 2024-05-17T21:43:16.228 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:14.588+0000 7f9345b0a700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9340101730 0x7f9340110530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:43:16.228 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:14.588+0000 7f9345b0a700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9340101730 0x7f9340110530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54386/0 (socket says 172.21.15.107:54386) 2024-05-17T21:43:16.229 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:14.588+0000 7f9345b0a700 1 -- 172.21.15.107:0/1203834316 learned_addr learned my addr 172.21.15.107:0/1203834316 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:43:16.229 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:14.588+0000 7f9345b0a700 1 -- 172.21.15.107:0/1203834316 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f933c0094f0 con 0x7f9340101730 2024-05-17T21:43:16.229 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:14.588+0000 7f9345b0a700 1 --2- 172.21.15.107:0/1203834316 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9340101730 0x7f9340110530 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f933c011680 tx=0x7f933c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:43:16.229 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:14.588+0000 7f93367fc700 1 -- 172.21.15.107:0/1203834316 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f933c010890 con 0x7f9340101730 2024-05-17T21:43:16.229 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:14.588+0000 7f93367fc700 1 -- 172.21.15.107:0/1203834316 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f933c010ec0 con 0x7f9340101730 2024-05-17T21:43:16.229 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:14.588+0000 7f9347d6e700 1 -- 172.21.15.107:0/1203834316 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9340110c70 con 0x7f9340101730 2024-05-17T21:43:16.229 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:14.588+0000 7f93367fc700 1 -- 172.21.15.107:0/1203834316 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f933c0243f0 con 0x7f9340101730 2024-05-17T21:43:16.229 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:14.588+0000 7f9347d6e700 1 -- 172.21.15.107:0/1203834316 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93401117e0 con 0x7f9340101730 2024-05-17T21:43:16.229 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:14.589+0000 7f93367fc700 1 -- 172.21.15.107:0/1203834316 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f933c02d3f0 con 0x7f9340101730 2024-05-17T21:43:16.229 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:14.589+0000 7f9347d6e700 1 -- 172.21.15.107:0/1203834316 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f934005f5c0 con 0x7f9340101730 2024-05-17T21:43:16.229 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:14.589+0000 7f93367fc700 1 --2- 172.21.15.107:0/1203834316 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7f932c036850 0x7f932c038d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:43:16.229 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:14.589+0000 7f93367fc700 1 -- 172.21.15.107:0/1203834316 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f933c04a3d0 con 0x7f9340101730 2024-05-17T21:43:16.229 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:14.589+0000 7f9345309700 1 --2- 172.21.15.107:0/1203834316 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7f932c036850 0x7f932c038d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:43:16.229 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:14.590+0000 7f9345309700 1 --2- 172.21.15.107:0/1203834316 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7f932c036850 0x7f932c038d00 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f9330009990 tx=0x7f9330006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:43:16.230 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:14.594+0000 7f93367fc700 1 -- 172.21.15.107:0/1203834316 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f933c018b40 con 0x7f9340101730 2024-05-17T21:43:16.230 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:14.834+0000 7f9347d6e700 1 -- 172.21.15.107:0/1203834316 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f934004bed0 con 0x7f9340101730 2024-05-17T21:43:16.230 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:15.607+0000 7f93367fc700 1 -- 172.21.15.107:0/1203834316 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mgrmap(e 9) v1 ==== 43389+0+0 (secure 0 0 0) 0x7f933c010c10 con 0x7f9340101730 2024-05-17T21:43:16.230 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:15.608+0000 7f93367fc700 1 -- 172.21.15.107:0/1203834316 <== mon.0 v2:172.21.15.107:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f933c04ae20 con 0x7f9340101730 2024-05-17T21:43:16.230 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:15.612+0000 7f9347d6e700 1 -- 172.21.15.107:0/1203834316 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7f932c036850 msgr2=0x7f932c038d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:43:16.230 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:15.612+0000 7f9347d6e700 1 --2- 172.21.15.107:0/1203834316 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7f932c036850 0x7f932c038d00 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f9330009990 tx=0x7f9330006e40 comp rx=0 tx=0).stop 2024-05-17T21:43:16.230 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:15.612+0000 7f9347d6e700 1 -- 172.21.15.107:0/1203834316 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9340101730 msgr2=0x7f9340110530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:43:16.230 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:15.612+0000 7f9347d6e700 1 --2- 172.21.15.107:0/1203834316 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9340101730 0x7f9340110530 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f933c011680 tx=0x7f933c009250 comp rx=0 tx=0).stop 2024-05-17T21:43:16.230 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:15.612+0000 7f9347d6e700 1 -- 172.21.15.107:0/1203834316 shutdown_connections 2024-05-17T21:43:16.230 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:15.612+0000 7f9347d6e700 1 --2- 172.21.15.107:0/1203834316 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7f932c036850 0x7f932c038d00 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:43:16.230 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:15.612+0000 7f9347d6e700 1 --2- 172.21.15.107:0/1203834316 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9340101730 0x7f9340110530 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:43:16.230 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:15.612+0000 7f9347d6e700 1 -- 172.21.15.107:0/1203834316 >> 172.21.15.107:0/1203834316 conn(0x7f93400fcef0 msgr2=0x7f93400fd9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:43:16.230 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:15.612+0000 7f9347d6e700 1 -- 172.21.15.107:0/1203834316 shutdown_connections 2024-05-17T21:43:16.230 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:15.612+0000 7f9347d6e700 1 -- 172.21.15.107:0/1203834316 wait complete. 2024-05-17T21:43:16.844 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:16 smithi107 ceph-mon[28434]: from='client.? 172.21.15.107:0/1203834316' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-05-17T21:43:16.845 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:16 smithi107 ceph-mon[28434]: mgrmap e9: smithi107.xhhyiw(active, since 23s) 2024-05-17T21:43:17.470 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout { 2024-05-17T21:43:17.470 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-05-17T21:43:17.470 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-17T21:43:17.470 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "active_name": "smithi107.xhhyiw", 2024-05-17T21:43:17.470 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-17T21:43:17.470 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout } 2024-05-17T21:43:17.470 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:16.644+0000 7f9bdb0eb700 1 Processor -- start 2024-05-17T21:43:17.470 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:16.644+0000 7f9bdb0eb700 1 -- start start 2024-05-17T21:43:17.471 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:16.644+0000 7f9bdb0eb700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9bd41017f0 0x7f9bd4101c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:43:17.471 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:16.644+0000 7f9bdb0eb700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd4102140 con 0x7f9bd41017f0 2024-05-17T21:43:17.471 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:16.645+0000 7f9bd8e87700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9bd41017f0 0x7f9bd4101c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:43:17.471 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:16.645+0000 7f9bd8e87700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9bd41017f0 0x7f9bd4101c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56596/0 (socket says 172.21.15.107:56596) 2024-05-17T21:43:17.471 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:16.645+0000 7f9bd8e87700 1 -- 172.21.15.107:0/2138761700 learned_addr learned my addr 172.21.15.107:0/2138761700 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:43:17.471 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:16.645+0000 7f9bd8e87700 1 -- 172.21.15.107:0/2138761700 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bd4102280 con 0x7f9bd41017f0 2024-05-17T21:43:17.471 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:16.645+0000 7f9bd8e87700 1 --2- 172.21.15.107:0/2138761700 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9bd41017f0 0x7f9bd4101c00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9bc80098e0 tx=0x7f9bc8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c42794aef4931ca server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:43:17.471 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:16.645+0000 7f9bd37fe700 1 -- 172.21.15.107:0/2138761700 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9bc800b860 con 0x7f9bd41017f0 2024-05-17T21:43:17.471 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:16.645+0000 7f9bd37fe700 1 -- 172.21.15.107:0/2138761700 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9bc800be90 con 0x7f9bd41017f0 2024-05-17T21:43:17.471 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:16.645+0000 7f9bd37fe700 1 -- 172.21.15.107:0/2138761700 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9bc80103c0 con 0x7f9bd41017f0 2024-05-17T21:43:17.471 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:16.646+0000 7f9bdb0eb700 1 -- 172.21.15.107:0/2138761700 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9bd41017f0 msgr2=0x7f9bd4101c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:43:17.471 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:16.646+0000 7f9bdb0eb700 1 --2- 172.21.15.107:0/2138761700 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9bd41017f0 0x7f9bd4101c00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9bc80098e0 tx=0x7f9bc8009250 comp rx=0 tx=0).stop 2024-05-17T21:43:17.471 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:16.646+0000 7f9bdb0eb700 1 -- 172.21.15.107:0/2138761700 shutdown_connections 2024-05-17T21:43:17.472 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:16.646+0000 7f9bdb0eb700 1 --2- 172.21.15.107:0/2138761700 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9bd41017f0 0x7f9bd4101c00 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:43:17.472 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:16.646+0000 7f9bdb0eb700 1 -- 172.21.15.107:0/2138761700 >> 172.21.15.107:0/2138761700 conn(0x7f9bd40fcf50 msgr2=0x7f9bd40ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:43:17.472 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:16.646+0000 7f9bdb0eb700 1 -- 172.21.15.107:0/2138761700 shutdown_connections 2024-05-17T21:43:17.472 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:16.646+0000 7f9bdb0eb700 1 -- 172.21.15.107:0/2138761700 wait complete. 2024-05-17T21:43:17.472 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:16.646+0000 7f9bdb0eb700 1 Processor -- start 2024-05-17T21:43:17.472 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:16.646+0000 7f9bdb0eb700 1 -- start start 2024-05-17T21:43:17.472 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:16.647+0000 7f9bdb0eb700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9bd41017f0 0x7f9bd41137c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:43:17.472 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:16.647+0000 7f9bdb0eb700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd4113d00 con 0x7f9bd41017f0 2024-05-17T21:43:17.472 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:16.647+0000 7f9bd8e87700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9bd41017f0 0x7f9bd41137c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:43:17.472 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:16.647+0000 7f9bd8e87700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9bd41017f0 0x7f9bd41137c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56600/0 (socket says 172.21.15.107:56600) 2024-05-17T21:43:17.472 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:16.647+0000 7f9bd8e87700 1 -- 172.21.15.107:0/1870122771 learned_addr learned my addr 172.21.15.107:0/1870122771 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:43:17.472 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:16.647+0000 7f9bd8e87700 1 -- 172.21.15.107:0/1870122771 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bc80094f0 con 0x7f9bd41017f0 2024-05-17T21:43:17.473 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:16.647+0000 7f9bd8e87700 1 --2- 172.21.15.107:0/1870122771 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9bd41017f0 0x7f9bd41137c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9bc8011680 tx=0x7f9bc8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:43:17.473 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:16.647+0000 7f9bd17fa700 1 -- 172.21.15.107:0/1870122771 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9bc8010890 con 0x7f9bd41017f0 2024-05-17T21:43:17.473 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:16.648+0000 7f9bd17fa700 1 -- 172.21.15.107:0/1870122771 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9bc8010ec0 con 0x7f9bd41017f0 2024-05-17T21:43:17.473 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:16.648+0000 7f9bdb0eb700 1 -- 172.21.15.107:0/1870122771 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bd4113f00 con 0x7f9bd41017f0 2024-05-17T21:43:17.473 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:16.648+0000 7f9bd17fa700 1 -- 172.21.15.107:0/1870122771 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9bc80243f0 con 0x7f9bd41017f0 2024-05-17T21:43:17.473 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:16.648+0000 7f9bdb0eb700 1 -- 172.21.15.107:0/1870122771 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bd4114a70 con 0x7f9bd41017f0 2024-05-17T21:43:17.473 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:16.648+0000 7f9bd17fa700 1 -- 172.21.15.107:0/1870122771 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 9) v1 ==== 43389+0+0 (secure 0 0 0) 0x7f9bc802d3f0 con 0x7f9bd41017f0 2024-05-17T21:43:17.473 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:16.649+0000 7f9bdb0eb700 1 -- 172.21.15.107:0/1870122771 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bd405f5c0 con 0x7f9bd41017f0 2024-05-17T21:43:17.473 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:16.649+0000 7f9bd17fa700 1 --2- 172.21.15.107:0/1870122771 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7f9bc4036890 0x7f9bc4038d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:43:17.473 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:16.649+0000 7f9bd3fff700 1 -- 172.21.15.107:0/1870122771 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7f9bc4036890 msgr2=0x7f9bc4038d40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/1718632476 2024-05-17T21:43:17.473 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:16.649+0000 7f9bd17fa700 1 -- 172.21.15.107:0/1870122771 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f9bc804a2e0 con 0x7f9bd41017f0 2024-05-17T21:43:17.473 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:16.649+0000 7f9bd3fff700 1 --2- 172.21.15.107:0/1870122771 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7f9bc4036890 0x7f9bc4038d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-17T21:43:17.473 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:16.653+0000 7f9bd17fa700 1 -- 172.21.15.107:0/1870122771 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f9bc80193c0 con 0x7f9bd41017f0 2024-05-17T21:43:17.474 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:16.850+0000 7f9bd3fff700 1 -- 172.21.15.107:0/1870122771 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7f9bc4036890 msgr2=0x7f9bc4038d40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/1718632476 2024-05-17T21:43:17.474 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:16.850+0000 7f9bd3fff700 1 --2- 172.21.15.107:0/1870122771 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7f9bc4036890 0x7f9bc4038d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-17T21:43:17.474 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:16.931+0000 7f9bdb0eb700 1 -- 172.21.15.107:0/1870122771 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f9bd4115580 con 0x7f9bd41017f0 2024-05-17T21:43:17.474 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:16.932+0000 7f9bd17fa700 1 -- 172.21.15.107:0/1870122771 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+103 (secure 0 0 0) 0x7f9bc8006d70 con 0x7f9bd41017f0 2024-05-17T21:43:17.474 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:16.934+0000 7f9bdb0eb700 1 -- 172.21.15.107:0/1870122771 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7f9bc4036890 msgr2=0x7f9bc4038d40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-17T21:43:17.474 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:16.934+0000 7f9bdb0eb700 1 --2- 172.21.15.107:0/1870122771 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7f9bc4036890 0x7f9bc4038d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:43:17.474 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:16.934+0000 7f9bdb0eb700 1 -- 172.21.15.107:0/1870122771 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9bd41017f0 msgr2=0x7f9bd41137c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:43:17.474 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:16.934+0000 7f9bdb0eb700 1 --2- 172.21.15.107:0/1870122771 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9bd41017f0 0x7f9bd41137c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9bc8011680 tx=0x7f9bc8009250 comp rx=0 tx=0).stop 2024-05-17T21:43:17.474 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:16.934+0000 7f9bdb0eb700 1 -- 172.21.15.107:0/1870122771 shutdown_connections 2024-05-17T21:43:17.474 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:16.934+0000 7f9bdb0eb700 1 --2- 172.21.15.107:0/1870122771 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7f9bc4036890 0x7f9bc4038d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:43:17.474 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:16.934+0000 7f9bdb0eb700 1 --2- 172.21.15.107:0/1870122771 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9bd41017f0 0x7f9bd41137c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:43:17.474 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:16.934+0000 7f9bdb0eb700 1 -- 172.21.15.107:0/1870122771 >> 172.21.15.107:0/1870122771 conn(0x7f9bd40fcf50 msgr2=0x7f9bd4105c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:43:17.474 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:16.934+0000 7f9bdb0eb700 1 -- 172.21.15.107:0/1870122771 shutdown_connections 2024-05-17T21:43:17.475 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:16.934+0000 7f9bdb0eb700 1 -- 172.21.15.107:0/1870122771 wait complete. 2024-05-17T21:43:17.475 INFO:teuthology.orchestra.run.smithi107.stdout:Waiting for the mgr to restart... 2024-05-17T21:43:17.475 INFO:teuthology.orchestra.run.smithi107.stdout:Waiting for mgr epoch 9... 2024-05-17T21:43:17.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:17 smithi107 ceph-mon[28434]: from='client.? 172.21.15.107:0/1870122771' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-17T21:43:22.178 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:22 smithi107 ceph-mon[28434]: Active manager daemon smithi107.xhhyiw restarted 2024-05-17T21:43:22.179 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:22 smithi107 ceph-mon[28434]: Activating manager daemon smithi107.xhhyiw 2024-05-17T21:43:22.179 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:22 smithi107 ceph-mon[28434]: osdmap e3: 0 total, 0 up, 0 in 2024-05-17T21:43:22.179 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:22 smithi107 ceph-mon[28434]: mgrmap e10: smithi107.xhhyiw(active, starting, since 0.00315518s) 2024-05-17T21:43:22.179 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:22 smithi107 ceph-mon[28434]: from='mgr.14162 172.21.15.107:0/3666393387' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mon metadata", "id": "smithi107"}]: dispatch 2024-05-17T21:43:22.179 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:22 smithi107 ceph-mon[28434]: from='mgr.14162 172.21.15.107:0/3666393387' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mgr metadata", "who": "smithi107.xhhyiw", "id": "smithi107.xhhyiw"}]: dispatch 2024-05-17T21:43:22.179 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:22 smithi107 ceph-mon[28434]: from='mgr.14162 172.21.15.107:0/3666393387' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-17T21:43:22.179 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:22 smithi107 ceph-mon[28434]: from='mgr.14162 172.21.15.107:0/3666393387' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-17T21:43:22.179 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:22 smithi107 ceph-mon[28434]: from='mgr.14162 172.21.15.107:0/3666393387' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-17T21:43:22.179 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:22 smithi107 ceph-mon[28434]: Manager daemon smithi107.xhhyiw is now available 2024-05-17T21:43:22.179 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:22 smithi107 ceph-mon[28434]: from='mgr.14162 172.21.15.107:0/3666393387' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:43:22.179 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:22 smithi107 ceph-mon[28434]: from='mgr.14162 172.21.15.107:0/3666393387' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:43:22.179 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:22 smithi107 ceph-mon[28434]: from='mgr.14162 172.21.15.107:0/3666393387' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi107.xhhyiw/mirror_snapshot_schedule"}]: dispatch 2024-05-17T21:43:22.179 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:22 smithi107 ceph-mon[28434]: from='mgr.14162 172.21.15.107:0/3666393387' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi107.xhhyiw/trash_purge_schedule"}]: dispatch 2024-05-17T21:43:23.711 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout { 2024-05-17T21:43:23.711 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-05-17T21:43:23.711 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-17T21:43:23.711 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout } 2024-05-17T21:43:23.711 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:17.918+0000 7f65dac66700 1 Processor -- start 2024-05-17T21:43:23.711 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:17.918+0000 7f65dac66700 1 -- start start 2024-05-17T21:43:23.711 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:17.918+0000 7f65dac66700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f65d4078cc0 0x7f65d4077270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:43:23.711 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:17.918+0000 7f65dac66700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65d40790d0 con 0x7f65d4078cc0 2024-05-17T21:43:23.712 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:17.918+0000 7f65d8a02700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f65d4078cc0 0x7f65d4077270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:43:23.712 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:17.919+0000 7f65d8a02700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f65d4078cc0 0x7f65d4077270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56616/0 (socket says 172.21.15.107:56616) 2024-05-17T21:43:23.712 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:17.919+0000 7f65d8a02700 1 -- 172.21.15.107:0/1798156295 learned_addr learned my addr 172.21.15.107:0/1798156295 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:43:23.712 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:17.919+0000 7f65d8a02700 1 -- 172.21.15.107:0/1798156295 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65d40777b0 con 0x7f65d4078cc0 2024-05-17T21:43:23.712 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:17.919+0000 7f65d8a02700 1 --2- 172.21.15.107:0/1798156295 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f65d4078cc0 0x7f65d4077270 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f65c80098e0 tx=0x7f65c8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cea2f0d11c58fa8 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:43:23.712 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:17.919+0000 7f65d37fe700 1 -- 172.21.15.107:0/1798156295 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f65c800b860 con 0x7f65d4078cc0 2024-05-17T21:43:23.712 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:17.919+0000 7f65d37fe700 1 -- 172.21.15.107:0/1798156295 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f65c800be90 con 0x7f65d4078cc0 2024-05-17T21:43:23.712 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:17.919+0000 7f65d37fe700 1 -- 172.21.15.107:0/1798156295 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f65c80103c0 con 0x7f65d4078cc0 2024-05-17T21:43:23.712 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:17.920+0000 7f65dac66700 1 -- 172.21.15.107:0/1798156295 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f65d4078cc0 msgr2=0x7f65d4077270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:43:23.712 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:17.920+0000 7f65dac66700 1 --2- 172.21.15.107:0/1798156295 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f65d4078cc0 0x7f65d4077270 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f65c80098e0 tx=0x7f65c8009250 comp rx=0 tx=0).stop 2024-05-17T21:43:23.712 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:17.920+0000 7f65dac66700 1 -- 172.21.15.107:0/1798156295 shutdown_connections 2024-05-17T21:43:23.712 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:17.920+0000 7f65dac66700 1 --2- 172.21.15.107:0/1798156295 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f65d4078cc0 0x7f65d4077270 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:43:23.712 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:17.920+0000 7f65dac66700 1 -- 172.21.15.107:0/1798156295 >> 172.21.15.107:0/1798156295 conn(0x7f65d40fe010 msgr2=0x7f65d4100420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:43:23.713 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:17.920+0000 7f65dac66700 1 -- 172.21.15.107:0/1798156295 shutdown_connections 2024-05-17T21:43:23.713 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:17.920+0000 7f65dac66700 1 -- 172.21.15.107:0/1798156295 wait complete. 2024-05-17T21:43:23.713 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:17.921+0000 7f65dac66700 1 Processor -- start 2024-05-17T21:43:23.713 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:17.921+0000 7f65dac66700 1 -- start start 2024-05-17T21:43:23.713 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:17.921+0000 7f65dac66700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f65d4078cc0 0x7f65d4113810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:43:23.713 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:17.921+0000 7f65dac66700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65d4113d50 con 0x7f65d4078cc0 2024-05-17T21:43:23.713 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:17.921+0000 7f65d8a02700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f65d4078cc0 0x7f65d4113810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:43:23.713 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:17.921+0000 7f65d8a02700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f65d4078cc0 0x7f65d4113810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56630/0 (socket says 172.21.15.107:56630) 2024-05-17T21:43:23.713 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:17.921+0000 7f65d8a02700 1 -- 172.21.15.107:0/2132625075 learned_addr learned my addr 172.21.15.107:0/2132625075 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:43:23.713 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:17.921+0000 7f65d8a02700 1 -- 172.21.15.107:0/2132625075 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65c80094f0 con 0x7f65d4078cc0 2024-05-17T21:43:23.713 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:17.921+0000 7f65d8a02700 1 --2- 172.21.15.107:0/2132625075 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f65d4078cc0 0x7f65d4113810 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f65c80094c0 tx=0x7f65c8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:43:23.713 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:17.922+0000 7f65d17fa700 1 -- 172.21.15.107:0/2132625075 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f65c8010890 con 0x7f65d4078cc0 2024-05-17T21:43:23.713 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:17.922+0000 7f65dac66700 1 -- 172.21.15.107:0/2132625075 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65d4113f50 con 0x7f65d4078cc0 2024-05-17T21:43:23.714 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:17.922+0000 7f65d17fa700 1 -- 172.21.15.107:0/2132625075 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f65c8010ec0 con 0x7f65d4078cc0 2024-05-17T21:43:23.714 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:17.922+0000 7f65d17fa700 1 -- 172.21.15.107:0/2132625075 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f65c80243f0 con 0x7f65d4078cc0 2024-05-17T21:43:23.714 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:17.922+0000 7f65dac66700 1 -- 172.21.15.107:0/2132625075 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65d4114ac0 con 0x7f65d4078cc0 2024-05-17T21:43:23.714 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:17.923+0000 7f65d17fa700 1 -- 172.21.15.107:0/2132625075 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 9) v1 ==== 43389+0+0 (secure 0 0 0) 0x7f65c802d3f0 con 0x7f65d4078cc0 2024-05-17T21:43:23.714 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:17.923+0000 7f65d17fa700 1 --2- 172.21.15.107:0/2132625075 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7f65c40368a0 0x7f65c4038d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:43:23.714 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:17.923+0000 7f65dac66700 1 -- 172.21.15.107:0/2132625075 --> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f65d405f5c0 con 0x7f65c40368a0 2024-05-17T21:43:23.714 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:17.923+0000 7f65d17fa700 1 -- 172.21.15.107:0/2132625075 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f65c804a340 con 0x7f65d4078cc0 2024-05-17T21:43:23.714 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:17.923+0000 7f65d3fff700 1 -- 172.21.15.107:0/2132625075 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7f65c40368a0 msgr2=0x7f65c4038d50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/1718632476 2024-05-17T21:43:23.714 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:17.923+0000 7f65d3fff700 1 --2- 172.21.15.107:0/2132625075 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7f65c40368a0 0x7f65c4038d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-17T21:43:23.714 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:18.124+0000 7f65d3fff700 1 -- 172.21.15.107:0/2132625075 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7f65c40368a0 msgr2=0x7f65c4038d50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/1718632476 2024-05-17T21:43:23.714 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:18.124+0000 7f65d3fff700 1 --2- 172.21.15.107:0/2132625075 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7f65c40368a0 0x7f65c4038d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-17T21:43:23.714 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:18.525+0000 7f65d3fff700 1 -- 172.21.15.107:0/2132625075 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7f65c40368a0 msgr2=0x7f65c4038d50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/1718632476 2024-05-17T21:43:23.714 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:18.525+0000 7f65d3fff700 1 --2- 172.21.15.107:0/2132625075 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7f65c40368a0 0x7f65c4038d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-17T21:43:23.715 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:19.326+0000 7f65d3fff700 1 -- 172.21.15.107:0/2132625075 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7f65c40368a0 msgr2=0x7f65c4038d50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/1718632476 2024-05-17T21:43:23.715 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:19.326+0000 7f65d3fff700 1 --2- 172.21.15.107:0/2132625075 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7f65c40368a0 0x7f65c4038d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-17T21:43:23.715 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:20.927+0000 7f65d3fff700 1 -- 172.21.15.107:0/2132625075 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7f65c40368a0 msgr2=0x7f65c4038d50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/1718632476 2024-05-17T21:43:23.715 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:20.927+0000 7f65d3fff700 1 --2- 172.21.15.107:0/2132625075 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7f65c40368a0 0x7f65c4038d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-17T21:43:23.715 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:22.098+0000 7f65d17fa700 1 -- 172.21.15.107:0/2132625075 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mgrmap(e 10) v1 ==== 43156+0+0 (secure 0 0 0) 0x7f65c801a7e0 con 0x7f65d4078cc0 2024-05-17T21:43:23.715 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:22.098+0000 7f65d17fa700 1 -- 172.21.15.107:0/2132625075 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7f65c40368a0 msgr2=0x7f65c4038d50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-17T21:43:23.715 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:22.098+0000 7f65d17fa700 1 --2- 172.21.15.107:0/2132625075 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7f65c40368a0 0x7f65c4038d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:43:23.715 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:23.100+0000 7f65d17fa700 1 -- 172.21.15.107:0/2132625075 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mgrmap(e 11) v1 ==== 43283+0+0 (secure 0 0 0) 0x7f65c804b940 con 0x7f65d4078cc0 2024-05-17T21:43:23.715 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:23.100+0000 7f65d17fa700 1 --2- 172.21.15.107:0/2132625075 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f65c4039fc0 0x7f65c403c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:43:23.715 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:23.100+0000 7f65d17fa700 1 -- 172.21.15.107:0/2132625075 --> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f65d405f5c0 con 0x7f65c4039fc0 2024-05-17T21:43:23.715 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:23.101+0000 7f65d3fff700 1 --2- 172.21.15.107:0/2132625075 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f65c4039fc0 0x7f65c403c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:43:23.715 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:23.102+0000 7f65d3fff700 1 --2- 172.21.15.107:0/2132625075 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f65c4039fc0 0x7f65c403c3a0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f65c00039d0 tx=0x7f65c00092f0 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:43:23.715 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:23.103+0000 7f65d17fa700 1 -- 172.21.15.107:0/2132625075 <== mgr.14162 v2:172.21.15.107:6800/3405524625 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6549 (secure 0 0 0) 0x7f65d405f5c0 con 0x7f65c4039fc0 2024-05-17T21:43:23.716 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:23.109+0000 7f65dac66700 1 -- 172.21.15.107:0/2132625075 --> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f65d41143e0 con 0x7f65c4039fc0 2024-05-17T21:43:23.716 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:23.109+0000 7f65d17fa700 1 -- 172.21.15.107:0/2132625075 <== mgr.14162 v2:172.21.15.107:6800/3405524625 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7f65d41143e0 con 0x7f65c4039fc0 2024-05-17T21:43:23.716 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:23.109+0000 7f65dac66700 1 -- 172.21.15.107:0/2132625075 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f65c4039fc0 msgr2=0x7f65c403c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:43:23.716 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:23.109+0000 7f65dac66700 1 --2- 172.21.15.107:0/2132625075 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f65c4039fc0 0x7f65c403c3a0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f65c00039d0 tx=0x7f65c00092f0 comp rx=0 tx=0).stop 2024-05-17T21:43:23.716 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:23.109+0000 7f65dac66700 1 -- 172.21.15.107:0/2132625075 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f65d4078cc0 msgr2=0x7f65d4113810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:43:23.716 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:23.109+0000 7f65dac66700 1 --2- 172.21.15.107:0/2132625075 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f65d4078cc0 0x7f65d4113810 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f65c80094c0 tx=0x7f65c8009250 comp rx=0 tx=0).stop 2024-05-17T21:43:23.716 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:23.109+0000 7f65dac66700 1 -- 172.21.15.107:0/2132625075 shutdown_connections 2024-05-17T21:43:23.716 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:23.109+0000 7f65dac66700 1 --2- 172.21.15.107:0/2132625075 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f65c4039fc0 0x7f65c403c3a0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:43:23.716 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:23.109+0000 7f65dac66700 1 --2- 172.21.15.107:0/2132625075 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f65d4078cc0 0x7f65d4113810 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:43:23.716 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:23.109+0000 7f65dac66700 1 --2- 172.21.15.107:0/2132625075 >> [v2:172.21.15.107:6800/1718632476,v1:172.21.15.107:6801/1718632476] conn(0x7f65c40368a0 0x7f65c4038d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:43:23.716 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:23.109+0000 7f65dac66700 1 -- 172.21.15.107:0/2132625075 >> 172.21.15.107:0/2132625075 conn(0x7f65d40fe010 msgr2=0x7f65d40feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:43:23.716 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:23.109+0000 7f65dac66700 1 -- 172.21.15.107:0/2132625075 shutdown_connections 2024-05-17T21:43:23.716 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:23.110+0000 7f65dac66700 1 -- 172.21.15.107:0/2132625075 wait complete. 2024-05-17T21:43:23.717 INFO:teuthology.orchestra.run.smithi107.stdout:mgr epoch 9 is available 2024-05-17T21:43:23.717 INFO:teuthology.orchestra.run.smithi107.stdout:Generating a dashboard self-signed certificate... 2024-05-17T21:43:23.952 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:23 smithi107 ceph-mon[28434]: [17/May/2024:21:43:22] ENGINE Bus STARTING 2024-05-17T21:43:23.953 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:23 smithi107 ceph-mon[28434]: [17/May/2024:21:43:22] ENGINE Serving on https://172.21.15.107:7150 2024-05-17T21:43:23.953 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:23 smithi107 ceph-mon[28434]: [17/May/2024:21:43:22] ENGINE Bus STARTED 2024-05-17T21:43:23.953 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:23 smithi107 ceph-mon[28434]: from='mgr.14162 172.21.15.107:0/3666393387' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:43:23.953 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:23 smithi107 ceph-mon[28434]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-17T21:43:23.953 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:23 smithi107 ceph-mon[28434]: mgrmap e11: smithi107.xhhyiw(active, since 1.00661s) 2024-05-17T21:43:23.953 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:23 smithi107 ceph-mon[28434]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-17T21:43:23.953 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:23 smithi107 ceph-mon[28434]: from='mgr.14162 172.21.15.107:0/3666393387' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:43:25.354 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:25 smithi107 ceph-mon[28434]: from='mgr.14162 172.21.15.107:0/3666393387' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:43:25.354 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:25 smithi107 ceph-mon[28434]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:43:25.354 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:25 smithi107 ceph-mon[28434]: from='mgr.14162 172.21.15.107:0/3666393387' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:43:25.354 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:25 smithi107 ceph-mon[28434]: from='mgr.14162 172.21.15.107:0/3666393387' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:43:25.354 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:25 smithi107 ceph-mon[28434]: mgrmap e12: smithi107.xhhyiw(active, since 2s) 2024-05-17T21:43:25.367 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-05-17T21:43:25.367 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:24.176+0000 7fbfd287c700 1 Processor -- start 2024-05-17T21:43:25.367 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:24.176+0000 7fbfd287c700 1 -- start start 2024-05-17T21:43:25.368 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:24.176+0000 7fbfd287c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbfcc102880 0x7fbfcc102c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:43:25.368 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:24.176+0000 7fbfd287c700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfcc1031d0 con 0x7fbfcc102880 2024-05-17T21:43:25.368 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:24.176+0000 7fbfcbfff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbfcc102880 0x7fbfcc102c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:43:25.368 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:24.176+0000 7fbfcbfff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbfcc102880 0x7fbfcc102c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56714/0 (socket says 172.21.15.107:56714) 2024-05-17T21:43:25.368 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:24.176+0000 7fbfcbfff700 1 -- 172.21.15.107:0/3802592765 learned_addr learned my addr 172.21.15.107:0/3802592765 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:43:25.368 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:24.176+0000 7fbfcbfff700 1 -- 172.21.15.107:0/3802592765 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfcc103310 con 0x7fbfcc102880 2024-05-17T21:43:25.368 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:24.177+0000 7fbfcbfff700 1 --2- 172.21.15.107:0/3802592765 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbfcc102880 0x7fbfcc102c90 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc0098e0 tx=0x7fbfbc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95296cdf50b23ba7 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:43:25.368 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:24.177+0000 7fbfcaffd700 1 -- 172.21.15.107:0/3802592765 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbfbc00b860 con 0x7fbfcc102880 2024-05-17T21:43:25.368 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:24.177+0000 7fbfcaffd700 1 -- 172.21.15.107:0/3802592765 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbfbc00be90 con 0x7fbfcc102880 2024-05-17T21:43:25.368 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:24.177+0000 7fbfcaffd700 1 -- 172.21.15.107:0/3802592765 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbfbc0103c0 con 0x7fbfcc102880 2024-05-17T21:43:25.368 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:24.177+0000 7fbfd287c700 1 -- 172.21.15.107:0/3802592765 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbfcc102880 msgr2=0x7fbfcc102c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:43:25.368 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:24.177+0000 7fbfd287c700 1 --2- 172.21.15.107:0/3802592765 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbfcc102880 0x7fbfcc102c90 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc0098e0 tx=0x7fbfbc009250 comp rx=0 tx=0).stop 2024-05-17T21:43:25.368 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:24.177+0000 7fbfd287c700 1 -- 172.21.15.107:0/3802592765 shutdown_connections 2024-05-17T21:43:25.369 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:24.177+0000 7fbfd287c700 1 --2- 172.21.15.107:0/3802592765 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbfcc102880 0x7fbfcc102c90 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:43:25.369 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:24.177+0000 7fbfd287c700 1 -- 172.21.15.107:0/3802592765 >> 172.21.15.107:0/3802592765 conn(0x7fbfcc0fe060 msgr2=0x7fbfcc100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:43:25.369 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:24.177+0000 7fbfd287c700 1 -- 172.21.15.107:0/3802592765 shutdown_connections 2024-05-17T21:43:25.369 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:24.177+0000 7fbfd287c700 1 -- 172.21.15.107:0/3802592765 wait complete. 2024-05-17T21:43:25.369 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:24.178+0000 7fbfd287c700 1 Processor -- start 2024-05-17T21:43:25.369 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:24.178+0000 7fbfd287c700 1 -- start start 2024-05-17T21:43:25.369 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:24.178+0000 7fbfd287c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbfcc102880 0x7fbfcc111620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:43:25.369 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:24.178+0000 7fbfd287c700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfcc111b60 con 0x7fbfcc102880 2024-05-17T21:43:25.369 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:24.178+0000 7fbfcbfff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbfcc102880 0x7fbfcc111620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:43:25.369 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:24.178+0000 7fbfcbfff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbfcc102880 0x7fbfcc111620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56718/0 (socket says 172.21.15.107:56718) 2024-05-17T21:43:25.369 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:24.178+0000 7fbfcbfff700 1 -- 172.21.15.107:0/2855046639 learned_addr learned my addr 172.21.15.107:0/2855046639 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:43:25.369 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:24.179+0000 7fbfcbfff700 1 -- 172.21.15.107:0/2855046639 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfbc0094f0 con 0x7fbfcc102880 2024-05-17T21:43:25.369 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:24.179+0000 7fbfcbfff700 1 --2- 172.21.15.107:0/2855046639 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbfcc102880 0x7fbfcc111620 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc0090b0 tx=0x7fbfbc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:43:25.370 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:24.179+0000 7fbfc8ff9700 1 -- 172.21.15.107:0/2855046639 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbfbc010840 con 0x7fbfcc102880 2024-05-17T21:43:25.370 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:24.179+0000 7fbfc8ff9700 1 -- 172.21.15.107:0/2855046639 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbfbc010e70 con 0x7fbfcc102880 2024-05-17T21:43:25.370 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:24.179+0000 7fbfd287c700 1 -- 172.21.15.107:0/2855046639 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfcc111d60 con 0x7fbfcc102880 2024-05-17T21:43:25.370 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:24.179+0000 7fbfc8ff9700 1 -- 172.21.15.107:0/2855046639 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbfbc024e40 con 0x7fbfcc102880 2024-05-17T21:43:25.370 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:24.179+0000 7fbfd287c700 1 -- 172.21.15.107:0/2855046639 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfcc1128d0 con 0x7fbfcc102880 2024-05-17T21:43:25.370 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:24.180+0000 7fbfc8ff9700 1 -- 172.21.15.107:0/2855046639 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 11) v1 ==== 43283+0+0 (secure 0 0 0) 0x7fbfbc01a530 con 0x7fbfcc102880 2024-05-17T21:43:25.370 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:24.180+0000 7fbfd287c700 1 -- 172.21.15.107:0/2855046639 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfcc05f5c0 con 0x7fbfcc102880 2024-05-17T21:43:25.370 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:24.180+0000 7fbfc8ff9700 1 --2- 172.21.15.107:0/2855046639 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7fbfb4036720 0x7fbfb4038bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:43:25.370 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:24.180+0000 7fbfc8ff9700 1 -- 172.21.15.107:0/2855046639 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fbfbc04a730 con 0x7fbfcc102880 2024-05-17T21:43:25.370 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:24.180+0000 7fbfcb7fe700 1 --2- 172.21.15.107:0/2855046639 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7fbfb4036720 0x7fbfb4038bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:43:25.370 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:24.180+0000 7fbfcb7fe700 1 --2- 172.21.15.107:0/2855046639 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7fbfb4036720 0x7fbfb4038bd0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbfc0009990 tx=0x7fbfc0006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:43:25.370 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:24.185+0000 7fbfc8ff9700 1 -- 172.21.15.107:0/2855046639 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fbfbc018940 con 0x7fbfcc102880 2024-05-17T21:43:25.371 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:24.383+0000 7fbfd287c700 1 -- 172.21.15.107:0/2855046639 --> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7fbfcc0ff250 con 0x7fbfb4036720 2024-05-17T21:43:25.371 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:24.696+0000 7fbfc8ff9700 1 -- 172.21.15.107:0/2855046639 <== mgr.14162 v2:172.21.15.107:6800/3405524625 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7fbfcc0ff250 con 0x7fbfb4036720 2024-05-17T21:43:25.371 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:24.698+0000 7fbfd287c700 1 -- 172.21.15.107:0/2855046639 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7fbfb4036720 msgr2=0x7fbfb4038bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:43:25.371 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:24.698+0000 7fbfd287c700 1 --2- 172.21.15.107:0/2855046639 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7fbfb4036720 0x7fbfb4038bd0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbfc0009990 tx=0x7fbfc0006e40 comp rx=0 tx=0).stop 2024-05-17T21:43:25.371 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:24.698+0000 7fbfd287c700 1 -- 172.21.15.107:0/2855046639 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbfcc102880 msgr2=0x7fbfcc111620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:43:25.371 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:24.698+0000 7fbfd287c700 1 --2- 172.21.15.107:0/2855046639 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbfcc102880 0x7fbfcc111620 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc0090b0 tx=0x7fbfbc009250 comp rx=0 tx=0).stop 2024-05-17T21:43:25.371 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:24.698+0000 7fbfd287c700 1 -- 172.21.15.107:0/2855046639 shutdown_connections 2024-05-17T21:43:25.371 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:24.698+0000 7fbfd287c700 1 --2- 172.21.15.107:0/2855046639 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7fbfb4036720 0x7fbfb4038bd0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:43:25.371 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:24.698+0000 7fbfd287c700 1 --2- 172.21.15.107:0/2855046639 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbfcc102880 0x7fbfcc111620 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:43:25.371 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:24.698+0000 7fbfd287c700 1 -- 172.21.15.107:0/2855046639 >> 172.21.15.107:0/2855046639 conn(0x7fbfcc0fe060 msgr2=0x7fbfcc0feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:43:25.371 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:24.698+0000 7fbfd287c700 1 -- 172.21.15.107:0/2855046639 shutdown_connections 2024-05-17T21:43:25.371 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:24.698+0000 7fbfd287c700 1 -- 172.21.15.107:0/2855046639 wait complete. 2024-05-17T21:43:25.371 INFO:teuthology.orchestra.run.smithi107.stdout:Creating initial admin user... 2024-05-17T21:43:26.750 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$SZiSGvo2lRzdR6YHh.Nyxuip9oRodnJOrGOQN7AisuANjuskcarei", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1715982206, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-05-17T21:43:26.750 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:25.752+0000 7f0ac1285700 1 Processor -- start 2024-05-17T21:43:26.750 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:25.752+0000 7f0ac1285700 1 -- start start 2024-05-17T21:43:26.750 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:25.753+0000 7f0ac1285700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0abc102c90 0x7f0abc1030a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:43:26.751 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:25.753+0000 7f0ac1285700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0abc103670 con 0x7f0abc102c90 2024-05-17T21:43:26.751 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:25.753+0000 7f0abaffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0abc102c90 0x7f0abc1030a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:43:26.751 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:25.753+0000 7f0abaffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0abc102c90 0x7f0abc1030a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33016/0 (socket says 172.21.15.107:33016) 2024-05-17T21:43:26.751 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:25.753+0000 7f0abaffd700 1 -- 172.21.15.107:0/832161078 learned_addr learned my addr 172.21.15.107:0/832161078 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:43:26.751 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:25.753+0000 7f0abaffd700 1 -- 172.21.15.107:0/832161078 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0abc1037b0 con 0x7f0abc102c90 2024-05-17T21:43:26.751 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:25.753+0000 7f0abaffd700 1 --2- 172.21.15.107:0/832161078 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0abc102c90 0x7f0abc1030a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0aac0098e0 tx=0x7f0aac009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73ba60b7d35fdc24 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:43:26.751 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:25.753+0000 7f0aba7fc700 1 -- 172.21.15.107:0/832161078 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0aac00b860 con 0x7f0abc102c90 2024-05-17T21:43:26.751 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:25.753+0000 7f0aba7fc700 1 -- 172.21.15.107:0/832161078 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0aac00be90 con 0x7f0abc102c90 2024-05-17T21:43:26.751 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:25.753+0000 7f0aba7fc700 1 -- 172.21.15.107:0/832161078 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0aac0103c0 con 0x7f0abc102c90 2024-05-17T21:43:26.751 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:25.754+0000 7f0ac1285700 1 -- 172.21.15.107:0/832161078 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0abc102c90 msgr2=0x7f0abc1030a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:43:26.751 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:25.754+0000 7f0ac1285700 1 --2- 172.21.15.107:0/832161078 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0abc102c90 0x7f0abc1030a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0aac0098e0 tx=0x7f0aac009250 comp rx=0 tx=0).stop 2024-05-17T21:43:26.751 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:25.754+0000 7f0ac1285700 1 -- 172.21.15.107:0/832161078 shutdown_connections 2024-05-17T21:43:26.751 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:25.754+0000 7f0ac1285700 1 --2- 172.21.15.107:0/832161078 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0abc102c90 0x7f0abc1030a0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:43:26.752 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:25.754+0000 7f0ac1285700 1 -- 172.21.15.107:0/832161078 >> 172.21.15.107:0/832161078 conn(0x7f0abc0fc6f0 msgr2=0x7f0abc0feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:43:26.752 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:25.754+0000 7f0ac1285700 1 -- 172.21.15.107:0/832161078 shutdown_connections 2024-05-17T21:43:26.752 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:25.754+0000 7f0ac1285700 1 -- 172.21.15.107:0/832161078 wait complete. 2024-05-17T21:43:26.752 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:25.754+0000 7f0ac1285700 1 Processor -- start 2024-05-17T21:43:26.752 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:25.755+0000 7f0ac1285700 1 -- start start 2024-05-17T21:43:26.752 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:25.755+0000 7f0ac1285700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0abc102c90 0x7f0abc1137e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:43:26.752 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:25.755+0000 7f0ac1285700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0abc113d20 con 0x7f0abc102c90 2024-05-17T21:43:26.752 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:25.755+0000 7f0abaffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0abc102c90 0x7f0abc1137e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:43:26.752 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:25.755+0000 7f0abaffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0abc102c90 0x7f0abc1137e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33020/0 (socket says 172.21.15.107:33020) 2024-05-17T21:43:26.752 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:25.755+0000 7f0abaffd700 1 -- 172.21.15.107:0/883451685 learned_addr learned my addr 172.21.15.107:0/883451685 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:43:26.752 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:25.755+0000 7f0abaffd700 1 -- 172.21.15.107:0/883451685 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0aac0094f0 con 0x7f0abc102c90 2024-05-17T21:43:26.752 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:25.755+0000 7f0abaffd700 1 --2- 172.21.15.107:0/883451685 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0abc102c90 0x7f0abc1137e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0aac0094c0 tx=0x7f0aac009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:43:26.753 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:25.755+0000 7f0ab3fff700 1 -- 172.21.15.107:0/883451685 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0aac010890 con 0x7f0abc102c90 2024-05-17T21:43:26.753 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:25.756+0000 7f0ac1285700 1 -- 172.21.15.107:0/883451685 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0abc113f20 con 0x7f0abc102c90 2024-05-17T21:43:26.753 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:25.756+0000 7f0ab3fff700 1 -- 172.21.15.107:0/883451685 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0aac010ec0 con 0x7f0abc102c90 2024-05-17T21:43:26.753 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:25.756+0000 7f0ab3fff700 1 -- 172.21.15.107:0/883451685 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0aac0243f0 con 0x7f0abc102c90 2024-05-17T21:43:26.753 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:25.756+0000 7f0ac1285700 1 -- 172.21.15.107:0/883451685 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0abc114a90 con 0x7f0abc102c90 2024-05-17T21:43:26.753 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:25.756+0000 7f0ab3fff700 1 -- 172.21.15.107:0/883451685 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 12) v1 ==== 43389+0+0 (secure 0 0 0) 0x7f0aac02d3f0 con 0x7f0abc102c90 2024-05-17T21:43:26.753 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:25.756+0000 7f0ac1285700 1 -- 172.21.15.107:0/883451685 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0abc05f5c0 con 0x7f0abc102c90 2024-05-17T21:43:26.753 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:25.757+0000 7f0ab3fff700 1 --2- 172.21.15.107:0/883451685 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f0aa80368a0 0x7f0aa8038d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:43:26.753 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:25.757+0000 7f0ab3fff700 1 -- 172.21.15.107:0/883451685 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f0aac04a400 con 0x7f0abc102c90 2024-05-17T21:43:26.753 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:25.757+0000 7f0ab27fc700 1 --2- 172.21.15.107:0/883451685 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f0aa80368a0 0x7f0aa8038d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:43:26.753 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:25.757+0000 7f0ab27fc700 1 --2- 172.21.15.107:0/883451685 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f0aa80368a0 0x7f0aa8038d50 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0aa4009990 tx=0x7f0aa4006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:43:26.753 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:25.761+0000 7f0ab3fff700 1 -- 172.21.15.107:0/883451685 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f0aac02b8e0 con 0x7f0abc102c90 2024-05-17T21:43:26.754 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:25.966+0000 7f0ac1285700 1 -- 172.21.15.107:0/883451685 --> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f0abc05f3b0 con 0x7f0aa80368a0 2024-05-17T21:43:26.754 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:26.188+0000 7f0ab3fff700 1 -- 172.21.15.107:0/883451685 <== mgr.14162 v2:172.21.15.107:6800/3405524625 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f0abc05f3b0 con 0x7f0aa80368a0 2024-05-17T21:43:26.754 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:26.190+0000 7f0ac1285700 1 -- 172.21.15.107:0/883451685 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f0aa80368a0 msgr2=0x7f0aa8038d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:43:26.754 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:26.190+0000 7f0ac1285700 1 --2- 172.21.15.107:0/883451685 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f0aa80368a0 0x7f0aa8038d50 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0aa4009990 tx=0x7f0aa4006e40 comp rx=0 tx=0).stop 2024-05-17T21:43:26.754 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:26.190+0000 7f0ac1285700 1 -- 172.21.15.107:0/883451685 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0abc102c90 msgr2=0x7f0abc1137e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:43:26.754 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:26.190+0000 7f0ac1285700 1 --2- 172.21.15.107:0/883451685 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0abc102c90 0x7f0abc1137e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0aac0094c0 tx=0x7f0aac009250 comp rx=0 tx=0).stop 2024-05-17T21:43:26.754 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:26.190+0000 7f0ac1285700 1 -- 172.21.15.107:0/883451685 shutdown_connections 2024-05-17T21:43:26.754 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:26.190+0000 7f0ac1285700 1 --2- 172.21.15.107:0/883451685 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f0aa80368a0 0x7f0aa8038d50 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:43:26.754 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:26.190+0000 7f0ac1285700 1 --2- 172.21.15.107:0/883451685 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0abc102c90 0x7f0abc1137e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:43:26.754 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:26.190+0000 7f0ac1285700 1 -- 172.21.15.107:0/883451685 >> 172.21.15.107:0/883451685 conn(0x7f0abc0fc6f0 msgr2=0x7f0abc0fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:43:26.754 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:26.190+0000 7f0ac1285700 1 -- 172.21.15.107:0/883451685 shutdown_connections 2024-05-17T21:43:26.754 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:26.190+0000 7f0ac1285700 1 -- 172.21.15.107:0/883451685 wait complete. 2024-05-17T21:43:26.754 INFO:teuthology.orchestra.run.smithi107.stdout:Fetching dashboard port number... 2024-05-17T21:43:27.354 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:27 smithi107 ceph-mon[28434]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:43:27.354 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:27 smithi107 ceph-mon[28434]: from='mgr.14162 172.21.15.107:0/3666393387' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:43:28.117 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 8443 2024-05-17T21:43:28.117 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:27.205+0000 7f857cc5d700 1 Processor -- start 2024-05-17T21:43:28.118 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:27.205+0000 7f857cc5d700 1 -- start start 2024-05-17T21:43:28.118 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:27.206+0000 7f857cc5d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8578076c30 0x7f8578105420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:43:28.118 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:27.206+0000 7f857cc5d700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8578105960 con 0x7f8578076c30 2024-05-17T21:43:28.118 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:27.206+0000 7f857659c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8578076c30 0x7f8578105420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:43:28.118 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:27.206+0000 7f857659c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8578076c30 0x7f8578105420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33024/0 (socket says 172.21.15.107:33024) 2024-05-17T21:43:28.118 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:27.206+0000 7f857659c700 1 -- 172.21.15.107:0/3745482280 learned_addr learned my addr 172.21.15.107:0/3745482280 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:43:28.118 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:27.206+0000 7f857659c700 1 -- 172.21.15.107:0/3745482280 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8578105aa0 con 0x7f8578076c30 2024-05-17T21:43:28.118 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:27.206+0000 7f857659c700 1 --2- 172.21.15.107:0/3745482280 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8578076c30 0x7f8578105420 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f85600098e0 tx=0x7f8560009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b38eef1c0be7a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:43:28.118 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:27.206+0000 7f857559a700 1 -- 172.21.15.107:0/3745482280 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f856000b860 con 0x7f8578076c30 2024-05-17T21:43:28.118 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:27.206+0000 7f857559a700 1 -- 172.21.15.107:0/3745482280 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f856000be90 con 0x7f8578076c30 2024-05-17T21:43:28.118 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:27.206+0000 7f857559a700 1 -- 172.21.15.107:0/3745482280 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f85600103c0 con 0x7f8578076c30 2024-05-17T21:43:28.119 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:27.207+0000 7f857cc5d700 1 -- 172.21.15.107:0/3745482280 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8578076c30 msgr2=0x7f8578105420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:43:28.119 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:27.207+0000 7f857cc5d700 1 --2- 172.21.15.107:0/3745482280 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8578076c30 0x7f8578105420 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f85600098e0 tx=0x7f8560009250 comp rx=0 tx=0).stop 2024-05-17T21:43:28.119 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:27.207+0000 7f857cc5d700 1 -- 172.21.15.107:0/3745482280 shutdown_connections 2024-05-17T21:43:28.119 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:27.207+0000 7f857cc5d700 1 --2- 172.21.15.107:0/3745482280 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8578076c30 0x7f8578105420 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:43:28.119 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:27.207+0000 7f857cc5d700 1 -- 172.21.15.107:0/3745482280 >> 172.21.15.107:0/3745482280 conn(0x7f85780fcef0 msgr2=0x7f85780ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:43:28.119 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:27.207+0000 7f857cc5d700 1 -- 172.21.15.107:0/3745482280 shutdown_connections 2024-05-17T21:43:28.119 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:27.207+0000 7f857cc5d700 1 -- 172.21.15.107:0/3745482280 wait complete. 2024-05-17T21:43:28.119 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:27.208+0000 7f857cc5d700 1 Processor -- start 2024-05-17T21:43:28.119 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:27.208+0000 7f857cc5d700 1 -- start start 2024-05-17T21:43:28.119 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:27.208+0000 7f857cc5d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8578076c30 0x7f8578102d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:43:28.119 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:27.208+0000 7f857cc5d700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85781032b0 con 0x7f8578076c30 2024-05-17T21:43:28.119 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:27.208+0000 7f857659c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8578076c30 0x7f8578102d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:43:28.119 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:27.208+0000 7f857659c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8578076c30 0x7f8578102d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33032/0 (socket says 172.21.15.107:33032) 2024-05-17T21:43:28.119 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:27.208+0000 7f857659c700 1 -- 172.21.15.107:0/3577601250 learned_addr learned my addr 172.21.15.107:0/3577601250 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:43:28.120 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:27.208+0000 7f857659c700 1 -- 172.21.15.107:0/3577601250 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85600094f0 con 0x7f8578076c30 2024-05-17T21:43:28.120 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:27.209+0000 7f857659c700 1 --2- 172.21.15.107:0/3577601250 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8578076c30 0x7f8578102d70 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f85600094c0 tx=0x7f8560009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:43:28.120 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:27.209+0000 7f856effd700 1 -- 172.21.15.107:0/3577601250 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8560010890 con 0x7f8578076c30 2024-05-17T21:43:28.120 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:27.209+0000 7f857cc5d700 1 -- 172.21.15.107:0/3577601250 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8578101320 con 0x7f8578076c30 2024-05-17T21:43:28.120 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:27.209+0000 7f856effd700 1 -- 172.21.15.107:0/3577601250 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8560010ec0 con 0x7f8578076c30 2024-05-17T21:43:28.120 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:27.209+0000 7f856effd700 1 -- 172.21.15.107:0/3577601250 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f85600243f0 con 0x7f8578076c30 2024-05-17T21:43:28.120 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:27.209+0000 7f857cc5d700 1 -- 172.21.15.107:0/3577601250 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8578101810 con 0x7f8578076c30 2024-05-17T21:43:28.120 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:27.210+0000 7f856effd700 1 -- 172.21.15.107:0/3577601250 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 12) v1 ==== 43389+0+0 (secure 0 0 0) 0x7f85600109f0 con 0x7f8578076c30 2024-05-17T21:43:28.120 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:27.210+0000 7f857cc5d700 1 -- 172.21.15.107:0/3577601250 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f857804bed0 con 0x7f8578076c30 2024-05-17T21:43:28.120 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:27.210+0000 7f856effd700 1 --2- 172.21.15.107:0/3577601250 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f85640368a0 0x7f8564038d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:43:28.120 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:27.210+0000 7f856effd700 1 -- 172.21.15.107:0/3577601250 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f856004a400 con 0x7f8578076c30 2024-05-17T21:43:28.120 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:27.210+0000 7f8575d9b700 1 --2- 172.21.15.107:0/3577601250 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f85640368a0 0x7f8564038d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:43:28.120 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:27.211+0000 7f8575d9b700 1 --2- 172.21.15.107:0/3577601250 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f85640368a0 0x7f8564038d50 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8568009990 tx=0x7f8568006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:43:28.120 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:27.216+0000 7f856effd700 1 -- 172.21.15.107:0/3577601250 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f856002b760 con 0x7f8578076c30 2024-05-17T21:43:28.121 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:27.407+0000 7f857cc5d700 1 -- 172.21.15.107:0/3577601250 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f8578106830 con 0x7f8578076c30 2024-05-17T21:43:28.121 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:27.407+0000 7f856effd700 1 -- 172.21.15.107:0/3577601250 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7f856002ce20 con 0x7f8578076c30 2024-05-17T21:43:28.121 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:27.409+0000 7f857cc5d700 1 -- 172.21.15.107:0/3577601250 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f85640368a0 msgr2=0x7f8564038d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:43:28.121 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:27.409+0000 7f857cc5d700 1 --2- 172.21.15.107:0/3577601250 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f85640368a0 0x7f8564038d50 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8568009990 tx=0x7f8568006e40 comp rx=0 tx=0).stop 2024-05-17T21:43:28.121 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:27.409+0000 7f857cc5d700 1 -- 172.21.15.107:0/3577601250 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8578076c30 msgr2=0x7f8578102d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:43:28.121 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:27.409+0000 7f857cc5d700 1 --2- 172.21.15.107:0/3577601250 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8578076c30 0x7f8578102d70 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f85600094c0 tx=0x7f8560009250 comp rx=0 tx=0).stop 2024-05-17T21:43:28.121 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:27.409+0000 7f857cc5d700 1 -- 172.21.15.107:0/3577601250 shutdown_connections 2024-05-17T21:43:28.121 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:27.409+0000 7f857cc5d700 1 --2- 172.21.15.107:0/3577601250 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f85640368a0 0x7f8564038d50 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:43:28.121 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:27.409+0000 7f857cc5d700 1 --2- 172.21.15.107:0/3577601250 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8578076c30 0x7f8578102d70 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:43:28.121 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:27.409+0000 7f857cc5d700 1 -- 172.21.15.107:0/3577601250 >> 172.21.15.107:0/3577601250 conn(0x7f85780fcef0 msgr2=0x7f85780fd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:43:28.121 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:27.409+0000 7f857cc5d700 1 -- 172.21.15.107:0/3577601250 shutdown_connections 2024-05-17T21:43:28.121 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:27.409+0000 7f857cc5d700 1 -- 172.21.15.107:0/3577601250 wait complete. 2024-05-17T21:43:28.125 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-17T21:43:28.125 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout disabled 2024-05-17T21:43:28.133 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-17T21:43:28.133 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout inactive 2024-05-17T21:43:28.133 INFO:teuthology.orchestra.run.smithi107.stdout:firewalld.service is not enabled 2024-05-17T21:43:28.133 INFO:teuthology.orchestra.run.smithi107.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-05-17T21:43:28.135 INFO:teuthology.orchestra.run.smithi107.stdout:Ceph Dashboard is now available at: 2024-05-17T21:43:28.135 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:43:28.135 INFO:teuthology.orchestra.run.smithi107.stdout: URL: https://smithi107.front.sepia.ceph.com:8443/ 2024-05-17T21:43:28.135 INFO:teuthology.orchestra.run.smithi107.stdout: User: admin 2024-05-17T21:43:28.135 INFO:teuthology.orchestra.run.smithi107.stdout: Password: zfvpwj6nih 2024-05-17T21:43:28.135 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:43:28.136 INFO:teuthology.orchestra.run.smithi107.stdout:Saving cluster configuration to /var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/config directory 2024-05-17T21:43:28.136 INFO:teuthology.orchestra.run.smithi107.stdout:Enabling autotune for osd_memory_target 2024-05-17T21:43:28.446 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:28 smithi107 ceph-mon[28434]: from='client.? 172.21.15.107:0/3577601250' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-05-17T21:43:29.299 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:28.554+0000 7f63827b7700 1 Processor -- start 2024-05-17T21:43:29.299 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:28.554+0000 7f63827b7700 1 -- start start 2024-05-17T21:43:29.299 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:28.554+0000 7f63827b7700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f637c076be0 0x7f637c105480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:43:29.299 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:28.554+0000 7f63827b7700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f637c1059c0 con 0x7f637c076be0 2024-05-17T21:43:29.299 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:28.554+0000 7f637bfff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f637c076be0 0x7f637c105480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:43:29.299 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:28.555+0000 7f637bfff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f637c076be0 0x7f637c105480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33036/0 (socket says 172.21.15.107:33036) 2024-05-17T21:43:29.300 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:28.555+0000 7f637bfff700 1 -- 172.21.15.107:0/2162367668 learned_addr learned my addr 172.21.15.107:0/2162367668 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:43:29.300 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:28.555+0000 7f637bfff700 1 -- 172.21.15.107:0/2162367668 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f637c105b00 con 0x7f637c076be0 2024-05-17T21:43:29.300 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:28.555+0000 7f637bfff700 1 --2- 172.21.15.107:0/2162367668 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f637c076be0 0x7f637c105480 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f636c0098e0 tx=0x7f636c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99d8ccc8697834cf server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:43:29.300 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:28.555+0000 7f637b7fe700 1 -- 172.21.15.107:0/2162367668 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f636c00b8f0 con 0x7f637c076be0 2024-05-17T21:43:29.300 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:28.555+0000 7f637b7fe700 1 -- 172.21.15.107:0/2162367668 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f636c011420 con 0x7f637c076be0 2024-05-17T21:43:29.300 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:28.555+0000 7f637b7fe700 1 -- 172.21.15.107:0/2162367668 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f636c010540 con 0x7f637c076be0 2024-05-17T21:43:29.300 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:28.556+0000 7f63827b7700 1 -- 172.21.15.107:0/2162367668 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f637c076be0 msgr2=0x7f637c105480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:43:29.300 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:28.556+0000 7f63827b7700 1 --2- 172.21.15.107:0/2162367668 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f637c076be0 0x7f637c105480 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f636c0098e0 tx=0x7f636c009250 comp rx=0 tx=0).stop 2024-05-17T21:43:29.300 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:28.556+0000 7f63827b7700 1 -- 172.21.15.107:0/2162367668 shutdown_connections 2024-05-17T21:43:29.300 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:28.556+0000 7f63827b7700 1 --2- 172.21.15.107:0/2162367668 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f637c076be0 0x7f637c105480 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:43:29.300 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:28.556+0000 7f63827b7700 1 -- 172.21.15.107:0/2162367668 >> 172.21.15.107:0/2162367668 conn(0x7f637c0fced0 msgr2=0x7f637c0ff2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:43:29.300 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:28.556+0000 7f63827b7700 1 -- 172.21.15.107:0/2162367668 shutdown_connections 2024-05-17T21:43:29.300 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:28.556+0000 7f63827b7700 1 -- 172.21.15.107:0/2162367668 wait complete. 2024-05-17T21:43:29.300 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:28.556+0000 7f63827b7700 1 Processor -- start 2024-05-17T21:43:29.300 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:28.556+0000 7f63827b7700 1 -- start start 2024-05-17T21:43:29.301 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:28.557+0000 7f63827b7700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f637c076be0 0x7f637c102dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:43:29.301 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:28.557+0000 7f63827b7700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f637c103310 con 0x7f637c076be0 2024-05-17T21:43:29.301 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:28.557+0000 7f637bfff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f637c076be0 0x7f637c102dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:43:29.301 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:28.557+0000 7f637bfff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f637c076be0 0x7f637c102dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33052/0 (socket says 172.21.15.107:33052) 2024-05-17T21:43:29.301 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:28.557+0000 7f637bfff700 1 -- 172.21.15.107:0/2467851297 learned_addr learned my addr 172.21.15.107:0/2467851297 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:43:29.301 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:28.557+0000 7f637bfff700 1 -- 172.21.15.107:0/2467851297 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f636c0094f0 con 0x7f637c076be0 2024-05-17T21:43:29.301 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:28.557+0000 7f637bfff700 1 --2- 172.21.15.107:0/2467851297 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f637c076be0 0x7f637c102dd0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f636c011cf0 tx=0x7f636c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:43:29.301 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:28.557+0000 7f63797fa700 1 -- 172.21.15.107:0/2467851297 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f636c010800 con 0x7f637c076be0 2024-05-17T21:43:29.301 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:28.557+0000 7f63797fa700 1 -- 172.21.15.107:0/2467851297 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f636c010e30 con 0x7f637c076be0 2024-05-17T21:43:29.301 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:28.557+0000 7f63827b7700 1 -- 172.21.15.107:0/2467851297 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f637c101380 con 0x7f637c076be0 2024-05-17T21:43:29.301 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:28.557+0000 7f63797fa700 1 -- 172.21.15.107:0/2467851297 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f636c01ae80 con 0x7f637c076be0 2024-05-17T21:43:29.301 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:28.557+0000 7f63827b7700 1 -- 172.21.15.107:0/2467851297 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f637c101870 con 0x7f637c076be0 2024-05-17T21:43:29.301 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:28.558+0000 7f63797fa700 1 -- 172.21.15.107:0/2467851297 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) v1 ==== 43433+0+0 (secure 0 0 0) 0x7f636c02d3f0 con 0x7f637c076be0 2024-05-17T21:43:29.301 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:28.558+0000 7f63797fa700 1 --2- 172.21.15.107:0/2467851297 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f6368036990 0x7f6368038e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:43:29.302 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:28.558+0000 7f63797fa700 1 -- 172.21.15.107:0/2467851297 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f636c04a380 con 0x7f637c076be0 2024-05-17T21:43:29.302 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:28.558+0000 7f6373fff700 1 --2- 172.21.15.107:0/2467851297 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f6368036990 0x7f6368038e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:43:29.302 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:28.559+0000 7f63827b7700 1 -- 172.21.15.107:0/2467851297 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f637c04bed0 con 0x7f637c076be0 2024-05-17T21:43:29.302 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:28.559+0000 7f6373fff700 1 --2- 172.21.15.107:0/2467851297 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f6368036990 0x7f6368038e40 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6364009990 tx=0x7f6364006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:43:29.302 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:28.563+0000 7f63797fa700 1 -- 172.21.15.107:0/2467851297 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f636c02b8e0 con 0x7f637c076be0 2024-05-17T21:43:29.302 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:28.754+0000 7f63827b7700 1 -- 172.21.15.107:0/2467851297 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f637c106880 con 0x7f637c076be0 2024-05-17T21:43:29.302 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:28.755+0000 7f63797fa700 1 -- 172.21.15.107:0/2467851297 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f636c02b2f0 con 0x7f637c076be0 2024-05-17T21:43:29.302 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:28.757+0000 7f63827b7700 1 -- 172.21.15.107:0/2467851297 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f6368036990 msgr2=0x7f6368038e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:43:29.302 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:28.757+0000 7f63827b7700 1 --2- 172.21.15.107:0/2467851297 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f6368036990 0x7f6368038e40 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6364009990 tx=0x7f6364006e40 comp rx=0 tx=0).stop 2024-05-17T21:43:29.302 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:28.757+0000 7f63827b7700 1 -- 172.21.15.107:0/2467851297 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f637c076be0 msgr2=0x7f637c102dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:43:29.302 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:28.757+0000 7f63827b7700 1 --2- 172.21.15.107:0/2467851297 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f637c076be0 0x7f637c102dd0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f636c011cf0 tx=0x7f636c009250 comp rx=0 tx=0).stop 2024-05-17T21:43:29.302 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:28.757+0000 7f63827b7700 1 -- 172.21.15.107:0/2467851297 shutdown_connections 2024-05-17T21:43:29.302 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:28.757+0000 7f63827b7700 1 --2- 172.21.15.107:0/2467851297 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f6368036990 0x7f6368038e40 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:43:29.303 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:28.757+0000 7f63827b7700 1 --2- 172.21.15.107:0/2467851297 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f637c076be0 0x7f637c102dd0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:43:29.303 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:28.757+0000 7f63827b7700 1 -- 172.21.15.107:0/2467851297 >> 172.21.15.107:0/2467851297 conn(0x7f637c0fced0 msgr2=0x7f637c0fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:43:29.303 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:28.757+0000 7f63827b7700 1 -- 172.21.15.107:0/2467851297 shutdown_connections 2024-05-17T21:43:29.303 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:28.757+0000 7f63827b7700 1 -- 172.21.15.107:0/2467851297 wait complete. 2024-05-17T21:43:29.354 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:29 smithi107 ceph-mon[28434]: mgrmap e13: smithi107.xhhyiw(active, since 6s) 2024-05-17T21:43:30.555 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:29.680+0000 7f31a73d2700 1 Processor -- start 2024-05-17T21:43:30.555 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:29.680+0000 7f31a73d2700 1 -- start start 2024-05-17T21:43:30.555 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:29.680+0000 7f31a73d2700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31a0102870 0x7f31a0102c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:43:30.556 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:29.681+0000 7f31a73d2700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31a01031c0 con 0x7f31a0102870 2024-05-17T21:43:30.556 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:29.681+0000 7f31a516e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31a0102870 0x7f31a0102c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:43:30.556 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:29.681+0000 7f31a516e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31a0102870 0x7f31a0102c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33058/0 (socket says 172.21.15.107:33058) 2024-05-17T21:43:30.556 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:29.681+0000 7f31a516e700 1 -- 172.21.15.107:0/2790002048 learned_addr learned my addr 172.21.15.107:0/2790002048 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:43:30.556 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:29.681+0000 7f31a516e700 1 -- 172.21.15.107:0/2790002048 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31a0103300 con 0x7f31a0102870 2024-05-17T21:43:30.556 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:29.681+0000 7f31a516e700 1 --2- 172.21.15.107:0/2790002048 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31a0102870 0x7f31a0102c80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f319c0098e0 tx=0x7f319c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=844eb0ef034bc558 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:43:30.556 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:29.681+0000 7f3197fff700 1 -- 172.21.15.107:0/2790002048 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f319c00b860 con 0x7f31a0102870 2024-05-17T21:43:30.556 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:29.681+0000 7f3197fff700 1 -- 172.21.15.107:0/2790002048 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f319c00be90 con 0x7f31a0102870 2024-05-17T21:43:30.556 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:29.681+0000 7f3197fff700 1 -- 172.21.15.107:0/2790002048 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f319c0103c0 con 0x7f31a0102870 2024-05-17T21:43:30.556 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:29.682+0000 7f31a73d2700 1 -- 172.21.15.107:0/2790002048 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31a0102870 msgr2=0x7f31a0102c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:43:30.556 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:29.682+0000 7f31a73d2700 1 --2- 172.21.15.107:0/2790002048 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31a0102870 0x7f31a0102c80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f319c0098e0 tx=0x7f319c009250 comp rx=0 tx=0).stop 2024-05-17T21:43:30.556 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:29.682+0000 7f31a73d2700 1 -- 172.21.15.107:0/2790002048 shutdown_connections 2024-05-17T21:43:30.556 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:29.682+0000 7f31a73d2700 1 --2- 172.21.15.107:0/2790002048 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31a0102870 0x7f31a0102c80 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:43:30.557 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:29.682+0000 7f31a73d2700 1 -- 172.21.15.107:0/2790002048 >> 172.21.15.107:0/2790002048 conn(0x7f31a00fdf90 msgr2=0x7f31a0100400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:43:30.557 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:29.682+0000 7f31a73d2700 1 -- 172.21.15.107:0/2790002048 shutdown_connections 2024-05-17T21:43:30.557 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:29.682+0000 7f31a73d2700 1 -- 172.21.15.107:0/2790002048 wait complete. 2024-05-17T21:43:30.557 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:29.683+0000 7f31a73d2700 1 Processor -- start 2024-05-17T21:43:30.557 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:29.683+0000 7f31a73d2700 1 -- start start 2024-05-17T21:43:30.557 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:29.683+0000 7f31a73d2700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31a0102870 0x7f31a0111600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:43:30.557 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:29.683+0000 7f31a73d2700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31a0111b40 con 0x7f31a0102870 2024-05-17T21:43:30.557 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:29.683+0000 7f31a516e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31a0102870 0x7f31a0111600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:43:30.557 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:29.683+0000 7f31a516e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31a0102870 0x7f31a0111600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33070/0 (socket says 172.21.15.107:33070) 2024-05-17T21:43:30.557 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:29.683+0000 7f31a516e700 1 -- 172.21.15.107:0/624168105 learned_addr learned my addr 172.21.15.107:0/624168105 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:43:30.558 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:29.683+0000 7f31a516e700 1 -- 172.21.15.107:0/624168105 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f319c0094f0 con 0x7f31a0102870 2024-05-17T21:43:30.558 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:29.683+0000 7f31a516e700 1 --2- 172.21.15.107:0/624168105 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31a0102870 0x7f31a0111600 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f319c0090b0 tx=0x7f319c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:43:30.558 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:29.683+0000 7f3195ffb700 1 -- 172.21.15.107:0/624168105 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f319c010840 con 0x7f31a0102870 2024-05-17T21:43:30.558 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:29.683+0000 7f3195ffb700 1 -- 172.21.15.107:0/624168105 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f319c010e70 con 0x7f31a0102870 2024-05-17T21:43:30.558 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:29.683+0000 7f31a73d2700 1 -- 172.21.15.107:0/624168105 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31a0111d40 con 0x7f31a0102870 2024-05-17T21:43:30.558 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:29.683+0000 7f3195ffb700 1 -- 172.21.15.107:0/624168105 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f319c024e40 con 0x7f31a0102870 2024-05-17T21:43:30.558 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:29.684+0000 7f31a73d2700 1 -- 172.21.15.107:0/624168105 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31a01128b0 con 0x7f31a0102870 2024-05-17T21:43:30.558 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:29.685+0000 7f3195ffb700 1 -- 172.21.15.107:0/624168105 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) v1 ==== 43433+0+0 (secure 0 0 0) 0x7f319c01a4e0 con 0x7f31a0102870 2024-05-17T21:43:30.558 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:29.685+0000 7f31a73d2700 1 -- 172.21.15.107:0/624168105 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31a005f5c0 con 0x7f31a0102870 2024-05-17T21:43:30.558 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:29.685+0000 7f3195ffb700 1 --2- 172.21.15.107:0/624168105 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f318c036940 0x7f318c038df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:43:30.558 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:29.685+0000 7f3195ffb700 1 -- 172.21.15.107:0/624168105 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f319c04abe0 con 0x7f31a0102870 2024-05-17T21:43:30.558 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:29.685+0000 7f31a496d700 1 --2- 172.21.15.107:0/624168105 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f318c036940 0x7f318c038df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:43:30.559 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:29.685+0000 7f31a496d700 1 --2- 172.21.15.107:0/624168105 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f318c036940 0x7f318c038df0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3190009990 tx=0x7f3190006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:43:30.559 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:29.689+0000 7f3195ffb700 1 -- 172.21.15.107:0/624168105 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f319c018940 con 0x7f31a0102870 2024-05-17T21:43:30.559 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:29.981+0000 7f31a73d2700 1 -- 172.21.15.107:0/624168105 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f31a004bed0 con 0x7f31a0102870 2024-05-17T21:43:30.559 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:29.983+0000 7f3195ffb700 1 -- 172.21.15.107:0/624168105 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v33)=0 set mgr/dashboard/cluster/status v33) v1 ==== 153+0+0 (secure 0 0 0) 0x7f319c018350 con 0x7f31a0102870 2024-05-17T21:43:30.559 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:29.986+0000 7f31a73d2700 1 -- 172.21.15.107:0/624168105 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f318c036940 msgr2=0x7f318c038df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:43:30.559 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:29.986+0000 7f31a73d2700 1 --2- 172.21.15.107:0/624168105 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f318c036940 0x7f318c038df0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3190009990 tx=0x7f3190006e40 comp rx=0 tx=0).stop 2024-05-17T21:43:30.559 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:29.986+0000 7f31a73d2700 1 -- 172.21.15.107:0/624168105 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31a0102870 msgr2=0x7f31a0111600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:43:30.559 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:29.986+0000 7f31a73d2700 1 --2- 172.21.15.107:0/624168105 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31a0102870 0x7f31a0111600 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f319c0090b0 tx=0x7f319c009250 comp rx=0 tx=0).stop 2024-05-17T21:43:30.559 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:29.986+0000 7f31a73d2700 1 -- 172.21.15.107:0/624168105 shutdown_connections 2024-05-17T21:43:30.559 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:29.986+0000 7f31a73d2700 1 --2- 172.21.15.107:0/624168105 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f318c036940 0x7f318c038df0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:43:30.559 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:29.986+0000 7f31a73d2700 1 --2- 172.21.15.107:0/624168105 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31a0102870 0x7f31a0111600 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:43:30.560 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:29.986+0000 7f31a73d2700 1 -- 172.21.15.107:0/624168105 >> 172.21.15.107:0/624168105 conn(0x7f31a00fdf90 msgr2=0x7f31a00feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:43:30.560 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:29.986+0000 7f31a73d2700 1 -- 172.21.15.107:0/624168105 shutdown_connections 2024-05-17T21:43:30.560 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-05-17T21:43:29.986+0000 7f31a73d2700 1 -- 172.21.15.107:0/624168105 wait complete. 2024-05-17T21:43:30.560 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-05-17T21:43:30.560 INFO:teuthology.orchestra.run.smithi107.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-05-17T21:43:30.560 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:43:30.560 INFO:teuthology.orchestra.run.smithi107.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-05-17T21:43:30.560 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:43:30.560 INFO:teuthology.orchestra.run.smithi107.stdout:Or, if you are only running a single cluster on this host: 2024-05-17T21:43:30.560 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:43:30.560 INFO:teuthology.orchestra.run.smithi107.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-05-17T21:43:30.560 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:43:30.560 INFO:teuthology.orchestra.run.smithi107.stdout:Please consider enabling telemetry to help improve Ceph: 2024-05-17T21:43:30.560 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:43:30.560 INFO:teuthology.orchestra.run.smithi107.stdout: ceph telemetry on 2024-05-17T21:43:30.560 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:43:30.561 INFO:teuthology.orchestra.run.smithi107.stdout:For more information see: 2024-05-17T21:43:30.561 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:43:30.561 INFO:teuthology.orchestra.run.smithi107.stdout: https://docs.ceph.com/docs/master/mgr/telemetry/ 2024-05-17T21:43:30.561 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:43:30.561 INFO:teuthology.orchestra.run.smithi107.stdout:Bootstrap complete. 2024-05-17T21:43:30.589 INFO:tasks.cephadm:Fetching config... 2024-05-17T21:43:30.589 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-05-17T21:43:30.589 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-05-17T21:43:30.646 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-05-17T21:43:30.646 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-05-17T21:43:30.646 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-05-17T21:43:30.702 INFO:tasks.cephadm:Fetching mon keyring... 2024-05-17T21:43:30.702 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-05-17T21:43:30.702 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/mon.smithi107/keyring of=/dev/stdout 2024-05-17T21:43:30.767 INFO:tasks.cephadm:Fetching pub ssh key... 2024-05-17T21:43:30.768 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-05-17T21:43:30.768 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-05-17T21:43:30.822 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-05-17T21:43:30.822 DEBUG:teuthology.orchestra.run.smithi107:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDC82J4v0yngLUWHqgMnkz1BtuSxjzef52E8Eig8IVeZp7HkNJWFBdaHG++rWmW7COJuA4L7cjFG/t0n2Cv0ZDRoAWT1PN0+5I17aLtivFr77qLoi0rOxfNhp4gFrFF+9OosLjlTNKZjk1kkojWJC8yCfWY5jcl6LY+5y1YnAYfuquqK8cjLrzCjg9vvoihv82tBe+p20KD0HlyLzARqILlktDOUOHnXeCmgRVPuw5T+PZ/Kiytb28gHHsxGiGhttgs0D4ImCNO7+MOLBALzECk7OjmPPaVYsbv3rA+P2n9vbGMCLuVhpWCKEfvLwr4WUueKnBgApmXWW6nu0bP2XqNamn79HdL71tAH3rb0n8A2CUwWRqCGhkRWYH5iKEd0RyqSbLDv1K2wGuCD+SYLwCN1I8K1IGwnhhFR/9W4lXf3hJnhUOgLw6XLQTlo826uKq+Fs9/OUjVHossT1BFiZIkug5LDvO354kaIum6uo144Ub6kqG7giLTFssS+H3vy9U= ceph-3722731a-1496-11ef-bc9a-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-17T21:43:30.896 INFO:teuthology.orchestra.run.smithi107.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDC82J4v0yngLUWHqgMnkz1BtuSxjzef52E8Eig8IVeZp7HkNJWFBdaHG++rWmW7COJuA4L7cjFG/t0n2Cv0ZDRoAWT1PN0+5I17aLtivFr77qLoi0rOxfNhp4gFrFF+9OosLjlTNKZjk1kkojWJC8yCfWY5jcl6LY+5y1YnAYfuquqK8cjLrzCjg9vvoihv82tBe+p20KD0HlyLzARqILlktDOUOHnXeCmgRVPuw5T+PZ/Kiytb28gHHsxGiGhttgs0D4ImCNO7+MOLBALzECk7OjmPPaVYsbv3rA+P2n9vbGMCLuVhpWCKEfvLwr4WUueKnBgApmXWW6nu0bP2XqNamn79HdL71tAH3rb0n8A2CUwWRqCGhkRWYH5iKEd0RyqSbLDv1K2wGuCD+SYLwCN1I8K1IGwnhhFR/9W4lXf3hJnhUOgLw6XLQTlo826uKq+Fs9/OUjVHossT1BFiZIkug5LDvO354kaIum6uo144Ub6kqG7giLTFssS+H3vy9U= ceph-3722731a-1496-11ef-bc9a-c7b262605968 2024-05-17T21:43:30.907 DEBUG:teuthology.orchestra.run.smithi138:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDC82J4v0yngLUWHqgMnkz1BtuSxjzef52E8Eig8IVeZp7HkNJWFBdaHG++rWmW7COJuA4L7cjFG/t0n2Cv0ZDRoAWT1PN0+5I17aLtivFr77qLoi0rOxfNhp4gFrFF+9OosLjlTNKZjk1kkojWJC8yCfWY5jcl6LY+5y1YnAYfuquqK8cjLrzCjg9vvoihv82tBe+p20KD0HlyLzARqILlktDOUOHnXeCmgRVPuw5T+PZ/Kiytb28gHHsxGiGhttgs0D4ImCNO7+MOLBALzECk7OjmPPaVYsbv3rA+P2n9vbGMCLuVhpWCKEfvLwr4WUueKnBgApmXWW6nu0bP2XqNamn79HdL71tAH3rb0n8A2CUwWRqCGhkRWYH5iKEd0RyqSbLDv1K2wGuCD+SYLwCN1I8K1IGwnhhFR/9W4lXf3hJnhUOgLw6XLQTlo826uKq+Fs9/OUjVHossT1BFiZIkug5LDvO354kaIum6uo144Ub6kqG7giLTFssS+H3vy9U= ceph-3722731a-1496-11ef-bc9a-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-17T21:43:30.948 INFO:teuthology.orchestra.run.smithi138.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDC82J4v0yngLUWHqgMnkz1BtuSxjzef52E8Eig8IVeZp7HkNJWFBdaHG++rWmW7COJuA4L7cjFG/t0n2Cv0ZDRoAWT1PN0+5I17aLtivFr77qLoi0rOxfNhp4gFrFF+9OosLjlTNKZjk1kkojWJC8yCfWY5jcl6LY+5y1YnAYfuquqK8cjLrzCjg9vvoihv82tBe+p20KD0HlyLzARqILlktDOUOHnXeCmgRVPuw5T+PZ/Kiytb28gHHsxGiGhttgs0D4ImCNO7+MOLBALzECk7OjmPPaVYsbv3rA+P2n9vbGMCLuVhpWCKEfvLwr4WUueKnBgApmXWW6nu0bP2XqNamn79HdL71tAH3rb0n8A2CUwWRqCGhkRWYH5iKEd0RyqSbLDv1K2wGuCD+SYLwCN1I8K1IGwnhhFR/9W4lXf3hJnhUOgLw6XLQTlo826uKq+Fs9/OUjVHossT1BFiZIkug5LDvO354kaIum6uo144Ub6kqG7giLTFssS+H3vy9U= ceph-3722731a-1496-11ef-bc9a-c7b262605968 2024-05-17T21:43:30.960 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-05-17T21:43:31.212 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/mon.smithi107/config 2024-05-17T21:43:31.334 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:30 smithi107 ceph-mon[28434]: from='client.? 172.21.15.107:0/624168105' entity='client.admin' 2024-05-17T21:43:33.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:33.801+0000 7f7a06b62700 1 -- 172.21.15.107:0/3967845524 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a00065d30 msgr2=0x7f7a000ff630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:43:33.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:33.801+0000 7f7a06b62700 1 --2- 172.21.15.107:0/3967845524 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a00065d30 0x7f7a000ff630 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f79f40048c0 tx=0x7f79f4009e30 comp rx=0 tx=0).stop 2024-05-17T21:43:33.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:33.802+0000 7f7a06b62700 1 -- 172.21.15.107:0/3967845524 shutdown_connections 2024-05-17T21:43:33.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:33.802+0000 7f7a06b62700 1 --2- 172.21.15.107:0/3967845524 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a00065d30 0x7f7a000ff630 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:43:33.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:33.802+0000 7f7a06b62700 1 -- 172.21.15.107:0/3967845524 >> 172.21.15.107:0/3967845524 conn(0x7f7a000f70a0 msgr2=0x7f7a000f94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:43:33.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:33.802+0000 7f7a06b62700 1 -- 172.21.15.107:0/3967845524 shutdown_connections 2024-05-17T21:43:33.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:33.802+0000 7f7a06b62700 1 -- 172.21.15.107:0/3967845524 wait complete. 2024-05-17T21:43:33.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:33.802+0000 7f7a06b62700 1 Processor -- start 2024-05-17T21:43:33.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:33.802+0000 7f7a06b62700 1 -- start start 2024-05-17T21:43:33.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:33.803+0000 7f7a06b62700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a00065d30 0x7f7a0010d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:43:33.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:33.803+0000 7f7a06b62700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a0010d5d0 con 0x7f7a00065d30 2024-05-17T21:43:33.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:33.803+0000 7f7a048fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a00065d30 0x7f7a0010d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:43:33.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:33.803+0000 7f7a048fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a00065d30 0x7f7a0010d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33094/0 (socket says 172.21.15.107:33094) 2024-05-17T21:43:33.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:33.803+0000 7f7a048fe700 1 -- 172.21.15.107:0/1665255867 learned_addr learned my addr 172.21.15.107:0/1665255867 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:43:33.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:33.803+0000 7f7a048fe700 1 -- 172.21.15.107:0/1665255867 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79f4005040 con 0x7f7a00065d30 2024-05-17T21:43:33.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:33.803+0000 7f7a048fe700 1 --2- 172.21.15.107:0/1665255867 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a00065d30 0x7f7a0010d090 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f79f4009680 tx=0x7f79f4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:43:33.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:33.803+0000 7f79fd7fa700 1 -- 172.21.15.107:0/1665255867 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f79f400bed0 con 0x7f7a00065d30 2024-05-17T21:43:33.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:33.803+0000 7f79fd7fa700 1 -- 172.21.15.107:0/1665255867 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f79f400b5c0 con 0x7f7a00065d30 2024-05-17T21:43:33.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:33.803+0000 7f7a06b62700 1 -- 172.21.15.107:0/1665255867 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a0010d7d0 con 0x7f7a00065d30 2024-05-17T21:43:33.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:33.803+0000 7f79fd7fa700 1 -- 172.21.15.107:0/1665255867 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f79f401c5b0 con 0x7f7a00065d30 2024-05-17T21:43:33.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:33.804+0000 7f7a06b62700 1 -- 172.21.15.107:0/1665255867 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a0010e340 con 0x7f7a00065d30 2024-05-17T21:43:33.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:33.804+0000 7f7a06b62700 1 -- 172.21.15.107:0/1665255867 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a00102f40 con 0x7f7a00065d30 2024-05-17T21:43:33.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:33.804+0000 7f79fd7fa700 1 -- 172.21.15.107:0/1665255867 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) v1 ==== 43433+0+0 (secure 0 0 0) 0x7f79f401c710 con 0x7f7a00065d30 2024-05-17T21:43:33.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:33.805+0000 7f79fd7fa700 1 --2- 172.21.15.107:0/1665255867 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f79f00368d0 0x7f79f0038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:43:33.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:33.805+0000 7f79fd7fa700 1 -- 172.21.15.107:0/1665255867 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f79f4049960 con 0x7f7a00065d30 2024-05-17T21:43:33.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:33.805+0000 7f79fffff700 1 --2- 172.21.15.107:0/1665255867 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f79f00368d0 0x7f79f0038d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:43:33.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:33.805+0000 7f79fffff700 1 --2- 172.21.15.107:0/1665255867 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f79f00368d0 0x7f79f0038d80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f79ec009990 tx=0x7f79ec006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:43:33.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:33.812+0000 7f79fd7fa700 1 -- 172.21.15.107:0/1665255867 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f79f4025400 con 0x7f7a00065d30 2024-05-17T21:43:34.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:34.030+0000 7f7a06b62700 1 -- 172.21.15.107:0/1665255867 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f7a0005f5c0 con 0x7f7a00065d30 2024-05-17T21:43:34.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:34.035+0000 7f79fd7fa700 1 -- 172.21.15.107:0/1665255867 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f79f4011b60 con 0x7f7a00065d30 2024-05-17T21:43:34.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:34.037+0000 7f7a06b62700 1 -- 172.21.15.107:0/1665255867 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f79f00368d0 msgr2=0x7f79f0038d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:43:34.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:34.037+0000 7f7a06b62700 1 --2- 172.21.15.107:0/1665255867 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f79f00368d0 0x7f79f0038d80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f79ec009990 tx=0x7f79ec006e40 comp rx=0 tx=0).stop 2024-05-17T21:43:34.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:34.037+0000 7f7a06b62700 1 -- 172.21.15.107:0/1665255867 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a00065d30 msgr2=0x7f7a0010d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:43:34.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:34.037+0000 7f7a06b62700 1 --2- 172.21.15.107:0/1665255867 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a00065d30 0x7f7a0010d090 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f79f4009680 tx=0x7f79f4009e30 comp rx=0 tx=0).stop 2024-05-17T21:43:34.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:34.037+0000 7f7a06b62700 1 -- 172.21.15.107:0/1665255867 shutdown_connections 2024-05-17T21:43:34.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:34.037+0000 7f7a06b62700 1 --2- 172.21.15.107:0/1665255867 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f79f00368d0 0x7f79f0038d80 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:43:34.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:34.037+0000 7f7a06b62700 1 --2- 172.21.15.107:0/1665255867 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a00065d30 0x7f7a0010d090 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:43:34.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:34.037+0000 7f7a06b62700 1 -- 172.21.15.107:0/1665255867 >> 172.21.15.107:0/1665255867 conn(0x7f7a000f70a0 msgr2=0x7f7a00103b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:43:34.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:34.038+0000 7f7a06b62700 1 -- 172.21.15.107:0/1665255867 shutdown_connections 2024-05-17T21:43:34.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:34.038+0000 7f7a06b62700 1 -- 172.21.15.107:0/1665255867 wait complete. 2024-05-17T21:43:34.227 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:33 smithi107 ceph-mon[28434]: from='mgr.14162 172.21.15.107:0/3666393387' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:43:34.227 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:33 smithi107 ceph-mon[28434]: from='mgr.14162 172.21.15.107:0/3666393387' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:43:34.227 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:33 smithi107 ceph-mon[28434]: from='mgr.14162 172.21.15.107:0/3666393387' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config rm", "who": "osd/host:smithi107", "name": "osd_memory_target"}]: dispatch 2024-05-17T21:43:34.228 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:33 smithi107 ceph-mon[28434]: from='mgr.14162 172.21.15.107:0/3666393387' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:43:34.228 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:33 smithi107 ceph-mon[28434]: from='mgr.14162 172.21.15.107:0/3666393387' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi107", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-17T21:43:34.228 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:33 smithi107 ceph-mon[28434]: from='mgr.14162 172.21.15.107:0/3666393387' entity='mgr.smithi107.xhhyiw' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi107", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-05-17T21:43:34.228 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:33 smithi107 ceph-mon[28434]: from='mgr.14162 172.21.15.107:0/3666393387' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:43:34.228 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:33 smithi107 ceph-mon[28434]: Deploying daemon crash.smithi107 on smithi107 2024-05-17T21:43:34.774 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-05-17T21:43:34.774 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-05-17T21:43:35.209 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/mon.smithi107/config 2024-05-17T21:43:35.332 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:35 smithi107 ceph-mon[28434]: from='client.? 172.21.15.107:0/1665255867' entity='client.admin' 2024-05-17T21:43:36.655 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:36.660+0000 7f8045b16700 1 -- 172.21.15.107:0/3768013066 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80400fca60 msgr2=0x7f80400fce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:43:36.655 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:36.660+0000 7f8045b16700 1 --2- 172.21.15.107:0/3768013066 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80400fca60 0x7f80400fce70 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f80300048c0 tx=0x7f8030009e30 comp rx=0 tx=0).stop 2024-05-17T21:43:36.655 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:36.660+0000 7f8045b16700 1 -- 172.21.15.107:0/3768013066 shutdown_connections 2024-05-17T21:43:36.655 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:36.660+0000 7f8045b16700 1 --2- 172.21.15.107:0/3768013066 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80400fca60 0x7f80400fce70 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:43:36.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:36.660+0000 7f8045b16700 1 -- 172.21.15.107:0/3768013066 >> 172.21.15.107:0/3768013066 conn(0x7f80400f81c0 msgr2=0x7f80400fa5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:43:36.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:36.660+0000 7f8045b16700 1 -- 172.21.15.107:0/3768013066 shutdown_connections 2024-05-17T21:43:36.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:36.660+0000 7f8045b16700 1 -- 172.21.15.107:0/3768013066 wait complete. 2024-05-17T21:43:36.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:36.661+0000 7f8045b16700 1 Processor -- start 2024-05-17T21:43:36.657 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:36.661+0000 7f8045b16700 1 -- start start 2024-05-17T21:43:36.657 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:36.661+0000 7f8045b16700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80400fca60 0x7f804010d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:43:36.657 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:36.661+0000 7f8045b16700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f804010d5d0 con 0x7f80400fca60 2024-05-17T21:43:36.657 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:36.661+0000 7f803f7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80400fca60 0x7f804010d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:43:36.657 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:36.661+0000 7f803f7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80400fca60 0x7f804010d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37914/0 (socket says 172.21.15.107:37914) 2024-05-17T21:43:36.657 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:36.661+0000 7f803f7fe700 1 -- 172.21.15.107:0/230802826 learned_addr learned my addr 172.21.15.107:0/230802826 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:43:36.657 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:36.662+0000 7f803f7fe700 1 -- 172.21.15.107:0/230802826 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8030005040 con 0x7f80400fca60 2024-05-17T21:43:36.657 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:36.662+0000 7f803f7fe700 1 --2- 172.21.15.107:0/230802826 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80400fca60 0x7f804010d090 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f80300096b0 tx=0x7f8030009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:43:36.657 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:36.662+0000 7f8027fff700 1 -- 172.21.15.107:0/230802826 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f803000bed0 con 0x7f80400fca60 2024-05-17T21:43:36.657 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:36.662+0000 7f8027fff700 1 -- 172.21.15.107:0/230802826 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f803000b5c0 con 0x7f80400fca60 2024-05-17T21:43:36.657 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:36.662+0000 7f8045b16700 1 -- 172.21.15.107:0/230802826 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f804010d7d0 con 0x7f80400fca60 2024-05-17T21:43:36.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:36.662+0000 7f8027fff700 1 -- 172.21.15.107:0/230802826 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f803001c5b0 con 0x7f80400fca60 2024-05-17T21:43:36.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:36.662+0000 7f8045b16700 1 -- 172.21.15.107:0/230802826 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f804010e340 con 0x7f80400fca60 2024-05-17T21:43:36.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:36.663+0000 7f8027fff700 1 -- 172.21.15.107:0/230802826 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) v1 ==== 43433+0+0 (secure 0 0 0) 0x7f803001c820 con 0x7f80400fca60 2024-05-17T21:43:36.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:36.663+0000 7f8045b16700 1 -- 172.21.15.107:0/230802826 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8040102f70 con 0x7f80400fca60 2024-05-17T21:43:36.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:36.663+0000 7f8027fff700 1 --2- 172.21.15.107:0/230802826 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f8028036880 0x7f8028038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:43:36.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:36.663+0000 7f8027fff700 1 -- 172.21.15.107:0/230802826 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f8030049c80 con 0x7f80400fca60 2024-05-17T21:43:36.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:36.663+0000 7f803effd700 1 --2- 172.21.15.107:0/230802826 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f8028036880 0x7f8028038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:43:36.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:36.664+0000 7f803effd700 1 --2- 172.21.15.107:0/230802826 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f8028036880 0x7f8028038d30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8034009990 tx=0x7f8034006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:43:36.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:36.668+0000 7f8027fff700 1 -- 172.21.15.107:0/230802826 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f8030050080 con 0x7f80400fca60 2024-05-17T21:43:36.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:36.862+0000 7f8045b16700 1 -- 172.21.15.107:0/230802826 --> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f804005f660 con 0x7f8028036880 2024-05-17T21:43:36.859 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:36.865+0000 7f8027fff700 1 -- 172.21.15.107:0/230802826 <== mgr.14162 v2:172.21.15.107:6800/3405524625 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f804005f660 con 0x7f8028036880 2024-05-17T21:43:36.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:36.867+0000 7f8045b16700 1 -- 172.21.15.107:0/230802826 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f8028036880 msgr2=0x7f8028038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:43:36.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:36.867+0000 7f8045b16700 1 --2- 172.21.15.107:0/230802826 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f8028036880 0x7f8028038d30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8034009990 tx=0x7f8034006e40 comp rx=0 tx=0).stop 2024-05-17T21:43:36.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:36.867+0000 7f8045b16700 1 -- 172.21.15.107:0/230802826 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80400fca60 msgr2=0x7f804010d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:43:36.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:36.867+0000 7f8045b16700 1 --2- 172.21.15.107:0/230802826 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80400fca60 0x7f804010d090 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f80300096b0 tx=0x7f8030009e30 comp rx=0 tx=0).stop 2024-05-17T21:43:36.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:36.867+0000 7f8045b16700 1 -- 172.21.15.107:0/230802826 shutdown_connections 2024-05-17T21:43:36.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:36.867+0000 7f8045b16700 1 --2- 172.21.15.107:0/230802826 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f8028036880 0x7f8028038d30 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:43:36.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:36.867+0000 7f8045b16700 1 --2- 172.21.15.107:0/230802826 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80400fca60 0x7f804010d090 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:43:36.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:36.867+0000 7f8045b16700 1 -- 172.21.15.107:0/230802826 >> 172.21.15.107:0/230802826 conn(0x7f80400f81c0 msgr2=0x7f8040103ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:43:36.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:36.867+0000 7f8045b16700 1 -- 172.21.15.107:0/230802826 shutdown_connections 2024-05-17T21:43:36.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:36.867+0000 7f8045b16700 1 -- 172.21.15.107:0/230802826 wait complete. 2024-05-17T21:43:38.009 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:37 smithi107 ceph-mon[28434]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:43:38.009 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:37 smithi107 ceph-mon[28434]: from='mgr.14162 172.21.15.107:0/3666393387' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:43:38.373 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi138 2024-05-17T21:43:38.373 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-17T21:43:38.373 DEBUG:teuthology.orchestra.run.smithi138:> dd of=/etc/ceph/ceph.conf 2024-05-17T21:43:38.392 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-17T21:43:38.392 DEBUG:teuthology.orchestra.run.smithi138:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-17T21:43:38.451 INFO:tasks.cephadm:Adding host smithi138 to orchestrator... 2024-05-17T21:43:38.451 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -- ceph orch host add smithi138 2024-05-17T21:43:38.919 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/mon.smithi107/config 2024-05-17T21:43:40.259 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:39 smithi107 ceph-mon[28434]: from='mgr.14162 172.21.15.107:0/3666393387' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:43:40.259 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:39 smithi107 ceph-mon[28434]: from='mgr.14162 172.21.15.107:0/3666393387' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:43:40.259 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:39 smithi107 ceph-mon[28434]: from='mgr.14162 172.21.15.107:0/3666393387' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:43:40.259 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:39 smithi107 ceph-mon[28434]: from='mgr.14162 172.21.15.107:0/3666393387' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:43:40.259 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:39 smithi107 ceph-mon[28434]: Deploying daemon node-exporter.smithi107 on smithi107 2024-05-17T21:43:40.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:40.487+0000 7f3fca71e700 1 -- 172.21.15.107:0/696088653 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3fc40665c0 msgr2=0x7f3fc40ff630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:43:40.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:40.487+0000 7f3fca71e700 1 --2- 172.21.15.107:0/696088653 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3fc40665c0 0x7f3fc40ff630 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3fb40048c0 tx=0x7f3fb4009e30 comp rx=0 tx=0).stop 2024-05-17T21:43:40.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:40.487+0000 7f3fca71e700 1 -- 172.21.15.107:0/696088653 shutdown_connections 2024-05-17T21:43:40.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:40.487+0000 7f3fca71e700 1 --2- 172.21.15.107:0/696088653 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3fc40665c0 0x7f3fc40ff630 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:43:40.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:40.487+0000 7f3fca71e700 1 -- 172.21.15.107:0/696088653 >> 172.21.15.107:0/696088653 conn(0x7f3fc40f70a0 msgr2=0x7f3fc40f94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:43:40.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:40.487+0000 7f3fca71e700 1 -- 172.21.15.107:0/696088653 shutdown_connections 2024-05-17T21:43:40.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:40.487+0000 7f3fca71e700 1 -- 172.21.15.107:0/696088653 wait complete. 2024-05-17T21:43:40.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:40.488+0000 7f3fca71e700 1 Processor -- start 2024-05-17T21:43:40.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:40.488+0000 7f3fca71e700 1 -- start start 2024-05-17T21:43:40.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:40.488+0000 7f3fca71e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3fc40665c0 0x7f3fc410d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:43:40.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:40.488+0000 7f3fca71e700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc410d5d0 con 0x7f3fc40665c0 2024-05-17T21:43:40.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:40.488+0000 7f3fc3fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3fc40665c0 0x7f3fc410d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:43:40.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:40.488+0000 7f3fc3fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3fc40665c0 0x7f3fc410d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37928/0 (socket says 172.21.15.107:37928) 2024-05-17T21:43:40.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:40.488+0000 7f3fc3fff700 1 -- 172.21.15.107:0/2942736491 learned_addr learned my addr 172.21.15.107:0/2942736491 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:43:40.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:40.489+0000 7f3fc3fff700 1 -- 172.21.15.107:0/2942736491 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fb4005040 con 0x7f3fc40665c0 2024-05-17T21:43:40.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:40.489+0000 7f3fc3fff700 1 --2- 172.21.15.107:0/2942736491 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3fc40665c0 0x7f3fc410d090 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3fb4009680 tx=0x7f3fb4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:43:40.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:40.489+0000 7f3fc0ff9700 1 -- 172.21.15.107:0/2942736491 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3fb400bed0 con 0x7f3fc40665c0 2024-05-17T21:43:40.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:40.489+0000 7f3fc0ff9700 1 -- 172.21.15.107:0/2942736491 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3fb400b5c0 con 0x7f3fc40665c0 2024-05-17T21:43:40.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:40.489+0000 7f3fc0ff9700 1 -- 172.21.15.107:0/2942736491 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3fb401c5b0 con 0x7f3fc40665c0 2024-05-17T21:43:40.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:40.489+0000 7f3fca71e700 1 -- 172.21.15.107:0/2942736491 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fc410d7d0 con 0x7f3fc40665c0 2024-05-17T21:43:40.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:40.489+0000 7f3fca71e700 1 -- 172.21.15.107:0/2942736491 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fc410e340 con 0x7f3fc40665c0 2024-05-17T21:43:40.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:40.490+0000 7f3fc0ff9700 1 -- 172.21.15.107:0/2942736491 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) v1 ==== 43433+0+0 (secure 0 0 0) 0x7f3fb401c710 con 0x7f3fc40665c0 2024-05-17T21:43:40.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:40.490+0000 7f3fca71e700 1 -- 172.21.15.107:0/2942736491 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fc4102f40 con 0x7f3fc40665c0 2024-05-17T21:43:40.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:40.490+0000 7f3fc0ff9700 1 --2- 172.21.15.107:0/2942736491 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f3fb0036880 0x7f3fb0038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:43:40.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:40.490+0000 7f3fc0ff9700 1 -- 172.21.15.107:0/2942736491 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f3fb40499b0 con 0x7f3fc40665c0 2024-05-17T21:43:40.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:40.490+0000 7f3fc37fe700 1 --2- 172.21.15.107:0/2942736491 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f3fb0036880 0x7f3fb0038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:43:40.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:40.491+0000 7f3fc37fe700 1 --2- 172.21.15.107:0/2942736491 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f3fb0036880 0x7f3fb0038d30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f3fac009990 tx=0x7f3fac006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:43:40.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:40.497+0000 7f3fc0ff9700 1 -- 172.21.15.107:0/2942736491 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f3fb4025400 con 0x7f3fc40665c0 2024-05-17T21:43:40.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:40.703+0000 7f3fca71e700 1 -- 172.21.15.107:0/2942736491 --> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi138", "target": ["mon-mgr", ""]}) v1 -- 0x7f3fc410e580 con 0x7f3fb0036880 2024-05-17T21:43:42.104 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:41 smithi107 ceph-mon[28434]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi138", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:43:42.104 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:41 smithi107 ceph-mon[28434]: Deploying cephadm binary to smithi138 2024-05-17T21:43:42.683 INFO:teuthology.orchestra.run.smithi107.stdout:Added host 'smithi138' with addr '172.21.15.138' 2024-05-17T21:43:42.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:42.688+0000 7f3fc0ff9700 1 -- 172.21.15.107:0/2942736491 <== mgr.14162 v2:172.21.15.107:6800/3405524625 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f3fc410e580 con 0x7f3fb0036880 2024-05-17T21:43:42.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:42.691+0000 7f3fca71e700 1 -- 172.21.15.107:0/2942736491 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f3fb0036880 msgr2=0x7f3fb0038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:43:42.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:42.691+0000 7f3fca71e700 1 --2- 172.21.15.107:0/2942736491 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f3fb0036880 0x7f3fb0038d30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f3fac009990 tx=0x7f3fac006e40 comp rx=0 tx=0).stop 2024-05-17T21:43:42.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:42.691+0000 7f3fca71e700 1 -- 172.21.15.107:0/2942736491 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3fc40665c0 msgr2=0x7f3fc410d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:43:42.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:42.691+0000 7f3fca71e700 1 --2- 172.21.15.107:0/2942736491 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3fc40665c0 0x7f3fc410d090 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3fb4009680 tx=0x7f3fb4009e30 comp rx=0 tx=0).stop 2024-05-17T21:43:42.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:42.691+0000 7f3fca71e700 1 -- 172.21.15.107:0/2942736491 shutdown_connections 2024-05-17T21:43:42.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:42.691+0000 7f3fca71e700 1 --2- 172.21.15.107:0/2942736491 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f3fb0036880 0x7f3fb0038d30 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:43:42.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:42.691+0000 7f3fca71e700 1 --2- 172.21.15.107:0/2942736491 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3fc40665c0 0x7f3fc410d090 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:43:42.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:42.691+0000 7f3fca71e700 1 -- 172.21.15.107:0/2942736491 >> 172.21.15.107:0/2942736491 conn(0x7f3fc40f70a0 msgr2=0x7f3fc4103b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:43:42.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:42.691+0000 7f3fca71e700 1 -- 172.21.15.107:0/2942736491 shutdown_connections 2024-05-17T21:43:42.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:42.691+0000 7f3fca71e700 1 -- 172.21.15.107:0/2942736491 wait complete. 2024-05-17T21:43:43.390 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:43 smithi107 ceph-mon[28434]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T21:43:43.390 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:43 smithi107 ceph-mon[28434]: from='mgr.14162 172.21.15.107:0/3666393387' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:43:43.390 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:43 smithi107 ceph-mon[28434]: from='mgr.14162 172.21.15.107:0/3666393387' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:43:43.415 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -- ceph orch host ls --format=json 2024-05-17T21:43:43.906 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/mon.smithi107/config 2024-05-17T21:43:44.347 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:44 smithi107 ceph-mon[28434]: Added host smithi138 2024-05-17T21:43:44.347 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:44 smithi107 ceph-mon[28434]: from='mgr.14162 172.21.15.107:0/3666393387' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:43:44.347 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:44 smithi107 ceph-mon[28434]: from='mgr.14162 172.21.15.107:0/3666393387' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:43:44.347 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:44 smithi107 ceph-mon[28434]: from='mgr.14162 172.21.15.107:0/3666393387' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:43:44.347 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:44 smithi107 ceph-mon[28434]: from='mgr.14162 172.21.15.107:0/3666393387' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:43:45.293 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:45 smithi107 ceph-mon[28434]: Deploying daemon alertmanager.smithi107 on smithi107 2024-05-17T21:43:45.294 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:45 smithi107 ceph-mon[28434]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T21:43:45.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:45.545+0000 7f997a6bc700 1 -- 172.21.15.107:0/2268458757 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9974065cd0 msgr2=0x7f99740ff690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:43:45.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:45.545+0000 7f997a6bc700 1 --2- 172.21.15.107:0/2268458757 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9974065cd0 0x7f99740ff690 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f99640048c0 tx=0x7f9964009e30 comp rx=0 tx=0).stop 2024-05-17T21:43:45.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:45.545+0000 7f997a6bc700 1 -- 172.21.15.107:0/2268458757 shutdown_connections 2024-05-17T21:43:45.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:45.545+0000 7f997a6bc700 1 --2- 172.21.15.107:0/2268458757 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9974065cd0 0x7f99740ff690 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:43:45.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:45.545+0000 7f997a6bc700 1 -- 172.21.15.107:0/2268458757 >> 172.21.15.107:0/2268458757 conn(0x7f99740f70a0 msgr2=0x7f99740f94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:43:45.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:45.545+0000 7f997a6bc700 1 -- 172.21.15.107:0/2268458757 shutdown_connections 2024-05-17T21:43:45.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:45.545+0000 7f997a6bc700 1 -- 172.21.15.107:0/2268458757 wait complete. 2024-05-17T21:43:45.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:45.546+0000 7f997a6bc700 1 Processor -- start 2024-05-17T21:43:45.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:45.546+0000 7f997a6bc700 1 -- start start 2024-05-17T21:43:45.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:45.546+0000 7f997a6bc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9974065cd0 0x7f997410f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:43:45.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:45.546+0000 7f997a6bc700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f997410f860 con 0x7f9974065cd0 2024-05-17T21:43:45.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:45.546+0000 7f9973fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9974065cd0 0x7f997410f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:43:45.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:45.546+0000 7f9973fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9974065cd0 0x7f997410f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44458/0 (socket says 172.21.15.107:44458) 2024-05-17T21:43:45.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:45.546+0000 7f9973fff700 1 -- 172.21.15.107:0/1508207014 learned_addr learned my addr 172.21.15.107:0/1508207014 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:43:45.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:45.546+0000 7f9973fff700 1 -- 172.21.15.107:0/1508207014 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9964005040 con 0x7f9974065cd0 2024-05-17T21:43:45.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:45.547+0000 7f9973fff700 1 --2- 172.21.15.107:0/1508207014 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9974065cd0 0x7f997410f320 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f996400b900 tx=0x7f9964009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:43:45.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:45.547+0000 7f99717fa700 1 -- 172.21.15.107:0/1508207014 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f996400b5c0 con 0x7f9974065cd0 2024-05-17T21:43:45.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:45.547+0000 7f99717fa700 1 -- 172.21.15.107:0/1508207014 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f996400be70 con 0x7f9974065cd0 2024-05-17T21:43:45.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:45.547+0000 7f997a6bc700 1 -- 172.21.15.107:0/1508207014 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f997410fa60 con 0x7f9974065cd0 2024-05-17T21:43:45.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:45.547+0000 7f99717fa700 1 -- 172.21.15.107:0/1508207014 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f996401c590 con 0x7f9974065cd0 2024-05-17T21:43:45.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:45.547+0000 7f997a6bc700 1 -- 172.21.15.107:0/1508207014 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9974110580 con 0x7f9974065cd0 2024-05-17T21:43:45.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:45.548+0000 7f99717fa700 1 -- 172.21.15.107:0/1508207014 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) v1 ==== 43433+0+0 (secure 0 0 0) 0x7f996401c800 con 0x7f9974065cd0 2024-05-17T21:43:45.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:45.548+0000 7f997a6bc700 1 -- 172.21.15.107:0/1508207014 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9974102f40 con 0x7f9974065cd0 2024-05-17T21:43:45.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:45.548+0000 7f99717fa700 1 --2- 172.21.15.107:0/1508207014 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f99600368d0 0x7f9960038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:43:45.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:45.548+0000 7f99717fa700 1 -- 172.21.15.107:0/1508207014 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f9964049490 con 0x7f9974065cd0 2024-05-17T21:43:45.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:45.548+0000 7f996bfff700 1 --2- 172.21.15.107:0/1508207014 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f99600368d0 0x7f9960038d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:43:45.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:45.548+0000 7f996bfff700 1 --2- 172.21.15.107:0/1508207014 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f99600368d0 0x7f9960038d80 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f995c009990 tx=0x7f995c006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:43:45.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:45.553+0000 7f99717fa700 1 -- 172.21.15.107:0/1508207014 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f9964011960 con 0x7f9974065cd0 2024-05-17T21:43:45.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:45.751+0000 7f997a6bc700 1 -- 172.21.15.107:0/1508207014 --> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f997405f660 con 0x7f99600368d0 2024-05-17T21:43:45.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:45.752+0000 7f99717fa700 1 -- 172.21.15.107:0/1508207014 <== mgr.14162 v2:172.21.15.107:6800/3405524625 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+161 (secure 0 0 0) 0x7f997405f660 con 0x7f99600368d0 2024-05-17T21:43:45.747 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:43:45.747 INFO:teuthology.orchestra.run.smithi107.stdout:[{"addr": "172.21.15.107", "hostname": "smithi107", "labels": [], "status": ""}, {"addr": "172.21.15.138", "hostname": "smithi138", "labels": [], "status": ""}] 2024-05-17T21:43:45.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:45.754+0000 7f997a6bc700 1 -- 172.21.15.107:0/1508207014 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f99600368d0 msgr2=0x7f9960038d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:43:45.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:45.754+0000 7f997a6bc700 1 --2- 172.21.15.107:0/1508207014 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f99600368d0 0x7f9960038d80 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f995c009990 tx=0x7f995c006e40 comp rx=0 tx=0).stop 2024-05-17T21:43:45.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:45.754+0000 7f997a6bc700 1 -- 172.21.15.107:0/1508207014 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9974065cd0 msgr2=0x7f997410f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:43:45.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:45.754+0000 7f997a6bc700 1 --2- 172.21.15.107:0/1508207014 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9974065cd0 0x7f997410f320 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f996400b900 tx=0x7f9964009e30 comp rx=0 tx=0).stop 2024-05-17T21:43:45.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:45.754+0000 7f997a6bc700 1 -- 172.21.15.107:0/1508207014 shutdown_connections 2024-05-17T21:43:45.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:45.754+0000 7f997a6bc700 1 --2- 172.21.15.107:0/1508207014 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f99600368d0 0x7f9960038d80 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:43:45.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:45.754+0000 7f997a6bc700 1 --2- 172.21.15.107:0/1508207014 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9974065cd0 0x7f997410f320 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:43:45.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:45.754+0000 7f997a6bc700 1 -- 172.21.15.107:0/1508207014 >> 172.21.15.107:0/1508207014 conn(0x7f99740f70a0 msgr2=0x7f99740f94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:43:45.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:45.754+0000 7f997a6bc700 1 -- 172.21.15.107:0/1508207014 shutdown_connections 2024-05-17T21:43:45.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:45.754+0000 7f997a6bc700 1 -- 172.21.15.107:0/1508207014 wait complete. 2024-05-17T21:43:46.893 INFO:tasks.cephadm:Setting crush tunables to default 2024-05-17T21:43:46.894 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -- ceph osd crush tunables default 2024-05-17T21:43:47.452 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/mon.smithi107/config 2024-05-17T21:43:47.575 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:47 smithi107 ceph-mon[28434]: from='client.14190 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-17T21:43:47.575 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:47 smithi107 ceph-mon[28434]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T21:43:48.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:48 smithi107 ceph-mon[28434]: from='mgr.14162 172.21.15.107:0/3666393387' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:43:49.272 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:49 smithi107 ceph-mon[28434]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T21:43:49.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:49.380+0000 7fe8a45ac700 1 -- 172.21.15.107:0/3085731879 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe89c0fe080 msgr2=0x7fe89c0fe490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:43:49.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:49.380+0000 7fe8a45ac700 1 --2- 172.21.15.107:0/3085731879 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe89c0fe080 0x7fe89c0fe490 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe8980048c0 tx=0x7fe898009e30 comp rx=0 tx=0).stop 2024-05-17T21:43:49.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:49.380+0000 7fe8a45ac700 1 -- 172.21.15.107:0/3085731879 shutdown_connections 2024-05-17T21:43:49.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:49.380+0000 7fe8a45ac700 1 --2- 172.21.15.107:0/3085731879 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe89c0fe080 0x7fe89c0fe490 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:43:49.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:49.380+0000 7fe8a45ac700 1 -- 172.21.15.107:0/3085731879 >> 172.21.15.107:0/3085731879 conn(0x7fe89c0f6820 msgr2=0x7fe89c0f8c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:43:49.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:49.380+0000 7fe8a45ac700 1 -- 172.21.15.107:0/3085731879 shutdown_connections 2024-05-17T21:43:49.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:49.381+0000 7fe8a45ac700 1 -- 172.21.15.107:0/3085731879 wait complete. 2024-05-17T21:43:49.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:49.381+0000 7fe8a45ac700 1 Processor -- start 2024-05-17T21:43:49.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:49.381+0000 7fe8a45ac700 1 -- start start 2024-05-17T21:43:49.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:49.381+0000 7fe8a45ac700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe89c0fe080 0x7fe89c10d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:43:49.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:49.381+0000 7fe8a45ac700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe89c10d5e0 con 0x7fe89c0fe080 2024-05-17T21:43:49.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:49.381+0000 7fe8a2348700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe89c0fe080 0x7fe89c10d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:43:49.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:49.381+0000 7fe8a2348700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe89c0fe080 0x7fe89c10d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44468/0 (socket says 172.21.15.107:44468) 2024-05-17T21:43:49.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:49.381+0000 7fe8a2348700 1 -- 172.21.15.107:0/2385318981 learned_addr learned my addr 172.21.15.107:0/2385318981 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:43:49.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:49.381+0000 7fe8a2348700 1 -- 172.21.15.107:0/2385318981 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe898005040 con 0x7fe89c0fe080 2024-05-17T21:43:49.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:49.382+0000 7fe8a2348700 1 --2- 172.21.15.107:0/2385318981 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe89c0fe080 0x7fe89c10d0a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe898009680 tx=0x7fe898009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:43:49.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:49.382+0000 7fe892ffd700 1 -- 172.21.15.107:0/2385318981 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe89800b5c0 con 0x7fe89c0fe080 2024-05-17T21:43:49.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:49.382+0000 7fe892ffd700 1 -- 172.21.15.107:0/2385318981 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe89800be70 con 0x7fe89c0fe080 2024-05-17T21:43:49.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:49.382+0000 7fe8a45ac700 1 -- 172.21.15.107:0/2385318981 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe89c10d7e0 con 0x7fe89c0fe080 2024-05-17T21:43:49.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:49.382+0000 7fe892ffd700 1 -- 172.21.15.107:0/2385318981 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe8980135a0 con 0x7fe89c0fe080 2024-05-17T21:43:49.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:49.382+0000 7fe8a45ac700 1 -- 172.21.15.107:0/2385318981 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe89c10e350 con 0x7fe89c0fe080 2024-05-17T21:43:49.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:49.383+0000 7fe892ffd700 1 -- 172.21.15.107:0/2385318981 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) v1 ==== 43433+0+0 (secure 0 0 0) 0x7fe898004d30 con 0x7fe89c0fe080 2024-05-17T21:43:49.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:49.383+0000 7fe892ffd700 1 --2- 172.21.15.107:0/2385318981 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7fe888036880 0x7fe888038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:43:49.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:49.383+0000 7fe8a45ac700 1 -- 172.21.15.107:0/2385318981 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe89c102f50 con 0x7fe89c0fe080 2024-05-17T21:43:49.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:49.383+0000 7fe892ffd700 1 -- 172.21.15.107:0/2385318981 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fe898049310 con 0x7fe89c0fe080 2024-05-17T21:43:49.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:49.383+0000 7fe8a1b47700 1 --2- 172.21.15.107:0/2385318981 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7fe888036880 0x7fe888038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:43:49.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:49.383+0000 7fe8a1b47700 1 --2- 172.21.15.107:0/2385318981 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7fe888036880 0x7fe888038d30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fe88c009990 tx=0x7fe88c006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:43:49.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:49.388+0000 7fe892ffd700 1 -- 172.21.15.107:0/2385318981 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fe898021020 con 0x7fe89c0fe080 2024-05-17T21:43:49.584 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:49.591+0000 7fe8a45ac700 1 -- 172.21.15.107:0/2385318981 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fe89c04bed0 con 0x7fe89c0fe080 2024-05-17T21:43:50.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:50.158+0000 7fe892ffd700 1 -- 172.21.15.107:0/2385318981 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7fe89801ca90 con 0x7fe89c0fe080 2024-05-17T21:43:50.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:50.160+0000 7fe8a45ac700 1 -- 172.21.15.107:0/2385318981 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7fe888036880 msgr2=0x7fe888038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:43:50.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:50.160+0000 7fe8a45ac700 1 --2- 172.21.15.107:0/2385318981 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7fe888036880 0x7fe888038d30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fe88c009990 tx=0x7fe88c006e40 comp rx=0 tx=0).stop 2024-05-17T21:43:50.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:50.160+0000 7fe8a45ac700 1 -- 172.21.15.107:0/2385318981 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe89c0fe080 msgr2=0x7fe89c10d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:43:50.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:50.160+0000 7fe8a45ac700 1 --2- 172.21.15.107:0/2385318981 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe89c0fe080 0x7fe89c10d0a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe898009680 tx=0x7fe898009e30 comp rx=0 tx=0).stop 2024-05-17T21:43:50.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:50.160+0000 7fe8a45ac700 1 -- 172.21.15.107:0/2385318981 shutdown_connections 2024-05-17T21:43:50.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:50.160+0000 7fe8a45ac700 1 --2- 172.21.15.107:0/2385318981 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7fe888036880 0x7fe888038d30 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:43:50.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:50.160+0000 7fe8a45ac700 1 --2- 172.21.15.107:0/2385318981 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe89c0fe080 0x7fe89c10d0a0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:43:50.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:50.160+0000 7fe8a45ac700 1 -- 172.21.15.107:0/2385318981 >> 172.21.15.107:0/2385318981 conn(0x7fe89c0f6820 msgr2=0x7fe89c103b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:43:50.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:50.160+0000 7fe8a45ac700 1 -- 172.21.15.107:0/2385318981 shutdown_connections 2024-05-17T21:43:50.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:43:50.160+0000 7fe8a45ac700 1 -- 172.21.15.107:0/2385318981 wait complete. 2024-05-17T21:43:50.155 INFO:teuthology.orchestra.run.smithi107.stderr:adjusted tunables profile to default 2024-05-17T21:43:50.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:50 smithi107 ceph-mon[28434]: from='client.? 172.21.15.107:0/2385318981' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-17T21:43:50.996 INFO:tasks.cephadm:Adding mon.smithi107 on smithi107 2024-05-17T21:43:50.996 INFO:tasks.cephadm:Adding mon.smithi138 on smithi138 2024-05-17T21:43:50.996 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -- ceph orch apply mon '2;smithi107:172.21.15.107=smithi107;smithi138:172.21.15.138=smithi138' 2024-05-17T21:43:51.247 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-17T21:43:51.294 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-17T21:43:51.347 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:51 smithi107 ceph-mon[28434]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T21:43:51.347 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:51 smithi107 ceph-mon[28434]: from='client.? 172.21.15.107:0/2385318981' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-17T21:43:51.347 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:51 smithi107 ceph-mon[28434]: osdmap e4: 0 total, 0 up, 0 in 2024-05-17T21:43:52.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:52.829+0000 7f004747b700 1 -- 172.21.15.138:0/723058013 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f00400fca30 msgr2=0x7f00400fce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:43:52.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:52.829+0000 7f004747b700 1 --2- 172.21.15.138:0/723058013 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f00400fca30 0x7f00400fce40 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f00340048c0 tx=0x7f0034009e30 comp rx=0 tx=0).stop 2024-05-17T21:43:52.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:52.829+0000 7f004747b700 1 -- 172.21.15.138:0/723058013 shutdown_connections 2024-05-17T21:43:52.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:52.829+0000 7f004747b700 1 --2- 172.21.15.138:0/723058013 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f00400fca30 0x7f00400fce40 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:43:52.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:52.829+0000 7f004747b700 1 -- 172.21.15.138:0/723058013 >> 172.21.15.138:0/723058013 conn(0x7f00400f8170 msgr2=0x7f00400fa5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:43:52.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:52.829+0000 7f004747b700 1 -- 172.21.15.138:0/723058013 shutdown_connections 2024-05-17T21:43:52.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:52.829+0000 7f004747b700 1 -- 172.21.15.138:0/723058013 wait complete. 2024-05-17T21:43:52.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:52.829+0000 7f004747b700 1 Processor -- start 2024-05-17T21:43:52.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:52.829+0000 7f004747b700 1 -- start start 2024-05-17T21:43:52.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:52.830+0000 7f004747b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f00400fca30 0x7f004010d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:43:52.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:52.830+0000 7f004747b700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f004010d5f0 con 0x7f00400fca30 2024-05-17T21:43:52.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:52.830+0000 7f0045217700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f00400fca30 0x7f004010d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:43:52.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:52.830+0000 7f0045217700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f00400fca30 0x7f004010d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.138:49206/0 (socket says 172.21.15.138:49206) 2024-05-17T21:43:52.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:52.830+0000 7f0045217700 1 -- 172.21.15.138:0/1169274123 learned_addr learned my addr 172.21.15.138:0/1169274123 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-17T21:43:52.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:52.830+0000 7f0045217700 1 -- 172.21.15.138:0/1169274123 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0034005040 con 0x7f00400fca30 2024-05-17T21:43:52.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:52.830+0000 7f0045217700 1 --2- 172.21.15.138:0/1169274123 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f00400fca30 0x7f004010d0b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f00340096b0 tx=0x7f0034009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:43:52.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:52.830+0000 7f0031ffb700 1 -- 172.21.15.138:0/1169274123 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f003400bed0 con 0x7f00400fca30 2024-05-17T21:43:52.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:52.831+0000 7f0031ffb700 1 -- 172.21.15.138:0/1169274123 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f003400b5c0 con 0x7f00400fca30 2024-05-17T21:43:52.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:52.831+0000 7f004747b700 1 -- 172.21.15.138:0/1169274123 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f004010d7f0 con 0x7f00400fca30 2024-05-17T21:43:52.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:52.831+0000 7f0031ffb700 1 -- 172.21.15.138:0/1169274123 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f003401c5b0 con 0x7f00400fca30 2024-05-17T21:43:52.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:52.831+0000 7f004747b700 1 -- 172.21.15.138:0/1169274123 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f004010e360 con 0x7f00400fca30 2024-05-17T21:43:52.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:52.832+0000 7f0031ffb700 1 -- 172.21.15.138:0/1169274123 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) v1 ==== 43433+0+0 (secure 0 0 0) 0x7f003401c820 con 0x7f00400fca30 2024-05-17T21:43:52.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:52.832+0000 7f0031ffb700 1 --2- 172.21.15.138:0/1169274123 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f002c036880 0x7f002c038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:43:52.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:52.832+0000 7f004747b700 1 -- 172.21.15.138:0/1169274123 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0040102f40 con 0x7f00400fca30 2024-05-17T21:43:52.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:52.832+0000 7f0031ffb700 1 -- 172.21.15.138:0/1169274123 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f0034049b00 con 0x7f00400fca30 2024-05-17T21:43:52.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:52.832+0000 7f0044a16700 1 --2- 172.21.15.138:0/1169274123 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f002c036880 0x7f002c038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:43:52.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:52.833+0000 7f0044a16700 1 --2- 172.21.15.138:0/1169274123 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f002c036880 0x7f002c038d30 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f003c009990 tx=0x7f003c006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:43:52.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:52.837+0000 7f0031ffb700 1 -- 172.21.15.138:0/1169274123 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f0034023300 con 0x7f00400fca30 2024-05-17T21:43:53.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:53.032+0000 7f004747b700 1 -- 172.21.15.138:0/1169274123 --> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi107:172.21.15.107=smithi107;smithi138:172.21.15.138=smithi138", "target": ["mon-mgr", ""]}) v1 -- 0x7f004010e650 con 0x7f002c036880 2024-05-17T21:43:53.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:53.041+0000 7f0031ffb700 1 -- 172.21.15.138:0/1169274123 <== mgr.14162 v2:172.21.15.107:6800/3405524625 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f004010e650 con 0x7f002c036880 2024-05-17T21:43:53.035 INFO:teuthology.orchestra.run.smithi138.stdout:Scheduled mon update... 2024-05-17T21:43:53.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:53.042+0000 7f004747b700 1 -- 172.21.15.138:0/1169274123 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f002c036880 msgr2=0x7f002c038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:43:53.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:53.042+0000 7f004747b700 1 --2- 172.21.15.138:0/1169274123 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f002c036880 0x7f002c038d30 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f003c009990 tx=0x7f003c006e40 comp rx=0 tx=0).stop 2024-05-17T21:43:53.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:53.042+0000 7f004747b700 1 -- 172.21.15.138:0/1169274123 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f00400fca30 msgr2=0x7f004010d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:43:53.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:53.042+0000 7f004747b700 1 --2- 172.21.15.138:0/1169274123 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f00400fca30 0x7f004010d0b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f00340096b0 tx=0x7f0034009e30 comp rx=0 tx=0).stop 2024-05-17T21:43:53.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:53.042+0000 7f004747b700 1 -- 172.21.15.138:0/1169274123 shutdown_connections 2024-05-17T21:43:53.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:53.042+0000 7f004747b700 1 --2- 172.21.15.138:0/1169274123 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f002c036880 0x7f002c038d30 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:43:53.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:53.042+0000 7f004747b700 1 --2- 172.21.15.138:0/1169274123 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f00400fca30 0x7f004010d0b0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:43:53.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:53.042+0000 7f004747b700 1 -- 172.21.15.138:0/1169274123 >> 172.21.15.138:0/1169274123 conn(0x7f00400f8170 msgr2=0x7f0040103b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:43:53.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:53.042+0000 7f004747b700 1 -- 172.21.15.138:0/1169274123 shutdown_connections 2024-05-17T21:43:53.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:53.042+0000 7f004747b700 1 -- 172.21.15.138:0/1169274123 wait complete. 2024-05-17T21:43:53.436 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:53 smithi107 ceph-mon[28434]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T21:43:53.436 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:53 smithi107 ceph-mon[28434]: from='mgr.14162 172.21.15.107:0/3666393387' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:43:53.621 DEBUG:teuthology.orchestra.run.smithi138:mon.smithi138> sudo journalctl -f -n 0 -u ceph-3722731a-1496-11ef-bc9a-c7b262605968@mon.smithi138.service 2024-05-17T21:43:53.624 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-17T21:43:53.624 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -- ceph mon dump -f json 2024-05-17T21:43:53.904 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-17T21:43:53.951 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-17T21:43:54.354 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:54 smithi107 ceph-mon[28434]: from='client.14194 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi107:172.21.15.107=smithi107;smithi138:172.21.15.138=smithi138", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:43:54.354 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:54 smithi107 ceph-mon[28434]: Saving service mon spec with placement smithi107:172.21.15.107=smithi107;smithi138:172.21.15.138=smithi138;count:2 2024-05-17T21:43:55.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:55.291+0000 7f3658942700 1 -- 172.21.15.138:0/3576665358 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3654066960 msgr2=0x7f3654101a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:43:55.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:55.291+0000 7f3658942700 1 --2- 172.21.15.138:0/3576665358 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3654066960 0x7f3654101a90 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f36440048c0 tx=0x7f3644009e30 comp rx=0 tx=0).stop 2024-05-17T21:43:55.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:55.291+0000 7f3658942700 1 -- 172.21.15.138:0/3576665358 shutdown_connections 2024-05-17T21:43:55.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:55.291+0000 7f3658942700 1 --2- 172.21.15.138:0/3576665358 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3654066960 0x7f3654101a90 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:43:55.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:55.291+0000 7f3658942700 1 -- 172.21.15.138:0/3576665358 >> 172.21.15.138:0/3576665358 conn(0x7f36540f70a0 msgr2=0x7f36540f94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:43:55.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:55.291+0000 7f3658942700 1 -- 172.21.15.138:0/3576665358 shutdown_connections 2024-05-17T21:43:55.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:55.291+0000 7f3658942700 1 -- 172.21.15.138:0/3576665358 wait complete. 2024-05-17T21:43:55.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:55.291+0000 7f3658942700 1 Processor -- start 2024-05-17T21:43:55.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:55.292+0000 7f3658942700 1 -- start start 2024-05-17T21:43:55.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:55.292+0000 7f3658942700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3654066960 0x7f365410f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:43:55.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:55.292+0000 7f3658942700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f365410f840 con 0x7f3654066960 2024-05-17T21:43:55.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:55.292+0000 7f365259c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3654066960 0x7f365410f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:43:55.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:55.292+0000 7f365259c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3654066960 0x7f365410f300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.138:60996/0 (socket says 172.21.15.138:60996) 2024-05-17T21:43:55.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:55.292+0000 7f365259c700 1 -- 172.21.15.138:0/1415944734 learned_addr learned my addr 172.21.15.138:0/1415944734 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-17T21:43:55.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:55.292+0000 7f365259c700 1 -- 172.21.15.138:0/1415944734 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3644005040 con 0x7f3654066960 2024-05-17T21:43:55.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:55.292+0000 7f365259c700 1 --2- 172.21.15.138:0/1415944734 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3654066960 0x7f365410f300 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3644009680 tx=0x7f3644009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:43:55.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:55.292+0000 7f3642ffd700 1 -- 172.21.15.138:0/1415944734 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f364400bed0 con 0x7f3654066960 2024-05-17T21:43:55.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:55.293+0000 7f3642ffd700 1 -- 172.21.15.138:0/1415944734 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f364400b5c0 con 0x7f3654066960 2024-05-17T21:43:55.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:55.293+0000 7f3658942700 1 -- 172.21.15.138:0/1415944734 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f365410fa40 con 0x7f3654066960 2024-05-17T21:43:55.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:55.293+0000 7f3642ffd700 1 -- 172.21.15.138:0/1415944734 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f364401c5b0 con 0x7f3654066960 2024-05-17T21:43:55.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:55.293+0000 7f3658942700 1 -- 172.21.15.138:0/1415944734 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36541105b0 con 0x7f3654066960 2024-05-17T21:43:55.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:55.293+0000 7f3642ffd700 1 -- 172.21.15.138:0/1415944734 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) v1 ==== 43433+0+0 (secure 0 0 0) 0x7f3644013420 con 0x7f3654066960 2024-05-17T21:43:55.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:55.293+0000 7f3658942700 1 -- 172.21.15.138:0/1415944734 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3654105150 con 0x7f3654066960 2024-05-17T21:43:55.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:55.294+0000 7f3642ffd700 1 --2- 172.21.15.138:0/1415944734 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f363c0368d0 0x7f363c038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:43:55.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:55.294+0000 7f3642ffd700 1 -- 172.21.15.138:0/1415944734 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f3644049a00 con 0x7f3654066960 2024-05-17T21:43:55.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:55.294+0000 7f3651d9b700 1 --2- 172.21.15.138:0/1415944734 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f363c0368d0 0x7f363c038d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:43:55.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:55.294+0000 7f3651d9b700 1 --2- 172.21.15.138:0/1415944734 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f363c0368d0 0x7f363c038d80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3648009990 tx=0x7f3648006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:43:55.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:55.298+0000 7f3642ffd700 1 -- 172.21.15.138:0/1415944734 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f3644013a80 con 0x7f3654066960 2024-05-17T21:43:55.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:55.556+0000 7f3658942700 1 -- 172.21.15.138:0/1415944734 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f3654110fc0 con 0x7f3654066960 2024-05-17T21:43:55.551 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:55.557+0000 7f3642ffd700 1 -- 172.21.15.138:0/1415944734 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7f3644011370 con 0x7f3654066960 2024-05-17T21:43:55.551 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-17T21:43:55.551 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":1,"fsid":"3722731a-1496-11ef-bc9a-c7b262605968","modified":"2024-05-17T21:42:10.472726Z","created":"2024-05-17T21:42:10.472726Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi107","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:3300","nonce":0},{"type":"v1","addr":"172.21.15.107:6789","nonce":0}]},"addr":"172.21.15.107:6789/0","public_addr":"172.21.15.107:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-17T21:43:55.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:55.559+0000 7f3658942700 1 -- 172.21.15.138:0/1415944734 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f363c0368d0 msgr2=0x7f363c038d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:43:55.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:55.559+0000 7f3658942700 1 --2- 172.21.15.138:0/1415944734 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f363c0368d0 0x7f363c038d80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3648009990 tx=0x7f3648006e40 comp rx=0 tx=0).stop 2024-05-17T21:43:55.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:55.559+0000 7f3658942700 1 -- 172.21.15.138:0/1415944734 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3654066960 msgr2=0x7f365410f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:43:55.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:55.559+0000 7f3658942700 1 --2- 172.21.15.138:0/1415944734 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3654066960 0x7f365410f300 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3644009680 tx=0x7f3644009e30 comp rx=0 tx=0).stop 2024-05-17T21:43:55.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:55.559+0000 7f3658942700 1 -- 172.21.15.138:0/1415944734 shutdown_connections 2024-05-17T21:43:55.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:55.559+0000 7f3658942700 1 --2- 172.21.15.138:0/1415944734 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f363c0368d0 0x7f363c038d80 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:43:55.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:55.559+0000 7f3658942700 1 --2- 172.21.15.138:0/1415944734 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3654066960 0x7f365410f300 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:43:55.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:55.559+0000 7f3658942700 1 -- 172.21.15.138:0/1415944734 >> 172.21.15.138:0/1415944734 conn(0x7f36540f70a0 msgr2=0x7f3654105d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:43:55.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:55.559+0000 7f3658942700 1 -- 172.21.15.138:0/1415944734 shutdown_connections 2024-05-17T21:43:55.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:55.559+0000 7f3658942700 1 -- 172.21.15.138:0/1415944734 wait complete. 2024-05-17T21:43:55.554 INFO:teuthology.orchestra.run.smithi138.stderr:dumped monmap epoch 1 2024-05-17T21:43:55.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:55 smithi107 ceph-mon[28434]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T21:43:55.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:55 smithi107 ceph-mon[28434]: from='mgr.14162 172.21.15.107:0/3666393387' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:43:55.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:55 smithi107 ceph-mon[28434]: from='mgr.14162 172.21.15.107:0/3666393387' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:43:55.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:55 smithi107 ceph-mon[28434]: from='mgr.14162 172.21.15.107:0/3666393387' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:43:55.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:55 smithi107 ceph-mon[28434]: from='mgr.14162 172.21.15.107:0/3666393387' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:43:55.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:55 smithi107 ceph-mon[28434]: from='mgr.14162 172.21.15.107:0/3666393387' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:43:55.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:55 smithi107 ceph-mon[28434]: from='mgr.14162 172.21.15.107:0/3666393387' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:43:55.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:55 smithi107 ceph-mon[28434]: from='mgr.14162 172.21.15.107:0/3666393387' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-05-17T21:43:55.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:55 smithi107 ceph-mon[28434]: from='mgr.14162 172.21.15.107:0/3666393387' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:43:56.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:56 smithi107 ceph-mon[28434]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-05-17T21:43:56.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:56 smithi107 ceph-mon[28434]: Deploying daemon grafana.smithi107 on smithi107 2024-05-17T21:43:56.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:56 smithi107 ceph-mon[28434]: from='client.? 172.21.15.138:0/1415944734' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-17T21:43:57.111 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-17T21:43:57.111 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -- ceph mon dump -f json 2024-05-17T21:43:57.359 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-17T21:43:57.383 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:57 smithi107 ceph-mon[28434]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T21:43:57.384 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:57 smithi107 ceph-mon[28434]: from='mgr.14162 172.21.15.107:0/3666393387' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:43:57.406 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-17T21:43:58.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:58.871+0000 7f483e648700 1 -- 172.21.15.138:0/2777995113 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4838071de0 msgr2=0x7f48380702f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:43:58.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:58.871+0000 7f483e648700 1 --2- 172.21.15.138:0/2777995113 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4838071de0 0x7f48380702f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4828009c60 tx=0x7f4828004d10 comp rx=0 tx=0).stop 2024-05-17T21:43:58.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:58.871+0000 7f483e648700 1 -- 172.21.15.138:0/2777995113 shutdown_connections 2024-05-17T21:43:58.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:58.871+0000 7f483e648700 1 --2- 172.21.15.138:0/2777995113 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4838071de0 0x7f48380702f0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:43:58.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:58.871+0000 7f483e648700 1 -- 172.21.15.138:0/2777995113 >> 172.21.15.138:0/2777995113 conn(0x7f48380f8200 msgr2=0x7f48380fa610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:43:58.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:58.871+0000 7f483e648700 1 -- 172.21.15.138:0/2777995113 shutdown_connections 2024-05-17T21:43:58.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:58.871+0000 7f483e648700 1 -- 172.21.15.138:0/2777995113 wait complete. 2024-05-17T21:43:58.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:58.872+0000 7f483e648700 1 Processor -- start 2024-05-17T21:43:58.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:58.872+0000 7f483e648700 1 -- start start 2024-05-17T21:43:58.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:58.872+0000 7f483e648700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4838071de0 0x7f483810f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:43:58.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:58.872+0000 7f483e648700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f483810f800 con 0x7f4838071de0 2024-05-17T21:43:58.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:58.872+0000 7f4837fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4838071de0 0x7f483810f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:43:58.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:58.873+0000 7f4837fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4838071de0 0x7f483810f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.138:32778/0 (socket says 172.21.15.138:32778) 2024-05-17T21:43:58.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:58.873+0000 7f4837fff700 1 -- 172.21.15.138:0/4087481936 learned_addr learned my addr 172.21.15.138:0/4087481936 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-17T21:43:58.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:58.873+0000 7f4837fff700 1 -- 172.21.15.138:0/4087481936 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4828009940 con 0x7f4838071de0 2024-05-17T21:43:58.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:58.873+0000 7f4837fff700 1 --2- 172.21.15.138:0/4087481936 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4838071de0 0x7f483810f2c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f48280097b0 tx=0x7f4828004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:43:58.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:58.873+0000 7f4834ff9700 1 -- 172.21.15.138:0/4087481936 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4828012410 con 0x7f4838071de0 2024-05-17T21:43:58.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:58.873+0000 7f4834ff9700 1 -- 172.21.15.138:0/4087481936 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4828012a40 con 0x7f4838071de0 2024-05-17T21:43:58.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:58.873+0000 7f483e648700 1 -- 172.21.15.138:0/4087481936 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f483810fa00 con 0x7f4838071de0 2024-05-17T21:43:58.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:58.873+0000 7f4834ff9700 1 -- 172.21.15.138:0/4087481936 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f482801b980 con 0x7f4838071de0 2024-05-17T21:43:58.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:58.873+0000 7f483e648700 1 -- 172.21.15.138:0/4087481936 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4838110570 con 0x7f4838071de0 2024-05-17T21:43:58.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:58.874+0000 7f4834ff9700 1 -- 172.21.15.138:0/4087481936 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) v1 ==== 43433+0+0 (secure 0 0 0) 0x7f4828019040 con 0x7f4838071de0 2024-05-17T21:43:58.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:58.874+0000 7f483e648700 1 -- 172.21.15.138:0/4087481936 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4838105180 con 0x7f4838071de0 2024-05-17T21:43:58.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:58.874+0000 7f4834ff9700 1 --2- 172.21.15.138:0/4087481936 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f48200368d0 0x7f4820038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:43:58.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:58.874+0000 7f4834ff9700 1 -- 172.21.15.138:0/4087481936 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f4828049c30 con 0x7f4838071de0 2024-05-17T21:43:58.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:58.875+0000 7f48377fe700 1 --2- 172.21.15.138:0/4087481936 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f48200368d0 0x7f4820038d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:43:58.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:58.875+0000 7f48377fe700 1 --2- 172.21.15.138:0/4087481936 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f48200368d0 0x7f4820038d80 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f482c009990 tx=0x7f482c006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:43:58.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:58.879+0000 7f4834ff9700 1 -- 172.21.15.138:0/4087481936 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f4828012bb0 con 0x7f4838071de0 2024-05-17T21:43:59.134 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:59.140+0000 7f483e648700 1 -- 172.21.15.138:0/4087481936 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f483805f5c0 con 0x7f4838071de0 2024-05-17T21:43:59.134 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:59.140+0000 7f4834ff9700 1 -- 172.21.15.138:0/4087481936 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7f4828017090 con 0x7f4838071de0 2024-05-17T21:43:59.134 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-17T21:43:59.135 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":1,"fsid":"3722731a-1496-11ef-bc9a-c7b262605968","modified":"2024-05-17T21:42:10.472726Z","created":"2024-05-17T21:42:10.472726Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi107","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:3300","nonce":0},{"type":"v1","addr":"172.21.15.107:6789","nonce":0}]},"addr":"172.21.15.107:6789/0","public_addr":"172.21.15.107:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-17T21:43:59.136 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:59.142+0000 7f483e648700 1 -- 172.21.15.138:0/4087481936 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f48200368d0 msgr2=0x7f4820038d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:43:59.136 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:59.142+0000 7f483e648700 1 --2- 172.21.15.138:0/4087481936 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f48200368d0 0x7f4820038d80 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f482c009990 tx=0x7f482c006e40 comp rx=0 tx=0).stop 2024-05-17T21:43:59.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:59.142+0000 7f483e648700 1 -- 172.21.15.138:0/4087481936 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4838071de0 msgr2=0x7f483810f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:43:59.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:59.142+0000 7f483e648700 1 --2- 172.21.15.138:0/4087481936 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4838071de0 0x7f483810f2c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f48280097b0 tx=0x7f4828004d10 comp rx=0 tx=0).stop 2024-05-17T21:43:59.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:59.142+0000 7f483e648700 1 -- 172.21.15.138:0/4087481936 shutdown_connections 2024-05-17T21:43:59.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:59.142+0000 7f483e648700 1 --2- 172.21.15.138:0/4087481936 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f48200368d0 0x7f4820038d80 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:43:59.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:59.142+0000 7f483e648700 1 --2- 172.21.15.138:0/4087481936 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4838071de0 0x7f483810f2c0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:43:59.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:59.142+0000 7f483e648700 1 -- 172.21.15.138:0/4087481936 >> 172.21.15.138:0/4087481936 conn(0x7f48380f8200 msgr2=0x7f4838105db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:43:59.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:59.143+0000 7f483e648700 1 -- 172.21.15.138:0/4087481936 shutdown_connections 2024-05-17T21:43:59.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:43:59.143+0000 7f483e648700 1 -- 172.21.15.138:0/4087481936 wait complete. 2024-05-17T21:43:59.137 INFO:teuthology.orchestra.run.smithi138.stderr:dumped monmap epoch 1 2024-05-17T21:43:59.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:59 smithi107 ceph-mon[28434]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T21:43:59.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:43:59 smithi107 ceph-mon[28434]: from='client.? 172.21.15.138:0/4087481936' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-17T21:44:00.764 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-17T21:44:00.765 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -- ceph mon dump -f json 2024-05-17T21:44:01.013 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-17T21:44:01.061 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-17T21:44:01.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:01 smithi107 ceph-mon[28434]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T21:44:02.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:02.504+0000 7f20242f0700 1 -- 172.21.15.138:0/3094002002 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f201c0fb0f0 msgr2=0x7f201c0fb500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:44:02.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:02.504+0000 7f20242f0700 1 --2- 172.21.15.138:0/3094002002 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f201c0fb0f0 0x7f201c0fb500 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f20180048c0 tx=0x7f2018009e30 comp rx=0 tx=0).stop 2024-05-17T21:44:02.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:02.504+0000 7f20242f0700 1 -- 172.21.15.138:0/3094002002 shutdown_connections 2024-05-17T21:44:02.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:02.504+0000 7f20242f0700 1 --2- 172.21.15.138:0/3094002002 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f201c0fb0f0 0x7f201c0fb500 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:44:02.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:02.504+0000 7f20242f0700 1 -- 172.21.15.138:0/3094002002 >> 172.21.15.138:0/3094002002 conn(0x7f201c0f6850 msgr2=0x7f201c0f8c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:44:02.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:02.504+0000 7f20242f0700 1 -- 172.21.15.138:0/3094002002 shutdown_connections 2024-05-17T21:44:02.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:02.504+0000 7f20242f0700 1 -- 172.21.15.138:0/3094002002 wait complete. 2024-05-17T21:44:02.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:02.504+0000 7f20242f0700 1 Processor -- start 2024-05-17T21:44:02.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:02.505+0000 7f20242f0700 1 -- start start 2024-05-17T21:44:02.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:02.505+0000 7f20242f0700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f201c0fb0f0 0x7f201c10d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:44:02.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:02.505+0000 7f20242f0700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f201c10d600 con 0x7f201c0fb0f0 2024-05-17T21:44:02.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:02.505+0000 7f202208c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f201c0fb0f0 0x7f201c10d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:44:02.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:02.505+0000 7f202208c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f201c0fb0f0 0x7f201c10d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.138:32796/0 (socket says 172.21.15.138:32796) 2024-05-17T21:44:02.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:02.505+0000 7f202208c700 1 -- 172.21.15.138:0/2275593744 learned_addr learned my addr 172.21.15.138:0/2275593744 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-17T21:44:02.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:02.505+0000 7f202208c700 1 -- 172.21.15.138:0/2275593744 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2018005040 con 0x7f201c0fb0f0 2024-05-17T21:44:02.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:02.505+0000 7f202208c700 1 --2- 172.21.15.138:0/2275593744 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f201c0fb0f0 0x7f201c10d0c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f20180096b0 tx=0x7f2018009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:44:02.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:02.506+0000 7f2012ffd700 1 -- 172.21.15.138:0/2275593744 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f201800bed0 con 0x7f201c0fb0f0 2024-05-17T21:44:02.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:02.506+0000 7f2012ffd700 1 -- 172.21.15.138:0/2275593744 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f201800b5c0 con 0x7f201c0fb0f0 2024-05-17T21:44:02.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:02.506+0000 7f20242f0700 1 -- 172.21.15.138:0/2275593744 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f201c10d800 con 0x7f201c0fb0f0 2024-05-17T21:44:02.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:02.506+0000 7f2012ffd700 1 -- 172.21.15.138:0/2275593744 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f201801c5b0 con 0x7f201c0fb0f0 2024-05-17T21:44:02.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:02.506+0000 7f20242f0700 1 -- 172.21.15.138:0/2275593744 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f201c10e370 con 0x7f201c0fb0f0 2024-05-17T21:44:02.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:02.506+0000 7f2012ffd700 1 -- 172.21.15.138:0/2275593744 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) v1 ==== 43433+0+0 (secure 0 0 0) 0x7f201801c820 con 0x7f201c0fb0f0 2024-05-17T21:44:02.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:02.507+0000 7f2012ffd700 1 --2- 172.21.15.138:0/2275593744 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f2008036880 0x7f2008038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:44:02.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:02.507+0000 7f2012ffd700 1 -- 172.21.15.138:0/2275593744 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f2018049ac0 con 0x7f201c0fb0f0 2024-05-17T21:44:02.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:02.507+0000 7f20242f0700 1 -- 172.21.15.138:0/2275593744 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f201c101600 con 0x7f201c0fb0f0 2024-05-17T21:44:02.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:02.507+0000 7f202188b700 1 --2- 172.21.15.138:0/2275593744 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f2008036880 0x7f2008038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:44:02.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:02.507+0000 7f202188b700 1 --2- 172.21.15.138:0/2275593744 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f2008036880 0x7f2008038d30 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f200c009990 tx=0x7f200c006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:44:02.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:02.512+0000 7f2012ffd700 1 -- 172.21.15.138:0/2275593744 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f2018023300 con 0x7f201c0fb0f0 2024-05-17T21:44:02.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:02.768+0000 7f20242f0700 1 -- 172.21.15.138:0/2275593744 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f201c05f5c0 con 0x7f201c0fb0f0 2024-05-17T21:44:02.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:02.769+0000 7f2012ffd700 1 -- 172.21.15.138:0/2275593744 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7f2018034090 con 0x7f201c0fb0f0 2024-05-17T21:44:02.764 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-17T21:44:02.764 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":1,"fsid":"3722731a-1496-11ef-bc9a-c7b262605968","modified":"2024-05-17T21:42:10.472726Z","created":"2024-05-17T21:42:10.472726Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi107","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:3300","nonce":0},{"type":"v1","addr":"172.21.15.107:6789","nonce":0}]},"addr":"172.21.15.107:6789/0","public_addr":"172.21.15.107:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-17T21:44:02.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:02.771+0000 7f20242f0700 1 -- 172.21.15.138:0/2275593744 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f2008036880 msgr2=0x7f2008038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:44:02.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:02.771+0000 7f20242f0700 1 --2- 172.21.15.138:0/2275593744 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f2008036880 0x7f2008038d30 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f200c009990 tx=0x7f200c006e40 comp rx=0 tx=0).stop 2024-05-17T21:44:02.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:02.771+0000 7f20242f0700 1 -- 172.21.15.138:0/2275593744 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f201c0fb0f0 msgr2=0x7f201c10d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:44:02.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:02.771+0000 7f20242f0700 1 --2- 172.21.15.138:0/2275593744 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f201c0fb0f0 0x7f201c10d0c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f20180096b0 tx=0x7f2018009e30 comp rx=0 tx=0).stop 2024-05-17T21:44:02.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:02.771+0000 7f20242f0700 1 -- 172.21.15.138:0/2275593744 shutdown_connections 2024-05-17T21:44:02.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:02.771+0000 7f20242f0700 1 --2- 172.21.15.138:0/2275593744 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f2008036880 0x7f2008038d30 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:44:02.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:02.771+0000 7f20242f0700 1 --2- 172.21.15.138:0/2275593744 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f201c0fb0f0 0x7f201c10d0c0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:44:02.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:02.771+0000 7f20242f0700 1 -- 172.21.15.138:0/2275593744 >> 172.21.15.138:0/2275593744 conn(0x7f201c0f6850 msgr2=0x7f201c0f8c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:44:02.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:02.771+0000 7f20242f0700 1 -- 172.21.15.138:0/2275593744 shutdown_connections 2024-05-17T21:44:02.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:02.771+0000 7f20242f0700 1 -- 172.21.15.138:0/2275593744 wait complete. 2024-05-17T21:44:02.767 INFO:teuthology.orchestra.run.smithi138.stderr:dumped monmap epoch 1 2024-05-17T21:44:03.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:03 smithi107 ceph-mon[28434]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T21:44:03.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:03 smithi107 ceph-mon[28434]: from='client.? 172.21.15.138:0/2275593744' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-17T21:44:04.344 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-17T21:44:04.345 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -- ceph mon dump -f json 2024-05-17T21:44:04.596 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-17T21:44:04.644 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-17T21:44:05.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:05 smithi107 ceph-mon[28434]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T21:44:06.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:06.021+0000 7fd66f3e5700 1 -- 172.21.15.138:0/1394622535 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd6680fca30 msgr2=0x7fd6680fce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:44:06.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:06.021+0000 7fd66f3e5700 1 --2- 172.21.15.138:0/1394622535 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd6680fca30 0x7fd6680fce40 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd6640048c0 tx=0x7fd664009e30 comp rx=0 tx=0).stop 2024-05-17T21:44:06.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:06.021+0000 7fd66f3e5700 1 -- 172.21.15.138:0/1394622535 shutdown_connections 2024-05-17T21:44:06.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:06.021+0000 7fd66f3e5700 1 --2- 172.21.15.138:0/1394622535 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd6680fca30 0x7fd6680fce40 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:44:06.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:06.022+0000 7fd66f3e5700 1 -- 172.21.15.138:0/1394622535 >> 172.21.15.138:0/1394622535 conn(0x7fd6680f8170 msgr2=0x7fd6680fa5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:44:06.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:06.022+0000 7fd66f3e5700 1 -- 172.21.15.138:0/1394622535 shutdown_connections 2024-05-17T21:44:06.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:06.022+0000 7fd66f3e5700 1 -- 172.21.15.138:0/1394622535 wait complete. 2024-05-17T21:44:06.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:06.022+0000 7fd66f3e5700 1 Processor -- start 2024-05-17T21:44:06.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:06.022+0000 7fd66f3e5700 1 -- start start 2024-05-17T21:44:06.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:06.022+0000 7fd66f3e5700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd6680fca30 0x7fd66810d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:44:06.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:06.022+0000 7fd66f3e5700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd66810d5f0 con 0x7fd6680fca30 2024-05-17T21:44:06.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:06.023+0000 7fd66d181700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd6680fca30 0x7fd66810d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:44:06.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:06.023+0000 7fd66d181700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd6680fca30 0x7fd66810d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.138:60814/0 (socket says 172.21.15.138:60814) 2024-05-17T21:44:06.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:06.023+0000 7fd66d181700 1 -- 172.21.15.138:0/3409066867 learned_addr learned my addr 172.21.15.138:0/3409066867 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-17T21:44:06.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:06.023+0000 7fd66d181700 1 -- 172.21.15.138:0/3409066867 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd664005040 con 0x7fd6680fca30 2024-05-17T21:44:06.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:06.023+0000 7fd66d181700 1 --2- 172.21.15.138:0/3409066867 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd6680fca30 0x7fd66810d0b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd6640096b0 tx=0x7fd664009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:44:06.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:06.023+0000 7fd659ffb700 1 -- 172.21.15.138:0/3409066867 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd66400bed0 con 0x7fd6680fca30 2024-05-17T21:44:06.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:06.023+0000 7fd659ffb700 1 -- 172.21.15.138:0/3409066867 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd66400b5c0 con 0x7fd6680fca30 2024-05-17T21:44:06.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:06.023+0000 7fd66f3e5700 1 -- 172.21.15.138:0/3409066867 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd66810d7f0 con 0x7fd6680fca30 2024-05-17T21:44:06.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:06.023+0000 7fd659ffb700 1 -- 172.21.15.138:0/3409066867 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd66401c5b0 con 0x7fd6680fca30 2024-05-17T21:44:06.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:06.024+0000 7fd66f3e5700 1 -- 172.21.15.138:0/3409066867 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd66810e360 con 0x7fd6680fca30 2024-05-17T21:44:06.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:06.024+0000 7fd659ffb700 1 -- 172.21.15.138:0/3409066867 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) v1 ==== 43433+0+0 (secure 0 0 0) 0x7fd664013420 con 0x7fd6680fca30 2024-05-17T21:44:06.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:06.024+0000 7fd66f3e5700 1 -- 172.21.15.138:0/3409066867 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd668102f40 con 0x7fd6680fca30 2024-05-17T21:44:06.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:06.024+0000 7fd659ffb700 1 --2- 172.21.15.138:0/3409066867 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7fd654036880 0x7fd654038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:44:06.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:06.025+0000 7fd659ffb700 1 -- 172.21.15.138:0/3409066867 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fd664049ae0 con 0x7fd6680fca30 2024-05-17T21:44:06.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:06.025+0000 7fd66c980700 1 --2- 172.21.15.138:0/3409066867 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7fd654036880 0x7fd654038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:44:06.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:06.025+0000 7fd66c980700 1 --2- 172.21.15.138:0/3409066867 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7fd654036880 0x7fd654038d30 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd65c009990 tx=0x7fd65c006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:44:06.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:06.029+0000 7fd659ffb700 1 -- 172.21.15.138:0/3409066867 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fd664013a80 con 0x7fd6680fca30 2024-05-17T21:44:06.281 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:06.287+0000 7fd66f3e5700 1 -- 172.21.15.138:0/3409066867 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fd66810ee30 con 0x7fd6680fca30 2024-05-17T21:44:06.282 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:06.288+0000 7fd659ffb700 1 -- 172.21.15.138:0/3409066867 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7fd664011370 con 0x7fd6680fca30 2024-05-17T21:44:06.282 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-17T21:44:06.282 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":1,"fsid":"3722731a-1496-11ef-bc9a-c7b262605968","modified":"2024-05-17T21:42:10.472726Z","created":"2024-05-17T21:42:10.472726Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi107","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:3300","nonce":0},{"type":"v1","addr":"172.21.15.107:6789","nonce":0}]},"addr":"172.21.15.107:6789/0","public_addr":"172.21.15.107:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-17T21:44:06.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:06.290+0000 7fd66f3e5700 1 -- 172.21.15.138:0/3409066867 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7fd654036880 msgr2=0x7fd654038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:44:06.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:06.290+0000 7fd66f3e5700 1 --2- 172.21.15.138:0/3409066867 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7fd654036880 0x7fd654038d30 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd65c009990 tx=0x7fd65c006e40 comp rx=0 tx=0).stop 2024-05-17T21:44:06.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:06.290+0000 7fd66f3e5700 1 -- 172.21.15.138:0/3409066867 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd6680fca30 msgr2=0x7fd66810d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:44:06.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:06.290+0000 7fd66f3e5700 1 --2- 172.21.15.138:0/3409066867 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd6680fca30 0x7fd66810d0b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd6640096b0 tx=0x7fd664009e30 comp rx=0 tx=0).stop 2024-05-17T21:44:06.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:06.290+0000 7fd66f3e5700 1 -- 172.21.15.138:0/3409066867 shutdown_connections 2024-05-17T21:44:06.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:06.290+0000 7fd66f3e5700 1 --2- 172.21.15.138:0/3409066867 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7fd654036880 0x7fd654038d30 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:44:06.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:06.290+0000 7fd66f3e5700 1 --2- 172.21.15.138:0/3409066867 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd6680fca30 0x7fd66810d0b0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:44:06.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:06.290+0000 7fd66f3e5700 1 -- 172.21.15.138:0/3409066867 >> 172.21.15.138:0/3409066867 conn(0x7fd6680f8170 msgr2=0x7fd668103b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:44:06.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:06.290+0000 7fd66f3e5700 1 -- 172.21.15.138:0/3409066867 shutdown_connections 2024-05-17T21:44:06.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:06.290+0000 7fd66f3e5700 1 -- 172.21.15.138:0/3409066867 wait complete. 2024-05-17T21:44:06.285 INFO:teuthology.orchestra.run.smithi138.stderr:dumped monmap epoch 1 2024-05-17T21:44:07.354 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:07 smithi107 ceph-mon[28434]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T21:44:07.354 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:07 smithi107 ceph-mon[28434]: from='client.? 172.21.15.138:0/3409066867' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-17T21:44:07.820 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-17T21:44:07.820 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -- ceph mon dump -f json 2024-05-17T21:44:08.068 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-17T21:44:08.116 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-17T21:44:09.465 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:09 smithi107 ceph-mon[28434]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T21:44:09.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:09.574+0000 7fb09e103700 1 -- 172.21.15.138:0/3995358504 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0980fca30 msgr2=0x7fb0980fce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:44:09.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:09.574+0000 7fb09e103700 1 --2- 172.21.15.138:0/3995358504 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0980fca30 0x7fb0980fce40 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb0880048c0 tx=0x7fb088009e30 comp rx=0 tx=0).stop 2024-05-17T21:44:09.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:09.574+0000 7fb09e103700 1 -- 172.21.15.138:0/3995358504 shutdown_connections 2024-05-17T21:44:09.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:09.574+0000 7fb09e103700 1 --2- 172.21.15.138:0/3995358504 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0980fca30 0x7fb0980fce40 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:44:09.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:09.574+0000 7fb09e103700 1 -- 172.21.15.138:0/3995358504 >> 172.21.15.138:0/3995358504 conn(0x7fb0980f8190 msgr2=0x7fb0980fa5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:44:09.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:09.574+0000 7fb09e103700 1 -- 172.21.15.138:0/3995358504 shutdown_connections 2024-05-17T21:44:09.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:09.574+0000 7fb09e103700 1 -- 172.21.15.138:0/3995358504 wait complete. 2024-05-17T21:44:09.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:09.574+0000 7fb09e103700 1 Processor -- start 2024-05-17T21:44:09.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:09.575+0000 7fb09e103700 1 -- start start 2024-05-17T21:44:09.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:09.575+0000 7fb09e103700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0980fca30 0x7fb098071d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:44:09.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:09.575+0000 7fb09e103700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb098072240 con 0x7fb0980fca30 2024-05-17T21:44:09.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:09.575+0000 7fb0977fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0980fca30 0x7fb098071d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:44:09.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:09.575+0000 7fb0977fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0980fca30 0x7fb098071d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.138:60822/0 (socket says 172.21.15.138:60822) 2024-05-17T21:44:09.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:09.575+0000 7fb0977fe700 1 -- 172.21.15.138:0/3076850121 learned_addr learned my addr 172.21.15.138:0/3076850121 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-17T21:44:09.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:09.575+0000 7fb0977fe700 1 -- 172.21.15.138:0/3076850121 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb088005040 con 0x7fb0980fca30 2024-05-17T21:44:09.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:09.575+0000 7fb0977fe700 1 --2- 172.21.15.138:0/3076850121 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0980fca30 0x7fb098071d00 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb0880096b0 tx=0x7fb088009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:44:09.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:09.575+0000 7fb094ff9700 1 -- 172.21.15.138:0/3076850121 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb08800bed0 con 0x7fb0980fca30 2024-05-17T21:44:09.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:09.575+0000 7fb09e103700 1 -- 172.21.15.138:0/3076850121 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb098070310 con 0x7fb0980fca30 2024-05-17T21:44:09.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:09.576+0000 7fb094ff9700 1 -- 172.21.15.138:0/3076850121 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb08800b5c0 con 0x7fb0980fca30 2024-05-17T21:44:09.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:09.576+0000 7fb094ff9700 1 -- 172.21.15.138:0/3076850121 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb08801c5b0 con 0x7fb0980fca30 2024-05-17T21:44:09.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:09.576+0000 7fb09e103700 1 -- 172.21.15.138:0/3076850121 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb098070800 con 0x7fb0980fca30 2024-05-17T21:44:09.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:09.576+0000 7fb094ff9700 1 -- 172.21.15.138:0/3076850121 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) v1 ==== 43433+0+0 (secure 0 0 0) 0x7fb08801c820 con 0x7fb0980fca30 2024-05-17T21:44:09.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:09.577+0000 7fb094ff9700 1 --2- 172.21.15.138:0/3076850121 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7fb084036880 0x7fb084038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:44:09.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:09.577+0000 7fb094ff9700 1 -- 172.21.15.138:0/3076850121 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fb088049ad0 con 0x7fb0980fca30 2024-05-17T21:44:09.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:09.577+0000 7fb09e103700 1 -- 172.21.15.138:0/3076850121 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb09805f5c0 con 0x7fb0980fca30 2024-05-17T21:44:09.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:09.577+0000 7fb08ffff700 1 --2- 172.21.15.138:0/3076850121 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7fb084036880 0x7fb084038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:44:09.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:09.577+0000 7fb08ffff700 1 --2- 172.21.15.138:0/3076850121 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7fb084036880 0x7fb084038d30 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb080009990 tx=0x7fb080006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:44:09.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:09.586+0000 7fb094ff9700 1 -- 172.21.15.138:0/3076850121 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fb088023300 con 0x7fb0980fca30 2024-05-17T21:44:09.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:09.845+0000 7fb09e103700 1 -- 172.21.15.138:0/3076850121 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fb09804bed0 con 0x7fb0980fca30 2024-05-17T21:44:09.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:09.845+0000 7fb094ff9700 1 -- 172.21.15.138:0/3076850121 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7fb088034090 con 0x7fb0980fca30 2024-05-17T21:44:09.839 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-17T21:44:09.839 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":1,"fsid":"3722731a-1496-11ef-bc9a-c7b262605968","modified":"2024-05-17T21:42:10.472726Z","created":"2024-05-17T21:42:10.472726Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi107","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:3300","nonce":0},{"type":"v1","addr":"172.21.15.107:6789","nonce":0}]},"addr":"172.21.15.107:6789/0","public_addr":"172.21.15.107:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-17T21:44:09.841 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:09.847+0000 7fb09e103700 1 -- 172.21.15.138:0/3076850121 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7fb084036880 msgr2=0x7fb084038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:44:09.841 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:09.847+0000 7fb09e103700 1 --2- 172.21.15.138:0/3076850121 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7fb084036880 0x7fb084038d30 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb080009990 tx=0x7fb080006e40 comp rx=0 tx=0).stop 2024-05-17T21:44:09.841 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:09.847+0000 7fb09e103700 1 -- 172.21.15.138:0/3076850121 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0980fca30 msgr2=0x7fb098071d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:44:09.841 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:09.847+0000 7fb09e103700 1 --2- 172.21.15.138:0/3076850121 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0980fca30 0x7fb098071d00 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb0880096b0 tx=0x7fb088009e30 comp rx=0 tx=0).stop 2024-05-17T21:44:09.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:09.847+0000 7fb09e103700 1 -- 172.21.15.138:0/3076850121 shutdown_connections 2024-05-17T21:44:09.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:09.847+0000 7fb09e103700 1 --2- 172.21.15.138:0/3076850121 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7fb084036880 0x7fb084038d30 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:44:09.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:09.847+0000 7fb09e103700 1 --2- 172.21.15.138:0/3076850121 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0980fca30 0x7fb098071d00 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:44:09.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:09.847+0000 7fb09e103700 1 -- 172.21.15.138:0/3076850121 >> 172.21.15.138:0/3076850121 conn(0x7fb0980f8190 msgr2=0x7fb098103b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:44:09.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:09.847+0000 7fb09e103700 1 -- 172.21.15.138:0/3076850121 shutdown_connections 2024-05-17T21:44:09.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:09.847+0000 7fb09e103700 1 -- 172.21.15.138:0/3076850121 wait complete. 2024-05-17T21:44:09.842 INFO:teuthology.orchestra.run.smithi138.stderr:dumped monmap epoch 1 2024-05-17T21:44:10.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:10 smithi107 ceph-mon[28434]: from='mgr.14162 172.21.15.107:0/3666393387' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:44:10.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:10 smithi107 ceph-mon[28434]: from='mgr.14162 172.21.15.107:0/3666393387' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:44:10.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:10 smithi107 ceph-mon[28434]: from='mgr.14162 172.21.15.107:0/3666393387' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:44:10.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:10 smithi107 ceph-mon[28434]: from='mgr.14162 172.21.15.107:0/3666393387' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:44:10.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:10 smithi107 ceph-mon[28434]: from='mgr.14162 172.21.15.107:0/3666393387' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:44:10.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:10 smithi107 ceph-mon[28434]: from='mgr.14162 172.21.15.107:0/3666393387' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:44:10.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:10 smithi107 ceph-mon[28434]: from='client.? 172.21.15.138:0/3076850121' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-17T21:44:11.431 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-17T21:44:11.432 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -- ceph mon dump -f json 2024-05-17T21:44:11.681 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-17T21:44:11.729 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-17T21:44:11.971 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:11 smithi107 ceph-mon[28434]: Deploying daemon prometheus.smithi107 on smithi107 2024-05-17T21:44:11.972 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:11 smithi107 ceph-mon[28434]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T21:44:13.222 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:13.228+0000 7f8cf17b0700 1 -- 172.21.15.138:0/266479640 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8cec0669a0 msgr2=0x7f8cec0ff620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:44:13.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:13.228+0000 7f8cf17b0700 1 --2- 172.21.15.138:0/266479640 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8cec0669a0 0x7f8cec0ff620 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc0048c0 tx=0x7f8cdc009e30 comp rx=0 tx=0).stop 2024-05-17T21:44:13.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:13.228+0000 7f8cf17b0700 1 -- 172.21.15.138:0/266479640 shutdown_connections 2024-05-17T21:44:13.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:13.228+0000 7f8cf17b0700 1 --2- 172.21.15.138:0/266479640 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8cec0669a0 0x7f8cec0ff620 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:44:13.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:13.228+0000 7f8cf17b0700 1 -- 172.21.15.138:0/266479640 >> 172.21.15.138:0/266479640 conn(0x7f8cec0f7070 msgr2=0x7f8cec0f94c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:44:13.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:13.228+0000 7f8cf17b0700 1 -- 172.21.15.138:0/266479640 shutdown_connections 2024-05-17T21:44:13.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:13.228+0000 7f8cf17b0700 1 -- 172.21.15.138:0/266479640 wait complete. 2024-05-17T21:44:13.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:13.229+0000 7f8cf17b0700 1 Processor -- start 2024-05-17T21:44:13.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:13.229+0000 7f8cf17b0700 1 -- start start 2024-05-17T21:44:13.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:13.229+0000 7f8cf17b0700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8cec0669a0 0x7f8cec0fcf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:44:13.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:13.229+0000 7f8cf17b0700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cec0fd4b0 con 0x7f8cec0669a0 2024-05-17T21:44:13.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:13.229+0000 7f8ceaffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8cec0669a0 0x7f8cec0fcf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:44:13.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:13.229+0000 7f8ceaffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8cec0669a0 0x7f8cec0fcf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.138:60840/0 (socket says 172.21.15.138:60840) 2024-05-17T21:44:13.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:13.229+0000 7f8ceaffd700 1 -- 172.21.15.138:0/3617572022 learned_addr learned my addr 172.21.15.138:0/3617572022 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-17T21:44:13.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:13.229+0000 7f8ceaffd700 1 -- 172.21.15.138:0/3617572022 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cdc005040 con 0x7f8cec0669a0 2024-05-17T21:44:13.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:13.229+0000 7f8ceaffd700 1 --2- 172.21.15.138:0/3617572022 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8cec0669a0 0x7f8cec0fcf70 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc00b900 tx=0x7f8cdc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:44:13.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:13.230+0000 7f8cd37fe700 1 -- 172.21.15.138:0/3617572022 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8cdc00b5c0 con 0x7f8cec0669a0 2024-05-17T21:44:13.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:13.230+0000 7f8cd37fe700 1 -- 172.21.15.138:0/3617572022 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8cdc00be70 con 0x7f8cec0669a0 2024-05-17T21:44:13.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:13.230+0000 7f8cf17b0700 1 -- 172.21.15.138:0/3617572022 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cec0fb580 con 0x7f8cec0669a0 2024-05-17T21:44:13.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:13.230+0000 7f8cd37fe700 1 -- 172.21.15.138:0/3617572022 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8cdc01c590 con 0x7f8cec0669a0 2024-05-17T21:44:13.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:13.230+0000 7f8cf17b0700 1 -- 172.21.15.138:0/3617572022 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cec0fba70 con 0x7f8cec0669a0 2024-05-17T21:44:13.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:13.230+0000 7f8cd37fe700 1 -- 172.21.15.138:0/3617572022 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) v1 ==== 43433+0+0 (secure 0 0 0) 0x7f8cdc01c800 con 0x7f8cec0669a0 2024-05-17T21:44:13.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:13.231+0000 7f8cf17b0700 1 -- 172.21.15.138:0/3617572022 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cec05f5c0 con 0x7f8cec0669a0 2024-05-17T21:44:13.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:13.231+0000 7f8cd37fe700 1 --2- 172.21.15.138:0/3617572022 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f8cd40368d0 0x7f8cd4038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:44:13.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:13.231+0000 7f8cd37fe700 1 -- 172.21.15.138:0/3617572022 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f8cdc049320 con 0x7f8cec0669a0 2024-05-17T21:44:13.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:13.231+0000 7f8cea7fc700 1 --2- 172.21.15.138:0/3617572022 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f8cd40368d0 0x7f8cd4038d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:44:13.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:13.231+0000 7f8cea7fc700 1 --2- 172.21.15.138:0/3617572022 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f8cd40368d0 0x7f8cd4038d80 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8ce0009990 tx=0x7f8ce0006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:44:13.229 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:13.236+0000 7f8cd37fe700 1 -- 172.21.15.138:0/3617572022 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f8cdc011960 con 0x7f8cec0669a0 2024-05-17T21:44:13.354 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:13 smithi107 ceph-mon[28434]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T21:44:13.354 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:13 smithi107 ceph-mon[28434]: from='mgr.14162 172.21.15.107:0/3666393387' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:44:13.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:13.495+0000 7f8cf17b0700 1 -- 172.21.15.138:0/3617572022 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f8cec106400 con 0x7f8cec0669a0 2024-05-17T21:44:13.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:13.496+0000 7f8cd37fe700 1 -- 172.21.15.138:0/3617572022 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7f8cdc021030 con 0x7f8cec0669a0 2024-05-17T21:44:13.490 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-17T21:44:13.490 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":1,"fsid":"3722731a-1496-11ef-bc9a-c7b262605968","modified":"2024-05-17T21:42:10.472726Z","created":"2024-05-17T21:42:10.472726Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi107","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:3300","nonce":0},{"type":"v1","addr":"172.21.15.107:6789","nonce":0}]},"addr":"172.21.15.107:6789/0","public_addr":"172.21.15.107:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-17T21:44:13.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:13.498+0000 7f8cf17b0700 1 -- 172.21.15.138:0/3617572022 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f8cd40368d0 msgr2=0x7f8cd4038d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:44:13.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:13.498+0000 7f8cf17b0700 1 --2- 172.21.15.138:0/3617572022 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f8cd40368d0 0x7f8cd4038d80 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8ce0009990 tx=0x7f8ce0006e40 comp rx=0 tx=0).stop 2024-05-17T21:44:13.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:13.498+0000 7f8cf17b0700 1 -- 172.21.15.138:0/3617572022 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8cec0669a0 msgr2=0x7f8cec0fcf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:44:13.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:13.498+0000 7f8cf17b0700 1 --2- 172.21.15.138:0/3617572022 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8cec0669a0 0x7f8cec0fcf70 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc00b900 tx=0x7f8cdc009e30 comp rx=0 tx=0).stop 2024-05-17T21:44:13.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:13.498+0000 7f8cf17b0700 1 -- 172.21.15.138:0/3617572022 shutdown_connections 2024-05-17T21:44:13.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:13.498+0000 7f8cf17b0700 1 --2- 172.21.15.138:0/3617572022 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f8cd40368d0 0x7f8cd4038d80 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:44:13.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:13.498+0000 7f8cf17b0700 1 --2- 172.21.15.138:0/3617572022 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8cec0669a0 0x7f8cec0fcf70 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:44:13.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:13.498+0000 7f8cf17b0700 1 -- 172.21.15.138:0/3617572022 >> 172.21.15.138:0/3617572022 conn(0x7f8cec0f7070 msgr2=0x7f8cec103b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:44:13.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:13.498+0000 7f8cf17b0700 1 -- 172.21.15.138:0/3617572022 shutdown_connections 2024-05-17T21:44:13.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:13.498+0000 7f8cf17b0700 1 -- 172.21.15.138:0/3617572022 wait complete. 2024-05-17T21:44:13.493 INFO:teuthology.orchestra.run.smithi138.stderr:dumped monmap epoch 1 2024-05-17T21:44:14.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:14 smithi107 ceph-mon[28434]: from='client.? 172.21.15.138:0/3617572022' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-17T21:44:15.034 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-17T21:44:15.034 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -- ceph mon dump -f json 2024-05-17T21:44:15.277 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-17T21:44:15.325 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-17T21:44:15.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:15 smithi107 ceph-mon[28434]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T21:44:16.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:16.799+0000 7faa30fa2700 1 -- 172.21.15.138:0/2907669553 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa2c0fe080 msgr2=0x7faa2c0fe490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:44:16.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:16.799+0000 7faa30fa2700 1 --2- 172.21.15.138:0/2907669553 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa2c0fe080 0x7faa2c0fe490 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7faa240048c0 tx=0x7faa24009e30 comp rx=0 tx=0).stop 2024-05-17T21:44:16.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:16.800+0000 7faa30fa2700 1 -- 172.21.15.138:0/2907669553 shutdown_connections 2024-05-17T21:44:16.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:16.800+0000 7faa30fa2700 1 --2- 172.21.15.138:0/2907669553 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa2c0fe080 0x7faa2c0fe490 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:44:16.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:16.800+0000 7faa30fa2700 1 -- 172.21.15.138:0/2907669553 >> 172.21.15.138:0/2907669553 conn(0x7faa2c0f6820 msgr2=0x7faa2c0f8c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:44:16.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:16.800+0000 7faa30fa2700 1 -- 172.21.15.138:0/2907669553 shutdown_connections 2024-05-17T21:44:16.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:16.800+0000 7faa30fa2700 1 -- 172.21.15.138:0/2907669553 wait complete. 2024-05-17T21:44:16.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:16.800+0000 7faa30fa2700 1 Processor -- start 2024-05-17T21:44:16.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:16.800+0000 7faa30fa2700 1 -- start start 2024-05-17T21:44:16.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:16.800+0000 7faa30fa2700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa2c0fe080 0x7faa2c0fd790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:44:16.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:16.800+0000 7faa30fa2700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa2c0fdcd0 con 0x7faa2c0fe080 2024-05-17T21:44:16.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:16.801+0000 7faa2a59c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa2c0fe080 0x7faa2c0fd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:44:16.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:16.801+0000 7faa2a59c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa2c0fe080 0x7faa2c0fd790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.138:60960/0 (socket says 172.21.15.138:60960) 2024-05-17T21:44:16.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:16.801+0000 7faa2a59c700 1 -- 172.21.15.138:0/304977011 learned_addr learned my addr 172.21.15.138:0/304977011 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-17T21:44:16.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:16.801+0000 7faa2a59c700 1 -- 172.21.15.138:0/304977011 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa24005040 con 0x7faa2c0fe080 2024-05-17T21:44:16.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:16.801+0000 7faa2a59c700 1 --2- 172.21.15.138:0/304977011 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa2c0fe080 0x7faa2c0fd790 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7faa240096b0 tx=0x7faa24009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:44:16.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:16.801+0000 7faa22ffd700 1 -- 172.21.15.138:0/304977011 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7faa2400bed0 con 0x7faa2c0fe080 2024-05-17T21:44:16.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:16.801+0000 7faa22ffd700 1 -- 172.21.15.138:0/304977011 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faa2400b5c0 con 0x7faa2c0fe080 2024-05-17T21:44:16.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:16.801+0000 7faa30fa2700 1 -- 172.21.15.138:0/304977011 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa2c0fbda0 con 0x7faa2c0fe080 2024-05-17T21:44:16.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:16.801+0000 7faa22ffd700 1 -- 172.21.15.138:0/304977011 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7faa2401c5b0 con 0x7faa2c0fe080 2024-05-17T21:44:16.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:16.801+0000 7faa30fa2700 1 -- 172.21.15.138:0/304977011 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa2c0fc290 con 0x7faa2c0fe080 2024-05-17T21:44:16.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:16.802+0000 7faa22ffd700 1 -- 172.21.15.138:0/304977011 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) v1 ==== 43433+0+0 (secure 0 0 0) 0x7faa24013420 con 0x7faa2c0fe080 2024-05-17T21:44:16.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:16.802+0000 7faa30fa2700 1 -- 172.21.15.138:0/304977011 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa2c05f5c0 con 0x7faa2c0fe080 2024-05-17T21:44:16.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:16.802+0000 7faa22ffd700 1 --2- 172.21.15.138:0/304977011 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7faa18036880 0x7faa18038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:44:16.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:16.802+0000 7faa22ffd700 1 -- 172.21.15.138:0/304977011 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7faa24049b10 con 0x7faa2c0fe080 2024-05-17T21:44:16.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:16.802+0000 7faa29d9b700 1 --2- 172.21.15.138:0/304977011 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7faa18036880 0x7faa18038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:44:16.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:16.803+0000 7faa29d9b700 1 --2- 172.21.15.138:0/304977011 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7faa18036880 0x7faa18038d30 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7faa14009990 tx=0x7faa14006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:44:16.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:16.807+0000 7faa22ffd700 1 -- 172.21.15.138:0/304977011 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7faa24025400 con 0x7faa2c0fe080 2024-05-17T21:44:17.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:17.065+0000 7faa30fa2700 1 -- 172.21.15.138:0/304977011 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7faa2c106420 con 0x7faa2c0fe080 2024-05-17T21:44:17.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:17.065+0000 7faa22ffd700 1 -- 172.21.15.138:0/304977011 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7faa24004d30 con 0x7faa2c0fe080 2024-05-17T21:44:17.060 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-17T21:44:17.060 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":1,"fsid":"3722731a-1496-11ef-bc9a-c7b262605968","modified":"2024-05-17T21:42:10.472726Z","created":"2024-05-17T21:42:10.472726Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi107","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:3300","nonce":0},{"type":"v1","addr":"172.21.15.107:6789","nonce":0}]},"addr":"172.21.15.107:6789/0","public_addr":"172.21.15.107:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-17T21:44:17.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:17.068+0000 7faa30fa2700 1 -- 172.21.15.138:0/304977011 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7faa18036880 msgr2=0x7faa18038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:44:17.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:17.068+0000 7faa30fa2700 1 --2- 172.21.15.138:0/304977011 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7faa18036880 0x7faa18038d30 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7faa14009990 tx=0x7faa14006e40 comp rx=0 tx=0).stop 2024-05-17T21:44:17.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:17.068+0000 7faa30fa2700 1 -- 172.21.15.138:0/304977011 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa2c0fe080 msgr2=0x7faa2c0fd790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:44:17.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:17.068+0000 7faa30fa2700 1 --2- 172.21.15.138:0/304977011 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa2c0fe080 0x7faa2c0fd790 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7faa240096b0 tx=0x7faa24009e30 comp rx=0 tx=0).stop 2024-05-17T21:44:17.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:17.068+0000 7faa30fa2700 1 -- 172.21.15.138:0/304977011 shutdown_connections 2024-05-17T21:44:17.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:17.068+0000 7faa30fa2700 1 --2- 172.21.15.138:0/304977011 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7faa18036880 0x7faa18038d30 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:44:17.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:17.068+0000 7faa30fa2700 1 --2- 172.21.15.138:0/304977011 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa2c0fe080 0x7faa2c0fd790 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:44:17.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:17.068+0000 7faa30fa2700 1 -- 172.21.15.138:0/304977011 >> 172.21.15.138:0/304977011 conn(0x7faa2c0f6820 msgr2=0x7faa2c103b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:44:17.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:17.068+0000 7faa30fa2700 1 -- 172.21.15.138:0/304977011 shutdown_connections 2024-05-17T21:44:17.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:17.068+0000 7faa30fa2700 1 -- 172.21.15.138:0/304977011 wait complete. 2024-05-17T21:44:17.063 INFO:teuthology.orchestra.run.smithi138.stderr:dumped monmap epoch 1 2024-05-17T21:44:17.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:17 smithi107 ceph-mon[28434]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T21:44:17.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:17 smithi107 ceph-mon[28434]: from='client.? 172.21.15.138:0/304977011' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-17T21:44:18.663 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-17T21:44:18.663 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -- ceph mon dump -f json 2024-05-17T21:44:18.905 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-17T21:44:18.953 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-17T21:44:19.346 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:19 smithi107 ceph-mon[28434]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T21:44:20.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:20.376+0000 7f617902d700 1 -- 172.21.15.138:0/2065150784 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61740fca40 msgr2=0x7f61740fce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:44:20.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:20.376+0000 7f617902d700 1 --2- 172.21.15.138:0/2065150784 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61740fca40 0x7f61740fce50 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f61640048c0 tx=0x7f6164009e30 comp rx=0 tx=0).stop 2024-05-17T21:44:20.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:20.376+0000 7f617902d700 1 -- 172.21.15.138:0/2065150784 shutdown_connections 2024-05-17T21:44:20.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:20.376+0000 7f617902d700 1 --2- 172.21.15.138:0/2065150784 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61740fca40 0x7f61740fce50 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:44:20.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:20.376+0000 7f617902d700 1 -- 172.21.15.138:0/2065150784 >> 172.21.15.138:0/2065150784 conn(0x7f61740f81c0 msgr2=0x7f61740fa5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:44:20.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:20.376+0000 7f617902d700 1 -- 172.21.15.138:0/2065150784 shutdown_connections 2024-05-17T21:44:20.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:20.376+0000 7f617902d700 1 -- 172.21.15.138:0/2065150784 wait complete. 2024-05-17T21:44:20.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:20.377+0000 7f617902d700 1 Processor -- start 2024-05-17T21:44:20.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:20.377+0000 7f617902d700 1 -- start start 2024-05-17T21:44:20.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:20.377+0000 7f617902d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61740fca40 0x7f617410d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:44:20.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:20.377+0000 7f617902d700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f617410d5d0 con 0x7f61740fca40 2024-05-17T21:44:20.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:20.377+0000 7f6172d9d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61740fca40 0x7f617410d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:44:20.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:20.377+0000 7f6172d9d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61740fca40 0x7f617410d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.138:60976/0 (socket says 172.21.15.138:60976) 2024-05-17T21:44:20.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:20.377+0000 7f6172d9d700 1 -- 172.21.15.138:0/3376948267 learned_addr learned my addr 172.21.15.138:0/3376948267 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-17T21:44:20.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:20.378+0000 7f6172d9d700 1 -- 172.21.15.138:0/3376948267 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6164005040 con 0x7f61740fca40 2024-05-17T21:44:20.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:20.378+0000 7f6172d9d700 1 --2- 172.21.15.138:0/3376948267 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61740fca40 0x7f617410d090 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f61640096b0 tx=0x7f6164009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:44:20.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:20.378+0000 7f616b7fe700 1 -- 172.21.15.138:0/3376948267 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f616400bed0 con 0x7f61740fca40 2024-05-17T21:44:20.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:20.378+0000 7f617902d700 1 -- 172.21.15.138:0/3376948267 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f617410d7d0 con 0x7f61740fca40 2024-05-17T21:44:20.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:20.378+0000 7f616b7fe700 1 -- 172.21.15.138:0/3376948267 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f616400b5c0 con 0x7f61740fca40 2024-05-17T21:44:20.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:20.378+0000 7f616b7fe700 1 -- 172.21.15.138:0/3376948267 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f616401c5b0 con 0x7f61740fca40 2024-05-17T21:44:20.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:20.378+0000 7f617902d700 1 -- 172.21.15.138:0/3376948267 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f617410e340 con 0x7f61740fca40 2024-05-17T21:44:20.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:20.379+0000 7f616b7fe700 1 -- 172.21.15.138:0/3376948267 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) v1 ==== 43433+0+0 (secure 0 0 0) 0x7f6164013420 con 0x7f61740fca40 2024-05-17T21:44:20.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:20.379+0000 7f617902d700 1 -- 172.21.15.138:0/3376948267 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6174102f50 con 0x7f61740fca40 2024-05-17T21:44:20.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:20.379+0000 7f616b7fe700 1 --2- 172.21.15.138:0/3376948267 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f61600368d0 0x7f6160038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:44:20.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:20.379+0000 7f616b7fe700 1 -- 172.21.15.138:0/3376948267 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f6164049ac0 con 0x7f61740fca40 2024-05-17T21:44:20.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:20.379+0000 7f617259c700 1 --2- 172.21.15.138:0/3376948267 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f61600368d0 0x7f6160038d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:44:20.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:20.380+0000 7f617259c700 1 --2- 172.21.15.138:0/3376948267 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f61600368d0 0x7f6160038d80 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f615c009990 tx=0x7f615c006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:44:20.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:20.384+0000 7f616b7fe700 1 -- 172.21.15.138:0/3376948267 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f6164004d30 con 0x7f61740fca40 2024-05-17T21:44:20.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:20.643+0000 7f617902d700 1 -- 172.21.15.138:0/3376948267 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f617410ed50 con 0x7f61740fca40 2024-05-17T21:44:20.638 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:20.644+0000 7f616b7fe700 1 -- 172.21.15.138:0/3376948267 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7f6164011370 con 0x7f61740fca40 2024-05-17T21:44:20.638 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-17T21:44:20.638 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":1,"fsid":"3722731a-1496-11ef-bc9a-c7b262605968","modified":"2024-05-17T21:42:10.472726Z","created":"2024-05-17T21:42:10.472726Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi107","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:3300","nonce":0},{"type":"v1","addr":"172.21.15.107:6789","nonce":0}]},"addr":"172.21.15.107:6789/0","public_addr":"172.21.15.107:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-17T21:44:20.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:20.646+0000 7f617902d700 1 -- 172.21.15.138:0/3376948267 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f61600368d0 msgr2=0x7f6160038d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:44:20.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:20.646+0000 7f617902d700 1 --2- 172.21.15.138:0/3376948267 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f61600368d0 0x7f6160038d80 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f615c009990 tx=0x7f615c006e40 comp rx=0 tx=0).stop 2024-05-17T21:44:20.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:20.646+0000 7f617902d700 1 -- 172.21.15.138:0/3376948267 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61740fca40 msgr2=0x7f617410d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:44:20.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:20.646+0000 7f617902d700 1 --2- 172.21.15.138:0/3376948267 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61740fca40 0x7f617410d090 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f61640096b0 tx=0x7f6164009e30 comp rx=0 tx=0).stop 2024-05-17T21:44:20.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:20.646+0000 7f617902d700 1 -- 172.21.15.138:0/3376948267 shutdown_connections 2024-05-17T21:44:20.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:20.646+0000 7f617902d700 1 --2- 172.21.15.138:0/3376948267 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f61600368d0 0x7f6160038d80 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:44:20.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:20.646+0000 7f617902d700 1 --2- 172.21.15.138:0/3376948267 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61740fca40 0x7f617410d090 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:44:20.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:20.646+0000 7f617902d700 1 -- 172.21.15.138:0/3376948267 >> 172.21.15.138:0/3376948267 conn(0x7f61740f81c0 msgr2=0x7f6174103b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:44:20.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:20.646+0000 7f617902d700 1 -- 172.21.15.138:0/3376948267 shutdown_connections 2024-05-17T21:44:20.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:20.646+0000 7f617902d700 1 -- 172.21.15.138:0/3376948267 wait complete. 2024-05-17T21:44:20.641 INFO:teuthology.orchestra.run.smithi138.stderr:dumped monmap epoch 1 2024-05-17T21:44:21.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:21 smithi107 ceph-mon[28434]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T21:44:21.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:21 smithi107 ceph-mon[28434]: from='client.? 172.21.15.138:0/3376948267' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-17T21:44:22.233 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-17T21:44:22.233 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -- ceph mon dump -f json 2024-05-17T21:44:22.482 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-17T21:44:22.530 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-17T21:44:23.353 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:23 smithi107 ceph-mon[28434]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T21:44:24.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:24.028+0000 7f90e5936700 1 -- 172.21.15.138:0/2264363332 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f90e00fca30 msgr2=0x7f90e00fce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:44:24.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:24.028+0000 7f90e5936700 1 --2- 172.21.15.138:0/2264363332 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f90e00fca30 0x7f90e00fce40 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f90d00048c0 tx=0x7f90d0009e30 comp rx=0 tx=0).stop 2024-05-17T21:44:24.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:24.028+0000 7f90e5936700 1 -- 172.21.15.138:0/2264363332 shutdown_connections 2024-05-17T21:44:24.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:24.028+0000 7f90e5936700 1 --2- 172.21.15.138:0/2264363332 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f90e00fca30 0x7f90e00fce40 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:44:24.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:24.028+0000 7f90e5936700 1 -- 172.21.15.138:0/2264363332 >> 172.21.15.138:0/2264363332 conn(0x7f90e00f8170 msgr2=0x7f90e00fa5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:44:24.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:24.028+0000 7f90e5936700 1 -- 172.21.15.138:0/2264363332 shutdown_connections 2024-05-17T21:44:24.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:24.029+0000 7f90e5936700 1 -- 172.21.15.138:0/2264363332 wait complete. 2024-05-17T21:44:24.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:24.029+0000 7f90e5936700 1 Processor -- start 2024-05-17T21:44:24.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:24.029+0000 7f90e5936700 1 -- start start 2024-05-17T21:44:24.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:24.029+0000 7f90e5936700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f90e00fca30 0x7f90e0071d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:44:24.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:24.029+0000 7f90e5936700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90e0072240 con 0x7f90e00fca30 2024-05-17T21:44:24.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:24.030+0000 7f90deffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f90e00fca30 0x7f90e0071d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:44:24.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:24.030+0000 7f90deffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f90e00fca30 0x7f90e0071d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.138:51726/0 (socket says 172.21.15.138:51726) 2024-05-17T21:44:24.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:24.030+0000 7f90deffd700 1 -- 172.21.15.138:0/1341998310 learned_addr learned my addr 172.21.15.138:0/1341998310 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-17T21:44:24.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:24.030+0000 7f90deffd700 1 -- 172.21.15.138:0/1341998310 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90d0005040 con 0x7f90e00fca30 2024-05-17T21:44:24.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:24.030+0000 7f90deffd700 1 --2- 172.21.15.138:0/1341998310 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f90e00fca30 0x7f90e0071d00 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f90d00096b0 tx=0x7f90d0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:44:24.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:24.030+0000 7f90c77fe700 1 -- 172.21.15.138:0/1341998310 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f90d000bed0 con 0x7f90e00fca30 2024-05-17T21:44:24.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:24.030+0000 7f90e5936700 1 -- 172.21.15.138:0/1341998310 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90e0070310 con 0x7f90e00fca30 2024-05-17T21:44:24.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:24.030+0000 7f90c77fe700 1 -- 172.21.15.138:0/1341998310 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f90d000b5c0 con 0x7f90e00fca30 2024-05-17T21:44:24.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:24.030+0000 7f90c77fe700 1 -- 172.21.15.138:0/1341998310 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f90d001c5b0 con 0x7f90e00fca30 2024-05-17T21:44:24.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:24.030+0000 7f90e5936700 1 -- 172.21.15.138:0/1341998310 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90e0070800 con 0x7f90e00fca30 2024-05-17T21:44:24.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:24.031+0000 7f90c77fe700 1 -- 172.21.15.138:0/1341998310 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) v1 ==== 43433+0+0 (secure 0 0 0) 0x7f90d001c820 con 0x7f90e00fca30 2024-05-17T21:44:24.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:24.031+0000 7f90e5936700 1 -- 172.21.15.138:0/1341998310 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90e005f5c0 con 0x7f90e00fca30 2024-05-17T21:44:24.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:24.031+0000 7f90c77fe700 1 --2- 172.21.15.138:0/1341998310 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f90c8036880 0x7f90c8038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:44:24.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:24.031+0000 7f90c77fe700 1 -- 172.21.15.138:0/1341998310 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f90d0049b10 con 0x7f90e00fca30 2024-05-17T21:44:24.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:24.032+0000 7f90de7fc700 1 --2- 172.21.15.138:0/1341998310 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f90c8036880 0x7f90c8038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:44:24.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:24.032+0000 7f90de7fc700 1 --2- 172.21.15.138:0/1341998310 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f90c8036880 0x7f90c8038d30 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f90d4009990 tx=0x7f90d4006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:44:24.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:24.037+0000 7f90c77fe700 1 -- 172.21.15.138:0/1341998310 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f90d0023300 con 0x7f90e00fca30 2024-05-17T21:44:24.295 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:24.301+0000 7f90e5936700 1 -- 172.21.15.138:0/1341998310 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f90e0106410 con 0x7f90e00fca30 2024-05-17T21:44:24.295 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:24.302+0000 7f90c77fe700 1 -- 172.21.15.138:0/1341998310 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7f90d0011b10 con 0x7f90e00fca30 2024-05-17T21:44:24.295 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-17T21:44:24.296 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":1,"fsid":"3722731a-1496-11ef-bc9a-c7b262605968","modified":"2024-05-17T21:42:10.472726Z","created":"2024-05-17T21:42:10.472726Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi107","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:3300","nonce":0},{"type":"v1","addr":"172.21.15.107:6789","nonce":0}]},"addr":"172.21.15.107:6789/0","public_addr":"172.21.15.107:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-17T21:44:24.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:24.304+0000 7f90e5936700 1 -- 172.21.15.138:0/1341998310 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f90c8036880 msgr2=0x7f90c8038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:44:24.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:24.304+0000 7f90e5936700 1 --2- 172.21.15.138:0/1341998310 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f90c8036880 0x7f90c8038d30 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f90d4009990 tx=0x7f90d4006e40 comp rx=0 tx=0).stop 2024-05-17T21:44:24.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:24.304+0000 7f90e5936700 1 -- 172.21.15.138:0/1341998310 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f90e00fca30 msgr2=0x7f90e0071d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:44:24.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:24.304+0000 7f90e5936700 1 --2- 172.21.15.138:0/1341998310 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f90e00fca30 0x7f90e0071d00 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f90d00096b0 tx=0x7f90d0009e30 comp rx=0 tx=0).stop 2024-05-17T21:44:24.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:24.304+0000 7f90e5936700 1 -- 172.21.15.138:0/1341998310 shutdown_connections 2024-05-17T21:44:24.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:24.304+0000 7f90e5936700 1 --2- 172.21.15.138:0/1341998310 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f90c8036880 0x7f90c8038d30 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:44:24.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:24.304+0000 7f90e5936700 1 --2- 172.21.15.138:0/1341998310 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f90e00fca30 0x7f90e0071d00 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:44:24.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:24.304+0000 7f90e5936700 1 -- 172.21.15.138:0/1341998310 >> 172.21.15.138:0/1341998310 conn(0x7f90e00f8170 msgr2=0x7f90e0103b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:44:24.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:24.304+0000 7f90e5936700 1 -- 172.21.15.138:0/1341998310 shutdown_connections 2024-05-17T21:44:24.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:24.304+0000 7f90e5936700 1 -- 172.21.15.138:0/1341998310 wait complete. 2024-05-17T21:44:24.299 INFO:teuthology.orchestra.run.smithi138.stderr:dumped monmap epoch 1 2024-05-17T21:44:25.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:25 smithi107 ceph-mon[28434]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T21:44:25.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:25 smithi107 ceph-mon[28434]: from='client.? 172.21.15.138:0/1341998310' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-17T21:44:25.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:25 smithi107 ceph-mon[28434]: from='mgr.14162 172.21.15.107:0/3666393387' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:44:25.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:25 smithi107 ceph-mon[28434]: from='mgr.14162 172.21.15.107:0/3666393387' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:44:25.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:25 smithi107 ceph-mon[28434]: from='mgr.14162 172.21.15.107:0/3666393387' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:44:25.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:25 smithi107 ceph-mon[28434]: from='mgr.14162 172.21.15.107:0/3666393387' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-05-17T21:44:25.861 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-17T21:44:25.861 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -- ceph mon dump -f json 2024-05-17T21:44:26.104 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-17T21:44:26.151 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-17T21:44:27.104 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:26 smithi107 ceph-mon[28434]: from='mgr.14162 172.21.15.107:0/3666393387' entity='mgr.smithi107.xhhyiw' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-05-17T21:44:27.104 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:26 smithi107 ceph-mon[28434]: mgrmap e14: smithi107.xhhyiw(active, since 63s) 2024-05-17T21:44:27.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:27.639+0000 7f9224ddc700 1 -- 172.21.15.138:0/1158754547 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92200fe080 msgr2=0x7f92200fe490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:44:27.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:27.639+0000 7f9224ddc700 1 --2- 172.21.15.138:0/1158754547 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92200fe080 0x7f92200fe490 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f92100048c0 tx=0x7f9210009e30 comp rx=0 tx=0).stop 2024-05-17T21:44:27.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:27.639+0000 7f9224ddc700 1 -- 172.21.15.138:0/1158754547 shutdown_connections 2024-05-17T21:44:27.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:27.639+0000 7f9224ddc700 1 --2- 172.21.15.138:0/1158754547 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92200fe080 0x7f92200fe490 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:44:27.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:27.639+0000 7f9224ddc700 1 -- 172.21.15.138:0/1158754547 >> 172.21.15.138:0/1158754547 conn(0x7f92200f6820 msgr2=0x7f92200f8c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:44:27.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:27.639+0000 7f9224ddc700 1 -- 172.21.15.138:0/1158754547 shutdown_connections 2024-05-17T21:44:27.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:27.639+0000 7f9224ddc700 1 -- 172.21.15.138:0/1158754547 wait complete. 2024-05-17T21:44:27.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:27.640+0000 7f9224ddc700 1 Processor -- start 2024-05-17T21:44:27.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:27.640+0000 7f9224ddc700 1 -- start start 2024-05-17T21:44:27.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:27.640+0000 7f9224ddc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92200fe080 0x7f922010d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:44:27.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:27.640+0000 7f9224ddc700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f922010d5e0 con 0x7f92200fe080 2024-05-17T21:44:27.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:27.640+0000 7f921e59c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92200fe080 0x7f922010d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:44:27.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:27.640+0000 7f921e59c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92200fe080 0x7f922010d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.138:51740/0 (socket says 172.21.15.138:51740) 2024-05-17T21:44:27.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:27.640+0000 7f921e59c700 1 -- 172.21.15.138:0/590217749 learned_addr learned my addr 172.21.15.138:0/590217749 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-17T21:44:27.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:27.641+0000 7f921e59c700 1 -- 172.21.15.138:0/590217749 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9210005040 con 0x7f92200fe080 2024-05-17T21:44:27.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:27.641+0000 7f921e59c700 1 --2- 172.21.15.138:0/590217749 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92200fe080 0x7f922010d0a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f92100096b0 tx=0x7f9210009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:44:27.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:27.641+0000 7f920effd700 1 -- 172.21.15.138:0/590217749 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f921000bed0 con 0x7f92200fe080 2024-05-17T21:44:27.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:27.641+0000 7f9224ddc700 1 -- 172.21.15.138:0/590217749 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f922010d7e0 con 0x7f92200fe080 2024-05-17T21:44:27.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:27.641+0000 7f920effd700 1 -- 172.21.15.138:0/590217749 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f921000b5c0 con 0x7f92200fe080 2024-05-17T21:44:27.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:27.641+0000 7f920effd700 1 -- 172.21.15.138:0/590217749 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f921001c5b0 con 0x7f92200fe080 2024-05-17T21:44:27.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:27.641+0000 7f9224ddc700 1 -- 172.21.15.138:0/590217749 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f922010e350 con 0x7f92200fe080 2024-05-17T21:44:27.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:27.642+0000 7f9224ddc700 1 -- 172.21.15.138:0/590217749 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9220102f50 con 0x7f92200fe080 2024-05-17T21:44:27.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:27.642+0000 7f920effd700 1 -- 172.21.15.138:0/590217749 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 14) v1 ==== 43447+0+0 (secure 0 0 0) 0x7f9210013a80 con 0x7f92200fe080 2024-05-17T21:44:27.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:27.642+0000 7f920effd700 1 --2- 172.21.15.138:0/590217749 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f92080368d0 0x7f9208038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:44:27.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:27.642+0000 7f920effd700 1 -- 172.21.15.138:0/590217749 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f9210049aa0 con 0x7f92200fe080 2024-05-17T21:44:27.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:27.642+0000 7f921dd9b700 1 -- 172.21.15.138:0/590217749 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f92080368d0 msgr2=0x7f9208038d80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/3405524625 2024-05-17T21:44:27.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:27.642+0000 7f921dd9b700 1 --2- 172.21.15.138:0/590217749 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f92080368d0 0x7f9208038d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-17T21:44:27.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:27.647+0000 7f920effd700 1 -- 172.21.15.138:0/590217749 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f9210011940 con 0x7f92200fe080 2024-05-17T21:44:27.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:27.843+0000 7f921dd9b700 1 -- 172.21.15.138:0/590217749 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f92080368d0 msgr2=0x7f9208038d80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/3405524625 2024-05-17T21:44:27.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:27.843+0000 7f921dd9b700 1 --2- 172.21.15.138:0/590217749 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f92080368d0 0x7f9208038d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-17T21:44:27.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:27.907+0000 7f9224ddc700 1 -- 172.21.15.138:0/590217749 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f922010ed60 con 0x7f92200fe080 2024-05-17T21:44:27.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:27.907+0000 7f920effd700 1 -- 172.21.15.138:0/590217749 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7f9210011370 con 0x7f92200fe080 2024-05-17T21:44:27.901 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-17T21:44:27.901 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":1,"fsid":"3722731a-1496-11ef-bc9a-c7b262605968","modified":"2024-05-17T21:42:10.472726Z","created":"2024-05-17T21:42:10.472726Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi107","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:3300","nonce":0},{"type":"v1","addr":"172.21.15.107:6789","nonce":0}]},"addr":"172.21.15.107:6789/0","public_addr":"172.21.15.107:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-17T21:44:27.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:27.909+0000 7f9224ddc700 1 -- 172.21.15.138:0/590217749 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f92080368d0 msgr2=0x7f9208038d80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-17T21:44:27.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:27.909+0000 7f9224ddc700 1 --2- 172.21.15.138:0/590217749 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f92080368d0 0x7f9208038d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:44:27.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:27.909+0000 7f9224ddc700 1 -- 172.21.15.138:0/590217749 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92200fe080 msgr2=0x7f922010d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:44:27.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:27.909+0000 7f9224ddc700 1 --2- 172.21.15.138:0/590217749 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92200fe080 0x7f922010d0a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f92100096b0 tx=0x7f9210009e30 comp rx=0 tx=0).stop 2024-05-17T21:44:27.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:27.909+0000 7f9224ddc700 1 -- 172.21.15.138:0/590217749 shutdown_connections 2024-05-17T21:44:27.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:27.909+0000 7f9224ddc700 1 --2- 172.21.15.138:0/590217749 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f92080368d0 0x7f9208038d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:44:27.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:27.909+0000 7f9224ddc700 1 --2- 172.21.15.138:0/590217749 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92200fe080 0x7f922010d0a0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:44:27.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:27.909+0000 7f9224ddc700 1 -- 172.21.15.138:0/590217749 >> 172.21.15.138:0/590217749 conn(0x7f92200f6820 msgr2=0x7f9220103b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:44:27.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:27.910+0000 7f9224ddc700 1 -- 172.21.15.138:0/590217749 shutdown_connections 2024-05-17T21:44:27.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:27.910+0000 7f9224ddc700 1 -- 172.21.15.138:0/590217749 wait complete. 2024-05-17T21:44:27.904 INFO:teuthology.orchestra.run.smithi138.stderr:dumped monmap epoch 1 2024-05-17T21:44:28.061 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:27 smithi107 ceph-mon[28434]: from='client.? 172.21.15.138:0/590217749' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-17T21:44:29.559 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-17T21:44:29.559 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -- ceph mon dump -f json 2024-05-17T21:44:29.809 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-17T21:44:29.856 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-17T21:44:31.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:31.293+0000 7f82b5763700 1 -- 172.21.15.138:0/3168331350 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82b00fca30 msgr2=0x7f82b00fce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:44:31.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:31.293+0000 7f82b5763700 1 --2- 172.21.15.138:0/3168331350 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82b00fca30 0x7f82b00fce40 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f82a80048c0 tx=0x7f82a8009e30 comp rx=0 tx=0).stop 2024-05-17T21:44:31.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:31.293+0000 7f82b5763700 1 -- 172.21.15.138:0/3168331350 shutdown_connections 2024-05-17T21:44:31.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:31.293+0000 7f82b5763700 1 --2- 172.21.15.138:0/3168331350 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82b00fca30 0x7f82b00fce40 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:44:31.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:31.293+0000 7f82b5763700 1 -- 172.21.15.138:0/3168331350 >> 172.21.15.138:0/3168331350 conn(0x7f82b00f8170 msgr2=0x7f82b00fa5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:44:31.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:31.293+0000 7f82b5763700 1 -- 172.21.15.138:0/3168331350 shutdown_connections 2024-05-17T21:44:31.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:31.294+0000 7f82b5763700 1 -- 172.21.15.138:0/3168331350 wait complete. 2024-05-17T21:44:31.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:31.294+0000 7f82b5763700 1 Processor -- start 2024-05-17T21:44:31.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:31.294+0000 7f82b5763700 1 -- start start 2024-05-17T21:44:31.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:31.294+0000 7f82b5763700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82b00fca30 0x7f82b010d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:44:31.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:31.294+0000 7f82b5763700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82b010d5c0 con 0x7f82b00fca30 2024-05-17T21:44:31.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:31.295+0000 7f82aeffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82b00fca30 0x7f82b010d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:44:31.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:31.295+0000 7f82aeffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82b00fca30 0x7f82b010d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.138:51758/0 (socket says 172.21.15.138:51758) 2024-05-17T21:44:31.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:31.295+0000 7f82aeffd700 1 -- 172.21.15.138:0/2632205170 learned_addr learned my addr 172.21.15.138:0/2632205170 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-17T21:44:31.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:31.295+0000 7f82aeffd700 1 -- 172.21.15.138:0/2632205170 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82a8005040 con 0x7f82b00fca30 2024-05-17T21:44:31.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:31.295+0000 7f82aeffd700 1 --2- 172.21.15.138:0/2632205170 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82b00fca30 0x7f82b010d080 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f82a80096b0 tx=0x7f82a8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:44:31.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:31.295+0000 7f82a77fe700 1 -- 172.21.15.138:0/2632205170 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f82a800bed0 con 0x7f82b00fca30 2024-05-17T21:44:31.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:31.295+0000 7f82a77fe700 1 -- 172.21.15.138:0/2632205170 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f82a800b5c0 con 0x7f82b00fca30 2024-05-17T21:44:31.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:31.295+0000 7f82b5763700 1 -- 172.21.15.138:0/2632205170 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82b010d7c0 con 0x7f82b00fca30 2024-05-17T21:44:31.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:31.295+0000 7f82a77fe700 1 -- 172.21.15.138:0/2632205170 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f82a801c5b0 con 0x7f82b00fca30 2024-05-17T21:44:31.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:31.295+0000 7f82b5763700 1 -- 172.21.15.138:0/2632205170 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82b010e330 con 0x7f82b00fca30 2024-05-17T21:44:31.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:31.296+0000 7f82a77fe700 1 -- 172.21.15.138:0/2632205170 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 14) v1 ==== 43447+0+0 (secure 0 0 0) 0x7f82a801c820 con 0x7f82b00fca30 2024-05-17T21:44:31.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:31.296+0000 7f82b5763700 1 -- 172.21.15.138:0/2632205170 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82b0102f40 con 0x7f82b00fca30 2024-05-17T21:44:31.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:31.296+0000 7f82a77fe700 1 --2- 172.21.15.138:0/2632205170 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f829c0368d0 0x7f829c038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:44:31.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:31.296+0000 7f82a77fe700 1 -- 172.21.15.138:0/2632205170 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f82a8049b90 con 0x7f82b00fca30 2024-05-17T21:44:31.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:31.296+0000 7f82ae7fc700 1 -- 172.21.15.138:0/2632205170 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f829c0368d0 msgr2=0x7f829c038d80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/3405524625 2024-05-17T21:44:31.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:31.296+0000 7f82ae7fc700 1 --2- 172.21.15.138:0/2632205170 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f829c0368d0 0x7f829c038d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-17T21:44:31.295 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:31.301+0000 7f82a77fe700 1 -- 172.21.15.138:0/2632205170 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f82a8011550 con 0x7f82b00fca30 2024-05-17T21:44:31.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:31.497+0000 7f82ae7fc700 1 -- 172.21.15.138:0/2632205170 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f829c0368d0 msgr2=0x7f829c038d80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/3405524625 2024-05-17T21:44:31.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:31.497+0000 7f82ae7fc700 1 --2- 172.21.15.138:0/2632205170 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f829c0368d0 0x7f829c038d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-17T21:44:31.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:31.562+0000 7f82b5763700 1 -- 172.21.15.138:0/2632205170 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f82b010ee00 con 0x7f82b00fca30 2024-05-17T21:44:31.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:31.563+0000 7f82a77fe700 1 -- 172.21.15.138:0/2632205170 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7f82a8004d30 con 0x7f82b00fca30 2024-05-17T21:44:31.557 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-17T21:44:31.557 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":1,"fsid":"3722731a-1496-11ef-bc9a-c7b262605968","modified":"2024-05-17T21:42:10.472726Z","created":"2024-05-17T21:42:10.472726Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi107","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:3300","nonce":0},{"type":"v1","addr":"172.21.15.107:6789","nonce":0}]},"addr":"172.21.15.107:6789/0","public_addr":"172.21.15.107:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-17T21:44:31.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:31.565+0000 7f82b5763700 1 -- 172.21.15.138:0/2632205170 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f829c0368d0 msgr2=0x7f829c038d80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-17T21:44:31.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:31.565+0000 7f82b5763700 1 --2- 172.21.15.138:0/2632205170 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f829c0368d0 0x7f829c038d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:44:31.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:31.565+0000 7f82b5763700 1 -- 172.21.15.138:0/2632205170 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82b00fca30 msgr2=0x7f82b010d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:44:31.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:31.565+0000 7f82b5763700 1 --2- 172.21.15.138:0/2632205170 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82b00fca30 0x7f82b010d080 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f82a80096b0 tx=0x7f82a8009e30 comp rx=0 tx=0).stop 2024-05-17T21:44:31.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:31.565+0000 7f82b5763700 1 -- 172.21.15.138:0/2632205170 shutdown_connections 2024-05-17T21:44:31.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:31.565+0000 7f82b5763700 1 --2- 172.21.15.138:0/2632205170 >> [v2:172.21.15.107:6800/3405524625,v1:172.21.15.107:6801/3405524625] conn(0x7f829c0368d0 0x7f829c038d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:44:31.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:31.565+0000 7f82b5763700 1 --2- 172.21.15.138:0/2632205170 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82b00fca30 0x7f82b010d080 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:44:31.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:31.565+0000 7f82b5763700 1 -- 172.21.15.138:0/2632205170 >> 172.21.15.138:0/2632205170 conn(0x7f82b00f8170 msgr2=0x7f82b0103b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:44:31.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:31.565+0000 7f82b5763700 1 -- 172.21.15.138:0/2632205170 shutdown_connections 2024-05-17T21:44:31.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:31.565+0000 7f82b5763700 1 -- 172.21.15.138:0/2632205170 wait complete. 2024-05-17T21:44:31.560 INFO:teuthology.orchestra.run.smithi138.stderr:dumped monmap epoch 1 2024-05-17T21:44:32.095 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:31 smithi107 ceph-mon[28434]: from='client.? 172.21.15.138:0/2632205170' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-17T21:44:32.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:32 smithi107 ceph-mon[28434]: Active manager daemon smithi107.xhhyiw restarted 2024-05-17T21:44:32.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:32 smithi107 ceph-mon[28434]: Activating manager daemon smithi107.xhhyiw 2024-05-17T21:44:32.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:32 smithi107 ceph-mon[28434]: osdmap e5: 0 total, 0 up, 0 in 2024-05-17T21:44:32.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:32 smithi107 ceph-mon[28434]: mgrmap e15: smithi107.xhhyiw(active, starting, since 0.00366439s) 2024-05-17T21:44:32.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:32 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mon metadata", "id": "smithi107"}]: dispatch 2024-05-17T21:44:32.855 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:32 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mgr metadata", "who": "smithi107.xhhyiw", "id": "smithi107.xhhyiw"}]: dispatch 2024-05-17T21:44:32.855 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:32 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-17T21:44:32.855 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:32 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-17T21:44:32.855 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:32 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-17T21:44:32.855 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:32 smithi107 ceph-mon[28434]: Manager daemon smithi107.xhhyiw is now available 2024-05-17T21:44:32.855 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:32 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:44:32.855 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:32 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:44:32.855 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:32 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:44:32.855 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:32 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:44:32.855 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:32 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi107.xhhyiw/mirror_snapshot_schedule"}]: dispatch 2024-05-17T21:44:32.855 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:32 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi107.xhhyiw/trash_purge_schedule"}]: dispatch 2024-05-17T21:44:33.120 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-17T21:44:33.120 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -- ceph mon dump -f json 2024-05-17T21:44:33.390 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-17T21:44:33.439 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-17T21:44:34.354 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:33 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:44:34.354 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:33 smithi107 ceph-mon[28434]: [17/May/2024:21:44:32] ENGINE Bus STARTING 2024-05-17T21:44:34.354 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:33 smithi107 ceph-mon[28434]: [17/May/2024:21:44:33] ENGINE Serving on https://172.21.15.107:7150 2024-05-17T21:44:34.354 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:33 smithi107 ceph-mon[28434]: [17/May/2024:21:44:33] ENGINE Bus STARTED 2024-05-17T21:44:34.354 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:33 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:44:34.354 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:33 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:44:34.354 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:33 smithi107 ceph-mon[28434]: mgrmap e16: smithi107.xhhyiw(active, since 1.00558s) 2024-05-17T21:44:35.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:35.526+0000 7f1461e85700 1 -- 172.21.15.138:0/3971841326 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f145c0fb930 msgr2=0x7f145c0fbd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:44:35.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:35.526+0000 7f1461e85700 1 --2- 172.21.15.138:0/3971841326 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f145c0fb930 0x7f145c0fbd40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1454009c60 tx=0x7f1454004d10 comp rx=0 tx=0).stop 2024-05-17T21:44:35.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:35.526+0000 7f1461e85700 1 -- 172.21.15.138:0/3971841326 shutdown_connections 2024-05-17T21:44:35.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:35.526+0000 7f1461e85700 1 --2- 172.21.15.138:0/3971841326 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f145c0fb930 0x7f145c0fbd40 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:44:35.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:35.526+0000 7f1461e85700 1 -- 172.21.15.138:0/3971841326 >> 172.21.15.138:0/3971841326 conn(0x7f145c0f7070 msgr2=0x7f145c0f94c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:44:35.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:35.526+0000 7f1461e85700 1 -- 172.21.15.138:0/3971841326 shutdown_connections 2024-05-17T21:44:35.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:35.527+0000 7f1461e85700 1 -- 172.21.15.138:0/3971841326 wait complete. 2024-05-17T21:44:35.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:35.527+0000 7f1461e85700 1 Processor -- start 2024-05-17T21:44:35.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:35.527+0000 7f1461e85700 1 -- start start 2024-05-17T21:44:35.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:35.527+0000 7f1461e85700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f145c0fb930 0x7f145c066840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:44:35.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:35.527+0000 7f1461e85700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f145c10fdb0 con 0x7f145c0fb930 2024-05-17T21:44:35.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:35.528+0000 7f145b7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f145c0fb930 0x7f145c066840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:44:35.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:35.528+0000 7f145b7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f145c0fb930 0x7f145c066840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.138:43526/0 (socket says 172.21.15.138:43526) 2024-05-17T21:44:35.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:35.528+0000 7f145b7fe700 1 -- 172.21.15.138:0/1230426615 learned_addr learned my addr 172.21.15.138:0/1230426615 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-17T21:44:35.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:35.528+0000 7f145b7fe700 1 -- 172.21.15.138:0/1230426615 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1454009940 con 0x7f145c0fb930 2024-05-17T21:44:35.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:35.528+0000 7f145b7fe700 1 --2- 172.21.15.138:0/1230426615 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f145c0fb930 0x7f145c066840 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f1454009500 tx=0x7f1454004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:44:35.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:35.528+0000 7f1453fff700 1 -- 172.21.15.138:0/1230426615 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1454007c80 con 0x7f145c0fb930 2024-05-17T21:44:35.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:35.528+0000 7f1453fff700 1 -- 172.21.15.138:0/1230426615 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f145401d910 con 0x7f145c0fb930 2024-05-17T21:44:35.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:35.528+0000 7f1461e85700 1 -- 172.21.15.138:0/1230426615 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f145c10fef0 con 0x7f145c0fb930 2024-05-17T21:44:35.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:35.528+0000 7f1453fff700 1 -- 172.21.15.138:0/1230426615 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f14540129d0 con 0x7f145c0fb930 2024-05-17T21:44:35.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:35.528+0000 7f1461e85700 1 -- 172.21.15.138:0/1230426615 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f145c1103e0 con 0x7f145c0fb930 2024-05-17T21:44:35.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:35.529+0000 7f1453fff700 1 -- 172.21.15.138:0/1230426615 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 43491+0+0 (secure 0 0 0) 0x7f1454019040 con 0x7f145c0fb930 2024-05-17T21:44:35.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:35.529+0000 7f1461e85700 1 -- 172.21.15.138:0/1230426615 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f145c1058f0 con 0x7f145c0fb930 2024-05-17T21:44:35.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:35.529+0000 7f1453fff700 1 --2- 172.21.15.138:0/1230426615 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f1448036970 0x7f1448038e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:44:35.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:35.529+0000 7f1453fff700 1 -- 172.21.15.138:0/1230426615 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f145404a690 con 0x7f145c0fb930 2024-05-17T21:44:35.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:35.529+0000 7f145affd700 1 --2- 172.21.15.138:0/1230426615 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f1448036970 0x7f1448038e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:44:35.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:35.530+0000 7f145affd700 1 --2- 172.21.15.138:0/1230426615 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f1448036970 0x7f1448038e20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1444009990 tx=0x7f1444006e40 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:44:35.528 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:35.534+0000 7f1453fff700 1 -- 172.21.15.138:0/1230426615 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f145402eb40 con 0x7f145c0fb930 2024-05-17T21:44:35.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:35 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:44:35.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:35.796+0000 7f1461e85700 1 -- 172.21.15.138:0/1230426615 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f145c05f5c0 con 0x7f145c0fb930 2024-05-17T21:44:35.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:35.796+0000 7f1453fff700 1 -- 172.21.15.138:0/1230426615 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7f1454017020 con 0x7f145c0fb930 2024-05-17T21:44:35.790 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-17T21:44:35.790 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":1,"fsid":"3722731a-1496-11ef-bc9a-c7b262605968","modified":"2024-05-17T21:42:10.472726Z","created":"2024-05-17T21:42:10.472726Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi107","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:3300","nonce":0},{"type":"v1","addr":"172.21.15.107:6789","nonce":0}]},"addr":"172.21.15.107:6789/0","public_addr":"172.21.15.107:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-17T21:44:35.792 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:35.798+0000 7f1461e85700 1 -- 172.21.15.138:0/1230426615 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f1448036970 msgr2=0x7f1448038e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:44:35.793 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:35.798+0000 7f1461e85700 1 --2- 172.21.15.138:0/1230426615 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f1448036970 0x7f1448038e20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1444009990 tx=0x7f1444006e40 comp rx=0 tx=0).stop 2024-05-17T21:44:35.793 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:35.798+0000 7f1461e85700 1 -- 172.21.15.138:0/1230426615 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f145c0fb930 msgr2=0x7f145c066840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:44:35.793 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:35.798+0000 7f1461e85700 1 --2- 172.21.15.138:0/1230426615 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f145c0fb930 0x7f145c066840 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f1454009500 tx=0x7f1454004d10 comp rx=0 tx=0).stop 2024-05-17T21:44:35.793 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:35.798+0000 7f1461e85700 1 -- 172.21.15.138:0/1230426615 shutdown_connections 2024-05-17T21:44:35.793 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:35.798+0000 7f1461e85700 1 --2- 172.21.15.138:0/1230426615 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f1448036970 0x7f1448038e20 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:44:35.793 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:35.798+0000 7f1461e85700 1 --2- 172.21.15.138:0/1230426615 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f145c0fb930 0x7f145c066840 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:44:35.793 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:35.798+0000 7f1461e85700 1 -- 172.21.15.138:0/1230426615 >> 172.21.15.138:0/1230426615 conn(0x7f145c0f7070 msgr2=0x7f145c102a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:44:35.793 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:35.799+0000 7f1461e85700 1 -- 172.21.15.138:0/1230426615 shutdown_connections 2024-05-17T21:44:35.793 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:35.799+0000 7f1461e85700 1 -- 172.21.15.138:0/1230426615 wait complete. 2024-05-17T21:44:35.794 INFO:teuthology.orchestra.run.smithi138.stderr:dumped monmap epoch 1 2024-05-17T21:44:36.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:36 smithi107 ceph-mon[28434]: mgrmap e17: smithi107.xhhyiw(active, since 2s) 2024-05-17T21:44:36.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:36 smithi107 ceph-mon[28434]: from='client.? 172.21.15.138:0/1230426615' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-17T21:44:37.389 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-17T21:44:37.389 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -- ceph mon dump -f json 2024-05-17T21:44:37.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:37 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:44:37.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:37 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:44:37.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:37 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:44:37.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:37 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:44:37.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:37 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config rm", "who": "osd/host:smithi107", "name": "osd_memory_target"}]: dispatch 2024-05-17T21:44:38.147 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-17T21:44:38.196 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-17T21:44:39.410 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:39 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:44:39.410 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:39 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:44:39.410 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:39 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:44:39.410 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:39 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:44:39.410 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:39 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config rm", "who": "osd/host:smithi138", "name": "osd_memory_target"}]: dispatch 2024-05-17T21:44:39.411 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:39 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:44:39.411 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:39 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T21:44:39.411 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:39 smithi107 ceph-mon[28434]: Updating smithi107:/etc/ceph/ceph.conf 2024-05-17T21:44:39.411 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:39 smithi107 ceph-mon[28434]: Updating smithi138:/etc/ceph/ceph.conf 2024-05-17T21:44:39.411 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:39 smithi107 ceph-mon[28434]: Updating smithi107:/var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-17T21:44:39.411 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:39 smithi107 ceph-mon[28434]: Updating smithi138:/var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-17T21:44:39.411 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:39 smithi107 ceph-mon[28434]: Updating smithi107:/etc/ceph/ceph.client.admin.keyring 2024-05-17T21:44:39.411 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:39 smithi107 ceph-mon[28434]: Updating smithi138:/etc/ceph/ceph.client.admin.keyring 2024-05-17T21:44:39.411 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:39 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:44:39.411 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:39 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:44:39.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:39.901+0000 7f38f2ced700 1 -- 172.21.15.138:0/2136235634 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f38ec102a90 msgr2=0x7f38ec102e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:44:39.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:39.901+0000 7f38f2ced700 1 --2- 172.21.15.138:0/2136235634 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f38ec102a90 0x7f38ec102e60 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f38e00048c0 tx=0x7f38e0009e30 comp rx=0 tx=0).stop 2024-05-17T21:44:39.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:39.901+0000 7f38f2ced700 1 -- 172.21.15.138:0/2136235634 shutdown_connections 2024-05-17T21:44:39.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:39.901+0000 7f38f2ced700 1 --2- 172.21.15.138:0/2136235634 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f38ec102a90 0x7f38ec102e60 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:44:39.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:39.901+0000 7f38f2ced700 1 -- 172.21.15.138:0/2136235634 >> 172.21.15.138:0/2136235634 conn(0x7f38ec0f85b0 msgr2=0x7f38ec0fa9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:44:39.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:39.901+0000 7f38f2ced700 1 -- 172.21.15.138:0/2136235634 shutdown_connections 2024-05-17T21:44:39.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:39.901+0000 7f38f2ced700 1 -- 172.21.15.138:0/2136235634 wait complete. 2024-05-17T21:44:39.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:39.902+0000 7f38f2ced700 1 Processor -- start 2024-05-17T21:44:39.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:39.902+0000 7f38f2ced700 1 -- start start 2024-05-17T21:44:39.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:39.902+0000 7f38f2ced700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f38ec102a90 0x7f38ec112010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:44:39.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:39.902+0000 7f38f2ced700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38ec112550 con 0x7f38ec102a90 2024-05-17T21:44:39.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:39.902+0000 7f38f0a89700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f38ec102a90 0x7f38ec112010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:44:39.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:39.902+0000 7f38f0a89700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f38ec102a90 0x7f38ec112010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.138:43546/0 (socket says 172.21.15.138:43546) 2024-05-17T21:44:39.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:39.902+0000 7f38f0a89700 1 -- 172.21.15.138:0/984069725 learned_addr learned my addr 172.21.15.138:0/984069725 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-17T21:44:39.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:39.902+0000 7f38f0a89700 1 -- 172.21.15.138:0/984069725 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38e0005040 con 0x7f38ec102a90 2024-05-17T21:44:39.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:39.903+0000 7f38f0a89700 1 --2- 172.21.15.138:0/984069725 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f38ec102a90 0x7f38ec112010 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f38e000b820 tx=0x7f38e0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:44:39.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:39.903+0000 7f38e97fa700 1 -- 172.21.15.138:0/984069725 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f38e00093c0 con 0x7f38ec102a90 2024-05-17T21:44:39.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:39.903+0000 7f38f2ced700 1 -- 172.21.15.138:0/984069725 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38ec1126f0 con 0x7f38ec102a90 2024-05-17T21:44:39.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:39.903+0000 7f38e97fa700 1 -- 172.21.15.138:0/984069725 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f38e0013420 con 0x7f38ec102a90 2024-05-17T21:44:39.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:39.903+0000 7f38e97fa700 1 -- 172.21.15.138:0/984069725 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f38e001c5b0 con 0x7f38ec102a90 2024-05-17T21:44:39.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:39.903+0000 7f38f2ced700 1 -- 172.21.15.138:0/984069725 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38ec112be0 con 0x7f38ec102a90 2024-05-17T21:44:39.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:39.904+0000 7f38e97fa700 1 -- 172.21.15.138:0/984069725 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 43491+0+0 (secure 0 0 0) 0x7f38e001c7f0 con 0x7f38ec102a90 2024-05-17T21:44:39.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:39.904+0000 7f38f2ced700 1 -- 172.21.15.138:0/984069725 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38ec0fe190 con 0x7f38ec102a90 2024-05-17T21:44:39.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:39.904+0000 7f38e97fa700 1 --2- 172.21.15.138:0/984069725 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f38dc036970 0x7f38dc038e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:44:39.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:39.904+0000 7f38e97fa700 1 -- 172.21.15.138:0/984069725 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f38e0049b20 con 0x7f38ec102a90 2024-05-17T21:44:39.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:39.904+0000 7f38ebfff700 1 --2- 172.21.15.138:0/984069725 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f38dc036970 0x7f38dc038e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:44:39.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:39.905+0000 7f38ebfff700 1 --2- 172.21.15.138:0/984069725 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f38dc036970 0x7f38dc038e20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f38d8009990 tx=0x7f38d8006e40 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:44:39.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:39.909+0000 7f38e97fa700 1 -- 172.21.15.138:0/984069725 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f38e0023300 con 0x7f38ec102a90 2024-05-17T21:44:40.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:40.179+0000 7f38f2ced700 1 -- 172.21.15.138:0/984069725 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f38ec05f5c0 con 0x7f38ec102a90 2024-05-17T21:44:40.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:40.180+0000 7f38e97fa700 1 -- 172.21.15.138:0/984069725 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7f38e0011b60 con 0x7f38ec102a90 2024-05-17T21:44:40.174 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-17T21:44:40.174 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":1,"fsid":"3722731a-1496-11ef-bc9a-c7b262605968","modified":"2024-05-17T21:42:10.472726Z","created":"2024-05-17T21:42:10.472726Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi107","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:3300","nonce":0},{"type":"v1","addr":"172.21.15.107:6789","nonce":0}]},"addr":"172.21.15.107:6789/0","public_addr":"172.21.15.107:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-17T21:44:40.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:40.182+0000 7f38f2ced700 1 -- 172.21.15.138:0/984069725 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f38dc036970 msgr2=0x7f38dc038e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:44:40.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:40.182+0000 7f38f2ced700 1 --2- 172.21.15.138:0/984069725 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f38dc036970 0x7f38dc038e20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f38d8009990 tx=0x7f38d8006e40 comp rx=0 tx=0).stop 2024-05-17T21:44:40.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:40.182+0000 7f38f2ced700 1 -- 172.21.15.138:0/984069725 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f38ec102a90 msgr2=0x7f38ec112010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:44:40.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:40.182+0000 7f38f2ced700 1 --2- 172.21.15.138:0/984069725 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f38ec102a90 0x7f38ec112010 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f38e000b820 tx=0x7f38e0009e30 comp rx=0 tx=0).stop 2024-05-17T21:44:40.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:40.182+0000 7f38f2ced700 1 -- 172.21.15.138:0/984069725 shutdown_connections 2024-05-17T21:44:40.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:40.182+0000 7f38f2ced700 1 --2- 172.21.15.138:0/984069725 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f38dc036970 0x7f38dc038e20 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:44:40.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:40.182+0000 7f38f2ced700 1 --2- 172.21.15.138:0/984069725 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f38ec102a90 0x7f38ec112010 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:44:40.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:40.182+0000 7f38f2ced700 1 -- 172.21.15.138:0/984069725 >> 172.21.15.138:0/984069725 conn(0x7f38ec0f85b0 msgr2=0x7f38ec0f9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:44:40.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:40.182+0000 7f38f2ced700 1 -- 172.21.15.138:0/984069725 shutdown_connections 2024-05-17T21:44:40.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:40.182+0000 7f38f2ced700 1 -- 172.21.15.138:0/984069725 wait complete. 2024-05-17T21:44:40.177 INFO:teuthology.orchestra.run.smithi138.stderr:dumped monmap epoch 1 2024-05-17T21:44:40.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:40 smithi107 ceph-mon[28434]: Updating smithi138:/var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-17T21:44:40.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:40 smithi107 ceph-mon[28434]: Updating smithi107:/var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-17T21:44:40.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:40 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:44:40.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:40 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:44:40.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:40 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:44:40.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:40 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi138", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-17T21:44:40.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:40 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi138", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-05-17T21:44:40.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:40 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:44:40.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:40 smithi107 ceph-mon[28434]: Deploying daemon crash.smithi138 on smithi138 2024-05-17T21:44:41.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:41 smithi107 ceph-mon[28434]: from='client.? 172.21.15.138:0/984069725' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-17T21:44:42.103 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-17T21:44:42.104 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -- ceph mon dump -f json 2024-05-17T21:44:42.372 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-17T21:44:44.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:44.203+0000 7f91a0a65700 1 -- 172.21.15.138:0/2489774301 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f919c102aa0 msgr2=0x7f919c102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:44:44.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:44.203+0000 7f91a0a65700 1 --2- 172.21.15.138:0/2489774301 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f919c102aa0 0x7f919c102e70 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f918c0048c0 tx=0x7f918c009e30 comp rx=0 tx=0).stop 2024-05-17T21:44:44.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:44.203+0000 7f91a0a65700 1 -- 172.21.15.138:0/2489774301 shutdown_connections 2024-05-17T21:44:44.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:44.203+0000 7f91a0a65700 1 --2- 172.21.15.138:0/2489774301 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f919c102aa0 0x7f919c102e70 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:44:44.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:44.203+0000 7f91a0a65700 1 -- 172.21.15.138:0/2489774301 >> 172.21.15.138:0/2489774301 conn(0x7f919c0f85c0 msgr2=0x7f919c0fa9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:44:44.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:44.203+0000 7f91a0a65700 1 -- 172.21.15.138:0/2489774301 shutdown_connections 2024-05-17T21:44:44.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:44.203+0000 7f91a0a65700 1 -- 172.21.15.138:0/2489774301 wait complete. 2024-05-17T21:44:44.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:44.204+0000 7f91a0a65700 1 Processor -- start 2024-05-17T21:44:44.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:44.204+0000 7f91a0a65700 1 -- start start 2024-05-17T21:44:44.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:44.204+0000 7f91a0a65700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f919c102aa0 0x7f919c1130d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:44:44.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:44.204+0000 7f91a0a65700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f919c113610 con 0x7f919c102aa0 2024-05-17T21:44:44.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:44.204+0000 7f919a59c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f919c102aa0 0x7f919c1130d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:44:44.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:44.205+0000 7f919a59c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f919c102aa0 0x7f919c1130d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.138:50904/0 (socket says 172.21.15.138:50904) 2024-05-17T21:44:44.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:44.205+0000 7f919a59c700 1 -- 172.21.15.138:0/1215344346 learned_addr learned my addr 172.21.15.138:0/1215344346 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-17T21:44:44.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:44.205+0000 7f919a59c700 1 -- 172.21.15.138:0/1215344346 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f918c005040 con 0x7f919c102aa0 2024-05-17T21:44:44.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:44.205+0000 7f919a59c700 1 --2- 172.21.15.138:0/1215344346 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f919c102aa0 0x7f919c1130d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f918c00b820 tx=0x7f918c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:44:44.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:44.205+0000 7f918affd700 1 -- 172.21.15.138:0/1215344346 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f918c0093c0 con 0x7f919c102aa0 2024-05-17T21:44:44.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:44.205+0000 7f91a0a65700 1 -- 172.21.15.138:0/1215344346 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f919c113810 con 0x7f919c102aa0 2024-05-17T21:44:44.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:44.205+0000 7f918affd700 1 -- 172.21.15.138:0/1215344346 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f918c013420 con 0x7f919c102aa0 2024-05-17T21:44:44.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:44.205+0000 7f918affd700 1 -- 172.21.15.138:0/1215344346 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f918c01c5b0 con 0x7f919c102aa0 2024-05-17T21:44:44.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:44.205+0000 7f91a0a65700 1 -- 172.21.15.138:0/1215344346 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f919c113d00 con 0x7f919c102aa0 2024-05-17T21:44:44.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:44.206+0000 7f918affd700 1 -- 172.21.15.138:0/1215344346 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 43491+0+0 (secure 0 0 0) 0x7f918c01c7f0 con 0x7f919c102aa0 2024-05-17T21:44:44.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:44.206+0000 7f91a0a65700 1 -- 172.21.15.138:0/1215344346 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f919c0fe1a0 con 0x7f919c102aa0 2024-05-17T21:44:44.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:44.206+0000 7f918affd700 1 --2- 172.21.15.138:0/1215344346 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f9184036970 0x7f9184038e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:44:44.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:44.207+0000 7f918affd700 1 -- 172.21.15.138:0/1215344346 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f918c049a70 con 0x7f919c102aa0 2024-05-17T21:44:44.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:44.207+0000 7f9199d9b700 1 --2- 172.21.15.138:0/1215344346 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f9184036970 0x7f9184038e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:44:44.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:44.207+0000 7f9199d9b700 1 --2- 172.21.15.138:0/1215344346 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f9184036970 0x7f9184038e20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9190009990 tx=0x7f9190006e40 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:44:44.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:44.211+0000 7f918affd700 1 -- 172.21.15.138:0/1215344346 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f918c023300 con 0x7f919c102aa0 2024-05-17T21:44:44.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:44.532+0000 7f91a0a65700 1 -- 172.21.15.138:0/1215344346 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f919c05f5c0 con 0x7f919c102aa0 2024-05-17T21:44:44.527 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:44.532+0000 7f918affd700 1 -- 172.21.15.138:0/1215344346 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7f918c011b60 con 0x7f919c102aa0 2024-05-17T21:44:44.527 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-17T21:44:44.527 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":1,"fsid":"3722731a-1496-11ef-bc9a-c7b262605968","modified":"2024-05-17T21:42:10.472726Z","created":"2024-05-17T21:42:10.472726Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi107","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:3300","nonce":0},{"type":"v1","addr":"172.21.15.107:6789","nonce":0}]},"addr":"172.21.15.107:6789/0","public_addr":"172.21.15.107:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-17T21:44:44.528 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:44.534+0000 7f91a0a65700 1 -- 172.21.15.138:0/1215344346 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f9184036970 msgr2=0x7f9184038e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:44:44.528 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:44.534+0000 7f91a0a65700 1 --2- 172.21.15.138:0/1215344346 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f9184036970 0x7f9184038e20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9190009990 tx=0x7f9190006e40 comp rx=0 tx=0).stop 2024-05-17T21:44:44.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:44.534+0000 7f91a0a65700 1 -- 172.21.15.138:0/1215344346 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f919c102aa0 msgr2=0x7f919c1130d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:44:44.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:44.534+0000 7f91a0a65700 1 --2- 172.21.15.138:0/1215344346 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f919c102aa0 0x7f919c1130d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f918c00b820 tx=0x7f918c009e30 comp rx=0 tx=0).stop 2024-05-17T21:44:44.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:44.534+0000 7f91a0a65700 1 -- 172.21.15.138:0/1215344346 shutdown_connections 2024-05-17T21:44:44.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:44.534+0000 7f91a0a65700 1 --2- 172.21.15.138:0/1215344346 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f9184036970 0x7f9184038e20 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:44:44.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:44.534+0000 7f91a0a65700 1 --2- 172.21.15.138:0/1215344346 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f919c102aa0 0x7f919c1130d0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:44:44.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:44.534+0000 7f91a0a65700 1 -- 172.21.15.138:0/1215344346 >> 172.21.15.138:0/1215344346 conn(0x7f919c0f85c0 msgr2=0x7f919c0f9b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:44:44.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:44.535+0000 7f91a0a65700 1 -- 172.21.15.138:0/1215344346 shutdown_connections 2024-05-17T21:44:44.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:44.535+0000 7f91a0a65700 1 -- 172.21.15.138:0/1215344346 wait complete. 2024-05-17T21:44:44.529 INFO:teuthology.orchestra.run.smithi138.stderr:dumped monmap epoch 1 2024-05-17T21:44:45.104 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:44 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:44:45.104 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:44 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:44:45.104 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:44 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:44:45.104 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:44 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:44:45.104 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:44 smithi107 ceph-mon[28434]: from='client.? 172.21.15.138:0/1215344346' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-17T21:44:46.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:45 smithi107 ceph-mon[28434]: Deploying daemon node-exporter.smithi138 on smithi138 2024-05-17T21:44:46.107 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-17T21:44:46.107 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -- ceph mon dump -f json 2024-05-17T21:44:46.944 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-17T21:44:48.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:48 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:44:48.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:48 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:44:49.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:49 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:44:49.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:49 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:44:49.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:49 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:44:49.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:49 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:44:49.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:49 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi138.napnll", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-17T21:44:49.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:49 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.smithi138.napnll", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-17T21:44:49.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:49 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-17T21:44:49.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:49 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:44:49.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:49 smithi107 ceph-mon[28434]: Deploying daemon mgr.smithi138.napnll on smithi138 2024-05-17T21:44:50.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:50.354+0000 7fa747d16700 1 -- 172.21.15.138:0/3023748239 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa7401019a0 msgr2=0x7fa740101d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:44:50.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:50.354+0000 7fa747d16700 1 --2- 172.21.15.138:0/3023748239 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa7401019a0 0x7fa740101d70 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa73c0048c0 tx=0x7fa73c009e30 comp rx=0 tx=0).stop 2024-05-17T21:44:50.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:50.354+0000 7fa747d16700 1 -- 172.21.15.138:0/3023748239 shutdown_connections 2024-05-17T21:44:50.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:50.354+0000 7fa747d16700 1 --2- 172.21.15.138:0/3023748239 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa7401019a0 0x7fa740101d70 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:44:50.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:50.354+0000 7fa747d16700 1 -- 172.21.15.138:0/3023748239 >> 172.21.15.138:0/3023748239 conn(0x7fa7400f74a0 msgr2=0x7fa7400f98b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:44:50.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:50.354+0000 7fa747d16700 1 -- 172.21.15.138:0/3023748239 shutdown_connections 2024-05-17T21:44:50.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:50.354+0000 7fa747d16700 1 -- 172.21.15.138:0/3023748239 wait complete. 2024-05-17T21:44:50.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:50.354+0000 7fa747d16700 1 Processor -- start 2024-05-17T21:44:50.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:50.354+0000 7fa747d16700 1 -- start start 2024-05-17T21:44:50.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:50.355+0000 7fa747d16700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa7401019a0 0x7fa7401152b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:44:50.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:50.355+0000 7fa747d16700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7401157f0 con 0x7fa7401019a0 2024-05-17T21:44:50.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:50.355+0000 7fa745ab2700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa7401019a0 0x7fa7401152b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:44:50.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:50.355+0000 7fa745ab2700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa7401019a0 0x7fa7401152b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.138:50936/0 (socket says 172.21.15.138:50936) 2024-05-17T21:44:50.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:50.355+0000 7fa745ab2700 1 -- 172.21.15.138:0/2264737959 learned_addr learned my addr 172.21.15.138:0/2264737959 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-17T21:44:50.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:50.355+0000 7fa745ab2700 1 -- 172.21.15.138:0/2264737959 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa73c005040 con 0x7fa7401019a0 2024-05-17T21:44:50.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:50.356+0000 7fa745ab2700 1 --2- 172.21.15.138:0/2264737959 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa7401019a0 0x7fa7401152b0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa73c00b7f0 tx=0x7fa73c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:44:50.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:50.356+0000 7fa7367fc700 1 -- 172.21.15.138:0/2264737959 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa73c009370 con 0x7fa7401019a0 2024-05-17T21:44:50.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:50.356+0000 7fa7367fc700 1 -- 172.21.15.138:0/2264737959 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa73c013420 con 0x7fa7401019a0 2024-05-17T21:44:50.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:50.356+0000 7fa747d16700 1 -- 172.21.15.138:0/2264737959 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7401159f0 con 0x7fa7401019a0 2024-05-17T21:44:50.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:50.356+0000 7fa7367fc700 1 -- 172.21.15.138:0/2264737959 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa73c01c590 con 0x7fa7401019a0 2024-05-17T21:44:50.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:50.356+0000 7fa747d16700 1 -- 172.21.15.138:0/2264737959 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa740115ee0 con 0x7fa7401019a0 2024-05-17T21:44:50.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:50.357+0000 7fa747d16700 1 -- 172.21.15.138:0/2264737959 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7400fd0a0 con 0x7fa7401019a0 2024-05-17T21:44:50.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:50.357+0000 7fa7367fc700 1 -- 172.21.15.138:0/2264737959 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 43491+0+0 (secure 0 0 0) 0x7fa73c026440 con 0x7fa7401019a0 2024-05-17T21:44:50.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:50.357+0000 7fa7367fc700 1 --2- 172.21.15.138:0/2264737959 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fa72c0369c0 0x7fa72c038e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:44:50.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:50.357+0000 7fa7367fc700 1 -- 172.21.15.138:0/2264737959 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7fa73c0494b0 con 0x7fa7401019a0 2024-05-17T21:44:50.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:50.357+0000 7fa7452b1700 1 --2- 172.21.15.138:0/2264737959 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fa72c0369c0 0x7fa72c038e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:44:50.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:50.358+0000 7fa7452b1700 1 --2- 172.21.15.138:0/2264737959 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fa72c0369c0 0x7fa72c038e70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa730009990 tx=0x7fa730006e40 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:44:50.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:50.363+0000 7fa7367fc700 1 -- 172.21.15.138:0/2264737959 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fa73c011780 con 0x7fa7401019a0 2024-05-17T21:44:50.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:50.647+0000 7fa747d16700 1 -- 172.21.15.138:0/2264737959 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fa74005f5c0 con 0x7fa7401019a0 2024-05-17T21:44:50.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:50.647+0000 7fa7367fc700 1 -- 172.21.15.138:0/2264737959 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7fa73c021030 con 0x7fa7401019a0 2024-05-17T21:44:50.642 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-17T21:44:50.643 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":1,"fsid":"3722731a-1496-11ef-bc9a-c7b262605968","modified":"2024-05-17T21:42:10.472726Z","created":"2024-05-17T21:42:10.472726Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi107","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:3300","nonce":0},{"type":"v1","addr":"172.21.15.107:6789","nonce":0}]},"addr":"172.21.15.107:6789/0","public_addr":"172.21.15.107:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-17T21:44:50.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:50.649+0000 7fa747d16700 1 -- 172.21.15.138:0/2264737959 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fa72c0369c0 msgr2=0x7fa72c038e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:44:50.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:50.649+0000 7fa747d16700 1 --2- 172.21.15.138:0/2264737959 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fa72c0369c0 0x7fa72c038e70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa730009990 tx=0x7fa730006e40 comp rx=0 tx=0).stop 2024-05-17T21:44:50.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:50.649+0000 7fa747d16700 1 -- 172.21.15.138:0/2264737959 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa7401019a0 msgr2=0x7fa7401152b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:44:50.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:50.649+0000 7fa747d16700 1 --2- 172.21.15.138:0/2264737959 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa7401019a0 0x7fa7401152b0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa73c00b7f0 tx=0x7fa73c009e30 comp rx=0 tx=0).stop 2024-05-17T21:44:50.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:50.649+0000 7fa747d16700 1 -- 172.21.15.138:0/2264737959 shutdown_connections 2024-05-17T21:44:50.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:50.649+0000 7fa747d16700 1 --2- 172.21.15.138:0/2264737959 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fa72c0369c0 0x7fa72c038e70 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:44:50.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:50.649+0000 7fa747d16700 1 --2- 172.21.15.138:0/2264737959 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa7401019a0 0x7fa7401152b0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:44:50.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:50.649+0000 7fa747d16700 1 -- 172.21.15.138:0/2264737959 >> 172.21.15.138:0/2264737959 conn(0x7fa7400f74a0 msgr2=0x7fa740065d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:44:50.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:50.650+0000 7fa747d16700 1 -- 172.21.15.138:0/2264737959 shutdown_connections 2024-05-17T21:44:50.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:50.650+0000 7fa747d16700 1 -- 172.21.15.138:0/2264737959 wait complete. 2024-05-17T21:44:50.645 INFO:teuthology.orchestra.run.smithi138.stderr:dumped monmap epoch 1 2024-05-17T21:44:51.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:50 smithi107 ceph-mon[28434]: from='client.? 172.21.15.138:0/2264737959' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-17T21:44:52.388 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-17T21:44:52.389 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -- ceph mon dump -f json 2024-05-17T21:44:52.992 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-17T21:44:53.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:52 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:44:53.104 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:52 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:44:53.104 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:52 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:44:53.104 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:52 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:44:53.104 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:52 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-17T21:44:53.104 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:52 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:44:53.104 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:52 smithi107 ceph-mon[28434]: Deploying daemon mon.smithi138 on smithi138 2024-05-17T21:44:53.104 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:52 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:44:54.104 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:53 smithi107 ceph-mon[28434]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T21:44:55.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:55.606+0000 7f7f9124e700 1 -- 172.21.15.138:0/3143814433 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f8c0fda70 msgr2=0x7f7f8c0fde40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:44:55.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:55.606+0000 7f7f9124e700 1 --2- 172.21.15.138:0/3143814433 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f8c0fda70 0x7f7f8c0fde40 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c0048c0 tx=0x7f7f7c009e30 comp rx=0 tx=0).stop 2024-05-17T21:44:55.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:55.606+0000 7f7f9124e700 1 -- 172.21.15.138:0/3143814433 shutdown_connections 2024-05-17T21:44:55.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:55.606+0000 7f7f9124e700 1 --2- 172.21.15.138:0/3143814433 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f8c0fda70 0x7f7f8c0fde40 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:44:55.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:55.607+0000 7f7f9124e700 1 -- 172.21.15.138:0/3143814433 >> 172.21.15.138:0/3143814433 conn(0x7f7f8c071800 msgr2=0x7f7f8c071c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:44:55.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:55.607+0000 7f7f9124e700 1 -- 172.21.15.138:0/3143814433 shutdown_connections 2024-05-17T21:44:55.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:55.607+0000 7f7f9124e700 1 -- 172.21.15.138:0/3143814433 wait complete. 2024-05-17T21:44:55.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:55.608+0000 7f7f9124e700 1 Processor -- start 2024-05-17T21:44:55.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:55.608+0000 7f7f9124e700 1 -- start start 2024-05-17T21:44:55.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:55.608+0000 7f7f9124e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f8c0fda70 0x7f7f8c113190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:44:55.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:55.608+0000 7f7f9124e700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f8c1136d0 con 0x7f7f8c0fda70 2024-05-17T21:44:55.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:55.608+0000 7f7f8ad9d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f8c0fda70 0x7f7f8c113190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:44:55.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:55.608+0000 7f7f8ad9d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f8c0fda70 0x7f7f8c113190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.138:50906/0 (socket says 172.21.15.138:50906) 2024-05-17T21:44:55.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:55.608+0000 7f7f8ad9d700 1 -- 172.21.15.138:0/2650597986 learned_addr learned my addr 172.21.15.138:0/2650597986 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-17T21:44:55.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:55.609+0000 7f7f8ad9d700 1 -- 172.21.15.138:0/2650597986 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f7c005040 con 0x7f7f8c0fda70 2024-05-17T21:44:55.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:55.609+0000 7f7f8ad9d700 1 --2- 172.21.15.138:0/2650597986 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f8c0fda70 0x7f7f8c113190 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c00b820 tx=0x7f7f7c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:44:55.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:55.609+0000 7f7f737fe700 1 -- 172.21.15.138:0/2650597986 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7f7c0093c0 con 0x7f7f8c0fda70 2024-05-17T21:44:55.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:55.609+0000 7f7f737fe700 1 -- 172.21.15.138:0/2650597986 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7f7c013420 con 0x7f7f8c0fda70 2024-05-17T21:44:55.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:55.609+0000 7f7f9124e700 1 -- 172.21.15.138:0/2650597986 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f8c1138d0 con 0x7f7f8c0fda70 2024-05-17T21:44:55.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:55.609+0000 7f7f9124e700 1 -- 172.21.15.138:0/2650597986 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f8c113d40 con 0x7f7f8c0fda70 2024-05-17T21:44:55.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:55.609+0000 7f7f737fe700 1 -- 172.21.15.138:0/2650597986 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7f7c01c5b0 con 0x7f7f8c0fda70 2024-05-17T21:44:55.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:55.610+0000 7f7f737fe700 1 -- 172.21.15.138:0/2650597986 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 43491+0+0 (secure 0 0 0) 0x7f7f7c01c7f0 con 0x7f7f8c0fda70 2024-05-17T21:44:55.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:55.610+0000 7f7f737fe700 1 --2- 172.21.15.138:0/2650597986 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f7f74036970 0x7f7f74038e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:44:55.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:55.610+0000 7f7f737fe700 1 -- 172.21.15.138:0/2650597986 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f7f7c049e00 con 0x7f7f8c0fda70 2024-05-17T21:44:55.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:55.610+0000 7f7f9124e700 1 -- 172.21.15.138:0/2650597986 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f780052b0 con 0x7f7f8c0fda70 2024-05-17T21:44:55.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:55.610+0000 7f7f8a59c700 1 --2- 172.21.15.138:0/2650597986 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f7f74036970 0x7f7f74038e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:44:55.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:55.611+0000 7f7f8a59c700 1 --2- 172.21.15.138:0/2650597986 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f7f74036970 0x7f7f74038e20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7f80009990 tx=0x7f7f80006e40 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:44:55.609 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:55.616+0000 7f7f737fe700 1 -- 172.21.15.138:0/2650597986 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f7f7c011dd0 con 0x7f7f8c0fda70 2024-05-17T21:44:55.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:55.892+0000 7f7f9124e700 1 -- 172.21.15.138:0/2650597986 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f7f78005560 con 0x7f7f8c0fda70 2024-05-17T21:44:55.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:55.893+0000 7f7f737fe700 1 -- 172.21.15.138:0/2650597986 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7f7f7c011370 con 0x7f7f8c0fda70 2024-05-17T21:44:55.887 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-17T21:44:55.887 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":1,"fsid":"3722731a-1496-11ef-bc9a-c7b262605968","modified":"2024-05-17T21:42:10.472726Z","created":"2024-05-17T21:42:10.472726Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi107","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:3300","nonce":0},{"type":"v1","addr":"172.21.15.107:6789","nonce":0}]},"addr":"172.21.15.107:6789/0","public_addr":"172.21.15.107:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-17T21:44:55.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:55.895+0000 7f7f9124e700 1 -- 172.21.15.138:0/2650597986 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f7f74036970 msgr2=0x7f7f74038e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:44:55.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:55.895+0000 7f7f9124e700 1 --2- 172.21.15.138:0/2650597986 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f7f74036970 0x7f7f74038e20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7f80009990 tx=0x7f7f80006e40 comp rx=0 tx=0).stop 2024-05-17T21:44:55.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:55.895+0000 7f7f9124e700 1 -- 172.21.15.138:0/2650597986 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f8c0fda70 msgr2=0x7f7f8c113190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:44:55.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:55.895+0000 7f7f9124e700 1 --2- 172.21.15.138:0/2650597986 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f8c0fda70 0x7f7f8c113190 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c00b820 tx=0x7f7f7c009e30 comp rx=0 tx=0).stop 2024-05-17T21:44:55.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:55.895+0000 7f7f9124e700 1 -- 172.21.15.138:0/2650597986 shutdown_connections 2024-05-17T21:44:55.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:55.895+0000 7f7f9124e700 1 --2- 172.21.15.138:0/2650597986 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f7f74036970 0x7f7f74038e20 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:44:55.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:55.895+0000 7f7f9124e700 1 --2- 172.21.15.138:0/2650597986 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f8c0fda70 0x7f7f8c113190 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:44:55.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:55.895+0000 7f7f9124e700 1 -- 172.21.15.138:0/2650597986 >> 172.21.15.138:0/2650597986 conn(0x7f7f8c071800 msgr2=0x7f7f8c103640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:44:55.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:55.895+0000 7f7f9124e700 1 -- 172.21.15.138:0/2650597986 shutdown_connections 2024-05-17T21:44:55.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:44:55.895+0000 7f7f9124e700 1 -- 172.21.15.138:0/2650597986 wait complete. 2024-05-17T21:44:55.890 INFO:teuthology.orchestra.run.smithi138.stderr:dumped monmap epoch 1 2024-05-17T21:44:56.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:55 smithi107 ceph-mon[28434]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T21:44:56.932 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:44:56 smithi138 ceph-mon[35948]: mon.smithi138@-1(synchronizing).paxosservice(auth 1..6) refresh upgraded, format 0 -> 3 2024-05-17T21:44:56.933 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:44:56 smithi138 bash[35885]: ee61a065778d248677a0f5a17cdd2e879d3fc7b25128d9700cc17c88f16c00c2 2024-05-17T21:44:57.005 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:44:56 smithi107 ceph-mon[28434]: from='client.? 172.21.15.138:0/2650597986' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-17T21:44:57.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:44:56 smithi138 systemd[1]: Started Ceph mon.smithi138 for 3722731a-1496-11ef-bc9a-c7b262605968. 2024-05-17T21:44:57.690 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-17T21:44:57.690 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -- ceph mon dump -f json 2024-05-17T21:44:57.942 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/mon.smithi138/config 2024-05-17T21:45:02.328 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:01 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mon metadata", "id": "smithi107"}]: dispatch 2024-05-17T21:45:02.328 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:01 smithi138 ceph-mon[35948]: mon.smithi107 calling monitor election 2024-05-17T21:45:02.328 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:01 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mon metadata", "id": "smithi138"}]: dispatch 2024-05-17T21:45:02.328 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:01 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mon metadata", "id": "smithi138"}]: dispatch 2024-05-17T21:45:02.328 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:01 smithi138 ceph-mon[35948]: from='mgr.? 172.21.15.138:0/2928095634' entity='mgr.smithi138.napnll' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi138.napnll/crt"}]: dispatch 2024-05-17T21:45:02.329 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:01 smithi138 ceph-mon[35948]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T21:45:02.329 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:01 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mon metadata", "id": "smithi138"}]: dispatch 2024-05-17T21:45:02.329 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:01 smithi138 ceph-mon[35948]: mon.smithi138 calling monitor election 2024-05-17T21:45:02.329 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:01 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mon metadata", "id": "smithi138"}]: dispatch 2024-05-17T21:45:02.329 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:01 smithi138 ceph-mon[35948]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T21:45:02.329 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:01 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mon metadata", "id": "smithi138"}]: dispatch 2024-05-17T21:45:02.329 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:01 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mon metadata", "id": "smithi138"}]: dispatch 2024-05-17T21:45:02.329 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:01 smithi138 ceph-mon[35948]: mon.smithi107 is new leader, mons smithi107,smithi138 in quorum (ranks 0,1) 2024-05-17T21:45:02.329 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:01 smithi138 ceph-mon[35948]: monmap e2: 2 mons at {smithi107=[v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0],smithi138=[v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-17T21:45:02.329 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:01 smithi138 ceph-mon[35948]: fsmap 2024-05-17T21:45:02.329 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:01 smithi138 ceph-mon[35948]: osdmap e5: 0 total, 0 up, 0 in 2024-05-17T21:45:02.329 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:01 smithi138 ceph-mon[35948]: mgrmap e17: smithi107.xhhyiw(active, since 29s) 2024-05-17T21:45:02.329 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:01 smithi138 ceph-mon[35948]: overall HEALTH_OK 2024-05-17T21:45:02.329 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:01 smithi138 ceph-mon[35948]: Standby manager daemon smithi138.napnll started 2024-05-17T21:45:02.330 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:01 smithi138 ceph-mon[35948]: from='mgr.? 172.21.15.138:0/2928095634' entity='mgr.smithi138.napnll' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-17T21:45:02.330 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:01 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:45:02.330 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:01 smithi138 ceph-mon[35948]: from='mgr.? 172.21.15.138:0/2928095634' entity='mgr.smithi138.napnll' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi138.napnll/key"}]: dispatch 2024-05-17T21:45:02.330 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:01 smithi138 ceph-mon[35948]: from='mgr.? 172.21.15.138:0/2928095634' entity='mgr.smithi138.napnll' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-17T21:45:02.330 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:01 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:45:02.330 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:01 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:45:02.330 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:01 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:45:02.330 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:01 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:45:02.354 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:01 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mon metadata", "id": "smithi107"}]: dispatch 2024-05-17T21:45:02.354 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:01 smithi107 ceph-mon[28434]: mon.smithi107 calling monitor election 2024-05-17T21:45:02.354 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:01 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mon metadata", "id": "smithi138"}]: dispatch 2024-05-17T21:45:02.354 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:01 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mon metadata", "id": "smithi138"}]: dispatch 2024-05-17T21:45:02.354 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:01 smithi107 ceph-mon[28434]: from='mgr.? 172.21.15.138:0/2928095634' entity='mgr.smithi138.napnll' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi138.napnll/crt"}]: dispatch 2024-05-17T21:45:02.355 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:01 smithi107 ceph-mon[28434]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T21:45:02.355 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:01 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mon metadata", "id": "smithi138"}]: dispatch 2024-05-17T21:45:02.355 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:01 smithi107 ceph-mon[28434]: mon.smithi138 calling monitor election 2024-05-17T21:45:02.355 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:01 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mon metadata", "id": "smithi138"}]: dispatch 2024-05-17T21:45:02.355 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:01 smithi107 ceph-mon[28434]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T21:45:02.355 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:01 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mon metadata", "id": "smithi138"}]: dispatch 2024-05-17T21:45:02.355 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:01 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mon metadata", "id": "smithi138"}]: dispatch 2024-05-17T21:45:02.355 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:01 smithi107 ceph-mon[28434]: mon.smithi107 is new leader, mons smithi107,smithi138 in quorum (ranks 0,1) 2024-05-17T21:45:02.355 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:01 smithi107 ceph-mon[28434]: monmap e2: 2 mons at {smithi107=[v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0],smithi138=[v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-17T21:45:02.355 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:01 smithi107 ceph-mon[28434]: fsmap 2024-05-17T21:45:02.355 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:01 smithi107 ceph-mon[28434]: osdmap e5: 0 total, 0 up, 0 in 2024-05-17T21:45:02.355 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:01 smithi107 ceph-mon[28434]: mgrmap e17: smithi107.xhhyiw(active, since 29s) 2024-05-17T21:45:02.355 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:01 smithi107 ceph-mon[28434]: overall HEALTH_OK 2024-05-17T21:45:02.355 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:01 smithi107 ceph-mon[28434]: Standby manager daemon smithi138.napnll started 2024-05-17T21:45:02.356 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:01 smithi107 ceph-mon[28434]: from='mgr.? 172.21.15.138:0/2928095634' entity='mgr.smithi138.napnll' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-17T21:45:02.356 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:01 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:45:02.356 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:01 smithi107 ceph-mon[28434]: from='mgr.? 172.21.15.138:0/2928095634' entity='mgr.smithi138.napnll' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi138.napnll/key"}]: dispatch 2024-05-17T21:45:02.356 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:01 smithi107 ceph-mon[28434]: from='mgr.? 172.21.15.138:0/2928095634' entity='mgr.smithi138.napnll' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-17T21:45:02.356 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:01 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:45:02.356 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:01 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:45:02.356 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:01 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:45:02.356 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:01 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:45:02.662 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:45:02.668+0000 7fb42a78b700 1 -- 172.21.15.138:0/62926680 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb40c003580 msgr2=0x7fb40c005a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:45:02.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:45:02.668+0000 7fb42a78b700 1 --2- 172.21.15.138:0/62926680 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb40c003580 0x7fb40c005a00 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb4240fcac0 tx=0x7fb41400b040 comp rx=0 tx=0).stop 2024-05-17T21:45:02.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:45:02.668+0000 7fb42a78b700 1 -- 172.21.15.138:0/62926680 shutdown_connections 2024-05-17T21:45:02.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:45:02.668+0000 7fb42a78b700 1 --2- 172.21.15.138:0/62926680 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb40c003580 0x7fb40c005a00 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:45:02.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:45:02.668+0000 7fb42a78b700 1 -- 172.21.15.138:0/62926680 >> 172.21.15.138:0/62926680 conn(0x7fb4240f86f0 msgr2=0x7fb4240fab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:45:02.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:45:02.668+0000 7fb42a78b700 1 -- 172.21.15.138:0/62926680 shutdown_connections 2024-05-17T21:45:02.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:45:02.668+0000 7fb42a78b700 1 -- 172.21.15.138:0/62926680 wait complete. 2024-05-17T21:45:02.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:45:02.669+0000 7fb42a78b700 1 Processor -- start 2024-05-17T21:45:02.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:45:02.669+0000 7fb42a78b700 1 -- start start 2024-05-17T21:45:02.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:45:02.669+0000 7fb42a78b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb40c003580 0x7fb4241132a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:45:02.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:45:02.669+0000 7fb42a78b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4241137e0 0x7fb42410d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:45:02.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:45:02.669+0000 7fb42a78b700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb424113df0 con 0x7fb40c003580 2024-05-17T21:45:02.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:45:02.669+0000 7fb42a78b700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb42410d740 con 0x7fb4241137e0 2024-05-17T21:45:02.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:45:02.669+0000 7fb4237fe700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4241137e0 0x7fb42410d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:45:02.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:45:02.669+0000 7fb4237fe700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4241137e0 0x7fb42410d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:44260/0 (socket says 172.21.15.138:44260) 2024-05-17T21:45:02.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:45:02.669+0000 7fb4237fe700 1 -- 172.21.15.138:0/3302909623 learned_addr learned my addr 172.21.15.138:0/3302909623 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-17T21:45:02.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:45:02.670+0000 7fb423fff700 1 --2- 172.21.15.138:0/3302909623 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb40c003580 0x7fb4241132a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:45:02.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:45:02.670+0000 7fb4237fe700 1 -- 172.21.15.138:0/3302909623 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4241137e0 msgr2=0x7fb42410d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-05-17T21:45:02.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:45:02.670+0000 7fb4237fe700 1 -- 172.21.15.138:0/3302909623 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4241137e0 msgr2=0x7fb42410d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-17T21:45:02.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:45:02.670+0000 7fb4237fe700 1 --2- 172.21.15.138:0/3302909623 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4241137e0 0x7fb42410d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-05-17T21:45:02.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:45:02.670+0000 7fb4237fe700 1 --2- 172.21.15.138:0/3302909623 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4241137e0 0x7fb42410d200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-17T21:45:02.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:45:02.670+0000 7fb423fff700 1 -- 172.21.15.138:0/3302909623 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4241137e0 msgr2=0x7fb42410d200 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-17T21:45:02.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:45:02.670+0000 7fb423fff700 1 --2- 172.21.15.138:0/3302909623 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4241137e0 0x7fb42410d200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:45:02.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:45:02.670+0000 7fb423fff700 1 -- 172.21.15.138:0/3302909623 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb414005040 con 0x7fb40c003580 2024-05-17T21:45:02.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:45:02.670+0000 7fb423fff700 1 --2- 172.21.15.138:0/3302909623 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb40c003580 0x7fb4241132a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb414000c00 tx=0x7fb41400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:45:02.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:45:02.670+0000 7fb420ff9700 1 -- 172.21.15.138:0/3302909623 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb41401b070 con 0x7fb40c003580 2024-05-17T21:45:02.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:45:02.670+0000 7fb420ff9700 1 -- 172.21.15.138:0/3302909623 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb414016cf0 con 0x7fb40c003580 2024-05-17T21:45:02.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:45:02.670+0000 7fb42a78b700 1 -- 172.21.15.138:0/3302909623 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb42410d9c0 con 0x7fb40c003580 2024-05-17T21:45:02.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:45:02.670+0000 7fb420ff9700 1 -- 172.21.15.138:0/3302909623 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb41401f6b0 con 0x7fb40c003580 2024-05-17T21:45:02.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:45:02.670+0000 7fb42a78b700 1 -- 172.21.15.138:0/3302909623 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb42410df00 con 0x7fb40c003580 2024-05-17T21:45:02.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:45:02.671+0000 7fb42a78b700 1 -- 172.21.15.138:0/3302909623 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4240fe2d0 con 0x7fb40c003580 2024-05-17T21:45:02.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:45:02.672+0000 7fb420ff9700 1 -- 172.21.15.138:0/3302909623 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 86575+0+0 (secure 0 0 0) 0x7fb414020440 con 0x7fb40c003580 2024-05-17T21:45:02.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:45:02.672+0000 7fb420ff9700 1 --2- 172.21.15.138:0/3302909623 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fb410069320 0x7fb41006b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:45:02.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:45:02.672+0000 7fb420ff9700 1 -- 172.21.15.138:0/3302909623 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7fb414087e20 con 0x7fb40c003580 2024-05-17T21:45:02.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:45:02.672+0000 7fb4237fe700 1 --2- 172.21.15.138:0/3302909623 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fb410069320 0x7fb41006b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:45:02.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:45:02.673+0000 7fb4237fe700 1 --2- 172.21.15.138:0/3302909623 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fb410069320 0x7fb41006b7d0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb418006fd0 tx=0x7fb418008040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:45:02.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:45:02.677+0000 7fb420ff9700 1 -- 172.21.15.138:0/3302909623 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fb414058c40 con 0x7fb40c003580 2024-05-17T21:45:02.939 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:45:02.945+0000 7fb42a78b700 1 -- 172.21.15.138:0/3302909623 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fb42405f5c0 con 0x7fb40c003580 2024-05-17T21:45:02.939 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:45:02.946+0000 7fb420ff9700 1 -- 172.21.15.138:0/3302909623 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1021 (secure 0 0 0) 0x7fb414024410 con 0x7fb40c003580 2024-05-17T21:45:02.940 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-17T21:45:02.940 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":2,"fsid":"3722731a-1496-11ef-bc9a-c7b262605968","modified":"2024-05-17T21:44:56.895553Z","created":"2024-05-17T21:42:10.472726Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi107","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:3300","nonce":0},{"type":"v1","addr":"172.21.15.107:6789","nonce":0}]},"addr":"172.21.15.107:6789/0","public_addr":"172.21.15.107:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"smithi138","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:3300","nonce":0},{"type":"v1","addr":"172.21.15.138:6789","nonce":0}]},"addr":"172.21.15.138:6789/0","public_addr":"172.21.15.138:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-05-17T21:45:02.942 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:45:02.948+0000 7fb42a78b700 1 -- 172.21.15.138:0/3302909623 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fb410069320 msgr2=0x7fb41006b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:45:02.942 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:45:02.948+0000 7fb42a78b700 1 --2- 172.21.15.138:0/3302909623 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fb410069320 0x7fb41006b7d0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb418006fd0 tx=0x7fb418008040 comp rx=0 tx=0).stop 2024-05-17T21:45:02.942 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:45:02.948+0000 7fb42a78b700 1 -- 172.21.15.138:0/3302909623 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb40c003580 msgr2=0x7fb4241132a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:45:02.942 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:45:02.948+0000 7fb42a78b700 1 --2- 172.21.15.138:0/3302909623 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb40c003580 0x7fb4241132a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb414000c00 tx=0x7fb41400b040 comp rx=0 tx=0).stop 2024-05-17T21:45:02.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:45:02.948+0000 7fb42a78b700 1 -- 172.21.15.138:0/3302909623 shutdown_connections 2024-05-17T21:45:02.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:45:02.948+0000 7fb42a78b700 1 --2- 172.21.15.138:0/3302909623 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fb410069320 0x7fb41006b7d0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:45:02.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:45:02.948+0000 7fb42a78b700 1 --2- 172.21.15.138:0/3302909623 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb40c003580 0x7fb4241132a0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:45:02.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:45:02.948+0000 7fb42a78b700 1 --2- 172.21.15.138:0/3302909623 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4241137e0 0x7fb42410d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:45:02.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:45:02.948+0000 7fb42a78b700 1 -- 172.21.15.138:0/3302909623 >> 172.21.15.138:0/3302909623 conn(0x7fb4240f86f0 msgr2=0x7fb4241060d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:45:02.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:45:02.948+0000 7fb42a78b700 1 -- 172.21.15.138:0/3302909623 shutdown_connections 2024-05-17T21:45:02.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:45:02.948+0000 7fb42a78b700 1 -- 172.21.15.138:0/3302909623 wait complete. 2024-05-17T21:45:02.943 INFO:teuthology.orchestra.run.smithi138.stderr:dumped monmap epoch 2 2024-05-17T21:45:03.232 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:02 smithi138 ceph-mon[35948]: mgrmap e18: smithi107.xhhyiw(active, since 29s), standbys: smithi138.napnll 2024-05-17T21:45:03.232 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:02 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mgr metadata", "who": "smithi138.napnll", "id": "smithi138.napnll"}]: dispatch 2024-05-17T21:45:03.232 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:02 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:45:03.232 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:02 smithi138 ceph-mon[35948]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T21:45:03.232 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:02 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:45:03.232 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:02 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:45:03.232 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:02 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mon metadata", "id": "smithi138"}]: dispatch 2024-05-17T21:45:03.232 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:02 smithi138 ceph-mon[35948]: from='client.? 172.21.15.138:0/3302909623' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-17T21:45:03.353 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:02 smithi107 ceph-mon[28434]: mgrmap e18: smithi107.xhhyiw(active, since 29s), standbys: smithi138.napnll 2024-05-17T21:45:03.354 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:02 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mgr metadata", "who": "smithi138.napnll", "id": "smithi138.napnll"}]: dispatch 2024-05-17T21:45:03.354 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:02 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:45:03.354 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:02 smithi107 ceph-mon[28434]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T21:45:03.354 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:02 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:45:03.354 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:02 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:45:03.354 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:02 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mon metadata", "id": "smithi138"}]: dispatch 2024-05-17T21:45:03.354 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:02 smithi107 ceph-mon[28434]: from='client.? 172.21.15.138:0/3302909623' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-17T21:45:03.621 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-05-17T21:45:03.621 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -- ceph config generate-minimal-conf 2024-05-17T21:45:03.883 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/mon.smithi107/config 2024-05-17T21:45:05.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:05.529+0000 7f4b13837700 1 -- 172.21.15.107:0/2686927141 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b0c0fda70 msgr2=0x7f4b0c0fde40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:45:05.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:05.529+0000 7f4b13837700 1 --2- 172.21.15.107:0/2686927141 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b0c0fda70 0x7f4b0c0fde40 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4b080048c0 tx=0x7f4b08009e30 comp rx=0 tx=0).stop 2024-05-17T21:45:05.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:05.529+0000 7f4b13837700 1 -- 172.21.15.107:0/2686927141 shutdown_connections 2024-05-17T21:45:05.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:05.529+0000 7f4b13837700 1 --2- 172.21.15.107:0/2686927141 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b0c0fda70 0x7f4b0c0fde40 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:45:05.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:05.529+0000 7f4b13837700 1 -- 172.21.15.107:0/2686927141 >> 172.21.15.107:0/2686927141 conn(0x7f4b0c071800 msgr2=0x7f4b0c071c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:45:05.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:05.529+0000 7f4b13837700 1 -- 172.21.15.107:0/2686927141 shutdown_connections 2024-05-17T21:45:05.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:05.529+0000 7f4b13837700 1 -- 172.21.15.107:0/2686927141 wait complete. 2024-05-17T21:45:05.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:05.530+0000 7f4b13837700 1 Processor -- start 2024-05-17T21:45:05.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:05.530+0000 7f4b13837700 1 -- start start 2024-05-17T21:45:05.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:05.530+0000 7f4b13837700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b0c0fda70 0x7f4b0c1132a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:45:05.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:05.530+0000 7f4b13837700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b0c1137e0 0x7f4b0c10d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:45:05.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:05.530+0000 7f4b13837700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b0c113df0 con 0x7f4b0c1137e0 2024-05-17T21:45:05.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:05.530+0000 7f4b13837700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b0c10d740 con 0x7f4b0c0fda70 2024-05-17T21:45:05.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:05.530+0000 7f4b10dd2700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b0c1137e0 0x7f4b0c10d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:45:05.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:05.531+0000 7f4b10dd2700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b0c1137e0 0x7f4b0c10d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54790/0 (socket says 172.21.15.107:54790) 2024-05-17T21:45:05.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:05.531+0000 7f4b10dd2700 1 -- 172.21.15.107:0/3342026884 learned_addr learned my addr 172.21.15.107:0/3342026884 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:45:05.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:05.531+0000 7f4b115d3700 1 --2- 172.21.15.107:0/3342026884 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b0c0fda70 0x7f4b0c1132a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:45:05.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:05.531+0000 7f4b10dd2700 1 -- 172.21.15.107:0/3342026884 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b0c0fda70 msgr2=0x7f4b0c1132a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:45:05.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:05.531+0000 7f4b10dd2700 1 --2- 172.21.15.107:0/3342026884 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b0c0fda70 0x7f4b0c1132a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:45:05.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:05.531+0000 7f4b10dd2700 1 -- 172.21.15.107:0/3342026884 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b08005040 con 0x7f4b0c1137e0 2024-05-17T21:45:05.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:05.531+0000 7f4b10dd2700 1 --2- 172.21.15.107:0/3342026884 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b0c1137e0 0x7f4b0c10d200 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4afc00b790 tx=0x7f4afc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:45:05.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:05.531+0000 7f4b01ffb700 1 -- 172.21.15.107:0/3342026884 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4afc010870 con 0x7f4b0c1137e0 2024-05-17T21:45:05.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:05.531+0000 7f4b01ffb700 1 -- 172.21.15.107:0/3342026884 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4afc00f3e0 con 0x7f4b0c1137e0 2024-05-17T21:45:05.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:05.531+0000 7f4b01ffb700 1 -- 172.21.15.107:0/3342026884 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4afc010ea0 con 0x7f4b0c1137e0 2024-05-17T21:45:05.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:05.531+0000 7f4b13837700 1 -- 172.21.15.107:0/3342026884 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b0c10da20 con 0x7f4b0c1137e0 2024-05-17T21:45:05.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:05.531+0000 7f4b13837700 1 -- 172.21.15.107:0/3342026884 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b0c10df60 con 0x7f4b0c1137e0 2024-05-17T21:45:05.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:05.532+0000 7f4b13837700 1 -- 172.21.15.107:0/3342026884 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b0c102710 con 0x7f4b0c1137e0 2024-05-17T21:45:05.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:05.533+0000 7f4b01ffb700 1 -- 172.21.15.107:0/3342026884 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 86575+0+0 (secure 0 0 0) 0x7f4afc00f550 con 0x7f4b0c1137e0 2024-05-17T21:45:05.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:05.533+0000 7f4b01ffb700 1 --2- 172.21.15.107:0/3342026884 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f4af8069370 0x7f4af806b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:45:05.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:05.533+0000 7f4b01ffb700 1 -- 172.21.15.107:0/3342026884 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f4afc0865b0 con 0x7f4b0c1137e0 2024-05-17T21:45:05.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:05.533+0000 7f4b115d3700 1 --2- 172.21.15.107:0/3342026884 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f4af8069370 0x7f4af806b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:45:05.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:05.534+0000 7f4b115d3700 1 --2- 172.21.15.107:0/3342026884 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f4af8069370 0x7f4af806b820 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f4b080043d0 tx=0x7f4b08009e30 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:45:05.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:05.537+0000 7f4b01ffb700 1 -- 172.21.15.107:0/3342026884 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f4afc057450 con 0x7f4b0c1137e0 2024-05-17T21:45:05.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:05 smithi107 ceph-mon[28434]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T21:45:05.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:05 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:45:05.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:05 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:45:05.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:05 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:45:05.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:05 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T21:45:05.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:05 smithi107 ceph-mon[28434]: Updating smithi107:/etc/ceph/ceph.conf 2024-05-17T21:45:05.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:05 smithi107 ceph-mon[28434]: Updating smithi138:/etc/ceph/ceph.conf 2024-05-17T21:45:05.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:05 smithi107 ceph-mon[28434]: Updating smithi107:/var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-17T21:45:05.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:05 smithi107 ceph-mon[28434]: Updating smithi138:/var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-17T21:45:05.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:05 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:45:05.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:05 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:45:05.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:05 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:45:05.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:05 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:45:05.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:05 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:45:05.605 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:05 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-17T21:45:05.605 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:05 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-17T21:45:05.605 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:05 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:45:05.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:05.729+0000 7f4b13837700 1 -- 172.21.15.107:0/3342026884 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f4b0c10e8f0 con 0x7f4b0c1137e0 2024-05-17T21:45:05.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:05.730+0000 7f4b01ffb700 1 -- 172.21.15.107:0/3342026884 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v10) v1 ==== 76+0+227 (secure 0 0 0) 0x7f4afc014100 con 0x7f4b0c1137e0 2024-05-17T21:45:05.723 INFO:teuthology.orchestra.run.smithi107.stdout:# minimal ceph.conf for 3722731a-1496-11ef-bc9a-c7b262605968 2024-05-17T21:45:05.723 INFO:teuthology.orchestra.run.smithi107.stdout:[global] 2024-05-17T21:45:05.723 INFO:teuthology.orchestra.run.smithi107.stdout: fsid = 3722731a-1496-11ef-bc9a-c7b262605968 2024-05-17T21:45:05.723 INFO:teuthology.orchestra.run.smithi107.stdout: mon_host = [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] 2024-05-17T21:45:05.725 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:05.732+0000 7f4b13837700 1 -- 172.21.15.107:0/3342026884 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f4af8069370 msgr2=0x7f4af806b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:45:05.725 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:05.732+0000 7f4b13837700 1 --2- 172.21.15.107:0/3342026884 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f4af8069370 0x7f4af806b820 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f4b080043d0 tx=0x7f4b08009e30 comp rx=0 tx=0).stop 2024-05-17T21:45:05.725 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:05.732+0000 7f4b13837700 1 -- 172.21.15.107:0/3342026884 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b0c1137e0 msgr2=0x7f4b0c10d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:45:05.725 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:05.732+0000 7f4b13837700 1 --2- 172.21.15.107:0/3342026884 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b0c1137e0 0x7f4b0c10d200 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4afc00b790 tx=0x7f4afc009250 comp rx=0 tx=0).stop 2024-05-17T21:45:05.725 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:05.732+0000 7f4b13837700 1 -- 172.21.15.107:0/3342026884 shutdown_connections 2024-05-17T21:45:05.725 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:05.732+0000 7f4b13837700 1 --2- 172.21.15.107:0/3342026884 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f4af8069370 0x7f4af806b820 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:45:05.725 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:05.732+0000 7f4b13837700 1 --2- 172.21.15.107:0/3342026884 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b0c0fda70 0x7f4b0c1132a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:45:05.725 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:05.732+0000 7f4b13837700 1 --2- 172.21.15.107:0/3342026884 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b0c1137e0 0x7f4b0c10d200 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:45:05.725 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:05.732+0000 7f4b13837700 1 -- 172.21.15.107:0/3342026884 >> 172.21.15.107:0/3342026884 conn(0x7f4b0c071800 msgr2=0x7f4b0c0fa860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:45:05.725 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:05.732+0000 7f4b13837700 1 -- 172.21.15.107:0/3342026884 shutdown_connections 2024-05-17T21:45:05.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:05.732+0000 7f4b13837700 1 -- 172.21.15.107:0/3342026884 wait complete. 2024-05-17T21:45:05.828 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:05 smithi138 ceph-mon[35948]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T21:45:05.828 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:05 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:45:05.828 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:05 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:45:05.828 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:05 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:45:05.828 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:05 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T21:45:05.828 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:05 smithi138 ceph-mon[35948]: Updating smithi107:/etc/ceph/ceph.conf 2024-05-17T21:45:05.828 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:05 smithi138 ceph-mon[35948]: Updating smithi138:/etc/ceph/ceph.conf 2024-05-17T21:45:05.828 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:05 smithi138 ceph-mon[35948]: Updating smithi107:/var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-17T21:45:05.828 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:05 smithi138 ceph-mon[35948]: Updating smithi138:/var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-17T21:45:05.828 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:05 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:45:05.829 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:05 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:45:05.829 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:05 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:45:05.829 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:05 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:45:05.829 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:05 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:45:05.829 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:05 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-17T21:45:05.829 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:05 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-17T21:45:05.829 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:05 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:45:06.828 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:06 smithi138 ceph-mon[35948]: Reconfiguring mon.smithi107 (unknown last config time)... 2024-05-17T21:45:06.828 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:06 smithi138 ceph-mon[35948]: Reconfiguring daemon mon.smithi107 on smithi107 2024-05-17T21:45:06.828 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:06 smithi138 ceph-mon[35948]: from='client.? 172.21.15.107:0/3342026884' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:45:06.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:06 smithi107 ceph-mon[28434]: Reconfiguring mon.smithi107 (unknown last config time)... 2024-05-17T21:45:06.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:06 smithi107 ceph-mon[28434]: Reconfiguring daemon mon.smithi107 on smithi107 2024-05-17T21:45:06.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:06 smithi107 ceph-mon[28434]: from='client.? 172.21.15.107:0/3342026884' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:45:07.201 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-05-17T21:45:07.202 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-05-17T21:45:07.202 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/ceph/ceph.conf 2024-05-17T21:45:07.229 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-05-17T21:45:07.229 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-17T21:45:07.295 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-17T21:45:07.295 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/etc/ceph/ceph.conf 2024-05-17T21:45:07.327 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-17T21:45:07.328 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-17T21:45:07.397 INFO:tasks.cephadm:Deploying OSDs... 2024-05-17T21:45:07.397 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-05-17T21:45:07.397 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/scratch_devs of=/dev/stdout 2024-05-17T21:45:07.412 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-17T21:45:07.412 DEBUG:teuthology.orchestra.run.smithi107:> stat /dev/vg_nvme/lv_1 2024-05-17T21:45:07.468 INFO:teuthology.orchestra.run.smithi107.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-17T21:45:07.468 INFO:teuthology.orchestra.run.smithi107.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-17T21:45:07.468 INFO:teuthology.orchestra.run.smithi107.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-05-17T21:45:07.468 INFO:teuthology.orchestra.run.smithi107.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-17T21:45:07.468 INFO:teuthology.orchestra.run.smithi107.stdout:Context: system_u:object_r:device_t:s0 2024-05-17T21:45:07.468 INFO:teuthology.orchestra.run.smithi107.stdout:Access: 2024-05-17 21:44:20.593784716 +0000 2024-05-17T21:45:07.468 INFO:teuthology.orchestra.run.smithi107.stdout:Modify: 2024-05-17 21:38:36.460237931 +0000 2024-05-17T21:45:07.468 INFO:teuthology.orchestra.run.smithi107.stdout:Change: 2024-05-17 21:38:36.460237931 +0000 2024-05-17T21:45:07.468 INFO:teuthology.orchestra.run.smithi107.stdout: Birth: 2024-05-17 21:38:36.460237931 +0000 2024-05-17T21:45:07.468 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-17T21:45:07.532 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records in 2024-05-17T21:45:07.532 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records out 2024-05-17T21:45:07.532 INFO:teuthology.orchestra.run.smithi107.stderr:512 bytes copied, 0.000189867 s, 2.7 MB/s 2024-05-17T21:45:07.533 DEBUG:teuthology.orchestra.run.smithi107:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-17T21:45:07.588 DEBUG:teuthology.orchestra.run.smithi107:> stat /dev/vg_nvme/lv_2 2024-05-17T21:45:07.644 INFO:teuthology.orchestra.run.smithi107.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-17T21:45:07.644 INFO:teuthology.orchestra.run.smithi107.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-17T21:45:07.644 INFO:teuthology.orchestra.run.smithi107.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-05-17T21:45:07.645 INFO:teuthology.orchestra.run.smithi107.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-17T21:45:07.645 INFO:teuthology.orchestra.run.smithi107.stdout:Context: system_u:object_r:device_t:s0 2024-05-17T21:45:07.645 INFO:teuthology.orchestra.run.smithi107.stdout:Access: 2024-05-17 21:44:20.593784716 +0000 2024-05-17T21:45:07.645 INFO:teuthology.orchestra.run.smithi107.stdout:Modify: 2024-05-17 21:38:36.818231893 +0000 2024-05-17T21:45:07.645 INFO:teuthology.orchestra.run.smithi107.stdout:Change: 2024-05-17 21:38:36.818231893 +0000 2024-05-17T21:45:07.645 INFO:teuthology.orchestra.run.smithi107.stdout: Birth: 2024-05-17 21:38:36.818231893 +0000 2024-05-17T21:45:07.645 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-17T21:45:07.708 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records in 2024-05-17T21:45:07.708 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records out 2024-05-17T21:45:07.708 INFO:teuthology.orchestra.run.smithi107.stderr:512 bytes copied, 0.000225327 s, 2.3 MB/s 2024-05-17T21:45:07.710 DEBUG:teuthology.orchestra.run.smithi107:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-17T21:45:07.766 DEBUG:teuthology.orchestra.run.smithi107:> stat /dev/vg_nvme/lv_3 2024-05-17T21:45:07.821 INFO:teuthology.orchestra.run.smithi107.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-17T21:45:07.821 INFO:teuthology.orchestra.run.smithi107.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-17T21:45:07.821 INFO:teuthology.orchestra.run.smithi107.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-05-17T21:45:07.821 INFO:teuthology.orchestra.run.smithi107.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-17T21:45:07.822 INFO:teuthology.orchestra.run.smithi107.stdout:Context: system_u:object_r:device_t:s0 2024-05-17T21:45:07.822 INFO:teuthology.orchestra.run.smithi107.stdout:Access: 2024-05-17 21:44:20.593784716 +0000 2024-05-17T21:45:07.822 INFO:teuthology.orchestra.run.smithi107.stdout:Modify: 2024-05-17 21:38:37.148226328 +0000 2024-05-17T21:45:07.822 INFO:teuthology.orchestra.run.smithi107.stdout:Change: 2024-05-17 21:38:37.148226328 +0000 2024-05-17T21:45:07.822 INFO:teuthology.orchestra.run.smithi107.stdout: Birth: 2024-05-17 21:38:37.148226328 +0000 2024-05-17T21:45:07.822 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-17T21:45:07.885 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records in 2024-05-17T21:45:07.885 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records out 2024-05-17T21:45:07.885 INFO:teuthology.orchestra.run.smithi107.stderr:512 bytes copied, 0.000186624 s, 2.7 MB/s 2024-05-17T21:45:07.887 DEBUG:teuthology.orchestra.run.smithi107:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-17T21:45:07.943 DEBUG:teuthology.orchestra.run.smithi107:> stat /dev/vg_nvme/lv_4 2024-05-17T21:45:07.998 INFO:teuthology.orchestra.run.smithi107.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-17T21:45:07.998 INFO:teuthology.orchestra.run.smithi107.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-17T21:45:07.998 INFO:teuthology.orchestra.run.smithi107.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-05-17T21:45:07.999 INFO:teuthology.orchestra.run.smithi107.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-17T21:45:07.999 INFO:teuthology.orchestra.run.smithi107.stdout:Context: system_u:object_r:device_t:s0 2024-05-17T21:45:07.999 INFO:teuthology.orchestra.run.smithi107.stdout:Access: 2024-05-17 21:44:20.594784700 +0000 2024-05-17T21:45:07.999 INFO:teuthology.orchestra.run.smithi107.stdout:Modify: 2024-05-17 21:38:37.492220526 +0000 2024-05-17T21:45:07.999 INFO:teuthology.orchestra.run.smithi107.stdout:Change: 2024-05-17 21:38:37.492220526 +0000 2024-05-17T21:45:07.999 INFO:teuthology.orchestra.run.smithi107.stdout: Birth: 2024-05-17 21:38:37.492220526 +0000 2024-05-17T21:45:07.999 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-17T21:45:08.061 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:07 smithi107 ceph-mon[28434]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T21:45:08.061 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:07 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:45:08.061 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:07 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:45:08.061 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:07 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi107.xhhyiw", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-17T21:45:08.061 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:07 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-17T21:45:08.061 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:07 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:45:08.062 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records in 2024-05-17T21:45:08.062 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records out 2024-05-17T21:45:08.062 INFO:teuthology.orchestra.run.smithi107.stderr:512 bytes copied, 0.000169098 s, 3.0 MB/s 2024-05-17T21:45:08.063 DEBUG:teuthology.orchestra.run.smithi107:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-17T21:45:08.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:07 smithi138 ceph-mon[35948]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T21:45:08.078 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:07 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:45:08.078 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:07 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:45:08.078 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:07 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi107.xhhyiw", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-17T21:45:08.078 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:07 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-17T21:45:08.078 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:07 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:45:08.120 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-17T21:45:08.120 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/scratch_devs of=/dev/stdout 2024-05-17T21:45:08.139 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-17T21:45:08.140 DEBUG:teuthology.orchestra.run.smithi138:> stat /dev/vg_nvme/lv_1 2024-05-17T21:45:08.198 INFO:teuthology.orchestra.run.smithi138.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-17T21:45:08.198 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-17T21:45:08.198 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-05-17T21:45:08.198 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-17T21:45:08.198 INFO:teuthology.orchestra.run.smithi138.stdout:Context: system_u:object_r:device_t:s0 2024-05-17T21:45:08.198 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2024-05-17 21:44:54.521804575 +0000 2024-05-17T21:45:08.199 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2024-05-17 21:38:36.946035982 +0000 2024-05-17T21:45:08.199 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2024-05-17 21:38:36.946035982 +0000 2024-05-17T21:45:08.199 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: 2024-05-17 21:38:36.946035982 +0000 2024-05-17T21:45:08.199 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-17T21:45:08.265 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records in 2024-05-17T21:45:08.266 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records out 2024-05-17T21:45:08.266 INFO:teuthology.orchestra.run.smithi138.stderr:512 bytes copied, 0.000291361 s, 1.8 MB/s 2024-05-17T21:45:08.267 DEBUG:teuthology.orchestra.run.smithi138:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-17T21:45:08.325 DEBUG:teuthology.orchestra.run.smithi138:> stat /dev/vg_nvme/lv_2 2024-05-17T21:45:08.383 INFO:teuthology.orchestra.run.smithi138.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-17T21:45:08.383 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-17T21:45:08.383 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-05-17T21:45:08.383 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-17T21:45:08.383 INFO:teuthology.orchestra.run.smithi138.stdout:Context: system_u:object_r:device_t:s0 2024-05-17T21:45:08.383 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2024-05-17 21:44:54.521804575 +0000 2024-05-17T21:45:08.383 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2024-05-17 21:38:37.281028234 +0000 2024-05-17T21:45:08.383 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2024-05-17 21:38:37.281028234 +0000 2024-05-17T21:45:08.384 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: 2024-05-17 21:38:37.281028234 +0000 2024-05-17T21:45:08.384 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-17T21:45:08.449 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records in 2024-05-17T21:45:08.450 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records out 2024-05-17T21:45:08.450 INFO:teuthology.orchestra.run.smithi138.stderr:512 bytes copied, 0.000231736 s, 2.2 MB/s 2024-05-17T21:45:08.451 DEBUG:teuthology.orchestra.run.smithi138:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-17T21:45:08.509 DEBUG:teuthology.orchestra.run.smithi138:> stat /dev/vg_nvme/lv_3 2024-05-17T21:45:08.566 INFO:teuthology.orchestra.run.smithi138.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-17T21:45:08.567 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-17T21:45:08.567 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-05-17T21:45:08.567 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-17T21:45:08.567 INFO:teuthology.orchestra.run.smithi138.stdout:Context: system_u:object_r:device_t:s0 2024-05-17T21:45:08.567 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2024-05-17 21:44:54.522804554 +0000 2024-05-17T21:45:08.567 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2024-05-17 21:38:37.616020486 +0000 2024-05-17T21:45:08.567 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2024-05-17 21:38:37.616020486 +0000 2024-05-17T21:45:08.567 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: 2024-05-17 21:38:37.616020486 +0000 2024-05-17T21:45:08.567 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-17T21:45:08.632 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records in 2024-05-17T21:45:08.632 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records out 2024-05-17T21:45:08.632 INFO:teuthology.orchestra.run.smithi138.stderr:512 bytes copied, 0.000188545 s, 2.7 MB/s 2024-05-17T21:45:08.633 DEBUG:teuthology.orchestra.run.smithi138:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-17T21:45:08.691 DEBUG:teuthology.orchestra.run.smithi138:> stat /dev/vg_nvme/lv_4 2024-05-17T21:45:08.748 INFO:teuthology.orchestra.run.smithi138.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-17T21:45:08.748 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-17T21:45:08.748 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-05-17T21:45:08.748 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-17T21:45:08.748 INFO:teuthology.orchestra.run.smithi138.stdout:Context: system_u:object_r:device_t:s0 2024-05-17T21:45:08.748 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2024-05-17 21:44:54.522804554 +0000 2024-05-17T21:45:08.748 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2024-05-17 21:38:37.949012785 +0000 2024-05-17T21:45:08.748 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2024-05-17 21:38:37.949012785 +0000 2024-05-17T21:45:08.748 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: 2024-05-17 21:38:37.949012785 +0000 2024-05-17T21:45:08.748 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-17T21:45:08.813 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records in 2024-05-17T21:45:08.813 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records out 2024-05-17T21:45:08.813 INFO:teuthology.orchestra.run.smithi138.stderr:512 bytes copied, 0.000213888 s, 2.4 MB/s 2024-05-17T21:45:08.814 DEBUG:teuthology.orchestra.run.smithi138:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-17T21:45:08.871 INFO:tasks.cephadm:Deploying osd.0 on smithi107 with /dev/vg_nvme/lv_4... 2024-05-17T21:45:08.871 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-17T21:45:09.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:08 smithi138 ceph-mon[35948]: Reconfiguring mgr.smithi107.xhhyiw (unknown last config time)... 2024-05-17T21:45:09.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:08 smithi138 ceph-mon[35948]: Reconfiguring daemon mgr.smithi107.xhhyiw on smithi107 2024-05-17T21:45:09.078 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:08 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:45:09.078 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:08 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:45:09.078 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:08 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi107", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-17T21:45:09.078 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:08 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:45:09.104 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:08 smithi107 ceph-mon[28434]: Reconfiguring mgr.smithi107.xhhyiw (unknown last config time)... 2024-05-17T21:45:09.104 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:08 smithi107 ceph-mon[28434]: Reconfiguring daemon mgr.smithi107.xhhyiw on smithi107 2024-05-17T21:45:09.104 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:08 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:45:09.104 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:08 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:45:09.104 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:08 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi107", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-17T21:45:09.104 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:08 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:45:09.648 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/mon.smithi107/config 2024-05-17T21:45:09.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:09 smithi107 ceph-mon[28434]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T21:45:09.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:09 smithi107 ceph-mon[28434]: Reconfiguring crash.smithi107 (monmap changed)... 2024-05-17T21:45:09.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:09 smithi107 ceph-mon[28434]: Reconfiguring daemon crash.smithi107 on smithi107 2024-05-17T21:45:09.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:09 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:45:09.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:09 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:45:10.078 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:09 smithi138 ceph-mon[35948]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T21:45:10.078 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:09 smithi138 ceph-mon[35948]: Reconfiguring crash.smithi107 (monmap changed)... 2024-05-17T21:45:10.078 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:09 smithi138 ceph-mon[35948]: Reconfiguring daemon crash.smithi107 on smithi107 2024-05-17T21:45:10.078 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:09 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:45:10.078 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:09 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:45:10.996 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:10 smithi107 ceph-mon[28434]: Reconfiguring alertmanager.smithi107 (dependencies changed)... 2024-05-17T21:45:10.996 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:10 smithi107 ceph-mon[28434]: Reconfiguring daemon alertmanager.smithi107 on smithi107 2024-05-17T21:45:11.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:10 smithi138 ceph-mon[35948]: Reconfiguring alertmanager.smithi107 (dependencies changed)... 2024-05-17T21:45:11.078 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:10 smithi138 ceph-mon[35948]: Reconfiguring daemon alertmanager.smithi107 on smithi107 2024-05-17T21:45:12.078 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:11 smithi138 ceph-mon[35948]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T21:45:12.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:11 smithi107 ceph-mon[28434]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T21:45:12.839 INFO:teuthology.orchestra.run.smithi107.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-17T21:45:12.839 INFO:teuthology.orchestra.run.smithi107.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-17T21:45:12.839 INFO:teuthology.orchestra.run.smithi107.stderr: stderr: 10+0 records in 2024-05-17T21:45:12.839 INFO:teuthology.orchestra.run.smithi107.stderr:10+0 records out 2024-05-17T21:45:12.839 INFO:teuthology.orchestra.run.smithi107.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0179318 s, 585 MB/s 2024-05-17T21:45:12.839 INFO:teuthology.orchestra.run.smithi107.stderr:--> Zapping successful for: 2024-05-17T21:45:12.966 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:12 smithi107 ceph-mon[28434]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T21:45:13.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:12 smithi138 ceph-mon[35948]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T21:45:13.486 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi107:vg_nvme/lv_4 2024-05-17T21:45:14.078 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/mon.smithi107/config 2024-05-17T21:45:15.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:15 smithi107 ceph-mon[28434]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T21:45:15.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:15.798+0000 7f4ff4fc2700 1 -- 172.21.15.107:0/488783142 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ff00fcf60 msgr2=0x7f4ff00fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:45:15.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:15.798+0000 7f4ff4fc2700 1 --2- 172.21.15.107:0/488783142 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ff00fcf60 0x7f4ff00fd3d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4fe4004940 tx=0x7f4fe4009e30 comp rx=0 tx=0).stop 2024-05-17T21:45:15.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:15.798+0000 7f4ff4fc2700 1 -- 172.21.15.107:0/488783142 shutdown_connections 2024-05-17T21:45:15.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:15.798+0000 7f4ff4fc2700 1 --2- 172.21.15.107:0/488783142 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ff00fcf60 0x7f4ff00fd3d0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:45:15.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:15.798+0000 7f4ff4fc2700 1 --2- 172.21.15.107:0/488783142 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ff00fbcb0 0x7f4ff00fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:45:15.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:15.798+0000 7f4ff4fc2700 1 -- 172.21.15.107:0/488783142 >> 172.21.15.107:0/488783142 conn(0x7f4ff00f7430 msgr2=0x7f4ff00f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:45:15.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:15.798+0000 7f4ff4fc2700 1 -- 172.21.15.107:0/488783142 shutdown_connections 2024-05-17T21:45:15.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:15.798+0000 7f4ff4fc2700 1 -- 172.21.15.107:0/488783142 wait complete. 2024-05-17T21:45:15.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:15.799+0000 7f4ff4fc2700 1 Processor -- start 2024-05-17T21:45:15.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:15.799+0000 7f4ff4fc2700 1 -- start start 2024-05-17T21:45:15.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:15.799+0000 7f4ff4fc2700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ff00fbcb0 0x7f4ff0110710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:45:15.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:15.799+0000 7f4ff4fc2700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ff00fcf60 0x7f4ff0112c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:45:15.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:15.799+0000 7f4ff4fc2700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff00663c0 con 0x7f4ff00fbcb0 2024-05-17T21:45:15.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:15.799+0000 7f4ff4fc2700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff0066500 con 0x7f4ff00fcf60 2024-05-17T21:45:15.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:15.799+0000 7f4fee59c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ff00fbcb0 0x7f4ff0110710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:45:15.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:15.799+0000 7f4fee59c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ff00fbcb0 0x7f4ff0110710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37036/0 (socket says 172.21.15.107:37036) 2024-05-17T21:45:15.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:15.799+0000 7f4fee59c700 1 -- 172.21.15.107:0/440606583 learned_addr learned my addr 172.21.15.107:0/440606583 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:45:15.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:15.799+0000 7f4fee59c700 1 -- 172.21.15.107:0/440606583 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ff00fcf60 msgr2=0x7f4ff0112c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:45:15.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:15.799+0000 7f4fee59c700 1 --2- 172.21.15.107:0/440606583 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ff00fcf60 0x7f4ff0112c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:45:15.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:15.799+0000 7f4fee59c700 1 -- 172.21.15.107:0/440606583 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fe4005040 con 0x7f4ff00fbcb0 2024-05-17T21:45:15.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:15.799+0000 7f4fee59c700 1 --2- 172.21.15.107:0/440606583 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ff00fbcb0 0x7f4ff0110710 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4fe000b790 tx=0x7f4fe0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:45:15.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:15.799+0000 7f4fdeffd700 1 -- 172.21.15.107:0/440606583 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4fe000f770 con 0x7f4ff00fbcb0 2024-05-17T21:45:15.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:15.800+0000 7f4fdeffd700 1 -- 172.21.15.107:0/440606583 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4fe000fda0 con 0x7f4ff00fbcb0 2024-05-17T21:45:15.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:15.800+0000 7f4ff4fc2700 1 -- 172.21.15.107:0/440606583 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ff01132c0 con 0x7f4ff00fbcb0 2024-05-17T21:45:15.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:15.800+0000 7f4fdeffd700 1 -- 172.21.15.107:0/440606583 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4fe000d310 con 0x7f4ff00fbcb0 2024-05-17T21:45:15.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:15.800+0000 7f4ff4fc2700 1 -- 172.21.15.107:0/440606583 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ff0113850 con 0x7f4ff00fbcb0 2024-05-17T21:45:15.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:15.801+0000 7f4ff4fc2700 1 -- 172.21.15.107:0/440606583 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ff005f5c0 con 0x7f4ff00fbcb0 2024-05-17T21:45:15.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:15.801+0000 7f4fdeffd700 1 -- 172.21.15.107:0/440606583 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 86575+0+0 (secure 0 0 0) 0x7f4fe0010420 con 0x7f4ff00fbcb0 2024-05-17T21:45:15.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:15.801+0000 7f4fdeffd700 1 --2- 172.21.15.107:0/440606583 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f4fd8069210 0x7f4fd806b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:45:15.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:15.802+0000 7f4fdeffd700 1 -- 172.21.15.107:0/440606583 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f4fe0086090 con 0x7f4ff00fbcb0 2024-05-17T21:45:15.797 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:15.802+0000 7f4fedd9b700 1 --2- 172.21.15.107:0/440606583 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f4fd8069210 0x7f4fd806b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:45:15.797 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:15.802+0000 7f4fedd9b700 1 --2- 172.21.15.107:0/440606583 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f4fd8069210 0x7f4fd806b6c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4fe4004f90 tx=0x7f4fe4009e30 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:45:15.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:15.806+0000 7f4fdeffd700 1 -- 172.21.15.107:0/440606583 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f4fe00534a0 con 0x7f4ff00fbcb0 2024-05-17T21:45:15.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:15 smithi138 ceph-mon[35948]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T21:45:15.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:15.998+0000 7f4ff4fc2700 1 -- 172.21.15.107:0/440606583 --> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f4ff0101200 con 0x7f4fd8069210 2024-05-17T21:45:16.713 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:16 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T21:45:16.713 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:16 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T21:45:16.713 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:16 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:45:16.828 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:16 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T21:45:16.828 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:16 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T21:45:16.828 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:16 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:45:17.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:17 smithi107 ceph-mon[28434]: from='client.14248 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:45:17.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:17 smithi107 ceph-mon[28434]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T21:45:17.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:17 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:45:17.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:17 smithi138 ceph-mon[35948]: from='client.14248 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:45:17.828 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:17 smithi138 ceph-mon[35948]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T21:45:17.828 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:17 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:45:19.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:19 smithi138 ceph-mon[35948]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T21:45:19.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:19 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:45:19.828 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:19 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:45:19.828 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:19 smithi138 ceph-mon[35948]: Reconfiguring grafana.smithi107 (dependencies changed)... 2024-05-17T21:45:19.828 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:19 smithi138 ceph-mon[35948]: Reconfiguring daemon grafana.smithi107 on smithi107 2024-05-17T21:45:19.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:19 smithi107 ceph-mon[28434]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T21:45:19.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:19 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:45:19.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:19 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:45:19.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:19 smithi107 ceph-mon[28434]: Reconfiguring grafana.smithi107 (dependencies changed)... 2024-05-17T21:45:19.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:19 smithi107 ceph-mon[28434]: Reconfiguring daemon grafana.smithi107 on smithi107 2024-05-17T21:45:20.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:20 smithi107 ceph-mon[28434]: from='client.? 172.21.15.107:0/2445761990' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "203debeb-d443-4b99-a929-64afe799acfa"}]: dispatch 2024-05-17T21:45:20.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:20 smithi107 ceph-mon[28434]: from='client.? 172.21.15.107:0/2445761990' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "203debeb-d443-4b99-a929-64afe799acfa"}]': finished 2024-05-17T21:45:20.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:20 smithi107 ceph-mon[28434]: osdmap e6: 1 total, 0 up, 1 in 2024-05-17T21:45:20.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:20 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-17T21:45:20.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:20 smithi107 ceph-mon[28434]: from='client.? 172.21.15.107:0/2138404209' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T21:45:20.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:20 smithi138 ceph-mon[35948]: from='client.? 172.21.15.107:0/2445761990' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "203debeb-d443-4b99-a929-64afe799acfa"}]: dispatch 2024-05-17T21:45:20.828 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:20 smithi138 ceph-mon[35948]: from='client.? 172.21.15.107:0/2445761990' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "203debeb-d443-4b99-a929-64afe799acfa"}]': finished 2024-05-17T21:45:20.828 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:20 smithi138 ceph-mon[35948]: osdmap e6: 1 total, 0 up, 1 in 2024-05-17T21:45:20.828 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:20 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-17T21:45:20.828 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:20 smithi138 ceph-mon[35948]: from='client.? 172.21.15.107:0/2138404209' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T21:45:21.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:21 smithi138 ceph-mon[35948]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T21:45:21.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:21 smithi107 ceph-mon[28434]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T21:45:23.748 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:23 smithi107 ceph-mon[28434]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T21:45:23.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:23 smithi138 ceph-mon[35948]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T21:45:25.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:25 smithi138 ceph-mon[35948]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T21:45:25.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:25 smithi107 ceph-mon[28434]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T21:45:27.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:27 smithi138 ceph-mon[35948]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T21:45:27.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:27 smithi107 ceph-mon[28434]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T21:45:30.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:29 smithi138 ceph-mon[35948]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T21:45:30.078 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:29 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:45:30.078 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:29 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:45:30.078 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:29 smithi138 ceph-mon[35948]: Reconfiguring prometheus.smithi107 (dependencies changed)... 2024-05-17T21:45:30.078 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:29 smithi138 ceph-mon[35948]: Reconfiguring daemon prometheus.smithi107 on smithi107 2024-05-17T21:45:30.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:29 smithi107 ceph-mon[28434]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T21:45:30.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:29 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:45:30.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:29 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:45:30.104 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:29 smithi107 ceph-mon[28434]: Reconfiguring prometheus.smithi107 (dependencies changed)... 2024-05-17T21:45:30.104 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:29 smithi107 ceph-mon[28434]: Reconfiguring daemon prometheus.smithi107 on smithi107 2024-05-17T21:45:31.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:30 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-17T21:45:31.078 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:30 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:45:31.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:30 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-17T21:45:31.104 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:30 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:45:32.064 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:31 smithi107 ceph-mon[28434]: Deploying daemon osd.0 on smithi107 2024-05-17T21:45:32.065 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:31 smithi107 ceph-mon[28434]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T21:45:32.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:31 smithi138 ceph-mon[35948]: Deploying daemon osd.0 on smithi107 2024-05-17T21:45:32.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:31 smithi138 ceph-mon[35948]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T21:45:33.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:32 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:45:33.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:32 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:45:34.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:33 smithi138 ceph-mon[35948]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T21:45:34.098 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:33 smithi107 ceph-mon[28434]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T21:45:35.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:35 smithi107 ceph-mon[28434]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T21:45:35.605 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:35 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:45:35.605 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:35 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:45:35.605 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:35 smithi107 ceph-mon[28434]: Reconfiguring crash.smithi138 (monmap changed)... 2024-05-17T21:45:35.605 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:35 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi138", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-17T21:45:35.605 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:35 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:45:35.605 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:35 smithi107 ceph-mon[28434]: Reconfiguring daemon crash.smithi138 on smithi138 2024-05-17T21:45:35.775 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:35 smithi138 ceph-mon[35948]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T21:45:35.775 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:35 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:45:35.775 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:35 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:45:35.775 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:35 smithi138 ceph-mon[35948]: Reconfiguring crash.smithi138 (monmap changed)... 2024-05-17T21:45:35.775 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:35 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi138", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-17T21:45:35.775 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:35 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:45:35.776 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:35 smithi138 ceph-mon[35948]: Reconfiguring daemon crash.smithi138 on smithi138 2024-05-17T21:45:36.966 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:36 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:45:36.966 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:36 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:45:36.966 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:36 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi138.napnll", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-17T21:45:36.966 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:36 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-17T21:45:36.967 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:36 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:45:37.155 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:36 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:45:37.156 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:36 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:45:37.156 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:36 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi138.napnll", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-17T21:45:37.156 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:36 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-17T21:45:37.156 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:36 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:45:37.882 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:37 smithi107 ceph-mon[28434]: Reconfiguring mgr.smithi138.napnll (monmap changed)... 2024-05-17T21:45:37.883 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:37 smithi107 ceph-mon[28434]: Reconfiguring daemon mgr.smithi138.napnll on smithi138 2024-05-17T21:45:37.883 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:37 smithi107 ceph-mon[28434]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T21:45:37.883 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:37 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:45:37.883 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:37 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:45:37.883 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:37 smithi107 ceph-mon[28434]: Reconfiguring mon.smithi138 (monmap changed)... 2024-05-17T21:45:37.883 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:37 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-17T21:45:37.883 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:37 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-17T21:45:37.883 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:37 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:45:37.883 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:37 smithi107 ceph-mon[28434]: Reconfiguring daemon mon.smithi138 on smithi138 2024-05-17T21:45:37.926 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:37 smithi138 ceph-mon[35948]: Reconfiguring mgr.smithi138.napnll (monmap changed)... 2024-05-17T21:45:37.926 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:37 smithi138 ceph-mon[35948]: Reconfiguring daemon mgr.smithi138.napnll on smithi138 2024-05-17T21:45:37.926 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:37 smithi138 ceph-mon[35948]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T21:45:37.926 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:37 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:45:37.926 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:37 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:45:37.926 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:37 smithi138 ceph-mon[35948]: Reconfiguring mon.smithi138 (monmap changed)... 2024-05-17T21:45:37.926 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:37 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-17T21:45:37.926 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:37 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-17T21:45:37.927 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:37 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:45:37.927 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:37 smithi138 ceph-mon[35948]: Reconfiguring daemon mon.smithi138 on smithi138 2024-05-17T21:45:39.802 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:39 smithi107 ceph-mon[28434]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T21:45:39.803 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:39 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:45:39.803 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:39 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:45:39.803 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:39 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-05-17T21:45:39.803 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:39 smithi107 ceph-mon[28434]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-05-17T21:45:39.803 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:39 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi107.front.sepia.ceph.com:9093"}]: dispatch 2024-05-17T21:45:39.803 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:39 smithi107 ceph-mon[28434]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi107.front.sepia.ceph.com:9093"}]: dispatch 2024-05-17T21:45:39.803 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:39 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:45:39.803 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:39 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-05-17T21:45:39.803 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:39 smithi107 ceph-mon[28434]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-05-17T21:45:39.803 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:39 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi107.front.sepia.ceph.com:3000"}]: dispatch 2024-05-17T21:45:39.803 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:39 smithi107 ceph-mon[28434]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi107.front.sepia.ceph.com:3000"}]: dispatch 2024-05-17T21:45:39.803 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:39 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:45:39.803 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:39 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-17T21:45:39.804 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:39 smithi107 ceph-mon[28434]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-17T21:45:39.804 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:39 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi107.front.sepia.ceph.com:9095"}]: dispatch 2024-05-17T21:45:39.804 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:39 smithi107 ceph-mon[28434]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi107.front.sepia.ceph.com:9095"}]: dispatch 2024-05-17T21:45:39.804 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:39 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:45:39.804 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:39 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:45:39.804 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:39 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:45:39.804 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:39 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:45:39.850 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:39 smithi138 ceph-mon[35948]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T21:45:39.850 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:39 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:45:39.851 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:39 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:45:39.851 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:39 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-05-17T21:45:39.851 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:39 smithi138 ceph-mon[35948]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-05-17T21:45:39.851 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:39 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi107.front.sepia.ceph.com:9093"}]: dispatch 2024-05-17T21:45:39.851 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:39 smithi138 ceph-mon[35948]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi107.front.sepia.ceph.com:9093"}]: dispatch 2024-05-17T21:45:39.851 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:39 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:45:39.851 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:39 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-05-17T21:45:39.851 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:39 smithi138 ceph-mon[35948]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-05-17T21:45:39.851 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:39 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi107.front.sepia.ceph.com:3000"}]: dispatch 2024-05-17T21:45:39.851 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:39 smithi138 ceph-mon[35948]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi107.front.sepia.ceph.com:3000"}]: dispatch 2024-05-17T21:45:39.851 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:39 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:45:39.851 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:39 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-17T21:45:39.852 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:39 smithi138 ceph-mon[35948]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-17T21:45:39.852 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:39 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi107.front.sepia.ceph.com:9095"}]: dispatch 2024-05-17T21:45:39.852 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:39 smithi138 ceph-mon[35948]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi107.front.sepia.ceph.com:9095"}]: dispatch 2024-05-17T21:45:39.852 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:39 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:45:39.852 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:39 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:45:39.852 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:39 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:45:39.852 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:39 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:45:41.717 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:41 smithi107 ceph-mon[28434]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T21:45:41.717 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:41 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:45:41.717 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:41 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:45:41.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:41 smithi138 ceph-mon[35948]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T21:45:41.828 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:41 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:45:41.828 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:41 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:45:43.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:43.276+0000 7f4fdeffd700 1 -- 172.21.15.107:0/440606583 <== mgr.14214 v2:172.21.15.107:6800/1294361014 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f4ff0101200 con 0x7f4fd8069210 2024-05-17T21:45:43.270 INFO:teuthology.orchestra.run.smithi107.stdout:Created osd(s) 0 on host 'smithi107' 2024-05-17T21:45:43.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:43.278+0000 7f4ff4fc2700 1 -- 172.21.15.107:0/440606583 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f4fd8069210 msgr2=0x7f4fd806b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:45:43.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:43.278+0000 7f4ff4fc2700 1 --2- 172.21.15.107:0/440606583 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f4fd8069210 0x7f4fd806b6c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4fe4004f90 tx=0x7f4fe4009e30 comp rx=0 tx=0).stop 2024-05-17T21:45:43.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:43.278+0000 7f4ff4fc2700 1 -- 172.21.15.107:0/440606583 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ff00fbcb0 msgr2=0x7f4ff0110710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:45:43.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:43.278+0000 7f4ff4fc2700 1 --2- 172.21.15.107:0/440606583 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ff00fbcb0 0x7f4ff0110710 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4fe000b790 tx=0x7f4fe0009250 comp rx=0 tx=0).stop 2024-05-17T21:45:43.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:43.278+0000 7f4ff4fc2700 1 -- 172.21.15.107:0/440606583 shutdown_connections 2024-05-17T21:45:43.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:43.278+0000 7f4ff4fc2700 1 --2- 172.21.15.107:0/440606583 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f4fd8069210 0x7f4fd806b6c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:45:43.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:43.278+0000 7f4ff4fc2700 1 --2- 172.21.15.107:0/440606583 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ff00fbcb0 0x7f4ff0110710 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:45:43.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:43.278+0000 7f4ff4fc2700 1 --2- 172.21.15.107:0/440606583 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ff00fcf60 0x7f4ff0112c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:45:43.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:43.278+0000 7f4ff4fc2700 1 -- 172.21.15.107:0/440606583 >> 172.21.15.107:0/440606583 conn(0x7f4ff00f7430 msgr2=0x7f4ff00f8a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:45:43.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:43.278+0000 7f4ff4fc2700 1 -- 172.21.15.107:0/440606583 shutdown_connections 2024-05-17T21:45:43.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:43.278+0000 7f4ff4fc2700 1 -- 172.21.15.107:0/440606583 wait complete. 2024-05-17T21:45:43.464 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:43 smithi107 ceph-mon[28434]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T21:45:43.464 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:43 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:45:43.464 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:43 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:45:43.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:43 smithi138 ceph-mon[35948]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T21:45:43.828 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:43 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:45:43.828 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:43 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:45:43.862 DEBUG:teuthology.orchestra.run.smithi107:osd.0> sudo journalctl -f -n 0 -u ceph-3722731a-1496-11ef-bc9a-c7b262605968@osd.0.service 2024-05-17T21:45:43.864 INFO:tasks.cephadm:Deploying osd.1 on smithi107 with /dev/vg_nvme/lv_3... 2024-05-17T21:45:43.864 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-17T21:45:44.577 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:44 smithi138 ceph-mon[35948]: from='osd.0 [v2:172.21.15.107:6802/710062679,v1:172.21.15.107:6803/710062679]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-17T21:45:44.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:44 smithi107 ceph-mon[28434]: from='osd.0 [v2:172.21.15.107:6802/710062679,v1:172.21.15.107:6803/710062679]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-17T21:45:44.863 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/mon.smithi107/config 2024-05-17T21:45:45.828 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:45 smithi138 ceph-mon[35948]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T21:45:45.828 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:45 smithi138 ceph-mon[35948]: from='osd.0 [v2:172.21.15.107:6802/710062679,v1:172.21.15.107:6803/710062679]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-17T21:45:45.828 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:45 smithi138 ceph-mon[35948]: osdmap e7: 1 total, 0 up, 1 in 2024-05-17T21:45:45.828 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:45 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-17T21:45:45.828 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:45 smithi138 ceph-mon[35948]: from='osd.0 [v2:172.21.15.107:6802/710062679,v1:172.21.15.107:6803/710062679]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-05-17T21:45:45.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:45 smithi107 ceph-mon[28434]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T21:45:45.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:45 smithi107 ceph-mon[28434]: from='osd.0 [v2:172.21.15.107:6802/710062679,v1:172.21.15.107:6803/710062679]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-17T21:45:45.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:45 smithi107 ceph-mon[28434]: osdmap e7: 1 total, 0 up, 1 in 2024-05-17T21:45:45.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:45 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-17T21:45:45.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:45 smithi107 ceph-mon[28434]: from='osd.0 [v2:172.21.15.107:6802/710062679,v1:172.21.15.107:6803/710062679]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-05-17T21:45:45.854 INFO:journalctl@ceph.osd.0.smithi107.stdout:May 17 21:45:45 smithi107 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-0[47179]: 2024-05-17T21:45:45.441+0000 7fb014201700 -1 osd.0 0 waiting for initial osdmap 2024-05-17T21:45:45.854 INFO:journalctl@ceph.osd.0.smithi107.stdout:May 17 21:45:45 smithi107 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-0[47179]: 2024-05-17T21:45:45.447+0000 7fb00effc700 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-17T21:45:46.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:46 smithi107 ceph-mon[28434]: from='osd.0 [v2:172.21.15.107:6802/710062679,v1:172.21.15.107:6803/710062679]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-05-17T21:45:46.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:46 smithi107 ceph-mon[28434]: osdmap e8: 1 total, 0 up, 1 in 2024-05-17T21:45:46.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:46 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-17T21:45:46.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:46 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-17T21:45:46.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:46 smithi138 ceph-mon[35948]: from='osd.0 [v2:172.21.15.107:6802/710062679,v1:172.21.15.107:6803/710062679]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-05-17T21:45:46.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:46 smithi138 ceph-mon[35948]: osdmap e8: 1 total, 0 up, 1 in 2024-05-17T21:45:46.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:46 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-17T21:45:46.828 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:46 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-17T21:45:47.370 INFO:teuthology.orchestra.run.smithi107.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-17T21:45:47.370 INFO:teuthology.orchestra.run.smithi107.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-17T21:45:47.370 INFO:teuthology.orchestra.run.smithi107.stderr: stderr: 10+0 records in 2024-05-17T21:45:47.370 INFO:teuthology.orchestra.run.smithi107.stderr:10+0 records out 2024-05-17T21:45:47.370 INFO:teuthology.orchestra.run.smithi107.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0121504 s, 863 MB/s 2024-05-17T21:45:47.370 INFO:teuthology.orchestra.run.smithi107.stderr:--> Zapping successful for: 2024-05-17T21:45:47.440 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:47 smithi107 ceph-mon[28434]: purged_snaps scrub starts 2024-05-17T21:45:47.440 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:47 smithi107 ceph-mon[28434]: purged_snaps scrub ok 2024-05-17T21:45:47.440 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:47 smithi107 ceph-mon[28434]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T21:45:47.440 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:47 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-17T21:45:47.441 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:47 smithi107 ceph-mon[28434]: osd.0 [v2:172.21.15.107:6802/710062679,v1:172.21.15.107:6803/710062679] boot 2024-05-17T21:45:47.441 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:47 smithi107 ceph-mon[28434]: osdmap e9: 1 total, 1 up, 1 in 2024-05-17T21:45:47.712 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:47 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-17T21:45:47.713 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:47 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:45:47.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:47 smithi138 ceph-mon[35948]: purged_snaps scrub starts 2024-05-17T21:45:47.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:47 smithi138 ceph-mon[35948]: purged_snaps scrub ok 2024-05-17T21:45:47.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:47 smithi138 ceph-mon[35948]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T21:45:47.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:47 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-17T21:45:47.828 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:47 smithi138 ceph-mon[35948]: osd.0 [v2:172.21.15.107:6802/710062679,v1:172.21.15.107:6803/710062679] boot 2024-05-17T21:45:47.828 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:47 smithi138 ceph-mon[35948]: osdmap e9: 1 total, 1 up, 1 in 2024-05-17T21:45:47.828 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:47 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-17T21:45:47.828 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:47 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:45:47.966 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi107:vg_nvme/lv_3 2024-05-17T21:45:48.229 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/mon.smithi107/config 2024-05-17T21:45:48.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:48 smithi107 ceph-mon[28434]: osdmap e10: 1 total, 1 up, 1 in 2024-05-17T21:45:48.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:48 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:45:48.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:48 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:45:48.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:48 smithi138 ceph-mon[35948]: osdmap e10: 1 total, 1 up, 1 in 2024-05-17T21:45:48.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:48 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:45:48.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:48 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:45:49.547 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:49 smithi107 ceph-mon[28434]: pgmap v36: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-17T21:45:49.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:49.757+0000 7f95d284b700 1 -- 172.21.15.107:0/212318424 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95cc0fce60 msgr2=0x7f95cc0fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:45:49.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:49.757+0000 7f95d284b700 1 --2- 172.21.15.107:0/212318424 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95cc0fce60 0x7f95cc0fd270 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f95bc004940 tx=0x7f95bc009e30 comp rx=0 tx=0).stop 2024-05-17T21:45:49.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:49.757+0000 7f95d284b700 1 -- 172.21.15.107:0/212318424 shutdown_connections 2024-05-17T21:45:49.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:49.757+0000 7f95d284b700 1 --2- 172.21.15.107:0/212318424 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95cc0fdfd0 0x7f95cc0fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:45:49.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:49.757+0000 7f95d284b700 1 --2- 172.21.15.107:0/212318424 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95cc0fce60 0x7f95cc0fd270 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:45:49.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:49.757+0000 7f95d284b700 1 -- 172.21.15.107:0/212318424 >> 172.21.15.107:0/212318424 conn(0x7f95cc0f8540 msgr2=0x7f95cc0fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:45:49.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:49.757+0000 7f95d284b700 1 -- 172.21.15.107:0/212318424 shutdown_connections 2024-05-17T21:45:49.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:49.757+0000 7f95d284b700 1 -- 172.21.15.107:0/212318424 wait complete. 2024-05-17T21:45:49.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:49.757+0000 7f95d284b700 1 Processor -- start 2024-05-17T21:45:49.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:49.757+0000 7f95d284b700 1 -- start start 2024-05-17T21:45:49.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:49.758+0000 7f95d284b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95cc0fce60 0x7f95cc071c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:45:49.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:49.758+0000 7f95d284b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95cc0fdfd0 0x7f95cc0702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:45:49.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:49.758+0000 7f95d284b700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95cc072200 con 0x7f95cc0fce60 2024-05-17T21:45:49.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:49.758+0000 7f95d284b700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95cc0707f0 con 0x7f95cc0fdfd0 2024-05-17T21:45:49.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:49.758+0000 7f95cbfff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95cc0fce60 0x7f95cc071c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:45:49.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:49.758+0000 7f95cbfff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95cc0fce60 0x7f95cc071c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:53746/0 (socket says 172.21.15.107:53746) 2024-05-17T21:45:49.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:49.758+0000 7f95cbfff700 1 -- 172.21.15.107:0/851337824 learned_addr learned my addr 172.21.15.107:0/851337824 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:45:49.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:49.758+0000 7f95c3fff700 1 --2- 172.21.15.107:0/851337824 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95cc0fdfd0 0x7f95cc0702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:45:49.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:49.758+0000 7f95cbfff700 1 -- 172.21.15.107:0/851337824 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95cc0fdfd0 msgr2=0x7f95cc0702b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:45:49.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:49.758+0000 7f95cbfff700 1 --2- 172.21.15.107:0/851337824 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95cc0fdfd0 0x7f95cc0702b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:45:49.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:49.758+0000 7f95cbfff700 1 -- 172.21.15.107:0/851337824 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95bc005040 con 0x7f95cc0fce60 2024-05-17T21:45:49.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:49.758+0000 7f95cbfff700 1 --2- 172.21.15.107:0/851337824 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95cc0fce60 0x7f95cc071c00 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f95bc00b3b0 tx=0x7f95bc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:45:49.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:49.758+0000 7f95c97fa700 1 -- 172.21.15.107:0/851337824 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f95bc00e040 con 0x7f95cc0fce60 2024-05-17T21:45:49.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:49.758+0000 7f95c97fa700 1 -- 172.21.15.107:0/851337824 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f95bc00bcc0 con 0x7f95cc0fce60 2024-05-17T21:45:49.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:49.758+0000 7f95d284b700 1 -- 172.21.15.107:0/851337824 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95cc070a40 con 0x7f95cc0fce60 2024-05-17T21:45:49.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:49.758+0000 7f95c97fa700 1 -- 172.21.15.107:0/851337824 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f95bc01c630 con 0x7f95cc0fce60 2024-05-17T21:45:49.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:49.758+0000 7f95d284b700 1 -- 172.21.15.107:0/851337824 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95cc071000 con 0x7f95cc0fce60 2024-05-17T21:45:49.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:49.759+0000 7f95d284b700 1 -- 172.21.15.107:0/851337824 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95cc04c710 con 0x7f95cc0fce60 2024-05-17T21:45:49.756 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:49.760+0000 7f95c97fa700 1 -- 172.21.15.107:0/851337824 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 86575+0+0 (secure 0 0 0) 0x7f95bc013980 con 0x7f95cc0fce60 2024-05-17T21:45:49.756 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:49.760+0000 7f95c97fa700 1 --2- 172.21.15.107:0/851337824 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f95b8069210 0x7f95b806b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:45:49.756 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:49.760+0000 7f95c97fa700 1 -- 172.21.15.107:0/851337824 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1978+0+0 (secure 0 0 0) 0x7f95bc087050 con 0x7f95cc0fce60 2024-05-17T21:45:49.756 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:49.760+0000 7f95c3fff700 1 --2- 172.21.15.107:0/851337824 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f95b8069210 0x7f95b806b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:45:49.756 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:49.761+0000 7f95c3fff700 1 --2- 172.21.15.107:0/851337824 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f95b8069210 0x7f95b806b6c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f95b4007fd0 tx=0x7f95b4009040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:45:49.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:49.765+0000 7f95c97fa700 1 -- 172.21.15.107:0/851337824 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f95bc057c90 con 0x7f95cc0fce60 2024-05-17T21:45:49.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:49 smithi138 ceph-mon[35948]: pgmap v36: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-17T21:45:49.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:45:49.977+0000 7f95d284b700 1 -- 172.21.15.107:0/851337824 --> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f95cc102360 con 0x7f95b8069210 2024-05-17T21:45:50.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:50 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T21:45:50.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:50 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T21:45:50.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:50 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:45:50.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:50 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T21:45:50.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:50 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T21:45:50.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:50 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:45:51.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:51 smithi138 ceph-mon[35948]: from='client.14266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:45:51.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:51 smithi138 ceph-mon[35948]: pgmap v37: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-17T21:45:51.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:51 smithi107 ceph-mon[28434]: from='client.14266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:45:51.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:51 smithi107 ceph-mon[28434]: pgmap v37: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-17T21:45:53.999 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:53 smithi107 ceph-mon[28434]: pgmap v38: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-17T21:45:53.999 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:53 smithi107 ceph-mon[28434]: Detected new or changed devices on smithi107 2024-05-17T21:45:54.000 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:53 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:45:54.000 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:53 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:45:54.000 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:53 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-17T21:45:54.000 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:53 smithi107 ceph-mon[28434]: Adjusting osd_memory_target on smithi107 to 12864M 2024-05-17T21:45:54.000 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:53 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:45:54.000 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:53 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:45:54.000 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:53 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T21:45:54.000 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:53 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:45:54.000 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:53 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:45:54.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:53 smithi138 ceph-mon[35948]: pgmap v38: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-17T21:45:54.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:53 smithi138 ceph-mon[35948]: Detected new or changed devices on smithi107 2024-05-17T21:45:54.078 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:53 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:45:54.078 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:53 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:45:54.078 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:53 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-17T21:45:54.078 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:53 smithi138 ceph-mon[35948]: Adjusting osd_memory_target on smithi107 to 12864M 2024-05-17T21:45:54.078 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:53 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:45:54.078 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:53 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:45:54.078 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:53 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T21:45:54.078 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:53 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:45:54.078 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:53 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:45:54.849 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:54 smithi107 ceph-mon[28434]: from='client.? 172.21.15.107:0/496440568' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2052c88c-a0b5-4f5b-8ee0-68528e607d9e"}]: dispatch 2024-05-17T21:45:54.849 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:54 smithi107 ceph-mon[28434]: from='client.? 172.21.15.107:0/496440568' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2052c88c-a0b5-4f5b-8ee0-68528e607d9e"}]': finished 2024-05-17T21:45:54.849 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:54 smithi107 ceph-mon[28434]: osdmap e11: 2 total, 1 up, 2 in 2024-05-17T21:45:54.850 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:54 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-17T21:45:54.850 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:54 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:45:54.850 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:54 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:45:55.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:54 smithi138 ceph-mon[35948]: from='client.? 172.21.15.107:0/496440568' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2052c88c-a0b5-4f5b-8ee0-68528e607d9e"}]: dispatch 2024-05-17T21:45:55.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:54 smithi138 ceph-mon[35948]: from='client.? 172.21.15.107:0/496440568' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2052c88c-a0b5-4f5b-8ee0-68528e607d9e"}]': finished 2024-05-17T21:45:55.078 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:54 smithi138 ceph-mon[35948]: osdmap e11: 2 total, 1 up, 2 in 2024-05-17T21:45:55.078 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:54 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-17T21:45:55.078 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:54 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:45:55.078 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:54 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:45:56.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:55 smithi138 ceph-mon[35948]: pgmap v40: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-17T21:45:56.078 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:55 smithi138 ceph-mon[35948]: from='client.? 172.21.15.107:0/1534785247' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T21:45:56.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:55 smithi107 ceph-mon[28434]: pgmap v40: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-17T21:45:56.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:55 smithi107 ceph-mon[28434]: from='client.? 172.21.15.107:0/1534785247' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T21:45:58.078 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:57 smithi138 ceph-mon[35948]: pgmap v41: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-17T21:45:58.078 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:57 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:45:58.078 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:57 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:45:58.078 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:57 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:45:58.078 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:57 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T21:45:58.078 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:57 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:45:58.104 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:57 smithi107 ceph-mon[28434]: pgmap v41: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-17T21:45:58.104 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:57 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:45:58.104 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:57 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:45:58.104 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:57 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:45:58.104 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:57 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T21:45:58.104 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:57 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:46:00.077 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:45:59 smithi107 ceph-mon[28434]: pgmap v42: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-17T21:46:00.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:45:59 smithi138 ceph-mon[35948]: pgmap v42: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-17T21:46:02.038 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:01 smithi107 ceph-mon[28434]: pgmap v43: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-17T21:46:02.038 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:01 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-17T21:46:02.038 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:01 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:46:02.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:01 smithi138 ceph-mon[35948]: pgmap v43: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-17T21:46:02.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:01 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-17T21:46:02.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:01 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:46:03.069 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:02 smithi107 ceph-mon[28434]: Deploying daemon osd.1 on smithi107 2024-05-17T21:46:03.069 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:02 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:46:03.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:02 smithi138 ceph-mon[35948]: Deploying daemon osd.1 on smithi107 2024-05-17T21:46:03.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:02 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:46:04.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:03 smithi138 ceph-mon[35948]: pgmap v44: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-17T21:46:04.081 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:03 smithi107 ceph-mon[28434]: pgmap v44: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-17T21:46:06.006 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:05 smithi107 ceph-mon[28434]: pgmap v45: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-17T21:46:06.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:05 smithi138 ceph-mon[35948]: pgmap v45: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-17T21:46:07.893 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:07 smithi107 ceph-mon[28434]: pgmap v46: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-17T21:46:07.894 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:07 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:46:07.894 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:07 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:46:07.894 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:07 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:46:08.020 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:07 smithi138 ceph-mon[35948]: pgmap v46: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-17T21:46:08.020 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:07 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:46:08.021 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:07 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:46:08.021 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:07 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:46:09.591 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:09 smithi107 ceph-mon[28434]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-17T21:46:09.591 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:09 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:46:09.592 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:09 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:46:09.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:09 smithi138 ceph-mon[35948]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-17T21:46:09.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:09 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:46:09.828 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:09 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:46:10.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:46:10.349+0000 7f95c97fa700 1 -- 172.21.15.107:0/851337824 <== mgr.14214 v2:172.21.15.107:6800/1294361014 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f95cc102360 con 0x7f95b8069210 2024-05-17T21:46:10.343 INFO:teuthology.orchestra.run.smithi107.stdout:Created osd(s) 1 on host 'smithi107' 2024-05-17T21:46:10.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:46:10.351+0000 7f95d284b700 1 -- 172.21.15.107:0/851337824 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f95b8069210 msgr2=0x7f95b806b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:46:10.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:46:10.351+0000 7f95d284b700 1 --2- 172.21.15.107:0/851337824 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f95b8069210 0x7f95b806b6c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f95b4007fd0 tx=0x7f95b4009040 comp rx=0 tx=0).stop 2024-05-17T21:46:10.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:46:10.351+0000 7f95d284b700 1 -- 172.21.15.107:0/851337824 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95cc0fce60 msgr2=0x7f95cc071c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:46:10.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:46:10.351+0000 7f95d284b700 1 --2- 172.21.15.107:0/851337824 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95cc0fce60 0x7f95cc071c00 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f95bc00b3b0 tx=0x7f95bc009e30 comp rx=0 tx=0).stop 2024-05-17T21:46:10.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:46:10.351+0000 7f95d284b700 1 -- 172.21.15.107:0/851337824 shutdown_connections 2024-05-17T21:46:10.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:46:10.351+0000 7f95d284b700 1 --2- 172.21.15.107:0/851337824 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f95b8069210 0x7f95b806b6c0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:46:10.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:46:10.351+0000 7f95d284b700 1 --2- 172.21.15.107:0/851337824 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95cc0fce60 0x7f95cc071c00 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:46:10.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:46:10.351+0000 7f95d284b700 1 --2- 172.21.15.107:0/851337824 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95cc0fdfd0 0x7f95cc0702b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:46:10.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:46:10.351+0000 7f95d284b700 1 -- 172.21.15.107:0/851337824 >> 172.21.15.107:0/851337824 conn(0x7f95cc0f8540 msgr2=0x7f95cc0f9c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:46:10.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:46:10.351+0000 7f95d284b700 1 -- 172.21.15.107:0/851337824 shutdown_connections 2024-05-17T21:46:10.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:46:10.352+0000 7f95d284b700 1 -- 172.21.15.107:0/851337824 wait complete. 2024-05-17T21:46:11.474 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:11 smithi107 ceph-mon[28434]: pgmap v48: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-17T21:46:11.474 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:11 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:46:11.474 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:11 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:46:11.474 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:11 smithi107 ceph-mon[28434]: from='osd.1 [v2:172.21.15.107:6810/686958797,v1:172.21.15.107:6811/686958797]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-17T21:46:11.498 DEBUG:teuthology.orchestra.run.smithi107:osd.1> sudo journalctl -f -n 0 -u ceph-3722731a-1496-11ef-bc9a-c7b262605968@osd.1.service 2024-05-17T21:46:11.500 INFO:tasks.cephadm:Deploying osd.2 on smithi107 with /dev/vg_nvme/lv_2... 2024-05-17T21:46:11.500 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-17T21:46:11.577 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:11 smithi138 ceph-mon[35948]: pgmap v48: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-17T21:46:11.577 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:11 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:46:11.577 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:11 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:46:11.577 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:11 smithi138 ceph-mon[35948]: from='osd.1 [v2:172.21.15.107:6810/686958797,v1:172.21.15.107:6811/686958797]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-17T21:46:11.967 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/mon.smithi107/config 2024-05-17T21:46:12.698 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:12 smithi107 ceph-mon[28434]: from='osd.1 [v2:172.21.15.107:6810/686958797,v1:172.21.15.107:6811/686958797]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-17T21:46:12.698 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:12 smithi107 ceph-mon[28434]: osdmap e12: 2 total, 1 up, 2 in 2024-05-17T21:46:12.699 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:12 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-17T21:46:12.699 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:12 smithi107 ceph-mon[28434]: from='osd.1 [v2:172.21.15.107:6810/686958797,v1:172.21.15.107:6811/686958797]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-05-17T21:46:12.699 INFO:journalctl@ceph.osd.1.smithi107.stdout:May 17 21:46:12 smithi107 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-1[54544]: 2024-05-17T21:46:12.480+0000 7f658f3af700 -1 osd.1 0 waiting for initial osdmap 2024-05-17T21:46:12.699 INFO:journalctl@ceph.osd.1.smithi107.stdout:May 17 21:46:12 smithi107 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-1[54544]: 2024-05-17T21:46:12.483+0000 7f65881a6700 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-17T21:46:12.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:12 smithi138 ceph-mon[35948]: from='osd.1 [v2:172.21.15.107:6810/686958797,v1:172.21.15.107:6811/686958797]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-17T21:46:12.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:12 smithi138 ceph-mon[35948]: osdmap e12: 2 total, 1 up, 2 in 2024-05-17T21:46:12.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:12 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-17T21:46:12.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:12 smithi138 ceph-mon[35948]: from='osd.1 [v2:172.21.15.107:6810/686958797,v1:172.21.15.107:6811/686958797]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-05-17T21:46:13.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:13 smithi138 ceph-mon[35948]: pgmap v50: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-17T21:46:13.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:13 smithi138 ceph-mon[35948]: from='osd.1 [v2:172.21.15.107:6810/686958797,v1:172.21.15.107:6811/686958797]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-05-17T21:46:13.828 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:13 smithi138 ceph-mon[35948]: osdmap e13: 2 total, 1 up, 2 in 2024-05-17T21:46:13.828 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:13 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-17T21:46:13.828 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:13 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-17T21:46:13.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:13 smithi107 ceph-mon[28434]: pgmap v50: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-17T21:46:13.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:13 smithi107 ceph-mon[28434]: from='osd.1 [v2:172.21.15.107:6810/686958797,v1:172.21.15.107:6811/686958797]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-05-17T21:46:13.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:13 smithi107 ceph-mon[28434]: osdmap e13: 2 total, 1 up, 2 in 2024-05-17T21:46:13.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:13 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-17T21:46:13.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:13 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-17T21:46:14.577 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:14 smithi138 ceph-mon[35948]: purged_snaps scrub starts 2024-05-17T21:46:14.577 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:14 smithi138 ceph-mon[35948]: purged_snaps scrub ok 2024-05-17T21:46:14.577 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:14 smithi138 ceph-mon[35948]: osd.1 [v2:172.21.15.107:6810/686958797,v1:172.21.15.107:6811/686958797] boot 2024-05-17T21:46:14.577 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:14 smithi138 ceph-mon[35948]: osdmap e14: 2 total, 2 up, 2 in 2024-05-17T21:46:14.578 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:14 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-17T21:46:14.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:14 smithi107 ceph-mon[28434]: purged_snaps scrub starts 2024-05-17T21:46:14.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:14 smithi107 ceph-mon[28434]: purged_snaps scrub ok 2024-05-17T21:46:14.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:14 smithi107 ceph-mon[28434]: osd.1 [v2:172.21.15.107:6810/686958797,v1:172.21.15.107:6811/686958797] boot 2024-05-17T21:46:14.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:14 smithi107 ceph-mon[28434]: osdmap e14: 2 total, 2 up, 2 in 2024-05-17T21:46:14.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:14 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-17T21:46:15.006 INFO:teuthology.orchestra.run.smithi107.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-17T21:46:15.007 INFO:teuthology.orchestra.run.smithi107.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-17T21:46:15.007 INFO:teuthology.orchestra.run.smithi107.stderr: stderr: 10+0 records in 2024-05-17T21:46:15.007 INFO:teuthology.orchestra.run.smithi107.stderr:10+0 records out 2024-05-17T21:46:15.007 INFO:teuthology.orchestra.run.smithi107.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0143396 s, 731 MB/s 2024-05-17T21:46:15.007 INFO:teuthology.orchestra.run.smithi107.stderr: stderr: 2024-05-17T21:46:15.007 INFO:teuthology.orchestra.run.smithi107.stderr:--> Zapping successful for: 2024-05-17T21:46:15.606 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi107:vg_nvme/lv_2 2024-05-17T21:46:15.629 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:15 smithi107 ceph-mon[28434]: pgmap v53: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-17T21:46:15.630 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:15 smithi107 ceph-mon[28434]: osdmap e15: 2 total, 2 up, 2 in 2024-05-17T21:46:15.630 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:15 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:46:15.630 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:15 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:46:15.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:15 smithi138 ceph-mon[35948]: pgmap v53: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-17T21:46:15.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:15 smithi138 ceph-mon[35948]: osdmap e15: 2 total, 2 up, 2 in 2024-05-17T21:46:15.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:15 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:46:15.828 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:15 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:46:17.336 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/mon.smithi107/config 2024-05-17T21:46:17.712 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:17 smithi107 ceph-mon[28434]: pgmap v55: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-17T21:46:17.712 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:17 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:46:17.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:17 smithi138 ceph-mon[35948]: pgmap v55: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-17T21:46:17.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:17 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:46:19.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:46:19.628+0000 7f0646d99700 1 -- 172.21.15.107:0/1839513107 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06400fe090 msgr2=0x7f06400fe500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:46:19.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:46:19.628+0000 7f0646d99700 1 --2- 172.21.15.107:0/1839513107 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06400fe090 0x7f06400fe500 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f062c004940 tx=0x7f062c009e30 comp rx=0 tx=0).stop 2024-05-17T21:46:19.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:46:19.628+0000 7f0646d99700 1 -- 172.21.15.107:0/1839513107 shutdown_connections 2024-05-17T21:46:19.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:46:19.628+0000 7f0646d99700 1 --2- 172.21.15.107:0/1839513107 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06400fe090 0x7f06400fe500 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:46:19.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:46:19.628+0000 7f0646d99700 1 --2- 172.21.15.107:0/1839513107 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06400fce90 0x7f06400fd2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:46:19.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:46:19.628+0000 7f0646d99700 1 -- 172.21.15.107:0/1839513107 >> 172.21.15.107:0/1839513107 conn(0x7f06400f85c0 msgr2=0x7f06400fa9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:46:19.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:46:19.628+0000 7f0646d99700 1 -- 172.21.15.107:0/1839513107 shutdown_connections 2024-05-17T21:46:19.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:46:19.628+0000 7f0646d99700 1 -- 172.21.15.107:0/1839513107 wait complete. 2024-05-17T21:46:19.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:46:19.629+0000 7f0646d99700 1 Processor -- start 2024-05-17T21:46:19.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:46:19.629+0000 7f0646d99700 1 -- start start 2024-05-17T21:46:19.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:46:19.629+0000 7f0646d99700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06400fce90 0x7f064010da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:46:19.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:46:19.630+0000 7f0646d99700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06400fe090 0x7f064010df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:46:19.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:46:19.630+0000 7f0646d99700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f064010e580 con 0x7f06400fce90 2024-05-17T21:46:19.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:46:19.630+0000 7f0646d99700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f064010edd0 con 0x7f06400fe090 2024-05-17T21:46:19.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:46:19.630+0000 7f0644b35700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06400fce90 0x7f064010da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:46:19.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:46:19.630+0000 7f0644b35700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06400fce90 0x7f064010da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37990/0 (socket says 172.21.15.107:37990) 2024-05-17T21:46:19.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:46:19.630+0000 7f0644b35700 1 -- 172.21.15.107:0/1208264069 learned_addr learned my addr 172.21.15.107:0/1208264069 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:46:19.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:46:19.630+0000 7f063ffff700 1 --2- 172.21.15.107:0/1208264069 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06400fe090 0x7f064010df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:46:19.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:46:19.630+0000 7f0644b35700 1 -- 172.21.15.107:0/1208264069 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06400fe090 msgr2=0x7f064010df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:46:19.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:46:19.630+0000 7f0644b35700 1 --2- 172.21.15.107:0/1208264069 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06400fe090 0x7f064010df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:46:19.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:46:19.630+0000 7f0644b35700 1 -- 172.21.15.107:0/1208264069 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f062c005040 con 0x7f06400fce90 2024-05-17T21:46:19.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:46:19.630+0000 7f0644b35700 1 --2- 172.21.15.107:0/1208264069 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06400fce90 0x7f064010da40 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0634002a80 tx=0x7f0634004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:46:19.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:46:19.630+0000 7f063d7fa700 1 -- 172.21.15.107:0/1208264069 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0634007cc0 con 0x7f06400fce90 2024-05-17T21:46:19.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:46:19.630+0000 7f0646d99700 1 -- 172.21.15.107:0/1208264069 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f064010f0b0 con 0x7f06400fce90 2024-05-17T21:46:19.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:46:19.630+0000 7f063d7fa700 1 -- 172.21.15.107:0/1208264069 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f063400f040 con 0x7f06400fce90 2024-05-17T21:46:19.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:46:19.630+0000 7f063d7fa700 1 -- 172.21.15.107:0/1208264069 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f063400dad0 con 0x7f06400fce90 2024-05-17T21:46:19.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:46:19.631+0000 7f0646d99700 1 -- 172.21.15.107:0/1208264069 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f064010f5f0 con 0x7f06400fce90 2024-05-17T21:46:19.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:46:19.632+0000 7f063d7fa700 1 -- 172.21.15.107:0/1208264069 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 86575+0+0 (secure 0 0 0) 0x7f063400dc30 con 0x7f06400fce90 2024-05-17T21:46:19.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:46:19.632+0000 7f0646d99700 1 -- 172.21.15.107:0/1208264069 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f064005f5c0 con 0x7f06400fce90 2024-05-17T21:46:19.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:46:19.633+0000 7f063d7fa700 1 --2- 172.21.15.107:0/1208264069 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f0630069270 0x7f063006b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:46:19.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:46:19.633+0000 7f063d7fa700 1 -- 172.21.15.107:0/1208264069 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 2410+0+0 (secure 0 0 0) 0x7f0634086300 con 0x7f06400fce90 2024-05-17T21:46:19.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:46:19.633+0000 7f063ffff700 1 --2- 172.21.15.107:0/1208264069 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f0630069270 0x7f063006b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:46:19.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:46:19.633+0000 7f063ffff700 1 --2- 172.21.15.107:0/1208264069 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f0630069270 0x7f063006b720 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f062c000c00 tx=0x7f062c009e30 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:46:19.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:46:19.641+0000 7f063d7fa700 1 -- 172.21.15.107:0/1208264069 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f0634056c70 con 0x7f06400fce90 2024-05-17T21:46:19.839 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:19 smithi107 ceph-mon[28434]: pgmap v56: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-17T21:46:19.839 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:19 smithi107 ceph-mon[28434]: Detected new or changed devices on smithi107 2024-05-17T21:46:19.839 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:19 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:46:19.839 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:19 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:46:19.840 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:19 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-17T21:46:19.840 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:19 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-17T21:46:19.840 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:19 smithi107 ceph-mon[28434]: Adjusting osd_memory_target on smithi107 to 6432M 2024-05-17T21:46:19.840 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:19 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:46:19.840 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:19 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:46:19.840 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:19 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T21:46:19.840 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:19 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:46:19.840 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:19 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:46:19.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:46:19.845+0000 7f0646d99700 1 -- 172.21.15.107:0/1208264069 --> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f064010f980 con 0x7f0630069270 2024-05-17T21:46:20.078 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:19 smithi138 ceph-mon[35948]: pgmap v56: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-17T21:46:20.078 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:19 smithi138 ceph-mon[35948]: Detected new or changed devices on smithi107 2024-05-17T21:46:20.078 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:19 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:46:20.078 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:19 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:46:20.078 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:19 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-17T21:46:20.078 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:19 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-17T21:46:20.078 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:19 smithi138 ceph-mon[35948]: Adjusting osd_memory_target on smithi107 to 6432M 2024-05-17T21:46:20.079 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:19 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:46:20.079 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:19 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:46:20.079 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:19 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T21:46:20.079 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:19 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:46:20.079 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:19 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:46:20.821 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:20 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T21:46:20.821 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:20 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T21:46:20.821 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:20 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:46:21.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:20 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T21:46:21.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:20 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T21:46:21.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:20 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:46:21.961 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:21 smithi107 ceph-mon[28434]: from='client.14284 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:46:21.961 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:21 smithi107 ceph-mon[28434]: pgmap v57: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-17T21:46:22.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:21 smithi138 ceph-mon[35948]: from='client.14284 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:46:22.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:21 smithi138 ceph-mon[35948]: pgmap v57: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-17T21:46:23.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:23 smithi107 ceph-mon[28434]: pgmap v58: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-17T21:46:23.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:23 smithi107 ceph-mon[28434]: from='client.? 172.21.15.107:0/769929552' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b5dc6be0-2ac3-4769-ae70-14c23381506b"}]: dispatch 2024-05-17T21:46:23.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:23 smithi107 ceph-mon[28434]: from='client.? 172.21.15.107:0/769929552' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b5dc6be0-2ac3-4769-ae70-14c23381506b"}]': finished 2024-05-17T21:46:23.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:23 smithi107 ceph-mon[28434]: osdmap e16: 3 total, 2 up, 3 in 2024-05-17T21:46:23.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:23 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-17T21:46:24.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:23 smithi138 ceph-mon[35948]: pgmap v58: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-17T21:46:24.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:23 smithi138 ceph-mon[35948]: from='client.? 172.21.15.107:0/769929552' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b5dc6be0-2ac3-4769-ae70-14c23381506b"}]: dispatch 2024-05-17T21:46:24.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:23 smithi138 ceph-mon[35948]: from='client.? 172.21.15.107:0/769929552' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b5dc6be0-2ac3-4769-ae70-14c23381506b"}]': finished 2024-05-17T21:46:24.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:23 smithi138 ceph-mon[35948]: osdmap e16: 3 total, 2 up, 3 in 2024-05-17T21:46:24.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:23 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-17T21:46:25.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:24 smithi138 ceph-mon[35948]: from='client.? 172.21.15.107:0/3943000944' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T21:46:25.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:24 smithi107 ceph-mon[28434]: from='client.? 172.21.15.107:0/3943000944' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T21:46:26.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:25 smithi138 ceph-mon[35948]: pgmap v60: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-17T21:46:26.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:25 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:46:26.078 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:25 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:46:26.078 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:25 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:46:26.078 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:25 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T21:46:26.078 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:25 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:46:26.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:25 smithi107 ceph-mon[28434]: pgmap v60: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-17T21:46:26.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:25 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:46:26.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:25 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:46:26.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:25 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:46:26.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:25 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T21:46:26.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:25 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:46:27.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:27 smithi138 ceph-mon[35948]: pgmap v61: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-17T21:46:28.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:27 smithi107 ceph-mon[28434]: pgmap v61: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-17T21:46:29.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:29 smithi107 ceph-mon[28434]: pgmap v62: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-17T21:46:30.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:29 smithi138 ceph-mon[35948]: pgmap v62: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-17T21:46:30.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:30 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-17T21:46:30.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:30 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:46:31.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:30 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-17T21:46:31.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:30 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:46:31.960 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:31 smithi107 ceph-mon[28434]: pgmap v63: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-17T21:46:31.961 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:31 smithi107 ceph-mon[28434]: Deploying daemon osd.2 on smithi107 2024-05-17T21:46:32.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:31 smithi138 ceph-mon[35948]: pgmap v63: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-17T21:46:32.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:31 smithi138 ceph-mon[35948]: Deploying daemon osd.2 on smithi107 2024-05-17T21:46:32.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:32 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:46:32.979 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:32 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:46:33.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:33 smithi138 ceph-mon[35948]: pgmap v64: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-17T21:46:33.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:33 smithi107 ceph-mon[28434]: pgmap v64: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-17T21:46:36.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:35 smithi138 ceph-mon[35948]: pgmap v65: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-17T21:46:36.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:35 smithi107 ceph-mon[28434]: pgmap v65: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-17T21:46:37.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:36 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:46:37.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:36 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:46:37.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:36 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:46:37.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:36 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:46:37.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:36 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:46:37.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:36 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:46:38.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:37 smithi138 ceph-mon[35948]: pgmap v66: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-17T21:46:38.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:37 smithi107 ceph-mon[28434]: pgmap v66: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-17T21:46:39.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:46:39.063+0000 7f063d7fa700 1 -- 172.21.15.107:0/1208264069 <== mgr.14214 v2:172.21.15.107:6800/1294361014 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f064010f980 con 0x7f0630069270 2024-05-17T21:46:39.058 INFO:teuthology.orchestra.run.smithi107.stdout:Created osd(s) 2 on host 'smithi107' 2024-05-17T21:46:39.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:46:39.066+0000 7f0646d99700 1 -- 172.21.15.107:0/1208264069 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f0630069270 msgr2=0x7f063006b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:46:39.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:46:39.066+0000 7f0646d99700 1 --2- 172.21.15.107:0/1208264069 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f0630069270 0x7f063006b720 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f062c000c00 tx=0x7f062c009e30 comp rx=0 tx=0).stop 2024-05-17T21:46:39.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:46:39.066+0000 7f0646d99700 1 -- 172.21.15.107:0/1208264069 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06400fce90 msgr2=0x7f064010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:46:39.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:46:39.066+0000 7f0646d99700 1 --2- 172.21.15.107:0/1208264069 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06400fce90 0x7f064010da40 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0634002a80 tx=0x7f0634004600 comp rx=0 tx=0).stop 2024-05-17T21:46:39.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:46:39.066+0000 7f0646d99700 1 -- 172.21.15.107:0/1208264069 shutdown_connections 2024-05-17T21:46:39.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:46:39.066+0000 7f0646d99700 1 --2- 172.21.15.107:0/1208264069 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f0630069270 0x7f063006b720 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:46:39.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:46:39.066+0000 7f0646d99700 1 --2- 172.21.15.107:0/1208264069 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06400fce90 0x7f064010da40 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:46:39.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:46:39.066+0000 7f0646d99700 1 --2- 172.21.15.107:0/1208264069 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06400fe090 0x7f064010df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:46:39.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:46:39.066+0000 7f0646d99700 1 -- 172.21.15.107:0/1208264069 >> 172.21.15.107:0/1208264069 conn(0x7f06400f85c0 msgr2=0x7f06400f9cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:46:39.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:46:39.066+0000 7f0646d99700 1 -- 172.21.15.107:0/1208264069 shutdown_connections 2024-05-17T21:46:39.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:46:39.066+0000 7f0646d99700 1 -- 172.21.15.107:0/1208264069 wait complete. 2024-05-17T21:46:39.690 DEBUG:teuthology.orchestra.run.smithi107:osd.2> sudo journalctl -f -n 0 -u ceph-3722731a-1496-11ef-bc9a-c7b262605968@osd.2.service 2024-05-17T21:46:39.732 INFO:tasks.cephadm:Deploying osd.3 on smithi138 with /dev/vg_nvme/lv_4... 2024-05-17T21:46:39.732 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-17T21:46:39.992 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/mon.smithi138/config 2024-05-17T21:46:40.007 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:39 smithi107 ceph-mon[28434]: pgmap v67: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-17T21:46:40.007 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:39 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:46:40.007 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:39 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:46:40.007 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:39 smithi107 ceph-mon[28434]: from='osd.2 [v2:172.21.15.107:6818/3962264726,v1:172.21.15.107:6819/3962264726]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-17T21:46:40.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:39 smithi138 ceph-mon[35948]: pgmap v67: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-17T21:46:40.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:39 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:46:40.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:39 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:46:40.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:39 smithi138 ceph-mon[35948]: from='osd.2 [v2:172.21.15.107:6818/3962264726,v1:172.21.15.107:6819/3962264726]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-17T21:46:41.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:41 smithi138 ceph-mon[35948]: from='osd.2 [v2:172.21.15.107:6818/3962264726,v1:172.21.15.107:6819/3962264726]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-17T21:46:41.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:41 smithi138 ceph-mon[35948]: osdmap e17: 3 total, 2 up, 3 in 2024-05-17T21:46:41.078 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:41 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-17T21:46:41.078 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:41 smithi138 ceph-mon[35948]: from='osd.2 [v2:172.21.15.107:6818/3962264726,v1:172.21.15.107:6819/3962264726]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-05-17T21:46:41.078 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:41 smithi138 ceph-mon[35948]: pgmap v69: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-17T21:46:41.353 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:41 smithi107 ceph-mon[28434]: from='osd.2 [v2:172.21.15.107:6818/3962264726,v1:172.21.15.107:6819/3962264726]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-17T21:46:41.353 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:41 smithi107 ceph-mon[28434]: osdmap e17: 3 total, 2 up, 3 in 2024-05-17T21:46:41.353 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:41 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-17T21:46:41.353 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:41 smithi107 ceph-mon[28434]: from='osd.2 [v2:172.21.15.107:6818/3962264726,v1:172.21.15.107:6819/3962264726]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-05-17T21:46:41.353 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:41 smithi107 ceph-mon[28434]: pgmap v69: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-17T21:46:41.354 INFO:journalctl@ceph.osd.2.smithi107.stdout:May 17 21:46:41 smithi107 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-2[61810]: 2024-05-17T21:46:41.062+0000 7f81a8bbe700 -1 osd.2 0 waiting for initial osdmap 2024-05-17T21:46:41.354 INFO:journalctl@ceph.osd.2.smithi107.stdout:May 17 21:46:41 smithi107 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-2[61810]: 2024-05-17T21:46:41.071+0000 7f81a59bd700 -1 osd.2 18 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-17T21:46:41.946 INFO:teuthology.orchestra.run.smithi138.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-17T21:46:41.947 INFO:teuthology.orchestra.run.smithi138.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-17T21:46:41.947 INFO:teuthology.orchestra.run.smithi138.stderr: stderr: 10+0 records in 2024-05-17T21:46:41.947 INFO:teuthology.orchestra.run.smithi138.stderr:10+0 records out 2024-05-17T21:46:41.947 INFO:teuthology.orchestra.run.smithi138.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0124339 s, 843 MB/s 2024-05-17T21:46:41.947 INFO:teuthology.orchestra.run.smithi138.stderr:--> Zapping successful for: 2024-05-17T21:46:42.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:42 smithi138 ceph-mon[35948]: from='osd.2 [v2:172.21.15.107:6818/3962264726,v1:172.21.15.107:6819/3962264726]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-05-17T21:46:42.328 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:42 smithi138 ceph-mon[35948]: osdmap e18: 3 total, 2 up, 3 in 2024-05-17T21:46:42.328 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:42 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-17T21:46:42.328 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:42 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-17T21:46:42.350 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:42 smithi107 ceph-mon[28434]: from='osd.2 [v2:172.21.15.107:6818/3962264726,v1:172.21.15.107:6819/3962264726]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-05-17T21:46:42.350 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:42 smithi107 ceph-mon[28434]: osdmap e18: 3 total, 2 up, 3 in 2024-05-17T21:46:42.350 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:42 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-17T21:46:42.350 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:42 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-17T21:46:42.630 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi138:vg_nvme/lv_4 2024-05-17T21:46:42.885 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/mon.smithi138/config 2024-05-17T21:46:43.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:43 smithi138 ceph-mon[35948]: purged_snaps scrub starts 2024-05-17T21:46:43.328 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:43 smithi138 ceph-mon[35948]: purged_snaps scrub ok 2024-05-17T21:46:43.328 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:43 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-17T21:46:43.328 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:43 smithi138 ceph-mon[35948]: osd.2 [v2:172.21.15.107:6818/3962264726,v1:172.21.15.107:6819/3962264726] boot 2024-05-17T21:46:43.328 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:43 smithi138 ceph-mon[35948]: osdmap e19: 3 total, 3 up, 3 in 2024-05-17T21:46:43.328 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:43 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-17T21:46:43.328 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:43 smithi138 ceph-mon[35948]: pgmap v72: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-17T21:46:43.328 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:43 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-05-17T21:46:43.328 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:43 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:46:43.328 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:43 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:46:43.353 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:43 smithi107 ceph-mon[28434]: purged_snaps scrub starts 2024-05-17T21:46:43.354 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:43 smithi107 ceph-mon[28434]: purged_snaps scrub ok 2024-05-17T21:46:43.354 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:43 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-17T21:46:43.354 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:43 smithi107 ceph-mon[28434]: osd.2 [v2:172.21.15.107:6818/3962264726,v1:172.21.15.107:6819/3962264726] boot 2024-05-17T21:46:43.354 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:43 smithi107 ceph-mon[28434]: osdmap e19: 3 total, 3 up, 3 in 2024-05-17T21:46:43.354 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:43 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-17T21:46:43.354 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:43 smithi107 ceph-mon[28434]: pgmap v72: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-17T21:46:43.354 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:43 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-05-17T21:46:43.354 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:43 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:46:43.354 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:43 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:46:44.114 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:44 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-05-17T21:46:44.115 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:44 smithi138 ceph-mon[35948]: osdmap e20: 3 total, 3 up, 3 in 2024-05-17T21:46:44.115 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:44 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-17T21:46:44.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:46:44.330+0000 7f23e263e700 1 -- 172.21.15.138:0/2103262908 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23dc0fdfd0 msgr2=0x7f23dc0fe440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:46:44.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:46:44.330+0000 7f23e263e700 1 --2- 172.21.15.138:0/2103262908 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23dc0fdfd0 0x7f23dc0fe440 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f23d0004940 tx=0x7f23d0009e30 comp rx=0 tx=0).stop 2024-05-17T21:46:44.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:46:44.330+0000 7f23e263e700 1 -- 172.21.15.138:0/2103262908 shutdown_connections 2024-05-17T21:46:44.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:46:44.330+0000 7f23e263e700 1 --2- 172.21.15.138:0/2103262908 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23dc0fdfd0 0x7f23dc0fe440 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:46:44.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:46:44.330+0000 7f23e263e700 1 --2- 172.21.15.138:0/2103262908 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23dc0fce60 0x7f23dc0fd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:46:44.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:46:44.330+0000 7f23e263e700 1 -- 172.21.15.138:0/2103262908 >> 172.21.15.138:0/2103262908 conn(0x7f23dc0f8540 msgr2=0x7f23dc0fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:46:44.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:46:44.330+0000 7f23e263e700 1 -- 172.21.15.138:0/2103262908 shutdown_connections 2024-05-17T21:46:44.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:46:44.330+0000 7f23e263e700 1 -- 172.21.15.138:0/2103262908 wait complete. 2024-05-17T21:46:44.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:46:44.330+0000 7f23e263e700 1 Processor -- start 2024-05-17T21:46:44.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:46:44.331+0000 7f23e263e700 1 -- start start 2024-05-17T21:46:44.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:46:44.331+0000 7f23e263e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23dc0fce60 0x7f23dc06fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:46:44.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:46:44.331+0000 7f23e263e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23dc0fdfd0 0x7f23dc06e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:46:44.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:46:44.331+0000 7f23e263e700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23dc06e980 con 0x7f23dc0fce60 2024-05-17T21:46:44.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:46:44.331+0000 7f23e263e700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23dc06eaf0 con 0x7f23dc0fdfd0 2024-05-17T21:46:44.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:46:44.331+0000 7f23db7fe700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23dc0fdfd0 0x7f23dc06e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:46:44.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:46:44.331+0000 7f23db7fe700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23dc0fdfd0 0x7f23dc06e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:54216/0 (socket says 172.21.15.138:54216) 2024-05-17T21:46:44.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:46:44.331+0000 7f23db7fe700 1 -- 172.21.15.138:0/3755931474 learned_addr learned my addr 172.21.15.138:0/3755931474 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-17T21:46:44.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:46:44.331+0000 7f23dbfff700 1 --2- 172.21.15.138:0/3755931474 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23dc0fce60 0x7f23dc06fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:46:44.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:46:44.331+0000 7f23db7fe700 1 -- 172.21.15.138:0/3755931474 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23dc0fce60 msgr2=0x7f23dc06fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:46:44.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:46:44.331+0000 7f23db7fe700 1 --2- 172.21.15.138:0/3755931474 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23dc0fce60 0x7f23dc06fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:46:44.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:46:44.331+0000 7f23db7fe700 1 -- 172.21.15.138:0/3755931474 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23d0005040 con 0x7f23dc0fdfd0 2024-05-17T21:46:44.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:46:44.331+0000 7f23dbfff700 1 --2- 172.21.15.138:0/3755931474 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23dc0fce60 0x7f23dc06fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T21:46:44.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:46:44.331+0000 7f23db7fe700 1 --2- 172.21.15.138:0/3755931474 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23dc0fdfd0 0x7f23dc06e330 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f23d000b3b0 tx=0x7f23d0009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:46:44.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:46:44.331+0000 7f23d8ff9700 1 -- 172.21.15.138:0/3755931474 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f23d000e040 con 0x7f23dc0fdfd0 2024-05-17T21:46:44.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:46:44.332+0000 7f23d8ff9700 1 -- 172.21.15.138:0/3755931474 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f23d0004290 con 0x7f23dc0fdfd0 2024-05-17T21:46:44.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:46:44.332+0000 7f23e263e700 1 -- 172.21.15.138:0/3755931474 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23dc06ed70 con 0x7f23dc0fdfd0 2024-05-17T21:46:44.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:46:44.332+0000 7f23d8ff9700 1 -- 172.21.15.138:0/3755931474 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f23d001c630 con 0x7f23dc0fdfd0 2024-05-17T21:46:44.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:46:44.332+0000 7f23e263e700 1 -- 172.21.15.138:0/3755931474 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23dc11bdb0 con 0x7f23dc0fdfd0 2024-05-17T21:46:44.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:46:44.332+0000 7f23e263e700 1 -- 172.21.15.138:0/3755931474 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23dc04c710 con 0x7f23dc0fdfd0 2024-05-17T21:46:44.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:46:44.333+0000 7f23d8ff9700 1 -- 172.21.15.138:0/3755931474 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 18) v1 ==== 86575+0+0 (secure 0 0 0) 0x7f23d000bbf0 con 0x7f23dc0fdfd0 2024-05-17T21:46:44.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:46:44.333+0000 7f23d8ff9700 1 --2- 172.21.15.138:0/3755931474 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f23c4069210 0x7f23c406b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:46:44.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:46:44.333+0000 7f23d8ff9700 1 -- 172.21.15.138:0/3755931474 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 3228+0+0 (secure 0 0 0) 0x7f23d0087c30 con 0x7f23dc0fdfd0 2024-05-17T21:46:44.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:46:44.334+0000 7f23dbfff700 1 --2- 172.21.15.138:0/3755931474 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f23c4069210 0x7f23c406b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:46:44.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:46:44.334+0000 7f23dbfff700 1 --2- 172.21.15.138:0/3755931474 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f23c4069210 0x7f23c406b6c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f23cc009b10 tx=0x7f23cc008040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:46:44.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:46:44.338+0000 7f23d8ff9700 1 -- 172.21.15.138:0/3755931474 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f23d0054900 con 0x7f23dc0fdfd0 2024-05-17T21:46:44.342 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:44 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-05-17T21:46:44.343 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:44 smithi107 ceph-mon[28434]: osdmap e20: 3 total, 3 up, 3 in 2024-05-17T21:46:44.343 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:44 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-17T21:46:44.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:46:44.541+0000 7f23e263e700 1 -- 172.21.15.138:0/3755931474 --> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f23dc102270 con 0x7f23c4069210 2024-05-17T21:46:45.232 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:45 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-17T21:46:45.232 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:45 smithi138 ceph-mon[35948]: osdmap e21: 3 total, 3 up, 3 in 2024-05-17T21:46:45.232 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:45 smithi138 ceph-mon[35948]: pgmap v75: 1 pgs: 1 creating+peering; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-05-17T21:46:45.232 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:45 smithi138 ceph-mon[35948]: from='client.24125 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:46:45.232 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:45 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T21:46:45.232 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:45 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T21:46:45.232 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:45 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:46:45.328 INFO:journalctl@ceph.osd.0.smithi107.stdout:May 17 21:46:45 smithi107 sudo[63683]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-17T21:46:45.328 INFO:journalctl@ceph.osd.0.smithi107.stdout:May 17 21:46:45 smithi107 sudo[63683]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-17T21:46:45.328 INFO:journalctl@ceph.osd.0.smithi107.stdout:May 17 21:46:45 smithi107 sudo[63683]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-17T21:46:45.328 INFO:journalctl@ceph.osd.0.smithi107.stdout:May 17 21:46:45 smithi107 sudo[63683]: pam_unix(sudo:session): session closed for user root 2024-05-17T21:46:45.328 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:45 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-17T21:46:45.328 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:45 smithi107 ceph-mon[28434]: osdmap e21: 3 total, 3 up, 3 in 2024-05-17T21:46:45.329 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:45 smithi107 ceph-mon[28434]: pgmap v75: 1 pgs: 1 creating+peering; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-05-17T21:46:45.329 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:45 smithi107 ceph-mon[28434]: from='client.24125 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:46:45.329 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:45 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T21:46:45.329 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:45 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T21:46:45.329 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:45 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:46:45.600 INFO:journalctl@ceph.osd.0.smithi107.stdout:May 17 21:46:45 smithi107 sudo[66329]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-17T21:46:45.600 INFO:journalctl@ceph.osd.0.smithi107.stdout:May 17 21:46:45 smithi107 sudo[66329]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-17T21:46:45.600 INFO:journalctl@ceph.osd.0.smithi107.stdout:May 17 21:46:45 smithi107 sudo[66329]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-17T21:46:45.600 INFO:journalctl@ceph.osd.0.smithi107.stdout:May 17 21:46:45 smithi107 sudo[66329]: pam_unix(sudo:session): session closed for user root 2024-05-17T21:46:45.853 INFO:journalctl@ceph.osd.1.smithi107.stdout:May 17 21:46:45 smithi107 sudo[66344]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-17T21:46:45.853 INFO:journalctl@ceph.osd.1.smithi107.stdout:May 17 21:46:45 smithi107 sudo[66344]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-17T21:46:45.853 INFO:journalctl@ceph.osd.1.smithi107.stdout:May 17 21:46:45 smithi107 sudo[66344]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-17T21:46:45.853 INFO:journalctl@ceph.osd.1.smithi107.stdout:May 17 21:46:45 smithi107 sudo[66344]: pam_unix(sudo:session): session closed for user root 2024-05-17T21:46:46.108 INFO:journalctl@ceph.osd.1.smithi107.stdout:May 17 21:46:45 smithi107 sudo[66349]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-17T21:46:46.108 INFO:journalctl@ceph.osd.1.smithi107.stdout:May 17 21:46:45 smithi107 sudo[66349]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-17T21:46:46.108 INFO:journalctl@ceph.osd.1.smithi107.stdout:May 17 21:46:45 smithi107 sudo[66349]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-17T21:46:46.108 INFO:journalctl@ceph.osd.1.smithi107.stdout:May 17 21:46:45 smithi107 sudo[66349]: pam_unix(sudo:session): session closed for user root 2024-05-17T21:46:46.108 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:46 smithi107 ceph-mon[28434]: osdmap e22: 3 total, 3 up, 3 in 2024-05-17T21:46:46.108 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:46 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:46:46.108 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:46 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:46:46.109 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:46 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:46:46.109 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:46 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-17T21:46:46.109 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:46 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-17T21:46:46.109 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:46 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-17T21:46:46.109 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:46 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:46:46.109 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:46 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:46:46.109 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:46 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T21:46:46.109 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:46 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:46:46.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:46 smithi138 ceph-mon[35948]: osdmap e22: 3 total, 3 up, 3 in 2024-05-17T21:46:46.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:46 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:46:46.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:46 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:46:46.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:46 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:46:46.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:46 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-17T21:46:46.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:46 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-17T21:46:46.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:46 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-17T21:46:46.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:46 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:46:46.137 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:46 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:46:46.137 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:46 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T21:46:46.137 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:46 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:46:46.371 INFO:journalctl@ceph.osd.2.smithi107.stdout:May 17 21:46:46 smithi107 sudo[66384]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-17T21:46:46.371 INFO:journalctl@ceph.osd.2.smithi107.stdout:May 17 21:46:46 smithi107 sudo[66384]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-17T21:46:46.371 INFO:journalctl@ceph.osd.2.smithi107.stdout:May 17 21:46:46 smithi107 sudo[66384]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-17T21:46:46.371 INFO:journalctl@ceph.osd.2.smithi107.stdout:May 17 21:46:46 smithi107 sudo[66384]: pam_unix(sudo:session): session closed for user root 2024-05-17T21:46:46.628 INFO:journalctl@ceph.osd.2.smithi107.stdout:May 17 21:46:46 smithi107 sudo[66453]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-17T21:46:46.628 INFO:journalctl@ceph.osd.2.smithi107.stdout:May 17 21:46:46 smithi107 sudo[66453]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-17T21:46:46.628 INFO:journalctl@ceph.osd.2.smithi107.stdout:May 17 21:46:46 smithi107 sudo[66453]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-17T21:46:46.629 INFO:journalctl@ceph.osd.2.smithi107.stdout:May 17 21:46:46 smithi107 sudo[66453]: pam_unix(sudo:session): session closed for user root 2024-05-17T21:46:46.889 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:46 smithi107 sudo[66541]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-17T21:46:46.890 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:46 smithi107 sudo[66541]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-17T21:46:46.890 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:46 smithi107 sudo[66541]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-17T21:46:46.890 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:46 smithi107 sudo[66541]: pam_unix(sudo:session): session closed for user root 2024-05-17T21:46:47.190 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:46 smithi107 sudo[66550]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-17T21:46:47.190 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:46 smithi107 sudo[66550]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-17T21:46:47.190 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:46 smithi107 sudo[66550]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-17T21:46:47.190 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:46 smithi107 sudo[66550]: pam_unix(sudo:session): session closed for user root 2024-05-17T21:46:47.388 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:47 smithi138 sudo[39402]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-17T21:46:47.388 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:47 smithi138 sudo[39402]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-17T21:46:47.388 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:47 smithi138 sudo[39402]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-17T21:46:47.388 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:47 smithi138 sudo[39402]: pam_unix(sudo:session): session closed for user root 2024-05-17T21:46:47.401 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:46:47.408+0000 7f23d8ff9700 1 -- 172.21.15.138:0/3755931474 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f23d0054cd0 con 0x7f23dc0fdfd0 2024-05-17T21:46:47.548 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:47 smithi107 ceph-mon[28434]: Detected new or changed devices on smithi107 2024-05-17T21:46:47.548 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:47 smithi107 ceph-mon[28434]: Adjusting osd_memory_target on smithi107 to 4288M 2024-05-17T21:46:47.548 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:47 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:46:47.548 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:47 smithi107 ceph-mon[28434]: pgmap v77: 1 pgs: 1 creating+peering; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-05-17T21:46:47.548 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:47 smithi107 ceph-mon[28434]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-17T21:46:47.548 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:47 smithi107 ceph-mon[28434]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-17T21:46:47.549 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:47 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mon metadata", "id": "smithi107"}]: dispatch 2024-05-17T21:46:47.549 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:47 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mon metadata", "id": "smithi138"}]: dispatch 2024-05-17T21:46:47.549 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:47 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mon metadata", "id": "smithi107"}]: dispatch 2024-05-17T21:46:47.549 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:47 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mon metadata", "id": "smithi138"}]: dispatch 2024-05-17T21:46:47.727 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:47 smithi138 sudo[39405]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-17T21:46:47.727 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:47 smithi138 sudo[39405]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-17T21:46:47.727 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:47 smithi138 sudo[39405]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-17T21:46:47.727 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:47 smithi138 sudo[39405]: pam_unix(sudo:session): session closed for user root 2024-05-17T21:46:47.728 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:47 smithi138 ceph-mon[35948]: Detected new or changed devices on smithi107 2024-05-17T21:46:47.728 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:47 smithi138 ceph-mon[35948]: Adjusting osd_memory_target on smithi107 to 4288M 2024-05-17T21:46:47.728 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:47 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:46:47.728 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:47 smithi138 ceph-mon[35948]: pgmap v77: 1 pgs: 1 creating+peering; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-05-17T21:46:47.728 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:47 smithi138 ceph-mon[35948]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-17T21:46:47.728 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:47 smithi138 ceph-mon[35948]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-17T21:46:47.728 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:47 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mon metadata", "id": "smithi107"}]: dispatch 2024-05-17T21:46:47.728 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:47 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mon metadata", "id": "smithi138"}]: dispatch 2024-05-17T21:46:47.728 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:47 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mon metadata", "id": "smithi107"}]: dispatch 2024-05-17T21:46:47.728 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:47 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mon metadata", "id": "smithi138"}]: dispatch 2024-05-17T21:46:48.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:48 smithi107 ceph-mon[28434]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-17T21:46:48.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:48 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:46:48.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:48 smithi107 ceph-mon[28434]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-17T21:46:48.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:48 smithi107 ceph-mon[28434]: from='client.? 172.21.15.138:0/959310742' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5f3d4515-afed-481f-8e08-debaf33a9e0b"}]: dispatch 2024-05-17T21:46:48.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:48 smithi107 ceph-mon[28434]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5f3d4515-afed-481f-8e08-debaf33a9e0b"}]: dispatch 2024-05-17T21:46:48.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:48 smithi107 ceph-mon[28434]: mgrmap e19: smithi107.xhhyiw(active, since 2m), standbys: smithi138.napnll 2024-05-17T21:46:48.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:48 smithi107 ceph-mon[28434]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5f3d4515-afed-481f-8e08-debaf33a9e0b"}]': finished 2024-05-17T21:46:48.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:48 smithi107 ceph-mon[28434]: osdmap e23: 4 total, 3 up, 4 in 2024-05-17T21:46:48.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:48 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-17T21:46:48.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:48 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:46:48.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:48 smithi107 ceph-mon[28434]: from='client.? 172.21.15.138:0/3659850022' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T21:46:48.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:48 smithi138 ceph-mon[35948]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-17T21:46:48.828 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:48 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:46:48.828 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:48 smithi138 ceph-mon[35948]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-17T21:46:48.828 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:48 smithi138 ceph-mon[35948]: from='client.? 172.21.15.138:0/959310742' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5f3d4515-afed-481f-8e08-debaf33a9e0b"}]: dispatch 2024-05-17T21:46:48.828 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:48 smithi138 ceph-mon[35948]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5f3d4515-afed-481f-8e08-debaf33a9e0b"}]: dispatch 2024-05-17T21:46:48.828 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:48 smithi138 ceph-mon[35948]: mgrmap e19: smithi107.xhhyiw(active, since 2m), standbys: smithi138.napnll 2024-05-17T21:46:48.828 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:48 smithi138 ceph-mon[35948]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5f3d4515-afed-481f-8e08-debaf33a9e0b"}]': finished 2024-05-17T21:46:48.828 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:48 smithi138 ceph-mon[35948]: osdmap e23: 4 total, 3 up, 4 in 2024-05-17T21:46:48.828 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:48 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-17T21:46:48.828 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:48 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:46:48.828 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:48 smithi138 ceph-mon[35948]: from='client.? 172.21.15.138:0/3659850022' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T21:46:49.774 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:49 smithi107 ceph-mon[28434]: pgmap v79: 1 pgs: 1 creating+peering; 0 B data, 873 MiB used, 267 GiB / 268 GiB avail 2024-05-17T21:46:49.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:49 smithi138 ceph-mon[35948]: pgmap v79: 1 pgs: 1 creating+peering; 0 B data, 873 MiB used, 267 GiB / 268 GiB avail 2024-05-17T21:46:51.577 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:51 smithi138 ceph-mon[35948]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-05-17T21:46:51.577 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:51 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:46:51.577 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:51 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:46:51.577 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:51 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:46:51.577 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:51 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T21:46:51.578 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:51 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:46:51.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:51 smithi107 ceph-mon[28434]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-05-17T21:46:51.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:51 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:46:51.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:51 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:46:51.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:51 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:46:51.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:51 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T21:46:51.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:51 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:46:53.499 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:53 smithi138 ceph-mon[35948]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-05-17T21:46:53.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:53 smithi107 ceph-mon[28434]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-05-17T21:46:55.577 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:55 smithi138 ceph-mon[35948]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-05-17T21:46:55.577 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:55 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-17T21:46:55.577 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:55 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:46:55.577 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:55 smithi138 ceph-mon[35948]: Deploying daemon osd.3 on smithi138 2024-05-17T21:46:55.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:55 smithi107 ceph-mon[28434]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-05-17T21:46:55.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:55 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-17T21:46:55.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:55 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:46:55.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:55 smithi107 ceph-mon[28434]: Deploying daemon osd.3 on smithi138 2024-05-17T21:46:57.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:57 smithi107 ceph-mon[28434]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-05-17T21:46:57.752 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:57 smithi138 ceph-mon[35948]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-05-17T21:46:59.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:46:59 smithi138 ceph-mon[35948]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-05-17T21:46:59.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:46:59 smithi107 ceph-mon[28434]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-05-17T21:47:01.037 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:00 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:47:01.037 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:00 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:47:01.037 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:00 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:47:01.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:00 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:47:01.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:00 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:47:01.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:00 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:47:02.009 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:01 smithi138 ceph-mon[35948]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-05-17T21:47:02.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:01 smithi107 ceph-mon[28434]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-05-17T21:47:02.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:02 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:47:03.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:02 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:47:03.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:47:03.454+0000 7f23d8ff9700 1 -- 172.21.15.138:0/3755931474 <== mgr.14214 v2:172.21.15.107:6800/1294361014 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f23dc102270 con 0x7f23c4069210 2024-05-17T21:47:03.449 INFO:teuthology.orchestra.run.smithi138.stdout:Created osd(s) 3 on host 'smithi138' 2024-05-17T21:47:03.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:47:03.456+0000 7f23e263e700 1 -- 172.21.15.138:0/3755931474 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f23c4069210 msgr2=0x7f23c406b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:47:03.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:47:03.456+0000 7f23e263e700 1 --2- 172.21.15.138:0/3755931474 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f23c4069210 0x7f23c406b6c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f23cc009b10 tx=0x7f23cc008040 comp rx=0 tx=0).stop 2024-05-17T21:47:03.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:47:03.457+0000 7f23e263e700 1 -- 172.21.15.138:0/3755931474 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23dc0fdfd0 msgr2=0x7f23dc06e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:47:03.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:47:03.457+0000 7f23e263e700 1 --2- 172.21.15.138:0/3755931474 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23dc0fdfd0 0x7f23dc06e330 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f23d000b3b0 tx=0x7f23d0009e30 comp rx=0 tx=0).stop 2024-05-17T21:47:03.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:47:03.457+0000 7f23e263e700 1 -- 172.21.15.138:0/3755931474 shutdown_connections 2024-05-17T21:47:03.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:47:03.457+0000 7f23e263e700 1 --2- 172.21.15.138:0/3755931474 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f23c4069210 0x7f23c406b6c0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:47:03.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:47:03.457+0000 7f23e263e700 1 --2- 172.21.15.138:0/3755931474 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23dc0fce60 0x7f23dc06fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:47:03.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:47:03.457+0000 7f23e263e700 1 --2- 172.21.15.138:0/3755931474 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23dc0fdfd0 0x7f23dc06e330 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:47:03.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:47:03.457+0000 7f23e263e700 1 -- 172.21.15.138:0/3755931474 >> 172.21.15.138:0/3755931474 conn(0x7f23dc0f8540 msgr2=0x7f23dc0f9b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:47:03.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:47:03.457+0000 7f23e263e700 1 -- 172.21.15.138:0/3755931474 shutdown_connections 2024-05-17T21:47:03.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:47:03.457+0000 7f23e263e700 1 -- 172.21.15.138:0/3755931474 wait complete. 2024-05-17T21:47:03.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:03 smithi138 ceph-mon[35948]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-05-17T21:47:03.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:03 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:47:03.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:03 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:47:03.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:03 smithi138 ceph-mon[35948]: from='osd.3 [v2:172.21.15.138:6800/2477238048,v1:172.21.15.138:6801/2477238048]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-17T21:47:03.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:03 smithi138 ceph-mon[35948]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-17T21:47:04.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:03 smithi107 ceph-mon[28434]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-05-17T21:47:04.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:03 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:47:04.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:03 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:47:04.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:03 smithi107 ceph-mon[28434]: from='osd.3 [v2:172.21.15.138:6800/2477238048,v1:172.21.15.138:6801/2477238048]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-17T21:47:04.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:03 smithi107 ceph-mon[28434]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-17T21:47:04.382 DEBUG:teuthology.orchestra.run.smithi138:osd.3> sudo journalctl -f -n 0 -u ceph-3722731a-1496-11ef-bc9a-c7b262605968@osd.3.service 2024-05-17T21:47:04.384 INFO:tasks.cephadm:Deploying osd.4 on smithi138 with /dev/vg_nvme/lv_3... 2024-05-17T21:47:04.385 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-17T21:47:04.686 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/mon.smithi138/config 2024-05-17T21:47:05.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:05 smithi138 ceph-mon[35948]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-05-17T21:47:05.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:05 smithi138 ceph-mon[35948]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-17T21:47:05.828 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:05 smithi138 ceph-mon[35948]: osdmap e24: 4 total, 3 up, 4 in 2024-05-17T21:47:05.828 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:05 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-17T21:47:05.828 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:05 smithi138 ceph-mon[35948]: from='osd.3 [v2:172.21.15.138:6800/2477238048,v1:172.21.15.138:6801/2477238048]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-05-17T21:47:05.828 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:05 smithi138 ceph-mon[35948]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-05-17T21:47:05.828 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:05 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:47:05.828 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:05 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:47:05.828 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 17 21:47:05 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-3[41471]: 2024-05-17T21:47:05.450+0000 7fbc8ea13700 -1 osd.3 0 waiting for initial osdmap 2024-05-17T21:47:05.828 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 17 21:47:05 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-3[41471]: 2024-05-17T21:47:05.457+0000 7fbc8980e700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-17T21:47:05.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:05 smithi107 ceph-mon[28434]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-05-17T21:47:05.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:05 smithi107 ceph-mon[28434]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-17T21:47:05.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:05 smithi107 ceph-mon[28434]: osdmap e24: 4 total, 3 up, 4 in 2024-05-17T21:47:05.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:05 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-17T21:47:05.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:05 smithi107 ceph-mon[28434]: from='osd.3 [v2:172.21.15.138:6800/2477238048,v1:172.21.15.138:6801/2477238048]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-05-17T21:47:05.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:05 smithi107 ceph-mon[28434]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-05-17T21:47:05.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:05 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:47:05.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:05 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:47:06.723 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:06 smithi138 ceph-mon[35948]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-05-17T21:47:06.723 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:06 smithi138 ceph-mon[35948]: osdmap e25: 4 total, 3 up, 4 in 2024-05-17T21:47:06.724 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:06 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-17T21:47:06.724 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:06 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-17T21:47:06.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:06 smithi107 ceph-mon[28434]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-05-17T21:47:06.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:06 smithi107 ceph-mon[28434]: osdmap e25: 4 total, 3 up, 4 in 2024-05-17T21:47:06.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:06 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-17T21:47:06.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:06 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-17T21:47:07.267 INFO:teuthology.orchestra.run.smithi138.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-17T21:47:07.267 INFO:teuthology.orchestra.run.smithi138.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-17T21:47:07.268 INFO:teuthology.orchestra.run.smithi138.stderr: stderr: 10+0 records in 2024-05-17T21:47:07.268 INFO:teuthology.orchestra.run.smithi138.stderr:10+0 records out 2024-05-17T21:47:07.268 INFO:teuthology.orchestra.run.smithi138.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0120971 s, 867 MB/s 2024-05-17T21:47:07.268 INFO:teuthology.orchestra.run.smithi138.stderr:--> Zapping successful for: 2024-05-17T21:47:07.682 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:07 smithi138 ceph-mon[35948]: purged_snaps scrub starts 2024-05-17T21:47:07.682 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:07 smithi138 ceph-mon[35948]: purged_snaps scrub ok 2024-05-17T21:47:07.682 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:07 smithi138 ceph-mon[35948]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-05-17T21:47:07.683 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:07 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-17T21:47:07.683 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:07 smithi138 ceph-mon[35948]: osd.3 [v2:172.21.15.138:6800/2477238048,v1:172.21.15.138:6801/2477238048] boot 2024-05-17T21:47:07.683 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:07 smithi138 ceph-mon[35948]: osdmap e26: 4 total, 4 up, 4 in 2024-05-17T21:47:07.683 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:07 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-17T21:47:07.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:07 smithi107 ceph-mon[28434]: purged_snaps scrub starts 2024-05-17T21:47:07.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:07 smithi107 ceph-mon[28434]: purged_snaps scrub ok 2024-05-17T21:47:07.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:07 smithi107 ceph-mon[28434]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-05-17T21:47:07.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:07 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-17T21:47:07.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:07 smithi107 ceph-mon[28434]: osd.3 [v2:172.21.15.138:6800/2477238048,v1:172.21.15.138:6801/2477238048] boot 2024-05-17T21:47:07.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:07 smithi107 ceph-mon[28434]: osdmap e26: 4 total, 4 up, 4 in 2024-05-17T21:47:07.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:07 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-17T21:47:07.970 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi138:vg_nvme/lv_3 2024-05-17T21:47:08.587 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/mon.smithi138/config 2024-05-17T21:47:08.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:08 smithi107 ceph-mon[28434]: osdmap e27: 4 total, 4 up, 4 in 2024-05-17T21:47:08.709 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:08 smithi138 ceph-mon[35948]: osdmap e27: 4 total, 4 up, 4 in 2024-05-17T21:47:09.577 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:09 smithi138 ceph-mon[35948]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-05-17T21:47:09.577 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:09 smithi138 ceph-mon[35948]: osdmap e28: 4 total, 4 up, 4 in 2024-05-17T21:47:09.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:09 smithi107 ceph-mon[28434]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-05-17T21:47:09.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:09 smithi107 ceph-mon[28434]: osdmap e28: 4 total, 4 up, 4 in 2024-05-17T21:47:10.664 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:10 smithi138 ceph-mon[35948]: osdmap e29: 4 total, 4 up, 4 in 2024-05-17T21:47:10.664 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:10 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:47:10.664 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:10 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:47:10.664 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:10 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-17T21:47:10.664 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:10 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:47:10.664 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:10 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:47:10.664 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:10 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T21:47:10.665 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:10 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:47:10.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:10 smithi107 ceph-mon[28434]: osdmap e29: 4 total, 4 up, 4 in 2024-05-17T21:47:10.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:10 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:47:10.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:10 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:47:10.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:10 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-17T21:47:10.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:10 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:47:10.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:10 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:47:10.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:10 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T21:47:10.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:10 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:47:10.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:47:10.890+0000 7fcdebbea700 1 -- 172.21.15.138:0/3596018379 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcde40fcf60 msgr2=0x7fcde40fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:47:10.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:47:10.890+0000 7fcdebbea700 1 --2- 172.21.15.138:0/3596018379 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcde40fcf60 0x7fcde40fd3d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fcde0004940 tx=0x7fcde0009e30 comp rx=0 tx=0).stop 2024-05-17T21:47:10.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:47:10.890+0000 7fcdebbea700 1 -- 172.21.15.138:0/3596018379 shutdown_connections 2024-05-17T21:47:10.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:47:10.890+0000 7fcdebbea700 1 --2- 172.21.15.138:0/3596018379 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcde40fcf60 0x7fcde40fd3d0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:47:10.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:47:10.890+0000 7fcdebbea700 1 --2- 172.21.15.138:0/3596018379 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcde40fbcb0 0x7fcde40fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:47:10.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:47:10.890+0000 7fcdebbea700 1 -- 172.21.15.138:0/3596018379 >> 172.21.15.138:0/3596018379 conn(0x7fcde40f7430 msgr2=0x7fcde40f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:47:10.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:47:10.890+0000 7fcdebbea700 1 -- 172.21.15.138:0/3596018379 shutdown_connections 2024-05-17T21:47:10.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:47:10.890+0000 7fcdebbea700 1 -- 172.21.15.138:0/3596018379 wait complete. 2024-05-17T21:47:10.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:47:10.891+0000 7fcdebbea700 1 Processor -- start 2024-05-17T21:47:10.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:47:10.891+0000 7fcdebbea700 1 -- start start 2024-05-17T21:47:10.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:47:10.891+0000 7fcdebbea700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcde40fbcb0 0x7fcde41106b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:47:10.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:47:10.891+0000 7fcdebbea700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcde40fcf60 0x7fcde4112c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:47:10.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:47:10.891+0000 7fcdebbea700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcde4066420 con 0x7fcde40fcf60 2024-05-17T21:47:10.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:47:10.891+0000 7fcdebbea700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcde4066560 con 0x7fcde40fbcb0 2024-05-17T21:47:10.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:47:10.891+0000 7fcde9986700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcde40fbcb0 0x7fcde41106b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:47:10.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:47:10.891+0000 7fcde9986700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcde40fbcb0 0x7fcde41106b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:41322/0 (socket says 172.21.15.138:41322) 2024-05-17T21:47:10.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:47:10.891+0000 7fcde9986700 1 -- 172.21.15.138:0/2016360709 learned_addr learned my addr 172.21.15.138:0/2016360709 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-17T21:47:10.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:47:10.891+0000 7fcde9185700 1 --2- 172.21.15.138:0/2016360709 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcde40fcf60 0x7fcde4112c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:47:10.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:47:10.891+0000 7fcde9986700 1 -- 172.21.15.138:0/2016360709 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcde40fcf60 msgr2=0x7fcde4112c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:47:10.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:47:10.891+0000 7fcde9986700 1 --2- 172.21.15.138:0/2016360709 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcde40fcf60 0x7fcde4112c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:47:10.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:47:10.891+0000 7fcde9986700 1 -- 172.21.15.138:0/2016360709 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcde0005040 con 0x7fcde40fbcb0 2024-05-17T21:47:10.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:47:10.891+0000 7fcde9986700 1 --2- 172.21.15.138:0/2016360709 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcde40fbcb0 0x7fcde41106b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fcdd400b790 tx=0x7fcdd4009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:47:10.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:47:10.891+0000 7fcdda7fc700 1 -- 172.21.15.138:0/2016360709 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcdd400f6f0 con 0x7fcde40fbcb0 2024-05-17T21:47:10.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:47:10.891+0000 7fcdebbea700 1 -- 172.21.15.138:0/2016360709 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcde4113290 con 0x7fcde40fbcb0 2024-05-17T21:47:10.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:47:10.891+0000 7fcdda7fc700 1 -- 172.21.15.138:0/2016360709 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcdd400fd20 con 0x7fcde40fbcb0 2024-05-17T21:47:10.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:47:10.891+0000 7fcdda7fc700 1 -- 172.21.15.138:0/2016360709 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcdd40103a0 con 0x7fcde40fbcb0 2024-05-17T21:47:10.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:47:10.892+0000 7fcdebbea700 1 -- 172.21.15.138:0/2016360709 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcde4113850 con 0x7fcde40fbcb0 2024-05-17T21:47:10.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:47:10.892+0000 7fcdebbea700 1 -- 172.21.15.138:0/2016360709 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcde405f5c0 con 0x7fcde40fbcb0 2024-05-17T21:47:10.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:47:10.893+0000 7fcdda7fc700 1 -- 172.21.15.138:0/2016360709 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7fcdd4011400 con 0x7fcde40fbcb0 2024-05-17T21:47:10.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:47:10.893+0000 7fcdda7fc700 1 --2- 172.21.15.138:0/2016360709 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fcdd00692a0 0x7fcdd006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:47:10.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:47:10.893+0000 7fcdda7fc700 1 -- 172.21.15.138:0/2016360709 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 3770+0+0 (secure 0 0 0) 0x7fcdd40868e0 con 0x7fcde40fbcb0 2024-05-17T21:47:10.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:47:10.893+0000 7fcde9185700 1 --2- 172.21.15.138:0/2016360709 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fcdd00692a0 0x7fcdd006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:47:10.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:47:10.894+0000 7fcde9185700 1 --2- 172.21.15.138:0/2016360709 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fcdd00692a0 0x7fcdd006b750 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fcde000b970 tx=0x7fcde0009e30 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:47:10.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:47:10.898+0000 7fcdda7fc700 1 -- 172.21.15.138:0/2016360709 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fcdd40532b0 con 0x7fcde40fbcb0 2024-05-17T21:47:11.082 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:47:11.089+0000 7fcdebbea700 1 -- 172.21.15.138:0/2016360709 --> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fcde4101200 con 0x7fcdd00692a0 2024-05-17T21:47:11.735 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:11 smithi138 ceph-mon[35948]: pgmap v96: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-17T21:47:11.736 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:11 smithi138 ceph-mon[35948]: Detected new or changed devices on smithi138 2024-05-17T21:47:11.736 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:11 smithi138 ceph-mon[35948]: Adjusting osd_memory_target on smithi138 to 15936M 2024-05-17T21:47:11.736 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:11 smithi138 ceph-mon[35948]: osdmap e30: 4 total, 4 up, 4 in 2024-05-17T21:47:11.736 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:11 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:47:11.736 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:11 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:47:11.736 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:11 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T21:47:11.736 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:11 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:47:11.736 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:11 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T21:47:11.736 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:11 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T21:47:11.736 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:11 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:47:11.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:11 smithi107 ceph-mon[28434]: pgmap v96: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-17T21:47:11.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:11 smithi107 ceph-mon[28434]: Detected new or changed devices on smithi138 2024-05-17T21:47:11.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:11 smithi107 ceph-mon[28434]: Adjusting osd_memory_target on smithi138 to 15936M 2024-05-17T21:47:11.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:11 smithi107 ceph-mon[28434]: osdmap e30: 4 total, 4 up, 4 in 2024-05-17T21:47:11.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:11 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:47:11.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:11 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:47:11.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:11 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T21:47:11.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:11 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:47:11.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:11 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T21:47:11.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:11 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T21:47:11.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:11 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:47:12.817 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:12 smithi138 ceph-mon[35948]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:47:12.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:12 smithi107 ceph-mon[28434]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:47:13.765 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:13 smithi138 ceph-mon[35948]: pgmap v98: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 118 KiB/s, 0 objects/s recovering 2024-05-17T21:47:13.765 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:13 smithi138 ceph-mon[35948]: from='client.? 172.21.15.138:0/3795639970' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d28515a1-4510-4f94-b0b1-3aca283eed05"}]: dispatch 2024-05-17T21:47:13.765 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:13 smithi138 ceph-mon[35948]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d28515a1-4510-4f94-b0b1-3aca283eed05"}]: dispatch 2024-05-17T21:47:13.765 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:13 smithi138 ceph-mon[35948]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d28515a1-4510-4f94-b0b1-3aca283eed05"}]': finished 2024-05-17T21:47:13.765 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:13 smithi138 ceph-mon[35948]: osdmap e31: 5 total, 4 up, 5 in 2024-05-17T21:47:13.765 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:13 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-17T21:47:13.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:13 smithi107 ceph-mon[28434]: pgmap v98: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 118 KiB/s, 0 objects/s recovering 2024-05-17T21:47:13.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:13 smithi107 ceph-mon[28434]: from='client.? 172.21.15.138:0/3795639970' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d28515a1-4510-4f94-b0b1-3aca283eed05"}]: dispatch 2024-05-17T21:47:13.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:13 smithi107 ceph-mon[28434]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d28515a1-4510-4f94-b0b1-3aca283eed05"}]: dispatch 2024-05-17T21:47:13.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:13 smithi107 ceph-mon[28434]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d28515a1-4510-4f94-b0b1-3aca283eed05"}]': finished 2024-05-17T21:47:13.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:13 smithi107 ceph-mon[28434]: osdmap e31: 5 total, 4 up, 5 in 2024-05-17T21:47:13.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:13 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-17T21:47:14.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:14 smithi138 ceph-mon[35948]: from='client.? 172.21.15.138:0/2864971971' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T21:47:14.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:14 smithi107 ceph-mon[28434]: from='client.? 172.21.15.138:0/2864971971' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T21:47:15.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:15 smithi138 ceph-mon[35948]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-05-17T21:47:15.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:15 smithi107 ceph-mon[28434]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-05-17T21:47:17.577 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:17 smithi138 ceph-mon[35948]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-05-17T21:47:17.577 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:17 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:47:17.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:17 smithi107 ceph-mon[28434]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-05-17T21:47:17.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:17 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:47:19.715 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:19 smithi138 ceph-mon[35948]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-05-17T21:47:19.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:19 smithi107 ceph-mon[28434]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-05-17T21:47:21.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:21 smithi138 ceph-mon[35948]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-05-17T21:47:21.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:21 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-17T21:47:21.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:21 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:47:21.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:21 smithi138 ceph-mon[35948]: Deploying daemon osd.4 on smithi138 2024-05-17T21:47:21.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:21 smithi107 ceph-mon[28434]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-05-17T21:47:21.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:21 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-17T21:47:21.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:21 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:47:21.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:21 smithi107 ceph-mon[28434]: Deploying daemon osd.4 on smithi138 2024-05-17T21:47:23.768 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:23 smithi138 ceph-mon[35948]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-05-17T21:47:23.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:23 smithi107 ceph-mon[28434]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-05-17T21:47:25.662 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:25 smithi138 ceph-mon[35948]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-05-17T21:47:25.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:25 smithi107 ceph-mon[28434]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-05-17T21:47:27.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:26 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:47:27.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:26 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:47:27.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:26 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:47:27.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:26 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:47:27.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:26 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:47:27.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:26 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:47:28.048 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:27 smithi138 ceph-mon[35948]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-05-17T21:47:28.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:27 smithi107 ceph-mon[28434]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-05-17T21:47:29.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:47:29.403+0000 7fcdda7fc700 1 -- 172.21.15.138:0/2016360709 <== mgr.14214 v2:172.21.15.107:6800/1294361014 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fcde4101200 con 0x7fcdd00692a0 2024-05-17T21:47:29.397 INFO:teuthology.orchestra.run.smithi138.stdout:Created osd(s) 4 on host 'smithi138' 2024-05-17T21:47:29.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:47:29.405+0000 7fcdebbea700 1 -- 172.21.15.138:0/2016360709 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fcdd00692a0 msgr2=0x7fcdd006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:47:29.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:47:29.405+0000 7fcdebbea700 1 --2- 172.21.15.138:0/2016360709 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fcdd00692a0 0x7fcdd006b750 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fcde000b970 tx=0x7fcde0009e30 comp rx=0 tx=0).stop 2024-05-17T21:47:29.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:47:29.406+0000 7fcdebbea700 1 -- 172.21.15.138:0/2016360709 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcde40fbcb0 msgr2=0x7fcde41106b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:47:29.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:47:29.406+0000 7fcdebbea700 1 --2- 172.21.15.138:0/2016360709 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcde40fbcb0 0x7fcde41106b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fcdd400b790 tx=0x7fcdd4009250 comp rx=0 tx=0).stop 2024-05-17T21:47:29.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:47:29.406+0000 7fcdebbea700 1 -- 172.21.15.138:0/2016360709 shutdown_connections 2024-05-17T21:47:29.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:47:29.406+0000 7fcdebbea700 1 --2- 172.21.15.138:0/2016360709 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fcdd00692a0 0x7fcdd006b750 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:47:29.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:47:29.406+0000 7fcdebbea700 1 --2- 172.21.15.138:0/2016360709 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcde40fbcb0 0x7fcde41106b0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:47:29.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:47:29.406+0000 7fcdebbea700 1 --2- 172.21.15.138:0/2016360709 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcde40fcf60 0x7fcde4112c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:47:29.400 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:47:29.406+0000 7fcdebbea700 1 -- 172.21.15.138:0/2016360709 >> 172.21.15.138:0/2016360709 conn(0x7fcde40f7430 msgr2=0x7fcde40f8a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:47:29.400 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:47:29.406+0000 7fcdebbea700 1 -- 172.21.15.138:0/2016360709 shutdown_connections 2024-05-17T21:47:29.400 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:47:29.406+0000 7fcdebbea700 1 -- 172.21.15.138:0/2016360709 wait complete. 2024-05-17T21:47:29.985 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:29 smithi138 ceph-mon[35948]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-05-17T21:47:29.985 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:29 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:47:29.986 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:29 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:47:29.986 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:29 smithi138 ceph-mon[35948]: from='osd.4 [v2:172.21.15.138:6808/792878465,v1:172.21.15.138:6809/792878465]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-17T21:47:29.986 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:29 smithi138 ceph-mon[35948]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-17T21:47:30.010 DEBUG:teuthology.orchestra.run.smithi138:osd.4> sudo journalctl -f -n 0 -u ceph-3722731a-1496-11ef-bc9a-c7b262605968@osd.4.service 2024-05-17T21:47:30.016 INFO:tasks.cephadm:Deploying osd.5 on smithi138 with /dev/vg_nvme/lv_2... 2024-05-17T21:47:30.017 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-17T21:47:30.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:29 smithi107 ceph-mon[28434]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-05-17T21:47:30.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:29 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:47:30.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:29 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:47:30.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:29 smithi107 ceph-mon[28434]: from='osd.4 [v2:172.21.15.138:6808/792878465,v1:172.21.15.138:6809/792878465]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-17T21:47:30.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:29 smithi107 ceph-mon[28434]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-17T21:47:30.341 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/mon.smithi138/config 2024-05-17T21:47:31.460 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:31 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:47:31.460 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:31 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:47:31.460 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:31 smithi138 ceph-mon[35948]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-05-17T21:47:31.460 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:31 smithi138 ceph-mon[35948]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-17T21:47:31.460 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:31 smithi138 ceph-mon[35948]: osdmap e32: 5 total, 4 up, 5 in 2024-05-17T21:47:31.460 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:31 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-17T21:47:31.460 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:31 smithi138 ceph-mon[35948]: from='osd.4 [v2:172.21.15.138:6808/792878465,v1:172.21.15.138:6809/792878465]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-05-17T21:47:31.460 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:31 smithi138 ceph-mon[35948]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-05-17T21:47:31.460 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 17 21:47:31 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-4[48290]: 2024-05-17T21:47:31.385+0000 7fce89b52700 -1 osd.4 0 waiting for initial osdmap 2024-05-17T21:47:31.461 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 17 21:47:31 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-4[48290]: 2024-05-17T21:47:31.393+0000 7fce8394b700 -1 osd.4 33 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-17T21:47:31.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:31 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:47:31.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:31 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:47:31.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:31 smithi107 ceph-mon[28434]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-05-17T21:47:31.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:31 smithi107 ceph-mon[28434]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-17T21:47:31.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:31 smithi107 ceph-mon[28434]: osdmap e32: 5 total, 4 up, 5 in 2024-05-17T21:47:31.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:31 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-17T21:47:31.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:31 smithi107 ceph-mon[28434]: from='osd.4 [v2:172.21.15.138:6808/792878465,v1:172.21.15.138:6809/792878465]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-05-17T21:47:31.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:31 smithi107 ceph-mon[28434]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-05-17T21:47:32.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:32 smithi107 ceph-mon[28434]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-05-17T21:47:32.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:32 smithi107 ceph-mon[28434]: osdmap e33: 5 total, 4 up, 5 in 2024-05-17T21:47:32.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:32 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-17T21:47:32.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:32 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-17T21:47:32.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:32 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:47:32.731 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:32 smithi138 ceph-mon[35948]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-05-17T21:47:32.731 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:32 smithi138 ceph-mon[35948]: osdmap e33: 5 total, 4 up, 5 in 2024-05-17T21:47:32.731 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:32 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-17T21:47:32.731 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:32 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-17T21:47:32.731 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:32 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:47:32.889 INFO:teuthology.orchestra.run.smithi138.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-17T21:47:32.890 INFO:teuthology.orchestra.run.smithi138.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-17T21:47:32.890 INFO:teuthology.orchestra.run.smithi138.stderr: stderr: 10+0 records in 2024-05-17T21:47:32.890 INFO:teuthology.orchestra.run.smithi138.stderr:10+0 records out 2024-05-17T21:47:32.890 INFO:teuthology.orchestra.run.smithi138.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0127222 s, 824 MB/s 2024-05-17T21:47:32.890 INFO:teuthology.orchestra.run.smithi138.stderr:--> Zapping successful for: 2024-05-17T21:47:33.467 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:33 smithi138 ceph-mon[35948]: purged_snaps scrub starts 2024-05-17T21:47:33.467 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:33 smithi138 ceph-mon[35948]: purged_snaps scrub ok 2024-05-17T21:47:33.467 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:33 smithi138 ceph-mon[35948]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-05-17T21:47:33.467 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:33 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-17T21:47:33.467 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:33 smithi138 ceph-mon[35948]: osd.4 [v2:172.21.15.138:6808/792878465,v1:172.21.15.138:6809/792878465] boot 2024-05-17T21:47:33.467 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:33 smithi138 ceph-mon[35948]: osdmap e34: 5 total, 5 up, 5 in 2024-05-17T21:47:33.467 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:33 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-17T21:47:33.491 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi138:vg_nvme/lv_2 2024-05-17T21:47:33.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:33 smithi107 ceph-mon[28434]: purged_snaps scrub starts 2024-05-17T21:47:33.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:33 smithi107 ceph-mon[28434]: purged_snaps scrub ok 2024-05-17T21:47:33.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:33 smithi107 ceph-mon[28434]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-05-17T21:47:33.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:33 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-17T21:47:33.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:33 smithi107 ceph-mon[28434]: osd.4 [v2:172.21.15.138:6808/792878465,v1:172.21.15.138:6809/792878465] boot 2024-05-17T21:47:33.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:33 smithi107 ceph-mon[28434]: osdmap e34: 5 total, 5 up, 5 in 2024-05-17T21:47:33.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:33 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-17T21:47:34.161 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/mon.smithi138/config 2024-05-17T21:47:34.742 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:34 smithi138 ceph-mon[35948]: osdmap e35: 5 total, 5 up, 5 in 2024-05-17T21:47:34.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:34 smithi107 ceph-mon[28434]: osdmap e35: 5 total, 5 up, 5 in 2024-05-17T21:47:35.577 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:35 smithi138 ceph-mon[35948]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-05-17T21:47:35.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:35 smithi107 ceph-mon[28434]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-05-17T21:47:35.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:47:35.889+0000 7f7e57153700 1 -- 172.21.15.138:0/467390451 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e500fce60 msgr2=0x7f7e500fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:47:35.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:47:35.889+0000 7f7e57153700 1 --2- 172.21.15.138:0/467390451 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e500fce60 0x7f7e500fd270 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c004940 tx=0x7f7e4c009e30 comp rx=0 tx=0).stop 2024-05-17T21:47:35.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:47:35.889+0000 7f7e57153700 1 -- 172.21.15.138:0/467390451 shutdown_connections 2024-05-17T21:47:35.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:47:35.889+0000 7f7e57153700 1 --2- 172.21.15.138:0/467390451 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e500fdfd0 0x7f7e500fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:47:35.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:47:35.889+0000 7f7e57153700 1 --2- 172.21.15.138:0/467390451 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e500fce60 0x7f7e500fd270 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:47:35.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:47:35.889+0000 7f7e57153700 1 -- 172.21.15.138:0/467390451 >> 172.21.15.138:0/467390451 conn(0x7f7e500f8540 msgr2=0x7f7e500fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:47:35.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:47:35.889+0000 7f7e57153700 1 -- 172.21.15.138:0/467390451 shutdown_connections 2024-05-17T21:47:35.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:47:35.889+0000 7f7e57153700 1 -- 172.21.15.138:0/467390451 wait complete. 2024-05-17T21:47:35.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:47:35.890+0000 7f7e57153700 1 Processor -- start 2024-05-17T21:47:35.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:47:35.890+0000 7f7e57153700 1 -- start start 2024-05-17T21:47:35.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:47:35.890+0000 7f7e57153700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e500fce60 0x7f7e5010d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:47:35.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:47:35.890+0000 7f7e57153700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e500fdfd0 0x7f7e5010de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:47:35.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:47:35.890+0000 7f7e57153700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e5010e460 con 0x7f7e500fdfd0 2024-05-17T21:47:35.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:47:35.890+0000 7f7e57153700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e5010e5d0 con 0x7f7e500fce60 2024-05-17T21:47:35.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:47:35.890+0000 7f7e54eef700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e500fce60 0x7f7e5010d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:47:35.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:47:35.890+0000 7f7e54eef700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e500fce60 0x7f7e5010d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:48714/0 (socket says 172.21.15.138:48714) 2024-05-17T21:47:35.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:47:35.890+0000 7f7e54eef700 1 -- 172.21.15.138:0/1239511915 learned_addr learned my addr 172.21.15.138:0/1239511915 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-17T21:47:35.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:47:35.890+0000 7f7e47fff700 1 --2- 172.21.15.138:0/1239511915 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e500fdfd0 0x7f7e5010de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:47:35.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:47:35.890+0000 7f7e54eef700 1 -- 172.21.15.138:0/1239511915 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e500fdfd0 msgr2=0x7f7e5010de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:47:35.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:47:35.890+0000 7f7e54eef700 1 --2- 172.21.15.138:0/1239511915 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e500fdfd0 0x7f7e5010de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:47:35.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:47:35.890+0000 7f7e54eef700 1 -- 172.21.15.138:0/1239511915 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e4c005040 con 0x7f7e500fce60 2024-05-17T21:47:35.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:47:35.891+0000 7f7e54eef700 1 --2- 172.21.15.138:0/1239511915 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e500fce60 0x7f7e5010d920 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c00b3b0 tx=0x7f7e4c009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:47:35.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:47:35.891+0000 7f7e457fa700 1 -- 172.21.15.138:0/1239511915 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7e4c00e040 con 0x7f7e500fce60 2024-05-17T21:47:35.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:47:35.891+0000 7f7e457fa700 1 -- 172.21.15.138:0/1239511915 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7e4c00bbf0 con 0x7f7e500fce60 2024-05-17T21:47:35.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:47:35.891+0000 7f7e57153700 1 -- 172.21.15.138:0/1239511915 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e50112fd0 con 0x7f7e500fce60 2024-05-17T21:47:35.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:47:35.891+0000 7f7e457fa700 1 -- 172.21.15.138:0/1239511915 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7e4c01c630 con 0x7f7e500fce60 2024-05-17T21:47:35.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:47:35.891+0000 7f7e57153700 1 -- 172.21.15.138:0/1239511915 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e50113500 con 0x7f7e500fce60 2024-05-17T21:47:35.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:47:35.892+0000 7f7e57153700 1 -- 172.21.15.138:0/1239511915 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e5004c710 con 0x7f7e500fce60 2024-05-17T21:47:35.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:47:35.892+0000 7f7e457fa700 1 -- 172.21.15.138:0/1239511915 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f7e4c00bd60 con 0x7f7e500fce60 2024-05-17T21:47:35.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:47:35.893+0000 7f7e457fa700 1 --2- 172.21.15.138:0/1239511915 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f7e380692a0 0x7f7e3806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:47:35.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:47:35.893+0000 7f7e457fa700 1 -- 172.21.15.138:0/1239511915 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 4202+0+0 (secure 0 0 0) 0x7f7e4c087960 con 0x7f7e500fce60 2024-05-17T21:47:35.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:47:35.893+0000 7f7e47fff700 1 --2- 172.21.15.138:0/1239511915 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f7e380692a0 0x7f7e3806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:47:35.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:47:35.893+0000 7f7e47fff700 1 --2- 172.21.15.138:0/1239511915 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f7e380692a0 0x7f7e3806b750 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7e40004f40 tx=0x7f7e4000b040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:47:35.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:47:35.897+0000 7f7e457fa700 1 -- 172.21.15.138:0/1239511915 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f7e4c057c10 con 0x7f7e500fce60 2024-05-17T21:47:36.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:47:36.099+0000 7f7e57153700 1 -- 172.21.15.138:0/1239511915 --> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f7e50113720 con 0x7f7e380692a0 2024-05-17T21:47:36.577 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:36 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T21:47:36.577 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:36 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T21:47:36.577 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:36 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:47:36.578 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:36 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:47:36.578 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:36 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:47:36.578 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:36 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-17T21:47:36.578 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:36 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-17T21:47:36.578 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:36 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:47:36.578 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:36 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:47:36.578 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:36 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T21:47:36.578 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:36 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:47:36.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:36 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T21:47:36.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:36 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T21:47:36.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:36 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:47:36.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:36 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:47:36.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:36 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:47:36.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:36 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-17T21:47:36.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:36 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-17T21:47:36.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:36 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:47:36.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:36 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:47:36.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:36 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T21:47:36.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:36 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:47:37.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:37 smithi107 ceph-mon[28434]: from='client.24167 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:47:37.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:37 smithi107 ceph-mon[28434]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-05-17T21:47:37.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:37 smithi107 ceph-mon[28434]: Detected new or changed devices on smithi138 2024-05-17T21:47:37.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:37 smithi107 ceph-mon[28434]: Adjusting osd_memory_target on smithi138 to 7968M 2024-05-17T21:47:37.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:37 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:47:37.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:37 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:47:37.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:37 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T21:47:37.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:37 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:47:37.716 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:37 smithi138 ceph-mon[35948]: from='client.24167 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:47:37.716 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:37 smithi138 ceph-mon[35948]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-05-17T21:47:37.716 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:37 smithi138 ceph-mon[35948]: Detected new or changed devices on smithi138 2024-05-17T21:47:37.716 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:37 smithi138 ceph-mon[35948]: Adjusting osd_memory_target on smithi138 to 7968M 2024-05-17T21:47:37.716 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:37 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:47:37.717 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:37 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:47:37.717 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:37 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T21:47:37.717 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:37 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:47:39.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:39 smithi107 ceph-mon[28434]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-05-17T21:47:39.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:39 smithi107 ceph-mon[28434]: from='client.? 172.21.15.138:0/2185859393' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ab57d399-f1cf-4139-abba-7cb15e433c25"}]: dispatch 2024-05-17T21:47:39.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:39 smithi107 ceph-mon[28434]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ab57d399-f1cf-4139-abba-7cb15e433c25"}]: dispatch 2024-05-17T21:47:39.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:39 smithi107 ceph-mon[28434]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ab57d399-f1cf-4139-abba-7cb15e433c25"}]': finished 2024-05-17T21:47:39.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:39 smithi107 ceph-mon[28434]: osdmap e36: 6 total, 5 up, 6 in 2024-05-17T21:47:39.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:39 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-17T21:47:39.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:39 smithi107 ceph-mon[28434]: from='client.? 172.21.15.138:0/122266545' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T21:47:39.831 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:39 smithi138 ceph-mon[35948]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-05-17T21:47:39.831 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:39 smithi138 ceph-mon[35948]: from='client.? 172.21.15.138:0/2185859393' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ab57d399-f1cf-4139-abba-7cb15e433c25"}]: dispatch 2024-05-17T21:47:39.831 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:39 smithi138 ceph-mon[35948]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ab57d399-f1cf-4139-abba-7cb15e433c25"}]: dispatch 2024-05-17T21:47:39.831 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:39 smithi138 ceph-mon[35948]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ab57d399-f1cf-4139-abba-7cb15e433c25"}]': finished 2024-05-17T21:47:39.831 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:39 smithi138 ceph-mon[35948]: osdmap e36: 6 total, 5 up, 6 in 2024-05-17T21:47:39.831 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:39 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-17T21:47:39.831 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:39 smithi138 ceph-mon[35948]: from='client.? 172.21.15.138:0/122266545' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T21:47:41.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:41 smithi138 ceph-mon[35948]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-05-17T21:47:41.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:41 smithi107 ceph-mon[28434]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-05-17T21:47:43.682 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:43 smithi138 ceph-mon[35948]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-05-17T21:47:43.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:43 smithi107 ceph-mon[28434]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-05-17T21:47:45.577 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:45 smithi138 ceph-mon[35948]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-05-17T21:47:45.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:45 smithi107 ceph-mon[28434]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-05-17T21:47:46.653 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:46 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-17T21:47:46.654 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:46 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:47:46.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:46 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-17T21:47:46.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:46 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:47:47.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:47 smithi138 ceph-mon[35948]: Deploying daemon osd.5 on smithi138 2024-05-17T21:47:47.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:47 smithi138 ceph-mon[35948]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-05-17T21:47:47.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:47 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:47:47.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:47 smithi107 ceph-mon[28434]: Deploying daemon osd.5 on smithi138 2024-05-17T21:47:47.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:47 smithi107 ceph-mon[28434]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-05-17T21:47:47.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:47 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:47:49.791 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:49 smithi138 ceph-mon[35948]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-05-17T21:47:49.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:49 smithi107 ceph-mon[28434]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-05-17T21:47:51.693 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:51 smithi138 ceph-mon[35948]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-05-17T21:47:51.694 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:51 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:47:51.694 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:51 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:47:51.694 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:51 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:47:51.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:51 smithi107 ceph-mon[28434]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-05-17T21:47:51.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:51 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:47:51.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:51 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:47:51.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:51 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:47:53.577 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:53 smithi138 ceph-mon[35948]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-05-17T21:47:53.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:53 smithi107 ceph-mon[28434]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-05-17T21:47:54.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:47:54.567+0000 7f7e457fa700 1 -- 172.21.15.138:0/1239511915 <== mgr.14214 v2:172.21.15.107:6800/1294361014 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f7e50113720 con 0x7f7e380692a0 2024-05-17T21:47:54.561 INFO:teuthology.orchestra.run.smithi138.stdout:Created osd(s) 5 on host 'smithi138' 2024-05-17T21:47:54.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:47:54.570+0000 7f7e57153700 1 -- 172.21.15.138:0/1239511915 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f7e380692a0 msgr2=0x7f7e3806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:47:54.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:47:54.570+0000 7f7e57153700 1 --2- 172.21.15.138:0/1239511915 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f7e380692a0 0x7f7e3806b750 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7e40004f40 tx=0x7f7e4000b040 comp rx=0 tx=0).stop 2024-05-17T21:47:54.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:47:54.570+0000 7f7e57153700 1 -- 172.21.15.138:0/1239511915 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e500fce60 msgr2=0x7f7e5010d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:47:54.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:47:54.570+0000 7f7e57153700 1 --2- 172.21.15.138:0/1239511915 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e500fce60 0x7f7e5010d920 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c00b3b0 tx=0x7f7e4c009e30 comp rx=0 tx=0).stop 2024-05-17T21:47:54.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:47:54.570+0000 7f7e57153700 1 -- 172.21.15.138:0/1239511915 shutdown_connections 2024-05-17T21:47:54.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:47:54.570+0000 7f7e57153700 1 --2- 172.21.15.138:0/1239511915 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f7e380692a0 0x7f7e3806b750 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:47:54.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:47:54.570+0000 7f7e57153700 1 --2- 172.21.15.138:0/1239511915 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e500fce60 0x7f7e5010d920 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:47:54.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:47:54.570+0000 7f7e57153700 1 --2- 172.21.15.138:0/1239511915 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e500fdfd0 0x7f7e5010de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:47:54.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:47:54.570+0000 7f7e57153700 1 -- 172.21.15.138:0/1239511915 >> 172.21.15.138:0/1239511915 conn(0x7f7e500f8540 msgr2=0x7f7e500f9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:47:54.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:47:54.570+0000 7f7e57153700 1 -- 172.21.15.138:0/1239511915 shutdown_connections 2024-05-17T21:47:54.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:47:54.570+0000 7f7e57153700 1 -- 172.21.15.138:0/1239511915 wait complete. 2024-05-17T21:47:55.169 DEBUG:teuthology.orchestra.run.smithi138:osd.5> sudo journalctl -f -n 0 -u ceph-3722731a-1496-11ef-bc9a-c7b262605968@osd.5.service 2024-05-17T21:47:55.171 INFO:tasks.cephadm:Waiting for 6 OSDs to come up... 2024-05-17T21:47:55.172 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -- ceph osd stat -f json 2024-05-17T21:47:55.436 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/mon.smithi107/config 2024-05-17T21:47:55.821 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:55 smithi107 ceph-mon[28434]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-05-17T21:47:55.821 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:55 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:47:55.821 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:55 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:47:55.821 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:55 smithi107 ceph-mon[28434]: from='osd.5 [v2:172.21.15.138:6816/406753522,v1:172.21.15.138:6817/406753522]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-17T21:47:55.822 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:55 smithi107 ceph-mon[28434]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-17T21:47:55.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:55 smithi138 ceph-mon[35948]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-05-17T21:47:55.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:55 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:47:55.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:55 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:47:55.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:55 smithi138 ceph-mon[35948]: from='osd.5 [v2:172.21.15.138:6816/406753522,v1:172.21.15.138:6817/406753522]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-17T21:47:55.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:55 smithi138 ceph-mon[35948]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-17T21:47:56.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:56 smithi138 ceph-mon[35948]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-17T21:47:56.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:56 smithi138 ceph-mon[35948]: osdmap e37: 6 total, 5 up, 6 in 2024-05-17T21:47:56.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:56 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-17T21:47:56.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:56 smithi138 ceph-mon[35948]: from='osd.5 [v2:172.21.15.138:6816/406753522,v1:172.21.15.138:6817/406753522]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-05-17T21:47:56.828 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:56 smithi138 ceph-mon[35948]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-05-17T21:47:56.828 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:56 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:47:56.828 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:56 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:47:56.828 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 17 21:47:56 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-5[55025]: 2024-05-17T21:47:56.571+0000 7f044d26e700 -1 osd.5 0 waiting for initial osdmap 2024-05-17T21:47:56.828 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 17 21:47:56 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-5[55025]: 2024-05-17T21:47:56.579+0000 7f0448069700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-17T21:47:56.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:56 smithi107 ceph-mon[28434]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-17T21:47:56.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:56 smithi107 ceph-mon[28434]: osdmap e37: 6 total, 5 up, 6 in 2024-05-17T21:47:56.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:56 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-17T21:47:56.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:56 smithi107 ceph-mon[28434]: from='osd.5 [v2:172.21.15.138:6816/406753522,v1:172.21.15.138:6817/406753522]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-05-17T21:47:56.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:56 smithi107 ceph-mon[28434]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-05-17T21:47:56.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:56 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:47:56.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:56 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:47:57.072 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:47:57.078+0000 7f164263b700 1 -- 172.21.15.107:0/3575534626 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f163c0fdfd0 msgr2=0x7f163c0fe440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:47:57.072 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:47:57.078+0000 7f164263b700 1 --2- 172.21.15.107:0/3575534626 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f163c0fdfd0 0x7f163c0fe440 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1624004940 tx=0x7f1624009e30 comp rx=0 tx=0).stop 2024-05-17T21:47:57.072 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:47:57.078+0000 7f164263b700 1 -- 172.21.15.107:0/3575534626 shutdown_connections 2024-05-17T21:47:57.072 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:47:57.078+0000 7f164263b700 1 --2- 172.21.15.107:0/3575534626 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f163c0fdfd0 0x7f163c0fe440 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:47:57.072 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:47:57.078+0000 7f164263b700 1 --2- 172.21.15.107:0/3575534626 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f163c0fce60 0x7f163c0fd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:47:57.072 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:47:57.078+0000 7f164263b700 1 -- 172.21.15.107:0/3575534626 >> 172.21.15.107:0/3575534626 conn(0x7f163c0f8540 msgr2=0x7f163c0fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:47:57.072 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:47:57.078+0000 7f164263b700 1 -- 172.21.15.107:0/3575534626 shutdown_connections 2024-05-17T21:47:57.072 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:47:57.078+0000 7f164263b700 1 -- 172.21.15.107:0/3575534626 wait complete. 2024-05-17T21:47:57.072 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:47:57.078+0000 7f164263b700 1 Processor -- start 2024-05-17T21:47:57.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:47:57.078+0000 7f164263b700 1 -- start start 2024-05-17T21:47:57.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:47:57.078+0000 7f164263b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f163c0fce60 0x7f163c10fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:47:57.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:47:57.078+0000 7f164263b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f163c0fdfd0 0x7f163c110060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:47:57.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:47:57.078+0000 7f164263b700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f163c110660 con 0x7f163c0fce60 2024-05-17T21:47:57.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:47:57.078+0000 7f164263b700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f163c1107d0 con 0x7f163c0fdfd0 2024-05-17T21:47:57.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:47:57.079+0000 7f163bfff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f163c0fce60 0x7f163c10fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:47:57.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:47:57.079+0000 7f163bfff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f163c0fce60 0x7f163c10fb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52226/0 (socket says 172.21.15.107:52226) 2024-05-17T21:47:57.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:47:57.079+0000 7f163bfff700 1 -- 172.21.15.107:0/2091788928 learned_addr learned my addr 172.21.15.107:0/2091788928 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:47:57.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:47:57.079+0000 7f1633fff700 1 --2- 172.21.15.107:0/2091788928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f163c0fdfd0 0x7f163c110060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:47:57.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:47:57.079+0000 7f163bfff700 1 -- 172.21.15.107:0/2091788928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f163c0fdfd0 msgr2=0x7f163c110060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:47:57.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:47:57.079+0000 7f163bfff700 1 --2- 172.21.15.107:0/2091788928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f163c0fdfd0 0x7f163c110060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:47:57.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:47:57.079+0000 7f163bfff700 1 -- 172.21.15.107:0/2091788928 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1624005040 con 0x7f163c0fce60 2024-05-17T21:47:57.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:47:57.079+0000 7f1633fff700 1 --2- 172.21.15.107:0/2091788928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f163c0fdfd0 0x7f163c110060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:47:57.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:47:57.079+0000 7f163bfff700 1 --2- 172.21.15.107:0/2091788928 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f163c0fce60 0x7f163c10fb20 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f162c00c830 tx=0x7f162c004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:47:57.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:47:57.079+0000 7f16397fa700 1 -- 172.21.15.107:0/2091788928 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f162c00a760 con 0x7f163c0fce60 2024-05-17T21:47:57.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:47:57.079+0000 7f164263b700 1 -- 172.21.15.107:0/2091788928 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f163c115230 con 0x7f163c0fce60 2024-05-17T21:47:57.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:47:57.079+0000 7f16397fa700 1 -- 172.21.15.107:0/2091788928 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f162c00ad90 con 0x7f163c0fce60 2024-05-17T21:47:57.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:47:57.079+0000 7f16397fa700 1 -- 172.21.15.107:0/2091788928 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f162c0113a0 con 0x7f163c0fce60 2024-05-17T21:47:57.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:47:57.079+0000 7f164263b700 1 -- 172.21.15.107:0/2091788928 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f163c1157f0 con 0x7f163c0fce60 2024-05-17T21:47:57.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:47:57.080+0000 7f164263b700 1 -- 172.21.15.107:0/2091788928 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f163c101960 con 0x7f163c0fce60 2024-05-17T21:47:57.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:47:57.081+0000 7f16397fa700 1 -- 172.21.15.107:0/2091788928 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f162c00a8c0 con 0x7f163c0fce60 2024-05-17T21:47:57.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:47:57.081+0000 7f16397fa700 1 --2- 172.21.15.107:0/2091788928 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f16280692a0 0x7f162806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:47:57.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:47:57.081+0000 7f16397fa700 1 -- 172.21.15.107:0/2091788928 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4354+0+0 (secure 0 0 0) 0x7f162c086db0 con 0x7f163c0fce60 2024-05-17T21:47:57.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:47:57.081+0000 7f1633fff700 1 --2- 172.21.15.107:0/2091788928 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f16280692a0 0x7f162806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:47:57.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:47:57.081+0000 7f1633fff700 1 --2- 172.21.15.107:0/2091788928 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f16280692a0 0x7f162806b750 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f162400b970 tx=0x7f1624009e30 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:47:57.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:47:57.086+0000 7f16397fa700 1 -- 172.21.15.107:0/2091788928 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f162c056fc0 con 0x7f163c0fce60 2024-05-17T21:47:57.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:47:57.278+0000 7f164263b700 1 -- 172.21.15.107:0/2091788928 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f163c04c710 con 0x7f163c0fce60 2024-05-17T21:47:57.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:47:57.278+0000 7f16397fa700 1 -- 172.21.15.107:0/2091788928 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v38) v1 ==== 74+0+130 (secure 0 0 0) 0x7f162c056b50 con 0x7f163c0fce60 2024-05-17T21:47:57.271 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:47:57.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:47:57.280+0000 7f164263b700 1 -- 172.21.15.107:0/2091788928 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f16280692a0 msgr2=0x7f162806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:47:57.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:47:57.280+0000 7f164263b700 1 --2- 172.21.15.107:0/2091788928 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f16280692a0 0x7f162806b750 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f162400b970 tx=0x7f1624009e30 comp rx=0 tx=0).stop 2024-05-17T21:47:57.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:47:57.280+0000 7f164263b700 1 -- 172.21.15.107:0/2091788928 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f163c0fce60 msgr2=0x7f163c10fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:47:57.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:47:57.280+0000 7f164263b700 1 --2- 172.21.15.107:0/2091788928 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f163c0fce60 0x7f163c10fb20 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f162c00c830 tx=0x7f162c004470 comp rx=0 tx=0).stop 2024-05-17T21:47:57.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:47:57.281+0000 7f164263b700 1 -- 172.21.15.107:0/2091788928 shutdown_connections 2024-05-17T21:47:57.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:47:57.281+0000 7f164263b700 1 --2- 172.21.15.107:0/2091788928 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f16280692a0 0x7f162806b750 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:47:57.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:47:57.281+0000 7f164263b700 1 --2- 172.21.15.107:0/2091788928 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f163c0fce60 0x7f163c10fb20 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:47:57.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:47:57.281+0000 7f164263b700 1 --2- 172.21.15.107:0/2091788928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f163c0fdfd0 0x7f163c110060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:47:57.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:47:57.281+0000 7f164263b700 1 -- 172.21.15.107:0/2091788928 >> 172.21.15.107:0/2091788928 conn(0x7f163c0f8540 msgr2=0x7f163c0f9c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:47:57.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:47:57.281+0000 7f164263b700 1 -- 172.21.15.107:0/2091788928 shutdown_connections 2024-05-17T21:47:57.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:47:57.281+0000 7f164263b700 1 -- 172.21.15.107:0/2091788928 wait complete. 2024-05-17T21:47:57.740 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:57 smithi138 ceph-mon[35948]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-05-17T21:47:57.740 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:57 smithi138 ceph-mon[35948]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-05-17T21:47:57.740 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:57 smithi138 ceph-mon[35948]: osdmap e38: 6 total, 5 up, 6 in 2024-05-17T21:47:57.740 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:57 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-17T21:47:57.740 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:57 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-17T21:47:57.741 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:57 smithi138 ceph-mon[35948]: from='client.? 172.21.15.107:0/2091788928' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-17T21:47:57.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:57 smithi107 ceph-mon[28434]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-05-17T21:47:57.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:57 smithi107 ceph-mon[28434]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-05-17T21:47:57.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:57 smithi107 ceph-mon[28434]: osdmap e38: 6 total, 5 up, 6 in 2024-05-17T21:47:57.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:57 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-17T21:47:57.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:57 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-17T21:47:57.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:57 smithi107 ceph-mon[28434]: from='client.? 172.21.15.107:0/2091788928' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-17T21:47:58.110 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":38,"num_osds":6,"num_up_osds":5,"osd_up_since":1715982452,"num_in_osds":6,"osd_in_since":1715982458,"num_remapped_pgs":0} 2024-05-17T21:47:58.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:58 smithi138 ceph-mon[35948]: purged_snaps scrub starts 2024-05-17T21:47:58.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:58 smithi138 ceph-mon[35948]: purged_snaps scrub ok 2024-05-17T21:47:58.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:58 smithi138 ceph-mon[35948]: osd.5 [v2:172.21.15.138:6816/406753522,v1:172.21.15.138:6817/406753522] boot 2024-05-17T21:47:58.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:58 smithi138 ceph-mon[35948]: osdmap e39: 6 total, 6 up, 6 in 2024-05-17T21:47:58.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:58 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-17T21:47:58.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:58 smithi107 ceph-mon[28434]: purged_snaps scrub starts 2024-05-17T21:47:58.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:58 smithi107 ceph-mon[28434]: purged_snaps scrub ok 2024-05-17T21:47:58.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:58 smithi107 ceph-mon[28434]: osd.5 [v2:172.21.15.138:6816/406753522,v1:172.21.15.138:6817/406753522] boot 2024-05-17T21:47:58.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:58 smithi107 ceph-mon[28434]: osdmap e39: 6 total, 6 up, 6 in 2024-05-17T21:47:58.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:58 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-17T21:47:59.110 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -- ceph osd stat -f json 2024-05-17T21:47:59.375 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/mon.smithi107/config 2024-05-17T21:47:59.812 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:59 smithi107 ceph-mon[28434]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-05-17T21:47:59.812 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:59 smithi107 ceph-mon[28434]: osdmap e40: 6 total, 6 up, 6 in 2024-05-17T21:47:59.812 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:59 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:47:59.812 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:59 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:47:59.812 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:59 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-17T21:47:59.812 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:59 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-17T21:47:59.813 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:59 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-17T21:47:59.813 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:59 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:47:59.813 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:59 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:47:59.813 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:59 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T21:47:59.813 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:59 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:47:59.813 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:59 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:47:59.813 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:59 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:47:59.813 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:59 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T21:47:59.813 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:47:59 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:47:59.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:59 smithi138 ceph-mon[35948]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-05-17T21:47:59.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:59 smithi138 ceph-mon[35948]: osdmap e40: 6 total, 6 up, 6 in 2024-05-17T21:47:59.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:59 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:47:59.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:59 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:47:59.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:59 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-17T21:47:59.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:59 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-17T21:47:59.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:59 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-17T21:47:59.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:59 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:47:59.828 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:59 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:47:59.828 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:59 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T21:47:59.828 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:59 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:47:59.828 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:59 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:47:59.828 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:59 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:47:59.828 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:59 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T21:47:59.828 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:47:59 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:48:00.726 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:48:00 smithi107 ceph-mon[28434]: Detected new or changed devices on smithi138 2024-05-17T21:48:00.726 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:48:00 smithi107 ceph-mon[28434]: Adjusting osd_memory_target on smithi138 to 5312M 2024-05-17T21:48:00.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:00.949+0000 7fd20ef22700 1 -- 172.21.15.107:0/1140352591 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2080fcf50 msgr2=0x7fd2080fd3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:48:00.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:00.949+0000 7fd20ef22700 1 --2- 172.21.15.107:0/1140352591 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2080fcf50 0x7fd2080fd3c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd1f4004940 tx=0x7fd1f4009e30 comp rx=0 tx=0).stop 2024-05-17T21:48:00.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:00.950+0000 7fd20ef22700 1 -- 172.21.15.107:0/1140352591 shutdown_connections 2024-05-17T21:48:00.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:00.950+0000 7fd20ef22700 1 --2- 172.21.15.107:0/1140352591 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2080fcf50 0x7fd2080fd3c0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:00.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:00.950+0000 7fd20ef22700 1 --2- 172.21.15.107:0/1140352591 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2080fbca0 0x7fd2080fc0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:00.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:00.950+0000 7fd20ef22700 1 -- 172.21.15.107:0/1140352591 >> 172.21.15.107:0/1140352591 conn(0x7fd2080f7420 msgr2=0x7fd2080f9830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:48:00.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:00.950+0000 7fd20ef22700 1 -- 172.21.15.107:0/1140352591 shutdown_connections 2024-05-17T21:48:00.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:00.950+0000 7fd20ef22700 1 -- 172.21.15.107:0/1140352591 wait complete. 2024-05-17T21:48:00.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:00.950+0000 7fd20ef22700 1 Processor -- start 2024-05-17T21:48:00.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:00.950+0000 7fd20ef22700 1 -- start start 2024-05-17T21:48:00.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:00.950+0000 7fd20ef22700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2080fbca0 0x7fd20810d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:48:00.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:00.950+0000 7fd20ef22700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2080fcf50 0x7fd20810de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:48:00.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:00.950+0000 7fd20ef22700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd20810e4c0 con 0x7fd2080fcf50 2024-05-17T21:48:00.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:00.950+0000 7fd20ef22700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd208112e60 con 0x7fd2080fbca0 2024-05-17T21:48:00.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:00.951+0000 7fd207fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2080fcf50 0x7fd20810de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:48:00.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:00.951+0000 7fd207fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2080fcf50 0x7fd20810de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52256/0 (socket says 172.21.15.107:52256) 2024-05-17T21:48:00.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:00.951+0000 7fd207fff700 1 -- 172.21.15.107:0/917588207 learned_addr learned my addr 172.21.15.107:0/917588207 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:48:00.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:00.951+0000 7fd20ccbe700 1 --2- 172.21.15.107:0/917588207 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2080fbca0 0x7fd20810d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:48:00.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:00.951+0000 7fd207fff700 1 -- 172.21.15.107:0/917588207 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2080fbca0 msgr2=0x7fd20810d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:48:00.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:00.951+0000 7fd207fff700 1 --2- 172.21.15.107:0/917588207 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2080fbca0 0x7fd20810d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:00.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:00.951+0000 7fd207fff700 1 -- 172.21.15.107:0/917588207 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1f4005040 con 0x7fd2080fcf50 2024-05-17T21:48:00.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:00.951+0000 7fd20ccbe700 1 --2- 172.21.15.107:0/917588207 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2080fbca0 0x7fd20810d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:48:00.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:00.951+0000 7fd207fff700 1 --2- 172.21.15.107:0/917588207 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2080fcf50 0x7fd20810de30 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd1f400b3b0 tx=0x7fd1f4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:48:00.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:00.951+0000 7fd2057fa700 1 -- 172.21.15.107:0/917588207 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd1f400e040 con 0x7fd2080fcf50 2024-05-17T21:48:00.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:00.951+0000 7fd2057fa700 1 -- 172.21.15.107:0/917588207 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd1f400bcc0 con 0x7fd2080fcf50 2024-05-17T21:48:00.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:00.951+0000 7fd20ef22700 1 -- 172.21.15.107:0/917588207 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd208113060 con 0x7fd2080fcf50 2024-05-17T21:48:00.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:00.951+0000 7fd2057fa700 1 -- 172.21.15.107:0/917588207 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd1f401c630 con 0x7fd2080fcf50 2024-05-17T21:48:00.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:00.951+0000 7fd20ef22700 1 -- 172.21.15.107:0/917588207 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2081135d0 con 0x7fd2080fcf50 2024-05-17T21:48:00.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:00.952+0000 7fd20ef22700 1 -- 172.21.15.107:0/917588207 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd208101ae0 con 0x7fd2080fcf50 2024-05-17T21:48:00.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:00.953+0000 7fd2057fa700 1 -- 172.21.15.107:0/917588207 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7fd1f4013980 con 0x7fd2080fcf50 2024-05-17T21:48:00.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:00.953+0000 7fd2057fa700 1 --2- 172.21.15.107:0/917588207 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fd1f80692a0 0x7fd1f806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:48:00.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:00.953+0000 7fd2057fa700 1 -- 172.21.15.107:0/917588207 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fd1f4087c80 con 0x7fd2080fcf50 2024-05-17T21:48:00.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:00.953+0000 7fd20ccbe700 1 --2- 172.21.15.107:0/917588207 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fd1f80692a0 0x7fd1f806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:48:00.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:00.953+0000 7fd20ccbe700 1 --2- 172.21.15.107:0/917588207 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fd1f80692a0 0x7fd1f806b750 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc006fd0 tx=0x7fd1fc008040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:48:00.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:00.958+0000 7fd2057fa700 1 -- 172.21.15.107:0/917588207 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fd1f4057d00 con 0x7fd2080fcf50 2024-05-17T21:48:01.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:48:00 smithi138 ceph-mon[35948]: Detected new or changed devices on smithi138 2024-05-17T21:48:01.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:48:00 smithi138 ceph-mon[35948]: Adjusting osd_memory_target on smithi138 to 5312M 2024-05-17T21:48:01.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:01.143+0000 7fd20ef22700 1 -- 172.21.15.107:0/917588207 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7fd20804c710 con 0x7fd2080fcf50 2024-05-17T21:48:01.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:01.143+0000 7fd2057fa700 1 -- 172.21.15.107:0/917588207 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v40) v1 ==== 74+0+130 (secure 0 0 0) 0x7fd1f4057890 con 0x7fd2080fcf50 2024-05-17T21:48:01.136 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:48:01.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:01.145+0000 7fd20ef22700 1 -- 172.21.15.107:0/917588207 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fd1f80692a0 msgr2=0x7fd1f806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:48:01.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:01.145+0000 7fd20ef22700 1 --2- 172.21.15.107:0/917588207 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fd1f80692a0 0x7fd1f806b750 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc006fd0 tx=0x7fd1fc008040 comp rx=0 tx=0).stop 2024-05-17T21:48:01.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:01.145+0000 7fd20ef22700 1 -- 172.21.15.107:0/917588207 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2080fcf50 msgr2=0x7fd20810de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:48:01.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:01.145+0000 7fd20ef22700 1 --2- 172.21.15.107:0/917588207 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2080fcf50 0x7fd20810de30 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd1f400b3b0 tx=0x7fd1f4009e30 comp rx=0 tx=0).stop 2024-05-17T21:48:01.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:01.146+0000 7fd20ef22700 1 -- 172.21.15.107:0/917588207 shutdown_connections 2024-05-17T21:48:01.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:01.146+0000 7fd20ef22700 1 --2- 172.21.15.107:0/917588207 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fd1f80692a0 0x7fd1f806b750 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:01.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:01.146+0000 7fd20ef22700 1 --2- 172.21.15.107:0/917588207 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2080fbca0 0x7fd20810d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:01.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:01.146+0000 7fd20ef22700 1 --2- 172.21.15.107:0/917588207 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2080fcf50 0x7fd20810de30 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:01.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:01.146+0000 7fd20ef22700 1 -- 172.21.15.107:0/917588207 >> 172.21.15.107:0/917588207 conn(0x7fd2080f7420 msgr2=0x7fd2080f89a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:48:01.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:01.146+0000 7fd20ef22700 1 -- 172.21.15.107:0/917588207 shutdown_connections 2024-05-17T21:48:01.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:01.146+0000 7fd20ef22700 1 -- 172.21.15.107:0/917588207 wait complete. 2024-05-17T21:48:01.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:48:01 smithi107 ceph-mon[28434]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-17T21:48:01.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:48:01 smithi107 ceph-mon[28434]: from='client.? 172.21.15.107:0/917588207' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-17T21:48:01.943 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":40,"num_osds":6,"num_up_osds":6,"osd_up_since":1715982477,"num_in_osds":6,"osd_in_since":1715982458,"num_remapped_pgs":0} 2024-05-17T21:48:01.943 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -- ceph osd dump --format=json 2024-05-17T21:48:02.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:48:01 smithi138 ceph-mon[35948]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-17T21:48:02.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:48:01 smithi138 ceph-mon[35948]: from='client.? 172.21.15.107:0/917588207' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-17T21:48:02.197 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/mon.smithi107/config 2024-05-17T21:48:02.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:48:02 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:48:02.978 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:48:02 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:48:03.702 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:48:03 smithi107 ceph-mon[28434]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-17T21:48:03.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:03.938+0000 7f40fc99f700 1 -- 172.21.15.107:0/1889834948 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40f8101b10 msgr2=0x7f40f8101ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:48:03.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:03.938+0000 7f40fc99f700 1 --2- 172.21.15.107:0/1889834948 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40f8101b10 0x7f40f8101ee0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f40e8004940 tx=0x7f40e8009e30 comp rx=0 tx=0).stop 2024-05-17T21:48:03.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:03.938+0000 7f40fc99f700 1 -- 172.21.15.107:0/1889834948 shutdown_connections 2024-05-17T21:48:03.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:03.938+0000 7f40fc99f700 1 --2- 172.21.15.107:0/1889834948 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f40f80fbb50 0x7f40f80fbfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:03.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:03.938+0000 7f40fc99f700 1 --2- 172.21.15.107:0/1889834948 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40f8101b10 0x7f40f8101ee0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:03.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:03.938+0000 7f40fc99f700 1 -- 172.21.15.107:0/1889834948 >> 172.21.15.107:0/1889834948 conn(0x7f40f80f7650 msgr2=0x7f40f80f9a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:48:03.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:03.938+0000 7f40fc99f700 1 -- 172.21.15.107:0/1889834948 shutdown_connections 2024-05-17T21:48:03.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:03.938+0000 7f40fc99f700 1 -- 172.21.15.107:0/1889834948 wait complete. 2024-05-17T21:48:03.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:03.939+0000 7f40fc99f700 1 Processor -- start 2024-05-17T21:48:03.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:03.939+0000 7f40fc99f700 1 -- start start 2024-05-17T21:48:03.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:03.939+0000 7f40fc99f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40f80fbb50 0x7f40f8113990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:48:03.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:03.939+0000 7f40fc99f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f40f8101b10 0x7f40f8113ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:48:03.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:03.939+0000 7f40fc99f700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40f8114520 con 0x7f40f80fbb50 2024-05-17T21:48:03.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:03.939+0000 7f40fc99f700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40f810d910 con 0x7f40f8101b10 2024-05-17T21:48:03.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:03.939+0000 7f40f77fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40f80fbb50 0x7f40f8113990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:48:03.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:03.939+0000 7f40f77fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40f80fbb50 0x7f40f8113990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52266/0 (socket says 172.21.15.107:52266) 2024-05-17T21:48:03.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:03.939+0000 7f40f77fe700 1 -- 172.21.15.107:0/2889215912 learned_addr learned my addr 172.21.15.107:0/2889215912 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:48:03.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:03.939+0000 7f40effff700 1 --2- 172.21.15.107:0/2889215912 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f40f8101b10 0x7f40f8113ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:48:03.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:03.939+0000 7f40f77fe700 1 -- 172.21.15.107:0/2889215912 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f40f8101b10 msgr2=0x7f40f8113ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:48:03.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:03.939+0000 7f40f77fe700 1 --2- 172.21.15.107:0/2889215912 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f40f8101b10 0x7f40f8113ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:03.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:03.939+0000 7f40f77fe700 1 -- 172.21.15.107:0/2889215912 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40e8005040 con 0x7f40f80fbb50 2024-05-17T21:48:03.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:03.939+0000 7f40f77fe700 1 --2- 172.21.15.107:0/2889215912 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40f80fbb50 0x7f40f8113990 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f40e800b3b0 tx=0x7f40e8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:48:03.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:03.939+0000 7f40f4ff9700 1 -- 172.21.15.107:0/2889215912 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f40e800e040 con 0x7f40f80fbb50 2024-05-17T21:48:03.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:03.940+0000 7f40fc99f700 1 -- 172.21.15.107:0/2889215912 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40f810db90 con 0x7f40f80fbb50 2024-05-17T21:48:03.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:03.940+0000 7f40f4ff9700 1 -- 172.21.15.107:0/2889215912 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f40e8013420 con 0x7f40f80fbb50 2024-05-17T21:48:03.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:03.940+0000 7f40f4ff9700 1 -- 172.21.15.107:0/2889215912 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f40e801c630 con 0x7f40f80fbb50 2024-05-17T21:48:03.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:03.940+0000 7f40fc99f700 1 -- 172.21.15.107:0/2889215912 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40f810e150 con 0x7f40f80fbb50 2024-05-17T21:48:03.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:03.940+0000 7f40fc99f700 1 -- 172.21.15.107:0/2889215912 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40f80fd200 con 0x7f40f80fbb50 2024-05-17T21:48:03.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:03.941+0000 7f40f4ff9700 1 -- 172.21.15.107:0/2889215912 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f40e8013590 con 0x7f40f80fbb50 2024-05-17T21:48:03.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:03.941+0000 7f40f4ff9700 1 --2- 172.21.15.107:0/2889215912 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f40e40692a0 0x7f40e406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:48:03.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:03.941+0000 7f40f4ff9700 1 -- 172.21.15.107:0/2889215912 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f40e8087fc0 con 0x7f40f80fbb50 2024-05-17T21:48:03.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:03.941+0000 7f40effff700 1 --2- 172.21.15.107:0/2889215912 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f40e40692a0 0x7f40e406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:48:03.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:03.942+0000 7f40effff700 1 --2- 172.21.15.107:0/2889215912 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f40e40692a0 0x7f40e406b750 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f40f810ead0 tx=0x7f40e000b040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:48:03.938 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:03.946+0000 7f40f4ff9700 1 -- 172.21.15.107:0/2889215912 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f40e80580c0 con 0x7f40f80fbb50 2024-05-17T21:48:04.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:48:03 smithi138 ceph-mon[35948]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-17T21:48:04.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:04.132+0000 7f40fc99f700 1 -- 172.21.15.107:0/2889215912 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f40f8108c00 con 0x7f40f80fbb50 2024-05-17T21:48:04.126 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:04.133+0000 7f40f4ff9700 1 -- 172.21.15.107:0/2889215912 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+11098 (secure 0 0 0) 0x7f40e8013840 con 0x7f40f80fbb50 2024-05-17T21:48:04.126 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:48:04.129 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":40,"fsid":"3722731a-1496-11ef-bc9a-c7b262605968","created":"2024-05-17T21:42:15.126179+0000","modified":"2024-05-17T21:47:58.575311+0000","last_up_change":"2024-05-17T21:47:57.569583+0000","last_in_change":"2024-05-17T21:47:38.526939+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-17T21:46:42.602884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"203debeb-d443-4b99-a929-64afe799acfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":710062679},{"type":"v1","addr":"172.21.15.107:6803","nonce":710062679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":710062679},{"type":"v1","addr":"172.21.15.107:6805","nonce":710062679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":710062679},{"type":"v1","addr":"172.21.15.107:6809","nonce":710062679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":710062679},{"type":"v1","addr":"172.21.15.107:6807","nonce":710062679}]},"public_addr":"172.21.15.107:6803/710062679","cluster_addr":"172.21.15.107:6805/710062679","heartbeat_back_addr":"172.21.15.107:6809/710062679","heartbeat_front_addr":"172.21.15.107:6807/710062679","state":["exists","up"]},{"osd":1,"uuid":"2052c88c-a0b5-4f5b-8ee0-68528e607d9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":686958797},{"type":"v1","addr":"172.21.15.107:6811","nonce":686958797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":686958797},{"type":"v1","addr":"172.21.15.107:6813","nonce":686958797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":686958797},{"type":"v1","addr":"172.21.15.107:6817","nonce":686958797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":686958797},{"type":"v1","addr":"172.21.15.107:6815","nonce":686958797}]},"public_addr":"172.21.15.107:6811/686958797","cluster_addr":"172.21.15.107:6813/686958797","heartbeat_back_addr":"172.21.15.107:6817/686958797","heartbeat_front_addr":"172.21.15.107:6815/686958797","state":["exists","up"]},{"osd":2,"uuid":"b5dc6be0-2ac3-4769-ae70-14c23381506b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":3962264726},{"type":"v1","addr":"172.21.15.107:6819","nonce":3962264726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":3962264726},{"type":"v1","addr":"172.21.15.107:6821","nonce":3962264726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3962264726},{"type":"v1","addr":"172.21.15.107:6825","nonce":3962264726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":3962264726},{"type":"v1","addr":"172.21.15.107:6823","nonce":3962264726}]},"public_addr":"172.21.15.107:6819/3962264726","cluster_addr":"172.21.15.107:6821/3962264726","heartbeat_back_addr":"172.21.15.107:6825/3962264726","heartbeat_front_addr":"172.21.15.107:6823/3962264726","state":["exists","up"]},{"osd":3,"uuid":"5f3d4515-afed-481f-8e08-debaf33a9e0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2477238048},{"type":"v1","addr":"172.21.15.138:6801","nonce":2477238048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2477238048},{"type":"v1","addr":"172.21.15.138:6803","nonce":2477238048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2477238048},{"type":"v1","addr":"172.21.15.138:6807","nonce":2477238048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2477238048},{"type":"v1","addr":"172.21.15.138:6805","nonce":2477238048}]},"public_addr":"172.21.15.138:6801/2477238048","cluster_addr":"172.21.15.138:6803/2477238048","heartbeat_back_addr":"172.21.15.138:6807/2477238048","heartbeat_front_addr":"172.21.15.138:6805/2477238048","state":["exists","up"]},{"osd":4,"uuid":"d28515a1-4510-4f94-b0b1-3aca283eed05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":792878465},{"type":"v1","addr":"172.21.15.138:6809","nonce":792878465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":792878465},{"type":"v1","addr":"172.21.15.138:6811","nonce":792878465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":792878465},{"type":"v1","addr":"172.21.15.138:6815","nonce":792878465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":792878465},{"type":"v1","addr":"172.21.15.138:6813","nonce":792878465}]},"public_addr":"172.21.15.138:6809/792878465","cluster_addr":"172.21.15.138:6811/792878465","heartbeat_back_addr":"172.21.15.138:6815/792878465","heartbeat_front_addr":"172.21.15.138:6813/792878465","state":["exists","up"]},{"osd":5,"uuid":"ab57d399-f1cf-4139-abba-7cb15e433c25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":406753522},{"type":"v1","addr":"172.21.15.138:6817","nonce":406753522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":406753522},{"type":"v1","addr":"172.21.15.138:6819","nonce":406753522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":406753522},{"type":"v1","addr":"172.21.15.138:6823","nonce":406753522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":406753522},{"type":"v1","addr":"172.21.15.138:6821","nonce":406753522}]},"public_addr":"172.21.15.138:6817/406753522","cluster_addr":"172.21.15.138:6819/406753522","heartbeat_back_addr":"172.21.15.138:6823/406753522","heartbeat_front_addr":"172.21.15.138:6821/406753522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:45:44.441319+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:46:11.518607+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:46:40.661238+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:47:04.556335+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:47:30.603032+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:47:55.950499+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/2414403835":"2024-05-18T21:44:32.293094+0000","172.21.15.107:6801/3405524625":"2024-05-18T21:44:32.293094+0000","172.21.15.107:6800/3405524625":"2024-05-18T21:44:32.293094+0000","172.21.15.107:0/771372867":"2024-05-18T21:44:32.293094+0000","172.21.15.107:0/498460124":"2024-05-18T21:43:22.097723+0000","172.21.15.107:0/3170894163":"2024-05-18T21:43:22.097723+0000","172.21.15.107:6801/1718632476":"2024-05-18T21:43:22.097723+0000","172.21.15.107:6801/1677232931":"2024-05-18T21:42:52.099188+0000","172.21.15.107:0/2856704707":"2024-05-18T21:43:22.097723+0000","172.21.15.107:0/3048013464":"2024-05-18T21:42:52.099188+0000","172.21.15.107:0/2872531133":"2024-05-18T21:42:52.099188+0000","172.21.15.107:0/1077444017":"2024-05-18T21:44:32.293094+0000","172.21.15.107:0/3105018805":"2024-05-18T21:42:52.099188+0000","172.21.15.107:6800/1677232931":"2024-05-18T21:42:52.099188+0000","172.21.15.107:6800/1718632476":"2024-05-18T21:43:22.097723+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-17T21:48:04.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:04.135+0000 7f40fc99f700 1 -- 172.21.15.107:0/2889215912 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f40e40692a0 msgr2=0x7f40e406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:48:04.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:04.135+0000 7f40fc99f700 1 --2- 172.21.15.107:0/2889215912 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f40e40692a0 0x7f40e406b750 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f40f810ead0 tx=0x7f40e000b040 comp rx=0 tx=0).stop 2024-05-17T21:48:04.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:04.135+0000 7f40fc99f700 1 -- 172.21.15.107:0/2889215912 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40f80fbb50 msgr2=0x7f40f8113990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:48:04.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:04.135+0000 7f40fc99f700 1 --2- 172.21.15.107:0/2889215912 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40f80fbb50 0x7f40f8113990 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f40e800b3b0 tx=0x7f40e8009e30 comp rx=0 tx=0).stop 2024-05-17T21:48:04.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:04.135+0000 7f40fc99f700 1 -- 172.21.15.107:0/2889215912 shutdown_connections 2024-05-17T21:48:04.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:04.135+0000 7f40fc99f700 1 --2- 172.21.15.107:0/2889215912 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f40e40692a0 0x7f40e406b750 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:04.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:04.135+0000 7f40fc99f700 1 --2- 172.21.15.107:0/2889215912 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40f80fbb50 0x7f40f8113990 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:04.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:04.135+0000 7f40fc99f700 1 --2- 172.21.15.107:0/2889215912 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f40f8101b10 0x7f40f8113ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:04.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:04.135+0000 7f40fc99f700 1 -- 172.21.15.107:0/2889215912 >> 172.21.15.107:0/2889215912 conn(0x7f40f80f7650 msgr2=0x7f40f8066050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:48:04.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:04.135+0000 7f40fc99f700 1 -- 172.21.15.107:0/2889215912 shutdown_connections 2024-05-17T21:48:04.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:04.135+0000 7f40fc99f700 1 -- 172.21.15.107:0/2889215912 wait complete. 2024-05-17T21:48:04.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:48:04 smithi107 ceph-mon[28434]: from='client.? 172.21.15.107:0/2889215912' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T21:48:04.877 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-17T21:46:42.602884+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2024-05-17T21:48:04.878 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-05-17T21:48:05.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:48:04 smithi138 ceph-mon[35948]: from='client.? 172.21.15.107:0/2889215912' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T21:48:05.132 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/mon.smithi107/config 2024-05-17T21:48:05.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:48:05 smithi107 ceph-mon[28434]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-17T21:48:06.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:48:05 smithi138 ceph-mon[35948]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-17T21:48:06.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:06.648+0000 7f53ddda6700 1 -- 172.21.15.107:0/1019214008 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53d8102e20 msgr2=0x7f53d81031f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:48:06.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:06.648+0000 7f53ddda6700 1 --2- 172.21.15.107:0/1019214008 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53d8102e20 0x7f53d81031f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f53c8009ce0 tx=0x7f53c8004d10 comp rx=0 tx=0).stop 2024-05-17T21:48:06.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:06.648+0000 7f53ddda6700 1 -- 172.21.15.107:0/1019214008 shutdown_connections 2024-05-17T21:48:06.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:06.648+0000 7f53ddda6700 1 --2- 172.21.15.107:0/1019214008 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53d81037c0 0x7f53d80fb3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:06.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:06.648+0000 7f53ddda6700 1 --2- 172.21.15.107:0/1019214008 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53d8102e20 0x7f53d81031f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:06.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:06.648+0000 7f53ddda6700 1 -- 172.21.15.107:0/1019214008 >> 172.21.15.107:0/1019214008 conn(0x7f53d8072380 msgr2=0x7f53d8072780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:48:06.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:06.648+0000 7f53ddda6700 1 -- 172.21.15.107:0/1019214008 shutdown_connections 2024-05-17T21:48:06.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:06.648+0000 7f53ddda6700 1 -- 172.21.15.107:0/1019214008 wait complete. 2024-05-17T21:48:06.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:06.648+0000 7f53ddda6700 1 Processor -- start 2024-05-17T21:48:06.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:06.648+0000 7f53ddda6700 1 -- start start 2024-05-17T21:48:06.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:06.649+0000 7f53ddda6700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53d8102e20 0x7f53d8113bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:48:06.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:06.649+0000 7f53ddda6700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53d81037c0 0x7f53d81140f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:48:06.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:06.649+0000 7f53ddda6700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53d81147d0 con 0x7f53d81037c0 2024-05-17T21:48:06.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:06.649+0000 7f53ddda6700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53d810dd20 con 0x7f53d8102e20 2024-05-17T21:48:06.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:06.649+0000 7f53d6ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53d81037c0 0x7f53d81140f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:48:06.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:06.649+0000 7f53d6ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53d81037c0 0x7f53d81140f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33896/0 (socket says 172.21.15.107:33896) 2024-05-17T21:48:06.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:06.649+0000 7f53d6ffd700 1 -- 172.21.15.107:0/2983543128 learned_addr learned my addr 172.21.15.107:0/2983543128 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:48:06.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:06.649+0000 7f53d77fe700 1 --2- 172.21.15.107:0/2983543128 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53d8102e20 0x7f53d8113bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:48:06.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:06.649+0000 7f53d6ffd700 1 -- 172.21.15.107:0/2983543128 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53d8102e20 msgr2=0x7f53d8113bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:48:06.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:06.649+0000 7f53d6ffd700 1 --2- 172.21.15.107:0/2983543128 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53d8102e20 0x7f53d8113bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:06.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:06.649+0000 7f53d6ffd700 1 -- 172.21.15.107:0/2983543128 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53c8009940 con 0x7f53d81037c0 2024-05-17T21:48:06.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:06.649+0000 7f53d6ffd700 1 --2- 172.21.15.107:0/2983543128 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53d81037c0 0x7f53d81140f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f53cc00b790 tx=0x7f53cc009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:48:06.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:06.649+0000 7f53bffff700 1 -- 172.21.15.107:0/2983543128 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f53cc00f6f0 con 0x7f53d81037c0 2024-05-17T21:48:06.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:06.649+0000 7f53bffff700 1 -- 172.21.15.107:0/2983543128 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f53cc00fd20 con 0x7f53d81037c0 2024-05-17T21:48:06.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:06.649+0000 7f53bffff700 1 -- 172.21.15.107:0/2983543128 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f53cc0103a0 con 0x7f53d81037c0 2024-05-17T21:48:06.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:06.649+0000 7f53ddda6700 1 -- 172.21.15.107:0/2983543128 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53d810dfa0 con 0x7f53d81037c0 2024-05-17T21:48:06.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:06.649+0000 7f53ddda6700 1 -- 172.21.15.107:0/2983543128 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53d810e4e0 con 0x7f53d81037c0 2024-05-17T21:48:06.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:06.650+0000 7f53ddda6700 1 -- 172.21.15.107:0/2983543128 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53d80fe6e0 con 0x7f53d81037c0 2024-05-17T21:48:06.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:06.651+0000 7f53bffff700 1 -- 172.21.15.107:0/2983543128 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f53cc011400 con 0x7f53d81037c0 2024-05-17T21:48:06.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:06.651+0000 7f53bffff700 1 --2- 172.21.15.107:0/2983543128 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f53c00692a0 0x7f53c006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:48:06.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:06.651+0000 7f53d77fe700 1 --2- 172.21.15.107:0/2983543128 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f53c00692a0 0x7f53c006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:48:06.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:06.651+0000 7f53bffff700 1 -- 172.21.15.107:0/2983543128 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f53cc086d20 con 0x7f53d81037c0 2024-05-17T21:48:06.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:06.652+0000 7f53d77fe700 1 --2- 172.21.15.107:0/2983543128 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f53c00692a0 0x7f53c006b750 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f53c8005010 tx=0x7f53c8004d10 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:48:06.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:06.656+0000 7f53bffff700 1 -- 172.21.15.107:0/2983543128 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f53cc056da0 con 0x7f53d81037c0 2024-05-17T21:48:06.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:06.837+0000 7f53ddda6700 1 -- 172.21.15.107:0/2983543128 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f53d804c710 con 0x7f53d81037c0 2024-05-17T21:48:06.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:06.838+0000 7f53bffff700 1 -- 172.21.15.107:0/2983543128 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) v1 ==== 93+0+10 (secure 0 0 0) 0x7f53cc056930 con 0x7f53d81037c0 2024-05-17T21:48:06.831 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 1 2024-05-17T21:48:06.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:06.840+0000 7f53ddda6700 1 -- 172.21.15.107:0/2983543128 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f53c00692a0 msgr2=0x7f53c006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:48:06.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:06.840+0000 7f53ddda6700 1 --2- 172.21.15.107:0/2983543128 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f53c00692a0 0x7f53c006b750 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f53c8005010 tx=0x7f53c8004d10 comp rx=0 tx=0).stop 2024-05-17T21:48:06.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:06.840+0000 7f53ddda6700 1 -- 172.21.15.107:0/2983543128 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53d81037c0 msgr2=0x7f53d81140f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:48:06.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:06.840+0000 7f53ddda6700 1 --2- 172.21.15.107:0/2983543128 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53d81037c0 0x7f53d81140f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f53cc00b790 tx=0x7f53cc009220 comp rx=0 tx=0).stop 2024-05-17T21:48:06.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:06.840+0000 7f53ddda6700 1 -- 172.21.15.107:0/2983543128 shutdown_connections 2024-05-17T21:48:06.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:06.840+0000 7f53ddda6700 1 --2- 172.21.15.107:0/2983543128 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f53c00692a0 0x7f53c006b750 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:06.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:06.840+0000 7f53ddda6700 1 --2- 172.21.15.107:0/2983543128 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53d8102e20 0x7f53d8113bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:06.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:06.840+0000 7f53ddda6700 1 --2- 172.21.15.107:0/2983543128 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53d81037c0 0x7f53d81140f0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:06.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:06.840+0000 7f53ddda6700 1 -- 172.21.15.107:0/2983543128 >> 172.21.15.107:0/2983543128 conn(0x7f53d8072380 msgr2=0x7f53d80f9d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:48:06.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:06.840+0000 7f53ddda6700 1 -- 172.21.15.107:0/2983543128 shutdown_connections 2024-05-17T21:48:06.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:06.840+0000 7f53ddda6700 1 -- 172.21.15.107:0/2983543128 wait complete. 2024-05-17T21:48:07.474 INFO:tasks.cephadm:Setting up client nodes... 2024-05-17T21:48:07.475 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-05-17T21:48:07.729 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/mon.smithi107/config 2024-05-17T21:48:07.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:48:07 smithi138 ceph-mon[35948]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-17T21:48:07.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:48:07 smithi138 ceph-mon[35948]: from='client.? 172.21.15.107:0/2983543128' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-17T21:48:07.849 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:48:07 smithi107 ceph-mon[28434]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-17T21:48:07.849 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:48:07 smithi107 ceph-mon[28434]: from='client.? 172.21.15.107:0/2983543128' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-17T21:48:09.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:09.431+0000 7f361a410700 1 -- 172.21.15.107:0/3055789239 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f36140fce60 msgr2=0x7f36140fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:48:09.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:09.431+0000 7f361a410700 1 --2- 172.21.15.107:0/3055789239 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f36140fce60 0x7f36140fd270 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f3604004940 tx=0x7f3604009e30 comp rx=0 tx=0).stop 2024-05-17T21:48:09.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:09.431+0000 7f361a410700 1 -- 172.21.15.107:0/3055789239 shutdown_connections 2024-05-17T21:48:09.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:09.431+0000 7f361a410700 1 --2- 172.21.15.107:0/3055789239 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f36140fdfd0 0x7f36140fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:09.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:09.431+0000 7f361a410700 1 --2- 172.21.15.107:0/3055789239 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f36140fce60 0x7f36140fd270 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:09.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:09.431+0000 7f361a410700 1 -- 172.21.15.107:0/3055789239 >> 172.21.15.107:0/3055789239 conn(0x7f36140f8540 msgr2=0x7f36140fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:48:09.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:09.431+0000 7f361a410700 1 -- 172.21.15.107:0/3055789239 shutdown_connections 2024-05-17T21:48:09.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:09.431+0000 7f361a410700 1 -- 172.21.15.107:0/3055789239 wait complete. 2024-05-17T21:48:09.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:09.432+0000 7f361a410700 1 Processor -- start 2024-05-17T21:48:09.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:09.432+0000 7f361a410700 1 -- start start 2024-05-17T21:48:09.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:09.432+0000 7f361a410700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f36140fce60 0x7f361410d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:48:09.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:09.432+0000 7f361a410700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f36140fdfd0 0x7f361410de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:48:09.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:09.432+0000 7f361a410700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f361410e460 con 0x7f36140fdfd0 2024-05-17T21:48:09.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:09.432+0000 7f361a410700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f361410e5d0 con 0x7f36140fce60 2024-05-17T21:48:09.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:09.432+0000 7f36137fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f36140fdfd0 0x7f361410de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:48:09.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:09.432+0000 7f36137fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f36140fdfd0 0x7f361410de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33906/0 (socket says 172.21.15.107:33906) 2024-05-17T21:48:09.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:09.432+0000 7f36137fe700 1 -- 172.21.15.107:0/2958294452 learned_addr learned my addr 172.21.15.107:0/2958294452 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:48:09.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:09.433+0000 7f3613fff700 1 --2- 172.21.15.107:0/2958294452 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f36140fce60 0x7f361410d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:48:09.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:09.433+0000 7f36137fe700 1 -- 172.21.15.107:0/2958294452 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f36140fce60 msgr2=0x7f361410d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:48:09.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:09.433+0000 7f36137fe700 1 --2- 172.21.15.107:0/2958294452 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f36140fce60 0x7f361410d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:09.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:09.433+0000 7f36137fe700 1 -- 172.21.15.107:0/2958294452 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3604005040 con 0x7f36140fdfd0 2024-05-17T21:48:09.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:09.433+0000 7f3613fff700 1 --2- 172.21.15.107:0/2958294452 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f36140fce60 0x7f361410d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T21:48:09.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:09.433+0000 7f36137fe700 1 --2- 172.21.15.107:0/2958294452 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f36140fdfd0 0x7f361410de60 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3608002a50 tx=0x7f36080046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:48:09.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:09.433+0000 7f3610ff9700 1 -- 172.21.15.107:0/2958294452 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3608007cc0 con 0x7f36140fdfd0 2024-05-17T21:48:09.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:09.433+0000 7f3610ff9700 1 -- 172.21.15.107:0/2958294452 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f360800f040 con 0x7f36140fdfd0 2024-05-17T21:48:09.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:09.433+0000 7f361a410700 1 -- 172.21.15.107:0/2958294452 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3614113030 con 0x7f36140fdfd0 2024-05-17T21:48:09.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:09.433+0000 7f3610ff9700 1 -- 172.21.15.107:0/2958294452 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3608009640 con 0x7f36140fdfd0 2024-05-17T21:48:09.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:09.433+0000 7f361a410700 1 -- 172.21.15.107:0/2958294452 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36141135c0 con 0x7f36140fdfd0 2024-05-17T21:48:09.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:09.434+0000 7f361a410700 1 -- 172.21.15.107:0/2958294452 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f361404c710 con 0x7f36140fdfd0 2024-05-17T21:48:09.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:09.435+0000 7f3610ff9700 1 -- 172.21.15.107:0/2958294452 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f36080097c0 con 0x7f36140fdfd0 2024-05-17T21:48:09.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:09.435+0000 7f3610ff9700 1 --2- 172.21.15.107:0/2958294452 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f35fc0692a0 0x7f35fc06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:48:09.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:09.435+0000 7f3610ff9700 1 -- 172.21.15.107:0/2958294452 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f3608086c70 con 0x7f36140fdfd0 2024-05-17T21:48:09.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:09.435+0000 7f3613fff700 1 --2- 172.21.15.107:0/2958294452 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f35fc0692a0 0x7f35fc06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:48:09.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:09.435+0000 7f3613fff700 1 --2- 172.21.15.107:0/2958294452 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f35fc0692a0 0x7f35fc06b750 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f360400b3b0 tx=0x7f3604009e30 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:48:09.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:09.439+0000 7f3610ff9700 1 -- 172.21.15.107:0/2958294452 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f3608056cf0 con 0x7f36140fdfd0 2024-05-17T21:48:09.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:09.701+0000 7f361a410700 1 -- 172.21.15.107:0/2958294452 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f36141058c0 con 0x7f36140fdfd0 2024-05-17T21:48:09.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:09.705+0000 7f3610ff9700 1 -- 172.21.15.107:0/2958294452 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v14) v1 ==== 170+0+59 (secure 0 0 0) 0x7f3608056880 con 0x7f36140fdfd0 2024-05-17T21:48:09.697 INFO:teuthology.orchestra.run.smithi107.stdout:[client.0] 2024-05-17T21:48:09.697 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQCZ0EdmBJrYKRAAlUJF9TuU2slZp6Zd0+4rxQ== 2024-05-17T21:48:09.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:09.707+0000 7f361a410700 1 -- 172.21.15.107:0/2958294452 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f35fc0692a0 msgr2=0x7f35fc06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:48:09.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:09.707+0000 7f361a410700 1 --2- 172.21.15.107:0/2958294452 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f35fc0692a0 0x7f35fc06b750 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f360400b3b0 tx=0x7f3604009e30 comp rx=0 tx=0).stop 2024-05-17T21:48:09.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:09.707+0000 7f361a410700 1 -- 172.21.15.107:0/2958294452 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f36140fdfd0 msgr2=0x7f361410de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:48:09.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:09.707+0000 7f361a410700 1 --2- 172.21.15.107:0/2958294452 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f36140fdfd0 0x7f361410de60 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3608002a50 tx=0x7f36080046b0 comp rx=0 tx=0).stop 2024-05-17T21:48:09.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:09.707+0000 7f361a410700 1 -- 172.21.15.107:0/2958294452 shutdown_connections 2024-05-17T21:48:09.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:09.707+0000 7f361a410700 1 --2- 172.21.15.107:0/2958294452 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f35fc0692a0 0x7f35fc06b750 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:09.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:09.707+0000 7f361a410700 1 --2- 172.21.15.107:0/2958294452 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f36140fce60 0x7f361410d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:09.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:09.707+0000 7f361a410700 1 --2- 172.21.15.107:0/2958294452 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f36140fdfd0 0x7f361410de60 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:09.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:09.707+0000 7f361a410700 1 -- 172.21.15.107:0/2958294452 >> 172.21.15.107:0/2958294452 conn(0x7f36140f8540 msgr2=0x7f36140f9bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:48:09.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:09.707+0000 7f361a410700 1 -- 172.21.15.107:0/2958294452 shutdown_connections 2024-05-17T21:48:09.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:09.707+0000 7f361a410700 1 -- 172.21.15.107:0/2958294452 wait complete. 2024-05-17T21:48:09.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:48:09 smithi107 ceph-mon[28434]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-17T21:48:10.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:48:09 smithi138 ceph-mon[35948]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-17T21:48:10.389 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-05-17T21:48:10.389 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-05-17T21:48:10.389 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-17T21:48:10.428 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-05-17T21:48:10.695 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/mon.smithi138/config 2024-05-17T21:48:10.818 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:48:10 smithi138 ceph-mon[35948]: from='client.? 172.21.15.107:0/2958294452' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-17T21:48:10.818 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:48:10 smithi138 ceph-mon[35948]: from='client.? 172.21.15.107:0/2958294452' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-17T21:48:10.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:48:10 smithi107 ceph-mon[28434]: from='client.? 172.21.15.107:0/2958294452' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-17T21:48:10.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:48:10 smithi107 ceph-mon[28434]: from='client.? 172.21.15.107:0/2958294452' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-17T21:48:11.960 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:48:11 smithi138 ceph-mon[35948]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-17T21:48:12.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:48:11 smithi107 ceph-mon[28434]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-17T21:48:12.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:48:12.321+0000 7f02ea852700 1 -- 172.21.15.138:0/3665809760 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02e40fce50 msgr2=0x7f02e40fd260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:48:12.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:48:12.321+0000 7f02ea852700 1 --2- 172.21.15.138:0/3665809760 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02e40fce50 0x7f02e40fd260 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f02d4004940 tx=0x7f02d4009e30 comp rx=0 tx=0).stop 2024-05-17T21:48:12.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:48:12.322+0000 7f02ea852700 1 -- 172.21.15.138:0/3665809760 shutdown_connections 2024-05-17T21:48:12.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:48:12.322+0000 7f02ea852700 1 --2- 172.21.15.138:0/3665809760 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f02e40fe050 0x7f02e40fe4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:12.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:48:12.322+0000 7f02ea852700 1 --2- 172.21.15.138:0/3665809760 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02e40fce50 0x7f02e40fd260 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:12.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:48:12.322+0000 7f02ea852700 1 -- 172.21.15.138:0/3665809760 >> 172.21.15.138:0/3665809760 conn(0x7f02e40f8580 msgr2=0x7f02e40fa990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:48:12.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:48:12.322+0000 7f02ea852700 1 -- 172.21.15.138:0/3665809760 shutdown_connections 2024-05-17T21:48:12.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:48:12.322+0000 7f02ea852700 1 -- 172.21.15.138:0/3665809760 wait complete. 2024-05-17T21:48:12.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:48:12.322+0000 7f02ea852700 1 Processor -- start 2024-05-17T21:48:12.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:48:12.322+0000 7f02ea852700 1 -- start start 2024-05-17T21:48:12.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:48:12.322+0000 7f02ea852700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f02e40fce50 0x7f02e410d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:48:12.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:48:12.322+0000 7f02ea852700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02e40fe050 0x7f02e410dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:48:12.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:48:12.322+0000 7f02ea852700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02e410e4a0 con 0x7f02e40fce50 2024-05-17T21:48:12.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:48:12.322+0000 7f02ea852700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02e410e610 con 0x7f02e40fe050 2024-05-17T21:48:12.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:48:12.323+0000 7f02e37fe700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02e40fe050 0x7f02e410dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:48:12.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:48:12.323+0000 7f02e37fe700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02e40fe050 0x7f02e410dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:55444/0 (socket says 172.21.15.138:55444) 2024-05-17T21:48:12.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:48:12.323+0000 7f02e37fe700 1 -- 172.21.15.138:0/406929411 learned_addr learned my addr 172.21.15.138:0/406929411 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-17T21:48:12.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:48:12.323+0000 7f02e3fff700 1 --2- 172.21.15.138:0/406929411 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f02e40fce50 0x7f02e410d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:48:12.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:48:12.323+0000 7f02e37fe700 1 -- 172.21.15.138:0/406929411 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f02e40fce50 msgr2=0x7f02e410d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:48:12.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:48:12.323+0000 7f02e37fe700 1 --2- 172.21.15.138:0/406929411 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f02e40fce50 0x7f02e410d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:12.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:48:12.323+0000 7f02e37fe700 1 -- 172.21.15.138:0/406929411 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02d4005040 con 0x7f02e40fe050 2024-05-17T21:48:12.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:48:12.323+0000 7f02e37fe700 1 --2- 172.21.15.138:0/406929411 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02e40fe050 0x7f02e410dea0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f02d8002a50 tx=0x7f02d80046b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:48:12.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:48:12.323+0000 7f02e0ff9700 1 -- 172.21.15.138:0/406929411 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f02d8007cc0 con 0x7f02e40fe050 2024-05-17T21:48:12.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:48:12.323+0000 7f02ea852700 1 -- 172.21.15.138:0/406929411 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02e4113070 con 0x7f02e40fe050 2024-05-17T21:48:12.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:48:12.323+0000 7f02e0ff9700 1 -- 172.21.15.138:0/406929411 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f02d800f040 con 0x7f02e40fe050 2024-05-17T21:48:12.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:48:12.323+0000 7f02e0ff9700 1 -- 172.21.15.138:0/406929411 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f02d800da70 con 0x7f02e40fe050 2024-05-17T21:48:12.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:48:12.323+0000 7f02ea852700 1 -- 172.21.15.138:0/406929411 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02e4113630 con 0x7f02e40fe050 2024-05-17T21:48:12.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:48:12.324+0000 7f02ea852700 1 -- 172.21.15.138:0/406929411 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02e404c710 con 0x7f02e40fe050 2024-05-17T21:48:12.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:48:12.325+0000 7f02e0ff9700 1 -- 172.21.15.138:0/406929411 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f02d800dc10 con 0x7f02e40fe050 2024-05-17T21:48:12.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:48:12.325+0000 7f02e0ff9700 1 --2- 172.21.15.138:0/406929411 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f02cc0692a0 0x7f02cc06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:48:12.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:48:12.325+0000 7f02e0ff9700 1 -- 172.21.15.138:0/406929411 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f02d80864f0 con 0x7f02e40fe050 2024-05-17T21:48:12.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:48:12.325+0000 7f02e3fff700 1 --2- 172.21.15.138:0/406929411 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f02cc0692a0 0x7f02cc06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:48:12.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:48:12.326+0000 7f02e3fff700 1 --2- 172.21.15.138:0/406929411 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f02cc0692a0 0x7f02cc06b750 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f02d4004890 tx=0x7f02d4009e30 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:48:12.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:48:12.329+0000 7f02e0ff9700 1 -- 172.21.15.138:0/406929411 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f02d8056570 con 0x7f02e40fe050 2024-05-17T21:48:12.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:48:12.590+0000 7f02ea852700 1 -- 172.21.15.138:0/406929411 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f02e4113a30 con 0x7f02e40fe050 2024-05-17T21:48:12.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:48:12.595+0000 7f02e0ff9700 1 -- 172.21.15.138:0/406929411 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v15) v1 ==== 170+0+59 (secure 0 0 0) 0x7f02d8056100 con 0x7f02e40fe050 2024-05-17T21:48:12.588 INFO:teuthology.orchestra.run.smithi138.stdout:[client.1] 2024-05-17T21:48:12.588 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQCc0EdmFhtMIxAAFLma3mgd4h/oAxkwO7vq8g== 2024-05-17T21:48:12.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:48:12.597+0000 7f02ea852700 1 -- 172.21.15.138:0/406929411 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f02cc0692a0 msgr2=0x7f02cc06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:48:12.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:48:12.597+0000 7f02ea852700 1 --2- 172.21.15.138:0/406929411 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f02cc0692a0 0x7f02cc06b750 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f02d4004890 tx=0x7f02d4009e30 comp rx=0 tx=0).stop 2024-05-17T21:48:12.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:48:12.597+0000 7f02ea852700 1 -- 172.21.15.138:0/406929411 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02e40fe050 msgr2=0x7f02e410dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:48:12.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:48:12.597+0000 7f02ea852700 1 --2- 172.21.15.138:0/406929411 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02e40fe050 0x7f02e410dea0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f02d8002a50 tx=0x7f02d80046b0 comp rx=0 tx=0).stop 2024-05-17T21:48:12.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:48:12.597+0000 7f02ea852700 1 -- 172.21.15.138:0/406929411 shutdown_connections 2024-05-17T21:48:12.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:48:12.597+0000 7f02ea852700 1 --2- 172.21.15.138:0/406929411 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f02cc0692a0 0x7f02cc06b750 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:12.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:48:12.597+0000 7f02ea852700 1 --2- 172.21.15.138:0/406929411 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f02e40fce50 0x7f02e410d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:12.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:48:12.597+0000 7f02ea852700 1 --2- 172.21.15.138:0/406929411 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02e40fe050 0x7f02e410dea0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:12.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:48:12.597+0000 7f02ea852700 1 -- 172.21.15.138:0/406929411 >> 172.21.15.138:0/406929411 conn(0x7f02e40f8580 msgr2=0x7f02e40f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:48:12.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:48:12.598+0000 7f02ea852700 1 -- 172.21.15.138:0/406929411 shutdown_connections 2024-05-17T21:48:12.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:48:12.598+0000 7f02ea852700 1 -- 172.21.15.138:0/406929411 wait complete. 2024-05-17T21:48:12.948 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:48:12 smithi138 ceph-mon[35948]: from='client.? 172.21.15.138:0/406929411' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-17T21:48:12.948 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:48:12 smithi138 ceph-mon[35948]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-17T21:48:12.948 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:48:12 smithi138 ceph-mon[35948]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-17T21:48:13.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:48:12 smithi107 ceph-mon[28434]: from='client.? 172.21.15.138:0/406929411' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-17T21:48:13.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:48:12 smithi107 ceph-mon[28434]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-17T21:48:13.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:48:12 smithi107 ceph-mon[28434]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-17T21:48:13.260 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-17T21:48:13.260 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-05-17T21:48:13.260 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-05-17T21:48:13.301 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-17T21:48:13.301 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-05-17T21:48:13.301 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -- ceph mgr dump --format=json 2024-05-17T21:48:13.569 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/mon.smithi107/config 2024-05-17T21:48:13.989 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:48:13 smithi107 ceph-mon[28434]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-17T21:48:14.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:48:13 smithi138 ceph-mon[35948]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-17T21:48:15.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:15.176+0000 7f0d552a3700 1 -- 172.21.15.107:0/265371098 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d50102de0 msgr2=0x7f0d501031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:48:15.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:15.176+0000 7f0d552a3700 1 --2- 172.21.15.107:0/265371098 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d50102de0 0x7f0d501031b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0d40004940 tx=0x7f0d40009e30 comp rx=0 tx=0).stop 2024-05-17T21:48:15.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:15.176+0000 7f0d552a3700 1 -- 172.21.15.107:0/265371098 shutdown_connections 2024-05-17T21:48:15.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:15.176+0000 7f0d552a3700 1 --2- 172.21.15.107:0/265371098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0d50103780 0x7f0d500fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:15.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:15.176+0000 7f0d552a3700 1 --2- 172.21.15.107:0/265371098 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d50102de0 0x7f0d501031b0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:15.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:15.176+0000 7f0d552a3700 1 -- 172.21.15.107:0/265371098 >> 172.21.15.107:0/265371098 conn(0x7f0d500f8990 msgr2=0x7f0d500fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:48:15.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:15.176+0000 7f0d552a3700 1 -- 172.21.15.107:0/265371098 shutdown_connections 2024-05-17T21:48:15.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:15.176+0000 7f0d552a3700 1 -- 172.21.15.107:0/265371098 wait complete. 2024-05-17T21:48:15.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:15.176+0000 7f0d552a3700 1 Processor -- start 2024-05-17T21:48:15.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:15.177+0000 7f0d552a3700 1 -- start start 2024-05-17T21:48:15.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:15.177+0000 7f0d552a3700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d50102de0 0x7f0d50113930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:48:15.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:15.177+0000 7f0d552a3700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0d50103780 0x7f0d50113e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:48:15.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:15.177+0000 7f0d552a3700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d50114550 con 0x7f0d50102de0 2024-05-17T21:48:15.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:15.177+0000 7f0d552a3700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d5010dac0 con 0x7f0d50103780 2024-05-17T21:48:15.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:15.177+0000 7f0d4effd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d50102de0 0x7f0d50113930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:48:15.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:15.177+0000 7f0d4effd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d50102de0 0x7f0d50113930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33926/0 (socket says 172.21.15.107:33926) 2024-05-17T21:48:15.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:15.177+0000 7f0d4effd700 1 -- 172.21.15.107:0/669483829 learned_addr learned my addr 172.21.15.107:0/669483829 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:48:15.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:15.177+0000 7f0d467fc700 1 --2- 172.21.15.107:0/669483829 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0d50103780 0x7f0d50113e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:48:15.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:15.177+0000 7f0d4effd700 1 -- 172.21.15.107:0/669483829 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0d50103780 msgr2=0x7f0d50113e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:48:15.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:15.177+0000 7f0d4effd700 1 --2- 172.21.15.107:0/669483829 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0d50103780 0x7f0d50113e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:15.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:15.177+0000 7f0d4effd700 1 -- 172.21.15.107:0/669483829 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d40005040 con 0x7f0d50102de0 2024-05-17T21:48:15.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:15.177+0000 7f0d4effd700 1 --2- 172.21.15.107:0/669483829 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d50102de0 0x7f0d50113930 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0d4000b3b0 tx=0x7f0d40009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:48:15.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:15.178+0000 7f0d47fff700 1 -- 172.21.15.107:0/669483829 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0d4000e040 con 0x7f0d50102de0 2024-05-17T21:48:15.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:15.178+0000 7f0d552a3700 1 -- 172.21.15.107:0/669483829 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d5010dd40 con 0x7f0d50102de0 2024-05-17T21:48:15.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:15.178+0000 7f0d47fff700 1 -- 172.21.15.107:0/669483829 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0d4000bbc0 con 0x7f0d50102de0 2024-05-17T21:48:15.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:15.178+0000 7f0d47fff700 1 -- 172.21.15.107:0/669483829 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0d4001c3f0 con 0x7f0d50102de0 2024-05-17T21:48:15.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:15.178+0000 7f0d552a3700 1 -- 172.21.15.107:0/669483829 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d5010e300 con 0x7f0d50102de0 2024-05-17T21:48:15.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:15.179+0000 7f0d552a3700 1 -- 172.21.15.107:0/669483829 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d501000f0 con 0x7f0d50102de0 2024-05-17T21:48:15.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:15.179+0000 7f0d47fff700 1 -- 172.21.15.107:0/669483829 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f0d4000bd30 con 0x7f0d50102de0 2024-05-17T21:48:15.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:15.179+0000 7f0d47fff700 1 --2- 172.21.15.107:0/669483829 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f0d3c0692a0 0x7f0d3c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:48:15.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:15.180+0000 7f0d47fff700 1 -- 172.21.15.107:0/669483829 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f0d400880f0 con 0x7f0d50102de0 2024-05-17T21:48:15.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:15.180+0000 7f0d467fc700 1 --2- 172.21.15.107:0/669483829 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f0d3c0692a0 0x7f0d3c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:48:15.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:15.180+0000 7f0d467fc700 1 --2- 172.21.15.107:0/669483829 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f0d3c0692a0 0x7f0d3c06b750 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0d38006fd0 tx=0x7f0d38008040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:48:15.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:15.184+0000 7f0d47fff700 1 -- 172.21.15.107:0/669483829 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f0d40058170 con 0x7f0d50102de0 2024-05-17T21:48:15.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:15.420+0000 7f0d552a3700 1 -- 172.21.15.107:0/669483829 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f0d5010ef80 con 0x7f0d50102de0 2024-05-17T21:48:15.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:15.423+0000 7f0d47fff700 1 -- 172.21.15.107:0/669483829 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v19) v1 ==== 74+0+167187 (secure 0 0 0) 0x7f0d40086e60 con 0x7f0d50102de0 2024-05-17T21:48:15.420 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:48:15.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:15.425+0000 7f0d552a3700 1 -- 172.21.15.107:0/669483829 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f0d3c0692a0 msgr2=0x7f0d3c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:48:15.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:15.425+0000 7f0d552a3700 1 --2- 172.21.15.107:0/669483829 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f0d3c0692a0 0x7f0d3c06b750 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0d38006fd0 tx=0x7f0d38008040 comp rx=0 tx=0).stop 2024-05-17T21:48:15.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:15.425+0000 7f0d552a3700 1 -- 172.21.15.107:0/669483829 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d50102de0 msgr2=0x7f0d50113930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:48:15.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:15.425+0000 7f0d552a3700 1 --2- 172.21.15.107:0/669483829 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d50102de0 0x7f0d50113930 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0d4000b3b0 tx=0x7f0d40009e30 comp rx=0 tx=0).stop 2024-05-17T21:48:15.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:15.425+0000 7f0d552a3700 1 -- 172.21.15.107:0/669483829 shutdown_connections 2024-05-17T21:48:15.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:15.425+0000 7f0d552a3700 1 --2- 172.21.15.107:0/669483829 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f0d3c0692a0 0x7f0d3c06b750 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:15.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:15.425+0000 7f0d552a3700 1 --2- 172.21.15.107:0/669483829 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d50102de0 0x7f0d50113930 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:15.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:15.425+0000 7f0d552a3700 1 --2- 172.21.15.107:0/669483829 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0d50103780 0x7f0d50113e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:15.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:15.425+0000 7f0d552a3700 1 -- 172.21.15.107:0/669483829 >> 172.21.15.107:0/669483829 conn(0x7f0d500f8990 msgr2=0x7f0d500fa6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:48:15.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:15.425+0000 7f0d552a3700 1 -- 172.21.15.107:0/669483829 shutdown_connections 2024-05-17T21:48:15.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:15.425+0000 7f0d552a3700 1 -- 172.21.15.107:0/669483829 wait complete. 2024-05-17T21:48:16.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:48:15 smithi138 ceph-mon[35948]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-17T21:48:16.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:48:15 smithi138 ceph-mon[35948]: from='client.? 172.21.15.107:0/669483829' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-17T21:48:16.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:48:15 smithi107 ceph-mon[28434]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-17T21:48:16.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:48:15 smithi107 ceph-mon[28434]: from='client.? 172.21.15.107:0/669483829' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-17T21:48:16.259 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":19,"flags":0,"active_gid":14214,"active_name":"smithi107.xhhyiw","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":1294361014},{"type":"v1","addr":"172.21.15.107:6801","nonce":1294361014}]},"active_addr":"172.21.15.107:6801/1294361014","active_change":"2024-05-17T21:44:32.293204+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":14234,"name":"smithi138.napnll","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.107:8443/","prometheus":"http://172.21.15.107:9283/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":5,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.107:0","nonce":1170468325}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.107:0","nonce":239507720}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.107:0","nonce":2269613822}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.107:0","nonce":2823347896}]}]} 2024-05-17T21:48:16.262 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-05-17T21:48:16.262 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-05-17T21:48:16.263 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -- ceph osd dump --format=json 2024-05-17T21:48:16.516 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/mon.smithi107/config 2024-05-17T21:48:17.975 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:48:17 smithi107 ceph-mon[28434]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-17T21:48:17.975 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:48:17 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:48:18.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:48:17 smithi138 ceph-mon[35948]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-17T21:48:18.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:48:17 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:48:18.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:18.215+0000 7f6fe1989700 1 -- 172.21.15.107:0/3106941803 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6fdc102f60 msgr2=0x7f6fdc103330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:48:18.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:18.215+0000 7f6fe1989700 1 --2- 172.21.15.107:0/3106941803 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6fdc102f60 0x7f6fdc103330 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc004940 tx=0x7f6fcc009e30 comp rx=0 tx=0).stop 2024-05-17T21:48:18.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:18.215+0000 7f6fe1989700 1 -- 172.21.15.107:0/3106941803 shutdown_connections 2024-05-17T21:48:18.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:18.215+0000 7f6fe1989700 1 --2- 172.21.15.107:0/3106941803 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6fdc0fb520 0x7f6fdc0fb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:18.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:18.215+0000 7f6fe1989700 1 --2- 172.21.15.107:0/3106941803 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6fdc102f60 0x7f6fdc103330 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:18.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:18.215+0000 7f6fe1989700 1 -- 172.21.15.107:0/3106941803 >> 172.21.15.107:0/3106941803 conn(0x7f6fdc0714d0 msgr2=0x7f6fdc0718d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:48:18.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:18.215+0000 7f6fe1989700 1 -- 172.21.15.107:0/3106941803 shutdown_connections 2024-05-17T21:48:18.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:18.215+0000 7f6fe1989700 1 -- 172.21.15.107:0/3106941803 wait complete. 2024-05-17T21:48:18.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:18.215+0000 7f6fe1989700 1 Processor -- start 2024-05-17T21:48:18.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:18.215+0000 7f6fe1989700 1 -- start start 2024-05-17T21:48:18.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:18.216+0000 7f6fe1989700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6fdc0fb520 0x7f6fdc1139a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:48:18.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:18.216+0000 7f6fe1989700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6fdc102f60 0x7f6fdc113ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:48:18.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:18.216+0000 7f6fe1989700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fdc114530 con 0x7f6fdc102f60 2024-05-17T21:48:18.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:18.216+0000 7f6fe1989700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fdc10db30 con 0x7f6fdc0fb520 2024-05-17T21:48:18.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:18.216+0000 7f6fd3fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6fdc102f60 0x7f6fdc113ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:48:18.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:18.216+0000 7f6fd3fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6fdc102f60 0x7f6fdc113ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44620/0 (socket says 172.21.15.107:44620) 2024-05-17T21:48:18.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:18.216+0000 7f6fd3fff700 1 -- 172.21.15.107:0/1161744053 learned_addr learned my addr 172.21.15.107:0/1161744053 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:48:18.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:18.216+0000 7f6fdaffd700 1 --2- 172.21.15.107:0/1161744053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6fdc0fb520 0x7f6fdc1139a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:48:18.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:18.216+0000 7f6fd3fff700 1 -- 172.21.15.107:0/1161744053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6fdc0fb520 msgr2=0x7f6fdc1139a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:48:18.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:18.216+0000 7f6fd3fff700 1 --2- 172.21.15.107:0/1161744053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6fdc0fb520 0x7f6fdc1139a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:18.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:18.216+0000 7f6fd3fff700 1 -- 172.21.15.107:0/1161744053 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fcc005040 con 0x7f6fdc102f60 2024-05-17T21:48:18.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:18.216+0000 7f6fd3fff700 1 --2- 172.21.15.107:0/1161744053 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6fdc102f60 0x7f6fdc113ee0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6fc4002a50 tx=0x7f6fc40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:48:18.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:18.216+0000 7f6fd37fe700 1 -- 172.21.15.107:0/1161744053 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6fc4007cc0 con 0x7f6fdc102f60 2024-05-17T21:48:18.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:18.216+0000 7f6fd37fe700 1 -- 172.21.15.107:0/1161744053 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6fc400f040 con 0x7f6fdc102f60 2024-05-17T21:48:18.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:18.216+0000 7f6fe1989700 1 -- 172.21.15.107:0/1161744053 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fdc10de10 con 0x7f6fdc102f60 2024-05-17T21:48:18.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:18.216+0000 7f6fd37fe700 1 -- 172.21.15.107:0/1161744053 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6fc400da30 con 0x7f6fdc102f60 2024-05-17T21:48:18.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:18.216+0000 7f6fe1989700 1 -- 172.21.15.107:0/1161744053 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fdc10e350 con 0x7f6fdc102f60 2024-05-17T21:48:18.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:18.217+0000 7f6fe1989700 1 -- 172.21.15.107:0/1161744053 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fdc100250 con 0x7f6fdc102f60 2024-05-17T21:48:18.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:18.218+0000 7f6fd37fe700 1 -- 172.21.15.107:0/1161744053 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f6fc400db90 con 0x7f6fdc102f60 2024-05-17T21:48:18.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:18.218+0000 7f6fd37fe700 1 --2- 172.21.15.107:0/1161744053 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f6fc80692a0 0x7f6fc806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:48:18.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:18.218+0000 7f6fdaffd700 1 --2- 172.21.15.107:0/1161744053 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f6fc80692a0 0x7f6fc806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:48:18.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:18.218+0000 7f6fd37fe700 1 -- 172.21.15.107:0/1161744053 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f6fc4086c10 con 0x7f6fdc102f60 2024-05-17T21:48:18.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:18.219+0000 7f6fdaffd700 1 --2- 172.21.15.107:0/1161744053 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f6fc80692a0 0x7f6fc806b750 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc004890 tx=0x7f6fcc009e30 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:48:18.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:18.223+0000 7f6fd37fe700 1 -- 172.21.15.107:0/1161744053 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f6fc4056c90 con 0x7f6fdc102f60 2024-05-17T21:48:18.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:18.410+0000 7f6fe1989700 1 -- 172.21.15.107:0/1161744053 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6fdc10eff0 con 0x7f6fdc102f60 2024-05-17T21:48:18.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:18.410+0000 7f6fd37fe700 1 -- 172.21.15.107:0/1161744053 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+11098 (secure 0 0 0) 0x7f6fc4056820 con 0x7f6fdc102f60 2024-05-17T21:48:18.404 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:48:18.404 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":40,"fsid":"3722731a-1496-11ef-bc9a-c7b262605968","created":"2024-05-17T21:42:15.126179+0000","modified":"2024-05-17T21:47:58.575311+0000","last_up_change":"2024-05-17T21:47:57.569583+0000","last_in_change":"2024-05-17T21:47:38.526939+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-17T21:46:42.602884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"203debeb-d443-4b99-a929-64afe799acfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":710062679},{"type":"v1","addr":"172.21.15.107:6803","nonce":710062679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":710062679},{"type":"v1","addr":"172.21.15.107:6805","nonce":710062679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":710062679},{"type":"v1","addr":"172.21.15.107:6809","nonce":710062679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":710062679},{"type":"v1","addr":"172.21.15.107:6807","nonce":710062679}]},"public_addr":"172.21.15.107:6803/710062679","cluster_addr":"172.21.15.107:6805/710062679","heartbeat_back_addr":"172.21.15.107:6809/710062679","heartbeat_front_addr":"172.21.15.107:6807/710062679","state":["exists","up"]},{"osd":1,"uuid":"2052c88c-a0b5-4f5b-8ee0-68528e607d9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":686958797},{"type":"v1","addr":"172.21.15.107:6811","nonce":686958797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":686958797},{"type":"v1","addr":"172.21.15.107:6813","nonce":686958797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":686958797},{"type":"v1","addr":"172.21.15.107:6817","nonce":686958797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":686958797},{"type":"v1","addr":"172.21.15.107:6815","nonce":686958797}]},"public_addr":"172.21.15.107:6811/686958797","cluster_addr":"172.21.15.107:6813/686958797","heartbeat_back_addr":"172.21.15.107:6817/686958797","heartbeat_front_addr":"172.21.15.107:6815/686958797","state":["exists","up"]},{"osd":2,"uuid":"b5dc6be0-2ac3-4769-ae70-14c23381506b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":3962264726},{"type":"v1","addr":"172.21.15.107:6819","nonce":3962264726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":3962264726},{"type":"v1","addr":"172.21.15.107:6821","nonce":3962264726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3962264726},{"type":"v1","addr":"172.21.15.107:6825","nonce":3962264726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":3962264726},{"type":"v1","addr":"172.21.15.107:6823","nonce":3962264726}]},"public_addr":"172.21.15.107:6819/3962264726","cluster_addr":"172.21.15.107:6821/3962264726","heartbeat_back_addr":"172.21.15.107:6825/3962264726","heartbeat_front_addr":"172.21.15.107:6823/3962264726","state":["exists","up"]},{"osd":3,"uuid":"5f3d4515-afed-481f-8e08-debaf33a9e0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2477238048},{"type":"v1","addr":"172.21.15.138:6801","nonce":2477238048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2477238048},{"type":"v1","addr":"172.21.15.138:6803","nonce":2477238048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2477238048},{"type":"v1","addr":"172.21.15.138:6807","nonce":2477238048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2477238048},{"type":"v1","addr":"172.21.15.138:6805","nonce":2477238048}]},"public_addr":"172.21.15.138:6801/2477238048","cluster_addr":"172.21.15.138:6803/2477238048","heartbeat_back_addr":"172.21.15.138:6807/2477238048","heartbeat_front_addr":"172.21.15.138:6805/2477238048","state":["exists","up"]},{"osd":4,"uuid":"d28515a1-4510-4f94-b0b1-3aca283eed05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":792878465},{"type":"v1","addr":"172.21.15.138:6809","nonce":792878465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":792878465},{"type":"v1","addr":"172.21.15.138:6811","nonce":792878465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":792878465},{"type":"v1","addr":"172.21.15.138:6815","nonce":792878465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":792878465},{"type":"v1","addr":"172.21.15.138:6813","nonce":792878465}]},"public_addr":"172.21.15.138:6809/792878465","cluster_addr":"172.21.15.138:6811/792878465","heartbeat_back_addr":"172.21.15.138:6815/792878465","heartbeat_front_addr":"172.21.15.138:6813/792878465","state":["exists","up"]},{"osd":5,"uuid":"ab57d399-f1cf-4139-abba-7cb15e433c25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":406753522},{"type":"v1","addr":"172.21.15.138:6817","nonce":406753522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":406753522},{"type":"v1","addr":"172.21.15.138:6819","nonce":406753522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":406753522},{"type":"v1","addr":"172.21.15.138:6823","nonce":406753522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":406753522},{"type":"v1","addr":"172.21.15.138:6821","nonce":406753522}]},"public_addr":"172.21.15.138:6817/406753522","cluster_addr":"172.21.15.138:6819/406753522","heartbeat_back_addr":"172.21.15.138:6823/406753522","heartbeat_front_addr":"172.21.15.138:6821/406753522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:45:44.441319+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:46:11.518607+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:46:40.661238+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:47:04.556335+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:47:30.603032+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:47:55.950499+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/2414403835":"2024-05-18T21:44:32.293094+0000","172.21.15.107:6801/3405524625":"2024-05-18T21:44:32.293094+0000","172.21.15.107:6800/3405524625":"2024-05-18T21:44:32.293094+0000","172.21.15.107:0/771372867":"2024-05-18T21:44:32.293094+0000","172.21.15.107:0/498460124":"2024-05-18T21:43:22.097723+0000","172.21.15.107:0/3170894163":"2024-05-18T21:43:22.097723+0000","172.21.15.107:6801/1718632476":"2024-05-18T21:43:22.097723+0000","172.21.15.107:6801/1677232931":"2024-05-18T21:42:52.099188+0000","172.21.15.107:0/2856704707":"2024-05-18T21:43:22.097723+0000","172.21.15.107:0/3048013464":"2024-05-18T21:42:52.099188+0000","172.21.15.107:0/2872531133":"2024-05-18T21:42:52.099188+0000","172.21.15.107:0/1077444017":"2024-05-18T21:44:32.293094+0000","172.21.15.107:0/3105018805":"2024-05-18T21:42:52.099188+0000","172.21.15.107:6800/1677232931":"2024-05-18T21:42:52.099188+0000","172.21.15.107:6800/1718632476":"2024-05-18T21:43:22.097723+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-17T21:48:18.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:18.413+0000 7f6fe1989700 1 -- 172.21.15.107:0/1161744053 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f6fc80692a0 msgr2=0x7f6fc806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:48:18.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:18.413+0000 7f6fe1989700 1 --2- 172.21.15.107:0/1161744053 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f6fc80692a0 0x7f6fc806b750 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc004890 tx=0x7f6fcc009e30 comp rx=0 tx=0).stop 2024-05-17T21:48:18.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:18.413+0000 7f6fe1989700 1 -- 172.21.15.107:0/1161744053 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6fdc102f60 msgr2=0x7f6fdc113ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:48:18.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:18.413+0000 7f6fe1989700 1 --2- 172.21.15.107:0/1161744053 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6fdc102f60 0x7f6fdc113ee0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6fc4002a50 tx=0x7f6fc40046b0 comp rx=0 tx=0).stop 2024-05-17T21:48:18.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:18.413+0000 7f6fe1989700 1 -- 172.21.15.107:0/1161744053 shutdown_connections 2024-05-17T21:48:18.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:18.413+0000 7f6fe1989700 1 --2- 172.21.15.107:0/1161744053 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f6fc80692a0 0x7f6fc806b750 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:18.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:18.413+0000 7f6fe1989700 1 --2- 172.21.15.107:0/1161744053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6fdc0fb520 0x7f6fdc1139a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:18.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:18.413+0000 7f6fe1989700 1 --2- 172.21.15.107:0/1161744053 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6fdc102f60 0x7f6fdc113ee0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:18.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:18.413+0000 7f6fe1989700 1 -- 172.21.15.107:0/1161744053 >> 172.21.15.107:0/1161744053 conn(0x7f6fdc0714d0 msgr2=0x7f6fdc0fa7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:48:18.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:18.413+0000 7f6fe1989700 1 -- 172.21.15.107:0/1161744053 shutdown_connections 2024-05-17T21:48:18.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:18.413+0000 7f6fe1989700 1 -- 172.21.15.107:0/1161744053 wait complete. 2024-05-17T21:48:18.946 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:48:18 smithi107 ceph-mon[28434]: from='client.? 172.21.15.107:0/1161744053' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T21:48:18.971 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-05-17T21:48:18.971 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -- ceph osd dump --format=json 2024-05-17T21:48:19.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:48:18 smithi138 ceph-mon[35948]: from='client.? 172.21.15.107:0/1161744053' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T21:48:19.227 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/mon.smithi107/config 2024-05-17T21:48:20.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:48:19 smithi138 ceph-mon[35948]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-17T21:48:20.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:48:19 smithi107 ceph-mon[28434]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-17T21:48:20.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:20.812+0000 7fdb96592700 1 -- 172.21.15.107:0/2061361924 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb90101d10 msgr2=0x7fdb901020e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:48:20.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:20.812+0000 7fdb96592700 1 --2- 172.21.15.107:0/2061361924 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb90101d10 0x7fdb901020e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fdb80004940 tx=0x7fdb80009e30 comp rx=0 tx=0).stop 2024-05-17T21:48:20.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:20.812+0000 7fdb96592700 1 -- 172.21.15.107:0/2061361924 shutdown_connections 2024-05-17T21:48:20.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:20.812+0000 7fdb96592700 1 --2- 172.21.15.107:0/2061361924 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb901026b0 0x7fdb900fbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:20.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:20.812+0000 7fdb96592700 1 --2- 172.21.15.107:0/2061361924 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb90101d10 0x7fdb901020e0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:20.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:20.812+0000 7fdb96592700 1 -- 172.21.15.107:0/2061361924 >> 172.21.15.107:0/2061361924 conn(0x7fdb900f7860 msgr2=0x7fdb900f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:48:20.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:20.812+0000 7fdb96592700 1 -- 172.21.15.107:0/2061361924 shutdown_connections 2024-05-17T21:48:20.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:20.812+0000 7fdb96592700 1 -- 172.21.15.107:0/2061361924 wait complete. 2024-05-17T21:48:20.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:20.812+0000 7fdb96592700 1 Processor -- start 2024-05-17T21:48:20.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:20.812+0000 7fdb96592700 1 -- start start 2024-05-17T21:48:20.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:20.812+0000 7fdb96592700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb90101d10 0x7fdb90113990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:48:20.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:20.812+0000 7fdb96592700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb901026b0 0x7fdb90113ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:48:20.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:20.812+0000 7fdb96592700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb901145b0 con 0x7fdb901026b0 2024-05-17T21:48:20.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:20.812+0000 7fdb96592700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb9010db40 con 0x7fdb90101d10 2024-05-17T21:48:20.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:20.813+0000 7fdb87fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb901026b0 0x7fdb90113ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:48:20.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:20.813+0000 7fdb87fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb901026b0 0x7fdb90113ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44638/0 (socket says 172.21.15.107:44638) 2024-05-17T21:48:20.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:20.813+0000 7fdb87fff700 1 -- 172.21.15.107:0/2504221669 learned_addr learned my addr 172.21.15.107:0/2504221669 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:48:20.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:20.813+0000 7fdb8ffff700 1 --2- 172.21.15.107:0/2504221669 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb90101d10 0x7fdb90113990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:48:20.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:20.813+0000 7fdb87fff700 1 -- 172.21.15.107:0/2504221669 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb90101d10 msgr2=0x7fdb90113990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:48:20.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:20.813+0000 7fdb87fff700 1 --2- 172.21.15.107:0/2504221669 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb90101d10 0x7fdb90113990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:20.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:20.813+0000 7fdb87fff700 1 -- 172.21.15.107:0/2504221669 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb80005040 con 0x7fdb901026b0 2024-05-17T21:48:20.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:20.813+0000 7fdb87fff700 1 --2- 172.21.15.107:0/2504221669 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb901026b0 0x7fdb90113ed0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fdb7800b790 tx=0x7fdb78009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:48:20.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:20.813+0000 7fdb8d7fa700 1 -- 172.21.15.107:0/2504221669 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb7800f6f0 con 0x7fdb901026b0 2024-05-17T21:48:20.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:20.813+0000 7fdb96592700 1 -- 172.21.15.107:0/2504221669 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb9010de20 con 0x7fdb901026b0 2024-05-17T21:48:20.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:20.813+0000 7fdb8d7fa700 1 -- 172.21.15.107:0/2504221669 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdb7800fd20 con 0x7fdb901026b0 2024-05-17T21:48:20.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:20.813+0000 7fdb8d7fa700 1 -- 172.21.15.107:0/2504221669 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb780103a0 con 0x7fdb901026b0 2024-05-17T21:48:20.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:20.813+0000 7fdb96592700 1 -- 172.21.15.107:0/2504221669 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb9010e3e0 con 0x7fdb901026b0 2024-05-17T21:48:20.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:20.814+0000 7fdb96592700 1 -- 172.21.15.107:0/2504221669 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb900fd460 con 0x7fdb901026b0 2024-05-17T21:48:20.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:20.815+0000 7fdb8d7fa700 1 -- 172.21.15.107:0/2504221669 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7fdb78011400 con 0x7fdb901026b0 2024-05-17T21:48:20.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:20.815+0000 7fdb8d7fa700 1 --2- 172.21.15.107:0/2504221669 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fdb7c0692a0 0x7fdb7c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:48:20.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:20.815+0000 7fdb8d7fa700 1 -- 172.21.15.107:0/2504221669 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fdb78086d20 con 0x7fdb901026b0 2024-05-17T21:48:20.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:20.815+0000 7fdb8ffff700 1 --2- 172.21.15.107:0/2504221669 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fdb7c0692a0 0x7fdb7c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:48:20.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:20.815+0000 7fdb8ffff700 1 --2- 172.21.15.107:0/2504221669 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fdb7c0692a0 0x7fdb7c06b750 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fdb8000b3b0 tx=0x7fdb80009e30 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:48:20.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:20.819+0000 7fdb8d7fa700 1 -- 172.21.15.107:0/2504221669 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fdb78053310 con 0x7fdb901026b0 2024-05-17T21:48:21.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:21.019+0000 7fdb96592700 1 -- 172.21.15.107:0/2504221669 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdb90108e80 con 0x7fdb901026b0 2024-05-17T21:48:21.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:21.020+0000 7fdb8d7fa700 1 -- 172.21.15.107:0/2504221669 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+11098 (secure 0 0 0) 0x7fdb78056930 con 0x7fdb901026b0 2024-05-17T21:48:21.014 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:48:21.014 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":40,"fsid":"3722731a-1496-11ef-bc9a-c7b262605968","created":"2024-05-17T21:42:15.126179+0000","modified":"2024-05-17T21:47:58.575311+0000","last_up_change":"2024-05-17T21:47:57.569583+0000","last_in_change":"2024-05-17T21:47:38.526939+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-17T21:46:42.602884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"203debeb-d443-4b99-a929-64afe799acfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":710062679},{"type":"v1","addr":"172.21.15.107:6803","nonce":710062679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":710062679},{"type":"v1","addr":"172.21.15.107:6805","nonce":710062679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":710062679},{"type":"v1","addr":"172.21.15.107:6809","nonce":710062679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":710062679},{"type":"v1","addr":"172.21.15.107:6807","nonce":710062679}]},"public_addr":"172.21.15.107:6803/710062679","cluster_addr":"172.21.15.107:6805/710062679","heartbeat_back_addr":"172.21.15.107:6809/710062679","heartbeat_front_addr":"172.21.15.107:6807/710062679","state":["exists","up"]},{"osd":1,"uuid":"2052c88c-a0b5-4f5b-8ee0-68528e607d9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":686958797},{"type":"v1","addr":"172.21.15.107:6811","nonce":686958797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":686958797},{"type":"v1","addr":"172.21.15.107:6813","nonce":686958797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":686958797},{"type":"v1","addr":"172.21.15.107:6817","nonce":686958797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":686958797},{"type":"v1","addr":"172.21.15.107:6815","nonce":686958797}]},"public_addr":"172.21.15.107:6811/686958797","cluster_addr":"172.21.15.107:6813/686958797","heartbeat_back_addr":"172.21.15.107:6817/686958797","heartbeat_front_addr":"172.21.15.107:6815/686958797","state":["exists","up"]},{"osd":2,"uuid":"b5dc6be0-2ac3-4769-ae70-14c23381506b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":3962264726},{"type":"v1","addr":"172.21.15.107:6819","nonce":3962264726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":3962264726},{"type":"v1","addr":"172.21.15.107:6821","nonce":3962264726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3962264726},{"type":"v1","addr":"172.21.15.107:6825","nonce":3962264726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":3962264726},{"type":"v1","addr":"172.21.15.107:6823","nonce":3962264726}]},"public_addr":"172.21.15.107:6819/3962264726","cluster_addr":"172.21.15.107:6821/3962264726","heartbeat_back_addr":"172.21.15.107:6825/3962264726","heartbeat_front_addr":"172.21.15.107:6823/3962264726","state":["exists","up"]},{"osd":3,"uuid":"5f3d4515-afed-481f-8e08-debaf33a9e0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2477238048},{"type":"v1","addr":"172.21.15.138:6801","nonce":2477238048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2477238048},{"type":"v1","addr":"172.21.15.138:6803","nonce":2477238048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2477238048},{"type":"v1","addr":"172.21.15.138:6807","nonce":2477238048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2477238048},{"type":"v1","addr":"172.21.15.138:6805","nonce":2477238048}]},"public_addr":"172.21.15.138:6801/2477238048","cluster_addr":"172.21.15.138:6803/2477238048","heartbeat_back_addr":"172.21.15.138:6807/2477238048","heartbeat_front_addr":"172.21.15.138:6805/2477238048","state":["exists","up"]},{"osd":4,"uuid":"d28515a1-4510-4f94-b0b1-3aca283eed05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":792878465},{"type":"v1","addr":"172.21.15.138:6809","nonce":792878465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":792878465},{"type":"v1","addr":"172.21.15.138:6811","nonce":792878465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":792878465},{"type":"v1","addr":"172.21.15.138:6815","nonce":792878465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":792878465},{"type":"v1","addr":"172.21.15.138:6813","nonce":792878465}]},"public_addr":"172.21.15.138:6809/792878465","cluster_addr":"172.21.15.138:6811/792878465","heartbeat_back_addr":"172.21.15.138:6815/792878465","heartbeat_front_addr":"172.21.15.138:6813/792878465","state":["exists","up"]},{"osd":5,"uuid":"ab57d399-f1cf-4139-abba-7cb15e433c25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":406753522},{"type":"v1","addr":"172.21.15.138:6817","nonce":406753522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":406753522},{"type":"v1","addr":"172.21.15.138:6819","nonce":406753522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":406753522},{"type":"v1","addr":"172.21.15.138:6823","nonce":406753522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":406753522},{"type":"v1","addr":"172.21.15.138:6821","nonce":406753522}]},"public_addr":"172.21.15.138:6817/406753522","cluster_addr":"172.21.15.138:6819/406753522","heartbeat_back_addr":"172.21.15.138:6823/406753522","heartbeat_front_addr":"172.21.15.138:6821/406753522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:45:44.441319+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:46:11.518607+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:46:40.661238+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:47:04.556335+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:47:30.603032+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:47:55.950499+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/2414403835":"2024-05-18T21:44:32.293094+0000","172.21.15.107:6801/3405524625":"2024-05-18T21:44:32.293094+0000","172.21.15.107:6800/3405524625":"2024-05-18T21:44:32.293094+0000","172.21.15.107:0/771372867":"2024-05-18T21:44:32.293094+0000","172.21.15.107:0/498460124":"2024-05-18T21:43:22.097723+0000","172.21.15.107:0/3170894163":"2024-05-18T21:43:22.097723+0000","172.21.15.107:6801/1718632476":"2024-05-18T21:43:22.097723+0000","172.21.15.107:6801/1677232931":"2024-05-18T21:42:52.099188+0000","172.21.15.107:0/2856704707":"2024-05-18T21:43:22.097723+0000","172.21.15.107:0/3048013464":"2024-05-18T21:42:52.099188+0000","172.21.15.107:0/2872531133":"2024-05-18T21:42:52.099188+0000","172.21.15.107:0/1077444017":"2024-05-18T21:44:32.293094+0000","172.21.15.107:0/3105018805":"2024-05-18T21:42:52.099188+0000","172.21.15.107:6800/1677232931":"2024-05-18T21:42:52.099188+0000","172.21.15.107:6800/1718632476":"2024-05-18T21:43:22.097723+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-17T21:48:21.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:21.022+0000 7fdb96592700 1 -- 172.21.15.107:0/2504221669 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fdb7c0692a0 msgr2=0x7fdb7c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:48:21.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:21.022+0000 7fdb96592700 1 --2- 172.21.15.107:0/2504221669 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fdb7c0692a0 0x7fdb7c06b750 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fdb8000b3b0 tx=0x7fdb80009e30 comp rx=0 tx=0).stop 2024-05-17T21:48:21.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:21.022+0000 7fdb96592700 1 -- 172.21.15.107:0/2504221669 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb901026b0 msgr2=0x7fdb90113ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:48:21.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:21.022+0000 7fdb96592700 1 --2- 172.21.15.107:0/2504221669 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb901026b0 0x7fdb90113ed0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fdb7800b790 tx=0x7fdb78009220 comp rx=0 tx=0).stop 2024-05-17T21:48:21.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:21.022+0000 7fdb96592700 1 -- 172.21.15.107:0/2504221669 shutdown_connections 2024-05-17T21:48:21.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:21.022+0000 7fdb96592700 1 --2- 172.21.15.107:0/2504221669 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fdb7c0692a0 0x7fdb7c06b750 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:21.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:21.022+0000 7fdb96592700 1 --2- 172.21.15.107:0/2504221669 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb90101d10 0x7fdb90113990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:21.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:21.022+0000 7fdb96592700 1 --2- 172.21.15.107:0/2504221669 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb901026b0 0x7fdb90113ed0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:21.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:21.022+0000 7fdb96592700 1 -- 172.21.15.107:0/2504221669 >> 172.21.15.107:0/2504221669 conn(0x7fdb900f7860 msgr2=0x7fdb900f87d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:48:21.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:21.022+0000 7fdb96592700 1 -- 172.21.15.107:0/2504221669 shutdown_connections 2024-05-17T21:48:21.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:21.022+0000 7fdb96592700 1 -- 172.21.15.107:0/2504221669 wait complete. 2024-05-17T21:48:21.644 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-05-17T21:48:21.644 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-05-17T21:48:21.644 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-05-17T21:48:21.645 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-05-17T21:48:21.645 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-05-17T21:48:21.646 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-05-17T21:48:21.954 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/mon.smithi107/config 2024-05-17T21:48:21.966 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/mon.smithi107/config 2024-05-17T21:48:22.076 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:48:21 smithi107 ceph-mon[28434]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-17T21:48:22.076 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:48:21 smithi107 ceph-mon[28434]: from='client.? 172.21.15.107:0/2504221669' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T21:48:22.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:48:21 smithi138 ceph-mon[35948]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-17T21:48:22.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:48:21 smithi138 ceph-mon[35948]: from='client.? 172.21.15.107:0/2504221669' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T21:48:22.610 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/mon.smithi107/config 2024-05-17T21:48:22.611 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/mon.smithi107/config 2024-05-17T21:48:23.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:48:23 smithi107 ceph-mon[28434]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-17T21:48:24.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:48:23 smithi138 ceph-mon[35948]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-17T21:48:24.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:24.186+0000 7fb2fc87d700 1 -- 172.21.15.107:0/1507336831 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2f8103780 msgr2=0x7fb2f80fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:48:24.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:24.186+0000 7fb2fc87d700 1 --2- 172.21.15.107:0/1507336831 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2f8103780 0x7fb2f80fcdf0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec004940 tx=0x7fb2ec009e30 comp rx=0 tx=0).stop 2024-05-17T21:48:24.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:24.186+0000 7fb2fc87d700 1 -- 172.21.15.107:0/1507336831 shutdown_connections 2024-05-17T21:48:24.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:24.186+0000 7fb2fc87d700 1 --2- 172.21.15.107:0/1507336831 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2f8103780 0x7fb2f80fcdf0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:24.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:24.186+0000 7fb2fc87d700 1 --2- 172.21.15.107:0/1507336831 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2f8102de0 0x7fb2f81031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:24.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:24.186+0000 7fb2fc87d700 1 -- 172.21.15.107:0/1507336831 >> 172.21.15.107:0/1507336831 conn(0x7fb2f80f8990 msgr2=0x7fb2f80fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:48:24.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:24.186+0000 7fb2fc87d700 1 -- 172.21.15.107:0/1507336831 shutdown_connections 2024-05-17T21:48:24.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:24.186+0000 7fb2fc87d700 1 -- 172.21.15.107:0/1507336831 wait complete. 2024-05-17T21:48:24.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:24.186+0000 7fb2fc87d700 1 Processor -- start 2024-05-17T21:48:24.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:24.186+0000 7fb2fc87d700 1 -- start start 2024-05-17T21:48:24.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:24.187+0000 7fb2fc87d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2f8102de0 0x7fb2f8113950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:48:24.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:24.187+0000 7fb2fc87d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2f8103780 0x7fb2f8113e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:48:24.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:24.187+0000 7fb2fc87d700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f81144e0 con 0x7fb2f8102de0 2024-05-17T21:48:24.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:24.187+0000 7fb2fc87d700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f810dae0 con 0x7fb2f8103780 2024-05-17T21:48:24.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:24.187+0000 7fb2f659c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2f8102de0 0x7fb2f8113950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:48:24.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:24.187+0000 7fb2f659c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2f8102de0 0x7fb2f8113950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44648/0 (socket says 172.21.15.107:44648) 2024-05-17T21:48:24.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:24.187+0000 7fb2f659c700 1 -- 172.21.15.107:0/570631157 learned_addr learned my addr 172.21.15.107:0/570631157 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:48:24.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:24.187+0000 7fb2f5d9b700 1 --2- 172.21.15.107:0/570631157 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2f8103780 0x7fb2f8113e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:48:24.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:24.187+0000 7fb2f659c700 1 -- 172.21.15.107:0/570631157 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2f8103780 msgr2=0x7fb2f8113e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:48:24.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:24.187+0000 7fb2f659c700 1 --2- 172.21.15.107:0/570631157 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2f8103780 0x7fb2f8113e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:24.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:24.187+0000 7fb2f659c700 1 -- 172.21.15.107:0/570631157 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2ec005040 con 0x7fb2f8102de0 2024-05-17T21:48:24.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:24.187+0000 7fb2f659c700 1 --2- 172.21.15.107:0/570631157 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2f8102de0 0x7fb2f8113950 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb2e8002a50 tx=0x7fb2e80046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:48:24.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:24.187+0000 7fb2e6ffd700 1 -- 172.21.15.107:0/570631157 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb2e8007cc0 con 0x7fb2f8102de0 2024-05-17T21:48:24.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:24.187+0000 7fb2fc87d700 1 -- 172.21.15.107:0/570631157 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2f810ddc0 con 0x7fb2f8102de0 2024-05-17T21:48:24.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:24.187+0000 7fb2e6ffd700 1 -- 172.21.15.107:0/570631157 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb2e800f040 con 0x7fb2f8102de0 2024-05-17T21:48:24.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:24.187+0000 7fb2e6ffd700 1 -- 172.21.15.107:0/570631157 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb2e800da30 con 0x7fb2f8102de0 2024-05-17T21:48:24.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:24.187+0000 7fb2fc87d700 1 -- 172.21.15.107:0/570631157 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2f810e350 con 0x7fb2f8102de0 2024-05-17T21:48:24.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:24.188+0000 7fb2fc87d700 1 -- 172.21.15.107:0/570631157 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb2f810e800 con 0x7fb2f8102de0 2024-05-17T21:48:24.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:24.189+0000 7fb2e6ffd700 1 -- 172.21.15.107:0/570631157 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7fb2e800db90 con 0x7fb2f8102de0 2024-05-17T21:48:24.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:24.189+0000 7fb2e6ffd700 1 --2- 172.21.15.107:0/570631157 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fb2e00692a0 0x7fb2e006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:48:24.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:24.189+0000 7fb2f5d9b700 1 --2- 172.21.15.107:0/570631157 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fb2e00692a0 0x7fb2e006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:48:24.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:24.189+0000 7fb2e6ffd700 1 -- 172.21.15.107:0/570631157 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fb2e8087690 con 0x7fb2f8102de0 2024-05-17T21:48:24.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:24.189+0000 7fb2e6ffd700 1 --2- 172.21.15.107:0/570631157 >> [v2:172.21.15.138:6808/792878465,v1:172.21.15.138:6809/792878465] conn(0x7fb2e006ef70 0x7fb2e0071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:48:24.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:24.189+0000 7fb2e6ffd700 1 -- 172.21.15.107:0/570631157 --> [v2:172.21.15.138:6808/792878465,v1:172.21.15.138:6809/792878465] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb2e0071a30 con 0x7fb2e006ef70 2024-05-17T21:48:24.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:24.189+0000 7fb2e6ffd700 1 -- 172.21.15.107:0/570631157 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb2e8087a70 con 0x7fb2f8102de0 2024-05-17T21:48:24.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:24.190+0000 7fb2f5d9b700 1 --2- 172.21.15.107:0/570631157 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fb2e00692a0 0x7fb2e006b750 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec004890 tx=0x7fb2ec009e30 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:48:24.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:24.190+0000 7fb2f6d9d700 1 --2- 172.21.15.107:0/570631157 >> [v2:172.21.15.138:6808/792878465,v1:172.21.15.138:6809/792878465] conn(0x7fb2e006ef70 0x7fb2e0071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:48:24.192 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:24.190+0000 7fb2f6d9d700 1 --2- 172.21.15.107:0/570631157 >> [v2:172.21.15.138:6808/792878465,v1:172.21.15.138:6809/792878465] conn(0x7fb2e006ef70 0x7fb2e0071380 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:48:24.192 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:24.191+0000 7fb2e6ffd700 1 -- 172.21.15.107:0/570631157 <== osd.4 v2:172.21.15.138:6808/792878465 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7fb2e0071a30 con 0x7fb2e006ef70 2024-05-17T21:48:24.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:24.206+0000 7fb2fc87d700 1 -- 172.21.15.107:0/570631157 --> [v2:172.21.15.138:6808/792878465,v1:172.21.15.138:6809/792878465] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb2f80fa360 con 0x7fb2e006ef70 2024-05-17T21:48:24.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:24.207+0000 7fb2e6ffd700 1 -- 172.21.15.107:0/570631157 <== osd.4 v2:172.21.15.138:6808/792878465 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fb2f80fa360 con 0x7fb2e006ef70 2024-05-17T21:48:24.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:24.207+0000 7fb2fc87d700 1 -- 172.21.15.107:0/570631157 >> [v2:172.21.15.138:6808/792878465,v1:172.21.15.138:6809/792878465] conn(0x7fb2e006ef70 msgr2=0x7fb2e0071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:48:24.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:24.207+0000 7fb2fc87d700 1 --2- 172.21.15.107:0/570631157 >> [v2:172.21.15.138:6808/792878465,v1:172.21.15.138:6809/792878465] conn(0x7fb2e006ef70 0x7fb2e0071380 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:24.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:24.207+0000 7fb2fc87d700 1 -- 172.21.15.107:0/570631157 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fb2e00692a0 msgr2=0x7fb2e006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:48:24.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:24.207+0000 7fb2fc87d700 1 --2- 172.21.15.107:0/570631157 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fb2e00692a0 0x7fb2e006b750 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec004890 tx=0x7fb2ec009e30 comp rx=0 tx=0).stop 2024-05-17T21:48:24.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:24.207+0000 7fb2fc87d700 1 -- 172.21.15.107:0/570631157 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2f8102de0 msgr2=0x7fb2f8113950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:48:24.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:24.207+0000 7fb2fc87d700 1 --2- 172.21.15.107:0/570631157 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2f8102de0 0x7fb2f8113950 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb2e8002a50 tx=0x7fb2e80046b0 comp rx=0 tx=0).stop 2024-05-17T21:48:24.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:24.207+0000 7fb2fc87d700 1 -- 172.21.15.107:0/570631157 shutdown_connections 2024-05-17T21:48:24.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:24.207+0000 7fb2fc87d700 1 --2- 172.21.15.107:0/570631157 >> [v2:172.21.15.138:6808/792878465,v1:172.21.15.138:6809/792878465] conn(0x7fb2e006ef70 0x7fb2e0071380 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:24.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:24.207+0000 7fb2fc87d700 1 --2- 172.21.15.107:0/570631157 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fb2e00692a0 0x7fb2e006b750 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:24.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:24.207+0000 7fb2fc87d700 1 --2- 172.21.15.107:0/570631157 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2f8102de0 0x7fb2f8113950 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:24.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:24.207+0000 7fb2fc87d700 1 --2- 172.21.15.107:0/570631157 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2f8103780 0x7fb2f8113e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:24.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:24.207+0000 7fb2fc87d700 1 -- 172.21.15.107:0/570631157 >> 172.21.15.107:0/570631157 conn(0x7fb2f80f8990 msgr2=0x7fb2f80f9990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:48:24.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:24.208+0000 7fb2fc87d700 1 -- 172.21.15.107:0/570631157 shutdown_connections 2024-05-17T21:48:24.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:24.208+0000 7fb2fc87d700 1 -- 172.21.15.107:0/570631157 wait complete. 2024-05-17T21:48:24.696 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/mon.smithi107/config 2024-05-17T21:48:26.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:48:25 smithi138 ceph-mon[35948]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-17T21:48:26.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:48:25 smithi107 ceph-mon[28434]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-17T21:48:26.521 INFO:teuthology.orchestra.run.smithi107.stdout:146028888078 2024-05-17T21:48:26.522 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-05-17T21:48:27.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.480+0000 7f83106bf700 1 -- 172.21.15.107:0/1399334673 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8308070300 msgr2=0x7f83080706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:48:27.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.480+0000 7f83106bf700 1 --2- 172.21.15.107:0/1399334673 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8308070300 0x7f83080706d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8304004970 tx=0x7f8304009e30 comp rx=0 tx=0).stop 2024-05-17T21:48:27.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.480+0000 7f83106bf700 1 -- 172.21.15.107:0/1399334673 shutdown_connections 2024-05-17T21:48:27.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.480+0000 7f83106bf700 1 --2- 172.21.15.107:0/1399334673 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8308070c10 0x7f83081090f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:27.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.480+0000 7f83106bf700 1 --2- 172.21.15.107:0/1399334673 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8308070300 0x7f83080706d0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:27.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.480+0000 7f83106bf700 1 -- 172.21.15.107:0/1399334673 >> 172.21.15.107:0/1399334673 conn(0x7f83080f89b0 msgr2=0x7f83080fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:48:27.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.481+0000 7f83106bf700 1 -- 172.21.15.107:0/1399334673 shutdown_connections 2024-05-17T21:48:27.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.481+0000 7f83106bf700 1 -- 172.21.15.107:0/1399334673 wait complete. 2024-05-17T21:48:27.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.481+0000 7f83106bf700 1 Processor -- start 2024-05-17T21:48:27.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.481+0000 7f83106bf700 1 -- start start 2024-05-17T21:48:27.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.482+0000 7f83106bf700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8308070300 0x7f8308115d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:48:27.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.482+0000 7f83106bf700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8308070c10 0x7f8308116270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:48:27.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.482+0000 7f83106bf700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83081168c0 con 0x7f8308070c10 2024-05-17T21:48:27.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.482+0000 7f83106bf700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f830810fc90 con 0x7f8308070300 2024-05-17T21:48:27.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.482+0000 7f830dc5a700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8308070c10 0x7f8308116270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:48:27.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.482+0000 7f830dc5a700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8308070c10 0x7f8308116270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39214/0 (socket says 172.21.15.107:39214) 2024-05-17T21:48:27.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.482+0000 7f830dc5a700 1 -- 172.21.15.107:0/3592353280 learned_addr learned my addr 172.21.15.107:0/3592353280 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:48:27.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.482+0000 7f830e45b700 1 --2- 172.21.15.107:0/3592353280 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8308070300 0x7f8308115d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:48:27.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.482+0000 7f830dc5a700 1 -- 172.21.15.107:0/3592353280 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8308070300 msgr2=0x7f8308115d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:48:27.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.482+0000 7f830dc5a700 1 --2- 172.21.15.107:0/3592353280 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8308070300 0x7f8308115d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:27.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.482+0000 7f830dc5a700 1 -- 172.21.15.107:0/3592353280 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8304005040 con 0x7f8308070c10 2024-05-17T21:48:27.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.482+0000 7f830e45b700 1 --2- 172.21.15.107:0/3592353280 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8308070300 0x7f8308115d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T21:48:27.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.482+0000 7f830dc5a700 1 --2- 172.21.15.107:0/3592353280 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8308070c10 0x7f8308116270 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f82f800c8c0 tx=0x7f82f80044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:48:27.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.483+0000 7f82feffd700 1 -- 172.21.15.107:0/3592353280 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f82f800a980 con 0x7f8308070c10 2024-05-17T21:48:27.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.483+0000 7f83106bf700 1 -- 172.21.15.107:0/3592353280 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f830810ff70 con 0x7f8308070c10 2024-05-17T21:48:27.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.483+0000 7f82feffd700 1 -- 172.21.15.107:0/3592353280 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f82f800ce60 con 0x7f8308070c10 2024-05-17T21:48:27.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.483+0000 7f82feffd700 1 -- 172.21.15.107:0/3592353280 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f82f80115e0 con 0x7f8308070c10 2024-05-17T21:48:27.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.483+0000 7f83106bf700 1 -- 172.21.15.107:0/3592353280 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83081104b0 con 0x7f8308070c10 2024-05-17T21:48:27.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.483+0000 7f83106bf700 1 -- 172.21.15.107:0/3592353280 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f83081109f0 con 0x7f8308070c10 2024-05-17T21:48:27.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.484+0000 7f82feffd700 1 -- 172.21.15.107:0/3592353280 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f82f8012400 con 0x7f8308070c10 2024-05-17T21:48:27.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.485+0000 7f82feffd700 1 --2- 172.21.15.107:0/3592353280 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f82f40692a0 0x7f82f406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:48:27.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.485+0000 7f830e45b700 1 --2- 172.21.15.107:0/3592353280 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f82f40692a0 0x7f82f406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:48:27.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.485+0000 7f82feffd700 1 -- 172.21.15.107:0/3592353280 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f82f8087bb0 con 0x7f8308070c10 2024-05-17T21:48:27.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.485+0000 7f82feffd700 1 --2- 172.21.15.107:0/3592353280 >> [v2:172.21.15.138:6816/406753522,v1:172.21.15.138:6817/406753522] conn(0x7f82f406ef70 0x7f82f4071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:48:27.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.485+0000 7f82feffd700 1 -- 172.21.15.107:0/3592353280 --> [v2:172.21.15.138:6816/406753522,v1:172.21.15.138:6817/406753522] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f82f4071a30 con 0x7f82f406ef70 2024-05-17T21:48:27.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.485+0000 7f82feffd700 1 -- 172.21.15.107:0/3592353280 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f82f8087ef0 con 0x7f8308070c10 2024-05-17T21:48:27.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.485+0000 7f830e45b700 1 --2- 172.21.15.107:0/3592353280 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f82f40692a0 0x7f82f406b750 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8304004f50 tx=0x7f8304009e30 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:48:27.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.485+0000 7f830ec5c700 1 --2- 172.21.15.107:0/3592353280 >> [v2:172.21.15.138:6816/406753522,v1:172.21.15.138:6817/406753522] conn(0x7f82f406ef70 0x7f82f4071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:48:27.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.486+0000 7f830ec5c700 1 --2- 172.21.15.107:0/3592353280 >> [v2:172.21.15.138:6816/406753522,v1:172.21.15.138:6817/406753522] conn(0x7f82f406ef70 0x7f82f4071380 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:48:27.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.486+0000 7f82feffd700 1 -- 172.21.15.107:0/3592353280 <== osd.5 v2:172.21.15.138:6816/406753522 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f82f4071a30 con 0x7f82f406ef70 2024-05-17T21:48:27.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.501+0000 7f83106bf700 1 -- 172.21.15.107:0/3592353280 --> [v2:172.21.15.138:6816/406753522,v1:172.21.15.138:6817/406753522] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f83080fa820 con 0x7f82f406ef70 2024-05-17T21:48:27.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.502+0000 7f82feffd700 1 -- 172.21.15.107:0/3592353280 <== osd.5 v2:172.21.15.138:6816/406753522 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f83080fa820 con 0x7f82f406ef70 2024-05-17T21:48:27.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.502+0000 7f83106bf700 1 -- 172.21.15.107:0/3592353280 >> [v2:172.21.15.138:6816/406753522,v1:172.21.15.138:6817/406753522] conn(0x7f82f406ef70 msgr2=0x7f82f4071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:48:27.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.502+0000 7f83106bf700 1 --2- 172.21.15.107:0/3592353280 >> [v2:172.21.15.138:6816/406753522,v1:172.21.15.138:6817/406753522] conn(0x7f82f406ef70 0x7f82f4071380 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:27.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.502+0000 7f83106bf700 1 -- 172.21.15.107:0/3592353280 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f82f40692a0 msgr2=0x7f82f406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:48:27.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.502+0000 7f83106bf700 1 --2- 172.21.15.107:0/3592353280 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f82f40692a0 0x7f82f406b750 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8304004f50 tx=0x7f8304009e30 comp rx=0 tx=0).stop 2024-05-17T21:48:27.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.502+0000 7f83106bf700 1 -- 172.21.15.107:0/3592353280 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8308070c10 msgr2=0x7f8308116270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:48:27.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.502+0000 7f83106bf700 1 --2- 172.21.15.107:0/3592353280 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8308070c10 0x7f8308116270 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f82f800c8c0 tx=0x7f82f80044c0 comp rx=0 tx=0).stop 2024-05-17T21:48:27.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.502+0000 7f83106bf700 1 -- 172.21.15.107:0/3592353280 shutdown_connections 2024-05-17T21:48:27.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.502+0000 7f83106bf700 1 --2- 172.21.15.107:0/3592353280 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f82f40692a0 0x7f82f406b750 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:27.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.502+0000 7f83106bf700 1 --2- 172.21.15.107:0/3592353280 >> [v2:172.21.15.138:6816/406753522,v1:172.21.15.138:6817/406753522] conn(0x7f82f406ef70 0x7f82f4071380 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:27.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.502+0000 7f83106bf700 1 --2- 172.21.15.107:0/3592353280 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8308070300 0x7f8308115d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:27.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.502+0000 7f83106bf700 1 --2- 172.21.15.107:0/3592353280 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8308070c10 0x7f8308116270 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:27.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.502+0000 7f83106bf700 1 -- 172.21.15.107:0/3592353280 >> 172.21.15.107:0/3592353280 conn(0x7f83080f89b0 msgr2=0x7f83080f9e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:48:27.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.502+0000 7f83106bf700 1 -- 172.21.15.107:0/3592353280 shutdown_connections 2024-05-17T21:48:27.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.502+0000 7f83106bf700 1 -- 172.21.15.107:0/3592353280 wait complete. 2024-05-17T21:48:27.813 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:48:27 smithi107 ceph-mon[28434]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-17T21:48:27.913 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.918+0000 7f639ab55700 1 -- 172.21.15.107:0/2229645739 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6394101cd0 msgr2=0x7f63941020a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:48:27.913 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.918+0000 7f639ab55700 1 --2- 172.21.15.107:0/2229645739 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6394101cd0 0x7f63941020a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6388004940 tx=0x7f6388009e30 comp rx=0 tx=0).stop 2024-05-17T21:48:27.913 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.918+0000 7f639ab55700 1 -- 172.21.15.107:0/2229645739 shutdown_connections 2024-05-17T21:48:27.913 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.918+0000 7f639ab55700 1 --2- 172.21.15.107:0/2229645739 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6394102670 0x7f63940fb600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:27.913 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.918+0000 7f639ab55700 1 --2- 172.21.15.107:0/2229645739 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6394101cd0 0x7f63941020a0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:27.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.918+0000 7f639ab55700 1 -- 172.21.15.107:0/2229645739 >> 172.21.15.107:0/2229645739 conn(0x7f6394071460 msgr2=0x7f6394071860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:48:27.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.918+0000 7f639ab55700 1 -- 172.21.15.107:0/2229645739 shutdown_connections 2024-05-17T21:48:27.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.918+0000 7f639ab55700 1 -- 172.21.15.107:0/2229645739 wait complete. 2024-05-17T21:48:27.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.918+0000 7f639ab55700 1 Processor -- start 2024-05-17T21:48:27.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.919+0000 7f639ab55700 1 -- start start 2024-05-17T21:48:27.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.919+0000 7f639ab55700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6394101cd0 0x7f63940fc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:48:27.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.919+0000 7f639ab55700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6394102670 0x7f63940fcd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:48:27.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.919+0000 7f639ab55700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63940fd2b0 con 0x7f6394102670 2024-05-17T21:48:27.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.919+0000 7f639ab55700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63940fd3f0 con 0x7f6394101cd0 2024-05-17T21:48:27.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.919+0000 7f6393fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6394102670 0x7f63940fcd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:48:27.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.919+0000 7f6393fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6394102670 0x7f63940fcd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39230/0 (socket says 172.21.15.107:39230) 2024-05-17T21:48:27.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.919+0000 7f6393fff700 1 -- 172.21.15.107:0/1473644981 learned_addr learned my addr 172.21.15.107:0/1473644981 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:48:27.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.919+0000 7f63988f1700 1 --2- 172.21.15.107:0/1473644981 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6394101cd0 0x7f63940fc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:48:27.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.919+0000 7f6393fff700 1 -- 172.21.15.107:0/1473644981 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6394101cd0 msgr2=0x7f63940fc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:48:27.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.919+0000 7f6393fff700 1 --2- 172.21.15.107:0/1473644981 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6394101cd0 0x7f63940fc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:27.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.919+0000 7f6393fff700 1 -- 172.21.15.107:0/1473644981 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6388005040 con 0x7f6394102670 2024-05-17T21:48:27.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.920+0000 7f6393fff700 1 --2- 172.21.15.107:0/1473644981 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6394102670 0x7f63940fcd20 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f638000c810 tx=0x7f63800043f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:48:27.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.920+0000 7f63917fa700 1 -- 172.21.15.107:0/1473644981 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6380012810 con 0x7f6394102670 2024-05-17T21:48:27.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.920+0000 7f639ab55700 1 -- 172.21.15.107:0/1473644981 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6394119e80 con 0x7f6394102670 2024-05-17T21:48:27.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.920+0000 7f63917fa700 1 -- 172.21.15.107:0/1473644981 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6380012e40 con 0x7f6394102670 2024-05-17T21:48:27.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.920+0000 7f63917fa700 1 -- 172.21.15.107:0/1473644981 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f638000a550 con 0x7f6394102670 2024-05-17T21:48:27.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.920+0000 7f639ab55700 1 -- 172.21.15.107:0/1473644981 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f639411a300 con 0x7f6394102670 2024-05-17T21:48:27.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.921+0000 7f639ab55700 1 -- 172.21.15.107:0/1473644981 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f639405e660 con 0x7f6394102670 2024-05-17T21:48:27.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.921+0000 7f63917fa700 1 -- 172.21.15.107:0/1473644981 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f6380012970 con 0x7f6394102670 2024-05-17T21:48:27.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.922+0000 7f63917fa700 1 --2- 172.21.15.107:0/1473644981 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f63840692a0 0x7f638406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:48:27.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.922+0000 7f63917fa700 1 -- 172.21.15.107:0/1473644981 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f6380087270 con 0x7f6394102670 2024-05-17T21:48:27.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.922+0000 7f63988f1700 1 --2- 172.21.15.107:0/1473644981 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f63840692a0 0x7f638406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:48:27.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.922+0000 7f63917fa700 1 --2- 172.21.15.107:0/1473644981 >> [v2:172.21.15.107:6810/686958797,v1:172.21.15.107:6811/686958797] conn(0x7f638406ef70 0x7f6384071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:48:27.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.922+0000 7f63917fa700 1 -- 172.21.15.107:0/1473644981 --> [v2:172.21.15.107:6810/686958797,v1:172.21.15.107:6811/686958797] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6384071a30 con 0x7f638406ef70 2024-05-17T21:48:27.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.922+0000 7f63917fa700 1 -- 172.21.15.107:0/1473644981 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6380089860 con 0x7f6394102670 2024-05-17T21:48:27.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.922+0000 7f63990f2700 1 --2- 172.21.15.107:0/1473644981 >> [v2:172.21.15.107:6810/686958797,v1:172.21.15.107:6811/686958797] conn(0x7f638406ef70 0x7f6384071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:48:27.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.922+0000 7f63988f1700 1 --2- 172.21.15.107:0/1473644981 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f63840692a0 0x7f638406b750 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6388004f90 tx=0x7f6388009e30 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:48:27.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.922+0000 7f63990f2700 1 --2- 172.21.15.107:0/1473644981 >> [v2:172.21.15.107:6810/686958797,v1:172.21.15.107:6811/686958797] conn(0x7f638406ef70 0x7f6384071380 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:48:27.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.923+0000 7f63917fa700 1 -- 172.21.15.107:0/1473644981 <== osd.1 v2:172.21.15.107:6810/686958797 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f6384071a30 con 0x7f638406ef70 2024-05-17T21:48:27.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.938+0000 7f639ab55700 1 -- 172.21.15.107:0/1473644981 --> [v2:172.21.15.107:6810/686958797,v1:172.21.15.107:6811/686958797] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f639405ea90 con 0x7f638406ef70 2024-05-17T21:48:27.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.939+0000 7f63917fa700 1 -- 172.21.15.107:0/1473644981 <== osd.1 v2:172.21.15.107:6810/686958797 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f639405ea90 con 0x7f638406ef70 2024-05-17T21:48:27.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.939+0000 7f639ab55700 1 -- 172.21.15.107:0/1473644981 >> [v2:172.21.15.107:6810/686958797,v1:172.21.15.107:6811/686958797] conn(0x7f638406ef70 msgr2=0x7f6384071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:48:27.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.939+0000 7f639ab55700 1 --2- 172.21.15.107:0/1473644981 >> [v2:172.21.15.107:6810/686958797,v1:172.21.15.107:6811/686958797] conn(0x7f638406ef70 0x7f6384071380 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:27.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.939+0000 7f639ab55700 1 -- 172.21.15.107:0/1473644981 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f63840692a0 msgr2=0x7f638406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:48:27.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.939+0000 7f639ab55700 1 --2- 172.21.15.107:0/1473644981 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f63840692a0 0x7f638406b750 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6388004f90 tx=0x7f6388009e30 comp rx=0 tx=0).stop 2024-05-17T21:48:27.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.939+0000 7f639ab55700 1 -- 172.21.15.107:0/1473644981 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6394102670 msgr2=0x7f63940fcd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:48:27.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.939+0000 7f639ab55700 1 --2- 172.21.15.107:0/1473644981 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6394102670 0x7f63940fcd20 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f638000c810 tx=0x7f63800043f0 comp rx=0 tx=0).stop 2024-05-17T21:48:27.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.939+0000 7f639ab55700 1 -- 172.21.15.107:0/1473644981 shutdown_connections 2024-05-17T21:48:27.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.939+0000 7f639ab55700 1 --2- 172.21.15.107:0/1473644981 >> [v2:172.21.15.107:6810/686958797,v1:172.21.15.107:6811/686958797] conn(0x7f638406ef70 0x7f6384071380 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:27.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.939+0000 7f639ab55700 1 --2- 172.21.15.107:0/1473644981 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f63840692a0 0x7f638406b750 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:27.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.939+0000 7f639ab55700 1 --2- 172.21.15.107:0/1473644981 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6394101cd0 0x7f63940fc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:27.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.939+0000 7f639ab55700 1 --2- 172.21.15.107:0/1473644981 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6394102670 0x7f63940fcd20 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:27.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.939+0000 7f639ab55700 1 -- 172.21.15.107:0/1473644981 >> 172.21.15.107:0/1473644981 conn(0x7f6394071460 msgr2=0x7f63940fabe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:48:27.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.939+0000 7f639ab55700 1 -- 172.21.15.107:0/1473644981 shutdown_connections 2024-05-17T21:48:27.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:27.939+0000 7f639ab55700 1 -- 172.21.15.107:0/1473644981 wait complete. 2024-05-17T21:48:28.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:48:27 smithi138 ceph-mon[35948]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-17T21:48:28.866 INFO:teuthology.orchestra.run.smithi107.stdout:167503724553 2024-05-17T21:48:28.867 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-05-17T21:48:29.878 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:48:29 smithi107 ceph-mon[28434]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-17T21:48:30.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:48:29 smithi138 ceph-mon[35948]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-17T21:48:30.208 INFO:teuthology.orchestra.run.smithi107.stdout:60129542173 2024-05-17T21:48:30.208 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-05-17T21:48:30.868 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/mon.smithi107/config 2024-05-17T21:48:30.869 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/mon.smithi107/config 2024-05-17T21:48:30.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:30.878+0000 7f16c3e62700 1 -- 172.21.15.107:0/2716079868 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f16bc103780 msgr2=0x7f16bc0fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:48:30.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:30.878+0000 7f16c3e62700 1 --2- 172.21.15.107:0/2716079868 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f16bc103780 0x7f16bc0fcdf0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f16ac004940 tx=0x7f16ac009e30 comp rx=0 tx=0).stop 2024-05-17T21:48:30.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:30.878+0000 7f16c3e62700 1 -- 172.21.15.107:0/2716079868 shutdown_connections 2024-05-17T21:48:30.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:30.878+0000 7f16c3e62700 1 --2- 172.21.15.107:0/2716079868 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f16bc103780 0x7f16bc0fcdf0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:30.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:30.878+0000 7f16c3e62700 1 --2- 172.21.15.107:0/2716079868 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16bc102de0 0x7f16bc1031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:30.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:30.878+0000 7f16c3e62700 1 -- 172.21.15.107:0/2716079868 >> 172.21.15.107:0/2716079868 conn(0x7f16bc0f8950 msgr2=0x7f16bc0fad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:48:30.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:30.879+0000 7f16c3e62700 1 -- 172.21.15.107:0/2716079868 shutdown_connections 2024-05-17T21:48:30.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:30.879+0000 7f16c3e62700 1 -- 172.21.15.107:0/2716079868 wait complete. 2024-05-17T21:48:30.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:30.880+0000 7f16c3e62700 1 Processor -- start 2024-05-17T21:48:30.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:30.880+0000 7f16c3e62700 1 -- start start 2024-05-17T21:48:30.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:30.880+0000 7f16c3e62700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f16bc102de0 0x7f16bc1129e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:48:30.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:30.880+0000 7f16c1bfe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f16bc102de0 0x7f16bc1129e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:48:30.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:30.880+0000 7f16c1bfe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f16bc102de0 0x7f16bc1129e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39246/0 (socket says 172.21.15.107:39246) 2024-05-17T21:48:30.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:30.881+0000 7f16c1bfe700 1 -- 172.21.15.107:0/979091002 learned_addr learned my addr 172.21.15.107:0/979091002 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:48:30.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:30.881+0000 7f16c3e62700 1 --2- 172.21.15.107:0/979091002 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16bc103780 0x7f16bc112f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:48:30.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:30.881+0000 7f16c3e62700 1 -- 172.21.15.107:0/979091002 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16bc113520 con 0x7f16bc102de0 2024-05-17T21:48:30.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:30.881+0000 7f16c3e62700 1 -- 172.21.15.107:0/979091002 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16bc10daf0 con 0x7f16bc103780 2024-05-17T21:48:30.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:30.881+0000 7f16c13fd700 1 --2- 172.21.15.107:0/979091002 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16bc103780 0x7f16bc112f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:48:30.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:30.881+0000 7f16c1bfe700 1 -- 172.21.15.107:0/979091002 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16bc103780 msgr2=0x7f16bc112f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:48:30.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:30.881+0000 7f16c1bfe700 1 --2- 172.21.15.107:0/979091002 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16bc103780 0x7f16bc112f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:30.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:30.881+0000 7f16c1bfe700 1 -- 172.21.15.107:0/979091002 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16ac005040 con 0x7f16bc102de0 2024-05-17T21:48:30.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:30.881+0000 7f16c1bfe700 1 --2- 172.21.15.107:0/979091002 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f16bc102de0 0x7f16bc1129e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f16b800eb70 tx=0x7f16b800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:48:30.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:30.881+0000 7f16b27fc700 1 -- 172.21.15.107:0/979091002 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f16b8007cc0 con 0x7f16bc102de0 2024-05-17T21:48:30.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:30.882+0000 7f16b27fc700 1 -- 172.21.15.107:0/979091002 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f16b800f040 con 0x7f16bc102de0 2024-05-17T21:48:30.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:30.882+0000 7f16c3e62700 1 -- 172.21.15.107:0/979091002 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16bc10dd70 con 0x7f16bc102de0 2024-05-17T21:48:30.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:30.882+0000 7f16b27fc700 1 -- 172.21.15.107:0/979091002 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f16b8009640 con 0x7f16bc102de0 2024-05-17T21:48:30.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:30.882+0000 7f16c3e62700 1 -- 172.21.15.107:0/979091002 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16bc10e2b0 con 0x7f16bc102de0 2024-05-17T21:48:30.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:30.883+0000 7f16c3e62700 1 -- 172.21.15.107:0/979091002 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f16bc10e730 con 0x7f16bc102de0 2024-05-17T21:48:30.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:30.884+0000 7f16b27fc700 1 -- 172.21.15.107:0/979091002 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f16b80097a0 con 0x7f16bc102de0 2024-05-17T21:48:30.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:30.884+0000 7f16b27fc700 1 --2- 172.21.15.107:0/979091002 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f16a80692b0 0x7f16a806b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:48:30.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:30.885+0000 7f16b27fc700 1 -- 172.21.15.107:0/979091002 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f16b8087460 con 0x7f16bc102de0 2024-05-17T21:48:30.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:30.885+0000 7f16b27fc700 1 --2- 172.21.15.107:0/979091002 >> [v2:172.21.15.138:6800/2477238048,v1:172.21.15.138:6801/2477238048] conn(0x7f16a806ef80 0x7f16a8071390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:48:30.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:30.885+0000 7f16b27fc700 1 -- 172.21.15.107:0/979091002 --> [v2:172.21.15.138:6800/2477238048,v1:172.21.15.138:6801/2477238048] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f16a8071a40 con 0x7f16a806ef80 2024-05-17T21:48:30.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:30.885+0000 7f16b27fc700 1 -- 172.21.15.107:0/979091002 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f16b8087840 con 0x7f16bc102de0 2024-05-17T21:48:30.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:30.885+0000 7f16c13fd700 1 --2- 172.21.15.107:0/979091002 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f16a80692b0 0x7f16a806b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:48:30.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:30.885+0000 7f16c23ff700 1 --2- 172.21.15.107:0/979091002 >> [v2:172.21.15.138:6800/2477238048,v1:172.21.15.138:6801/2477238048] conn(0x7f16a806ef80 0x7f16a8071390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:48:30.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:30.885+0000 7f16c13fd700 1 --2- 172.21.15.107:0/979091002 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f16a80692b0 0x7f16a806b760 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f16ac00b0d0 tx=0x7f16ac009e30 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:48:30.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:30.886+0000 7f16c23ff700 1 --2- 172.21.15.107:0/979091002 >> [v2:172.21.15.138:6800/2477238048,v1:172.21.15.138:6801/2477238048] conn(0x7f16a806ef80 0x7f16a8071390 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:48:30.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:30.886+0000 7f16b27fc700 1 -- 172.21.15.107:0/979091002 <== osd.3 v2:172.21.15.138:6800/2477238048 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f16a8071a40 con 0x7f16a806ef80 2024-05-17T21:48:30.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:30.909+0000 7f16c3e62700 1 -- 172.21.15.107:0/979091002 --> [v2:172.21.15.138:6800/2477238048,v1:172.21.15.138:6801/2477238048] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f16bc05e660 con 0x7f16a806ef80 2024-05-17T21:48:30.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:30.910+0000 7f16b27fc700 1 -- 172.21.15.107:0/979091002 <== osd.3 v2:172.21.15.138:6800/2477238048 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f16bc05e660 con 0x7f16a806ef80 2024-05-17T21:48:30.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:30.910+0000 7f16c3e62700 1 -- 172.21.15.107:0/979091002 >> [v2:172.21.15.138:6800/2477238048,v1:172.21.15.138:6801/2477238048] conn(0x7f16a806ef80 msgr2=0x7f16a8071390 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:48:30.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:30.910+0000 7f16c3e62700 1 --2- 172.21.15.107:0/979091002 >> [v2:172.21.15.138:6800/2477238048,v1:172.21.15.138:6801/2477238048] conn(0x7f16a806ef80 0x7f16a8071390 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:30.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:30.910+0000 7f16c3e62700 1 -- 172.21.15.107:0/979091002 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f16a80692b0 msgr2=0x7f16a806b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:48:30.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:30.910+0000 7f16c3e62700 1 --2- 172.21.15.107:0/979091002 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f16a80692b0 0x7f16a806b760 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f16ac00b0d0 tx=0x7f16ac009e30 comp rx=0 tx=0).stop 2024-05-17T21:48:30.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:30.910+0000 7f16c3e62700 1 -- 172.21.15.107:0/979091002 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f16bc102de0 msgr2=0x7f16bc1129e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:48:30.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:30.910+0000 7f16c3e62700 1 --2- 172.21.15.107:0/979091002 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f16bc102de0 0x7f16bc1129e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f16b800eb70 tx=0x7f16b800b040 comp rx=0 tx=0).stop 2024-05-17T21:48:30.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:30.910+0000 7f16c3e62700 1 -- 172.21.15.107:0/979091002 shutdown_connections 2024-05-17T21:48:30.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:30.910+0000 7f16c3e62700 1 --2- 172.21.15.107:0/979091002 >> [v2:172.21.15.138:6800/2477238048,v1:172.21.15.138:6801/2477238048] conn(0x7f16a806ef80 0x7f16a8071390 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:30.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:30.910+0000 7f16c3e62700 1 --2- 172.21.15.107:0/979091002 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f16a80692b0 0x7f16a806b760 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:30.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:30.910+0000 7f16c3e62700 1 --2- 172.21.15.107:0/979091002 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f16bc102de0 0x7f16bc1129e0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:30.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:30.910+0000 7f16c3e62700 1 --2- 172.21.15.107:0/979091002 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16bc103780 0x7f16bc112f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:30.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:30.911+0000 7f16c3e62700 1 -- 172.21.15.107:0/979091002 >> 172.21.15.107:0/979091002 conn(0x7f16bc0f8950 msgr2=0x7f16bc1021f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:48:30.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:30.911+0000 7f16c3e62700 1 -- 172.21.15.107:0/979091002 shutdown_connections 2024-05-17T21:48:30.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:30.911+0000 7f16c3e62700 1 -- 172.21.15.107:0/979091002 wait complete. 2024-05-17T21:48:31.242 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/mon.smithi107/config 2024-05-17T21:48:31.243 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/mon.smithi107/config 2024-05-17T21:48:32.010 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:48:31 smithi107 ceph-mon[28434]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-17T21:48:32.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:48:31 smithi138 ceph-mon[35948]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-17T21:48:32.988 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:48:32 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:48:33.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:48:32 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:48:33.994 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:48:33 smithi138 ceph-mon[35948]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-17T21:48:34.031 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:48:33 smithi107 ceph-mon[28434]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-17T21:48:34.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:34.138+0000 7f0bd20ea700 1 -- 172.21.15.107:0/869277356 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0bcc0fbf40 msgr2=0x7f0bcc101a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:48:34.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:34.138+0000 7f0bd20ea700 1 --2- 172.21.15.107:0/869277356 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0bcc0fbf40 0x7f0bcc101a30 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc004940 tx=0x7f0bbc009e30 comp rx=0 tx=0).stop 2024-05-17T21:48:34.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:34.138+0000 7f0bd20ea700 1 -- 172.21.15.107:0/869277356 shutdown_connections 2024-05-17T21:48:34.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:34.138+0000 7f0bd20ea700 1 --2- 172.21.15.107:0/869277356 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0bcc0fbf40 0x7f0bcc101a30 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:34.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:34.138+0000 7f0bd20ea700 1 --2- 172.21.15.107:0/869277356 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0bcc0fb5a0 0x7f0bcc0fb970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:34.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:34.138+0000 7f0bd20ea700 1 -- 172.21.15.107:0/869277356 >> 172.21.15.107:0/869277356 conn(0x7f0bcc073350 msgr2=0x7f0bcc073750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:48:34.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:34.138+0000 7f0bd20ea700 1 -- 172.21.15.107:0/869277356 shutdown_connections 2024-05-17T21:48:34.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:34.138+0000 7f0bd20ea700 1 -- 172.21.15.107:0/869277356 wait complete. 2024-05-17T21:48:34.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:34.139+0000 7f0bd20ea700 1 Processor -- start 2024-05-17T21:48:34.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:34.139+0000 7f0bd20ea700 1 -- start start 2024-05-17T21:48:34.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:34.139+0000 7f0bd20ea700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0bcc0fb5a0 0x7f0bcc113a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:48:34.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:34.139+0000 7f0bd20ea700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0bcc0fbf40 0x7f0bcc113f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:48:34.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:34.139+0000 7f0bd20ea700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bcc1145d0 con 0x7f0bcc0fb5a0 2024-05-17T21:48:34.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:34.139+0000 7f0bd20ea700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bcc10dbb0 con 0x7f0bcc0fbf40 2024-05-17T21:48:34.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:34.139+0000 7f0bcb7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0bcc0fb5a0 0x7f0bcc113a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:48:34.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:34.139+0000 7f0bcb7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0bcc0fb5a0 0x7f0bcc113a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39274/0 (socket says 172.21.15.107:39274) 2024-05-17T21:48:34.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:34.139+0000 7f0bcb7fe700 1 -- 172.21.15.107:0/4083700613 learned_addr learned my addr 172.21.15.107:0/4083700613 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:48:34.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:34.140+0000 7f0bcaffd700 1 --2- 172.21.15.107:0/4083700613 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0bcc0fbf40 0x7f0bcc113f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:48:34.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:34.140+0000 7f0bcb7fe700 1 -- 172.21.15.107:0/4083700613 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0bcc0fbf40 msgr2=0x7f0bcc113f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:48:34.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:34.140+0000 7f0bcb7fe700 1 --2- 172.21.15.107:0/4083700613 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0bcc0fbf40 0x7f0bcc113f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:34.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:34.140+0000 7f0bcb7fe700 1 -- 172.21.15.107:0/4083700613 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bbc005040 con 0x7f0bcc0fb5a0 2024-05-17T21:48:34.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:34.140+0000 7f0bcb7fe700 1 --2- 172.21.15.107:0/4083700613 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0bcc0fb5a0 0x7f0bcc113a40 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0bb400eb40 tx=0x7f0bb400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:48:34.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:34.140+0000 7f0bc3fff700 1 -- 172.21.15.107:0/4083700613 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0bb4007cc0 con 0x7f0bcc0fb5a0 2024-05-17T21:48:34.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:34.140+0000 7f0bd20ea700 1 -- 172.21.15.107:0/4083700613 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bcc10de30 con 0x7f0bcc0fb5a0 2024-05-17T21:48:34.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:34.140+0000 7f0bc3fff700 1 -- 172.21.15.107:0/4083700613 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0bb400f040 con 0x7f0bcc0fb5a0 2024-05-17T21:48:34.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:34.140+0000 7f0bc3fff700 1 -- 172.21.15.107:0/4083700613 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0bb400ca10 con 0x7f0bcc0fb5a0 2024-05-17T21:48:34.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:34.140+0000 7f0bd20ea700 1 -- 172.21.15.107:0/4083700613 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bcc10e340 con 0x7f0bcc0fb5a0 2024-05-17T21:48:34.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:34.141+0000 7f0bd20ea700 1 -- 172.21.15.107:0/4083700613 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0bcc10e7a0 con 0x7f0bcc0fb5a0 2024-05-17T21:48:34.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:34.142+0000 7f0bc3fff700 1 -- 172.21.15.107:0/4083700613 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f0bb400cc80 con 0x7f0bcc0fb5a0 2024-05-17T21:48:34.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:34.142+0000 7f0bc3fff700 1 --2- 172.21.15.107:0/4083700613 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f0bb80692a0 0x7f0bb806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:48:34.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:34.142+0000 7f0bc3fff700 1 -- 172.21.15.107:0/4083700613 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f0bb4086a50 con 0x7f0bcc0fb5a0 2024-05-17T21:48:34.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:34.142+0000 7f0bcaffd700 1 --2- 172.21.15.107:0/4083700613 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f0bb80692a0 0x7f0bb806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:48:34.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:34.142+0000 7f0bc3fff700 1 --2- 172.21.15.107:0/4083700613 >> [v2:172.21.15.107:6818/3962264726,v1:172.21.15.107:6819/3962264726] conn(0x7f0bb806ef70 0x7f0bb8071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:48:34.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:34.142+0000 7f0bc3fff700 1 -- 172.21.15.107:0/4083700613 --> [v2:172.21.15.107:6818/3962264726,v1:172.21.15.107:6819/3962264726] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0bb8071a30 con 0x7f0bb806ef70 2024-05-17T21:48:34.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:34.142+0000 7f0bc3fff700 1 -- 172.21.15.107:0/4083700613 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0bb408b220 con 0x7f0bcc0fb5a0 2024-05-17T21:48:34.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:34.142+0000 7f0bcbfff700 1 --2- 172.21.15.107:0/4083700613 >> [v2:172.21.15.107:6818/3962264726,v1:172.21.15.107:6819/3962264726] conn(0x7f0bb806ef70 0x7f0bb8071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:48:34.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:34.143+0000 7f0bcaffd700 1 --2- 172.21.15.107:0/4083700613 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f0bb80692a0 0x7f0bb806b750 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc00b0d0 tx=0x7f0bbc009e30 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:48:34.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:34.143+0000 7f0bcbfff700 1 --2- 172.21.15.107:0/4083700613 >> [v2:172.21.15.107:6818/3962264726,v1:172.21.15.107:6819/3962264726] conn(0x7f0bb806ef70 0x7f0bb8071380 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:48:34.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:34.143+0000 7f0bc3fff700 1 -- 172.21.15.107:0/4083700613 <== osd.2 v2:172.21.15.107:6818/3962264726 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f0bb8071a30 con 0x7f0bb806ef70 2024-05-17T21:48:34.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:34.158+0000 7f0bd20ea700 1 -- 172.21.15.107:0/4083700613 --> [v2:172.21.15.107:6818/3962264726,v1:172.21.15.107:6819/3962264726] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0bcc0fe670 con 0x7f0bb806ef70 2024-05-17T21:48:34.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:34.158+0000 7f0bc3fff700 1 -- 172.21.15.107:0/4083700613 <== osd.2 v2:172.21.15.107:6818/3962264726 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f0bcc0fe670 con 0x7f0bb806ef70 2024-05-17T21:48:34.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:34.158+0000 7f0bd20ea700 1 -- 172.21.15.107:0/4083700613 >> [v2:172.21.15.107:6818/3962264726,v1:172.21.15.107:6819/3962264726] conn(0x7f0bb806ef70 msgr2=0x7f0bb8071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:48:34.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:34.158+0000 7f0bd20ea700 1 --2- 172.21.15.107:0/4083700613 >> [v2:172.21.15.107:6818/3962264726,v1:172.21.15.107:6819/3962264726] conn(0x7f0bb806ef70 0x7f0bb8071380 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:34.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:34.158+0000 7f0bd20ea700 1 -- 172.21.15.107:0/4083700613 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f0bb80692a0 msgr2=0x7f0bb806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:48:34.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:34.158+0000 7f0bd20ea700 1 --2- 172.21.15.107:0/4083700613 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f0bb80692a0 0x7f0bb806b750 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc00b0d0 tx=0x7f0bbc009e30 comp rx=0 tx=0).stop 2024-05-17T21:48:34.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:34.158+0000 7f0bd20ea700 1 -- 172.21.15.107:0/4083700613 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0bcc0fb5a0 msgr2=0x7f0bcc113a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:48:34.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:34.158+0000 7f0bd20ea700 1 --2- 172.21.15.107:0/4083700613 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0bcc0fb5a0 0x7f0bcc113a40 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0bb400eb40 tx=0x7f0bb400b040 comp rx=0 tx=0).stop 2024-05-17T21:48:34.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:34.158+0000 7f0bd20ea700 1 -- 172.21.15.107:0/4083700613 shutdown_connections 2024-05-17T21:48:34.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:34.158+0000 7f0bd20ea700 1 --2- 172.21.15.107:0/4083700613 >> [v2:172.21.15.107:6818/3962264726,v1:172.21.15.107:6819/3962264726] conn(0x7f0bb806ef70 0x7f0bb8071380 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:34.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:34.158+0000 7f0bd20ea700 1 --2- 172.21.15.107:0/4083700613 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f0bb80692a0 0x7f0bb806b750 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:34.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:34.158+0000 7f0bd20ea700 1 --2- 172.21.15.107:0/4083700613 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0bcc0fb5a0 0x7f0bcc113a40 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:34.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:34.158+0000 7f0bd20ea700 1 --2- 172.21.15.107:0/4083700613 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0bcc0fbf40 0x7f0bcc113f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:34.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:34.158+0000 7f0bd20ea700 1 -- 172.21.15.107:0/4083700613 >> 172.21.15.107:0/4083700613 conn(0x7f0bcc073350 msgr2=0x7f0bcc100a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:48:34.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:34.159+0000 7f0bd20ea700 1 -- 172.21.15.107:0/4083700613 shutdown_connections 2024-05-17T21:48:34.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:34.159+0000 7f0bd20ea700 1 -- 172.21.15.107:0/4083700613 wait complete. 2024-05-17T21:48:34.886 INFO:teuthology.orchestra.run.smithi107.stdout:111669149716 2024-05-17T21:48:34.887 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-05-17T21:48:36.042 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:48:35 smithi107 ceph-mon[28434]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-17T21:48:36.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:48:35 smithi138 ceph-mon[35948]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-17T21:48:37.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:37.163+0000 7f773e5eb700 1 -- 172.21.15.107:0/1030186147 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7738103780 msgr2=0x7f77380fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:48:37.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:37.163+0000 7f773e5eb700 1 --2- 172.21.15.107:0/1030186147 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7738103780 0x7f77380fcdf0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7720004940 tx=0x7f7720009e30 comp rx=0 tx=0).stop 2024-05-17T21:48:37.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:37.163+0000 7f773e5eb700 1 -- 172.21.15.107:0/1030186147 shutdown_connections 2024-05-17T21:48:37.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:37.163+0000 7f773e5eb700 1 --2- 172.21.15.107:0/1030186147 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7738103780 0x7f77380fcdf0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:37.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:37.163+0000 7f773e5eb700 1 --2- 172.21.15.107:0/1030186147 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7738102de0 0x7f77381031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:37.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:37.163+0000 7f773e5eb700 1 -- 172.21.15.107:0/1030186147 >> 172.21.15.107:0/1030186147 conn(0x7f77380f89b0 msgr2=0x7f77380fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:48:37.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:37.163+0000 7f773e5eb700 1 -- 172.21.15.107:0/1030186147 shutdown_connections 2024-05-17T21:48:37.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:37.163+0000 7f773e5eb700 1 -- 172.21.15.107:0/1030186147 wait complete. 2024-05-17T21:48:37.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:37.164+0000 7f773e5eb700 1 Processor -- start 2024-05-17T21:48:37.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:37.164+0000 7f773e5eb700 1 -- start start 2024-05-17T21:48:37.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:37.164+0000 7f773e5eb700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7738102de0 0x7f7738113c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:48:37.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:37.164+0000 7f773e5eb700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7738103780 0x7f77381141a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:48:37.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:37.164+0000 7f773e5eb700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7738114880 con 0x7f7738102de0 2024-05-17T21:48:37.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:37.164+0000 7f773e5eb700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f773810dbc0 con 0x7f7738103780 2024-05-17T21:48:37.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:37.164+0000 7f7737fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7738102de0 0x7f7738113c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:48:37.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:37.164+0000 7f7737fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7738102de0 0x7f7738113c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41594/0 (socket says 172.21.15.107:41594) 2024-05-17T21:48:37.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:37.164+0000 7f7737fff700 1 -- 172.21.15.107:0/1507373917 learned_addr learned my addr 172.21.15.107:0/1507373917 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:48:37.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:37.164+0000 7f772ffff700 1 --2- 172.21.15.107:0/1507373917 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7738103780 0x7f77381141a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:48:37.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:37.164+0000 7f7737fff700 1 -- 172.21.15.107:0/1507373917 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7738103780 msgr2=0x7f77381141a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:48:37.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:37.164+0000 7f7737fff700 1 --2- 172.21.15.107:0/1507373917 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7738103780 0x7f77381141a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:37.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:37.164+0000 7f7737fff700 1 -- 172.21.15.107:0/1507373917 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7720005040 con 0x7f7738102de0 2024-05-17T21:48:37.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:37.165+0000 7f7737fff700 1 --2- 172.21.15.107:0/1507373917 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7738102de0 0x7f7738113c60 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f772800b790 tx=0x7f7728009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:48:37.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:37.165+0000 7f77357fa700 1 -- 172.21.15.107:0/1507373917 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f772800f6f0 con 0x7f7738102de0 2024-05-17T21:48:37.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:37.165+0000 7f77357fa700 1 -- 172.21.15.107:0/1507373917 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f772800fd20 con 0x7f7738102de0 2024-05-17T21:48:37.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:37.165+0000 7f773e5eb700 1 -- 172.21.15.107:0/1507373917 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f773810de40 con 0x7f7738102de0 2024-05-17T21:48:37.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:37.165+0000 7f77357fa700 1 -- 172.21.15.107:0/1507373917 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f77280103a0 con 0x7f7738102de0 2024-05-17T21:48:37.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:37.165+0000 7f773e5eb700 1 -- 172.21.15.107:0/1507373917 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f773810e350 con 0x7f7738102de0 2024-05-17T21:48:37.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:37.166+0000 7f773e5eb700 1 -- 172.21.15.107:0/1507373917 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77381000f0 con 0x7f7738102de0 2024-05-17T21:48:37.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:37.166+0000 7f77357fa700 1 -- 172.21.15.107:0/1507373917 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f772800f850 con 0x7f7738102de0 2024-05-17T21:48:37.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:37.167+0000 7f77357fa700 1 --2- 172.21.15.107:0/1507373917 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f77240692a0 0x7f772406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:48:37.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:37.167+0000 7f77357fa700 1 -- 172.21.15.107:0/1507373917 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f7728086b80 con 0x7f7738102de0 2024-05-17T21:48:37.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:37.167+0000 7f772ffff700 1 --2- 172.21.15.107:0/1507373917 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f77240692a0 0x7f772406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:48:37.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:37.167+0000 7f772ffff700 1 --2- 172.21.15.107:0/1507373917 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f77240692a0 0x7f772406b750 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7720004890 tx=0x7f7720009e30 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:48:37.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:37.171+0000 7f77357fa700 1 -- 172.21.15.107:0/1507373917 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f7728056d20 con 0x7f7738102de0 2024-05-17T21:48:37.355 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:37.362+0000 7f773e5eb700 1 -- 172.21.15.107:0/1507373917 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f773804bed0 con 0x7f7738102de0 2024-05-17T21:48:37.355 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:37.362+0000 7f77357fa700 1 -- 172.21.15.107:0/1507373917 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f77280568b0 con 0x7f7738102de0 2024-05-17T21:48:37.355 INFO:teuthology.orchestra.run.smithi107.stdout:167503724555 2024-05-17T21:48:37.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:37.364+0000 7f773e5eb700 1 -- 172.21.15.107:0/1507373917 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f77240692a0 msgr2=0x7f772406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:48:37.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:37.364+0000 7f773e5eb700 1 --2- 172.21.15.107:0/1507373917 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f77240692a0 0x7f772406b750 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7720004890 tx=0x7f7720009e30 comp rx=0 tx=0).stop 2024-05-17T21:48:37.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:37.364+0000 7f773e5eb700 1 -- 172.21.15.107:0/1507373917 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7738102de0 msgr2=0x7f7738113c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:48:37.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:37.364+0000 7f773e5eb700 1 --2- 172.21.15.107:0/1507373917 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7738102de0 0x7f7738113c60 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f772800b790 tx=0x7f7728009250 comp rx=0 tx=0).stop 2024-05-17T21:48:37.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:37.364+0000 7f773e5eb700 1 -- 172.21.15.107:0/1507373917 shutdown_connections 2024-05-17T21:48:37.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:37.364+0000 7f773e5eb700 1 --2- 172.21.15.107:0/1507373917 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f77240692a0 0x7f772406b750 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:37.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:37.364+0000 7f773e5eb700 1 --2- 172.21.15.107:0/1507373917 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7738102de0 0x7f7738113c60 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:37.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:37.364+0000 7f773e5eb700 1 --2- 172.21.15.107:0/1507373917 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7738103780 0x7f77381141a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:37.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:37.364+0000 7f773e5eb700 1 -- 172.21.15.107:0/1507373917 >> 172.21.15.107:0/1507373917 conn(0x7f77380f89b0 msgr2=0x7f7738107380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:48:37.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:37.364+0000 7f773e5eb700 1 -- 172.21.15.107:0/1507373917 shutdown_connections 2024-05-17T21:48:37.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:37.364+0000 7f773e5eb700 1 -- 172.21.15.107:0/1507373917 wait complete. 2024-05-17T21:48:37.883 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:48:37 smithi107 ceph-mon[28434]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-17T21:48:37.883 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:48:37 smithi107 ceph-mon[28434]: from='client.? 172.21.15.107:0/1507373917' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-17T21:48:38.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:48:37 smithi138 ceph-mon[35948]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-17T21:48:38.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:48:37 smithi138 ceph-mon[35948]: from='client.? 172.21.15.107:0/1507373917' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-17T21:48:38.499 INFO:teuthology.orchestra.run.smithi107.stdout:81604378649 2024-05-17T21:48:38.500 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-05-17T21:48:39.431 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724553 got 167503724555 for osd.5 2024-05-17T21:48:39.431 DEBUG:teuthology.parallel:result is None 2024-05-17T21:48:39.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:39.614+0000 7fc487920700 1 -- 172.21.15.107:0/2397844425 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc480103780 msgr2=0x7fc4800fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:48:39.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:39.614+0000 7fc487920700 1 --2- 172.21.15.107:0/2397844425 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc480103780 0x7fc4800fcdf0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc47c004940 tx=0x7fc47c009e30 comp rx=0 tx=0).stop 2024-05-17T21:48:39.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:39.614+0000 7fc487920700 1 -- 172.21.15.107:0/2397844425 shutdown_connections 2024-05-17T21:48:39.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:39.614+0000 7fc487920700 1 --2- 172.21.15.107:0/2397844425 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc480103780 0x7fc4800fcdf0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:39.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:39.614+0000 7fc487920700 1 --2- 172.21.15.107:0/2397844425 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc480102de0 0x7fc4801031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:39.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:39.614+0000 7fc487920700 1 -- 172.21.15.107:0/2397844425 >> 172.21.15.107:0/2397844425 conn(0x7fc4800f8990 msgr2=0x7fc4800fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:48:39.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:39.614+0000 7fc487920700 1 -- 172.21.15.107:0/2397844425 shutdown_connections 2024-05-17T21:48:39.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:39.614+0000 7fc487920700 1 -- 172.21.15.107:0/2397844425 wait complete. 2024-05-17T21:48:39.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:39.614+0000 7fc487920700 1 Processor -- start 2024-05-17T21:48:39.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:39.614+0000 7fc487920700 1 -- start start 2024-05-17T21:48:39.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:39.615+0000 7fc487920700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc480102de0 0x7fc480115b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:48:39.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:39.615+0000 7fc487920700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc480103780 0x7fc480116070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:48:39.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:39.615+0000 7fc487920700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4801166c0 con 0x7fc480102de0 2024-05-17T21:48:39.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:39.615+0000 7fc487920700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc48010fce0 con 0x7fc480103780 2024-05-17T21:48:39.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:39.615+0000 7fc4856bc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc480102de0 0x7fc480115b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:48:39.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:39.615+0000 7fc4856bc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc480102de0 0x7fc480115b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41614/0 (socket says 172.21.15.107:41614) 2024-05-17T21:48:39.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:39.615+0000 7fc4856bc700 1 -- 172.21.15.107:0/2117176048 learned_addr learned my addr 172.21.15.107:0/2117176048 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:48:39.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:39.615+0000 7fc484ebb700 1 --2- 172.21.15.107:0/2117176048 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc480103780 0x7fc480116070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:48:39.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:39.615+0000 7fc4856bc700 1 -- 172.21.15.107:0/2117176048 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc480103780 msgr2=0x7fc480116070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:48:39.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:39.615+0000 7fc4856bc700 1 --2- 172.21.15.107:0/2117176048 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc480103780 0x7fc480116070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:39.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:39.615+0000 7fc4856bc700 1 -- 172.21.15.107:0/2117176048 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc47c005040 con 0x7fc480102de0 2024-05-17T21:48:39.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:39.615+0000 7fc4856bc700 1 --2- 172.21.15.107:0/2117176048 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc480102de0 0x7fc480115b30 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc47000c830 tx=0x7fc470004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:48:39.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:39.615+0000 7fc475ffb700 1 -- 172.21.15.107:0/2117176048 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc4700106f0 con 0x7fc480102de0 2024-05-17T21:48:39.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:39.615+0000 7fc475ffb700 1 -- 172.21.15.107:0/2117176048 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc470010d20 con 0x7fc480102de0 2024-05-17T21:48:39.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:39.615+0000 7fc487920700 1 -- 172.21.15.107:0/2117176048 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc48010ffc0 con 0x7fc480102de0 2024-05-17T21:48:39.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:39.615+0000 7fc475ffb700 1 -- 172.21.15.107:0/2117176048 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc4700113a0 con 0x7fc480102de0 2024-05-17T21:48:39.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:39.616+0000 7fc487920700 1 -- 172.21.15.107:0/2117176048 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc480110550 con 0x7fc480102de0 2024-05-17T21:48:39.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:39.616+0000 7fc487920700 1 -- 172.21.15.107:0/2117176048 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4800fe530 con 0x7fc480102de0 2024-05-17T21:48:39.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:39.617+0000 7fc475ffb700 1 -- 172.21.15.107:0/2117176048 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7fc470012400 con 0x7fc480102de0 2024-05-17T21:48:39.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:39.617+0000 7fc475ffb700 1 --2- 172.21.15.107:0/2117176048 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fc46c0692a0 0x7fc46c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:48:39.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:39.618+0000 7fc475ffb700 1 -- 172.21.15.107:0/2117176048 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fc470086d30 con 0x7fc480102de0 2024-05-17T21:48:39.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:39.618+0000 7fc484ebb700 1 --2- 172.21.15.107:0/2117176048 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fc46c0692a0 0x7fc46c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:48:39.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:39.618+0000 7fc484ebb700 1 --2- 172.21.15.107:0/2117176048 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fc46c0692a0 0x7fc46c06b750 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc47c004890 tx=0x7fc47c009e30 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:48:39.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:39.622+0000 7fc475ffb700 1 -- 172.21.15.107:0/2117176048 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fc470053440 con 0x7fc480102de0 2024-05-17T21:48:39.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:39.808+0000 7f7ba89c8700 1 -- 172.21.15.107:0/2212013962 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ba4102680 msgr2=0x7f7ba40fbcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:48:39.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:39.808+0000 7f7ba89c8700 1 --2- 172.21.15.107:0/2212013962 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ba4102680 0x7f7ba40fbcf0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7b98004940 tx=0x7f7b98009e30 comp rx=0 tx=0).stop 2024-05-17T21:48:39.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:39.808+0000 7f7ba89c8700 1 -- 172.21.15.107:0/2212013962 shutdown_connections 2024-05-17T21:48:39.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:39.808+0000 7f7ba89c8700 1 --2- 172.21.15.107:0/2212013962 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ba4102680 0x7f7ba40fbcf0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:39.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:39.808+0000 7f7ba89c8700 1 --2- 172.21.15.107:0/2212013962 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ba4101ce0 0x7f7ba41020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:39.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:39.808+0000 7f7ba89c8700 1 -- 172.21.15.107:0/2212013962 >> 172.21.15.107:0/2212013962 conn(0x7f7ba40f7850 msgr2=0x7f7ba40f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:48:39.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:39.808+0000 7f7ba89c8700 1 -- 172.21.15.107:0/2212013962 shutdown_connections 2024-05-17T21:48:39.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:39.808+0000 7f7ba89c8700 1 -- 172.21.15.107:0/2212013962 wait complete. 2024-05-17T21:48:39.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:39.808+0000 7f7ba89c8700 1 Processor -- start 2024-05-17T21:48:39.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:39.809+0000 7f7ba89c8700 1 -- start start 2024-05-17T21:48:39.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:39.809+0000 7f7ba89c8700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ba4101ce0 0x7f7ba4112a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:48:39.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:39.809+0000 7f7ba89c8700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ba4102680 0x7f7ba4112fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:48:39.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:39.809+0000 7f7ba89c8700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba4113600 con 0x7f7ba4101ce0 2024-05-17T21:48:39.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:39.809+0000 7f7ba89c8700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba410c9f0 con 0x7f7ba4102680 2024-05-17T21:48:39.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:39.809+0000 7f7ba259c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ba4101ce0 0x7f7ba4112a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:48:39.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:39.809+0000 7f7ba259c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ba4101ce0 0x7f7ba4112a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41638/0 (socket says 172.21.15.107:41638) 2024-05-17T21:48:39.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:39.809+0000 7f7ba259c700 1 -- 172.21.15.107:0/4178383302 learned_addr learned my addr 172.21.15.107:0/4178383302 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:48:39.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:39.809+0000 7f7ba1d9b700 1 --2- 172.21.15.107:0/4178383302 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ba4102680 0x7f7ba4112fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:48:39.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:39.809+0000 7f7ba259c700 1 -- 172.21.15.107:0/4178383302 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ba4102680 msgr2=0x7f7ba4112fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:48:39.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:39.809+0000 7f7ba259c700 1 --2- 172.21.15.107:0/4178383302 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ba4102680 0x7f7ba4112fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:39.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:39.809+0000 7f7ba259c700 1 -- 172.21.15.107:0/4178383302 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b98005040 con 0x7f7ba4101ce0 2024-05-17T21:48:39.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:39.809+0000 7f7ba1d9b700 1 --2- 172.21.15.107:0/4178383302 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ba4102680 0x7f7ba4112fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T21:48:39.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:39.809+0000 7f7ba259c700 1 --2- 172.21.15.107:0/4178383302 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ba4101ce0 0x7f7ba4112a70 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7b94002a50 tx=0x7f7b940046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:48:39.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:39.810+0000 7f7b92ffd700 1 -- 172.21.15.107:0/4178383302 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7b94007cc0 con 0x7f7ba4101ce0 2024-05-17T21:48:39.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:39.810+0000 7f7ba89c8700 1 -- 172.21.15.107:0/4178383302 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ba410ccd0 con 0x7f7ba4101ce0 2024-05-17T21:48:39.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:39.810+0000 7f7b92ffd700 1 -- 172.21.15.107:0/4178383302 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7b9400f040 con 0x7f7ba4101ce0 2024-05-17T21:48:39.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:39.810+0000 7f7b92ffd700 1 -- 172.21.15.107:0/4178383302 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7b9400db70 con 0x7f7ba4101ce0 2024-05-17T21:48:39.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:39.810+0000 7f7ba89c8700 1 -- 172.21.15.107:0/4178383302 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ba410d260 con 0x7f7ba4101ce0 2024-05-17T21:48:39.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:39.811+0000 7f7ba89c8700 1 -- 172.21.15.107:0/4178383302 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ba40fd430 con 0x7f7ba4101ce0 2024-05-17T21:48:39.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:39.811+0000 7f7b92ffd700 1 -- 172.21.15.107:0/4178383302 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f7b9401c400 con 0x7f7ba4101ce0 2024-05-17T21:48:39.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:39.812+0000 7f7b92ffd700 1 --2- 172.21.15.107:0/4178383302 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f7b8c0692a0 0x7f7b8c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:48:39.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:39.812+0000 7f7b92ffd700 1 -- 172.21.15.107:0/4178383302 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f7b940865e0 con 0x7f7ba4101ce0 2024-05-17T21:48:39.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:39.812+0000 7f7ba1d9b700 1 --2- 172.21.15.107:0/4178383302 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f7b8c0692a0 0x7f7b8c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:48:39.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:39.812+0000 7f7ba1d9b700 1 --2- 172.21.15.107:0/4178383302 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f7b8c0692a0 0x7f7b8c06b750 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7ba410dcd0 tx=0x7f7b98009e30 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:48:39.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:39.820+0000 7f7b92ffd700 1 -- 172.21.15.107:0/4178383302 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f7b94056660 con 0x7f7ba4101ce0 2024-05-17T21:48:39.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:39.821+0000 7fc487920700 1 -- 172.21.15.107:0/2117176048 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fc48004c710 con 0x7fc480102de0 2024-05-17T21:48:39.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:39.821+0000 7fc475ffb700 1 -- 172.21.15.107:0/2117176048 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc470056a60 con 0x7fc480102de0 2024-05-17T21:48:39.814 INFO:teuthology.orchestra.run.smithi107.stdout:146028888080 2024-05-17T21:48:39.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:39.823+0000 7fc487920700 1 -- 172.21.15.107:0/2117176048 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fc46c0692a0 msgr2=0x7fc46c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:48:39.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:39.823+0000 7fc487920700 1 --2- 172.21.15.107:0/2117176048 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fc46c0692a0 0x7fc46c06b750 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc47c004890 tx=0x7fc47c009e30 comp rx=0 tx=0).stop 2024-05-17T21:48:39.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:39.823+0000 7fc487920700 1 -- 172.21.15.107:0/2117176048 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc480102de0 msgr2=0x7fc480115b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:48:39.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:39.823+0000 7fc487920700 1 --2- 172.21.15.107:0/2117176048 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc480102de0 0x7fc480115b30 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc47000c830 tx=0x7fc470004470 comp rx=0 tx=0).stop 2024-05-17T21:48:39.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:39.823+0000 7fc487920700 1 -- 172.21.15.107:0/2117176048 shutdown_connections 2024-05-17T21:48:39.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:39.823+0000 7fc487920700 1 --2- 172.21.15.107:0/2117176048 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fc46c0692a0 0x7fc46c06b750 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:39.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:39.823+0000 7fc487920700 1 --2- 172.21.15.107:0/2117176048 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc480102de0 0x7fc480115b30 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:39.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:39.823+0000 7fc487920700 1 --2- 172.21.15.107:0/2117176048 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc480103780 0x7fc480116070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:39.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:39.823+0000 7fc487920700 1 -- 172.21.15.107:0/2117176048 >> 172.21.15.107:0/2117176048 conn(0x7fc4800f8990 msgr2=0x7fc480071a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:48:39.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:39.823+0000 7fc487920700 1 -- 172.21.15.107:0/2117176048 shutdown_connections 2024-05-17T21:48:39.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:39.823+0000 7fc487920700 1 -- 172.21.15.107:0/2117176048 wait complete. 2024-05-17T21:48:40.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:40.011+0000 7f7ba89c8700 1 -- 172.21.15.107:0/4178383302 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f7ba4108de0 con 0x7f7ba4101ce0 2024-05-17T21:48:40.005 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:48:39 smithi107 ceph-mon[28434]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-17T21:48:40.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:40.011+0000 7f7b92ffd700 1 -- 172.21.15.107:0/4178383302 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7b940561f0 con 0x7f7ba4101ce0 2024-05-17T21:48:40.005 INFO:teuthology.orchestra.run.smithi107.stdout:60129542175 2024-05-17T21:48:40.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:40.014+0000 7f7ba89c8700 1 -- 172.21.15.107:0/4178383302 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f7b8c0692a0 msgr2=0x7f7b8c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:48:40.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:40.014+0000 7f7ba89c8700 1 --2- 172.21.15.107:0/4178383302 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f7b8c0692a0 0x7f7b8c06b750 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7ba410dcd0 tx=0x7f7b98009e30 comp rx=0 tx=0).stop 2024-05-17T21:48:40.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:40.014+0000 7f7ba89c8700 1 -- 172.21.15.107:0/4178383302 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ba4101ce0 msgr2=0x7f7ba4112a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:48:40.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:40.014+0000 7f7ba89c8700 1 --2- 172.21.15.107:0/4178383302 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ba4101ce0 0x7f7ba4112a70 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7b94002a50 tx=0x7f7b940046b0 comp rx=0 tx=0).stop 2024-05-17T21:48:40.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:40.014+0000 7f7ba89c8700 1 -- 172.21.15.107:0/4178383302 shutdown_connections 2024-05-17T21:48:40.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:40.014+0000 7f7ba89c8700 1 --2- 172.21.15.107:0/4178383302 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f7b8c0692a0 0x7f7b8c06b750 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:40.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:40.014+0000 7f7ba89c8700 1 --2- 172.21.15.107:0/4178383302 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ba4101ce0 0x7f7ba4112a70 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:40.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:40.014+0000 7f7ba89c8700 1 --2- 172.21.15.107:0/4178383302 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ba4102680 0x7f7ba4112fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:40.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:40.014+0000 7f7ba89c8700 1 -- 172.21.15.107:0/4178383302 >> 172.21.15.107:0/4178383302 conn(0x7f7ba40f7850 msgr2=0x7f7ba40f8880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:48:40.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:40.014+0000 7f7ba89c8700 1 -- 172.21.15.107:0/4178383302 shutdown_connections 2024-05-17T21:48:40.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:40.014+0000 7f7ba89c8700 1 -- 172.21.15.107:0/4178383302 wait complete. 2024-05-17T21:48:40.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:48:39 smithi138 ceph-mon[35948]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-17T21:48:40.391 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/mon.smithi107/config 2024-05-17T21:48:40.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:40.413+0000 7f28ae7bf700 1 -- 172.21.15.107:0/3358078050 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28a81026a0 msgr2=0x7f28a80fbd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:48:40.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:40.413+0000 7f28ae7bf700 1 --2- 172.21.15.107:0/3358078050 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28a81026a0 0x7f28a80fbd10 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f289c004940 tx=0x7f289c009e30 comp rx=0 tx=0).stop 2024-05-17T21:48:40.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:40.413+0000 7f28ae7bf700 1 -- 172.21.15.107:0/3358078050 shutdown_connections 2024-05-17T21:48:40.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:40.413+0000 7f28ae7bf700 1 --2- 172.21.15.107:0/3358078050 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28a81026a0 0x7f28a80fbd10 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:40.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:40.413+0000 7f28ae7bf700 1 --2- 172.21.15.107:0/3358078050 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f28a8101d00 0x7f28a81020d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:40.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:40.413+0000 7f28ae7bf700 1 -- 172.21.15.107:0/3358078050 >> 172.21.15.107:0/3358078050 conn(0x7f28a80f7870 msgr2=0x7f28a80f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:48:40.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:40.413+0000 7f28ae7bf700 1 -- 172.21.15.107:0/3358078050 shutdown_connections 2024-05-17T21:48:40.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:40.413+0000 7f28ae7bf700 1 -- 172.21.15.107:0/3358078050 wait complete. 2024-05-17T21:48:40.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:40.414+0000 7f28ae7bf700 1 Processor -- start 2024-05-17T21:48:40.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:40.414+0000 7f28ae7bf700 1 -- start start 2024-05-17T21:48:40.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:40.414+0000 7f28ae7bf700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28a8101d00 0x7f28a81138f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:48:40.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:40.414+0000 7f28ae7bf700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f28a81026a0 0x7f28a8113e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:48:40.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:40.414+0000 7f28ae7bf700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28a8114480 con 0x7f28a8101d00 2024-05-17T21:48:40.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:40.414+0000 7f28ae7bf700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28a810da80 con 0x7f28a81026a0 2024-05-17T21:48:40.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:40.414+0000 7f28a7fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28a8101d00 0x7f28a81138f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:48:40.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:40.414+0000 7f28a7fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28a8101d00 0x7f28a81138f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41646/0 (socket says 172.21.15.107:41646) 2024-05-17T21:48:40.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:40.414+0000 7f28a7fff700 1 -- 172.21.15.107:0/1613249070 learned_addr learned my addr 172.21.15.107:0/1613249070 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:48:40.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:40.414+0000 7f28a77fe700 1 --2- 172.21.15.107:0/1613249070 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f28a81026a0 0x7f28a8113e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:48:40.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:40.415+0000 7f28a7fff700 1 -- 172.21.15.107:0/1613249070 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f28a81026a0 msgr2=0x7f28a8113e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:48:40.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:40.415+0000 7f28a7fff700 1 --2- 172.21.15.107:0/1613249070 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f28a81026a0 0x7f28a8113e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:40.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:40.415+0000 7f28a7fff700 1 -- 172.21.15.107:0/1613249070 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f289c005040 con 0x7f28a8101d00 2024-05-17T21:48:40.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:40.415+0000 7f28a77fe700 1 --2- 172.21.15.107:0/1613249070 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f28a81026a0 0x7f28a8113e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T21:48:40.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:40.415+0000 7f28a7fff700 1 --2- 172.21.15.107:0/1613249070 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28a8101d00 0x7f28a81138f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f289800eb40 tx=0x7f289800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:48:40.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:40.415+0000 7f28a4ff9700 1 -- 172.21.15.107:0/1613249070 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2898007cc0 con 0x7f28a8101d00 2024-05-17T21:48:40.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:40.415+0000 7f28a4ff9700 1 -- 172.21.15.107:0/1613249070 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f289800f040 con 0x7f28a8101d00 2024-05-17T21:48:40.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:40.415+0000 7f28ae7bf700 1 -- 172.21.15.107:0/1613249070 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28a810dd00 con 0x7f28a8101d00 2024-05-17T21:48:40.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:40.415+0000 7f28a4ff9700 1 -- 172.21.15.107:0/1613249070 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f289800ca00 con 0x7f28a8101d00 2024-05-17T21:48:40.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:40.415+0000 7f28ae7bf700 1 -- 172.21.15.107:0/1613249070 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28a810e210 con 0x7f28a8101d00 2024-05-17T21:48:40.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:40.416+0000 7f28ae7bf700 1 -- 172.21.15.107:0/1613249070 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f28a810e7d0 con 0x7f28a8101d00 2024-05-17T21:48:40.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:40.417+0000 7f28a4ff9700 1 -- 172.21.15.107:0/1613249070 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f289800cb60 con 0x7f28a8101d00 2024-05-17T21:48:40.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:40.417+0000 7f28a4ff9700 1 --2- 172.21.15.107:0/1613249070 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f28900692a0 0x7f289006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:48:40.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:40.417+0000 7f28a4ff9700 1 -- 172.21.15.107:0/1613249070 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f2898086ae0 con 0x7f28a8101d00 2024-05-17T21:48:40.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:40.417+0000 7f28a77fe700 1 --2- 172.21.15.107:0/1613249070 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f28900692a0 0x7f289006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:48:40.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:40.418+0000 7f28a4ff9700 1 --2- 172.21.15.107:0/1613249070 >> [v2:172.21.15.107:6802/710062679,v1:172.21.15.107:6803/710062679] conn(0x7f289006ef70 0x7f2890071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:48:40.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:40.418+0000 7f28a4ff9700 1 -- 172.21.15.107:0/1613249070 --> [v2:172.21.15.107:6802/710062679,v1:172.21.15.107:6803/710062679] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2890071a30 con 0x7f289006ef70 2024-05-17T21:48:40.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:40.418+0000 7f28a4ff9700 1 -- 172.21.15.107:0/1613249070 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2898086ec0 con 0x7f28a8101d00 2024-05-17T21:48:40.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:40.418+0000 7f28acd5c700 1 --2- 172.21.15.107:0/1613249070 >> [v2:172.21.15.107:6802/710062679,v1:172.21.15.107:6803/710062679] conn(0x7f289006ef70 0x7f2890071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:48:40.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:40.418+0000 7f28a77fe700 1 --2- 172.21.15.107:0/1613249070 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f28900692a0 0x7f289006b750 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f289c004890 tx=0x7f289c009e30 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:48:40.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:40.418+0000 7f28acd5c700 1 --2- 172.21.15.107:0/1613249070 >> [v2:172.21.15.107:6802/710062679,v1:172.21.15.107:6803/710062679] conn(0x7f289006ef70 0x7f2890071380 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:48:40.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:40.419+0000 7f28a4ff9700 1 -- 172.21.15.107:0/1613249070 <== osd.0 v2:172.21.15.107:6802/710062679 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f2890071a30 con 0x7f289006ef70 2024-05-17T21:48:40.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:40.436+0000 7f28ae7bf700 1 -- 172.21.15.107:0/1613249070 --> [v2:172.21.15.107:6802/710062679,v1:172.21.15.107:6803/710062679] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f28a810de90 con 0x7f289006ef70 2024-05-17T21:48:40.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:40.436+0000 7f28a4ff9700 1 -- 172.21.15.107:0/1613249070 <== osd.0 v2:172.21.15.107:6802/710062679 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f28a810de90 con 0x7f289006ef70 2024-05-17T21:48:40.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:40.437+0000 7f28ae7bf700 1 -- 172.21.15.107:0/1613249070 >> [v2:172.21.15.107:6802/710062679,v1:172.21.15.107:6803/710062679] conn(0x7f289006ef70 msgr2=0x7f2890071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:48:40.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:40.437+0000 7f28ae7bf700 1 --2- 172.21.15.107:0/1613249070 >> [v2:172.21.15.107:6802/710062679,v1:172.21.15.107:6803/710062679] conn(0x7f289006ef70 0x7f2890071380 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:40.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:40.437+0000 7f28ae7bf700 1 -- 172.21.15.107:0/1613249070 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f28900692a0 msgr2=0x7f289006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:48:40.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:40.437+0000 7f28ae7bf700 1 --2- 172.21.15.107:0/1613249070 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f28900692a0 0x7f289006b750 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f289c004890 tx=0x7f289c009e30 comp rx=0 tx=0).stop 2024-05-17T21:48:40.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:40.437+0000 7f28ae7bf700 1 -- 172.21.15.107:0/1613249070 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28a8101d00 msgr2=0x7f28a81138f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:48:40.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:40.437+0000 7f28ae7bf700 1 --2- 172.21.15.107:0/1613249070 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28a8101d00 0x7f28a81138f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f289800eb40 tx=0x7f289800b040 comp rx=0 tx=0).stop 2024-05-17T21:48:40.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:40.437+0000 7f28ae7bf700 1 -- 172.21.15.107:0/1613249070 shutdown_connections 2024-05-17T21:48:40.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:40.437+0000 7f28ae7bf700 1 --2- 172.21.15.107:0/1613249070 >> [v2:172.21.15.107:6802/710062679,v1:172.21.15.107:6803/710062679] conn(0x7f289006ef70 0x7f2890071380 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:40.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:40.437+0000 7f28ae7bf700 1 --2- 172.21.15.107:0/1613249070 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f28900692a0 0x7f289006b750 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:40.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:40.437+0000 7f28ae7bf700 1 --2- 172.21.15.107:0/1613249070 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28a8101d00 0x7f28a81138f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:40.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:40.437+0000 7f28ae7bf700 1 --2- 172.21.15.107:0/1613249070 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f28a81026a0 0x7f28a8113e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:40.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:40.437+0000 7f28ae7bf700 1 -- 172.21.15.107:0/1613249070 >> 172.21.15.107:0/1613249070 conn(0x7f28a80f7870 msgr2=0x7f28a8066c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:48:40.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:40.437+0000 7f28ae7bf700 1 -- 172.21.15.107:0/1613249070 shutdown_connections 2024-05-17T21:48:40.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:40.437+0000 7f28ae7bf700 1 -- 172.21.15.107:0/1613249070 wait complete. 2024-05-17T21:48:41.070 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:48:40 smithi107 ceph-mon[28434]: from='client.? 172.21.15.107:0/2117176048' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-17T21:48:41.070 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:48:40 smithi107 ceph-mon[28434]: from='client.? 172.21.15.107:0/4178383302' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-17T21:48:41.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:48:40 smithi138 ceph-mon[35948]: from='client.? 172.21.15.107:0/2117176048' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-17T21:48:41.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:48:40 smithi138 ceph-mon[35948]: from='client.? 172.21.15.107:0/4178383302' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-17T21:48:41.609 INFO:teuthology.orchestra.run.smithi107.stdout:38654705701 2024-05-17T21:48:41.609 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-05-17T21:48:42.054 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:48:41 smithi107 ceph-mon[28434]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-17T21:48:42.076 INFO:tasks.cephadm.ceph_manager.ceph:need seq 60129542173 got 60129542175 for osd.1 2024-05-17T21:48:42.077 DEBUG:teuthology.parallel:result is None 2024-05-17T21:48:42.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:48:41 smithi138 ceph-mon[35948]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-17T21:48:42.582 INFO:tasks.cephadm.ceph_manager.ceph:need seq 146028888078 got 146028888080 for osd.4 2024-05-17T21:48:42.582 DEBUG:teuthology.parallel:result is None 2024-05-17T21:48:43.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:43.527+0000 7f242d37f700 1 -- 172.21.15.107:0/2074072291 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24280fde30 msgr2=0x7f24280fe200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:48:43.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:43.527+0000 7f242d37f700 1 --2- 172.21.15.107:0/2074072291 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24280fde30 0x7f24280fe200 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2418004940 tx=0x7f2418009e30 comp rx=0 tx=0).stop 2024-05-17T21:48:43.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:43.527+0000 7f242d37f700 1 -- 172.21.15.107:0/2074072291 shutdown_connections 2024-05-17T21:48:43.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:43.527+0000 7f242d37f700 1 --2- 172.21.15.107:0/2074072291 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f24280fe7d0 0x7f2428106e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:43.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:43.527+0000 7f242d37f700 1 --2- 172.21.15.107:0/2074072291 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24280fde30 0x7f24280fe200 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:43.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:43.527+0000 7f242d37f700 1 -- 172.21.15.107:0/2074072291 >> 172.21.15.107:0/2074072291 conn(0x7f24280f7860 msgr2=0x7f24280f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:48:43.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:43.527+0000 7f242d37f700 1 -- 172.21.15.107:0/2074072291 shutdown_connections 2024-05-17T21:48:43.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:43.527+0000 7f242d37f700 1 -- 172.21.15.107:0/2074072291 wait complete. 2024-05-17T21:48:43.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:43.528+0000 7f242d37f700 1 Processor -- start 2024-05-17T21:48:43.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:43.528+0000 7f242d37f700 1 -- start start 2024-05-17T21:48:43.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:43.528+0000 7f242d37f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24280fde30 0x7f24280fbc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:48:43.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:43.528+0000 7f242d37f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f24280fe7d0 0x7f24280fc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:48:43.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:43.528+0000 7f242d37f700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24280fdbb0 con 0x7f24280fde30 2024-05-17T21:48:43.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:43.528+0000 7f242d37f700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24280fc6e0 con 0x7f24280fe7d0 2024-05-17T21:48:43.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:43.528+0000 7f2426ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24280fde30 0x7f24280fbc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:48:43.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:43.528+0000 7f2426ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24280fde30 0x7f24280fbc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41658/0 (socket says 172.21.15.107:41658) 2024-05-17T21:48:43.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:43.528+0000 7f2426ffd700 1 -- 172.21.15.107:0/2401683885 learned_addr learned my addr 172.21.15.107:0/2401683885 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:48:43.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:43.529+0000 7f24267fc700 1 --2- 172.21.15.107:0/2401683885 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f24280fe7d0 0x7f24280fc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:48:43.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:43.529+0000 7f2426ffd700 1 -- 172.21.15.107:0/2401683885 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f24280fe7d0 msgr2=0x7f24280fc1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:48:43.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:43.529+0000 7f2426ffd700 1 --2- 172.21.15.107:0/2401683885 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f24280fe7d0 0x7f24280fc1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:43.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:43.529+0000 7f2426ffd700 1 -- 172.21.15.107:0/2401683885 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2418005040 con 0x7f24280fde30 2024-05-17T21:48:43.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:43.529+0000 7f2426ffd700 1 --2- 172.21.15.107:0/2401683885 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24280fde30 0x7f24280fbc60 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2418009380 tx=0x7f2418009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:48:43.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:43.529+0000 7f240f7fe700 1 -- 172.21.15.107:0/2401683885 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f241800e040 con 0x7f24280fde30 2024-05-17T21:48:43.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:43.530+0000 7f240f7fe700 1 -- 172.21.15.107:0/2401683885 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2418013420 con 0x7f24280fde30 2024-05-17T21:48:43.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:43.530+0000 7f240f7fe700 1 -- 172.21.15.107:0/2401683885 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f241801c610 con 0x7f24280fde30 2024-05-17T21:48:43.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:43.530+0000 7f242d37f700 1 -- 172.21.15.107:0/2401683885 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24280fc9c0 con 0x7f24280fde30 2024-05-17T21:48:43.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:43.530+0000 7f242d37f700 1 -- 172.21.15.107:0/2401683885 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2428119fb0 con 0x7f24280fde30 2024-05-17T21:48:43.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:43.531+0000 7f242d37f700 1 -- 172.21.15.107:0/2401683885 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2428066b10 con 0x7f24280fde30 2024-05-17T21:48:43.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:43.532+0000 7f240f7fe700 1 -- 172.21.15.107:0/2401683885 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f2418013a50 con 0x7f24280fde30 2024-05-17T21:48:43.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:43.532+0000 7f240f7fe700 1 --2- 172.21.15.107:0/2401683885 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f2410069360 0x7f241006b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:48:43.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:43.532+0000 7f240f7fe700 1 -- 172.21.15.107:0/2401683885 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f2418087e20 con 0x7f24280fde30 2024-05-17T21:48:43.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:43.532+0000 7f24267fc700 1 --2- 172.21.15.107:0/2401683885 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f2410069360 0x7f241006b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:48:43.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:43.533+0000 7f24267fc700 1 --2- 172.21.15.107:0/2401683885 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f2410069360 0x7f241006b810 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f24280fd390 tx=0x7f241c00b040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:48:43.529 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/mon.smithi107/config 2024-05-17T21:48:43.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:43.539+0000 7f240f7fe700 1 -- 172.21.15.107:0/2401683885 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f2418054530 con 0x7f24280fde30 2024-05-17T21:48:43.564 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/mon.smithi107/config 2024-05-17T21:48:43.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:43.750+0000 7f242d37f700 1 -- 172.21.15.107:0/2401683885 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f242804c710 con 0x7f24280fde30 2024-05-17T21:48:43.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:43.750+0000 7f240f7fe700 1 -- 172.21.15.107:0/2401683885 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2418057b50 con 0x7f24280fde30 2024-05-17T21:48:43.744 INFO:teuthology.orchestra.run.smithi107.stdout:111669149718 2024-05-17T21:48:43.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:43.752+0000 7f242d37f700 1 -- 172.21.15.107:0/2401683885 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f2410069360 msgr2=0x7f241006b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:48:43.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:43.752+0000 7f242d37f700 1 --2- 172.21.15.107:0/2401683885 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f2410069360 0x7f241006b810 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f24280fd390 tx=0x7f241c00b040 comp rx=0 tx=0).stop 2024-05-17T21:48:43.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:43.752+0000 7f242d37f700 1 -- 172.21.15.107:0/2401683885 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24280fde30 msgr2=0x7f24280fbc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:48:43.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:43.752+0000 7f242d37f700 1 --2- 172.21.15.107:0/2401683885 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24280fde30 0x7f24280fbc60 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2418009380 tx=0x7f2418009e30 comp rx=0 tx=0).stop 2024-05-17T21:48:43.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:43.753+0000 7f242d37f700 1 -- 172.21.15.107:0/2401683885 shutdown_connections 2024-05-17T21:48:43.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:43.753+0000 7f242d37f700 1 --2- 172.21.15.107:0/2401683885 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f2410069360 0x7f241006b810 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:43.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:43.753+0000 7f242d37f700 1 --2- 172.21.15.107:0/2401683885 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24280fde30 0x7f24280fbc60 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:43.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:43.753+0000 7f242d37f700 1 --2- 172.21.15.107:0/2401683885 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f24280fe7d0 0x7f24280fc1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:43.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:43.753+0000 7f242d37f700 1 -- 172.21.15.107:0/2401683885 >> 172.21.15.107:0/2401683885 conn(0x7f24280f7860 msgr2=0x7f24280f8dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:48:43.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:43.753+0000 7f242d37f700 1 -- 172.21.15.107:0/2401683885 shutdown_connections 2024-05-17T21:48:43.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:43.753+0000 7f242d37f700 1 -- 172.21.15.107:0/2401683885 wait complete. 2024-05-17T21:48:44.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:48:43 smithi138 ceph-mon[35948]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-17T21:48:44.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:48:43 smithi138 ceph-mon[35948]: from='client.? 172.21.15.107:0/2401683885' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-17T21:48:44.083 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:48:43 smithi107 ceph-mon[28434]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-17T21:48:44.083 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:48:43 smithi107 ceph-mon[28434]: from='client.? 172.21.15.107:0/2401683885' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-17T21:48:45.984 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:48:45 smithi107 ceph-mon[28434]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-17T21:48:46.018 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149716 got 111669149718 for osd.3 2024-05-17T21:48:46.018 DEBUG:teuthology.parallel:result is None 2024-05-17T21:48:46.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:48:45 smithi138 ceph-mon[35948]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-17T21:48:46.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:46.874+0000 7f8c73062700 1 -- 172.21.15.107:0/2855792509 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8c6c0fbd40 msgr2=0x7f8c6c0fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:48:46.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:46.874+0000 7f8c73062700 1 --2- 172.21.15.107:0/2855792509 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8c6c0fbd40 0x7f8c6c0fc1b0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c009ce0 tx=0x7f8c5c004d10 comp rx=0 tx=0).stop 2024-05-17T21:48:46.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:46.874+0000 7f8c73062700 1 -- 172.21.15.107:0/2855792509 shutdown_connections 2024-05-17T21:48:46.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:46.874+0000 7f8c73062700 1 --2- 172.21.15.107:0/2855792509 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8c6c0fbd40 0x7f8c6c0fc1b0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:46.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:46.874+0000 7f8c73062700 1 --2- 172.21.15.107:0/2855792509 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c6c101d90 0x7f8c6c102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:46.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:46.874+0000 7f8c73062700 1 -- 172.21.15.107:0/2855792509 >> 172.21.15.107:0/2855792509 conn(0x7f8c6c071d00 msgr2=0x7f8c6c072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:48:46.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:46.874+0000 7f8c73062700 1 -- 172.21.15.107:0/2855792509 shutdown_connections 2024-05-17T21:48:46.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:46.874+0000 7f8c73062700 1 -- 172.21.15.107:0/2855792509 wait complete. 2024-05-17T21:48:46.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:46.875+0000 7f8c73062700 1 Processor -- start 2024-05-17T21:48:46.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:46.875+0000 7f8c73062700 1 -- start start 2024-05-17T21:48:46.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:46.875+0000 7f8c73062700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8c6c0fbd40 0x7f8c6c10ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:48:46.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:46.875+0000 7f8c73062700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c6c101d90 0x7f8c6c10f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:48:46.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:46.875+0000 7f8c73062700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c6c10f8e0 con 0x7f8c6c0fbd40 2024-05-17T21:48:46.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:46.875+0000 7f8c73062700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c6c10fa20 con 0x7f8c6c101d90 2024-05-17T21:48:46.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:46.875+0000 7f8c70dfe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8c6c0fbd40 0x7f8c6c10ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:48:46.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:46.875+0000 7f8c70dfe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8c6c0fbd40 0x7f8c6c10ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40260/0 (socket says 172.21.15.107:40260) 2024-05-17T21:48:46.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:46.875+0000 7f8c70dfe700 1 -- 172.21.15.107:0/3903789121 learned_addr learned my addr 172.21.15.107:0/3903789121 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:48:46.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:46.876+0000 7f8c6bfff700 1 --2- 172.21.15.107:0/3903789121 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c6c101d90 0x7f8c6c10f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:48:46.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:46.876+0000 7f8c70dfe700 1 -- 172.21.15.107:0/3903789121 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c6c101d90 msgr2=0x7f8c6c10f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:48:46.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:46.876+0000 7f8c70dfe700 1 --2- 172.21.15.107:0/3903789121 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c6c101d90 0x7f8c6c10f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:46.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:46.876+0000 7f8c70dfe700 1 -- 172.21.15.107:0/3903789121 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c5c009940 con 0x7f8c6c0fbd40 2024-05-17T21:48:46.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:46.876+0000 7f8c6bfff700 1 --2- 172.21.15.107:0/3903789121 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c6c101d90 0x7f8c6c10f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:48:46.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:46.876+0000 7f8c70dfe700 1 --2- 172.21.15.107:0/3903789121 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8c6c0fbd40 0x7f8c6c10ee60 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8c54002a50 tx=0x7f8c540046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:48:46.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:46.876+0000 7f8c697fa700 1 -- 172.21.15.107:0/3903789121 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8c54007cc0 con 0x7f8c6c0fbd40 2024-05-17T21:48:46.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:46.876+0000 7f8c697fa700 1 -- 172.21.15.107:0/3903789121 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8c5400f040 con 0x7f8c6c0fbd40 2024-05-17T21:48:46.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:46.876+0000 7f8c73062700 1 -- 172.21.15.107:0/3903789121 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c6c117d10 con 0x7f8c6c0fbd40 2024-05-17T21:48:46.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:46.876+0000 7f8c697fa700 1 -- 172.21.15.107:0/3903789121 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8c54009640 con 0x7f8c6c0fbd40 2024-05-17T21:48:46.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:46.876+0000 7f8c73062700 1 -- 172.21.15.107:0/3903789121 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c6c118240 con 0x7f8c6c0fbd40 2024-05-17T21:48:46.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:46.877+0000 7f8c73062700 1 -- 172.21.15.107:0/3903789121 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c6c0fd430 con 0x7f8c6c0fbd40 2024-05-17T21:48:46.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:46.877+0000 7f8c697fa700 1 -- 172.21.15.107:0/3903789121 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f8c54009820 con 0x7f8c6c0fbd40 2024-05-17T21:48:46.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:46.878+0000 7f8c697fa700 1 --2- 172.21.15.107:0/3903789121 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f8c580692a0 0x7f8c5806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:48:46.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:46.878+0000 7f8c697fa700 1 -- 172.21.15.107:0/3903789121 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f8c54086c60 con 0x7f8c6c0fbd40 2024-05-17T21:48:46.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:46.878+0000 7f8c6bfff700 1 --2- 172.21.15.107:0/3903789121 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f8c580692a0 0x7f8c5806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:48:46.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:46.878+0000 7f8c6bfff700 1 --2- 172.21.15.107:0/3903789121 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f8c580692a0 0x7f8c5806b750 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c0048c0 tx=0x7f8c5c004d10 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:48:46.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:46.882+0000 7f8c697fa700 1 -- 172.21.15.107:0/3903789121 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f8c54053250 con 0x7f8c6c0fbd40 2024-05-17T21:48:47.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:47.079+0000 7f8c73062700 1 -- 172.21.15.107:0/3903789121 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f8c6c065db0 con 0x7f8c6c0fbd40 2024-05-17T21:48:47.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:47.080+0000 7f8c697fa700 1 -- 172.21.15.107:0/3903789121 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8c54056870 con 0x7f8c6c0fbd40 2024-05-17T21:48:47.073 INFO:teuthology.orchestra.run.smithi107.stdout:81604378651 2024-05-17T21:48:47.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:47.082+0000 7f8c73062700 1 -- 172.21.15.107:0/3903789121 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f8c580692a0 msgr2=0x7f8c5806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:48:47.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:47.082+0000 7f8c73062700 1 --2- 172.21.15.107:0/3903789121 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f8c580692a0 0x7f8c5806b750 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c0048c0 tx=0x7f8c5c004d10 comp rx=0 tx=0).stop 2024-05-17T21:48:47.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:47.082+0000 7f8c73062700 1 -- 172.21.15.107:0/3903789121 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8c6c0fbd40 msgr2=0x7f8c6c10ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:48:47.076 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:47.082+0000 7f8c73062700 1 --2- 172.21.15.107:0/3903789121 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8c6c0fbd40 0x7f8c6c10ee60 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8c54002a50 tx=0x7f8c540046b0 comp rx=0 tx=0).stop 2024-05-17T21:48:47.076 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:47.082+0000 7f8c73062700 1 -- 172.21.15.107:0/3903789121 shutdown_connections 2024-05-17T21:48:47.076 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:47.082+0000 7f8c73062700 1 --2- 172.21.15.107:0/3903789121 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f8c580692a0 0x7f8c5806b750 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:47.076 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:47.082+0000 7f8c73062700 1 --2- 172.21.15.107:0/3903789121 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8c6c0fbd40 0x7f8c6c10ee60 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:47.076 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:47.082+0000 7f8c73062700 1 --2- 172.21.15.107:0/3903789121 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c6c101d90 0x7f8c6c10f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:47.076 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:47.082+0000 7f8c73062700 1 -- 172.21.15.107:0/3903789121 >> 172.21.15.107:0/3903789121 conn(0x7f8c6c071d00 msgr2=0x7f8c6c0fa0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:48:47.076 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:47.082+0000 7f8c73062700 1 -- 172.21.15.107:0/3903789121 shutdown_connections 2024-05-17T21:48:47.076 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:47.082+0000 7f8c73062700 1 -- 172.21.15.107:0/3903789121 wait complete. 2024-05-17T21:48:47.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:47.548+0000 7f9cc5bf6700 1 -- 172.21.15.107:0/3542130767 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9cc0102690 msgr2=0x7f9cc00fbd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:48:47.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:47.548+0000 7f9cc5bf6700 1 --2- 172.21.15.107:0/3542130767 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9cc0102690 0x7f9cc00fbd00 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9cb4004940 tx=0x7f9cb4009e30 comp rx=0 tx=0).stop 2024-05-17T21:48:47.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:47.548+0000 7f9cc5bf6700 1 -- 172.21.15.107:0/3542130767 shutdown_connections 2024-05-17T21:48:47.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:47.548+0000 7f9cc5bf6700 1 --2- 172.21.15.107:0/3542130767 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9cc0102690 0x7f9cc00fbd00 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:47.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:47.549+0000 7f9cc5bf6700 1 --2- 172.21.15.107:0/3542130767 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9cc0101cf0 0x7f9cc01020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:47.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:47.549+0000 7f9cc5bf6700 1 -- 172.21.15.107:0/3542130767 >> 172.21.15.107:0/3542130767 conn(0x7f9cc00f7860 msgr2=0x7f9cc00f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:48:47.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:47.549+0000 7f9cc5bf6700 1 -- 172.21.15.107:0/3542130767 shutdown_connections 2024-05-17T21:48:47.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:47.549+0000 7f9cc5bf6700 1 -- 172.21.15.107:0/3542130767 wait complete. 2024-05-17T21:48:47.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:47.549+0000 7f9cc5bf6700 1 Processor -- start 2024-05-17T21:48:47.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:47.549+0000 7f9cc5bf6700 1 -- start start 2024-05-17T21:48:47.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:47.549+0000 7f9cc5bf6700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9cc0101cf0 0x7f9cc010eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:48:47.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:47.549+0000 7f9cc5bf6700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9cc0102690 0x7f9cc010f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:48:47.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:47.549+0000 7f9cc5bf6700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc010f560 con 0x7f9cc0102690 2024-05-17T21:48:47.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:47.549+0000 7f9cc5bf6700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc010f6d0 con 0x7f9cc0101cf0 2024-05-17T21:48:47.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:47.550+0000 7f9cbeffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9cc0102690 0x7f9cc010f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:48:47.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:47.550+0000 7f9cbeffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9cc0102690 0x7f9cc010f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40280/0 (socket says 172.21.15.107:40280) 2024-05-17T21:48:47.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:47.550+0000 7f9cbeffd700 1 -- 172.21.15.107:0/2505550592 learned_addr learned my addr 172.21.15.107:0/2505550592 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:48:47.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:47.550+0000 7f9cbf7fe700 1 --2- 172.21.15.107:0/2505550592 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9cc0101cf0 0x7f9cc010eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:48:47.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:47.550+0000 7f9cbeffd700 1 -- 172.21.15.107:0/2505550592 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9cc0101cf0 msgr2=0x7f9cc010eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:48:47.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:47.550+0000 7f9cbeffd700 1 --2- 172.21.15.107:0/2505550592 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9cc0101cf0 0x7f9cc010eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:47.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:47.550+0000 7f9cbeffd700 1 -- 172.21.15.107:0/2505550592 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cb4005040 con 0x7f9cc0102690 2024-05-17T21:48:47.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:47.550+0000 7f9cbeffd700 1 --2- 172.21.15.107:0/2505550592 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9cc0102690 0x7f9cc010f020 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9cb4009100 tx=0x7f9cb4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:48:47.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:47.550+0000 7f9ca7fff700 1 -- 172.21.15.107:0/2505550592 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9cb400e040 con 0x7f9cc0102690 2024-05-17T21:48:47.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:47.550+0000 7f9cc5bf6700 1 -- 172.21.15.107:0/2505550592 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cc010f950 con 0x7f9cc0102690 2024-05-17T21:48:47.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:47.550+0000 7f9cc5bf6700 1 -- 172.21.15.107:0/2505550592 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cc0117e20 con 0x7f9cc0102690 2024-05-17T21:48:47.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:47.551+0000 7f9ca7fff700 1 -- 172.21.15.107:0/2505550592 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9cb4009af0 con 0x7f9cc0102690 2024-05-17T21:48:47.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:47.551+0000 7f9ca7fff700 1 -- 172.21.15.107:0/2505550592 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9cb4013d10 con 0x7f9cc0102690 2024-05-17T21:48:47.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:47.552+0000 7f9cc5bf6700 1 -- 172.21.15.107:0/2505550592 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cac0052b0 con 0x7f9cc0102690 2024-05-17T21:48:47.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:47.552+0000 7f9ca7fff700 1 -- 172.21.15.107:0/2505550592 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f9cb400baf0 con 0x7f9cc0102690 2024-05-17T21:48:47.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:47.553+0000 7f9ca7fff700 1 --2- 172.21.15.107:0/2505550592 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f9ca80692a0 0x7f9ca806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:48:47.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:47.553+0000 7f9cbf7fe700 1 --2- 172.21.15.107:0/2505550592 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f9ca80692a0 0x7f9ca806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:48:47.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:47.553+0000 7f9ca7fff700 1 -- 172.21.15.107:0/2505550592 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f9cb4087d90 con 0x7f9cc0102690 2024-05-17T21:48:47.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:47.553+0000 7f9cbf7fe700 1 --2- 172.21.15.107:0/2505550592 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f9ca80692a0 0x7f9ca806b750 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9cb0004eb0 tx=0x7f9cb000b040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:48:47.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:47.557+0000 7f9ca7fff700 1 -- 172.21.15.107:0/2505550592 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f9cb40544a0 con 0x7f9cc0102690 2024-05-17T21:48:47.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:47.746+0000 7f9cc5bf6700 1 -- 172.21.15.107:0/2505550592 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f9cac005560 con 0x7f9cc0102690 2024-05-17T21:48:47.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:47.746+0000 7f9ca7fff700 1 -- 172.21.15.107:0/2505550592 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9cb4057ac0 con 0x7f9cc0102690 2024-05-17T21:48:47.740 INFO:teuthology.orchestra.run.smithi107.stdout:38654705702 2024-05-17T21:48:47.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:47.749+0000 7f9cc5bf6700 1 -- 172.21.15.107:0/2505550592 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f9ca80692a0 msgr2=0x7f9ca806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:48:47.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:47.749+0000 7f9cc5bf6700 1 --2- 172.21.15.107:0/2505550592 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f9ca80692a0 0x7f9ca806b750 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9cb0004eb0 tx=0x7f9cb000b040 comp rx=0 tx=0).stop 2024-05-17T21:48:47.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:47.749+0000 7f9cc5bf6700 1 -- 172.21.15.107:0/2505550592 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9cc0102690 msgr2=0x7f9cc010f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:48:47.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:47.749+0000 7f9cc5bf6700 1 --2- 172.21.15.107:0/2505550592 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9cc0102690 0x7f9cc010f020 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9cb4009100 tx=0x7f9cb4009e30 comp rx=0 tx=0).stop 2024-05-17T21:48:47.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:47.749+0000 7f9cc5bf6700 1 -- 172.21.15.107:0/2505550592 shutdown_connections 2024-05-17T21:48:47.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:47.749+0000 7f9cc5bf6700 1 --2- 172.21.15.107:0/2505550592 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f9ca80692a0 0x7f9ca806b750 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:47.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:47.749+0000 7f9cc5bf6700 1 --2- 172.21.15.107:0/2505550592 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9cc0101cf0 0x7f9cc010eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:47.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:47.749+0000 7f9cc5bf6700 1 --2- 172.21.15.107:0/2505550592 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9cc0102690 0x7f9cc010f020 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:47.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:47.749+0000 7f9cc5bf6700 1 -- 172.21.15.107:0/2505550592 >> 172.21.15.107:0/2505550592 conn(0x7f9cc00f7860 msgr2=0x7f9cc00f8800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:48:47.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:47.749+0000 7f9cc5bf6700 1 -- 172.21.15.107:0/2505550592 shutdown_connections 2024-05-17T21:48:47.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:47.749+0000 7f9cc5bf6700 1 -- 172.21.15.107:0/2505550592 wait complete. 2024-05-17T21:48:48.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:48:47 smithi138 ceph-mon[35948]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-17T21:48:48.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:48:47 smithi138 ceph-mon[35948]: from='client.? 172.21.15.107:0/3903789121' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-17T21:48:48.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:48:47 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:48:48.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:48:47 smithi138 ceph-mon[35948]: from='client.? 172.21.15.107:0/2505550592' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-17T21:48:48.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:48:47 smithi107 ceph-mon[28434]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-17T21:48:48.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:48:47 smithi107 ceph-mon[28434]: from='client.? 172.21.15.107:0/3903789121' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-17T21:48:48.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:48:47 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:48:48.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:48:47 smithi107 ceph-mon[28434]: from='client.? 172.21.15.107:0/2505550592' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-17T21:48:48.205 INFO:tasks.cephadm.ceph_manager.ceph:need seq 81604378649 got 81604378651 for osd.2 2024-05-17T21:48:48.205 DEBUG:teuthology.parallel:result is None 2024-05-17T21:48:48.793 INFO:tasks.cephadm.ceph_manager.ceph:need seq 38654705701 got 38654705702 for osd.0 2024-05-17T21:48:48.793 DEBUG:teuthology.parallel:result is None 2024-05-17T21:48:48.793 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-17T21:48:48.793 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -- ceph pg dump --format=json 2024-05-17T21:48:49.050 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/mon.smithi107/config 2024-05-17T21:48:50.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:48:49 smithi138 ceph-mon[35948]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-17T21:48:50.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:48:49 smithi107 ceph-mon[28434]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-17T21:48:50.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:50.633+0000 7fc02e6e4700 1 -- 172.21.15.107:0/2870886311 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc028103780 msgr2=0x7fc0280fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:48:50.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:50.633+0000 7fc02e6e4700 1 --2- 172.21.15.107:0/2870886311 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc028103780 0x7fc0280fcdf0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc01c004940 tx=0x7fc01c009e30 comp rx=0 tx=0).stop 2024-05-17T21:48:50.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:50.633+0000 7fc02e6e4700 1 -- 172.21.15.107:0/2870886311 shutdown_connections 2024-05-17T21:48:50.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:50.633+0000 7fc02e6e4700 1 --2- 172.21.15.107:0/2870886311 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc028103780 0x7fc0280fcdf0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:50.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:50.633+0000 7fc02e6e4700 1 --2- 172.21.15.107:0/2870886311 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc028102de0 0x7fc0281031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:50.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:50.633+0000 7fc02e6e4700 1 -- 172.21.15.107:0/2870886311 >> 172.21.15.107:0/2870886311 conn(0x7fc0280f8990 msgr2=0x7fc0280fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:48:50.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:50.633+0000 7fc02e6e4700 1 -- 172.21.15.107:0/2870886311 shutdown_connections 2024-05-17T21:48:50.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:50.633+0000 7fc02e6e4700 1 -- 172.21.15.107:0/2870886311 wait complete. 2024-05-17T21:48:50.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:50.634+0000 7fc02e6e4700 1 Processor -- start 2024-05-17T21:48:50.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:50.634+0000 7fc02e6e4700 1 -- start start 2024-05-17T21:48:50.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:50.635+0000 7fc02e6e4700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc028102de0 0x7fc028113980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:48:50.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:50.635+0000 7fc02e6e4700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc028103780 0x7fc028113ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:48:50.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:50.635+0000 7fc02e6e4700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0281145a0 con 0x7fc028102de0 2024-05-17T21:48:50.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:50.635+0000 7fc02e6e4700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc02810db60 con 0x7fc028103780 2024-05-17T21:48:50.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:50.635+0000 7fc027fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc028102de0 0x7fc028113980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:48:50.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:50.635+0000 7fc027fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc028102de0 0x7fc028113980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40296/0 (socket says 172.21.15.107:40296) 2024-05-17T21:48:50.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:50.635+0000 7fc027fff700 1 -- 172.21.15.107:0/3215330034 learned_addr learned my addr 172.21.15.107:0/3215330034 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:48:50.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:50.635+0000 7fc027fff700 1 -- 172.21.15.107:0/3215330034 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc028103780 msgr2=0x7fc028113ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:48:50.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:50.635+0000 7fc027fff700 1 --2- 172.21.15.107:0/3215330034 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc028103780 0x7fc028113ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:50.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:50.635+0000 7fc027fff700 1 -- 172.21.15.107:0/3215330034 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc01c005040 con 0x7fc028102de0 2024-05-17T21:48:50.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:50.635+0000 7fc027fff700 1 --2- 172.21.15.107:0/3215330034 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc028102de0 0x7fc028113980 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc01800b790 tx=0x7fc018009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:48:50.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:50.635+0000 7fc024ff9700 1 -- 172.21.15.107:0/3215330034 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc01800f770 con 0x7fc028102de0 2024-05-17T21:48:50.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:50.635+0000 7fc02e6e4700 1 -- 172.21.15.107:0/3215330034 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc02810de40 con 0x7fc028102de0 2024-05-17T21:48:50.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:50.635+0000 7fc024ff9700 1 -- 172.21.15.107:0/3215330034 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc01800fda0 con 0x7fc028102de0 2024-05-17T21:48:50.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:50.635+0000 7fc024ff9700 1 -- 172.21.15.107:0/3215330034 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc01800d310 con 0x7fc028102de0 2024-05-17T21:48:50.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:50.635+0000 7fc02e6e4700 1 -- 172.21.15.107:0/3215330034 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc02810e400 con 0x7fc028102de0 2024-05-17T21:48:50.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:50.636+0000 7fc02e6e4700 1 -- 172.21.15.107:0/3215330034 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0281000f0 con 0x7fc028102de0 2024-05-17T21:48:50.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:50.637+0000 7fc024ff9700 1 -- 172.21.15.107:0/3215330034 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7fc018010420 con 0x7fc028102de0 2024-05-17T21:48:50.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:50.637+0000 7fc024ff9700 1 --2- 172.21.15.107:0/3215330034 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fc0100692a0 0x7fc01006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:48:50.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:50.637+0000 7fc0277fe700 1 --2- 172.21.15.107:0/3215330034 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fc0100692a0 0x7fc01006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:48:50.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:50.637+0000 7fc024ff9700 1 -- 172.21.15.107:0/3215330034 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fc018086e80 con 0x7fc028102de0 2024-05-17T21:48:50.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:50.638+0000 7fc0277fe700 1 --2- 172.21.15.107:0/3215330034 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fc0100692a0 0x7fc01006b750 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc01c004f90 tx=0x7fc01c009e30 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:48:50.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:50.642+0000 7fc024ff9700 1 -- 172.21.15.107:0/3215330034 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fc018056f80 con 0x7fc028102de0 2024-05-17T21:48:50.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:50.823+0000 7fc02e6e4700 1 -- 172.21.15.107:0/3215330034 --> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc02810f040 con 0x7fc0100692a0 2024-05-17T21:48:50.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:50.824+0000 7fc024ff9700 1 -- 172.21.15.107:0/3215330034 <== mgr.14214 v2:172.21.15.107:6800/1294361014 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+28124 (secure 0 0 0) 0x7fc02810f040 con 0x7fc0100692a0 2024-05-17T21:48:50.818 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:48:50.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:50.826+0000 7fc02e6e4700 1 -- 172.21.15.107:0/3215330034 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fc0100692a0 msgr2=0x7fc01006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:48:50.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:50.827+0000 7fc02e6e4700 1 --2- 172.21.15.107:0/3215330034 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fc0100692a0 0x7fc01006b750 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc01c004f90 tx=0x7fc01c009e30 comp rx=0 tx=0).stop 2024-05-17T21:48:50.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:50.827+0000 7fc02e6e4700 1 -- 172.21.15.107:0/3215330034 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc028102de0 msgr2=0x7fc028113980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:48:50.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:50.827+0000 7fc02e6e4700 1 --2- 172.21.15.107:0/3215330034 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc028102de0 0x7fc028113980 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc01800b790 tx=0x7fc018009250 comp rx=0 tx=0).stop 2024-05-17T21:48:50.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:50.827+0000 7fc02e6e4700 1 -- 172.21.15.107:0/3215330034 shutdown_connections 2024-05-17T21:48:50.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:50.827+0000 7fc02e6e4700 1 --2- 172.21.15.107:0/3215330034 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fc0100692a0 0x7fc01006b750 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:50.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:50.827+0000 7fc02e6e4700 1 --2- 172.21.15.107:0/3215330034 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc028102de0 0x7fc028113980 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:50.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:50.827+0000 7fc02e6e4700 1 --2- 172.21.15.107:0/3215330034 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc028103780 0x7fc028113ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:50.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:50.827+0000 7fc02e6e4700 1 -- 172.21.15.107:0/3215330034 >> 172.21.15.107:0/3215330034 conn(0x7fc0280f8990 msgr2=0x7fc028107380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:48:50.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:50.827+0000 7fc02e6e4700 1 -- 172.21.15.107:0/3215330034 shutdown_connections 2024-05-17T21:48:50.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:50.827+0000 7fc02e6e4700 1 -- 172.21.15.107:0/3215330034 wait complete. 2024-05-17T21:48:50.823 INFO:teuthology.orchestra.run.smithi107.stderr:dumped all 2024-05-17T21:48:51.517 INFO:teuthology.orchestra.run.smithi107.stdout:{"pg_ready":true,"pg_map":{"version":157,"stamp":"2024-05-17T21:48:50.337981+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":1788036,"kb_used_data":3876,"kb_used_omap":9,"kb_used_meta":1784054,"kb_avail":560682876,"statfs":{"total":575970213888,"available":574139265024,"internally_reserved":0,"allocated":3969024,"data_stored":2606262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10224,"internal_metadata":1826871312},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002030"},"pg_stats":[{"pgid":"1.0","version":"23'117","reported_seq":190,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-17T21:47:57.572607+0000","last_change":"2024-05-17T21:47:32.393796+0000","last_active":"2024-05-17T21:47:57.572607+0000","last_peered":"2024-05-17T21:47:57.572607+0000","last_clean":"2024-05-17T21:47:57.572607+0000","last_became_active":"2024-05-17T21:47:32.393465+0000","last_became_peered":"2024-05-17T21:47:32.393465+0000","last_unstale":"2024-05-17T21:47:57.572607+0000","last_undegraded":"2024-05-17T21:47:57.572607+0000","last_fullsized":"2024-05-17T21:47:57.572607+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:46:43.071604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:46:43.071604+0000","last_clean_scrub_stamp":"2024-05-17T21:46:43.071604+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:48:48.458751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":39,"seq":167503724558,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297712,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447440,"statfs":{"total":95995035648,"available":95690178560,"internally_reserved":0,"allocated":364544,"data_stored":139193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66300000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45400000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46700000000000003}]}]},{"osd":4,"up_from":34,"seq":146028888083,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297716,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447436,"statfs":{"total":95995035648,"available":95690174464,"internally_reserved":0,"allocated":364544,"data_stored":139193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 17 21:48:33 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":2.7909999999999999,"5min":2.7909999999999999,"15min":2.7909999999999999},"last":0.627},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.5649999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.48099999999999998}]},{"osd":1,"last update":"Fri May 17 21:48:33 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.4710000000000001,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.49,"5min":1.49,"15min":1.49},"last":0.59699999999999998}]},{"osd":2,"last update":"Fri May 17 21:48:33 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":2.7349999999999999,"5min":2.7349999999999999,"15min":2.7349999999999999},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.52900000000000003}]},{"osd":3,"last update":"Fri May 17 21:48:33 2024","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.379,"5min":0.379,"15min":0.379},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.54500000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]}]},{"osd":3,"up_from":26,"seq":111669149720,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298292,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446860,"statfs":{"total":95995035648,"available":95689584640,"internally_reserved":0,"allocated":958464,"data_stored":729561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 17 21:48:07 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.70299999999999996}]},{"osd":1,"last update":"Fri May 17 21:48:07 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.75600000000000001}]},{"osd":2,"last update":"Fri May 17 21:48:07 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.74099999999999999}]},{"osd":4,"last update":"Fri May 17 21:48:35 2024","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.36799999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39300000000000002}]}]},{"osd":2,"up_from":19,"seq":81604378652,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297720,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447432,"statfs":{"total":95995035648,"available":95690170368,"internally_reserved":0,"allocated":364544,"data_stored":139193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 17 21:48:46 2024","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.34799999999999998,"15min":0.34999999999999998},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"last":0.376},{"interface":"front","average":{"1min":0.32500000000000001,"5min":0.35899999999999999,"15min":0.36499999999999999},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"last":0.40300000000000002}]},{"osd":1,"last update":"Fri May 17 21:48:46 2024","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":0.35099999999999998,"15min":0.35499999999999998},"min":{"1min":0.23999999999999999,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.47399999999999998,"5min":0.48399999999999999,"15min":0.48399999999999999},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.32500000000000001,"5min":0.35899999999999999,"15min":0.36499999999999999},"min":{"1min":0.25800000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.42799999999999999,"5min":0.45100000000000001,"15min":0.45100000000000001},"last":0.34300000000000003}]},{"osd":3,"last update":"Fri May 17 21:48:07 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.44600000000000001}]},{"osd":4,"last update":"Fri May 17 21:48:36 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.437}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46500000000000002}]}]},{"osd":0,"up_from":9,"seq":38654705703,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298300,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446852,"statfs":{"total":95995035648,"available":95689576448,"internally_reserved":0,"allocated":958464,"data_stored":729561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri May 17 21:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.32200000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"min":{"1min":0.20599999999999999,"5min":0.182,"15min":0.182},"max":{"1min":0.46400000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"last":0.252},{"interface":"front","average":{"1min":0.33500000000000002,"5min":0.32900000000000001,"15min":0.32800000000000001},"min":{"1min":0.247,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.42099999999999999,"5min":0.58199999999999996,"15min":0.58199999999999996},"last":0.30399999999999999}]},{"osd":2,"last update":"Fri May 17 21:48:45 2024","interfaces":[{"interface":"back","average":{"1min":0.31,"5min":0.36599999999999999,"15min":0.376},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.437,"5min":0.627,"15min":0.627},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.38100000000000001,"15min":0.38900000000000001},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.44800000000000001,"5min":0.61199999999999999,"15min":0.61199999999999999},"last":0.318}]},{"osd":3,"last update":"Fri May 17 21:48:07 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.53700000000000003}]},{"osd":4,"last update":"Fri May 17 21:48:32 2024","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":1.246,"15min":1.246},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":13.632999999999999,"5min":13.632999999999999,"15min":13.632999999999999},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.42799999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44900000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542178,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298296,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446856,"statfs":{"total":95995035648,"available":95689580544,"internally_reserved":0,"allocated":958464,"data_stored":729561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 17 21:48:20 2024","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.31900000000000001,"15min":0.313},"min":{"1min":0.249,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.32400000000000001,"15min":0.32000000000000001},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.48099999999999998,"5min":0.502,"15min":0.502},"last":0.377}]},{"osd":2,"last update":"Fri May 17 21:48:47 2024","interfaces":[{"interface":"back","average":{"1min":0.32400000000000001,"5min":0.36199999999999999,"15min":0.36799999999999999},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.318},{"interface":"front","average":{"1min":0.317,"5min":0.376,"15min":0.38600000000000001},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.46600000000000003,"5min":0.57199999999999995,"15min":0.57199999999999995},"last":0.35699999999999998}]},{"osd":3,"last update":"Fri May 17 21:48:10 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.498},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.50700000000000001}]},{"osd":4,"last update":"Fri May 17 21:48:34 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.44700000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-17T21:48:51.518 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -- ceph pg dump --format=json 2024-05-17T21:48:51.772 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/mon.smithi107/config 2024-05-17T21:48:52.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:48:51 smithi138 ceph-mon[35948]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-17T21:48:52.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:48:51 smithi138 ceph-mon[35948]: from='client.14426 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-17T21:48:52.085 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:48:51 smithi107 ceph-mon[28434]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-17T21:48:52.085 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:48:51 smithi107 ceph-mon[28434]: from='client.14426 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-17T21:48:53.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:53.365+0000 7ff611647700 1 -- 172.21.15.107:0/2482092396 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff60c0fc710 msgr2=0x7ff60c0fcb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:48:53.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:53.365+0000 7ff611647700 1 --2- 172.21.15.107:0/2482092396 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff60c0fc710 0x7ff60c0fcb80 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff5f4004940 tx=0x7ff5f4009e30 comp rx=0 tx=0).stop 2024-05-17T21:48:53.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:53.365+0000 7ff611647700 1 -- 172.21.15.107:0/2482092396 shutdown_connections 2024-05-17T21:48:53.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:53.365+0000 7ff611647700 1 --2- 172.21.15.107:0/2482092396 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff60c0fc710 0x7ff60c0fcb80 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:53.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:53.365+0000 7ff611647700 1 --2- 172.21.15.107:0/2482092396 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff60c101e20 0x7ff60c1021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:53.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:53.365+0000 7ff611647700 1 -- 172.21.15.107:0/2482092396 >> 172.21.15.107:0/2482092396 conn(0x7ff60c071460 msgr2=0x7ff60c071860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:48:53.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:53.365+0000 7ff611647700 1 -- 172.21.15.107:0/2482092396 shutdown_connections 2024-05-17T21:48:53.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:53.365+0000 7ff611647700 1 -- 172.21.15.107:0/2482092396 wait complete. 2024-05-17T21:48:53.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:53.366+0000 7ff611647700 1 Processor -- start 2024-05-17T21:48:53.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:53.366+0000 7ff611647700 1 -- start start 2024-05-17T21:48:53.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:53.366+0000 7ff611647700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff60c0fc710 0x7ff60c112a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:48:53.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:53.366+0000 7ff611647700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff60c101e20 0x7ff60c112f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:48:53.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:53.366+0000 7ff611647700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff60c1135d0 con 0x7ff60c101e20 2024-05-17T21:48:53.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:53.366+0000 7ff611647700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff60c10c9c0 con 0x7ff60c0fc710 2024-05-17T21:48:53.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:53.366+0000 7ff60a7fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff60c101e20 0x7ff60c112f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:48:53.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:53.366+0000 7ff60a7fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff60c101e20 0x7ff60c112f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40304/0 (socket says 172.21.15.107:40304) 2024-05-17T21:48:53.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:53.366+0000 7ff60a7fc700 1 -- 172.21.15.107:0/2188173077 learned_addr learned my addr 172.21.15.107:0/2188173077 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:48:53.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:53.366+0000 7ff60affd700 1 --2- 172.21.15.107:0/2188173077 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff60c0fc710 0x7ff60c112a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:48:53.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:53.366+0000 7ff60a7fc700 1 -- 172.21.15.107:0/2188173077 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff60c0fc710 msgr2=0x7ff60c112a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:48:53.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:53.366+0000 7ff60a7fc700 1 --2- 172.21.15.107:0/2188173077 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff60c0fc710 0x7ff60c112a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:53.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:53.366+0000 7ff60a7fc700 1 -- 172.21.15.107:0/2188173077 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5f4005040 con 0x7ff60c101e20 2024-05-17T21:48:53.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:53.366+0000 7ff60a7fc700 1 --2- 172.21.15.107:0/2188173077 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff60c101e20 0x7ff60c112f80 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff5f400b3b0 tx=0x7ff5f4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:48:53.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:53.367+0000 7ff6037fe700 1 -- 172.21.15.107:0/2188173077 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff5f400e040 con 0x7ff60c101e20 2024-05-17T21:48:53.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:53.367+0000 7ff6037fe700 1 -- 172.21.15.107:0/2188173077 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff5f4013420 con 0x7ff60c101e20 2024-05-17T21:48:53.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:53.367+0000 7ff6037fe700 1 -- 172.21.15.107:0/2188173077 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff5f401c630 con 0x7ff60c101e20 2024-05-17T21:48:53.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:53.367+0000 7ff611647700 1 -- 172.21.15.107:0/2188173077 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff60c10cc40 con 0x7ff60c101e20 2024-05-17T21:48:53.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:53.367+0000 7ff611647700 1 -- 172.21.15.107:0/2188173077 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff60c10d1b0 con 0x7ff60c101e20 2024-05-17T21:48:53.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:53.368+0000 7ff611647700 1 -- 172.21.15.107:0/2188173077 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff60c0652f0 con 0x7ff60c101e20 2024-05-17T21:48:53.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:53.368+0000 7ff6037fe700 1 -- 172.21.15.107:0/2188173077 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7ff5f4013590 con 0x7ff60c101e20 2024-05-17T21:48:53.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:53.369+0000 7ff6037fe700 1 --2- 172.21.15.107:0/2188173077 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7ff5f806b4b0 0x7ff5f806d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:48:53.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:53.369+0000 7ff6037fe700 1 -- 172.21.15.107:0/2188173077 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7ff5f4088040 con 0x7ff60c101e20 2024-05-17T21:48:53.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:53.369+0000 7ff60affd700 1 --2- 172.21.15.107:0/2188173077 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7ff5f806b4b0 0x7ff5f806d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:48:53.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:53.369+0000 7ff60affd700 1 --2- 172.21.15.107:0/2188173077 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7ff5f806b4b0 0x7ff5f806d960 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff604004fd0 tx=0x7ff604009040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:48:53.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:53.373+0000 7ff6037fe700 1 -- 172.21.15.107:0/2188173077 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7ff5f4058140 con 0x7ff60c101e20 2024-05-17T21:48:53.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:53.553+0000 7ff611647700 1 -- 172.21.15.107:0/2188173077 --> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff60c112710 con 0x7ff5f806b4b0 2024-05-17T21:48:53.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:53.554+0000 7ff6037fe700 1 -- 172.21.15.107:0/2188173077 <== mgr.14214 v2:172.21.15.107:6800/1294361014 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+28124 (secure 0 0 0) 0x7ff60c112710 con 0x7ff5f806b4b0 2024-05-17T21:48:53.548 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:48:53.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:53.557+0000 7ff611647700 1 -- 172.21.15.107:0/2188173077 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7ff5f806b4b0 msgr2=0x7ff5f806d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:48:53.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:53.557+0000 7ff611647700 1 --2- 172.21.15.107:0/2188173077 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7ff5f806b4b0 0x7ff5f806d960 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff604004fd0 tx=0x7ff604009040 comp rx=0 tx=0).stop 2024-05-17T21:48:53.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:53.557+0000 7ff611647700 1 -- 172.21.15.107:0/2188173077 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff60c101e20 msgr2=0x7ff60c112f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:48:53.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:53.557+0000 7ff611647700 1 --2- 172.21.15.107:0/2188173077 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff60c101e20 0x7ff60c112f80 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff5f400b3b0 tx=0x7ff5f4009e30 comp rx=0 tx=0).stop 2024-05-17T21:48:53.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:53.557+0000 7ff611647700 1 -- 172.21.15.107:0/2188173077 shutdown_connections 2024-05-17T21:48:53.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:53.557+0000 7ff611647700 1 --2- 172.21.15.107:0/2188173077 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7ff5f806b4b0 0x7ff5f806d960 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:53.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:53.557+0000 7ff611647700 1 --2- 172.21.15.107:0/2188173077 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff60c0fc710 0x7ff60c112a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:53.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:53.557+0000 7ff611647700 1 --2- 172.21.15.107:0/2188173077 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff60c101e20 0x7ff60c112f80 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:53.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:53.557+0000 7ff611647700 1 -- 172.21.15.107:0/2188173077 >> 172.21.15.107:0/2188173077 conn(0x7ff60c071460 msgr2=0x7ff60c0f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:48:53.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:53.557+0000 7ff611647700 1 -- 172.21.15.107:0/2188173077 shutdown_connections 2024-05-17T21:48:53.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:53.557+0000 7ff611647700 1 -- 172.21.15.107:0/2188173077 wait complete. 2024-05-17T21:48:53.551 INFO:teuthology.orchestra.run.smithi107.stderr:dumped all 2024-05-17T21:48:53.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:48:53 smithi107 ceph-mon[28434]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-17T21:48:54.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:48:53 smithi138 ceph-mon[35948]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-17T21:48:54.162 INFO:teuthology.orchestra.run.smithi107.stdout:{"pg_ready":true,"pg_map":{"version":158,"stamp":"2024-05-17T21:48:52.338280+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":1788036,"kb_used_data":3876,"kb_used_omap":9,"kb_used_meta":1784054,"kb_avail":560682876,"statfs":{"total":575970213888,"available":574139265024,"internally_reserved":0,"allocated":3969024,"data_stored":2606262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10224,"internal_metadata":1826871312},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002038"},"pg_stats":[{"pgid":"1.0","version":"23'117","reported_seq":190,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-17T21:47:57.572607+0000","last_change":"2024-05-17T21:47:32.393796+0000","last_active":"2024-05-17T21:47:57.572607+0000","last_peered":"2024-05-17T21:47:57.572607+0000","last_clean":"2024-05-17T21:47:57.572607+0000","last_became_active":"2024-05-17T21:47:32.393465+0000","last_became_peered":"2024-05-17T21:47:32.393465+0000","last_unstale":"2024-05-17T21:47:57.572607+0000","last_undegraded":"2024-05-17T21:47:57.572607+0000","last_fullsized":"2024-05-17T21:47:57.572607+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:46:43.071604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:46:43.071604+0000","last_clean_scrub_stamp":"2024-05-17T21:46:43.071604+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:48:48.458751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":39,"seq":167503724558,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297712,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447440,"statfs":{"total":95995035648,"available":95690178560,"internally_reserved":0,"allocated":364544,"data_stored":139193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66300000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45400000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46700000000000003}]}]},{"osd":4,"up_from":34,"seq":146028888083,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297716,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447436,"statfs":{"total":95995035648,"available":95690174464,"internally_reserved":0,"allocated":364544,"data_stored":139193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 17 21:48:33 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":2.7909999999999999,"5min":2.7909999999999999,"15min":2.7909999999999999},"last":0.627},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.5649999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.48099999999999998}]},{"osd":1,"last update":"Fri May 17 21:48:33 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.4710000000000001,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.49,"5min":1.49,"15min":1.49},"last":0.59699999999999998}]},{"osd":2,"last update":"Fri May 17 21:48:33 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":2.7349999999999999,"5min":2.7349999999999999,"15min":2.7349999999999999},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.52900000000000003}]},{"osd":3,"last update":"Fri May 17 21:48:33 2024","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.379,"5min":0.379,"15min":0.379},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.54500000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]}]},{"osd":3,"up_from":26,"seq":111669149720,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298292,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446860,"statfs":{"total":95995035648,"available":95689584640,"internally_reserved":0,"allocated":958464,"data_stored":729561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 17 21:48:07 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.70299999999999996}]},{"osd":1,"last update":"Fri May 17 21:48:07 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.75600000000000001}]},{"osd":2,"last update":"Fri May 17 21:48:07 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.74099999999999999}]},{"osd":4,"last update":"Fri May 17 21:48:35 2024","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.36799999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39300000000000002}]}]},{"osd":2,"up_from":19,"seq":81604378652,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297720,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447432,"statfs":{"total":95995035648,"available":95690170368,"internally_reserved":0,"allocated":364544,"data_stored":139193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 17 21:48:46 2024","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.34799999999999998,"15min":0.34999999999999998},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"last":0.376},{"interface":"front","average":{"1min":0.32500000000000001,"5min":0.35899999999999999,"15min":0.36499999999999999},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"last":0.40300000000000002}]},{"osd":1,"last update":"Fri May 17 21:48:46 2024","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":0.35099999999999998,"15min":0.35499999999999998},"min":{"1min":0.23999999999999999,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.47399999999999998,"5min":0.48399999999999999,"15min":0.48399999999999999},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.32500000000000001,"5min":0.35899999999999999,"15min":0.36499999999999999},"min":{"1min":0.25800000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.42799999999999999,"5min":0.45100000000000001,"15min":0.45100000000000001},"last":0.34300000000000003}]},{"osd":3,"last update":"Fri May 17 21:48:07 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.44600000000000001}]},{"osd":4,"last update":"Fri May 17 21:48:36 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.437}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46500000000000002}]}]},{"osd":0,"up_from":9,"seq":38654705703,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298300,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446852,"statfs":{"total":95995035648,"available":95689576448,"internally_reserved":0,"allocated":958464,"data_stored":729561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri May 17 21:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.32200000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"min":{"1min":0.20599999999999999,"5min":0.182,"15min":0.182},"max":{"1min":0.46400000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"last":0.252},{"interface":"front","average":{"1min":0.33500000000000002,"5min":0.32900000000000001,"15min":0.32800000000000001},"min":{"1min":0.247,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.42099999999999999,"5min":0.58199999999999996,"15min":0.58199999999999996},"last":0.30399999999999999}]},{"osd":2,"last update":"Fri May 17 21:48:45 2024","interfaces":[{"interface":"back","average":{"1min":0.31,"5min":0.36599999999999999,"15min":0.376},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.437,"5min":0.627,"15min":0.627},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.38100000000000001,"15min":0.38900000000000001},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.44800000000000001,"5min":0.61199999999999999,"15min":0.61199999999999999},"last":0.318}]},{"osd":3,"last update":"Fri May 17 21:48:07 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.53700000000000003}]},{"osd":4,"last update":"Fri May 17 21:48:32 2024","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":1.246,"15min":1.246},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":13.632999999999999,"5min":13.632999999999999,"15min":13.632999999999999},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.42799999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44900000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542178,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298296,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446856,"statfs":{"total":95995035648,"available":95689580544,"internally_reserved":0,"allocated":958464,"data_stored":729561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 17 21:48:20 2024","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.31900000000000001,"15min":0.313},"min":{"1min":0.249,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.32400000000000001,"15min":0.32000000000000001},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.48099999999999998,"5min":0.502,"15min":0.502},"last":0.377}]},{"osd":2,"last update":"Fri May 17 21:48:47 2024","interfaces":[{"interface":"back","average":{"1min":0.32400000000000001,"5min":0.36199999999999999,"15min":0.36799999999999999},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.318},{"interface":"front","average":{"1min":0.317,"5min":0.376,"15min":0.38600000000000001},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.46600000000000003,"5min":0.57199999999999995,"15min":0.57199999999999995},"last":0.35699999999999998}]},{"osd":3,"last update":"Fri May 17 21:48:10 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.498},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.50700000000000001}]},{"osd":4,"last update":"Fri May 17 21:48:34 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.44700000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-17T21:48:54.163 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-17T21:48:54.163 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-17T21:48:54.164 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-05-17T21:48:54.164 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -- ceph health --format=json 2024-05-17T21:48:54.416 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/mon.smithi107/config 2024-05-17T21:48:54.824 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:48:54 smithi107 ceph-mon[28434]: from='client.14430 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-17T21:48:55.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:48:54 smithi138 ceph-mon[35948]: from='client.14430 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-17T21:48:55.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:55.968+0000 7f3c4ea62700 1 -- 172.21.15.107:0/1150965654 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3c480fe800 msgr2=0x7f3c48106ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:48:55.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:55.968+0000 7f3c4ea62700 1 --2- 172.21.15.107:0/1150965654 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3c480fe800 0x7f3c48106ed0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c004940 tx=0x7f3c3c009e30 comp rx=0 tx=0).stop 2024-05-17T21:48:55.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:55.968+0000 7f3c4ea62700 1 -- 172.21.15.107:0/1150965654 shutdown_connections 2024-05-17T21:48:55.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:55.968+0000 7f3c4ea62700 1 --2- 172.21.15.107:0/1150965654 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3c480fe800 0x7f3c48106ed0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:55.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:55.968+0000 7f3c4ea62700 1 --2- 172.21.15.107:0/1150965654 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c480fde60 0x7f3c480fe230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:55.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:55.968+0000 7f3c4ea62700 1 -- 172.21.15.107:0/1150965654 >> 172.21.15.107:0/1150965654 conn(0x7f3c480f78e0 msgr2=0x7f3c480f9cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:48:55.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:55.968+0000 7f3c4ea62700 1 -- 172.21.15.107:0/1150965654 shutdown_connections 2024-05-17T21:48:55.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:55.968+0000 7f3c4ea62700 1 -- 172.21.15.107:0/1150965654 wait complete. 2024-05-17T21:48:55.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:55.968+0000 7f3c4ea62700 1 Processor -- start 2024-05-17T21:48:55.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:55.969+0000 7f3c4ea62700 1 -- start start 2024-05-17T21:48:55.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:55.969+0000 7f3c4ea62700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3c480fde60 0x7f3c48113950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:48:55.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:55.969+0000 7f3c4ea62700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c480fe800 0x7f3c48113e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:48:55.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:55.969+0000 7f3c4ea62700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c481144e0 con 0x7f3c480fde60 2024-05-17T21:48:55.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:55.969+0000 7f3c4ea62700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c4810dae0 con 0x7f3c480fe800 2024-05-17T21:48:55.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:55.969+0000 7f3c47fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3c480fde60 0x7f3c48113950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:48:55.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:55.969+0000 7f3c47fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3c480fde60 0x7f3c48113950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46026/0 (socket says 172.21.15.107:46026) 2024-05-17T21:48:55.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:55.969+0000 7f3c47fff700 1 -- 172.21.15.107:0/1152876340 learned_addr learned my addr 172.21.15.107:0/1152876340 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:48:55.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:55.969+0000 7f3c47fff700 1 -- 172.21.15.107:0/1152876340 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c480fe800 msgr2=0x7f3c48113e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:48:55.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:55.969+0000 7f3c47fff700 1 --2- 172.21.15.107:0/1152876340 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c480fe800 0x7f3c48113e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:55.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:55.969+0000 7f3c47fff700 1 -- 172.21.15.107:0/1152876340 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c3c005040 con 0x7f3c480fde60 2024-05-17T21:48:55.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:55.969+0000 7f3c47fff700 1 --2- 172.21.15.107:0/1152876340 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3c480fde60 0x7f3c48113950 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3c3800c8c0 tx=0x7f3c380044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:48:55.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:55.969+0000 7f3c44ff9700 1 -- 172.21.15.107:0/1152876340 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3c3800a980 con 0x7f3c480fde60 2024-05-17T21:48:55.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:55.969+0000 7f3c4ea62700 1 -- 172.21.15.107:0/1152876340 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c4810ddc0 con 0x7f3c480fde60 2024-05-17T21:48:55.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:55.969+0000 7f3c44ff9700 1 -- 172.21.15.107:0/1152876340 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3c3800ce60 con 0x7f3c480fde60 2024-05-17T21:48:55.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:55.969+0000 7f3c44ff9700 1 -- 172.21.15.107:0/1152876340 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3c380115e0 con 0x7f3c480fde60 2024-05-17T21:48:55.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:55.969+0000 7f3c4ea62700 1 -- 172.21.15.107:0/1152876340 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c4810e300 con 0x7f3c480fde60 2024-05-17T21:48:55.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:55.971+0000 7f3c4ea62700 1 -- 172.21.15.107:0/1152876340 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c48101890 con 0x7f3c480fde60 2024-05-17T21:48:55.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:55.971+0000 7f3c44ff9700 1 -- 172.21.15.107:0/1152876340 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f3c3800aae0 con 0x7f3c480fde60 2024-05-17T21:48:55.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:55.971+0000 7f3c44ff9700 1 --2- 172.21.15.107:0/1152876340 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f3c300692a0 0x7f3c3006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:48:55.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:55.971+0000 7f3c44ff9700 1 -- 172.21.15.107:0/1152876340 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f3c38086eb0 con 0x7f3c480fde60 2024-05-17T21:48:55.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:55.971+0000 7f3c477fe700 1 --2- 172.21.15.107:0/1152876340 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f3c300692a0 0x7f3c3006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:48:55.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:55.972+0000 7f3c477fe700 1 --2- 172.21.15.107:0/1152876340 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f3c300692a0 0x7f3c3006b750 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c004890 tx=0x7f3c3c009e30 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:48:55.969 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:55.976+0000 7f3c44ff9700 1 -- 172.21.15.107:0/1152876340 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f3c38057050 con 0x7f3c480fde60 2024-05-17T21:48:56.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:48:55 smithi138 ceph-mon[35948]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-17T21:48:56.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:48:55 smithi107 ceph-mon[28434]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-17T21:48:56.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:56.202+0000 7f3c4ea62700 1 -- 172.21.15.107:0/1152876340 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f3c4810efa0 con 0x7f3c480fde60 2024-05-17T21:48:56.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:56.202+0000 7f3c44ff9700 1 -- 172.21.15.107:0/1152876340 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f3c38056be0 con 0x7f3c480fde60 2024-05-17T21:48:56.196 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:48:56.196 INFO:teuthology.orchestra.run.smithi107.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-17T21:48:56.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:56.204+0000 7f3c4ea62700 1 -- 172.21.15.107:0/1152876340 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f3c300692a0 msgr2=0x7f3c3006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:48:56.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:56.204+0000 7f3c4ea62700 1 --2- 172.21.15.107:0/1152876340 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f3c300692a0 0x7f3c3006b750 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c004890 tx=0x7f3c3c009e30 comp rx=0 tx=0).stop 2024-05-17T21:48:56.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:56.204+0000 7f3c4ea62700 1 -- 172.21.15.107:0/1152876340 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3c480fde60 msgr2=0x7f3c48113950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:48:56.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:56.204+0000 7f3c4ea62700 1 --2- 172.21.15.107:0/1152876340 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3c480fde60 0x7f3c48113950 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3c3800c8c0 tx=0x7f3c380044c0 comp rx=0 tx=0).stop 2024-05-17T21:48:56.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:56.204+0000 7f3c4ea62700 1 -- 172.21.15.107:0/1152876340 shutdown_connections 2024-05-17T21:48:56.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:56.204+0000 7f3c4ea62700 1 --2- 172.21.15.107:0/1152876340 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f3c300692a0 0x7f3c3006b750 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:56.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:56.204+0000 7f3c4ea62700 1 --2- 172.21.15.107:0/1152876340 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3c480fde60 0x7f3c48113950 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:56.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:56.204+0000 7f3c4ea62700 1 --2- 172.21.15.107:0/1152876340 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c480fe800 0x7f3c48113e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:56.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:56.204+0000 7f3c4ea62700 1 -- 172.21.15.107:0/1152876340 >> 172.21.15.107:0/1152876340 conn(0x7f3c480f78e0 msgr2=0x7f3c480ff350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:48:56.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:56.205+0000 7f3c4ea62700 1 -- 172.21.15.107:0/1152876340 shutdown_connections 2024-05-17T21:48:56.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:56.205+0000 7f3c4ea62700 1 -- 172.21.15.107:0/1152876340 wait complete. 2024-05-17T21:48:56.809 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:48:56 smithi107 ceph-mon[28434]: from='client.? 172.21.15.107:0/1152876340' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-17T21:48:56.834 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-05-17T21:48:56.834 INFO:tasks.cephadm:Setup complete, yielding 2024-05-17T21:48:56.835 INFO:teuthology.run_tasks:Running task print... 2024-05-17T21:48:56.843 INFO:teuthology.task.print:**** done end installing quincy cephadm ... 2024-05-17T21:48:56.843 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-17T21:48:56.851 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi107.front.sepia.ceph.com 2024-05-17T21:48:56.851 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -- bash -c 'ceph config set mgr mgr/cephadm/use_repo_digest true --force' 2024-05-17T21:48:57.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:48:56 smithi138 ceph-mon[35948]: from='client.? 172.21.15.107:0/1152876340' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-17T21:48:57.109 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/mon.smithi107/config 2024-05-17T21:48:58.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:48:57 smithi138 ceph-mon[35948]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-17T21:48:58.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:48:57 smithi107 ceph-mon[28434]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-17T21:48:58.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:58.691+0000 7fd0d8ff5700 1 -- 172.21.15.107:0/126336564 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0d40fcff0 msgr2=0x7fd0d40fd460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:48:58.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:58.691+0000 7fd0d8ff5700 1 --2- 172.21.15.107:0/126336564 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0d40fcff0 0x7fd0d40fd460 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd0c8004940 tx=0x7fd0c8009e30 comp rx=0 tx=0).stop 2024-05-17T21:48:58.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:58.691+0000 7fd0d8ff5700 1 -- 172.21.15.107:0/126336564 shutdown_connections 2024-05-17T21:48:58.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:58.691+0000 7fd0d8ff5700 1 --2- 172.21.15.107:0/126336564 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0d40fcff0 0x7fd0d40fd460 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:58.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:58.691+0000 7fd0d8ff5700 1 --2- 172.21.15.107:0/126336564 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd0d40fbd40 0x7fd0d40fc150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:58.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:58.691+0000 7fd0d8ff5700 1 -- 172.21.15.107:0/126336564 >> 172.21.15.107:0/126336564 conn(0x7fd0d40f74c0 msgr2=0x7fd0d40f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:48:58.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:58.691+0000 7fd0d8ff5700 1 -- 172.21.15.107:0/126336564 shutdown_connections 2024-05-17T21:48:58.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:58.691+0000 7fd0d8ff5700 1 -- 172.21.15.107:0/126336564 wait complete. 2024-05-17T21:48:58.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:58.691+0000 7fd0d8ff5700 1 Processor -- start 2024-05-17T21:48:58.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:58.691+0000 7fd0d8ff5700 1 -- start start 2024-05-17T21:48:58.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:58.692+0000 7fd0d8ff5700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0d40fbd40 0x7fd0d4110720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:48:58.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:58.692+0000 7fd0d8ff5700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd0d40fcff0 0x7fd0d4112c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:48:58.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:58.692+0000 7fd0d8ff5700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0d4066c70 con 0x7fd0d40fbd40 2024-05-17T21:48:58.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:58.692+0000 7fd0d8ff5700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0d4066db0 con 0x7fd0d40fcff0 2024-05-17T21:48:58.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:58.692+0000 7fd0d259c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0d40fbd40 0x7fd0d4110720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:48:58.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:58.692+0000 7fd0d259c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0d40fbd40 0x7fd0d4110720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46040/0 (socket says 172.21.15.107:46040) 2024-05-17T21:48:58.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:58.692+0000 7fd0d259c700 1 -- 172.21.15.107:0/2125627181 learned_addr learned my addr 172.21.15.107:0/2125627181 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:48:58.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:58.692+0000 7fd0d259c700 1 -- 172.21.15.107:0/2125627181 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd0d40fcff0 msgr2=0x7fd0d4112c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:48:58.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:58.692+0000 7fd0d1d9b700 1 --2- 172.21.15.107:0/2125627181 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd0d40fcff0 0x7fd0d4112c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:48:58.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:58.692+0000 7fd0d259c700 1 --2- 172.21.15.107:0/2125627181 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd0d40fcff0 0x7fd0d4112c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:58.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:58.692+0000 7fd0d259c700 1 -- 172.21.15.107:0/2125627181 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0c8005040 con 0x7fd0d40fbd40 2024-05-17T21:48:58.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:58.692+0000 7fd0d259c700 1 --2- 172.21.15.107:0/2125627181 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0d40fbd40 0x7fd0d4110720 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd0c400b7c0 tx=0x7fd0c4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:48:58.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:58.692+0000 7fd0c2ffd700 1 -- 172.21.15.107:0/2125627181 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd0c400f6f0 con 0x7fd0d40fbd40 2024-05-17T21:48:58.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:58.692+0000 7fd0d8ff5700 1 -- 172.21.15.107:0/2125627181 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0d41132d0 con 0x7fd0d40fbd40 2024-05-17T21:48:58.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:58.692+0000 7fd0c2ffd700 1 -- 172.21.15.107:0/2125627181 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd0c400fd20 con 0x7fd0d40fbd40 2024-05-17T21:48:58.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:58.692+0000 7fd0c2ffd700 1 -- 172.21.15.107:0/2125627181 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd0c40103a0 con 0x7fd0d40fbd40 2024-05-17T21:48:58.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:58.693+0000 7fd0d8ff5700 1 -- 172.21.15.107:0/2125627181 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0d4113890 con 0x7fd0d40fbd40 2024-05-17T21:48:58.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:58.694+0000 7fd0d8ff5700 1 -- 172.21.15.107:0/2125627181 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0d405f5c0 con 0x7fd0d40fbd40 2024-05-17T21:48:58.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:58.694+0000 7fd0c2ffd700 1 -- 172.21.15.107:0/2125627181 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7fd0c4011400 con 0x7fd0d40fbd40 2024-05-17T21:48:58.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:58.694+0000 7fd0c2ffd700 1 --2- 172.21.15.107:0/2125627181 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fd0bc0692a0 0x7fd0bc06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:48:58.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:58.694+0000 7fd0c2ffd700 1 -- 172.21.15.107:0/2125627181 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fd0c4086bf0 con 0x7fd0d40fbd40 2024-05-17T21:48:58.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:58.695+0000 7fd0d1d9b700 1 --2- 172.21.15.107:0/2125627181 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fd0bc0692a0 0x7fd0bc06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:48:58.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:58.695+0000 7fd0d1d9b700 1 --2- 172.21.15.107:0/2125627181 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fd0bc0692a0 0x7fd0bc06b750 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd0c8000c00 tx=0x7fd0c8009e30 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:48:58.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:58.699+0000 7fd0c2ffd700 1 -- 172.21.15.107:0/2125627181 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fd0c4053300 con 0x7fd0d40fbd40 2024-05-17T21:48:58.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:58.891+0000 7fd0d8ff5700 1 -- 172.21.15.107:0/2125627181 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/use_repo_digest}] v 0) v1 -- 0x7fd0d404bed0 con 0x7fd0d40fbd40 2024-05-17T21:48:58.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:58.896+0000 7fd0c2ffd700 1 -- 172.21.15.107:0/2125627181 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/use_repo_digest}]=0 v20)=0 v20) v1 ==== 143+0+0 (secure 0 0 0) 0x7fd0c4056920 con 0x7fd0d40fbd40 2024-05-17T21:48:58.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:58.899+0000 7fd0d8ff5700 1 -- 172.21.15.107:0/2125627181 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fd0bc0692a0 msgr2=0x7fd0bc06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:48:58.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:58.899+0000 7fd0d8ff5700 1 --2- 172.21.15.107:0/2125627181 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fd0bc0692a0 0x7fd0bc06b750 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd0c8000c00 tx=0x7fd0c8009e30 comp rx=0 tx=0).stop 2024-05-17T21:48:58.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:58.899+0000 7fd0d8ff5700 1 -- 172.21.15.107:0/2125627181 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0d40fbd40 msgr2=0x7fd0d4110720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:48:58.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:58.899+0000 7fd0d8ff5700 1 --2- 172.21.15.107:0/2125627181 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0d40fbd40 0x7fd0d4110720 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd0c400b7c0 tx=0x7fd0c4009250 comp rx=0 tx=0).stop 2024-05-17T21:48:58.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:58.899+0000 7fd0d8ff5700 1 -- 172.21.15.107:0/2125627181 shutdown_connections 2024-05-17T21:48:58.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:58.899+0000 7fd0d8ff5700 1 --2- 172.21.15.107:0/2125627181 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fd0bc0692a0 0x7fd0bc06b750 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:58.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:58.899+0000 7fd0d8ff5700 1 --2- 172.21.15.107:0/2125627181 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0d40fbd40 0x7fd0d4110720 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:58.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:58.899+0000 7fd0d8ff5700 1 --2- 172.21.15.107:0/2125627181 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd0d40fcff0 0x7fd0d4112c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:48:58.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:58.899+0000 7fd0d8ff5700 1 -- 172.21.15.107:0/2125627181 >> 172.21.15.107:0/2125627181 conn(0x7fd0d40f74c0 msgr2=0x7fd0d40f8ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:48:58.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:58.899+0000 7fd0d8ff5700 1 -- 172.21.15.107:0/2125627181 shutdown_connections 2024-05-17T21:48:58.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:48:58.899+0000 7fd0d8ff5700 1 -- 172.21.15.107:0/2125627181 wait complete. 2024-05-17T21:48:59.613 INFO:teuthology.run_tasks:Running task print... 2024-05-17T21:48:59.622 INFO:teuthology.task.print:**** done cephadm.shell ceph config set mgr... 2024-05-17T21:48:59.622 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-17T21:48:59.631 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi107.front.sepia.ceph.com 2024-05-17T21:48:59.632 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -- bash -c 'ceph orch status' 2024-05-17T21:48:59.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:48:59 smithi107 ceph-mon[28434]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-17T21:48:59.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:48:59 smithi107 ceph-mon[28434]: from='client.? 172.21.15.107:0/2125627181' entity='client.admin' 2024-05-17T21:48:59.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:48:59 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:48:59.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:48:59 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:48:59.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:48:59 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T21:48:59.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:48:59 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:48:59.892 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/mon.smithi107/config 2024-05-17T21:49:00.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:48:59 smithi138 ceph-mon[35948]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-17T21:49:00.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:48:59 smithi138 ceph-mon[35948]: from='client.? 172.21.15.107:0/2125627181' entity='client.admin' 2024-05-17T21:49:00.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:48:59 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:49:00.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:48:59 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:49:00.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:48:59 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T21:49:00.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:48:59 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:49:01.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:01.424+0000 7ff4d4378700 1 -- 172.21.15.107:0/2705262396 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff4cc0fb420 msgr2=0x7ff4cc0fb830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:49:01.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:01.424+0000 7ff4d4378700 1 --2- 172.21.15.107:0/2705262396 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff4cc0fb420 0x7ff4cc0fb830 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4004a50 tx=0x7ff4c400af40 comp rx=0 tx=0).stop 2024-05-17T21:49:01.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:01.424+0000 7ff4d4378700 1 -- 172.21.15.107:0/2705262396 shutdown_connections 2024-05-17T21:49:01.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:01.424+0000 7ff4d4378700 1 --2- 172.21.15.107:0/2705262396 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4cc0fc6d0 0x7ff4cc0fcb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:49:01.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:01.424+0000 7ff4d4378700 1 --2- 172.21.15.107:0/2705262396 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff4cc0fb420 0x7ff4cc0fb830 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:49:01.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:01.424+0000 7ff4d4378700 1 -- 172.21.15.107:0/2705262396 >> 172.21.15.107:0/2705262396 conn(0x7ff4cc0f6b80 msgr2=0x7ff4cc0f8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:49:01.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:01.424+0000 7ff4d4378700 1 -- 172.21.15.107:0/2705262396 shutdown_connections 2024-05-17T21:49:01.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:01.424+0000 7ff4d4378700 1 -- 172.21.15.107:0/2705262396 wait complete. 2024-05-17T21:49:01.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:01.425+0000 7ff4d4378700 1 Processor -- start 2024-05-17T21:49:01.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:01.425+0000 7ff4d4378700 1 -- start start 2024-05-17T21:49:01.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:01.425+0000 7ff4d4378700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4cc0fb420 0x7ff4cc10d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:49:01.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:01.425+0000 7ff4d4378700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff4cc0fc6d0 0x7ff4cc10def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:49:01.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:01.425+0000 7ff4d4378700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4cc10e580 con 0x7ff4cc0fc6d0 2024-05-17T21:49:01.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:01.425+0000 7ff4d4378700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4cc10ed70 con 0x7ff4cc0fb420 2024-05-17T21:49:01.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:01.425+0000 7ff4d1913700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff4cc0fc6d0 0x7ff4cc10def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:49:01.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:01.425+0000 7ff4d1913700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff4cc0fc6d0 0x7ff4cc10def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46064/0 (socket says 172.21.15.107:46064) 2024-05-17T21:49:01.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:01.425+0000 7ff4d1913700 1 -- 172.21.15.107:0/182260377 learned_addr learned my addr 172.21.15.107:0/182260377 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:49:01.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:01.425+0000 7ff4d2114700 1 --2- 172.21.15.107:0/182260377 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4cc0fb420 0x7ff4cc10d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:49:01.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:01.425+0000 7ff4d1913700 1 -- 172.21.15.107:0/182260377 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4cc0fb420 msgr2=0x7ff4cc10d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:49:01.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:01.425+0000 7ff4d1913700 1 --2- 172.21.15.107:0/182260377 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4cc0fb420 0x7ff4cc10d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:49:01.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:01.425+0000 7ff4d1913700 1 -- 172.21.15.107:0/182260377 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4c4005040 con 0x7ff4cc0fc6d0 2024-05-17T21:49:01.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:01.425+0000 7ff4d1913700 1 --2- 172.21.15.107:0/182260377 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff4cc0fc6d0 0x7ff4cc10def0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff4c000b790 tx=0x7ff4c0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:49:01.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:01.425+0000 7ff4caffd700 1 -- 172.21.15.107:0/182260377 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff4c000f770 con 0x7ff4cc0fc6d0 2024-05-17T21:49:01.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:01.426+0000 7ff4caffd700 1 -- 172.21.15.107:0/182260377 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff4c000fda0 con 0x7ff4cc0fc6d0 2024-05-17T21:49:01.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:01.426+0000 7ff4caffd700 1 -- 172.21.15.107:0/182260377 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff4c000d310 con 0x7ff4cc0fc6d0 2024-05-17T21:49:01.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:01.426+0000 7ff4d4378700 1 -- 172.21.15.107:0/182260377 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4cc10f050 con 0x7ff4cc0fc6d0 2024-05-17T21:49:01.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:01.426+0000 7ff4d4378700 1 -- 172.21.15.107:0/182260377 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4cc10f5c0 con 0x7ff4cc0fc6d0 2024-05-17T21:49:01.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:01.427+0000 7ff4d4378700 1 -- 172.21.15.107:0/182260377 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4cc05f5c0 con 0x7ff4cc0fc6d0 2024-05-17T21:49:01.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:01.427+0000 7ff4caffd700 1 -- 172.21.15.107:0/182260377 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7ff4c0010420 con 0x7ff4cc0fc6d0 2024-05-17T21:49:01.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:01.428+0000 7ff4caffd700 1 --2- 172.21.15.107:0/182260377 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7ff4b40692a0 0x7ff4b406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:49:01.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:01.428+0000 7ff4caffd700 1 -- 172.21.15.107:0/182260377 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7ff4c0086e80 con 0x7ff4cc0fc6d0 2024-05-17T21:49:01.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:01.428+0000 7ff4d2114700 1 --2- 172.21.15.107:0/182260377 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7ff4b40692a0 0x7ff4b406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:49:01.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:01.428+0000 7ff4d2114700 1 --2- 172.21.15.107:0/182260377 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7ff4b40692a0 0x7ff4b406b750 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4000c00 tx=0x7ff4c400af40 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:49:01.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:01.432+0000 7ff4caffd700 1 -- 172.21.15.107:0/182260377 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7ff4c0056f80 con 0x7ff4cc0fc6d0 2024-05-17T21:49:01.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:01.632+0000 7ff4d4378700 1 -- 172.21.15.107:0/182260377 --> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff4cc10f960 con 0x7ff4b40692a0 2024-05-17T21:49:01.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:01.633+0000 7ff4caffd700 1 -- 172.21.15.107:0/182260377 <== mgr.14214 v2:172.21.15.107:6800/1294361014 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7ff4cc10f960 con 0x7ff4b40692a0 2024-05-17T21:49:01.625 INFO:teuthology.orchestra.run.smithi107.stdout:Backend: cephadm 2024-05-17T21:49:01.625 INFO:teuthology.orchestra.run.smithi107.stdout:Available: Yes 2024-05-17T21:49:01.625 INFO:teuthology.orchestra.run.smithi107.stdout:Paused: No 2024-05-17T21:49:01.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:01.635+0000 7ff4d4378700 1 -- 172.21.15.107:0/182260377 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7ff4b40692a0 msgr2=0x7ff4b406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:49:01.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:01.635+0000 7ff4d4378700 1 --2- 172.21.15.107:0/182260377 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7ff4b40692a0 0x7ff4b406b750 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4000c00 tx=0x7ff4c400af40 comp rx=0 tx=0).stop 2024-05-17T21:49:01.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:01.635+0000 7ff4d4378700 1 -- 172.21.15.107:0/182260377 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff4cc0fc6d0 msgr2=0x7ff4cc10def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:49:01.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:01.635+0000 7ff4d4378700 1 --2- 172.21.15.107:0/182260377 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff4cc0fc6d0 0x7ff4cc10def0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff4c000b790 tx=0x7ff4c0009250 comp rx=0 tx=0).stop 2024-05-17T21:49:01.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:01.635+0000 7ff4d4378700 1 -- 172.21.15.107:0/182260377 shutdown_connections 2024-05-17T21:49:01.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:01.635+0000 7ff4d4378700 1 --2- 172.21.15.107:0/182260377 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7ff4b40692a0 0x7ff4b406b750 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:49:01.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:01.635+0000 7ff4d4378700 1 --2- 172.21.15.107:0/182260377 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4cc0fb420 0x7ff4cc10d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:49:01.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:01.635+0000 7ff4d4378700 1 --2- 172.21.15.107:0/182260377 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff4cc0fc6d0 0x7ff4cc10def0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:49:01.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:01.635+0000 7ff4d4378700 1 -- 172.21.15.107:0/182260377 >> 172.21.15.107:0/182260377 conn(0x7ff4cc0f6b80 msgr2=0x7ff4cc0f8140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:49:01.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:01.635+0000 7ff4d4378700 1 -- 172.21.15.107:0/182260377 shutdown_connections 2024-05-17T21:49:01.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:01.635+0000 7ff4d4378700 1 -- 172.21.15.107:0/182260377 wait complete. 2024-05-17T21:49:02.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:01 smithi138 ceph-mon[35948]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-17T21:49:02.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:01 smithi107 ceph-mon[28434]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-17T21:49:02.203 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -- bash -c 'ceph orch ps' 2024-05-17T21:49:02.458 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/mon.smithi107/config 2024-05-17T21:49:02.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:02 smithi107 ceph-mon[28434]: from='client.14442 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:49:02.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:02 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:49:03.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:02 smithi138 ceph-mon[35948]: from='client.14442 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:49:03.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:02 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:49:03.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:03 smithi107 ceph-mon[28434]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-17T21:49:04.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:03 smithi138 ceph-mon[35948]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-17T21:49:04.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:04.196+0000 7fb55cda4700 1 -- 172.21.15.107:0/492681691 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb5580fdfd0 msgr2=0x7fb5580fe440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:49:04.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:04.196+0000 7fb55cda4700 1 --2- 172.21.15.107:0/492681691 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb5580fdfd0 0x7fb5580fe440 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb540004940 tx=0x7fb540009e30 comp rx=0 tx=0).stop 2024-05-17T21:49:04.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:04.196+0000 7fb55cda4700 1 -- 172.21.15.107:0/492681691 shutdown_connections 2024-05-17T21:49:04.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:04.196+0000 7fb55cda4700 1 --2- 172.21.15.107:0/492681691 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb5580fdfd0 0x7fb5580fe440 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:49:04.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:04.196+0000 7fb55cda4700 1 --2- 172.21.15.107:0/492681691 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5580fce60 0x7fb5580fd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:49:04.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:04.196+0000 7fb55cda4700 1 -- 172.21.15.107:0/492681691 >> 172.21.15.107:0/492681691 conn(0x7fb5580f8540 msgr2=0x7fb5580fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:49:04.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:04.196+0000 7fb55cda4700 1 -- 172.21.15.107:0/492681691 shutdown_connections 2024-05-17T21:49:04.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:04.196+0000 7fb55cda4700 1 -- 172.21.15.107:0/492681691 wait complete. 2024-05-17T21:49:04.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:04.197+0000 7fb55cda4700 1 Processor -- start 2024-05-17T21:49:04.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:04.197+0000 7fb55cda4700 1 -- start start 2024-05-17T21:49:04.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:04.197+0000 7fb55cda4700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5580fce60 0x7fb55810d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:49:04.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:04.197+0000 7fb55cda4700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb5580fdfd0 0x7fb55810df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:49:04.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:04.197+0000 7fb55cda4700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb55810e500 con 0x7fb5580fdfd0 2024-05-17T21:49:04.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:04.197+0000 7fb55cda4700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb55810ed00 con 0x7fb5580fce60 2024-05-17T21:49:04.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:04.197+0000 7fb54dd9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb5580fdfd0 0x7fb55810df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:49:04.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:04.197+0000 7fb54dd9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb5580fdfd0 0x7fb55810df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46082/0 (socket says 172.21.15.107:46082) 2024-05-17T21:49:04.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:04.197+0000 7fb54dd9b700 1 -- 172.21.15.107:0/799060466 learned_addr learned my addr 172.21.15.107:0/799060466 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:49:04.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:04.197+0000 7fb55659c700 1 --2- 172.21.15.107:0/799060466 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5580fce60 0x7fb55810d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:49:04.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:04.197+0000 7fb54dd9b700 1 -- 172.21.15.107:0/799060466 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5580fce60 msgr2=0x7fb55810d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:49:04.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:04.197+0000 7fb54dd9b700 1 --2- 172.21.15.107:0/799060466 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5580fce60 0x7fb55810d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:49:04.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:04.197+0000 7fb54dd9b700 1 -- 172.21.15.107:0/799060466 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb540005040 con 0x7fb5580fdfd0 2024-05-17T21:49:04.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:04.197+0000 7fb54dd9b700 1 --2- 172.21.15.107:0/799060466 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb5580fdfd0 0x7fb55810df00 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fb54000b3b0 tx=0x7fb540009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:49:04.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:04.198+0000 7fb54f7fe700 1 -- 172.21.15.107:0/799060466 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb54000e040 con 0x7fb5580fdfd0 2024-05-17T21:49:04.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:04.198+0000 7fb54f7fe700 1 -- 172.21.15.107:0/799060466 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb54000bbf0 con 0x7fb5580fdfd0 2024-05-17T21:49:04.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:04.198+0000 7fb55cda4700 1 -- 172.21.15.107:0/799060466 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb55810ef80 con 0x7fb5580fdfd0 2024-05-17T21:49:04.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:04.198+0000 7fb54f7fe700 1 -- 172.21.15.107:0/799060466 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb54001c630 con 0x7fb5580fdfd0 2024-05-17T21:49:04.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:04.198+0000 7fb55cda4700 1 -- 172.21.15.107:0/799060466 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb55810f4c0 con 0x7fb5580fdfd0 2024-05-17T21:49:04.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:04.199+0000 7fb55cda4700 1 -- 172.21.15.107:0/799060466 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb55805f5c0 con 0x7fb5580fdfd0 2024-05-17T21:49:04.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:04.199+0000 7fb54f7fe700 1 -- 172.21.15.107:0/799060466 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7fb540013420 con 0x7fb5580fdfd0 2024-05-17T21:49:04.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:04.199+0000 7fb54f7fe700 1 --2- 172.21.15.107:0/799060466 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fb5440692a0 0x7fb54406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:49:04.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:04.199+0000 7fb54f7fe700 1 -- 172.21.15.107:0/799060466 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fb540087b30 con 0x7fb5580fdfd0 2024-05-17T21:49:04.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:04.199+0000 7fb55659c700 1 --2- 172.21.15.107:0/799060466 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fb5440692a0 0x7fb54406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:49:04.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:04.200+0000 7fb55659c700 1 --2- 172.21.15.107:0/799060466 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fb5440692a0 0x7fb54406b750 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb548009fd0 tx=0x7fb54800b040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:49:04.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:04.204+0000 7fb54f7fe700 1 -- 172.21.15.107:0/799060466 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fb540057c30 con 0x7fb5580fdfd0 2024-05-17T21:49:04.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:04.396+0000 7fb55cda4700 1 -- 172.21.15.107:0/799060466 --> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fb55810f7f0 con 0x7fb5440692a0 2024-05-17T21:49:04.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:04.404+0000 7fb54f7fe700 1 -- 172.21.15.107:0/799060466 <== mgr.14214 v2:172.21.15.107:6800/1294361014 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+2682 (secure 0 0 0) 0x7fb55810f7f0 con 0x7fb5440692a0 2024-05-17T21:49:04.396 INFO:teuthology.orchestra.run.smithi107.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-17T21:49:04.396 INFO:teuthology.orchestra.run.smithi107.stdout:alertmanager.smithi107 smithi107 *:9093,9094 running (3m) 2m ago 5m 26.2M - 0.25.0 c8568f914cd2 a2d817bf64d3 2024-05-17T21:49:04.397 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi107 smithi107 running (5m) 2m ago 5m 6660k - 17.2.7-952-g9903f776 d15a36853796 828741c3a89c 2024-05-17T21:49:04.397 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi138 smithi138 running (4m) 68s ago 4m 6660k - 17.2.7-952-g9903f776 d15a36853796 20e9ccf1fadd 2024-05-17T21:49:04.397 INFO:teuthology.orchestra.run.smithi107.stdout:grafana.smithi107 smithi107 *:3000 running (3m) 2m ago 4m 73.8M - 9.4.7 954c08fa6188 0bc11311fb71 2024-05-17T21:49:04.397 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi107.xhhyiw smithi107 *:9283 running (6m) 2m ago 6m 474M - 17.2.7-952-g9903f776 d15a36853796 b566f08938ae 2024-05-17T21:49:04.397 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi138.napnll smithi138 *:8443,9283 running (4m) 68s ago 4m 435M - 17.2.7-952-g9903f776 d15a36853796 b0a8c6e63a55 2024-05-17T21:49:04.397 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi107 smithi107 running (6m) 2m ago 6m 42.6M 2048M 17.2.7-952-g9903f776 d15a36853796 7de9c23eee97 2024-05-17T21:49:04.397 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi138 smithi138 running (4m) 68s ago 4m 38.4M 2048M 17.2.7-952-g9903f776 d15a36853796 ee61a065778d 2024-05-17T21:49:04.397 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi107 smithi107 *:9100 running (5m) 2m ago 5m 16.0M - 1.5.0 0da6a335fe13 730e9bfe00b2 2024-05-17T21:49:04.397 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi138 smithi138 *:9100 running (4m) 68s ago 4m 15.6M - 1.5.0 0da6a335fe13 296b996124a1 2024-05-17T21:49:04.397 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0 smithi107 running (3m) 2m ago 3m 39.6M 4288M 17.2.7-952-g9903f776 d15a36853796 f2d20009361d 2024-05-17T21:49:04.397 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1 smithi107 running (2m) 2m ago 2m 40.9M 4288M 17.2.7-952-g9903f776 d15a36853796 e5d3e99844a9 2024-05-17T21:49:04.397 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2 smithi107 running (2m) 2m ago 2m 40.5M 4288M 17.2.7-952-g9903f776 d15a36853796 e044d7b78088 2024-05-17T21:49:04.397 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3 smithi138 running (2m) 68s ago 2m 38.7M 5312M 17.2.7-952-g9903f776 d15a36853796 8c980cc8af87 2024-05-17T21:49:04.398 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4 smithi138 running (99s) 68s ago 98s 36.2M 5312M 17.2.7-952-g9903f776 d15a36853796 b306a36a744e 2024-05-17T21:49:04.398 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5 smithi138 running (73s) 68s ago 73s 13.4M 5312M 17.2.7-952-g9903f776 d15a36853796 7c8d8ed74974 2024-05-17T21:49:04.398 INFO:teuthology.orchestra.run.smithi107.stdout:prometheus.smithi107 smithi107 *:9095 running (3m) 2m ago 4m 35.8M - 2.43.0 a07b618ecd1d 38259629b8f2 2024-05-17T21:49:04.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:04.406+0000 7fb55cda4700 1 -- 172.21.15.107:0/799060466 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fb5440692a0 msgr2=0x7fb54406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:49:04.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:04.406+0000 7fb55cda4700 1 --2- 172.21.15.107:0/799060466 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fb5440692a0 0x7fb54406b750 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb548009fd0 tx=0x7fb54800b040 comp rx=0 tx=0).stop 2024-05-17T21:49:04.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:04.406+0000 7fb55cda4700 1 -- 172.21.15.107:0/799060466 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb5580fdfd0 msgr2=0x7fb55810df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:49:04.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:04.406+0000 7fb55cda4700 1 --2- 172.21.15.107:0/799060466 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb5580fdfd0 0x7fb55810df00 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fb54000b3b0 tx=0x7fb540009e30 comp rx=0 tx=0).stop 2024-05-17T21:49:04.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:04.407+0000 7fb55cda4700 1 -- 172.21.15.107:0/799060466 shutdown_connections 2024-05-17T21:49:04.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:04.407+0000 7fb55cda4700 1 --2- 172.21.15.107:0/799060466 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fb5440692a0 0x7fb54406b750 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:49:04.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:04.407+0000 7fb55cda4700 1 --2- 172.21.15.107:0/799060466 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5580fce60 0x7fb55810d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:49:04.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:04.407+0000 7fb55cda4700 1 --2- 172.21.15.107:0/799060466 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb5580fdfd0 0x7fb55810df00 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:49:04.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:04.407+0000 7fb55cda4700 1 -- 172.21.15.107:0/799060466 >> 172.21.15.107:0/799060466 conn(0x7fb5580f8540 msgr2=0x7fb5580f9b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:49:04.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:04.407+0000 7fb55cda4700 1 -- 172.21.15.107:0/799060466 shutdown_connections 2024-05-17T21:49:04.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:04.407+0000 7fb55cda4700 1 -- 172.21.15.107:0/799060466 wait complete. 2024-05-17T21:49:05.043 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -- bash -c 'ceph orch ls' 2024-05-17T21:49:05.298 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/mon.smithi107/config 2024-05-17T21:49:06.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:05 smithi138 ceph-mon[35948]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-17T21:49:06.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:05 smithi138 ceph-mon[35948]: from='client.14446 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:49:06.097 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:05 smithi107 ceph-mon[28434]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-17T21:49:06.097 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:05 smithi107 ceph-mon[28434]: from='client.14446 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:49:06.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:06.833+0000 7f4254480700 1 -- 172.21.15.107:0/2029903620 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f424c0fd2f0 msgr2=0x7f424c0fd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:49:06.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:06.833+0000 7f4254480700 1 --2- 172.21.15.107:0/2029903620 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f424c0fd2f0 0x7f424c0fd700 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f423c004940 tx=0x7f423c009e30 comp rx=0 tx=0).stop 2024-05-17T21:49:06.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:06.833+0000 7f4254480700 1 -- 172.21.15.107:0/2029903620 shutdown_connections 2024-05-17T21:49:06.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:06.833+0000 7f4254480700 1 --2- 172.21.15.107:0/2029903620 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f424c0fdc40 0x7f424c0fc050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:49:06.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:06.833+0000 7f4254480700 1 --2- 172.21.15.107:0/2029903620 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f424c0fd2f0 0x7f424c0fd700 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:49:06.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:06.833+0000 7f4254480700 1 -- 172.21.15.107:0/2029903620 >> 172.21.15.107:0/2029903620 conn(0x7f424c0f6bd0 msgr2=0x7f424c0f9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:49:06.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:06.833+0000 7f4254480700 1 -- 172.21.15.107:0/2029903620 shutdown_connections 2024-05-17T21:49:06.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:06.833+0000 7f4254480700 1 -- 172.21.15.107:0/2029903620 wait complete. 2024-05-17T21:49:06.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:06.834+0000 7f4254480700 1 Processor -- start 2024-05-17T21:49:06.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:06.834+0000 7f4254480700 1 -- start start 2024-05-17T21:49:06.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:06.834+0000 7f4254480700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f424c0fd2f0 0x7f424c10d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:49:06.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:06.834+0000 7f4254480700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f424c0fdc40 0x7f424c10dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:49:06.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:06.834+0000 7f4254480700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f424c10e570 con 0x7f424c0fdc40 2024-05-17T21:49:06.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:06.834+0000 7f4254480700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f424c112f10 con 0x7f424c0fd2f0 2024-05-17T21:49:06.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:06.834+0000 7f4251a1b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f424c0fdc40 0x7f424c10dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:49:06.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:06.834+0000 7f4251a1b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f424c0fdc40 0x7f424c10dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43226/0 (socket says 172.21.15.107:43226) 2024-05-17T21:49:06.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:06.834+0000 7f4251a1b700 1 -- 172.21.15.107:0/556007307 learned_addr learned my addr 172.21.15.107:0/556007307 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:49:06.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:06.834+0000 7f425221c700 1 --2- 172.21.15.107:0/556007307 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f424c0fd2f0 0x7f424c10d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:49:06.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:06.834+0000 7f4251a1b700 1 -- 172.21.15.107:0/556007307 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f424c0fd2f0 msgr2=0x7f424c10d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:49:06.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:06.834+0000 7f4251a1b700 1 --2- 172.21.15.107:0/556007307 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f424c0fd2f0 0x7f424c10d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:49:06.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:06.834+0000 7f4251a1b700 1 -- 172.21.15.107:0/556007307 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f423c005040 con 0x7f424c0fdc40 2024-05-17T21:49:06.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:06.834+0000 7f425221c700 1 --2- 172.21.15.107:0/556007307 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f424c0fd2f0 0x7f424c10d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:49:06.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:06.834+0000 7f4251a1b700 1 --2- 172.21.15.107:0/556007307 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f424c0fdc40 0x7f424c10dee0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f424800b790 tx=0x7f4248009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:49:06.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:06.835+0000 7f4242ffd700 1 -- 172.21.15.107:0/556007307 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f424800f6f0 con 0x7f424c0fdc40 2024-05-17T21:49:06.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:06.835+0000 7f4254480700 1 -- 172.21.15.107:0/556007307 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f424c113170 con 0x7f424c0fdc40 2024-05-17T21:49:06.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:06.835+0000 7f4242ffd700 1 -- 172.21.15.107:0/556007307 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f424800fd20 con 0x7f424c0fdc40 2024-05-17T21:49:06.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:06.835+0000 7f4242ffd700 1 -- 172.21.15.107:0/556007307 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f42480103a0 con 0x7f424c0fdc40 2024-05-17T21:49:06.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:06.835+0000 7f4254480700 1 -- 172.21.15.107:0/556007307 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f424c1136e0 con 0x7f424c0fdc40 2024-05-17T21:49:06.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:06.836+0000 7f4254480700 1 -- 172.21.15.107:0/556007307 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f424c101bf0 con 0x7f424c0fdc40 2024-05-17T21:49:06.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:06.836+0000 7f4242ffd700 1 -- 172.21.15.107:0/556007307 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f4248011400 con 0x7f424c0fdc40 2024-05-17T21:49:06.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:06.837+0000 7f4242ffd700 1 --2- 172.21.15.107:0/556007307 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f42380692a0 0x7f423806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:49:06.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:06.837+0000 7f425221c700 1 --2- 172.21.15.107:0/556007307 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f42380692a0 0x7f423806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:49:06.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:06.837+0000 7f4242ffd700 1 -- 172.21.15.107:0/556007307 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f4248086d20 con 0x7f424c0fdc40 2024-05-17T21:49:06.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:06.837+0000 7f425221c700 1 --2- 172.21.15.107:0/556007307 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f42380692a0 0x7f423806b750 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f423c00b7a0 tx=0x7f423c009e30 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:49:06.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:06.841+0000 7f4242ffd700 1 -- 172.21.15.107:0/556007307 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f4248056da0 con 0x7f424c0fdc40 2024-05-17T21:49:07.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:07.033+0000 7f4254480700 1 -- 172.21.15.107:0/556007307 --> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f424c02b390 con 0x7f42380692a0 2024-05-17T21:49:07.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:07.037+0000 7f4242ffd700 1 -- 172.21.15.107:0/556007307 <== mgr.14214 v2:172.21.15.107:6800/1294361014 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+1179 (secure 0 0 0) 0x7f424c02b390 con 0x7f42380692a0 2024-05-17T21:49:07.030 INFO:teuthology.orchestra.run.smithi107.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-05-17T21:49:07.030 INFO:teuthology.orchestra.run.smithi107.stdout:alertmanager ?:9093,9094 1/1 2m ago 5m count:1 2024-05-17T21:49:07.030 INFO:teuthology.orchestra.run.smithi107.stdout:crash 2/2 2m ago 6m * 2024-05-17T21:49:07.030 INFO:teuthology.orchestra.run.smithi107.stdout:grafana ?:3000 1/1 2m ago 5m count:1 2024-05-17T21:49:07.030 INFO:teuthology.orchestra.run.smithi107.stdout:mgr 2/2 2m ago 6m count:2 2024-05-17T21:49:07.030 INFO:teuthology.orchestra.run.smithi107.stdout:mon 2/2 2m ago 5m smithi107:172.21.15.107=smithi107;smithi138:172.21.15.138=smithi138;count:2 2024-05-17T21:49:07.030 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter ?:9100 2/2 2m ago 5m * 2024-05-17T21:49:07.030 INFO:teuthology.orchestra.run.smithi107.stdout:osd 6 2m ago - 2024-05-17T21:49:07.030 INFO:teuthology.orchestra.run.smithi107.stdout:prometheus ?:9095 1/1 2m ago 6m count:1 2024-05-17T21:49:07.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:07.039+0000 7f4254480700 1 -- 172.21.15.107:0/556007307 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f42380692a0 msgr2=0x7f423806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:49:07.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:07.039+0000 7f4254480700 1 --2- 172.21.15.107:0/556007307 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f42380692a0 0x7f423806b750 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f423c00b7a0 tx=0x7f423c009e30 comp rx=0 tx=0).stop 2024-05-17T21:49:07.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:07.039+0000 7f4254480700 1 -- 172.21.15.107:0/556007307 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f424c0fdc40 msgr2=0x7f424c10dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:49:07.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:07.039+0000 7f4254480700 1 --2- 172.21.15.107:0/556007307 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f424c0fdc40 0x7f424c10dee0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f424800b790 tx=0x7f4248009220 comp rx=0 tx=0).stop 2024-05-17T21:49:07.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:07.040+0000 7f4254480700 1 -- 172.21.15.107:0/556007307 shutdown_connections 2024-05-17T21:49:07.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:07.040+0000 7f4254480700 1 --2- 172.21.15.107:0/556007307 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f42380692a0 0x7f423806b750 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:49:07.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:07.040+0000 7f4254480700 1 --2- 172.21.15.107:0/556007307 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f424c0fd2f0 0x7f424c10d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:49:07.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:07.040+0000 7f4254480700 1 --2- 172.21.15.107:0/556007307 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f424c0fdc40 0x7f424c10dee0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:49:07.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:07.040+0000 7f4254480700 1 -- 172.21.15.107:0/556007307 >> 172.21.15.107:0/556007307 conn(0x7f424c0f6bd0 msgr2=0x7f424c0f8210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:49:07.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:07.040+0000 7f4254480700 1 -- 172.21.15.107:0/556007307 shutdown_connections 2024-05-17T21:49:07.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:07.040+0000 7f4254480700 1 -- 172.21.15.107:0/556007307 wait complete. 2024-05-17T21:49:07.872 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:07 smithi107 ceph-mon[28434]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-17T21:49:07.901 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -- bash -c 'ceph orch host ls' 2024-05-17T21:49:08.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:07 smithi138 ceph-mon[35948]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-17T21:49:08.159 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/mon.smithi107/config 2024-05-17T21:49:09.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:08 smithi138 ceph-mon[35948]: from='client.14450 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:49:09.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:08 smithi107 ceph-mon[28434]: from='client.14450 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:49:09.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:09.831+0000 7f74bd241700 1 -- 172.21.15.107:0/2091330864 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74b80faa10 msgr2=0x7f74b80fae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:49:09.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:09.831+0000 7f74bd241700 1 --2- 172.21.15.107:0/2091330864 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74b80faa10 0x7f74b80fae20 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f74a8004940 tx=0x7f74a8009e30 comp rx=0 tx=0).stop 2024-05-17T21:49:09.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:09.831+0000 7f74bd241700 1 -- 172.21.15.107:0/2091330864 shutdown_connections 2024-05-17T21:49:09.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:09.831+0000 7f74bd241700 1 --2- 172.21.15.107:0/2091330864 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74b80fb360 0x7f74b81025e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:49:09.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:09.831+0000 7f74bd241700 1 --2- 172.21.15.107:0/2091330864 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74b80faa10 0x7f74b80fae20 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:49:09.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:09.831+0000 7f74bd241700 1 -- 172.21.15.107:0/2091330864 >> 172.21.15.107:0/2091330864 conn(0x7f74b80f6740 msgr2=0x7f74b80f8b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:49:09.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:09.831+0000 7f74bd241700 1 -- 172.21.15.107:0/2091330864 shutdown_connections 2024-05-17T21:49:09.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:09.831+0000 7f74bd241700 1 -- 172.21.15.107:0/2091330864 wait complete. 2024-05-17T21:49:09.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:09.832+0000 7f74bd241700 1 Processor -- start 2024-05-17T21:49:09.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:09.832+0000 7f74bd241700 1 -- start start 2024-05-17T21:49:09.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:09.832+0000 7f74bd241700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74b80faa10 0x7f74b810d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:49:09.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:09.832+0000 7f74bd241700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74b80fb360 0x7f74b810de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:49:09.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:09.832+0000 7f74bd241700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74b810e480 con 0x7f74b80faa10 2024-05-17T21:49:09.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:09.832+0000 7f74bd241700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74b810e5f0 con 0x7f74b80fb360 2024-05-17T21:49:09.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:09.832+0000 7f74b6d9d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74b80faa10 0x7f74b810d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:49:09.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:09.832+0000 7f74b6d9d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74b80faa10 0x7f74b810d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43248/0 (socket says 172.21.15.107:43248) 2024-05-17T21:49:09.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:09.832+0000 7f74b6d9d700 1 -- 172.21.15.107:0/4241403434 learned_addr learned my addr 172.21.15.107:0/4241403434 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:49:09.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:09.832+0000 7f74b659c700 1 --2- 172.21.15.107:0/4241403434 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74b80fb360 0x7f74b810de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:49:09.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:09.833+0000 7f74b6d9d700 1 -- 172.21.15.107:0/4241403434 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74b80fb360 msgr2=0x7f74b810de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:49:09.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:09.833+0000 7f74b6d9d700 1 --2- 172.21.15.107:0/4241403434 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74b80fb360 0x7f74b810de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:49:09.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:09.833+0000 7f74b6d9d700 1 -- 172.21.15.107:0/4241403434 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74a8005040 con 0x7f74b80faa10 2024-05-17T21:49:09.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:09.833+0000 7f74b659c700 1 --2- 172.21.15.107:0/4241403434 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74b80fb360 0x7f74b810de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:49:09.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:09.833+0000 7f74b6d9d700 1 --2- 172.21.15.107:0/4241403434 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74b80faa10 0x7f74b810d940 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f74a8000c00 tx=0x7f74a8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:49:09.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:09.833+0000 7f749f7fe700 1 -- 172.21.15.107:0/4241403434 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f74a800e040 con 0x7f74b80faa10 2024-05-17T21:49:09.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:09.833+0000 7f749f7fe700 1 -- 172.21.15.107:0/4241403434 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f74a800bcc0 con 0x7f74b80faa10 2024-05-17T21:49:09.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:09.833+0000 7f74bd241700 1 -- 172.21.15.107:0/4241403434 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74b8112ff0 con 0x7f74b80faa10 2024-05-17T21:49:09.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:09.833+0000 7f749f7fe700 1 -- 172.21.15.107:0/4241403434 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f74a801c630 con 0x7f74b80faa10 2024-05-17T21:49:09.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:09.833+0000 7f74bd241700 1 -- 172.21.15.107:0/4241403434 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74b8113520 con 0x7f74b80faa10 2024-05-17T21:49:09.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:09.834+0000 7f74bd241700 1 -- 172.21.15.107:0/4241403434 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74b804c710 con 0x7f74b80faa10 2024-05-17T21:49:09.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:09.834+0000 7f749f7fe700 1 -- 172.21.15.107:0/4241403434 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f74a8013980 con 0x7f74b80faa10 2024-05-17T21:49:09.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:09.835+0000 7f749f7fe700 1 --2- 172.21.15.107:0/4241403434 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f74a00692a0 0x7f74a006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:49:09.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:09.835+0000 7f749f7fe700 1 -- 172.21.15.107:0/4241403434 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f74a80881e0 con 0x7f74b80faa10 2024-05-17T21:49:09.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:09.835+0000 7f74b659c700 1 --2- 172.21.15.107:0/4241403434 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f74a00692a0 0x7f74a006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:49:09.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:09.835+0000 7f74b659c700 1 --2- 172.21.15.107:0/4241403434 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f74a00692a0 0x7f74a006b750 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f74ac007fd0 tx=0x7f74ac009040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:49:09.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:09.839+0000 7f749f7fe700 1 -- 172.21.15.107:0/4241403434 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f74a8058390 con 0x7f74b80faa10 2024-05-17T21:49:10.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:10.031+0000 7f74bd241700 1 -- 172.21.15.107:0/4241403434 --> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f74b8101e40 con 0x7f74a00692a0 2024-05-17T21:49:10.024 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:09 smithi107 ceph-mon[28434]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-17T21:49:10.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:10.032+0000 7f749f7fe700 1 -- 172.21.15.107:0/4241403434 <== mgr.14214 v2:172.21.15.107:6800/1294361014 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+148 (secure 0 0 0) 0x7f74b8101e40 con 0x7f74a00692a0 2024-05-17T21:49:10.024 INFO:teuthology.orchestra.run.smithi107.stdout:HOST ADDR LABELS STATUS 2024-05-17T21:49:10.025 INFO:teuthology.orchestra.run.smithi107.stdout:smithi107 172.21.15.107 2024-05-17T21:49:10.025 INFO:teuthology.orchestra.run.smithi107.stdout:smithi138 172.21.15.138 2024-05-17T21:49:10.025 INFO:teuthology.orchestra.run.smithi107.stdout:2 hosts in cluster 2024-05-17T21:49:10.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:10.034+0000 7f74bd241700 1 -- 172.21.15.107:0/4241403434 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f74a00692a0 msgr2=0x7f74a006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:49:10.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:10.034+0000 7f74bd241700 1 --2- 172.21.15.107:0/4241403434 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f74a00692a0 0x7f74a006b750 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f74ac007fd0 tx=0x7f74ac009040 comp rx=0 tx=0).stop 2024-05-17T21:49:10.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:10.034+0000 7f74bd241700 1 -- 172.21.15.107:0/4241403434 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74b80faa10 msgr2=0x7f74b810d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:49:10.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:10.034+0000 7f74bd241700 1 --2- 172.21.15.107:0/4241403434 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74b80faa10 0x7f74b810d940 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f74a8000c00 tx=0x7f74a8009e30 comp rx=0 tx=0).stop 2024-05-17T21:49:10.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:10.035+0000 7f74bd241700 1 -- 172.21.15.107:0/4241403434 shutdown_connections 2024-05-17T21:49:10.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:10.035+0000 7f74bd241700 1 --2- 172.21.15.107:0/4241403434 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f74a00692a0 0x7f74a006b750 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:49:10.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:10.035+0000 7f74bd241700 1 --2- 172.21.15.107:0/4241403434 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74b80faa10 0x7f74b810d940 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:49:10.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:10.035+0000 7f74bd241700 1 --2- 172.21.15.107:0/4241403434 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74b80fb360 0x7f74b810de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:49:10.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:10.035+0000 7f74bd241700 1 -- 172.21.15.107:0/4241403434 >> 172.21.15.107:0/4241403434 conn(0x7f74b80f6740 msgr2=0x7f74b8100720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:49:10.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:10.035+0000 7f74bd241700 1 -- 172.21.15.107:0/4241403434 shutdown_connections 2024-05-17T21:49:10.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:10.035+0000 7f74bd241700 1 -- 172.21.15.107:0/4241403434 wait complete. 2024-05-17T21:49:10.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:09 smithi138 ceph-mon[35948]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-17T21:49:10.670 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -- bash -c 'ceph orch device ls' 2024-05-17T21:49:10.928 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/mon.smithi107/config 2024-05-17T21:49:12.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:11 smithi138 ceph-mon[35948]: from='client.14454 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:49:12.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:11 smithi138 ceph-mon[35948]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-17T21:49:12.089 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:11 smithi107 ceph-mon[28434]: from='client.14454 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:49:12.089 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:11 smithi107 ceph-mon[28434]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-17T21:49:12.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:12.462+0000 7f245a041700 1 -- 172.21.15.107:0/2176984809 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24540fdc00 msgr2=0x7f24540fc020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:49:12.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:12.462+0000 7f245a041700 1 --2- 172.21.15.107:0/2176984809 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24540fdc00 0x7f24540fc020 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f2448004940 tx=0x7f2448009e30 comp rx=0 tx=0).stop 2024-05-17T21:49:12.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:12.462+0000 7f245a041700 1 -- 172.21.15.107:0/2176984809 shutdown_connections 2024-05-17T21:49:12.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:12.462+0000 7f245a041700 1 --2- 172.21.15.107:0/2176984809 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24540fdc00 0x7f24540fc020 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:49:12.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:12.462+0000 7f245a041700 1 --2- 172.21.15.107:0/2176984809 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f24540fd2b0 0x7f24540fd6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:49:12.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:12.462+0000 7f245a041700 1 -- 172.21.15.107:0/2176984809 >> 172.21.15.107:0/2176984809 conn(0x7f24540f6be0 msgr2=0x7f24540f8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:49:12.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:12.462+0000 7f245a041700 1 -- 172.21.15.107:0/2176984809 shutdown_connections 2024-05-17T21:49:12.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:12.462+0000 7f245a041700 1 -- 172.21.15.107:0/2176984809 wait complete. 2024-05-17T21:49:12.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:12.463+0000 7f245a041700 1 Processor -- start 2024-05-17T21:49:12.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:12.463+0000 7f245a041700 1 -- start start 2024-05-17T21:49:12.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:12.463+0000 7f245a041700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f24540fd2b0 0x7f245410c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:49:12.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:12.463+0000 7f245a041700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f245410cd70 0x7f2454111da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:49:12.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:12.463+0000 7f245a041700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f245410d2c0 con 0x7f245410cd70 2024-05-17T21:49:12.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:12.463+0000 7f245a041700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f245410d400 con 0x7f24540fd2b0 2024-05-17T21:49:12.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:12.463+0000 7f2452ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f245410cd70 0x7f2454111da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:49:12.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:12.463+0000 7f2452ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f245410cd70 0x7f2454111da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43272/0 (socket says 172.21.15.107:43272) 2024-05-17T21:49:12.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:12.463+0000 7f2452ffd700 1 -- 172.21.15.107:0/3187952239 learned_addr learned my addr 172.21.15.107:0/3187952239 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:49:12.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:12.463+0000 7f24537fe700 1 --2- 172.21.15.107:0/3187952239 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f24540fd2b0 0x7f245410c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:49:12.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:12.464+0000 7f2452ffd700 1 -- 172.21.15.107:0/3187952239 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f24540fd2b0 msgr2=0x7f245410c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:49:12.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:12.464+0000 7f2452ffd700 1 --2- 172.21.15.107:0/3187952239 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f24540fd2b0 0x7f245410c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:49:12.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:12.464+0000 7f2452ffd700 1 -- 172.21.15.107:0/3187952239 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2448005040 con 0x7f245410cd70 2024-05-17T21:49:12.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:12.464+0000 7f2452ffd700 1 --2- 172.21.15.107:0/3187952239 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f245410cd70 0x7f2454111da0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f244800b3b0 tx=0x7f2448009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:49:12.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:12.464+0000 7f243bfff700 1 -- 172.21.15.107:0/3187952239 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f244800e040 con 0x7f245410cd70 2024-05-17T21:49:12.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:12.464+0000 7f243bfff700 1 -- 172.21.15.107:0/3187952239 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f244800bbf0 con 0x7f245410cd70 2024-05-17T21:49:12.458 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:12.464+0000 7f245a041700 1 -- 172.21.15.107:0/3187952239 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2454112340 con 0x7f245410cd70 2024-05-17T21:49:12.458 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:12.464+0000 7f243bfff700 1 -- 172.21.15.107:0/3187952239 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f244801c630 con 0x7f245410cd70 2024-05-17T21:49:12.458 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:12.464+0000 7f245a041700 1 -- 172.21.15.107:0/3187952239 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2454112900 con 0x7f245410cd70 2024-05-17T21:49:12.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:12.465+0000 7f245a041700 1 -- 172.21.15.107:0/3187952239 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f245404c710 con 0x7f245410cd70 2024-05-17T21:49:12.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:12.465+0000 7f243bfff700 1 -- 172.21.15.107:0/3187952239 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f2448026440 con 0x7f245410cd70 2024-05-17T21:49:12.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:12.466+0000 7f243bfff700 1 --2- 172.21.15.107:0/3187952239 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f243c0692a0 0x7f243c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:49:12.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:12.466+0000 7f24537fe700 1 --2- 172.21.15.107:0/3187952239 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f243c0692a0 0x7f243c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:49:12.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:12.466+0000 7f243bfff700 1 -- 172.21.15.107:0/3187952239 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f2448087b00 con 0x7f245410cd70 2024-05-17T21:49:12.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:12.466+0000 7f24537fe700 1 --2- 172.21.15.107:0/3187952239 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f243c0692a0 0x7f243c06b750 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2454065d20 tx=0x7f24440091a0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:49:12.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:12.470+0000 7f243bfff700 1 -- 172.21.15.107:0/3187952239 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f2448057c00 con 0x7f245410cd70 2024-05-17T21:49:12.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:12.663+0000 7f245a041700 1 -- 172.21.15.107:0/3187952239 --> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f24540fdd40 con 0x7f243c0692a0 2024-05-17T21:49:12.657 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:12.665+0000 7f243bfff700 1 -- 172.21.15.107:0/3187952239 <== mgr.14214 v2:172.21.15.107:6800/1294361014 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+399 (secure 0 0 0) 0x7f24540fdd40 con 0x7f243c0692a0 2024-05-17T21:49:12.657 INFO:teuthology.orchestra.run.smithi107.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-05-17T21:49:12.657 INFO:teuthology.orchestra.run.smithi107.stdout:smithi107 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400NK400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-05-17T21:49:12.657 INFO:teuthology.orchestra.run.smithi107.stdout:smithi138 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6161003C400BGN 372G No 73s ago Has a FileSystem, LVM detected 2024-05-17T21:49:12.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:12.668+0000 7f245a041700 1 -- 172.21.15.107:0/3187952239 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f243c0692a0 msgr2=0x7f243c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:49:12.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:12.668+0000 7f245a041700 1 --2- 172.21.15.107:0/3187952239 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f243c0692a0 0x7f243c06b750 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2454065d20 tx=0x7f24440091a0 comp rx=0 tx=0).stop 2024-05-17T21:49:12.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:12.668+0000 7f245a041700 1 -- 172.21.15.107:0/3187952239 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f245410cd70 msgr2=0x7f2454111da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:49:12.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:12.668+0000 7f245a041700 1 --2- 172.21.15.107:0/3187952239 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f245410cd70 0x7f2454111da0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f244800b3b0 tx=0x7f2448009e30 comp rx=0 tx=0).stop 2024-05-17T21:49:12.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:12.668+0000 7f245a041700 1 -- 172.21.15.107:0/3187952239 shutdown_connections 2024-05-17T21:49:12.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:12.668+0000 7f245a041700 1 --2- 172.21.15.107:0/3187952239 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f243c0692a0 0x7f243c06b750 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:49:12.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:12.668+0000 7f245a041700 1 --2- 172.21.15.107:0/3187952239 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f24540fd2b0 0x7f245410c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:49:12.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:12.668+0000 7f245a041700 1 --2- 172.21.15.107:0/3187952239 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f245410cd70 0x7f2454111da0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:49:12.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:12.668+0000 7f245a041700 1 -- 172.21.15.107:0/3187952239 >> 172.21.15.107:0/3187952239 conn(0x7f24540f6be0 msgr2=0x7f24540f81b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:49:12.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:12.668+0000 7f245a041700 1 -- 172.21.15.107:0/3187952239 shutdown_connections 2024-05-17T21:49:12.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:12.668+0000 7f245a041700 1 -- 172.21.15.107:0/3187952239 wait complete. 2024-05-17T21:49:13.264 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-17T21:49:13.274 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi107.front.sepia.ceph.com 2024-05-17T21:49:13.275 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -- bash -c 'ceph fs volume create cephfs --placement=4' 2024-05-17T21:49:13.529 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/mon.smithi107/config 2024-05-17T21:49:13.918 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:13 smithi107 ceph-mon[28434]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-17T21:49:13.918 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:13 smithi107 ceph-mon[28434]: from='client.14458 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:49:14.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:13 smithi138 ceph-mon[35948]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-17T21:49:14.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:13 smithi138 ceph-mon[35948]: from='client.14458 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:49:15.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:15.100+0000 7fe777e3d700 1 -- 172.21.15.107:0/2192817899 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe7700fbb70 msgr2=0x7fe7701023a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:49:15.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:15.100+0000 7fe777e3d700 1 --2- 172.21.15.107:0/2192817899 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe7700fbb70 0x7fe7701023a0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe760004940 tx=0x7fe760009e30 comp rx=0 tx=0).stop 2024-05-17T21:49:15.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:15.100+0000 7fe777e3d700 1 -- 172.21.15.107:0/2192817899 shutdown_connections 2024-05-17T21:49:15.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:15.100+0000 7fe777e3d700 1 --2- 172.21.15.107:0/2192817899 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe7700fbb70 0x7fe7701023a0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:49:15.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:15.100+0000 7fe777e3d700 1 --2- 172.21.15.107:0/2192817899 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7700fb220 0x7fe7700fb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:49:15.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:15.100+0000 7fe777e3d700 1 -- 172.21.15.107:0/2192817899 >> 172.21.15.107:0/2192817899 conn(0x7fe7700f6be0 msgr2=0x7fe7700f8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:49:15.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:15.101+0000 7fe777e3d700 1 -- 172.21.15.107:0/2192817899 shutdown_connections 2024-05-17T21:49:15.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:15.101+0000 7fe777e3d700 1 -- 172.21.15.107:0/2192817899 wait complete. 2024-05-17T21:49:15.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:15.101+0000 7fe777e3d700 1 Processor -- start 2024-05-17T21:49:15.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:15.101+0000 7fe777e3d700 1 -- start start 2024-05-17T21:49:15.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:15.101+0000 7fe777e3d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe7700fb220 0x7fe77010d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:49:15.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:15.101+0000 7fe777e3d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7700fbb70 0x7fe77010de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:49:15.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:15.101+0000 7fe777e3d700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe77010e4c0 con 0x7fe7700fb220 2024-05-17T21:49:15.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:15.101+0000 7fe777e3d700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe770112e60 con 0x7fe7700fbb70 2024-05-17T21:49:15.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:15.101+0000 7fe775bd9700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe7700fb220 0x7fe77010d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:49:15.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:15.101+0000 7fe775bd9700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe7700fb220 0x7fe77010d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43296/0 (socket says 172.21.15.107:43296) 2024-05-17T21:49:15.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:15.101+0000 7fe775bd9700 1 -- 172.21.15.107:0/1765478230 learned_addr learned my addr 172.21.15.107:0/1765478230 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:49:15.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:15.102+0000 7fe7753d8700 1 --2- 172.21.15.107:0/1765478230 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7700fbb70 0x7fe77010de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:49:15.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:15.102+0000 7fe775bd9700 1 -- 172.21.15.107:0/1765478230 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7700fbb70 msgr2=0x7fe77010de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:49:15.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:15.102+0000 7fe775bd9700 1 --2- 172.21.15.107:0/1765478230 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7700fbb70 0x7fe77010de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:49:15.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:15.102+0000 7fe775bd9700 1 -- 172.21.15.107:0/1765478230 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe760005040 con 0x7fe7700fb220 2024-05-17T21:49:15.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:15.102+0000 7fe7753d8700 1 --2- 172.21.15.107:0/1765478230 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7700fbb70 0x7fe77010de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T21:49:15.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:15.102+0000 7fe775bd9700 1 --2- 172.21.15.107:0/1765478230 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe7700fb220 0x7fe77010d8f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe76c00b790 tx=0x7fe76c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:49:15.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:15.102+0000 7fe7667fc700 1 -- 172.21.15.107:0/1765478230 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe76c00f770 con 0x7fe7700fb220 2024-05-17T21:49:15.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:15.102+0000 7fe7667fc700 1 -- 172.21.15.107:0/1765478230 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe76c00fda0 con 0x7fe7700fb220 2024-05-17T21:49:15.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:15.102+0000 7fe777e3d700 1 -- 172.21.15.107:0/1765478230 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7701130c0 con 0x7fe7700fb220 2024-05-17T21:49:15.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:15.102+0000 7fe7667fc700 1 -- 172.21.15.107:0/1765478230 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe76c00d310 con 0x7fe7700fb220 2024-05-17T21:49:15.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:15.102+0000 7fe777e3d700 1 -- 172.21.15.107:0/1765478230 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe770113630 con 0x7fe7700fb220 2024-05-17T21:49:15.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:15.103+0000 7fe777e3d700 1 -- 172.21.15.107:0/1765478230 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe77004c710 con 0x7fe7700fb220 2024-05-17T21:49:15.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:15.104+0000 7fe7667fc700 1 -- 172.21.15.107:0/1765478230 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7fe76c010420 con 0x7fe7700fb220 2024-05-17T21:49:15.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:15.104+0000 7fe7667fc700 1 --2- 172.21.15.107:0/1765478230 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fe75c0692a0 0x7fe75c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:49:15.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:15.104+0000 7fe7667fc700 1 -- 172.21.15.107:0/1765478230 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fe76c086e80 con 0x7fe7700fb220 2024-05-17T21:49:15.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:15.104+0000 7fe7753d8700 1 --2- 172.21.15.107:0/1765478230 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fe75c0692a0 0x7fe75c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:49:15.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:15.104+0000 7fe7753d8700 1 --2- 172.21.15.107:0/1765478230 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fe75c0692a0 0x7fe75c06b750 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe76000b3b0 tx=0x7fe760009e30 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:49:15.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:15.108+0000 7fe7667fc700 1 -- 172.21.15.107:0/1765478230 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fe76c056f80 con 0x7fe7700fb220 2024-05-17T21:49:15.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:15.320+0000 7fe777e3d700 1 -- 172.21.15.107:0/1765478230 --> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] -- mgr_command(tid 0: {"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7fe770101ab0 con 0x7fe75c0692a0 2024-05-17T21:49:16.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:16 smithi138 ceph-mon[35948]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-17T21:49:16.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:16 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-05-17T21:49:16.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:16 smithi107 ceph-mon[28434]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-17T21:49:16.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:16 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-05-17T21:49:16.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:16.858+0000 7fe7667fc700 1 -- 172.21.15.107:0/1765478230 <== mgr.14214 v2:172.21.15.107:6800/1294361014 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fe770101ab0 con 0x7fe75c0692a0 2024-05-17T21:49:16.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:16.860+0000 7fe777e3d700 1 -- 172.21.15.107:0/1765478230 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fe75c0692a0 msgr2=0x7fe75c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:49:16.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:16.860+0000 7fe777e3d700 1 --2- 172.21.15.107:0/1765478230 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fe75c0692a0 0x7fe75c06b750 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe76000b3b0 tx=0x7fe760009e30 comp rx=0 tx=0).stop 2024-05-17T21:49:16.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:16.860+0000 7fe777e3d700 1 -- 172.21.15.107:0/1765478230 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe7700fb220 msgr2=0x7fe77010d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:49:16.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:16.860+0000 7fe777e3d700 1 --2- 172.21.15.107:0/1765478230 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe7700fb220 0x7fe77010d8f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe76c00b790 tx=0x7fe76c009250 comp rx=0 tx=0).stop 2024-05-17T21:49:16.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:16.860+0000 7fe777e3d700 1 -- 172.21.15.107:0/1765478230 shutdown_connections 2024-05-17T21:49:16.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:16.860+0000 7fe777e3d700 1 --2- 172.21.15.107:0/1765478230 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fe75c0692a0 0x7fe75c06b750 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:49:16.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:16.860+0000 7fe777e3d700 1 --2- 172.21.15.107:0/1765478230 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe7700fb220 0x7fe77010d8f0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:49:16.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:16.860+0000 7fe777e3d700 1 --2- 172.21.15.107:0/1765478230 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7700fbb70 0x7fe77010de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:49:16.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:16.860+0000 7fe777e3d700 1 -- 172.21.15.107:0/1765478230 >> 172.21.15.107:0/1765478230 conn(0x7fe7700f6be0 msgr2=0x7fe770100390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:49:16.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:16.860+0000 7fe777e3d700 1 -- 172.21.15.107:0/1765478230 shutdown_connections 2024-05-17T21:49:16.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:16.860+0000 7fe777e3d700 1 -- 172.21.15.107:0/1765478230 wait complete. 2024-05-17T21:49:17.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:16 smithi107 ceph-3722731a-1496-11ef-bc9a-c7b262605968-mon-smithi107[28430]: 2024-05-17T21:49:16.835+0000 7f898b7f1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-17T21:49:17.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:17 smithi107 ceph-mon[28434]: from='client.14462 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:49:17.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:17 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-05-17T21:49:17.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:17 smithi107 ceph-mon[28434]: osdmap e41: 6 total, 6 up, 6 in 2024-05-17T21:49:17.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:17 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-05-17T21:49:17.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:17 smithi107 ceph-mon[28434]: pgmap v171: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-17T21:49:17.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:17 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-05-17T21:49:17.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:17 smithi107 ceph-mon[28434]: osdmap e42: 6 total, 6 up, 6 in 2024-05-17T21:49:17.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:17 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-05-17T21:49:17.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:17 smithi107 ceph-mon[28434]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-17T21:49:17.104 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:17 smithi107 ceph-mon[28434]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-17T21:49:17.104 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:17 smithi107 ceph-mon[28434]: osdmap e43: 6 total, 6 up, 6 in 2024-05-17T21:49:17.104 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:17 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-05-17T21:49:17.104 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:17 smithi107 ceph-mon[28434]: fsmap cephfs:0 2024-05-17T21:49:17.104 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:17 smithi107 ceph-mon[28434]: Saving service mds.cephfs spec with placement count:4 2024-05-17T21:49:17.104 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:17 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:49:17.104 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:17 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:49:17.104 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:17 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:49:17.104 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:17 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T21:49:17.104 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:17 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:49:17.104 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:17 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi107.idmbgj", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T21:49:17.104 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:17 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi107.idmbgj", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T21:49:17.104 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:17 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:49:17.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:17 smithi138 ceph-mon[35948]: from='client.14462 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:49:17.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:17 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-05-17T21:49:17.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:17 smithi138 ceph-mon[35948]: osdmap e41: 6 total, 6 up, 6 in 2024-05-17T21:49:17.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:17 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-05-17T21:49:17.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:17 smithi138 ceph-mon[35948]: pgmap v171: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-17T21:49:17.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:17 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-05-17T21:49:17.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:17 smithi138 ceph-mon[35948]: osdmap e42: 6 total, 6 up, 6 in 2024-05-17T21:49:17.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:17 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-05-17T21:49:17.328 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:17 smithi138 ceph-mon[35948]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-17T21:49:17.328 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:17 smithi138 ceph-mon[35948]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-17T21:49:17.328 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:17 smithi138 ceph-mon[35948]: osdmap e43: 6 total, 6 up, 6 in 2024-05-17T21:49:17.328 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:17 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-05-17T21:49:17.328 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:17 smithi138 ceph-mon[35948]: fsmap cephfs:0 2024-05-17T21:49:17.328 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:17 smithi138 ceph-mon[35948]: Saving service mds.cephfs spec with placement count:4 2024-05-17T21:49:17.328 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:17 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:49:17.328 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:17 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:49:17.328 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:17 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:49:17.328 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:17 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T21:49:17.328 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:17 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:49:17.328 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:17 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi107.idmbgj", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T21:49:17.328 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:17 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi107.idmbgj", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T21:49:17.329 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:17 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:49:17.527 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -- bash -c 'ceph fs dump' 2024-05-17T21:49:18.320 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/mon.smithi107/config 2024-05-17T21:49:18.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:18 smithi138 ceph-mon[35948]: Deploying daemon mds.cephfs.smithi107.idmbgj on smithi107 2024-05-17T21:49:18.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:18 smithi138 ceph-mon[35948]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-17T21:49:18.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:18 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:49:18.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:18 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:49:18.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:18 smithi138 ceph-mon[35948]: osdmap e44: 6 total, 6 up, 6 in 2024-05-17T21:49:18.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:18 smithi107 ceph-mon[28434]: Deploying daemon mds.cephfs.smithi107.idmbgj on smithi107 2024-05-17T21:49:18.353 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:18 smithi107 ceph-mon[28434]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-17T21:49:18.353 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:18 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:49:18.353 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:18 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:49:18.353 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:18 smithi107 ceph-mon[28434]: osdmap e44: 6 total, 6 up, 6 in 2024-05-17T21:49:20.105 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:19 smithi107 ceph-mon[28434]: pgmap v175: 65 pgs: 12 creating+peering, 31 unknown, 22 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-17T21:49:20.105 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:19 smithi107 ceph-mon[28434]: osdmap e45: 6 total, 6 up, 6 in 2024-05-17T21:49:20.105 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:19 smithi107 ceph-mon[28434]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-17T21:49:20.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:19 smithi138 ceph-mon[35948]: pgmap v175: 65 pgs: 12 creating+peering, 31 unknown, 22 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-17T21:49:20.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:19 smithi138 ceph-mon[35948]: osdmap e45: 6 total, 6 up, 6 in 2024-05-17T21:49:20.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:19 smithi138 ceph-mon[35948]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-17T21:49:20.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:20.720+0000 7f560e174700 1 -- 172.21.15.107:0/382351863 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5608070830 msgr2=0x7f5608070ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:49:20.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:20.720+0000 7f560e174700 1 --2- 172.21.15.107:0/382351863 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5608070830 0x7f5608070ca0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f55fc004940 tx=0x7f55fc009e30 comp rx=0 tx=0).stop 2024-05-17T21:49:20.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:20.720+0000 7f560e174700 1 -- 172.21.15.107:0/382351863 shutdown_connections 2024-05-17T21:49:20.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:20.720+0000 7f560e174700 1 --2- 172.21.15.107:0/382351863 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5608070830 0x7f5608070ca0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:49:20.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:20.720+0000 7f560e174700 1 --2- 172.21.15.107:0/382351863 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5608071de0 0x7f56080702f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:49:20.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:20.720+0000 7f560e174700 1 -- 172.21.15.107:0/382351863 >> 172.21.15.107:0/382351863 conn(0x7f56080f8540 msgr2=0x7f56080fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:49:20.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:20.720+0000 7f560e174700 1 -- 172.21.15.107:0/382351863 shutdown_connections 2024-05-17T21:49:20.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:20.720+0000 7f560e174700 1 -- 172.21.15.107:0/382351863 wait complete. 2024-05-17T21:49:20.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:20.721+0000 7f560e174700 1 Processor -- start 2024-05-17T21:49:20.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:20.721+0000 7f560e174700 1 -- start start 2024-05-17T21:49:20.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:20.721+0000 7f560e174700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5608070830 0x7f560810faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:49:20.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:20.721+0000 7f560e174700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5608071de0 0x7f5608110030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:49:20.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:20.721+0000 7f560e174700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5608110630 con 0x7f5608070830 2024-05-17T21:49:20.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:20.721+0000 7f560e174700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56081107a0 con 0x7f5608071de0 2024-05-17T21:49:20.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:20.722+0000 7f56077fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5608070830 0x7f560810faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:49:20.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:20.722+0000 7f56077fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5608070830 0x7f560810faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41778/0 (socket says 172.21.15.107:41778) 2024-05-17T21:49:20.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:20.722+0000 7f56077fe700 1 -- 172.21.15.107:0/3985642891 learned_addr learned my addr 172.21.15.107:0/3985642891 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:49:20.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:20.722+0000 7f5606ffd700 1 --2- 172.21.15.107:0/3985642891 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5608071de0 0x7f5608110030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:49:20.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:20.722+0000 7f56077fe700 1 -- 172.21.15.107:0/3985642891 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5608071de0 msgr2=0x7f5608110030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:49:20.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:20.722+0000 7f56077fe700 1 --2- 172.21.15.107:0/3985642891 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5608071de0 0x7f5608110030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:49:20.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:20.722+0000 7f56077fe700 1 -- 172.21.15.107:0/3985642891 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55fc005040 con 0x7f5608070830 2024-05-17T21:49:20.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:20.722+0000 7f56077fe700 1 --2- 172.21.15.107:0/3985642891 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5608070830 0x7f560810faf0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f55f8002a80 tx=0x7f55f8004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:49:20.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:20.722+0000 7f55effff700 1 -- 172.21.15.107:0/3985642891 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f55f8007cc0 con 0x7f5608070830 2024-05-17T21:49:20.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:20.722+0000 7f560e174700 1 -- 172.21.15.107:0/3985642891 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5608115200 con 0x7f5608070830 2024-05-17T21:49:20.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:20.722+0000 7f55effff700 1 -- 172.21.15.107:0/3985642891 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f55f800f040 con 0x7f5608070830 2024-05-17T21:49:20.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:20.722+0000 7f55effff700 1 -- 172.21.15.107:0/3985642891 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f55f800da30 con 0x7f5608070830 2024-05-17T21:49:20.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:20.722+0000 7f560e174700 1 -- 172.21.15.107:0/3985642891 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56081157c0 con 0x7f5608070830 2024-05-17T21:49:20.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:20.724+0000 7f560e174700 1 -- 172.21.15.107:0/3985642891 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f560804c710 con 0x7f5608070830 2024-05-17T21:49:20.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:20.725+0000 7f55effff700 1 -- 172.21.15.107:0/3985642891 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f55f800db90 con 0x7f5608070830 2024-05-17T21:49:20.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:20.725+0000 7f55effff700 1 --2- 172.21.15.107:0/3985642891 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f55f00692a0 0x7f55f006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:49:20.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:20.725+0000 7f55effff700 1 -- 172.21.15.107:0/3985642891 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f55f8087580 con 0x7f5608070830 2024-05-17T21:49:20.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:20.725+0000 7f5606ffd700 1 --2- 172.21.15.107:0/3985642891 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f55f00692a0 0x7f55f006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:49:20.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:20.726+0000 7f5606ffd700 1 --2- 172.21.15.107:0/3985642891 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f55f00692a0 0x7f55f006b750 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f55fc00b970 tx=0x7f55fc009e30 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:49:20.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:20.729+0000 7f55effff700 1 -- 172.21.15.107:0/3985642891 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f55f80573f0 con 0x7f5608070830 2024-05-17T21:49:20.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:20.968+0000 7f560e174700 1 -- 172.21.15.107:0/3985642891 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f5608115ca0 con 0x7f5608070830 2024-05-17T21:49:20.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:20.969+0000 7f55effff700 1 -- 172.21.15.107:0/3985642891 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 2 v2) v1 ==== 75+0+1076 (secure 0 0 0) 0x7f55f8056f80 con 0x7f5608070830 2024-05-17T21:49:20.963 INFO:teuthology.orchestra.run.smithi107.stdout:e2 2024-05-17T21:49:20.963 INFO:teuthology.orchestra.run.smithi107.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-17T21:49:20.963 INFO:teuthology.orchestra.run.smithi107.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-17T21:49:20.963 INFO:teuthology.orchestra.run.smithi107.stdout:legacy client fscid: 1 2024-05-17T21:49:20.963 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:49:20.963 INFO:teuthology.orchestra.run.smithi107.stdout:Filesystem 'cephfs' (1) 2024-05-17T21:49:20.963 INFO:teuthology.orchestra.run.smithi107.stdout:fs_name cephfs 2024-05-17T21:49:20.963 INFO:teuthology.orchestra.run.smithi107.stdout:epoch 2 2024-05-17T21:49:20.963 INFO:teuthology.orchestra.run.smithi107.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-05-17T21:49:20.963 INFO:teuthology.orchestra.run.smithi107.stdout:created 2024-05-17T21:49:16.836333+0000 2024-05-17T21:49:20.963 INFO:teuthology.orchestra.run.smithi107.stdout:modified 2024-05-17T21:49:16.836401+0000 2024-05-17T21:49:20.963 INFO:teuthology.orchestra.run.smithi107.stdout:tableserver 0 2024-05-17T21:49:20.964 INFO:teuthology.orchestra.run.smithi107.stdout:root 0 2024-05-17T21:49:20.964 INFO:teuthology.orchestra.run.smithi107.stdout:session_timeout 60 2024-05-17T21:49:20.964 INFO:teuthology.orchestra.run.smithi107.stdout:session_autoclose 300 2024-05-17T21:49:20.964 INFO:teuthology.orchestra.run.smithi107.stdout:max_file_size 1099511627776 2024-05-17T21:49:20.964 INFO:teuthology.orchestra.run.smithi107.stdout:required_client_features {} 2024-05-17T21:49:20.964 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure 0 2024-05-17T21:49:20.964 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure_osd_epoch 0 2024-05-17T21:49:20.964 INFO:teuthology.orchestra.run.smithi107.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-17T21:49:20.964 INFO:teuthology.orchestra.run.smithi107.stdout:max_mds 1 2024-05-17T21:49:20.964 INFO:teuthology.orchestra.run.smithi107.stdout:in 2024-05-17T21:49:20.964 INFO:teuthology.orchestra.run.smithi107.stdout:up {} 2024-05-17T21:49:20.964 INFO:teuthology.orchestra.run.smithi107.stdout:failed 2024-05-17T21:49:20.964 INFO:teuthology.orchestra.run.smithi107.stdout:damaged 2024-05-17T21:49:20.965 INFO:teuthology.orchestra.run.smithi107.stdout:stopped 2024-05-17T21:49:20.965 INFO:teuthology.orchestra.run.smithi107.stdout:data_pools [3] 2024-05-17T21:49:20.965 INFO:teuthology.orchestra.run.smithi107.stdout:metadata_pool 2 2024-05-17T21:49:20.965 INFO:teuthology.orchestra.run.smithi107.stdout:inline_data disabled 2024-05-17T21:49:20.965 INFO:teuthology.orchestra.run.smithi107.stdout:balancer 2024-05-17T21:49:20.965 INFO:teuthology.orchestra.run.smithi107.stdout:standby_count_wanted 0 2024-05-17T21:49:20.965 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:49:20.965 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:49:20.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:20.971+0000 7f560e174700 1 -- 172.21.15.107:0/3985642891 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f55f00692a0 msgr2=0x7f55f006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:49:20.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:20.971+0000 7f560e174700 1 --2- 172.21.15.107:0/3985642891 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f55f00692a0 0x7f55f006b750 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f55fc00b970 tx=0x7f55fc009e30 comp rx=0 tx=0).stop 2024-05-17T21:49:20.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:20.971+0000 7f560e174700 1 -- 172.21.15.107:0/3985642891 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5608070830 msgr2=0x7f560810faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:49:20.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:20.971+0000 7f560e174700 1 --2- 172.21.15.107:0/3985642891 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5608070830 0x7f560810faf0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f55f8002a80 tx=0x7f55f8004600 comp rx=0 tx=0).stop 2024-05-17T21:49:20.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:20.971+0000 7f560e174700 1 -- 172.21.15.107:0/3985642891 shutdown_connections 2024-05-17T21:49:20.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:20.971+0000 7f560e174700 1 --2- 172.21.15.107:0/3985642891 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f55f00692a0 0x7f55f006b750 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:49:20.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:20.971+0000 7f560e174700 1 --2- 172.21.15.107:0/3985642891 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5608070830 0x7f560810faf0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:49:20.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:20.971+0000 7f560e174700 1 --2- 172.21.15.107:0/3985642891 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5608071de0 0x7f5608110030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:49:20.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:20.971+0000 7f560e174700 1 -- 172.21.15.107:0/3985642891 >> 172.21.15.107:0/3985642891 conn(0x7f56080f8540 msgr2=0x7f56080f9c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:49:20.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:20.972+0000 7f560e174700 1 -- 172.21.15.107:0/3985642891 shutdown_connections 2024-05-17T21:49:20.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:20.972+0000 7f560e174700 1 -- 172.21.15.107:0/3985642891 wait complete. 2024-05-17T21:49:20.966 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 2 2024-05-17T21:49:21.944 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-17T21:49:21.954 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi107.front.sepia.ceph.com 2024-05-17T21:49:21.954 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -- bash -c 'ceph fs set cephfs max_mds 1' 2024-05-17T21:49:22.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:21 smithi107 ceph-mon[28434]: pgmap v177: 65 pgs: 12 creating+peering, 53 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-17T21:49:22.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:21 smithi107 ceph-mon[28434]: from='client.? 172.21.15.107:0/3985642891' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-17T21:49:22.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:21 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:49:22.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:21 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:49:22.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:21 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:49:22.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:21 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi138.gwbtyd", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T21:49:22.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:21 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi138.gwbtyd", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T21:49:22.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:21 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:49:22.137 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:21 smithi138 ceph-mon[35948]: pgmap v177: 65 pgs: 12 creating+peering, 53 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-17T21:49:22.137 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:21 smithi138 ceph-mon[35948]: from='client.? 172.21.15.107:0/3985642891' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-17T21:49:22.138 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:21 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:49:22.138 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:21 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:49:22.138 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:21 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:49:22.138 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:21 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi138.gwbtyd", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T21:49:22.138 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:21 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi138.gwbtyd", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T21:49:22.138 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:21 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:49:22.210 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/mon.smithi107/config 2024-05-17T21:49:22.910 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:22 smithi107 ceph-mon[28434]: Deploying daemon mds.cephfs.smithi138.gwbtyd on smithi138 2024-05-17T21:49:22.910 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:22 smithi107 ceph-mon[28434]: mds.? [v2:172.21.15.107:6826/2751824191,v1:172.21.15.107:6827/2751824191] up:boot 2024-05-17T21:49:22.911 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:22 smithi107 ceph-mon[28434]: daemon mds.cephfs.smithi107.idmbgj assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-05-17T21:49:22.911 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:22 smithi107 ceph-mon[28434]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-17T21:49:22.911 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:22 smithi107 ceph-mon[28434]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-17T21:49:22.911 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:22 smithi107 ceph-mon[28434]: Cluster is now healthy 2024-05-17T21:49:22.911 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:22 smithi107 ceph-mon[28434]: fsmap cephfs:0 1 up:standby 2024-05-17T21:49:22.911 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:22 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi107.idmbgj"}]: dispatch 2024-05-17T21:49:22.911 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:22 smithi107 ceph-mon[28434]: fsmap cephfs:1 {0=cephfs.smithi107.idmbgj=up:creating} 2024-05-17T21:49:22.911 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:22 smithi107 ceph-mon[28434]: daemon mds.cephfs.smithi107.idmbgj is now active in filesystem cephfs as rank 0 2024-05-17T21:49:23.042 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:22 smithi138 ceph-mon[35948]: Deploying daemon mds.cephfs.smithi138.gwbtyd on smithi138 2024-05-17T21:49:23.042 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:22 smithi138 ceph-mon[35948]: mds.? [v2:172.21.15.107:6826/2751824191,v1:172.21.15.107:6827/2751824191] up:boot 2024-05-17T21:49:23.042 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:22 smithi138 ceph-mon[35948]: daemon mds.cephfs.smithi107.idmbgj assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-05-17T21:49:23.042 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:22 smithi138 ceph-mon[35948]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-17T21:49:23.043 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:22 smithi138 ceph-mon[35948]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-17T21:49:23.043 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:22 smithi138 ceph-mon[35948]: Cluster is now healthy 2024-05-17T21:49:23.043 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:22 smithi138 ceph-mon[35948]: fsmap cephfs:0 1 up:standby 2024-05-17T21:49:23.043 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:22 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi107.idmbgj"}]: dispatch 2024-05-17T21:49:23.043 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:22 smithi138 ceph-mon[35948]: fsmap cephfs:1 {0=cephfs.smithi107.idmbgj=up:creating} 2024-05-17T21:49:23.043 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:22 smithi138 ceph-mon[35948]: daemon mds.cephfs.smithi107.idmbgj is now active in filesystem cephfs as rank 0 2024-05-17T21:49:23.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:23.706+0000 7f7c4ccd8700 1 -- 172.21.15.107:0/4122547605 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c480fdfd0 msgr2=0x7f7c480fe440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:49:23.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:23.706+0000 7f7c4ccd8700 1 --2- 172.21.15.107:0/4122547605 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c480fdfd0 0x7f7c480fe440 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f7c30004940 tx=0x7f7c30009e30 comp rx=0 tx=0).stop 2024-05-17T21:49:23.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:23.706+0000 7f7c4ccd8700 1 -- 172.21.15.107:0/4122547605 shutdown_connections 2024-05-17T21:49:23.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:23.706+0000 7f7c4ccd8700 1 --2- 172.21.15.107:0/4122547605 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c480fdfd0 0x7f7c480fe440 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:49:23.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:23.706+0000 7f7c4ccd8700 1 --2- 172.21.15.107:0/4122547605 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c480fce60 0x7f7c480fd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:49:23.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:23.706+0000 7f7c4ccd8700 1 -- 172.21.15.107:0/4122547605 >> 172.21.15.107:0/4122547605 conn(0x7f7c480f8540 msgr2=0x7f7c480fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:49:23.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:23.706+0000 7f7c4ccd8700 1 -- 172.21.15.107:0/4122547605 shutdown_connections 2024-05-17T21:49:23.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:23.706+0000 7f7c4ccd8700 1 -- 172.21.15.107:0/4122547605 wait complete. 2024-05-17T21:49:23.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:23.707+0000 7f7c4ccd8700 1 Processor -- start 2024-05-17T21:49:23.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:23.707+0000 7f7c4ccd8700 1 -- start start 2024-05-17T21:49:23.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:23.707+0000 7f7c4ccd8700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c480fce60 0x7f7c4810d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:49:23.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:23.707+0000 7f7c4ccd8700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c480fdfd0 0x7f7c4810de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:49:23.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:23.707+0000 7f7c4ccd8700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c4810e450 con 0x7f7c480fce60 2024-05-17T21:49:23.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:23.707+0000 7f7c4ccd8700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c4810e5c0 con 0x7f7c480fdfd0 2024-05-17T21:49:23.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:23.707+0000 7f7c4659c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c480fce60 0x7f7c4810d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:49:23.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:23.707+0000 7f7c4659c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c480fce60 0x7f7c4810d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41818/0 (socket says 172.21.15.107:41818) 2024-05-17T21:49:23.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:23.707+0000 7f7c4659c700 1 -- 172.21.15.107:0/2685863006 learned_addr learned my addr 172.21.15.107:0/2685863006 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:49:23.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:23.707+0000 7f7c3dd9b700 1 --2- 172.21.15.107:0/2685863006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c480fdfd0 0x7f7c4810de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:49:23.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:23.707+0000 7f7c4659c700 1 -- 172.21.15.107:0/2685863006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c480fdfd0 msgr2=0x7f7c4810de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:49:23.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:23.707+0000 7f7c4659c700 1 --2- 172.21.15.107:0/2685863006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c480fdfd0 0x7f7c4810de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:49:23.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:23.707+0000 7f7c4659c700 1 -- 172.21.15.107:0/2685863006 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c30005040 con 0x7f7c480fce60 2024-05-17T21:49:23.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:23.707+0000 7f7c4659c700 1 --2- 172.21.15.107:0/2685863006 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c480fce60 0x7f7c4810d910 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f7c3800c8c0 tx=0x7f7c380044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:49:23.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:23.707+0000 7f7c3f7fe700 1 -- 172.21.15.107:0/2685863006 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c3800a980 con 0x7f7c480fce60 2024-05-17T21:49:23.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:23.708+0000 7f7c4ccd8700 1 -- 172.21.15.107:0/2685863006 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c48112fc0 con 0x7f7c480fce60 2024-05-17T21:49:23.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:23.708+0000 7f7c3f7fe700 1 -- 172.21.15.107:0/2685863006 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7c3800ce60 con 0x7f7c480fce60 2024-05-17T21:49:23.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:23.708+0000 7f7c3f7fe700 1 -- 172.21.15.107:0/2685863006 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c380115e0 con 0x7f7c480fce60 2024-05-17T21:49:23.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:23.708+0000 7f7c4ccd8700 1 -- 172.21.15.107:0/2685863006 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c48113550 con 0x7f7c480fce60 2024-05-17T21:49:23.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:23.708+0000 7f7c4ccd8700 1 -- 172.21.15.107:0/2685863006 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c4804c710 con 0x7f7c480fce60 2024-05-17T21:49:23.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:23.709+0000 7f7c3f7fe700 1 -- 172.21.15.107:0/2685863006 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f7c38012400 con 0x7f7c480fce60 2024-05-17T21:49:23.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:23.709+0000 7f7c3f7fe700 1 --2- 172.21.15.107:0/2685863006 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f7c340692a0 0x7f7c3406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:49:23.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:23.710+0000 7f7c3dd9b700 1 --2- 172.21.15.107:0/2685863006 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f7c340692a0 0x7f7c3406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:49:23.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:23.710+0000 7f7c3f7fe700 1 -- 172.21.15.107:0/2685863006 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f7c38087280 con 0x7f7c480fce60 2024-05-17T21:49:23.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:23.710+0000 7f7c3dd9b700 1 --2- 172.21.15.107:0/2685863006 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f7c340692a0 0x7f7c3406b750 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7c3000b3b0 tx=0x7f7c30009e30 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:49:23.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:23.714+0000 7f7c3f7fe700 1 -- 172.21.15.107:0/2685863006 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f7c38057040 con 0x7f7c480fce60 2024-05-17T21:49:23.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:23.949+0000 7f7c4ccd8700 1 -- 172.21.15.107:0/2685863006 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"} v 0) v1 -- 0x7f7c481137d0 con 0x7f7c480fce60 2024-05-17T21:49:24.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:23 smithi138 ceph-mon[35948]: pgmap v178: 65 pgs: 12 creating+peering, 53 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-17T21:49:24.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:23 smithi138 ceph-mon[35948]: mds.? [v2:172.21.15.107:6826/2751824191,v1:172.21.15.107:6827/2751824191] up:active 2024-05-17T21:49:24.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:23 smithi138 ceph-mon[35948]: fsmap cephfs:1 {0=cephfs.smithi107.idmbgj=up:active} 2024-05-17T21:49:24.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:23 smithi107 ceph-mon[28434]: pgmap v178: 65 pgs: 12 creating+peering, 53 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-17T21:49:24.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:23 smithi107 ceph-mon[28434]: mds.? [v2:172.21.15.107:6826/2751824191,v1:172.21.15.107:6827/2751824191] up:active 2024-05-17T21:49:24.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:23 smithi107 ceph-mon[28434]: fsmap cephfs:1 {0=cephfs.smithi107.idmbgj=up:active} 2024-05-17T21:49:24.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:24.870+0000 7f7c3f7fe700 1 -- 172.21.15.107:0/2685863006 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]=0 v6) v1 ==== 105+0+0 (secure 0 0 0) 0x7f7c380126b0 con 0x7f7c480fce60 2024-05-17T21:49:24.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:24.873+0000 7f7c4ccd8700 1 -- 172.21.15.107:0/2685863006 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f7c340692a0 msgr2=0x7f7c3406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:49:24.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:24.873+0000 7f7c4ccd8700 1 --2- 172.21.15.107:0/2685863006 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f7c340692a0 0x7f7c3406b750 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7c3000b3b0 tx=0x7f7c30009e30 comp rx=0 tx=0).stop 2024-05-17T21:49:24.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:24.873+0000 7f7c4ccd8700 1 -- 172.21.15.107:0/2685863006 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c480fce60 msgr2=0x7f7c4810d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:49:24.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:24.873+0000 7f7c4ccd8700 1 --2- 172.21.15.107:0/2685863006 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c480fce60 0x7f7c4810d910 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f7c3800c8c0 tx=0x7f7c380044c0 comp rx=0 tx=0).stop 2024-05-17T21:49:24.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:24.873+0000 7f7c4ccd8700 1 -- 172.21.15.107:0/2685863006 shutdown_connections 2024-05-17T21:49:24.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:24.873+0000 7f7c4ccd8700 1 --2- 172.21.15.107:0/2685863006 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f7c340692a0 0x7f7c3406b750 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:49:24.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:24.873+0000 7f7c4ccd8700 1 --2- 172.21.15.107:0/2685863006 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c480fce60 0x7f7c4810d910 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:49:24.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:24.873+0000 7f7c4ccd8700 1 --2- 172.21.15.107:0/2685863006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c480fdfd0 0x7f7c4810de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:49:24.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:24.873+0000 7f7c4ccd8700 1 -- 172.21.15.107:0/2685863006 >> 172.21.15.107:0/2685863006 conn(0x7f7c480f8540 msgr2=0x7f7c480f9c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:49:24.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:24.873+0000 7f7c4ccd8700 1 -- 172.21.15.107:0/2685863006 shutdown_connections 2024-05-17T21:49:24.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:24.873+0000 7f7c4ccd8700 1 -- 172.21.15.107:0/2685863006 wait complete. 2024-05-17T21:49:25.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:24 smithi107 ceph-mon[28434]: from='client.? 172.21.15.107:0/2685863006' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]: dispatch 2024-05-17T21:49:25.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:24 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:49:25.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:24 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:49:25.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:24 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:49:25.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:24 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi107.drqody", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T21:49:25.104 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:24 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi107.drqody", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T21:49:25.104 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:24 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:49:25.294 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:24 smithi138 ceph-mon[35948]: from='client.? 172.21.15.107:0/2685863006' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]: dispatch 2024-05-17T21:49:25.294 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:24 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:49:25.294 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:24 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:49:25.294 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:24 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:49:25.294 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:24 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi107.drqody", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T21:49:25.294 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:24 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi107.drqody", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T21:49:25.294 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:24 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:49:25.637 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-17T21:49:25.648 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi107.front.sepia.ceph.com 2024-05-17T21:49:25.648 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -- bash -c 'ceph fs set cephfs allow_standby_replay true' 2024-05-17T21:49:26.049 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:25 smithi107 ceph-mon[28434]: Deploying daemon mds.cephfs.smithi107.drqody on smithi107 2024-05-17T21:49:26.049 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:25 smithi107 ceph-mon[28434]: pgmap v179: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-17T21:49:26.049 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:25 smithi107 ceph-mon[28434]: mds.? [v2:172.21.15.138:6824/2779634919,v1:172.21.15.138:6825/2779634919] up:boot 2024-05-17T21:49:26.049 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:25 smithi107 ceph-mon[28434]: from='client.? 172.21.15.107:0/2685863006' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]': finished 2024-05-17T21:49:26.049 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:25 smithi107 ceph-mon[28434]: fsmap cephfs:1 {0=cephfs.smithi107.idmbgj=up:active} 1 up:standby 2024-05-17T21:49:26.049 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:25 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi138.gwbtyd"}]: dispatch 2024-05-17T21:49:26.049 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:25 smithi107 ceph-mon[28434]: fsmap cephfs:1 {0=cephfs.smithi107.idmbgj=up:active} 1 up:standby 2024-05-17T21:49:26.160 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/mon.smithi107/config 2024-05-17T21:49:26.222 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:25 smithi138 ceph-mon[35948]: Deploying daemon mds.cephfs.smithi107.drqody on smithi107 2024-05-17T21:49:26.223 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:25 smithi138 ceph-mon[35948]: pgmap v179: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-17T21:49:26.223 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:25 smithi138 ceph-mon[35948]: mds.? [v2:172.21.15.138:6824/2779634919,v1:172.21.15.138:6825/2779634919] up:boot 2024-05-17T21:49:26.223 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:25 smithi138 ceph-mon[35948]: from='client.? 172.21.15.107:0/2685863006' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]': finished 2024-05-17T21:49:26.223 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:25 smithi138 ceph-mon[35948]: fsmap cephfs:1 {0=cephfs.smithi107.idmbgj=up:active} 1 up:standby 2024-05-17T21:49:26.223 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:25 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi138.gwbtyd"}]: dispatch 2024-05-17T21:49:26.223 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:25 smithi138 ceph-mon[35948]: fsmap cephfs:1 {0=cephfs.smithi107.idmbgj=up:active} 1 up:standby 2024-05-17T21:49:27.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:27.929+0000 7fa43d0d3700 1 -- 172.21.15.107:0/3317498374 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4380fdfd0 msgr2=0x7fa4380fe440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:49:27.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:27.929+0000 7fa43d0d3700 1 --2- 172.21.15.107:0/3317498374 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4380fdfd0 0x7fa4380fe440 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa420004940 tx=0x7fa420009e30 comp rx=0 tx=0).stop 2024-05-17T21:49:27.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:27.929+0000 7fa43d0d3700 1 -- 172.21.15.107:0/3317498374 shutdown_connections 2024-05-17T21:49:27.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:27.929+0000 7fa43d0d3700 1 --2- 172.21.15.107:0/3317498374 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4380fdfd0 0x7fa4380fe440 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:49:27.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:27.929+0000 7fa43d0d3700 1 --2- 172.21.15.107:0/3317498374 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4380fce60 0x7fa4380fd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:49:27.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:27.929+0000 7fa43d0d3700 1 -- 172.21.15.107:0/3317498374 >> 172.21.15.107:0/3317498374 conn(0x7fa4380f8540 msgr2=0x7fa4380fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:49:27.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:27.929+0000 7fa43d0d3700 1 -- 172.21.15.107:0/3317498374 shutdown_connections 2024-05-17T21:49:27.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:27.929+0000 7fa43d0d3700 1 -- 172.21.15.107:0/3317498374 wait complete. 2024-05-17T21:49:27.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:27.929+0000 7fa43d0d3700 1 Processor -- start 2024-05-17T21:49:27.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:27.929+0000 7fa43d0d3700 1 -- start start 2024-05-17T21:49:27.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:27.930+0000 7fa43d0d3700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4380fce60 0x7fa43810d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:49:27.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:27.930+0000 7fa43d0d3700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4380fdfd0 0x7fa43810de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:49:27.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:27.930+0000 7fa43d0d3700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa43810e4f0 con 0x7fa4380fdfd0 2024-05-17T21:49:27.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:27.930+0000 7fa43d0d3700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa438112e90 con 0x7fa4380fce60 2024-05-17T21:49:27.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:27.930+0000 7fa42ffff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4380fdfd0 0x7fa43810de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:49:27.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:27.930+0000 7fa42ffff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4380fdfd0 0x7fa43810de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:53812/0 (socket says 172.21.15.107:53812) 2024-05-17T21:49:27.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:27.930+0000 7fa42ffff700 1 -- 172.21.15.107:0/3078823705 learned_addr learned my addr 172.21.15.107:0/3078823705 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:49:27.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:27.930+0000 7fa436d9d700 1 --2- 172.21.15.107:0/3078823705 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4380fce60 0x7fa43810d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:49:27.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:27.930+0000 7fa42ffff700 1 -- 172.21.15.107:0/3078823705 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4380fce60 msgr2=0x7fa43810d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:49:27.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:27.930+0000 7fa42ffff700 1 --2- 172.21.15.107:0/3078823705 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4380fce60 0x7fa43810d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:49:27.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:27.930+0000 7fa42ffff700 1 -- 172.21.15.107:0/3078823705 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa420005040 con 0x7fa4380fdfd0 2024-05-17T21:49:27.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:27.930+0000 7fa42ffff700 1 --2- 172.21.15.107:0/3078823705 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4380fdfd0 0x7fa43810de60 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa42000b3b0 tx=0x7fa420009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:49:27.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:27.930+0000 7fa42f7fe700 1 -- 172.21.15.107:0/3078823705 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa42000e040 con 0x7fa4380fdfd0 2024-05-17T21:49:27.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:27.930+0000 7fa42f7fe700 1 -- 172.21.15.107:0/3078823705 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa42000bcc0 con 0x7fa4380fdfd0 2024-05-17T21:49:27.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:27.930+0000 7fa43d0d3700 1 -- 172.21.15.107:0/3078823705 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa438113090 con 0x7fa4380fdfd0 2024-05-17T21:49:27.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:27.930+0000 7fa42f7fe700 1 -- 172.21.15.107:0/3078823705 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa42001c630 con 0x7fa4380fdfd0 2024-05-17T21:49:27.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:27.930+0000 7fa43d0d3700 1 -- 172.21.15.107:0/3078823705 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa438113600 con 0x7fa4380fdfd0 2024-05-17T21:49:27.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:27.931+0000 7fa43d0d3700 1 -- 172.21.15.107:0/3078823705 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa43804c710 con 0x7fa4380fdfd0 2024-05-17T21:49:27.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:27.932+0000 7fa42f7fe700 1 -- 172.21.15.107:0/3078823705 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7fa420013980 con 0x7fa4380fdfd0 2024-05-17T21:49:27.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:27.932+0000 7fa42f7fe700 1 --2- 172.21.15.107:0/3078823705 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fa4240692a0 0x7fa42406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:49:27.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:27.933+0000 7fa42f7fe700 1 -- 172.21.15.107:0/3078823705 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fa420088620 con 0x7fa4380fdfd0 2024-05-17T21:49:27.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:27.933+0000 7fa436d9d700 1 --2- 172.21.15.107:0/3078823705 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fa4240692a0 0x7fa42406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:49:27.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:27.933+0000 7fa436d9d700 1 --2- 172.21.15.107:0/3078823705 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fa4240692a0 0x7fa42406b750 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa428006fd0 tx=0x7fa428008040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:49:27.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:27.936+0000 7fa42f7fe700 1 -- 172.21.15.107:0/3078823705 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fa420058480 con 0x7fa4380fdfd0 2024-05-17T21:49:28.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:27 smithi107 ceph-mon[28434]: pgmap v180: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s wr, 5 op/s 2024-05-17T21:49:28.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:27 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:49:28.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:28.171+0000 7fa43d0d3700 1 -- 172.21.15.107:0/3078823705 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7fa438070190 con 0x7fa4380fdfd0 2024-05-17T21:49:28.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:27 smithi138 ceph-mon[35948]: pgmap v180: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s wr, 5 op/s 2024-05-17T21:49:28.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:27 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:49:28.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:28.879+0000 7fa42f7fe700 1 -- 172.21.15.107:0/3078823705 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v8) v1 ==== 121+0+0 (secure 0 0 0) 0x7fa420058010 con 0x7fa4380fdfd0 2024-05-17T21:49:28.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:28.881+0000 7fa43d0d3700 1 -- 172.21.15.107:0/3078823705 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fa4240692a0 msgr2=0x7fa42406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:49:28.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:28.881+0000 7fa43d0d3700 1 --2- 172.21.15.107:0/3078823705 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fa4240692a0 0x7fa42406b750 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa428006fd0 tx=0x7fa428008040 comp rx=0 tx=0).stop 2024-05-17T21:49:28.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:28.881+0000 7fa43d0d3700 1 -- 172.21.15.107:0/3078823705 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4380fdfd0 msgr2=0x7fa43810de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:49:28.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:28.881+0000 7fa43d0d3700 1 --2- 172.21.15.107:0/3078823705 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4380fdfd0 0x7fa43810de60 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa42000b3b0 tx=0x7fa420009e30 comp rx=0 tx=0).stop 2024-05-17T21:49:28.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:28.882+0000 7fa43d0d3700 1 -- 172.21.15.107:0/3078823705 shutdown_connections 2024-05-17T21:49:28.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:28.882+0000 7fa43d0d3700 1 --2- 172.21.15.107:0/3078823705 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fa4240692a0 0x7fa42406b750 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:49:28.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:28.882+0000 7fa43d0d3700 1 --2- 172.21.15.107:0/3078823705 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4380fce60 0x7fa43810d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:49:28.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:28.882+0000 7fa43d0d3700 1 --2- 172.21.15.107:0/3078823705 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4380fdfd0 0x7fa43810de60 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:49:28.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:28.882+0000 7fa43d0d3700 1 -- 172.21.15.107:0/3078823705 >> 172.21.15.107:0/3078823705 conn(0x7fa4380f8540 msgr2=0x7fa4380f9b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:49:28.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:28.882+0000 7fa43d0d3700 1 -- 172.21.15.107:0/3078823705 shutdown_connections 2024-05-17T21:49:28.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:28.882+0000 7fa43d0d3700 1 -- 172.21.15.107:0/3078823705 wait complete. 2024-05-17T21:49:29.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:28 smithi138 ceph-mon[35948]: from='client.? 172.21.15.107:0/3078823705' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-17T21:49:29.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:28 smithi138 ceph-mon[35948]: pgmap v181: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s wr, 4 op/s 2024-05-17T21:49:29.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:28 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:49:29.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:28 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:49:29.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:28 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:49:29.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:28 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi138.zckrvs", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T21:49:29.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:28 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi138.zckrvs", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T21:49:29.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:28 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:49:29.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:28 smithi138 ceph-mon[35948]: Deploying daemon mds.cephfs.smithi138.zckrvs on smithi138 2024-05-17T21:49:29.208 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:28 smithi107 ceph-mon[28434]: from='client.? 172.21.15.107:0/3078823705' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-17T21:49:29.208 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:28 smithi107 ceph-mon[28434]: pgmap v181: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s wr, 4 op/s 2024-05-17T21:49:29.208 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:28 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:49:29.208 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:28 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:49:29.208 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:28 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:49:29.208 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:28 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi138.zckrvs", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T21:49:29.209 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:28 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi138.zckrvs", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T21:49:29.209 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:28 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:49:29.209 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:28 smithi107 ceph-mon[28434]: Deploying daemon mds.cephfs.smithi138.zckrvs on smithi138 2024-05-17T21:49:29.549 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-17T21:49:29.559 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi107.front.sepia.ceph.com 2024-05-17T21:49:29.560 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -- bash -c 'ceph fs set cephfs inline_data true --yes-i-really-really-mean-it' 2024-05-17T21:49:29.817 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/mon.smithi107/config 2024-05-17T21:49:30.116 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:29 smithi138 ceph-mon[35948]: from='client.? 172.21.15.107:0/3078823705' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-17T21:49:30.116 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:29 smithi138 ceph-mon[35948]: mds.? [v2:172.21.15.107:6828/2275807100,v1:172.21.15.107:6829/2275807100] up:boot 2024-05-17T21:49:30.116 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:29 smithi138 ceph-mon[35948]: fsmap cephfs:1 {0=cephfs.smithi107.idmbgj=up:active} 2 up:standby 2024-05-17T21:49:30.116 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:29 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi107.drqody"}]: dispatch 2024-05-17T21:49:30.116 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:29 smithi138 ceph-mon[35948]: fsmap cephfs:1 {0=cephfs.smithi107.idmbgj=up:active} 1 up:standby-replay 1 up:standby 2024-05-17T21:49:30.152 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:29 smithi107 ceph-mon[28434]: from='client.? 172.21.15.107:0/3078823705' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-17T21:49:30.152 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:29 smithi107 ceph-mon[28434]: mds.? [v2:172.21.15.107:6828/2275807100,v1:172.21.15.107:6829/2275807100] up:boot 2024-05-17T21:49:30.152 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:29 smithi107 ceph-mon[28434]: fsmap cephfs:1 {0=cephfs.smithi107.idmbgj=up:active} 2 up:standby 2024-05-17T21:49:30.152 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:29 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi107.drqody"}]: dispatch 2024-05-17T21:49:30.152 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:29 smithi107 ceph-mon[28434]: fsmap cephfs:1 {0=cephfs.smithi107.idmbgj=up:active} 1 up:standby-replay 1 up:standby 2024-05-17T21:49:31.194 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:30 smithi107 ceph-mon[28434]: pgmap v182: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s wr, 3 op/s 2024-05-17T21:49:31.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:30 smithi138 ceph-mon[35948]: pgmap v182: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s wr, 3 op/s 2024-05-17T21:49:31.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:31.409+0000 7fcd02eee700 1 -- 172.21.15.107:0/2638984611 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fccfc0fff70 msgr2=0x7fccfc102350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:49:31.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:31.409+0000 7fcd02eee700 1 --2- 172.21.15.107:0/2638984611 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fccfc0fff70 0x7fccfc102350 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fcce8004940 tx=0x7fcce8009e30 comp rx=0 tx=0).stop 2024-05-17T21:49:31.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:31.409+0000 7fcd02eee700 1 -- 172.21.15.107:0/2638984611 shutdown_connections 2024-05-17T21:49:31.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:31.409+0000 7fcd02eee700 1 --2- 172.21.15.107:0/2638984611 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fccfc0fff70 0x7fccfc102350 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:49:31.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:31.409+0000 7fcd02eee700 1 --2- 172.21.15.107:0/2638984611 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fccfc066af0 0x7fccfc0ffa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:49:31.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:31.409+0000 7fcd02eee700 1 -- 172.21.15.107:0/2638984611 >> 172.21.15.107:0/2638984611 conn(0x7fccfc0f7440 msgr2=0x7fccfc0f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:49:31.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:31.409+0000 7fcd02eee700 1 -- 172.21.15.107:0/2638984611 shutdown_connections 2024-05-17T21:49:31.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:31.409+0000 7fcd02eee700 1 -- 172.21.15.107:0/2638984611 wait complete. 2024-05-17T21:49:31.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:31.410+0000 7fcd02eee700 1 Processor -- start 2024-05-17T21:49:31.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:31.410+0000 7fcd02eee700 1 -- start start 2024-05-17T21:49:31.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:31.410+0000 7fcd02eee700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fccfc066af0 0x7fccfc10da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:49:31.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:31.410+0000 7fcd02eee700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fccfc0fff70 0x7fccfc10dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:49:31.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:31.410+0000 7fcd02eee700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccfc10e5a0 con 0x7fccfc066af0 2024-05-17T21:49:31.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:31.410+0000 7fcd02eee700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccfc10ed50 con 0x7fccfc0fff70 2024-05-17T21:49:31.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:31.410+0000 7fcd00c8a700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fccfc066af0 0x7fccfc10da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:49:31.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:31.410+0000 7fcd00c8a700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fccfc066af0 0x7fccfc10da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:53852/0 (socket says 172.21.15.107:53852) 2024-05-17T21:49:31.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:31.410+0000 7fcd00c8a700 1 -- 172.21.15.107:0/4141013327 learned_addr learned my addr 172.21.15.107:0/4141013327 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:49:31.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:31.410+0000 7fccfbfff700 1 --2- 172.21.15.107:0/4141013327 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fccfc0fff70 0x7fccfc10dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:49:31.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:31.411+0000 7fcd00c8a700 1 -- 172.21.15.107:0/4141013327 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fccfc0fff70 msgr2=0x7fccfc10dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:49:31.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:31.411+0000 7fcd00c8a700 1 --2- 172.21.15.107:0/4141013327 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fccfc0fff70 0x7fccfc10dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:49:31.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:31.411+0000 7fcd00c8a700 1 -- 172.21.15.107:0/4141013327 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcce8005040 con 0x7fccfc066af0 2024-05-17T21:49:31.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:31.411+0000 7fccfbfff700 1 --2- 172.21.15.107:0/4141013327 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fccfc0fff70 0x7fccfc10dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T21:49:31.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:31.411+0000 7fcd00c8a700 1 --2- 172.21.15.107:0/4141013327 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fccfc066af0 0x7fccfc10da60 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fccf000eb70 tx=0x7fccf000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:49:31.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:31.411+0000 7fccf97fa700 1 -- 172.21.15.107:0/4141013327 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fccf0007cc0 con 0x7fccfc066af0 2024-05-17T21:49:31.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:31.411+0000 7fccf97fa700 1 -- 172.21.15.107:0/4141013327 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fccf000f040 con 0x7fccfc066af0 2024-05-17T21:49:31.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:31.411+0000 7fcd02eee700 1 -- 172.21.15.107:0/4141013327 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccfc10efd0 con 0x7fccfc066af0 2024-05-17T21:49:31.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:31.411+0000 7fccf97fa700 1 -- 172.21.15.107:0/4141013327 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fccf000cb20 con 0x7fccfc066af0 2024-05-17T21:49:31.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:31.411+0000 7fcd02eee700 1 -- 172.21.15.107:0/4141013327 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccfc10f4c0 con 0x7fccfc066af0 2024-05-17T21:49:31.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:31.412+0000 7fcd02eee700 1 -- 172.21.15.107:0/4141013327 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccfc05f5c0 con 0x7fccfc066af0 2024-05-17T21:49:31.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:31.412+0000 7fccf97fa700 1 -- 172.21.15.107:0/4141013327 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7fccf000cc80 con 0x7fccfc066af0 2024-05-17T21:49:31.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:31.413+0000 7fccf97fa700 1 --2- 172.21.15.107:0/4141013327 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fccec0692a0 0x7fccec06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:49:31.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:31.413+0000 7fccf97fa700 1 -- 172.21.15.107:0/4141013327 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fccf0086c00 con 0x7fccfc066af0 2024-05-17T21:49:31.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:31.413+0000 7fccfbfff700 1 --2- 172.21.15.107:0/4141013327 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fccec0692a0 0x7fccec06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:49:31.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:31.414+0000 7fccfbfff700 1 --2- 172.21.15.107:0/4141013327 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fccec0692a0 0x7fccec06b750 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fcce8009b30 tx=0x7fcce8009e30 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:49:31.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:31.419+0000 7fccf97fa700 1 -- 172.21.15.107:0/4141013327 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fccf004e250 con 0x7fccfc066af0 2024-05-17T21:49:31.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:31.686+0000 7fcd02eee700 1 -- 172.21.15.107:0/4141013327 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fccfc04bed0 con 0x7fccfc066af0 2024-05-17T21:49:32.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:32.031+0000 7fccf97fa700 1 -- 172.21.15.107:0/4141013327 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]=0 inline data enabled v11) v1 ==== 168+0+0 (secure 0 0 0) 0x7fccf0019020 con 0x7fccfc066af0 2024-05-17T21:49:32.026 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:31 smithi107 ceph-mon[28434]: mds.? [v2:172.21.15.138:6826/170628661,v1:172.21.15.138:6827/170628661] up:boot 2024-05-17T21:49:32.026 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:31 smithi107 ceph-mon[28434]: fsmap cephfs:1 {0=cephfs.smithi107.idmbgj=up:active} 1 up:standby-replay 2 up:standby 2024-05-17T21:49:32.026 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:31 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi138.zckrvs"}]: dispatch 2024-05-17T21:49:32.026 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:31 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:49:32.026 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:31 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:49:32.026 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:31 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:49:32.026 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:31 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:49:32.026 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:31 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:49:32.027 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:31 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:49:32.027 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:31 smithi107 ceph-mon[28434]: from='client.? 172.21.15.107:0/4141013327' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-17T21:49:32.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:32.034+0000 7fcd02eee700 1 -- 172.21.15.107:0/4141013327 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fccec0692a0 msgr2=0x7fccec06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:49:32.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:32.034+0000 7fcd02eee700 1 --2- 172.21.15.107:0/4141013327 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fccec0692a0 0x7fccec06b750 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fcce8009b30 tx=0x7fcce8009e30 comp rx=0 tx=0).stop 2024-05-17T21:49:32.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:32.034+0000 7fcd02eee700 1 -- 172.21.15.107:0/4141013327 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fccfc066af0 msgr2=0x7fccfc10da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:49:32.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:32.034+0000 7fcd02eee700 1 --2- 172.21.15.107:0/4141013327 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fccfc066af0 0x7fccfc10da60 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fccf000eb70 tx=0x7fccf000b040 comp rx=0 tx=0).stop 2024-05-17T21:49:32.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:32.034+0000 7fcd02eee700 1 -- 172.21.15.107:0/4141013327 shutdown_connections 2024-05-17T21:49:32.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:32.034+0000 7fcd02eee700 1 --2- 172.21.15.107:0/4141013327 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fccec0692a0 0x7fccec06b750 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:49:32.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:32.034+0000 7fcd02eee700 1 --2- 172.21.15.107:0/4141013327 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fccfc066af0 0x7fccfc10da60 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:49:32.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:32.034+0000 7fcd02eee700 1 --2- 172.21.15.107:0/4141013327 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fccfc0fff70 0x7fccfc10dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:49:32.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:32.034+0000 7fcd02eee700 1 -- 172.21.15.107:0/4141013327 >> 172.21.15.107:0/4141013327 conn(0x7fccfc0f7440 msgr2=0x7fccfc0f7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:49:32.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:32.034+0000 7fcd02eee700 1 -- 172.21.15.107:0/4141013327 shutdown_connections 2024-05-17T21:49:32.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:32.034+0000 7fcd02eee700 1 -- 172.21.15.107:0/4141013327 wait complete. 2024-05-17T21:49:32.028 INFO:teuthology.orchestra.run.smithi107.stderr:inline data enabled 2024-05-17T21:49:32.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:31 smithi138 ceph-mon[35948]: mds.? [v2:172.21.15.138:6826/170628661,v1:172.21.15.138:6827/170628661] up:boot 2024-05-17T21:49:32.078 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:31 smithi138 ceph-mon[35948]: fsmap cephfs:1 {0=cephfs.smithi107.idmbgj=up:active} 1 up:standby-replay 2 up:standby 2024-05-17T21:49:32.078 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:31 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi138.zckrvs"}]: dispatch 2024-05-17T21:49:32.078 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:31 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:49:32.078 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:31 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:49:32.078 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:31 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:49:32.078 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:31 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:49:32.078 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:31 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:49:32.078 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:31 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:49:32.078 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:31 smithi138 ceph-mon[35948]: from='client.? 172.21.15.107:0/4141013327' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-17T21:49:32.836 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-17T21:49:32.846 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi107.front.sepia.ceph.com 2024-05-17T21:49:32.846 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -- bash -c 'ceph fs dump' 2024-05-17T21:49:33.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:32 smithi107 ceph-mon[28434]: Health check failed: 1 filesystem with deprecated feature inline_data (FS_INLINE_DATA_DEPRECATED) 2024-05-17T21:49:33.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:32 smithi107 ceph-mon[28434]: from='client.? 172.21.15.107:0/4141013327' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]': finished 2024-05-17T21:49:33.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:32 smithi107 ceph-mon[28434]: mds.? [v2:172.21.15.138:6824/2779634919,v1:172.21.15.138:6825/2779634919] up:standby-replay 2024-05-17T21:49:33.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:32 smithi107 ceph-mon[28434]: Dropping low affinity active daemon mds.cephfs.smithi107.idmbgj in favor of higher affinity standby. 2024-05-17T21:49:33.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:32 smithi107 ceph-mon[28434]: Replacing daemon mds.cephfs.smithi107.idmbgj as rank 0 with standby daemon mds.cephfs.smithi138.gwbtyd 2024-05-17T21:49:33.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:32 smithi107 ceph-mon[28434]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-17T21:49:33.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:32 smithi107 ceph-mon[28434]: fsmap cephfs:1 {0=cephfs.smithi107.idmbgj=up:active} 1 up:standby-replay 2 up:standby 2024-05-17T21:49:33.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:32 smithi107 ceph-mon[28434]: osdmap e46: 6 total, 6 up, 6 in 2024-05-17T21:49:33.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:32 smithi107 ceph-mon[28434]: fsmap cephfs:1/1 {0=cephfs.smithi138.gwbtyd=up:replay} 2 up:standby 2024-05-17T21:49:33.104 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:32 smithi107 ceph-mon[28434]: pgmap v184: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s wr, 4 op/s 2024-05-17T21:49:33.104 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:32 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:49:33.104 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:32 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:49:33.104 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:32 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:49:33.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:32 smithi138 ceph-mon[35948]: Health check failed: 1 filesystem with deprecated feature inline_data (FS_INLINE_DATA_DEPRECATED) 2024-05-17T21:49:33.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:32 smithi138 ceph-mon[35948]: from='client.? 172.21.15.107:0/4141013327' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]': finished 2024-05-17T21:49:33.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:32 smithi138 ceph-mon[35948]: mds.? [v2:172.21.15.138:6824/2779634919,v1:172.21.15.138:6825/2779634919] up:standby-replay 2024-05-17T21:49:33.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:32 smithi138 ceph-mon[35948]: Dropping low affinity active daemon mds.cephfs.smithi107.idmbgj in favor of higher affinity standby. 2024-05-17T21:49:33.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:32 smithi138 ceph-mon[35948]: Replacing daemon mds.cephfs.smithi107.idmbgj as rank 0 with standby daemon mds.cephfs.smithi138.gwbtyd 2024-05-17T21:49:33.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:32 smithi138 ceph-mon[35948]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-17T21:49:33.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:32 smithi138 ceph-mon[35948]: fsmap cephfs:1 {0=cephfs.smithi107.idmbgj=up:active} 1 up:standby-replay 2 up:standby 2024-05-17T21:49:33.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:32 smithi138 ceph-mon[35948]: osdmap e46: 6 total, 6 up, 6 in 2024-05-17T21:49:33.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:32 smithi138 ceph-mon[35948]: fsmap cephfs:1/1 {0=cephfs.smithi138.gwbtyd=up:replay} 2 up:standby 2024-05-17T21:49:33.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:32 smithi138 ceph-mon[35948]: pgmap v184: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s wr, 4 op/s 2024-05-17T21:49:33.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:32 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:49:33.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:32 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:49:33.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:32 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:49:34.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:34 smithi138 ceph-mon[35948]: mds.? [v2:172.21.15.107:6826/1460186079,v1:172.21.15.107:6827/1460186079] up:boot 2024-05-17T21:49:34.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:34 smithi138 ceph-mon[35948]: mds.? [v2:172.21.15.107:6828/2275807100,v1:172.21.15.107:6829/2275807100] up:standby 2024-05-17T21:49:34.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:34 smithi138 ceph-mon[35948]: mds.? [v2:172.21.15.138:6824/2779634919,v1:172.21.15.138:6825/2779634919] up:reconnect 2024-05-17T21:49:34.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:34 smithi138 ceph-mon[35948]: fsmap cephfs:1/1 {0=cephfs.smithi138.gwbtyd=up:reconnect} 3 up:standby 2024-05-17T21:49:34.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:34 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi107.idmbgj"}]: dispatch 2024-05-17T21:49:34.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:34 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:49:34.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:34 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:49:34.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:34 smithi107 ceph-mon[28434]: mds.? [v2:172.21.15.107:6826/1460186079,v1:172.21.15.107:6827/1460186079] up:boot 2024-05-17T21:49:34.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:34 smithi107 ceph-mon[28434]: mds.? [v2:172.21.15.107:6828/2275807100,v1:172.21.15.107:6829/2275807100] up:standby 2024-05-17T21:49:34.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:34 smithi107 ceph-mon[28434]: mds.? [v2:172.21.15.138:6824/2779634919,v1:172.21.15.138:6825/2779634919] up:reconnect 2024-05-17T21:49:34.353 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:34 smithi107 ceph-mon[28434]: fsmap cephfs:1/1 {0=cephfs.smithi138.gwbtyd=up:reconnect} 3 up:standby 2024-05-17T21:49:34.353 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:34 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi107.idmbgj"}]: dispatch 2024-05-17T21:49:34.353 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:34 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:49:34.353 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:34 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:49:34.382 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/mon.smithi107/config 2024-05-17T21:49:35.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:35 smithi107 ceph-mon[28434]: mds.? [v2:172.21.15.138:6824/2779634919,v1:172.21.15.138:6825/2779634919] up:rejoin 2024-05-17T21:49:35.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:35 smithi107 ceph-mon[28434]: fsmap cephfs:1/1 {0=cephfs.smithi138.gwbtyd=up:rejoin} 3 up:standby 2024-05-17T21:49:35.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:35 smithi107 ceph-mon[28434]: daemon mds.cephfs.smithi138.gwbtyd is now active in filesystem cephfs as rank 0 2024-05-17T21:49:35.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:35 smithi107 ceph-mon[28434]: pgmap v185: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1 op/s 2024-05-17T21:49:35.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:35 smithi107 ceph-mon[28434]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-17T21:49:35.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:35 smithi107 ceph-mon[28434]: mds.? [v2:172.21.15.138:6824/2779634919,v1:172.21.15.138:6825/2779634919] up:active 2024-05-17T21:49:35.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:35 smithi107 ceph-mon[28434]: mds.? [v2:172.21.15.138:6826/170628661,v1:172.21.15.138:6827/170628661] up:standby 2024-05-17T21:49:35.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:35 smithi107 ceph-mon[28434]: fsmap cephfs:1 {0=cephfs.smithi138.gwbtyd=up:active} 1 up:standby-replay 2 up:standby 2024-05-17T21:49:35.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:35 smithi138 ceph-mon[35948]: mds.? [v2:172.21.15.138:6824/2779634919,v1:172.21.15.138:6825/2779634919] up:rejoin 2024-05-17T21:49:35.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:35 smithi138 ceph-mon[35948]: fsmap cephfs:1/1 {0=cephfs.smithi138.gwbtyd=up:rejoin} 3 up:standby 2024-05-17T21:49:35.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:35 smithi138 ceph-mon[35948]: daemon mds.cephfs.smithi138.gwbtyd is now active in filesystem cephfs as rank 0 2024-05-17T21:49:35.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:35 smithi138 ceph-mon[35948]: pgmap v185: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1 op/s 2024-05-17T21:49:35.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:35 smithi138 ceph-mon[35948]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-17T21:49:35.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:35 smithi138 ceph-mon[35948]: mds.? [v2:172.21.15.138:6824/2779634919,v1:172.21.15.138:6825/2779634919] up:active 2024-05-17T21:49:35.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:35 smithi138 ceph-mon[35948]: mds.? [v2:172.21.15.138:6826/170628661,v1:172.21.15.138:6827/170628661] up:standby 2024-05-17T21:49:35.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:35 smithi138 ceph-mon[35948]: fsmap cephfs:1 {0=cephfs.smithi138.gwbtyd=up:active} 1 up:standby-replay 2 up:standby 2024-05-17T21:49:36.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:36.865+0000 7f50d6961700 1 -- 172.21.15.107:0/2683972883 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f50d00fce60 msgr2=0x7f50d00fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:49:36.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:36.865+0000 7f50d6961700 1 --2- 172.21.15.107:0/2683972883 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f50d00fce60 0x7f50d00fd270 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f50c0004940 tx=0x7f50c0009e30 comp rx=0 tx=0).stop 2024-05-17T21:49:36.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:36.865+0000 7f50d6961700 1 -- 172.21.15.107:0/2683972883 shutdown_connections 2024-05-17T21:49:36.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:36.865+0000 7f50d6961700 1 --2- 172.21.15.107:0/2683972883 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f50d00fdfd0 0x7f50d00fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:49:36.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:36.865+0000 7f50d6961700 1 --2- 172.21.15.107:0/2683972883 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f50d00fce60 0x7f50d00fd270 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:49:36.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:36.865+0000 7f50d6961700 1 -- 172.21.15.107:0/2683972883 >> 172.21.15.107:0/2683972883 conn(0x7f50d00f8540 msgr2=0x7f50d00fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:49:36.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:36.865+0000 7f50d6961700 1 -- 172.21.15.107:0/2683972883 shutdown_connections 2024-05-17T21:49:36.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:36.865+0000 7f50d6961700 1 -- 172.21.15.107:0/2683972883 wait complete. 2024-05-17T21:49:36.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:36.865+0000 7f50d6961700 1 Processor -- start 2024-05-17T21:49:36.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:36.866+0000 7f50d6961700 1 -- start start 2024-05-17T21:49:36.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:36.866+0000 7f50d6961700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f50d00fce60 0x7f50d010d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:49:36.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:36.866+0000 7f50d6961700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f50d00fdfd0 0x7f50d010de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:49:36.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:36.866+0000 7f50d6961700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50d010e470 con 0x7f50d00fce60 2024-05-17T21:49:36.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:36.866+0000 7f50d6961700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50d010e5e0 con 0x7f50d00fdfd0 2024-05-17T21:49:36.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:36.866+0000 7f50cffff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f50d00fce60 0x7f50d010d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:49:36.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:36.866+0000 7f50cffff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f50d00fce60 0x7f50d010d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54644/0 (socket says 172.21.15.107:54644) 2024-05-17T21:49:36.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:36.866+0000 7f50cffff700 1 -- 172.21.15.107:0/2220533172 learned_addr learned my addr 172.21.15.107:0/2220533172 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:49:36.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:36.866+0000 7f50cf7fe700 1 --2- 172.21.15.107:0/2220533172 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f50d00fdfd0 0x7f50d010de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:49:36.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:36.866+0000 7f50cffff700 1 -- 172.21.15.107:0/2220533172 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f50d00fdfd0 msgr2=0x7f50d010de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:49:36.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:36.866+0000 7f50cffff700 1 --2- 172.21.15.107:0/2220533172 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f50d00fdfd0 0x7f50d010de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:49:36.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:36.866+0000 7f50cffff700 1 -- 172.21.15.107:0/2220533172 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50c0005040 con 0x7f50d00fce60 2024-05-17T21:49:36.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:36.866+0000 7f50cf7fe700 1 --2- 172.21.15.107:0/2220533172 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f50d00fdfd0 0x7f50d010de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T21:49:36.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:36.866+0000 7f50cffff700 1 --2- 172.21.15.107:0/2220533172 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f50d00fce60 0x7f50d010d930 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f50c000b3b0 tx=0x7f50c0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:49:36.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:36.866+0000 7f50ccff9700 1 -- 172.21.15.107:0/2220533172 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f50c000e040 con 0x7f50d00fce60 2024-05-17T21:49:36.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:36.867+0000 7f50d6961700 1 -- 172.21.15.107:0/2220533172 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50d0112fe0 con 0x7f50d00fce60 2024-05-17T21:49:36.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:36.867+0000 7f50ccff9700 1 -- 172.21.15.107:0/2220533172 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f50c000bcc0 con 0x7f50d00fce60 2024-05-17T21:49:36.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:36.867+0000 7f50ccff9700 1 -- 172.21.15.107:0/2220533172 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f50c001c630 con 0x7f50d00fce60 2024-05-17T21:49:36.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:36.867+0000 7f50d6961700 1 -- 172.21.15.107:0/2220533172 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50d0113510 con 0x7f50d00fce60 2024-05-17T21:49:36.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:36.867+0000 7f50d6961700 1 -- 172.21.15.107:0/2220533172 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50d004c710 con 0x7f50d00fce60 2024-05-17T21:49:36.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:36.868+0000 7f50ccff9700 1 -- 172.21.15.107:0/2220533172 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f50c0013980 con 0x7f50d00fce60 2024-05-17T21:49:36.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:36.868+0000 7f50ccff9700 1 --2- 172.21.15.107:0/2220533172 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f50b80692a0 0x7f50b806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:49:36.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:36.868+0000 7f50ccff9700 1 -- 172.21.15.107:0/2220533172 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f50c0088000 con 0x7f50d00fce60 2024-05-17T21:49:36.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:36.869+0000 7f50cf7fe700 1 --2- 172.21.15.107:0/2220533172 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f50b80692a0 0x7f50b806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:49:36.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:36.869+0000 7f50cf7fe700 1 --2- 172.21.15.107:0/2220533172 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f50b80692a0 0x7f50b806b750 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f50c4009fd0 tx=0x7f50c4009f20 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:49:36.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:36.873+0000 7f50ccff9700 1 -- 172.21.15.107:0/2220533172 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f50c0057cf0 con 0x7f50d00fce60 2024-05-17T21:49:37.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:37.105+0000 7f50d6961700 1 -- 172.21.15.107:0/2220533172 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f50d01058e0 con 0x7f50d00fce60 2024-05-17T21:49:37.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:37.105+0000 7f50ccff9700 1 -- 172.21.15.107:0/2220533172 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1840 (secure 0 0 0) 0x7f50c0057880 con 0x7f50d00fce60 2024-05-17T21:49:37.098 INFO:teuthology.orchestra.run.smithi107.stdout:e15 2024-05-17T21:49:37.098 INFO:teuthology.orchestra.run.smithi107.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-17T21:49:37.099 INFO:teuthology.orchestra.run.smithi107.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-17T21:49:37.099 INFO:teuthology.orchestra.run.smithi107.stdout:legacy client fscid: 1 2024-05-17T21:49:37.099 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:49:37.099 INFO:teuthology.orchestra.run.smithi107.stdout:Filesystem 'cephfs' (1) 2024-05-17T21:49:37.099 INFO:teuthology.orchestra.run.smithi107.stdout:fs_name cephfs 2024-05-17T21:49:37.099 INFO:teuthology.orchestra.run.smithi107.stdout:epoch 15 2024-05-17T21:49:37.099 INFO:teuthology.orchestra.run.smithi107.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-17T21:49:37.099 INFO:teuthology.orchestra.run.smithi107.stdout:created 2024-05-17T21:49:16.836333+0000 2024-05-17T21:49:37.099 INFO:teuthology.orchestra.run.smithi107.stdout:modified 2024-05-17T21:49:34.873628+0000 2024-05-17T21:49:37.099 INFO:teuthology.orchestra.run.smithi107.stdout:tableserver 0 2024-05-17T21:49:37.099 INFO:teuthology.orchestra.run.smithi107.stdout:root 0 2024-05-17T21:49:37.099 INFO:teuthology.orchestra.run.smithi107.stdout:session_timeout 60 2024-05-17T21:49:37.099 INFO:teuthology.orchestra.run.smithi107.stdout:session_autoclose 300 2024-05-17T21:49:37.100 INFO:teuthology.orchestra.run.smithi107.stdout:max_file_size 1099511627776 2024-05-17T21:49:37.100 INFO:teuthology.orchestra.run.smithi107.stdout:required_client_features {} 2024-05-17T21:49:37.100 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure 0 2024-05-17T21:49:37.100 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure_osd_epoch 46 2024-05-17T21:49:37.100 INFO:teuthology.orchestra.run.smithi107.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-17T21:49:37.100 INFO:teuthology.orchestra.run.smithi107.stdout:max_mds 1 2024-05-17T21:49:37.100 INFO:teuthology.orchestra.run.smithi107.stdout:in 0 2024-05-17T21:49:37.100 INFO:teuthology.orchestra.run.smithi107.stdout:up {0=24259} 2024-05-17T21:49:37.100 INFO:teuthology.orchestra.run.smithi107.stdout:failed 2024-05-17T21:49:37.100 INFO:teuthology.orchestra.run.smithi107.stdout:damaged 2024-05-17T21:49:37.100 INFO:teuthology.orchestra.run.smithi107.stdout:stopped 2024-05-17T21:49:37.100 INFO:teuthology.orchestra.run.smithi107.stdout:data_pools [3] 2024-05-17T21:49:37.100 INFO:teuthology.orchestra.run.smithi107.stdout:metadata_pool 2 2024-05-17T21:49:37.100 INFO:teuthology.orchestra.run.smithi107.stdout:inline_data enabled 2024-05-17T21:49:37.101 INFO:teuthology.orchestra.run.smithi107.stdout:balancer 2024-05-17T21:49:37.101 INFO:teuthology.orchestra.run.smithi107.stdout:standby_count_wanted 1 2024-05-17T21:49:37.101 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi138.gwbtyd{0:24259} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.138:6824/2779634919,v1:172.21.15.138:6825/2779634919] compat {c=[1],r=[1],i=[7ff]}] 2024-05-17T21:49:37.101 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.drqody{0:14486} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.107:6828/2275807100,v1:172.21.15.107:6829/2275807100] compat {c=[1],r=[1],i=[7ff]}] 2024-05-17T21:49:37.101 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:49:37.101 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:49:37.101 INFO:teuthology.orchestra.run.smithi107.stdout:Standby daemons: 2024-05-17T21:49:37.101 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:49:37.101 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.idmbgj{-1:14498} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.107:6826/1460186079,v1:172.21.15.107:6827/1460186079] compat {c=[1],r=[1],i=[7ff]}] 2024-05-17T21:49:37.101 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi138.zckrvs{-1:24267} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.138:6826/170628661,v1:172.21.15.138:6827/170628661] compat {c=[1],r=[1],i=[7ff]}] 2024-05-17T21:49:37.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:37.107+0000 7f50d6961700 1 -- 172.21.15.107:0/2220533172 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f50b80692a0 msgr2=0x7f50b806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:49:37.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:37.107+0000 7f50d6961700 1 --2- 172.21.15.107:0/2220533172 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f50b80692a0 0x7f50b806b750 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f50c4009fd0 tx=0x7f50c4009f20 comp rx=0 tx=0).stop 2024-05-17T21:49:37.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:37.107+0000 7f50d6961700 1 -- 172.21.15.107:0/2220533172 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f50d00fce60 msgr2=0x7f50d010d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:49:37.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:37.107+0000 7f50d6961700 1 --2- 172.21.15.107:0/2220533172 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f50d00fce60 0x7f50d010d930 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f50c000b3b0 tx=0x7f50c0009e30 comp rx=0 tx=0).stop 2024-05-17T21:49:37.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:37.107+0000 7f50d6961700 1 -- 172.21.15.107:0/2220533172 shutdown_connections 2024-05-17T21:49:37.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:37.107+0000 7f50d6961700 1 --2- 172.21.15.107:0/2220533172 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f50b80692a0 0x7f50b806b750 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:49:37.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:37.107+0000 7f50d6961700 1 --2- 172.21.15.107:0/2220533172 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f50d00fce60 0x7f50d010d930 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:49:37.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:37.107+0000 7f50d6961700 1 --2- 172.21.15.107:0/2220533172 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f50d00fdfd0 0x7f50d010de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:49:37.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:37.107+0000 7f50d6961700 1 -- 172.21.15.107:0/2220533172 >> 172.21.15.107:0/2220533172 conn(0x7f50d00f8540 msgr2=0x7f50d00f9c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:49:37.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:37.107+0000 7f50d6961700 1 -- 172.21.15.107:0/2220533172 shutdown_connections 2024-05-17T21:49:37.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:37.107+0000 7f50d6961700 1 -- 172.21.15.107:0/2220533172 wait complete. 2024-05-17T21:49:37.103 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 15 2024-05-17T21:49:37.771 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:37 smithi107 ceph-mon[28434]: pgmap v186: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1 op/s 2024-05-17T21:49:37.771 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:37 smithi107 ceph-mon[28434]: from='client.? 172.21.15.107:0/2220533172' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-17T21:49:37.799 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -- bash -c 'ceph --format=json fs dump | jq -e ".filesystems | length == 1"' 2024-05-17T21:49:37.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:37 smithi138 ceph-mon[35948]: pgmap v186: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1 op/s 2024-05-17T21:49:37.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:37 smithi138 ceph-mon[35948]: from='client.? 172.21.15.107:0/2220533172' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-17T21:49:38.082 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/mon.smithi107/config 2024-05-17T21:49:39.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:39 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:49:39.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:39 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:49:39.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:39 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:49:39.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:39 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T21:49:39.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:39 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:49:39.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:39 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:49:39.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:39 smithi107 ceph-mon[28434]: pgmap v187: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 5.9 KiB/s rd, 0 B/s wr, 6 op/s 2024-05-17T21:49:39.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:39 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:49:39.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:39 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:49:39.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:39 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:49:39.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:39 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T21:49:39.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:39 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:49:39.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:39 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:49:39.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:39 smithi138 ceph-mon[35948]: pgmap v187: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 5.9 KiB/s rd, 0 B/s wr, 6 op/s 2024-05-17T21:49:39.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:39.815+0000 7f22954c0700 1 -- 172.21.15.107:0/303896647 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22900fce70 msgr2=0x7f22900fd280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:49:39.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:39.815+0000 7f22954c0700 1 --2- 172.21.15.107:0/303896647 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22900fce70 0x7f22900fd280 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2280004940 tx=0x7f2280009e30 comp rx=0 tx=0).stop 2024-05-17T21:49:39.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:39.815+0000 7f22954c0700 1 -- 172.21.15.107:0/303896647 shutdown_connections 2024-05-17T21:49:39.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:39.815+0000 7f22954c0700 1 --2- 172.21.15.107:0/303896647 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f22900fdfe0 0x7f22900fe450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:49:39.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:39.815+0000 7f22954c0700 1 --2- 172.21.15.107:0/303896647 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22900fce70 0x7f22900fd280 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:49:39.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:39.815+0000 7f22954c0700 1 -- 172.21.15.107:0/303896647 >> 172.21.15.107:0/303896647 conn(0x7f22900f8510 msgr2=0x7f22900fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:49:39.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:39.815+0000 7f22954c0700 1 -- 172.21.15.107:0/303896647 shutdown_connections 2024-05-17T21:49:39.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:39.815+0000 7f22954c0700 1 -- 172.21.15.107:0/303896647 wait complete. 2024-05-17T21:49:39.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:39.815+0000 7f22954c0700 1 Processor -- start 2024-05-17T21:49:39.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:39.815+0000 7f22954c0700 1 -- start start 2024-05-17T21:49:39.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:39.816+0000 7f22954c0700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22900fce70 0x7f229010d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:49:39.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:39.816+0000 7f22954c0700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f22900fdfe0 0x7f229010ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:49:39.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:39.816+0000 7f22954c0700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f229010e4d0 con 0x7f22900fce70 2024-05-17T21:49:39.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:39.816+0000 7f22954c0700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f229010e640 con 0x7f22900fdfe0 2024-05-17T21:49:39.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:39.816+0000 7f228effd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22900fce70 0x7f229010d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:49:39.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:39.816+0000 7f228effd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22900fce70 0x7f229010d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54666/0 (socket says 172.21.15.107:54666) 2024-05-17T21:49:39.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:39.816+0000 7f228effd700 1 -- 172.21.15.107:0/150275288 learned_addr learned my addr 172.21.15.107:0/150275288 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:49:39.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:39.816+0000 7f228e7fc700 1 --2- 172.21.15.107:0/150275288 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f22900fdfe0 0x7f229010ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:49:39.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:39.816+0000 7f228effd700 1 -- 172.21.15.107:0/150275288 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f22900fdfe0 msgr2=0x7f229010ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:49:39.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:39.816+0000 7f228effd700 1 --2- 172.21.15.107:0/150275288 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f22900fdfe0 0x7f229010ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:49:39.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:39.816+0000 7f228effd700 1 -- 172.21.15.107:0/150275288 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2280005040 con 0x7f22900fce70 2024-05-17T21:49:39.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:39.816+0000 7f228effd700 1 --2- 172.21.15.107:0/150275288 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22900fce70 0x7f229010d990 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f2280009150 tx=0x7f2280009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:49:39.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:39.816+0000 7f22777fe700 1 -- 172.21.15.107:0/150275288 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f228000e040 con 0x7f22900fce70 2024-05-17T21:49:39.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:39.816+0000 7f22954c0700 1 -- 172.21.15.107:0/150275288 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2290112fa0 con 0x7f22900fce70 2024-05-17T21:49:39.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:39.816+0000 7f22777fe700 1 -- 172.21.15.107:0/150275288 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f228000bbf0 con 0x7f22900fce70 2024-05-17T21:49:39.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:39.816+0000 7f22777fe700 1 -- 172.21.15.107:0/150275288 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f228001c630 con 0x7f22900fce70 2024-05-17T21:49:39.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:39.817+0000 7f22954c0700 1 -- 172.21.15.107:0/150275288 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22901134d0 con 0x7f22900fce70 2024-05-17T21:49:39.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:39.818+0000 7f22954c0700 1 -- 172.21.15.107:0/150275288 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f229004c710 con 0x7f22900fce70 2024-05-17T21:49:39.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:39.818+0000 7f22777fe700 1 -- 172.21.15.107:0/150275288 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f228001c8a0 con 0x7f22900fce70 2024-05-17T21:49:39.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:39.818+0000 7f22777fe700 1 --2- 172.21.15.107:0/150275288 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f22780692a0 0x7f227806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:49:39.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:39.818+0000 7f228e7fc700 1 --2- 172.21.15.107:0/150275288 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f22780692a0 0x7f227806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:49:39.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:39.818+0000 7f22777fe700 1 -- 172.21.15.107:0/150275288 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f2280088500 con 0x7f22900fce70 2024-05-17T21:49:39.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:39.819+0000 7f228e7fc700 1 --2- 172.21.15.107:0/150275288 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f22780692a0 0x7f227806b750 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2284004f40 tx=0x7f228400b040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:49:39.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:39.823+0000 7f22777fe700 1 -- 172.21.15.107:0/150275288 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f22800582e0 con 0x7f22900fce70 2024-05-17T21:49:40.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:40.060+0000 7f22954c0700 1 -- 172.21.15.107:0/150275288 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f22901058d0 con 0x7f22900fce70 2024-05-17T21:49:40.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:40.061+0000 7f22777fe700 1 -- 172.21.15.107:0/150275288 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4705 (secure 0 0 0) 0x7f2280013e60 con 0x7f22900fce70 2024-05-17T21:49:40.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:40.063+0000 7f22954c0700 1 -- 172.21.15.107:0/150275288 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f22780692a0 msgr2=0x7f227806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:49:40.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:40.063+0000 7f22954c0700 1 --2- 172.21.15.107:0/150275288 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f22780692a0 0x7f227806b750 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2284004f40 tx=0x7f228400b040 comp rx=0 tx=0).stop 2024-05-17T21:49:40.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:40.063+0000 7f22954c0700 1 -- 172.21.15.107:0/150275288 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22900fce70 msgr2=0x7f229010d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:49:40.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:40.063+0000 7f22954c0700 1 --2- 172.21.15.107:0/150275288 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22900fce70 0x7f229010d990 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f2280009150 tx=0x7f2280009e30 comp rx=0 tx=0).stop 2024-05-17T21:49:40.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:40.063+0000 7f22954c0700 1 -- 172.21.15.107:0/150275288 shutdown_connections 2024-05-17T21:49:40.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:40.063+0000 7f22954c0700 1 --2- 172.21.15.107:0/150275288 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f22780692a0 0x7f227806b750 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:49:40.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:40.063+0000 7f22954c0700 1 --2- 172.21.15.107:0/150275288 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22900fce70 0x7f229010d990 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:49:40.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:40.063+0000 7f22954c0700 1 --2- 172.21.15.107:0/150275288 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f22900fdfe0 0x7f229010ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:49:40.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:40.063+0000 7f22954c0700 1 -- 172.21.15.107:0/150275288 >> 172.21.15.107:0/150275288 conn(0x7f22900f8510 msgr2=0x7f22900f9bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:49:40.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:40.063+0000 7f22954c0700 1 -- 172.21.15.107:0/150275288 shutdown_connections 2024-05-17T21:49:40.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:40.063+0000 7f22954c0700 1 -- 172.21.15.107:0/150275288 wait complete. 2024-05-17T21:49:40.057 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 15 2024-05-17T21:49:40.067 INFO:teuthology.orchestra.run.smithi107.stdout:true 2024-05-17T21:49:40.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:40 smithi107 ceph-mon[28434]: from='client.? 172.21.15.107:0/150275288' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T21:49:40.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:40 smithi138 ceph-mon[35948]: from='client.? 172.21.15.107:0/150275288' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T21:49:40.745 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -- bash -c 'while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done' 2024-05-17T21:49:41.129 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/mon.smithi107/config 2024-05-17T21:49:41.534 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:41 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:49:41.534 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:41 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:49:41.535 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:41 smithi107 ceph-mon[28434]: pgmap v188: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 6.7 KiB/s rd, 7 op/s 2024-05-17T21:49:41.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:41 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:49:41.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:41 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:49:41.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:41 smithi138 ceph-mon[35948]: pgmap v188: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 6.7 KiB/s rd, 7 op/s 2024-05-17T21:49:42.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:42.657+0000 7fb1d65d1700 1 -- 172.21.15.107:0/4276077760 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb1d00fe420 msgr2=0x7fb1d00fe830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:49:42.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:42.657+0000 7fb1d65d1700 1 --2- 172.21.15.107:0/4276077760 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb1d00fe420 0x7fb1d00fe830 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fb1c0004940 tx=0x7fb1c0009e30 comp rx=0 tx=0).stop 2024-05-17T21:49:42.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:42.657+0000 7fb1d65d1700 1 -- 172.21.15.107:0/4276077760 shutdown_connections 2024-05-17T21:49:42.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:42.657+0000 7fb1d65d1700 1 --2- 172.21.15.107:0/4276077760 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1d00fed70 0x7fb1d00faeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:49:42.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:42.657+0000 7fb1d65d1700 1 --2- 172.21.15.107:0/4276077760 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb1d00fe420 0x7fb1d00fe830 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:49:42.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:42.657+0000 7fb1d65d1700 1 -- 172.21.15.107:0/4276077760 >> 172.21.15.107:0/4276077760 conn(0x7fb1d00f6be0 msgr2=0x7fb1d00f8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:49:42.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:42.657+0000 7fb1d65d1700 1 -- 172.21.15.107:0/4276077760 shutdown_connections 2024-05-17T21:49:42.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:42.657+0000 7fb1d65d1700 1 -- 172.21.15.107:0/4276077760 wait complete. 2024-05-17T21:49:42.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:42.657+0000 7fb1d65d1700 1 Processor -- start 2024-05-17T21:49:42.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:42.657+0000 7fb1d65d1700 1 -- start start 2024-05-17T21:49:42.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:42.658+0000 7fb1d65d1700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb1d00fe420 0x7fb1d010d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:49:42.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:42.658+0000 7fb1d65d1700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1d00fed70 0x7fb1d010de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:49:42.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:42.658+0000 7fb1d65d1700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d010e430 con 0x7fb1d00fe420 2024-05-17T21:49:42.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:42.658+0000 7fb1d65d1700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d010e5a0 con 0x7fb1d00fed70 2024-05-17T21:49:42.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:42.658+0000 7fb1cffff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb1d00fe420 0x7fb1d010d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:49:42.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:42.658+0000 7fb1cffff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb1d00fe420 0x7fb1d010d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54678/0 (socket says 172.21.15.107:54678) 2024-05-17T21:49:42.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:42.658+0000 7fb1cffff700 1 -- 172.21.15.107:0/567273748 learned_addr learned my addr 172.21.15.107:0/567273748 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:49:42.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:42.658+0000 7fb1c7fff700 1 --2- 172.21.15.107:0/567273748 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1d00fed70 0x7fb1d010de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:49:42.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:42.658+0000 7fb1cffff700 1 -- 172.21.15.107:0/567273748 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1d00fed70 msgr2=0x7fb1d010de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:49:42.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:42.658+0000 7fb1cffff700 1 --2- 172.21.15.107:0/567273748 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1d00fed70 0x7fb1d010de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:49:42.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:42.658+0000 7fb1cffff700 1 -- 172.21.15.107:0/567273748 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1c0005040 con 0x7fb1d00fe420 2024-05-17T21:49:42.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:42.658+0000 7fb1c7fff700 1 --2- 172.21.15.107:0/567273748 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1d00fed70 0x7fb1d010de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T21:49:42.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:42.658+0000 7fb1cffff700 1 --2- 172.21.15.107:0/567273748 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb1d00fe420 0x7fb1d010d8f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fb1c0009150 tx=0x7fb1c0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:49:42.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:42.658+0000 7fb1cd7fa700 1 -- 172.21.15.107:0/567273748 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb1c000e040 con 0x7fb1d00fe420 2024-05-17T21:49:42.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:42.658+0000 7fb1cd7fa700 1 -- 172.21.15.107:0/567273748 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb1c000bcc0 con 0x7fb1d00fe420 2024-05-17T21:49:42.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:42.658+0000 7fb1cd7fa700 1 -- 172.21.15.107:0/567273748 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb1c001c630 con 0x7fb1d00fe420 2024-05-17T21:49:42.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:42.658+0000 7fb1d65d1700 1 -- 172.21.15.107:0/567273748 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1d0112fa0 con 0x7fb1d00fe420 2024-05-17T21:49:42.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:42.658+0000 7fb1d65d1700 1 -- 172.21.15.107:0/567273748 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1d01134d0 con 0x7fb1d00fe420 2024-05-17T21:49:42.655 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:42.659+0000 7fb1d65d1700 1 -- 172.21.15.107:0/567273748 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1d004c710 con 0x7fb1d00fe420 2024-05-17T21:49:42.655 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:42.660+0000 7fb1cd7fa700 1 -- 172.21.15.107:0/567273748 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7fb1c0013420 con 0x7fb1d00fe420 2024-05-17T21:49:42.655 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:42.660+0000 7fb1cd7fa700 1 --2- 172.21.15.107:0/567273748 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fb1bc0692a0 0x7fb1bc06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:49:42.655 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:42.660+0000 7fb1c7fff700 1 --2- 172.21.15.107:0/567273748 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fb1bc0692a0 0x7fb1bc06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:49:42.655 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:42.660+0000 7fb1cd7fa700 1 -- 172.21.15.107:0/567273748 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fb1c0087e70 con 0x7fb1d00fe420 2024-05-17T21:49:42.655 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:42.661+0000 7fb1c7fff700 1 --2- 172.21.15.107:0/567273748 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fb1bc0692a0 0x7fb1bc06b750 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb1b8007fd0 tx=0x7fb1b8009040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:49:42.657 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:42.665+0000 7fb1cd7fa700 1 -- 172.21.15.107:0/567273748 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fb1c0057c80 con 0x7fb1d00fe420 2024-05-17T21:49:42.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:42.943+0000 7fb1d65d1700 1 -- 172.21.15.107:0/567273748 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mds versions", "format": "json"} v 0) v1 -- 0x7fb1d0113960 con 0x7fb1d00fe420 2024-05-17T21:49:42.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:42.944+0000 7fb1cd7fa700 1 -- 172.21.15.107:0/567273748 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mds versions", "format": "json"}]=0 v15) v1 ==== 78+0+99 (secure 0 0 0) 0x7fb1c0057810 con 0x7fb1d00fe420 2024-05-17T21:49:42.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:42.946+0000 7fb1d65d1700 1 -- 172.21.15.107:0/567273748 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fb1bc0692a0 msgr2=0x7fb1bc06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:49:42.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:42.946+0000 7fb1d65d1700 1 --2- 172.21.15.107:0/567273748 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fb1bc0692a0 0x7fb1bc06b750 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb1b8007fd0 tx=0x7fb1b8009040 comp rx=0 tx=0).stop 2024-05-17T21:49:42.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:42.946+0000 7fb1d65d1700 1 -- 172.21.15.107:0/567273748 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb1d00fe420 msgr2=0x7fb1d010d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:49:42.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:42.946+0000 7fb1d65d1700 1 --2- 172.21.15.107:0/567273748 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb1d00fe420 0x7fb1d010d8f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fb1c0009150 tx=0x7fb1c0009e30 comp rx=0 tx=0).stop 2024-05-17T21:49:42.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:42.946+0000 7fb1d65d1700 1 -- 172.21.15.107:0/567273748 shutdown_connections 2024-05-17T21:49:42.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:42.946+0000 7fb1d65d1700 1 --2- 172.21.15.107:0/567273748 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fb1bc0692a0 0x7fb1bc06b750 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:49:42.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:42.946+0000 7fb1d65d1700 1 --2- 172.21.15.107:0/567273748 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb1d00fe420 0x7fb1d010d8f0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:49:42.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:42.946+0000 7fb1d65d1700 1 --2- 172.21.15.107:0/567273748 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1d00fed70 0x7fb1d010de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:49:42.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:42.946+0000 7fb1d65d1700 1 -- 172.21.15.107:0/567273748 >> 172.21.15.107:0/567273748 conn(0x7fb1d00f6be0 msgr2=0x7fb1d00f8290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:49:42.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:42.946+0000 7fb1d65d1700 1 -- 172.21.15.107:0/567273748 shutdown_connections 2024-05-17T21:49:42.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:42.946+0000 7fb1d65d1700 1 -- 172.21.15.107:0/567273748 wait complete. 2024-05-17T21:49:42.949 INFO:teuthology.orchestra.run.smithi107.stdout:true 2024-05-17T21:49:43.573 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:43 smithi107 ceph-mon[28434]: pgmap v189: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 6.5 KiB/s rd, 7 op/s 2024-05-17T21:49:43.573 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:43 smithi107 ceph-mon[28434]: from='client.? 172.21.15.107:0/567273748' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-05-17T21:49:43.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:43 smithi138 ceph-mon[35948]: pgmap v189: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 6.5 KiB/s rd, 7 op/s 2024-05-17T21:49:43.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:43 smithi138 ceph-mon[35948]: from='client.? 172.21.15.107:0/567273748' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-05-17T21:49:43.892 INFO:teuthology.run_tasks:Running task fs.pre_upgrade_save... 2024-05-17T21:49:43.904 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T21:49:45.702 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:45 smithi107 ceph-mon[28434]: pgmap v190: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 6.0 KiB/s rd, 0 B/s wr, 6 op/s 2024-05-17T21:49:45.796 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/mon.smithi107/config 2024-05-17T21:49:45.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:45 smithi138 ceph-mon[35948]: pgmap v190: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 6.0 KiB/s rd, 0 B/s wr, 6 op/s 2024-05-17T21:49:47.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:47 smithi107 ceph-mon[28434]: pgmap v191: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 4.7 KiB/s rd, 0 B/s wr, 5 op/s 2024-05-17T21:49:47.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:47 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:49:47.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:47 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:49:47.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:47 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:49:47.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:47 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T21:49:47.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:47 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:49:47.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:47 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:49:47.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:47 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:49:47.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:47.807+0000 7f9f663ba700 1 -- 172.21.15.107:0/4132093824 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f600fbd40 msgr2=0x7f9f600fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:49:47.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:47.807+0000 7f9f663ba700 1 --2- 172.21.15.107:0/4132093824 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f600fbd40 0x7f9f600fc1b0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f9f50004940 tx=0x7f9f50009e30 comp rx=0 tx=0).stop 2024-05-17T21:49:47.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:47.807+0000 7f9f663ba700 1 -- 172.21.15.107:0/4132093824 shutdown_connections 2024-05-17T21:49:47.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:47.807+0000 7f9f663ba700 1 --2- 172.21.15.107:0/4132093824 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f600fbd40 0x7f9f600fc1b0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:49:47.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:47.807+0000 7f9f663ba700 1 --2- 172.21.15.107:0/4132093824 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f60101d90 0x7f9f60102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:49:47.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:47.807+0000 7f9f663ba700 1 -- 172.21.15.107:0/4132093824 >> 172.21.15.107:0/4132093824 conn(0x7f9f60071d00 msgr2=0x7f9f60072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:49:47.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:47.807+0000 7f9f663ba700 1 -- 172.21.15.107:0/4132093824 shutdown_connections 2024-05-17T21:49:47.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:47.807+0000 7f9f663ba700 1 -- 172.21.15.107:0/4132093824 wait complete. 2024-05-17T21:49:47.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:47.808+0000 7f9f663ba700 1 Processor -- start 2024-05-17T21:49:47.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:47.808+0000 7f9f663ba700 1 -- start start 2024-05-17T21:49:47.804 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:47.808+0000 7f9f663ba700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f600fbd40 0x7f9f6010eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:49:47.804 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:47.808+0000 7f9f663ba700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f60101d90 0x7f9f6010f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:49:47.804 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:47.808+0000 7f9f663ba700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f6010f6c0 con 0x7f9f600fbd40 2024-05-17T21:49:47.804 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:47.808+0000 7f9f663ba700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f6010f800 con 0x7f9f60101d90 2024-05-17T21:49:47.804 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:47.808+0000 7f9f5ffff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f600fbd40 0x7f9f6010eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:49:47.804 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:47.808+0000 7f9f5ffff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f600fbd40 0x7f9f6010eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59192/0 (socket says 172.21.15.107:59192) 2024-05-17T21:49:47.804 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:47.808+0000 7f9f5ffff700 1 -- 172.21.15.107:0/4154070183 learned_addr learned my addr 172.21.15.107:0/4154070183 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:49:47.804 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:47.809+0000 7f9f5ffff700 1 -- 172.21.15.107:0/4154070183 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f60101d90 msgr2=0x7f9f6010f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:49:47.804 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:47.809+0000 7f9f5ffff700 1 --2- 172.21.15.107:0/4154070183 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f60101d90 0x7f9f6010f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:49:47.804 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:47.809+0000 7f9f5ffff700 1 -- 172.21.15.107:0/4154070183 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f50005040 con 0x7f9f600fbd40 2024-05-17T21:49:47.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:47.809+0000 7f9f5ffff700 1 --2- 172.21.15.107:0/4154070183 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f600fbd40 0x7f9f6010eb30 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f9f4800c8c0 tx=0x7f9f480044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:49:47.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:47.809+0000 7f9f5cff9700 1 -- 172.21.15.107:0/4154070183 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f4800a980 con 0x7f9f600fbd40 2024-05-17T21:49:47.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:47.809+0000 7f9f663ba700 1 -- 172.21.15.107:0/4154070183 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f60117ba0 con 0x7f9f600fbd40 2024-05-17T21:49:47.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:47.809+0000 7f9f5cff9700 1 -- 172.21.15.107:0/4154070183 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9f4800ce60 con 0x7f9f600fbd40 2024-05-17T21:49:47.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:47.809+0000 7f9f5cff9700 1 -- 172.21.15.107:0/4154070183 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f480115e0 con 0x7f9f600fbd40 2024-05-17T21:49:47.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:47.809+0000 7f9f663ba700 1 -- 172.21.15.107:0/4154070183 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f60118100 con 0x7f9f600fbd40 2024-05-17T21:49:47.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:47.810+0000 7f9f663ba700 1 -- 172.21.15.107:0/4154070183 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f600fd430 con 0x7f9f600fbd40 2024-05-17T21:49:47.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:47.811+0000 7f9f5cff9700 1 -- 172.21.15.107:0/4154070183 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f9f48012400 con 0x7f9f600fbd40 2024-05-17T21:49:47.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:47.811+0000 7f9f5cff9700 1 --2- 172.21.15.107:0/4154070183 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f9f4c0692a0 0x7f9f4c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:49:47.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:47.811+0000 7f9f5cff9700 1 -- 172.21.15.107:0/4154070183 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f9f480872f0 con 0x7f9f600fbd40 2024-05-17T21:49:47.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:47.811+0000 7f9f5f7fe700 1 --2- 172.21.15.107:0/4154070183 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f9f4c0692a0 0x7f9f4c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:49:47.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:47.811+0000 7f9f5f7fe700 1 --2- 172.21.15.107:0/4154070183 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f9f4c0692a0 0x7f9f4c06b750 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9f50004f90 tx=0x7f9f50009e30 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:49:47.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:47.815+0000 7f9f5cff9700 1 -- 172.21.15.107:0/4154070183 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f9f480535d0 con 0x7f9f600fbd40 2024-05-17T21:49:47.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:47 smithi138 ceph-mon[35948]: pgmap v191: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 4.7 KiB/s rd, 0 B/s wr, 5 op/s 2024-05-17T21:49:47.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:47 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:49:47.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:47 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:49:47.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:47 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:49:47.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:47 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T21:49:47.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:47 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:49:47.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:47 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:49:47.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:47 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:49:48.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:48.048+0000 7f9f663ba700 1 -- 172.21.15.107:0/4154070183 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9f6004bed0 con 0x7f9f600fbd40 2024-05-17T21:49:48.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:48.049+0000 7f9f5cff9700 1 -- 172.21.15.107:0/4154070183 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4705 (secure 0 0 0) 0x7f9f480126b0 con 0x7f9f600fbd40 2024-05-17T21:49:48.042 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:49:48.042 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":15,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14498,"name":"cephfs.smithi107.idmbgj","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6827/1460186079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1460186079},{"type":"v1","addr":"172.21.15.107:6827","nonce":1460186079}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":24267,"name":"cephfs.smithi138.zckrvs","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.138:6827/170628661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":170628661},{"type":"v1","addr":"172.21.15.138:6827","nonce":170628661}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15}],"filesystems":[{"mdsmap":{"epoch":15,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T21:49:16.836333+0000","modified":"2024-05-17T21:49:34.873628+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24259},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14486":{"gid":14486,"name":"cephfs.smithi107.drqody","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":2,"addr":"172.21.15.107:6829/2275807100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":2275807100},{"type":"v1","addr":"172.21.15.107:6829","nonce":2275807100}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24259":{"gid":24259,"name":"cephfs.smithi138.gwbtyd","rank":0,"incarnation":12,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6825/2779634919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":2779634919},{"type":"v1","addr":"172.21.15.138:6825","nonce":2779634919}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":1},"id":1}]} 2024-05-17T21:49:48.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:48.051+0000 7f9f663ba700 1 -- 172.21.15.107:0/4154070183 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f9f4c0692a0 msgr2=0x7f9f4c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:49:48.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:48.051+0000 7f9f663ba700 1 --2- 172.21.15.107:0/4154070183 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f9f4c0692a0 0x7f9f4c06b750 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9f50004f90 tx=0x7f9f50009e30 comp rx=0 tx=0).stop 2024-05-17T21:49:48.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:48.051+0000 7f9f663ba700 1 -- 172.21.15.107:0/4154070183 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f600fbd40 msgr2=0x7f9f6010eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:49:48.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:48.051+0000 7f9f663ba700 1 --2- 172.21.15.107:0/4154070183 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f600fbd40 0x7f9f6010eb30 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f9f4800c8c0 tx=0x7f9f480044c0 comp rx=0 tx=0).stop 2024-05-17T21:49:48.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:48.051+0000 7f9f663ba700 1 -- 172.21.15.107:0/4154070183 shutdown_connections 2024-05-17T21:49:48.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:48.051+0000 7f9f663ba700 1 --2- 172.21.15.107:0/4154070183 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f9f4c0692a0 0x7f9f4c06b750 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:49:48.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:48.051+0000 7f9f663ba700 1 --2- 172.21.15.107:0/4154070183 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f600fbd40 0x7f9f6010eb30 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:49:48.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:48.051+0000 7f9f663ba700 1 --2- 172.21.15.107:0/4154070183 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f60101d90 0x7f9f6010f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:49:48.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:48.051+0000 7f9f663ba700 1 -- 172.21.15.107:0/4154070183 >> 172.21.15.107:0/4154070183 conn(0x7f9f60071d00 msgr2=0x7f9f60104a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:49:48.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:48.051+0000 7f9f663ba700 1 -- 172.21.15.107:0/4154070183 shutdown_connections 2024-05-17T21:49:48.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:48.051+0000 7f9f663ba700 1 -- 172.21.15.107:0/4154070183 wait complete. 2024-05-17T21:49:48.045 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 15 2024-05-17T21:49:48.672 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:48 smithi107 ceph-mon[28434]: from='client.? 172.21.15.107:0/4154070183' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T21:49:48.698 DEBUG:tasks.fs:fs fscid=1,name=cephfs state = {'epoch': 15, 'max_mds': 1, 'flags': 50} 2024-05-17T21:49:48.699 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-05-17T21:49:48.726 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-05-17T21:49:48.726 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.1': {}} 2024-05-17T21:49:48.727 INFO:tasks.ceph_fuse:client.0 config is {} 2024-05-17T21:49:48.727 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-17T21:49:48.727 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-17T21:49:48.727 INFO:tasks.ceph_fuse:client.1 config is {} 2024-05-17T21:49:48.727 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-17T21:49:48.727 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-05-17T21:49:48.727 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:49:48.727 DEBUG:teuthology.orchestra.run.smithi107:> ip netns list 2024-05-17T21:49:48.744 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:49:48.744 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link delete ceph-brx 2024-05-17T21:49:48.812 INFO:teuthology.orchestra.run.smithi107.stderr:Cannot find device "ceph-brx" 2024-05-17T21:49:48.813 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-17T21:49:48.814 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:49:48.814 DEBUG:teuthology.orchestra.run.smithi138:> ip netns list 2024-05-17T21:49:48.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:48 smithi138 ceph-mon[35948]: from='client.? 172.21.15.107:0/4154070183' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T21:49:48.835 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:49:48.835 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link delete ceph-brx 2024-05-17T21:49:48.909 INFO:teuthology.orchestra.run.smithi138.stderr:Cannot find device "ceph-brx" 2024-05-17T21:49:48.910 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-17T21:49:48.911 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-05-17T21:49:48.911 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -- ceph fs ls 2024-05-17T21:49:49.164 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/mon.smithi107/config 2024-05-17T21:49:49.558 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:49 smithi107 ceph-mon[28434]: pgmap v192: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 5.2 KiB/s rd, 0 B/s wr, 6 op/s 2024-05-17T21:49:49.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:49 smithi138 ceph-mon[35948]: pgmap v192: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 5.2 KiB/s rd, 0 B/s wr, 6 op/s 2024-05-17T21:49:50.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:50.720+0000 7fe3ad331700 1 -- 172.21.15.107:0/800569699 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3a8102de0 msgr2=0x7fe3a81031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:49:50.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:50.720+0000 7fe3ad331700 1 --2- 172.21.15.107:0/800569699 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3a8102de0 0x7fe3a81031b0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe398004940 tx=0x7fe398009e30 comp rx=0 tx=0).stop 2024-05-17T21:49:50.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:50.720+0000 7fe3ad331700 1 -- 172.21.15.107:0/800569699 shutdown_connections 2024-05-17T21:49:50.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:50.720+0000 7fe3ad331700 1 --2- 172.21.15.107:0/800569699 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3a8103780 0x7fe3a80fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:49:50.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:50.720+0000 7fe3ad331700 1 --2- 172.21.15.107:0/800569699 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3a8102de0 0x7fe3a81031b0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:49:50.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:50.720+0000 7fe3ad331700 1 -- 172.21.15.107:0/800569699 >> 172.21.15.107:0/800569699 conn(0x7fe3a80f8990 msgr2=0x7fe3a80fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:49:50.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:50.720+0000 7fe3ad331700 1 -- 172.21.15.107:0/800569699 shutdown_connections 2024-05-17T21:49:50.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:50.720+0000 7fe3ad331700 1 -- 172.21.15.107:0/800569699 wait complete. 2024-05-17T21:49:50.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:50.720+0000 7fe3ad331700 1 Processor -- start 2024-05-17T21:49:50.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:50.721+0000 7fe3ad331700 1 -- start start 2024-05-17T21:49:50.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:50.721+0000 7fe3ad331700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3a8102de0 0x7fe3a8070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:49:50.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:50.721+0000 7fe3ad331700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3a8103780 0x7fe3a80707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:49:50.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:50.721+0000 7fe3ad331700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a80721b0 con 0x7fe3a8103780 2024-05-17T21:49:50.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:50.721+0000 7fe3ad331700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a8070ce0 con 0x7fe3a8102de0 2024-05-17T21:49:50.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:50.721+0000 7fe39e7fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3a8103780 0x7fe3a80707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:49:50.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:50.721+0000 7fe39e7fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3a8103780 0x7fe3a80707a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59208/0 (socket says 172.21.15.107:59208) 2024-05-17T21:49:50.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:50.721+0000 7fe39e7fc700 1 -- 172.21.15.107:0/3954086758 learned_addr learned my addr 172.21.15.107:0/3954086758 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:49:50.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:50.721+0000 7fe3a6ffd700 1 --2- 172.21.15.107:0/3954086758 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3a8102de0 0x7fe3a8070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:49:50.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:50.721+0000 7fe39e7fc700 1 -- 172.21.15.107:0/3954086758 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3a8102de0 msgr2=0x7fe3a8070260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:49:50.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:50.721+0000 7fe39e7fc700 1 --2- 172.21.15.107:0/3954086758 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3a8102de0 0x7fe3a8070260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:49:50.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:50.721+0000 7fe39e7fc700 1 -- 172.21.15.107:0/3954086758 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe398005040 con 0x7fe3a8103780 2024-05-17T21:49:50.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:50.721+0000 7fe3a6ffd700 1 --2- 172.21.15.107:0/3954086758 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3a8102de0 0x7fe3a8070260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:49:50.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:50.721+0000 7fe39e7fc700 1 --2- 172.21.15.107:0/3954086758 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3a8103780 0x7fe3a80707a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe390002a50 tx=0x7fe3900046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:49:50.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:50.721+0000 7fe39ffff700 1 -- 172.21.15.107:0/3954086758 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe390007cc0 con 0x7fe3a8103780 2024-05-17T21:49:50.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:50.722+0000 7fe39ffff700 1 -- 172.21.15.107:0/3954086758 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe39000f040 con 0x7fe3a8103780 2024-05-17T21:49:50.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:50.722+0000 7fe3ad331700 1 -- 172.21.15.107:0/3954086758 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3a8070fc0 con 0x7fe3a8103780 2024-05-17T21:49:50.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:50.722+0000 7fe39ffff700 1 -- 172.21.15.107:0/3954086758 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe390009630 con 0x7fe3a8103780 2024-05-17T21:49:50.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:50.722+0000 7fe3ad331700 1 -- 172.21.15.107:0/3954086758 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3a811a090 con 0x7fe3a8103780 2024-05-17T21:49:50.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:50.723+0000 7fe3ad331700 1 -- 172.21.15.107:0/3954086758 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3a81000f0 con 0x7fe3a8103780 2024-05-17T21:49:50.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:50.723+0000 7fe39ffff700 1 -- 172.21.15.107:0/3954086758 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7fe390009790 con 0x7fe3a8103780 2024-05-17T21:49:50.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:50.724+0000 7fe39ffff700 1 --2- 172.21.15.107:0/3954086758 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fe3940692a0 0x7fe39406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:49:50.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:50.724+0000 7fe39ffff700 1 -- 172.21.15.107:0/3954086758 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fe390086ea0 con 0x7fe3a8103780 2024-05-17T21:49:50.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:50.724+0000 7fe3a6ffd700 1 --2- 172.21.15.107:0/3954086758 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fe3940692a0 0x7fe39406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:49:50.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:50.724+0000 7fe3a6ffd700 1 --2- 172.21.15.107:0/3954086758 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fe3940692a0 0x7fe39406b750 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe398004560 tx=0x7fe398009e30 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:49:50.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:50.728+0000 7fe39ffff700 1 -- 172.21.15.107:0/3954086758 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fe3900531b0 con 0x7fe3a8103780 2024-05-17T21:49:50.968 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:50.975+0000 7fe3ad331700 1 -- 172.21.15.107:0/3954086758 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe3a8109ee0 con 0x7fe3a8103780 2024-05-17T21:49:50.968 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:50.975+0000 7fe39ffff700 1 -- 172.21.15.107:0/3954086758 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v15) v1 ==== 53+0+83 (secure 0 0 0) 0x7fe3900567d0 con 0x7fe3a8103780 2024-05-17T21:49:50.968 INFO:teuthology.orchestra.run.smithi107.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-05-17T21:49:50.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:50.977+0000 7fe3ad331700 1 -- 172.21.15.107:0/3954086758 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fe3940692a0 msgr2=0x7fe39406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:49:50.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:50.977+0000 7fe3ad331700 1 --2- 172.21.15.107:0/3954086758 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fe3940692a0 0x7fe39406b750 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe398004560 tx=0x7fe398009e30 comp rx=0 tx=0).stop 2024-05-17T21:49:50.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:50.977+0000 7fe3ad331700 1 -- 172.21.15.107:0/3954086758 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3a8103780 msgr2=0x7fe3a80707a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:49:50.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:50.977+0000 7fe3ad331700 1 --2- 172.21.15.107:0/3954086758 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3a8103780 0x7fe3a80707a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe390002a50 tx=0x7fe3900046b0 comp rx=0 tx=0).stop 2024-05-17T21:49:50.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:50.978+0000 7fe3ad331700 1 -- 172.21.15.107:0/3954086758 shutdown_connections 2024-05-17T21:49:50.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:50.978+0000 7fe3ad331700 1 --2- 172.21.15.107:0/3954086758 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fe3940692a0 0x7fe39406b750 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:49:50.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:50.978+0000 7fe3ad331700 1 --2- 172.21.15.107:0/3954086758 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3a8102de0 0x7fe3a8070260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:49:50.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:50.978+0000 7fe3ad331700 1 --2- 172.21.15.107:0/3954086758 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3a8103780 0x7fe3a80707a0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:49:50.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:50.978+0000 7fe3ad331700 1 -- 172.21.15.107:0/3954086758 >> 172.21.15.107:0/3954086758 conn(0x7fe3a80f8990 msgr2=0x7fe3a80fa6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:49:50.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:50.978+0000 7fe3ad331700 1 -- 172.21.15.107:0/3954086758 shutdown_connections 2024-05-17T21:49:50.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:50.978+0000 7fe3ad331700 1 -- 172.21.15.107:0/3954086758 wait complete. 2024-05-17T21:49:51.569 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:51 smithi107 ceph-mon[28434]: pgmap v193: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 2 op/s 2024-05-17T21:49:51.570 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:51 smithi107 ceph-mon[28434]: from='client.? 172.21.15.107:0/3954086758' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-17T21:49:51.594 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-17T21:49:51.594 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-17T21:49:51.594 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi107.front.sepia.ceph.com 2024-05-17T21:49:51.594 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-17T21:49:51.594 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-17T21:49:51.595 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-17T21:49:51.595 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-17T21:49:51.595 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-17T21:49:51.595 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-17T21:49:51.595 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:49:51.596 DEBUG:teuthology.orchestra.run.smithi107:> ip addr 2024-05-17T21:49:51.613 INFO:teuthology.orchestra.run.smithi107.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-17T21:49:51.613 INFO:teuthology.orchestra.run.smithi107.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-17T21:49:51.613 INFO:teuthology.orchestra.run.smithi107.stdout: inet 127.0.0.1/8 scope host lo 2024-05-17T21:49:51.613 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2024-05-17T21:49:51.613 INFO:teuthology.orchestra.run.smithi107.stdout: inet6 ::1/128 scope host 2024-05-17T21:49:51.613 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2024-05-17T21:49:51.614 INFO:teuthology.orchestra.run.smithi107.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-17T21:49:51.614 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 0c:c4:7a:d8:a2:0a brd ff:ff:ff:ff:ff:ff 2024-05-17T21:49:51.614 INFO:teuthology.orchestra.run.smithi107.stdout: altname enp7s0f0 2024-05-17T21:49:51.614 INFO:teuthology.orchestra.run.smithi107.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-17T21:49:51.614 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 0c:c4:7a:d8:a2:0b brd ff:ff:ff:ff:ff:ff 2024-05-17T21:49:51.614 INFO:teuthology.orchestra.run.smithi107.stdout: altname enp7s0f1 2024-05-17T21:49:51.614 INFO:teuthology.orchestra.run.smithi107.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-17T21:49:51.614 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 0c:c4:7a:8f:47:ea brd ff:ff:ff:ff:ff:ff 2024-05-17T21:49:51.614 INFO:teuthology.orchestra.run.smithi107.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-17T21:49:51.614 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 0c:c4:7a:8f:47:eb brd ff:ff:ff:ff:ff:ff 2024-05-17T21:49:51.615 INFO:teuthology.orchestra.run.smithi107.stdout: inet 172.21.15.107/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-05-17T21:49:51.615 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft 42302sec preferred_lft 42302sec 2024-05-17T21:49:51.615 INFO:teuthology.orchestra.run.smithi107.stdout: inet6 fe80::ec4:7aff:fe8f:47eb/64 scope link 2024-05-17T21:49:51.615 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2024-05-17T21:49:51.618 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-17T21:49:51.618 DEBUG:teuthology.orchestra.run.smithi107:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-17T21:49:51.618 DEBUG:teuthology.orchestra.run.smithi107:> set -e 2024-05-17T21:49:51.618 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link add name ceph-brx type bridge 2024-05-17T21:49:51.618 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip addr flush dev ceph-brx 2024-05-17T21:49:51.618 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link set ceph-brx up 2024-05-17T21:49:51.618 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-17T21:49:51.618 DEBUG:teuthology.orchestra.run.smithi107:> ') 2024-05-17T21:49:51.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-17T21:49:51.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T21:49:51.804 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:49:51.804 DEBUG:teuthology.orchestra.run.smithi107:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-17T21:49:51.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:51 smithi138 ceph-mon[35948]: pgmap v193: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 2 op/s 2024-05-17T21:49:51.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:51 smithi138 ceph-mon[35948]: from='client.? 172.21.15.107:0/3954086758' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-17T21:49:51.829 INFO:teuthology.orchestra.run.smithi107.stdout:1 2024-05-17T21:49:51.830 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:49:51.830 DEBUG:teuthology.orchestra.run.smithi107:> route 2024-05-17T21:49:51.905 INFO:teuthology.orchestra.run.smithi107.stdout:Kernel IP routing table 2024-05-17T21:49:51.905 INFO:teuthology.orchestra.run.smithi107.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-17T21:49:51.905 INFO:teuthology.orchestra.run.smithi107.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-17T21:49:51.905 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-17T21:49:51.905 INFO:teuthology.orchestra.run.smithi107.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-17T21:49:51.905 DEBUG:teuthology.orchestra.run.smithi107:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-17T21:49:51.906 DEBUG:teuthology.orchestra.run.smithi107:> set -e 2024-05-17T21:49:51.906 DEBUG:teuthology.orchestra.run.smithi107:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-17T21:49:51.906 DEBUG:teuthology.orchestra.run.smithi107:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-17T21:49:51.906 DEBUG:teuthology.orchestra.run.smithi107:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-17T21:49:51.906 DEBUG:teuthology.orchestra.run.smithi107:> ') 2024-05-17T21:49:51.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-17T21:49:52.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T21:49:52.205 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:49:52.205 DEBUG:teuthology.orchestra.run.smithi107:> ip netns list 2024-05-17T21:49:52.221 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:49:52.221 DEBUG:teuthology.orchestra.run.smithi107:> ip netns list-id 2024-05-17T21:49:52.277 DEBUG:teuthology.orchestra.run.smithi107:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-17T21:49:52.277 DEBUG:teuthology.orchestra.run.smithi107:> set -e 2024-05-17T21:49:52.277 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T21:49:52.277 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-05-17T21:49:52.277 DEBUG:teuthology.orchestra.run.smithi107:> ') 2024-05-17T21:49:52.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-17T21:49:52.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T21:49:52.396 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-05-17T21:49:52.396 DEBUG:teuthology.orchestra.run.smithi107:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-17T21:49:52.396 DEBUG:teuthology.orchestra.run.smithi107:> set -e 2024-05-17T21:49:52.396 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-05-17T21:49:52.396 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-17T21:49:52.396 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-05-17T21:49:52.397 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-05-17T21:49:52.397 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-05-17T21:49:52.397 DEBUG:teuthology.orchestra.run.smithi107:> ') 2024-05-17T21:49:52.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-17T21:49:52.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T21:49:52.522 DEBUG:teuthology.orchestra.run.smithi107:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-17T21:49:52.522 DEBUG:teuthology.orchestra.run.smithi107:> set -e 2024-05-17T21:49:52.522 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link set brx.0 up 2024-05-17T21:49:52.522 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link set dev brx.0 master ceph-brx 2024-05-17T21:49:52.522 DEBUG:teuthology.orchestra.run.smithi107:> ') 2024-05-17T21:49:52.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-17T21:49:52.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T21:49:52.599 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-17T21:49:52.599 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-17T21:49:52.599 DEBUG:teuthology.orchestra.run.smithi107:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-17T21:49:52.614 INFO:teuthology.orchestra.run.smithi107.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-17T21:49:52.615 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-17T21:49:52.615 DEBUG:teuthology.orchestra.run.smithi107:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-17T21:49:52.670 DEBUG:teuthology.orchestra.run.smithi107:> sudo modprobe fuse 2024-05-17T21:49:52.735 DEBUG:teuthology.orchestra.run.smithi107:> cat /proc/self/mounts | awk '{print $2}' 2024-05-17T21:49:52.792 INFO:teuthology.orchestra.run.smithi107.stdout:/proc 2024-05-17T21:49:52.792 INFO:teuthology.orchestra.run.smithi107.stdout:/sys 2024-05-17T21:49:52.793 INFO:teuthology.orchestra.run.smithi107.stdout:/dev 2024-05-17T21:49:52.793 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/kernel/security 2024-05-17T21:49:52.793 INFO:teuthology.orchestra.run.smithi107.stdout:/dev/shm 2024-05-17T21:49:52.793 INFO:teuthology.orchestra.run.smithi107.stdout:/dev/pts 2024-05-17T21:49:52.793 INFO:teuthology.orchestra.run.smithi107.stdout:/run 2024-05-17T21:49:52.793 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/fs/cgroup 2024-05-17T21:49:52.793 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/fs/pstore 2024-05-17T21:49:52.793 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/fs/bpf 2024-05-17T21:49:52.793 INFO:teuthology.orchestra.run.smithi107.stdout:/ 2024-05-17T21:49:52.793 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/fs/selinux 2024-05-17T21:49:52.793 INFO:teuthology.orchestra.run.smithi107.stdout:/proc/sys/fs/binfmt_misc 2024-05-17T21:49:52.793 INFO:teuthology.orchestra.run.smithi107.stdout:/dev/hugepages 2024-05-17T21:49:52.793 INFO:teuthology.orchestra.run.smithi107.stdout:/dev/mqueue 2024-05-17T21:49:52.793 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/kernel/debug 2024-05-17T21:49:52.793 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/kernel/tracing 2024-05-17T21:49:52.794 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/fs/fuse/connections 2024-05-17T21:49:52.794 INFO:teuthology.orchestra.run.smithi107.stdout:/run/credentials/systemd-sysctl.service 2024-05-17T21:49:52.794 INFO:teuthology.orchestra.run.smithi107.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-17T21:49:52.794 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/kernel/config 2024-05-17T21:49:52.794 INFO:teuthology.orchestra.run.smithi107.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-17T21:49:52.794 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/nfs/rpc_pipefs 2024-05-17T21:49:52.794 INFO:teuthology.orchestra.run.smithi107.stdout:/run/user/1000 2024-05-17T21:49:52.794 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/ceph 2024-05-17T21:49:52.794 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay 2024-05-17T21:49:52.794 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/610d72d82b79351f7c5fbd3e7779546dd4b8e2f6a01f82cfcd3f5229ceff9617/merged 2024-05-17T21:49:52.794 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/0a0408c6a615242b8d17c4431a780d670b77e129ae89083b2929f6c8ff978855/merged 2024-05-17T21:49:52.794 INFO:teuthology.orchestra.run.smithi107.stdout:/run/user/0 2024-05-17T21:49:52.794 INFO:teuthology.orchestra.run.smithi107.stdout:/proc/sys/fs/binfmt_misc 2024-05-17T21:49:52.795 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/9381a36ade860e6b15f6e6cf506e5aa69858f89b4b1902055f15ba6f73ffad27/merged 2024-05-17T21:49:52.795 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/f646dd11480b7be6e75b143db72b9459df68ba5420f9035b59e8ea50d1746fbb/merged 2024-05-17T21:49:52.795 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/27a5fffcea787df19abe9e6fa8e40ff99bb376c968a41e2c0d1e5943476a302d/merged 2024-05-17T21:49:52.795 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/d3ede44c1b5567f0a6ad92c17f3bff07c5b4d221a56358983f9fc1764d2b6521/merged 2024-05-17T21:49:52.795 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/6f086ae2021a3ad7d4c71822b6459288d191b7e57f06aec3cfd3f842b98b8f18/merged 2024-05-17T21:49:52.795 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/a39a7e9ff95cbb38afa23c13c67947c4ef11001f9e05af58ddaff00ed235f426/merged 2024-05-17T21:49:52.795 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/207dbd72bbb7b6f39c892b0f75e14b80ca9b7b4468c360f18d7646a32eb5e0fd/merged 2024-05-17T21:49:52.795 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/333f76c9031a380ae2d65a6981859b7b23261979818e8c34bdc0da233e65766f/merged 2024-05-17T21:49:52.795 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/396b62cced7aec1fe573bc2b2e1c107802001c84e8ae938f6c7fb0e1e8e09667/merged 2024-05-17T21:49:52.795 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/75b37426d8b94efa3aed5b75e338c5495fa1d8bb5dcc4b79a20e9a3db834179c/merged 2024-05-17T21:49:52.795 INFO:teuthology.orchestra.run.smithi107.stdout:/run/netns 2024-05-17T21:49:52.795 INFO:teuthology.orchestra.run.smithi107.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T21:49:52.795 INFO:teuthology.orchestra.run.smithi107.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T21:49:52.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:49:52.796 DEBUG:teuthology.orchestra.run.smithi107:> ls /sys/fs/fuse/connections 2024-05-17T21:49:52.848 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-17T21:49:52.848 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-17T21:49:52.890 DEBUG:teuthology.orchestra.run.smithi107:> sudo modprobe fuse 2024-05-17T21:49:52.914 DEBUG:teuthology.orchestra.run.smithi107:> cat /proc/self/mounts | awk '{print $2}' 2024-05-17T21:49:52.971 INFO:teuthology.orchestra.run.smithi107.stdout:/proc 2024-05-17T21:49:52.971 INFO:teuthology.orchestra.run.smithi107.stdout:/sys 2024-05-17T21:49:52.971 INFO:teuthology.orchestra.run.smithi107.stdout:/dev 2024-05-17T21:49:52.971 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/kernel/security 2024-05-17T21:49:52.971 INFO:teuthology.orchestra.run.smithi107.stdout:/dev/shm 2024-05-17T21:49:52.971 INFO:teuthology.orchestra.run.smithi107.stdout:/dev/pts 2024-05-17T21:49:52.972 INFO:teuthology.orchestra.run.smithi107.stdout:/run 2024-05-17T21:49:52.972 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/fs/cgroup 2024-05-17T21:49:52.972 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/fs/pstore 2024-05-17T21:49:52.972 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/fs/bpf 2024-05-17T21:49:52.972 INFO:teuthology.orchestra.run.smithi107.stdout:/ 2024-05-17T21:49:52.972 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/fs/selinux 2024-05-17T21:49:52.972 INFO:teuthology.orchestra.run.smithi107.stdout:/proc/sys/fs/binfmt_misc 2024-05-17T21:49:52.972 INFO:teuthology.orchestra.run.smithi107.stdout:/dev/hugepages 2024-05-17T21:49:52.972 INFO:teuthology.orchestra.run.smithi107.stdout:/dev/mqueue 2024-05-17T21:49:52.972 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/kernel/debug 2024-05-17T21:49:52.972 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/kernel/tracing 2024-05-17T21:49:52.972 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/fs/fuse/connections 2024-05-17T21:49:52.972 INFO:teuthology.orchestra.run.smithi107.stdout:/run/credentials/systemd-sysctl.service 2024-05-17T21:49:52.973 INFO:teuthology.orchestra.run.smithi107.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-17T21:49:52.973 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/kernel/config 2024-05-17T21:49:52.973 INFO:teuthology.orchestra.run.smithi107.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-17T21:49:52.973 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/nfs/rpc_pipefs 2024-05-17T21:49:52.973 INFO:teuthology.orchestra.run.smithi107.stdout:/run/user/1000 2024-05-17T21:49:52.973 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/ceph 2024-05-17T21:49:52.973 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay 2024-05-17T21:49:52.973 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/610d72d82b79351f7c5fbd3e7779546dd4b8e2f6a01f82cfcd3f5229ceff9617/merged 2024-05-17T21:49:52.973 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/0a0408c6a615242b8d17c4431a780d670b77e129ae89083b2929f6c8ff978855/merged 2024-05-17T21:49:52.973 INFO:teuthology.orchestra.run.smithi107.stdout:/run/user/0 2024-05-17T21:49:52.973 INFO:teuthology.orchestra.run.smithi107.stdout:/proc/sys/fs/binfmt_misc 2024-05-17T21:49:52.973 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/9381a36ade860e6b15f6e6cf506e5aa69858f89b4b1902055f15ba6f73ffad27/merged 2024-05-17T21:49:52.973 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/f646dd11480b7be6e75b143db72b9459df68ba5420f9035b59e8ea50d1746fbb/merged 2024-05-17T21:49:52.974 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/27a5fffcea787df19abe9e6fa8e40ff99bb376c968a41e2c0d1e5943476a302d/merged 2024-05-17T21:49:52.974 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/d3ede44c1b5567f0a6ad92c17f3bff07c5b4d221a56358983f9fc1764d2b6521/merged 2024-05-17T21:49:52.974 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/6f086ae2021a3ad7d4c71822b6459288d191b7e57f06aec3cfd3f842b98b8f18/merged 2024-05-17T21:49:52.974 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/a39a7e9ff95cbb38afa23c13c67947c4ef11001f9e05af58ddaff00ed235f426/merged 2024-05-17T21:49:52.974 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/207dbd72bbb7b6f39c892b0f75e14b80ca9b7b4468c360f18d7646a32eb5e0fd/merged 2024-05-17T21:49:52.974 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/333f76c9031a380ae2d65a6981859b7b23261979818e8c34bdc0da233e65766f/merged 2024-05-17T21:49:52.974 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/396b62cced7aec1fe573bc2b2e1c107802001c84e8ae938f6c7fb0e1e8e09667/merged 2024-05-17T21:49:52.974 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/75b37426d8b94efa3aed5b75e338c5495fa1d8bb5dcc4b79a20e9a3db834179c/merged 2024-05-17T21:49:52.974 INFO:teuthology.orchestra.run.smithi107.stdout:/run/netns 2024-05-17T21:49:52.974 INFO:teuthology.orchestra.run.smithi107.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T21:49:52.974 INFO:teuthology.orchestra.run.smithi107.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T21:49:52.975 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:49:52.975 DEBUG:teuthology.orchestra.run.smithi107:> ls /sys/fs/fuse/connections 2024-05-17T21:49:52.995 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi107.stderr:2024-05-17T21:49:53.002+0000 7fb420eca300 -1 init, newargv = 0x55c2fc08d8a0 newargc=15 2024-05-17T21:49:52.995 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi107.stderr:ceph-fuse[76055]: starting ceph client 2024-05-17T21:49:53.000 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi107.stderr:ceph-fuse[76055]: starting fuse 2024-05-17T21:49:53.011 INFO:teuthology.orchestra.run.smithi107.stdout:119 2024-05-17T21:49:53.011 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [119] 2024-05-17T21:49:53.011 DEBUG:teuthology.orchestra.run.smithi107:> sudo stdin-killer -- python3 -c ' 2024-05-17T21:49:53.011 DEBUG:teuthology.orchestra.run.smithi107:> import glob 2024-05-17T21:49:53.011 DEBUG:teuthology.orchestra.run.smithi107:> import re 2024-05-17T21:49:53.011 DEBUG:teuthology.orchestra.run.smithi107:> import os 2024-05-17T21:49:53.011 DEBUG:teuthology.orchestra.run.smithi107:> import subprocess 2024-05-17T21:49:53.012 DEBUG:teuthology.orchestra.run.smithi107:> 2024-05-17T21:49:53.012 DEBUG:teuthology.orchestra.run.smithi107:> def _find_admin_socket(client_name): 2024-05-17T21:49:53.012 DEBUG:teuthology.orchestra.run.smithi107:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-17T21:49:53.012 DEBUG:teuthology.orchestra.run.smithi107:> files = glob.glob(asok_path) 2024-05-17T21:49:53.012 DEBUG:teuthology.orchestra.run.smithi107:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-17T21:49:53.012 DEBUG:teuthology.orchestra.run.smithi107:> 2024-05-17T21:49:53.012 DEBUG:teuthology.orchestra.run.smithi107:> # Given a non-glob path, it better be there 2024-05-17T21:49:53.012 DEBUG:teuthology.orchestra.run.smithi107:> if "*" not in asok_path: 2024-05-17T21:49:53.012 DEBUG:teuthology.orchestra.run.smithi107:> assert(len(files) == 1) 2024-05-17T21:49:53.012 DEBUG:teuthology.orchestra.run.smithi107:> return files[0] 2024-05-17T21:49:53.012 DEBUG:teuthology.orchestra.run.smithi107:> 2024-05-17T21:49:53.012 DEBUG:teuthology.orchestra.run.smithi107:> for f in files: 2024-05-17T21:49:53.012 DEBUG:teuthology.orchestra.run.smithi107:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-17T21:49:53.012 DEBUG:teuthology.orchestra.run.smithi107:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-17T21:49:53.012 DEBUG:teuthology.orchestra.run.smithi107:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-17T21:49:53.013 DEBUG:teuthology.orchestra.run.smithi107:> contents = proc_f.read() 2024-05-17T21:49:53.013 DEBUG:teuthology.orchestra.run.smithi107:> if mountpoint in contents: 2024-05-17T21:49:53.013 DEBUG:teuthology.orchestra.run.smithi107:> return f 2024-05-17T21:49:53.013 DEBUG:teuthology.orchestra.run.smithi107:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-17T21:49:53.013 DEBUG:teuthology.orchestra.run.smithi107:> 2024-05-17T21:49:53.013 DEBUG:teuthology.orchestra.run.smithi107:> print(_find_admin_socket("client.0")) 2024-05-17T21:49:53.013 DEBUG:teuthology.orchestra.run.smithi107:> ' 2024-05-17T21:49:53.129 INFO:teuthology.orchestra.run.smithi107.stdout:/var/run/ceph/ceph-client.0.76055.asok 2024-05-17T21:49:53.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T21:49:53.140 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.76055.asok 2024-05-17T21:49:53.140 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:49:53.140 DEBUG:teuthology.orchestra.run.smithi107:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.76055.asok status 2024-05-17T21:49:53.270 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-05-17T21:49:53.270 INFO:teuthology.orchestra.run.smithi107.stdout: "metadata": { 2024-05-17T21:49:53.270 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_sha1": "9903f7767386a9d9efadb7c1effdcd3064d71804", 2024-05-17T21:49:53.270 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_version": "ceph version 17.2.7-952-g9903f776 (9903f7767386a9d9efadb7c1effdcd3064d71804) quincy (stable)", 2024-05-17T21:49:53.270 INFO:teuthology.orchestra.run.smithi107.stdout: "entity_id": "0", 2024-05-17T21:49:53.271 INFO:teuthology.orchestra.run.smithi107.stdout: "hostname": "smithi107", 2024-05-17T21:49:53.271 INFO:teuthology.orchestra.run.smithi107.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-17T21:49:53.271 INFO:teuthology.orchestra.run.smithi107.stdout: "pid": "76055", 2024-05-17T21:49:53.271 INFO:teuthology.orchestra.run.smithi107.stdout: "root": "/" 2024-05-17T21:49:53.271 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-05-17T21:49:53.271 INFO:teuthology.orchestra.run.smithi107.stdout: "dentry_count": 0, 2024-05-17T21:49:53.271 INFO:teuthology.orchestra.run.smithi107.stdout: "dentry_pinned_count": 0, 2024-05-17T21:49:53.271 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 14522, 2024-05-17T21:49:53.271 INFO:teuthology.orchestra.run.smithi107.stdout: "inst": { 2024-05-17T21:49:53.271 INFO:teuthology.orchestra.run.smithi107.stdout: "name": { 2024-05-17T21:49:53.271 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "client", 2024-05-17T21:49:53.271 INFO:teuthology.orchestra.run.smithi107.stdout: "num": 14522 2024-05-17T21:49:53.271 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-05-17T21:49:53.272 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": { 2024-05-17T21:49:53.272 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-05-17T21:49:53.272 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "192.168.0.1:0", 2024-05-17T21:49:53.272 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3969818366 2024-05-17T21:49:53.272 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-05-17T21:49:53.272 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-05-17T21:49:53.272 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": { 2024-05-17T21:49:53.272 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-05-17T21:49:53.272 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "192.168.0.1:0", 2024-05-17T21:49:53.272 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3969818366 2024-05-17T21:49:53.272 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-05-17T21:49:53.272 INFO:teuthology.orchestra.run.smithi107.stdout: "inst_str": "client.14522 192.168.0.1:0/3969818366", 2024-05-17T21:49:53.272 INFO:teuthology.orchestra.run.smithi107.stdout: "addr_str": "192.168.0.1:0/3969818366", 2024-05-17T21:49:53.273 INFO:teuthology.orchestra.run.smithi107.stdout: "inode_count": 1, 2024-05-17T21:49:53.273 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_epoch": 15, 2024-05-17T21:49:53.273 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_epoch": 46, 2024-05-17T21:49:53.273 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_epoch_barrier": 0, 2024-05-17T21:49:53.273 INFO:teuthology.orchestra.run.smithi107.stdout: "blocklisted": false, 2024-05-17T21:49:53.273 INFO:teuthology.orchestra.run.smithi107.stdout: "fs_name": "cephfs" 2024-05-17T21:49:53.273 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-05-17T21:49:53.279 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -- ceph fs ls 2024-05-17T21:49:53.536 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/mon.smithi107/config 2024-05-17T21:49:53.664 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:53 smithi107 ceph-mon[28434]: pgmap v194: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 0 B/s wr, 1 op/s 2024-05-17T21:49:53.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:53 smithi138 ceph-mon[35948]: pgmap v194: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 0 B/s wr, 1 op/s 2024-05-17T21:49:55.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:55.233+0000 7f8269284700 1 -- 172.21.15.107:0/388908684 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8264103780 msgr2=0x7f82640fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:49:55.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:55.233+0000 7f8269284700 1 --2- 172.21.15.107:0/388908684 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8264103780 0x7f82640fcdf0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f8258004940 tx=0x7f8258009e30 comp rx=0 tx=0).stop 2024-05-17T21:49:55.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:55.233+0000 7f8269284700 1 -- 172.21.15.107:0/388908684 shutdown_connections 2024-05-17T21:49:55.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:55.233+0000 7f8269284700 1 --2- 172.21.15.107:0/388908684 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8264103780 0x7f82640fcdf0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:49:55.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:55.233+0000 7f8269284700 1 --2- 172.21.15.107:0/388908684 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8264102de0 0x7f82641031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:49:55.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:55.233+0000 7f8269284700 1 -- 172.21.15.107:0/388908684 >> 172.21.15.107:0/388908684 conn(0x7f82640f8990 msgr2=0x7f82640fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:49:55.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:55.233+0000 7f8269284700 1 -- 172.21.15.107:0/388908684 shutdown_connections 2024-05-17T21:49:55.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:55.233+0000 7f8269284700 1 -- 172.21.15.107:0/388908684 wait complete. 2024-05-17T21:49:55.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:55.233+0000 7f8269284700 1 Processor -- start 2024-05-17T21:49:55.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:55.233+0000 7f8269284700 1 -- start start 2024-05-17T21:49:55.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:55.234+0000 7f8269284700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8264102de0 0x7f82641138e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:49:55.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:55.234+0000 7f8269284700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8264103780 0x7f8264113e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:49:55.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:55.234+0000 7f8269284700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8264114470 con 0x7f8264103780 2024-05-17T21:49:55.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:55.234+0000 7f8269284700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f826410da70 con 0x7f8264102de0 2024-05-17T21:49:55.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:55.234+0000 7f82627fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8264103780 0x7f8264113e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:49:55.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:55.234+0000 7f82627fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8264103780 0x7f8264113e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59220/0 (socket says 172.21.15.107:59220) 2024-05-17T21:49:55.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:55.234+0000 7f82627fc700 1 -- 172.21.15.107:0/263359310 learned_addr learned my addr 172.21.15.107:0/263359310 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:49:55.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:55.234+0000 7f8262ffd700 1 --2- 172.21.15.107:0/263359310 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8264102de0 0x7f82641138e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:49:55.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:55.234+0000 7f82627fc700 1 -- 172.21.15.107:0/263359310 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8264102de0 msgr2=0x7f82641138e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:49:55.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:55.234+0000 7f82627fc700 1 --2- 172.21.15.107:0/263359310 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8264102de0 0x7f82641138e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:49:55.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:55.234+0000 7f82627fc700 1 -- 172.21.15.107:0/263359310 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8258005040 con 0x7f8264103780 2024-05-17T21:49:55.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:55.234+0000 7f82627fc700 1 --2- 172.21.15.107:0/263359310 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8264103780 0x7f8264113e20 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f8258000c00 tx=0x7f8258009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:49:55.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:55.234+0000 7f824b7fe700 1 -- 172.21.15.107:0/263359310 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8258021070 con 0x7f8264103780 2024-05-17T21:49:55.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:55.234+0000 7f824b7fe700 1 -- 172.21.15.107:0/263359310 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8258013cd0 con 0x7f8264103780 2024-05-17T21:49:55.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:55.234+0000 7f8269284700 1 -- 172.21.15.107:0/263359310 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f826410dcf0 con 0x7f8264103780 2024-05-17T21:49:55.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:55.234+0000 7f824b7fe700 1 -- 172.21.15.107:0/263359310 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f825800e070 con 0x7f8264103780 2024-05-17T21:49:55.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:55.234+0000 7f8269284700 1 -- 172.21.15.107:0/263359310 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f826410e1d0 con 0x7f8264103780 2024-05-17T21:49:55.230 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:55.235+0000 7f8269284700 1 -- 172.21.15.107:0/263359310 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82640fe530 con 0x7f8264103780 2024-05-17T21:49:55.230 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:55.236+0000 7f824b7fe700 1 -- 172.21.15.107:0/263359310 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f825801caf0 con 0x7f8264103780 2024-05-17T21:49:55.230 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:55.236+0000 7f824b7fe700 1 --2- 172.21.15.107:0/263359310 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f824c0692a0 0x7f824c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:49:55.230 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:55.236+0000 7f824b7fe700 1 -- 172.21.15.107:0/263359310 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f8258088710 con 0x7f8264103780 2024-05-17T21:49:55.230 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:55.236+0000 7f8262ffd700 1 --2- 172.21.15.107:0/263359310 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f824c0692a0 0x7f824c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:49:55.230 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:55.237+0000 7f8262ffd700 1 --2- 172.21.15.107:0/263359310 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f824c0692a0 0x7f824c06b750 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8254004600 tx=0x7f825400b040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:49:55.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:55.241+0000 7f824b7fe700 1 -- 172.21.15.107:0/263359310 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f8258058460 con 0x7f8264103780 2024-05-17T21:49:55.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:55.469+0000 7f8269284700 1 -- 172.21.15.107:0/263359310 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f8264109ec0 con 0x7f8264103780 2024-05-17T21:49:55.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:55.469+0000 7f824b7fe700 1 -- 172.21.15.107:0/263359310 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v15) v1 ==== 53+0+83 (secure 0 0 0) 0x7f8258057ff0 con 0x7f8264103780 2024-05-17T21:49:55.463 INFO:teuthology.orchestra.run.smithi107.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-05-17T21:49:55.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:55.472+0000 7f8269284700 1 -- 172.21.15.107:0/263359310 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f824c0692a0 msgr2=0x7f824c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:49:55.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:55.472+0000 7f8269284700 1 --2- 172.21.15.107:0/263359310 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f824c0692a0 0x7f824c06b750 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8254004600 tx=0x7f825400b040 comp rx=0 tx=0).stop 2024-05-17T21:49:55.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:55.472+0000 7f8269284700 1 -- 172.21.15.107:0/263359310 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8264103780 msgr2=0x7f8264113e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:49:55.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:55.472+0000 7f8269284700 1 --2- 172.21.15.107:0/263359310 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8264103780 0x7f8264113e20 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f8258000c00 tx=0x7f8258009e30 comp rx=0 tx=0).stop 2024-05-17T21:49:55.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:55.472+0000 7f8269284700 1 -- 172.21.15.107:0/263359310 shutdown_connections 2024-05-17T21:49:55.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:55.472+0000 7f8269284700 1 --2- 172.21.15.107:0/263359310 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f824c0692a0 0x7f824c06b750 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:49:55.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:55.472+0000 7f8269284700 1 --2- 172.21.15.107:0/263359310 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8264102de0 0x7f82641138e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:49:55.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:55.472+0000 7f8269284700 1 --2- 172.21.15.107:0/263359310 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8264103780 0x7f8264113e20 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:49:55.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:55.472+0000 7f8269284700 1 -- 172.21.15.107:0/263359310 >> 172.21.15.107:0/263359310 conn(0x7f82640f8990 msgr2=0x7f82640f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:49:55.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:55.472+0000 7f8269284700 1 -- 172.21.15.107:0/263359310 shutdown_connections 2024-05-17T21:49:55.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:49:55.472+0000 7f8269284700 1 -- 172.21.15.107:0/263359310 wait complete. 2024-05-17T21:49:55.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:55 smithi138 ceph-mon[35948]: pgmap v195: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-05-17T21:49:55.828 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:55 smithi107 ceph-mon[28434]: pgmap v195: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-05-17T21:49:56.138 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-17T21:49:56.138 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-17T21:49:56.138 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi138.front.sepia.ceph.com 2024-05-17T21:49:56.138 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-05-17T21:49:56.138 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-05-17T21:49:56.138 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-17T21:49:56.138 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-17T21:49:56.138 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-17T21:49:56.139 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-05-17T21:49:56.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:49:56.139 DEBUG:teuthology.orchestra.run.smithi138:> ip addr 2024-05-17T21:49:56.158 INFO:teuthology.orchestra.run.smithi138.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-17T21:49:56.158 INFO:teuthology.orchestra.run.smithi138.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-17T21:49:56.158 INFO:teuthology.orchestra.run.smithi138.stdout: inet 127.0.0.1/8 scope host lo 2024-05-17T21:49:56.159 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-05-17T21:49:56.159 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 ::1/128 scope host 2024-05-17T21:49:56.159 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-05-17T21:49:56.159 INFO:teuthology.orchestra.run.smithi138.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-17T21:49:56.159 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:d9:36:7a brd ff:ff:ff:ff:ff:ff 2024-05-17T21:49:56.159 INFO:teuthology.orchestra.run.smithi138.stdout: altname enp7s0f0 2024-05-17T21:49:56.159 INFO:teuthology.orchestra.run.smithi138.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-17T21:49:56.159 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:d9:36:7b brd ff:ff:ff:ff:ff:ff 2024-05-17T21:49:56.159 INFO:teuthology.orchestra.run.smithi138.stdout: altname enp7s0f1 2024-05-17T21:49:56.159 INFO:teuthology.orchestra.run.smithi138.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-17T21:49:56.159 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:88:6f:70 brd ff:ff:ff:ff:ff:ff 2024-05-17T21:49:56.159 INFO:teuthology.orchestra.run.smithi138.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-17T21:49:56.159 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:88:6f:71 brd ff:ff:ff:ff:ff:ff 2024-05-17T21:49:56.159 INFO:teuthology.orchestra.run.smithi138.stdout: inet 172.21.15.138/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-05-17T21:49:56.159 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft 42297sec preferred_lft 42297sec 2024-05-17T21:49:56.160 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::ec4:7aff:fe88:6f71/64 scope link 2024-05-17T21:49:56.160 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-05-17T21:49:56.160 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-17T21:49:56.160 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-17T21:49:56.160 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2024-05-17T21:49:56.160 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link add name ceph-brx type bridge 2024-05-17T21:49:56.160 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip addr flush dev ceph-brx 2024-05-17T21:49:56.160 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link set ceph-brx up 2024-05-17T21:49:56.160 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-17T21:49:56.160 DEBUG:teuthology.orchestra.run.smithi138:> ') 2024-05-17T21:49:56.253 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:49:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-17T21:49:56.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:49:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T21:49:56.382 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:49:56.382 DEBUG:teuthology.orchestra.run.smithi138:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-17T21:49:56.407 INFO:teuthology.orchestra.run.smithi138.stdout:1 2024-05-17T21:49:56.408 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:49:56.408 DEBUG:teuthology.orchestra.run.smithi138:> route 2024-05-17T21:49:56.472 INFO:teuthology.orchestra.run.smithi138.stdout:Kernel IP routing table 2024-05-17T21:49:56.472 INFO:teuthology.orchestra.run.smithi138.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-17T21:49:56.472 INFO:teuthology.orchestra.run.smithi138.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-17T21:49:56.472 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-17T21:49:56.472 INFO:teuthology.orchestra.run.smithi138.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-17T21:49:56.473 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-17T21:49:56.473 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2024-05-17T21:49:56.473 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-17T21:49:56.473 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-17T21:49:56.473 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-17T21:49:56.473 DEBUG:teuthology.orchestra.run.smithi138:> ') 2024-05-17T21:49:56.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:49:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-17T21:49:56.569 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:56 smithi138 ceph-mon[35948]: from='client.? 172.21.15.107:0/263359310' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-17T21:49:56.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:49:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T21:49:56.773 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:49:56.773 DEBUG:teuthology.orchestra.run.smithi138:> ip netns list 2024-05-17T21:49:56.789 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:49:56.789 DEBUG:teuthology.orchestra.run.smithi138:> ip netns list-id 2024-05-17T21:49:56.846 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-17T21:49:56.846 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2024-05-17T21:49:56.847 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-17T21:49:56.847 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-05-17T21:49:56.847 DEBUG:teuthology.orchestra.run.smithi138:> ') 2024-05-17T21:49:56.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:56 smithi107 ceph-mon[28434]: from='client.? 172.21.15.107:0/263359310' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-17T21:49:56.936 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:49:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-17T21:49:56.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:49:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T21:49:56.968 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-05-17T21:49:56.968 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-17T21:49:56.969 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2024-05-17T21:49:56.969 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-05-17T21:49:56.969 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-17T21:49:56.969 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-05-17T21:49:56.969 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-05-17T21:49:56.970 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-05-17T21:49:56.970 DEBUG:teuthology.orchestra.run.smithi138:> ') 2024-05-17T21:49:57.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:49:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-17T21:49:57.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:49:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T21:49:57.100 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-17T21:49:57.100 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2024-05-17T21:49:57.100 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link set brx.0 up 2024-05-17T21:49:57.100 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link set dev brx.0 master ceph-brx 2024-05-17T21:49:57.100 DEBUG:teuthology.orchestra.run.smithi138:> ') 2024-05-17T21:49:57.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:49:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-17T21:49:57.171 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:49:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T21:49:57.177 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-05-17T21:49:57.177 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-17T21:49:57.177 DEBUG:teuthology.orchestra.run.smithi138:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-05-17T21:49:57.194 INFO:teuthology.orchestra.run.smithi138.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-05-17T21:49:57.194 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-17T21:49:57.194 DEBUG:teuthology.orchestra.run.smithi138:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-05-17T21:49:57.251 DEBUG:teuthology.orchestra.run.smithi138:> sudo modprobe fuse 2024-05-17T21:49:57.319 DEBUG:teuthology.orchestra.run.smithi138:> cat /proc/self/mounts | awk '{print $2}' 2024-05-17T21:49:57.375 INFO:teuthology.orchestra.run.smithi138.stdout:/proc 2024-05-17T21:49:57.375 INFO:teuthology.orchestra.run.smithi138.stdout:/sys 2024-05-17T21:49:57.375 INFO:teuthology.orchestra.run.smithi138.stdout:/dev 2024-05-17T21:49:57.376 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/security 2024-05-17T21:49:57.376 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/shm 2024-05-17T21:49:57.376 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/pts 2024-05-17T21:49:57.376 INFO:teuthology.orchestra.run.smithi138.stdout:/run 2024-05-17T21:49:57.376 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/cgroup 2024-05-17T21:49:57.376 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/pstore 2024-05-17T21:49:57.376 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/bpf 2024-05-17T21:49:57.376 INFO:teuthology.orchestra.run.smithi138.stdout:/ 2024-05-17T21:49:57.376 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/selinux 2024-05-17T21:49:57.376 INFO:teuthology.orchestra.run.smithi138.stdout:/proc/sys/fs/binfmt_misc 2024-05-17T21:49:57.376 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/hugepages 2024-05-17T21:49:57.376 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/mqueue 2024-05-17T21:49:57.377 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/debug 2024-05-17T21:49:57.377 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/tracing 2024-05-17T21:49:57.377 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/fuse/connections 2024-05-17T21:49:57.377 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/config 2024-05-17T21:49:57.377 INFO:teuthology.orchestra.run.smithi138.stdout:/run/credentials/systemd-sysctl.service 2024-05-17T21:49:57.377 INFO:teuthology.orchestra.run.smithi138.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-17T21:49:57.377 INFO:teuthology.orchestra.run.smithi138.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-17T21:49:57.377 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/nfs/rpc_pipefs 2024-05-17T21:49:57.377 INFO:teuthology.orchestra.run.smithi138.stdout:/run/user/1000 2024-05-17T21:49:57.377 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph 2024-05-17T21:49:57.377 INFO:teuthology.orchestra.run.smithi138.stdout:/run/user/0 2024-05-17T21:49:57.377 INFO:teuthology.orchestra.run.smithi138.stdout:/proc/sys/fs/binfmt_misc 2024-05-17T21:49:57.377 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/containers/storage/overlay 2024-05-17T21:49:57.378 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/containers/storage/overlay/d31444b4766199614e81b4808646e2a6b9b0bcecb2baac4e075e6fb05de6f8a4/merged 2024-05-17T21:49:57.378 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/containers/storage/overlay/e7f1c18918232206279571c3f9abb1c3d06634f680ce5b2ab2ff0474d94a83d7/merged 2024-05-17T21:49:57.378 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/containers/storage/overlay/d06121b5c4d49574aad5b599011eaf8cd9a12f75779f30a2ef1fe5862dc880fe/merged 2024-05-17T21:49:57.378 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/containers/storage/overlay/03241031caa59cf6c5ea30f3e8da7905bb16c83e49ce314977e4b8ed756880db/merged 2024-05-17T21:49:57.378 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/containers/storage/overlay/071b0db963d2a7835ca58394edb2b5fad9beff40d118e686e5d1ba04ee6e94af/merged 2024-05-17T21:49:57.378 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/containers/storage/overlay/7c2fb4b2fc18630eb8b406391f8f1f47daefb478b6598f6e1c18f5e60610abce/merged 2024-05-17T21:49:57.378 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/containers/storage/overlay/e235935c510922c491132467909a2a3185f9bab822cf6bcbff79e5e585676d47/merged 2024-05-17T21:49:57.378 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/containers/storage/overlay/c6dea341f3770ca369a8b4e59330202b6a90ac7e3efe9b9932dd2038f3f9490b/merged 2024-05-17T21:49:57.378 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/containers/storage/overlay/4326b0f52c36178a201357dee4c89de1e0a7a6e7c47cb64b3850e9423947ae49/merged 2024-05-17T21:49:57.378 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns 2024-05-17T21:49:57.378 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-17T21:49:57.378 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-17T21:49:57.379 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:49:57.379 DEBUG:teuthology.orchestra.run.smithi138:> ls /sys/fs/fuse/connections 2024-05-17T21:49:57.430 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-17T21:49:57.431 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-05-17T21:49:57.473 DEBUG:teuthology.orchestra.run.smithi138:> sudo modprobe fuse 2024-05-17T21:49:57.499 DEBUG:teuthology.orchestra.run.smithi138:> cat /proc/self/mounts | awk '{print $2}' 2024-05-17T21:49:57.556 INFO:teuthology.orchestra.run.smithi138.stdout:/proc 2024-05-17T21:49:57.556 INFO:teuthology.orchestra.run.smithi138.stdout:/sys 2024-05-17T21:49:57.556 INFO:teuthology.orchestra.run.smithi138.stdout:/dev 2024-05-17T21:49:57.557 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/security 2024-05-17T21:49:57.557 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/shm 2024-05-17T21:49:57.557 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/pts 2024-05-17T21:49:57.557 INFO:teuthology.orchestra.run.smithi138.stdout:/run 2024-05-17T21:49:57.557 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/cgroup 2024-05-17T21:49:57.557 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/pstore 2024-05-17T21:49:57.557 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/bpf 2024-05-17T21:49:57.557 INFO:teuthology.orchestra.run.smithi138.stdout:/ 2024-05-17T21:49:57.557 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/selinux 2024-05-17T21:49:57.557 INFO:teuthology.orchestra.run.smithi138.stdout:/proc/sys/fs/binfmt_misc 2024-05-17T21:49:57.557 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/hugepages 2024-05-17T21:49:57.557 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/mqueue 2024-05-17T21:49:57.557 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/debug 2024-05-17T21:49:57.557 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/tracing 2024-05-17T21:49:57.557 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/fuse/connections 2024-05-17T21:49:57.558 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/config 2024-05-17T21:49:57.558 INFO:teuthology.orchestra.run.smithi138.stdout:/run/credentials/systemd-sysctl.service 2024-05-17T21:49:57.558 INFO:teuthology.orchestra.run.smithi138.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-17T21:49:57.558 INFO:teuthology.orchestra.run.smithi138.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-17T21:49:57.558 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/nfs/rpc_pipefs 2024-05-17T21:49:57.558 INFO:teuthology.orchestra.run.smithi138.stdout:/run/user/1000 2024-05-17T21:49:57.558 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph 2024-05-17T21:49:57.558 INFO:teuthology.orchestra.run.smithi138.stdout:/run/user/0 2024-05-17T21:49:57.558 INFO:teuthology.orchestra.run.smithi138.stdout:/proc/sys/fs/binfmt_misc 2024-05-17T21:49:57.558 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/containers/storage/overlay 2024-05-17T21:49:57.558 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/containers/storage/overlay/d31444b4766199614e81b4808646e2a6b9b0bcecb2baac4e075e6fb05de6f8a4/merged 2024-05-17T21:49:57.558 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/containers/storage/overlay/e7f1c18918232206279571c3f9abb1c3d06634f680ce5b2ab2ff0474d94a83d7/merged 2024-05-17T21:49:57.558 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/containers/storage/overlay/d06121b5c4d49574aad5b599011eaf8cd9a12f75779f30a2ef1fe5862dc880fe/merged 2024-05-17T21:49:57.559 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/containers/storage/overlay/03241031caa59cf6c5ea30f3e8da7905bb16c83e49ce314977e4b8ed756880db/merged 2024-05-17T21:49:57.559 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/containers/storage/overlay/071b0db963d2a7835ca58394edb2b5fad9beff40d118e686e5d1ba04ee6e94af/merged 2024-05-17T21:49:57.559 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/containers/storage/overlay/7c2fb4b2fc18630eb8b406391f8f1f47daefb478b6598f6e1c18f5e60610abce/merged 2024-05-17T21:49:57.559 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/containers/storage/overlay/e235935c510922c491132467909a2a3185f9bab822cf6bcbff79e5e585676d47/merged 2024-05-17T21:49:57.559 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/containers/storage/overlay/c6dea341f3770ca369a8b4e59330202b6a90ac7e3efe9b9932dd2038f3f9490b/merged 2024-05-17T21:49:57.559 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/containers/storage/overlay/4326b0f52c36178a201357dee4c89de1e0a7a6e7c47cb64b3850e9423947ae49/merged 2024-05-17T21:49:57.559 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns 2024-05-17T21:49:57.559 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-17T21:49:57.559 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-17T21:49:57.559 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:49:57.559 DEBUG:teuthology.orchestra.run.smithi138:> ls /sys/fs/fuse/connections 2024-05-17T21:49:57.588 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi138.stderr:2024-05-17T21:49:57.595+0000 7f3eb3ec1300 -1 init, newargv = 0x5603b3dac9b0 newargc=15 2024-05-17T21:49:57.588 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi138.stderr:ceph-fuse[61222]: starting ceph client 2024-05-17T21:49:57.592 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi138.stderr:ceph-fuse[61222]: starting fuse 2024-05-17T21:49:57.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:57 smithi107 ceph-mon[28434]: pgmap v196: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T21:49:57.604 INFO:teuthology.orchestra.run.smithi138.stdout:116 2024-05-17T21:49:57.604 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [116] 2024-05-17T21:49:57.604 DEBUG:teuthology.orchestra.run.smithi138:> sudo stdin-killer -- python3 -c ' 2024-05-17T21:49:57.604 DEBUG:teuthology.orchestra.run.smithi138:> import glob 2024-05-17T21:49:57.604 DEBUG:teuthology.orchestra.run.smithi138:> import re 2024-05-17T21:49:57.604 DEBUG:teuthology.orchestra.run.smithi138:> import os 2024-05-17T21:49:57.604 DEBUG:teuthology.orchestra.run.smithi138:> import subprocess 2024-05-17T21:49:57.604 DEBUG:teuthology.orchestra.run.smithi138:> 2024-05-17T21:49:57.605 DEBUG:teuthology.orchestra.run.smithi138:> def _find_admin_socket(client_name): 2024-05-17T21:49:57.605 DEBUG:teuthology.orchestra.run.smithi138:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-05-17T21:49:57.605 DEBUG:teuthology.orchestra.run.smithi138:> files = glob.glob(asok_path) 2024-05-17T21:49:57.605 DEBUG:teuthology.orchestra.run.smithi138:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-05-17T21:49:57.605 DEBUG:teuthology.orchestra.run.smithi138:> 2024-05-17T21:49:57.605 DEBUG:teuthology.orchestra.run.smithi138:> # Given a non-glob path, it better be there 2024-05-17T21:49:57.605 DEBUG:teuthology.orchestra.run.smithi138:> if "*" not in asok_path: 2024-05-17T21:49:57.605 DEBUG:teuthology.orchestra.run.smithi138:> assert(len(files) == 1) 2024-05-17T21:49:57.605 DEBUG:teuthology.orchestra.run.smithi138:> return files[0] 2024-05-17T21:49:57.605 DEBUG:teuthology.orchestra.run.smithi138:> 2024-05-17T21:49:57.605 DEBUG:teuthology.orchestra.run.smithi138:> for f in files: 2024-05-17T21:49:57.605 DEBUG:teuthology.orchestra.run.smithi138:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-17T21:49:57.605 DEBUG:teuthology.orchestra.run.smithi138:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-17T21:49:57.605 DEBUG:teuthology.orchestra.run.smithi138:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-17T21:49:57.606 DEBUG:teuthology.orchestra.run.smithi138:> contents = proc_f.read() 2024-05-17T21:49:57.606 DEBUG:teuthology.orchestra.run.smithi138:> if mountpoint in contents: 2024-05-17T21:49:57.606 DEBUG:teuthology.orchestra.run.smithi138:> return f 2024-05-17T21:49:57.606 DEBUG:teuthology.orchestra.run.smithi138:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-17T21:49:57.606 DEBUG:teuthology.orchestra.run.smithi138:> 2024-05-17T21:49:57.606 DEBUG:teuthology.orchestra.run.smithi138:> print(_find_admin_socket("client.1")) 2024-05-17T21:49:57.606 DEBUG:teuthology.orchestra.run.smithi138:> ' 2024-05-17T21:49:57.722 INFO:teuthology.orchestra.run.smithi138.stdout:/var/run/ceph/ceph-client.1.61222.asok 2024-05-17T21:49:57.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-17T21:49:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T21:49:57.731 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:57 smithi138 ceph-mon[35948]: pgmap v196: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T21:49:57.732 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.61222.asok 2024-05-17T21:49:57.733 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:49:57.733 DEBUG:teuthology.orchestra.run.smithi138:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.61222.asok status 2024-05-17T21:49:57.865 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-17T21:49:57.866 INFO:teuthology.orchestra.run.smithi138.stdout: "metadata": { 2024-05-17T21:49:57.866 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_sha1": "9903f7767386a9d9efadb7c1effdcd3064d71804", 2024-05-17T21:49:57.866 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_version": "ceph version 17.2.7-952-g9903f776 (9903f7767386a9d9efadb7c1effdcd3064d71804) quincy (stable)", 2024-05-17T21:49:57.866 INFO:teuthology.orchestra.run.smithi138.stdout: "entity_id": "1", 2024-05-17T21:49:57.867 INFO:teuthology.orchestra.run.smithi138.stdout: "hostname": "smithi138", 2024-05-17T21:49:57.867 INFO:teuthology.orchestra.run.smithi138.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-05-17T21:49:57.867 INFO:teuthology.orchestra.run.smithi138.stdout: "pid": "61222", 2024-05-17T21:49:57.867 INFO:teuthology.orchestra.run.smithi138.stdout: "root": "/" 2024-05-17T21:49:57.867 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-17T21:49:57.867 INFO:teuthology.orchestra.run.smithi138.stdout: "dentry_count": 0, 2024-05-17T21:49:57.868 INFO:teuthology.orchestra.run.smithi138.stdout: "dentry_pinned_count": 0, 2024-05-17T21:49:57.868 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 24295, 2024-05-17T21:49:57.868 INFO:teuthology.orchestra.run.smithi138.stdout: "inst": { 2024-05-17T21:49:57.868 INFO:teuthology.orchestra.run.smithi138.stdout: "name": { 2024-05-17T21:49:57.868 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "client", 2024-05-17T21:49:57.868 INFO:teuthology.orchestra.run.smithi138.stdout: "num": 24295 2024-05-17T21:49:57.869 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-17T21:49:57.869 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": { 2024-05-17T21:49:57.869 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2024-05-17T21:49:57.869 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "192.168.0.1:0", 2024-05-17T21:49:57.869 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 262157930 2024-05-17T21:49:57.870 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-17T21:49:57.870 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-17T21:49:57.870 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": { 2024-05-17T21:49:57.870 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2024-05-17T21:49:57.870 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "192.168.0.1:0", 2024-05-17T21:49:57.870 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 262157930 2024-05-17T21:49:57.870 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-17T21:49:57.870 INFO:teuthology.orchestra.run.smithi138.stdout: "inst_str": "client.24295 192.168.0.1:0/262157930", 2024-05-17T21:49:57.870 INFO:teuthology.orchestra.run.smithi138.stdout: "addr_str": "192.168.0.1:0/262157930", 2024-05-17T21:49:57.870 INFO:teuthology.orchestra.run.smithi138.stdout: "inode_count": 1, 2024-05-17T21:49:57.870 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_epoch": 15, 2024-05-17T21:49:57.870 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_epoch": 46, 2024-05-17T21:49:57.871 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_epoch_barrier": 0, 2024-05-17T21:49:57.871 INFO:teuthology.orchestra.run.smithi138.stdout: "blocklisted": false, 2024-05-17T21:49:57.871 INFO:teuthology.orchestra.run.smithi138.stdout: "fs_name": "cephfs" 2024-05-17T21:49:57.871 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-17T21:49:57.874 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:49:57.874 DEBUG:teuthology.orchestra.run.smithi107:> stat --file-system '--printf=%T 2024-05-17T21:49:57.874 DEBUG:teuthology.orchestra.run.smithi107:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-17T21:49:57.892 INFO:teuthology.orchestra.run.smithi107.stdout:fuseblk 2024-05-17T21:49:57.892 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-17T21:49:57.892 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:49:57.893 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-17T21:49:57.962 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:49:57.962 DEBUG:teuthology.orchestra.run.smithi138:> stat --file-system '--printf=%T 2024-05-17T21:49:57.962 DEBUG:teuthology.orchestra.run.smithi138:> ' -- /home/ubuntu/cephtest/mnt.1 2024-05-17T21:49:57.978 INFO:teuthology.orchestra.run.smithi138.stdout:fuseblk 2024-05-17T21:49:57.978 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-05-17T21:49:57.979 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:49:57.979 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-05-17T21:49:58.046 INFO:teuthology.run_tasks:Running task print... 2024-05-17T21:49:58.057 INFO:teuthology.task.print:**** done client 2024-05-17T21:49:58.057 INFO:teuthology.run_tasks:Running task parallel... 2024-05-17T21:49:58.140 INFO:teuthology.task.parallel:starting parallel... 2024-05-17T21:49:58.141 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-05-17T21:49:58.141 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-05-17T21:49:58.142 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi107.front.sepia.ceph.com 2024-05-17T21:49:58.142 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -e sha1=931809a9e5d295bd61984693cac44fe131ded4bc -- bash -c 'ceph config set mgr mgr/orchestrator/fail_fs false || true' 2024-05-17T21:49:58.143 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-05-17T21:49:58.143 INFO:teuthology.task.sequential:In sequential, running task workunit... 2024-05-17T21:49:58.149 INFO:tasks.workunit:Pulling workunits from ref 931809a9e5d295bd61984693cac44fe131ded4bc 2024-05-17T21:49:58.150 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-05-17T21:49:58.150 INFO:tasks.workunit:timeout=3h 2024-05-17T21:49:58.150 INFO:tasks.workunit:cleanup=True 2024-05-17T21:49:58.151 DEBUG:teuthology.orchestra.run.smithi107:> stat -- /home/ubuntu/cephtest/mnt.0 2024-05-17T21:49:58.168 INFO:teuthology.orchestra.run.smithi107.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-05-17T21:49:58.168 INFO:teuthology.orchestra.run.smithi107.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-05-17T21:49:58.169 INFO:teuthology.orchestra.run.smithi107.stdout:Device: 77h/119d Inode: 1 Links: 2 2024-05-17T21:49:58.169 INFO:teuthology.orchestra.run.smithi107.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-17T21:49:58.169 INFO:teuthology.orchestra.run.smithi107.stdout:Context: system_u:object_r:fusefs_t:s0 2024-05-17T21:49:58.169 INFO:teuthology.orchestra.run.smithi107.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-05-17T21:49:58.169 INFO:teuthology.orchestra.run.smithi107.stdout:Modify: 2024-05-17 21:49:21.858043569 +0000 2024-05-17T21:49:58.169 INFO:teuthology.orchestra.run.smithi107.stdout:Change: 2024-05-17 21:49:57.966915089 +0000 2024-05-17T21:49:58.169 INFO:teuthology.orchestra.run.smithi107.stdout: Birth: - 2024-05-17T21:49:58.169 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-05-17T21:49:58.169 DEBUG:teuthology.orchestra.run.smithi107:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-05-17T21:49:58.239 DEBUG:teuthology.orchestra.run.smithi138:> stat -- /home/ubuntu/cephtest/mnt.1 2024-05-17T21:49:58.327 INFO:teuthology.orchestra.run.smithi138.stdout: File: /home/ubuntu/cephtest/mnt.1 2024-05-17T21:49:58.327 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-05-17T21:49:58.327 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 74h/116d Inode: 1 Links: 3 2024-05-17T21:49:58.327 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-17T21:49:58.327 INFO:teuthology.orchestra.run.smithi138.stdout:Context: system_u:object_r:fusefs_t:s0 2024-05-17T21:49:58.327 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-05-17T21:49:58.327 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2024-05-17 21:49:58.242161834 +0000 2024-05-17T21:49:58.327 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2024-05-17 21:49:58.242161834 +0000 2024-05-17T21:49:58.327 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: - 2024-05-17T21:49:58.328 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.1 2024-05-17T21:49:58.328 DEBUG:teuthology.orchestra.run.smithi138:> cd -- /home/ubuntu/cephtest/mnt.1 && sudo install -d -m 0755 --owner=ubuntu -- client.1 2024-05-17T21:49:58.397 DEBUG:teuthology.orchestra.run.smithi107:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 931809a9e5d295bd61984693cac44fe131ded4bc 2024-05-17T21:49:58.398 DEBUG:teuthology.orchestra.run.smithi138:> rm -rf /home/ubuntu/cephtest/clone.client.1 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.1 && cd /home/ubuntu/cephtest/clone.client.1 && git checkout 931809a9e5d295bd61984693cac44fe131ded4bc 2024-05-17T21:49:58.416 INFO:tasks.workunit.client.0.smithi107.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-05-17T21:49:58.426 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/mon.smithi107/config 2024-05-17T21:49:58.454 INFO:tasks.workunit.client.1.smithi138.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.1'... 2024-05-17T21:49:59.696 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:49:59 smithi107 ceph-mon[28434]: pgmap v197: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-05-17T21:49:59.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:49:59 smithi138 ceph-mon[35948]: pgmap v197: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-05-17T21:50:00.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:00.046+0000 7f8c5daa3700 1 -- 172.21.15.107:0/2010752366 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8c580fdfd0 msgr2=0x7f8c580fe440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:50:00.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:00.046+0000 7f8c5daa3700 1 --2- 172.21.15.107:0/2010752366 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8c580fdfd0 0x7f8c580fe440 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f8c40004940 tx=0x7f8c40009e30 comp rx=0 tx=0).stop 2024-05-17T21:50:00.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:00.046+0000 7f8c5daa3700 1 -- 172.21.15.107:0/2010752366 shutdown_connections 2024-05-17T21:50:00.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:00.046+0000 7f8c5daa3700 1 --2- 172.21.15.107:0/2010752366 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8c580fdfd0 0x7f8c580fe440 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:00.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:00.046+0000 7f8c5daa3700 1 --2- 172.21.15.107:0/2010752366 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c580fce60 0x7f8c580fd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:00.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:00.046+0000 7f8c5daa3700 1 -- 172.21.15.107:0/2010752366 >> 172.21.15.107:0/2010752366 conn(0x7f8c580f8540 msgr2=0x7f8c580fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:50:00.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:00.046+0000 7f8c5daa3700 1 -- 172.21.15.107:0/2010752366 shutdown_connections 2024-05-17T21:50:00.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:00.046+0000 7f8c5daa3700 1 -- 172.21.15.107:0/2010752366 wait complete. 2024-05-17T21:50:00.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:00.046+0000 7f8c5daa3700 1 Processor -- start 2024-05-17T21:50:00.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:00.047+0000 7f8c5daa3700 1 -- start start 2024-05-17T21:50:00.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:00.047+0000 7f8c5daa3700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8c580fce60 0x7f8c58071c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:50:00.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:00.047+0000 7f8c5daa3700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c580fdfd0 0x7f8c580702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:50:00.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:00.047+0000 7f8c5daa3700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c580707f0 con 0x7f8c580fce60 2024-05-17T21:50:00.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:00.047+0000 7f8c5daa3700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c58070960 con 0x7f8c580fdfd0 2024-05-17T21:50:00.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:00.047+0000 7f8c577fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8c580fce60 0x7f8c58071c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:50:00.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:00.047+0000 7f8c577fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8c580fce60 0x7f8c58071c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46676/0 (socket says 172.21.15.107:46676) 2024-05-17T21:50:00.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:00.047+0000 7f8c577fe700 1 -- 172.21.15.107:0/116386817 learned_addr learned my addr 172.21.15.107:0/116386817 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:50:00.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:00.047+0000 7f8c4effd700 1 --2- 172.21.15.107:0/116386817 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c580fdfd0 0x7f8c580702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:50:00.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:00.047+0000 7f8c577fe700 1 -- 172.21.15.107:0/116386817 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c580fdfd0 msgr2=0x7f8c580702b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:50:00.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:00.047+0000 7f8c577fe700 1 --2- 172.21.15.107:0/116386817 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c580fdfd0 0x7f8c580702b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:00.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:00.047+0000 7f8c577fe700 1 -- 172.21.15.107:0/116386817 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c40005040 con 0x7f8c580fce60 2024-05-17T21:50:00.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:00.047+0000 7f8c4effd700 1 --2- 172.21.15.107:0/116386817 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c580fdfd0 0x7f8c580702b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T21:50:00.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:00.048+0000 7f8c577fe700 1 --2- 172.21.15.107:0/116386817 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8c580fce60 0x7f8c58071c00 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f8c4800b790 tx=0x7f8c48009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:50:00.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:00.048+0000 7f8c54ff9700 1 -- 172.21.15.107:0/116386817 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8c4800f770 con 0x7f8c580fce60 2024-05-17T21:50:00.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:00.048+0000 7f8c54ff9700 1 -- 172.21.15.107:0/116386817 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8c4800fda0 con 0x7f8c580fce60 2024-05-17T21:50:00.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:00.048+0000 7f8c54ff9700 1 -- 172.21.15.107:0/116386817 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8c4800d310 con 0x7f8c580fce60 2024-05-17T21:50:00.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:00.048+0000 7f8c5daa3700 1 -- 172.21.15.107:0/116386817 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c58070c40 con 0x7f8c580fce60 2024-05-17T21:50:00.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:00.048+0000 7f8c5daa3700 1 -- 172.21.15.107:0/116386817 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c58115a30 con 0x7f8c580fce60 2024-05-17T21:50:00.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:00.049+0000 7f8c5daa3700 1 -- 172.21.15.107:0/116386817 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c5804c710 con 0x7f8c580fce60 2024-05-17T21:50:00.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:00.049+0000 7f8c54ff9700 1 -- 172.21.15.107:0/116386817 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f8c48010420 con 0x7f8c580fce60 2024-05-17T21:50:00.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:00.050+0000 7f8c54ff9700 1 --2- 172.21.15.107:0/116386817 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f8c440692a0 0x7f8c4406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:50:00.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:00.050+0000 7f8c54ff9700 1 -- 172.21.15.107:0/116386817 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f8c48087210 con 0x7f8c580fce60 2024-05-17T21:50:00.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:00.050+0000 7f8c4effd700 1 --2- 172.21.15.107:0/116386817 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f8c440692a0 0x7f8c4406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:50:00.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:00.050+0000 7f8c4effd700 1 --2- 172.21.15.107:0/116386817 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f8c440692a0 0x7f8c4406b750 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8c4000b3b0 tx=0x7f8c40009e30 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:50:00.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:00.054+0000 7f8c54ff9700 1 -- 172.21.15.107:0/116386817 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f8c48056f80 con 0x7f8c580fce60 2024-05-17T21:50:00.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:00.239+0000 7f8c5daa3700 1 -- 172.21.15.107:0/116386817 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command([{prefix=config set, name=mgr/orchestrator/fail_fs}] v 0) v1 -- 0x7f8c58105940 con 0x7f8c580fce60 2024-05-17T21:50:00.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:00.239+0000 7f8c54ff9700 1 -- 172.21.15.107:0/116386817 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/orchestrator/fail_fs}]=-22 unrecognized config option 'mgr/orchestrator/fail_fs' v21)=-22 unrecognized config option 'mgr/orchestrator/fail_fs' v21) v1 ==== 179+0+0 (secure 0 0 0) 0x7f8c48056b10 con 0x7f8c580fce60 2024-05-17T21:50:00.233 INFO:teuthology.orchestra.run.smithi107.stderr:Error EINVAL: unrecognized config option 'mgr/orchestrator/fail_fs' 2024-05-17T21:50:00.768 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:50:00 smithi138 ceph-mon[35948]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-17T21:50:00.768 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:50:00 smithi138 ceph-mon[35948]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-17T21:50:00.769 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:50:00 smithi138 ceph-mon[35948]: fs cephfs has deprecated feature inline_data enabled. 2024-05-17T21:50:00.836 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:50:00 smithi107 ceph-mon[28434]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-17T21:50:00.836 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:50:00 smithi107 ceph-mon[28434]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-17T21:50:00.836 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:50:00 smithi107 ceph-mon[28434]: fs cephfs has deprecated feature inline_data enabled. 2024-05-17T21:50:00.861 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-05-17T21:50:00.861 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi107.front.sepia.ceph.com 2024-05-17T21:50:00.861 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -e sha1=931809a9e5d295bd61984693cac44fe131ded4bc -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force' 2024-05-17T21:50:01.160 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/mon.smithi107/config 2024-05-17T21:50:01.514 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:50:01 smithi107 ceph-mon[28434]: pgmap v198: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-05-17T21:50:01.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:50:01 smithi138 ceph-mon[35948]: pgmap v198: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-05-17T21:50:02.476 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:50:02 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:50:02.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:02.725+0000 7fae2f91d700 1 -- 172.21.15.107:0/2173109719 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae280fce60 msgr2=0x7fae280fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:50:02.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:02.725+0000 7fae2f91d700 1 --2- 172.21.15.107:0/2173109719 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae280fce60 0x7fae280fd270 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fae1c004940 tx=0x7fae1c009e30 comp rx=0 tx=0).stop 2024-05-17T21:50:02.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:02.725+0000 7fae2f91d700 1 -- 172.21.15.107:0/2173109719 shutdown_connections 2024-05-17T21:50:02.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:02.725+0000 7fae2f91d700 1 --2- 172.21.15.107:0/2173109719 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae280fdfd0 0x7fae280fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:02.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:02.725+0000 7fae2f91d700 1 --2- 172.21.15.107:0/2173109719 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae280fce60 0x7fae280fd270 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:02.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:02.725+0000 7fae2f91d700 1 -- 172.21.15.107:0/2173109719 >> 172.21.15.107:0/2173109719 conn(0x7fae280f8540 msgr2=0x7fae280fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:50:02.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:02.725+0000 7fae2f91d700 1 -- 172.21.15.107:0/2173109719 shutdown_connections 2024-05-17T21:50:02.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:02.725+0000 7fae2f91d700 1 -- 172.21.15.107:0/2173109719 wait complete. 2024-05-17T21:50:02.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:02.726+0000 7fae2f91d700 1 Processor -- start 2024-05-17T21:50:02.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:02.726+0000 7fae2f91d700 1 -- start start 2024-05-17T21:50:02.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:02.726+0000 7fae2f91d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae280fce60 0x7fae2810d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:50:02.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:02.726+0000 7fae2f91d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae280fdfd0 0x7fae2810de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:50:02.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:02.726+0000 7fae2f91d700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae2810e480 con 0x7fae280fdfd0 2024-05-17T21:50:02.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:02.726+0000 7fae2f91d700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae2810e5f0 con 0x7fae280fce60 2024-05-17T21:50:02.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:02.726+0000 7fae2ceb8700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae280fdfd0 0x7fae2810de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:50:02.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:02.726+0000 7fae2ceb8700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae280fdfd0 0x7fae2810de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46692/0 (socket says 172.21.15.107:46692) 2024-05-17T21:50:02.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:02.726+0000 7fae2ceb8700 1 -- 172.21.15.107:0/216715169 learned_addr learned my addr 172.21.15.107:0/216715169 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:50:02.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:02.726+0000 7fae2d6b9700 1 --2- 172.21.15.107:0/216715169 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae280fce60 0x7fae2810d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:50:02.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:02.726+0000 7fae2ceb8700 1 -- 172.21.15.107:0/216715169 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae280fce60 msgr2=0x7fae2810d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:50:02.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:02.726+0000 7fae2ceb8700 1 --2- 172.21.15.107:0/216715169 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae280fce60 0x7fae2810d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:02.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:02.726+0000 7fae2ceb8700 1 -- 172.21.15.107:0/216715169 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae1c005040 con 0x7fae280fdfd0 2024-05-17T21:50:02.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:02.726+0000 7fae2d6b9700 1 --2- 172.21.15.107:0/216715169 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae280fce60 0x7fae2810d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:50:02.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:02.727+0000 7fae2ceb8700 1 --2- 172.21.15.107:0/216715169 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae280fdfd0 0x7fae2810de80 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fae2400c8c0 tx=0x7fae240044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:50:02.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:02.727+0000 7fae19ffb700 1 -- 172.21.15.107:0/216715169 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fae2400a980 con 0x7fae280fdfd0 2024-05-17T21:50:02.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:02.727+0000 7fae19ffb700 1 -- 172.21.15.107:0/216715169 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fae2400ce60 con 0x7fae280fdfd0 2024-05-17T21:50:02.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:02.727+0000 7fae2f91d700 1 -- 172.21.15.107:0/216715169 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae28113050 con 0x7fae280fdfd0 2024-05-17T21:50:02.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:02.727+0000 7fae19ffb700 1 -- 172.21.15.107:0/216715169 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fae240115e0 con 0x7fae280fdfd0 2024-05-17T21:50:02.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:02.727+0000 7fae2f91d700 1 -- 172.21.15.107:0/216715169 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae281135e0 con 0x7fae280fdfd0 2024-05-17T21:50:02.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:02.728+0000 7fae2f91d700 1 -- 172.21.15.107:0/216715169 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae2804c710 con 0x7fae280fdfd0 2024-05-17T21:50:02.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:02.728+0000 7fae19ffb700 1 -- 172.21.15.107:0/216715169 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7fae24012400 con 0x7fae280fdfd0 2024-05-17T21:50:02.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:02.728+0000 7fae19ffb700 1 --2- 172.21.15.107:0/216715169 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fae140692a0 0x7fae1406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:50:02.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:02.729+0000 7fae19ffb700 1 -- 172.21.15.107:0/216715169 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fae240871e0 con 0x7fae280fdfd0 2024-05-17T21:50:02.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:02.729+0000 7fae2d6b9700 1 --2- 172.21.15.107:0/216715169 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fae140692a0 0x7fae1406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:50:02.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:02.729+0000 7fae2d6b9700 1 --2- 172.21.15.107:0/216715169 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fae140692a0 0x7fae1406b750 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fae1c00b970 tx=0x7fae1c009e30 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:50:02.725 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:02.733+0000 7fae19ffb700 1 -- 172.21.15.107:0/216715169 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fae24056ff0 con 0x7fae280fdfd0 2024-05-17T21:50:02.728 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:50:02 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:50:02.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:02.920+0000 7fae2f91d700 1 -- 172.21.15.107:0/216715169 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}] v 0) v1 -- 0x7fae281058c0 con 0x7fae280fdfd0 2024-05-17T21:50:02.913 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:02.921+0000 7fae19ffb700 1 -- 172.21.15.107:0/216715169 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}]=0 v21)=0 v21) v1 ==== 155+0+0 (secure 0 0 0) 0x7fae240126b0 con 0x7fae280fdfd0 2024-05-17T21:50:02.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:02.923+0000 7fae2f91d700 1 -- 172.21.15.107:0/216715169 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fae140692a0 msgr2=0x7fae1406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:50:02.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:02.923+0000 7fae2f91d700 1 --2- 172.21.15.107:0/216715169 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fae140692a0 0x7fae1406b750 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fae1c00b970 tx=0x7fae1c009e30 comp rx=0 tx=0).stop 2024-05-17T21:50:02.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:02.923+0000 7fae2f91d700 1 -- 172.21.15.107:0/216715169 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae280fdfd0 msgr2=0x7fae2810de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:50:02.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:02.923+0000 7fae2f91d700 1 --2- 172.21.15.107:0/216715169 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae280fdfd0 0x7fae2810de80 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fae2400c8c0 tx=0x7fae240044c0 comp rx=0 tx=0).stop 2024-05-17T21:50:02.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:02.923+0000 7fae2f91d700 1 -- 172.21.15.107:0/216715169 shutdown_connections 2024-05-17T21:50:02.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:02.923+0000 7fae2f91d700 1 --2- 172.21.15.107:0/216715169 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fae140692a0 0x7fae1406b750 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:02.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:02.923+0000 7fae2f91d700 1 --2- 172.21.15.107:0/216715169 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae280fce60 0x7fae2810d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:02.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:02.923+0000 7fae2f91d700 1 --2- 172.21.15.107:0/216715169 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae280fdfd0 0x7fae2810de80 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:02.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:02.923+0000 7fae2f91d700 1 -- 172.21.15.107:0/216715169 >> 172.21.15.107:0/216715169 conn(0x7fae280f8540 msgr2=0x7fae280f9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:50:02.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:02.923+0000 7fae2f91d700 1 -- 172.21.15.107:0/216715169 shutdown_connections 2024-05-17T21:50:02.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:02.923+0000 7fae2f91d700 1 -- 172.21.15.107:0/216715169 wait complete. 2024-05-17T21:50:03.533 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:50:03 smithi107 ceph-mon[28434]: pgmap v199: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-05-17T21:50:03.557 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -e sha1=931809a9e5d295bd61984693cac44fe131ded4bc -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force' 2024-05-17T21:50:03.741 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:50:03 smithi138 ceph-mon[35948]: pgmap v199: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-05-17T21:50:03.811 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/mon.smithi107/config 2024-05-17T21:50:05.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:05.449+0000 7fcc10dee700 1 -- 172.21.15.107:0/2878205355 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc0c0fd9c0 msgr2=0x7fcc0c0ffda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:50:05.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:05.449+0000 7fcc10dee700 1 --2- 172.21.15.107:0/2878205355 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc0c0fd9c0 0x7fcc0c0ffda0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc004940 tx=0x7fcbfc009e30 comp rx=0 tx=0).stop 2024-05-17T21:50:05.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:05.449+0000 7fcc10dee700 1 -- 172.21.15.107:0/2878205355 shutdown_connections 2024-05-17T21:50:05.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:05.449+0000 7fcc10dee700 1 --2- 172.21.15.107:0/2878205355 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcc0c1002e0 0x7fcc0c1026c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:05.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:05.449+0000 7fcc10dee700 1 --2- 172.21.15.107:0/2878205355 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc0c0fd9c0 0x7fcc0c0ffda0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:05.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:05.449+0000 7fcc10dee700 1 -- 172.21.15.107:0/2878205355 >> 172.21.15.107:0/2878205355 conn(0x7fcc0c0f7440 msgr2=0x7fcc0c0f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:50:05.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:05.449+0000 7fcc10dee700 1 -- 172.21.15.107:0/2878205355 shutdown_connections 2024-05-17T21:50:05.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:05.449+0000 7fcc10dee700 1 -- 172.21.15.107:0/2878205355 wait complete. 2024-05-17T21:50:05.442 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:50:05 smithi107 ceph-mon[28434]: pgmap v200: 65 pgs: 65 active+clean; 585 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 767 B/s wr, 3 op/s 2024-05-17T21:50:05.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:05.449+0000 7fcc10dee700 1 Processor -- start 2024-05-17T21:50:05.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:05.449+0000 7fcc10dee700 1 -- start start 2024-05-17T21:50:05.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:05.450+0000 7fcc10dee700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc0c0fd9c0 0x7fcc0c0fd400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:50:05.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:05.450+0000 7fcc10dee700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcc0c1002e0 0x7fcc0c0fb8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:50:05.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:05.450+0000 7fcc10dee700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0c0fbe80 con 0x7fcc0c0fd9c0 2024-05-17T21:50:05.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:05.450+0000 7fcc10dee700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0c0fbff0 con 0x7fcc0c1002e0 2024-05-17T21:50:05.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:05.450+0000 7fcc0a59c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc0c0fd9c0 0x7fcc0c0fd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:50:05.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:05.450+0000 7fcc0a59c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc0c0fd9c0 0x7fcc0c0fd400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46702/0 (socket says 172.21.15.107:46702) 2024-05-17T21:50:05.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:05.450+0000 7fcc0a59c700 1 -- 172.21.15.107:0/2832547545 learned_addr learned my addr 172.21.15.107:0/2832547545 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:50:05.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:05.450+0000 7fcc09d9b700 1 --2- 172.21.15.107:0/2832547545 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcc0c1002e0 0x7fcc0c0fb8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:50:05.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:05.450+0000 7fcc0a59c700 1 -- 172.21.15.107:0/2832547545 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcc0c1002e0 msgr2=0x7fcc0c0fb8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:50:05.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:05.450+0000 7fcc0a59c700 1 --2- 172.21.15.107:0/2832547545 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcc0c1002e0 0x7fcc0c0fb8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:05.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:05.450+0000 7fcc0a59c700 1 -- 172.21.15.107:0/2832547545 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbfc005040 con 0x7fcc0c0fd9c0 2024-05-17T21:50:05.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:05.450+0000 7fcc0a59c700 1 --2- 172.21.15.107:0/2832547545 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc0c0fd9c0 0x7fcc0c0fd400 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc00b970 tx=0x7fcbfc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:50:05.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:05.450+0000 7fcbfaffd700 1 -- 172.21.15.107:0/2832547545 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcbfc00e040 con 0x7fcc0c0fd9c0 2024-05-17T21:50:05.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:05.450+0000 7fcbfaffd700 1 -- 172.21.15.107:0/2832547545 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcbfc00bbf0 con 0x7fcc0c0fd9c0 2024-05-17T21:50:05.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:05.450+0000 7fcc10dee700 1 -- 172.21.15.107:0/2832547545 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc0c0fc270 con 0x7fcc0c0fd9c0 2024-05-17T21:50:05.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:05.450+0000 7fcbfaffd700 1 -- 172.21.15.107:0/2832547545 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcbfc01c3f0 con 0x7fcc0c0fd9c0 2024-05-17T21:50:05.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:05.451+0000 7fcc10dee700 1 -- 172.21.15.107:0/2832547545 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc0c115a90 con 0x7fcc0c0fd9c0 2024-05-17T21:50:05.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:05.452+0000 7fcc10dee700 1 -- 172.21.15.107:0/2832547545 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc0c05f5c0 con 0x7fcc0c0fd9c0 2024-05-17T21:50:05.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:05.452+0000 7fcbfaffd700 1 -- 172.21.15.107:0/2832547545 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7fcbfc00bd60 con 0x7fcc0c0fd9c0 2024-05-17T21:50:05.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:05.452+0000 7fcbfaffd700 1 --2- 172.21.15.107:0/2832547545 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fcbf40692b0 0x7fcbf406b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:50:05.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:05.453+0000 7fcbfaffd700 1 -- 172.21.15.107:0/2832547545 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fcbfc088400 con 0x7fcc0c0fd9c0 2024-05-17T21:50:05.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:05.453+0000 7fcc09d9b700 1 --2- 172.21.15.107:0/2832547545 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fcbf40692b0 0x7fcbf406b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:50:05.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:05.453+0000 7fcc09d9b700 1 --2- 172.21.15.107:0/2832547545 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fcbf40692b0 0x7fcbf406b760 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fcc000102e0 tx=0x7fcc0000b040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:50:05.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:05.457+0000 7fcbfaffd700 1 -- 172.21.15.107:0/2832547545 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fcbfc054830 con 0x7fcc0c0fd9c0 2024-05-17T21:50:05.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:05.642+0000 7fcc10dee700 1 -- 172.21.15.107:0/2832547545 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}] v 0) v1 -- 0x7fcc0c04bed0 con 0x7fcc0c0fd9c0 2024-05-17T21:50:05.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:05.642+0000 7fcbfaffd700 1 -- 172.21.15.107:0/2832547545 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}]=0 v21)=0 v21) v1 ==== 163+0+0 (secure 0 0 0) 0x7fcbfc057e50 con 0x7fcc0c0fd9c0 2024-05-17T21:50:05.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:05.644+0000 7fcc10dee700 1 -- 172.21.15.107:0/2832547545 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fcbf40692b0 msgr2=0x7fcbf406b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:50:05.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:05.644+0000 7fcc10dee700 1 --2- 172.21.15.107:0/2832547545 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fcbf40692b0 0x7fcbf406b760 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fcc000102e0 tx=0x7fcc0000b040 comp rx=0 tx=0).stop 2024-05-17T21:50:05.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:05.644+0000 7fcc10dee700 1 -- 172.21.15.107:0/2832547545 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc0c0fd9c0 msgr2=0x7fcc0c0fd400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:50:05.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:05.644+0000 7fcc10dee700 1 --2- 172.21.15.107:0/2832547545 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc0c0fd9c0 0x7fcc0c0fd400 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc00b970 tx=0x7fcbfc009e30 comp rx=0 tx=0).stop 2024-05-17T21:50:05.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:05.644+0000 7fcc10dee700 1 -- 172.21.15.107:0/2832547545 shutdown_connections 2024-05-17T21:50:05.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:05.644+0000 7fcc10dee700 1 --2- 172.21.15.107:0/2832547545 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fcbf40692b0 0x7fcbf406b760 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:05.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:05.644+0000 7fcc10dee700 1 --2- 172.21.15.107:0/2832547545 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc0c0fd9c0 0x7fcc0c0fd400 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:05.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:05.644+0000 7fcc10dee700 1 --2- 172.21.15.107:0/2832547545 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcc0c1002e0 0x7fcc0c0fb8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:05.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:05.644+0000 7fcc10dee700 1 -- 172.21.15.107:0/2832547545 >> 172.21.15.107:0/2832547545 conn(0x7fcc0c0f7440 msgr2=0x7fcc0c0f7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:50:05.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:05.645+0000 7fcc10dee700 1 -- 172.21.15.107:0/2832547545 shutdown_connections 2024-05-17T21:50:05.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:05.645+0000 7fcc10dee700 1 -- 172.21.15.107:0/2832547545 wait complete. 2024-05-17T21:50:05.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:50:05 smithi138 ceph-mon[35948]: pgmap v200: 65 pgs: 65 active+clean; 585 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 767 B/s wr, 3 op/s 2024-05-17T21:50:06.365 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -e sha1=931809a9e5d295bd61984693cac44fe131ded4bc -- bash -c 'ceph config set global log_to_journald false --force' 2024-05-17T21:50:06.621 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/mon.smithi107/config 2024-05-17T21:50:07.697 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:50:07 smithi107 ceph-mon[28434]: pgmap v201: 65 pgs: 65 active+clean; 585 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.4 KiB/s rd, 767 B/s wr, 2 op/s 2024-05-17T21:50:07.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:50:07 smithi138 ceph-mon[35948]: pgmap v201: 65 pgs: 65 active+clean; 585 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.4 KiB/s rd, 767 B/s wr, 2 op/s 2024-05-17T21:50:08.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:08.255+0000 7efe22cf3700 1 -- 172.21.15.107:0/1061507343 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe1c0fbd40 msgr2=0x7efe1c0fc150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:50:08.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:08.255+0000 7efe22cf3700 1 --2- 172.21.15.107:0/1061507343 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe1c0fbd40 0x7efe1c0fc150 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7efe10004940 tx=0x7efe10009e30 comp rx=0 tx=0).stop 2024-05-17T21:50:08.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:08.255+0000 7efe22cf3700 1 -- 172.21.15.107:0/1061507343 shutdown_connections 2024-05-17T21:50:08.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:08.255+0000 7efe22cf3700 1 --2- 172.21.15.107:0/1061507343 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe1c0fcff0 0x7efe1c0fd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:08.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:08.255+0000 7efe22cf3700 1 --2- 172.21.15.107:0/1061507343 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe1c0fbd40 0x7efe1c0fc150 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:08.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:08.255+0000 7efe22cf3700 1 -- 172.21.15.107:0/1061507343 >> 172.21.15.107:0/1061507343 conn(0x7efe1c0f74c0 msgr2=0x7efe1c0f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:50:08.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:08.255+0000 7efe22cf3700 1 -- 172.21.15.107:0/1061507343 shutdown_connections 2024-05-17T21:50:08.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:08.255+0000 7efe22cf3700 1 -- 172.21.15.107:0/1061507343 wait complete. 2024-05-17T21:50:08.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:08.256+0000 7efe22cf3700 1 Processor -- start 2024-05-17T21:50:08.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:08.256+0000 7efe22cf3700 1 -- start start 2024-05-17T21:50:08.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:08.256+0000 7efe22cf3700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe1c0fbd40 0x7efe1c10da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:50:08.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:08.256+0000 7efe22cf3700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe1c0fcff0 0x7efe1c10df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:50:08.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:08.256+0000 7efe22cf3700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe1c10e5d0 con 0x7efe1c0fcff0 2024-05-17T21:50:08.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:08.256+0000 7efe22cf3700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe1c10edc0 con 0x7efe1c0fbd40 2024-05-17T21:50:08.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:08.256+0000 7efe1bfff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe1c0fcff0 0x7efe1c10df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:50:08.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:08.256+0000 7efe1bfff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe1c0fcff0 0x7efe1c10df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50078/0 (socket says 172.21.15.107:50078) 2024-05-17T21:50:08.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:08.256+0000 7efe1bfff700 1 -- 172.21.15.107:0/633871833 learned_addr learned my addr 172.21.15.107:0/633871833 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:50:08.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:08.257+0000 7efe20a8f700 1 --2- 172.21.15.107:0/633871833 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe1c0fbd40 0x7efe1c10da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:50:08.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:08.257+0000 7efe1bfff700 1 -- 172.21.15.107:0/633871833 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe1c0fbd40 msgr2=0x7efe1c10da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:50:08.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:08.257+0000 7efe1bfff700 1 --2- 172.21.15.107:0/633871833 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe1c0fbd40 0x7efe1c10da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:08.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:08.257+0000 7efe1bfff700 1 -- 172.21.15.107:0/633871833 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe10005040 con 0x7efe1c0fcff0 2024-05-17T21:50:08.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:08.257+0000 7efe1bfff700 1 --2- 172.21.15.107:0/633871833 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe1c0fcff0 0x7efe1c10df40 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7efe0800b790 tx=0x7efe08009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:50:08.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:08.257+0000 7efe197fa700 1 -- 172.21.15.107:0/633871833 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe0800f770 con 0x7efe1c0fcff0 2024-05-17T21:50:08.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:08.257+0000 7efe197fa700 1 -- 172.21.15.107:0/633871833 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efe0800fda0 con 0x7efe1c0fcff0 2024-05-17T21:50:08.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:08.257+0000 7efe197fa700 1 -- 172.21.15.107:0/633871833 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe0800d310 con 0x7efe1c0fcff0 2024-05-17T21:50:08.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:08.257+0000 7efe22cf3700 1 -- 172.21.15.107:0/633871833 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe1c10f0a0 con 0x7efe1c0fcff0 2024-05-17T21:50:08.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:08.257+0000 7efe22cf3700 1 -- 172.21.15.107:0/633871833 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe1c10f610 con 0x7efe1c0fcff0 2024-05-17T21:50:08.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:08.258+0000 7efe22cf3700 1 -- 172.21.15.107:0/633871833 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe1c101b80 con 0x7efe1c0fcff0 2024-05-17T21:50:08.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:08.259+0000 7efe197fa700 1 -- 172.21.15.107:0/633871833 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7efe08010420 con 0x7efe1c0fcff0 2024-05-17T21:50:08.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:08.259+0000 7efe197fa700 1 --2- 172.21.15.107:0/633871833 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7efe0c0692a0 0x7efe0c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:50:08.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:08.259+0000 7efe197fa700 1 -- 172.21.15.107:0/633871833 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7efe08087210 con 0x7efe1c0fcff0 2024-05-17T21:50:08.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:08.259+0000 7efe20a8f700 1 --2- 172.21.15.107:0/633871833 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7efe0c0692a0 0x7efe0c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:50:08.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:08.259+0000 7efe20a8f700 1 --2- 172.21.15.107:0/633871833 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7efe0c0692a0 0x7efe0c06b750 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7efe1000b3b0 tx=0x7efe10009e30 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:50:08.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:08.263+0000 7efe197fa700 1 -- 172.21.15.107:0/633871833 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7efe08056f80 con 0x7efe1c0fcff0 2024-05-17T21:50:08.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:08.452+0000 7efe22cf3700 1 -- 172.21.15.107:0/633871833 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command([{prefix=config set, name=log_to_journald}] v 0) v1 -- 0x7efe1c1049b0 con 0x7efe1c0fcff0 2024-05-17T21:50:08.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:08.452+0000 7efe197fa700 1 -- 172.21.15.107:0/633871833 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{prefix=config set, name=log_to_journald}]=0 v21)=0 v21) v1 ==== 135+0+0 (secure 0 0 0) 0x7efe08056b10 con 0x7efe1c0fcff0 2024-05-17T21:50:08.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:08.454+0000 7efe22cf3700 1 -- 172.21.15.107:0/633871833 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7efe0c0692a0 msgr2=0x7efe0c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:50:08.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:08.454+0000 7efe22cf3700 1 --2- 172.21.15.107:0/633871833 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7efe0c0692a0 0x7efe0c06b750 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7efe1000b3b0 tx=0x7efe10009e30 comp rx=0 tx=0).stop 2024-05-17T21:50:08.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:08.454+0000 7efe22cf3700 1 -- 172.21.15.107:0/633871833 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe1c0fcff0 msgr2=0x7efe1c10df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:50:08.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:08.454+0000 7efe22cf3700 1 --2- 172.21.15.107:0/633871833 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe1c0fcff0 0x7efe1c10df40 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7efe0800b790 tx=0x7efe08009250 comp rx=0 tx=0).stop 2024-05-17T21:50:08.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:08.455+0000 7efe22cf3700 1 -- 172.21.15.107:0/633871833 shutdown_connections 2024-05-17T21:50:08.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:08.455+0000 7efe22cf3700 1 --2- 172.21.15.107:0/633871833 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7efe0c0692a0 0x7efe0c06b750 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:08.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:08.455+0000 7efe22cf3700 1 --2- 172.21.15.107:0/633871833 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe1c0fbd40 0x7efe1c10da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:08.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:08.455+0000 7efe22cf3700 1 --2- 172.21.15.107:0/633871833 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe1c0fcff0 0x7efe1c10df40 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:08.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:08.455+0000 7efe22cf3700 1 -- 172.21.15.107:0/633871833 >> 172.21.15.107:0/633871833 conn(0x7efe1c0f74c0 msgr2=0x7efe1c0f8b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:50:08.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:08.455+0000 7efe22cf3700 1 -- 172.21.15.107:0/633871833 shutdown_connections 2024-05-17T21:50:08.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:08.455+0000 7efe22cf3700 1 -- 172.21.15.107:0/633871833 wait complete. 2024-05-17T21:50:09.082 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -e sha1=931809a9e5d295bd61984693cac44fe131ded4bc -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1' 2024-05-17T21:50:09.340 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/mon.smithi107/config 2024-05-17T21:50:09.707 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:50:09 smithi107 ceph-mon[28434]: pgmap v202: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 938 B/s wr, 3 op/s 2024-05-17T21:50:09.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:50:09 smithi138 ceph-mon[35948]: pgmap v202: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 938 B/s wr, 3 op/s 2024-05-17T21:50:11.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:11.016+0000 7f438e1b3700 1 -- 172.21.15.107:0/922188945 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f43880fe080 msgr2=0x7f43880fe4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:50:11.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:11.016+0000 7f438e1b3700 1 --2- 172.21.15.107:0/922188945 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f43880fe080 0x7f43880fe4f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f437c004940 tx=0x7f437c009e30 comp rx=0 tx=0).stop 2024-05-17T21:50:11.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:11.016+0000 7f438e1b3700 1 -- 172.21.15.107:0/922188945 shutdown_connections 2024-05-17T21:50:11.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:11.016+0000 7f438e1b3700 1 --2- 172.21.15.107:0/922188945 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f43880fe080 0x7f43880fe4f0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:11.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:11.016+0000 7f438e1b3700 1 --2- 172.21.15.107:0/922188945 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43880fcf10 0x7f43880fd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:11.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:11.016+0000 7f438e1b3700 1 -- 172.21.15.107:0/922188945 >> 172.21.15.107:0/922188945 conn(0x7f43880f8560 msgr2=0x7f43880fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:50:11.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:11.016+0000 7f438e1b3700 1 -- 172.21.15.107:0/922188945 shutdown_connections 2024-05-17T21:50:11.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:11.016+0000 7f438e1b3700 1 -- 172.21.15.107:0/922188945 wait complete. 2024-05-17T21:50:11.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:11.017+0000 7f438e1b3700 1 Processor -- start 2024-05-17T21:50:11.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:11.017+0000 7f438e1b3700 1 -- start start 2024-05-17T21:50:11.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:11.017+0000 7f438e1b3700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f43880fcf10 0x7f438810da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:50:11.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:11.017+0000 7f438e1b3700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43880fe080 0x7f438810df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:50:11.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:11.017+0000 7f438e1b3700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f438810e5d0 con 0x7f43880fcf10 2024-05-17T21:50:11.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:11.017+0000 7f438e1b3700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f438810edc0 con 0x7f43880fe080 2024-05-17T21:50:11.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:11.017+0000 7f43877fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f43880fcf10 0x7f438810da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:50:11.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:11.017+0000 7f43877fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f43880fcf10 0x7f438810da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50090/0 (socket says 172.21.15.107:50090) 2024-05-17T21:50:11.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:11.017+0000 7f43877fe700 1 -- 172.21.15.107:0/3387318187 learned_addr learned my addr 172.21.15.107:0/3387318187 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:50:11.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:11.017+0000 7f4386ffd700 1 --2- 172.21.15.107:0/3387318187 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43880fe080 0x7f438810df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:50:11.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:11.017+0000 7f43877fe700 1 -- 172.21.15.107:0/3387318187 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43880fe080 msgr2=0x7f438810df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:50:11.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:11.018+0000 7f43877fe700 1 --2- 172.21.15.107:0/3387318187 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43880fe080 0x7f438810df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:11.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:11.018+0000 7f43877fe700 1 -- 172.21.15.107:0/3387318187 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f437c005040 con 0x7f43880fcf10 2024-05-17T21:50:11.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:11.018+0000 7f43877fe700 1 --2- 172.21.15.107:0/3387318187 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f43880fcf10 0x7f438810da00 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f437800b790 tx=0x7f4378009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:50:11.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:11.018+0000 7f436ffff700 1 -- 172.21.15.107:0/3387318187 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f437800f6f0 con 0x7f43880fcf10 2024-05-17T21:50:11.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:11.018+0000 7f438e1b3700 1 -- 172.21.15.107:0/3387318187 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f438810f0a0 con 0x7f43880fcf10 2024-05-17T21:50:11.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:11.018+0000 7f436ffff700 1 -- 172.21.15.107:0/3387318187 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f437800fd20 con 0x7f43880fcf10 2024-05-17T21:50:11.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:11.018+0000 7f436ffff700 1 -- 172.21.15.107:0/3387318187 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f43780103a0 con 0x7f43880fcf10 2024-05-17T21:50:11.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:11.018+0000 7f438e1b3700 1 -- 172.21.15.107:0/3387318187 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f438810f660 con 0x7f43880fcf10 2024-05-17T21:50:11.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:11.019+0000 7f438e1b3700 1 -- 172.21.15.107:0/3387318187 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f438805f5c0 con 0x7f43880fcf10 2024-05-17T21:50:11.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:11.019+0000 7f436ffff700 1 -- 172.21.15.107:0/3387318187 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f4378011400 con 0x7f43880fcf10 2024-05-17T21:50:11.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:11.020+0000 7f436ffff700 1 --2- 172.21.15.107:0/3387318187 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f43700692a0 0x7f437006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:50:11.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:11.020+0000 7f436ffff700 1 -- 172.21.15.107:0/3387318187 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f4378086fc0 con 0x7f43880fcf10 2024-05-17T21:50:11.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:11.020+0000 7f4386ffd700 1 --2- 172.21.15.107:0/3387318187 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f43700692a0 0x7f437006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:50:11.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:11.020+0000 7f4386ffd700 1 --2- 172.21.15.107:0/3387318187 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f43700692a0 0x7f437006b750 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f437c00b970 tx=0x7f437c009e30 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:50:11.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:11.024+0000 7f436ffff700 1 -- 172.21.15.107:0/3387318187 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f4378056d30 con 0x7f43880fcf10 2024-05-17T21:50:11.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:11.225+0000 7f438e1b3700 1 -- 172.21.15.107:0/3387318187 --> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] -- mgr_command(tid 0: {"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc", "target": ["mon-mgr", ""]}) v1 -- 0x7f438810f940 con 0x7f43700692a0 2024-05-17T21:50:11.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:11.233+0000 7f436ffff700 1 -- 172.21.15.107:0/3387318187 <== mgr.14214 v2:172.21.15.107:6800/1294361014 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+89 (secure 0 0 0) 0x7f438810f940 con 0x7f43700692a0 2024-05-17T21:50:11.225 INFO:teuthology.orchestra.run.smithi107.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc 2024-05-17T21:50:11.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:11.235+0000 7f438e1b3700 1 -- 172.21.15.107:0/3387318187 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f43700692a0 msgr2=0x7f437006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:50:11.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:11.235+0000 7f438e1b3700 1 --2- 172.21.15.107:0/3387318187 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f43700692a0 0x7f437006b750 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f437c00b970 tx=0x7f437c009e30 comp rx=0 tx=0).stop 2024-05-17T21:50:11.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:11.235+0000 7f438e1b3700 1 -- 172.21.15.107:0/3387318187 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f43880fcf10 msgr2=0x7f438810da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:50:11.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:11.235+0000 7f438e1b3700 1 --2- 172.21.15.107:0/3387318187 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f43880fcf10 0x7f438810da00 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f437800b790 tx=0x7f4378009250 comp rx=0 tx=0).stop 2024-05-17T21:50:11.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:11.235+0000 7f438e1b3700 1 -- 172.21.15.107:0/3387318187 shutdown_connections 2024-05-17T21:50:11.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:11.235+0000 7f438e1b3700 1 --2- 172.21.15.107:0/3387318187 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f43700692a0 0x7f437006b750 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:11.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:11.235+0000 7f438e1b3700 1 --2- 172.21.15.107:0/3387318187 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f43880fcf10 0x7f438810da00 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:11.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:11.235+0000 7f438e1b3700 1 --2- 172.21.15.107:0/3387318187 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43880fe080 0x7f438810df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:11.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:11.235+0000 7f438e1b3700 1 -- 172.21.15.107:0/3387318187 >> 172.21.15.107:0/3387318187 conn(0x7f43880f8560 msgr2=0x7f43880f9ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:50:11.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:11.235+0000 7f438e1b3700 1 -- 172.21.15.107:0/3387318187 shutdown_connections 2024-05-17T21:50:11.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:11.235+0000 7f438e1b3700 1 -- 172.21.15.107:0/3387318187 wait complete. 2024-05-17T21:50:11.552 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:50:11 smithi107 ceph-mon[28434]: pgmap v203: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 853 B/s wr, 2 op/s 2024-05-17T21:50:11.552 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:50:11 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:50:11.552 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:50:11 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:50:11.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:50:11 smithi138 ceph-mon[35948]: pgmap v203: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 853 B/s wr, 2 op/s 2024-05-17T21:50:11.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:50:11 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:50:11.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:50:11 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:50:12.000 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-05-17T21:50:12.001 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi107.front.sepia.ceph.com 2024-05-17T21:50:12.001 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -e sha1=931809a9e5d295bd61984693cac44fe131ded4bc -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true && ! ceph orch upgrade status | jq '"'"'.message'"'"' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done' 2024-05-17T21:50:12.267 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/mon.smithi107/config 2024-05-17T21:50:12.631 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:50:12 smithi107 ceph-mon[28434]: from='client.14548 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:50:12.631 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:50:12 smithi107 ceph-mon[28434]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc 2024-05-17T21:50:12.631 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:50:12 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:50:12.632 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:50:12 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T21:50:12.632 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:50:12 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:50:12.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:50:12 smithi138 ceph-mon[35948]: from='client.14548 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:50:12.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:50:12 smithi138 ceph-mon[35948]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc 2024-05-17T21:50:12.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:50:12 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:50:12.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:50:12 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T21:50:12.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:50:12 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:50:13.708 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:50:13 smithi107 ceph-mon[28434]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc 2024-05-17T21:50:13.708 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:50:13 smithi107 ceph-mon[28434]: pgmap v204: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 853 B/s wr, 2 op/s 2024-05-17T21:50:13.780 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:50:13 smithi138 ceph-mon[35948]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc 2024-05-17T21:50:13.780 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:50:13 smithi138 ceph-mon[35948]: pgmap v204: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 853 B/s wr, 2 op/s 2024-05-17T21:50:14.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.181+0000 7f067ae8c700 1 -- 172.21.15.107:0/791249764 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06740fff80 msgr2=0x7f0674102360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:50:14.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.181+0000 7f067ae8c700 1 --2- 172.21.15.107:0/791249764 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06740fff80 0x7f0674102360 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f0660004940 tx=0x7f0660009e30 comp rx=0 tx=0).stop 2024-05-17T21:50:14.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.181+0000 7f067ae8c700 1 -- 172.21.15.107:0/791249764 shutdown_connections 2024-05-17T21:50:14.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.181+0000 7f067ae8c700 1 --2- 172.21.15.107:0/791249764 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06740fff80 0x7f0674102360 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:14.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.181+0000 7f067ae8c700 1 --2- 172.21.15.107:0/791249764 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0674066220 0x7f06740ffa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:14.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.181+0000 7f067ae8c700 1 -- 172.21.15.107:0/791249764 >> 172.21.15.107:0/791249764 conn(0x7f06740f7430 msgr2=0x7f06740f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:50:14.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.181+0000 7f067ae8c700 1 -- 172.21.15.107:0/791249764 shutdown_connections 2024-05-17T21:50:14.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.181+0000 7f067ae8c700 1 -- 172.21.15.107:0/791249764 wait complete. 2024-05-17T21:50:14.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.182+0000 7f067ae8c700 1 Processor -- start 2024-05-17T21:50:14.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.182+0000 7f067ae8c700 1 -- start start 2024-05-17T21:50:14.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.182+0000 7f067ae8c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0674066220 0x7f067410fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:50:14.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.182+0000 7f067ae8c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06740fff80 0x7f0674110160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:50:14.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.182+0000 7f067ae8c700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0674110760 con 0x7f0674066220 2024-05-17T21:50:14.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.182+0000 7f067ae8c700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0674110fb0 con 0x7f06740fff80 2024-05-17T21:50:14.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.182+0000 7f0678c28700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0674066220 0x7f067410fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:50:14.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.182+0000 7f0678c28700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0674066220 0x7f067410fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50106/0 (socket says 172.21.15.107:50106) 2024-05-17T21:50:14.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.182+0000 7f0678c28700 1 -- 172.21.15.107:0/1282365648 learned_addr learned my addr 172.21.15.107:0/1282365648 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:50:14.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.182+0000 7f0673fff700 1 --2- 172.21.15.107:0/1282365648 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06740fff80 0x7f0674110160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:50:14.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.183+0000 7f0678c28700 1 -- 172.21.15.107:0/1282365648 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06740fff80 msgr2=0x7f0674110160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:50:14.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.183+0000 7f0678c28700 1 --2- 172.21.15.107:0/1282365648 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06740fff80 0x7f0674110160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:14.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.183+0000 7f0678c28700 1 -- 172.21.15.107:0/1282365648 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0660005040 con 0x7f0674066220 2024-05-17T21:50:14.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.183+0000 7f0678c28700 1 --2- 172.21.15.107:0/1282365648 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0674066220 0x7f067410fc20 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f0668002a80 tx=0x7f0668004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:50:14.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.183+0000 7f06717fa700 1 -- 172.21.15.107:0/1282365648 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0668007cc0 con 0x7f0674066220 2024-05-17T21:50:14.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.183+0000 7f067ae8c700 1 -- 172.21.15.107:0/1282365648 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0674111290 con 0x7f0674066220 2024-05-17T21:50:14.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.183+0000 7f06717fa700 1 -- 172.21.15.107:0/1282365648 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f066800f040 con 0x7f0674066220 2024-05-17T21:50:14.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.183+0000 7f06717fa700 1 -- 172.21.15.107:0/1282365648 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f066800da30 con 0x7f0674066220 2024-05-17T21:50:14.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.183+0000 7f067ae8c700 1 -- 172.21.15.107:0/1282365648 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06741117d0 con 0x7f0674066220 2024-05-17T21:50:14.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.184+0000 7f067ae8c700 1 -- 172.21.15.107:0/1282365648 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f067405f5c0 con 0x7f0674066220 2024-05-17T21:50:14.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.185+0000 7f06717fa700 1 -- 172.21.15.107:0/1282365648 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f066800db90 con 0x7f0674066220 2024-05-17T21:50:14.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.185+0000 7f06717fa700 1 --2- 172.21.15.107:0/1282365648 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f06640692b0 0x7f066406b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:50:14.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.185+0000 7f06717fa700 1 -- 172.21.15.107:0/1282365648 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f0668086f70 con 0x7f0674066220 2024-05-17T21:50:14.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.185+0000 7f0673fff700 1 --2- 172.21.15.107:0/1282365648 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f06640692b0 0x7f066406b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:50:14.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.186+0000 7f0673fff700 1 --2- 172.21.15.107:0/1282365648 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f06640692b0 0x7f066406b760 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f0660009fd0 tx=0x7f0660009e30 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:50:14.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.190+0000 7f06717fa700 1 -- 172.21.15.107:0/1282365648 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f0668056c60 con 0x7f0674066220 2024-05-17T21:50:14.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.421+0000 7f067ae8c700 1 -- 172.21.15.107:0/1282365648 --> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0674111b60 con 0x7f06640692b0 2024-05-17T21:50:14.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.422+0000 7f06717fa700 1 -- 172.21.15.107:0/1282365648 <== mgr.14214 v2:172.21.15.107:6800/1294361014 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7f0674111b60 con 0x7f06640692b0 2024-05-17T21:50:14.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.424+0000 7f067ae8c700 1 -- 172.21.15.107:0/1282365648 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f06640692b0 msgr2=0x7f066406b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:50:14.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.424+0000 7f067ae8c700 1 --2- 172.21.15.107:0/1282365648 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f06640692b0 0x7f066406b760 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f0660009fd0 tx=0x7f0660009e30 comp rx=0 tx=0).stop 2024-05-17T21:50:14.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.424+0000 7f067ae8c700 1 -- 172.21.15.107:0/1282365648 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0674066220 msgr2=0x7f067410fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:50:14.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.424+0000 7f067ae8c700 1 --2- 172.21.15.107:0/1282365648 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0674066220 0x7f067410fc20 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f0668002a80 tx=0x7f0668004600 comp rx=0 tx=0).stop 2024-05-17T21:50:14.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.424+0000 7f067ae8c700 1 -- 172.21.15.107:0/1282365648 shutdown_connections 2024-05-17T21:50:14.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.424+0000 7f067ae8c700 1 --2- 172.21.15.107:0/1282365648 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f06640692b0 0x7f066406b760 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:14.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.424+0000 7f067ae8c700 1 --2- 172.21.15.107:0/1282365648 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0674066220 0x7f067410fc20 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:14.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.424+0000 7f067ae8c700 1 --2- 172.21.15.107:0/1282365648 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06740fff80 0x7f0674110160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:14.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.424+0000 7f067ae8c700 1 -- 172.21.15.107:0/1282365648 >> 172.21.15.107:0/1282365648 conn(0x7f06740f7430 msgr2=0x7f06740fbb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:50:14.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.424+0000 7f067ae8c700 1 -- 172.21.15.107:0/1282365648 shutdown_connections 2024-05-17T21:50:14.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.424+0000 7f067ae8c700 1 -- 172.21.15.107:0/1282365648 wait complete. 2024-05-17T21:50:14.426 INFO:teuthology.orchestra.run.smithi107.stdout:true 2024-05-17T21:50:14.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.535+0000 7fad3d90e700 1 -- 172.21.15.107:0/2077491219 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad380fe080 msgr2=0x7fad380fe4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:50:14.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.535+0000 7fad3d90e700 1 --2- 172.21.15.107:0/2077491219 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad380fe080 0x7fad380fe4f0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fad2c004940 tx=0x7fad2c009e30 comp rx=0 tx=0).stop 2024-05-17T21:50:14.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.535+0000 7fad3d90e700 1 -- 172.21.15.107:0/2077491219 shutdown_connections 2024-05-17T21:50:14.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.535+0000 7fad3d90e700 1 --2- 172.21.15.107:0/2077491219 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad380fe080 0x7fad380fe4f0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:14.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.535+0000 7fad3d90e700 1 --2- 172.21.15.107:0/2077491219 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad380fcf10 0x7fad380fd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:14.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.535+0000 7fad3d90e700 1 -- 172.21.15.107:0/2077491219 >> 172.21.15.107:0/2077491219 conn(0x7fad380f8560 msgr2=0x7fad380fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:50:14.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.535+0000 7fad3d90e700 1 -- 172.21.15.107:0/2077491219 shutdown_connections 2024-05-17T21:50:14.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.536+0000 7fad3d90e700 1 -- 172.21.15.107:0/2077491219 wait complete. 2024-05-17T21:50:14.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.536+0000 7fad3d90e700 1 Processor -- start 2024-05-17T21:50:14.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.536+0000 7fad3d90e700 1 -- start start 2024-05-17T21:50:14.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.537+0000 7fad3d90e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad380fcf10 0x7fad3810da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:50:14.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.537+0000 7fad3d90e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad380fe080 0x7fad3810df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:50:14.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.537+0000 7fad3d90e700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad3810e5d0 con 0x7fad380fcf10 2024-05-17T21:50:14.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.537+0000 7fad3d90e700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad3810edc0 con 0x7fad380fe080 2024-05-17T21:50:14.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.537+0000 7fad36ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad380fcf10 0x7fad3810da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:50:14.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.537+0000 7fad36ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad380fcf10 0x7fad3810da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50116/0 (socket says 172.21.15.107:50116) 2024-05-17T21:50:14.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.537+0000 7fad36ffd700 1 -- 172.21.15.107:0/3389134628 learned_addr learned my addr 172.21.15.107:0/3389134628 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:50:14.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.537+0000 7fad367fc700 1 --2- 172.21.15.107:0/3389134628 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad380fe080 0x7fad3810df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:50:14.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.537+0000 7fad36ffd700 1 -- 172.21.15.107:0/3389134628 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad380fe080 msgr2=0x7fad3810df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:50:14.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.537+0000 7fad36ffd700 1 --2- 172.21.15.107:0/3389134628 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad380fe080 0x7fad3810df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:14.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.537+0000 7fad36ffd700 1 -- 172.21.15.107:0/3389134628 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad2c005040 con 0x7fad380fcf10 2024-05-17T21:50:14.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.537+0000 7fad367fc700 1 --2- 172.21.15.107:0/3389134628 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad380fe080 0x7fad3810df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:50:14.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.538+0000 7fad36ffd700 1 --2- 172.21.15.107:0/3389134628 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad380fcf10 0x7fad3810da00 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fad2800b790 tx=0x7fad28009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:50:14.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.538+0000 7fad1f7fe700 1 -- 172.21.15.107:0/3389134628 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fad2800f770 con 0x7fad380fcf10 2024-05-17T21:50:14.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.538+0000 7fad3d90e700 1 -- 172.21.15.107:0/3389134628 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad3810f0a0 con 0x7fad380fcf10 2024-05-17T21:50:14.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.538+0000 7fad3d90e700 1 -- 172.21.15.107:0/3389134628 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad3810f660 con 0x7fad380fcf10 2024-05-17T21:50:14.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.538+0000 7fad1f7fe700 1 -- 172.21.15.107:0/3389134628 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fad2800fda0 con 0x7fad380fcf10 2024-05-17T21:50:14.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.538+0000 7fad1f7fe700 1 -- 172.21.15.107:0/3389134628 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fad2800d310 con 0x7fad380fcf10 2024-05-17T21:50:14.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.539+0000 7fad3d90e700 1 -- 172.21.15.107:0/3389134628 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad3805f5c0 con 0x7fad380fcf10 2024-05-17T21:50:14.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.539+0000 7fad1f7fe700 1 -- 172.21.15.107:0/3389134628 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7fad2800f8d0 con 0x7fad380fcf10 2024-05-17T21:50:14.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.540+0000 7fad1f7fe700 1 --2- 172.21.15.107:0/3389134628 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fad200692a0 0x7fad2006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:50:14.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.540+0000 7fad1f7fe700 1 -- 172.21.15.107:0/3389134628 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fad28087160 con 0x7fad380fcf10 2024-05-17T21:50:14.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.540+0000 7fad367fc700 1 --2- 172.21.15.107:0/3389134628 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fad200692a0 0x7fad2006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:50:14.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.540+0000 7fad367fc700 1 --2- 172.21.15.107:0/3389134628 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fad200692a0 0x7fad2006b750 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fad2c00b970 tx=0x7fad2c009e30 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:50:14.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.544+0000 7fad1f7fe700 1 -- 172.21.15.107:0/3389134628 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fad28056f70 con 0x7fad380fcf10 2024-05-17T21:50:14.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.778+0000 7fad3d90e700 1 -- 172.21.15.107:0/3389134628 --> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fad3810f9d0 con 0x7fad200692a0 2024-05-17T21:50:14.771 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.779+0000 7fad1f7fe700 1 -- 172.21.15.107:0/3389134628 <== mgr.14214 v2:172.21.15.107:6800/1294361014 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7fad3810f9d0 con 0x7fad200692a0 2024-05-17T21:50:14.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.782+0000 7fad3d90e700 1 -- 172.21.15.107:0/3389134628 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fad200692a0 msgr2=0x7fad2006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:50:14.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.782+0000 7fad3d90e700 1 --2- 172.21.15.107:0/3389134628 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fad200692a0 0x7fad2006b750 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fad2c00b970 tx=0x7fad2c009e30 comp rx=0 tx=0).stop 2024-05-17T21:50:14.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.782+0000 7fad3d90e700 1 -- 172.21.15.107:0/3389134628 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad380fcf10 msgr2=0x7fad3810da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:50:14.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.782+0000 7fad3d90e700 1 --2- 172.21.15.107:0/3389134628 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad380fcf10 0x7fad3810da00 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fad2800b790 tx=0x7fad28009250 comp rx=0 tx=0).stop 2024-05-17T21:50:14.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.782+0000 7fad3d90e700 1 -- 172.21.15.107:0/3389134628 shutdown_connections 2024-05-17T21:50:14.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.782+0000 7fad3d90e700 1 --2- 172.21.15.107:0/3389134628 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fad200692a0 0x7fad2006b750 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:14.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.782+0000 7fad3d90e700 1 --2- 172.21.15.107:0/3389134628 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad380fcf10 0x7fad3810da00 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:14.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.782+0000 7fad3d90e700 1 --2- 172.21.15.107:0/3389134628 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad380fe080 0x7fad3810df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:14.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.782+0000 7fad3d90e700 1 -- 172.21.15.107:0/3389134628 >> 172.21.15.107:0/3389134628 conn(0x7fad380f8560 msgr2=0x7fad380f9b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:50:14.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.782+0000 7fad3d90e700 1 -- 172.21.15.107:0/3389134628 shutdown_connections 2024-05-17T21:50:14.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.782+0000 7fad3d90e700 1 -- 172.21.15.107:0/3389134628 wait complete. 2024-05-17T21:50:14.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.894+0000 7f0106c3c700 1 -- 172.21.15.107:0/1354437247 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01000fff40 msgr2=0x7f0100102320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:50:14.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.894+0000 7f0106c3c700 1 --2- 172.21.15.107:0/1354437247 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01000fff40 0x7f0100102320 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f00ec004940 tx=0x7f00ec009e30 comp rx=0 tx=0).stop 2024-05-17T21:50:14.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.894+0000 7f0106c3c700 1 -- 172.21.15.107:0/1354437247 shutdown_connections 2024-05-17T21:50:14.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.894+0000 7f0106c3c700 1 --2- 172.21.15.107:0/1354437247 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01000fff40 0x7f0100102320 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:14.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.894+0000 7f0106c3c700 1 --2- 172.21.15.107:0/1354437247 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0100066260 0x7f01000ffa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:14.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.894+0000 7f0106c3c700 1 -- 172.21.15.107:0/1354437247 >> 172.21.15.107:0/1354437247 conn(0x7f01000f7430 msgr2=0x7f01000f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:50:14.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.895+0000 7f0106c3c700 1 -- 172.21.15.107:0/1354437247 shutdown_connections 2024-05-17T21:50:14.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.895+0000 7f0106c3c700 1 -- 172.21.15.107:0/1354437247 wait complete. 2024-05-17T21:50:14.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.895+0000 7f0106c3c700 1 Processor -- start 2024-05-17T21:50:14.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.895+0000 7f0106c3c700 1 -- start start 2024-05-17T21:50:14.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.896+0000 7f0106c3c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0100066260 0x7f01000fd430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:50:14.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.896+0000 7f0106c3c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f01000fff40 0x7f01000fb900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:50:14.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.896+0000 7f0106c3c700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01000fbeb0 con 0x7f0100066260 2024-05-17T21:50:14.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.896+0000 7f0106c3c700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01000fc020 con 0x7f01000fff40 2024-05-17T21:50:14.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.896+0000 7f01049d8700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0100066260 0x7f01000fd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:50:14.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.896+0000 7f01049d8700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0100066260 0x7f01000fd430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50128/0 (socket says 172.21.15.107:50128) 2024-05-17T21:50:14.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.896+0000 7f01049d8700 1 -- 172.21.15.107:0/168167242 learned_addr learned my addr 172.21.15.107:0/168167242 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:50:14.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.896+0000 7f00fffff700 1 --2- 172.21.15.107:0/168167242 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f01000fff40 0x7f01000fb900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:50:14.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.896+0000 7f01049d8700 1 -- 172.21.15.107:0/168167242 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f01000fff40 msgr2=0x7f01000fb900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:50:14.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.896+0000 7f01049d8700 1 --2- 172.21.15.107:0/168167242 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f01000fff40 0x7f01000fb900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:14.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.896+0000 7f01049d8700 1 -- 172.21.15.107:0/168167242 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00ec005040 con 0x7f0100066260 2024-05-17T21:50:14.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.896+0000 7f01049d8700 1 --2- 172.21.15.107:0/168167242 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0100066260 0x7f01000fd430 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f00f4002a80 tx=0x7f00f4004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:50:14.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.897+0000 7f00fd7fa700 1 -- 172.21.15.107:0/168167242 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f00f4007cc0 con 0x7f0100066260 2024-05-17T21:50:14.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.897+0000 7f00fd7fa700 1 -- 172.21.15.107:0/168167242 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f00f400f040 con 0x7f0100066260 2024-05-17T21:50:14.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.897+0000 7f0106c3c700 1 -- 172.21.15.107:0/168167242 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01000fc300 con 0x7f0100066260 2024-05-17T21:50:14.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.897+0000 7f00fd7fa700 1 -- 172.21.15.107:0/168167242 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f00f400da30 con 0x7f0100066260 2024-05-17T21:50:14.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.897+0000 7f0106c3c700 1 -- 172.21.15.107:0/168167242 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0100115a10 con 0x7f0100066260 2024-05-17T21:50:14.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.898+0000 7f0106c3c700 1 -- 172.21.15.107:0/168167242 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f010005f5c0 con 0x7f0100066260 2024-05-17T21:50:14.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.899+0000 7f00fd7fa700 1 -- 172.21.15.107:0/168167242 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f00f400db90 con 0x7f0100066260 2024-05-17T21:50:14.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.899+0000 7f00fd7fa700 1 --2- 172.21.15.107:0/168167242 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f00f00692a0 0x7f00f006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:50:14.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.899+0000 7f00fd7fa700 1 -- 172.21.15.107:0/168167242 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f00f4086f70 con 0x7f0100066260 2024-05-17T21:50:14.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.899+0000 7f00fffff700 1 --2- 172.21.15.107:0/168167242 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f00f00692a0 0x7f00f006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:50:14.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.900+0000 7f00fffff700 1 --2- 172.21.15.107:0/168167242 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f00f00692a0 0x7f00f006b750 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f00ec00b3b0 tx=0x7f00ec009e30 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:50:14.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:14.907+0000 7f00fd7fa700 1 -- 172.21.15.107:0/168167242 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f00f40531d0 con 0x7f0100066260 2024-05-17T21:50:15.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:15.136+0000 7f0106c3c700 1 -- 172.21.15.107:0/168167242 --> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f01000f8f20 con 0x7f00f00692a0 2024-05-17T21:50:15.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:15.147+0000 7f00fd7fa700 1 -- 172.21.15.107:0/168167242 <== mgr.14214 v2:172.21.15.107:6800/1294361014 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3366 (secure 0 0 0) 0x7f01000f8f20 con 0x7f00f00692a0 2024-05-17T21:50:15.139 INFO:teuthology.orchestra.run.smithi107.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-17T21:50:15.139 INFO:teuthology.orchestra.run.smithi107.stdout:alertmanager.smithi107 smithi107 *:9093,9094 running (4m) 27s ago 6m 25.2M - 0.25.0 c8568f914cd2 a2d817bf64d3 2024-05-17T21:50:15.139 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi107 smithi107 running (6m) 27s ago 6m 6660k - 17.2.7-952-g9903f776 d15a36853796 828741c3a89c 2024-05-17T21:50:15.140 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi138 smithi138 running (5m) 34s ago 5m 6660k - 17.2.7-952-g9903f776 d15a36853796 20e9ccf1fadd 2024-05-17T21:50:15.140 INFO:teuthology.orchestra.run.smithi107.stdout:grafana.smithi107 smithi107 *:3000 running (4m) 27s ago 6m 80.5M - 9.4.7 954c08fa6188 0bc11311fb71 2024-05-17T21:50:15.140 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.drqody smithi107 running (47s) 27s ago 46s 14.5M - 17.2.7-952-g9903f776 d15a36853796 0d851bc912f0 2024-05-17T21:50:15.140 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.idmbgj smithi107 running (54s) 27s ago 53s 10.4M - 17.2.7-952-g9903f776 d15a36853796 1348b09ab29e 2024-05-17T21:50:15.140 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi138.gwbtyd smithi138 running (51s) 34s ago 51s 14.4M - 17.2.7-952-g9903f776 d15a36853796 fd2ba75354c0 2024-05-17T21:50:15.140 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi138.zckrvs smithi138 running (44s) 34s ago 44s 11.2M - 17.2.7-952-g9903f776 d15a36853796 a1cfd2fe4cda 2024-05-17T21:50:15.140 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi107.xhhyiw smithi107 *:9283 running (7m) 27s ago 7m 483M - 17.2.7-952-g9903f776 d15a36853796 b566f08938ae 2024-05-17T21:50:15.140 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi138.napnll smithi138 *:8443,9283 running (5m) 34s ago 5m 436M - 17.2.7-952-g9903f776 d15a36853796 b0a8c6e63a55 2024-05-17T21:50:15.140 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi107 smithi107 running (7m) 27s ago 7m 47.5M 2048M 17.2.7-952-g9903f776 d15a36853796 7de9c23eee97 2024-05-17T21:50:15.140 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi138 smithi138 running (5m) 34s ago 5m 32.4M 2048M 17.2.7-952-g9903f776 d15a36853796 ee61a065778d 2024-05-17T21:50:15.140 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi107 smithi107 *:9100 running (6m) 27s ago 6m 15.4M - 1.5.0 0da6a335fe13 730e9bfe00b2 2024-05-17T21:50:15.140 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi138 smithi138 *:9100 running (5m) 34s ago 5m 16.2M - 1.5.0 0da6a335fe13 296b996124a1 2024-05-17T21:50:15.140 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0 smithi107 running (4m) 27s ago 4m 46.0M 4288M 17.2.7-952-g9903f776 d15a36853796 f2d20009361d 2024-05-17T21:50:15.140 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1 smithi107 running (4m) 27s ago 4m 48.3M 4288M 17.2.7-952-g9903f776 d15a36853796 e5d3e99844a9 2024-05-17T21:50:15.141 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2 smithi107 running (3m) 27s ago 3m 46.5M 4288M 17.2.7-952-g9903f776 d15a36853796 e044d7b78088 2024-05-17T21:50:15.141 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3 smithi138 running (3m) 34s ago 3m 44.5M 5312M 17.2.7-952-g9903f776 d15a36853796 8c980cc8af87 2024-05-17T21:50:15.141 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4 smithi138 running (2m) 34s ago 2m 41.1M 5312M 17.2.7-952-g9903f776 d15a36853796 b306a36a744e 2024-05-17T21:50:15.141 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5 smithi138 running (2m) 34s ago 2m 43.6M 5312M 17.2.7-952-g9903f776 d15a36853796 7c8d8ed74974 2024-05-17T21:50:15.141 INFO:teuthology.orchestra.run.smithi107.stdout:prometheus.smithi107 smithi107 *:9095 running (4m) 27s ago 5m 53.9M - 2.43.0 a07b618ecd1d 38259629b8f2 2024-05-17T21:50:15.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:15.149+0000 7f0106c3c700 1 -- 172.21.15.107:0/168167242 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f00f00692a0 msgr2=0x7f00f006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:50:15.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:15.149+0000 7f0106c3c700 1 --2- 172.21.15.107:0/168167242 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f00f00692a0 0x7f00f006b750 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f00ec00b3b0 tx=0x7f00ec009e30 comp rx=0 tx=0).stop 2024-05-17T21:50:15.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:15.149+0000 7f0106c3c700 1 -- 172.21.15.107:0/168167242 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0100066260 msgr2=0x7f01000fd430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:50:15.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:15.149+0000 7f0106c3c700 1 --2- 172.21.15.107:0/168167242 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0100066260 0x7f01000fd430 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f00f4002a80 tx=0x7f00f4004600 comp rx=0 tx=0).stop 2024-05-17T21:50:15.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:15.150+0000 7f0106c3c700 1 -- 172.21.15.107:0/168167242 shutdown_connections 2024-05-17T21:50:15.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:15.150+0000 7f0106c3c700 1 --2- 172.21.15.107:0/168167242 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f00f00692a0 0x7f00f006b750 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:15.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:15.150+0000 7f0106c3c700 1 --2- 172.21.15.107:0/168167242 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0100066260 0x7f01000fd430 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:15.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:15.150+0000 7f0106c3c700 1 --2- 172.21.15.107:0/168167242 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f01000fff40 0x7f01000fb900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:15.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:15.150+0000 7f0106c3c700 1 -- 172.21.15.107:0/168167242 >> 172.21.15.107:0/168167242 conn(0x7f01000f7430 msgr2=0x7f01000f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:50:15.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:15.150+0000 7f0106c3c700 1 -- 172.21.15.107:0/168167242 shutdown_connections 2024-05-17T21:50:15.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:15.150+0000 7f0106c3c700 1 -- 172.21.15.107:0/168167242 wait complete. 2024-05-17T21:50:15.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:15.262+0000 7fec23dd2700 1 -- 172.21.15.107:0/2647820293 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec1c0fcf60 msgr2=0x7fec1c0fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:50:15.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:15.262+0000 7fec23dd2700 1 --2- 172.21.15.107:0/2647820293 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec1c0fcf60 0x7fec1c0fd3d0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fec0c004940 tx=0x7fec0c009e30 comp rx=0 tx=0).stop 2024-05-17T21:50:15.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:15.262+0000 7fec23dd2700 1 -- 172.21.15.107:0/2647820293 shutdown_connections 2024-05-17T21:50:15.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:15.262+0000 7fec23dd2700 1 --2- 172.21.15.107:0/2647820293 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec1c0fcf60 0x7fec1c0fd3d0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:15.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:15.262+0000 7fec23dd2700 1 --2- 172.21.15.107:0/2647820293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec1c0fbcb0 0x7fec1c0fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:15.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:15.262+0000 7fec23dd2700 1 -- 172.21.15.107:0/2647820293 >> 172.21.15.107:0/2647820293 conn(0x7fec1c0f7430 msgr2=0x7fec1c0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:50:15.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:15.262+0000 7fec23dd2700 1 -- 172.21.15.107:0/2647820293 shutdown_connections 2024-05-17T21:50:15.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:15.262+0000 7fec23dd2700 1 -- 172.21.15.107:0/2647820293 wait complete. 2024-05-17T21:50:15.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:15.263+0000 7fec23dd2700 1 Processor -- start 2024-05-17T21:50:15.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:15.263+0000 7fec23dd2700 1 -- start start 2024-05-17T21:50:15.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:15.263+0000 7fec23dd2700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec1c0fbcb0 0x7fec1c10db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:50:15.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:15.263+0000 7fec23dd2700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec1c0fcf60 0x7fec1c10e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:50:15.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:15.263+0000 7fec23dd2700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec1c10e640 con 0x7fec1c0fcf60 2024-05-17T21:50:15.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:15.263+0000 7fec23dd2700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec1c10edf0 con 0x7fec1c0fbcb0 2024-05-17T21:50:15.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:15.263+0000 7fec2136d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec1c0fcf60 0x7fec1c10e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:50:15.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:15.263+0000 7fec2136d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec1c0fcf60 0x7fec1c10e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50156/0 (socket says 172.21.15.107:50156) 2024-05-17T21:50:15.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:15.263+0000 7fec2136d700 1 -- 172.21.15.107:0/2256487915 learned_addr learned my addr 172.21.15.107:0/2256487915 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:50:15.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:15.263+0000 7fec21b6e700 1 --2- 172.21.15.107:0/2256487915 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec1c0fbcb0 0x7fec1c10db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:50:15.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:15.264+0000 7fec2136d700 1 -- 172.21.15.107:0/2256487915 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec1c0fbcb0 msgr2=0x7fec1c10db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:50:15.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:15.264+0000 7fec2136d700 1 --2- 172.21.15.107:0/2256487915 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec1c0fbcb0 0x7fec1c10db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:15.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:15.264+0000 7fec2136d700 1 -- 172.21.15.107:0/2256487915 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec0c005040 con 0x7fec1c0fcf60 2024-05-17T21:50:15.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:15.264+0000 7fec21b6e700 1 --2- 172.21.15.107:0/2256487915 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec1c0fbcb0 0x7fec1c10db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T21:50:15.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:15.264+0000 7fec2136d700 1 --2- 172.21.15.107:0/2256487915 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec1c0fcf60 0x7fec1c10e040 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fec0c00b3b0 tx=0x7fec0c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:50:15.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:15.264+0000 7fec127fc700 1 -- 172.21.15.107:0/2256487915 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fec0c00e040 con 0x7fec1c0fcf60 2024-05-17T21:50:15.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:15.264+0000 7fec127fc700 1 -- 172.21.15.107:0/2256487915 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fec0c00bcc0 con 0x7fec1c0fcf60 2024-05-17T21:50:15.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:15.264+0000 7fec127fc700 1 -- 172.21.15.107:0/2256487915 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fec0c01c630 con 0x7fec1c0fcf60 2024-05-17T21:50:15.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:15.264+0000 7fec23dd2700 1 -- 172.21.15.107:0/2256487915 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec1c10f070 con 0x7fec1c0fcf60 2024-05-17T21:50:15.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:15.264+0000 7fec23dd2700 1 -- 172.21.15.107:0/2256487915 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec1c10f560 con 0x7fec1c0fcf60 2024-05-17T21:50:15.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:15.265+0000 7fec23dd2700 1 -- 172.21.15.107:0/2256487915 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec1c101af0 con 0x7fec1c0fcf60 2024-05-17T21:50:15.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:15.265+0000 7fec127fc700 1 -- 172.21.15.107:0/2256487915 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7fec0c013980 con 0x7fec1c0fcf60 2024-05-17T21:50:15.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:15.266+0000 7fec127fc700 1 --2- 172.21.15.107:0/2256487915 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fec080692a0 0x7fec0806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:50:15.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:15.266+0000 7fec127fc700 1 -- 172.21.15.107:0/2256487915 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fec0c0880b0 con 0x7fec1c0fcf60 2024-05-17T21:50:15.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:15.266+0000 7fec21b6e700 1 --2- 172.21.15.107:0/2256487915 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fec080692a0 0x7fec0806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:50:15.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:15.266+0000 7fec21b6e700 1 --2- 172.21.15.107:0/2256487915 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fec080692a0 0x7fec0806b750 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fec180041d0 tx=0x7fec18009040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:50:15.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:15.270+0000 7fec127fc700 1 -- 172.21.15.107:0/2256487915 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fec0c057d00 con 0x7fec1c0fcf60 2024-05-17T21:50:15.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:15.568+0000 7fec23dd2700 1 -- 172.21.15.107:0/2256487915 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fec1c05f5c0 con 0x7fec1c0fcf60 2024-05-17T21:50:15.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:15.568+0000 7fec127fc700 1 -- 172.21.15.107:0/2256487915 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+638 (secure 0 0 0) 0x7fec0c057890 con 0x7fec1c0fcf60 2024-05-17T21:50:15.560 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-05-17T21:50:15.560 INFO:teuthology.orchestra.run.smithi107.stdout: "mon": { 2024-05-17T21:50:15.560 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-952-g9903f776 (9903f7767386a9d9efadb7c1effdcd3064d71804) quincy (stable)": 2 2024-05-17T21:50:15.560 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-05-17T21:50:15.561 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr": { 2024-05-17T21:50:15.561 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-952-g9903f776 (9903f7767386a9d9efadb7c1effdcd3064d71804) quincy (stable)": 2 2024-05-17T21:50:15.561 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-05-17T21:50:15.561 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": { 2024-05-17T21:50:15.561 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-952-g9903f776 (9903f7767386a9d9efadb7c1effdcd3064d71804) quincy (stable)": 6 2024-05-17T21:50:15.561 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-05-17T21:50:15.561 INFO:teuthology.orchestra.run.smithi107.stdout: "mds": { 2024-05-17T21:50:15.561 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-952-g9903f776 (9903f7767386a9d9efadb7c1effdcd3064d71804) quincy (stable)": 4 2024-05-17T21:50:15.561 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-05-17T21:50:15.561 INFO:teuthology.orchestra.run.smithi107.stdout: "overall": { 2024-05-17T21:50:15.561 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-952-g9903f776 (9903f7767386a9d9efadb7c1effdcd3064d71804) quincy (stable)": 14 2024-05-17T21:50:15.561 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-05-17T21:50:15.561 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-05-17T21:50:15.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:15.570+0000 7fec23dd2700 1 -- 172.21.15.107:0/2256487915 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fec080692a0 msgr2=0x7fec0806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:50:15.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:15.570+0000 7fec23dd2700 1 --2- 172.21.15.107:0/2256487915 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fec080692a0 0x7fec0806b750 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fec180041d0 tx=0x7fec18009040 comp rx=0 tx=0).stop 2024-05-17T21:50:15.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:15.570+0000 7fec23dd2700 1 -- 172.21.15.107:0/2256487915 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec1c0fcf60 msgr2=0x7fec1c10e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:50:15.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:15.570+0000 7fec23dd2700 1 --2- 172.21.15.107:0/2256487915 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec1c0fcf60 0x7fec1c10e040 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fec0c00b3b0 tx=0x7fec0c009e30 comp rx=0 tx=0).stop 2024-05-17T21:50:15.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:15.570+0000 7fec23dd2700 1 -- 172.21.15.107:0/2256487915 shutdown_connections 2024-05-17T21:50:15.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:15.570+0000 7fec23dd2700 1 --2- 172.21.15.107:0/2256487915 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fec080692a0 0x7fec0806b750 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:15.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:15.570+0000 7fec23dd2700 1 --2- 172.21.15.107:0/2256487915 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec1c0fbcb0 0x7fec1c10db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:15.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:15.570+0000 7fec23dd2700 1 --2- 172.21.15.107:0/2256487915 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec1c0fcf60 0x7fec1c10e040 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:15.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:15.570+0000 7fec23dd2700 1 -- 172.21.15.107:0/2256487915 >> 172.21.15.107:0/2256487915 conn(0x7fec1c0f7430 msgr2=0x7fec1c065c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:50:15.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:15.571+0000 7fec23dd2700 1 -- 172.21.15.107:0/2256487915 shutdown_connections 2024-05-17T21:50:15.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:15.571+0000 7fec23dd2700 1 -- 172.21.15.107:0/2256487915 wait complete. 2024-05-17T21:50:15.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:15.676+0000 7fbd6c783700 1 -- 172.21.15.107:0/1342472938 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbd640fff50 msgr2=0x7fbd64102330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:50:15.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:15.676+0000 7fbd6c783700 1 --2- 172.21.15.107:0/1342472938 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbd640fff50 0x7fbd64102330 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fbd54004940 tx=0x7fbd54009e30 comp rx=0 tx=0).stop 2024-05-17T21:50:15.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:15.676+0000 7fbd6c783700 1 -- 172.21.15.107:0/1342472938 shutdown_connections 2024-05-17T21:50:15.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:15.676+0000 7fbd6c783700 1 --2- 172.21.15.107:0/1342472938 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbd640fff50 0x7fbd64102330 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:15.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:15.676+0000 7fbd6c783700 1 --2- 172.21.15.107:0/1342472938 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd64066af0 0x7fbd640ffa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:15.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:15.676+0000 7fbd6c783700 1 -- 172.21.15.107:0/1342472938 >> 172.21.15.107:0/1342472938 conn(0x7fbd640f7420 msgr2=0x7fbd640f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:50:15.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:15.676+0000 7fbd6c783700 1 -- 172.21.15.107:0/1342472938 shutdown_connections 2024-05-17T21:50:15.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:15.676+0000 7fbd6c783700 1 -- 172.21.15.107:0/1342472938 wait complete. 2024-05-17T21:50:15.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:15.676+0000 7fbd6c783700 1 Processor -- start 2024-05-17T21:50:15.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:15.676+0000 7fbd6c783700 1 -- start start 2024-05-17T21:50:15.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:15.677+0000 7fbd6c783700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd64066af0 0x7fbd6410da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:50:15.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:15.677+0000 7fbd6c783700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbd640fff50 0x7fbd6410df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:50:15.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:15.677+0000 7fbd6c783700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd6410e580 con 0x7fbd640fff50 2024-05-17T21:50:15.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:15.677+0000 7fbd6c783700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd6410edd0 con 0x7fbd64066af0 2024-05-17T21:50:15.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:15.677+0000 7fbd69d1e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbd640fff50 0x7fbd6410df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:50:15.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:15.677+0000 7fbd69d1e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbd640fff50 0x7fbd6410df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50658/0 (socket says 172.21.15.107:50658) 2024-05-17T21:50:15.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:15.677+0000 7fbd69d1e700 1 -- 172.21.15.107:0/112644970 learned_addr learned my addr 172.21.15.107:0/112644970 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:50:15.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:15.677+0000 7fbd69d1e700 1 -- 172.21.15.107:0/112644970 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd64066af0 msgr2=0x7fbd6410da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:50:15.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:15.677+0000 7fbd6a51f700 1 --2- 172.21.15.107:0/112644970 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd64066af0 0x7fbd6410da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:50:15.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:15.677+0000 7fbd69d1e700 1 --2- 172.21.15.107:0/112644970 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd64066af0 0x7fbd6410da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:15.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:15.677+0000 7fbd69d1e700 1 -- 172.21.15.107:0/112644970 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd54005040 con 0x7fbd640fff50 2024-05-17T21:50:15.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:15.677+0000 7fbd69d1e700 1 --2- 172.21.15.107:0/112644970 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbd640fff50 0x7fbd6410df80 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fbd54009150 tx=0x7fbd54009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:50:15.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:15.677+0000 7fbd5affd700 1 -- 172.21.15.107:0/112644970 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbd5400e040 con 0x7fbd640fff50 2024-05-17T21:50:15.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:15.677+0000 7fbd5affd700 1 -- 172.21.15.107:0/112644970 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbd5400bbf0 con 0x7fbd640fff50 2024-05-17T21:50:15.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:15.677+0000 7fbd6c783700 1 -- 172.21.15.107:0/112644970 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd6410f050 con 0x7fbd640fff50 2024-05-17T21:50:15.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:15.678+0000 7fbd5affd700 1 -- 172.21.15.107:0/112644970 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbd5401c630 con 0x7fbd640fff50 2024-05-17T21:50:15.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:15.678+0000 7fbd6c783700 1 -- 172.21.15.107:0/112644970 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd6410f590 con 0x7fbd640fff50 2024-05-17T21:50:15.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:15.678+0000 7fbd6c783700 1 -- 172.21.15.107:0/112644970 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd6405f5c0 con 0x7fbd640fff50 2024-05-17T21:50:15.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:15.679+0000 7fbd5affd700 1 -- 172.21.15.107:0/112644970 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7fbd5400bd60 con 0x7fbd640fff50 2024-05-17T21:50:15.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:15.680+0000 7fbd5affd700 1 --2- 172.21.15.107:0/112644970 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fbd500692b0 0x7fbd5006b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:50:15.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:15.680+0000 7fbd5affd700 1 -- 172.21.15.107:0/112644970 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fbd54088550 con 0x7fbd640fff50 2024-05-17T21:50:15.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:15.680+0000 7fbd6a51f700 1 --2- 172.21.15.107:0/112644970 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fbd500692b0 0x7fbd5006b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:50:15.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:15.680+0000 7fbd6a51f700 1 --2- 172.21.15.107:0/112644970 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fbd500692b0 0x7fbd5006b760 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fbd64066830 tx=0x7fbd6000b040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:50:15.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:15.684+0000 7fbd5affd700 1 -- 172.21.15.107:0/112644970 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fbd54058410 con 0x7fbd640fff50 2024-05-17T21:50:15.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:50:15 smithi138 ceph-mon[35948]: pgmap v205: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-05-17T21:50:15.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:50:15 smithi138 ceph-mon[35948]: from='client.14552 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:50:15.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:50:15 smithi138 ceph-mon[35948]: from='client.14556 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:50:15.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:50:15 smithi107 ceph-mon[28434]: pgmap v205: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-05-17T21:50:15.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:50:15 smithi107 ceph-mon[28434]: from='client.14552 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:50:15.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:50:15 smithi107 ceph-mon[28434]: from='client.14556 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:50:15.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:15.954+0000 7fbd6c783700 1 -- 172.21.15.107:0/112644970 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fbd6404bed0 con 0x7fbd640fff50 2024-05-17T21:50:15.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:15.954+0000 7fbd5affd700 1 -- 172.21.15.107:0/112644970 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1840 (secure 0 0 0) 0x7fbd54057fa0 con 0x7fbd640fff50 2024-05-17T21:50:15.946 INFO:teuthology.orchestra.run.smithi107.stdout:e15 2024-05-17T21:50:15.946 INFO:teuthology.orchestra.run.smithi107.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-17T21:50:15.947 INFO:teuthology.orchestra.run.smithi107.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-17T21:50:15.947 INFO:teuthology.orchestra.run.smithi107.stdout:legacy client fscid: 1 2024-05-17T21:50:15.947 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:50:15.947 INFO:teuthology.orchestra.run.smithi107.stdout:Filesystem 'cephfs' (1) 2024-05-17T21:50:15.947 INFO:teuthology.orchestra.run.smithi107.stdout:fs_name cephfs 2024-05-17T21:50:15.947 INFO:teuthology.orchestra.run.smithi107.stdout:epoch 15 2024-05-17T21:50:15.947 INFO:teuthology.orchestra.run.smithi107.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-17T21:50:15.947 INFO:teuthology.orchestra.run.smithi107.stdout:created 2024-05-17T21:49:16.836333+0000 2024-05-17T21:50:15.947 INFO:teuthology.orchestra.run.smithi107.stdout:modified 2024-05-17T21:49:34.873628+0000 2024-05-17T21:50:15.947 INFO:teuthology.orchestra.run.smithi107.stdout:tableserver 0 2024-05-17T21:50:15.947 INFO:teuthology.orchestra.run.smithi107.stdout:root 0 2024-05-17T21:50:15.947 INFO:teuthology.orchestra.run.smithi107.stdout:session_timeout 60 2024-05-17T21:50:15.947 INFO:teuthology.orchestra.run.smithi107.stdout:session_autoclose 300 2024-05-17T21:50:15.947 INFO:teuthology.orchestra.run.smithi107.stdout:max_file_size 1099511627776 2024-05-17T21:50:15.948 INFO:teuthology.orchestra.run.smithi107.stdout:required_client_features {} 2024-05-17T21:50:15.948 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure 0 2024-05-17T21:50:15.948 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure_osd_epoch 46 2024-05-17T21:50:15.948 INFO:teuthology.orchestra.run.smithi107.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-17T21:50:15.948 INFO:teuthology.orchestra.run.smithi107.stdout:max_mds 1 2024-05-17T21:50:15.948 INFO:teuthology.orchestra.run.smithi107.stdout:in 0 2024-05-17T21:50:15.948 INFO:teuthology.orchestra.run.smithi107.stdout:up {0=24259} 2024-05-17T21:50:15.948 INFO:teuthology.orchestra.run.smithi107.stdout:failed 2024-05-17T21:50:15.948 INFO:teuthology.orchestra.run.smithi107.stdout:damaged 2024-05-17T21:50:15.948 INFO:teuthology.orchestra.run.smithi107.stdout:stopped 2024-05-17T21:50:15.948 INFO:teuthology.orchestra.run.smithi107.stdout:data_pools [3] 2024-05-17T21:50:15.948 INFO:teuthology.orchestra.run.smithi107.stdout:metadata_pool 2 2024-05-17T21:50:15.948 INFO:teuthology.orchestra.run.smithi107.stdout:inline_data enabled 2024-05-17T21:50:15.949 INFO:teuthology.orchestra.run.smithi107.stdout:balancer 2024-05-17T21:50:15.949 INFO:teuthology.orchestra.run.smithi107.stdout:standby_count_wanted 1 2024-05-17T21:50:15.949 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi138.gwbtyd{0:24259} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.138:6824/2779634919,v1:172.21.15.138:6825/2779634919] compat {c=[1],r=[1],i=[7ff]}] 2024-05-17T21:50:15.949 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.drqody{0:14486} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.107:6828/2275807100,v1:172.21.15.107:6829/2275807100] compat {c=[1],r=[1],i=[7ff]}] 2024-05-17T21:50:15.949 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:50:15.949 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:50:15.949 INFO:teuthology.orchestra.run.smithi107.stdout:Standby daemons: 2024-05-17T21:50:15.949 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:50:15.949 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.idmbgj{-1:14498} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.107:6826/1460186079,v1:172.21.15.107:6827/1460186079] compat {c=[1],r=[1],i=[7ff]}] 2024-05-17T21:50:15.949 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi138.zckrvs{-1:24267} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.138:6826/170628661,v1:172.21.15.138:6827/170628661] compat {c=[1],r=[1],i=[7ff]}] 2024-05-17T21:50:15.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:15.956+0000 7fbd6c783700 1 -- 172.21.15.107:0/112644970 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fbd500692b0 msgr2=0x7fbd5006b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:50:15.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:15.956+0000 7fbd6c783700 1 --2- 172.21.15.107:0/112644970 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fbd500692b0 0x7fbd5006b760 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fbd64066830 tx=0x7fbd6000b040 comp rx=0 tx=0).stop 2024-05-17T21:50:15.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:15.956+0000 7fbd6c783700 1 -- 172.21.15.107:0/112644970 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbd640fff50 msgr2=0x7fbd6410df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:50:15.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:15.956+0000 7fbd6c783700 1 --2- 172.21.15.107:0/112644970 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbd640fff50 0x7fbd6410df80 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fbd54009150 tx=0x7fbd54009e30 comp rx=0 tx=0).stop 2024-05-17T21:50:15.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:15.956+0000 7fbd6c783700 1 -- 172.21.15.107:0/112644970 shutdown_connections 2024-05-17T21:50:15.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:15.956+0000 7fbd6c783700 1 --2- 172.21.15.107:0/112644970 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fbd500692b0 0x7fbd5006b760 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:15.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:15.956+0000 7fbd6c783700 1 --2- 172.21.15.107:0/112644970 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd64066af0 0x7fbd6410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:15.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:15.956+0000 7fbd6c783700 1 --2- 172.21.15.107:0/112644970 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbd640fff50 0x7fbd6410df80 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:15.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:15.956+0000 7fbd6c783700 1 -- 172.21.15.107:0/112644970 >> 172.21.15.107:0/112644970 conn(0x7fbd640f7420 msgr2=0x7fbd640f7840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:50:15.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:15.956+0000 7fbd6c783700 1 -- 172.21.15.107:0/112644970 shutdown_connections 2024-05-17T21:50:15.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:15.956+0000 7fbd6c783700 1 -- 172.21.15.107:0/112644970 wait complete. 2024-05-17T21:50:15.951 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 15 2024-05-17T21:50:16.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:16.064+0000 7f4d1c27d700 1 -- 172.21.15.107:0/2860570588 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d140fff00 msgr2=0x7f4d14102300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:50:16.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:16.064+0000 7f4d1c27d700 1 --2- 172.21.15.107:0/2860570588 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d140fff00 0x7f4d14102300 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f4d04004940 tx=0x7f4d04009e30 comp rx=0 tx=0).stop 2024-05-17T21:50:16.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:16.064+0000 7f4d1c27d700 1 -- 172.21.15.107:0/2860570588 shutdown_connections 2024-05-17T21:50:16.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:16.064+0000 7f4d1c27d700 1 --2- 172.21.15.107:0/2860570588 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d140fff00 0x7f4d14102300 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:16.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:16.064+0000 7f4d1c27d700 1 --2- 172.21.15.107:0/2860570588 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d140662a0 0x7f4d140ff9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:16.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:16.064+0000 7f4d1c27d700 1 -- 172.21.15.107:0/2860570588 >> 172.21.15.107:0/2860570588 conn(0x7f4d140f7430 msgr2=0x7f4d140f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:50:16.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:16.064+0000 7f4d1c27d700 1 -- 172.21.15.107:0/2860570588 shutdown_connections 2024-05-17T21:50:16.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:16.064+0000 7f4d1c27d700 1 -- 172.21.15.107:0/2860570588 wait complete. 2024-05-17T21:50:16.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:16.065+0000 7f4d1c27d700 1 Processor -- start 2024-05-17T21:50:16.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:16.065+0000 7f4d1c27d700 1 -- start start 2024-05-17T21:50:16.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:16.065+0000 7f4d1c27d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d140662a0 0x7f4d1410da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:50:16.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:16.065+0000 7f4d1c27d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d140fff00 0x7f4d1410df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:50:16.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:16.065+0000 7f4d1c27d700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1410e550 con 0x7f4d140662a0 2024-05-17T21:50:16.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:16.065+0000 7f4d1c27d700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1410eda0 con 0x7f4d140fff00 2024-05-17T21:50:16.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:16.065+0000 7f4d1a019700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d140662a0 0x7f4d1410da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:50:16.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:16.065+0000 7f4d1a019700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d140662a0 0x7f4d1410da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50680/0 (socket says 172.21.15.107:50680) 2024-05-17T21:50:16.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:16.065+0000 7f4d1a019700 1 -- 172.21.15.107:0/284591380 learned_addr learned my addr 172.21.15.107:0/284591380 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:50:16.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:16.066+0000 7f4d19818700 1 --2- 172.21.15.107:0/284591380 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d140fff00 0x7f4d1410df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:50:16.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:16.066+0000 7f4d1a019700 1 -- 172.21.15.107:0/284591380 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d140fff00 msgr2=0x7f4d1410df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:50:16.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:16.066+0000 7f4d1a019700 1 --2- 172.21.15.107:0/284591380 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d140fff00 0x7f4d1410df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:16.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:16.066+0000 7f4d1a019700 1 -- 172.21.15.107:0/284591380 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d04005040 con 0x7f4d140662a0 2024-05-17T21:50:16.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:16.066+0000 7f4d1a019700 1 --2- 172.21.15.107:0/284591380 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d140662a0 0x7f4d1410da10 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f4d10002a80 tx=0x7f4d10004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:50:16.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:16.066+0000 7f4d0affd700 1 -- 172.21.15.107:0/284591380 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4d10007cc0 con 0x7f4d140662a0 2024-05-17T21:50:16.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:16.066+0000 7f4d0affd700 1 -- 172.21.15.107:0/284591380 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4d1000f040 con 0x7f4d140662a0 2024-05-17T21:50:16.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:16.066+0000 7f4d1c27d700 1 -- 172.21.15.107:0/284591380 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d1410f080 con 0x7f4d140662a0 2024-05-17T21:50:16.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:16.066+0000 7f4d0affd700 1 -- 172.21.15.107:0/284591380 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4d1000da30 con 0x7f4d140662a0 2024-05-17T21:50:16.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:16.066+0000 7f4d1c27d700 1 -- 172.21.15.107:0/284591380 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d1410f5c0 con 0x7f4d140662a0 2024-05-17T21:50:16.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:16.067+0000 7f4d1c27d700 1 -- 172.21.15.107:0/284591380 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d1405f5c0 con 0x7f4d140662a0 2024-05-17T21:50:16.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:16.068+0000 7f4d0affd700 1 -- 172.21.15.107:0/284591380 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f4d1000db90 con 0x7f4d140662a0 2024-05-17T21:50:16.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:16.068+0000 7f4d0affd700 1 --2- 172.21.15.107:0/284591380 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f4d000692a0 0x7f4d0006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:50:16.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:16.068+0000 7f4d0affd700 1 -- 172.21.15.107:0/284591380 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f4d10086f70 con 0x7f4d140662a0 2024-05-17T21:50:16.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:16.068+0000 7f4d19818700 1 --2- 172.21.15.107:0/284591380 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f4d000692a0 0x7f4d0006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:50:16.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:16.069+0000 7f4d19818700 1 --2- 172.21.15.107:0/284591380 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f4d000692a0 0x7f4d0006b750 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4d0400b970 tx=0x7f4d04009e30 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:50:16.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:16.073+0000 7f4d0affd700 1 -- 172.21.15.107:0/284591380 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f4d10056c60 con 0x7f4d140662a0 2024-05-17T21:50:16.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:16.297+0000 7f4d1c27d700 1 -- 172.21.15.107:0/284591380 --> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4d1410f920 con 0x7f4d000692a0 2024-05-17T21:50:16.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:16.298+0000 7f4d0affd700 1 -- 172.21.15.107:0/284591380 <== mgr.14214 v2:172.21.15.107:6800/1294361014 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7f4d1410f920 con 0x7f4d000692a0 2024-05-17T21:50:16.290 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-05-17T21:50:16.290 INFO:teuthology.orchestra.run.smithi107.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc", 2024-05-17T21:50:16.290 INFO:teuthology.orchestra.run.smithi107.stdout: "in_progress": true, 2024-05-17T21:50:16.290 INFO:teuthology.orchestra.run.smithi107.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-17T21:50:16.290 INFO:teuthology.orchestra.run.smithi107.stdout: "services_complete": [], 2024-05-17T21:50:16.290 INFO:teuthology.orchestra.run.smithi107.stdout: "progress": "", 2024-05-17T21:50:16.290 INFO:teuthology.orchestra.run.smithi107.stdout: "message": "Doing first pull of quay.ceph.io/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc image", 2024-05-17T21:50:16.291 INFO:teuthology.orchestra.run.smithi107.stdout: "is_paused": false 2024-05-17T21:50:16.291 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-05-17T21:50:16.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:16.300+0000 7f4d1c27d700 1 -- 172.21.15.107:0/284591380 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f4d000692a0 msgr2=0x7f4d0006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:50:16.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:16.300+0000 7f4d1c27d700 1 --2- 172.21.15.107:0/284591380 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f4d000692a0 0x7f4d0006b750 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4d0400b970 tx=0x7f4d04009e30 comp rx=0 tx=0).stop 2024-05-17T21:50:16.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:16.300+0000 7f4d1c27d700 1 -- 172.21.15.107:0/284591380 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d140662a0 msgr2=0x7f4d1410da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:50:16.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:16.300+0000 7f4d1c27d700 1 --2- 172.21.15.107:0/284591380 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d140662a0 0x7f4d1410da10 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f4d10002a80 tx=0x7f4d10004600 comp rx=0 tx=0).stop 2024-05-17T21:50:16.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:16.301+0000 7f4d1c27d700 1 -- 172.21.15.107:0/284591380 shutdown_connections 2024-05-17T21:50:16.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:16.301+0000 7f4d1c27d700 1 --2- 172.21.15.107:0/284591380 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f4d000692a0 0x7f4d0006b750 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:16.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:16.301+0000 7f4d1c27d700 1 --2- 172.21.15.107:0/284591380 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d140662a0 0x7f4d1410da10 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:16.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:16.301+0000 7f4d1c27d700 1 --2- 172.21.15.107:0/284591380 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d140fff00 0x7f4d1410df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:16.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:16.301+0000 7f4d1c27d700 1 -- 172.21.15.107:0/284591380 >> 172.21.15.107:0/284591380 conn(0x7f4d140f7430 msgr2=0x7f4d140f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:50:16.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:16.301+0000 7f4d1c27d700 1 -- 172.21.15.107:0/284591380 shutdown_connections 2024-05-17T21:50:16.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:16.301+0000 7f4d1c27d700 1 -- 172.21.15.107:0/284591380 wait complete. 2024-05-17T21:50:16.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:16.414+0000 7f7551764700 1 -- 172.21.15.107:0/103801841 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f754c0fcf60 msgr2=0x7f754c0fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:50:16.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:16.414+0000 7f7551764700 1 --2- 172.21.15.107:0/103801841 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f754c0fcf60 0x7f754c0fd3d0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f7540004940 tx=0x7f7540009e30 comp rx=0 tx=0).stop 2024-05-17T21:50:16.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:16.414+0000 7f7551764700 1 -- 172.21.15.107:0/103801841 shutdown_connections 2024-05-17T21:50:16.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:16.414+0000 7f7551764700 1 --2- 172.21.15.107:0/103801841 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f754c0fcf60 0x7f754c0fd3d0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:16.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:16.414+0000 7f7551764700 1 --2- 172.21.15.107:0/103801841 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f754c0fbcb0 0x7f754c0fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:16.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:16.414+0000 7f7551764700 1 -- 172.21.15.107:0/103801841 >> 172.21.15.107:0/103801841 conn(0x7f754c0f7430 msgr2=0x7f754c0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:50:16.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:16.414+0000 7f7551764700 1 -- 172.21.15.107:0/103801841 shutdown_connections 2024-05-17T21:50:16.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:16.414+0000 7f7551764700 1 -- 172.21.15.107:0/103801841 wait complete. 2024-05-17T21:50:16.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:16.415+0000 7f7551764700 1 Processor -- start 2024-05-17T21:50:16.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:16.415+0000 7f7551764700 1 -- start start 2024-05-17T21:50:16.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:16.415+0000 7f7551764700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f754c0fbcb0 0x7f754c10db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:50:16.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:16.415+0000 7f7551764700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f754c0fcf60 0x7f754c10e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:50:16.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:16.415+0000 7f7551764700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f754c10e640 con 0x7f754c0fcf60 2024-05-17T21:50:16.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:16.415+0000 7f7551764700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f754c10edf0 con 0x7f754c0fbcb0 2024-05-17T21:50:16.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:16.415+0000 7f754a7fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f754c0fcf60 0x7f754c10e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:50:16.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:16.415+0000 7f754a7fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f754c0fcf60 0x7f754c10e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50708/0 (socket says 172.21.15.107:50708) 2024-05-17T21:50:16.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:16.415+0000 7f754a7fc700 1 -- 172.21.15.107:0/3774497344 learned_addr learned my addr 172.21.15.107:0/3774497344 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:50:16.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:16.415+0000 7f754affd700 1 --2- 172.21.15.107:0/3774497344 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f754c0fbcb0 0x7f754c10db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:50:16.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:16.415+0000 7f754a7fc700 1 -- 172.21.15.107:0/3774497344 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f754c0fbcb0 msgr2=0x7f754c10db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:50:16.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:16.415+0000 7f754a7fc700 1 --2- 172.21.15.107:0/3774497344 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f754c0fbcb0 0x7f754c10db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:16.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:16.415+0000 7f754a7fc700 1 -- 172.21.15.107:0/3774497344 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7540005040 con 0x7f754c0fcf60 2024-05-17T21:50:16.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:16.416+0000 7f754a7fc700 1 --2- 172.21.15.107:0/3774497344 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f754c0fcf60 0x7f754c10e040 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f754000b3b0 tx=0x7f7540009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:50:16.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:16.416+0000 7f75337fe700 1 -- 172.21.15.107:0/3774497344 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f754000e040 con 0x7f754c0fcf60 2024-05-17T21:50:16.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:16.416+0000 7f75337fe700 1 -- 172.21.15.107:0/3774497344 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f754000bcc0 con 0x7f754c0fcf60 2024-05-17T21:50:16.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:16.416+0000 7f7551764700 1 -- 172.21.15.107:0/3774497344 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f754c10f070 con 0x7f754c0fcf60 2024-05-17T21:50:16.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:16.416+0000 7f75337fe700 1 -- 172.21.15.107:0/3774497344 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f754001c630 con 0x7f754c0fcf60 2024-05-17T21:50:16.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:16.416+0000 7f7551764700 1 -- 172.21.15.107:0/3774497344 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f754c10f5b0 con 0x7f754c0fcf60 2024-05-17T21:50:16.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:16.417+0000 7f7551764700 1 -- 172.21.15.107:0/3774497344 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f754c101af0 con 0x7f754c0fcf60 2024-05-17T21:50:16.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:16.417+0000 7f75337fe700 1 -- 172.21.15.107:0/3774497344 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f7540013980 con 0x7f754c0fcf60 2024-05-17T21:50:16.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:16.418+0000 7f75337fe700 1 --2- 172.21.15.107:0/3774497344 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f75340692a0 0x7f753406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:50:16.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:16.418+0000 7f75337fe700 1 -- 172.21.15.107:0/3774497344 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f7540087fa0 con 0x7f754c0fcf60 2024-05-17T21:50:16.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:16.418+0000 7f754affd700 1 --2- 172.21.15.107:0/3774497344 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f75340692a0 0x7f753406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:50:16.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:16.418+0000 7f754affd700 1 --2- 172.21.15.107:0/3774497344 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f75340692a0 0x7f753406b750 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f753c007fd0 tx=0x7f753c009040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:50:16.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:16.422+0000 7f75337fe700 1 -- 172.21.15.107:0/3774497344 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f75400ba250 con 0x7f754c0fcf60 2024-05-17T21:50:16.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:50:16 smithi107 ceph-mon[28434]: from='client.14560 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:50:16.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:50:16 smithi107 ceph-mon[28434]: from='client.? 172.21.15.107:0/2256487915' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:50:16.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:50:16 smithi107 ceph-mon[28434]: from='client.? 172.21.15.107:0/112644970' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-17T21:50:16.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:16.714+0000 7f7551764700 1 -- 172.21.15.107:0/3774497344 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f754c05f5c0 con 0x7f754c0fcf60 2024-05-17T21:50:16.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:16.715+0000 7f75337fe700 1 -- 172.21.15.107:0/3774497344 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f7540057cb0 con 0x7f754c0fcf60 2024-05-17T21:50:16.708 INFO:teuthology.orchestra.run.smithi107.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-17T21:50:16.708 INFO:teuthology.orchestra.run.smithi107.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-17T21:50:16.708 INFO:teuthology.orchestra.run.smithi107.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-17T21:50:16.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:16.717+0000 7f7551764700 1 -- 172.21.15.107:0/3774497344 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f75340692a0 msgr2=0x7f753406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:50:16.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:16.717+0000 7f7551764700 1 --2- 172.21.15.107:0/3774497344 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f75340692a0 0x7f753406b750 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f753c007fd0 tx=0x7f753c009040 comp rx=0 tx=0).stop 2024-05-17T21:50:16.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:16.717+0000 7f7551764700 1 -- 172.21.15.107:0/3774497344 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f754c0fcf60 msgr2=0x7f754c10e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:50:16.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:16.717+0000 7f7551764700 1 --2- 172.21.15.107:0/3774497344 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f754c0fcf60 0x7f754c10e040 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f754000b3b0 tx=0x7f7540009e30 comp rx=0 tx=0).stop 2024-05-17T21:50:16.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:16.717+0000 7f7551764700 1 -- 172.21.15.107:0/3774497344 shutdown_connections 2024-05-17T21:50:16.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:16.717+0000 7f7551764700 1 --2- 172.21.15.107:0/3774497344 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f75340692a0 0x7f753406b750 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:16.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:16.717+0000 7f7551764700 1 --2- 172.21.15.107:0/3774497344 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f754c0fbcb0 0x7f754c10db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:16.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:16.717+0000 7f7551764700 1 --2- 172.21.15.107:0/3774497344 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f754c0fcf60 0x7f754c10e040 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:16.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:16.717+0000 7f7551764700 1 -- 172.21.15.107:0/3774497344 >> 172.21.15.107:0/3774497344 conn(0x7f754c0f7430 msgr2=0x7f754c065c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:50:16.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:16.717+0000 7f7551764700 1 -- 172.21.15.107:0/3774497344 shutdown_connections 2024-05-17T21:50:16.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:16.717+0000 7f7551764700 1 -- 172.21.15.107:0/3774497344 wait complete. 2024-05-17T21:50:16.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:50:16 smithi138 ceph-mon[35948]: from='client.14560 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:50:16.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:50:16 smithi138 ceph-mon[35948]: from='client.? 172.21.15.107:0/2256487915' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:50:16.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:50:16 smithi138 ceph-mon[35948]: from='client.? 172.21.15.107:0/112644970' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-17T21:50:17.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:50:17 smithi138 ceph-mon[35948]: from='client.14572 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:50:17.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:50:17 smithi138 ceph-mon[35948]: pgmap v206: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 597 B/s wr, 2 op/s 2024-05-17T21:50:17.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:50:17 smithi138 ceph-mon[35948]: from='client.? 172.21.15.107:0/3774497344' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-17T21:50:17.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:50:17 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:50:17.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:50:17 smithi107 ceph-mon[28434]: from='client.14572 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:50:17.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:50:17 smithi107 ceph-mon[28434]: pgmap v206: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 597 B/s wr, 2 op/s 2024-05-17T21:50:17.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:50:17 smithi107 ceph-mon[28434]: from='client.? 172.21.15.107:0/3774497344' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-17T21:50:17.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:50:17 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:50:19.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:50:19 smithi138 ceph-mon[35948]: pgmap v207: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 597 B/s wr, 3 op/s 2024-05-17T21:50:19.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:50:19 smithi107 ceph-mon[28434]: pgmap v207: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 597 B/s wr, 3 op/s 2024-05-17T21:50:21.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:50:20 smithi138 ceph-mon[35948]: pgmap v208: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 426 B/s wr, 2 op/s 2024-05-17T21:50:21.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:50:20 smithi107 ceph-mon[28434]: pgmap v208: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 426 B/s wr, 2 op/s 2024-05-17T21:50:23.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:50:23 smithi138 ceph-mon[35948]: pgmap v209: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 341 B/s wr, 2 op/s 2024-05-17T21:50:23.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:50:23 smithi107 ceph-mon[28434]: pgmap v209: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 341 B/s wr, 2 op/s 2024-05-17T21:50:25.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:50:25 smithi138 ceph-mon[35948]: pgmap v210: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.4 KiB/s rd, 341 B/s wr, 2 op/s 2024-05-17T21:50:25.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:50:25 smithi107 ceph-mon[28434]: pgmap v210: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.4 KiB/s rd, 341 B/s wr, 2 op/s 2024-05-17T21:50:27.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:50:27 smithi107 ceph-mon[28434]: pgmap v211: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T21:50:27.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:50:27 smithi138 ceph-mon[35948]: pgmap v211: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T21:50:29.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:50:29 smithi138 ceph-mon[35948]: pgmap v212: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T21:50:29.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:50:29 smithi107 ceph-mon[28434]: pgmap v212: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T21:50:31.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:50:31 smithi138 ceph-mon[35948]: pgmap v213: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T21:50:31.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:50:31 smithi107 ceph-mon[28434]: pgmap v213: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T21:50:32.728 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:50:32 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:50:32.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:50:32 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:50:33.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:50:33 smithi138 ceph-mon[35948]: pgmap v214: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T21:50:33.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:50:33 smithi107 ceph-mon[28434]: pgmap v214: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T21:50:35.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:50:35 smithi138 ceph-mon[35948]: pgmap v215: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T21:50:35.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:50:35 smithi107 ceph-mon[28434]: pgmap v215: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T21:50:37.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:50:37 smithi107 ceph-mon[28434]: pgmap v216: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T21:50:37.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:50:37 smithi138 ceph-mon[35948]: pgmap v216: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T21:50:39.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:50:39 smithi138 ceph-mon[35948]: pgmap v217: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T21:50:39.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:50:39 smithi107 ceph-mon[28434]: pgmap v217: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T21:50:41.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:50:41 smithi138 ceph-mon[35948]: pgmap v218: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T21:50:41.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:50:41 smithi107 ceph-mon[28434]: pgmap v218: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T21:50:43.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:50:43 smithi138 ceph-mon[35948]: pgmap v219: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T21:50:43.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:50:43 smithi107 ceph-mon[28434]: pgmap v219: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T21:50:45.701 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:50:45 smithi107 ceph-mon[28434]: pgmap v220: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T21:50:45.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:50:45 smithi138 ceph-mon[35948]: pgmap v220: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T21:50:46.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:46.824+0000 7f8919b78700 1 -- 172.21.15.107:0/2759906302 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89140fbcc0 msgr2=0x7f89140fc0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:50:46.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:46.824+0000 7f8919b78700 1 --2- 172.21.15.107:0/2759906302 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89140fbcc0 0x7f89140fc0d0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f8904004940 tx=0x7f8904009e30 comp rx=0 tx=0).stop 2024-05-17T21:50:46.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:46.824+0000 7f8919b78700 1 -- 172.21.15.107:0/2759906302 shutdown_connections 2024-05-17T21:50:46.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:46.825+0000 7f8919b78700 1 --2- 172.21.15.107:0/2759906302 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89140fcf70 0x7f89140fd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:46.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:46.825+0000 7f8919b78700 1 --2- 172.21.15.107:0/2759906302 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89140fbcc0 0x7f89140fc0d0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:46.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:46.825+0000 7f8919b78700 1 -- 172.21.15.107:0/2759906302 >> 172.21.15.107:0/2759906302 conn(0x7f89140f7440 msgr2=0x7f89140f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:50:46.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:46.825+0000 7f8919b78700 1 -- 172.21.15.107:0/2759906302 shutdown_connections 2024-05-17T21:50:46.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:46.825+0000 7f8919b78700 1 -- 172.21.15.107:0/2759906302 wait complete. 2024-05-17T21:50:46.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:46.825+0000 7f8919b78700 1 Processor -- start 2024-05-17T21:50:46.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:46.825+0000 7f8919b78700 1 -- start start 2024-05-17T21:50:46.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:46.825+0000 7f8919b78700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89140fbcc0 0x7f891410c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:50:46.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:46.825+0000 7f8919b78700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89140fcf70 0x7f891410cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:50:46.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:46.825+0000 7f8919b78700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f891410d350 con 0x7f89140fbcc0 2024-05-17T21:50:46.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:46.825+0000 7f8919b78700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f891410d4c0 con 0x7f89140fcf70 2024-05-17T21:50:46.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:46.826+0000 7f89137fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89140fbcc0 0x7f891410c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:50:46.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:46.826+0000 7f89137fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89140fbcc0 0x7f891410c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55750/0 (socket says 172.21.15.107:55750) 2024-05-17T21:50:46.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:46.826+0000 7f89137fe700 1 -- 172.21.15.107:0/933455934 learned_addr learned my addr 172.21.15.107:0/933455934 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:50:46.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:46.826+0000 7f8912ffd700 1 --2- 172.21.15.107:0/933455934 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89140fcf70 0x7f891410cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:50:46.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:46.826+0000 7f89137fe700 1 -- 172.21.15.107:0/933455934 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89140fcf70 msgr2=0x7f891410cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:50:46.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:46.826+0000 7f89137fe700 1 --2- 172.21.15.107:0/933455934 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89140fcf70 0x7f891410cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:46.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:46.826+0000 7f89137fe700 1 -- 172.21.15.107:0/933455934 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8904005040 con 0x7f89140fbcc0 2024-05-17T21:50:46.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:46.826+0000 7f8912ffd700 1 --2- 172.21.15.107:0/933455934 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89140fcf70 0x7f891410cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T21:50:46.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:46.826+0000 7f89137fe700 1 --2- 172.21.15.107:0/933455934 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89140fbcc0 0x7f891410c810 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f8904013870 tx=0x7f8904009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:50:46.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:46.826+0000 7f88fbfff700 1 -- 172.21.15.107:0/933455934 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f890400e070 con 0x7f89140fbcc0 2024-05-17T21:50:46.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:46.826+0000 7f88fbfff700 1 -- 172.21.15.107:0/933455934 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f890401c400 con 0x7f89140fbcc0 2024-05-17T21:50:46.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:46.826+0000 7f8919b78700 1 -- 172.21.15.107:0/933455934 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8914111ec0 con 0x7f89140fbcc0 2024-05-17T21:50:46.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:46.826+0000 7f88fbfff700 1 -- 172.21.15.107:0/933455934 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f890400b650 con 0x7f89140fbcc0 2024-05-17T21:50:46.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:46.826+0000 7f8919b78700 1 -- 172.21.15.107:0/933455934 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8914112320 con 0x7f89140fbcc0 2024-05-17T21:50:46.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:46.827+0000 7f8919b78700 1 -- 172.21.15.107:0/933455934 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f891404c710 con 0x7f89140fbcc0 2024-05-17T21:50:46.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:46.828+0000 7f88fbfff700 1 -- 172.21.15.107:0/933455934 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f890401ca00 con 0x7f89140fbcc0 2024-05-17T21:50:46.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:46.828+0000 7f88fbfff700 1 --2- 172.21.15.107:0/933455934 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f88fc0692a0 0x7f88fc06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:50:46.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:46.828+0000 7f88fbfff700 1 -- 172.21.15.107:0/933455934 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f8904088910 con 0x7f89140fbcc0 2024-05-17T21:50:46.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:46.828+0000 7f8912ffd700 1 --2- 172.21.15.107:0/933455934 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f88fc0692a0 0x7f88fc06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:50:46.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:46.829+0000 7f8912ffd700 1 --2- 172.21.15.107:0/933455934 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f88fc0692a0 0x7f88fc06b750 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8908004600 tx=0x7f890800b040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:50:46.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:46.832+0000 7f88fbfff700 1 -- 172.21.15.107:0/933455934 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f8904058680 con 0x7f89140fbcc0 2024-05-17T21:50:47.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.033+0000 7f8919b78700 1 -- 172.21.15.107:0/933455934 --> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8914101280 con 0x7f88fc0692a0 2024-05-17T21:50:47.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.034+0000 7f88fbfff700 1 -- 172.21.15.107:0/933455934 <== mgr.14214 v2:172.21.15.107:6800/1294361014 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+349 (secure 0 0 0) 0x7f8914101280 con 0x7f88fc0692a0 2024-05-17T21:50:47.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.037+0000 7f8919b78700 1 -- 172.21.15.107:0/933455934 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f88fc0692a0 msgr2=0x7f88fc06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:50:47.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.037+0000 7f8919b78700 1 --2- 172.21.15.107:0/933455934 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f88fc0692a0 0x7f88fc06b750 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8908004600 tx=0x7f890800b040 comp rx=0 tx=0).stop 2024-05-17T21:50:47.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.037+0000 7f8919b78700 1 -- 172.21.15.107:0/933455934 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89140fbcc0 msgr2=0x7f891410c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:50:47.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.037+0000 7f8919b78700 1 --2- 172.21.15.107:0/933455934 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89140fbcc0 0x7f891410c810 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f8904013870 tx=0x7f8904009e30 comp rx=0 tx=0).stop 2024-05-17T21:50:47.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.037+0000 7f8919b78700 1 -- 172.21.15.107:0/933455934 shutdown_connections 2024-05-17T21:50:47.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.037+0000 7f8919b78700 1 --2- 172.21.15.107:0/933455934 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f88fc0692a0 0x7f88fc06b750 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:47.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.037+0000 7f8919b78700 1 --2- 172.21.15.107:0/933455934 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89140fbcc0 0x7f891410c810 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:47.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.037+0000 7f8919b78700 1 --2- 172.21.15.107:0/933455934 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89140fcf70 0x7f891410cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:47.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.037+0000 7f8919b78700 1 -- 172.21.15.107:0/933455934 >> 172.21.15.107:0/933455934 conn(0x7f89140f7440 msgr2=0x7f89140f8a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:50:47.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.038+0000 7f8919b78700 1 -- 172.21.15.107:0/933455934 shutdown_connections 2024-05-17T21:50:47.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.038+0000 7f8919b78700 1 -- 172.21.15.107:0/933455934 wait complete. 2024-05-17T21:50:47.041 INFO:teuthology.orchestra.run.smithi107.stdout:true 2024-05-17T21:50:47.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.142+0000 7f6815bc1700 1 -- 172.21.15.107:0/4164609943 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68100fb1f0 msgr2=0x7f68100fb600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:50:47.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.142+0000 7f6815bc1700 1 --2- 172.21.15.107:0/4164609943 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68100fb1f0 0x7f68100fb600 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f6800004940 tx=0x7f6800009e30 comp rx=0 tx=0).stop 2024-05-17T21:50:47.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.142+0000 7f6815bc1700 1 -- 172.21.15.107:0/4164609943 shutdown_connections 2024-05-17T21:50:47.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.142+0000 7f6815bc1700 1 --2- 172.21.15.107:0/4164609943 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68100fbb40 0x7f68100fe3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:47.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.142+0000 7f6815bc1700 1 --2- 172.21.15.107:0/4164609943 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68100fb1f0 0x7f68100fb600 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:47.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.142+0000 7f6815bc1700 1 -- 172.21.15.107:0/4164609943 >> 172.21.15.107:0/4164609943 conn(0x7f68100f6bd0 msgr2=0x7f68100f9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:50:47.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.142+0000 7f6815bc1700 1 -- 172.21.15.107:0/4164609943 shutdown_connections 2024-05-17T21:50:47.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.142+0000 7f6815bc1700 1 -- 172.21.15.107:0/4164609943 wait complete. 2024-05-17T21:50:47.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.142+0000 7f6815bc1700 1 Processor -- start 2024-05-17T21:50:47.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.142+0000 7f6815bc1700 1 -- start start 2024-05-17T21:50:47.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.143+0000 7f6815bc1700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68100fb1f0 0x7f681010fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:50:47.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.143+0000 7f6815bc1700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68100fbb40 0x7f68101102c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:50:47.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.143+0000 7f6815bc1700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68101108c0 con 0x7f68100fbb40 2024-05-17T21:50:47.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.143+0000 7f6815bc1700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6810110a30 con 0x7f68100fb1f0 2024-05-17T21:50:47.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.143+0000 7f680effd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68100fbb40 0x7f68101102c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:50:47.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.143+0000 7f680effd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68100fbb40 0x7f68101102c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55778/0 (socket says 172.21.15.107:55778) 2024-05-17T21:50:47.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.143+0000 7f680effd700 1 -- 172.21.15.107:0/1615576400 learned_addr learned my addr 172.21.15.107:0/1615576400 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:50:47.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.143+0000 7f680f7fe700 1 --2- 172.21.15.107:0/1615576400 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68100fb1f0 0x7f681010fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:50:47.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.143+0000 7f680effd700 1 -- 172.21.15.107:0/1615576400 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68100fb1f0 msgr2=0x7f681010fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:50:47.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.143+0000 7f680effd700 1 --2- 172.21.15.107:0/1615576400 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68100fb1f0 0x7f681010fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:47.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.143+0000 7f680effd700 1 -- 172.21.15.107:0/1615576400 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6800005040 con 0x7f68100fbb40 2024-05-17T21:50:47.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.143+0000 7f680effd700 1 --2- 172.21.15.107:0/1615576400 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68100fbb40 0x7f68101102c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f6804002a50 tx=0x7f68040046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:50:47.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.143+0000 7f67f7fff700 1 -- 172.21.15.107:0/1615576400 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6804007cc0 con 0x7f68100fbb40 2024-05-17T21:50:47.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.143+0000 7f67f7fff700 1 -- 172.21.15.107:0/1615576400 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f680400f040 con 0x7f68100fbb40 2024-05-17T21:50:47.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.143+0000 7f6815bc1700 1 -- 172.21.15.107:0/1615576400 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6810115490 con 0x7f68100fbb40 2024-05-17T21:50:47.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.143+0000 7f67f7fff700 1 -- 172.21.15.107:0/1615576400 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f680400da70 con 0x7f68100fbb40 2024-05-17T21:50:47.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.143+0000 7f6815bc1700 1 -- 172.21.15.107:0/1615576400 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6810115a50 con 0x7f68100fbb40 2024-05-17T21:50:47.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.144+0000 7f6815bc1700 1 -- 172.21.15.107:0/1615576400 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6810101d30 con 0x7f68100fbb40 2024-05-17T21:50:47.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.145+0000 7f67f7fff700 1 -- 172.21.15.107:0/1615576400 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f680400dc10 con 0x7f68100fbb40 2024-05-17T21:50:47.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.145+0000 7f67f7fff700 1 --2- 172.21.15.107:0/1615576400 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f67f80692a0 0x7f67f806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:50:47.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.145+0000 7f67f7fff700 1 -- 172.21.15.107:0/1615576400 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f6804086880 con 0x7f68100fbb40 2024-05-17T21:50:47.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.145+0000 7f680f7fe700 1 --2- 172.21.15.107:0/1615576400 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f67f80692a0 0x7f67f806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:50:47.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.146+0000 7f680f7fe700 1 --2- 172.21.15.107:0/1615576400 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f67f80692a0 0x7f67f806b750 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6800009fd0 tx=0x7f6800009e30 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:50:47.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.150+0000 7f67f7fff700 1 -- 172.21.15.107:0/1615576400 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f6804056570 con 0x7f68100fbb40 2024-05-17T21:50:47.355 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.362+0000 7f6815bc1700 1 -- 172.21.15.107:0/1615576400 --> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6810105c30 con 0x7f67f80692a0 2024-05-17T21:50:47.356 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.363+0000 7f67f7fff700 1 -- 172.21.15.107:0/1615576400 <== mgr.14214 v2:172.21.15.107:6800/1294361014 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+447 (secure 0 0 0) 0x7f6810105c30 con 0x7f67f80692a0 2024-05-17T21:50:47.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.366+0000 7f6815bc1700 1 -- 172.21.15.107:0/1615576400 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f67f80692a0 msgr2=0x7f67f806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:50:47.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.366+0000 7f6815bc1700 1 --2- 172.21.15.107:0/1615576400 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f67f80692a0 0x7f67f806b750 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6800009fd0 tx=0x7f6800009e30 comp rx=0 tx=0).stop 2024-05-17T21:50:47.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.366+0000 7f6815bc1700 1 -- 172.21.15.107:0/1615576400 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68100fbb40 msgr2=0x7f68101102c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:50:47.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.366+0000 7f6815bc1700 1 --2- 172.21.15.107:0/1615576400 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68100fbb40 0x7f68101102c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f6804002a50 tx=0x7f68040046b0 comp rx=0 tx=0).stop 2024-05-17T21:50:47.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.366+0000 7f6815bc1700 1 -- 172.21.15.107:0/1615576400 shutdown_connections 2024-05-17T21:50:47.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.366+0000 7f6815bc1700 1 --2- 172.21.15.107:0/1615576400 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f67f80692a0 0x7f67f806b750 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:47.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.366+0000 7f6815bc1700 1 --2- 172.21.15.107:0/1615576400 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68100fb1f0 0x7f681010fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:47.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.366+0000 7f6815bc1700 1 --2- 172.21.15.107:0/1615576400 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68100fbb40 0x7f68101102c0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:47.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.366+0000 7f6815bc1700 1 -- 172.21.15.107:0/1615576400 >> 172.21.15.107:0/1615576400 conn(0x7f68100f6bd0 msgr2=0x7f68100f8220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:50:47.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.366+0000 7f6815bc1700 1 -- 172.21.15.107:0/1615576400 shutdown_connections 2024-05-17T21:50:47.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.366+0000 7f6815bc1700 1 -- 172.21.15.107:0/1615576400 wait complete. 2024-05-17T21:50:47.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.468+0000 7f8227888700 1 -- 172.21.15.107:0/3255908652 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82200fffb0 msgr2=0x7f8220102390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:50:47.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.468+0000 7f8227888700 1 --2- 172.21.15.107:0/3255908652 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82200fffb0 0x7f8220102390 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f8214004940 tx=0x7f8214009e30 comp rx=0 tx=0).stop 2024-05-17T21:50:47.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.468+0000 7f8227888700 1 -- 172.21.15.107:0/3255908652 shutdown_connections 2024-05-17T21:50:47.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.468+0000 7f8227888700 1 --2- 172.21.15.107:0/3255908652 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82200fffb0 0x7f8220102390 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:47.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.468+0000 7f8227888700 1 --2- 172.21.15.107:0/3255908652 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8220066e90 0x7f82200ffa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:47.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.468+0000 7f8227888700 1 -- 172.21.15.107:0/3255908652 >> 172.21.15.107:0/3255908652 conn(0x7f82200f7440 msgr2=0x7f82200f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:50:47.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.468+0000 7f8227888700 1 -- 172.21.15.107:0/3255908652 shutdown_connections 2024-05-17T21:50:47.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.468+0000 7f8227888700 1 -- 172.21.15.107:0/3255908652 wait complete. 2024-05-17T21:50:47.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.469+0000 7f8227888700 1 Processor -- start 2024-05-17T21:50:47.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.469+0000 7f8227888700 1 -- start start 2024-05-17T21:50:47.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.469+0000 7f8227888700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8220066e90 0x7f822010d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:50:47.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.469+0000 7f8227888700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82200fffb0 0x7f822010df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:50:47.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.469+0000 7f8227888700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f822010e5c0 con 0x7f82200fffb0 2024-05-17T21:50:47.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.469+0000 7f8227888700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8220112f60 con 0x7f8220066e90 2024-05-17T21:50:47.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.469+0000 7f8224e23700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82200fffb0 0x7f822010df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:50:47.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.469+0000 7f8224e23700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82200fffb0 0x7f822010df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55786/0 (socket says 172.21.15.107:55786) 2024-05-17T21:50:47.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.469+0000 7f8224e23700 1 -- 172.21.15.107:0/1118507568 learned_addr learned my addr 172.21.15.107:0/1118507568 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:50:47.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.469+0000 7f8225624700 1 --2- 172.21.15.107:0/1118507568 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8220066e90 0x7f822010d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:50:47.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.470+0000 7f8224e23700 1 -- 172.21.15.107:0/1118507568 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8220066e90 msgr2=0x7f822010d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:50:47.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.470+0000 7f8224e23700 1 --2- 172.21.15.107:0/1118507568 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8220066e90 0x7f822010d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:47.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.470+0000 7f8224e23700 1 -- 172.21.15.107:0/1118507568 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8214005040 con 0x7f82200fffb0 2024-05-17T21:50:47.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.470+0000 7f8224e23700 1 --2- 172.21.15.107:0/1118507568 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82200fffb0 0x7f822010df30 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f8214009fd0 tx=0x7f8214009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:50:47.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.470+0000 7f821dffb700 1 -- 172.21.15.107:0/1118507568 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f821400e040 con 0x7f82200fffb0 2024-05-17T21:50:47.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.470+0000 7f821dffb700 1 -- 172.21.15.107:0/1118507568 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f821400bbf0 con 0x7f82200fffb0 2024-05-17T21:50:47.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.470+0000 7f8227888700 1 -- 172.21.15.107:0/1118507568 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8220113160 con 0x7f82200fffb0 2024-05-17T21:50:47.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.470+0000 7f821dffb700 1 -- 172.21.15.107:0/1118507568 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f821401c630 con 0x7f82200fffb0 2024-05-17T21:50:47.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.470+0000 7f8227888700 1 -- 172.21.15.107:0/1118507568 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82201136d0 con 0x7f82200fffb0 2024-05-17T21:50:47.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.471+0000 7f8227888700 1 -- 172.21.15.107:0/1118507568 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f822004c710 con 0x7f82200fffb0 2024-05-17T21:50:47.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.472+0000 7f821dffb700 1 -- 172.21.15.107:0/1118507568 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f821400bd60 con 0x7f82200fffb0 2024-05-17T21:50:47.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.472+0000 7f821dffb700 1 --2- 172.21.15.107:0/1118507568 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f82080692a0 0x7f820806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:50:47.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.472+0000 7f8225624700 1 --2- 172.21.15.107:0/1118507568 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f82080692a0 0x7f820806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:50:47.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.472+0000 7f821dffb700 1 -- 172.21.15.107:0/1118507568 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f82140884b0 con 0x7f82200fffb0 2024-05-17T21:50:47.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.473+0000 7f8225624700 1 --2- 172.21.15.107:0/1118507568 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f82080692a0 0x7f820806b750 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8220066bd0 tx=0x7f8210009780 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:50:47.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.477+0000 7f821dffb700 1 -- 172.21.15.107:0/1118507568 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f8214058220 con 0x7f82200fffb0 2024-05-17T21:50:47.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.671+0000 7f8227888700 1 -- 172.21.15.107:0/1118507568 --> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f82200f8f30 con 0x7f82080692a0 2024-05-17T21:50:47.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.681+0000 7f821dffb700 1 -- 172.21.15.107:0/1118507568 <== mgr.14214 v2:172.21.15.107:6800/1294361014 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3366 (secure 0 0 0) 0x7f82200f8f30 con 0x7f82080692a0 2024-05-17T21:50:47.675 INFO:teuthology.orchestra.run.smithi107.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-17T21:50:47.675 INFO:teuthology.orchestra.run.smithi107.stdout:alertmanager.smithi107 smithi107 *:9093,9094 running (5m) 60s ago 6m 25.2M - 0.25.0 c8568f914cd2 a2d817bf64d3 2024-05-17T21:50:47.675 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi107 smithi107 running (7m) 60s ago 7m 6660k - 17.2.7-952-g9903f776 d15a36853796 828741c3a89c 2024-05-17T21:50:47.675 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi138 smithi138 running (6m) 67s ago 6m 6660k - 17.2.7-952-g9903f776 d15a36853796 20e9ccf1fadd 2024-05-17T21:50:47.675 INFO:teuthology.orchestra.run.smithi107.stdout:grafana.smithi107 smithi107 *:3000 running (5m) 60s ago 6m 80.5M - 9.4.7 954c08fa6188 0bc11311fb71 2024-05-17T21:50:47.675 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.drqody smithi107 running (79s) 60s ago 79s 14.5M - 17.2.7-952-g9903f776 d15a36853796 0d851bc912f0 2024-05-17T21:50:47.675 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.idmbgj smithi107 running (86s) 60s ago 86s 10.4M - 17.2.7-952-g9903f776 d15a36853796 1348b09ab29e 2024-05-17T21:50:47.675 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi138.gwbtyd smithi138 running (84s) 67s ago 83s 14.4M - 17.2.7-952-g9903f776 d15a36853796 fd2ba75354c0 2024-05-17T21:50:47.675 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi138.zckrvs smithi138 running (77s) 67s ago 76s 11.2M - 17.2.7-952-g9903f776 d15a36853796 a1cfd2fe4cda 2024-05-17T21:50:47.675 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi107.xhhyiw smithi107 *:9283 running (8m) 60s ago 8m 483M - 17.2.7-952-g9903f776 d15a36853796 b566f08938ae 2024-05-17T21:50:47.675 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi138.napnll smithi138 *:8443,9283 running (5m) 67s ago 5m 436M - 17.2.7-952-g9903f776 d15a36853796 b0a8c6e63a55 2024-05-17T21:50:47.676 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi107 smithi107 running (8m) 60s ago 8m 47.5M 2048M 17.2.7-952-g9903f776 d15a36853796 7de9c23eee97 2024-05-17T21:50:47.676 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi138 smithi138 running (5m) 67s ago 5m 32.4M 2048M 17.2.7-952-g9903f776 d15a36853796 ee61a065778d 2024-05-17T21:50:47.676 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi107 smithi107 *:9100 running (7m) 60s ago 7m 15.4M - 1.5.0 0da6a335fe13 730e9bfe00b2 2024-05-17T21:50:47.676 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi138 smithi138 *:9100 running (5m) 67s ago 5m 16.2M - 1.5.0 0da6a335fe13 296b996124a1 2024-05-17T21:50:47.676 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0 smithi107 running (5m) 60s ago 5m 46.0M 4288M 17.2.7-952-g9903f776 d15a36853796 f2d20009361d 2024-05-17T21:50:47.676 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1 smithi107 running (4m) 60s ago 4m 48.3M 4288M 17.2.7-952-g9903f776 d15a36853796 e5d3e99844a9 2024-05-17T21:50:47.676 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2 smithi107 running (4m) 60s ago 4m 46.5M 4288M 17.2.7-952-g9903f776 d15a36853796 e044d7b78088 2024-05-17T21:50:47.676 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3 smithi138 running (3m) 67s ago 3m 44.5M 5312M 17.2.7-952-g9903f776 d15a36853796 8c980cc8af87 2024-05-17T21:50:47.676 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4 smithi138 running (3m) 67s ago 3m 41.1M 5312M 17.2.7-952-g9903f776 d15a36853796 b306a36a744e 2024-05-17T21:50:47.676 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5 smithi138 running (2m) 67s ago 2m 43.6M 5312M 17.2.7-952-g9903f776 d15a36853796 7c8d8ed74974 2024-05-17T21:50:47.676 INFO:teuthology.orchestra.run.smithi107.stdout:prometheus.smithi107 smithi107 *:9095 running (5m) 60s ago 6m 53.9M - 2.43.0 a07b618ecd1d 38259629b8f2 2024-05-17T21:50:47.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.683+0000 7f8227888700 1 -- 172.21.15.107:0/1118507568 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f82080692a0 msgr2=0x7f820806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:50:47.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.683+0000 7f8227888700 1 --2- 172.21.15.107:0/1118507568 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f82080692a0 0x7f820806b750 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8220066bd0 tx=0x7f8210009780 comp rx=0 tx=0).stop 2024-05-17T21:50:47.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.683+0000 7f8227888700 1 -- 172.21.15.107:0/1118507568 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82200fffb0 msgr2=0x7f822010df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:50:47.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.683+0000 7f8227888700 1 --2- 172.21.15.107:0/1118507568 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82200fffb0 0x7f822010df30 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f8214009fd0 tx=0x7f8214009e30 comp rx=0 tx=0).stop 2024-05-17T21:50:47.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.683+0000 7f8227888700 1 -- 172.21.15.107:0/1118507568 shutdown_connections 2024-05-17T21:50:47.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.683+0000 7f8227888700 1 --2- 172.21.15.107:0/1118507568 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f82080692a0 0x7f820806b750 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:47.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.683+0000 7f8227888700 1 --2- 172.21.15.107:0/1118507568 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8220066e90 0x7f822010d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:47.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.683+0000 7f8227888700 1 --2- 172.21.15.107:0/1118507568 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82200fffb0 0x7f822010df30 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:47.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.683+0000 7f8227888700 1 -- 172.21.15.107:0/1118507568 >> 172.21.15.107:0/1118507568 conn(0x7f82200f7440 msgr2=0x7f82200f7810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:50:47.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.683+0000 7f8227888700 1 -- 172.21.15.107:0/1118507568 shutdown_connections 2024-05-17T21:50:47.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.683+0000 7f8227888700 1 -- 172.21.15.107:0/1118507568 wait complete. 2024-05-17T21:50:47.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.786+0000 7fbee2b04700 1 -- 172.21.15.107:0/2745331731 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbedc0fb460 msgr2=0x7fbedc0fb870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:50:47.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.786+0000 7fbee2b04700 1 --2- 172.21.15.107:0/2745331731 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbedc0fb460 0x7fbedc0fb870 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fbed4004940 tx=0x7fbed4009e30 comp rx=0 tx=0).stop 2024-05-17T21:50:47.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.786+0000 7fbee2b04700 1 -- 172.21.15.107:0/2745331731 shutdown_connections 2024-05-17T21:50:47.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.786+0000 7fbee2b04700 1 --2- 172.21.15.107:0/2745331731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbedc0fc710 0x7fbedc0fcb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:47.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.786+0000 7fbee2b04700 1 --2- 172.21.15.107:0/2745331731 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbedc0fb460 0x7fbedc0fb870 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:47.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.786+0000 7fbee2b04700 1 -- 172.21.15.107:0/2745331731 >> 172.21.15.107:0/2745331731 conn(0x7fbedc0f6be0 msgr2=0x7fbedc0f8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:50:47.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.786+0000 7fbee2b04700 1 -- 172.21.15.107:0/2745331731 shutdown_connections 2024-05-17T21:50:47.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.786+0000 7fbee2b04700 1 -- 172.21.15.107:0/2745331731 wait complete. 2024-05-17T21:50:47.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.787+0000 7fbee2b04700 1 Processor -- start 2024-05-17T21:50:47.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.787+0000 7fbee2b04700 1 -- start start 2024-05-17T21:50:47.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.787+0000 7fbee2b04700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbedc0fb460 0x7fbedc0ff3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:50:47.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.787+0000 7fbee2b04700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbedc0fc710 0x7fbedc0fda50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:50:47.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.787+0000 7fbee2b04700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbedc0ff9a0 con 0x7fbedc0fb460 2024-05-17T21:50:47.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.787+0000 7fbee2b04700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbedc0fdf90 con 0x7fbedc0fc710 2024-05-17T21:50:47.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.787+0000 7fbee08a0700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbedc0fb460 0x7fbedc0ff3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:50:47.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.787+0000 7fbee08a0700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbedc0fb460 0x7fbedc0ff3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55808/0 (socket says 172.21.15.107:55808) 2024-05-17T21:50:47.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.787+0000 7fbee08a0700 1 -- 172.21.15.107:0/96246212 learned_addr learned my addr 172.21.15.107:0/96246212 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:50:47.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.787+0000 7fbedbfff700 1 --2- 172.21.15.107:0/96246212 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbedc0fc710 0x7fbedc0fda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:50:47.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.788+0000 7fbee08a0700 1 -- 172.21.15.107:0/96246212 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbedc0fc710 msgr2=0x7fbedc0fda50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:50:47.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.788+0000 7fbee08a0700 1 --2- 172.21.15.107:0/96246212 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbedc0fc710 0x7fbedc0fda50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:47.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.788+0000 7fbee08a0700 1 -- 172.21.15.107:0/96246212 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbed4005040 con 0x7fbedc0fb460 2024-05-17T21:50:47.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.788+0000 7fbee08a0700 1 --2- 172.21.15.107:0/96246212 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbedc0fb460 0x7fbedc0ff3a0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fbed400b3b0 tx=0x7fbed4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:50:47.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.788+0000 7fbed97fa700 1 -- 172.21.15.107:0/96246212 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbed400e040 con 0x7fbedc0fb460 2024-05-17T21:50:47.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.788+0000 7fbee2b04700 1 -- 172.21.15.107:0/96246212 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbedc0fe1e0 con 0x7fbedc0fb460 2024-05-17T21:50:47.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.788+0000 7fbed97fa700 1 -- 172.21.15.107:0/96246212 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbed400bbf0 con 0x7fbedc0fb460 2024-05-17T21:50:47.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.788+0000 7fbed97fa700 1 -- 172.21.15.107:0/96246212 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbed401c630 con 0x7fbedc0fb460 2024-05-17T21:50:47.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.788+0000 7fbee2b04700 1 -- 172.21.15.107:0/96246212 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbedc0fe7a0 con 0x7fbedc0fb460 2024-05-17T21:50:47.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.789+0000 7fbee2b04700 1 -- 172.21.15.107:0/96246212 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbedc04c710 con 0x7fbedc0fb460 2024-05-17T21:50:47.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.790+0000 7fbed97fa700 1 -- 172.21.15.107:0/96246212 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7fbed400bd60 con 0x7fbedc0fb460 2024-05-17T21:50:47.784 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.790+0000 7fbed97fa700 1 --2- 172.21.15.107:0/96246212 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fbec80692a0 0x7fbec806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:50:47.784 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.790+0000 7fbed97fa700 1 -- 172.21.15.107:0/96246212 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fbed4087f10 con 0x7fbedc0fb460 2024-05-17T21:50:47.784 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.790+0000 7fbedbfff700 1 --2- 172.21.15.107:0/96246212 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fbec80692a0 0x7fbec806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:50:47.784 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.790+0000 7fbedbfff700 1 --2- 172.21.15.107:0/96246212 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fbec80692a0 0x7fbec806b750 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fbecc004c30 tx=0x7fbecc00c040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:50:47.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:47.794+0000 7fbed97fa700 1 -- 172.21.15.107:0/96246212 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fbed4057c80 con 0x7fbedc0fb460 2024-05-17T21:50:48.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.065+0000 7fbee2b04700 1 -- 172.21.15.107:0/96246212 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fbedc115a10 con 0x7fbedc0fb460 2024-05-17T21:50:48.058 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:50:47 smithi107 ceph-mon[28434]: pgmap v221: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T21:50:48.058 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:50:47 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:50:48.058 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:50:47 smithi107 ceph-mon[28434]: Upgrade: Target is version 19.0.0-3794-g931809a9 (unknown) 2024-05-17T21:50:48.058 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:50:47 smithi107 ceph-mon[28434]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, digests ['quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b'] 2024-05-17T21:50:48.058 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:50:47 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:50:48.058 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:50:47 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:50:48.058 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:50:47 smithi107 ceph-mon[28434]: Upgrade: Need to upgrade myself (mgr.smithi107.xhhyiw) 2024-05-17T21:50:48.058 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:50:47 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:50:48.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.065+0000 7fbed97fa700 1 -- 172.21.15.107:0/96246212 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+638 (secure 0 0 0) 0x7fbed4057810 con 0x7fbedc0fb460 2024-05-17T21:50:48.059 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-05-17T21:50:48.059 INFO:teuthology.orchestra.run.smithi107.stdout: "mon": { 2024-05-17T21:50:48.059 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-952-g9903f776 (9903f7767386a9d9efadb7c1effdcd3064d71804) quincy (stable)": 2 2024-05-17T21:50:48.059 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-05-17T21:50:48.059 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr": { 2024-05-17T21:50:48.059 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-952-g9903f776 (9903f7767386a9d9efadb7c1effdcd3064d71804) quincy (stable)": 2 2024-05-17T21:50:48.059 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-05-17T21:50:48.060 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": { 2024-05-17T21:50:48.060 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-952-g9903f776 (9903f7767386a9d9efadb7c1effdcd3064d71804) quincy (stable)": 6 2024-05-17T21:50:48.060 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-05-17T21:50:48.060 INFO:teuthology.orchestra.run.smithi107.stdout: "mds": { 2024-05-17T21:50:48.060 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-952-g9903f776 (9903f7767386a9d9efadb7c1effdcd3064d71804) quincy (stable)": 4 2024-05-17T21:50:48.060 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-05-17T21:50:48.060 INFO:teuthology.orchestra.run.smithi107.stdout: "overall": { 2024-05-17T21:50:48.060 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-952-g9903f776 (9903f7767386a9d9efadb7c1effdcd3064d71804) quincy (stable)": 14 2024-05-17T21:50:48.060 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-05-17T21:50:48.060 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-05-17T21:50:48.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.067+0000 7fbee2b04700 1 -- 172.21.15.107:0/96246212 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fbec80692a0 msgr2=0x7fbec806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:50:48.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.067+0000 7fbee2b04700 1 --2- 172.21.15.107:0/96246212 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fbec80692a0 0x7fbec806b750 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fbecc004c30 tx=0x7fbecc00c040 comp rx=0 tx=0).stop 2024-05-17T21:50:48.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.067+0000 7fbee2b04700 1 -- 172.21.15.107:0/96246212 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbedc0fb460 msgr2=0x7fbedc0ff3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:50:48.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.067+0000 7fbee2b04700 1 --2- 172.21.15.107:0/96246212 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbedc0fb460 0x7fbedc0ff3a0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fbed400b3b0 tx=0x7fbed4009e30 comp rx=0 tx=0).stop 2024-05-17T21:50:48.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.067+0000 7fbee2b04700 1 -- 172.21.15.107:0/96246212 shutdown_connections 2024-05-17T21:50:48.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.067+0000 7fbee2b04700 1 --2- 172.21.15.107:0/96246212 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fbec80692a0 0x7fbec806b750 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:48.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.067+0000 7fbee2b04700 1 --2- 172.21.15.107:0/96246212 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbedc0fb460 0x7fbedc0ff3a0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:48.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.067+0000 7fbee2b04700 1 --2- 172.21.15.107:0/96246212 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbedc0fc710 0x7fbedc0fda50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:48.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.067+0000 7fbee2b04700 1 -- 172.21.15.107:0/96246212 >> 172.21.15.107:0/96246212 conn(0x7fbedc0f6be0 msgr2=0x7fbedc0f80f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:50:48.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.067+0000 7fbee2b04700 1 -- 172.21.15.107:0/96246212 shutdown_connections 2024-05-17T21:50:48.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.067+0000 7fbee2b04700 1 -- 172.21.15.107:0/96246212 wait complete. 2024-05-17T21:50:48.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:50:47 smithi138 ceph-mon[35948]: pgmap v221: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T21:50:48.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:50:47 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:50:48.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:50:47 smithi138 ceph-mon[35948]: Upgrade: Target is version 19.0.0-3794-g931809a9 (unknown) 2024-05-17T21:50:48.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:50:47 smithi138 ceph-mon[35948]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, digests ['quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b'] 2024-05-17T21:50:48.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:50:47 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:50:48.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:50:47 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:50:48.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:50:47 smithi138 ceph-mon[35948]: Upgrade: Need to upgrade myself (mgr.smithi107.xhhyiw) 2024-05-17T21:50:48.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:50:47 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:50:48.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.171+0000 7fb4c8e79700 1 -- 172.21.15.107:0/3381852097 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4c40fbcc0 msgr2=0x7fb4c40fc0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:50:48.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.171+0000 7fb4c8e79700 1 --2- 172.21.15.107:0/3381852097 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4c40fbcc0 0x7fb4c40fc0d0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fb4b4004940 tx=0x7fb4b4009e30 comp rx=0 tx=0).stop 2024-05-17T21:50:48.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.171+0000 7fb4c8e79700 1 -- 172.21.15.107:0/3381852097 shutdown_connections 2024-05-17T21:50:48.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.171+0000 7fb4c8e79700 1 --2- 172.21.15.107:0/3381852097 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4c40fcf70 0x7fb4c40fd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:48.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.171+0000 7fb4c8e79700 1 --2- 172.21.15.107:0/3381852097 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4c40fbcc0 0x7fb4c40fc0d0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:48.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.171+0000 7fb4c8e79700 1 -- 172.21.15.107:0/3381852097 >> 172.21.15.107:0/3381852097 conn(0x7fb4c40f7440 msgr2=0x7fb4c40f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:50:48.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.171+0000 7fb4c8e79700 1 -- 172.21.15.107:0/3381852097 shutdown_connections 2024-05-17T21:50:48.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.171+0000 7fb4c8e79700 1 -- 172.21.15.107:0/3381852097 wait complete. 2024-05-17T21:50:48.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.172+0000 7fb4c8e79700 1 Processor -- start 2024-05-17T21:50:48.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.172+0000 7fb4c8e79700 1 -- start start 2024-05-17T21:50:48.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.172+0000 7fb4c8e79700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4c40fbcc0 0x7fb4c410fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:50:48.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.172+0000 7fb4c8e79700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4c40fcf70 0x7fb4c410fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:50:48.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.172+0000 7fb4c8e79700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c41105f0 con 0x7fb4c40fcf70 2024-05-17T21:50:48.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.172+0000 7fb4c8e79700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c4110760 con 0x7fb4c40fbcc0 2024-05-17T21:50:48.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.172+0000 7fb4c1d9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4c40fcf70 0x7fb4c410fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:50:48.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.172+0000 7fb4c1d9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4c40fcf70 0x7fb4c410fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55834/0 (socket says 172.21.15.107:55834) 2024-05-17T21:50:48.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.172+0000 7fb4c259c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4c40fbcc0 0x7fb4c410fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:50:48.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.172+0000 7fb4c1d9b700 1 -- 172.21.15.107:0/3894634561 learned_addr learned my addr 172.21.15.107:0/3894634561 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:50:48.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.172+0000 7fb4c1d9b700 1 -- 172.21.15.107:0/3894634561 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4c40fbcc0 msgr2=0x7fb4c410fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:50:48.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.172+0000 7fb4c1d9b700 1 --2- 172.21.15.107:0/3894634561 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4c40fbcc0 0x7fb4c410fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:48.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.172+0000 7fb4c1d9b700 1 -- 172.21.15.107:0/3894634561 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4b4005040 con 0x7fb4c40fcf70 2024-05-17T21:50:48.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.173+0000 7fb4c1d9b700 1 --2- 172.21.15.107:0/3894634561 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4c40fcf70 0x7fb4c410fff0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fb4b8002a80 tx=0x7fb4b80046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:50:48.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.173+0000 7fb4b2ffd700 1 -- 172.21.15.107:0/3894634561 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb4b8007cc0 con 0x7fb4c40fcf70 2024-05-17T21:50:48.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.173+0000 7fb4b2ffd700 1 -- 172.21.15.107:0/3894634561 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb4b800f040 con 0x7fb4c40fcf70 2024-05-17T21:50:48.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.173+0000 7fb4b2ffd700 1 -- 172.21.15.107:0/3894634561 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb4b800dac0 con 0x7fb4c40fcf70 2024-05-17T21:50:48.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.173+0000 7fb4c8e79700 1 -- 172.21.15.107:0/3894634561 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4c41151c0 con 0x7fb4c40fcf70 2024-05-17T21:50:48.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.173+0000 7fb4c8e79700 1 -- 172.21.15.107:0/3894634561 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4c4115780 con 0x7fb4c40fcf70 2024-05-17T21:50:48.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.174+0000 7fb4c8e79700 1 -- 172.21.15.107:0/3894634561 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4c41029f0 con 0x7fb4c40fcf70 2024-05-17T21:50:48.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.174+0000 7fb4b2ffd700 1 -- 172.21.15.107:0/3894634561 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7fb4b800dc20 con 0x7fb4c40fcf70 2024-05-17T21:50:48.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.175+0000 7fb4b2ffd700 1 --2- 172.21.15.107:0/3894634561 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fb4ac0692a0 0x7fb4ac06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:50:48.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.175+0000 7fb4b2ffd700 1 -- 172.21.15.107:0/3894634561 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fb4b8086e10 con 0x7fb4c40fcf70 2024-05-17T21:50:48.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.175+0000 7fb4c259c700 1 --2- 172.21.15.107:0/3894634561 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fb4ac0692a0 0x7fb4ac06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:50:48.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.175+0000 7fb4c259c700 1 --2- 172.21.15.107:0/3894634561 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fb4ac0692a0 0x7fb4ac06b750 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb4b400b3b0 tx=0x7fb4b4009e30 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:50:48.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.179+0000 7fb4b2ffd700 1 -- 172.21.15.107:0/3894634561 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fb4b8056ae0 con 0x7fb4c40fcf70 2024-05-17T21:50:48.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.409+0000 7fb4c8e79700 1 -- 172.21.15.107:0/3894634561 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fb4c41058b0 con 0x7fb4c40fcf70 2024-05-17T21:50:48.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.409+0000 7fb4b2ffd700 1 -- 172.21.15.107:0/3894634561 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1840 (secure 0 0 0) 0x7fb4b8056670 con 0x7fb4c40fcf70 2024-05-17T21:50:48.402 INFO:teuthology.orchestra.run.smithi107.stdout:e15 2024-05-17T21:50:48.402 INFO:teuthology.orchestra.run.smithi107.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-17T21:50:48.402 INFO:teuthology.orchestra.run.smithi107.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-17T21:50:48.402 INFO:teuthology.orchestra.run.smithi107.stdout:legacy client fscid: 1 2024-05-17T21:50:48.402 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:50:48.402 INFO:teuthology.orchestra.run.smithi107.stdout:Filesystem 'cephfs' (1) 2024-05-17T21:50:48.403 INFO:teuthology.orchestra.run.smithi107.stdout:fs_name cephfs 2024-05-17T21:50:48.403 INFO:teuthology.orchestra.run.smithi107.stdout:epoch 15 2024-05-17T21:50:48.403 INFO:teuthology.orchestra.run.smithi107.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-17T21:50:48.403 INFO:teuthology.orchestra.run.smithi107.stdout:created 2024-05-17T21:49:16.836333+0000 2024-05-17T21:50:48.403 INFO:teuthology.orchestra.run.smithi107.stdout:modified 2024-05-17T21:49:34.873628+0000 2024-05-17T21:50:48.403 INFO:teuthology.orchestra.run.smithi107.stdout:tableserver 0 2024-05-17T21:50:48.403 INFO:teuthology.orchestra.run.smithi107.stdout:root 0 2024-05-17T21:50:48.403 INFO:teuthology.orchestra.run.smithi107.stdout:session_timeout 60 2024-05-17T21:50:48.403 INFO:teuthology.orchestra.run.smithi107.stdout:session_autoclose 300 2024-05-17T21:50:48.403 INFO:teuthology.orchestra.run.smithi107.stdout:max_file_size 1099511627776 2024-05-17T21:50:48.403 INFO:teuthology.orchestra.run.smithi107.stdout:required_client_features {} 2024-05-17T21:50:48.403 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure 0 2024-05-17T21:50:48.403 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure_osd_epoch 46 2024-05-17T21:50:48.403 INFO:teuthology.orchestra.run.smithi107.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-17T21:50:48.404 INFO:teuthology.orchestra.run.smithi107.stdout:max_mds 1 2024-05-17T21:50:48.404 INFO:teuthology.orchestra.run.smithi107.stdout:in 0 2024-05-17T21:50:48.404 INFO:teuthology.orchestra.run.smithi107.stdout:up {0=24259} 2024-05-17T21:50:48.404 INFO:teuthology.orchestra.run.smithi107.stdout:failed 2024-05-17T21:50:48.404 INFO:teuthology.orchestra.run.smithi107.stdout:damaged 2024-05-17T21:50:48.404 INFO:teuthology.orchestra.run.smithi107.stdout:stopped 2024-05-17T21:50:48.404 INFO:teuthology.orchestra.run.smithi107.stdout:data_pools [3] 2024-05-17T21:50:48.404 INFO:teuthology.orchestra.run.smithi107.stdout:metadata_pool 2 2024-05-17T21:50:48.404 INFO:teuthology.orchestra.run.smithi107.stdout:inline_data enabled 2024-05-17T21:50:48.404 INFO:teuthology.orchestra.run.smithi107.stdout:balancer 2024-05-17T21:50:48.404 INFO:teuthology.orchestra.run.smithi107.stdout:standby_count_wanted 1 2024-05-17T21:50:48.404 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi138.gwbtyd{0:24259} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.138:6824/2779634919,v1:172.21.15.138:6825/2779634919] compat {c=[1],r=[1],i=[7ff]}] 2024-05-17T21:50:48.404 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.drqody{0:14486} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.107:6828/2275807100,v1:172.21.15.107:6829/2275807100] compat {c=[1],r=[1],i=[7ff]}] 2024-05-17T21:50:48.405 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:50:48.405 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:50:48.405 INFO:teuthology.orchestra.run.smithi107.stdout:Standby daemons: 2024-05-17T21:50:48.405 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:50:48.405 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.idmbgj{-1:14498} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.107:6826/1460186079,v1:172.21.15.107:6827/1460186079] compat {c=[1],r=[1],i=[7ff]}] 2024-05-17T21:50:48.405 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi138.zckrvs{-1:24267} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.138:6826/170628661,v1:172.21.15.138:6827/170628661] compat {c=[1],r=[1],i=[7ff]}] 2024-05-17T21:50:48.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.411+0000 7fb4c8e79700 1 -- 172.21.15.107:0/3894634561 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fb4ac0692a0 msgr2=0x7fb4ac06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:50:48.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.411+0000 7fb4c8e79700 1 --2- 172.21.15.107:0/3894634561 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fb4ac0692a0 0x7fb4ac06b750 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb4b400b3b0 tx=0x7fb4b4009e30 comp rx=0 tx=0).stop 2024-05-17T21:50:48.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.411+0000 7fb4c8e79700 1 -- 172.21.15.107:0/3894634561 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4c40fcf70 msgr2=0x7fb4c410fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:50:48.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.411+0000 7fb4c8e79700 1 --2- 172.21.15.107:0/3894634561 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4c40fcf70 0x7fb4c410fff0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fb4b8002a80 tx=0x7fb4b80046b0 comp rx=0 tx=0).stop 2024-05-17T21:50:48.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.411+0000 7fb4c8e79700 1 -- 172.21.15.107:0/3894634561 shutdown_connections 2024-05-17T21:50:48.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.411+0000 7fb4c8e79700 1 --2- 172.21.15.107:0/3894634561 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fb4ac0692a0 0x7fb4ac06b750 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:48.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.411+0000 7fb4c8e79700 1 --2- 172.21.15.107:0/3894634561 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4c40fbcc0 0x7fb4c410fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:48.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.411+0000 7fb4c8e79700 1 --2- 172.21.15.107:0/3894634561 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4c40fcf70 0x7fb4c410fff0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:48.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.411+0000 7fb4c8e79700 1 -- 172.21.15.107:0/3894634561 >> 172.21.15.107:0/3894634561 conn(0x7fb4c40f7440 msgr2=0x7fb4c40f8a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:50:48.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.411+0000 7fb4c8e79700 1 -- 172.21.15.107:0/3894634561 shutdown_connections 2024-05-17T21:50:48.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.411+0000 7fb4c8e79700 1 -- 172.21.15.107:0/3894634561 wait complete. 2024-05-17T21:50:48.406 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 15 2024-05-17T21:50:48.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.514+0000 7fdc6f745700 1 -- 172.21.15.107:0/1566525727 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc680fbaf0 msgr2=0x7fdc680fe350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:50:48.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.514+0000 7fdc6f745700 1 --2- 172.21.15.107:0/1566525727 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc680fbaf0 0x7fdc680fe350 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fdc58004940 tx=0x7fdc58009e30 comp rx=0 tx=0).stop 2024-05-17T21:50:48.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.514+0000 7fdc6f745700 1 -- 172.21.15.107:0/1566525727 shutdown_connections 2024-05-17T21:50:48.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.514+0000 7fdc6f745700 1 --2- 172.21.15.107:0/1566525727 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc680fbaf0 0x7fdc680fe350 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:48.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.514+0000 7fdc6f745700 1 --2- 172.21.15.107:0/1566525727 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc680fb1a0 0x7fdc680fb5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:48.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.514+0000 7fdc6f745700 1 -- 172.21.15.107:0/1566525727 >> 172.21.15.107:0/1566525727 conn(0x7fdc680f6b80 msgr2=0x7fdc680f8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:50:48.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.514+0000 7fdc6f745700 1 -- 172.21.15.107:0/1566525727 shutdown_connections 2024-05-17T21:50:48.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.514+0000 7fdc6f745700 1 -- 172.21.15.107:0/1566525727 wait complete. 2024-05-17T21:50:48.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.514+0000 7fdc6f745700 1 Processor -- start 2024-05-17T21:50:48.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.514+0000 7fdc6f745700 1 -- start start 2024-05-17T21:50:48.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.515+0000 7fdc6f745700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc680fb1a0 0x7fdc680fdcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:50:48.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.515+0000 7fdc6f745700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc680fbaf0 0x7fdc680fc180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:50:48.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.515+0000 7fdc6f745700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc680fc730 con 0x7fdc680fb1a0 2024-05-17T21:50:48.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.515+0000 7fdc6f745700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc680fc8a0 con 0x7fdc680fbaf0 2024-05-17T21:50:48.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.515+0000 7fdc6d4e1700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc680fb1a0 0x7fdc680fdcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:50:48.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.515+0000 7fdc6d4e1700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc680fb1a0 0x7fdc680fdcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55850/0 (socket says 172.21.15.107:55850) 2024-05-17T21:50:48.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.515+0000 7fdc6d4e1700 1 -- 172.21.15.107:0/2406449656 learned_addr learned my addr 172.21.15.107:0/2406449656 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:50:48.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.515+0000 7fdc6cce0700 1 --2- 172.21.15.107:0/2406449656 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc680fbaf0 0x7fdc680fc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:50:48.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.515+0000 7fdc6d4e1700 1 -- 172.21.15.107:0/2406449656 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc680fbaf0 msgr2=0x7fdc680fc180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:50:48.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.515+0000 7fdc6d4e1700 1 --2- 172.21.15.107:0/2406449656 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc680fbaf0 0x7fdc680fc180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:48.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.515+0000 7fdc6d4e1700 1 -- 172.21.15.107:0/2406449656 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc58005040 con 0x7fdc680fb1a0 2024-05-17T21:50:48.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.515+0000 7fdc6d4e1700 1 --2- 172.21.15.107:0/2406449656 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc680fb1a0 0x7fdc680fdcb0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fdc6400eb40 tx=0x7fdc6400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:50:48.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.515+0000 7fdc5dffb700 1 -- 172.21.15.107:0/2406449656 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdc64007cc0 con 0x7fdc680fb1a0 2024-05-17T21:50:48.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.515+0000 7fdc5dffb700 1 -- 172.21.15.107:0/2406449656 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdc6400f040 con 0x7fdc680fb1a0 2024-05-17T21:50:48.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.515+0000 7fdc6f745700 1 -- 172.21.15.107:0/2406449656 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc680fcb20 con 0x7fdc680fb1a0 2024-05-17T21:50:48.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.515+0000 7fdc5dffb700 1 -- 172.21.15.107:0/2406449656 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdc6400ca10 con 0x7fdc680fb1a0 2024-05-17T21:50:48.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.516+0000 7fdc6f745700 1 -- 172.21.15.107:0/2406449656 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc68115bb0 con 0x7fdc680fb1a0 2024-05-17T21:50:48.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.517+0000 7fdc6f745700 1 -- 172.21.15.107:0/2406449656 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc6805f5c0 con 0x7fdc680fb1a0 2024-05-17T21:50:48.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.517+0000 7fdc5dffb700 1 -- 172.21.15.107:0/2406449656 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7fdc6400cb70 con 0x7fdc680fb1a0 2024-05-17T21:50:48.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.517+0000 7fdc5dffb700 1 --2- 172.21.15.107:0/2406449656 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fdc540692a0 0x7fdc5406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:50:48.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.517+0000 7fdc6cce0700 1 --2- 172.21.15.107:0/2406449656 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fdc540692a0 0x7fdc5406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:50:48.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.517+0000 7fdc5dffb700 1 -- 172.21.15.107:0/2406449656 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fdc64086d00 con 0x7fdc680fb1a0 2024-05-17T21:50:48.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.518+0000 7fdc6cce0700 1 --2- 172.21.15.107:0/2406449656 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fdc540692a0 0x7fdc5406b750 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fdc58009b30 tx=0x7fdc58009e30 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:50:48.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.522+0000 7fdc5dffb700 1 -- 172.21.15.107:0/2406449656 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fdc6404e250 con 0x7fdc680fb1a0 2024-05-17T21:50:48.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.721+0000 7fdc6f745700 1 -- 172.21.15.107:0/2406449656 --> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdc681025f0 con 0x7fdc540692a0 2024-05-17T21:50:48.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.722+0000 7fdc5dffb700 1 -- 172.21.15.107:0/2406449656 <== mgr.14214 v2:172.21.15.107:6800/1294361014 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+447 (secure 0 0 0) 0x7fdc681025f0 con 0x7fdc540692a0 2024-05-17T21:50:48.715 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-05-17T21:50:48.715 INFO:teuthology.orchestra.run.smithi107.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b", 2024-05-17T21:50:48.715 INFO:teuthology.orchestra.run.smithi107.stdout: "in_progress": true, 2024-05-17T21:50:48.716 INFO:teuthology.orchestra.run.smithi107.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-17T21:50:48.716 INFO:teuthology.orchestra.run.smithi107.stdout: "services_complete": [], 2024-05-17T21:50:48.716 INFO:teuthology.orchestra.run.smithi107.stdout: "progress": "0/21 daemons upgraded", 2024-05-17T21:50:48.716 INFO:teuthology.orchestra.run.smithi107.stdout: "message": "Pulling quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b image on host smithi138", 2024-05-17T21:50:48.716 INFO:teuthology.orchestra.run.smithi107.stdout: "is_paused": false 2024-05-17T21:50:48.716 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-05-17T21:50:48.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.724+0000 7fdc6f745700 1 -- 172.21.15.107:0/2406449656 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fdc540692a0 msgr2=0x7fdc5406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:50:48.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.724+0000 7fdc6f745700 1 --2- 172.21.15.107:0/2406449656 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fdc540692a0 0x7fdc5406b750 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fdc58009b30 tx=0x7fdc58009e30 comp rx=0 tx=0).stop 2024-05-17T21:50:48.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.724+0000 7fdc6f745700 1 -- 172.21.15.107:0/2406449656 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc680fb1a0 msgr2=0x7fdc680fdcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:50:48.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.724+0000 7fdc6f745700 1 --2- 172.21.15.107:0/2406449656 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc680fb1a0 0x7fdc680fdcb0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fdc6400eb40 tx=0x7fdc6400b040 comp rx=0 tx=0).stop 2024-05-17T21:50:48.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.724+0000 7fdc6f745700 1 -- 172.21.15.107:0/2406449656 shutdown_connections 2024-05-17T21:50:48.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.724+0000 7fdc6f745700 1 --2- 172.21.15.107:0/2406449656 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fdc540692a0 0x7fdc5406b750 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:48.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.724+0000 7fdc6f745700 1 --2- 172.21.15.107:0/2406449656 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc680fb1a0 0x7fdc680fdcb0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:48.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.724+0000 7fdc6f745700 1 --2- 172.21.15.107:0/2406449656 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc680fbaf0 0x7fdc680fc180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:48.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.725+0000 7fdc6f745700 1 -- 172.21.15.107:0/2406449656 >> 172.21.15.107:0/2406449656 conn(0x7fdc680f6b80 msgr2=0x7fdc680f8180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:50:48.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.725+0000 7fdc6f745700 1 -- 172.21.15.107:0/2406449656 shutdown_connections 2024-05-17T21:50:48.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.725+0000 7fdc6f745700 1 -- 172.21.15.107:0/2406449656 wait complete. 2024-05-17T21:50:48.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.827+0000 7fa7486d8700 1 -- 172.21.15.107:0/2610086200 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa7400fdfd0 msgr2=0x7fa7400fe440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:50:48.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.827+0000 7fa7486d8700 1 --2- 172.21.15.107:0/2610086200 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa7400fdfd0 0x7fa7400fe440 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fa73c004940 tx=0x7fa73c009e30 comp rx=0 tx=0).stop 2024-05-17T21:50:48.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.827+0000 7fa7486d8700 1 -- 172.21.15.107:0/2610086200 shutdown_connections 2024-05-17T21:50:48.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.827+0000 7fa7486d8700 1 --2- 172.21.15.107:0/2610086200 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa7400fdfd0 0x7fa7400fe440 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:48.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.827+0000 7fa7486d8700 1 --2- 172.21.15.107:0/2610086200 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7400fce60 0x7fa7400fd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:48.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.827+0000 7fa7486d8700 1 -- 172.21.15.107:0/2610086200 >> 172.21.15.107:0/2610086200 conn(0x7fa7400f8540 msgr2=0x7fa7400fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:50:48.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.827+0000 7fa7486d8700 1 -- 172.21.15.107:0/2610086200 shutdown_connections 2024-05-17T21:50:48.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.828+0000 7fa7486d8700 1 -- 172.21.15.107:0/2610086200 wait complete. 2024-05-17T21:50:48.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.828+0000 7fa7486d8700 1 Processor -- start 2024-05-17T21:50:48.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.828+0000 7fa7486d8700 1 -- start start 2024-05-17T21:50:48.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.828+0000 7fa7486d8700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7400fce60 0x7fa74010d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:50:48.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.828+0000 7fa7486d8700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa7400fdfd0 0x7fa74010de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:50:48.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.828+0000 7fa7486d8700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa74010e400 con 0x7fa7400fdfd0 2024-05-17T21:50:48.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.828+0000 7fa7486d8700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa74010e570 con 0x7fa7400fce60 2024-05-17T21:50:48.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.828+0000 7fa745c73700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa7400fdfd0 0x7fa74010de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:50:48.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.828+0000 7fa745c73700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa7400fdfd0 0x7fa74010de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55868/0 (socket says 172.21.15.107:55868) 2024-05-17T21:50:48.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.828+0000 7fa745c73700 1 -- 172.21.15.107:0/1661572746 learned_addr learned my addr 172.21.15.107:0/1661572746 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:50:48.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.829+0000 7fa746474700 1 --2- 172.21.15.107:0/1661572746 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7400fce60 0x7fa74010d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:50:48.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.829+0000 7fa745c73700 1 -- 172.21.15.107:0/1661572746 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7400fce60 msgr2=0x7fa74010d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:50:48.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.829+0000 7fa745c73700 1 --2- 172.21.15.107:0/1661572746 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7400fce60 0x7fa74010d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:48.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.829+0000 7fa745c73700 1 -- 172.21.15.107:0/1661572746 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa73c005040 con 0x7fa7400fdfd0 2024-05-17T21:50:48.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.829+0000 7fa745c73700 1 --2- 172.21.15.107:0/1661572746 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa7400fdfd0 0x7fa74010de00 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fa73c013870 tx=0x7fa73c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:50:48.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.829+0000 7fa736ffd700 1 -- 172.21.15.107:0/1661572746 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa73c021070 con 0x7fa7400fdfd0 2024-05-17T21:50:48.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.829+0000 7fa736ffd700 1 -- 172.21.15.107:0/1661572746 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa73c009430 con 0x7fa7400fdfd0 2024-05-17T21:50:48.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.829+0000 7fa7486d8700 1 -- 172.21.15.107:0/1661572746 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa740112f70 con 0x7fa7400fdfd0 2024-05-17T21:50:48.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.829+0000 7fa736ffd700 1 -- 172.21.15.107:0/1661572746 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa73c00e070 con 0x7fa7400fdfd0 2024-05-17T21:50:48.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.829+0000 7fa7486d8700 1 -- 172.21.15.107:0/1661572746 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa740113480 con 0x7fa7400fdfd0 2024-05-17T21:50:48.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.830+0000 7fa7486d8700 1 -- 172.21.15.107:0/1661572746 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa74004c710 con 0x7fa7400fdfd0 2024-05-17T21:50:48.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.831+0000 7fa736ffd700 1 -- 172.21.15.107:0/1661572746 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7fa73c01c750 con 0x7fa7400fdfd0 2024-05-17T21:50:48.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.831+0000 7fa736ffd700 1 --2- 172.21.15.107:0/1661572746 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fa72c069240 0x7fa72c06b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:50:48.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.831+0000 7fa746474700 1 --2- 172.21.15.107:0/1661572746 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fa72c069240 0x7fa72c06b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:50:48.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.831+0000 7fa736ffd700 1 -- 172.21.15.107:0/1661572746 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fa73c0884e0 con 0x7fa7400fdfd0 2024-05-17T21:50:48.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.831+0000 7fa746474700 1 --2- 172.21.15.107:0/1661572746 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fa72c069240 0x7fa72c06b6f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa730009b80 tx=0x7fa73000b040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:50:48.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:48.835+0000 7fa736ffd700 1 -- 172.21.15.107:0/1661572746 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fa73c058250 con 0x7fa7400fdfd0 2024-05-17T21:50:49.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:50:48 smithi138 ceph-mon[35948]: from='client.14580 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:50:49.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:50:48 smithi138 ceph-mon[35948]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b on smithi138 2024-05-17T21:50:49.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:50:48 smithi138 ceph-mon[35948]: from='client.14584 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:50:49.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:50:48 smithi138 ceph-mon[35948]: from='client.14588 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:50:49.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:50:48 smithi138 ceph-mon[35948]: from='client.? 172.21.15.107:0/96246212' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:50:49.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:50:48 smithi138 ceph-mon[35948]: from='client.? 172.21.15.107:0/3894634561' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-17T21:50:49.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:49.098+0000 7fa7486d8700 1 -- 172.21.15.107:0/1661572746 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fa740113990 con 0x7fa7400fdfd0 2024-05-17T21:50:49.091 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:50:48 smithi107 ceph-mon[28434]: from='client.14580 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:50:49.091 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:50:48 smithi107 ceph-mon[28434]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b on smithi138 2024-05-17T21:50:49.091 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:50:48 smithi107 ceph-mon[28434]: from='client.14584 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:50:49.091 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:50:48 smithi107 ceph-mon[28434]: from='client.14588 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:50:49.091 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:50:48 smithi107 ceph-mon[28434]: from='client.? 172.21.15.107:0/96246212' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:50:49.091 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:50:48 smithi107 ceph-mon[28434]: from='client.? 172.21.15.107:0/3894634561' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-17T21:50:49.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:49.098+0000 7fa736ffd700 1 -- 172.21.15.107:0/1661572746 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7fa73c057de0 con 0x7fa7400fdfd0 2024-05-17T21:50:49.092 INFO:teuthology.orchestra.run.smithi107.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-17T21:50:49.092 INFO:teuthology.orchestra.run.smithi107.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-17T21:50:49.092 INFO:teuthology.orchestra.run.smithi107.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-17T21:50:49.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:49.101+0000 7fa7486d8700 1 -- 172.21.15.107:0/1661572746 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fa72c069240 msgr2=0x7fa72c06b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:50:49.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:49.101+0000 7fa7486d8700 1 --2- 172.21.15.107:0/1661572746 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fa72c069240 0x7fa72c06b6f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa730009b80 tx=0x7fa73000b040 comp rx=0 tx=0).stop 2024-05-17T21:50:49.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:49.101+0000 7fa7486d8700 1 -- 172.21.15.107:0/1661572746 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa7400fdfd0 msgr2=0x7fa74010de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:50:49.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:49.101+0000 7fa7486d8700 1 --2- 172.21.15.107:0/1661572746 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa7400fdfd0 0x7fa74010de00 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fa73c013870 tx=0x7fa73c009e30 comp rx=0 tx=0).stop 2024-05-17T21:50:49.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:49.101+0000 7fa7486d8700 1 -- 172.21.15.107:0/1661572746 shutdown_connections 2024-05-17T21:50:49.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:49.101+0000 7fa7486d8700 1 --2- 172.21.15.107:0/1661572746 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fa72c069240 0x7fa72c06b6f0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:49.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:49.101+0000 7fa7486d8700 1 --2- 172.21.15.107:0/1661572746 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7400fce60 0x7fa74010d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:49.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:49.101+0000 7fa7486d8700 1 --2- 172.21.15.107:0/1661572746 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa7400fdfd0 0x7fa74010de00 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:50:49.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:49.101+0000 7fa7486d8700 1 -- 172.21.15.107:0/1661572746 >> 172.21.15.107:0/1661572746 conn(0x7fa7400f8540 msgr2=0x7fa7400f9b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:50:49.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:49.101+0000 7fa7486d8700 1 -- 172.21.15.107:0/1661572746 shutdown_connections 2024-05-17T21:50:49.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:50:49.101+0000 7fa7486d8700 1 -- 172.21.15.107:0/1661572746 wait complete. 2024-05-17T21:50:50.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:50:49 smithi138 ceph-mon[35948]: pgmap v222: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T21:50:50.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:50:49 smithi138 ceph-mon[35948]: from='client.14600 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:50:50.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:50:49 smithi138 ceph-mon[35948]: from='client.? 172.21.15.107:0/1661572746' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-17T21:50:50.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:50:49 smithi107 ceph-mon[28434]: pgmap v222: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T21:50:50.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:50:49 smithi107 ceph-mon[28434]: from='client.14600 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:50:50.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:50:49 smithi107 ceph-mon[28434]: from='client.? 172.21.15.107:0/1661572746' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-17T21:50:52.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:50:51 smithi138 ceph-mon[35948]: pgmap v223: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T21:50:52.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:50:51 smithi107 ceph-mon[28434]: pgmap v223: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T21:50:54.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:50:53 smithi138 ceph-mon[35948]: pgmap v224: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T21:50:54.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:50:53 smithi107 ceph-mon[28434]: pgmap v224: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T21:50:56.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:50:55 smithi138 ceph-mon[35948]: pgmap v225: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T21:50:56.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:50:55 smithi107 ceph-mon[28434]: pgmap v225: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T21:50:57.872 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:50:57 smithi107 ceph-mon[28434]: pgmap v226: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T21:50:58.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:50:57 smithi138 ceph-mon[35948]: pgmap v226: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T21:51:00.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:50:59 smithi138 ceph-mon[35948]: pgmap v227: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T21:51:00.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:50:59 smithi107 ceph-mon[28434]: pgmap v227: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T21:51:02.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:01 smithi138 ceph-mon[35948]: pgmap v228: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T21:51:02.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:01 smithi107 ceph-mon[28434]: pgmap v228: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T21:51:03.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:02 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:51:03.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:02 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:51:04.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:03 smithi138 ceph-mon[35948]: pgmap v229: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T21:51:04.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:03 smithi107 ceph-mon[28434]: pgmap v229: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T21:51:06.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:05 smithi138 ceph-mon[35948]: pgmap v230: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T21:51:06.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:05 smithi107 ceph-mon[28434]: pgmap v230: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T21:51:08.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:07 smithi138 ceph-mon[35948]: pgmap v231: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T21:51:08.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:07 smithi107 ceph-mon[28434]: pgmap v231: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T21:51:10.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:09 smithi138 ceph-mon[35948]: pgmap v232: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T21:51:10.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:09 smithi107 ceph-mon[28434]: pgmap v232: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T21:51:12.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:11 smithi138 ceph-mon[35948]: pgmap v233: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T21:51:12.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:11 smithi107 ceph-mon[28434]: pgmap v233: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T21:51:14.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:13 smithi138 ceph-mon[35948]: pgmap v234: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T21:51:14.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:13 smithi107 ceph-mon[28434]: pgmap v234: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T21:51:16.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:15 smithi138 ceph-mon[35948]: pgmap v235: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T21:51:16.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:15 smithi107 ceph-mon[28434]: pgmap v235: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T21:51:17.787 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:17 smithi138 ceph-mon[35948]: pgmap v236: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T21:51:17.787 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:17 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:51:18.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:17 smithi107 ceph-mon[28434]: pgmap v236: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T21:51:18.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:17 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:51:19.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:19.245+0000 7fb621b27700 1 -- 172.21.15.107:0/754109175 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb61c0fcf60 msgr2=0x7fb61c0fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:51:19.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:19.245+0000 7fb621b27700 1 --2- 172.21.15.107:0/754109175 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb61c0fcf60 0x7fb61c0fd3d0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fb610004940 tx=0x7fb610009e30 comp rx=0 tx=0).stop 2024-05-17T21:51:19.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:19.245+0000 7fb621b27700 1 -- 172.21.15.107:0/754109175 shutdown_connections 2024-05-17T21:51:19.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:19.245+0000 7fb621b27700 1 --2- 172.21.15.107:0/754109175 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb61c0fcf60 0x7fb61c0fd3d0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:51:19.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:19.245+0000 7fb621b27700 1 --2- 172.21.15.107:0/754109175 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb61c0fbcb0 0x7fb61c0fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:51:19.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:19.245+0000 7fb621b27700 1 -- 172.21.15.107:0/754109175 >> 172.21.15.107:0/754109175 conn(0x7fb61c0f7430 msgr2=0x7fb61c0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:51:19.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:19.245+0000 7fb621b27700 1 -- 172.21.15.107:0/754109175 shutdown_connections 2024-05-17T21:51:19.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:19.245+0000 7fb621b27700 1 -- 172.21.15.107:0/754109175 wait complete. 2024-05-17T21:51:19.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:19.246+0000 7fb621b27700 1 Processor -- start 2024-05-17T21:51:19.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:19.246+0000 7fb621b27700 1 -- start start 2024-05-17T21:51:19.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:19.246+0000 7fb621b27700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb61c0fbcb0 0x7fb61c10c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:51:19.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:19.246+0000 7fb621b27700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb61c0fcf60 0x7fb61c10cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:51:19.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:19.246+0000 7fb621b27700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb61c10d140 con 0x7fb61c0fbcb0 2024-05-17T21:51:19.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:19.246+0000 7fb621b27700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb61c10d2b0 con 0x7fb61c0fcf60 2024-05-17T21:51:19.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:19.247+0000 7fb61affd700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb61c0fcf60 0x7fb61c10cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:51:19.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:19.247+0000 7fb61affd700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb61c0fcf60 0x7fb61c10cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.107:48626/0 (socket says 172.21.15.107:48626) 2024-05-17T21:51:19.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:19.247+0000 7fb61affd700 1 -- 172.21.15.107:0/1743812269 learned_addr learned my addr 172.21.15.107:0/1743812269 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:51:19.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:19.247+0000 7fb61b7fe700 1 --2- 172.21.15.107:0/1743812269 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb61c0fbcb0 0x7fb61c10c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:51:19.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:19.247+0000 7fb61affd700 1 -- 172.21.15.107:0/1743812269 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb61c0fbcb0 msgr2=0x7fb61c10c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:51:19.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:19.247+0000 7fb61affd700 1 --2- 172.21.15.107:0/1743812269 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb61c0fbcb0 0x7fb61c10c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:51:19.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:19.247+0000 7fb61affd700 1 -- 172.21.15.107:0/1743812269 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb610005040 con 0x7fb61c0fcf60 2024-05-17T21:51:19.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:19.247+0000 7fb61affd700 1 --2- 172.21.15.107:0/1743812269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb61c0fcf60 0x7fb61c10cb40 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb61000bf40 tx=0x7fb610009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:51:19.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:19.248+0000 7fb603fff700 1 -- 172.21.15.107:0/1743812269 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb61000e030 con 0x7fb61c0fcf60 2024-05-17T21:51:19.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:19.248+0000 7fb621b27700 1 -- 172.21.15.107:0/1743812269 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb61c0661f0 con 0x7fb61c0fcf60 2024-05-17T21:51:19.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:19.248+0000 7fb603fff700 1 -- 172.21.15.107:0/1743812269 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb6100092b0 con 0x7fb61c0fcf60 2024-05-17T21:51:19.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:19.248+0000 7fb603fff700 1 -- 172.21.15.107:0/1743812269 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb61001c640 con 0x7fb61c0fcf60 2024-05-17T21:51:19.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:19.248+0000 7fb621b27700 1 -- 172.21.15.107:0/1743812269 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb61c066730 con 0x7fb61c0fcf60 2024-05-17T21:51:19.242 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:19.249+0000 7fb621b27700 1 -- 172.21.15.107:0/1743812269 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb61c04c710 con 0x7fb61c0fcf60 2024-05-17T21:51:19.242 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:19.249+0000 7fb603fff700 1 -- 172.21.15.107:0/1743812269 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7fb610013980 con 0x7fb61c0fcf60 2024-05-17T21:51:19.242 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:19.250+0000 7fb603fff700 1 --2- 172.21.15.107:0/1743812269 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fb6040692a0 0x7fb60406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:51:19.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:19.250+0000 7fb603fff700 1 -- 172.21.15.107:0/1743812269 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fb610088ff0 con 0x7fb61c0fcf60 2024-05-17T21:51:19.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:19.250+0000 7fb61b7fe700 1 --2- 172.21.15.107:0/1743812269 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fb6040692a0 0x7fb60406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:51:19.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:19.250+0000 7fb61b7fe700 1 --2- 172.21.15.107:0/1743812269 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fb6040692a0 0x7fb60406b750 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb60c007fd0 tx=0x7fb60c0044c0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:51:19.248 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:19.256+0000 7fb603fff700 1 -- 172.21.15.107:0/1743812269 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fb610058ca0 con 0x7fb61c0fcf60 2024-05-17T21:51:19.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:19.557+0000 7fb621b27700 1 -- 172.21.15.107:0/1743812269 --> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb61c066d90 con 0x7fb6040692a0 2024-05-17T21:51:19.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:19.558+0000 7fb603fff700 1 -- 172.21.15.107:0/1743812269 <== mgr.14214 v2:172.21.15.107:6800/1294361014 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+447 (secure 0 0 0) 0x7fb61c066d90 con 0x7fb6040692a0 2024-05-17T21:51:19.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:19.560+0000 7fb621b27700 1 -- 172.21.15.107:0/1743812269 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fb6040692a0 msgr2=0x7fb60406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:51:19.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:19.560+0000 7fb621b27700 1 --2- 172.21.15.107:0/1743812269 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fb6040692a0 0x7fb60406b750 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb60c007fd0 tx=0x7fb60c0044c0 comp rx=0 tx=0).stop 2024-05-17T21:51:19.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:19.560+0000 7fb621b27700 1 -- 172.21.15.107:0/1743812269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb61c0fcf60 msgr2=0x7fb61c10cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:51:19.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:19.560+0000 7fb621b27700 1 --2- 172.21.15.107:0/1743812269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb61c0fcf60 0x7fb61c10cb40 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb61000bf40 tx=0x7fb610009e30 comp rx=0 tx=0).stop 2024-05-17T21:51:19.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:19.561+0000 7fb621b27700 1 -- 172.21.15.107:0/1743812269 shutdown_connections 2024-05-17T21:51:19.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:19.561+0000 7fb621b27700 1 --2- 172.21.15.107:0/1743812269 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fb6040692a0 0x7fb60406b750 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:51:19.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:19.561+0000 7fb621b27700 1 --2- 172.21.15.107:0/1743812269 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb61c0fbcb0 0x7fb61c10c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:51:19.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:19.561+0000 7fb621b27700 1 --2- 172.21.15.107:0/1743812269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb61c0fcf60 0x7fb61c10cb40 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:51:19.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:19.561+0000 7fb621b27700 1 -- 172.21.15.107:0/1743812269 >> 172.21.15.107:0/1743812269 conn(0x7fb61c0f7430 msgr2=0x7fb61c0f8760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:51:19.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:19.561+0000 7fb621b27700 1 -- 172.21.15.107:0/1743812269 shutdown_connections 2024-05-17T21:51:19.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:19.561+0000 7fb621b27700 1 -- 172.21.15.107:0/1743812269 wait complete. 2024-05-17T21:51:19.566 INFO:teuthology.orchestra.run.smithi107.stdout:true 2024-05-17T21:51:19.713 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:19.720+0000 7fbcaeec0700 1 -- 172.21.15.107:0/2317322774 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbca80fcf60 msgr2=0x7fbca80fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:51:19.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:19.720+0000 7fbcaeec0700 1 --2- 172.21.15.107:0/2317322774 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbca80fcf60 0x7fbca80fd3d0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fbc94004940 tx=0x7fbc94009e30 comp rx=0 tx=0).stop 2024-05-17T21:51:19.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:19.720+0000 7fbcaeec0700 1 -- 172.21.15.107:0/2317322774 shutdown_connections 2024-05-17T21:51:19.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:19.720+0000 7fbcaeec0700 1 --2- 172.21.15.107:0/2317322774 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbca80fcf60 0x7fbca80fd3d0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:51:19.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:19.720+0000 7fbcaeec0700 1 --2- 172.21.15.107:0/2317322774 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbca80fbcb0 0x7fbca80fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:51:19.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:19.720+0000 7fbcaeec0700 1 -- 172.21.15.107:0/2317322774 >> 172.21.15.107:0/2317322774 conn(0x7fbca80f7430 msgr2=0x7fbca80f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:51:19.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:19.720+0000 7fbcaeec0700 1 -- 172.21.15.107:0/2317322774 shutdown_connections 2024-05-17T21:51:19.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:19.720+0000 7fbcaeec0700 1 -- 172.21.15.107:0/2317322774 wait complete. 2024-05-17T21:51:19.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:19.721+0000 7fbcaeec0700 1 Processor -- start 2024-05-17T21:51:19.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:19.721+0000 7fbcaeec0700 1 -- start start 2024-05-17T21:51:19.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:19.721+0000 7fbcaeec0700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbca80fbcb0 0x7fbca810c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:51:19.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:19.721+0000 7fbcaeec0700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbca80fcf60 0x7fbca810ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:51:19.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:19.721+0000 7fbcaeec0700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbca810d400 con 0x7fbca80fcf60 2024-05-17T21:51:19.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:19.721+0000 7fbcaeec0700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbca810dc00 con 0x7fbca80fbcb0 2024-05-17T21:51:19.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:19.722+0000 7fbcacc5c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbca80fbcb0 0x7fbca810c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:51:19.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:19.722+0000 7fbcacc5c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbca80fbcb0 0x7fbca810c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.107:48650/0 (socket says 172.21.15.107:48650) 2024-05-17T21:51:19.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:19.722+0000 7fbcacc5c700 1 -- 172.21.15.107:0/2075542138 learned_addr learned my addr 172.21.15.107:0/2075542138 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:51:19.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:19.722+0000 7fbca7fff700 1 --2- 172.21.15.107:0/2075542138 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbca80fcf60 0x7fbca810ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:51:19.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:19.722+0000 7fbcacc5c700 1 -- 172.21.15.107:0/2075542138 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbca80fcf60 msgr2=0x7fbca810ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:51:19.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:19.722+0000 7fbcacc5c700 1 --2- 172.21.15.107:0/2075542138 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbca80fcf60 0x7fbca810ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:51:19.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:19.722+0000 7fbcacc5c700 1 -- 172.21.15.107:0/2075542138 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc94005040 con 0x7fbca80fbcb0 2024-05-17T21:51:19.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:19.722+0000 7fbca7fff700 1 --2- 172.21.15.107:0/2075542138 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbca80fcf60 0x7fbca810ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T21:51:19.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:19.722+0000 7fbcacc5c700 1 --2- 172.21.15.107:0/2075542138 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbca80fbcb0 0x7fbca810c8c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c002a80 tx=0x7fbc9c0046b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:51:19.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:19.723+0000 7fbca57fa700 1 -- 172.21.15.107:0/2075542138 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc9c007cc0 con 0x7fbca80fbcb0 2024-05-17T21:51:19.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:19.723+0000 7fbcaeec0700 1 -- 172.21.15.107:0/2075542138 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbca810dee0 con 0x7fbca80fbcb0 2024-05-17T21:51:19.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:19.723+0000 7fbcaeec0700 1 -- 172.21.15.107:0/2075542138 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbca810e420 con 0x7fbca80fbcb0 2024-05-17T21:51:19.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:19.723+0000 7fbca57fa700 1 -- 172.21.15.107:0/2075542138 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbc9c00f040 con 0x7fbca80fbcb0 2024-05-17T21:51:19.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:19.723+0000 7fbca57fa700 1 -- 172.21.15.107:0/2075542138 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc9c009670 con 0x7fbca80fbcb0 2024-05-17T21:51:19.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:19.724+0000 7fbcaeec0700 1 -- 172.21.15.107:0/2075542138 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbca805f5c0 con 0x7fbca80fbcb0 2024-05-17T21:51:19.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:19.726+0000 7fbca57fa700 1 -- 172.21.15.107:0/2075542138 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7fbc9c0097d0 con 0x7fbca80fbcb0 2024-05-17T21:51:19.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:19.726+0000 7fbca57fa700 1 --2- 172.21.15.107:0/2075542138 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fbc98069300 0x7fbc9806b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:51:19.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:19.727+0000 7fbca57fa700 1 -- 172.21.15.107:0/2075542138 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fbc9c086b90 con 0x7fbca80fbcb0 2024-05-17T21:51:19.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:19.727+0000 7fbca7fff700 1 --2- 172.21.15.107:0/2075542138 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fbc98069300 0x7fbc9806b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:51:19.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:19.727+0000 7fbca7fff700 1 --2- 172.21.15.107:0/2075542138 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fbc98069300 0x7fbc9806b7b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fbc9400b020 tx=0x7fbc94009e30 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:51:19.725 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:19.732+0000 7fbca57fa700 1 -- 172.21.15.107:0/2075542138 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fbc9c056900 con 0x7fbca80fbcb0 2024-05-17T21:51:19.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:19 smithi107 ceph-mon[28434]: pgmap v237: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T21:51:20.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:19 smithi138 ceph-mon[35948]: pgmap v237: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T21:51:20.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:20.113+0000 7fbcaeec0700 1 -- 172.21.15.107:0/2075542138 --> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbca810e870 con 0x7fbc98069300 2024-05-17T21:51:20.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:20.114+0000 7fbca57fa700 1 -- 172.21.15.107:0/2075542138 <== mgr.14214 v2:172.21.15.107:6800/1294361014 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+447 (secure 0 0 0) 0x7fbca810e870 con 0x7fbc98069300 2024-05-17T21:51:20.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:20.117+0000 7fbcaeec0700 1 -- 172.21.15.107:0/2075542138 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fbc98069300 msgr2=0x7fbc9806b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:51:20.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:20.117+0000 7fbcaeec0700 1 --2- 172.21.15.107:0/2075542138 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fbc98069300 0x7fbc9806b7b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fbc9400b020 tx=0x7fbc94009e30 comp rx=0 tx=0).stop 2024-05-17T21:51:20.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:20.117+0000 7fbcaeec0700 1 -- 172.21.15.107:0/2075542138 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbca80fbcb0 msgr2=0x7fbca810c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:51:20.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:20.117+0000 7fbcaeec0700 1 --2- 172.21.15.107:0/2075542138 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbca80fbcb0 0x7fbca810c8c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c002a80 tx=0x7fbc9c0046b0 comp rx=0 tx=0).stop 2024-05-17T21:51:20.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:20.117+0000 7fbcaeec0700 1 -- 172.21.15.107:0/2075542138 shutdown_connections 2024-05-17T21:51:20.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:20.117+0000 7fbcaeec0700 1 --2- 172.21.15.107:0/2075542138 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fbc98069300 0x7fbc9806b7b0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:51:20.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:20.117+0000 7fbcaeec0700 1 --2- 172.21.15.107:0/2075542138 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbca80fbcb0 0x7fbca810c8c0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:51:20.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:20.117+0000 7fbcaeec0700 1 --2- 172.21.15.107:0/2075542138 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbca80fcf60 0x7fbca810ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:51:20.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:20.117+0000 7fbcaeec0700 1 -- 172.21.15.107:0/2075542138 >> 172.21.15.107:0/2075542138 conn(0x7fbca80f7430 msgr2=0x7fbca80f8ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:51:20.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:20.118+0000 7fbcaeec0700 1 -- 172.21.15.107:0/2075542138 shutdown_connections 2024-05-17T21:51:20.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:20.118+0000 7fbcaeec0700 1 -- 172.21.15.107:0/2075542138 wait complete. 2024-05-17T21:51:20.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:20.279+0000 7f63534c5700 1 -- 172.21.15.107:0/1950446613 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f634c0fed30 msgr2=0x7f634c0fae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:51:20.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:20.279+0000 7f63534c5700 1 --2- 172.21.15.107:0/1950446613 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f634c0fed30 0x7f634c0fae70 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f633c004940 tx=0x7f633c009e30 comp rx=0 tx=0).stop 2024-05-17T21:51:20.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:20.279+0000 7f63534c5700 1 -- 172.21.15.107:0/1950446613 shutdown_connections 2024-05-17T21:51:20.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:20.279+0000 7f63534c5700 1 --2- 172.21.15.107:0/1950446613 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f634c0fed30 0x7f634c0fae70 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:51:20.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:20.279+0000 7f63534c5700 1 --2- 172.21.15.107:0/1950446613 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f634c0fe3e0 0x7f634c0fe7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:51:20.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:20.279+0000 7f63534c5700 1 -- 172.21.15.107:0/1950446613 >> 172.21.15.107:0/1950446613 conn(0x7f634c0f6b80 msgr2=0x7f634c0f8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:51:20.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:20.279+0000 7f63534c5700 1 -- 172.21.15.107:0/1950446613 shutdown_connections 2024-05-17T21:51:20.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:20.279+0000 7f63534c5700 1 -- 172.21.15.107:0/1950446613 wait complete. 2024-05-17T21:51:20.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:20.280+0000 7f63534c5700 1 Processor -- start 2024-05-17T21:51:20.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:20.280+0000 7f63534c5700 1 -- start start 2024-05-17T21:51:20.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:20.280+0000 7f63534c5700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f634c0fe3e0 0x7f634c113dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:51:20.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:20.280+0000 7f63534c5700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f634c0fed30 0x7f634c114300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:51:20.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:20.280+0000 7f63534c5700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f634c114930 con 0x7f634c0fe3e0 2024-05-17T21:51:20.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:20.280+0000 7f63534c5700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f634c1150f0 con 0x7f634c0fed30 2024-05-17T21:51:20.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:20.280+0000 7f6351261700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f634c0fe3e0 0x7f634c113dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:51:20.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:20.281+0000 7f6351261700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f634c0fe3e0 0x7f634c113dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:53172/0 (socket says 172.21.15.107:53172) 2024-05-17T21:51:20.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:20.281+0000 7f6351261700 1 -- 172.21.15.107:0/4142207425 learned_addr learned my addr 172.21.15.107:0/4142207425 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:51:20.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:20.281+0000 7f6350a60700 1 --2- 172.21.15.107:0/4142207425 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f634c0fed30 0x7f634c114300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:51:20.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:20.281+0000 7f6351261700 1 -- 172.21.15.107:0/4142207425 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f634c0fed30 msgr2=0x7f634c114300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:51:20.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:20.281+0000 7f6351261700 1 --2- 172.21.15.107:0/4142207425 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f634c0fed30 0x7f634c114300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:51:20.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:20.281+0000 7f6351261700 1 -- 172.21.15.107:0/4142207425 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f633c005040 con 0x7f634c0fe3e0 2024-05-17T21:51:20.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:20.282+0000 7f6351261700 1 --2- 172.21.15.107:0/4142207425 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f634c0fe3e0 0x7f634c113dc0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f634800eb40 tx=0x7f634800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:51:20.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:20.282+0000 7f6341ffb700 1 -- 172.21.15.107:0/4142207425 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6348007cc0 con 0x7f634c0fe3e0 2024-05-17T21:51:20.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:20.282+0000 7f6341ffb700 1 -- 172.21.15.107:0/4142207425 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f634800f040 con 0x7f634c0fe3e0 2024-05-17T21:51:20.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:20.282+0000 7f6341ffb700 1 -- 172.21.15.107:0/4142207425 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f634800ca10 con 0x7f634c0fe3e0 2024-05-17T21:51:20.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:20.282+0000 7f63534c5700 1 -- 172.21.15.107:0/4142207425 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f634c115320 con 0x7f634c0fe3e0 2024-05-17T21:51:20.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:20.283+0000 7f63534c5700 1 -- 172.21.15.107:0/4142207425 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f634c115860 con 0x7f634c0fe3e0 2024-05-17T21:51:20.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:20.284+0000 7f63534c5700 1 -- 172.21.15.107:0/4142207425 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f634c101b20 con 0x7f634c0fe3e0 2024-05-17T21:51:20.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:20.285+0000 7f6341ffb700 1 -- 172.21.15.107:0/4142207425 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f634800cb70 con 0x7f634c0fe3e0 2024-05-17T21:51:20.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:20.285+0000 7f6341ffb700 1 --2- 172.21.15.107:0/4142207425 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f63380692b0 0x7f633806b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:51:20.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:20.285+0000 7f6341ffb700 1 -- 172.21.15.107:0/4142207425 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f6348086d30 con 0x7f634c0fe3e0 2024-05-17T21:51:20.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:20.286+0000 7f6350a60700 1 --2- 172.21.15.107:0/4142207425 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f63380692b0 0x7f633806b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:51:20.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:20.287+0000 7f6350a60700 1 --2- 172.21.15.107:0/4142207425 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f63380692b0 0x7f633806b760 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f633c000c00 tx=0x7f633c009e30 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:51:20.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:20.294+0000 7f6341ffb700 1 -- 172.21.15.107:0/4142207425 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f634804e250 con 0x7f634c0fe3e0 2024-05-17T21:51:20.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:20.695+0000 7f63534c5700 1 -- 172.21.15.107:0/4142207425 --> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f634c1059f0 con 0x7f63380692b0 2024-05-17T21:51:20.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:20.712+0000 7f6341ffb700 1 -- 172.21.15.107:0/4142207425 <== mgr.14214 v2:172.21.15.107:6800/1294361014 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3410 (secure 0 0 0) 0x7f634c1059f0 con 0x7f63380692b0 2024-05-17T21:51:20.705 INFO:teuthology.orchestra.run.smithi107.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-17T21:51:20.705 INFO:teuthology.orchestra.run.smithi107.stdout:alertmanager.smithi107 smithi107 *:9093,9094 running (6m) 93s ago 7m 25.2M - 0.25.0 c8568f914cd2 a2d817bf64d3 2024-05-17T21:51:20.705 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi107 smithi107 running (7m) 93s ago 7m 6660k - 17.2.7-952-g9903f776 d15a36853796 828741c3a89c 2024-05-17T21:51:20.705 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi138 smithi138 running (6m) 100s ago 6m 6660k - 17.2.7-952-g9903f776 d15a36853796 20e9ccf1fadd 2024-05-17T21:51:20.705 INFO:teuthology.orchestra.run.smithi107.stdout:grafana.smithi107 smithi107 *:3000 running (5m) 93s ago 7m 80.5M - 9.4.7 954c08fa6188 0bc11311fb71 2024-05-17T21:51:20.705 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.drqody smithi107 running (112s) 93s ago 112s 14.5M - 17.2.7-952-g9903f776 d15a36853796 0d851bc912f0 2024-05-17T21:51:20.705 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.idmbgj smithi107 running (119s) 93s ago 119s 10.4M - 17.2.7-952-g9903f776 d15a36853796 1348b09ab29e 2024-05-17T21:51:20.705 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi138.gwbtyd smithi138 running (117s) 100s ago 116s 14.4M - 17.2.7-952-g9903f776 d15a36853796 fd2ba75354c0 2024-05-17T21:51:20.705 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi138.zckrvs smithi138 running (110s) 100s ago 109s 11.2M - 17.2.7-952-g9903f776 d15a36853796 a1cfd2fe4cda 2024-05-17T21:51:20.705 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi107.xhhyiw smithi107 *:9283 running (8m) 93s ago 8m 483M - 17.2.7-952-g9903f776 d15a36853796 b566f08938ae 2024-05-17T21:51:20.706 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi138.napnll smithi138 *:8443,9283 running (6m) 100s ago 6m 436M - 17.2.7-952-g9903f776 d15a36853796 b0a8c6e63a55 2024-05-17T21:51:20.706 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi107 smithi107 running (8m) 93s ago 9m 47.5M 2048M 17.2.7-952-g9903f776 d15a36853796 7de9c23eee97 2024-05-17T21:51:20.706 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi138 smithi138 running (6m) 100s ago 6m 32.4M 2048M 17.2.7-952-g9903f776 d15a36853796 ee61a065778d 2024-05-17T21:51:20.706 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi107 smithi107 *:9100 running (7m) 93s ago 7m 15.4M - 1.5.0 0da6a335fe13 730e9bfe00b2 2024-05-17T21:51:20.706 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi138 smithi138 *:9100 running (6m) 100s ago 6m 16.2M - 1.5.0 0da6a335fe13 296b996124a1 2024-05-17T21:51:20.706 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0 smithi107 running (5m) 93s ago 5m 46.0M 4288M 17.2.7-952-g9903f776 d15a36853796 f2d20009361d 2024-05-17T21:51:20.706 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1 smithi107 running (5m) 93s ago 5m 48.3M 4288M 17.2.7-952-g9903f776 d15a36853796 e5d3e99844a9 2024-05-17T21:51:20.706 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2 smithi107 running (4m) 93s ago 4m 46.5M 4288M 17.2.7-952-g9903f776 d15a36853796 e044d7b78088 2024-05-17T21:51:20.706 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3 smithi138 running (4m) 100s ago 4m 44.5M 5312M 17.2.7-952-g9903f776 d15a36853796 8c980cc8af87 2024-05-17T21:51:20.706 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4 smithi138 running (3m) 100s ago 3m 41.1M 5312M 17.2.7-952-g9903f776 d15a36853796 b306a36a744e 2024-05-17T21:51:20.706 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5 smithi138 running (3m) 100s ago 3m 43.6M 5312M 17.2.7-952-g9903f776 d15a36853796 7c8d8ed74974 2024-05-17T21:51:20.706 INFO:teuthology.orchestra.run.smithi107.stdout:prometheus.smithi107 smithi107 *:9095 running (5m) 93s ago 6m 53.9M - 2.43.0 a07b618ecd1d 38259629b8f2 2024-05-17T21:51:20.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:20.715+0000 7f63534c5700 1 -- 172.21.15.107:0/4142207425 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f63380692b0 msgr2=0x7f633806b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:51:20.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:20.715+0000 7f63534c5700 1 --2- 172.21.15.107:0/4142207425 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f63380692b0 0x7f633806b760 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f633c000c00 tx=0x7f633c009e30 comp rx=0 tx=0).stop 2024-05-17T21:51:20.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:20.715+0000 7f63534c5700 1 -- 172.21.15.107:0/4142207425 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f634c0fe3e0 msgr2=0x7f634c113dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:51:20.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:20.715+0000 7f63534c5700 1 --2- 172.21.15.107:0/4142207425 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f634c0fe3e0 0x7f634c113dc0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f634800eb40 tx=0x7f634800b040 comp rx=0 tx=0).stop 2024-05-17T21:51:20.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:20.715+0000 7f63534c5700 1 -- 172.21.15.107:0/4142207425 shutdown_connections 2024-05-17T21:51:20.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:20.715+0000 7f63534c5700 1 --2- 172.21.15.107:0/4142207425 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f63380692b0 0x7f633806b760 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:51:20.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:20.715+0000 7f63534c5700 1 --2- 172.21.15.107:0/4142207425 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f634c0fe3e0 0x7f634c113dc0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:51:20.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:20.715+0000 7f63534c5700 1 --2- 172.21.15.107:0/4142207425 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f634c0fed30 0x7f634c114300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:51:20.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:20.715+0000 7f63534c5700 1 -- 172.21.15.107:0/4142207425 >> 172.21.15.107:0/4142207425 conn(0x7f634c0f6b80 msgr2=0x7f634c0f8f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:51:20.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:20.715+0000 7f63534c5700 1 -- 172.21.15.107:0/4142207425 shutdown_connections 2024-05-17T21:51:20.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:20.715+0000 7f63534c5700 1 -- 172.21.15.107:0/4142207425 wait complete. 2024-05-17T21:51:20.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:20.858+0000 7f5e43eb3700 1 -- 172.21.15.107:0/72376324 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e3c0fbcc0 msgr2=0x7f5e3c0fc0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:51:20.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:20.858+0000 7f5e43eb3700 1 --2- 172.21.15.107:0/72376324 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e3c0fbcc0 0x7f5e3c0fc0d0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f5e38004940 tx=0x7f5e38009e30 comp rx=0 tx=0).stop 2024-05-17T21:51:20.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:20.858+0000 7f5e43eb3700 1 -- 172.21.15.107:0/72376324 shutdown_connections 2024-05-17T21:51:20.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:20.858+0000 7f5e43eb3700 1 --2- 172.21.15.107:0/72376324 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e3c0fcf70 0x7f5e3c0fd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:51:20.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:20.858+0000 7f5e43eb3700 1 --2- 172.21.15.107:0/72376324 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e3c0fbcc0 0x7f5e3c0fc0d0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:51:20.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:20.858+0000 7f5e43eb3700 1 -- 172.21.15.107:0/72376324 >> 172.21.15.107:0/72376324 conn(0x7f5e3c0f7420 msgr2=0x7f5e3c0f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:51:20.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:20.858+0000 7f5e43eb3700 1 -- 172.21.15.107:0/72376324 shutdown_connections 2024-05-17T21:51:20.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:20.858+0000 7f5e43eb3700 1 -- 172.21.15.107:0/72376324 wait complete. 2024-05-17T21:51:20.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:20.859+0000 7f5e43eb3700 1 Processor -- start 2024-05-17T21:51:20.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:20.859+0000 7f5e43eb3700 1 -- start start 2024-05-17T21:51:20.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:20.859+0000 7f5e43eb3700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e3c0fbcc0 0x7f5e3c1106d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:51:20.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:20.859+0000 7f5e43eb3700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e3c0fcf70 0x7f5e3c112c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:51:20.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:20.859+0000 7f5e43eb3700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3c066b90 con 0x7f5e3c0fbcc0 2024-05-17T21:51:20.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:20.859+0000 7f5e43eb3700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3c066cd0 con 0x7f5e3c0fcf70 2024-05-17T21:51:20.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:20.859+0000 7f5e41c4f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e3c0fbcc0 0x7f5e3c1106d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:51:20.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:20.859+0000 7f5e41c4f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e3c0fbcc0 0x7f5e3c1106d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:53188/0 (socket says 172.21.15.107:53188) 2024-05-17T21:51:20.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:20.860+0000 7f5e41c4f700 1 -- 172.21.15.107:0/322921897 learned_addr learned my addr 172.21.15.107:0/322921897 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:51:20.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:20.860+0000 7f5e4144e700 1 --2- 172.21.15.107:0/322921897 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e3c0fcf70 0x7f5e3c112c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:51:20.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:20.860+0000 7f5e41c4f700 1 -- 172.21.15.107:0/322921897 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e3c0fcf70 msgr2=0x7f5e3c112c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:51:20.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:20.860+0000 7f5e41c4f700 1 --2- 172.21.15.107:0/322921897 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e3c0fcf70 0x7f5e3c112c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:51:20.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:20.860+0000 7f5e41c4f700 1 -- 172.21.15.107:0/322921897 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e38005040 con 0x7f5e3c0fbcc0 2024-05-17T21:51:20.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:20.860+0000 7f5e4144e700 1 --2- 172.21.15.107:0/322921897 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e3c0fcf70 0x7f5e3c112c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:51:20.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:20.860+0000 7f5e41c4f700 1 --2- 172.21.15.107:0/322921897 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e3c0fbcc0 0x7f5e3c1106d0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f5e3800b0d0 tx=0x7f5e38009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:51:20.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:20.861+0000 7f5e327fc700 1 -- 172.21.15.107:0/322921897 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e3800e030 con 0x7f5e3c0fbcc0 2024-05-17T21:51:20.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:20.861+0000 7f5e327fc700 1 -- 172.21.15.107:0/322921897 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5e3801c400 con 0x7f5e3c0fbcc0 2024-05-17T21:51:20.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:20.861+0000 7f5e43eb3700 1 -- 172.21.15.107:0/322921897 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e3c113160 con 0x7f5e3c0fbcc0 2024-05-17T21:51:20.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:20.863+0000 7f5e327fc700 1 -- 172.21.15.107:0/322921897 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e3800b650 con 0x7f5e3c0fbcc0 2024-05-17T21:51:20.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:20.863+0000 7f5e43eb3700 1 -- 172.21.15.107:0/322921897 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e3c1135e0 con 0x7f5e3c0fbcc0 2024-05-17T21:51:20.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:20.865+0000 7f5e43eb3700 1 -- 172.21.15.107:0/322921897 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e3c05f5c0 con 0x7f5e3c0fbcc0 2024-05-17T21:51:20.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:20.865+0000 7f5e327fc700 1 -- 172.21.15.107:0/322921897 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f5e3800b8c0 con 0x7f5e3c0fbcc0 2024-05-17T21:51:20.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:20.865+0000 7f5e327fc700 1 --2- 172.21.15.107:0/322921897 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f5e280692a0 0x7f5e2806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:51:20.859 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:20.866+0000 7f5e4144e700 1 --2- 172.21.15.107:0/322921897 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f5e280692a0 0x7f5e2806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:51:20.859 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:20.866+0000 7f5e327fc700 1 -- 172.21.15.107:0/322921897 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f5e38059800 con 0x7f5e3c0fbcc0 2024-05-17T21:51:20.859 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:20.867+0000 7f5e4144e700 1 --2- 172.21.15.107:0/322921897 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f5e280692a0 0x7f5e2806b750 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c004600 tx=0x7f5e2c00b040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:51:20.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:20.874+0000 7f5e327fc700 1 -- 172.21.15.107:0/322921897 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f5e3808d060 con 0x7f5e3c0fbcc0 2024-05-17T21:51:21.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:20 smithi138 ceph-mon[35948]: from='client.24351 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:51:21.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:20 smithi107 ceph-mon[28434]: from='client.24351 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:51:21.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:21.225+0000 7f5e43eb3700 1 -- 172.21.15.107:0/322921897 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f5e3c1047b0 con 0x7f5e3c0fbcc0 2024-05-17T21:51:21.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:21.226+0000 7f5e327fc700 1 -- 172.21.15.107:0/322921897 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+638 (secure 0 0 0) 0x7f5e38055440 con 0x7f5e3c0fbcc0 2024-05-17T21:51:21.219 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-05-17T21:51:21.219 INFO:teuthology.orchestra.run.smithi107.stdout: "mon": { 2024-05-17T21:51:21.219 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-952-g9903f776 (9903f7767386a9d9efadb7c1effdcd3064d71804) quincy (stable)": 2 2024-05-17T21:51:21.219 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-05-17T21:51:21.219 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr": { 2024-05-17T21:51:21.219 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-952-g9903f776 (9903f7767386a9d9efadb7c1effdcd3064d71804) quincy (stable)": 2 2024-05-17T21:51:21.219 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-05-17T21:51:21.219 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": { 2024-05-17T21:51:21.219 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-952-g9903f776 (9903f7767386a9d9efadb7c1effdcd3064d71804) quincy (stable)": 6 2024-05-17T21:51:21.220 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-05-17T21:51:21.220 INFO:teuthology.orchestra.run.smithi107.stdout: "mds": { 2024-05-17T21:51:21.220 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-952-g9903f776 (9903f7767386a9d9efadb7c1effdcd3064d71804) quincy (stable)": 4 2024-05-17T21:51:21.220 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-05-17T21:51:21.220 INFO:teuthology.orchestra.run.smithi107.stdout: "overall": { 2024-05-17T21:51:21.220 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-952-g9903f776 (9903f7767386a9d9efadb7c1effdcd3064d71804) quincy (stable)": 14 2024-05-17T21:51:21.220 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-05-17T21:51:21.220 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-05-17T21:51:21.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:21.229+0000 7f5e43eb3700 1 -- 172.21.15.107:0/322921897 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f5e280692a0 msgr2=0x7f5e2806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:51:21.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:21.229+0000 7f5e43eb3700 1 --2- 172.21.15.107:0/322921897 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f5e280692a0 0x7f5e2806b750 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c004600 tx=0x7f5e2c00b040 comp rx=0 tx=0).stop 2024-05-17T21:51:21.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:21.229+0000 7f5e43eb3700 1 -- 172.21.15.107:0/322921897 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e3c0fbcc0 msgr2=0x7f5e3c1106d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:51:21.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:21.229+0000 7f5e43eb3700 1 --2- 172.21.15.107:0/322921897 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e3c0fbcc0 0x7f5e3c1106d0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f5e3800b0d0 tx=0x7f5e38009e30 comp rx=0 tx=0).stop 2024-05-17T21:51:21.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:21.229+0000 7f5e43eb3700 1 -- 172.21.15.107:0/322921897 shutdown_connections 2024-05-17T21:51:21.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:21.229+0000 7f5e43eb3700 1 --2- 172.21.15.107:0/322921897 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f5e280692a0 0x7f5e2806b750 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:51:21.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:21.229+0000 7f5e43eb3700 1 --2- 172.21.15.107:0/322921897 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e3c0fbcc0 0x7f5e3c1106d0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:51:21.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:21.229+0000 7f5e43eb3700 1 --2- 172.21.15.107:0/322921897 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e3c0fcf70 0x7f5e3c112c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:51:21.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:21.229+0000 7f5e43eb3700 1 -- 172.21.15.107:0/322921897 >> 172.21.15.107:0/322921897 conn(0x7f5e3c0f7420 msgr2=0x7f5e3c0f8a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:51:21.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:21.229+0000 7f5e43eb3700 1 -- 172.21.15.107:0/322921897 shutdown_connections 2024-05-17T21:51:21.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:21.229+0000 7f5e43eb3700 1 -- 172.21.15.107:0/322921897 wait complete. 2024-05-17T21:51:21.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:21.333+0000 7f561039e700 1 -- 172.21.15.107:0/489892275 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56080662c0 msgr2=0x7f56080ff9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:51:21.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:21.333+0000 7f561039e700 1 --2- 172.21.15.107:0/489892275 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56080662c0 0x7f56080ff9a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f5604004940 tx=0x7f5604009e30 comp rx=0 tx=0).stop 2024-05-17T21:51:21.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:21.333+0000 7f561039e700 1 -- 172.21.15.107:0/489892275 shutdown_connections 2024-05-17T21:51:21.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:21.333+0000 7f561039e700 1 --2- 172.21.15.107:0/489892275 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56080ffee0 0x7f5608102300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:51:21.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:21.333+0000 7f561039e700 1 --2- 172.21.15.107:0/489892275 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56080662c0 0x7f56080ff9a0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:51:21.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:21.333+0000 7f561039e700 1 -- 172.21.15.107:0/489892275 >> 172.21.15.107:0/489892275 conn(0x7f56080f7430 msgr2=0x7f56080f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:51:21.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:21.333+0000 7f561039e700 1 -- 172.21.15.107:0/489892275 shutdown_connections 2024-05-17T21:51:21.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:21.333+0000 7f561039e700 1 -- 172.21.15.107:0/489892275 wait complete. 2024-05-17T21:51:21.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:21.334+0000 7f561039e700 1 Processor -- start 2024-05-17T21:51:21.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:21.334+0000 7f561039e700 1 -- start start 2024-05-17T21:51:21.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:21.334+0000 7f561039e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56080662c0 0x7f56080fd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:51:21.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:21.334+0000 7f561039e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56080ffee0 0x7f56080fb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:51:21.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:21.334+0000 7f561039e700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56080fbe50 con 0x7f56080662c0 2024-05-17T21:51:21.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:21.334+0000 7f561039e700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56080fbfc0 con 0x7f56080ffee0 2024-05-17T21:51:21.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:21.334+0000 7f560d939700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56080ffee0 0x7f56080fb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:51:21.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:21.334+0000 7f560e13a700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56080662c0 0x7f56080fd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:51:21.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:21.334+0000 7f560d939700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56080ffee0 0x7f56080fb8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.107:48716/0 (socket says 172.21.15.107:48716) 2024-05-17T21:51:21.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:21.334+0000 7f560e13a700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56080662c0 0x7f56080fd3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:53210/0 (socket says 172.21.15.107:53210) 2024-05-17T21:51:21.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:21.334+0000 7f560d939700 1 -- 172.21.15.107:0/3299882437 learned_addr learned my addr 172.21.15.107:0/3299882437 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:51:21.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:21.335+0000 7f560e13a700 1 -- 172.21.15.107:0/3299882437 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56080ffee0 msgr2=0x7f56080fb8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:51:21.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:21.335+0000 7f560e13a700 1 --2- 172.21.15.107:0/3299882437 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56080ffee0 0x7f56080fb8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:51:21.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:21.335+0000 7f560e13a700 1 -- 172.21.15.107:0/3299882437 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5604005040 con 0x7f56080662c0 2024-05-17T21:51:21.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:21.335+0000 7f560e13a700 1 --2- 172.21.15.107:0/3299882437 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56080662c0 0x7f56080fd3d0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f5604000c00 tx=0x7f5604009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:51:21.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:21.335+0000 7f55feffd700 1 -- 172.21.15.107:0/3299882437 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f560400e040 con 0x7f56080662c0 2024-05-17T21:51:21.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:21.335+0000 7f561039e700 1 -- 172.21.15.107:0/3299882437 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56080fc240 con 0x7f56080662c0 2024-05-17T21:51:21.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:21.335+0000 7f55feffd700 1 -- 172.21.15.107:0/3299882437 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f560400bbf0 con 0x7f56080662c0 2024-05-17T21:51:21.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:21.335+0000 7f55feffd700 1 -- 172.21.15.107:0/3299882437 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5604013660 con 0x7f56080662c0 2024-05-17T21:51:21.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:21.335+0000 7f561039e700 1 -- 172.21.15.107:0/3299882437 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5608115a50 con 0x7f56080662c0 2024-05-17T21:51:21.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:21.336+0000 7f561039e700 1 -- 172.21.15.107:0/3299882437 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f560805f5c0 con 0x7f56080662c0 2024-05-17T21:51:21.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:21.337+0000 7f55feffd700 1 -- 172.21.15.107:0/3299882437 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f560400bd60 con 0x7f56080662c0 2024-05-17T21:51:21.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:21.337+0000 7f55feffd700 1 --2- 172.21.15.107:0/3299882437 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f55f40692a0 0x7f55f406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:51:21.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:21.338+0000 7f55feffd700 1 -- 172.21.15.107:0/3299882437 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f56040884e0 con 0x7f56080662c0 2024-05-17T21:51:21.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:21.338+0000 7f560d939700 1 --2- 172.21.15.107:0/3299882437 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f55f40692a0 0x7f55f406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:51:21.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:21.338+0000 7f560d939700 1 --2- 172.21.15.107:0/3299882437 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f55f40692a0 0x7f55f406b750 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f55f80041d0 tx=0x7f55f800b040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:51:21.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:21.342+0000 7f55feffd700 1 -- 172.21.15.107:0/3299882437 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f5604054860 con 0x7f56080662c0 2024-05-17T21:51:21.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:21.615+0000 7f561039e700 1 -- 172.21.15.107:0/3299882437 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f5608115da0 con 0x7f56080662c0 2024-05-17T21:51:21.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:21.615+0000 7f55feffd700 1 -- 172.21.15.107:0/3299882437 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1840 (secure 0 0 0) 0x7f5604057e80 con 0x7f56080662c0 2024-05-17T21:51:21.608 INFO:teuthology.orchestra.run.smithi107.stdout:e15 2024-05-17T21:51:21.608 INFO:teuthology.orchestra.run.smithi107.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-17T21:51:21.609 INFO:teuthology.orchestra.run.smithi107.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-17T21:51:21.609 INFO:teuthology.orchestra.run.smithi107.stdout:legacy client fscid: 1 2024-05-17T21:51:21.609 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:51:21.609 INFO:teuthology.orchestra.run.smithi107.stdout:Filesystem 'cephfs' (1) 2024-05-17T21:51:21.609 INFO:teuthology.orchestra.run.smithi107.stdout:fs_name cephfs 2024-05-17T21:51:21.609 INFO:teuthology.orchestra.run.smithi107.stdout:epoch 15 2024-05-17T21:51:21.609 INFO:teuthology.orchestra.run.smithi107.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-17T21:51:21.609 INFO:teuthology.orchestra.run.smithi107.stdout:created 2024-05-17T21:49:16.836333+0000 2024-05-17T21:51:21.609 INFO:teuthology.orchestra.run.smithi107.stdout:modified 2024-05-17T21:49:34.873628+0000 2024-05-17T21:51:21.609 INFO:teuthology.orchestra.run.smithi107.stdout:tableserver 0 2024-05-17T21:51:21.609 INFO:teuthology.orchestra.run.smithi107.stdout:root 0 2024-05-17T21:51:21.609 INFO:teuthology.orchestra.run.smithi107.stdout:session_timeout 60 2024-05-17T21:51:21.609 INFO:teuthology.orchestra.run.smithi107.stdout:session_autoclose 300 2024-05-17T21:51:21.610 INFO:teuthology.orchestra.run.smithi107.stdout:max_file_size 1099511627776 2024-05-17T21:51:21.610 INFO:teuthology.orchestra.run.smithi107.stdout:required_client_features {} 2024-05-17T21:51:21.610 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure 0 2024-05-17T21:51:21.610 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure_osd_epoch 46 2024-05-17T21:51:21.610 INFO:teuthology.orchestra.run.smithi107.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-17T21:51:21.610 INFO:teuthology.orchestra.run.smithi107.stdout:max_mds 1 2024-05-17T21:51:21.610 INFO:teuthology.orchestra.run.smithi107.stdout:in 0 2024-05-17T21:51:21.610 INFO:teuthology.orchestra.run.smithi107.stdout:up {0=24259} 2024-05-17T21:51:21.610 INFO:teuthology.orchestra.run.smithi107.stdout:failed 2024-05-17T21:51:21.610 INFO:teuthology.orchestra.run.smithi107.stdout:damaged 2024-05-17T21:51:21.610 INFO:teuthology.orchestra.run.smithi107.stdout:stopped 2024-05-17T21:51:21.610 INFO:teuthology.orchestra.run.smithi107.stdout:data_pools [3] 2024-05-17T21:51:21.611 INFO:teuthology.orchestra.run.smithi107.stdout:metadata_pool 2 2024-05-17T21:51:21.611 INFO:teuthology.orchestra.run.smithi107.stdout:inline_data enabled 2024-05-17T21:51:21.611 INFO:teuthology.orchestra.run.smithi107.stdout:balancer 2024-05-17T21:51:21.611 INFO:teuthology.orchestra.run.smithi107.stdout:standby_count_wanted 1 2024-05-17T21:51:21.611 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi138.gwbtyd{0:24259} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.138:6824/2779634919,v1:172.21.15.138:6825/2779634919] compat {c=[1],r=[1],i=[7ff]}] 2024-05-17T21:51:21.611 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.drqody{0:14486} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.107:6828/2275807100,v1:172.21.15.107:6829/2275807100] compat {c=[1],r=[1],i=[7ff]}] 2024-05-17T21:51:21.611 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:51:21.611 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:51:21.611 INFO:teuthology.orchestra.run.smithi107.stdout:Standby daemons: 2024-05-17T21:51:21.611 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:51:21.611 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.idmbgj{-1:14498} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.107:6826/1460186079,v1:172.21.15.107:6827/1460186079] compat {c=[1],r=[1],i=[7ff]}] 2024-05-17T21:51:21.611 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi138.zckrvs{-1:24267} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.138:6826/170628661,v1:172.21.15.138:6827/170628661] compat {c=[1],r=[1],i=[7ff]}] 2024-05-17T21:51:21.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:21.618+0000 7f561039e700 1 -- 172.21.15.107:0/3299882437 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f55f40692a0 msgr2=0x7f55f406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:51:21.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:21.618+0000 7f561039e700 1 --2- 172.21.15.107:0/3299882437 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f55f40692a0 0x7f55f406b750 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f55f80041d0 tx=0x7f55f800b040 comp rx=0 tx=0).stop 2024-05-17T21:51:21.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:21.618+0000 7f561039e700 1 -- 172.21.15.107:0/3299882437 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56080662c0 msgr2=0x7f56080fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:51:21.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:21.618+0000 7f561039e700 1 --2- 172.21.15.107:0/3299882437 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56080662c0 0x7f56080fd3d0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f5604000c00 tx=0x7f5604009e30 comp rx=0 tx=0).stop 2024-05-17T21:51:21.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:21.618+0000 7f561039e700 1 -- 172.21.15.107:0/3299882437 shutdown_connections 2024-05-17T21:51:21.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:21.618+0000 7f561039e700 1 --2- 172.21.15.107:0/3299882437 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7f55f40692a0 0x7f55f406b750 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:51:21.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:21.618+0000 7f561039e700 1 --2- 172.21.15.107:0/3299882437 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56080662c0 0x7f56080fd3d0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:51:21.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:21.618+0000 7f561039e700 1 --2- 172.21.15.107:0/3299882437 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56080ffee0 0x7f56080fb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:51:21.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:21.618+0000 7f561039e700 1 -- 172.21.15.107:0/3299882437 >> 172.21.15.107:0/3299882437 conn(0x7f56080f7430 msgr2=0x7f56080f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:51:21.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:21.618+0000 7f561039e700 1 -- 172.21.15.107:0/3299882437 shutdown_connections 2024-05-17T21:51:21.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:21.618+0000 7f561039e700 1 -- 172.21.15.107:0/3299882437 wait complete. 2024-05-17T21:51:21.613 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 15 2024-05-17T21:51:21.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:21.726+0000 7eff841f0700 1 -- 172.21.15.107:0/3952536038 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff7c0ffee0 msgr2=0x7eff7c102300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:51:21.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:21.726+0000 7eff841f0700 1 --2- 172.21.15.107:0/3952536038 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff7c0ffee0 0x7eff7c102300 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7eff6c004940 tx=0x7eff6c009e30 comp rx=0 tx=0).stop 2024-05-17T21:51:21.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:21.726+0000 7eff841f0700 1 -- 172.21.15.107:0/3952536038 shutdown_connections 2024-05-17T21:51:21.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:21.726+0000 7eff841f0700 1 --2- 172.21.15.107:0/3952536038 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff7c0ffee0 0x7eff7c102300 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:51:21.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:21.726+0000 7eff841f0700 1 --2- 172.21.15.107:0/3952536038 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff7c0662c0 0x7eff7c0ff9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:51:21.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:21.726+0000 7eff841f0700 1 -- 172.21.15.107:0/3952536038 >> 172.21.15.107:0/3952536038 conn(0x7eff7c0f7430 msgr2=0x7eff7c0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:51:21.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:21.726+0000 7eff841f0700 1 -- 172.21.15.107:0/3952536038 shutdown_connections 2024-05-17T21:51:21.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:21.726+0000 7eff841f0700 1 -- 172.21.15.107:0/3952536038 wait complete. 2024-05-17T21:51:21.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:21.727+0000 7eff841f0700 1 Processor -- start 2024-05-17T21:51:21.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:21.727+0000 7eff841f0700 1 -- start start 2024-05-17T21:51:21.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:21.727+0000 7eff841f0700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff7c0662c0 0x7eff7c0fd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:51:21.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:21.727+0000 7eff841f0700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff7c0ffee0 0x7eff7c0fb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:51:21.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:21.727+0000 7eff841f0700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff7c0fbe50 con 0x7eff7c0ffee0 2024-05-17T21:51:21.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:21.727+0000 7eff841f0700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff7c0fbfc0 con 0x7eff7c0662c0 2024-05-17T21:51:21.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:21.727+0000 7eff8178b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff7c0ffee0 0x7eff7c0fb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:51:21.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:21.728+0000 7eff8178b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff7c0ffee0 0x7eff7c0fb8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:53230/0 (socket says 172.21.15.107:53230) 2024-05-17T21:51:21.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:21.728+0000 7eff8178b700 1 -- 172.21.15.107:0/887353621 learned_addr learned my addr 172.21.15.107:0/887353621 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:51:21.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:21.728+0000 7eff81f8c700 1 --2- 172.21.15.107:0/887353621 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff7c0662c0 0x7eff7c0fd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:51:21.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:21.728+0000 7eff8178b700 1 -- 172.21.15.107:0/887353621 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff7c0662c0 msgr2=0x7eff7c0fd3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:51:21.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:21.728+0000 7eff8178b700 1 --2- 172.21.15.107:0/887353621 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff7c0662c0 0x7eff7c0fd3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:51:21.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:21.728+0000 7eff8178b700 1 -- 172.21.15.107:0/887353621 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff6c005040 con 0x7eff7c0ffee0 2024-05-17T21:51:21.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:21.728+0000 7eff8178b700 1 --2- 172.21.15.107:0/887353621 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff7c0ffee0 0x7eff7c0fb8a0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7eff6c00b3b0 tx=0x7eff6c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:51:21.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:21.728+0000 7eff727fc700 1 -- 172.21.15.107:0/887353621 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7eff6c00e040 con 0x7eff7c0ffee0 2024-05-17T21:51:21.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:21.728+0000 7eff727fc700 1 -- 172.21.15.107:0/887353621 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7eff6c004290 con 0x7eff7c0ffee0 2024-05-17T21:51:21.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:21.728+0000 7eff841f0700 1 -- 172.21.15.107:0/887353621 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff7c0fc240 con 0x7eff7c0ffee0 2024-05-17T21:51:21.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:21.729+0000 7eff841f0700 1 -- 172.21.15.107:0/887353621 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff7c1159e0 con 0x7eff7c0ffee0 2024-05-17T21:51:21.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:21.729+0000 7eff727fc700 1 -- 172.21.15.107:0/887353621 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7eff6c01c630 con 0x7eff7c0ffee0 2024-05-17T21:51:21.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:21.730+0000 7eff841f0700 1 -- 172.21.15.107:0/887353621 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff7c05f5c0 con 0x7eff7c0ffee0 2024-05-17T21:51:21.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:21.731+0000 7eff727fc700 1 -- 172.21.15.107:0/887353621 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7eff6c01c790 con 0x7eff7c0ffee0 2024-05-17T21:51:21.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:21.731+0000 7eff727fc700 1 --2- 172.21.15.107:0/887353621 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7eff68069360 0x7eff6806b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:51:21.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:21.731+0000 7eff727fc700 1 -- 172.21.15.107:0/887353621 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7eff6c088570 con 0x7eff7c0ffee0 2024-05-17T21:51:21.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:21.732+0000 7eff81f8c700 1 --2- 172.21.15.107:0/887353621 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7eff68069360 0x7eff6806b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:51:21.725 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:21.732+0000 7eff81f8c700 1 --2- 172.21.15.107:0/887353621 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7eff68069360 0x7eff6806b810 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7eff78009fd0 tx=0x7eff78009f60 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:51:21.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:21.736+0000 7eff727fc700 1 -- 172.21.15.107:0/887353621 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7eff6c055880 con 0x7eff7c0ffee0 2024-05-17T21:51:21.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:21.949+0000 7eff841f0700 1 -- 172.21.15.107:0/887353621 --> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7eff7c0f8f20 con 0x7eff68069360 2024-05-17T21:51:21.942 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:21 smithi107 ceph-mon[28434]: from='client.24355 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:51:21.942 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:21 smithi107 ceph-mon[28434]: pgmap v238: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T21:51:21.942 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:21 smithi107 ceph-mon[28434]: from='client.14616 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:51:21.943 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:21 smithi107 ceph-mon[28434]: from='client.? 172.21.15.107:0/322921897' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:51:21.943 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:21 smithi107 ceph-mon[28434]: from='client.? 172.21.15.107:0/3299882437' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-17T21:51:21.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:21.951+0000 7eff727fc700 1 -- 172.21.15.107:0/887353621 <== mgr.14214 v2:172.21.15.107:6800/1294361014 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+447 (secure 0 0 0) 0x7eff7c0f8f20 con 0x7eff68069360 2024-05-17T21:51:21.943 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-05-17T21:51:21.944 INFO:teuthology.orchestra.run.smithi107.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b", 2024-05-17T21:51:21.944 INFO:teuthology.orchestra.run.smithi107.stdout: "in_progress": true, 2024-05-17T21:51:21.944 INFO:teuthology.orchestra.run.smithi107.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-17T21:51:21.944 INFO:teuthology.orchestra.run.smithi107.stdout: "services_complete": [], 2024-05-17T21:51:21.944 INFO:teuthology.orchestra.run.smithi107.stdout: "progress": "0/21 daemons upgraded", 2024-05-17T21:51:21.944 INFO:teuthology.orchestra.run.smithi107.stdout: "message": "Pulling quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b image on host smithi138", 2024-05-17T21:51:21.944 INFO:teuthology.orchestra.run.smithi107.stdout: "is_paused": false 2024-05-17T21:51:21.944 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-05-17T21:51:21.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:21.953+0000 7eff841f0700 1 -- 172.21.15.107:0/887353621 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7eff68069360 msgr2=0x7eff6806b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:51:21.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:21.953+0000 7eff841f0700 1 --2- 172.21.15.107:0/887353621 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7eff68069360 0x7eff6806b810 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7eff78009fd0 tx=0x7eff78009f60 comp rx=0 tx=0).stop 2024-05-17T21:51:21.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:21.953+0000 7eff841f0700 1 -- 172.21.15.107:0/887353621 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff7c0ffee0 msgr2=0x7eff7c0fb8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:51:21.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:21.953+0000 7eff841f0700 1 --2- 172.21.15.107:0/887353621 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff7c0ffee0 0x7eff7c0fb8a0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7eff6c00b3b0 tx=0x7eff6c009e30 comp rx=0 tx=0).stop 2024-05-17T21:51:21.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:21.953+0000 7eff841f0700 1 -- 172.21.15.107:0/887353621 shutdown_connections 2024-05-17T21:51:21.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:21.953+0000 7eff841f0700 1 --2- 172.21.15.107:0/887353621 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7eff68069360 0x7eff6806b810 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:51:21.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:21.953+0000 7eff841f0700 1 --2- 172.21.15.107:0/887353621 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff7c0662c0 0x7eff7c0fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:51:21.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:21.953+0000 7eff841f0700 1 --2- 172.21.15.107:0/887353621 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff7c0ffee0 0x7eff7c0fb8a0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:51:21.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:21.953+0000 7eff841f0700 1 -- 172.21.15.107:0/887353621 >> 172.21.15.107:0/887353621 conn(0x7eff7c0f7430 msgr2=0x7eff7c0f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:51:21.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:21.953+0000 7eff841f0700 1 -- 172.21.15.107:0/887353621 shutdown_connections 2024-05-17T21:51:21.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:21.953+0000 7eff841f0700 1 -- 172.21.15.107:0/887353621 wait complete. 2024-05-17T21:51:22.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:21 smithi138 ceph-mon[35948]: from='client.24355 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:51:22.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:21 smithi138 ceph-mon[35948]: pgmap v238: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T21:51:22.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:21 smithi138 ceph-mon[35948]: from='client.14616 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:51:22.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:21 smithi138 ceph-mon[35948]: from='client.? 172.21.15.107:0/322921897' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:51:22.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:21 smithi138 ceph-mon[35948]: from='client.? 172.21.15.107:0/3299882437' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-17T21:51:22.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:22.092+0000 7fd2176be700 1 -- 172.21.15.107:0/2545365760 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2100fe4a0 msgr2=0x7fd2100fe8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:51:22.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:22.092+0000 7fd2176be700 1 --2- 172.21.15.107:0/2545365760 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2100fe4a0 0x7fd2100fe8b0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd20c004940 tx=0x7fd20c009e30 comp rx=0 tx=0).stop 2024-05-17T21:51:22.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:22.092+0000 7fd2176be700 1 -- 172.21.15.107:0/2545365760 shutdown_connections 2024-05-17T21:51:22.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:22.092+0000 7fd2176be700 1 --2- 172.21.15.107:0/2545365760 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2100fedf0 0x7fd2100faf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:51:22.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:22.092+0000 7fd2176be700 1 --2- 172.21.15.107:0/2545365760 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2100fe4a0 0x7fd2100fe8b0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:51:22.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:22.092+0000 7fd2176be700 1 -- 172.21.15.107:0/2545365760 >> 172.21.15.107:0/2545365760 conn(0x7fd2100f6b80 msgr2=0x7fd2100f8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:51:22.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:22.092+0000 7fd2176be700 1 -- 172.21.15.107:0/2545365760 shutdown_connections 2024-05-17T21:51:22.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:22.092+0000 7fd2176be700 1 -- 172.21.15.107:0/2545365760 wait complete. 2024-05-17T21:51:22.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:22.093+0000 7fd2176be700 1 Processor -- start 2024-05-17T21:51:22.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:22.093+0000 7fd2176be700 1 -- start start 2024-05-17T21:51:22.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:22.093+0000 7fd2176be700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2100fe4a0 0x7fd2101070a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:51:22.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:22.093+0000 7fd2176be700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2100fedf0 0x7fd2101075e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:51:22.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:22.093+0000 7fd2176be700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd210107c30 con 0x7fd2100fe4a0 2024-05-17T21:51:22.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:22.093+0000 7fd2176be700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd210108480 con 0x7fd2100fedf0 2024-05-17T21:51:22.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:22.093+0000 7fd21545a700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2100fe4a0 0x7fd2101070a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:51:22.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:22.093+0000 7fd21545a700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2100fe4a0 0x7fd2101070a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:53260/0 (socket says 172.21.15.107:53260) 2024-05-17T21:51:22.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:22.093+0000 7fd21545a700 1 -- 172.21.15.107:0/4127870851 learned_addr learned my addr 172.21.15.107:0/4127870851 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:51:22.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:22.093+0000 7fd214c59700 1 --2- 172.21.15.107:0/4127870851 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2100fedf0 0x7fd2101075e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:51:22.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:22.093+0000 7fd21545a700 1 -- 172.21.15.107:0/4127870851 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2100fedf0 msgr2=0x7fd2101075e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:51:22.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:22.094+0000 7fd21545a700 1 --2- 172.21.15.107:0/4127870851 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2100fedf0 0x7fd2101075e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:51:22.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:22.094+0000 7fd21545a700 1 -- 172.21.15.107:0/4127870851 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd20c005040 con 0x7fd2100fe4a0 2024-05-17T21:51:22.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:22.094+0000 7fd21545a700 1 --2- 172.21.15.107:0/4127870851 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2100fe4a0 0x7fd2101070a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd20c013870 tx=0x7fd20c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:51:22.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:22.094+0000 7fd205ffb700 1 -- 172.21.15.107:0/4127870851 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd20c021070 con 0x7fd2100fe4a0 2024-05-17T21:51:22.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:22.094+0000 7fd205ffb700 1 -- 172.21.15.107:0/4127870851 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd20c009430 con 0x7fd2100fe4a0 2024-05-17T21:51:22.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:22.094+0000 7fd2176be700 1 -- 172.21.15.107:0/4127870851 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd210108700 con 0x7fd2100fe4a0 2024-05-17T21:51:22.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:22.094+0000 7fd205ffb700 1 -- 172.21.15.107:0/4127870851 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd20c00e070 con 0x7fd2100fe4a0 2024-05-17T21:51:22.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:22.094+0000 7fd2176be700 1 -- 172.21.15.107:0/4127870851 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd210108c40 con 0x7fd2100fe4a0 2024-05-17T21:51:22.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:22.096+0000 7fd2176be700 1 -- 172.21.15.107:0/4127870851 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd210101be0 con 0x7fd2100fe4a0 2024-05-17T21:51:22.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:22.096+0000 7fd205ffb700 1 -- 172.21.15.107:0/4127870851 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7fd20c01c750 con 0x7fd2100fe4a0 2024-05-17T21:51:22.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:22.097+0000 7fd205ffb700 1 --2- 172.21.15.107:0/4127870851 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fd1fc0692a0 0x7fd1fc06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:51:22.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:22.097+0000 7fd205ffb700 1 -- 172.21.15.107:0/4127870851 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fd20c0883f0 con 0x7fd2100fe4a0 2024-05-17T21:51:22.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:22.098+0000 7fd214c59700 1 --2- 172.21.15.107:0/4127870851 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fd1fc0692a0 0x7fd1fc06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:51:22.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:22.098+0000 7fd214c59700 1 --2- 172.21.15.107:0/4127870851 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fd1fc0692a0 0x7fd1fc06b750 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd200009b80 tx=0x7fd20000b040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:51:22.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:22.105+0000 7fd205ffb700 1 -- 172.21.15.107:0/4127870851 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fd20c054770 con 0x7fd2100fe4a0 2024-05-17T21:51:22.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:22.382+0000 7fd2176be700 1 -- 172.21.15.107:0/4127870851 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fd210105950 con 0x7fd2100fe4a0 2024-05-17T21:51:22.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:22.382+0000 7fd205ffb700 1 -- 172.21.15.107:0/4127870851 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7fd20c057d90 con 0x7fd2100fe4a0 2024-05-17T21:51:22.376 INFO:teuthology.orchestra.run.smithi107.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-17T21:51:22.376 INFO:teuthology.orchestra.run.smithi107.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-17T21:51:22.376 INFO:teuthology.orchestra.run.smithi107.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-17T21:51:22.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:22.385+0000 7fd2176be700 1 -- 172.21.15.107:0/4127870851 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fd1fc0692a0 msgr2=0x7fd1fc06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:51:22.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:22.385+0000 7fd2176be700 1 --2- 172.21.15.107:0/4127870851 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fd1fc0692a0 0x7fd1fc06b750 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd200009b80 tx=0x7fd20000b040 comp rx=0 tx=0).stop 2024-05-17T21:51:22.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:22.385+0000 7fd2176be700 1 -- 172.21.15.107:0/4127870851 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2100fe4a0 msgr2=0x7fd2101070a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:51:22.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:22.385+0000 7fd2176be700 1 --2- 172.21.15.107:0/4127870851 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2100fe4a0 0x7fd2101070a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd20c013870 tx=0x7fd20c009e30 comp rx=0 tx=0).stop 2024-05-17T21:51:22.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:22.385+0000 7fd2176be700 1 -- 172.21.15.107:0/4127870851 shutdown_connections 2024-05-17T21:51:22.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:22.385+0000 7fd2176be700 1 --2- 172.21.15.107:0/4127870851 >> [v2:172.21.15.107:6800/1294361014,v1:172.21.15.107:6801/1294361014] conn(0x7fd1fc0692a0 0x7fd1fc06b750 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:51:22.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:22.385+0000 7fd2176be700 1 --2- 172.21.15.107:0/4127870851 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2100fe4a0 0x7fd2101070a0 secure :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd20c013870 tx=0x7fd20c009e30 comp rx=0 tx=0).stop 2024-05-17T21:51:22.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:22.385+0000 7fd2176be700 1 --2- 172.21.15.107:0/4127870851 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2100fedf0 0x7fd2101075e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:51:22.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:22.385+0000 7fd2176be700 1 -- 172.21.15.107:0/4127870851 >> 172.21.15.107:0/4127870851 conn(0x7fd2100f6b80 msgr2=0x7fd2100f8120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:51:22.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:22.385+0000 7fd2176be700 1 -- 172.21.15.107:0/4127870851 shutdown_connections 2024-05-17T21:51:22.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:22.385+0000 7fd2176be700 1 -- 172.21.15.107:0/4127870851 wait complete. 2024-05-17T21:51:23.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:22 smithi138 ceph-mon[35948]: from='client.? 172.21.15.107:0/4127870851' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-17T21:51:23.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:22 smithi107 ceph-mon[28434]: from='client.? 172.21.15.107:0/4127870851' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-17T21:51:24.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:23 smithi138 ceph-mon[35948]: from='client.14628 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:51:24.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:23 smithi138 ceph-mon[35948]: pgmap v239: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T21:51:24.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:23 smithi107 ceph-mon[28434]: from='client.14628 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:51:24.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:23 smithi107 ceph-mon[28434]: pgmap v239: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T21:51:26.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:25 smithi138 ceph-mon[35948]: pgmap v240: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T21:51:26.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:25 smithi107 ceph-mon[28434]: pgmap v240: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T21:51:28.018 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:27 smithi138 ceph-mon[35948]: pgmap v241: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T21:51:28.019 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:27 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:51:28.019 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:27 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi138.napnll", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-17T21:51:28.019 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:27 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-17T21:51:28.019 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:27 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:51:28.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:27 smithi107 ceph-mon[28434]: pgmap v241: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T21:51:28.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:27 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:51:28.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:27 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi138.napnll", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-17T21:51:28.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:27 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-17T21:51:28.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:27 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:51:29.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:28 smithi138 ceph-mon[35948]: Upgrade: Updating mgr.smithi138.napnll 2024-05-17T21:51:29.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:28 smithi138 ceph-mon[35948]: Deploying daemon mgr.smithi138.napnll on smithi138 2024-05-17T21:51:29.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:28 smithi107 ceph-mon[28434]: Upgrade: Updating mgr.smithi138.napnll 2024-05-17T21:51:29.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:28 smithi107 ceph-mon[28434]: Deploying daemon mgr.smithi138.napnll on smithi138 2024-05-17T21:51:30.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:29 smithi138 ceph-mon[35948]: pgmap v242: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T21:51:30.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:29 smithi107 ceph-mon[28434]: pgmap v242: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T21:51:32.054 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:31 smithi138 ceph-mon[35948]: pgmap v243: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T21:51:32.054 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:31 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:51:32.054 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:31 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:51:32.054 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:31 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:51:32.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:31 smithi107 ceph-mon[28434]: pgmap v243: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T21:51:32.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:31 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:51:32.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:31 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:51:32.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:31 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:51:33.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:32 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:51:33.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:32 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:51:34.026 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:33 smithi138 ceph-mon[35948]: pgmap v244: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T21:51:34.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:33 smithi107 ceph-mon[28434]: pgmap v244: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T21:51:36.054 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:35 smithi138 ceph-mon[35948]: pgmap v245: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T21:51:36.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:35 smithi107 ceph-mon[28434]: pgmap v245: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T21:51:38.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:37 smithi138 ceph-mon[35948]: pgmap v246: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T21:51:38.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:37 smithi107 ceph-mon[28434]: pgmap v246: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T21:51:40.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:39 smithi107 ceph-mon[28434]: pgmap v247: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T21:51:40.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:39 smithi107 ceph-mon[28434]: Standby manager daemon smithi138.napnll restarted 2024-05-17T21:51:40.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:39 smithi107 ceph-mon[28434]: Standby manager daemon smithi138.napnll started 2024-05-17T21:51:40.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:39 smithi107 ceph-mon[28434]: from='mgr.? 172.21.15.138:0/3120106723' entity='mgr.smithi138.napnll' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi138.napnll/crt"}]: dispatch 2024-05-17T21:51:40.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:39 smithi107 ceph-mon[28434]: from='mgr.? 172.21.15.138:0/3120106723' entity='mgr.smithi138.napnll' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-17T21:51:40.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:39 smithi107 ceph-mon[28434]: from='mgr.? 172.21.15.138:0/3120106723' entity='mgr.smithi138.napnll' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi138.napnll/key"}]: dispatch 2024-05-17T21:51:40.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:39 smithi107 ceph-mon[28434]: from='mgr.? 172.21.15.138:0/3120106723' entity='mgr.smithi138.napnll' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-17T21:51:40.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:39 smithi138 ceph-mon[35948]: pgmap v247: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T21:51:40.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:39 smithi138 ceph-mon[35948]: Standby manager daemon smithi138.napnll restarted 2024-05-17T21:51:40.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:39 smithi138 ceph-mon[35948]: Standby manager daemon smithi138.napnll started 2024-05-17T21:51:40.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:39 smithi138 ceph-mon[35948]: from='mgr.? 172.21.15.138:0/3120106723' entity='mgr.smithi138.napnll' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi138.napnll/crt"}]: dispatch 2024-05-17T21:51:40.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:39 smithi138 ceph-mon[35948]: from='mgr.? 172.21.15.138:0/3120106723' entity='mgr.smithi138.napnll' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-17T21:51:40.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:39 smithi138 ceph-mon[35948]: from='mgr.? 172.21.15.138:0/3120106723' entity='mgr.smithi138.napnll' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi138.napnll/key"}]: dispatch 2024-05-17T21:51:40.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:39 smithi138 ceph-mon[35948]: from='mgr.? 172.21.15.138:0/3120106723' entity='mgr.smithi138.napnll' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-17T21:51:41.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:40 smithi107 ceph-mon[28434]: mgrmap e20: smithi107.xhhyiw(active, since 7m), standbys: smithi138.napnll 2024-05-17T21:51:41.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:40 smithi138 ceph-mon[35948]: mgrmap e20: smithi107.xhhyiw(active, since 7m), standbys: smithi138.napnll 2024-05-17T21:51:42.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:41 smithi107 ceph-mon[28434]: pgmap v248: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T21:51:42.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:41 smithi138 ceph-mon[35948]: pgmap v248: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T21:51:44.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:43 smithi107 ceph-mon[28434]: pgmap v249: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T21:51:44.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:43 smithi138 ceph-mon[35948]: pgmap v249: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T21:51:44.333 INFO:tasks.workunit.client.0.smithi107.stderr:Note: switching to '931809a9e5d295bd61984693cac44fe131ded4bc'. 2024-05-17T21:51:44.333 INFO:tasks.workunit.client.0.smithi107.stderr: 2024-05-17T21:51:44.333 INFO:tasks.workunit.client.0.smithi107.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-05-17T21:51:44.333 INFO:tasks.workunit.client.0.smithi107.stderr:changes and commit them, and you can discard any commits you make in this 2024-05-17T21:51:44.334 INFO:tasks.workunit.client.0.smithi107.stderr:state without impacting any branches by switching back to a branch. 2024-05-17T21:51:44.334 INFO:tasks.workunit.client.0.smithi107.stderr: 2024-05-17T21:51:44.334 INFO:tasks.workunit.client.0.smithi107.stderr:If you want to create a new branch to retain commits you create, you may 2024-05-17T21:51:44.334 INFO:tasks.workunit.client.0.smithi107.stderr:do so (now or later) by using -c with the switch command. Example: 2024-05-17T21:51:44.334 INFO:tasks.workunit.client.0.smithi107.stderr: 2024-05-17T21:51:44.334 INFO:tasks.workunit.client.0.smithi107.stderr: git switch -c 2024-05-17T21:51:44.335 INFO:tasks.workunit.client.0.smithi107.stderr: 2024-05-17T21:51:44.335 INFO:tasks.workunit.client.0.smithi107.stderr:Or undo this operation with: 2024-05-17T21:51:44.335 INFO:tasks.workunit.client.0.smithi107.stderr: 2024-05-17T21:51:44.335 INFO:tasks.workunit.client.0.smithi107.stderr: git switch - 2024-05-17T21:51:44.335 INFO:tasks.workunit.client.0.smithi107.stderr: 2024-05-17T21:51:44.336 INFO:tasks.workunit.client.0.smithi107.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-05-17T21:51:44.336 INFO:tasks.workunit.client.0.smithi107.stderr: 2024-05-17T21:51:44.336 INFO:tasks.workunit.client.0.smithi107.stderr:HEAD is now at 931809a9e5d Merge PR #57081 into wip-pdonnell-testing-20240517.012458-debug 2024-05-17T21:51:44.340 DEBUG:teuthology.orchestra.run.smithi107:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-05-17T21:51:44.369 INFO:tasks.workunit.client.0.smithi107.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-05-17T21:51:44.371 INFO:tasks.workunit.client.0.smithi107.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-17T21:51:44.372 INFO:tasks.workunit.client.0.smithi107.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-05-17T21:51:45.018 INFO:tasks.workunit.client.0.smithi107.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-05-17T21:51:45.072 INFO:tasks.workunit.client.0.smithi107.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-05-17T21:51:45.112 INFO:tasks.workunit.client.0.smithi107.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-17T21:51:45.114 INFO:tasks.workunit.client.0.smithi107.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-17T21:51:45.114 INFO:tasks.workunit.client.0.smithi107.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-05-17T21:51:45.154 INFO:tasks.workunit.client.0.smithi107.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-17T21:51:45.157 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-05-17T21:51:45.157 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-05-17T21:51:45.216 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.0... 2024-05-17T21:51:45.218 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-05-17T21:51:45.218 DEBUG:teuthology.orchestra.run.smithi107:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=931809a9e5d295bd61984693cac44fe131ded4bc TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh 2024-05-17T21:51:45.281 INFO:tasks.workunit.client.0.smithi107.stderr:+ mkdir -p fsstress 2024-05-17T21:51:45.283 INFO:tasks.workunit.client.0.smithi107.stderr:+ pushd fsstress 2024-05-17T21:51:45.284 INFO:tasks.workunit.client.0.smithi107.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-05-17T21:51:45.285 INFO:tasks.workunit.client.0.smithi107.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-05-17T21:51:45.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:45 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:51:45.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:45 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:51:45.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:45 smithi138 ceph-mon[35948]: pgmap v250: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T21:51:45.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:45 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:51:45.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:45 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:51:45.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:45 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:51:45.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:45 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:51:45.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:45 smithi107 ceph-mon[28434]: pgmap v250: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T21:51:45.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:45 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:51:45.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:45 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:51:47.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:47 smithi107 ceph-mon[28434]: pgmap v251: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T21:51:47.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:47 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:51:47.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:47 smithi138 ceph-mon[35948]: pgmap v251: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T21:51:47.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:47 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:51:48.278 INFO:tasks.workunit.client.1.smithi138.stderr:Updating files: 94% (11626/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-05-17T21:51:48.845 INFO:tasks.workunit.client.1.smithi138.stderr:Note: switching to '931809a9e5d295bd61984693cac44fe131ded4bc'. 2024-05-17T21:51:48.845 INFO:tasks.workunit.client.1.smithi138.stderr: 2024-05-17T21:51:48.845 INFO:tasks.workunit.client.1.smithi138.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-05-17T21:51:48.846 INFO:tasks.workunit.client.1.smithi138.stderr:changes and commit them, and you can discard any commits you make in this 2024-05-17T21:51:48.846 INFO:tasks.workunit.client.1.smithi138.stderr:state without impacting any branches by switching back to a branch. 2024-05-17T21:51:48.846 INFO:tasks.workunit.client.1.smithi138.stderr: 2024-05-17T21:51:48.846 INFO:tasks.workunit.client.1.smithi138.stderr:If you want to create a new branch to retain commits you create, you may 2024-05-17T21:51:48.846 INFO:tasks.workunit.client.1.smithi138.stderr:do so (now or later) by using -c with the switch command. Example: 2024-05-17T21:51:48.846 INFO:tasks.workunit.client.1.smithi138.stderr: 2024-05-17T21:51:48.846 INFO:tasks.workunit.client.1.smithi138.stderr: git switch -c 2024-05-17T21:51:48.846 INFO:tasks.workunit.client.1.smithi138.stderr: 2024-05-17T21:51:48.846 INFO:tasks.workunit.client.1.smithi138.stderr:Or undo this operation with: 2024-05-17T21:51:48.846 INFO:tasks.workunit.client.1.smithi138.stderr: 2024-05-17T21:51:48.846 INFO:tasks.workunit.client.1.smithi138.stderr: git switch - 2024-05-17T21:51:48.846 INFO:tasks.workunit.client.1.smithi138.stderr: 2024-05-17T21:51:48.846 INFO:tasks.workunit.client.1.smithi138.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-05-17T21:51:48.847 INFO:tasks.workunit.client.1.smithi138.stderr: 2024-05-17T21:51:48.847 INFO:tasks.workunit.client.1.smithi138.stderr:HEAD is now at 931809a9e5d Merge PR #57081 into wip-pdonnell-testing-20240517.012458-debug 2024-05-17T21:51:48.852 DEBUG:teuthology.orchestra.run.smithi138:> cd -- /home/ubuntu/cephtest/clone.client.1/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.1 2024-05-17T21:51:48.901 INFO:tasks.workunit.client.0.smithi107.stderr:+ tar xzf ltp-full.tgz 2024-05-17T21:51:48.945 INFO:tasks.workunit.client.1.smithi138.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-05-17T21:51:48.947 INFO:tasks.workunit.client.1.smithi138.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-05-17T21:51:48.947 INFO:tasks.workunit.client.1.smithi138.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-05-17T21:51:49.577 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:49 smithi138 ceph-mon[35948]: pgmap v252: 65 pgs: 65 active+clean; 599 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-05-17T21:51:49.577 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:49 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:51:49.577 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:49 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:51:49.578 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:49 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:51:49.578 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:49 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T21:51:49.578 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:49 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:51:49.578 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:49 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:51:49.578 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:49 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mgr fail", "who": "smithi107.xhhyiw"}]: dispatch 2024-05-17T21:51:49.578 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:49 smithi138 ceph-mon[35948]: Activating manager daemon smithi138.napnll 2024-05-17T21:51:49.578 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:49 smithi138 ceph-mon[35948]: osdmap e47: 6 total, 6 up, 6 in 2024-05-17T21:51:49.579 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:49 smithi138 ceph-mon[35948]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd='[{"prefix": "mgr fail", "who": "smithi107.xhhyiw"}]': finished 2024-05-17T21:51:49.579 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:49 smithi138 ceph-mon[35948]: mgrmap e21: smithi138.napnll(active, starting, since 0.00729892s) 2024-05-17T21:51:49.579 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:49 smithi138 ceph-mon[35948]: from='mgr.14636 172.21.15.138:0/3120106723' entity='mgr.smithi138.napnll' cmd=[{"prefix": "mon metadata", "id": "smithi107"}]: dispatch 2024-05-17T21:51:49.579 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:49 smithi138 ceph-mon[35948]: from='mgr.14636 172.21.15.138:0/3120106723' entity='mgr.smithi138.napnll' cmd=[{"prefix": "mon metadata", "id": "smithi138"}]: dispatch 2024-05-17T21:51:49.579 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:49 smithi138 ceph-mon[35948]: from='mgr.14636 172.21.15.138:0/3120106723' entity='mgr.smithi138.napnll' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi107.drqody"}]: dispatch 2024-05-17T21:51:49.579 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:49 smithi138 ceph-mon[35948]: from='mgr.14636 172.21.15.138:0/3120106723' entity='mgr.smithi138.napnll' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi107.idmbgj"}]: dispatch 2024-05-17T21:51:49.579 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:49 smithi138 ceph-mon[35948]: from='mgr.14636 172.21.15.138:0/3120106723' entity='mgr.smithi138.napnll' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi138.gwbtyd"}]: dispatch 2024-05-17T21:51:49.579 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:49 smithi138 ceph-mon[35948]: from='mgr.14636 172.21.15.138:0/3120106723' entity='mgr.smithi138.napnll' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi138.zckrvs"}]: dispatch 2024-05-17T21:51:49.580 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:49 smithi138 ceph-mon[35948]: from='mgr.14636 172.21.15.138:0/3120106723' entity='mgr.smithi138.napnll' cmd=[{"prefix": "mgr metadata", "who": "smithi138.napnll", "id": "smithi138.napnll"}]: dispatch 2024-05-17T21:51:49.580 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:49 smithi138 ceph-mon[35948]: from='mgr.14636 172.21.15.138:0/3120106723' entity='mgr.smithi138.napnll' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-17T21:51:49.580 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:49 smithi138 ceph-mon[35948]: from='mgr.14636 172.21.15.138:0/3120106723' entity='mgr.smithi138.napnll' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-17T21:51:49.580 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:49 smithi138 ceph-mon[35948]: from='mgr.14636 172.21.15.138:0/3120106723' entity='mgr.smithi138.napnll' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-17T21:51:49.580 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:49 smithi138 ceph-mon[35948]: from='mgr.14636 172.21.15.138:0/3120106723' entity='mgr.smithi138.napnll' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-17T21:51:49.580 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:49 smithi138 ceph-mon[35948]: from='mgr.14636 172.21.15.138:0/3120106723' entity='mgr.smithi138.napnll' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-17T21:51:49.580 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:49 smithi138 ceph-mon[35948]: from='mgr.14636 172.21.15.138:0/3120106723' entity='mgr.smithi138.napnll' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-17T21:51:49.581 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:49 smithi138 ceph-mon[35948]: from='mgr.14636 172.21.15.138:0/3120106723' entity='mgr.smithi138.napnll' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-17T21:51:49.581 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:49 smithi138 ceph-mon[35948]: from='mgr.14636 172.21.15.138:0/3120106723' entity='mgr.smithi138.napnll' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-17T21:51:49.581 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:49 smithi138 ceph-mon[35948]: from='mgr.14636 172.21.15.138:0/3120106723' entity='mgr.smithi138.napnll' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-17T21:51:49.581 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:49 smithi138 ceph-mon[35948]: Manager daemon smithi138.napnll is now available 2024-05-17T21:51:49.581 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:49 smithi138 ceph-mon[35948]: from='mgr.14636 172.21.15.138:0/3120106723' entity='mgr.smithi138.napnll' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:51:49.581 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:49 smithi138 ceph-mon[35948]: from='mgr.14636 172.21.15.138:0/3120106723' entity='mgr.smithi138.napnll' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:51:49.581 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:49 smithi138 ceph-mon[35948]: from='mgr.14636 172.21.15.138:0/3120106723' entity='mgr.smithi138.napnll' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi138.napnll/mirror_snapshot_schedule"}]: dispatch 2024-05-17T21:51:49.582 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:49 smithi138 ceph-mon[35948]: from='mgr.14636 ' entity='mgr.smithi138.napnll' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi138.napnll/mirror_snapshot_schedule"}]: dispatch 2024-05-17T21:51:49.582 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:49 smithi138 ceph-mon[35948]: from='mgr.14636 172.21.15.138:0/3120106723' entity='mgr.smithi138.napnll' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi138.napnll/trash_purge_schedule"}]: dispatch 2024-05-17T21:51:49.582 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:49 smithi138 ceph-mon[35948]: from='mgr.14636 ' entity='mgr.smithi138.napnll' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi138.napnll/trash_purge_schedule"}]: dispatch 2024-05-17T21:51:49.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:49 smithi107 ceph-mon[28434]: pgmap v252: 65 pgs: 65 active+clean; 599 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-05-17T21:51:49.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:49 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:51:49.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:49 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:51:49.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:49 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:51:49.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:49 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T21:51:49.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:49 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:51:49.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:49 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:51:49.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:49 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mgr fail", "who": "smithi107.xhhyiw"}]: dispatch 2024-05-17T21:51:49.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:49 smithi107 ceph-mon[28434]: Activating manager daemon smithi138.napnll 2024-05-17T21:51:49.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:49 smithi107 ceph-mon[28434]: osdmap e47: 6 total, 6 up, 6 in 2024-05-17T21:51:49.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:49 smithi107 ceph-mon[28434]: from='mgr.14214 172.21.15.107:0/3734573034' entity='mgr.smithi107.xhhyiw' cmd='[{"prefix": "mgr fail", "who": "smithi107.xhhyiw"}]': finished 2024-05-17T21:51:49.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:49 smithi107 ceph-mon[28434]: mgrmap e21: smithi138.napnll(active, starting, since 0.00729892s) 2024-05-17T21:51:49.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:49 smithi107 ceph-mon[28434]: from='mgr.14636 172.21.15.138:0/3120106723' entity='mgr.smithi138.napnll' cmd=[{"prefix": "mon metadata", "id": "smithi107"}]: dispatch 2024-05-17T21:51:49.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:49 smithi107 ceph-mon[28434]: from='mgr.14636 172.21.15.138:0/3120106723' entity='mgr.smithi138.napnll' cmd=[{"prefix": "mon metadata", "id": "smithi138"}]: dispatch 2024-05-17T21:51:49.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:49 smithi107 ceph-mon[28434]: from='mgr.14636 172.21.15.138:0/3120106723' entity='mgr.smithi138.napnll' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi107.drqody"}]: dispatch 2024-05-17T21:51:49.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:49 smithi107 ceph-mon[28434]: from='mgr.14636 172.21.15.138:0/3120106723' entity='mgr.smithi138.napnll' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi107.idmbgj"}]: dispatch 2024-05-17T21:51:49.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:49 smithi107 ceph-mon[28434]: from='mgr.14636 172.21.15.138:0/3120106723' entity='mgr.smithi138.napnll' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi138.gwbtyd"}]: dispatch 2024-05-17T21:51:49.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:49 smithi107 ceph-mon[28434]: from='mgr.14636 172.21.15.138:0/3120106723' entity='mgr.smithi138.napnll' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi138.zckrvs"}]: dispatch 2024-05-17T21:51:49.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:49 smithi107 ceph-mon[28434]: from='mgr.14636 172.21.15.138:0/3120106723' entity='mgr.smithi138.napnll' cmd=[{"prefix": "mgr metadata", "who": "smithi138.napnll", "id": "smithi138.napnll"}]: dispatch 2024-05-17T21:51:49.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:49 smithi107 ceph-mon[28434]: from='mgr.14636 172.21.15.138:0/3120106723' entity='mgr.smithi138.napnll' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-17T21:51:49.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:49 smithi107 ceph-mon[28434]: from='mgr.14636 172.21.15.138:0/3120106723' entity='mgr.smithi138.napnll' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-17T21:51:49.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:49 smithi107 ceph-mon[28434]: from='mgr.14636 172.21.15.138:0/3120106723' entity='mgr.smithi138.napnll' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-17T21:51:49.605 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:49 smithi107 ceph-mon[28434]: from='mgr.14636 172.21.15.138:0/3120106723' entity='mgr.smithi138.napnll' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-17T21:51:49.605 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:49 smithi107 ceph-mon[28434]: from='mgr.14636 172.21.15.138:0/3120106723' entity='mgr.smithi138.napnll' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-17T21:51:49.605 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:49 smithi107 ceph-mon[28434]: from='mgr.14636 172.21.15.138:0/3120106723' entity='mgr.smithi138.napnll' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-17T21:51:49.605 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:49 smithi107 ceph-mon[28434]: from='mgr.14636 172.21.15.138:0/3120106723' entity='mgr.smithi138.napnll' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-17T21:51:49.605 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:49 smithi107 ceph-mon[28434]: from='mgr.14636 172.21.15.138:0/3120106723' entity='mgr.smithi138.napnll' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-17T21:51:49.605 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:49 smithi107 ceph-mon[28434]: from='mgr.14636 172.21.15.138:0/3120106723' entity='mgr.smithi138.napnll' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-17T21:51:49.605 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:49 smithi107 ceph-mon[28434]: Manager daemon smithi138.napnll is now available 2024-05-17T21:51:49.605 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:49 smithi107 ceph-mon[28434]: from='mgr.14636 172.21.15.138:0/3120106723' entity='mgr.smithi138.napnll' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:51:49.605 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:49 smithi107 ceph-mon[28434]: from='mgr.14636 172.21.15.138:0/3120106723' entity='mgr.smithi138.napnll' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:51:49.606 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:49 smithi107 ceph-mon[28434]: from='mgr.14636 172.21.15.138:0/3120106723' entity='mgr.smithi138.napnll' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi138.napnll/mirror_snapshot_schedule"}]: dispatch 2024-05-17T21:51:49.606 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:49 smithi107 ceph-mon[28434]: from='mgr.14636 ' entity='mgr.smithi138.napnll' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi138.napnll/mirror_snapshot_schedule"}]: dispatch 2024-05-17T21:51:49.606 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:49 smithi107 ceph-mon[28434]: from='mgr.14636 172.21.15.138:0/3120106723' entity='mgr.smithi138.napnll' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi138.napnll/trash_purge_schedule"}]: dispatch 2024-05-17T21:51:49.606 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:49 smithi107 ceph-mon[28434]: from='mgr.14636 ' entity='mgr.smithi138.napnll' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi138.napnll/trash_purge_schedule"}]: dispatch 2024-05-17T21:51:49.867 INFO:tasks.workunit.client.1.smithi138.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-05-17T21:51:49.936 INFO:tasks.workunit.client.1.smithi138.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-05-17T21:51:49.986 INFO:tasks.workunit.client.1.smithi138.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-05-17T21:51:49.988 INFO:tasks.workunit.client.1.smithi138.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-05-17T21:51:49.988 INFO:tasks.workunit.client.1.smithi138.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-05-17T21:51:50.030 INFO:tasks.workunit.client.1.smithi138.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-05-17T21:51:50.033 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-17T21:51:50.033 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/home/ubuntu/cephtest/workunits.list.client.1 of=/dev/stdout 2024-05-17T21:51:50.050 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.1... 2024-05-17T21:51:50.051 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-05-17T21:51:50.051 DEBUG:teuthology.orchestra.run.smithi138:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=931809a9e5d295bd61984693cac44fe131ded4bc TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/suites/fsstress.sh 2024-05-17T21:51:50.117 INFO:tasks.workunit.client.1.smithi138.stderr:+ mkdir -p fsstress 2024-05-17T21:51:50.120 INFO:tasks.workunit.client.1.smithi138.stderr:+ pushd fsstress 2024-05-17T21:51:50.121 INFO:tasks.workunit.client.1.smithi138.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-05-17T21:51:50.122 INFO:tasks.workunit.client.1.smithi138.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-05-17T21:51:51.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:50 smithi138 ceph-mon[35948]: Deploying cephadm binary to smithi138 2024-05-17T21:51:51.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:50 smithi138 ceph-mon[35948]: Deploying cephadm binary to smithi107 2024-05-17T21:51:51.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:50 smithi138 ceph-mon[35948]: from='mgr.14636 ' entity='mgr.smithi138.napnll' 2024-05-17T21:51:51.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:50 smithi138 ceph-mon[35948]: from='mgr.14636 ' entity='mgr.smithi138.napnll' 2024-05-17T21:51:51.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:50 smithi138 ceph-mon[35948]: mgrmap e22: smithi138.napnll(active, since 1.00992s) 2024-05-17T21:51:51.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:50 smithi138 ceph-mon[35948]: pgmap v3: 65 pgs: 65 active+clean; 13 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-17T21:51:51.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:50 smithi138 ceph-mon[35948]: [17/May/2024:21:51:50] ENGINE Bus STARTING 2024-05-17T21:51:51.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:50 smithi138 ceph-mon[35948]: [17/May/2024:21:51:50] ENGINE Serving on http://172.21.15.138:8765 2024-05-17T21:51:51.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:50 smithi107 ceph-mon[28434]: Deploying cephadm binary to smithi138 2024-05-17T21:51:51.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:50 smithi107 ceph-mon[28434]: Deploying cephadm binary to smithi107 2024-05-17T21:51:51.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:50 smithi107 ceph-mon[28434]: from='mgr.14636 ' entity='mgr.smithi138.napnll' 2024-05-17T21:51:51.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:50 smithi107 ceph-mon[28434]: from='mgr.14636 ' entity='mgr.smithi138.napnll' 2024-05-17T21:51:51.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:50 smithi107 ceph-mon[28434]: mgrmap e22: smithi138.napnll(active, since 1.00992s) 2024-05-17T21:51:51.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:50 smithi107 ceph-mon[28434]: pgmap v3: 65 pgs: 65 active+clean; 13 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-17T21:51:51.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:50 smithi107 ceph-mon[28434]: [17/May/2024:21:51:50] ENGINE Bus STARTING 2024-05-17T21:51:51.353 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:50 smithi107 ceph-mon[28434]: [17/May/2024:21:51:50] ENGINE Serving on http://172.21.15.138:8765 2024-05-17T21:51:52.482 INFO:tasks.workunit.client.1.smithi138.stderr:+ tar xzf ltp-full.tgz 2024-05-17T21:51:52.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:52.519+0000 7fa277628700 1 -- 172.21.15.107:0/1930857414 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa2700fcfa0 msgr2=0x7fa2700fd410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:51:52.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:52.519+0000 7fa277628700 1 --2- 172.21.15.107:0/1930857414 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa2700fcfa0 0x7fa2700fd410 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fa260004940 tx=0x7fa260009e30 comp rx=0 tx=0).stop 2024-05-17T21:51:52.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:52.519+0000 7fa277628700 1 -- 172.21.15.107:0/1930857414 shutdown_connections 2024-05-17T21:51:52.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:52.519+0000 7fa277628700 1 --2- 172.21.15.107:0/1930857414 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa2700fcfa0 0x7fa2700fd410 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:51:52.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:52.519+0000 7fa277628700 1 --2- 172.21.15.107:0/1930857414 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2700fbe30 0x7fa2700fc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:51:52.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:52.519+0000 7fa277628700 1 -- 172.21.15.107:0/1930857414 >> 172.21.15.107:0/1930857414 conn(0x7fa2700f7440 msgr2=0x7fa2700f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:51:52.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:52.519+0000 7fa277628700 1 -- 172.21.15.107:0/1930857414 shutdown_connections 2024-05-17T21:51:52.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:52.519+0000 7fa277628700 1 -- 172.21.15.107:0/1930857414 wait complete. 2024-05-17T21:51:52.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:52.520+0000 7fa277628700 1 Processor -- start 2024-05-17T21:51:52.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:52.520+0000 7fa277628700 1 -- start start 2024-05-17T21:51:52.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:52.520+0000 7fa277628700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2700fbe30 0x7fa270110640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:51:52.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:52.520+0000 7fa277628700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa2700fcfa0 0x7fa270112b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:51:52.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:52.520+0000 7fa277628700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa270066c20 con 0x7fa2700fcfa0 2024-05-17T21:51:52.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:52.520+0000 7fa277628700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa270066d60 con 0x7fa2700fbe30 2024-05-17T21:51:52.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:52.520+0000 7fa274bc3700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa2700fcfa0 0x7fa270112b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:51:52.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:52.520+0000 7fa274bc3700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa2700fcfa0 0x7fa270112b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45524/0 (socket says 172.21.15.107:45524) 2024-05-17T21:51:52.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:52.520+0000 7fa274bc3700 1 -- 172.21.15.107:0/1326480818 learned_addr learned my addr 172.21.15.107:0/1326480818 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:51:52.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:52.520+0000 7fa2753c4700 1 --2- 172.21.15.107:0/1326480818 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2700fbe30 0x7fa270110640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:51:52.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:52.521+0000 7fa274bc3700 1 -- 172.21.15.107:0/1326480818 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2700fbe30 msgr2=0x7fa270110640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:51:52.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:52.521+0000 7fa274bc3700 1 --2- 172.21.15.107:0/1326480818 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2700fbe30 0x7fa270110640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:51:52.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:52.521+0000 7fa274bc3700 1 -- 172.21.15.107:0/1326480818 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa260005040 con 0x7fa2700fcfa0 2024-05-17T21:51:52.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:52.521+0000 7fa2753c4700 1 --2- 172.21.15.107:0/1326480818 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2700fbe30 0x7fa270110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T21:51:52.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:52.521+0000 7fa274bc3700 1 --2- 172.21.15.107:0/1326480818 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa2700fcfa0 0x7fa270112b90 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fa260013870 tx=0x7fa260009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:51:52.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:52.521+0000 7fa265ffb700 1 -- 172.21.15.107:0/1326480818 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa260021070 con 0x7fa2700fcfa0 2024-05-17T21:51:52.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:52.521+0000 7fa277628700 1 -- 172.21.15.107:0/1326480818 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa270113130 con 0x7fa2700fcfa0 2024-05-17T21:51:52.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:52.521+0000 7fa265ffb700 1 -- 172.21.15.107:0/1326480818 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa260009430 con 0x7fa2700fcfa0 2024-05-17T21:51:52.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:52.521+0000 7fa265ffb700 1 -- 172.21.15.107:0/1326480818 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa26000e070 con 0x7fa2700fcfa0 2024-05-17T21:51:52.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:52.521+0000 7fa277628700 1 -- 172.21.15.107:0/1326480818 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa270113670 con 0x7fa2700fcfa0 2024-05-17T21:51:52.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:52.522+0000 7fa277628700 1 -- 172.21.15.107:0/1326480818 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa27005f5c0 con 0x7fa2700fcfa0 2024-05-17T21:51:52.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:52.522+0000 7fa265ffb700 1 -- 172.21.15.107:0/1326480818 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 23) v1 ==== 50176+0+0 (secure 0 0 0) 0x7fa260026410 con 0x7fa2700fcfa0 2024-05-17T21:51:52.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:52.523+0000 7fa265ffb700 1 --2- 172.21.15.107:0/1326480818 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7fa25c03e2b0 0x7fa25c040760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:51:52.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:52.523+0000 7fa265ffb700 1 -- 172.21.15.107:0/1326480818 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7fa260054c80 con 0x7fa2700fcfa0 2024-05-17T21:51:52.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:52.523+0000 7fa2753c4700 1 --2- 172.21.15.107:0/1326480818 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7fa25c03e2b0 0x7fa25c040760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:51:52.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:52.524+0000 7fa2753c4700 1 --2- 172.21.15.107:0/1326480818 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7fa25c03e2b0 0x7fa25c040760 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fa2700668a0 tx=0x7fa26c00b040 comp rx=0 tx=0).ready entity=mgr.14636 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:51:52.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:52.528+0000 7fa265ffb700 1 -- 172.21.15.107:0/1326480818 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+190121 (secure 0 0 0) 0x7fa2600123c0 con 0x7fa2700fcfa0 2024-05-17T21:51:52.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:52 smithi138 ceph-mon[35948]: [17/May/2024:21:51:50] ENGINE Serving on https://172.21.15.138:7150 2024-05-17T21:51:52.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:52 smithi138 ceph-mon[35948]: [17/May/2024:21:51:50] ENGINE Bus STARTED 2024-05-17T21:51:52.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:52 smithi138 ceph-mon[35948]: pgmap v4: 65 pgs: 65 active+clean; 13 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-17T21:51:52.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:52 smithi107 ceph-mon[28434]: [17/May/2024:21:51:50] ENGINE Serving on https://172.21.15.138:7150 2024-05-17T21:51:52.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:52 smithi107 ceph-mon[28434]: [17/May/2024:21:51:50] ENGINE Bus STARTED 2024-05-17T21:51:52.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:52 smithi107 ceph-mon[28434]: pgmap v4: 65 pgs: 65 active+clean; 13 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-17T21:51:52.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:52.806+0000 7fa277628700 1 -- 172.21.15.107:0/1326480818 --> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa270101240 con 0x7fa25c03e2b0 2024-05-17T21:51:52.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:52.819+0000 7fa265ffb700 1 -- 172.21.15.107:0/1326480818 <== mgr.14636 v2:172.21.15.138:6828/1369055740 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7fa270101240 con 0x7fa25c03e2b0 2024-05-17T21:51:52.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:52.822+0000 7fa277628700 1 -- 172.21.15.107:0/1326480818 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7fa25c03e2b0 msgr2=0x7fa25c040760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:51:52.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:52.822+0000 7fa277628700 1 --2- 172.21.15.107:0/1326480818 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7fa25c03e2b0 0x7fa25c040760 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fa2700668a0 tx=0x7fa26c00b040 comp rx=0 tx=0).stop 2024-05-17T21:51:52.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:52.822+0000 7fa277628700 1 -- 172.21.15.107:0/1326480818 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa2700fcfa0 msgr2=0x7fa270112b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:51:52.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:52.822+0000 7fa277628700 1 --2- 172.21.15.107:0/1326480818 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa2700fcfa0 0x7fa270112b90 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fa260013870 tx=0x7fa260009e30 comp rx=0 tx=0).stop 2024-05-17T21:51:52.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:52.822+0000 7fa277628700 1 -- 172.21.15.107:0/1326480818 shutdown_connections 2024-05-17T21:51:52.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:52.822+0000 7fa277628700 1 --2- 172.21.15.107:0/1326480818 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7fa25c03e2b0 0x7fa25c040760 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:51:52.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:52.822+0000 7fa277628700 1 --2- 172.21.15.107:0/1326480818 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2700fbe30 0x7fa270110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:51:52.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:52.822+0000 7fa277628700 1 --2- 172.21.15.107:0/1326480818 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa2700fcfa0 0x7fa270112b90 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:51:52.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:52.822+0000 7fa277628700 1 -- 172.21.15.107:0/1326480818 >> 172.21.15.107:0/1326480818 conn(0x7fa2700f7440 msgr2=0x7fa2700f8a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:51:52.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:52.823+0000 7fa277628700 1 -- 172.21.15.107:0/1326480818 shutdown_connections 2024-05-17T21:51:52.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:52.823+0000 7fa277628700 1 -- 172.21.15.107:0/1326480818 wait complete. 2024-05-17T21:51:52.828 INFO:teuthology.orchestra.run.smithi107.stdout:true 2024-05-17T21:51:52.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:52.947+0000 7f6c76b03700 1 -- 172.21.15.107:0/846917188 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c700fcfa0 msgr2=0x7f6c700fd410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:51:52.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:52.947+0000 7f6c76b03700 1 --2- 172.21.15.107:0/846917188 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c700fcfa0 0x7f6c700fd410 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c004940 tx=0x7f6c5c009e30 comp rx=0 tx=0).stop 2024-05-17T21:51:52.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:52.947+0000 7f6c76b03700 1 -- 172.21.15.107:0/846917188 shutdown_connections 2024-05-17T21:51:52.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:52.947+0000 7f6c76b03700 1 --2- 172.21.15.107:0/846917188 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c700fcfa0 0x7f6c700fd410 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:51:52.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:52.947+0000 7f6c76b03700 1 --2- 172.21.15.107:0/846917188 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c700fbe30 0x7f6c700fc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:51:52.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:52.947+0000 7f6c76b03700 1 -- 172.21.15.107:0/846917188 >> 172.21.15.107:0/846917188 conn(0x7f6c700f7440 msgr2=0x7f6c700f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:51:52.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:52.947+0000 7f6c76b03700 1 -- 172.21.15.107:0/846917188 shutdown_connections 2024-05-17T21:51:52.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:52.947+0000 7f6c76b03700 1 -- 172.21.15.107:0/846917188 wait complete. 2024-05-17T21:51:52.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:52.947+0000 7f6c76b03700 1 Processor -- start 2024-05-17T21:51:52.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:52.947+0000 7f6c76b03700 1 -- start start 2024-05-17T21:51:52.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:52.948+0000 7f6c76b03700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c700fbe30 0x7f6c701106a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:51:52.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:52.948+0000 7f6c76b03700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c700fcfa0 0x7f6c70112bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:51:52.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:52.948+0000 7f6c76b03700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c70066bc0 con 0x7f6c700fcfa0 2024-05-17T21:51:52.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:52.948+0000 7f6c76b03700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c70066d00 con 0x7f6c700fbe30 2024-05-17T21:51:52.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:52.948+0000 7f6c6ffff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c700fcfa0 0x7f6c70112bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:51:52.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:52.948+0000 7f6c7489f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c700fbe30 0x7f6c701106a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:51:52.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:52.948+0000 7f6c6ffff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c700fcfa0 0x7f6c70112bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45536/0 (socket says 172.21.15.107:45536) 2024-05-17T21:51:52.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:52.948+0000 7f6c6ffff700 1 -- 172.21.15.107:0/285136484 learned_addr learned my addr 172.21.15.107:0/285136484 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:51:52.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:52.948+0000 7f6c6ffff700 1 -- 172.21.15.107:0/285136484 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c700fbe30 msgr2=0x7f6c701106a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:51:52.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:52.948+0000 7f6c6ffff700 1 --2- 172.21.15.107:0/285136484 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c700fbe30 0x7f6c701106a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:51:52.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:52.948+0000 7f6c6ffff700 1 -- 172.21.15.107:0/285136484 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c5c005040 con 0x7f6c700fcfa0 2024-05-17T21:51:52.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:52.948+0000 7f6c7489f700 1 --2- 172.21.15.107:0/285136484 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c700fbe30 0x7f6c701106a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T21:51:52.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:52.949+0000 7f6c6ffff700 1 --2- 172.21.15.107:0/285136484 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c700fcfa0 0x7f6c70112bf0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c013870 tx=0x7f6c5c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:51:52.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:52.949+0000 7f6c6d7fa700 1 -- 172.21.15.107:0/285136484 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c5c00e070 con 0x7f6c700fcfa0 2024-05-17T21:51:52.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:52.949+0000 7f6c6d7fa700 1 -- 172.21.15.107:0/285136484 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6c5c013de0 con 0x7f6c700fcfa0 2024-05-17T21:51:52.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:52.949+0000 7f6c6d7fa700 1 -- 172.21.15.107:0/285136484 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c5c01ce70 con 0x7f6c700fcfa0 2024-05-17T21:51:52.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:52.949+0000 7f6c76b03700 1 -- 172.21.15.107:0/285136484 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c70113130 con 0x7f6c700fcfa0 2024-05-17T21:51:52.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:52.949+0000 7f6c76b03700 1 -- 172.21.15.107:0/285136484 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c701135c0 con 0x7f6c700fcfa0 2024-05-17T21:51:52.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:52.950+0000 7f6c76b03700 1 -- 172.21.15.107:0/285136484 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c7005f5c0 con 0x7f6c700fcfa0 2024-05-17T21:51:52.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:52.950+0000 7f6c6d7fa700 1 -- 172.21.15.107:0/285136484 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 23) v1 ==== 50176+0+0 (secure 0 0 0) 0x7f6c5c01c690 con 0x7f6c700fcfa0 2024-05-17T21:51:52.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:52.951+0000 7f6c6d7fa700 1 --2- 172.21.15.107:0/285136484 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7f6c6003e2a0 0x7f6c60040750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:51:52.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:52.951+0000 7f6c6d7fa700 1 -- 172.21.15.107:0/285136484 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f6c5c054750 con 0x7f6c700fcfa0 2024-05-17T21:51:52.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:52.951+0000 7f6c7489f700 1 --2- 172.21.15.107:0/285136484 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7f6c6003e2a0 0x7f6c60040750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:51:52.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:52.952+0000 7f6c7489f700 1 --2- 172.21.15.107:0/285136484 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7f6c6003e2a0 0x7f6c60040750 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f6c70066840 tx=0x7f6c6400b040 comp rx=0 tx=0).ready entity=mgr.14636 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:51:52.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:52.960+0000 7f6c6d7fa700 1 -- 172.21.15.107:0/285136484 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+190121 (secure 0 0 0) 0x7f6c5c011970 con 0x7f6c700fcfa0 2024-05-17T21:51:53.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:53.261+0000 7f6c76b03700 1 -- 172.21.15.107:0/285136484 --> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6c70101240 con 0x7f6c6003e2a0 2024-05-17T21:51:53.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:53.271+0000 7f6c6d7fa700 1 -- 172.21.15.107:0/285136484 <== mgr.14636 v2:172.21.15.138:6828/1369055740 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7f6c70101240 con 0x7f6c6003e2a0 2024-05-17T21:51:53.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:53.274+0000 7f6c76b03700 1 -- 172.21.15.107:0/285136484 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7f6c6003e2a0 msgr2=0x7f6c60040750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:51:53.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:53.274+0000 7f6c76b03700 1 --2- 172.21.15.107:0/285136484 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7f6c6003e2a0 0x7f6c60040750 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f6c70066840 tx=0x7f6c6400b040 comp rx=0 tx=0).stop 2024-05-17T21:51:53.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:53.274+0000 7f6c76b03700 1 -- 172.21.15.107:0/285136484 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c700fcfa0 msgr2=0x7f6c70112bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:51:53.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:53.274+0000 7f6c76b03700 1 --2- 172.21.15.107:0/285136484 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c700fcfa0 0x7f6c70112bf0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c013870 tx=0x7f6c5c009e30 comp rx=0 tx=0).stop 2024-05-17T21:51:53.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:53.274+0000 7f6c76b03700 1 -- 172.21.15.107:0/285136484 shutdown_connections 2024-05-17T21:51:53.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:53.274+0000 7f6c76b03700 1 --2- 172.21.15.107:0/285136484 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7f6c6003e2a0 0x7f6c60040750 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:51:53.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:53.274+0000 7f6c76b03700 1 --2- 172.21.15.107:0/285136484 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c700fbe30 0x7f6c701106a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:51:53.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:53.274+0000 7f6c76b03700 1 --2- 172.21.15.107:0/285136484 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c700fcfa0 0x7f6c70112bf0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:51:53.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:53.274+0000 7f6c76b03700 1 -- 172.21.15.107:0/285136484 >> 172.21.15.107:0/285136484 conn(0x7f6c700f7440 msgr2=0x7f6c700f8a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:51:53.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:53.274+0000 7f6c76b03700 1 -- 172.21.15.107:0/285136484 shutdown_connections 2024-05-17T21:51:53.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:53.274+0000 7f6c76b03700 1 -- 172.21.15.107:0/285136484 wait complete. 2024-05-17T21:51:53.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:53.392+0000 7f2928565700 1 -- 172.21.15.107:0/2333100331 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29200fcf90 msgr2=0x7f29200fd400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:51:53.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:53.392+0000 7f2928565700 1 --2- 172.21.15.107:0/2333100331 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29200fcf90 0x7f29200fd400 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f2910004940 tx=0x7f2910009e30 comp rx=0 tx=0).stop 2024-05-17T21:51:53.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:53.393+0000 7f2928565700 1 -- 172.21.15.107:0/2333100331 shutdown_connections 2024-05-17T21:51:53.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:53.393+0000 7f2928565700 1 --2- 172.21.15.107:0/2333100331 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29200fcf90 0x7f29200fd400 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:51:53.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:53.393+0000 7f2928565700 1 --2- 172.21.15.107:0/2333100331 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f29200fbce0 0x7f29200fc0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:51:53.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:53.393+0000 7f2928565700 1 -- 172.21.15.107:0/2333100331 >> 172.21.15.107:0/2333100331 conn(0x7f29200f7420 msgr2=0x7f29200f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:51:53.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:53.393+0000 7f2928565700 1 -- 172.21.15.107:0/2333100331 shutdown_connections 2024-05-17T21:51:53.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:53.393+0000 7f2928565700 1 -- 172.21.15.107:0/2333100331 wait complete. 2024-05-17T21:51:53.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:53.393+0000 7f2928565700 1 Processor -- start 2024-05-17T21:51:53.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:53.393+0000 7f2928565700 1 -- start start 2024-05-17T21:51:53.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:53.393+0000 7f2928565700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f29200fbce0 0x7f292010c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:51:53.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:53.393+0000 7f2928565700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29200fcf90 0x7f292010cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:51:53.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:53.393+0000 7f2928565700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f292010d2e0 con 0x7f29200fcf90 2024-05-17T21:51:53.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:53.393+0000 7f2928565700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f292010d450 con 0x7f29200fbce0 2024-05-17T21:51:53.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:53.394+0000 7f2925b00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29200fcf90 0x7f292010cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:51:53.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:53.394+0000 7f2925b00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29200fcf90 0x7f292010cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45560/0 (socket says 172.21.15.107:45560) 2024-05-17T21:51:53.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:53.394+0000 7f2925b00700 1 -- 172.21.15.107:0/111191774 learned_addr learned my addr 172.21.15.107:0/111191774 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:51:53.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:53.394+0000 7f2926301700 1 --2- 172.21.15.107:0/111191774 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f29200fbce0 0x7f292010c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:51:53.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:53.394+0000 7f2925b00700 1 -- 172.21.15.107:0/111191774 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f29200fbce0 msgr2=0x7f292010c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:51:53.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:53.394+0000 7f2925b00700 1 --2- 172.21.15.107:0/111191774 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f29200fbce0 0x7f292010c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:51:53.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:53.394+0000 7f2925b00700 1 -- 172.21.15.107:0/111191774 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2910005040 con 0x7f29200fcf90 2024-05-17T21:51:53.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:53.394+0000 7f2926301700 1 --2- 172.21.15.107:0/111191774 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f29200fbce0 0x7f292010c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T21:51:53.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:53.394+0000 7f2925b00700 1 --2- 172.21.15.107:0/111191774 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29200fcf90 0x7f292010cce0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f2910013870 tx=0x7f2910009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:51:53.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:53.394+0000 7f2916ffd700 1 -- 172.21.15.107:0/111191774 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2910021070 con 0x7f29200fcf90 2024-05-17T21:51:53.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:53.394+0000 7f2916ffd700 1 -- 172.21.15.107:0/111191774 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2910009430 con 0x7f29200fcf90 2024-05-17T21:51:53.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:53.395+0000 7f2928565700 1 -- 172.21.15.107:0/111191774 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29200661f0 con 0x7f29200fcf90 2024-05-17T21:51:53.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:53.395+0000 7f2916ffd700 1 -- 172.21.15.107:0/111191774 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f291000e070 con 0x7f29200fcf90 2024-05-17T21:51:53.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:53.395+0000 7f2928565700 1 -- 172.21.15.107:0/111191774 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2920066730 con 0x7f29200fcf90 2024-05-17T21:51:53.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:53.395+0000 7f2916ffd700 1 -- 172.21.15.107:0/111191774 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 23) v1 ==== 50176+0+0 (secure 0 0 0) 0x7f2910026410 con 0x7f29200fcf90 2024-05-17T21:51:53.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:53.396+0000 7f2928565700 1 -- 172.21.15.107:0/111191774 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f292004c710 con 0x7f29200fcf90 2024-05-17T21:51:53.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:53.396+0000 7f2916ffd700 1 --2- 172.21.15.107:0/111191774 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7f290c03e250 0x7f290c040700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:51:53.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:53.396+0000 7f2916ffd700 1 -- 172.21.15.107:0/111191774 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f29100541e0 con 0x7f29200fcf90 2024-05-17T21:51:53.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:53.396+0000 7f2926301700 1 --2- 172.21.15.107:0/111191774 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7f290c03e250 0x7f290c040700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:51:53.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:53.397+0000 7f2926301700 1 --2- 172.21.15.107:0/111191774 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7f290c03e250 0x7f290c040700 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f291c009b80 tx=0x7f291c00b040 comp rx=0 tx=0).ready entity=mgr.14636 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:51:53.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:53.403+0000 7f2916ffd700 1 -- 172.21.15.107:0/111191774 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+190121 (secure 0 0 0) 0x7f29100123c0 con 0x7f29200fcf90 2024-05-17T21:51:53.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:53 smithi138 ceph-mon[35948]: mgrmap e23: smithi138.napnll(active, since 3s) 2024-05-17T21:51:53.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:53 smithi107 ceph-mon[28434]: mgrmap e23: smithi138.napnll(active, since 3s) 2024-05-17T21:51:53.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:53.674+0000 7f2928565700 1 -- 172.21.15.107:0/111191774 --> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f2920066d50 con 0x7f290c03e250 2024-05-17T21:51:53.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:53.702+0000 7f2916ffd700 1 -- 172.21.15.107:0/111191774 <== mgr.14636 v2:172.21.15.138:6828/1369055740 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3388 (secure 0 0 0) 0x7f2920066d50 con 0x7f290c03e250 2024-05-17T21:51:53.696 INFO:teuthology.orchestra.run.smithi107.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-17T21:51:53.696 INFO:teuthology.orchestra.run.smithi107.stdout:alertmanager.smithi107 smithi107 *:9093,9094 running (6m) 2m ago 7m 25.2M - 0.25.0 c8568f914cd2 a2d817bf64d3 2024-05-17T21:51:53.696 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi107 smithi107 running (8m) 2m ago 8m 6660k - 17.2.7-952-g9903f776 d15a36853796 828741c3a89c 2024-05-17T21:51:53.696 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi138 smithi138 running (7m) 9s ago 7m 6660k - 17.2.7-952-g9903f776 d15a36853796 20e9ccf1fadd 2024-05-17T21:51:53.696 INFO:teuthology.orchestra.run.smithi107.stdout:grafana.smithi107 smithi107 *:3000 running (6m) 2m ago 7m 80.5M - 9.4.7 954c08fa6188 0bc11311fb71 2024-05-17T21:51:53.696 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.drqody smithi107 running (2m) 2m ago 2m 14.5M - 17.2.7-952-g9903f776 d15a36853796 0d851bc912f0 2024-05-17T21:51:53.696 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.idmbgj smithi107 running (2m) 2m ago 2m 10.4M - 17.2.7-952-g9903f776 d15a36853796 1348b09ab29e 2024-05-17T21:51:53.696 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi138.gwbtyd smithi138 running (2m) 9s ago 2m 22.2M - 17.2.7-952-g9903f776 d15a36853796 fd2ba75354c0 2024-05-17T21:51:53.696 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi138.zckrvs smithi138 running (2m) 9s ago 2m 11.8M - 17.2.7-952-g9903f776 d15a36853796 a1cfd2fe4cda 2024-05-17T21:51:53.696 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi107.xhhyiw smithi107 *:9283 running (9m) 2m ago 9m 483M - 17.2.7-952-g9903f776 d15a36853796 b566f08938ae 2024-05-17T21:51:53.696 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi138.napnll smithi138 *:8443,9283 running (23s) 9s ago 7m 113M - 19.0.0-3794-g931809a9 8880f7613d02 3f320d02d8e2 2024-05-17T21:51:53.696 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi107 smithi107 running (9m) 2m ago 9m 47.5M 2048M 17.2.7-952-g9903f776 d15a36853796 7de9c23eee97 2024-05-17T21:51:53.697 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi138 smithi138 running (6m) 9s ago 6m 34.1M 2048M 17.2.7-952-g9903f776 d15a36853796 ee61a065778d 2024-05-17T21:51:53.697 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi107 smithi107 *:9100 running (8m) 2m ago 8m 15.4M - 1.5.0 0da6a335fe13 730e9bfe00b2 2024-05-17T21:51:53.697 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi138 smithi138 *:9100 running (7m) 9s ago 7m 15.4M - 1.5.0 0da6a335fe13 296b996124a1 2024-05-17T21:51:53.697 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0 smithi107 running (6m) 2m ago 6m 46.0M 4288M 17.2.7-952-g9903f776 d15a36853796 f2d20009361d 2024-05-17T21:51:53.697 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1 smithi107 running (5m) 2m ago 5m 48.3M 4288M 17.2.7-952-g9903f776 d15a36853796 e5d3e99844a9 2024-05-17T21:51:53.697 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2 smithi107 running (5m) 2m ago 5m 46.5M 4288M 17.2.7-952-g9903f776 d15a36853796 e044d7b78088 2024-05-17T21:51:53.697 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3 smithi138 running (4m) 9s ago 4m 45.6M 5312M 17.2.7-952-g9903f776 d15a36853796 8c980cc8af87 2024-05-17T21:51:53.697 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4 smithi138 running (4m) 9s ago 4m 42.7M 5312M 17.2.7-952-g9903f776 d15a36853796 b306a36a744e 2024-05-17T21:51:53.697 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5 smithi138 running (4m) 9s ago 4m 45.0M 5312M 17.2.7-952-g9903f776 d15a36853796 7c8d8ed74974 2024-05-17T21:51:53.697 INFO:teuthology.orchestra.run.smithi107.stdout:prometheus.smithi107 smithi107 *:9095 running (6m) 2m ago 7m 53.9M - 2.43.0 a07b618ecd1d 38259629b8f2 2024-05-17T21:51:53.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:53.706+0000 7f2928565700 1 -- 172.21.15.107:0/111191774 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7f290c03e250 msgr2=0x7f290c040700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:51:53.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:53.706+0000 7f2928565700 1 --2- 172.21.15.107:0/111191774 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7f290c03e250 0x7f290c040700 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f291c009b80 tx=0x7f291c00b040 comp rx=0 tx=0).stop 2024-05-17T21:51:53.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:53.706+0000 7f2928565700 1 -- 172.21.15.107:0/111191774 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29200fcf90 msgr2=0x7f292010cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:51:53.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:53.706+0000 7f2928565700 1 --2- 172.21.15.107:0/111191774 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29200fcf90 0x7f292010cce0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f2910013870 tx=0x7f2910009e30 comp rx=0 tx=0).stop 2024-05-17T21:51:53.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:53.706+0000 7f2928565700 1 -- 172.21.15.107:0/111191774 shutdown_connections 2024-05-17T21:51:53.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:53.706+0000 7f2928565700 1 --2- 172.21.15.107:0/111191774 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7f290c03e250 0x7f290c040700 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:51:53.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:53.706+0000 7f2928565700 1 --2- 172.21.15.107:0/111191774 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f29200fbce0 0x7f292010c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:51:53.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:53.706+0000 7f2928565700 1 --2- 172.21.15.107:0/111191774 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29200fcf90 0x7f292010cce0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:51:53.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:53.706+0000 7f2928565700 1 -- 172.21.15.107:0/111191774 >> 172.21.15.107:0/111191774 conn(0x7f29200f7420 msgr2=0x7f29200f8a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:51:53.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:53.706+0000 7f2928565700 1 -- 172.21.15.107:0/111191774 shutdown_connections 2024-05-17T21:51:53.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:53.706+0000 7f2928565700 1 -- 172.21.15.107:0/111191774 wait complete. 2024-05-17T21:51:53.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:53.828+0000 7f01e4166700 1 -- 172.21.15.107:0/833228344 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01dc0fff10 msgr2=0x7f01dc102330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:51:53.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:53.828+0000 7f01e4166700 1 --2- 172.21.15.107:0/833228344 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01dc0fff10 0x7f01dc102330 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f01cc004940 tx=0x7f01cc009e30 comp rx=0 tx=0).stop 2024-05-17T21:51:53.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:53.828+0000 7f01e4166700 1 -- 172.21.15.107:0/833228344 shutdown_connections 2024-05-17T21:51:53.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:53.828+0000 7f01e4166700 1 --2- 172.21.15.107:0/833228344 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01dc0fff10 0x7f01dc102330 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:51:53.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:53.828+0000 7f01e4166700 1 --2- 172.21.15.107:0/833228344 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f01dc066ed0 0x7f01dc0ff9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:51:53.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:53.828+0000 7f01e4166700 1 -- 172.21.15.107:0/833228344 >> 172.21.15.107:0/833228344 conn(0x7f01dc0f7420 msgr2=0x7f01dc0f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:51:53.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:53.828+0000 7f01e4166700 1 -- 172.21.15.107:0/833228344 shutdown_connections 2024-05-17T21:51:53.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:53.828+0000 7f01e4166700 1 -- 172.21.15.107:0/833228344 wait complete. 2024-05-17T21:51:53.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:53.828+0000 7f01e4166700 1 Processor -- start 2024-05-17T21:51:53.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:53.829+0000 7f01e4166700 1 -- start start 2024-05-17T21:51:53.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:53.829+0000 7f01e4166700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01dc066ed0 0x7f01dc10d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:51:53.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:53.829+0000 7f01e4166700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f01dc0fff10 0x7f01dc10de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:51:53.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:53.829+0000 7f01e4166700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01dc10e450 con 0x7f01dc066ed0 2024-05-17T21:51:53.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:53.829+0000 7f01e4166700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01dc10e5c0 con 0x7f01dc0fff10 2024-05-17T21:51:53.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:53.829+0000 7f01e1f02700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01dc066ed0 0x7f01dc10d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:51:53.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:53.829+0000 7f01e1f02700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01dc066ed0 0x7f01dc10d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45576/0 (socket says 172.21.15.107:45576) 2024-05-17T21:51:53.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:53.829+0000 7f01e1f02700 1 -- 172.21.15.107:0/1233833864 learned_addr learned my addr 172.21.15.107:0/1233833864 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:51:53.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:53.829+0000 7f01e1701700 1 --2- 172.21.15.107:0/1233833864 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f01dc0fff10 0x7f01dc10de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:51:53.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:53.829+0000 7f01e1f02700 1 -- 172.21.15.107:0/1233833864 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f01dc0fff10 msgr2=0x7f01dc10de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:51:53.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:53.829+0000 7f01e1f02700 1 --2- 172.21.15.107:0/1233833864 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f01dc0fff10 0x7f01dc10de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:51:53.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:53.829+0000 7f01e1f02700 1 -- 172.21.15.107:0/1233833864 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01cc005040 con 0x7f01dc066ed0 2024-05-17T21:51:53.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:53.829+0000 7f01e1f02700 1 --2- 172.21.15.107:0/1233833864 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01dc066ed0 0x7f01dc10d910 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f01d800eb40 tx=0x7f01d800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:51:53.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:53.830+0000 7f01d27fc700 1 -- 172.21.15.107:0/1233833864 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f01d8007cc0 con 0x7f01dc066ed0 2024-05-17T21:51:53.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:53.830+0000 7f01d27fc700 1 -- 172.21.15.107:0/1233833864 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f01d800f040 con 0x7f01dc066ed0 2024-05-17T21:51:53.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:53.830+0000 7f01d27fc700 1 -- 172.21.15.107:0/1233833864 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f01d800c9d0 con 0x7f01dc066ed0 2024-05-17T21:51:53.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:53.830+0000 7f01e4166700 1 -- 172.21.15.107:0/1233833864 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01dc112f70 con 0x7f01dc066ed0 2024-05-17T21:51:53.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:53.830+0000 7f01e4166700 1 -- 172.21.15.107:0/1233833864 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01dc113420 con 0x7f01dc066ed0 2024-05-17T21:51:53.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:53.831+0000 7f01e4166700 1 -- 172.21.15.107:0/1233833864 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01dc04c710 con 0x7f01dc066ed0 2024-05-17T21:51:53.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:53.831+0000 7f01d27fc700 1 -- 172.21.15.107:0/1233833864 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 23) v1 ==== 50176+0+0 (secure 0 0 0) 0x7f01d80090d0 con 0x7f01dc066ed0 2024-05-17T21:51:53.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:53.831+0000 7f01d27fc700 1 --2- 172.21.15.107:0/1233833864 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7f01c803e2a0 0x7f01c8040750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:51:53.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:53.831+0000 7f01d27fc700 1 -- 172.21.15.107:0/1233833864 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f01d8052a10 con 0x7f01dc066ed0 2024-05-17T21:51:53.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:53.832+0000 7f01e1701700 1 --2- 172.21.15.107:0/1233833864 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7f01c803e2a0 0x7f01c8040750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:51:53.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:53.832+0000 7f01e1701700 1 --2- 172.21.15.107:0/1233833864 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7f01c803e2a0 0x7f01c8040750 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f01cc00b0d0 tx=0x7f01cc009e30 comp rx=0 tx=0).ready entity=mgr.14636 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:51:53.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:53.836+0000 7f01d27fc700 1 -- 172.21.15.107:0/1233833864 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+190121 (secure 0 0 0) 0x7f01d801d3b0 con 0x7f01dc066ed0 2024-05-17T21:51:54.192 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:54.199+0000 7f01e4166700 1 -- 172.21.15.107:0/1233833864 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f01dc113a50 con 0x7f01dc066ed0 2024-05-17T21:51:54.192 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:54.200+0000 7f01d27fc700 1 -- 172.21.15.107:0/1233833864 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+739 (secure 0 0 0) 0x7f01d801b2c0 con 0x7f01dc066ed0 2024-05-17T21:51:54.192 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-05-17T21:51:54.192 INFO:teuthology.orchestra.run.smithi107.stdout: "mon": { 2024-05-17T21:51:54.192 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-952-g9903f776 (9903f7767386a9d9efadb7c1effdcd3064d71804) quincy (stable)": 2 2024-05-17T21:51:54.193 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-05-17T21:51:54.193 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr": { 2024-05-17T21:51:54.193 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev)": 1 2024-05-17T21:51:54.193 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-05-17T21:51:54.193 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": { 2024-05-17T21:51:54.193 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-952-g9903f776 (9903f7767386a9d9efadb7c1effdcd3064d71804) quincy (stable)": 6 2024-05-17T21:51:54.193 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-05-17T21:51:54.193 INFO:teuthology.orchestra.run.smithi107.stdout: "mds": { 2024-05-17T21:51:54.193 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-952-g9903f776 (9903f7767386a9d9efadb7c1effdcd3064d71804) quincy (stable)": 4 2024-05-17T21:51:54.193 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-05-17T21:51:54.193 INFO:teuthology.orchestra.run.smithi107.stdout: "overall": { 2024-05-17T21:51:54.193 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-952-g9903f776 (9903f7767386a9d9efadb7c1effdcd3064d71804) quincy (stable)": 12, 2024-05-17T21:51:54.193 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev)": 1 2024-05-17T21:51:54.193 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-05-17T21:51:54.194 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-05-17T21:51:54.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:54.202+0000 7f01e4166700 1 -- 172.21.15.107:0/1233833864 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7f01c803e2a0 msgr2=0x7f01c8040750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:51:54.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:54.202+0000 7f01e4166700 1 --2- 172.21.15.107:0/1233833864 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7f01c803e2a0 0x7f01c8040750 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f01cc00b0d0 tx=0x7f01cc009e30 comp rx=0 tx=0).stop 2024-05-17T21:51:54.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:54.202+0000 7f01e4166700 1 -- 172.21.15.107:0/1233833864 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01dc066ed0 msgr2=0x7f01dc10d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:51:54.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:54.202+0000 7f01e4166700 1 --2- 172.21.15.107:0/1233833864 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01dc066ed0 0x7f01dc10d910 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f01d800eb40 tx=0x7f01d800b040 comp rx=0 tx=0).stop 2024-05-17T21:51:54.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:54.202+0000 7f01e4166700 1 -- 172.21.15.107:0/1233833864 shutdown_connections 2024-05-17T21:51:54.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:54.202+0000 7f01e4166700 1 --2- 172.21.15.107:0/1233833864 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7f01c803e2a0 0x7f01c8040750 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:51:54.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:54.202+0000 7f01e4166700 1 --2- 172.21.15.107:0/1233833864 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01dc066ed0 0x7f01dc10d910 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:51:54.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:54.202+0000 7f01e4166700 1 --2- 172.21.15.107:0/1233833864 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f01dc0fff10 0x7f01dc10de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:51:54.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:54.202+0000 7f01e4166700 1 -- 172.21.15.107:0/1233833864 >> 172.21.15.107:0/1233833864 conn(0x7f01dc0f7420 msgr2=0x7f01dc0f7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:51:54.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:54.203+0000 7f01e4166700 1 -- 172.21.15.107:0/1233833864 shutdown_connections 2024-05-17T21:51:54.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:54.203+0000 7f01e4166700 1 -- 172.21.15.107:0/1233833864 wait complete. 2024-05-17T21:51:54.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:54.349+0000 7fbf6b954700 1 -- 172.21.15.107:0/1349438616 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf640fcf60 msgr2=0x7fbf640fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:51:54.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:54.349+0000 7fbf6b954700 1 --2- 172.21.15.107:0/1349438616 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf640fcf60 0x7fbf640fd3d0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fbf54004940 tx=0x7fbf54009e30 comp rx=0 tx=0).stop 2024-05-17T21:51:54.342 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:54 smithi107 ceph-mon[28434]: from='client.14666 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:51:54.342 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:54 smithi107 ceph-mon[28434]: from='client.14670 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:51:54.342 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:54 smithi107 ceph-mon[28434]: pgmap v5: 65 pgs: 65 active+clean; 13 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-17T21:51:54.343 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:54 smithi107 ceph-mon[28434]: from='client.14674 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:51:54.343 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:54 smithi107 ceph-mon[28434]: from='client.? 172.21.15.107:0/1233833864' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:51:54.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:54.349+0000 7fbf6b954700 1 -- 172.21.15.107:0/1349438616 shutdown_connections 2024-05-17T21:51:54.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:54.349+0000 7fbf6b954700 1 --2- 172.21.15.107:0/1349438616 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf640fcf60 0x7fbf640fd3d0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:51:54.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:54.349+0000 7fbf6b954700 1 --2- 172.21.15.107:0/1349438616 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf640fbcb0 0x7fbf640fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:51:54.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:54.349+0000 7fbf6b954700 1 -- 172.21.15.107:0/1349438616 >> 172.21.15.107:0/1349438616 conn(0x7fbf640f7430 msgr2=0x7fbf640f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:51:54.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:54.350+0000 7fbf6b954700 1 -- 172.21.15.107:0/1349438616 shutdown_connections 2024-05-17T21:51:54.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:54.350+0000 7fbf6b954700 1 -- 172.21.15.107:0/1349438616 wait complete. 2024-05-17T21:51:54.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:54.350+0000 7fbf6b954700 1 Processor -- start 2024-05-17T21:51:54.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:54.351+0000 7fbf6b954700 1 -- start start 2024-05-17T21:51:54.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:54.351+0000 7fbf6b954700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf640fbcb0 0x7fbf6410d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:51:54.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:54.351+0000 7fbf6b954700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf640fcf60 0x7fbf6410df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:51:54.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:54.351+0000 7fbf6b954700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf6410e500 con 0x7fbf640fcf60 2024-05-17T21:51:54.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:54.351+0000 7fbf6b954700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf6410ed50 con 0x7fbf640fbcb0 2024-05-17T21:51:54.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:54.351+0000 7fbf696f0700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf640fbcb0 0x7fbf6410d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:51:54.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:54.351+0000 7fbf696f0700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf640fbcb0 0x7fbf6410d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.107:57474/0 (socket says 172.21.15.107:57474) 2024-05-17T21:51:54.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:54.351+0000 7fbf696f0700 1 -- 172.21.15.107:0/1521955854 learned_addr learned my addr 172.21.15.107:0/1521955854 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:51:54.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:54.351+0000 7fbf68eef700 1 --2- 172.21.15.107:0/1521955854 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf640fcf60 0x7fbf6410df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:51:54.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:54.352+0000 7fbf696f0700 1 -- 172.21.15.107:0/1521955854 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf640fcf60 msgr2=0x7fbf6410df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:51:54.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:54.352+0000 7fbf696f0700 1 --2- 172.21.15.107:0/1521955854 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf640fcf60 0x7fbf6410df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:51:54.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:54.352+0000 7fbf696f0700 1 -- 172.21.15.107:0/1521955854 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf54005040 con 0x7fbf640fbcb0 2024-05-17T21:51:54.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:54.352+0000 7fbf696f0700 1 --2- 172.21.15.107:0/1521955854 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf640fbcb0 0x7fbf6410d9c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbf6000eb40 tx=0x7fbf6000b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:51:54.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:54.352+0000 7fbf59ffb700 1 -- 172.21.15.107:0/1521955854 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbf60007cc0 con 0x7fbf640fbcb0 2024-05-17T21:51:54.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:54.352+0000 7fbf59ffb700 1 -- 172.21.15.107:0/1521955854 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbf6000f040 con 0x7fbf640fbcb0 2024-05-17T21:51:54.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:54.352+0000 7fbf59ffb700 1 -- 172.21.15.107:0/1521955854 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbf6000ca10 con 0x7fbf640fbcb0 2024-05-17T21:51:54.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:54.353+0000 7fbf6b954700 1 -- 172.21.15.107:0/1521955854 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf6410efd0 con 0x7fbf640fbcb0 2024-05-17T21:51:54.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:54.353+0000 7fbf6b954700 1 -- 172.21.15.107:0/1521955854 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf6410f450 con 0x7fbf640fbcb0 2024-05-17T21:51:54.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:54.354+0000 7fbf59ffb700 1 -- 172.21.15.107:0/1521955854 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 24) v1 ==== 50232+0+0 (secure 0 0 0) 0x7fbf6000cb70 con 0x7fbf640fbcb0 2024-05-17T21:51:54.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:54.355+0000 7fbf6b954700 1 -- 172.21.15.107:0/1521955854 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf64101af0 con 0x7fbf640fbcb0 2024-05-17T21:51:54.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:54.355+0000 7fbf59ffb700 1 --2- 172.21.15.107:0/1521955854 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7fbf5003e380 0x7fbf50040830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:51:54.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:54.356+0000 7fbf59ffb700 1 -- 172.21.15.107:0/1521955854 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7fbf60052d20 con 0x7fbf640fbcb0 2024-05-17T21:51:54.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:54.356+0000 7fbf68eef700 1 --2- 172.21.15.107:0/1521955854 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7fbf5003e380 0x7fbf50040830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:51:54.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:54.356+0000 7fbf68eef700 1 --2- 172.21.15.107:0/1521955854 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7fbf5003e380 0x7fbf50040830 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fbf5400b0d0 tx=0x7fbf54009e30 comp rx=0 tx=0).ready entity=mgr.14636 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:51:54.355 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:54.362+0000 7fbf59ffb700 1 -- 172.21.15.107:0/1521955854 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+190121 (secure 0 0 0) 0x7fbf60057070 con 0x7fbf640fbcb0 2024-05-17T21:51:54.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:54 smithi138 ceph-mon[35948]: from='client.14666 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:51:54.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:54 smithi138 ceph-mon[35948]: from='client.14670 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:51:54.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:54 smithi138 ceph-mon[35948]: pgmap v5: 65 pgs: 65 active+clean; 13 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-17T21:51:54.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:54 smithi138 ceph-mon[35948]: from='client.14674 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:51:54.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:54 smithi138 ceph-mon[35948]: from='client.? 172.21.15.107:0/1233833864' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:51:54.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:54.677+0000 7fbf6b954700 1 -- 172.21.15.107:0/1521955854 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fbf64101510 con 0x7fbf640fbcb0 2024-05-17T21:51:54.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:54.678+0000 7fbf59ffb700 1 -- 172.21.15.107:0/1521955854 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1840 (secure 0 0 0) 0x7fbf60012900 con 0x7fbf640fbcb0 2024-05-17T21:51:54.670 INFO:teuthology.orchestra.run.smithi107.stdout:e15 2024-05-17T21:51:54.671 INFO:teuthology.orchestra.run.smithi107.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-17T21:51:54.671 INFO:teuthology.orchestra.run.smithi107.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-17T21:51:54.671 INFO:teuthology.orchestra.run.smithi107.stdout:legacy client fscid: 1 2024-05-17T21:51:54.671 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:51:54.671 INFO:teuthology.orchestra.run.smithi107.stdout:Filesystem 'cephfs' (1) 2024-05-17T21:51:54.671 INFO:teuthology.orchestra.run.smithi107.stdout:fs_name cephfs 2024-05-17T21:51:54.671 INFO:teuthology.orchestra.run.smithi107.stdout:epoch 15 2024-05-17T21:51:54.671 INFO:teuthology.orchestra.run.smithi107.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-17T21:51:54.671 INFO:teuthology.orchestra.run.smithi107.stdout:created 2024-05-17T21:49:16.836333+0000 2024-05-17T21:51:54.671 INFO:teuthology.orchestra.run.smithi107.stdout:modified 2024-05-17T21:49:34.873628+0000 2024-05-17T21:51:54.671 INFO:teuthology.orchestra.run.smithi107.stdout:tableserver 0 2024-05-17T21:51:54.671 INFO:teuthology.orchestra.run.smithi107.stdout:root 0 2024-05-17T21:51:54.671 INFO:teuthology.orchestra.run.smithi107.stdout:session_timeout 60 2024-05-17T21:51:54.671 INFO:teuthology.orchestra.run.smithi107.stdout:session_autoclose 300 2024-05-17T21:51:54.672 INFO:teuthology.orchestra.run.smithi107.stdout:max_file_size 1099511627776 2024-05-17T21:51:54.672 INFO:teuthology.orchestra.run.smithi107.stdout:required_client_features {} 2024-05-17T21:51:54.672 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure 0 2024-05-17T21:51:54.672 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure_osd_epoch 46 2024-05-17T21:51:54.672 INFO:teuthology.orchestra.run.smithi107.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-17T21:51:54.672 INFO:teuthology.orchestra.run.smithi107.stdout:max_mds 1 2024-05-17T21:51:54.672 INFO:teuthology.orchestra.run.smithi107.stdout:in 0 2024-05-17T21:51:54.672 INFO:teuthology.orchestra.run.smithi107.stdout:up {0=24259} 2024-05-17T21:51:54.672 INFO:teuthology.orchestra.run.smithi107.stdout:failed 2024-05-17T21:51:54.672 INFO:teuthology.orchestra.run.smithi107.stdout:damaged 2024-05-17T21:51:54.672 INFO:teuthology.orchestra.run.smithi107.stdout:stopped 2024-05-17T21:51:54.672 INFO:teuthology.orchestra.run.smithi107.stdout:data_pools [3] 2024-05-17T21:51:54.672 INFO:teuthology.orchestra.run.smithi107.stdout:metadata_pool 2 2024-05-17T21:51:54.672 INFO:teuthology.orchestra.run.smithi107.stdout:inline_data enabled 2024-05-17T21:51:54.673 INFO:teuthology.orchestra.run.smithi107.stdout:balancer 2024-05-17T21:51:54.673 INFO:teuthology.orchestra.run.smithi107.stdout:standby_count_wanted 1 2024-05-17T21:51:54.673 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi138.gwbtyd{0:24259} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.138:6824/2779634919,v1:172.21.15.138:6825/2779634919] compat {c=[1],r=[1],i=[7ff]}] 2024-05-17T21:51:54.673 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.drqody{0:14486} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.107:6828/2275807100,v1:172.21.15.107:6829/2275807100] compat {c=[1],r=[1],i=[7ff]}] 2024-05-17T21:51:54.673 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:51:54.673 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:51:54.673 INFO:teuthology.orchestra.run.smithi107.stdout:Standby daemons: 2024-05-17T21:51:54.673 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:51:54.673 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.idmbgj{-1:14498} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.107:6826/1460186079,v1:172.21.15.107:6827/1460186079] compat {c=[1],r=[1],i=[7ff]}] 2024-05-17T21:51:54.673 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi138.zckrvs{-1:24267} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.138:6826/170628661,v1:172.21.15.138:6827/170628661] compat {c=[1],r=[1],i=[7ff]}] 2024-05-17T21:51:54.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:54.681+0000 7fbf6b954700 1 -- 172.21.15.107:0/1521955854 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7fbf5003e380 msgr2=0x7fbf50040830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:51:54.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:54.681+0000 7fbf6b954700 1 --2- 172.21.15.107:0/1521955854 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7fbf5003e380 0x7fbf50040830 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fbf5400b0d0 tx=0x7fbf54009e30 comp rx=0 tx=0).stop 2024-05-17T21:51:54.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:54.681+0000 7fbf6b954700 1 -- 172.21.15.107:0/1521955854 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf640fbcb0 msgr2=0x7fbf6410d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:51:54.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:54.681+0000 7fbf6b954700 1 --2- 172.21.15.107:0/1521955854 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf640fbcb0 0x7fbf6410d9c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbf6000eb40 tx=0x7fbf6000b040 comp rx=0 tx=0).stop 2024-05-17T21:51:54.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:54.681+0000 7fbf6b954700 1 -- 172.21.15.107:0/1521955854 shutdown_connections 2024-05-17T21:51:54.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:54.681+0000 7fbf6b954700 1 --2- 172.21.15.107:0/1521955854 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7fbf5003e380 0x7fbf50040830 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:51:54.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:54.681+0000 7fbf6b954700 1 --2- 172.21.15.107:0/1521955854 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf640fbcb0 0x7fbf6410d9c0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:51:54.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:54.681+0000 7fbf6b954700 1 --2- 172.21.15.107:0/1521955854 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf640fcf60 0x7fbf6410df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:51:54.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:54.681+0000 7fbf6b954700 1 -- 172.21.15.107:0/1521955854 >> 172.21.15.107:0/1521955854 conn(0x7fbf640f7430 msgr2=0x7fbf640f8a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:51:54.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:54.681+0000 7fbf6b954700 1 -- 172.21.15.107:0/1521955854 shutdown_connections 2024-05-17T21:51:54.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:54.681+0000 7fbf6b954700 1 -- 172.21.15.107:0/1521955854 wait complete. 2024-05-17T21:51:54.675 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 15 2024-05-17T21:51:54.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:54.812+0000 7f8a8c342700 1 -- 172.21.15.107:0/1560048159 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a840fdbc0 msgr2=0x7f8a840fbfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:51:54.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:54.812+0000 7f8a8c342700 1 --2- 172.21.15.107:0/1560048159 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a840fdbc0 0x7f8a840fbfe0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f8a74004940 tx=0x7f8a74009e30 comp rx=0 tx=0).stop 2024-05-17T21:51:54.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:54.812+0000 7f8a8c342700 1 -- 172.21.15.107:0/1560048159 shutdown_connections 2024-05-17T21:51:54.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:54.812+0000 7f8a8c342700 1 --2- 172.21.15.107:0/1560048159 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a840fdbc0 0x7f8a840fbfe0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:51:54.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:54.812+0000 7f8a8c342700 1 --2- 172.21.15.107:0/1560048159 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a840fd270 0x7f8a840fd680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:51:54.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:54.812+0000 7f8a8c342700 1 -- 172.21.15.107:0/1560048159 >> 172.21.15.107:0/1560048159 conn(0x7f8a840f6b80 msgr2=0x7f8a840f8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:51:54.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:54.812+0000 7f8a8c342700 1 -- 172.21.15.107:0/1560048159 shutdown_connections 2024-05-17T21:51:54.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:54.812+0000 7f8a8c342700 1 -- 172.21.15.107:0/1560048159 wait complete. 2024-05-17T21:51:54.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:54.813+0000 7f8a8c342700 1 Processor -- start 2024-05-17T21:51:54.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:54.813+0000 7f8a8c342700 1 -- start start 2024-05-17T21:51:54.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:54.813+0000 7f8a8c342700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a840fd270 0x7f8a8410c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:51:54.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:54.813+0000 7f8a8c342700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a8410cdd0 0x7f8a8410dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:51:54.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:54.813+0000 7f8a8c342700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a8410d290 con 0x7f8a840fd270 2024-05-17T21:51:54.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:54.813+0000 7f8a8c342700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a8410d3d0 con 0x7f8a8410cdd0 2024-05-17T21:51:54.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:54.813+0000 7f8a8a0de700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a840fd270 0x7f8a8410c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:51:54.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:54.813+0000 7f8a8a0de700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a840fd270 0x7f8a8410c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45602/0 (socket says 172.21.15.107:45602) 2024-05-17T21:51:54.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:54.813+0000 7f8a8a0de700 1 -- 172.21.15.107:0/3913104763 learned_addr learned my addr 172.21.15.107:0/3913104763 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:51:54.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:54.813+0000 7f8a8a0de700 1 -- 172.21.15.107:0/3913104763 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a8410cdd0 msgr2=0x7f8a8410dc20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-17T21:51:54.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:54.813+0000 7f8a8a0de700 1 --2- 172.21.15.107:0/3913104763 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a8410cdd0 0x7f8a8410dc20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:51:54.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:54.813+0000 7f8a8a0de700 1 -- 172.21.15.107:0/3913104763 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a74005040 con 0x7f8a840fd270 2024-05-17T21:51:54.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:54.814+0000 7f8a8a0de700 1 --2- 172.21.15.107:0/3913104763 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a840fd270 0x7f8a8410c890 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f8a8000eb40 tx=0x7f8a8000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:51:54.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:54.814+0000 7f8a7affd700 1 -- 172.21.15.107:0/3913104763 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8a80007cc0 con 0x7f8a840fd270 2024-05-17T21:51:54.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:54.814+0000 7f8a8c342700 1 -- 172.21.15.107:0/3913104763 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a8410e220 con 0x7f8a840fd270 2024-05-17T21:51:54.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:54.814+0000 7f8a7affd700 1 -- 172.21.15.107:0/3913104763 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8a8000f040 con 0x7f8a840fd270 2024-05-17T21:51:54.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:54.814+0000 7f8a7affd700 1 -- 172.21.15.107:0/3913104763 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8a8000cb30 con 0x7f8a840fd270 2024-05-17T21:51:54.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:54.814+0000 7f8a8c342700 1 -- 172.21.15.107:0/3913104763 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a8410e730 con 0x7f8a840fd270 2024-05-17T21:51:54.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:54.815+0000 7f8a8c342700 1 -- 172.21.15.107:0/3913104763 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a8405f5c0 con 0x7f8a840fd270 2024-05-17T21:51:54.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:54.815+0000 7f8a7affd700 1 -- 172.21.15.107:0/3913104763 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) v1 ==== 50232+0+0 (secure 0 0 0) 0x7f8a8000cc90 con 0x7f8a840fd270 2024-05-17T21:51:54.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:54.816+0000 7f8a7affd700 1 --2- 172.21.15.107:0/3913104763 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7f8a7003e330 0x7f8a700407e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:51:54.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:54.816+0000 7f8a7affd700 1 -- 172.21.15.107:0/3913104763 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f8a80053180 con 0x7f8a840fd270 2024-05-17T21:51:54.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:54.816+0000 7f8a898dd700 1 --2- 172.21.15.107:0/3913104763 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7f8a7003e330 0x7f8a700407e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:51:54.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:54.816+0000 7f8a898dd700 1 --2- 172.21.15.107:0/3913104763 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7f8a7003e330 0x7f8a700407e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8a74013870 tx=0x7f8a74009e30 comp rx=0 tx=0).ready entity=mgr.14636 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:51:54.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:54.821+0000 7f8a7affd700 1 -- 172.21.15.107:0/3913104763 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+190121 (secure 0 0 0) 0x7f8a8001c870 con 0x7f8a840fd270 2024-05-17T21:51:55.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:55.095+0000 7f8a8c342700 1 -- 172.21.15.107:0/3913104763 --> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8a840fdbc0 con 0x7f8a7003e330 2024-05-17T21:51:55.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:55.104+0000 7f8a7affd700 1 -- 172.21.15.107:0/3913104763 <== mgr.14636 v2:172.21.15.138:6828/1369055740 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7f8a840fdbc0 con 0x7f8a7003e330 2024-05-17T21:51:55.097 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-05-17T21:51:55.097 INFO:teuthology.orchestra.run.smithi107.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b", 2024-05-17T21:51:55.097 INFO:teuthology.orchestra.run.smithi107.stdout: "in_progress": true, 2024-05-17T21:51:55.097 INFO:teuthology.orchestra.run.smithi107.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-17T21:51:55.097 INFO:teuthology.orchestra.run.smithi107.stdout: "services_complete": [], 2024-05-17T21:51:55.097 INFO:teuthology.orchestra.run.smithi107.stdout: "progress": "1/21 daemons upgraded", 2024-05-17T21:51:55.097 INFO:teuthology.orchestra.run.smithi107.stdout: "message": "", 2024-05-17T21:51:55.097 INFO:teuthology.orchestra.run.smithi107.stdout: "is_paused": false 2024-05-17T21:51:55.097 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-05-17T21:51:55.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:55.106+0000 7f8a8c342700 1 -- 172.21.15.107:0/3913104763 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7f8a7003e330 msgr2=0x7f8a700407e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:51:55.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:55.106+0000 7f8a8c342700 1 --2- 172.21.15.107:0/3913104763 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7f8a7003e330 0x7f8a700407e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8a74013870 tx=0x7f8a74009e30 comp rx=0 tx=0).stop 2024-05-17T21:51:55.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:55.106+0000 7f8a8c342700 1 -- 172.21.15.107:0/3913104763 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a840fd270 msgr2=0x7f8a8410c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:51:55.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:55.106+0000 7f8a8c342700 1 --2- 172.21.15.107:0/3913104763 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a840fd270 0x7f8a8410c890 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f8a8000eb40 tx=0x7f8a8000b040 comp rx=0 tx=0).stop 2024-05-17T21:51:55.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:55.106+0000 7f8a8c342700 1 -- 172.21.15.107:0/3913104763 shutdown_connections 2024-05-17T21:51:55.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:55.106+0000 7f8a8c342700 1 --2- 172.21.15.107:0/3913104763 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7f8a7003e330 0x7f8a700407e0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:51:55.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:55.106+0000 7f8a8c342700 1 --2- 172.21.15.107:0/3913104763 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a840fd270 0x7f8a8410c890 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:51:55.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:55.106+0000 7f8a8c342700 1 --2- 172.21.15.107:0/3913104763 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a8410cdd0 0x7f8a8410dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:51:55.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:55.106+0000 7f8a8c342700 1 -- 172.21.15.107:0/3913104763 >> 172.21.15.107:0/3913104763 conn(0x7f8a840f6b80 msgr2=0x7f8a840f81d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:51:55.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:55.106+0000 7f8a8c342700 1 -- 172.21.15.107:0/3913104763 shutdown_connections 2024-05-17T21:51:55.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:55.106+0000 7f8a8c342700 1 -- 172.21.15.107:0/3913104763 wait complete. 2024-05-17T21:51:55.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:55.211+0000 7fe11176e700 1 -- 172.21.15.107:0/706962948 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe10c0fd9c0 msgr2=0x7fe10c0ffda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:51:55.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:55.211+0000 7fe11176e700 1 --2- 172.21.15.107:0/706962948 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe10c0fd9c0 0x7fe10c0ffda0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc004940 tx=0x7fe0fc009e30 comp rx=0 tx=0).stop 2024-05-17T21:51:55.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:55.211+0000 7fe11176e700 1 -- 172.21.15.107:0/706962948 shutdown_connections 2024-05-17T21:51:55.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:55.211+0000 7fe11176e700 1 --2- 172.21.15.107:0/706962948 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe10c1002e0 0x7fe10c1026c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:51:55.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:55.211+0000 7fe11176e700 1 --2- 172.21.15.107:0/706962948 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe10c0fd9c0 0x7fe10c0ffda0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:51:55.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:55.211+0000 7fe11176e700 1 -- 172.21.15.107:0/706962948 >> 172.21.15.107:0/706962948 conn(0x7fe10c0f7440 msgr2=0x7fe10c0f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:51:55.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:55.211+0000 7fe11176e700 1 -- 172.21.15.107:0/706962948 shutdown_connections 2024-05-17T21:51:55.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:55.211+0000 7fe11176e700 1 -- 172.21.15.107:0/706962948 wait complete. 2024-05-17T21:51:55.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:55.212+0000 7fe11176e700 1 Processor -- start 2024-05-17T21:51:55.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:55.212+0000 7fe11176e700 1 -- start start 2024-05-17T21:51:55.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:55.212+0000 7fe11176e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe10c0fd9c0 0x7fe10c10da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:51:55.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:55.212+0000 7fe11176e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe10c1002e0 0x7fe10c10df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:51:55.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:55.212+0000 7fe11176e700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe10c10e550 con 0x7fe10c1002e0 2024-05-17T21:51:55.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:55.212+0000 7fe11176e700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe10c10ed50 con 0x7fe10c0fd9c0 2024-05-17T21:51:55.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:55.212+0000 7fe10a7fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe10c1002e0 0x7fe10c10df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:51:55.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:55.212+0000 7fe10a7fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe10c1002e0 0x7fe10c10df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45610/0 (socket says 172.21.15.107:45610) 2024-05-17T21:51:55.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:55.212+0000 7fe10a7fc700 1 -- 172.21.15.107:0/3274946955 learned_addr learned my addr 172.21.15.107:0/3274946955 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:51:55.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:55.212+0000 7fe10affd700 1 --2- 172.21.15.107:0/3274946955 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe10c0fd9c0 0x7fe10c10da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:51:55.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:55.212+0000 7fe10a7fc700 1 -- 172.21.15.107:0/3274946955 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe10c0fd9c0 msgr2=0x7fe10c10da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:51:55.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:55.212+0000 7fe10a7fc700 1 --2- 172.21.15.107:0/3274946955 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe10c0fd9c0 0x7fe10c10da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:51:55.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:55.212+0000 7fe10a7fc700 1 -- 172.21.15.107:0/3274946955 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0fc005040 con 0x7fe10c1002e0 2024-05-17T21:51:55.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:55.212+0000 7fe10affd700 1 --2- 172.21.15.107:0/3274946955 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe10c0fd9c0 0x7fe10c10da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T21:51:55.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:55.212+0000 7fe10a7fc700 1 --2- 172.21.15.107:0/3274946955 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe10c1002e0 0x7fe10c10df50 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fe0f4002a80 tx=0x7fe0f40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:51:55.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:55.213+0000 7fe1037fe700 1 -- 172.21.15.107:0/3274946955 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe0f4007cc0 con 0x7fe10c1002e0 2024-05-17T21:51:55.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:55.213+0000 7fe1037fe700 1 -- 172.21.15.107:0/3274946955 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe0f400f040 con 0x7fe10c1002e0 2024-05-17T21:51:55.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:55.213+0000 7fe11176e700 1 -- 172.21.15.107:0/3274946955 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe10c10f030 con 0x7fe10c1002e0 2024-05-17T21:51:55.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:55.213+0000 7fe1037fe700 1 -- 172.21.15.107:0/3274946955 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe0f40096f0 con 0x7fe10c1002e0 2024-05-17T21:51:55.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:55.213+0000 7fe11176e700 1 -- 172.21.15.107:0/3274946955 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe10c10f570 con 0x7fe10c1002e0 2024-05-17T21:51:55.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:55.214+0000 7fe1037fe700 1 -- 172.21.15.107:0/3274946955 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) v1 ==== 50232+0+0 (secure 0 0 0) 0x7fe0f401c400 con 0x7fe10c1002e0 2024-05-17T21:51:55.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:55.214+0000 7fe1037fe700 1 --2- 172.21.15.107:0/3274946955 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7fe0f803e330 0x7fe0f80407e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:51:55.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:55.214+0000 7fe1037fe700 1 -- 172.21.15.107:0/3274946955 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7fe0f40528c0 con 0x7fe10c1002e0 2024-05-17T21:51:55.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:55.214+0000 7fe11176e700 1 -- 172.21.15.107:0/3274946955 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe10c05f5c0 con 0x7fe10c1002e0 2024-05-17T21:51:55.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:55.214+0000 7fe10affd700 1 --2- 172.21.15.107:0/3274946955 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7fe0f803e330 0x7fe0f80407e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:51:55.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:55.214+0000 7fe10affd700 1 --2- 172.21.15.107:0/3274946955 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7fe0f803e330 0x7fe0f80407e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc00b970 tx=0x7fe0fc009e30 comp rx=0 tx=0).ready entity=mgr.14636 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:51:55.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:55.219+0000 7fe1037fe700 1 -- 172.21.15.107:0/3274946955 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+190121 (secure 0 0 0) 0x7fe0f40133f0 con 0x7fe10c1002e0 2024-05-17T21:51:55.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:55 smithi107 ceph-mon[28434]: mgrmap e24: smithi138.napnll(active, since 5s) 2024-05-17T21:51:55.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:55 smithi107 ceph-mon[28434]: from='client.? 172.21.15.107:0/1521955854' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-17T21:51:55.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:55.580+0000 7fe11176e700 1 -- 172.21.15.107:0/3274946955 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fe10c105970 con 0x7fe10c1002e0 2024-05-17T21:51:55.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:55.580+0000 7fe1037fe700 1 -- 172.21.15.107:0/3274946955 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7fe0f40112f0 con 0x7fe10c1002e0 2024-05-17T21:51:55.573 INFO:teuthology.orchestra.run.smithi107.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-17T21:51:55.573 INFO:teuthology.orchestra.run.smithi107.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-17T21:51:55.573 INFO:teuthology.orchestra.run.smithi107.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-17T21:51:55.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:55.582+0000 7fe11176e700 1 -- 172.21.15.107:0/3274946955 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7fe0f803e330 msgr2=0x7fe0f80407e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:51:55.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:55.582+0000 7fe11176e700 1 --2- 172.21.15.107:0/3274946955 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7fe0f803e330 0x7fe0f80407e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc00b970 tx=0x7fe0fc009e30 comp rx=0 tx=0).stop 2024-05-17T21:51:55.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:55.583+0000 7fe11176e700 1 -- 172.21.15.107:0/3274946955 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe10c1002e0 msgr2=0x7fe10c10df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:51:55.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:55.583+0000 7fe11176e700 1 --2- 172.21.15.107:0/3274946955 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe10c1002e0 0x7fe10c10df50 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fe0f4002a80 tx=0x7fe0f40046b0 comp rx=0 tx=0).stop 2024-05-17T21:51:55.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:55.583+0000 7fe11176e700 1 -- 172.21.15.107:0/3274946955 shutdown_connections 2024-05-17T21:51:55.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:55.583+0000 7fe11176e700 1 --2- 172.21.15.107:0/3274946955 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7fe0f803e330 0x7fe0f80407e0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:51:55.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:55.583+0000 7fe11176e700 1 --2- 172.21.15.107:0/3274946955 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe10c0fd9c0 0x7fe10c10da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:51:55.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:55.583+0000 7fe11176e700 1 --2- 172.21.15.107:0/3274946955 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe10c1002e0 0x7fe10c10df50 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:51:55.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:55.583+0000 7fe11176e700 1 -- 172.21.15.107:0/3274946955 >> 172.21.15.107:0/3274946955 conn(0x7fe10c0f7440 msgr2=0x7fe10c0f7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:51:55.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:55.583+0000 7fe11176e700 1 -- 172.21.15.107:0/3274946955 shutdown_connections 2024-05-17T21:51:55.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:51:55.583+0000 7fe11176e700 1 -- 172.21.15.107:0/3274946955 wait complete. 2024-05-17T21:51:55.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:55 smithi138 ceph-mon[35948]: mgrmap e24: smithi138.napnll(active, since 5s) 2024-05-17T21:51:55.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:55 smithi138 ceph-mon[35948]: from='client.? 172.21.15.107:0/1521955854' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-17T21:51:56.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:56 smithi138 ceph-mon[35948]: from='client.14686 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:51:56.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:56 smithi138 ceph-mon[35948]: pgmap v6: 65 pgs: 65 active+clean; 18 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 757 KiB/s rd, 1.1 MiB/s wr, 95 op/s 2024-05-17T21:51:56.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:56 smithi138 ceph-mon[35948]: from='client.? 172.21.15.107:0/3274946955' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-17T21:51:56.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:56 smithi107 ceph-mon[28434]: from='client.14686 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:51:56.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:56 smithi107 ceph-mon[28434]: pgmap v6: 65 pgs: 65 active+clean; 18 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 757 KiB/s rd, 1.1 MiB/s wr, 95 op/s 2024-05-17T21:51:56.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:56 smithi107 ceph-mon[28434]: from='client.? 172.21.15.107:0/3274946955' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-17T21:51:57.390 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:57 smithi107 ceph-mon[28434]: Standby manager daemon smithi107.xhhyiw started 2024-05-17T21:51:57.390 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:57 smithi107 ceph-mon[28434]: from='mgr.? 172.21.15.107:0/3509655778' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi107.xhhyiw/crt"}]: dispatch 2024-05-17T21:51:57.390 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:57 smithi107 ceph-mon[28434]: from='mgr.? 172.21.15.107:0/3509655778' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-17T21:51:57.390 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:57 smithi107 ceph-mon[28434]: from='mgr.? 172.21.15.107:0/3509655778' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi107.xhhyiw/key"}]: dispatch 2024-05-17T21:51:57.390 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:57 smithi107 ceph-mon[28434]: from='mgr.? 172.21.15.107:0/3509655778' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-17T21:51:57.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:57 smithi138 ceph-mon[35948]: Standby manager daemon smithi107.xhhyiw started 2024-05-17T21:51:57.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:57 smithi138 ceph-mon[35948]: from='mgr.? 172.21.15.107:0/3509655778' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi107.xhhyiw/crt"}]: dispatch 2024-05-17T21:51:57.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:57 smithi138 ceph-mon[35948]: from='mgr.? 172.21.15.107:0/3509655778' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-17T21:51:57.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:57 smithi138 ceph-mon[35948]: from='mgr.? 172.21.15.107:0/3509655778' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi107.xhhyiw/key"}]: dispatch 2024-05-17T21:51:57.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:57 smithi138 ceph-mon[35948]: from='mgr.? 172.21.15.107:0/3509655778' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-17T21:51:58.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:58 smithi138 ceph-mon[35948]: pgmap v7: 65 pgs: 65 active+clean; 33 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 692 KiB/s rd, 2.9 MiB/s wr, 155 op/s 2024-05-17T21:51:58.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:58 smithi138 ceph-mon[35948]: mgrmap e25: smithi138.napnll(active, since 8s), standbys: smithi107.xhhyiw 2024-05-17T21:51:58.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:51:58 smithi138 ceph-mon[35948]: from='mgr.14636 172.21.15.138:0/3120106723' entity='mgr.smithi138.napnll' cmd=[{"prefix": "mgr metadata", "who": "smithi107.xhhyiw", "id": "smithi107.xhhyiw"}]: dispatch 2024-05-17T21:51:58.595 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:58 smithi107 ceph-mon[28434]: pgmap v7: 65 pgs: 65 active+clean; 33 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 692 KiB/s rd, 2.9 MiB/s wr, 155 op/s 2024-05-17T21:51:58.595 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:58 smithi107 ceph-mon[28434]: mgrmap e25: smithi138.napnll(active, since 8s), standbys: smithi107.xhhyiw 2024-05-17T21:51:58.595 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:51:58 smithi107 ceph-mon[28434]: from='mgr.14636 172.21.15.138:0/3120106723' entity='mgr.smithi138.napnll' cmd=[{"prefix": "mgr metadata", "who": "smithi107.xhhyiw", "id": "smithi107.xhhyiw"}]: dispatch 2024-05-17T21:52:00.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:00 smithi138 ceph-mon[35948]: pgmap v8: 65 pgs: 65 active+clean; 33 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 538 KiB/s rd, 2.3 MiB/s wr, 121 op/s 2024-05-17T21:52:00.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:00 smithi138 ceph-mon[35948]: from='mgr.14636 ' entity='mgr.smithi138.napnll' 2024-05-17T21:52:00.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:00 smithi138 ceph-mon[35948]: from='mgr.14636 ' entity='mgr.smithi138.napnll' 2024-05-17T21:52:00.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:00 smithi107 ceph-mon[28434]: pgmap v8: 65 pgs: 65 active+clean; 33 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 538 KiB/s rd, 2.3 MiB/s wr, 121 op/s 2024-05-17T21:52:00.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:00 smithi107 ceph-mon[28434]: from='mgr.14636 ' entity='mgr.smithi138.napnll' 2024-05-17T21:52:00.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:00 smithi107 ceph-mon[28434]: from='mgr.14636 ' entity='mgr.smithi138.napnll' 2024-05-17T21:52:02.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:01 smithi138 ceph-mon[35948]: from='mgr.14636 ' entity='mgr.smithi138.napnll' 2024-05-17T21:52:02.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:01 smithi138 ceph-mon[35948]: from='mgr.14636 ' entity='mgr.smithi138.napnll' 2024-05-17T21:52:02.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:01 smithi138 ceph-mon[35948]: from='mgr.14636 ' entity='mgr.smithi138.napnll' 2024-05-17T21:52:02.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:01 smithi138 ceph-mon[35948]: from='mgr.14636 ' entity='mgr.smithi138.napnll' 2024-05-17T21:52:02.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:01 smithi138 ceph-mon[35948]: pgmap v9: 65 pgs: 65 active+clean; 54 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 1.7 MiB/s rd, 3.8 MiB/s wr, 166 op/s 2024-05-17T21:52:02.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:01 smithi107 ceph-mon[28434]: from='mgr.14636 ' entity='mgr.smithi138.napnll' 2024-05-17T21:52:02.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:01 smithi107 ceph-mon[28434]: from='mgr.14636 ' entity='mgr.smithi138.napnll' 2024-05-17T21:52:02.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:01 smithi107 ceph-mon[28434]: from='mgr.14636 ' entity='mgr.smithi138.napnll' 2024-05-17T21:52:02.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:01 smithi107 ceph-mon[28434]: from='mgr.14636 ' entity='mgr.smithi138.napnll' 2024-05-17T21:52:02.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:01 smithi107 ceph-mon[28434]: pgmap v9: 65 pgs: 65 active+clean; 54 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 1.7 MiB/s rd, 3.8 MiB/s wr, 166 op/s 2024-05-17T21:52:03.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:03 smithi138 ceph-mon[35948]: from='mgr.14636 ' entity='mgr.smithi138.napnll' 2024-05-17T21:52:03.577 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:03 smithi138 ceph-mon[35948]: from='mgr.14636 ' entity='mgr.smithi138.napnll' 2024-05-17T21:52:03.577 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:03 smithi138 ceph-mon[35948]: from='mgr.14636 172.21.15.138:0/3120106723' entity='mgr.smithi138.napnll' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-17T21:52:03.577 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:03 smithi138 ceph-mon[35948]: from='mgr.14636 ' entity='mgr.smithi138.napnll' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-17T21:52:03.577 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:03 smithi138 ceph-mon[35948]: from='mgr.14636 172.21.15.138:0/3120106723' entity='mgr.smithi138.napnll' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-17T21:52:03.577 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:03 smithi138 ceph-mon[35948]: from='mgr.14636 ' entity='mgr.smithi138.napnll' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-17T21:52:03.577 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:03 smithi138 ceph-mon[35948]: from='mgr.14636 172.21.15.138:0/3120106723' entity='mgr.smithi138.napnll' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-17T21:52:03.577 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:03 smithi138 ceph-mon[35948]: from='mgr.14636 ' entity='mgr.smithi138.napnll' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-17T21:52:03.577 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:03 smithi138 ceph-mon[35948]: Adjusting osd_memory_target on smithi107 to 1557M 2024-05-17T21:52:03.577 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:03 smithi138 ceph-mon[35948]: from='mgr.14636 ' entity='mgr.smithi138.napnll' 2024-05-17T21:52:03.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:03 smithi107 ceph-mon[28434]: from='mgr.14636 ' entity='mgr.smithi138.napnll' 2024-05-17T21:52:03.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:03 smithi107 ceph-mon[28434]: from='mgr.14636 ' entity='mgr.smithi138.napnll' 2024-05-17T21:52:03.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:03 smithi107 ceph-mon[28434]: from='mgr.14636 172.21.15.138:0/3120106723' entity='mgr.smithi138.napnll' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-17T21:52:03.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:03 smithi107 ceph-mon[28434]: from='mgr.14636 ' entity='mgr.smithi138.napnll' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-17T21:52:03.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:03 smithi107 ceph-mon[28434]: from='mgr.14636 172.21.15.138:0/3120106723' entity='mgr.smithi138.napnll' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-17T21:52:03.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:03 smithi107 ceph-mon[28434]: from='mgr.14636 ' entity='mgr.smithi138.napnll' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-17T21:52:03.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:03 smithi107 ceph-mon[28434]: from='mgr.14636 172.21.15.138:0/3120106723' entity='mgr.smithi138.napnll' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-17T21:52:03.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:03 smithi107 ceph-mon[28434]: from='mgr.14636 ' entity='mgr.smithi138.napnll' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-17T21:52:03.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:03 smithi107 ceph-mon[28434]: Adjusting osd_memory_target on smithi107 to 1557M 2024-05-17T21:52:03.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:03 smithi107 ceph-mon[28434]: from='mgr.14636 ' entity='mgr.smithi138.napnll' 2024-05-17T21:52:04.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:04 smithi138 ceph-mon[35948]: pgmap v10: 65 pgs: 65 active+clean; 54 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 1.6 MiB/s rd, 3.5 MiB/s wr, 152 op/s 2024-05-17T21:52:04.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:04 smithi107 ceph-mon[28434]: pgmap v10: 65 pgs: 65 active+clean; 54 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 1.6 MiB/s rd, 3.5 MiB/s wr, 152 op/s 2024-05-17T21:52:05.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:05 smithi138 ceph-mon[35948]: from='mgr.14636 172.21.15.138:0/3120106723' entity='mgr.smithi138.napnll' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:52:05.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:05 smithi107 ceph-mon[28434]: from='mgr.14636 172.21.15.138:0/3120106723' entity='mgr.smithi138.napnll' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:52:06.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:06 smithi138 ceph-mon[35948]: pgmap v11: 65 pgs: 65 active+clean; 68 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 2.3 MiB/s rd, 4.6 MiB/s wr, 182 op/s 2024-05-17T21:52:06.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:06 smithi107 ceph-mon[28434]: pgmap v11: 65 pgs: 65 active+clean; 68 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 2.3 MiB/s rd, 4.6 MiB/s wr, 182 op/s 2024-05-17T21:52:08.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:08 smithi107 ceph-mon[28434]: pgmap v12: 65 pgs: 65 active+clean; 77 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 2.3 MiB/s rd, 5.0 MiB/s wr, 182 op/s 2024-05-17T21:52:08.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:08 smithi138 ceph-mon[35948]: pgmap v12: 65 pgs: 65 active+clean; 77 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 2.3 MiB/s rd, 5.0 MiB/s wr, 182 op/s 2024-05-17T21:52:10.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:10 smithi138 ceph-mon[35948]: pgmap v13: 65 pgs: 65 active+clean; 77 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 2.2 MiB/s rd, 3.7 MiB/s wr, 131 op/s 2024-05-17T21:52:10.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:10 smithi107 ceph-mon[28434]: pgmap v13: 65 pgs: 65 active+clean; 77 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 2.2 MiB/s rd, 3.7 MiB/s wr, 131 op/s 2024-05-17T21:52:12.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:12 smithi138 ceph-mon[35948]: pgmap v14: 65 pgs: 65 active+clean; 99 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 3.2 MiB/s rd, 5.6 MiB/s wr, 228 op/s 2024-05-17T21:52:12.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:12 smithi107 ceph-mon[28434]: pgmap v14: 65 pgs: 65 active+clean; 99 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 3.2 MiB/s rd, 5.6 MiB/s wr, 228 op/s 2024-05-17T21:52:14.612 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:14 smithi138 ceph-mon[35948]: pgmap v15: 65 pgs: 65 active+clean; 99 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 2.0 MiB/s rd, 3.8 MiB/s wr, 166 op/s 2024-05-17T21:52:14.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:14 smithi107 ceph-mon[28434]: pgmap v15: 65 pgs: 65 active+clean; 99 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 2.0 MiB/s rd, 3.8 MiB/s wr, 166 op/s 2024-05-17T21:52:16.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:16 smithi138 ceph-mon[35948]: pgmap v16: 65 pgs: 65 active+clean; 107 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 2.6 MiB/s rd, 4.5 MiB/s wr, 183 op/s 2024-05-17T21:52:16.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:16 smithi107 ceph-mon[28434]: pgmap v16: 65 pgs: 65 active+clean; 107 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 2.6 MiB/s rd, 4.5 MiB/s wr, 183 op/s 2024-05-17T21:52:18.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:18 smithi138 ceph-mon[35948]: from='mgr.14636 ' entity='mgr.smithi138.napnll' 2024-05-17T21:52:18.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:18 smithi138 ceph-mon[35948]: from='mgr.14636 ' entity='mgr.smithi138.napnll' 2024-05-17T21:52:18.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:18 smithi138 ceph-mon[35948]: from='mgr.14636 172.21.15.138:0/3120106723' entity='mgr.smithi138.napnll' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-17T21:52:18.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:18 smithi138 ceph-mon[35948]: from='mgr.14636 ' entity='mgr.smithi138.napnll' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-17T21:52:18.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:18 smithi138 ceph-mon[35948]: from='mgr.14636 172.21.15.138:0/3120106723' entity='mgr.smithi138.napnll' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-17T21:52:18.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:18 smithi138 ceph-mon[35948]: from='mgr.14636 ' entity='mgr.smithi138.napnll' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-17T21:52:18.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:18 smithi138 ceph-mon[35948]: from='mgr.14636 172.21.15.138:0/3120106723' entity='mgr.smithi138.napnll' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-17T21:52:18.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:18 smithi138 ceph-mon[35948]: from='mgr.14636 ' entity='mgr.smithi138.napnll' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-17T21:52:18.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:18 smithi138 ceph-mon[35948]: Adjusting osd_memory_target on smithi138 to 2581M 2024-05-17T21:52:18.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:18 smithi138 ceph-mon[35948]: from='mgr.14636 ' entity='mgr.smithi138.napnll' 2024-05-17T21:52:18.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:18 smithi138 ceph-mon[35948]: from='mgr.14636 172.21.15.138:0/3120106723' entity='mgr.smithi138.napnll' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:52:18.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:18 smithi138 ceph-mon[35948]: from='mgr.14636 172.21.15.138:0/3120106723' entity='mgr.smithi138.napnll' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T21:52:18.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:18 smithi138 ceph-mon[35948]: Updating smithi107:/etc/ceph/ceph.conf 2024-05-17T21:52:18.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:18 smithi138 ceph-mon[35948]: Updating smithi138:/etc/ceph/ceph.conf 2024-05-17T21:52:18.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:18 smithi138 ceph-mon[35948]: pgmap v17: 65 pgs: 65 active+clean; 111 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 2.3 MiB/s rd, 3.7 MiB/s wr, 179 op/s 2024-05-17T21:52:18.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:18 smithi138 ceph-mon[35948]: Updating smithi107:/var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-17T21:52:18.328 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:18 smithi138 ceph-mon[35948]: Updating smithi138:/var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-17T21:52:18.328 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:18 smithi138 ceph-mon[35948]: Updating smithi107:/etc/ceph/ceph.client.admin.keyring 2024-05-17T21:52:18.328 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:18 smithi138 ceph-mon[35948]: Updating smithi138:/etc/ceph/ceph.client.admin.keyring 2024-05-17T21:52:18.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:18 smithi107 ceph-mon[28434]: from='mgr.14636 ' entity='mgr.smithi138.napnll' 2024-05-17T21:52:18.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:18 smithi107 ceph-mon[28434]: from='mgr.14636 ' entity='mgr.smithi138.napnll' 2024-05-17T21:52:18.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:18 smithi107 ceph-mon[28434]: from='mgr.14636 172.21.15.138:0/3120106723' entity='mgr.smithi138.napnll' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-17T21:52:18.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:18 smithi107 ceph-mon[28434]: from='mgr.14636 ' entity='mgr.smithi138.napnll' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-17T21:52:18.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:18 smithi107 ceph-mon[28434]: from='mgr.14636 172.21.15.138:0/3120106723' entity='mgr.smithi138.napnll' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-17T21:52:18.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:18 smithi107 ceph-mon[28434]: from='mgr.14636 ' entity='mgr.smithi138.napnll' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-17T21:52:18.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:18 smithi107 ceph-mon[28434]: from='mgr.14636 172.21.15.138:0/3120106723' entity='mgr.smithi138.napnll' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-17T21:52:18.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:18 smithi107 ceph-mon[28434]: from='mgr.14636 ' entity='mgr.smithi138.napnll' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-17T21:52:18.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:18 smithi107 ceph-mon[28434]: Adjusting osd_memory_target on smithi138 to 2581M 2024-05-17T21:52:18.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:18 smithi107 ceph-mon[28434]: from='mgr.14636 ' entity='mgr.smithi138.napnll' 2024-05-17T21:52:18.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:18 smithi107 ceph-mon[28434]: from='mgr.14636 172.21.15.138:0/3120106723' entity='mgr.smithi138.napnll' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:52:18.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:18 smithi107 ceph-mon[28434]: from='mgr.14636 172.21.15.138:0/3120106723' entity='mgr.smithi138.napnll' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T21:52:18.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:18 smithi107 ceph-mon[28434]: Updating smithi107:/etc/ceph/ceph.conf 2024-05-17T21:52:18.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:18 smithi107 ceph-mon[28434]: Updating smithi138:/etc/ceph/ceph.conf 2024-05-17T21:52:18.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:18 smithi107 ceph-mon[28434]: pgmap v17: 65 pgs: 65 active+clean; 111 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 2.3 MiB/s rd, 3.7 MiB/s wr, 179 op/s 2024-05-17T21:52:18.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:18 smithi107 ceph-mon[28434]: Updating smithi107:/var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-17T21:52:18.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:18 smithi107 ceph-mon[28434]: Updating smithi138:/var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-17T21:52:18.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:18 smithi107 ceph-mon[28434]: Updating smithi107:/etc/ceph/ceph.client.admin.keyring 2024-05-17T21:52:18.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:18 smithi107 ceph-mon[28434]: Updating smithi138:/etc/ceph/ceph.client.admin.keyring 2024-05-17T21:52:19.416 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:19 smithi107 ceph-mon[28434]: Updating smithi107:/var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-17T21:52:19.416 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:19 smithi107 ceph-mon[28434]: Updating smithi138:/var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-17T21:52:19.416 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:19 smithi107 ceph-mon[28434]: from='mgr.14636 ' entity='mgr.smithi138.napnll' 2024-05-17T21:52:19.416 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:19 smithi107 ceph-mon[28434]: from='mgr.14636 ' entity='mgr.smithi138.napnll' 2024-05-17T21:52:19.416 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:19 smithi107 ceph-mon[28434]: from='mgr.14636 ' entity='mgr.smithi138.napnll' 2024-05-17T21:52:19.416 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:19 smithi107 ceph-mon[28434]: from='mgr.14636 ' entity='mgr.smithi138.napnll' 2024-05-17T21:52:19.417 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:19 smithi107 ceph-mon[28434]: from='mgr.14636 ' entity='mgr.smithi138.napnll' 2024-05-17T21:52:19.417 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:19 smithi107 ceph-mon[28434]: from='mgr.14636 ' entity='mgr.smithi138.napnll' 2024-05-17T21:52:19.417 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:19 smithi107 ceph-mon[28434]: from='mgr.14636 ' entity='mgr.smithi138.napnll' 2024-05-17T21:52:19.417 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:19 smithi107 ceph-mon[28434]: from='mgr.14636 ' entity='mgr.smithi138.napnll' 2024-05-17T21:52:19.417 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:19 smithi107 ceph-mon[28434]: from='mgr.14636 ' entity='mgr.smithi138.napnll' 2024-05-17T21:52:19.417 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:19 smithi107 ceph-mon[28434]: Reconfiguring node-exporter.smithi107 (dependencies changed)... 2024-05-17T21:52:19.417 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:19 smithi107 ceph-mon[28434]: Reconfiguring daemon node-exporter.smithi107 on smithi107 2024-05-17T21:52:19.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:19 smithi138 ceph-mon[35948]: Updating smithi107:/var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-17T21:52:19.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:19 smithi138 ceph-mon[35948]: Updating smithi138:/var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-17T21:52:19.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:19 smithi138 ceph-mon[35948]: from='mgr.14636 ' entity='mgr.smithi138.napnll' 2024-05-17T21:52:19.577 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:19 smithi138 ceph-mon[35948]: from='mgr.14636 ' entity='mgr.smithi138.napnll' 2024-05-17T21:52:19.577 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:19 smithi138 ceph-mon[35948]: from='mgr.14636 ' entity='mgr.smithi138.napnll' 2024-05-17T21:52:19.577 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:19 smithi138 ceph-mon[35948]: from='mgr.14636 ' entity='mgr.smithi138.napnll' 2024-05-17T21:52:19.577 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:19 smithi138 ceph-mon[35948]: from='mgr.14636 ' entity='mgr.smithi138.napnll' 2024-05-17T21:52:19.577 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:19 smithi138 ceph-mon[35948]: from='mgr.14636 ' entity='mgr.smithi138.napnll' 2024-05-17T21:52:19.577 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:19 smithi138 ceph-mon[35948]: from='mgr.14636 ' entity='mgr.smithi138.napnll' 2024-05-17T21:52:19.577 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:19 smithi138 ceph-mon[35948]: from='mgr.14636 ' entity='mgr.smithi138.napnll' 2024-05-17T21:52:19.577 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:19 smithi138 ceph-mon[35948]: from='mgr.14636 ' entity='mgr.smithi138.napnll' 2024-05-17T21:52:19.577 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:19 smithi138 ceph-mon[35948]: Reconfiguring node-exporter.smithi107 (dependencies changed)... 2024-05-17T21:52:19.577 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:19 smithi138 ceph-mon[35948]: Reconfiguring daemon node-exporter.smithi107 on smithi107 2024-05-17T21:52:20.439 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:20 smithi107 ceph-mon[28434]: pgmap v18: 65 pgs: 65 active+clean; 111 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 2.0 MiB/s rd, 2.9 MiB/s wr, 139 op/s 2024-05-17T21:52:20.439 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:20 smithi107 ceph-mon[28434]: from='mgr.14636 172.21.15.138:0/3120106723' entity='mgr.smithi138.napnll' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:52:20.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:20 smithi138 ceph-mon[35948]: pgmap v18: 65 pgs: 65 active+clean; 111 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 2.0 MiB/s rd, 2.9 MiB/s wr, 139 op/s 2024-05-17T21:52:20.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:20 smithi138 ceph-mon[35948]: from='mgr.14636 172.21.15.138:0/3120106723' entity='mgr.smithi138.napnll' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:52:22.365 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:22 smithi107 ceph-mon[28434]: from='mgr.14636 ' entity='mgr.smithi138.napnll' 2024-05-17T21:52:22.365 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:22 smithi107 ceph-mon[28434]: from='mgr.14636 ' entity='mgr.smithi138.napnll' 2024-05-17T21:52:22.365 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:22 smithi107 ceph-mon[28434]: Reconfiguring alertmanager.smithi107 (dependencies changed)... 2024-05-17T21:52:22.366 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:22 smithi107 ceph-mon[28434]: Reconfiguring daemon alertmanager.smithi107 on smithi107 2024-05-17T21:52:22.366 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:22 smithi107 ceph-mon[28434]: pgmap v19: 65 pgs: 65 active+clean; 128 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 2.8 MiB/s rd, 4.3 MiB/s wr, 234 op/s 2024-05-17T21:52:22.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:22 smithi138 ceph-mon[35948]: from='mgr.14636 ' entity='mgr.smithi138.napnll' 2024-05-17T21:52:22.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:22 smithi138 ceph-mon[35948]: from='mgr.14636 ' entity='mgr.smithi138.napnll' 2024-05-17T21:52:22.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:22 smithi138 ceph-mon[35948]: Reconfiguring alertmanager.smithi107 (dependencies changed)... 2024-05-17T21:52:22.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:22 smithi138 ceph-mon[35948]: Reconfiguring daemon alertmanager.smithi107 on smithi107 2024-05-17T21:52:22.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:22 smithi138 ceph-mon[35948]: pgmap v19: 65 pgs: 65 active+clean; 128 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 2.8 MiB/s rd, 4.3 MiB/s wr, 234 op/s 2024-05-17T21:52:24.530 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:24 smithi107 ceph-mon[28434]: pgmap v20: 65 pgs: 65 active+clean; 128 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.9 MiB/s rd, 2.5 MiB/s wr, 137 op/s 2024-05-17T21:52:24.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:24 smithi138 ceph-mon[35948]: pgmap v20: 65 pgs: 65 active+clean; 128 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.9 MiB/s rd, 2.5 MiB/s wr, 137 op/s 2024-05-17T21:52:25.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:25.694+0000 7feb2aa7e700 1 -- 172.21.15.107:0/1718510620 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb240fce90 msgr2=0x7feb240fd2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:52:25.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:25.694+0000 7feb2aa7e700 1 --2- 172.21.15.107:0/1718510620 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb240fce90 0x7feb240fd2a0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7feb14009ce0 tx=0x7feb14004d10 comp rx=0 tx=0).stop 2024-05-17T21:52:25.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:25.694+0000 7feb2aa7e700 1 -- 172.21.15.107:0/1718510620 shutdown_connections 2024-05-17T21:52:25.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:25.694+0000 7feb2aa7e700 1 --2- 172.21.15.107:0/1718510620 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb240fe090 0x7feb240fe500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:52:25.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:25.694+0000 7feb2aa7e700 1 --2- 172.21.15.107:0/1718510620 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb240fce90 0x7feb240fd2a0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:52:25.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:25.694+0000 7feb2aa7e700 1 -- 172.21.15.107:0/1718510620 >> 172.21.15.107:0/1718510620 conn(0x7feb240f85c0 msgr2=0x7feb240fa9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:52:25.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:25.694+0000 7feb2aa7e700 1 -- 172.21.15.107:0/1718510620 shutdown_connections 2024-05-17T21:52:25.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:25.694+0000 7feb2aa7e700 1 -- 172.21.15.107:0/1718510620 wait complete. 2024-05-17T21:52:25.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:25.695+0000 7feb2aa7e700 1 Processor -- start 2024-05-17T21:52:25.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:25.695+0000 7feb2aa7e700 1 -- start start 2024-05-17T21:52:25.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:25.695+0000 7feb2aa7e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb240fce90 0x7feb24071de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:52:25.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:25.695+0000 7feb2aa7e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb240fe090 0x7feb240702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:52:25.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:25.695+0000 7feb2aa7e700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb24070860 con 0x7feb240fce90 2024-05-17T21:52:25.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:25.695+0000 7feb2aa7e700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb240709d0 con 0x7feb240fe090 2024-05-17T21:52:25.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:25.695+0000 7feb2881a700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb240fce90 0x7feb24071de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:52:25.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:25.695+0000 7feb2881a700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb240fce90 0x7feb24071de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38920/0 (socket says 172.21.15.107:38920) 2024-05-17T21:52:25.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:25.695+0000 7feb2881a700 1 -- 172.21.15.107:0/2597329224 learned_addr learned my addr 172.21.15.107:0/2597329224 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:52:25.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:25.695+0000 7feb1bfff700 1 --2- 172.21.15.107:0/2597329224 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb240fe090 0x7feb240702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:52:25.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:25.696+0000 7feb2881a700 1 -- 172.21.15.107:0/2597329224 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb240fe090 msgr2=0x7feb240702b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:52:25.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:25.696+0000 7feb2881a700 1 --2- 172.21.15.107:0/2597329224 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb240fe090 0x7feb240702b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:52:25.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:25.696+0000 7feb2881a700 1 -- 172.21.15.107:0/2597329224 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb14009940 con 0x7feb240fce90 2024-05-17T21:52:25.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:25.696+0000 7feb1bfff700 1 --2- 172.21.15.107:0/2597329224 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb240fe090 0x7feb240702b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T21:52:25.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:25.696+0000 7feb2881a700 1 --2- 172.21.15.107:0/2597329224 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb240fce90 0x7feb24071de0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7feb14009380 tx=0x7feb14004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:52:25.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:25.696+0000 7feb197fa700 1 -- 172.21.15.107:0/2597329224 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feb1400e040 con 0x7feb240fce90 2024-05-17T21:52:25.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:25.696+0000 7feb2aa7e700 1 -- 172.21.15.107:0/2597329224 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb24070c50 con 0x7feb240fce90 2024-05-17T21:52:25.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:25.696+0000 7feb197fa700 1 -- 172.21.15.107:0/2597329224 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7feb1401cd00 con 0x7feb240fce90 2024-05-17T21:52:25.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:25.696+0000 7feb197fa700 1 -- 172.21.15.107:0/2597329224 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feb14012b90 con 0x7feb240fce90 2024-05-17T21:52:25.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:25.696+0000 7feb2aa7e700 1 -- 172.21.15.107:0/2597329224 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb24115aa0 con 0x7feb240fce90 2024-05-17T21:52:25.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:25.697+0000 7feb2aa7e700 1 -- 172.21.15.107:0/2597329224 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb2405f5c0 con 0x7feb240fce90 2024-05-17T21:52:25.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:25.698+0000 7feb197fa700 1 -- 172.21.15.107:0/2597329224 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) v1 ==== 93316+0+0 (secure 0 0 0) 0x7feb1401c830 con 0x7feb240fce90 2024-05-17T21:52:25.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:25.698+0000 7feb197fa700 1 --2- 172.21.15.107:0/2597329224 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7feb0c070bb0 0x7feb0c073060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:52:25.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:25.698+0000 7feb197fa700 1 -- 172.21.15.107:0/2597329224 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7feb14092d90 con 0x7feb240fce90 2024-05-17T21:52:25.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:25.698+0000 7feb1bfff700 1 --2- 172.21.15.107:0/2597329224 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7feb0c070bb0 0x7feb0c073060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:52:25.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:25.699+0000 7feb1bfff700 1 --2- 172.21.15.107:0/2597329224 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7feb0c070bb0 0x7feb0c073060 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7feb20009b80 tx=0x7feb2000b040 comp rx=0 tx=0).ready entity=mgr.14636 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:52:25.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:25.703+0000 7feb197fa700 1 -- 172.21.15.107:0/2597329224 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+190121 (secure 0 0 0) 0x7feb1405b140 con 0x7feb240fce90 2024-05-17T21:52:25.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:25.988+0000 7feb2aa7e700 1 -- 172.21.15.107:0/2597329224 --> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7feb24102450 con 0x7feb0c070bb0 2024-05-17T21:52:25.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:25.990+0000 7feb197fa700 1 -- 172.21.15.107:0/2597329224 <== mgr.14636 v2:172.21.15.138:6828/1369055740 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7feb24102450 con 0x7feb0c070bb0 2024-05-17T21:52:25.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:25.993+0000 7feb2aa7e700 1 -- 172.21.15.107:0/2597329224 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7feb0c070bb0 msgr2=0x7feb0c073060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:52:25.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:25.993+0000 7feb2aa7e700 1 --2- 172.21.15.107:0/2597329224 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7feb0c070bb0 0x7feb0c073060 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7feb20009b80 tx=0x7feb2000b040 comp rx=0 tx=0).stop 2024-05-17T21:52:25.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:25.993+0000 7feb2aa7e700 1 -- 172.21.15.107:0/2597329224 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb240fce90 msgr2=0x7feb24071de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:52:25.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:25.993+0000 7feb2aa7e700 1 --2- 172.21.15.107:0/2597329224 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb240fce90 0x7feb24071de0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7feb14009380 tx=0x7feb14004d10 comp rx=0 tx=0).stop 2024-05-17T21:52:25.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:25.993+0000 7feb2aa7e700 1 -- 172.21.15.107:0/2597329224 shutdown_connections 2024-05-17T21:52:25.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:25.993+0000 7feb2aa7e700 1 --2- 172.21.15.107:0/2597329224 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7feb0c070bb0 0x7feb0c073060 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:52:25.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:25.993+0000 7feb2aa7e700 1 --2- 172.21.15.107:0/2597329224 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb240fce90 0x7feb24071de0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:52:25.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:25.993+0000 7feb2aa7e700 1 --2- 172.21.15.107:0/2597329224 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb240fe090 0x7feb240702b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:52:25.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:25.993+0000 7feb2aa7e700 1 -- 172.21.15.107:0/2597329224 >> 172.21.15.107:0/2597329224 conn(0x7feb240f85c0 msgr2=0x7feb240f9ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:52:25.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:25.993+0000 7feb2aa7e700 1 -- 172.21.15.107:0/2597329224 shutdown_connections 2024-05-17T21:52:25.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:25.993+0000 7feb2aa7e700 1 -- 172.21.15.107:0/2597329224 wait complete. 2024-05-17T21:52:25.999 INFO:teuthology.orchestra.run.smithi107.stdout:true 2024-05-17T21:52:26.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.138+0000 7faa88dd3700 1 -- 172.21.15.107:0/174161362 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa840fed80 msgr2=0x7faa840faec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:52:26.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.138+0000 7faa88dd3700 1 --2- 172.21.15.107:0/174161362 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa840fed80 0x7faa840faec0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7faa78004940 tx=0x7faa78009e30 comp rx=0 tx=0).stop 2024-05-17T21:52:26.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.138+0000 7faa88dd3700 1 -- 172.21.15.107:0/174161362 shutdown_connections 2024-05-17T21:52:26.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.138+0000 7faa88dd3700 1 --2- 172.21.15.107:0/174161362 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa840fed80 0x7faa840faec0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:52:26.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.138+0000 7faa88dd3700 1 --2- 172.21.15.107:0/174161362 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa840fe430 0x7faa840fe840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:52:26.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.138+0000 7faa88dd3700 1 -- 172.21.15.107:0/174161362 >> 172.21.15.107:0/174161362 conn(0x7faa840f6bb0 msgr2=0x7faa840f9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:52:26.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.138+0000 7faa88dd3700 1 -- 172.21.15.107:0/174161362 shutdown_connections 2024-05-17T21:52:26.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.138+0000 7faa88dd3700 1 -- 172.21.15.107:0/174161362 wait complete. 2024-05-17T21:52:26.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.139+0000 7faa88dd3700 1 Processor -- start 2024-05-17T21:52:26.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.139+0000 7faa88dd3700 1 -- start start 2024-05-17T21:52:26.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.139+0000 7faa88dd3700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa840fe430 0x7faa8410d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:52:26.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.139+0000 7faa88dd3700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa840fed80 0x7faa8410deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:52:26.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.139+0000 7faa88dd3700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa8410e4b0 con 0x7faa840fed80 2024-05-17T21:52:26.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.139+0000 7faa88dd3700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa8410e620 con 0x7faa840fe430 2024-05-17T21:52:26.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.140+0000 7faa81d9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa840fed80 0x7faa8410deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:52:26.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.140+0000 7faa81d9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa840fed80 0x7faa8410deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38938/0 (socket says 172.21.15.107:38938) 2024-05-17T21:52:26.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.140+0000 7faa81d9b700 1 -- 172.21.15.107:0/579625610 learned_addr learned my addr 172.21.15.107:0/579625610 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:52:26.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.140+0000 7faa8259c700 1 --2- 172.21.15.107:0/579625610 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa840fe430 0x7faa8410d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:52:26.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.140+0000 7faa81d9b700 1 -- 172.21.15.107:0/579625610 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa840fe430 msgr2=0x7faa8410d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:52:26.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.140+0000 7faa81d9b700 1 --2- 172.21.15.107:0/579625610 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa840fe430 0x7faa8410d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:52:26.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.140+0000 7faa81d9b700 1 -- 172.21.15.107:0/579625610 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa78005040 con 0x7faa840fed80 2024-05-17T21:52:26.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.140+0000 7faa81d9b700 1 --2- 172.21.15.107:0/579625610 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa840fed80 0x7faa8410deb0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7faa78009fd0 tx=0x7faa78009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:52:26.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.140+0000 7faa72ffd700 1 -- 172.21.15.107:0/579625610 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa7800e070 con 0x7faa840fed80 2024-05-17T21:52:26.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.141+0000 7faa88dd3700 1 -- 172.21.15.107:0/579625610 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa84113020 con 0x7faa840fed80 2024-05-17T21:52:26.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.141+0000 7faa72ffd700 1 -- 172.21.15.107:0/579625610 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faa7801c400 con 0x7faa840fed80 2024-05-17T21:52:26.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.141+0000 7faa88dd3700 1 -- 172.21.15.107:0/579625610 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa841134a0 con 0x7faa840fed80 2024-05-17T21:52:26.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.142+0000 7faa72ffd700 1 -- 172.21.15.107:0/579625610 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa780255e0 con 0x7faa840fed80 2024-05-17T21:52:26.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.143+0000 7faa88dd3700 1 -- 172.21.15.107:0/579625610 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa8404c710 con 0x7faa840fed80 2024-05-17T21:52:26.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.144+0000 7faa72ffd700 1 -- 172.21.15.107:0/579625610 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) v1 ==== 93316+0+0 (secure 0 0 0) 0x7faa7801ca90 con 0x7faa840fed80 2024-05-17T21:52:26.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.144+0000 7faa72ffd700 1 --2- 172.21.15.107:0/579625610 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7faa6c070bc0 0x7faa6c073070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:52:26.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.145+0000 7faa8259c700 1 --2- 172.21.15.107:0/579625610 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7faa6c070bc0 0x7faa6c073070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:52:26.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.145+0000 7faa72ffd700 1 -- 172.21.15.107:0/579625610 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7faa78091ba0 con 0x7faa840fed80 2024-05-17T21:52:26.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.146+0000 7faa8259c700 1 --2- 172.21.15.107:0/579625610 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7faa6c070bc0 0x7faa6c073070 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7faa740095c0 tx=0x7faa7400b040 comp rx=0 tx=0).ready entity=mgr.14636 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:52:26.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.154+0000 7faa72ffd700 1 -- 172.21.15.107:0/579625610 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+190121 (secure 0 0 0) 0x7faa7805a0f0 con 0x7faa840fed80 2024-05-17T21:52:26.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.439+0000 7faa88dd3700 1 -- 172.21.15.107:0/579625610 --> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7faa84102550 con 0x7faa6c070bc0 2024-05-17T21:52:26.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.440+0000 7faa72ffd700 1 -- 172.21.15.107:0/579625610 <== mgr.14636 v2:172.21.15.138:6828/1369055740 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7faa84102550 con 0x7faa6c070bc0 2024-05-17T21:52:26.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.442+0000 7faa88dd3700 1 -- 172.21.15.107:0/579625610 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7faa6c070bc0 msgr2=0x7faa6c073070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:52:26.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.442+0000 7faa88dd3700 1 --2- 172.21.15.107:0/579625610 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7faa6c070bc0 0x7faa6c073070 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7faa740095c0 tx=0x7faa7400b040 comp rx=0 tx=0).stop 2024-05-17T21:52:26.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.442+0000 7faa88dd3700 1 -- 172.21.15.107:0/579625610 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa840fed80 msgr2=0x7faa8410deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:52:26.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.442+0000 7faa88dd3700 1 --2- 172.21.15.107:0/579625610 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa840fed80 0x7faa8410deb0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7faa78009fd0 tx=0x7faa78009e30 comp rx=0 tx=0).stop 2024-05-17T21:52:26.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.442+0000 7faa88dd3700 1 -- 172.21.15.107:0/579625610 shutdown_connections 2024-05-17T21:52:26.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.442+0000 7faa88dd3700 1 --2- 172.21.15.107:0/579625610 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7faa6c070bc0 0x7faa6c073070 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:52:26.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.442+0000 7faa88dd3700 1 --2- 172.21.15.107:0/579625610 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa840fe430 0x7faa8410d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:52:26.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.442+0000 7faa88dd3700 1 --2- 172.21.15.107:0/579625610 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa840fed80 0x7faa8410deb0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:52:26.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.442+0000 7faa88dd3700 1 -- 172.21.15.107:0/579625610 >> 172.21.15.107:0/579625610 conn(0x7faa840f6bb0 msgr2=0x7faa840f82c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:52:26.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.443+0000 7faa88dd3700 1 -- 172.21.15.107:0/579625610 shutdown_connections 2024-05-17T21:52:26.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.443+0000 7faa88dd3700 1 -- 172.21.15.107:0/579625610 wait complete. 2024-05-17T21:52:26.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.551+0000 7fb1a09c9700 1 -- 172.21.15.107:0/579204588 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb19c0fcf60 msgr2=0x7fb19c0fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:52:26.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.551+0000 7fb1a09c9700 1 --2- 172.21.15.107:0/579204588 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb19c0fcf60 0x7fb19c0fd3d0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fb190004940 tx=0x7fb190009e30 comp rx=0 tx=0).stop 2024-05-17T21:52:26.544 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:26 smithi107 ceph-mon[28434]: pgmap v21: 65 pgs: 65 active+clean; 134 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.9 MiB/s rd, 3.0 MiB/s wr, 184 op/s 2024-05-17T21:52:26.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.551+0000 7fb1a09c9700 1 -- 172.21.15.107:0/579204588 shutdown_connections 2024-05-17T21:52:26.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.551+0000 7fb1a09c9700 1 --2- 172.21.15.107:0/579204588 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb19c0fcf60 0x7fb19c0fd3d0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:52:26.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.551+0000 7fb1a09c9700 1 --2- 172.21.15.107:0/579204588 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb19c0fbcb0 0x7fb19c0fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:52:26.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.551+0000 7fb1a09c9700 1 -- 172.21.15.107:0/579204588 >> 172.21.15.107:0/579204588 conn(0x7fb19c0f7430 msgr2=0x7fb19c0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:52:26.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.551+0000 7fb1a09c9700 1 -- 172.21.15.107:0/579204588 shutdown_connections 2024-05-17T21:52:26.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.551+0000 7fb1a09c9700 1 -- 172.21.15.107:0/579204588 wait complete. 2024-05-17T21:52:26.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.552+0000 7fb1a09c9700 1 Processor -- start 2024-05-17T21:52:26.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.552+0000 7fb1a09c9700 1 -- start start 2024-05-17T21:52:26.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.552+0000 7fb1a09c9700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb19c0fbcb0 0x7fb19c1106c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:52:26.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.552+0000 7fb1a09c9700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb19c0fcf60 0x7fb19c112c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:52:26.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.552+0000 7fb1a09c9700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb19c066330 con 0x7fb19c0fcf60 2024-05-17T21:52:26.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.552+0000 7fb1a09c9700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb19c066470 con 0x7fb19c0fbcb0 2024-05-17T21:52:26.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.552+0000 7fb199d9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb19c0fcf60 0x7fb19c112c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:52:26.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.552+0000 7fb199d9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb19c0fcf60 0x7fb19c112c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38958/0 (socket says 172.21.15.107:38958) 2024-05-17T21:52:26.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.552+0000 7fb199d9b700 1 -- 172.21.15.107:0/3911364084 learned_addr learned my addr 172.21.15.107:0/3911364084 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:52:26.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.552+0000 7fb19a59c700 1 --2- 172.21.15.107:0/3911364084 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb19c0fbcb0 0x7fb19c1106c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:52:26.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.552+0000 7fb199d9b700 1 -- 172.21.15.107:0/3911364084 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb19c0fbcb0 msgr2=0x7fb19c1106c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:52:26.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.552+0000 7fb199d9b700 1 --2- 172.21.15.107:0/3911364084 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb19c0fbcb0 0x7fb19c1106c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:52:26.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.552+0000 7fb199d9b700 1 -- 172.21.15.107:0/3911364084 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb190005040 con 0x7fb19c0fcf60 2024-05-17T21:52:26.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.552+0000 7fb199d9b700 1 --2- 172.21.15.107:0/3911364084 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb19c0fcf60 0x7fb19c112c10 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fb190013870 tx=0x7fb190009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:52:26.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.553+0000 7fb18affd700 1 -- 172.21.15.107:0/3911364084 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb19000e070 con 0x7fb19c0fcf60 2024-05-17T21:52:26.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.553+0000 7fb18affd700 1 -- 172.21.15.107:0/3911364084 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb190013de0 con 0x7fb19c0fcf60 2024-05-17T21:52:26.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.553+0000 7fb1a09c9700 1 -- 172.21.15.107:0/3911364084 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb19c113150 con 0x7fb19c0fcf60 2024-05-17T21:52:26.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.553+0000 7fb18affd700 1 -- 172.21.15.107:0/3911364084 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb19001cea0 con 0x7fb19c0fcf60 2024-05-17T21:52:26.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.553+0000 7fb1a09c9700 1 -- 172.21.15.107:0/3911364084 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb19c113630 con 0x7fb19c0fcf60 2024-05-17T21:52:26.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.554+0000 7fb1a09c9700 1 -- 172.21.15.107:0/3911364084 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb19c05f5c0 con 0x7fb19c0fcf60 2024-05-17T21:52:26.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.554+0000 7fb18affd700 1 -- 172.21.15.107:0/3911364084 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) v1 ==== 93316+0+0 (secure 0 0 0) 0x7fb19001c690 con 0x7fb19c0fcf60 2024-05-17T21:52:26.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.555+0000 7fb18affd700 1 --2- 172.21.15.107:0/3911364084 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7fb184070bb0 0x7fb184073060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:52:26.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.555+0000 7fb18affd700 1 -- 172.21.15.107:0/3911364084 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7fb190092110 con 0x7fb19c0fcf60 2024-05-17T21:52:26.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.555+0000 7fb19a59c700 1 --2- 172.21.15.107:0/3911364084 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7fb184070bb0 0x7fb184073060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:52:26.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.556+0000 7fb19a59c700 1 --2- 172.21.15.107:0/3911364084 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7fb184070bb0 0x7fb184073060 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fb19c065fb0 tx=0x7fb18c00b040 comp rx=0 tx=0).ready entity=mgr.14636 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:52:26.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.560+0000 7fb18affd700 1 -- 172.21.15.107:0/3911364084 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+190121 (secure 0 0 0) 0x7fb19005a540 con 0x7fb19c0fcf60 2024-05-17T21:52:26.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.814+0000 7fb1a09c9700 1 -- 172.21.15.107:0/3911364084 --> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fb19c101290 con 0x7fb184070bb0 2024-05-17T21:52:26.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.829+0000 7fb18affd700 1 -- 172.21.15.107:0/3911364084 <== mgr.14636 v2:172.21.15.138:6828/1369055740 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3388 (secure 0 0 0) 0x7fb19c101290 con 0x7fb184070bb0 2024-05-17T21:52:26.822 INFO:teuthology.orchestra.run.smithi107.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-17T21:52:26.822 INFO:teuthology.orchestra.run.smithi107.stdout:alertmanager.smithi107 smithi107 *:9093,9094 running (7m) 25s ago 8m 25.2M - 0.25.0 c8568f914cd2 a2d817bf64d3 2024-05-17T21:52:26.822 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi107 smithi107 running (8m) 25s ago 8m 6660k - 17.2.7-952-g9903f776 d15a36853796 828741c3a89c 2024-05-17T21:52:26.822 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi138 smithi138 running (7m) 27s ago 7m 6660k - 17.2.7-952-g9903f776 d15a36853796 20e9ccf1fadd 2024-05-17T21:52:26.823 INFO:teuthology.orchestra.run.smithi107.stdout:grafana.smithi107 smithi107 *:3000 running (6m) 25s ago 8m 81.2M - 9.4.7 954c08fa6188 0bc11311fb71 2024-05-17T21:52:26.823 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.drqody smithi107 running (2m) 25s ago 2m 38.1M - 17.2.7-952-g9903f776 d15a36853796 0d851bc912f0 2024-05-17T21:52:26.823 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.idmbgj smithi107 running (3m) 25s ago 3m 11.6M - 17.2.7-952-g9903f776 d15a36853796 1348b09ab29e 2024-05-17T21:52:26.823 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi138.gwbtyd smithi138 running (3m) 27s ago 3m 70.7M - 17.2.7-952-g9903f776 d15a36853796 fd2ba75354c0 2024-05-17T21:52:26.823 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi138.zckrvs smithi138 running (2m) 27s ago 2m 12.4M - 17.2.7-952-g9903f776 d15a36853796 a1cfd2fe4cda 2024-05-17T21:52:26.823 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi107.xhhyiw smithi107 *:9283 running (9m) 25s ago 9m 398M - 17.2.7-952-g9903f776 d15a36853796 b566f08938ae 2024-05-17T21:52:26.823 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi138.napnll smithi138 *:8443,9283 running (56s) 27s ago 7m 570M - 19.0.0-3794-g931809a9 8880f7613d02 3f320d02d8e2 2024-05-17T21:52:26.823 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi107 smithi107 running (9m) 25s ago 10m 51.2M 2048M 17.2.7-952-g9903f776 d15a36853796 7de9c23eee97 2024-05-17T21:52:26.823 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi138 smithi138 running (7m) 27s ago 7m 39.5M 2048M 17.2.7-952-g9903f776 d15a36853796 ee61a065778d 2024-05-17T21:52:26.823 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi107 smithi107 *:9100 starting - - - - 2024-05-17T21:52:26.823 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi138 smithi138 *:9100 running (7m) 27s ago 7m 16.5M - 1.5.0 0da6a335fe13 296b996124a1 2024-05-17T21:52:26.823 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0 smithi107 running (6m) 25s ago 6m 57.7M 1557M 17.2.7-952-g9903f776 d15a36853796 f2d20009361d 2024-05-17T21:52:26.823 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1 smithi107 running (6m) 25s ago 6m 62.5M 1557M 17.2.7-952-g9903f776 d15a36853796 e5d3e99844a9 2024-05-17T21:52:26.823 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2 smithi107 running (5m) 25s ago 5m 54.8M 1557M 17.2.7-952-g9903f776 d15a36853796 e044d7b78088 2024-05-17T21:52:26.824 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3 smithi138 running (5m) 27s ago 5m 56.2M 2581M 17.2.7-952-g9903f776 d15a36853796 8c980cc8af87 2024-05-17T21:52:26.824 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4 smithi138 running (5m) 27s ago 5m 52.6M 2581M 17.2.7-952-g9903f776 d15a36853796 b306a36a744e 2024-05-17T21:52:26.824 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5 smithi138 running (4m) 27s ago 4m 53.9M 2581M 17.2.7-952-g9903f776 d15a36853796 7c8d8ed74974 2024-05-17T21:52:26.824 INFO:teuthology.orchestra.run.smithi107.stdout:prometheus.smithi107 smithi107 *:9095 running (6m) 25s ago 8m 58.2M - 2.43.0 a07b618ecd1d 38259629b8f2 2024-05-17T21:52:26.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.832+0000 7fb1a09c9700 1 -- 172.21.15.107:0/3911364084 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7fb184070bb0 msgr2=0x7fb184073060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:52:26.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.832+0000 7fb1a09c9700 1 --2- 172.21.15.107:0/3911364084 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7fb184070bb0 0x7fb184073060 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fb19c065fb0 tx=0x7fb18c00b040 comp rx=0 tx=0).stop 2024-05-17T21:52:26.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.832+0000 7fb1a09c9700 1 -- 172.21.15.107:0/3911364084 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb19c0fcf60 msgr2=0x7fb19c112c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:52:26.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.832+0000 7fb1a09c9700 1 --2- 172.21.15.107:0/3911364084 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb19c0fcf60 0x7fb19c112c10 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fb190013870 tx=0x7fb190009e30 comp rx=0 tx=0).stop 2024-05-17T21:52:26.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.832+0000 7fb1a09c9700 1 -- 172.21.15.107:0/3911364084 shutdown_connections 2024-05-17T21:52:26.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.832+0000 7fb1a09c9700 1 --2- 172.21.15.107:0/3911364084 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7fb184070bb0 0x7fb184073060 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:52:26.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.832+0000 7fb1a09c9700 1 --2- 172.21.15.107:0/3911364084 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb19c0fbcb0 0x7fb19c1106c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:52:26.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.832+0000 7fb1a09c9700 1 --2- 172.21.15.107:0/3911364084 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb19c0fcf60 0x7fb19c112c10 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:52:26.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.832+0000 7fb1a09c9700 1 -- 172.21.15.107:0/3911364084 >> 172.21.15.107:0/3911364084 conn(0x7fb19c0f7430 msgr2=0x7fb19c0f8a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:52:26.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.832+0000 7fb1a09c9700 1 -- 172.21.15.107:0/3911364084 shutdown_connections 2024-05-17T21:52:26.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.832+0000 7fb1a09c9700 1 -- 172.21.15.107:0/3911364084 wait complete. 2024-05-17T21:52:26.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:26 smithi138 ceph-mon[35948]: pgmap v21: 65 pgs: 65 active+clean; 134 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.9 MiB/s rd, 3.0 MiB/s wr, 184 op/s 2024-05-17T21:52:26.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.951+0000 7fa95e040700 1 -- 172.21.15.107:0/1591223227 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa958066260 msgr2=0x7fa9580ffa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:52:26.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.951+0000 7fa95e040700 1 --2- 172.21.15.107:0/1591223227 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa958066260 0x7fa9580ffa00 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fa948004940 tx=0x7fa948009e30 comp rx=0 tx=0).stop 2024-05-17T21:52:26.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.952+0000 7fa95e040700 1 -- 172.21.15.107:0/1591223227 shutdown_connections 2024-05-17T21:52:26.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.952+0000 7fa95e040700 1 --2- 172.21.15.107:0/1591223227 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9580fff40 0x7fa958102320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:52:26.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.952+0000 7fa95e040700 1 --2- 172.21.15.107:0/1591223227 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa958066260 0x7fa9580ffa00 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:52:26.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.952+0000 7fa95e040700 1 -- 172.21.15.107:0/1591223227 >> 172.21.15.107:0/1591223227 conn(0x7fa9580f7430 msgr2=0x7fa9580f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:52:26.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.952+0000 7fa95e040700 1 -- 172.21.15.107:0/1591223227 shutdown_connections 2024-05-17T21:52:26.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.952+0000 7fa95e040700 1 -- 172.21.15.107:0/1591223227 wait complete. 2024-05-17T21:52:26.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.952+0000 7fa95e040700 1 Processor -- start 2024-05-17T21:52:26.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.953+0000 7fa95e040700 1 -- start start 2024-05-17T21:52:26.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.953+0000 7fa95e040700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa958066260 0x7fa95810da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:52:26.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.953+0000 7fa95e040700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9580fff40 0x7fa95810df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:52:26.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.953+0000 7fa95e040700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa95810e570 con 0x7fa9580fff40 2024-05-17T21:52:26.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.953+0000 7fa95e040700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa95810edc0 con 0x7fa958066260 2024-05-17T21:52:26.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.953+0000 7fa956ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9580fff40 0x7fa95810df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:52:26.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.953+0000 7fa956ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9580fff40 0x7fa95810df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38972/0 (socket says 172.21.15.107:38972) 2024-05-17T21:52:26.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.953+0000 7fa9577fe700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa958066260 0x7fa95810da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:52:26.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.953+0000 7fa956ffd700 1 -- 172.21.15.107:0/2788907618 learned_addr learned my addr 172.21.15.107:0/2788907618 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:52:26.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.953+0000 7fa956ffd700 1 -- 172.21.15.107:0/2788907618 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa958066260 msgr2=0x7fa95810da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:52:26.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.953+0000 7fa956ffd700 1 --2- 172.21.15.107:0/2788907618 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa958066260 0x7fa95810da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:52:26.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.953+0000 7fa956ffd700 1 -- 172.21.15.107:0/2788907618 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa948005040 con 0x7fa9580fff40 2024-05-17T21:52:26.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.953+0000 7fa9577fe700 1 --2- 172.21.15.107:0/2788907618 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa958066260 0x7fa95810da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T21:52:26.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.954+0000 7fa956ffd700 1 --2- 172.21.15.107:0/2788907618 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9580fff40 0x7fa95810df70 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fa94c002a50 tx=0x7fa94c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:52:26.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.954+0000 7fa93ffff700 1 -- 172.21.15.107:0/2788907618 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa94c007cc0 con 0x7fa9580fff40 2024-05-17T21:52:26.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.954+0000 7fa93ffff700 1 -- 172.21.15.107:0/2788907618 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa94c00f040 con 0x7fa9580fff40 2024-05-17T21:52:26.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.954+0000 7fa95e040700 1 -- 172.21.15.107:0/2788907618 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa95810f0a0 con 0x7fa9580fff40 2024-05-17T21:52:26.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.954+0000 7fa95e040700 1 -- 172.21.15.107:0/2788907618 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa95810f5e0 con 0x7fa9580fff40 2024-05-17T21:52:26.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.954+0000 7fa93ffff700 1 -- 172.21.15.107:0/2788907618 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa94c00da30 con 0x7fa9580fff40 2024-05-17T21:52:26.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.955+0000 7fa95e040700 1 -- 172.21.15.107:0/2788907618 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa95805f5c0 con 0x7fa9580fff40 2024-05-17T21:52:26.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.956+0000 7fa93ffff700 1 -- 172.21.15.107:0/2788907618 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) v1 ==== 93316+0+0 (secure 0 0 0) 0x7fa94c00db90 con 0x7fa9580fff40 2024-05-17T21:52:26.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.956+0000 7fa93ffff700 1 --2- 172.21.15.107:0/2788907618 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7fa940070bb0 0x7fa940073060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:52:26.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.956+0000 7fa93ffff700 1 -- 172.21.15.107:0/2788907618 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7fa94c090710 con 0x7fa9580fff40 2024-05-17T21:52:26.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.956+0000 7fa9577fe700 1 --2- 172.21.15.107:0/2788907618 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7fa940070bb0 0x7fa940073060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:52:26.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.957+0000 7fa9577fe700 1 --2- 172.21.15.107:0/2788907618 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7fa940070bb0 0x7fa940073060 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fa94800b970 tx=0x7fa948009e30 comp rx=0 tx=0).ready entity=mgr.14636 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:52:26.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:26.962+0000 7fa93ffff700 1 -- 172.21.15.107:0/2788907618 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+190121 (secure 0 0 0) 0x7fa94c058b40 con 0x7fa9580fff40 2024-05-17T21:52:27.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:27.480+0000 7fa95e040700 1 -- 172.21.15.107:0/2788907618 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fa9581058a0 con 0x7fa9580fff40 2024-05-17T21:52:27.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:27.481+0000 7fa93ffff700 1 -- 172.21.15.107:0/2788907618 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+846 (secure 0 0 0) 0x7fa94c0584e0 con 0x7fa9580fff40 2024-05-17T21:52:27.473 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-05-17T21:52:27.473 INFO:teuthology.orchestra.run.smithi107.stdout: "mon": { 2024-05-17T21:52:27.473 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-952-g9903f776 (9903f7767386a9d9efadb7c1effdcd3064d71804) quincy (stable)": 2 2024-05-17T21:52:27.474 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-05-17T21:52:27.474 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr": { 2024-05-17T21:52:27.474 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-952-g9903f776 (9903f7767386a9d9efadb7c1effdcd3064d71804) quincy (stable)": 1, 2024-05-17T21:52:27.474 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev)": 1 2024-05-17T21:52:27.474 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-05-17T21:52:27.474 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": { 2024-05-17T21:52:27.474 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-952-g9903f776 (9903f7767386a9d9efadb7c1effdcd3064d71804) quincy (stable)": 6 2024-05-17T21:52:27.474 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-05-17T21:52:27.474 INFO:teuthology.orchestra.run.smithi107.stdout: "mds": { 2024-05-17T21:52:27.474 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-952-g9903f776 (9903f7767386a9d9efadb7c1effdcd3064d71804) quincy (stable)": 4 2024-05-17T21:52:27.474 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-05-17T21:52:27.474 INFO:teuthology.orchestra.run.smithi107.stdout: "overall": { 2024-05-17T21:52:27.474 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-952-g9903f776 (9903f7767386a9d9efadb7c1effdcd3064d71804) quincy (stable)": 13, 2024-05-17T21:52:27.475 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev)": 1 2024-05-17T21:52:27.475 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-05-17T21:52:27.475 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-05-17T21:52:27.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:27.483+0000 7fa95e040700 1 -- 172.21.15.107:0/2788907618 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7fa940070bb0 msgr2=0x7fa940073060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:52:27.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:27.483+0000 7fa95e040700 1 --2- 172.21.15.107:0/2788907618 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7fa940070bb0 0x7fa940073060 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fa94800b970 tx=0x7fa948009e30 comp rx=0 tx=0).stop 2024-05-17T21:52:27.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:27.483+0000 7fa95e040700 1 -- 172.21.15.107:0/2788907618 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9580fff40 msgr2=0x7fa95810df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:52:27.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:27.483+0000 7fa95e040700 1 --2- 172.21.15.107:0/2788907618 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9580fff40 0x7fa95810df70 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fa94c002a50 tx=0x7fa94c0046b0 comp rx=0 tx=0).stop 2024-05-17T21:52:27.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:27.483+0000 7fa95e040700 1 -- 172.21.15.107:0/2788907618 shutdown_connections 2024-05-17T21:52:27.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:27.483+0000 7fa95e040700 1 --2- 172.21.15.107:0/2788907618 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7fa940070bb0 0x7fa940073060 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:52:27.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:27.483+0000 7fa95e040700 1 --2- 172.21.15.107:0/2788907618 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa958066260 0x7fa95810da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:52:27.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:27.483+0000 7fa95e040700 1 --2- 172.21.15.107:0/2788907618 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9580fff40 0x7fa95810df70 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:52:27.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:27.483+0000 7fa95e040700 1 -- 172.21.15.107:0/2788907618 >> 172.21.15.107:0/2788907618 conn(0x7fa9580f7430 msgr2=0x7fa9580f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:52:27.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:27.483+0000 7fa95e040700 1 -- 172.21.15.107:0/2788907618 shutdown_connections 2024-05-17T21:52:27.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:27.483+0000 7fa95e040700 1 -- 172.21.15.107:0/2788907618 wait complete. 2024-05-17T21:52:27.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:27 smithi138 ceph-mon[35948]: from='client.14694 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:52:27.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:27 smithi138 ceph-mon[35948]: from='client.14698 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:52:27.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:27 smithi138 ceph-mon[35948]: from='mgr.14636 ' entity='mgr.smithi138.napnll' 2024-05-17T21:52:27.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:27 smithi138 ceph-mon[35948]: from='mgr.14636 ' entity='mgr.smithi138.napnll' 2024-05-17T21:52:27.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:27.594+0000 7f64ebd9c700 1 -- 172.21.15.107:0/953066227 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f64e40fbcb0 msgr2=0x7f64e40fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:52:27.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:27.594+0000 7f64ebd9c700 1 --2- 172.21.15.107:0/953066227 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f64e40fbcb0 0x7f64e40fc0c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f64e0004940 tx=0x7f64e0009e30 comp rx=0 tx=0).stop 2024-05-17T21:52:27.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:27.594+0000 7f64ebd9c700 1 -- 172.21.15.107:0/953066227 shutdown_connections 2024-05-17T21:52:27.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:27.594+0000 7f64ebd9c700 1 --2- 172.21.15.107:0/953066227 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f64e40fcf60 0x7f64e40fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:52:27.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:27.594+0000 7f64ebd9c700 1 --2- 172.21.15.107:0/953066227 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f64e40fbcb0 0x7f64e40fc0c0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:52:27.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:27.594+0000 7f64ebd9c700 1 -- 172.21.15.107:0/953066227 >> 172.21.15.107:0/953066227 conn(0x7f64e40f7430 msgr2=0x7f64e40f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:52:27.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:27.594+0000 7f64ebd9c700 1 -- 172.21.15.107:0/953066227 shutdown_connections 2024-05-17T21:52:27.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:27.594+0000 7f64ebd9c700 1 -- 172.21.15.107:0/953066227 wait complete. 2024-05-17T21:52:27.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:27.595+0000 7f64ebd9c700 1 Processor -- start 2024-05-17T21:52:27.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:27.595+0000 7f64ebd9c700 1 -- start start 2024-05-17T21:52:27.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:27.595+0000 7f64ebd9c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f64e40fbcb0 0x7f64e410c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:52:27.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:27.595+0000 7f64ebd9c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f64e40fcf60 0x7f64e410cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:52:27.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:27.595+0000 7f64ebd9c700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64e410d3c0 con 0x7f64e40fbcb0 2024-05-17T21:52:27.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:27.595+0000 7f64ebd9c700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64e410dc10 con 0x7f64e40fcf60 2024-05-17T21:52:27.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:27.595+0000 7f64e9b38700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f64e40fbcb0 0x7f64e410c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:52:27.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:27.595+0000 7f64e9b38700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f64e40fbcb0 0x7f64e410c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38982/0 (socket says 172.21.15.107:38982) 2024-05-17T21:52:27.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:27.595+0000 7f64e9b38700 1 -- 172.21.15.107:0/2178255023 learned_addr learned my addr 172.21.15.107:0/2178255023 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:52:27.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:27.595+0000 7f64e9337700 1 --2- 172.21.15.107:0/2178255023 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f64e40fcf60 0x7f64e410cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:52:27.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:27.595+0000 7f64e9b38700 1 -- 172.21.15.107:0/2178255023 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f64e40fcf60 msgr2=0x7f64e410cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:52:27.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:27.595+0000 7f64e9b38700 1 --2- 172.21.15.107:0/2178255023 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f64e40fcf60 0x7f64e410cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:52:27.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:27.595+0000 7f64e9b38700 1 -- 172.21.15.107:0/2178255023 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64e0005040 con 0x7f64e40fbcb0 2024-05-17T21:52:27.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:27.596+0000 7f64e9b38700 1 --2- 172.21.15.107:0/2178255023 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f64e40fbcb0 0x7f64e410c880 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f64e0013870 tx=0x7f64e0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:52:27.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:27.596+0000 7f64da7fc700 1 -- 172.21.15.107:0/2178255023 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f64e000e070 con 0x7f64e40fbcb0 2024-05-17T21:52:27.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:27.596+0000 7f64da7fc700 1 -- 172.21.15.107:0/2178255023 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f64e0013d00 con 0x7f64e40fbcb0 2024-05-17T21:52:27.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:27.596+0000 7f64ebd9c700 1 -- 172.21.15.107:0/2178255023 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64e410de90 con 0x7f64e40fbcb0 2024-05-17T21:52:27.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:27.596+0000 7f64da7fc700 1 -- 172.21.15.107:0/2178255023 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f64e0023650 con 0x7f64e40fbcb0 2024-05-17T21:52:27.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:27.596+0000 7f64ebd9c700 1 -- 172.21.15.107:0/2178255023 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64e410e310 con 0x7f64e40fbcb0 2024-05-17T21:52:27.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:27.597+0000 7f64ebd9c700 1 -- 172.21.15.107:0/2178255023 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64e405f5c0 con 0x7f64e40fbcb0 2024-05-17T21:52:27.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:27.598+0000 7f64da7fc700 1 -- 172.21.15.107:0/2178255023 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) v1 ==== 93316+0+0 (secure 0 0 0) 0x7f64e001aa30 con 0x7f64e40fbcb0 2024-05-17T21:52:27.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:27.598+0000 7f64da7fc700 1 --2- 172.21.15.107:0/2178255023 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7f64d0070bb0 0x7f64d0073060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:52:27.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:27.598+0000 7f64da7fc700 1 -- 172.21.15.107:0/2178255023 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f64e00924c0 con 0x7f64e40fbcb0 2024-05-17T21:52:27.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:27.599+0000 7f64e9337700 1 --2- 172.21.15.107:0/2178255023 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7f64d0070bb0 0x7f64d0073060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:52:27.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:27.599+0000 7f64e9337700 1 --2- 172.21.15.107:0/2178255023 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7f64d0070bb0 0x7f64d0073060 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f64d4009b80 tx=0x7f64d400b040 comp rx=0 tx=0).ready entity=mgr.14636 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:52:27.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:27.604+0000 7f64da7fc700 1 -- 172.21.15.107:0/2178255023 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+190121 (secure 0 0 0) 0x7f64e005aa10 con 0x7f64e40fbcb0 2024-05-17T21:52:27.682 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:27 smithi107 ceph-mon[28434]: from='client.14694 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:52:27.682 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:27 smithi107 ceph-mon[28434]: from='client.14698 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:52:27.682 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:27 smithi107 ceph-mon[28434]: from='mgr.14636 ' entity='mgr.smithi138.napnll' 2024-05-17T21:52:27.683 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:27 smithi107 ceph-mon[28434]: from='mgr.14636 ' entity='mgr.smithi138.napnll' 2024-05-17T21:52:27.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:27.891+0000 7f64ebd9c700 1 -- 172.21.15.107:0/2178255023 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f64e41047a0 con 0x7f64e40fbcb0 2024-05-17T21:52:27.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:27.892+0000 7f64da7fc700 1 -- 172.21.15.107:0/2178255023 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1840 (secure 0 0 0) 0x7f64e0004490 con 0x7f64e40fbcb0 2024-05-17T21:52:27.885 INFO:teuthology.orchestra.run.smithi107.stdout:e15 2024-05-17T21:52:27.885 INFO:teuthology.orchestra.run.smithi107.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-17T21:52:27.885 INFO:teuthology.orchestra.run.smithi107.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-17T21:52:27.885 INFO:teuthology.orchestra.run.smithi107.stdout:legacy client fscid: 1 2024-05-17T21:52:27.885 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:52:27.885 INFO:teuthology.orchestra.run.smithi107.stdout:Filesystem 'cephfs' (1) 2024-05-17T21:52:27.885 INFO:teuthology.orchestra.run.smithi107.stdout:fs_name cephfs 2024-05-17T21:52:27.885 INFO:teuthology.orchestra.run.smithi107.stdout:epoch 15 2024-05-17T21:52:27.885 INFO:teuthology.orchestra.run.smithi107.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-17T21:52:27.886 INFO:teuthology.orchestra.run.smithi107.stdout:created 2024-05-17T21:49:16.836333+0000 2024-05-17T21:52:27.886 INFO:teuthology.orchestra.run.smithi107.stdout:modified 2024-05-17T21:49:34.873628+0000 2024-05-17T21:52:27.886 INFO:teuthology.orchestra.run.smithi107.stdout:tableserver 0 2024-05-17T21:52:27.886 INFO:teuthology.orchestra.run.smithi107.stdout:root 0 2024-05-17T21:52:27.886 INFO:teuthology.orchestra.run.smithi107.stdout:session_timeout 60 2024-05-17T21:52:27.886 INFO:teuthology.orchestra.run.smithi107.stdout:session_autoclose 300 2024-05-17T21:52:27.886 INFO:teuthology.orchestra.run.smithi107.stdout:max_file_size 1099511627776 2024-05-17T21:52:27.886 INFO:teuthology.orchestra.run.smithi107.stdout:required_client_features {} 2024-05-17T21:52:27.886 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure 0 2024-05-17T21:52:27.886 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure_osd_epoch 46 2024-05-17T21:52:27.886 INFO:teuthology.orchestra.run.smithi107.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-17T21:52:27.887 INFO:teuthology.orchestra.run.smithi107.stdout:max_mds 1 2024-05-17T21:52:27.887 INFO:teuthology.orchestra.run.smithi107.stdout:in 0 2024-05-17T21:52:27.887 INFO:teuthology.orchestra.run.smithi107.stdout:up {0=24259} 2024-05-17T21:52:27.887 INFO:teuthology.orchestra.run.smithi107.stdout:failed 2024-05-17T21:52:27.887 INFO:teuthology.orchestra.run.smithi107.stdout:damaged 2024-05-17T21:52:27.887 INFO:teuthology.orchestra.run.smithi107.stdout:stopped 2024-05-17T21:52:27.887 INFO:teuthology.orchestra.run.smithi107.stdout:data_pools [3] 2024-05-17T21:52:27.887 INFO:teuthology.orchestra.run.smithi107.stdout:metadata_pool 2 2024-05-17T21:52:27.887 INFO:teuthology.orchestra.run.smithi107.stdout:inline_data enabled 2024-05-17T21:52:27.887 INFO:teuthology.orchestra.run.smithi107.stdout:balancer 2024-05-17T21:52:27.887 INFO:teuthology.orchestra.run.smithi107.stdout:standby_count_wanted 1 2024-05-17T21:52:27.887 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi138.gwbtyd{0:24259} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.138:6824/2779634919,v1:172.21.15.138:6825/2779634919] compat {c=[1],r=[1],i=[7ff]}] 2024-05-17T21:52:27.887 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.drqody{0:14486} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.107:6828/2275807100,v1:172.21.15.107:6829/2275807100] compat {c=[1],r=[1],i=[7ff]}] 2024-05-17T21:52:27.887 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:52:27.887 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:52:27.888 INFO:teuthology.orchestra.run.smithi107.stdout:Standby daemons: 2024-05-17T21:52:27.888 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:52:27.888 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.idmbgj{-1:14498} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.107:6826/1460186079,v1:172.21.15.107:6827/1460186079] compat {c=[1],r=[1],i=[7ff]}] 2024-05-17T21:52:27.888 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi138.zckrvs{-1:24267} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.138:6826/170628661,v1:172.21.15.138:6827/170628661] compat {c=[1],r=[1],i=[7ff]}] 2024-05-17T21:52:27.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:27.894+0000 7f64ebd9c700 1 -- 172.21.15.107:0/2178255023 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7f64d0070bb0 msgr2=0x7f64d0073060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:52:27.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:27.894+0000 7f64ebd9c700 1 --2- 172.21.15.107:0/2178255023 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7f64d0070bb0 0x7f64d0073060 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f64d4009b80 tx=0x7f64d400b040 comp rx=0 tx=0).stop 2024-05-17T21:52:27.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:27.894+0000 7f64ebd9c700 1 -- 172.21.15.107:0/2178255023 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f64e40fbcb0 msgr2=0x7f64e410c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:52:27.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:27.894+0000 7f64ebd9c700 1 --2- 172.21.15.107:0/2178255023 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f64e40fbcb0 0x7f64e410c880 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f64e0013870 tx=0x7f64e0009e30 comp rx=0 tx=0).stop 2024-05-17T21:52:27.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:27.895+0000 7f64ebd9c700 1 -- 172.21.15.107:0/2178255023 shutdown_connections 2024-05-17T21:52:27.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:27.895+0000 7f64ebd9c700 1 --2- 172.21.15.107:0/2178255023 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7f64d0070bb0 0x7f64d0073060 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:52:27.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:27.895+0000 7f64ebd9c700 1 --2- 172.21.15.107:0/2178255023 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f64e40fbcb0 0x7f64e410c880 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:52:27.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:27.895+0000 7f64ebd9c700 1 --2- 172.21.15.107:0/2178255023 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f64e40fcf60 0x7f64e410cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:52:27.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:27.895+0000 7f64ebd9c700 1 -- 172.21.15.107:0/2178255023 >> 172.21.15.107:0/2178255023 conn(0x7f64e40f7430 msgr2=0x7f64e40f8a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:52:27.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:27.895+0000 7f64ebd9c700 1 -- 172.21.15.107:0/2178255023 shutdown_connections 2024-05-17T21:52:27.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:27.895+0000 7f64ebd9c700 1 -- 172.21.15.107:0/2178255023 wait complete. 2024-05-17T21:52:27.890 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 15 2024-05-17T21:52:28.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:28.029+0000 7fb670169700 1 -- 172.21.15.107:0/1571029249 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb6680fcf60 msgr2=0x7fb6680fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:52:28.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:28.029+0000 7fb670169700 1 --2- 172.21.15.107:0/1571029249 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb6680fcf60 0x7fb6680fd3d0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb658004940 tx=0x7fb658009e30 comp rx=0 tx=0).stop 2024-05-17T21:52:28.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:28.029+0000 7fb670169700 1 -- 172.21.15.107:0/1571029249 shutdown_connections 2024-05-17T21:52:28.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:28.029+0000 7fb670169700 1 --2- 172.21.15.107:0/1571029249 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb6680fcf60 0x7fb6680fd3d0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:52:28.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:28.029+0000 7fb670169700 1 --2- 172.21.15.107:0/1571029249 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6680fbcb0 0x7fb6680fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:52:28.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:28.029+0000 7fb670169700 1 -- 172.21.15.107:0/1571029249 >> 172.21.15.107:0/1571029249 conn(0x7fb6680f7430 msgr2=0x7fb6680f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:52:28.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:28.029+0000 7fb670169700 1 -- 172.21.15.107:0/1571029249 shutdown_connections 2024-05-17T21:52:28.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:28.030+0000 7fb670169700 1 -- 172.21.15.107:0/1571029249 wait complete. 2024-05-17T21:52:28.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:28.030+0000 7fb670169700 1 Processor -- start 2024-05-17T21:52:28.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:28.030+0000 7fb670169700 1 -- start start 2024-05-17T21:52:28.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:28.031+0000 7fb670169700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb6680fbcb0 0x7fb66810d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:52:28.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:28.031+0000 7fb670169700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6680fcf60 0x7fb66810de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:52:28.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:28.031+0000 7fb670169700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb66810e490 con 0x7fb6680fbcb0 2024-05-17T21:52:28.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:28.031+0000 7fb670169700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb66810ece0 con 0x7fb6680fcf60 2024-05-17T21:52:28.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:28.031+0000 7fb66d704700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6680fcf60 0x7fb66810de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:52:28.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:28.031+0000 7fb66d704700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6680fcf60 0x7fb66810de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.107:60494/0 (socket says 172.21.15.107:60494) 2024-05-17T21:52:28.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:28.031+0000 7fb66d704700 1 -- 172.21.15.107:0/2866534518 learned_addr learned my addr 172.21.15.107:0/2866534518 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:52:28.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:28.031+0000 7fb66df05700 1 --2- 172.21.15.107:0/2866534518 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb6680fbcb0 0x7fb66810d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:52:28.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:28.032+0000 7fb66df05700 1 -- 172.21.15.107:0/2866534518 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6680fcf60 msgr2=0x7fb66810de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:52:28.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:28.032+0000 7fb66df05700 1 --2- 172.21.15.107:0/2866534518 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6680fcf60 0x7fb66810de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:52:28.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:28.032+0000 7fb66df05700 1 -- 172.21.15.107:0/2866534518 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb658005040 con 0x7fb6680fbcb0 2024-05-17T21:52:28.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:28.032+0000 7fb66d704700 1 --2- 172.21.15.107:0/2866534518 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6680fcf60 0x7fb66810de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T21:52:28.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:28.032+0000 7fb66df05700 1 --2- 172.21.15.107:0/2866534518 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb6680fbcb0 0x7fb66810d950 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb664009fd0 tx=0x7fb66400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:52:28.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:28.032+0000 7fb65e7fc700 1 -- 172.21.15.107:0/2866534518 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb664004e90 con 0x7fb6680fbcb0 2024-05-17T21:52:28.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:28.032+0000 7fb670169700 1 -- 172.21.15.107:0/2866534518 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb66810ef60 con 0x7fb6680fbcb0 2024-05-17T21:52:28.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:28.032+0000 7fb670169700 1 -- 172.21.15.107:0/2866534518 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb66810f4a0 con 0x7fb6680fbcb0 2024-05-17T21:52:28.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:28.032+0000 7fb65e7fc700 1 -- 172.21.15.107:0/2866534518 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb66400f040 con 0x7fb6680fbcb0 2024-05-17T21:52:28.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:28.032+0000 7fb65e7fc700 1 -- 172.21.15.107:0/2866534518 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb66400ca50 con 0x7fb6680fbcb0 2024-05-17T21:52:28.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:28.033+0000 7fb653fff700 1 -- 172.21.15.107:0/2866534518 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb66805f5c0 con 0x7fb6680fbcb0 2024-05-17T21:52:28.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:28.034+0000 7fb65e7fc700 1 -- 172.21.15.107:0/2866534518 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) v1 ==== 93316+0+0 (secure 0 0 0) 0x7fb664007cc0 con 0x7fb6680fbcb0 2024-05-17T21:52:28.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:28.035+0000 7fb65e7fc700 1 --2- 172.21.15.107:0/2866534518 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7fb654070bb0 0x7fb654073060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:52:28.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:28.035+0000 7fb65e7fc700 1 -- 172.21.15.107:0/2866534518 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7fb664090490 con 0x7fb6680fbcb0 2024-05-17T21:52:28.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:28.035+0000 7fb66d704700 1 --2- 172.21.15.107:0/2866534518 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7fb654070bb0 0x7fb654073060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:52:28.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:28.036+0000 7fb66d704700 1 --2- 172.21.15.107:0/2866534518 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7fb654070bb0 0x7fb654073060 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fb658013740 tx=0x7fb658009e30 comp rx=0 tx=0).ready entity=mgr.14636 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:52:28.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:28.042+0000 7fb65e7fc700 1 -- 172.21.15.107:0/2866534518 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+190121 (secure 0 0 0) 0x7fb6640516c0 con 0x7fb6680fbcb0 2024-05-17T21:52:28.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:28.327+0000 7fb653fff700 1 -- 172.21.15.107:0/2866534518 --> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb66804bed0 con 0x7fb654070bb0 2024-05-17T21:52:28.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:28.329+0000 7fb65e7fc700 1 -- 172.21.15.107:0/2866534518 <== mgr.14636 v2:172.21.15.138:6828/1369055740 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7fb66804bed0 con 0x7fb654070bb0 2024-05-17T21:52:28.322 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-05-17T21:52:28.323 INFO:teuthology.orchestra.run.smithi107.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b", 2024-05-17T21:52:28.323 INFO:teuthology.orchestra.run.smithi107.stdout: "in_progress": true, 2024-05-17T21:52:28.324 INFO:teuthology.orchestra.run.smithi107.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-17T21:52:28.325 INFO:teuthology.orchestra.run.smithi107.stdout: "services_complete": [], 2024-05-17T21:52:28.325 INFO:teuthology.orchestra.run.smithi107.stdout: "progress": "1/21 daemons upgraded", 2024-05-17T21:52:28.325 INFO:teuthology.orchestra.run.smithi107.stdout: "message": "", 2024-05-17T21:52:28.326 INFO:teuthology.orchestra.run.smithi107.stdout: "is_paused": false 2024-05-17T21:52:28.326 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-05-17T21:52:28.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:28.331+0000 7fb653fff700 1 -- 172.21.15.107:0/2866534518 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7fb654070bb0 msgr2=0x7fb654073060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:52:28.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:28.331+0000 7fb653fff700 1 --2- 172.21.15.107:0/2866534518 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7fb654070bb0 0x7fb654073060 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fb658013740 tx=0x7fb658009e30 comp rx=0 tx=0).stop 2024-05-17T21:52:28.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:28.331+0000 7fb653fff700 1 -- 172.21.15.107:0/2866534518 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb6680fbcb0 msgr2=0x7fb66810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:52:28.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:28.331+0000 7fb653fff700 1 --2- 172.21.15.107:0/2866534518 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb6680fbcb0 0x7fb66810d950 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb664009fd0 tx=0x7fb66400b040 comp rx=0 tx=0).stop 2024-05-17T21:52:28.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:28.331+0000 7fb653fff700 1 -- 172.21.15.107:0/2866534518 shutdown_connections 2024-05-17T21:52:28.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:28.331+0000 7fb653fff700 1 --2- 172.21.15.107:0/2866534518 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7fb654070bb0 0x7fb654073060 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:52:28.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:28.331+0000 7fb653fff700 1 --2- 172.21.15.107:0/2866534518 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb6680fbcb0 0x7fb66810d950 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:52:28.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:28.331+0000 7fb653fff700 1 --2- 172.21.15.107:0/2866534518 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6680fcf60 0x7fb66810de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:52:28.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:28.331+0000 7fb653fff700 1 -- 172.21.15.107:0/2866534518 >> 172.21.15.107:0/2866534518 conn(0x7fb6680f7430 msgr2=0x7fb6680f8f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:52:28.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:28.331+0000 7fb653fff700 1 -- 172.21.15.107:0/2866534518 shutdown_connections 2024-05-17T21:52:28.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:28.331+0000 7fb653fff700 1 -- 172.21.15.107:0/2866534518 wait complete. 2024-05-17T21:52:28.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:28.466+0000 7f4956702700 1 -- 172.21.15.107:0/3025583591 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49500fd2a0 msgr2=0x7f49500fd6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:52:28.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:28.466+0000 7f4956702700 1 --2- 172.21.15.107:0/3025583591 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49500fd2a0 0x7f49500fd6b0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f4940004940 tx=0x7f4940009e30 comp rx=0 tx=0).stop 2024-05-17T21:52:28.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:28.467+0000 7f4956702700 1 -- 172.21.15.107:0/3025583591 shutdown_connections 2024-05-17T21:52:28.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:28.467+0000 7f4956702700 1 --2- 172.21.15.107:0/3025583591 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f49500fdbf0 0x7f49500fc000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:52:28.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:28.467+0000 7f4956702700 1 --2- 172.21.15.107:0/3025583591 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49500fd2a0 0x7f49500fd6b0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:52:28.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:28.467+0000 7f4956702700 1 -- 172.21.15.107:0/3025583591 >> 172.21.15.107:0/3025583591 conn(0x7f49500f6b80 msgr2=0x7f49500f8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:52:28.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:28.467+0000 7f4956702700 1 -- 172.21.15.107:0/3025583591 shutdown_connections 2024-05-17T21:52:28.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:28.467+0000 7f4956702700 1 -- 172.21.15.107:0/3025583591 wait complete. 2024-05-17T21:52:28.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:28.467+0000 7f4956702700 1 Processor -- start 2024-05-17T21:52:28.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:28.468+0000 7f4956702700 1 -- start start 2024-05-17T21:52:28.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:28.468+0000 7f4956702700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f49500fd2a0 0x7f495010c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:52:28.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:28.468+0000 7f4956702700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49500fdbf0 0x7f495010ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:52:28.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:28.468+0000 7f4956702700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f495010d430 con 0x7f49500fdbf0 2024-05-17T21:52:28.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:28.468+0000 7f4956702700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f495010dc80 con 0x7f49500fd2a0 2024-05-17T21:52:28.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:28.468+0000 7f494f7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49500fdbf0 0x7f495010ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:52:28.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:28.468+0000 7f494f7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49500fdbf0 0x7f495010ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39026/0 (socket says 172.21.15.107:39026) 2024-05-17T21:52:28.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:28.468+0000 7f494f7fe700 1 -- 172.21.15.107:0/2181575980 learned_addr learned my addr 172.21.15.107:0/2181575980 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:52:28.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:28.468+0000 7f494ffff700 1 --2- 172.21.15.107:0/2181575980 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f49500fd2a0 0x7f495010c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:52:28.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:28.468+0000 7f494f7fe700 1 -- 172.21.15.107:0/2181575980 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f49500fd2a0 msgr2=0x7f495010c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:52:28.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:28.468+0000 7f494f7fe700 1 --2- 172.21.15.107:0/2181575980 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f49500fd2a0 0x7f495010c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:52:28.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:28.468+0000 7f494f7fe700 1 -- 172.21.15.107:0/2181575980 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4940005040 con 0x7f49500fdbf0 2024-05-17T21:52:28.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:28.468+0000 7f494ffff700 1 --2- 172.21.15.107:0/2181575980 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f49500fd2a0 0x7f495010c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T21:52:28.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:28.469+0000 7f494f7fe700 1 --2- 172.21.15.107:0/2181575980 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49500fdbf0 0x7f495010ce30 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f494400eb40 tx=0x7f494400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:52:28.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:28.469+0000 7f494cff9700 1 -- 172.21.15.107:0/2181575980 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4944007cc0 con 0x7f49500fdbf0 2024-05-17T21:52:28.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:28.469+0000 7f4956702700 1 -- 172.21.15.107:0/2181575980 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f495010de80 con 0x7f49500fdbf0 2024-05-17T21:52:28.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:28.469+0000 7f4956702700 1 -- 172.21.15.107:0/2181575980 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f495010e3c0 con 0x7f49500fdbf0 2024-05-17T21:52:28.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:28.469+0000 7f494cff9700 1 -- 172.21.15.107:0/2181575980 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f494400f040 con 0x7f49500fdbf0 2024-05-17T21:52:28.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:28.469+0000 7f494cff9700 1 -- 172.21.15.107:0/2181575980 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f494400cad0 con 0x7f49500fdbf0 2024-05-17T21:52:28.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:28.470+0000 7f4956702700 1 -- 172.21.15.107:0/2181575980 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f493c0052e0 con 0x7f49500fdbf0 2024-05-17T21:52:28.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:28.471+0000 7f494cff9700 1 -- 172.21.15.107:0/2181575980 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) v1 ==== 93316+0+0 (secure 0 0 0) 0x7f494400cc30 con 0x7f49500fdbf0 2024-05-17T21:52:28.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:28.472+0000 7f494cff9700 1 --2- 172.21.15.107:0/2181575980 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7f4938070bb0 0x7f4938073060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:52:28.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:28.472+0000 7f494ffff700 1 --2- 172.21.15.107:0/2181575980 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7f4938070bb0 0x7f4938073060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:52:28.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:28.472+0000 7f494cff9700 1 -- 172.21.15.107:0/2181575980 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f4944090430 con 0x7f49500fdbf0 2024-05-17T21:52:28.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:28.473+0000 7f494ffff700 1 --2- 172.21.15.107:0/2181575980 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7f4938070bb0 0x7f4938073060 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f4940009b30 tx=0x7f4940009e30 comp rx=0 tx=0).ready entity=mgr.14636 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:52:28.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:28.481+0000 7f494cff9700 1 -- 172.21.15.107:0/2181575980 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+190121 (secure 0 0 0) 0x7f4944058860 con 0x7f49500fdbf0 2024-05-17T21:52:28.597 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:28 smithi107 ceph-mon[28434]: from='client.14702 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:52:28.597 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:28 smithi107 ceph-mon[28434]: Reconfiguring grafana.smithi107 (dependencies changed)... 2024-05-17T21:52:28.597 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:28 smithi107 ceph-mon[28434]: Reconfiguring daemon grafana.smithi107 on smithi107 2024-05-17T21:52:28.597 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:28 smithi107 ceph-mon[28434]: pgmap v22: 65 pgs: 65 active+clean; 152 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.3 MiB/s rd, 3.9 MiB/s wr, 240 op/s 2024-05-17T21:52:28.597 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:28 smithi107 ceph-mon[28434]: from='client.? 172.21.15.107:0/2788907618' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:52:28.597 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:28 smithi107 ceph-mon[28434]: from='client.? 172.21.15.107:0/2178255023' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-17T21:52:28.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:28.816+0000 7f4956702700 1 -- 172.21.15.107:0/2181575980 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f493c005860 con 0x7f49500fdbf0 2024-05-17T21:52:28.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:28.816+0000 7f494cff9700 1 -- 172.21.15.107:0/2181575980 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f4944058200 con 0x7f49500fdbf0 2024-05-17T21:52:28.809 INFO:teuthology.orchestra.run.smithi107.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-17T21:52:28.809 INFO:teuthology.orchestra.run.smithi107.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-17T21:52:28.809 INFO:teuthology.orchestra.run.smithi107.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-17T21:52:28.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:28.819+0000 7f4956702700 1 -- 172.21.15.107:0/2181575980 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7f4938070bb0 msgr2=0x7f4938073060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:52:28.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:28.819+0000 7f4956702700 1 --2- 172.21.15.107:0/2181575980 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7f4938070bb0 0x7f4938073060 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f4940009b30 tx=0x7f4940009e30 comp rx=0 tx=0).stop 2024-05-17T21:52:28.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:28.819+0000 7f4956702700 1 -- 172.21.15.107:0/2181575980 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49500fdbf0 msgr2=0x7f495010ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:52:28.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:28.819+0000 7f4956702700 1 --2- 172.21.15.107:0/2181575980 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49500fdbf0 0x7f495010ce30 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f494400eb40 tx=0x7f494400b040 comp rx=0 tx=0).stop 2024-05-17T21:52:28.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:28.819+0000 7f4956702700 1 -- 172.21.15.107:0/2181575980 shutdown_connections 2024-05-17T21:52:28.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:28.819+0000 7f4956702700 1 --2- 172.21.15.107:0/2181575980 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7f4938070bb0 0x7f4938073060 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:52:28.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:28.819+0000 7f4956702700 1 --2- 172.21.15.107:0/2181575980 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f49500fd2a0 0x7f495010c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:52:28.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:28.819+0000 7f4956702700 1 --2- 172.21.15.107:0/2181575980 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49500fdbf0 0x7f495010ce30 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:52:28.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:28.819+0000 7f4956702700 1 -- 172.21.15.107:0/2181575980 >> 172.21.15.107:0/2181575980 conn(0x7f49500f6b80 msgr2=0x7f49500f8260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:52:28.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:28.819+0000 7f4956702700 1 -- 172.21.15.107:0/2181575980 shutdown_connections 2024-05-17T21:52:28.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:28.819+0000 7f4956702700 1 -- 172.21.15.107:0/2181575980 wait complete. 2024-05-17T21:52:28.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:28 smithi138 ceph-mon[35948]: from='client.14702 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:52:28.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:28 smithi138 ceph-mon[35948]: Reconfiguring grafana.smithi107 (dependencies changed)... 2024-05-17T21:52:28.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:28 smithi138 ceph-mon[35948]: Reconfiguring daemon grafana.smithi107 on smithi107 2024-05-17T21:52:28.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:28 smithi138 ceph-mon[35948]: pgmap v22: 65 pgs: 65 active+clean; 152 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.3 MiB/s rd, 3.9 MiB/s wr, 240 op/s 2024-05-17T21:52:28.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:28 smithi138 ceph-mon[35948]: from='client.? 172.21.15.107:0/2788907618' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:52:28.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:28 smithi138 ceph-mon[35948]: from='client.? 172.21.15.107:0/2178255023' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-17T21:52:29.637 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:29 smithi107 ceph-mon[28434]: from='client.14714 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:52:29.637 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:29 smithi107 ceph-mon[28434]: from='client.? 172.21.15.107:0/2181575980' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-17T21:52:29.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:29 smithi138 ceph-mon[35948]: from='client.14714 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:52:29.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:29 smithi138 ceph-mon[35948]: from='client.? 172.21.15.107:0/2181575980' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-17T21:52:30.526 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:30 smithi107 ceph-mon[28434]: pgmap v23: 65 pgs: 65 active+clean; 152 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.0 MiB/s rd, 3.5 MiB/s wr, 215 op/s 2024-05-17T21:52:30.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:30 smithi138 ceph-mon[35948]: pgmap v23: 65 pgs: 65 active+clean; 152 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.0 MiB/s rd, 3.5 MiB/s wr, 215 op/s 2024-05-17T21:52:32.628 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:32 smithi138 ceph-mon[35948]: pgmap v24: 65 pgs: 65 active+clean; 170 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 2.8 MiB/s rd, 5.0 MiB/s wr, 374 op/s 2024-05-17T21:52:32.687 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:32 smithi107 ceph-mon[28434]: pgmap v24: 65 pgs: 65 active+clean; 170 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 2.8 MiB/s rd, 5.0 MiB/s wr, 374 op/s 2024-05-17T21:52:33.914 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:33 smithi107 ceph-mon[28434]: from='mgr.14636 ' entity='mgr.smithi138.napnll' 2024-05-17T21:52:33.914 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:33 smithi107 ceph-mon[28434]: from='mgr.14636 ' entity='mgr.smithi138.napnll' 2024-05-17T21:52:33.914 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:33 smithi107 ceph-mon[28434]: Reconfiguring prometheus.smithi107 (dependencies changed)... 2024-05-17T21:52:33.914 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:33 smithi107 ceph-mon[28434]: Reconfiguring daemon prometheus.smithi107 on smithi107 2024-05-17T21:52:33.914 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:33 smithi107 ceph-mon[28434]: pgmap v25: 65 pgs: 65 active+clean; 170 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.9 MiB/s rd, 3.7 MiB/s wr, 279 op/s 2024-05-17T21:52:34.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:33 smithi138 ceph-mon[35948]: from='mgr.14636 ' entity='mgr.smithi138.napnll' 2024-05-17T21:52:34.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:33 smithi138 ceph-mon[35948]: from='mgr.14636 ' entity='mgr.smithi138.napnll' 2024-05-17T21:52:34.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:33 smithi138 ceph-mon[35948]: Reconfiguring prometheus.smithi107 (dependencies changed)... 2024-05-17T21:52:34.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:33 smithi138 ceph-mon[35948]: Reconfiguring daemon prometheus.smithi107 on smithi107 2024-05-17T21:52:34.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:33 smithi138 ceph-mon[35948]: pgmap v25: 65 pgs: 65 active+clean; 170 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.9 MiB/s rd, 3.7 MiB/s wr, 279 op/s 2024-05-17T21:52:35.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:34 smithi138 ceph-mon[35948]: from='mgr.14636 172.21.15.138:0/3120106723' entity='mgr.smithi138.napnll' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:52:35.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:34 smithi107 ceph-mon[28434]: from='mgr.14636 172.21.15.138:0/3120106723' entity='mgr.smithi138.napnll' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:52:36.055 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:35 smithi107 ceph-mon[28434]: pgmap v26: 65 pgs: 65 active+clean; 186 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 2.7 MiB/s rd, 5.0 MiB/s wr, 361 op/s 2024-05-17T21:52:36.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:35 smithi138 ceph-mon[35948]: pgmap v26: 65 pgs: 65 active+clean; 186 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 2.7 MiB/s rd, 5.0 MiB/s wr, 361 op/s 2024-05-17T21:52:37.229 INFO:tasks.workunit.client.0.smithi107.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-05-17T21:52:37.234 INFO:tasks.workunit.client.0.smithi107.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-05-17T21:52:37.234 INFO:tasks.workunit.client.0.smithi107.stderr:+ make 2024-05-17T21:52:37.986 INFO:tasks.workunit.client.0.smithi107.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-05-17T21:52:38.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:38 smithi107 ceph-mon[28434]: pgmap v27: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 2.9 MiB/s rd, 5.1 MiB/s wr, 436 op/s 2024-05-17T21:52:38.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:38 smithi138 ceph-mon[35948]: pgmap v27: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 2.9 MiB/s rd, 5.1 MiB/s wr, 436 op/s 2024-05-17T21:52:38.826 INFO:tasks.workunit.client.0.smithi107.stderr:++ readlink -f fsstress 2024-05-17T21:52:38.830 INFO:tasks.workunit.client.0.smithi107.stderr:+ BIN=/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-05-17T21:52:38.830 INFO:tasks.workunit.client.0.smithi107.stderr:+ popd 2024-05-17T21:52:38.832 INFO:tasks.workunit.client.0.smithi107.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-05-17T21:52:38.832 INFO:tasks.workunit.client.0.smithi107.stderr:+ popd 2024-05-17T21:52:38.832 INFO:tasks.workunit.client.0.smithi107.stdout:~/cephtest/mnt.0/client.0/tmp 2024-05-17T21:52:38.832 INFO:tasks.workunit.client.0.smithi107.stderr:++ mktemp -d -p . 2024-05-17T21:52:38.833 INFO:tasks.workunit.client.0.smithi107.stderr:+ T=./tmp.8ZNzSwV3Np 2024-05-17T21:52:38.833 INFO:tasks.workunit.client.0.smithi107.stderr:+ /home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.8ZNzSwV3Np -l 1 -n 1000 -p 10 -v 2024-05-17T21:52:38.837 INFO:tasks.workunit.client.0.smithi107.stdout:seed = 1716206391 2024-05-17T21:52:38.843 INFO:tasks.workunit.client.0.smithi107.stdout:0/0: write - no filename 2024-05-17T21:52:38.845 INFO:tasks.workunit.client.0.smithi107.stdout:1/0: stat - no entries 2024-05-17T21:52:38.847 INFO:tasks.workunit.client.0.smithi107.stdout:0/1: symlink l0 0 2024-05-17T21:52:38.851 INFO:tasks.workunit.client.0.smithi107.stdout:0/2: creat f1 x:0 0 0 2024-05-17T21:52:38.852 INFO:tasks.workunit.client.0.smithi107.stdout:1/1: creat f0 x:0 0 0 2024-05-17T21:52:38.853 INFO:tasks.workunit.client.0.smithi107.stdout:1/2: readlink - no filename 2024-05-17T21:52:38.854 INFO:tasks.workunit.client.0.smithi107.stdout:4/0: chown . 99 1 2024-05-17T21:52:38.854 INFO:tasks.workunit.client.0.smithi107.stdout:4/1: rename - no filename 2024-05-17T21:52:38.855 INFO:tasks.workunit.client.0.smithi107.stdout:4/2: dwrite - no filename 2024-05-17T21:52:38.855 INFO:tasks.workunit.client.0.smithi107.stdout:4/3: dread - no filename 2024-05-17T21:52:38.855 INFO:tasks.workunit.client.0.smithi107.stdout:4/4: rename - no filename 2024-05-17T21:52:38.855 INFO:tasks.workunit.client.0.smithi107.stdout:4/5: rmdir - no directory 2024-05-17T21:52:38.855 INFO:tasks.workunit.client.0.smithi107.stdout:4/6: dwrite - no filename 2024-05-17T21:52:38.855 INFO:tasks.workunit.client.0.smithi107.stdout:4/7: dwrite - no filename 2024-05-17T21:52:38.855 INFO:tasks.workunit.client.0.smithi107.stdout:4/8: dread - no filename 2024-05-17T21:52:38.856 INFO:tasks.workunit.client.0.smithi107.stdout:2/0: getdents . 0 2024-05-17T21:52:38.857 INFO:tasks.workunit.client.0.smithi107.stdout:0/3: rename l0 to l2 0 2024-05-17T21:52:38.858 INFO:tasks.workunit.client.0.smithi107.stdout:1/3: mknod c1 0 2024-05-17T21:52:38.858 INFO:tasks.workunit.client.0.smithi107.stdout:1/4: dread - f0 zero size 2024-05-17T21:52:38.861 INFO:tasks.workunit.client.0.smithi107.stdout:5/0: readlink - no filename 2024-05-17T21:52:38.863 INFO:tasks.workunit.client.0.smithi107.stdout:3/0: symlink l0 0 2024-05-17T21:52:38.863 INFO:tasks.workunit.client.0.smithi107.stdout:3/1: fdatasync - no filename 2024-05-17T21:52:38.866 INFO:tasks.workunit.client.0.smithi107.stdout:0/4: mkdir d3 0 2024-05-17T21:52:38.868 INFO:tasks.workunit.client.0.smithi107.stdout:1/5: unlink f0 0 2024-05-17T21:52:38.868 INFO:tasks.workunit.client.0.smithi107.stdout:1/6: write - no filename 2024-05-17T21:52:38.870 INFO:tasks.workunit.client.0.smithi107.stdout:4/9: symlink l0 0 2024-05-17T21:52:38.870 INFO:tasks.workunit.client.0.smithi107.stdout:4/10: dread - no filename 2024-05-17T21:52:38.870 INFO:tasks.workunit.client.0.smithi107.stdout:4/11: fsync - no filename 2024-05-17T21:52:38.870 INFO:tasks.workunit.client.0.smithi107.stdout:4/12: fsync - no filename 2024-05-17T21:52:38.870 INFO:tasks.workunit.client.0.smithi107.stdout:4/13: chown l0 695 1 2024-05-17T21:52:38.873 INFO:tasks.workunit.client.0.smithi107.stdout:6/0: dread - no filename 2024-05-17T21:52:38.873 INFO:tasks.workunit.client.0.smithi107.stdout:6/1: write - no filename 2024-05-17T21:52:38.873 INFO:tasks.workunit.client.0.smithi107.stdout:6/2: truncate - no filename 2024-05-17T21:52:38.873 INFO:tasks.workunit.client.0.smithi107.stdout:6/3: dwrite - no filename 2024-05-17T21:52:38.875 INFO:tasks.workunit.client.0.smithi107.stdout:3/2: creat f1 x:0 0 0 2024-05-17T21:52:38.876 INFO:tasks.workunit.client.0.smithi107.stdout:3/3: read - f1 zero size 2024-05-17T21:52:38.876 INFO:tasks.workunit.client.0.smithi107.stdout:2/1: mkdir d0 0 2024-05-17T21:52:38.876 INFO:tasks.workunit.client.0.smithi107.stdout:2/2: write - no filename 2024-05-17T21:52:38.880 INFO:tasks.workunit.client.0.smithi107.stdout:1/7: symlink l2 0 2024-05-17T21:52:38.881 INFO:tasks.workunit.client.0.smithi107.stdout:4/14: rename l0 to l1 0 2024-05-17T21:52:38.883 INFO:tasks.workunit.client.0.smithi107.stdout:5/1: symlink l0 0 2024-05-17T21:52:38.888 INFO:tasks.workunit.client.0.smithi107.stdout:3/4: rename f1 to f2 0 2024-05-17T21:52:38.889 INFO:tasks.workunit.client.0.smithi107.stdout:0/5: symlink d3/l4 0 2024-05-17T21:52:38.890 INFO:tasks.workunit.client.0.smithi107.stdout:1/8: creat f3 x:0 0 0 2024-05-17T21:52:38.891 INFO:tasks.workunit.client.0.smithi107.stdout:1/9: dread - f3 zero size 2024-05-17T21:52:38.893 INFO:tasks.workunit.client.0.smithi107.stdout:5/2: rename l0 to l1 0 2024-05-17T21:52:38.893 INFO:tasks.workunit.client.0.smithi107.stdout:5/3: rmdir - no directory 2024-05-17T21:52:38.894 INFO:tasks.workunit.client.0.smithi107.stdout:8/0: dwrite - no filename 2024-05-17T21:52:38.896 INFO:tasks.workunit.client.0.smithi107.stdout:6/4: creat f0 x:0 0 0 2024-05-17T21:52:38.902 INFO:tasks.workunit.client.0.smithi107.stdout:5/4: creat f2 x:0 0 0 2024-05-17T21:52:38.904 INFO:tasks.workunit.client.0.smithi107.stdout:5/5: chown f2 99 1 2024-05-17T21:52:38.905 INFO:tasks.workunit.client.0.smithi107.stdout:7/0: mknod c0 0 2024-05-17T21:52:38.905 INFO:tasks.workunit.client.0.smithi107.stdout:7/1: write - no filename 2024-05-17T21:52:38.905 INFO:tasks.workunit.client.0.smithi107.stdout:7/2: truncate - no filename 2024-05-17T21:52:38.905 INFO:tasks.workunit.client.0.smithi107.stdout:7/3: readlink - no filename 2024-05-17T21:52:38.905 INFO:tasks.workunit.client.0.smithi107.stdout:7/4: write - no filename 2024-05-17T21:52:38.906 INFO:tasks.workunit.client.0.smithi107.stdout:3/5: dwrite f2 [0,4194304] 0 2024-05-17T21:52:38.906 INFO:tasks.workunit.client.0.smithi107.stdout:6/5: creat f1 x:0 0 0 2024-05-17T21:52:38.906 INFO:tasks.workunit.client.0.smithi107.stdout:6/6: dread - f0 zero size 2024-05-17T21:52:38.909 INFO:tasks.workunit.client.0.smithi107.stdout:8/1: symlink l0 0 2024-05-17T21:52:38.910 INFO:tasks.workunit.client.0.smithi107.stdout:5/6: creat f3 x:0 0 0 2024-05-17T21:52:38.911 INFO:tasks.workunit.client.0.smithi107.stdout:5/7: read - f2 zero size 2024-05-17T21:52:38.912 INFO:tasks.workunit.client.0.smithi107.stdout:7/5: symlink l1 0 2024-05-17T21:52:38.912 INFO:tasks.workunit.client.0.smithi107.stdout:5/8: chown f2 505 1 2024-05-17T21:52:38.913 INFO:tasks.workunit.client.0.smithi107.stdout:5/9: dread - f2 zero size 2024-05-17T21:52:38.915 INFO:tasks.workunit.client.0.smithi107.stdout:5/10: truncate f3 1022217 0 2024-05-17T21:52:38.916 INFO:tasks.workunit.client.0.smithi107.stdout:3/6: dwrite f2 [0,4194304] 0 2024-05-17T21:52:38.916 INFO:tasks.workunit.client.0.smithi107.stdout:6/7: mknod c2 0 2024-05-17T21:52:38.917 INFO:tasks.workunit.client.0.smithi107.stdout:3/7: stat l0 0 2024-05-17T21:52:38.917 INFO:tasks.workunit.client.0.smithi107.stdout:8/2: rename l0 to l1 0 2024-05-17T21:52:38.917 INFO:tasks.workunit.client.0.smithi107.stdout:6/8: truncate f1 760561 0 2024-05-17T21:52:38.919 INFO:tasks.workunit.client.0.smithi107.stdout:9/0: mkdir d0 0 2024-05-17T21:52:38.919 INFO:tasks.workunit.client.0.smithi107.stdout:9/1: dread - no filename 2024-05-17T21:52:38.919 INFO:tasks.workunit.client.0.smithi107.stdout:9/2: unlink - no file 2024-05-17T21:52:38.919 INFO:tasks.workunit.client.0.smithi107.stdout:9/3: fsync - no filename 2024-05-17T21:52:38.919 INFO:tasks.workunit.client.0.smithi107.stdout:9/4: write - no filename 2024-05-17T21:52:38.920 INFO:tasks.workunit.client.0.smithi107.stdout:7/6: symlink l2 0 2024-05-17T21:52:38.921 INFO:tasks.workunit.client.0.smithi107.stdout:7/7: chown c0 0 1 2024-05-17T21:52:38.921 INFO:tasks.workunit.client.0.smithi107.stdout:7/8: fdatasync - no filename 2024-05-17T21:52:38.922 INFO:tasks.workunit.client.0.smithi107.stdout:5/11: symlink l4 0 2024-05-17T21:52:38.926 INFO:tasks.workunit.client.0.smithi107.stdout:6/9: symlink l3 0 2024-05-17T21:52:38.929 INFO:tasks.workunit.client.0.smithi107.stdout:7/9: mknod c3 0 2024-05-17T21:52:38.929 INFO:tasks.workunit.client.0.smithi107.stdout:7/10: dwrite - no filename 2024-05-17T21:52:38.930 INFO:tasks.workunit.client.0.smithi107.stdout:5/12: mkdir d5 0 2024-05-17T21:52:38.942 INFO:tasks.workunit.client.0.smithi107.stdout:3/8: link f2 f3 0 2024-05-17T21:52:38.951 INFO:tasks.workunit.client.0.smithi107.stdout:9/5: symlink d0/l1 0 2024-05-17T21:52:38.951 INFO:tasks.workunit.client.0.smithi107.stdout:9/6: write - no filename 2024-05-17T21:52:38.952 INFO:tasks.workunit.client.0.smithi107.stdout:7/11: creat f4 x:0 0 0 2024-05-17T21:52:38.955 INFO:tasks.workunit.client.0.smithi107.stdout:3/9: mknod c4 0 2024-05-17T21:52:38.955 INFO:tasks.workunit.client.0.smithi107.stdout:3/10: stat l0 0 2024-05-17T21:52:38.956 INFO:tasks.workunit.client.0.smithi107.stdout:5/13: symlink d5/l6 0 2024-05-17T21:52:38.957 INFO:tasks.workunit.client.0.smithi107.stdout:5/14: stat d5/l6 0 2024-05-17T21:52:38.959 INFO:tasks.workunit.client.0.smithi107.stdout:3/11: write f3 [2285546,55287] 0 2024-05-17T21:52:38.960 INFO:tasks.workunit.client.0.smithi107.stdout:7/12: dwrite f4 [0,4194304] 0 2024-05-17T21:52:38.960 INFO:tasks.workunit.client.0.smithi107.stdout:3/12: chown l0 31514988 1 2024-05-17T21:52:38.961 INFO:tasks.workunit.client.0.smithi107.stdout:5/15: unlink d5/l6 0 2024-05-17T21:52:38.961 INFO:tasks.workunit.client.0.smithi107.stdout:3/13: stat c4 0 2024-05-17T21:52:38.962 INFO:tasks.workunit.client.0.smithi107.stdout:5/16: write f3 [1912460,1094] 0 2024-05-17T21:52:38.962 INFO:tasks.workunit.client.0.smithi107.stdout:7/13: mkdir d5 0 2024-05-17T21:52:38.962 INFO:tasks.workunit.client.0.smithi107.stdout:7/14: chown c0 1068 1 2024-05-17T21:52:38.966 INFO:tasks.workunit.client.0.smithi107.stdout:5/17: creat d5/f7 x:0 0 0 2024-05-17T21:52:39.232 INFO:tasks.workunit.client.0.smithi107.stdout:0/6: rename l2 to d3/l5 0 2024-05-17T21:52:39.233 INFO:tasks.workunit.client.0.smithi107.stdout:0/7: read - f1 zero size 2024-05-17T21:52:39.233 INFO:tasks.workunit.client.0.smithi107.stdout:0/8: dread - f1 zero size 2024-05-17T21:52:39.235 INFO:tasks.workunit.client.0.smithi107.stdout:0/9: mknod d3/c6 0 2024-05-17T21:52:39.235 INFO:tasks.workunit.client.0.smithi107.stdout:0/10: truncate f1 657745 0 2024-05-17T21:52:39.236 INFO:tasks.workunit.client.0.smithi107.stdout:0/11: read f1 [484282,198] 0 2024-05-17T21:52:39.236 INFO:tasks.workunit.client.0.smithi107.stdout:0/12: read f1 [402303,42140] 0 2024-05-17T21:52:39.238 INFO:tasks.workunit.client.0.smithi107.stdout:0/13: write f1 [128369,15920] 0 2024-05-17T21:52:39.240 INFO:tasks.workunit.client.0.smithi107.stdout:4/15: chown l1 26676 1 2024-05-17T21:52:39.240 INFO:tasks.workunit.client.0.smithi107.stdout:4/16: rmdir - no directory 2024-05-17T21:52:39.240 INFO:tasks.workunit.client.0.smithi107.stdout:4/17: stat l1 0 2024-05-17T21:52:39.240 INFO:tasks.workunit.client.0.smithi107.stdout:4/18: truncate - no filename 2024-05-17T21:52:39.241 INFO:tasks.workunit.client.0.smithi107.stdout:0/14: dread f1 [0,4194304] 0 2024-05-17T21:52:39.242 INFO:tasks.workunit.client.0.smithi107.stdout:4/19: creat f2 x:0 0 0 2024-05-17T21:52:39.243 INFO:tasks.workunit.client.0.smithi107.stdout:0/15: rmdir d3 39 2024-05-17T21:52:39.243 INFO:tasks.workunit.client.0.smithi107.stdout:1/10: fsync f3 0 2024-05-17T21:52:39.245 INFO:tasks.workunit.client.0.smithi107.stdout:0/16: dread f1 [0,4194304] 0 2024-05-17T21:52:39.245 INFO:tasks.workunit.client.0.smithi107.stdout:1/11: mkdir d4 0 2024-05-17T21:52:39.246 INFO:tasks.workunit.client.0.smithi107.stdout:0/17: fsync f1 0 2024-05-17T21:52:39.247 INFO:tasks.workunit.client.0.smithi107.stdout:1/12: write f3 [937508,3952] 0 2024-05-17T21:52:39.250 INFO:tasks.workunit.client.0.smithi107.stdout:4/20: dwrite f2 [0,4194304] 0 2024-05-17T21:52:39.251 INFO:tasks.workunit.client.0.smithi107.stdout:1/13: creat d4/f5 x:0 0 0 2024-05-17T21:52:39.251 INFO:tasks.workunit.client.0.smithi107.stdout:1/14: dread - d4/f5 zero size 2024-05-17T21:52:39.251 INFO:tasks.workunit.client.0.smithi107.stdout:1/15: readlink l2 0 2024-05-17T21:52:39.258 INFO:tasks.workunit.client.0.smithi107.stdout:8/3: rename l1 to l2 0 2024-05-17T21:52:39.259 INFO:tasks.workunit.client.0.smithi107.stdout:0/18: link d3/c6 d3/c7 0 2024-05-17T21:52:39.260 INFO:tasks.workunit.client.0.smithi107.stdout:1/16: dwrite f3 [0,4194304] 0 2024-05-17T21:52:39.260 INFO:tasks.workunit.client.0.smithi107.stdout:1/17: fdatasync d4/f5 0 2024-05-17T21:52:39.261 INFO:tasks.workunit.client.0.smithi107.stdout:8/4: creat f3 x:0 0 0 2024-05-17T21:52:39.263 INFO:tasks.workunit.client.0.smithi107.stdout:4/21: fdatasync f2 0 2024-05-17T21:52:39.267 INFO:tasks.workunit.client.0.smithi107.stdout:4/22: mkdir d3 0 2024-05-17T21:52:39.269 INFO:tasks.workunit.client.0.smithi107.stdout:0/19: dwrite f1 [0,4194304] 0 2024-05-17T21:52:39.269 INFO:tasks.workunit.client.0.smithi107.stdout:4/23: chown f2 1044851 1 2024-05-17T21:52:39.270 INFO:tasks.workunit.client.0.smithi107.stdout:4/24: chown f2 282081 1 2024-05-17T21:52:39.274 INFO:tasks.workunit.client.0.smithi107.stdout:3/14: getdents . 0 2024-05-17T21:52:39.275 INFO:tasks.workunit.client.0.smithi107.stdout:4/25: creat d3/f4 x:0 0 0 2024-05-17T21:52:39.276 INFO:tasks.workunit.client.0.smithi107.stdout:3/15: creat f5 x:0 0 0 2024-05-17T21:52:39.277 INFO:tasks.workunit.client.0.smithi107.stdout:3/16: dread - f5 zero size 2024-05-17T21:52:39.278 INFO:tasks.workunit.client.0.smithi107.stdout:3/17: write f5 [376182,79150] 0 2024-05-17T21:52:39.280 INFO:tasks.workunit.client.0.smithi107.stdout:4/26: link l1 d3/l5 0 2024-05-17T21:52:39.280 INFO:tasks.workunit.client.0.smithi107.stdout:5/18: truncate f3 1844787 0 2024-05-17T21:52:39.283 INFO:tasks.workunit.client.0.smithi107.stdout:5/19: truncate d5/f7 80030 0 2024-05-17T21:52:39.284 INFO:tasks.workunit.client.0.smithi107.stdout:5/20: creat d5/f8 x:0 0 0 2024-05-17T21:52:39.286 INFO:tasks.workunit.client.0.smithi107.stdout:3/18: dwrite f2 [0,4194304] 0 2024-05-17T21:52:39.294 INFO:tasks.workunit.client.0.smithi107.stdout:5/21: dwrite d5/f7 [0,4194304] 0 2024-05-17T21:52:39.295 INFO:tasks.workunit.client.0.smithi107.stdout:5/22: write d5/f8 [462148,127669] 0 2024-05-17T21:52:39.301 INFO:tasks.workunit.client.0.smithi107.stdout:5/23: rename f2 to d5/f9 0 2024-05-17T21:52:39.301 INFO:tasks.workunit.client.0.smithi107.stdout:5/24: dread - d5/f9 zero size 2024-05-17T21:52:39.302 INFO:tasks.workunit.client.0.smithi107.stdout:5/25: write d5/f8 [1636291,121433] 0 2024-05-17T21:52:39.303 INFO:tasks.workunit.client.0.smithi107.stdout:5/26: dread - d5/f9 zero size 2024-05-17T21:52:39.304 INFO:tasks.workunit.client.0.smithi107.stdout:4/27: fsync f2 0 2024-05-17T21:52:39.304 INFO:tasks.workunit.client.0.smithi107.stdout:5/27: truncate d5/f9 550280 0 2024-05-17T21:52:39.306 INFO:tasks.workunit.client.0.smithi107.stdout:4/28: mkdir d3/d6 0 2024-05-17T21:52:39.310 INFO:tasks.workunit.client.0.smithi107.stdout:8/5: fsync f3 0 2024-05-17T21:52:39.310 INFO:tasks.workunit.client.0.smithi107.stdout:1/18: truncate f3 3479566 0 2024-05-17T21:52:39.312 INFO:tasks.workunit.client.0.smithi107.stdout:8/6: write f3 [638846,83588] 0 2024-05-17T21:52:39.313 INFO:tasks.workunit.client.0.smithi107.stdout:0/20: truncate f1 1674585 0 2024-05-17T21:52:39.314 INFO:tasks.workunit.client.0.smithi107.stdout:4/29: dwrite f2 [0,4194304] 0 2024-05-17T21:52:39.316 INFO:tasks.workunit.client.0.smithi107.stdout:1/19: rename c1 to d4/c6 0 2024-05-17T21:52:39.316 INFO:tasks.workunit.client.0.smithi107.stdout:1/20: readlink l2 0 2024-05-17T21:52:39.317 INFO:tasks.workunit.client.0.smithi107.stdout:0/21: symlink d3/l8 0 2024-05-17T21:52:39.317 INFO:tasks.workunit.client.0.smithi107.stdout:3/19: fsync f2 0 2024-05-17T21:52:39.320 INFO:tasks.workunit.client.0.smithi107.stdout:8/7: link f3 f4 0 2024-05-17T21:52:39.323 INFO:tasks.workunit.client.0.smithi107.stdout:3/20: mkdir d6 0 2024-05-17T21:52:39.324 INFO:tasks.workunit.client.0.smithi107.stdout:4/30: dwrite d3/f4 [0,4194304] 0 2024-05-17T21:52:39.329 INFO:tasks.workunit.client.0.smithi107.stdout:3/21: mknod d6/c7 0 2024-05-17T21:52:39.332 INFO:tasks.workunit.client.0.smithi107.stdout:4/31: rename d3/l5 to d3/l7 0 2024-05-17T21:52:39.335 INFO:tasks.workunit.client.0.smithi107.stdout:3/22: mkdir d6/d8 0 2024-05-17T21:52:39.335 INFO:tasks.workunit.client.0.smithi107.stdout:3/23: chown d6/c7 1 1 2024-05-17T21:52:39.336 INFO:tasks.workunit.client.0.smithi107.stdout:3/24: write f2 [921485,55673] 0 2024-05-17T21:52:39.336 INFO:tasks.workunit.client.0.smithi107.stdout:5/28: truncate d5/f8 42358 0 2024-05-17T21:52:39.337 INFO:tasks.workunit.client.0.smithi107.stdout:4/32: dread d3/f4 [0,4194304] 0 2024-05-17T21:52:39.338 INFO:tasks.workunit.client.0.smithi107.stdout:4/33: write f2 [3349647,62037] 0 2024-05-17T21:52:39.341 INFO:tasks.workunit.client.0.smithi107.stdout:0/22: write f1 [2525566,34793] 0 2024-05-17T21:52:39.342 INFO:tasks.workunit.client.0.smithi107.stdout:1/21: getdents d4 0 2024-05-17T21:52:39.346 INFO:tasks.workunit.client.0.smithi107.stdout:8/8: fsync f3 0 2024-05-17T21:52:39.349 INFO:tasks.workunit.client.0.smithi107.stdout:0/23: chown d3/l5 1932051409 1 2024-05-17T21:52:39.349 INFO:tasks.workunit.client.0.smithi107.stdout:4/34: write d3/f4 [5093554,113156] 0 2024-05-17T21:52:39.351 INFO:tasks.workunit.client.0.smithi107.stdout:8/9: dread f3 [0,4194304] 0 2024-05-17T21:52:39.352 INFO:tasks.workunit.client.0.smithi107.stdout:3/25: creat d6/d8/f9 x:0 0 0 2024-05-17T21:52:39.355 INFO:tasks.workunit.client.0.smithi107.stdout:8/10: dread f3 [0,4194304] 0 2024-05-17T21:52:39.355 INFO:tasks.workunit.client.0.smithi107.stdout:0/24: dread f1 [0,4194304] 0 2024-05-17T21:52:39.356 INFO:tasks.workunit.client.0.smithi107.stdout:5/29: rmdir d5 39 2024-05-17T21:52:39.358 INFO:tasks.workunit.client.0.smithi107.stdout:0/25: chown d3/l4 190756 1 2024-05-17T21:52:39.360 INFO:tasks.workunit.client.0.smithi107.stdout:4/35: dread d3/f4 [0,4194304] 0 2024-05-17T21:52:39.360 INFO:tasks.workunit.client.0.smithi107.stdout:1/22: dwrite f3 [4194304,4194304] 0 2024-05-17T21:52:39.362 INFO:tasks.workunit.client.0.smithi107.stdout:8/11: write f4 [1168199,48939] 0 2024-05-17T21:52:39.369 INFO:tasks.workunit.client.0.smithi107.stdout:0/26: creat d3/f9 x:0 0 0 2024-05-17T21:52:39.371 INFO:tasks.workunit.client.0.smithi107.stdout:1/23: rmdir d4 39 2024-05-17T21:52:39.373 INFO:tasks.workunit.client.0.smithi107.stdout:5/30: creat d5/fa x:0 0 0 2024-05-17T21:52:39.374 INFO:tasks.workunit.client.0.smithi107.stdout:8/12: rename f3 to f5 0 2024-05-17T21:52:39.376 INFO:tasks.workunit.client.0.smithi107.stdout:0/27: symlink d3/la 0 2024-05-17T21:52:39.383 INFO:tasks.workunit.client.0.smithi107.stdout:0/28: creat d3/fb x:0 0 0 2024-05-17T21:52:39.383 INFO:tasks.workunit.client.0.smithi107.stdout:3/26: getdents d6/d8 0 2024-05-17T21:52:39.388 INFO:tasks.workunit.client.0.smithi107.stdout:8/13: rename l2 to l6 0 2024-05-17T21:52:39.389 INFO:tasks.workunit.client.0.smithi107.stdout:0/29: stat d3/c6 0 2024-05-17T21:52:39.391 INFO:tasks.workunit.client.0.smithi107.stdout:3/27: rename c4 to d6/ca 0 2024-05-17T21:52:39.392 INFO:tasks.workunit.client.0.smithi107.stdout:8/14: mknod c7 0 2024-05-17T21:52:39.392 INFO:tasks.workunit.client.0.smithi107.stdout:1/24: truncate f3 5226315 0 2024-05-17T21:52:39.394 INFO:tasks.workunit.client.0.smithi107.stdout:0/30: rmdir d3 39 2024-05-17T21:52:39.395 INFO:tasks.workunit.client.0.smithi107.stdout:5/31: truncate d5/f9 149415 0 2024-05-17T21:52:39.397 INFO:tasks.workunit.client.0.smithi107.stdout:3/28: creat d6/d8/fb x:0 0 0 2024-05-17T21:52:39.397 INFO:tasks.workunit.client.0.smithi107.stdout:5/32: dread d5/f8 [0,4194304] 0 2024-05-17T21:52:39.398 INFO:tasks.workunit.client.0.smithi107.stdout:8/15: symlink l8 0 2024-05-17T21:52:39.400 INFO:tasks.workunit.client.0.smithi107.stdout:8/16: dread f5 [0,4194304] 0 2024-05-17T21:52:39.402 INFO:tasks.workunit.client.0.smithi107.stdout:0/31: chown d3/l8 2010391 1 2024-05-17T21:52:39.406 INFO:tasks.workunit.client.0.smithi107.stdout:3/29: creat d6/fc x:0 0 0 2024-05-17T21:52:39.406 INFO:tasks.workunit.client.0.smithi107.stdout:3/30: dread - d6/d8/fb zero size 2024-05-17T21:52:39.406 INFO:tasks.workunit.client.0.smithi107.stdout:5/33: mkdir d5/db 0 2024-05-17T21:52:39.406 INFO:tasks.workunit.client.0.smithi107.stdout:3/31: write d6/d8/fb [400117,57761] 0 2024-05-17T21:52:39.406 INFO:tasks.workunit.client.0.smithi107.stdout:1/25: symlink d4/l7 0 2024-05-17T21:52:39.407 INFO:tasks.workunit.client.0.smithi107.stdout:1/26: write d4/f5 [542440,82488] 0 2024-05-17T21:52:39.408 INFO:tasks.workunit.client.0.smithi107.stdout:1/27: truncate d4/f5 776150 0 2024-05-17T21:52:39.409 INFO:tasks.workunit.client.0.smithi107.stdout:0/32: mkdir d3/dc 0 2024-05-17T21:52:39.411 INFO:tasks.workunit.client.0.smithi107.stdout:3/32: symlink d6/d8/ld 0 2024-05-17T21:52:39.413 INFO:tasks.workunit.client.0.smithi107.stdout:5/34: mkdir d5/db/dc 0 2024-05-17T21:52:39.413 INFO:tasks.workunit.client.0.smithi107.stdout:1/28: dread d4/f5 [0,4194304] 0 2024-05-17T21:52:39.420 INFO:tasks.workunit.client.0.smithi107.stdout:1/29: creat d4/f8 x:0 0 0 2024-05-17T21:52:39.420 INFO:tasks.workunit.client.0.smithi107.stdout:8/17: dwrite f5 [0,4194304] 0 2024-05-17T21:52:39.421 INFO:tasks.workunit.client.0.smithi107.stdout:5/35: write d5/f8 [270440,79619] 0 2024-05-17T21:52:39.426 INFO:tasks.workunit.client.0.smithi107.stdout:0/33: dwrite d3/f9 [0,4194304] 0 2024-05-17T21:52:39.427 INFO:tasks.workunit.client.0.smithi107.stdout:1/30: write d4/f5 [1218915,55184] 0 2024-05-17T21:52:39.428 INFO:tasks.workunit.client.0.smithi107.stdout:0/34: stat d3/dc 0 2024-05-17T21:52:39.429 INFO:tasks.workunit.client.0.smithi107.stdout:3/33: dwrite d6/d8/fb [0,4194304] 0 2024-05-17T21:52:39.430 INFO:tasks.workunit.client.0.smithi107.stdout:0/35: write d3/fb [1003884,19712] 0 2024-05-17T21:52:39.448 INFO:tasks.workunit.client.0.smithi107.stdout:3/34: rename d6/ca to d6/d8/ce 0 2024-05-17T21:52:39.449 INFO:tasks.workunit.client.0.smithi107.stdout:0/36: symlink d3/dc/ld 0 2024-05-17T21:52:39.453 INFO:tasks.workunit.client.0.smithi107.stdout:0/37: mkdir d3/dc/de 0 2024-05-17T21:52:39.454 INFO:tasks.workunit.client.0.smithi107.stdout:1/31: rmdir d4 39 2024-05-17T21:52:39.457 INFO:tasks.workunit.client.0.smithi107.stdout:3/35: link d6/fc d6/ff 0 2024-05-17T21:52:39.457 INFO:tasks.workunit.client.0.smithi107.stdout:3/36: dread - d6/ff zero size 2024-05-17T21:52:39.465 INFO:tasks.workunit.client.0.smithi107.stdout:8/18: truncate f5 3335293 0 2024-05-17T21:52:39.466 INFO:tasks.workunit.client.0.smithi107.stdout:1/32: dwrite d4/f5 [0,4194304] 0 2024-05-17T21:52:39.466 INFO:tasks.workunit.client.0.smithi107.stdout:1/33: fdatasync d4/f8 0 2024-05-17T21:52:39.472 INFO:tasks.workunit.client.0.smithi107.stdout:1/34: write f3 [392323,45530] 0 2024-05-17T21:52:39.474 INFO:tasks.workunit.client.0.smithi107.stdout:1/35: write f3 [2092496,34473] 0 2024-05-17T21:52:39.478 INFO:tasks.workunit.client.0.smithi107.stdout:5/36: dwrite d5/f9 [0,4194304] 0 2024-05-17T21:52:39.478 INFO:tasks.workunit.client.0.smithi107.stdout:5/37: chown d5 0 1 2024-05-17T21:52:39.479 INFO:tasks.workunit.client.0.smithi107.stdout:5/38: rename d5 to d5/dd 22 2024-05-17T21:52:39.483 INFO:tasks.workunit.client.0.smithi107.stdout:0/38: dwrite f1 [0,4194304] 0 2024-05-17T21:52:39.487 INFO:tasks.workunit.client.0.smithi107.stdout:8/19: truncate f5 3849841 0 2024-05-17T21:52:39.492 INFO:tasks.workunit.client.0.smithi107.stdout:1/36: link d4/c6 d4/c9 0 2024-05-17T21:52:39.493 INFO:tasks.workunit.client.0.smithi107.stdout:1/37: fsync d4/f8 0 2024-05-17T21:52:39.493 INFO:tasks.workunit.client.0.smithi107.stdout:1/38: dread - d4/f8 zero size 2024-05-17T21:52:39.497 INFO:tasks.workunit.client.0.smithi107.stdout:1/39: dread f3 [0,4194304] 0 2024-05-17T21:52:39.499 INFO:tasks.workunit.client.0.smithi107.stdout:1/40: dread f3 [4194304,4194304] 0 2024-05-17T21:52:39.499 INFO:tasks.workunit.client.0.smithi107.stdout:1/41: chown d4 114264033 1 2024-05-17T21:52:39.499 INFO:tasks.workunit.client.0.smithi107.stdout:1/42: dread - d4/f8 zero size 2024-05-17T21:52:39.500 INFO:tasks.workunit.client.0.smithi107.stdout:5/39: link l1 d5/le 0 2024-05-17T21:52:39.501 INFO:tasks.workunit.client.0.smithi107.stdout:5/40: write d5/fa [860635,6701] 0 2024-05-17T21:52:39.502 INFO:tasks.workunit.client.0.smithi107.stdout:5/41: write d5/f7 [2629796,35150] 0 2024-05-17T21:52:39.502 INFO:tasks.workunit.client.0.smithi107.stdout:5/42: chown d5/db 1721361 1 2024-05-17T21:52:39.503 INFO:tasks.workunit.client.0.smithi107.stdout:5/43: fdatasync d5/f8 0 2024-05-17T21:52:39.506 INFO:tasks.workunit.client.0.smithi107.stdout:1/43: dread f3 [0,4194304] 0 2024-05-17T21:52:39.507 INFO:tasks.workunit.client.0.smithi107.stdout:1/44: chown d4/f5 0 1 2024-05-17T21:52:39.509 INFO:tasks.workunit.client.0.smithi107.stdout:5/44: symlink d5/db/dc/lf 0 2024-05-17T21:52:39.512 INFO:tasks.workunit.client.0.smithi107.stdout:8/20: truncate f5 1785448 0 2024-05-17T21:52:39.512 INFO:tasks.workunit.client.0.smithi107.stdout:1/45: dwrite d4/f5 [0,4194304] 0 2024-05-17T21:52:39.517 INFO:tasks.workunit.client.0.smithi107.stdout:1/46: creat d4/fa x:0 0 0 2024-05-17T21:52:39.520 INFO:tasks.workunit.client.0.smithi107.stdout:0/39: truncate d3/fb 419810 0 2024-05-17T21:52:39.520 INFO:tasks.workunit.client.0.smithi107.stdout:0/40: chown d3/f9 33395 1 2024-05-17T21:52:39.527 INFO:tasks.workunit.client.0.smithi107.stdout:0/41: creat d3/ff x:0 0 0 2024-05-17T21:52:39.537 INFO:tasks.workunit.client.0.smithi107.stdout:8/21: unlink f5 0 2024-05-17T21:52:39.542 INFO:tasks.workunit.client.0.smithi107.stdout:5/45: write f3 [591467,5171] 0 2024-05-17T21:52:39.546 INFO:tasks.workunit.client.0.smithi107.stdout:5/46: mknod d5/db/dc/c10 0 2024-05-17T21:52:39.550 INFO:tasks.workunit.client.0.smithi107.stdout:1/47: truncate f3 773972 0 2024-05-17T21:52:39.550 INFO:tasks.workunit.client.0.smithi107.stdout:5/47: dread d5/f7 [0,4194304] 0 2024-05-17T21:52:39.555 INFO:tasks.workunit.client.0.smithi107.stdout:5/48: dwrite f3 [0,4194304] 0 2024-05-17T21:52:39.556 INFO:tasks.workunit.client.0.smithi107.stdout:5/49: fdatasync d5/fa 0 2024-05-17T21:52:39.589 INFO:tasks.workunit.client.0.smithi107.stdout:8/22: dwrite f4 [0,4194304] 0 2024-05-17T21:52:39.609 INFO:tasks.workunit.client.0.smithi107.stdout:8/23: dread f4 [0,4194304] 0 2024-05-17T21:52:39.610 INFO:tasks.workunit.client.0.smithi107.stdout:1/48: truncate f3 771370 0 2024-05-17T21:52:39.613 INFO:tasks.workunit.client.0.smithi107.stdout:1/49: dwrite d4/fa [0,4194304] 0 2024-05-17T21:52:39.726 INFO:tasks.workunit.client.0.smithi107.stdout:8/24: write f4 [5041724,1890] 0 2024-05-17T21:52:39.729 INFO:tasks.workunit.client.0.smithi107.stdout:8/25: symlink l9 0 2024-05-17T21:52:39.734 INFO:tasks.workunit.client.0.smithi107.stdout:8/26: link c7 ca 0 2024-05-17T21:52:39.740 INFO:tasks.workunit.client.0.smithi107.stdout:8/27: dwrite f4 [0,4194304] 0 2024-05-17T21:52:39.745 INFO:tasks.workunit.client.0.smithi107.stdout:8/28: creat fb x:0 0 0 2024-05-17T21:52:39.745 INFO:tasks.workunit.client.0.smithi107.stdout:8/29: chown f4 6 1 2024-05-17T21:52:39.749 INFO:tasks.workunit.client.0.smithi107.stdout:8/30: mkdir dc 0 2024-05-17T21:52:39.751 INFO:tasks.workunit.client.0.smithi107.stdout:8/31: getdents dc 0 2024-05-17T21:52:39.752 INFO:tasks.workunit.client.0.smithi107.stdout:8/32: dread - fb zero size 2024-05-17T21:52:39.756 INFO:tasks.workunit.client.0.smithi107.stdout:8/33: rename l8 to dc/ld 0 2024-05-17T21:52:39.756 INFO:tasks.workunit.client.0.smithi107.stdout:8/34: readlink l9 0 2024-05-17T21:52:39.758 INFO:tasks.workunit.client.0.smithi107.stdout:8/35: symlink dc/le 0 2024-05-17T21:52:39.759 INFO:tasks.workunit.client.0.smithi107.stdout:8/36: mkdir dc/df 0 2024-05-17T21:52:39.761 INFO:tasks.workunit.client.0.smithi107.stdout:8/37: mkdir dc/d10 0 2024-05-17T21:52:39.761 INFO:tasks.workunit.client.0.smithi107.stdout:8/38: fsync f4 0 2024-05-17T21:52:39.761 INFO:tasks.workunit.client.0.smithi107.stdout:8/39: chown dc/df 81025 1 2024-05-17T21:52:39.763 INFO:tasks.workunit.client.0.smithi107.stdout:8/40: unlink l9 0 2024-05-17T21:52:39.765 INFO:tasks.workunit.client.0.smithi107.stdout:8/41: creat dc/df/f11 x:0 0 0 2024-05-17T21:52:39.766 INFO:tasks.workunit.client.0.smithi107.stdout:8/42: truncate fb 600030 0 2024-05-17T21:52:39.767 INFO:tasks.workunit.client.0.smithi107.stdout:8/43: mknod dc/df/c12 0 2024-05-17T21:52:39.769 INFO:tasks.workunit.client.0.smithi107.stdout:8/44: creat dc/f13 x:0 0 0 2024-05-17T21:52:39.772 INFO:tasks.workunit.client.0.smithi107.stdout:8/45: link dc/df/c12 dc/d10/c14 0 2024-05-17T21:52:39.776 INFO:tasks.workunit.client.0.smithi107.stdout:8/46: creat dc/d10/f15 x:0 0 0 2024-05-17T21:52:39.875 INFO:tasks.workunit.client.0.smithi107.stdout:8/47: truncate fb 354320 0 2024-05-17T21:52:39.876 INFO:tasks.workunit.client.0.smithi107.stdout:8/48: creat dc/df/f16 x:0 0 0 2024-05-17T21:52:39.877 INFO:tasks.workunit.client.0.smithi107.stdout:8/49: write dc/d10/f15 [92958,85387] 0 2024-05-17T21:52:39.901 INFO:tasks.workunit.client.0.smithi107.stdout:8/50: rmdir dc/df 39 2024-05-17T21:52:39.902 INFO:tasks.workunit.client.0.smithi107.stdout:8/51: chown dc/df 27231051 1 2024-05-17T21:52:39.903 INFO:tasks.workunit.client.0.smithi107.stdout:8/52: write dc/f13 [587506,114980] 0 2024-05-17T21:52:39.904 INFO:tasks.workunit.client.0.smithi107.stdout:8/53: mkdir dc/d10/d17 0 2024-05-17T21:52:39.905 INFO:tasks.workunit.client.0.smithi107.stdout:8/54: dread - dc/df/f16 zero size 2024-05-17T21:52:39.909 INFO:tasks.workunit.client.0.smithi107.stdout:8/55: creat dc/f18 x:0 0 0 2024-05-17T21:52:39.911 INFO:tasks.workunit.client.0.smithi107.stdout:8/56: symlink dc/df/l19 0 2024-05-17T21:52:39.912 INFO:tasks.workunit.client.0.smithi107.stdout:8/57: read - dc/f18 zero size 2024-05-17T21:52:39.915 INFO:tasks.workunit.client.0.smithi107.stdout:8/58: dread f4 [0,4194304] 0 2024-05-17T21:52:39.921 INFO:tasks.workunit.client.0.smithi107.stdout:8/59: dwrite dc/f18 [0,4194304] 0 2024-05-17T21:52:39.925 INFO:tasks.workunit.client.0.smithi107.stdout:8/60: mknod dc/c1a 0 2024-05-17T21:52:39.929 INFO:tasks.workunit.client.0.smithi107.stdout:8/61: creat dc/d10/d17/f1b x:0 0 0 2024-05-17T21:52:39.940 INFO:tasks.workunit.client.0.smithi107.stdout:8/62: truncate dc/d10/f15 100844 0 2024-05-17T21:52:39.942 INFO:tasks.workunit.client.0.smithi107.stdout:8/63: mknod dc/d10/d17/c1c 0 2024-05-17T21:52:39.953 INFO:tasks.workunit.client.0.smithi107.stdout:8/64: dwrite fb [0,4194304] 0 2024-05-17T21:52:39.954 INFO:tasks.workunit.client.0.smithi107.stdout:8/65: chown fb 1 1 2024-05-17T21:52:39.956 INFO:tasks.workunit.client.0.smithi107.stdout:8/66: readlink l6 0 2024-05-17T21:52:40.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:40 smithi138 ceph-mon[35948]: pgmap v28: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.9 MiB/s rd, 3.5 MiB/s wr, 362 op/s 2024-05-17T21:52:40.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:40 smithi107 ceph-mon[28434]: pgmap v28: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.9 MiB/s rd, 3.5 MiB/s wr, 362 op/s 2024-05-17T21:52:40.858 INFO:tasks.workunit.client.1.smithi138.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-05-17T21:52:40.863 INFO:tasks.workunit.client.1.smithi138.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-05-17T21:52:40.863 INFO:tasks.workunit.client.1.smithi138.stderr:+ make 2024-05-17T21:52:41.005 INFO:tasks.workunit.client.1.smithi138.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-05-17T21:52:41.330 INFO:tasks.workunit.client.1.smithi138.stderr:++ readlink -f fsstress 2024-05-17T21:52:41.331 INFO:tasks.workunit.client.1.smithi138.stderr:+ BIN=/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-05-17T21:52:41.331 INFO:tasks.workunit.client.1.smithi138.stderr:+ popd 2024-05-17T21:52:41.332 INFO:tasks.workunit.client.1.smithi138.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-05-17T21:52:41.332 INFO:tasks.workunit.client.1.smithi138.stderr:+ popd 2024-05-17T21:52:41.332 INFO:tasks.workunit.client.1.smithi138.stdout:~/cephtest/mnt.1/client.1/tmp 2024-05-17T21:52:41.333 INFO:tasks.workunit.client.1.smithi138.stderr:++ mktemp -d -p . 2024-05-17T21:52:41.335 INFO:tasks.workunit.client.1.smithi138.stderr:+ T=./tmp.T32dW2XgB2 2024-05-17T21:52:41.335 INFO:tasks.workunit.client.1.smithi138.stderr:+ /home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.T32dW2XgB2 -l 1 -n 1000 -p 10 -v 2024-05-17T21:52:41.337 INFO:tasks.workunit.client.1.smithi138.stdout:seed = 1715640848 2024-05-17T21:52:41.340 INFO:tasks.workunit.client.1.smithi138.stdout:0/0: dwrite - no filename 2024-05-17T21:52:41.340 INFO:tasks.workunit.client.1.smithi138.stdout:0/1: dwrite - no filename 2024-05-17T21:52:41.341 INFO:tasks.workunit.client.1.smithi138.stdout:1/0: dwrite - no filename 2024-05-17T21:52:41.341 INFO:tasks.workunit.client.1.smithi138.stdout:1/1: dwrite - no filename 2024-05-17T21:52:41.341 INFO:tasks.workunit.client.1.smithi138.stdout:1/2: fsync - no filename 2024-05-17T21:52:41.341 INFO:tasks.workunit.client.1.smithi138.stdout:1/3: chown . 3045 1 2024-05-17T21:52:41.342 INFO:tasks.workunit.client.1.smithi138.stdout:1/4: link - no file 2024-05-17T21:52:41.342 INFO:tasks.workunit.client.1.smithi138.stdout:1/5: dwrite - no filename 2024-05-17T21:52:41.342 INFO:tasks.workunit.client.1.smithi138.stdout:1/6: dwrite - no filename 2024-05-17T21:52:41.342 INFO:tasks.workunit.client.1.smithi138.stdout:0/2: creat f0 x:0 0 0 2024-05-17T21:52:41.343 INFO:tasks.workunit.client.1.smithi138.stdout:0/3: symlink l1 0 2024-05-17T21:52:41.344 INFO:tasks.workunit.client.1.smithi138.stdout:1/7: creat f0 x:0 0 0 2024-05-17T21:52:41.345 INFO:tasks.workunit.client.1.smithi138.stdout:0/4: creat f2 x:0 0 0 2024-05-17T21:52:41.346 INFO:tasks.workunit.client.1.smithi138.stdout:1/8: symlink l1 0 2024-05-17T21:52:41.347 INFO:tasks.workunit.client.1.smithi138.stdout:1/9: write f0 [840118,65661] 0 2024-05-17T21:52:41.347 INFO:tasks.workunit.client.1.smithi138.stdout:2/0: creat f0 x:0 0 0 2024-05-17T21:52:41.349 INFO:tasks.workunit.client.1.smithi138.stdout:1/10: write f0 [741457,31311] 0 2024-05-17T21:52:41.350 INFO:tasks.workunit.client.1.smithi138.stdout:0/5: mkdir d3 0 2024-05-17T21:52:41.351 INFO:tasks.workunit.client.1.smithi138.stdout:2/1: mknod c1 0 2024-05-17T21:52:41.351 INFO:tasks.workunit.client.1.smithi138.stdout:3/0: mknod c0 0 2024-05-17T21:52:41.352 INFO:tasks.workunit.client.1.smithi138.stdout:3/1: dwrite - no filename 2024-05-17T21:52:41.352 INFO:tasks.workunit.client.1.smithi138.stdout:3/2: dwrite - no filename 2024-05-17T21:52:41.352 INFO:tasks.workunit.client.1.smithi138.stdout:3/3: fdatasync - no filename 2024-05-17T21:52:41.352 INFO:tasks.workunit.client.1.smithi138.stdout:3/4: dread - no filename 2024-05-17T21:52:41.352 INFO:tasks.workunit.client.1.smithi138.stdout:3/5: fdatasync - no filename 2024-05-17T21:52:41.352 INFO:tasks.workunit.client.1.smithi138.stdout:3/6: dwrite - no filename 2024-05-17T21:52:41.352 INFO:tasks.workunit.client.1.smithi138.stdout:3/7: fsync - no filename 2024-05-17T21:52:41.352 INFO:tasks.workunit.client.1.smithi138.stdout:3/8: truncate - no filename 2024-05-17T21:52:41.353 INFO:tasks.workunit.client.1.smithi138.stdout:5/0: dwrite - no filename 2024-05-17T21:52:41.353 INFO:tasks.workunit.client.1.smithi138.stdout:5/1: chown . 49813325 1 2024-05-17T21:52:41.353 INFO:tasks.workunit.client.1.smithi138.stdout:5/2: read - no filename 2024-05-17T21:52:41.353 INFO:tasks.workunit.client.1.smithi138.stdout:5/3: unlink - no file 2024-05-17T21:52:41.353 INFO:tasks.workunit.client.1.smithi138.stdout:5/4: fdatasync - no filename 2024-05-17T21:52:41.353 INFO:tasks.workunit.client.1.smithi138.stdout:5/5: link - no file 2024-05-17T21:52:41.353 INFO:tasks.workunit.client.1.smithi138.stdout:5/6: truncate - no filename 2024-05-17T21:52:41.353 INFO:tasks.workunit.client.1.smithi138.stdout:1/11: unlink l1 0 2024-05-17T21:52:41.356 INFO:tasks.workunit.client.1.smithi138.stdout:2/2: rename f0 to f2 0 2024-05-17T21:52:41.356 INFO:tasks.workunit.client.1.smithi138.stdout:3/9: mkdir d1 0 2024-05-17T21:52:41.356 INFO:tasks.workunit.client.1.smithi138.stdout:3/10: chown c0 6252762 1 2024-05-17T21:52:41.357 INFO:tasks.workunit.client.1.smithi138.stdout:3/11: dwrite - no filename 2024-05-17T21:52:41.357 INFO:tasks.workunit.client.1.smithi138.stdout:3/12: truncate - no filename 2024-05-17T21:52:41.357 INFO:tasks.workunit.client.1.smithi138.stdout:3/13: dread - no filename 2024-05-17T21:52:41.357 INFO:tasks.workunit.client.1.smithi138.stdout:3/14: stat d1 0 2024-05-17T21:52:41.357 INFO:tasks.workunit.client.1.smithi138.stdout:3/15: dread - no filename 2024-05-17T21:52:41.358 INFO:tasks.workunit.client.1.smithi138.stdout:6/0: dread - no filename 2024-05-17T21:52:41.358 INFO:tasks.workunit.client.1.smithi138.stdout:6/1: readlink - no filename 2024-05-17T21:52:41.358 INFO:tasks.workunit.client.1.smithi138.stdout:6/2: write - no filename 2024-05-17T21:52:41.358 INFO:tasks.workunit.client.1.smithi138.stdout:6/3: dread - no filename 2024-05-17T21:52:41.359 INFO:tasks.workunit.client.1.smithi138.stdout:6/4: fdatasync - no filename 2024-05-17T21:52:41.359 INFO:tasks.workunit.client.1.smithi138.stdout:6/5: write - no filename 2024-05-17T21:52:41.359 INFO:tasks.workunit.client.1.smithi138.stdout:6/6: unlink - no file 2024-05-17T21:52:41.359 INFO:tasks.workunit.client.1.smithi138.stdout:6/7: chown . 39 1 2024-05-17T21:52:41.359 INFO:tasks.workunit.client.1.smithi138.stdout:6/8: unlink - no file 2024-05-17T21:52:41.359 INFO:tasks.workunit.client.1.smithi138.stdout:4/0: mkdir d0 0 2024-05-17T21:52:41.359 INFO:tasks.workunit.client.1.smithi138.stdout:4/1: chown d0 214794 1 2024-05-17T21:52:41.359 INFO:tasks.workunit.client.1.smithi138.stdout:4/2: write - no filename 2024-05-17T21:52:41.359 INFO:tasks.workunit.client.1.smithi138.stdout:1/12: creat f2 x:0 0 0 2024-05-17T21:52:41.360 INFO:tasks.workunit.client.1.smithi138.stdout:0/6: symlink d3/l4 0 2024-05-17T21:52:41.360 INFO:tasks.workunit.client.1.smithi138.stdout:1/13: write f2 [584892,127177] 0 2024-05-17T21:52:41.360 INFO:tasks.workunit.client.1.smithi138.stdout:5/7: mknod c0 0 2024-05-17T21:52:41.360 INFO:tasks.workunit.client.1.smithi138.stdout:1/14: readlink - no filename 2024-05-17T21:52:41.360 INFO:tasks.workunit.client.1.smithi138.stdout:1/15: truncate f0 1835842 0 2024-05-17T21:52:41.363 INFO:tasks.workunit.client.1.smithi138.stdout:6/9: mknod c0 0 2024-05-17T21:52:41.363 INFO:tasks.workunit.client.1.smithi138.stdout:6/10: dwrite - no filename 2024-05-17T21:52:41.363 INFO:tasks.workunit.client.1.smithi138.stdout:6/11: dread - no filename 2024-05-17T21:52:41.363 INFO:tasks.workunit.client.1.smithi138.stdout:7/0: dwrite - no filename 2024-05-17T21:52:41.363 INFO:tasks.workunit.client.1.smithi138.stdout:4/3: mknod d0/c1 0 2024-05-17T21:52:41.365 INFO:tasks.workunit.client.1.smithi138.stdout:8/0: unlink - no file 2024-05-17T21:52:41.366 INFO:tasks.workunit.client.1.smithi138.stdout:8/1: truncate - no filename 2024-05-17T21:52:41.366 INFO:tasks.workunit.client.1.smithi138.stdout:1/16: creat f3 x:0 0 0 2024-05-17T21:52:41.366 INFO:tasks.workunit.client.1.smithi138.stdout:1/17: rmdir - no directory 2024-05-17T21:52:41.366 INFO:tasks.workunit.client.1.smithi138.stdout:6/12: creat f1 x:0 0 0 2024-05-17T21:52:41.367 INFO:tasks.workunit.client.1.smithi138.stdout:4/4: creat d0/f2 x:0 0 0 2024-05-17T21:52:41.369 INFO:tasks.workunit.client.1.smithi138.stdout:6/13: write f1 [550904,18997] 0 2024-05-17T21:52:41.370 INFO:tasks.workunit.client.1.smithi138.stdout:0/7: dwrite f2 [0,4194304] 0 2024-05-17T21:52:41.371 INFO:tasks.workunit.client.1.smithi138.stdout:9/0: rmdir - no directory 2024-05-17T21:52:41.371 INFO:tasks.workunit.client.1.smithi138.stdout:4/5: dread - d0/f2 zero size 2024-05-17T21:52:41.371 INFO:tasks.workunit.client.1.smithi138.stdout:8/2: mkdir d0 0 2024-05-17T21:52:41.371 INFO:tasks.workunit.client.1.smithi138.stdout:8/3: readlink - no filename 2024-05-17T21:52:41.373 INFO:tasks.workunit.client.1.smithi138.stdout:4/6: chown d0 5 1 2024-05-17T21:52:41.374 INFO:tasks.workunit.client.1.smithi138.stdout:0/8: mkdir d3/d5 0 2024-05-17T21:52:41.376 INFO:tasks.workunit.client.1.smithi138.stdout:0/9: truncate f0 602018 0 2024-05-17T21:52:41.377 INFO:tasks.workunit.client.1.smithi138.stdout:6/14: read f1 [91257,66729] 0 2024-05-17T21:52:41.379 INFO:tasks.workunit.client.1.smithi138.stdout:4/7: unlink d0/f2 0 2024-05-17T21:52:41.379 INFO:tasks.workunit.client.1.smithi138.stdout:4/8: truncate - no filename 2024-05-17T21:52:41.379 INFO:tasks.workunit.client.1.smithi138.stdout:4/9: dwrite - no filename 2024-05-17T21:52:41.379 INFO:tasks.workunit.client.1.smithi138.stdout:4/10: dread - no filename 2024-05-17T21:52:41.379 INFO:tasks.workunit.client.1.smithi138.stdout:4/11: write - no filename 2024-05-17T21:52:41.379 INFO:tasks.workunit.client.1.smithi138.stdout:1/18: dwrite f2 [0,4194304] 0 2024-05-17T21:52:41.381 INFO:tasks.workunit.client.1.smithi138.stdout:9/1: mknod c0 0 2024-05-17T21:52:41.382 INFO:tasks.workunit.client.1.smithi138.stdout:8/4: getdents d0 0 2024-05-17T21:52:41.382 INFO:tasks.workunit.client.1.smithi138.stdout:8/5: dread - no filename 2024-05-17T21:52:41.382 INFO:tasks.workunit.client.1.smithi138.stdout:8/6: stat d0 0 2024-05-17T21:52:41.383 INFO:tasks.workunit.client.1.smithi138.stdout:0/10: creat d3/f6 x:0 0 0 2024-05-17T21:52:41.386 INFO:tasks.workunit.client.1.smithi138.stdout:6/15: creat f2 x:0 0 0 2024-05-17T21:52:41.387 INFO:tasks.workunit.client.1.smithi138.stdout:4/12: mkdir d0/d3 0 2024-05-17T21:52:41.387 INFO:tasks.workunit.client.1.smithi138.stdout:6/16: chown f1 2959 1 2024-05-17T21:52:41.388 INFO:tasks.workunit.client.1.smithi138.stdout:1/19: mkdir d4 0 2024-05-17T21:52:41.391 INFO:tasks.workunit.client.1.smithi138.stdout:9/2: creat f1 x:0 0 0 2024-05-17T21:52:41.393 INFO:tasks.workunit.client.1.smithi138.stdout:0/11: rename d3/f6 to d3/f7 0 2024-05-17T21:52:41.393 INFO:tasks.workunit.client.1.smithi138.stdout:0/12: fdatasync f0 0 2024-05-17T21:52:41.394 INFO:tasks.workunit.client.1.smithi138.stdout:4/13: symlink d0/l4 0 2024-05-17T21:52:41.396 INFO:tasks.workunit.client.1.smithi138.stdout:6/17: unlink f2 0 2024-05-17T21:52:41.398 INFO:tasks.workunit.client.1.smithi138.stdout:8/7: creat d0/f1 x:0 0 0 2024-05-17T21:52:41.400 INFO:tasks.workunit.client.1.smithi138.stdout:0/13: fdatasync d3/f7 0 2024-05-17T21:52:41.402 INFO:tasks.workunit.client.1.smithi138.stdout:0/14: dread f0 [0,4194304] 0 2024-05-17T21:52:41.403 INFO:tasks.workunit.client.1.smithi138.stdout:6/18: creat f3 x:0 0 0 2024-05-17T21:52:41.403 INFO:tasks.workunit.client.1.smithi138.stdout:6/19: fdatasync f3 0 2024-05-17T21:52:41.403 INFO:tasks.workunit.client.1.smithi138.stdout:6/20: chown f1 0 1 2024-05-17T21:52:41.403 INFO:tasks.workunit.client.1.smithi138.stdout:1/20: mknod d4/c5 0 2024-05-17T21:52:41.404 INFO:tasks.workunit.client.1.smithi138.stdout:6/21: chown f1 1 1 2024-05-17T21:52:41.404 INFO:tasks.workunit.client.1.smithi138.stdout:1/21: chown f2 23 1 2024-05-17T21:52:41.405 INFO:tasks.workunit.client.1.smithi138.stdout:8/8: creat d0/f2 x:0 0 0 2024-05-17T21:52:41.405 INFO:tasks.workunit.client.1.smithi138.stdout:1/22: fdatasync f2 0 2024-05-17T21:52:41.405 INFO:tasks.workunit.client.1.smithi138.stdout:8/9: readlink - no filename 2024-05-17T21:52:41.405 INFO:tasks.workunit.client.1.smithi138.stdout:6/22: chown f3 2042 1 2024-05-17T21:52:41.405 INFO:tasks.workunit.client.1.smithi138.stdout:6/23: stat f3 0 2024-05-17T21:52:41.405 INFO:tasks.workunit.client.1.smithi138.stdout:4/14: mknod d0/d3/c5 0 2024-05-17T21:52:41.406 INFO:tasks.workunit.client.1.smithi138.stdout:6/24: read - f3 zero size 2024-05-17T21:52:41.406 INFO:tasks.workunit.client.1.smithi138.stdout:0/15: creat d3/f8 x:0 0 0 2024-05-17T21:52:41.408 INFO:tasks.workunit.client.1.smithi138.stdout:8/10: mknod d0/c3 0 2024-05-17T21:52:41.409 INFO:tasks.workunit.client.1.smithi138.stdout:8/11: write d0/f2 [241682,105213] 0 2024-05-17T21:52:41.410 INFO:tasks.workunit.client.1.smithi138.stdout:1/23: creat d4/f6 x:0 0 0 2024-05-17T21:52:41.411 INFO:tasks.workunit.client.1.smithi138.stdout:6/25: creat f4 x:0 0 0 2024-05-17T21:52:41.417 INFO:tasks.workunit.client.1.smithi138.stdout:4/15: link d0/l4 d0/l6 0 2024-05-17T21:52:41.419 INFO:tasks.workunit.client.1.smithi138.stdout:0/16: rmdir d3/d5 0 2024-05-17T21:52:41.420 INFO:tasks.workunit.client.1.smithi138.stdout:4/16: rmdir d0/d3 39 2024-05-17T21:52:41.421 INFO:tasks.workunit.client.1.smithi138.stdout:6/26: dwrite f1 [0,4194304] 0 2024-05-17T21:52:41.421 INFO:tasks.workunit.client.1.smithi138.stdout:6/27: readlink - no filename 2024-05-17T21:52:41.421 INFO:tasks.workunit.client.1.smithi138.stdout:0/17: write f2 [4616154,51374] 0 2024-05-17T21:52:41.421 INFO:tasks.workunit.client.1.smithi138.stdout:6/28: stat f4 0 2024-05-17T21:52:41.425 INFO:tasks.workunit.client.1.smithi138.stdout:0/18: mknod d3/c9 0 2024-05-17T21:52:41.445 INFO:tasks.workunit.client.1.smithi138.stdout:0/19: rename d3/c9 to d3/ca 0 2024-05-17T21:52:41.446 INFO:tasks.workunit.client.1.smithi138.stdout:4/17: creat d0/f7 x:0 0 0 2024-05-17T21:52:41.452 INFO:tasks.workunit.client.1.smithi138.stdout:6/29: truncate f1 719800 0 2024-05-17T21:52:41.580 INFO:tasks.workunit.client.1.smithi138.stdout:0/20: symlink d3/lb 0 2024-05-17T21:52:41.581 INFO:tasks.workunit.client.1.smithi138.stdout:6/30: fdatasync f1 0 2024-05-17T21:52:41.582 INFO:tasks.workunit.client.1.smithi138.stdout:6/31: write f4 [62555,19794] 0 2024-05-17T21:52:41.582 INFO:tasks.workunit.client.1.smithi138.stdout:4/18: symlink d0/d3/l8 0 2024-05-17T21:52:41.583 INFO:tasks.workunit.client.1.smithi138.stdout:6/32: truncate f3 59638 0 2024-05-17T21:52:41.584 INFO:tasks.workunit.client.1.smithi138.stdout:0/21: rename d3/f8 to d3/fc 0 2024-05-17T21:52:41.585 INFO:tasks.workunit.client.1.smithi138.stdout:0/22: truncate d3/fc 561924 0 2024-05-17T21:52:41.585 INFO:tasks.workunit.client.1.smithi138.stdout:0/23: read - d3/f7 zero size 2024-05-17T21:52:41.587 INFO:tasks.workunit.client.1.smithi138.stdout:0/24: dread f0 [0,4194304] 0 2024-05-17T21:52:41.587 INFO:tasks.workunit.client.1.smithi138.stdout:4/19: symlink d0/l9 0 2024-05-17T21:52:41.587 INFO:tasks.workunit.client.1.smithi138.stdout:4/20: dread - d0/f7 zero size 2024-05-17T21:52:41.588 INFO:tasks.workunit.client.1.smithi138.stdout:0/25: truncate d3/f7 846404 0 2024-05-17T21:52:41.588 INFO:tasks.workunit.client.1.smithi138.stdout:4/21: stat d0/d3/c5 0 2024-05-17T21:52:41.591 INFO:tasks.workunit.client.1.smithi138.stdout:0/26: creat d3/fd x:0 0 0 2024-05-17T21:52:41.596 INFO:tasks.workunit.client.1.smithi138.stdout:6/33: dread f1 [0,4194304] 0 2024-05-17T21:52:41.597 INFO:tasks.workunit.client.1.smithi138.stdout:0/27: dwrite d3/f7 [0,4194304] 0 2024-05-17T21:52:41.600 INFO:tasks.workunit.client.1.smithi138.stdout:0/28: symlink d3/le 0 2024-05-17T21:52:41.602 INFO:tasks.workunit.client.1.smithi138.stdout:0/29: mknod d3/cf 0 2024-05-17T21:52:41.606 INFO:tasks.workunit.client.1.smithi138.stdout:6/34: getdents . 0 2024-05-17T21:52:41.610 INFO:tasks.workunit.client.1.smithi138.stdout:0/30: dwrite f2 [0,4194304] 0 2024-05-17T21:52:41.612 INFO:tasks.workunit.client.1.smithi138.stdout:0/31: dread f0 [0,4194304] 0 2024-05-17T21:52:41.615 INFO:tasks.workunit.client.1.smithi138.stdout:0/32: mkdir d3/d10 0 2024-05-17T21:52:41.615 INFO:tasks.workunit.client.1.smithi138.stdout:6/35: dwrite f1 [0,4194304] 0 2024-05-17T21:52:41.616 INFO:tasks.workunit.client.1.smithi138.stdout:6/36: fsync f4 0 2024-05-17T21:52:41.616 INFO:tasks.workunit.client.1.smithi138.stdout:0/33: read - d3/fd zero size 2024-05-17T21:52:41.620 INFO:tasks.workunit.client.1.smithi138.stdout:6/37: symlink l5 0 2024-05-17T21:52:41.626 INFO:tasks.workunit.client.1.smithi138.stdout:0/34: creat d3/d10/f11 x:0 0 0 2024-05-17T21:52:41.627 INFO:tasks.workunit.client.1.smithi138.stdout:0/35: read - d3/fd zero size 2024-05-17T21:52:41.627 INFO:tasks.workunit.client.1.smithi138.stdout:0/36: readlink d3/l4 0 2024-05-17T21:52:41.628 INFO:tasks.workunit.client.1.smithi138.stdout:0/37: write d3/fd [182517,1814] 0 2024-05-17T21:52:41.629 INFO:tasks.workunit.client.1.smithi138.stdout:0/38: chown d3/cf 1525993 1 2024-05-17T21:52:41.641 INFO:tasks.workunit.client.1.smithi138.stdout:0/39: fdatasync d3/fd 0 2024-05-17T21:52:41.641 INFO:tasks.workunit.client.1.smithi138.stdout:6/38: fdatasync f1 0 2024-05-17T21:52:41.642 INFO:tasks.workunit.client.1.smithi138.stdout:0/40: dread f0 [0,4194304] 0 2024-05-17T21:52:41.644 INFO:tasks.workunit.client.1.smithi138.stdout:6/39: creat f6 x:0 0 0 2024-05-17T21:52:41.645 INFO:tasks.workunit.client.1.smithi138.stdout:6/40: write f4 [1107748,61538] 0 2024-05-17T21:52:41.648 INFO:tasks.workunit.client.1.smithi138.stdout:6/41: mkdir d7 0 2024-05-17T21:52:41.648 INFO:tasks.workunit.client.1.smithi138.stdout:6/42: chown f4 13573 1 2024-05-17T21:52:41.652 INFO:tasks.workunit.client.1.smithi138.stdout:6/43: dread f4 [0,4194304] 0 2024-05-17T21:52:41.656 INFO:tasks.workunit.client.1.smithi138.stdout:0/41: link d3/ca d3/d10/c12 0 2024-05-17T21:52:41.657 INFO:tasks.workunit.client.1.smithi138.stdout:0/42: write d3/d10/f11 [276080,63122] 0 2024-05-17T21:52:41.658 INFO:tasks.workunit.client.1.smithi138.stdout:0/43: write d3/f7 [1979385,68033] 0 2024-05-17T21:52:41.658 INFO:tasks.workunit.client.1.smithi138.stdout:6/44: mkdir d7/d8 0 2024-05-17T21:52:41.667 INFO:tasks.workunit.client.1.smithi138.stdout:6/45: dwrite f1 [4194304,4194304] 0 2024-05-17T21:52:41.669 INFO:tasks.workunit.client.1.smithi138.stdout:6/46: truncate f3 922655 0 2024-05-17T21:52:41.670 INFO:tasks.workunit.client.1.smithi138.stdout:6/47: dread f3 [0,4194304] 0 2024-05-17T21:52:41.672 INFO:tasks.workunit.client.1.smithi138.stdout:6/48: dread f4 [0,4194304] 0 2024-05-17T21:52:41.678 INFO:tasks.workunit.client.1.smithi138.stdout:6/49: creat d7/d8/f9 x:0 0 0 2024-05-17T21:52:41.685 INFO:tasks.workunit.client.1.smithi138.stdout:6/50: dwrite f6 [0,4194304] 0 2024-05-17T21:52:41.685 INFO:tasks.workunit.client.1.smithi138.stdout:6/51: chown f3 3224320 1 2024-05-17T21:52:41.689 INFO:tasks.workunit.client.1.smithi138.stdout:6/52: rename f4 to d7/fa 0 2024-05-17T21:52:41.693 INFO:tasks.workunit.client.1.smithi138.stdout:6/53: dread f1 [0,4194304] 0 2024-05-17T21:52:41.697 INFO:tasks.workunit.client.1.smithi138.stdout:6/54: mkdir d7/db 0 2024-05-17T21:52:41.701 INFO:tasks.workunit.client.1.smithi138.stdout:6/55: write f3 [1514798,110655] 0 2024-05-17T21:52:41.708 INFO:tasks.workunit.client.1.smithi138.stdout:6/56: dwrite f3 [0,4194304] 0 2024-05-17T21:52:41.711 INFO:tasks.workunit.client.1.smithi138.stdout:6/57: creat d7/fc x:0 0 0 2024-05-17T21:52:41.713 INFO:tasks.workunit.client.1.smithi138.stdout:6/58: mknod d7/cd 0 2024-05-17T21:52:41.717 INFO:tasks.workunit.client.1.smithi138.stdout:6/59: unlink f6 0 2024-05-17T21:52:41.721 INFO:tasks.workunit.client.1.smithi138.stdout:6/60: mknod d7/db/ce 0 2024-05-17T21:52:41.721 INFO:tasks.workunit.client.1.smithi138.stdout:6/61: chown d7/d8 90217455 1 2024-05-17T21:52:41.721 INFO:tasks.workunit.client.1.smithi138.stdout:6/62: readlink l5 0 2024-05-17T21:52:41.722 INFO:tasks.workunit.client.1.smithi138.stdout:6/63: read f1 [8120995,42290] 0 2024-05-17T21:52:41.724 INFO:tasks.workunit.client.1.smithi138.stdout:6/64: creat d7/d8/ff x:0 0 0 2024-05-17T21:52:41.733 INFO:tasks.workunit.client.1.smithi138.stdout:6/65: dwrite d7/fa [0,4194304] 0 2024-05-17T21:52:41.736 INFO:tasks.workunit.client.1.smithi138.stdout:6/66: mknod d7/c10 0 2024-05-17T21:52:41.740 INFO:tasks.workunit.client.1.smithi138.stdout:6/67: rename d7/d8/f9 to d7/db/f11 0 2024-05-17T21:52:41.740 INFO:tasks.workunit.client.1.smithi138.stdout:6/68: stat l5 0 2024-05-17T21:52:41.743 INFO:tasks.workunit.client.1.smithi138.stdout:6/69: mkdir d7/db/d12 0 2024-05-17T21:52:41.746 INFO:tasks.workunit.client.1.smithi138.stdout:6/70: mknod d7/d8/c13 0 2024-05-17T21:52:41.749 INFO:tasks.workunit.client.1.smithi138.stdout:6/71: rename d7/d8/c13 to d7/d8/c14 0 2024-05-17T21:52:41.750 INFO:tasks.workunit.client.1.smithi138.stdout:6/72: chown d7/c10 6430424 1 2024-05-17T21:52:41.750 INFO:tasks.workunit.client.1.smithi138.stdout:6/73: stat d7/db 0 2024-05-17T21:52:41.751 INFO:tasks.workunit.client.1.smithi138.stdout:6/74: read - d7/d8/ff zero size 2024-05-17T21:52:41.753 INFO:tasks.workunit.client.1.smithi138.stdout:6/75: mknod d7/db/c15 0 2024-05-17T21:52:41.754 INFO:tasks.workunit.client.1.smithi138.stdout:6/76: fdatasync d7/d8/ff 0 2024-05-17T21:52:41.761 INFO:tasks.workunit.client.1.smithi138.stdout:6/77: dwrite d7/d8/ff [0,4194304] 0 2024-05-17T21:52:41.762 INFO:tasks.workunit.client.1.smithi138.stdout:6/78: dread - d7/fc zero size 2024-05-17T21:52:41.763 INFO:tasks.workunit.client.1.smithi138.stdout:6/79: write d7/d8/ff [396334,59646] 0 2024-05-17T21:52:41.767 INFO:tasks.workunit.client.1.smithi138.stdout:6/80: dread f3 [0,4194304] 0 2024-05-17T21:52:41.774 INFO:tasks.workunit.client.1.smithi138.stdout:6/81: dwrite d7/fc [0,4194304] 0 2024-05-17T21:52:41.783 INFO:tasks.workunit.client.1.smithi138.stdout:6/82: dwrite f1 [4194304,4194304] 0 2024-05-17T21:52:41.787 INFO:tasks.workunit.client.1.smithi138.stdout:6/83: symlink d7/l16 0 2024-05-17T21:52:41.791 INFO:tasks.workunit.client.1.smithi138.stdout:6/84: dread d7/d8/ff [0,4194304] 0 2024-05-17T21:52:41.795 INFO:tasks.workunit.client.1.smithi138.stdout:6/85: dread d7/d8/ff [0,4194304] 0 2024-05-17T21:52:41.799 INFO:tasks.workunit.client.1.smithi138.stdout:6/86: symlink d7/db/d12/l17 0 2024-05-17T21:52:41.800 INFO:tasks.workunit.client.1.smithi138.stdout:6/87: write f1 [9207419,124520] 0 2024-05-17T21:52:41.800 INFO:tasks.workunit.client.1.smithi138.stdout:6/88: chown d7/db/d12 459 1 2024-05-17T21:52:41.803 INFO:tasks.workunit.client.1.smithi138.stdout:6/89: creat d7/f18 x:0 0 0 2024-05-17T21:52:41.810 INFO:tasks.workunit.client.1.smithi138.stdout:6/90: dwrite d7/d8/ff [0,4194304] 0 2024-05-17T21:52:41.846 INFO:tasks.workunit.client.1.smithi138.stdout:4/22: rmdir d0 39 2024-05-17T21:52:41.849 INFO:tasks.workunit.client.1.smithi138.stdout:4/23: truncate d0/f7 711970 0 2024-05-17T21:52:41.852 INFO:tasks.workunit.client.1.smithi138.stdout:4/24: symlink d0/d3/la 0 2024-05-17T21:52:41.855 INFO:tasks.workunit.client.1.smithi138.stdout:4/25: mkdir d0/d3/db 0 2024-05-17T21:52:41.859 INFO:tasks.workunit.client.1.smithi138.stdout:4/26: mknod d0/d3/cc 0 2024-05-17T21:52:41.867 INFO:tasks.workunit.client.1.smithi138.stdout:4/27: dwrite d0/f7 [0,4194304] 0 2024-05-17T21:52:41.892 INFO:tasks.workunit.client.1.smithi138.stdout:0/44: truncate d3/f7 3397493 0 2024-05-17T21:52:41.892 INFO:tasks.workunit.client.1.smithi138.stdout:0/45: chown d3/d10 10 1 2024-05-17T21:52:41.899 INFO:tasks.workunit.client.1.smithi138.stdout:0/46: dwrite d3/fc [0,4194304] 0 2024-05-17T21:52:41.900 INFO:tasks.workunit.client.1.smithi138.stdout:0/47: write d3/fd [484724,59811] 0 2024-05-17T21:52:41.901 INFO:tasks.workunit.client.1.smithi138.stdout:0/48: fsync f0 0 2024-05-17T21:52:41.901 INFO:tasks.workunit.client.1.smithi138.stdout:0/49: fdatasync f2 0 2024-05-17T21:52:41.907 INFO:tasks.workunit.client.1.smithi138.stdout:0/50: rmdir d3/d10 39 2024-05-17T21:52:41.913 INFO:tasks.workunit.client.1.smithi138.stdout:0/51: symlink d3/d10/l13 0 2024-05-17T21:52:41.913 INFO:tasks.workunit.client.1.smithi138.stdout:0/52: chown f0 3955590 1 2024-05-17T21:52:41.914 INFO:tasks.workunit.client.1.smithi138.stdout:0/53: dread f0 [0,4194304] 0 2024-05-17T21:52:41.917 INFO:tasks.workunit.client.1.smithi138.stdout:0/54: creat d3/d10/f14 x:0 0 0 2024-05-17T21:52:41.917 INFO:tasks.workunit.client.1.smithi138.stdout:0/55: fdatasync d3/d10/f14 0 2024-05-17T21:52:41.964 INFO:tasks.workunit.client.1.smithi138.stdout:6/91: truncate f1 9012073 0 2024-05-17T21:52:41.967 INFO:tasks.workunit.client.1.smithi138.stdout:6/92: dread d7/d8/ff [0,4194304] 0 2024-05-17T21:52:41.968 INFO:tasks.workunit.client.1.smithi138.stdout:6/93: fdatasync d7/fc 0 2024-05-17T21:52:41.987 INFO:tasks.workunit.client.1.smithi138.stdout:4/28: dwrite d0/f7 [4194304,4194304] 0 2024-05-17T21:52:41.987 INFO:tasks.workunit.client.1.smithi138.stdout:4/29: write d0/f7 [3840354,52656] 0 2024-05-17T21:52:42.002 INFO:tasks.workunit.client.1.smithi138.stdout:0/56: getdents d3/d10 0 2024-05-17T21:52:42.002 INFO:tasks.workunit.client.1.smithi138.stdout:0/57: stat f2 0 2024-05-17T21:52:42.003 INFO:tasks.workunit.client.1.smithi138.stdout:0/58: creat d3/d10/f15 x:0 0 0 2024-05-17T21:52:42.005 INFO:tasks.workunit.client.1.smithi138.stdout:0/59: creat d3/d10/f16 x:0 0 0 2024-05-17T21:52:42.012 INFO:tasks.workunit.client.1.smithi138.stdout:0/60: dwrite d3/d10/f11 [0,4194304] 0 2024-05-17T21:52:42.013 INFO:tasks.workunit.client.1.smithi138.stdout:0/61: dread - d3/d10/f14 zero size 2024-05-17T21:52:42.014 INFO:tasks.workunit.client.1.smithi138.stdout:6/94: write f1 [4303084,55331] 0 2024-05-17T21:52:42.016 INFO:tasks.workunit.client.1.smithi138.stdout:0/62: creat d3/d10/f17 x:0 0 0 2024-05-17T21:52:42.016 INFO:tasks.workunit.client.1.smithi138.stdout:0/63: stat d3/d10 0 2024-05-17T21:52:42.017 INFO:tasks.workunit.client.1.smithi138.stdout:6/95: symlink d7/db/d12/l19 0 2024-05-17T21:52:42.021 INFO:tasks.workunit.client.1.smithi138.stdout:0/64: creat d3/f18 x:0 0 0 2024-05-17T21:52:42.025 INFO:tasks.workunit.client.1.smithi138.stdout:0/65: creat d3/d10/f19 x:0 0 0 2024-05-17T21:52:42.025 INFO:tasks.workunit.client.1.smithi138.stdout:6/96: link d7/db/c15 d7/db/d12/c1a 0 2024-05-17T21:52:42.027 INFO:tasks.workunit.client.1.smithi138.stdout:0/66: creat d3/d10/f1a x:0 0 0 2024-05-17T21:52:42.028 INFO:tasks.workunit.client.1.smithi138.stdout:0/67: read d3/fd [340749,66936] 0 2024-05-17T21:52:42.030 INFO:tasks.workunit.client.1.smithi138.stdout:0/68: unlink d3/d10/c12 0 2024-05-17T21:52:42.030 INFO:tasks.workunit.client.1.smithi138.stdout:0/69: chown d3/d10/f11 1451 1 2024-05-17T21:52:42.032 INFO:tasks.workunit.client.1.smithi138.stdout:0/70: mknod d3/c1b 0 2024-05-17T21:52:42.033 INFO:tasks.workunit.client.1.smithi138.stdout:0/71: write d3/d10/f16 [697346,99770] 0 2024-05-17T21:52:42.034 INFO:tasks.workunit.client.1.smithi138.stdout:0/72: mknod d3/d10/c1c 0 2024-05-17T21:52:42.034 INFO:tasks.workunit.client.1.smithi138.stdout:0/73: chown d3/fd 430 1 2024-05-17T21:52:42.036 INFO:tasks.workunit.client.1.smithi138.stdout:0/74: mknod d3/c1d 0 2024-05-17T21:52:42.038 INFO:tasks.workunit.client.1.smithi138.stdout:0/75: symlink d3/d10/l1e 0 2024-05-17T21:52:42.039 INFO:tasks.workunit.client.1.smithi138.stdout:0/76: rename d3/d10/l1e to d3/d10/l1f 0 2024-05-17T21:52:42.040 INFO:tasks.workunit.client.1.smithi138.stdout:0/77: creat d3/d10/f20 x:0 0 0 2024-05-17T21:52:42.041 INFO:tasks.workunit.client.1.smithi138.stdout:0/78: truncate d3/d10/f20 754127 0 2024-05-17T21:52:42.042 INFO:tasks.workunit.client.1.smithi138.stdout:0/79: chown d3/f18 26 1 2024-05-17T21:52:42.042 INFO:tasks.workunit.client.1.smithi138.stdout:0/80: dread - d3/d10/f14 zero size 2024-05-17T21:52:42.043 INFO:tasks.workunit.client.1.smithi138.stdout:0/81: truncate d3/d10/f15 85393 0 2024-05-17T21:52:42.049 INFO:tasks.workunit.client.1.smithi138.stdout:0/82: getdents d3 0 2024-05-17T21:52:42.051 INFO:tasks.workunit.client.1.smithi138.stdout:0/83: rename d3/d10/f16 to d3/f21 0 2024-05-17T21:52:42.052 INFO:tasks.workunit.client.1.smithi138.stdout:0/84: rmdir d3 39 2024-05-17T21:52:42.055 INFO:tasks.workunit.client.1.smithi138.stdout:0/85: creat d3/d10/f22 x:0 0 0 2024-05-17T21:52:42.056 INFO:tasks.workunit.client.1.smithi138.stdout:0/86: creat d3/d10/f23 x:0 0 0 2024-05-17T21:52:42.056 INFO:tasks.workunit.client.1.smithi138.stdout:0/87: dread - d3/d10/f23 zero size 2024-05-17T21:52:42.060 INFO:tasks.workunit.client.1.smithi138.stdout:0/88: rmdir d3/d10 39 2024-05-17T21:52:42.064 INFO:tasks.workunit.client.1.smithi138.stdout:0/89: rename d3/d10/f23 to d3/f24 0 2024-05-17T21:52:42.064 INFO:tasks.workunit.client.1.smithi138.stdout:0/90: fdatasync d3/d10/f15 0 2024-05-17T21:52:42.066 INFO:tasks.workunit.client.1.smithi138.stdout:0/91: rename d3/d10/f19 to d3/d10/f25 0 2024-05-17T21:52:42.067 INFO:tasks.workunit.client.1.smithi138.stdout:0/92: dread - d3/d10/f1a zero size 2024-05-17T21:52:42.067 INFO:tasks.workunit.client.1.smithi138.stdout:0/93: readlink l1 0 2024-05-17T21:52:42.068 INFO:tasks.workunit.client.1.smithi138.stdout:0/94: truncate d3/fd 1311162 0 2024-05-17T21:52:42.069 INFO:tasks.workunit.client.1.smithi138.stdout:0/95: read d3/f7 [2861459,103764] 0 2024-05-17T21:52:42.070 INFO:tasks.workunit.client.1.smithi138.stdout:0/96: write d3/f21 [1655785,86503] 0 2024-05-17T21:52:42.072 INFO:tasks.workunit.client.1.smithi138.stdout:0/97: mknod d3/d10/c26 0 2024-05-17T21:52:42.621 INFO:tasks.workunit.client.0.smithi107.stdout:6/10: sync 2024-05-17T21:52:42.622 INFO:tasks.workunit.client.0.smithi107.stdout:2/3: sync 2024-05-17T21:52:42.624 INFO:tasks.workunit.client.0.smithi107.stdout:2/4: symlink d0/l1 0 2024-05-17T21:52:42.625 INFO:tasks.workunit.client.0.smithi107.stdout:2/5: creat d0/f2 x:0 0 0 2024-05-17T21:52:42.626 INFO:tasks.workunit.client.0.smithi107.stdout:2/6: truncate d0/f2 724263 0 2024-05-17T21:52:42.724 INFO:tasks.workunit.client.0.smithi107.stdout:0/42: sync 2024-05-17T21:52:42.724 INFO:tasks.workunit.client.0.smithi107.stdout:5/50: sync 2024-05-17T21:52:42.724 INFO:tasks.workunit.client.0.smithi107.stdout:9/7: sync 2024-05-17T21:52:42.725 INFO:tasks.workunit.client.0.smithi107.stdout:4/36: sync 2024-05-17T21:52:42.725 INFO:tasks.workunit.client.0.smithi107.stdout:1/50: sync 2024-05-17T21:52:42.725 INFO:tasks.workunit.client.0.smithi107.stdout:3/37: sync 2024-05-17T21:52:42.725 INFO:tasks.workunit.client.0.smithi107.stdout:7/15: sync 2024-05-17T21:52:42.725 INFO:tasks.workunit.client.0.smithi107.stdout:8/67: sync 2024-05-17T21:52:42.725 INFO:tasks.workunit.client.0.smithi107.stdout:5/51: unlink l4 0 2024-05-17T21:52:42.726 INFO:tasks.workunit.client.0.smithi107.stdout:8/68: fsync dc/df/f11 0 2024-05-17T21:52:42.726 INFO:tasks.workunit.client.0.smithi107.stdout:7/16: write f4 [3952004,104528] 0 2024-05-17T21:52:42.726 INFO:tasks.workunit.client.0.smithi107.stdout:1/51: mknod d4/cb 0 2024-05-17T21:52:42.726 INFO:tasks.workunit.client.0.smithi107.stdout:5/52: write f3 [129105,15071] 0 2024-05-17T21:52:42.728 INFO:tasks.workunit.client.0.smithi107.stdout:9/8: mknod d0/c2 0 2024-05-17T21:52:42.728 INFO:tasks.workunit.client.0.smithi107.stdout:9/9: truncate - no filename 2024-05-17T21:52:42.729 INFO:tasks.workunit.client.0.smithi107.stdout:9/10: chown d0 258808707 1 2024-05-17T21:52:42.729 INFO:tasks.workunit.client.0.smithi107.stdout:9/11: dwrite - no filename 2024-05-17T21:52:42.730 INFO:tasks.workunit.client.0.smithi107.stdout:3/38: mkdir d6/d8/d10 0 2024-05-17T21:52:42.732 INFO:tasks.workunit.client.0.smithi107.stdout:0/43: creat d3/dc/f10 x:0 0 0 2024-05-17T21:52:42.734 INFO:tasks.workunit.client.0.smithi107.stdout:1/52: dwrite d4/fa [0,4194304] 0 2024-05-17T21:52:42.735 INFO:tasks.workunit.client.0.smithi107.stdout:4/37: rename d3/f4 to d3/f8 0 2024-05-17T21:52:42.739 INFO:tasks.workunit.client.0.smithi107.stdout:4/38: dread d3/f8 [0,4194304] 0 2024-05-17T21:52:42.739 INFO:tasks.workunit.client.0.smithi107.stdout:5/53: symlink d5/l11 0 2024-05-17T21:52:42.740 INFO:tasks.workunit.client.0.smithi107.stdout:5/54: fdatasync d5/f9 0 2024-05-17T21:52:42.742 INFO:tasks.workunit.client.0.smithi107.stdout:3/39: rename d6/ff to d6/f11 0 2024-05-17T21:52:42.743 INFO:tasks.workunit.client.0.smithi107.stdout:0/44: creat d3/f11 x:0 0 0 2024-05-17T21:52:42.744 INFO:tasks.workunit.client.0.smithi107.stdout:0/45: chown d3/l8 10730 1 2024-05-17T21:52:42.747 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:42 smithi107 ceph-mon[28434]: pgmap v29: 65 pgs: 65 active+clean; 283 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 3.0 MiB/s rd, 14 MiB/s wr, 496 op/s 2024-05-17T21:52:42.747 INFO:tasks.workunit.client.0.smithi107.stdout:7/17: symlink d5/l6 0 2024-05-17T21:52:42.747 INFO:tasks.workunit.client.0.smithi107.stdout:4/39: creat d3/f9 x:0 0 0 2024-05-17T21:52:42.750 INFO:tasks.workunit.client.0.smithi107.stdout:5/55: symlink d5/db/dc/l12 0 2024-05-17T21:52:42.751 INFO:tasks.workunit.client.0.smithi107.stdout:1/53: fdatasync f3 0 2024-05-17T21:52:42.753 INFO:tasks.workunit.client.0.smithi107.stdout:7/18: rmdir d5 39 2024-05-17T21:52:42.754 INFO:tasks.workunit.client.0.smithi107.stdout:7/19: fdatasync f4 0 2024-05-17T21:52:42.755 INFO:tasks.workunit.client.0.smithi107.stdout:9/12: link d0/c2 d0/c3 0 2024-05-17T21:52:42.756 INFO:tasks.workunit.client.0.smithi107.stdout:3/40: link f3 d6/d8/f12 0 2024-05-17T21:52:42.758 INFO:tasks.workunit.client.0.smithi107.stdout:1/54: creat d4/fc x:0 0 0 2024-05-17T21:52:42.759 INFO:tasks.workunit.client.0.smithi107.stdout:5/56: dwrite d5/fa [0,4194304] 0 2024-05-17T21:52:42.761 INFO:tasks.workunit.client.0.smithi107.stdout:9/13: creat d0/f4 x:0 0 0 2024-05-17T21:52:42.761 INFO:tasks.workunit.client.0.smithi107.stdout:1/55: write d4/f5 [4224744,76594] 0 2024-05-17T21:52:42.762 INFO:tasks.workunit.client.0.smithi107.stdout:3/41: creat d6/d8/f13 x:0 0 0 2024-05-17T21:52:42.765 INFO:tasks.workunit.client.0.smithi107.stdout:1/56: fdatasync f3 0 2024-05-17T21:52:42.766 INFO:tasks.workunit.client.0.smithi107.stdout:7/20: dwrite f4 [0,4194304] 0 2024-05-17T21:52:42.767 INFO:tasks.workunit.client.0.smithi107.stdout:3/42: creat d6/d8/f14 x:0 0 0 2024-05-17T21:52:42.767 INFO:tasks.workunit.client.0.smithi107.stdout:1/57: write d4/fc [107638,112928] 0 2024-05-17T21:52:42.768 INFO:tasks.workunit.client.0.smithi107.stdout:5/57: dread f3 [0,4194304] 0 2024-05-17T21:52:42.768 INFO:tasks.workunit.client.0.smithi107.stdout:9/14: unlink d0/c2 0 2024-05-17T21:52:42.768 INFO:tasks.workunit.client.0.smithi107.stdout:9/15: stat d0/l1 0 2024-05-17T21:52:42.770 INFO:tasks.workunit.client.0.smithi107.stdout:1/58: dread f3 [0,4194304] 0 2024-05-17T21:52:42.771 INFO:tasks.workunit.client.0.smithi107.stdout:3/43: mknod d6/d8/c15 0 2024-05-17T21:52:42.771 INFO:tasks.workunit.client.0.smithi107.stdout:6/11: sync 2024-05-17T21:52:42.772 INFO:tasks.workunit.client.0.smithi107.stdout:5/58: mkdir d5/db/d13 0 2024-05-17T21:52:42.773 INFO:tasks.workunit.client.0.smithi107.stdout:7/21: symlink d5/l7 0 2024-05-17T21:52:42.777 INFO:tasks.workunit.client.0.smithi107.stdout:6/12: mknod c4 0 2024-05-17T21:52:42.778 INFO:tasks.workunit.client.0.smithi107.stdout:2/7: getdents d0 0 2024-05-17T21:52:42.781 INFO:tasks.workunit.client.0.smithi107.stdout:9/16: dwrite d0/f4 [0,4194304] 0 2024-05-17T21:52:42.782 INFO:tasks.workunit.client.0.smithi107.stdout:2/8: write d0/f2 [315252,78462] 0 2024-05-17T21:52:42.784 INFO:tasks.workunit.client.0.smithi107.stdout:6/13: dread f1 [0,4194304] 0 2024-05-17T21:52:42.785 INFO:tasks.workunit.client.0.smithi107.stdout:1/59: dwrite d4/fc [0,4194304] 0 2024-05-17T21:52:42.787 INFO:tasks.workunit.client.0.smithi107.stdout:9/17: creat d0/f5 x:0 0 0 2024-05-17T21:52:42.789 INFO:tasks.workunit.client.0.smithi107.stdout:6/14: readlink l3 0 2024-05-17T21:52:42.789 INFO:tasks.workunit.client.0.smithi107.stdout:2/9: rmdir d0 39 2024-05-17T21:52:42.790 INFO:tasks.workunit.client.0.smithi107.stdout:5/59: dread d5/f7 [0,4194304] 0 2024-05-17T21:52:42.794 INFO:tasks.workunit.client.0.smithi107.stdout:6/15: write f1 [447719,56038] 0 2024-05-17T21:52:42.796 INFO:tasks.workunit.client.0.smithi107.stdout:6/16: truncate f1 1375721 0 2024-05-17T21:52:42.796 INFO:tasks.workunit.client.0.smithi107.stdout:6/17: read - f0 zero size 2024-05-17T21:52:42.797 INFO:tasks.workunit.client.0.smithi107.stdout:9/18: dread d0/f4 [0,4194304] 0 2024-05-17T21:52:42.797 INFO:tasks.workunit.client.0.smithi107.stdout:2/10: mknod d0/c3 0 2024-05-17T21:52:42.799 INFO:tasks.workunit.client.0.smithi107.stdout:2/11: read d0/f2 [135610,22280] 0 2024-05-17T21:52:42.801 INFO:tasks.workunit.client.0.smithi107.stdout:9/19: mknod d0/c6 0 2024-05-17T21:52:42.807 INFO:tasks.workunit.client.0.smithi107.stdout:2/12: dwrite d0/f2 [0,4194304] 0 2024-05-17T21:52:42.807 INFO:tasks.workunit.client.0.smithi107.stdout:2/13: stat d0/l1 0 2024-05-17T21:52:42.808 INFO:tasks.workunit.client.0.smithi107.stdout:2/14: chown d0/l1 0 1 2024-05-17T21:52:42.809 INFO:tasks.workunit.client.0.smithi107.stdout:2/15: write d0/f2 [1481571,34303] 0 2024-05-17T21:52:42.813 INFO:tasks.workunit.client.0.smithi107.stdout:7/22: fdatasync f4 0 2024-05-17T21:52:42.816 INFO:tasks.workunit.client.0.smithi107.stdout:7/23: unlink l2 0 2024-05-17T21:52:42.817 INFO:tasks.workunit.client.0.smithi107.stdout:7/24: creat d5/f8 x:0 0 0 2024-05-17T21:52:42.819 INFO:tasks.workunit.client.0.smithi107.stdout:7/25: symlink d5/l9 0 2024-05-17T21:52:42.822 INFO:tasks.workunit.client.0.smithi107.stdout:7/26: dread f4 [0,4194304] 0 2024-05-17T21:52:42.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:42 smithi138 ceph-mon[35948]: pgmap v29: 65 pgs: 65 active+clean; 283 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 3.0 MiB/s rd, 14 MiB/s wr, 496 op/s 2024-05-17T21:52:42.828 INFO:tasks.workunit.client.0.smithi107.stdout:7/27: dwrite d5/f8 [0,4194304] 0 2024-05-17T21:52:42.833 INFO:tasks.workunit.client.0.smithi107.stdout:0/46: rmdir d3 39 2024-05-17T21:52:42.835 INFO:tasks.workunit.client.0.smithi107.stdout:4/40: fsync d3/f9 0 2024-05-17T21:52:42.836 INFO:tasks.workunit.client.0.smithi107.stdout:5/60: getdents d5/db/dc 0 2024-05-17T21:52:42.836 INFO:tasks.workunit.client.0.smithi107.stdout:0/47: chown d3/dc/ld 0 1 2024-05-17T21:52:42.838 INFO:tasks.workunit.client.0.smithi107.stdout:3/44: fsync f2 0 2024-05-17T21:52:42.838 INFO:tasks.workunit.client.0.smithi107.stdout:1/60: rmdir d4 39 2024-05-17T21:52:42.838 INFO:tasks.workunit.client.0.smithi107.stdout:0/48: readlink d3/l5 0 2024-05-17T21:52:42.839 INFO:tasks.workunit.client.0.smithi107.stdout:7/28: dwrite f4 [0,4194304] 0 2024-05-17T21:52:42.841 INFO:tasks.workunit.client.0.smithi107.stdout:3/45: write d6/d8/f9 [795402,81733] 0 2024-05-17T21:52:42.841 INFO:tasks.workunit.client.0.smithi107.stdout:9/20: fsync d0/f4 0 2024-05-17T21:52:42.842 INFO:tasks.workunit.client.0.smithi107.stdout:8/69: dwrite dc/d10/f15 [0,4194304] 0 2024-05-17T21:52:42.843 INFO:tasks.workunit.client.0.smithi107.stdout:8/70: write dc/df/f16 [393825,112921] 0 2024-05-17T21:52:42.844 INFO:tasks.workunit.client.0.smithi107.stdout:1/61: stat d4 0 2024-05-17T21:52:42.844 INFO:tasks.workunit.client.0.smithi107.stdout:8/71: readlink dc/le 0 2024-05-17T21:52:42.844 INFO:tasks.workunit.client.0.smithi107.stdout:1/62: stat d4/fc 0 2024-05-17T21:52:42.845 INFO:tasks.workunit.client.0.smithi107.stdout:1/63: chown d4/l7 0 1 2024-05-17T21:52:42.846 INFO:tasks.workunit.client.0.smithi107.stdout:7/29: mknod d5/ca 0 2024-05-17T21:52:42.849 INFO:tasks.workunit.client.0.smithi107.stdout:4/41: link f2 d3/fa 0 2024-05-17T21:52:42.849 INFO:tasks.workunit.client.0.smithi107.stdout:4/42: chown d3/f8 4 1 2024-05-17T21:52:42.852 INFO:tasks.workunit.client.0.smithi107.stdout:9/21: creat d0/f7 x:0 0 0 2024-05-17T21:52:42.854 INFO:tasks.workunit.client.0.smithi107.stdout:5/61: mknod d5/db/d13/c14 0 2024-05-17T21:52:42.858 INFO:tasks.workunit.client.0.smithi107.stdout:1/64: creat d4/fd x:0 0 0 2024-05-17T21:52:42.859 INFO:tasks.workunit.client.0.smithi107.stdout:6/18: getdents . 0 2024-05-17T21:52:42.859 INFO:tasks.workunit.client.0.smithi107.stdout:1/65: stat d4/fd 0 2024-05-17T21:52:42.859 INFO:tasks.workunit.client.0.smithi107.stdout:6/19: dread - f0 zero size 2024-05-17T21:52:42.859 INFO:tasks.workunit.client.0.smithi107.stdout:1/66: read d4/f5 [2550158,86175] 0 2024-05-17T21:52:42.860 INFO:tasks.workunit.client.0.smithi107.stdout:6/20: write f0 [60288,97108] 0 2024-05-17T21:52:42.861 INFO:tasks.workunit.client.0.smithi107.stdout:4/43: write d3/f8 [6187779,59594] 0 2024-05-17T21:52:42.865 INFO:tasks.workunit.client.0.smithi107.stdout:8/72: mknod dc/c1d 0 2024-05-17T21:52:42.867 INFO:tasks.workunit.client.0.smithi107.stdout:1/67: chown d4/c6 150709 1 2024-05-17T21:52:42.868 INFO:tasks.workunit.client.0.smithi107.stdout:1/68: write d4/fd [560831,18802] 0 2024-05-17T21:52:42.869 INFO:tasks.workunit.client.0.smithi107.stdout:6/21: rename f0 to f5 0 2024-05-17T21:52:42.869 INFO:tasks.workunit.client.0.smithi107.stdout:2/16: truncate d0/f2 3455627 0 2024-05-17T21:52:42.870 INFO:tasks.workunit.client.0.smithi107.stdout:4/44: mkdir d3/db 0 2024-05-17T21:52:42.873 INFO:tasks.workunit.client.0.smithi107.stdout:8/73: unlink dc/f13 0 2024-05-17T21:52:42.874 INFO:tasks.workunit.client.0.smithi107.stdout:7/30: link d5/l7 d5/lb 0 2024-05-17T21:52:42.876 INFO:tasks.workunit.client.0.smithi107.stdout:1/69: mkdir d4/de 0 2024-05-17T21:52:42.877 INFO:tasks.workunit.client.0.smithi107.stdout:6/22: mknod c6 0 2024-05-17T21:52:42.878 INFO:tasks.workunit.client.0.smithi107.stdout:6/23: write f1 [2124712,3377] 0 2024-05-17T21:52:42.880 INFO:tasks.workunit.client.0.smithi107.stdout:2/17: mkdir d0/d4 0 2024-05-17T21:52:42.883 INFO:tasks.workunit.client.0.smithi107.stdout:1/70: dwrite d4/fc [0,4194304] 0 2024-05-17T21:52:42.885 INFO:tasks.workunit.client.0.smithi107.stdout:8/74: rmdir dc/df 39 2024-05-17T21:52:42.886 INFO:tasks.workunit.client.0.smithi107.stdout:1/71: write d4/f8 [597224,6257] 0 2024-05-17T21:52:42.890 INFO:tasks.workunit.client.0.smithi107.stdout:1/72: chown d4/cb 504359 1 2024-05-17T21:52:42.893 INFO:tasks.workunit.client.0.smithi107.stdout:6/24: fdatasync f5 0 2024-05-17T21:52:42.895 INFO:tasks.workunit.client.0.smithi107.stdout:0/49: dwrite d3/fb [0,4194304] 0 2024-05-17T21:52:42.895 INFO:tasks.workunit.client.0.smithi107.stdout:3/46: dwrite d6/fc [0,4194304] 0 2024-05-17T21:52:42.895 INFO:tasks.workunit.client.0.smithi107.stdout:6/25: write f1 [3015832,79237] 0 2024-05-17T21:52:42.897 INFO:tasks.workunit.client.0.smithi107.stdout:6/26: stat c2 0 2024-05-17T21:52:42.897 INFO:tasks.workunit.client.0.smithi107.stdout:5/62: getdents d5/db/d13 0 2024-05-17T21:52:42.898 INFO:tasks.workunit.client.0.smithi107.stdout:0/50: dread - d3/dc/f10 zero size 2024-05-17T21:52:42.898 INFO:tasks.workunit.client.0.smithi107.stdout:8/75: mknod dc/d10/c1e 0 2024-05-17T21:52:42.900 INFO:tasks.workunit.client.0.smithi107.stdout:9/22: dread - d0/f7 zero size 2024-05-17T21:52:42.905 INFO:tasks.workunit.client.0.smithi107.stdout:2/18: mknod d0/d4/c5 0 2024-05-17T21:52:42.906 INFO:tasks.workunit.client.0.smithi107.stdout:4/45: truncate d3/f8 180734 0 2024-05-17T21:52:42.909 INFO:tasks.workunit.client.0.smithi107.stdout:6/27: mknod c7 0 2024-05-17T21:52:42.909 INFO:tasks.workunit.client.0.smithi107.stdout:5/63: write d5/f7 [3770358,105474] 0 2024-05-17T21:52:42.910 INFO:tasks.workunit.client.0.smithi107.stdout:0/51: creat d3/dc/f12 x:0 0 0 2024-05-17T21:52:42.910 INFO:tasks.workunit.client.0.smithi107.stdout:8/76: symlink dc/d10/l1f 0 2024-05-17T21:52:42.911 INFO:tasks.workunit.client.0.smithi107.stdout:5/64: chown f3 0 1 2024-05-17T21:52:42.911 INFO:tasks.workunit.client.0.smithi107.stdout:5/65: readlink d5/db/dc/lf 0 2024-05-17T21:52:42.914 INFO:tasks.workunit.client.0.smithi107.stdout:2/19: creat d0/f6 x:0 0 0 2024-05-17T21:52:42.916 INFO:tasks.workunit.client.0.smithi107.stdout:6/28: rename c2 to c8 0 2024-05-17T21:52:42.916 INFO:tasks.workunit.client.0.smithi107.stdout:6/29: write f5 [713281,88920] 0 2024-05-17T21:52:42.917 INFO:tasks.workunit.client.0.smithi107.stdout:0/52: mknod d3/c13 0 2024-05-17T21:52:42.918 INFO:tasks.workunit.client.0.smithi107.stdout:0/53: dread - d3/ff zero size 2024-05-17T21:52:42.919 INFO:tasks.workunit.client.0.smithi107.stdout:5/66: mknod d5/db/c15 0 2024-05-17T21:52:42.920 INFO:tasks.workunit.client.0.smithi107.stdout:7/31: link d5/l7 d5/lc 0 2024-05-17T21:52:42.921 INFO:tasks.workunit.client.0.smithi107.stdout:6/30: read f5 [200428,92171] 0 2024-05-17T21:52:42.921 INFO:tasks.workunit.client.0.smithi107.stdout:9/23: write d0/f4 [5099450,67953] 0 2024-05-17T21:52:42.925 INFO:tasks.workunit.client.0.smithi107.stdout:4/46: creat d3/db/fc x:0 0 0 2024-05-17T21:52:42.926 INFO:tasks.workunit.client.0.smithi107.stdout:9/24: dread - d0/f7 zero size 2024-05-17T21:52:42.927 INFO:tasks.workunit.client.0.smithi107.stdout:3/47: dwrite d6/fc [4194304,4194304] 0 2024-05-17T21:52:42.927 INFO:tasks.workunit.client.0.smithi107.stdout:8/77: creat dc/df/f20 x:0 0 0 2024-05-17T21:52:42.928 INFO:tasks.workunit.client.0.smithi107.stdout:0/54: creat d3/dc/f14 x:0 0 0 2024-05-17T21:52:42.928 INFO:tasks.workunit.client.0.smithi107.stdout:8/78: fdatasync dc/df/f20 0 2024-05-17T21:52:42.928 INFO:tasks.workunit.client.0.smithi107.stdout:3/48: truncate f5 1065821 0 2024-05-17T21:52:42.928 INFO:tasks.workunit.client.0.smithi107.stdout:9/25: chown d0/f4 374954 1 2024-05-17T21:52:42.929 INFO:tasks.workunit.client.0.smithi107.stdout:0/55: fsync d3/dc/f10 0 2024-05-17T21:52:42.930 INFO:tasks.workunit.client.0.smithi107.stdout:8/79: fsync dc/df/f16 0 2024-05-17T21:52:42.930 INFO:tasks.workunit.client.0.smithi107.stdout:7/32: creat d5/fd x:0 0 0 2024-05-17T21:52:42.931 INFO:tasks.workunit.client.0.smithi107.stdout:6/31: mkdir d9 0 2024-05-17T21:52:42.932 INFO:tasks.workunit.client.0.smithi107.stdout:7/33: write d5/fd [144693,97503] 0 2024-05-17T21:52:42.937 INFO:tasks.workunit.client.0.smithi107.stdout:7/34: dread d5/fd [0,4194304] 0 2024-05-17T21:52:42.938 INFO:tasks.workunit.client.0.smithi107.stdout:0/56: rename d3/dc/f10 to d3/f15 0 2024-05-17T21:52:42.939 INFO:tasks.workunit.client.0.smithi107.stdout:0/57: readlink d3/la 0 2024-05-17T21:52:42.940 INFO:tasks.workunit.client.0.smithi107.stdout:0/58: chown d3/dc/ld 1 1 2024-05-17T21:52:42.943 INFO:tasks.workunit.client.0.smithi107.stdout:8/80: mkdir dc/d21 0 2024-05-17T21:52:42.943 INFO:tasks.workunit.client.0.smithi107.stdout:6/32: truncate f5 354791 0 2024-05-17T21:52:42.947 INFO:tasks.workunit.client.0.smithi107.stdout:7/35: rename d5/f8 to d5/fe 0 2024-05-17T21:52:42.951 INFO:tasks.workunit.client.0.smithi107.stdout:8/81: creat dc/df/f22 x:0 0 0 2024-05-17T21:52:42.951 INFO:tasks.workunit.client.0.smithi107.stdout:2/20: dwrite d0/f2 [0,4194304] 0 2024-05-17T21:52:42.952 INFO:tasks.workunit.client.0.smithi107.stdout:3/49: getdents d6/d8 0 2024-05-17T21:52:42.957 INFO:tasks.workunit.client.0.smithi107.stdout:3/50: fdatasync d6/fc 0 2024-05-17T21:52:42.957 INFO:tasks.workunit.client.0.smithi107.stdout:6/33: creat d9/fa x:0 0 0 2024-05-17T21:52:42.957 INFO:tasks.workunit.client.0.smithi107.stdout:8/82: chown dc/df/f11 10442535 1 2024-05-17T21:52:42.962 INFO:tasks.workunit.client.0.smithi107.stdout:9/26: truncate d0/f4 3185218 0 2024-05-17T21:52:42.963 INFO:tasks.workunit.client.0.smithi107.stdout:0/59: dwrite d3/f15 [0,4194304] 0 2024-05-17T21:52:42.963 INFO:tasks.workunit.client.0.smithi107.stdout:9/27: chown d0 77441012 1 2024-05-17T21:52:42.963 INFO:tasks.workunit.client.0.smithi107.stdout:3/51: write d6/d8/f14 [368046,9787] 0 2024-05-17T21:52:42.965 INFO:tasks.workunit.client.0.smithi107.stdout:2/21: creat d0/f7 x:0 0 0 2024-05-17T21:52:42.966 INFO:tasks.workunit.client.0.smithi107.stdout:0/60: readlink d3/la 0 2024-05-17T21:52:42.970 INFO:tasks.workunit.client.0.smithi107.stdout:8/83: unlink dc/df/f16 0 2024-05-17T21:52:42.970 INFO:tasks.workunit.client.0.smithi107.stdout:4/47: dread d3/f8 [0,4194304] 0 2024-05-17T21:52:42.971 INFO:tasks.workunit.client.0.smithi107.stdout:7/36: dwrite d5/fe [4194304,4194304] 0 2024-05-17T21:52:42.972 INFO:tasks.workunit.client.0.smithi107.stdout:9/28: readlink d0/l1 0 2024-05-17T21:52:42.973 INFO:tasks.workunit.client.0.smithi107.stdout:9/29: chown d0/f7 809 1 2024-05-17T21:52:42.974 INFO:tasks.workunit.client.0.smithi107.stdout:8/84: mkdir dc/df/d23 0 2024-05-17T21:52:42.975 INFO:tasks.workunit.client.0.smithi107.stdout:4/48: mkdir d3/db/dd 0 2024-05-17T21:52:42.975 INFO:tasks.workunit.client.0.smithi107.stdout:4/49: dread - d3/db/fc zero size 2024-05-17T21:52:42.977 INFO:tasks.workunit.client.0.smithi107.stdout:7/37: rmdir d5 39 2024-05-17T21:52:42.981 INFO:tasks.workunit.client.0.smithi107.stdout:9/30: read - d0/f5 zero size 2024-05-17T21:52:42.983 INFO:tasks.workunit.client.0.smithi107.stdout:3/52: link l0 d6/d8/d10/l16 0 2024-05-17T21:52:42.987 INFO:tasks.workunit.client.0.smithi107.stdout:0/61: rmdir d3/dc/de 0 2024-05-17T21:52:42.989 INFO:tasks.workunit.client.0.smithi107.stdout:0/62: read d3/f9 [3196233,59399] 0 2024-05-17T21:52:42.990 INFO:tasks.workunit.client.0.smithi107.stdout:3/53: creat d6/d8/f17 x:0 0 0 2024-05-17T21:52:42.992 INFO:tasks.workunit.client.0.smithi107.stdout:2/22: getdents d0 0 2024-05-17T21:52:42.993 INFO:tasks.workunit.client.0.smithi107.stdout:7/38: dwrite d5/fe [4194304,4194304] 0 2024-05-17T21:52:42.994 INFO:tasks.workunit.client.0.smithi107.stdout:0/63: creat d3/dc/f16 x:0 0 0 2024-05-17T21:52:42.994 INFO:tasks.workunit.client.0.smithi107.stdout:9/31: read d0/f4 [2657531,66213] 0 2024-05-17T21:52:42.995 INFO:tasks.workunit.client.0.smithi107.stdout:6/34: truncate f5 1030613 0 2024-05-17T21:52:42.997 INFO:tasks.workunit.client.0.smithi107.stdout:2/23: creat d0/f8 x:0 0 0 2024-05-17T21:52:42.997 INFO:tasks.workunit.client.0.smithi107.stdout:7/39: dread d5/fd [0,4194304] 0 2024-05-17T21:52:42.998 INFO:tasks.workunit.client.0.smithi107.stdout:0/64: creat d3/f17 x:0 0 0 2024-05-17T21:52:42.999 INFO:tasks.workunit.client.0.smithi107.stdout:4/50: dwrite f2 [0,4194304] 0 2024-05-17T21:52:42.999 INFO:tasks.workunit.client.0.smithi107.stdout:6/35: unlink c4 0 2024-05-17T21:52:43.002 INFO:tasks.workunit.client.0.smithi107.stdout:3/54: write d6/d8/f12 [5009426,109280] 0 2024-05-17T21:52:43.004 INFO:tasks.workunit.client.0.smithi107.stdout:7/40: mkdir d5/df 0 2024-05-17T21:52:43.005 INFO:tasks.workunit.client.0.smithi107.stdout:2/24: unlink d0/f8 0 2024-05-17T21:52:43.006 INFO:tasks.workunit.client.0.smithi107.stdout:2/25: dread - d0/f7 zero size 2024-05-17T21:52:43.006 INFO:tasks.workunit.client.0.smithi107.stdout:2/26: dread - d0/f7 zero size 2024-05-17T21:52:43.007 INFO:tasks.workunit.client.0.smithi107.stdout:2/27: readlink d0/l1 0 2024-05-17T21:52:43.008 INFO:tasks.workunit.client.0.smithi107.stdout:9/32: symlink d0/l8 0 2024-05-17T21:52:43.008 INFO:tasks.workunit.client.0.smithi107.stdout:9/33: dread - d0/f7 zero size 2024-05-17T21:52:43.009 INFO:tasks.workunit.client.0.smithi107.stdout:3/55: mknod d6/c18 0 2024-05-17T21:52:43.011 INFO:tasks.workunit.client.0.smithi107.stdout:6/36: dread f1 [0,4194304] 0 2024-05-17T21:52:43.012 INFO:tasks.workunit.client.0.smithi107.stdout:6/37: chown l3 141 1 2024-05-17T21:52:43.012 INFO:tasks.workunit.client.0.smithi107.stdout:4/51: link d3/f9 d3/db/fe 0 2024-05-17T21:52:43.013 INFO:tasks.workunit.client.0.smithi107.stdout:2/28: mkdir d0/d9 0 2024-05-17T21:52:43.014 INFO:tasks.workunit.client.0.smithi107.stdout:7/41: creat d5/df/f10 x:0 0 0 2024-05-17T21:52:43.017 INFO:tasks.workunit.client.0.smithi107.stdout:3/56: chown d6/d8/d10/l16 741388269 1 2024-05-17T21:52:43.018 INFO:tasks.workunit.client.0.smithi107.stdout:9/34: rename d0/l1 to d0/l9 0 2024-05-17T21:52:43.019 INFO:tasks.workunit.client.0.smithi107.stdout:4/52: rename d3/l7 to d3/lf 0 2024-05-17T21:52:43.019 INFO:tasks.workunit.client.0.smithi107.stdout:7/42: write d5/df/f10 [113510,93513] 0 2024-05-17T21:52:43.021 INFO:tasks.workunit.client.0.smithi107.stdout:9/35: dread - d0/f7 zero size 2024-05-17T21:52:43.021 INFO:tasks.workunit.client.0.smithi107.stdout:3/57: mknod d6/c19 0 2024-05-17T21:52:43.024 INFO:tasks.workunit.client.0.smithi107.stdout:4/53: write d3/db/fc [177439,98100] 0 2024-05-17T21:52:43.025 INFO:tasks.workunit.client.0.smithi107.stdout:7/43: truncate d5/fd 1052220 0 2024-05-17T21:52:43.026 INFO:tasks.workunit.client.0.smithi107.stdout:6/38: dwrite d9/fa [0,4194304] 0 2024-05-17T21:52:43.026 INFO:tasks.workunit.client.0.smithi107.stdout:0/65: truncate d3/f15 1330833 0 2024-05-17T21:52:43.027 INFO:tasks.workunit.client.0.smithi107.stdout:3/58: write d6/d8/fb [516986,43234] 0 2024-05-17T21:52:43.028 INFO:tasks.workunit.client.0.smithi107.stdout:2/29: dwrite d0/f2 [4194304,4194304] 0 2024-05-17T21:52:43.030 INFO:tasks.workunit.client.0.smithi107.stdout:9/36: readlink d0/l9 0 2024-05-17T21:52:43.030 INFO:tasks.workunit.client.0.smithi107.stdout:7/44: chown d5/df/f10 5829399 1 2024-05-17T21:52:43.032 INFO:tasks.workunit.client.0.smithi107.stdout:0/66: creat d3/f18 x:0 0 0 2024-05-17T21:52:43.035 INFO:tasks.workunit.client.0.smithi107.stdout:2/30: creat d0/fa x:0 0 0 2024-05-17T21:52:43.036 INFO:tasks.workunit.client.0.smithi107.stdout:2/31: rename d0/d4 to d0/d4/db 22 2024-05-17T21:52:43.038 INFO:tasks.workunit.client.0.smithi107.stdout:3/59: dwrite d6/d8/fb [0,4194304] 0 2024-05-17T21:52:43.039 INFO:tasks.workunit.client.0.smithi107.stdout:2/32: write d0/f6 [46807,25231] 0 2024-05-17T21:52:43.039 INFO:tasks.workunit.client.0.smithi107.stdout:7/45: mkdir d5/df/d11 0 2024-05-17T21:52:43.040 INFO:tasks.workunit.client.0.smithi107.stdout:6/39: fdatasync f5 0 2024-05-17T21:52:43.041 INFO:tasks.workunit.client.0.smithi107.stdout:2/33: truncate d0/f6 728052 0 2024-05-17T21:52:43.041 INFO:tasks.workunit.client.0.smithi107.stdout:7/46: truncate f4 4564669 0 2024-05-17T21:52:43.041 INFO:tasks.workunit.client.0.smithi107.stdout:0/67: creat d3/f19 x:0 0 0 2024-05-17T21:52:43.042 INFO:tasks.workunit.client.0.smithi107.stdout:4/54: link d3/db/fc d3/db/dd/f10 0 2024-05-17T21:52:43.043 INFO:tasks.workunit.client.0.smithi107.stdout:4/55: truncate d3/f9 679594 0 2024-05-17T21:52:43.044 INFO:tasks.workunit.client.0.smithi107.stdout:3/60: mkdir d6/d1a 0 2024-05-17T21:52:43.045 INFO:tasks.workunit.client.0.smithi107.stdout:6/40: rmdir d9 39 2024-05-17T21:52:43.049 INFO:tasks.workunit.client.0.smithi107.stdout:7/47: rename l1 to d5/df/d11/l12 0 2024-05-17T21:52:43.050 INFO:tasks.workunit.client.0.smithi107.stdout:0/68: mknod d3/dc/c1a 0 2024-05-17T21:52:43.051 INFO:tasks.workunit.client.0.smithi107.stdout:9/37: rename d0/f5 to d0/fa 0 2024-05-17T21:52:43.051 INFO:tasks.workunit.client.0.smithi107.stdout:9/38: dread - d0/f7 zero size 2024-05-17T21:52:43.052 INFO:tasks.workunit.client.0.smithi107.stdout:4/56: creat d3/db/f11 x:0 0 0 2024-05-17T21:52:43.054 INFO:tasks.workunit.client.0.smithi107.stdout:2/34: symlink d0/d9/lc 0 2024-05-17T21:52:43.055 INFO:tasks.workunit.client.0.smithi107.stdout:7/48: mknod d5/df/d11/c13 0 2024-05-17T21:52:43.057 INFO:tasks.workunit.client.0.smithi107.stdout:0/69: symlink d3/dc/l1b 0 2024-05-17T21:52:43.058 INFO:tasks.workunit.client.0.smithi107.stdout:0/70: truncate d3/f19 475981 0 2024-05-17T21:52:43.059 INFO:tasks.workunit.client.0.smithi107.stdout:0/71: write d3/dc/f16 [289815,83874] 0 2024-05-17T21:52:43.059 INFO:tasks.workunit.client.0.smithi107.stdout:3/61: truncate d6/fc 6698607 0 2024-05-17T21:52:43.061 INFO:tasks.workunit.client.0.smithi107.stdout:4/57: fsync d3/db/dd/f10 0 2024-05-17T21:52:43.062 INFO:tasks.workunit.client.0.smithi107.stdout:4/58: fdatasync d3/fa 0 2024-05-17T21:52:43.064 INFO:tasks.workunit.client.0.smithi107.stdout:2/35: rmdir d0/d4 39 2024-05-17T21:52:43.064 INFO:tasks.workunit.client.0.smithi107.stdout:2/36: dread - d0/f7 zero size 2024-05-17T21:52:43.065 INFO:tasks.workunit.client.0.smithi107.stdout:2/37: fsync d0/fa 0 2024-05-17T21:52:43.065 INFO:tasks.workunit.client.0.smithi107.stdout:2/38: fdatasync d0/f7 0 2024-05-17T21:52:43.065 INFO:tasks.workunit.client.0.smithi107.stdout:2/39: dread - d0/f7 zero size 2024-05-17T21:52:43.066 INFO:tasks.workunit.client.0.smithi107.stdout:7/49: symlink d5/df/l14 0 2024-05-17T21:52:43.067 INFO:tasks.workunit.client.0.smithi107.stdout:3/62: creat d6/f1b x:0 0 0 2024-05-17T21:52:43.068 INFO:tasks.workunit.client.0.smithi107.stdout:3/63: write d6/d8/f17 [448283,95331] 0 2024-05-17T21:52:43.068 INFO:tasks.workunit.client.0.smithi107.stdout:3/64: stat d6/d8/f12 0 2024-05-17T21:52:43.071 INFO:tasks.workunit.client.0.smithi107.stdout:6/41: rename c8 to d9/cb 0 2024-05-17T21:52:43.074 INFO:tasks.workunit.client.0.smithi107.stdout:4/59: creat d3/db/dd/f12 x:0 0 0 2024-05-17T21:52:43.075 INFO:tasks.workunit.client.0.smithi107.stdout:0/72: rename d3/l8 to d3/dc/l1c 0 2024-05-17T21:52:43.076 INFO:tasks.workunit.client.0.smithi107.stdout:2/40: creat d0/d4/fd x:0 0 0 2024-05-17T21:52:43.077 INFO:tasks.workunit.client.0.smithi107.stdout:0/73: chown d3/l4 0 1 2024-05-17T21:52:43.077 INFO:tasks.workunit.client.0.smithi107.stdout:9/39: rename d0/c6 to d0/cb 0 2024-05-17T21:52:43.078 INFO:tasks.workunit.client.0.smithi107.stdout:3/65: rename d6/d8/f13 to d6/f1c 0 2024-05-17T21:52:43.079 INFO:tasks.workunit.client.0.smithi107.stdout:9/40: fsync d0/fa 0 2024-05-17T21:52:43.081 INFO:tasks.workunit.client.0.smithi107.stdout:7/50: dwrite d5/fd [0,4194304] 0 2024-05-17T21:52:43.081 INFO:tasks.workunit.client.0.smithi107.stdout:3/66: symlink d6/l1d 0 2024-05-17T21:52:43.082 INFO:tasks.workunit.client.0.smithi107.stdout:3/67: fsync f3 0 2024-05-17T21:52:43.083 INFO:tasks.workunit.client.0.smithi107.stdout:9/41: write d0/fa [848658,5168] 0 2024-05-17T21:52:43.083 INFO:tasks.workunit.client.0.smithi107.stdout:0/74: link d3/dc/f14 d3/f1d 0 2024-05-17T21:52:43.084 INFO:tasks.workunit.client.0.smithi107.stdout:4/60: rename d3/lf to d3/d6/l13 0 2024-05-17T21:52:43.085 INFO:tasks.workunit.client.0.smithi107.stdout:9/42: chown d0/fa 0 1 2024-05-17T21:52:43.086 INFO:tasks.workunit.client.0.smithi107.stdout:0/75: write d3/dc/f12 [851163,101597] 0 2024-05-17T21:52:43.087 INFO:tasks.workunit.client.0.smithi107.stdout:7/51: creat d5/df/d11/f15 x:0 0 0 2024-05-17T21:52:43.088 INFO:tasks.workunit.client.0.smithi107.stdout:2/41: dwrite d0/f7 [0,4194304] 0 2024-05-17T21:52:43.089 INFO:tasks.workunit.client.0.smithi107.stdout:4/61: dread d3/db/fe [0,4194304] 0 2024-05-17T21:52:43.093 INFO:tasks.workunit.client.0.smithi107.stdout:9/43: rmdir d0 39 2024-05-17T21:52:43.094 INFO:tasks.workunit.client.0.smithi107.stdout:0/76: dread d3/dc/f12 [0,4194304] 0 2024-05-17T21:52:43.096 INFO:tasks.workunit.client.0.smithi107.stdout:7/52: mkdir d5/d16 0 2024-05-17T21:52:43.099 INFO:tasks.workunit.client.0.smithi107.stdout:2/42: rmdir d0/d4 39 2024-05-17T21:52:43.103 INFO:tasks.workunit.client.0.smithi107.stdout:7/53: mknod d5/c17 0 2024-05-17T21:52:43.105 INFO:tasks.workunit.client.0.smithi107.stdout:0/77: dread d3/f9 [0,4194304] 0 2024-05-17T21:52:43.107 INFO:tasks.workunit.client.0.smithi107.stdout:7/54: creat d5/f18 x:0 0 0 2024-05-17T21:52:43.107 INFO:tasks.workunit.client.0.smithi107.stdout:3/68: write d6/f11 [5510010,41845] 0 2024-05-17T21:52:43.107 INFO:tasks.workunit.client.0.smithi107.stdout:2/43: link d0/d9/lc d0/d9/le 0 2024-05-17T21:52:43.107 INFO:tasks.workunit.client.0.smithi107.stdout:4/62: write d3/db/fc [314455,87096] 0 2024-05-17T21:52:43.108 INFO:tasks.workunit.client.0.smithi107.stdout:7/55: write d5/f18 [748796,124416] 0 2024-05-17T21:52:43.109 INFO:tasks.workunit.client.0.smithi107.stdout:0/78: creat d3/f1e x:0 0 0 2024-05-17T21:52:43.110 INFO:tasks.workunit.client.0.smithi107.stdout:9/44: creat d0/fc x:0 0 0 2024-05-17T21:52:43.111 INFO:tasks.workunit.client.0.smithi107.stdout:2/44: dread - d0/d4/fd zero size 2024-05-17T21:52:43.112 INFO:tasks.workunit.client.0.smithi107.stdout:7/56: mknod d5/df/c19 0 2024-05-17T21:52:43.112 INFO:tasks.workunit.client.0.smithi107.stdout:9/45: write d0/fc [289291,52676] 0 2024-05-17T21:52:43.112 INFO:tasks.workunit.client.0.smithi107.stdout:4/63: dread d3/db/dd/f10 [0,4194304] 0 2024-05-17T21:52:43.113 INFO:tasks.workunit.client.0.smithi107.stdout:4/64: fdatasync d3/db/f11 0 2024-05-17T21:52:43.114 INFO:tasks.workunit.client.0.smithi107.stdout:7/57: readlink d5/lb 0 2024-05-17T21:52:43.114 INFO:tasks.workunit.client.0.smithi107.stdout:2/45: mkdir d0/d4/df 0 2024-05-17T21:52:43.114 INFO:tasks.workunit.client.0.smithi107.stdout:0/79: link d3/f1d d3/dc/f1f 0 2024-05-17T21:52:43.116 INFO:tasks.workunit.client.0.smithi107.stdout:0/80: chown d3/c6 4967 1 2024-05-17T21:52:43.117 INFO:tasks.workunit.client.0.smithi107.stdout:7/58: creat d5/df/f1a x:0 0 0 2024-05-17T21:52:43.119 INFO:tasks.workunit.client.0.smithi107.stdout:0/81: read - d3/f11 zero size 2024-05-17T21:52:43.123 INFO:tasks.workunit.client.0.smithi107.stdout:9/46: dwrite d0/fa [0,4194304] 0 2024-05-17T21:52:43.125 INFO:tasks.workunit.client.0.smithi107.stdout:0/82: chown d3/dc/f1f 1581 1 2024-05-17T21:52:43.126 INFO:tasks.workunit.client.0.smithi107.stdout:7/59: rename d5/l7 to d5/df/l1b 0 2024-05-17T21:52:43.126 INFO:tasks.workunit.client.0.smithi107.stdout:2/46: dread d0/f7 [0,4194304] 0 2024-05-17T21:52:43.127 INFO:tasks.workunit.client.0.smithi107.stdout:4/65: dwrite d3/db/dd/f12 [0,4194304] 0 2024-05-17T21:52:43.129 INFO:tasks.workunit.client.0.smithi107.stdout:9/47: mkdir d0/dd 0 2024-05-17T21:52:43.129 INFO:tasks.workunit.client.0.smithi107.stdout:7/60: write d5/df/d11/f15 [291936,120275] 0 2024-05-17T21:52:43.130 INFO:tasks.workunit.client.0.smithi107.stdout:4/66: creat d3/db/f14 x:0 0 0 2024-05-17T21:52:43.132 INFO:tasks.workunit.client.0.smithi107.stdout:2/47: rename d0/f7 to d0/d4/f10 0 2024-05-17T21:52:43.134 INFO:tasks.workunit.client.0.smithi107.stdout:7/61: mkdir d5/df/d11/d1c 0 2024-05-17T21:52:43.135 INFO:tasks.workunit.client.0.smithi107.stdout:2/48: write d0/f2 [2068232,33286] 0 2024-05-17T21:52:43.136 INFO:tasks.workunit.client.0.smithi107.stdout:4/67: dread d3/db/dd/f10 [0,4194304] 0 2024-05-17T21:52:43.139 INFO:tasks.workunit.client.0.smithi107.stdout:2/49: truncate d0/d4/fd 149330 0 2024-05-17T21:52:43.139 INFO:tasks.workunit.client.0.smithi107.stdout:7/62: mkdir d5/df/d11/d1d 0 2024-05-17T21:52:43.139 INFO:tasks.workunit.client.0.smithi107.stdout:0/83: dwrite d3/dc/f12 [0,4194304] 0 2024-05-17T21:52:43.140 INFO:tasks.workunit.client.0.smithi107.stdout:7/63: chown c0 404 1 2024-05-17T21:52:43.140 INFO:tasks.workunit.client.0.smithi107.stdout:7/64: stat f4 0 2024-05-17T21:52:43.141 INFO:tasks.workunit.client.0.smithi107.stdout:4/68: link l1 d3/l15 0 2024-05-17T21:52:43.145 INFO:tasks.workunit.client.0.smithi107.stdout:7/65: link d5/df/d11/f15 d5/d16/f1e 0 2024-05-17T21:52:43.145 INFO:tasks.workunit.client.0.smithi107.stdout:4/69: dread f2 [0,4194304] 0 2024-05-17T21:52:43.146 INFO:tasks.workunit.client.0.smithi107.stdout:4/70: read d3/db/dd/f10 [167797,108323] 0 2024-05-17T21:52:43.149 INFO:tasks.workunit.client.0.smithi107.stdout:9/48: write d0/f4 [3949568,1951] 0 2024-05-17T21:52:43.150 INFO:tasks.workunit.client.0.smithi107.stdout:9/49: truncate d0/fc 872635 0 2024-05-17T21:52:43.151 INFO:tasks.workunit.client.0.smithi107.stdout:7/66: link d5/c17 d5/df/c1f 0 2024-05-17T21:52:43.152 INFO:tasks.workunit.client.0.smithi107.stdout:7/67: write d5/fe [2454820,53684] 0 2024-05-17T21:52:43.154 INFO:tasks.workunit.client.0.smithi107.stdout:7/68: creat d5/df/f20 x:0 0 0 2024-05-17T21:52:43.156 INFO:tasks.workunit.client.0.smithi107.stdout:7/69: creat d5/df/d11/f21 x:0 0 0 2024-05-17T21:52:43.157 INFO:tasks.workunit.client.0.smithi107.stdout:2/50: truncate d0/f2 3593874 0 2024-05-17T21:52:43.158 INFO:tasks.workunit.client.0.smithi107.stdout:9/50: dwrite d0/fa [0,4194304] 0 2024-05-17T21:52:43.159 INFO:tasks.workunit.client.0.smithi107.stdout:9/51: write d0/fa [3464989,63909] 0 2024-05-17T21:52:43.159 INFO:tasks.workunit.client.0.smithi107.stdout:2/51: dread d0/d4/fd [0,4194304] 0 2024-05-17T21:52:43.160 INFO:tasks.workunit.client.0.smithi107.stdout:7/70: creat d5/f22 x:0 0 0 2024-05-17T21:52:43.160 INFO:tasks.workunit.client.0.smithi107.stdout:4/71: write d3/db/fe [590113,4689] 0 2024-05-17T21:52:43.166 INFO:tasks.workunit.client.0.smithi107.stdout:9/52: link d0/f7 d0/fe 0 2024-05-17T21:52:43.167 INFO:tasks.workunit.client.0.smithi107.stdout:9/53: dread - d0/f7 zero size 2024-05-17T21:52:43.169 INFO:tasks.workunit.client.0.smithi107.stdout:7/71: dwrite d5/f18 [0,4194304] 0 2024-05-17T21:52:43.171 INFO:tasks.workunit.client.0.smithi107.stdout:7/72: write d5/df/f1a [756409,94273] 0 2024-05-17T21:52:43.171 INFO:tasks.workunit.client.0.smithi107.stdout:4/72: write d3/f8 [406405,21316] 0 2024-05-17T21:52:43.172 INFO:tasks.workunit.client.0.smithi107.stdout:9/54: write d0/fe [338518,81471] 0 2024-05-17T21:52:43.172 INFO:tasks.workunit.client.0.smithi107.stdout:7/73: chown d5/fd 9748181 1 2024-05-17T21:52:43.173 INFO:tasks.workunit.client.0.smithi107.stdout:4/73: write d3/fa [3531281,101676] 0 2024-05-17T21:52:43.173 INFO:tasks.workunit.client.0.smithi107.stdout:2/52: dread d0/f2 [0,4194304] 0 2024-05-17T21:52:43.175 INFO:tasks.workunit.client.0.smithi107.stdout:9/55: write d0/fa [4399540,55718] 0 2024-05-17T21:52:43.179 INFO:tasks.workunit.client.0.smithi107.stdout:4/74: mkdir d3/db/d16 0 2024-05-17T21:52:43.180 INFO:tasks.workunit.client.0.smithi107.stdout:9/56: dread d0/fe [0,4194304] 0 2024-05-17T21:52:43.180 INFO:tasks.workunit.client.0.smithi107.stdout:7/74: creat d5/d16/f23 x:0 0 0 2024-05-17T21:52:43.181 INFO:tasks.workunit.client.0.smithi107.stdout:9/57: truncate d0/fc 872956 0 2024-05-17T21:52:43.182 INFO:tasks.workunit.client.0.smithi107.stdout:2/53: rename d0/d4 to d0/d9/d11 0 2024-05-17T21:52:43.183 INFO:tasks.workunit.client.0.smithi107.stdout:2/54: truncate d0/fa 680425 0 2024-05-17T21:52:43.187 INFO:tasks.workunit.client.0.smithi107.stdout:4/75: creat d3/db/d16/f17 x:0 0 0 2024-05-17T21:52:43.188 INFO:tasks.workunit.client.0.smithi107.stdout:7/75: symlink d5/df/d11/d1d/l24 0 2024-05-17T21:52:43.188 INFO:tasks.workunit.client.0.smithi107.stdout:7/76: fdatasync d5/fd 0 2024-05-17T21:52:43.189 INFO:tasks.workunit.client.0.smithi107.stdout:7/77: write f4 [5133500,43185] 0 2024-05-17T21:52:43.190 INFO:tasks.workunit.client.0.smithi107.stdout:2/55: symlink d0/d9/d11/df/l12 0 2024-05-17T21:52:43.193 INFO:tasks.workunit.client.0.smithi107.stdout:4/76: rename d3/d6/l13 to d3/db/d16/l18 0 2024-05-17T21:52:43.195 INFO:tasks.workunit.client.0.smithi107.stdout:2/56: fdatasync d0/d9/d11/fd 0 2024-05-17T21:52:43.198 INFO:tasks.workunit.client.0.smithi107.stdout:2/57: dread d0/f2 [0,4194304] 0 2024-05-17T21:52:43.199 INFO:tasks.workunit.client.0.smithi107.stdout:7/78: link d5/f22 d5/df/d11/f25 0 2024-05-17T21:52:43.201 INFO:tasks.workunit.client.0.smithi107.stdout:9/58: link d0/l9 d0/lf 0 2024-05-17T21:52:43.202 INFO:tasks.workunit.client.0.smithi107.stdout:4/77: chown d3/l15 7892915 1 2024-05-17T21:52:43.204 INFO:tasks.workunit.client.0.smithi107.stdout:2/58: mknod d0/d9/d11/c13 0 2024-05-17T21:52:43.205 INFO:tasks.workunit.client.0.smithi107.stdout:4/78: creat d3/db/d16/f19 x:0 0 0 2024-05-17T21:52:43.206 INFO:tasks.workunit.client.0.smithi107.stdout:7/79: dwrite d5/df/d11/f21 [0,4194304] 0 2024-05-17T21:52:43.207 INFO:tasks.workunit.client.0.smithi107.stdout:4/79: readlink d3/l15 0 2024-05-17T21:52:43.209 INFO:tasks.workunit.client.0.smithi107.stdout:4/80: fdatasync d3/db/d16/f19 0 2024-05-17T21:52:43.209 INFO:tasks.workunit.client.0.smithi107.stdout:9/59: creat d0/f10 x:0 0 0 2024-05-17T21:52:43.211 INFO:tasks.workunit.client.0.smithi107.stdout:9/60: write d0/f10 [660749,43524] 0 2024-05-17T21:52:43.211 INFO:tasks.workunit.client.0.smithi107.stdout:7/80: unlink c3 0 2024-05-17T21:52:43.211 INFO:tasks.workunit.client.0.smithi107.stdout:4/81: write d3/db/d16/f17 [739180,25190] 0 2024-05-17T21:52:43.214 INFO:tasks.workunit.client.0.smithi107.stdout:2/59: dwrite d0/d9/d11/f10 [0,4194304] 0 2024-05-17T21:52:43.215 INFO:tasks.workunit.client.0.smithi107.stdout:4/82: rename d3/db/dd to d3/db/dd/d1a 22 2024-05-17T21:52:43.216 INFO:tasks.workunit.client.0.smithi107.stdout:4/83: fdatasync d3/fa 0 2024-05-17T21:52:43.216 INFO:tasks.workunit.client.0.smithi107.stdout:2/60: read d0/f2 [1044989,42301] 0 2024-05-17T21:52:43.218 INFO:tasks.workunit.client.0.smithi107.stdout:2/61: readlink d0/l1 0 2024-05-17T21:52:43.218 INFO:tasks.workunit.client.0.smithi107.stdout:7/81: truncate d5/fd 1922775 0 2024-05-17T21:52:43.219 INFO:tasks.workunit.client.0.smithi107.stdout:2/62: write d0/d9/d11/f10 [139223,120906] 0 2024-05-17T21:52:43.220 INFO:tasks.workunit.client.0.smithi107.stdout:4/84: symlink d3/db/dd/l1b 0 2024-05-17T21:52:43.221 INFO:tasks.workunit.client.0.smithi107.stdout:7/82: creat d5/df/f26 x:0 0 0 2024-05-17T21:52:43.221 INFO:tasks.workunit.client.0.smithi107.stdout:2/63: write d0/fa [511919,113309] 0 2024-05-17T21:52:43.224 INFO:tasks.workunit.client.0.smithi107.stdout:7/83: creat d5/df/d11/d1d/f27 x:0 0 0 2024-05-17T21:52:43.227 INFO:tasks.workunit.client.0.smithi107.stdout:7/84: chown d5/l6 15778 1 2024-05-17T21:52:43.229 INFO:tasks.workunit.client.0.smithi107.stdout:7/85: read f4 [2237849,75641] 0 2024-05-17T21:52:43.231 INFO:tasks.workunit.client.0.smithi107.stdout:4/85: dwrite d3/db/f11 [0,4194304] 0 2024-05-17T21:52:43.237 INFO:tasks.workunit.client.0.smithi107.stdout:9/61: dwrite d0/f7 [0,4194304] 0 2024-05-17T21:52:43.238 INFO:tasks.workunit.client.0.smithi107.stdout:2/64: dwrite d0/f2 [0,4194304] 0 2024-05-17T21:52:43.238 INFO:tasks.workunit.client.0.smithi107.stdout:4/86: mknod d3/db/dd/c1c 0 2024-05-17T21:52:43.241 INFO:tasks.workunit.client.0.smithi107.stdout:7/86: rename d5/df/d11/d1d/f27 to d5/d16/f28 0 2024-05-17T21:52:43.243 INFO:tasks.workunit.client.0.smithi107.stdout:4/87: mknod d3/c1d 0 2024-05-17T21:52:43.250 INFO:tasks.workunit.client.0.smithi107.stdout:4/88: mkdir d3/db/d1e 0 2024-05-17T21:52:43.250 INFO:tasks.workunit.client.0.smithi107.stdout:9/62: getdents d0/dd 0 2024-05-17T21:52:43.256 INFO:tasks.workunit.client.0.smithi107.stdout:4/89: rename f2 to d3/db/f1f 0 2024-05-17T21:52:43.261 INFO:tasks.workunit.client.0.smithi107.stdout:9/63: link d0/cb d0/c11 0 2024-05-17T21:52:43.261 INFO:tasks.workunit.client.0.smithi107.stdout:4/90: dread d3/db/d16/f17 [0,4194304] 0 2024-05-17T21:52:43.263 INFO:tasks.workunit.client.0.smithi107.stdout:9/64: write d0/fe [605754,18257] 0 2024-05-17T21:52:43.263 INFO:tasks.workunit.client.0.smithi107.stdout:2/65: dwrite d0/d9/d11/fd [0,4194304] 0 2024-05-17T21:52:43.263 INFO:tasks.workunit.client.0.smithi107.stdout:9/65: stat d0 0 2024-05-17T21:52:43.264 INFO:tasks.workunit.client.0.smithi107.stdout:4/91: read d3/db/fe [492164,19719] 0 2024-05-17T21:52:43.265 INFO:tasks.workunit.client.0.smithi107.stdout:9/66: mknod d0/dd/c12 0 2024-05-17T21:52:43.268 INFO:tasks.workunit.client.0.smithi107.stdout:9/67: rmdir d0/dd 39 2024-05-17T21:52:43.268 INFO:tasks.workunit.client.0.smithi107.stdout:2/66: dread d0/fa [0,4194304] 0 2024-05-17T21:52:43.271 INFO:tasks.workunit.client.0.smithi107.stdout:7/87: truncate d5/fe 8075281 0 2024-05-17T21:52:43.272 INFO:tasks.workunit.client.0.smithi107.stdout:9/68: creat d0/dd/f13 x:0 0 0 2024-05-17T21:52:43.273 INFO:tasks.workunit.client.0.smithi107.stdout:1/73: sync 2024-05-17T21:52:43.274 INFO:tasks.workunit.client.0.smithi107.stdout:1/74: fdatasync d4/fa 0 2024-05-17T21:52:43.275 INFO:tasks.workunit.client.0.smithi107.stdout:4/92: dwrite d3/db/fe [0,4194304] 0 2024-05-17T21:52:43.276 INFO:tasks.workunit.client.0.smithi107.stdout:9/69: chown d0/l9 285 1 2024-05-17T21:52:43.277 INFO:tasks.workunit.client.0.smithi107.stdout:7/88: dread d5/df/f1a [0,4194304] 0 2024-05-17T21:52:43.280 INFO:tasks.workunit.client.0.smithi107.stdout:7/89: chown d5/df/l1b 11483 1 2024-05-17T21:52:43.280 INFO:tasks.workunit.client.0.smithi107.stdout:9/70: creat d0/f14 x:0 0 0 2024-05-17T21:52:43.285 INFO:tasks.workunit.client.0.smithi107.stdout:4/93: dwrite d3/db/d16/f19 [0,4194304] 0 2024-05-17T21:52:43.285 INFO:tasks.workunit.client.0.smithi107.stdout:2/67: truncate d0/f6 426396 0 2024-05-17T21:52:43.287 INFO:tasks.workunit.client.0.smithi107.stdout:2/68: write d0/d9/d11/f10 [1808702,4406] 0 2024-05-17T21:52:43.287 INFO:tasks.workunit.client.0.smithi107.stdout:9/71: symlink d0/l15 0 2024-05-17T21:52:43.289 INFO:tasks.workunit.client.0.smithi107.stdout:1/75: write d4/f5 [5235220,34307] 0 2024-05-17T21:52:43.291 INFO:tasks.workunit.client.0.smithi107.stdout:7/90: link d5/c17 d5/df/d11/d1d/c29 0 2024-05-17T21:52:43.292 INFO:tasks.workunit.client.0.smithi107.stdout:1/76: chown d4/c6 123737 1 2024-05-17T21:52:43.292 INFO:tasks.workunit.client.0.smithi107.stdout:7/91: read d5/df/f10 [133672,43573] 0 2024-05-17T21:52:43.293 INFO:tasks.workunit.client.0.smithi107.stdout:9/72: rename d0/l9 to d0/dd/l16 0 2024-05-17T21:52:43.294 INFO:tasks.workunit.client.0.smithi107.stdout:4/94: dread d3/db/d16/f19 [0,4194304] 0 2024-05-17T21:52:43.295 INFO:tasks.workunit.client.0.smithi107.stdout:2/69: dread d0/f2 [0,4194304] 0 2024-05-17T21:52:43.298 INFO:tasks.workunit.client.0.smithi107.stdout:9/73: dread d0/fc [0,4194304] 0 2024-05-17T21:52:43.298 INFO:tasks.workunit.client.0.smithi107.stdout:1/77: dread d4/f5 [0,4194304] 0 2024-05-17T21:52:43.300 INFO:tasks.workunit.client.0.smithi107.stdout:4/95: rename d3/db/fe to d3/db/d1e/f20 0 2024-05-17T21:52:43.305 INFO:tasks.workunit.client.0.smithi107.stdout:2/70: fsync d0/f2 0 2024-05-17T21:52:43.305 INFO:tasks.workunit.client.0.smithi107.stdout:9/74: unlink d0/l15 0 2024-05-17T21:52:43.306 INFO:tasks.workunit.client.0.smithi107.stdout:1/78: mkdir d4/df 0 2024-05-17T21:52:43.308 INFO:tasks.workunit.client.0.smithi107.stdout:2/71: rename d0/d9/d11/fd to d0/d9/d11/f14 0 2024-05-17T21:52:43.309 INFO:tasks.workunit.client.0.smithi107.stdout:9/75: rmdir d0/dd 39 2024-05-17T21:52:43.311 INFO:tasks.workunit.client.0.smithi107.stdout:1/79: write d4/f5 [789440,46176] 0 2024-05-17T21:52:43.313 INFO:tasks.workunit.client.0.smithi107.stdout:4/96: dwrite d3/db/f1f [0,4194304] 0 2024-05-17T21:52:43.316 INFO:tasks.workunit.client.0.smithi107.stdout:1/80: creat d4/f10 x:0 0 0 2024-05-17T21:52:43.319 INFO:tasks.workunit.client.0.smithi107.stdout:9/76: stat d0/dd/l16 0 2024-05-17T21:52:43.320 INFO:tasks.workunit.client.0.smithi107.stdout:1/81: rename d4/l7 to d4/de/l11 0 2024-05-17T21:52:43.323 INFO:tasks.workunit.client.0.smithi107.stdout:1/82: creat d4/de/f12 x:0 0 0 2024-05-17T21:52:43.324 INFO:tasks.workunit.client.0.smithi107.stdout:2/72: write d0/f6 [401986,39717] 0 2024-05-17T21:52:43.325 INFO:tasks.workunit.client.0.smithi107.stdout:9/77: creat d0/f17 x:0 0 0 2024-05-17T21:52:43.326 INFO:tasks.workunit.client.0.smithi107.stdout:9/78: readlink d0/dd/l16 0 2024-05-17T21:52:43.326 INFO:tasks.workunit.client.0.smithi107.stdout:1/83: creat d4/df/f13 x:0 0 0 2024-05-17T21:52:43.327 INFO:tasks.workunit.client.0.smithi107.stdout:2/73: mkdir d0/d9/d11/d15 0 2024-05-17T21:52:43.327 INFO:tasks.workunit.client.0.smithi107.stdout:1/84: write d4/fa [4738117,123263] 0 2024-05-17T21:52:43.328 INFO:tasks.workunit.client.0.smithi107.stdout:1/85: chown d4/f8 984421 1 2024-05-17T21:52:43.328 INFO:tasks.workunit.client.0.smithi107.stdout:2/74: fsync d0/d9/d11/f14 0 2024-05-17T21:52:43.329 INFO:tasks.workunit.client.0.smithi107.stdout:2/75: chown d0/f2 1358 1 2024-05-17T21:52:43.330 INFO:tasks.workunit.client.0.smithi107.stdout:1/86: unlink d4/cb 0 2024-05-17T21:52:43.331 INFO:tasks.workunit.client.0.smithi107.stdout:9/79: link d0/c11 d0/dd/c18 0 2024-05-17T21:52:43.332 INFO:tasks.workunit.client.0.smithi107.stdout:2/76: mknod d0/d9/d11/d15/c16 0 2024-05-17T21:52:43.333 INFO:tasks.workunit.client.0.smithi107.stdout:9/80: symlink d0/l19 0 2024-05-17T21:52:43.333 INFO:tasks.workunit.client.0.smithi107.stdout:2/77: rmdir d0/d9/d11/d15 39 2024-05-17T21:52:43.334 INFO:tasks.workunit.client.0.smithi107.stdout:9/81: rmdir d0/dd 39 2024-05-17T21:52:43.334 INFO:tasks.workunit.client.0.smithi107.stdout:9/82: stat d0/fa 0 2024-05-17T21:52:43.335 INFO:tasks.workunit.client.0.smithi107.stdout:2/78: mknod d0/c17 0 2024-05-17T21:52:43.336 INFO:tasks.workunit.client.0.smithi107.stdout:9/83: rmdir d0/dd 39 2024-05-17T21:52:43.338 INFO:tasks.workunit.client.0.smithi107.stdout:2/79: dread d0/d9/d11/f14 [0,4194304] 0 2024-05-17T21:52:43.340 INFO:tasks.workunit.client.0.smithi107.stdout:9/84: link d0/dd/l16 d0/l1a 0 2024-05-17T21:52:43.341 INFO:tasks.workunit.client.0.smithi107.stdout:2/80: truncate d0/f2 1782893 0 2024-05-17T21:52:43.343 INFO:tasks.workunit.client.0.smithi107.stdout:9/85: dread d0/f7 [0,4194304] 0 2024-05-17T21:52:43.343 INFO:tasks.workunit.client.0.smithi107.stdout:9/86: dread - d0/f14 zero size 2024-05-17T21:52:43.344 INFO:tasks.workunit.client.0.smithi107.stdout:9/87: write d0/fe [4104210,65621] 0 2024-05-17T21:52:43.348 INFO:tasks.workunit.client.0.smithi107.stdout:9/88: rmdir d0 39 2024-05-17T21:52:43.353 INFO:tasks.workunit.client.0.smithi107.stdout:2/81: dread d0/f2 [0,4194304] 0 2024-05-17T21:52:43.355 INFO:tasks.workunit.client.0.smithi107.stdout:9/89: dwrite d0/fa [4194304,4194304] 0 2024-05-17T21:52:43.356 INFO:tasks.workunit.client.0.smithi107.stdout:9/90: fdatasync d0/f17 0 2024-05-17T21:52:43.358 INFO:tasks.workunit.client.0.smithi107.stdout:2/82: truncate d0/d9/d11/f14 1419362 0 2024-05-17T21:52:43.359 INFO:tasks.workunit.client.0.smithi107.stdout:9/91: mkdir d0/dd/d1b 0 2024-05-17T21:52:43.365 INFO:tasks.workunit.client.0.smithi107.stdout:2/83: dwrite d0/f2 [0,4194304] 0 2024-05-17T21:52:43.369 INFO:tasks.workunit.client.0.smithi107.stdout:2/84: mkdir d0/d9/d11/d18 0 2024-05-17T21:52:43.373 INFO:tasks.workunit.client.0.smithi107.stdout:9/92: dwrite d0/fc [0,4194304] 0 2024-05-17T21:52:43.374 INFO:tasks.workunit.client.0.smithi107.stdout:9/93: truncate d0/f17 720141 0 2024-05-17T21:52:43.382 INFO:tasks.workunit.client.0.smithi107.stdout:9/94: dwrite d0/fa [0,4194304] 0 2024-05-17T21:52:43.383 INFO:tasks.workunit.client.0.smithi107.stdout:9/95: write d0/fe [4350705,107094] 0 2024-05-17T21:52:43.384 INFO:tasks.workunit.client.0.smithi107.stdout:9/96: write d0/f10 [59305,12860] 0 2024-05-17T21:52:43.399 INFO:tasks.workunit.client.0.smithi107.stdout:9/97: dread d0/f10 [0,4194304] 0 2024-05-17T21:52:43.399 INFO:tasks.workunit.client.0.smithi107.stdout:9/98: dread - d0/f14 zero size 2024-05-17T21:52:43.400 INFO:tasks.workunit.client.0.smithi107.stdout:9/99: chown d0/l19 0 1 2024-05-17T21:52:43.403 INFO:tasks.workunit.client.0.smithi107.stdout:9/100: creat d0/f1c x:0 0 0 2024-05-17T21:52:43.456 INFO:tasks.workunit.client.0.smithi107.stdout:5/67: sync 2024-05-17T21:52:43.457 INFO:tasks.workunit.client.0.smithi107.stdout:5/68: write d5/f7 [2968846,116058] 0 2024-05-17T21:52:43.459 INFO:tasks.workunit.client.0.smithi107.stdout:5/69: rename d5/db/d13 to d5/db/d16 0 2024-05-17T21:52:43.461 INFO:tasks.workunit.client.0.smithi107.stdout:5/70: creat d5/f17 x:0 0 0 2024-05-17T21:52:43.467 INFO:tasks.workunit.client.0.smithi107.stdout:5/71: dwrite d5/f17 [0,4194304] 0 2024-05-17T21:52:43.469 INFO:tasks.workunit.client.0.smithi107.stdout:5/72: mkdir d5/db/dc/d18 0 2024-05-17T21:52:43.470 INFO:tasks.workunit.client.0.smithi107.stdout:5/73: stat d5/db/dc 0 2024-05-17T21:52:43.473 INFO:tasks.workunit.client.0.smithi107.stdout:5/74: creat d5/f19 x:0 0 0 2024-05-17T21:52:43.480 INFO:tasks.workunit.client.0.smithi107.stdout:5/75: truncate d5/f9 2406553 0 2024-05-17T21:52:43.486 INFO:tasks.workunit.client.0.smithi107.stdout:5/76: link d5/f7 d5/db/f1a 0 2024-05-17T21:52:43.486 INFO:tasks.workunit.client.0.smithi107.stdout:5/77: truncate d5/f19 779088 0 2024-05-17T21:52:43.487 INFO:tasks.workunit.client.0.smithi107.stdout:5/78: readlink d5/db/dc/l12 0 2024-05-17T21:52:43.487 INFO:tasks.workunit.client.0.smithi107.stdout:5/79: chown d5/db/dc/lf 16320 1 2024-05-17T21:52:43.493 INFO:tasks.workunit.client.0.smithi107.stdout:5/80: link d5/db/f1a d5/f1b 0 2024-05-17T21:52:43.495 INFO:tasks.workunit.client.0.smithi107.stdout:5/81: symlink d5/db/l1c 0 2024-05-17T21:52:43.499 INFO:tasks.workunit.client.0.smithi107.stdout:5/82: dread d5/fa [0,4194304] 0 2024-05-17T21:52:43.517 INFO:tasks.workunit.client.0.smithi107.stdout:5/83: write d5/f7 [4143272,74214] 0 2024-05-17T21:52:43.517 INFO:tasks.workunit.client.0.smithi107.stdout:5/84: chown d5/f17 1852 1 2024-05-17T21:52:43.518 INFO:tasks.workunit.client.0.smithi107.stdout:5/85: read d5/f7 [1998068,109134] 0 2024-05-17T21:52:43.521 INFO:tasks.workunit.client.0.smithi107.stdout:5/86: write f3 [824941,77470] 0 2024-05-17T21:52:43.522 INFO:tasks.workunit.client.0.smithi107.stdout:5/87: readlink d5/l11 0 2024-05-17T21:52:43.523 INFO:tasks.workunit.client.0.smithi107.stdout:5/88: write d5/f17 [3279038,65524] 0 2024-05-17T21:52:43.526 INFO:tasks.workunit.client.0.smithi107.stdout:5/89: stat l1 0 2024-05-17T21:52:43.531 INFO:tasks.workunit.client.0.smithi107.stdout:5/90: dwrite f3 [0,4194304] 0 2024-05-17T21:52:43.536 INFO:tasks.workunit.client.0.smithi107.stdout:5/91: symlink d5/db/d16/l1d 0 2024-05-17T21:52:43.604 INFO:tasks.workunit.client.0.smithi107.stdout:8/85: sync 2024-05-17T21:52:43.604 INFO:tasks.workunit.client.0.smithi107.stdout:6/42: sync 2024-05-17T21:52:43.607 INFO:tasks.workunit.client.0.smithi107.stdout:6/43: write f1 [1378364,62802] 0 2024-05-17T21:52:43.607 INFO:tasks.workunit.client.0.smithi107.stdout:6/44: chown d9/fa 2085670 1 2024-05-17T21:52:43.609 INFO:tasks.workunit.client.0.smithi107.stdout:6/45: chown d9/cb 48 1 2024-05-17T21:52:43.611 INFO:tasks.workunit.client.0.smithi107.stdout:6/46: mknod d9/cc 0 2024-05-17T21:52:43.613 INFO:tasks.workunit.client.0.smithi107.stdout:6/47: creat d9/fd x:0 0 0 2024-05-17T21:52:43.615 INFO:tasks.workunit.client.0.smithi107.stdout:6/48: fsync f5 0 2024-05-17T21:52:43.616 INFO:tasks.workunit.client.0.smithi107.stdout:6/49: fdatasync f1 0 2024-05-17T21:52:43.622 INFO:tasks.workunit.client.0.smithi107.stdout:6/50: unlink d9/cb 0 2024-05-17T21:52:43.623 INFO:tasks.workunit.client.0.smithi107.stdout:6/51: write f1 [3814133,71963] 0 2024-05-17T21:52:43.626 INFO:tasks.workunit.client.0.smithi107.stdout:6/52: dread d9/fa [0,4194304] 0 2024-05-17T21:52:43.626 INFO:tasks.workunit.client.0.smithi107.stdout:6/53: chown d9/fa 540787 1 2024-05-17T21:52:43.630 INFO:tasks.workunit.client.0.smithi107.stdout:6/54: fdatasync f5 0 2024-05-17T21:52:43.631 INFO:tasks.workunit.client.0.smithi107.stdout:6/55: rename d9 to d9/de 22 2024-05-17T21:52:43.631 INFO:tasks.workunit.client.0.smithi107.stdout:6/56: read f5 [274035,84233] 0 2024-05-17T21:52:43.640 INFO:tasks.workunit.client.0.smithi107.stdout:6/57: dwrite d9/fa [0,4194304] 0 2024-05-17T21:52:43.703 INFO:tasks.workunit.client.0.smithi107.stdout:3/69: sync 2024-05-17T21:52:43.710 INFO:tasks.workunit.client.0.smithi107.stdout:3/70: creat d6/d8/d10/f1e x:0 0 0 2024-05-17T21:52:43.710 INFO:tasks.workunit.client.0.smithi107.stdout:3/71: dread - d6/f1c zero size 2024-05-17T21:52:43.710 INFO:tasks.workunit.client.0.smithi107.stdout:3/72: write d6/f11 [714377,123014] 0 2024-05-17T21:52:43.712 INFO:tasks.workunit.client.0.smithi107.stdout:3/73: rename d6/d8/fb to d6/d1a/f1f 0 2024-05-17T21:52:43.712 INFO:tasks.workunit.client.0.smithi107.stdout:3/74: chown f5 723120 1 2024-05-17T21:52:43.735 INFO:tasks.workunit.client.0.smithi107.stdout:6/58: truncate d9/fa 865075 0 2024-05-17T21:52:43.737 INFO:tasks.workunit.client.0.smithi107.stdout:6/59: fdatasync d9/fa 0 2024-05-17T21:52:43.758 INFO:tasks.workunit.client.0.smithi107.stdout:3/75: truncate d6/fc 68420 0 2024-05-17T21:52:43.762 INFO:tasks.workunit.client.0.smithi107.stdout:6/60: rmdir d9 39 2024-05-17T21:52:43.764 INFO:tasks.workunit.client.0.smithi107.stdout:6/61: dread d9/fa [0,4194304] 0 2024-05-17T21:52:43.767 INFO:tasks.workunit.client.0.smithi107.stdout:6/62: dread f5 [0,4194304] 0 2024-05-17T21:52:43.774 INFO:tasks.workunit.client.0.smithi107.stdout:6/63: dwrite d9/fd [0,4194304] 0 2024-05-17T21:52:43.776 INFO:tasks.workunit.client.0.smithi107.stdout:6/64: creat d9/ff x:0 0 0 2024-05-17T21:52:43.776 INFO:tasks.workunit.client.0.smithi107.stdout:6/65: readlink l3 0 2024-05-17T21:52:43.778 INFO:tasks.workunit.client.0.smithi107.stdout:6/66: creat d9/f10 x:0 0 0 2024-05-17T21:52:43.781 INFO:tasks.workunit.client.0.smithi107.stdout:6/67: rename d9/cc to d9/c11 0 2024-05-17T21:52:43.784 INFO:tasks.workunit.client.0.smithi107.stdout:6/68: rename d9/fd to d9/f12 0 2024-05-17T21:52:43.785 INFO:tasks.workunit.client.0.smithi107.stdout:6/69: symlink d9/l13 0 2024-05-17T21:52:43.786 INFO:tasks.workunit.client.0.smithi107.stdout:6/70: creat d9/f14 x:0 0 0 2024-05-17T21:52:43.787 INFO:tasks.workunit.client.0.smithi107.stdout:6/71: truncate d9/ff 941631 0 2024-05-17T21:52:43.787 INFO:tasks.workunit.client.0.smithi107.stdout:6/72: readlink d9/l13 0 2024-05-17T21:52:43.788 INFO:tasks.workunit.client.0.smithi107.stdout:6/73: creat d9/f15 x:0 0 0 2024-05-17T21:52:43.790 INFO:tasks.workunit.client.0.smithi107.stdout:6/74: mkdir d9/d16 0 2024-05-17T21:52:43.793 INFO:tasks.workunit.client.0.smithi107.stdout:6/75: fsync d9/f15 0 2024-05-17T21:52:43.794 INFO:tasks.workunit.client.0.smithi107.stdout:6/76: symlink d9/d16/l17 0 2024-05-17T21:52:43.796 INFO:tasks.workunit.client.0.smithi107.stdout:6/77: creat d9/d16/f18 x:0 0 0 2024-05-17T21:52:43.801 INFO:tasks.workunit.client.0.smithi107.stdout:6/78: dwrite d9/f10 [0,4194304] 0 2024-05-17T21:52:43.803 INFO:tasks.workunit.client.0.smithi107.stdout:6/79: mkdir d9/d16/d19 0 2024-05-17T21:52:43.803 INFO:tasks.workunit.client.0.smithi107.stdout:6/80: chown d9/f10 208196 1 2024-05-17T21:52:43.805 INFO:tasks.workunit.client.0.smithi107.stdout:6/81: write f1 [2774038,37679] 0 2024-05-17T21:52:43.808 INFO:tasks.workunit.client.0.smithi107.stdout:6/82: link d9/f15 d9/f1a 0 2024-05-17T21:52:43.809 INFO:tasks.workunit.client.0.smithi107.stdout:6/83: read d9/fa [450303,93611] 0 2024-05-17T21:52:43.857 INFO:tasks.workunit.client.0.smithi107.stdout:0/84: sync 2024-05-17T21:52:43.870 INFO:tasks.workunit.client.0.smithi107.stdout:0/85: write d3/f17 [566822,108964] 0 2024-05-17T21:52:43.870 INFO:tasks.workunit.client.0.smithi107.stdout:0/86: dwrite d3/f17 [0,4194304] 0 2024-05-17T21:52:43.871 INFO:tasks.workunit.client.0.smithi107.stdout:0/87: truncate d3/f15 1241358 0 2024-05-17T21:52:43.871 INFO:tasks.workunit.client.0.smithi107.stdout:0/88: fsync d3/f9 0 2024-05-17T21:52:43.872 INFO:tasks.workunit.client.0.smithi107.stdout:0/89: dread f1 [0,4194304] 0 2024-05-17T21:52:43.872 INFO:tasks.workunit.client.0.smithi107.stdout:0/90: readlink d3/dc/ld 0 2024-05-17T21:52:43.875 INFO:tasks.workunit.client.0.smithi107.stdout:0/91: rename d3/l4 to d3/l20 0 2024-05-17T21:52:43.887 INFO:tasks.workunit.client.0.smithi107.stdout:0/92: write d3/f15 [2242698,94227] 0 2024-05-17T21:52:43.887 INFO:tasks.workunit.client.0.smithi107.stdout:0/93: stat d3/f18 0 2024-05-17T21:52:43.889 INFO:tasks.workunit.client.0.smithi107.stdout:0/94: unlink d3/c7 0 2024-05-17T21:52:43.890 INFO:tasks.workunit.client.0.smithi107.stdout:0/95: write d3/f1e [120098,42639] 0 2024-05-17T21:52:43.891 INFO:tasks.workunit.client.0.smithi107.stdout:0/96: creat d3/f21 x:0 0 0 2024-05-17T21:52:43.892 INFO:tasks.workunit.client.0.smithi107.stdout:0/97: mkdir d3/dc/d22 0 2024-05-17T21:52:43.893 INFO:tasks.workunit.client.0.smithi107.stdout:0/98: dread - d3/dc/f14 zero size 2024-05-17T21:52:43.894 INFO:tasks.workunit.client.0.smithi107.stdout:0/99: mkdir d3/dc/d22/d23 0 2024-05-17T21:52:43.895 INFO:tasks.workunit.client.0.smithi107.stdout:0/100: write d3/f1e [163847,109480] 0 2024-05-17T21:52:43.896 INFO:tasks.workunit.client.0.smithi107.stdout:0/101: mkdir d3/d24 0 2024-05-17T21:52:43.897 INFO:tasks.workunit.client.0.smithi107.stdout:0/102: truncate d3/f19 1281619 0 2024-05-17T21:52:43.898 INFO:tasks.workunit.client.0.smithi107.stdout:0/103: mknod d3/d24/c25 0 2024-05-17T21:52:43.899 INFO:tasks.workunit.client.0.smithi107.stdout:0/104: creat d3/dc/f26 x:0 0 0 2024-05-17T21:52:43.901 INFO:tasks.workunit.client.0.smithi107.stdout:0/105: symlink d3/dc/d22/l27 0 2024-05-17T21:52:43.901 INFO:tasks.workunit.client.0.smithi107.stdout:0/106: fsync d3/ff 0 2024-05-17T21:52:43.902 INFO:tasks.workunit.client.0.smithi107.stdout:0/107: mknod d3/dc/d22/d23/c28 0 2024-05-17T21:52:43.905 INFO:tasks.workunit.client.0.smithi107.stdout:0/108: rename d3/fb to d3/dc/d22/d23/f29 0 2024-05-17T21:52:43.906 INFO:tasks.workunit.client.0.smithi107.stdout:0/109: unlink d3/l5 0 2024-05-17T21:52:43.909 INFO:tasks.workunit.client.0.smithi107.stdout:0/110: dread d3/dc/f12 [0,4194304] 0 2024-05-17T21:52:43.914 INFO:tasks.workunit.client.0.smithi107.stdout:0/111: write d3/dc/f14 [452037,65169] 0 2024-05-17T21:52:43.916 INFO:tasks.workunit.client.0.smithi107.stdout:0/112: creat d3/d24/f2a x:0 0 0 2024-05-17T21:52:43.918 INFO:tasks.workunit.client.0.smithi107.stdout:0/113: rename d3/la to d3/d24/l2b 0 2024-05-17T21:52:43.921 INFO:tasks.workunit.client.0.smithi107.stdout:0/114: dread d3/dc/f16 [0,4194304] 0 2024-05-17T21:52:43.922 INFO:tasks.workunit.client.0.smithi107.stdout:0/115: creat d3/dc/d22/d23/f2c x:0 0 0 2024-05-17T21:52:43.925 INFO:tasks.workunit.client.0.smithi107.stdout:0/116: rename d3/dc/f1f to d3/dc/d22/f2d 0 2024-05-17T21:52:43.926 INFO:tasks.workunit.client.0.smithi107.stdout:0/117: mknod d3/dc/c2e 0 2024-05-17T21:52:43.927 INFO:tasks.workunit.client.0.smithi107.stdout:8/86: sync 2024-05-17T21:52:43.927 INFO:tasks.workunit.client.0.smithi107.stdout:1/87: sync 2024-05-17T21:52:43.927 INFO:tasks.workunit.client.0.smithi107.stdout:9/101: sync 2024-05-17T21:52:43.927 INFO:tasks.workunit.client.0.smithi107.stdout:5/92: sync 2024-05-17T21:52:43.927 INFO:tasks.workunit.client.0.smithi107.stdout:3/76: sync 2024-05-17T21:52:43.927 INFO:tasks.workunit.client.0.smithi107.stdout:7/92: sync 2024-05-17T21:52:43.927 INFO:tasks.workunit.client.0.smithi107.stdout:4/97: sync 2024-05-17T21:52:43.927 INFO:tasks.workunit.client.0.smithi107.stdout:2/85: sync 2024-05-17T21:52:43.928 INFO:tasks.workunit.client.0.smithi107.stdout:1/88: chown l2 41 1 2024-05-17T21:52:43.929 INFO:tasks.workunit.client.0.smithi107.stdout:0/118: write d3/f1d [695119,89053] 0 2024-05-17T21:52:43.929 INFO:tasks.workunit.client.0.smithi107.stdout:5/93: write d5/f8 [299507,73112] 0 2024-05-17T21:52:43.931 INFO:tasks.workunit.client.0.smithi107.stdout:8/87: symlink dc/l24 0 2024-05-17T21:52:43.932 INFO:tasks.workunit.client.0.smithi107.stdout:3/77: rmdir d6/d8/d10 39 2024-05-17T21:52:43.934 INFO:tasks.workunit.client.0.smithi107.stdout:5/94: write d5/f19 [855616,51714] 0 2024-05-17T21:52:43.941 INFO:tasks.workunit.client.0.smithi107.stdout:4/98: dwrite d3/db/f14 [0,4194304] 0 2024-05-17T21:52:43.943 INFO:tasks.workunit.client.0.smithi107.stdout:1/89: dwrite d4/fa [0,4194304] 0 2024-05-17T21:52:43.945 INFO:tasks.workunit.client.0.smithi107.stdout:1/90: stat d4/c9 0 2024-05-17T21:52:43.947 INFO:tasks.workunit.client.0.smithi107.stdout:3/78: dwrite f2 [0,4194304] 0 2024-05-17T21:52:43.950 INFO:tasks.workunit.client.0.smithi107.stdout:3/79: truncate d6/d8/f9 998290 0 2024-05-17T21:52:43.951 INFO:tasks.workunit.client.0.smithi107.stdout:3/80: stat d6/c18 0 2024-05-17T21:52:43.953 INFO:tasks.workunit.client.0.smithi107.stdout:7/93: mkdir d5/df/d11/d2a 0 2024-05-17T21:52:43.954 INFO:tasks.workunit.client.0.smithi107.stdout:3/81: truncate d6/f1b 759516 0 2024-05-17T21:52:43.956 INFO:tasks.workunit.client.0.smithi107.stdout:3/82: stat d6/f1c 0 2024-05-17T21:52:43.956 INFO:tasks.workunit.client.0.smithi107.stdout:1/91: dwrite d4/f10 [0,4194304] 0 2024-05-17T21:52:43.957 INFO:tasks.workunit.client.0.smithi107.stdout:3/83: write f2 [4506003,27268] 0 2024-05-17T21:52:43.958 INFO:tasks.workunit.client.0.smithi107.stdout:3/84: fdatasync d6/d8/f17 0 2024-05-17T21:52:43.967 INFO:tasks.workunit.client.0.smithi107.stdout:6/84: sync 2024-05-17T21:52:43.968 INFO:tasks.workunit.client.0.smithi107.stdout:6/85: chown d9/f12 45990558 1 2024-05-17T21:52:43.982 INFO:tasks.workunit.client.0.smithi107.stdout:1/92: fdatasync d4/fa 0 2024-05-17T21:52:43.984 INFO:tasks.workunit.client.0.smithi107.stdout:1/93: dread f3 [0,4194304] 0 2024-05-17T21:52:43.999 INFO:tasks.workunit.client.0.smithi107.stdout:8/88: mkdir dc/d10/d17/d25 0 2024-05-17T21:52:44.001 INFO:tasks.workunit.client.0.smithi107.stdout:5/95: mkdir d5/db/dc/d1e 0 2024-05-17T21:52:44.002 INFO:tasks.workunit.client.0.smithi107.stdout:4/99: mkdir d3/db/dd/d21 0 2024-05-17T21:52:44.003 INFO:tasks.workunit.client.0.smithi107.stdout:4/100: stat d3/db/fc 0 2024-05-17T21:52:44.004 INFO:tasks.workunit.client.0.smithi107.stdout:7/94: creat d5/d16/f2b x:0 0 0 2024-05-17T21:52:44.005 INFO:tasks.workunit.client.0.smithi107.stdout:5/96: dread d5/f17 [0,4194304] 0 2024-05-17T21:52:44.011 INFO:tasks.workunit.client.0.smithi107.stdout:6/86: rmdir d9 39 2024-05-17T21:52:44.016 INFO:tasks.workunit.client.0.smithi107.stdout:1/94: mkdir d4/df/d14 0 2024-05-17T21:52:44.016 INFO:tasks.workunit.client.0.smithi107.stdout:1/95: stat d4/f8 0 2024-05-17T21:52:44.017 INFO:tasks.workunit.client.0.smithi107.stdout:1/96: chown d4/c6 102 1 2024-05-17T21:52:44.017 INFO:tasks.workunit.client.0.smithi107.stdout:9/102: mknod d0/dd/d1b/c1d 0 2024-05-17T21:52:44.021 INFO:tasks.workunit.client.0.smithi107.stdout:8/89: rename fb to dc/d10/f26 0 2024-05-17T21:52:44.023 INFO:tasks.workunit.client.0.smithi107.stdout:4/101: write d3/db/d16/f19 [1304461,75566] 0 2024-05-17T21:52:44.024 INFO:tasks.workunit.client.0.smithi107.stdout:7/95: symlink d5/d16/l2c 0 2024-05-17T21:52:44.025 INFO:tasks.workunit.client.0.smithi107.stdout:5/97: rmdir d5/db 39 2024-05-17T21:52:44.025 INFO:tasks.workunit.client.0.smithi107.stdout:7/96: readlink d5/df/l1b 0 2024-05-17T21:52:44.026 INFO:tasks.workunit.client.0.smithi107.stdout:3/85: mknod d6/d8/d10/c20 0 2024-05-17T21:52:44.027 INFO:tasks.workunit.client.0.smithi107.stdout:3/86: write d6/d8/f9 [1724986,18325] 0 2024-05-17T21:52:44.031 INFO:tasks.workunit.client.0.smithi107.stdout:9/103: rename d0/fa to d0/dd/f1e 0 2024-05-17T21:52:44.031 INFO:tasks.workunit.client.0.smithi107.stdout:9/104: write d0/f4 [3363945,113156] 0 2024-05-17T21:52:44.036 INFO:tasks.workunit.client.0.smithi107.stdout:6/87: mkdir d9/d1b 0 2024-05-17T21:52:44.038 INFO:tasks.workunit.client.0.smithi107.stdout:3/87: write d6/d1a/f1f [94058,53714] 0 2024-05-17T21:52:44.040 INFO:tasks.workunit.client.0.smithi107.stdout:4/102: dwrite d3/db/fc [0,4194304] 0 2024-05-17T21:52:44.040 INFO:tasks.workunit.client.0.smithi107.stdout:9/105: dwrite d0/fc [0,4194304] 0 2024-05-17T21:52:44.042 INFO:tasks.workunit.client.0.smithi107.stdout:8/90: rename dc/df/d23 to dc/d21/d27 0 2024-05-17T21:52:44.046 INFO:tasks.workunit.client.0.smithi107.stdout:5/98: creat d5/db/dc/f1f x:0 0 0 2024-05-17T21:52:44.049 INFO:tasks.workunit.client.0.smithi107.stdout:7/97: unlink d5/df/d11/l12 0 2024-05-17T21:52:44.056 INFO:tasks.workunit.client.0.smithi107.stdout:4/103: creat d3/f22 x:0 0 0 2024-05-17T21:52:44.056 INFO:tasks.workunit.client.0.smithi107.stdout:4/104: read - d3/f22 zero size 2024-05-17T21:52:44.057 INFO:tasks.workunit.client.0.smithi107.stdout:9/106: mkdir d0/dd/d1f 0 2024-05-17T21:52:44.058 INFO:tasks.workunit.client.0.smithi107.stdout:9/107: read d0/f4 [3595527,29824] 0 2024-05-17T21:52:44.063 INFO:tasks.workunit.client.0.smithi107.stdout:4/105: dwrite d3/f22 [0,4194304] 0 2024-05-17T21:52:44.064 INFO:tasks.workunit.client.0.smithi107.stdout:3/88: truncate d6/f11 106338 0 2024-05-17T21:52:44.065 INFO:tasks.workunit.client.0.smithi107.stdout:8/91: symlink dc/d10/d17/d25/l28 0 2024-05-17T21:52:44.066 INFO:tasks.workunit.client.0.smithi107.stdout:9/108: symlink d0/dd/l20 0 2024-05-17T21:52:44.070 INFO:tasks.workunit.client.0.smithi107.stdout:6/88: rmdir d9/d1b 0 2024-05-17T21:52:44.070 INFO:tasks.workunit.client.0.smithi107.stdout:6/89: chown l3 0 1 2024-05-17T21:52:44.072 INFO:tasks.workunit.client.0.smithi107.stdout:4/106: mknod d3/d6/c23 0 2024-05-17T21:52:44.073 INFO:tasks.workunit.client.0.smithi107.stdout:3/89: mknod d6/d8/d10/c21 0 2024-05-17T21:52:44.075 INFO:tasks.workunit.client.0.smithi107.stdout:8/92: mknod dc/d10/c29 0 2024-05-17T21:52:44.077 INFO:tasks.workunit.client.0.smithi107.stdout:9/109: chown d0/lf 0 1 2024-05-17T21:52:44.078 INFO:tasks.workunit.client.0.smithi107.stdout:6/90: fsync f5 0 2024-05-17T21:52:44.079 INFO:tasks.workunit.client.0.smithi107.stdout:4/107: write d3/db/d16/f17 [1378426,33082] 0 2024-05-17T21:52:44.080 INFO:tasks.workunit.client.0.smithi107.stdout:8/93: dread dc/f18 [0,4194304] 0 2024-05-17T21:52:44.083 INFO:tasks.workunit.client.0.smithi107.stdout:9/110: unlink d0/l8 0 2024-05-17T21:52:44.085 INFO:tasks.workunit.client.0.smithi107.stdout:6/91: rmdir d9/d16 39 2024-05-17T21:52:44.086 INFO:tasks.workunit.client.0.smithi107.stdout:2/86: dwrite d0/d9/d11/f14 [0,4194304] 0 2024-05-17T21:52:44.086 INFO:tasks.workunit.client.0.smithi107.stdout:6/92: stat d9/f1a 0 2024-05-17T21:52:44.087 INFO:tasks.workunit.client.0.smithi107.stdout:4/108: mknod d3/d6/c24 0 2024-05-17T21:52:44.088 INFO:tasks.workunit.client.0.smithi107.stdout:3/90: chown d6/f11 6 1 2024-05-17T21:52:44.089 INFO:tasks.workunit.client.0.smithi107.stdout:4/109: truncate d3/db/d1e/f20 4393491 0 2024-05-17T21:52:44.090 INFO:tasks.workunit.client.0.smithi107.stdout:8/94: symlink dc/d21/l2a 0 2024-05-17T21:52:44.090 INFO:tasks.workunit.client.0.smithi107.stdout:3/91: truncate d6/f1c 840417 0 2024-05-17T21:52:44.091 INFO:tasks.workunit.client.0.smithi107.stdout:6/93: dread d9/f10 [0,4194304] 0 2024-05-17T21:52:44.091 INFO:tasks.workunit.client.0.smithi107.stdout:3/92: fdatasync f5 0 2024-05-17T21:52:44.091 INFO:tasks.workunit.client.0.smithi107.stdout:6/94: chown l3 1178 1 2024-05-17T21:52:44.092 INFO:tasks.workunit.client.0.smithi107.stdout:3/93: chown d6/d8/ld 487771 1 2024-05-17T21:52:44.093 INFO:tasks.workunit.client.0.smithi107.stdout:3/94: truncate d6/f1c 1668988 0 2024-05-17T21:52:44.097 INFO:tasks.workunit.client.0.smithi107.stdout:4/110: mkdir d3/db/d25 0 2024-05-17T21:52:44.098 INFO:tasks.workunit.client.0.smithi107.stdout:9/111: dwrite d0/dd/f1e [8388608,4194304] 0 2024-05-17T21:52:44.103 INFO:tasks.workunit.client.0.smithi107.stdout:2/87: unlink d0/d9/lc 0 2024-05-17T21:52:44.107 INFO:tasks.workunit.client.0.smithi107.stdout:9/112: mkdir d0/dd/d21 0 2024-05-17T21:52:44.107 INFO:tasks.workunit.client.0.smithi107.stdout:9/113: dread - d0/dd/f13 zero size 2024-05-17T21:52:44.108 INFO:tasks.workunit.client.0.smithi107.stdout:9/114: fsync d0/fc 0 2024-05-17T21:52:44.110 INFO:tasks.workunit.client.0.smithi107.stdout:8/95: rename dc/ld to dc/d21/l2b 0 2024-05-17T21:52:44.114 INFO:tasks.workunit.client.0.smithi107.stdout:0/119: truncate d3/dc/f12 3040603 0 2024-05-17T21:52:44.128 INFO:tasks.workunit.client.0.smithi107.stdout:1/97: sync 2024-05-17T21:52:44.138 INFO:tasks.workunit.client.0.smithi107.stdout:0/120: chown d3/dc/ld 143261 1 2024-05-17T21:52:44.138 INFO:tasks.workunit.client.0.smithi107.stdout:6/95: creat d9/f1c x:0 0 0 2024-05-17T21:52:44.138 INFO:tasks.workunit.client.0.smithi107.stdout:8/96: creat dc/d10/d17/d25/f2c x:0 0 0 2024-05-17T21:52:44.138 INFO:tasks.workunit.client.0.smithi107.stdout:2/88: mknod d0/c19 0 2024-05-17T21:52:44.140 INFO:tasks.workunit.client.0.smithi107.stdout:0/121: read d3/f19 [1026637,71398] 0 2024-05-17T21:52:44.140 INFO:tasks.workunit.client.0.smithi107.stdout:6/96: chown d9/fa 186386 1 2024-05-17T21:52:44.141 INFO:tasks.workunit.client.0.smithi107.stdout:5/99: write d5/f7 [21751,44818] 0 2024-05-17T21:52:44.142 INFO:tasks.workunit.client.0.smithi107.stdout:2/89: dread d0/f6 [0,4194304] 0 2024-05-17T21:52:44.144 INFO:tasks.workunit.client.0.smithi107.stdout:9/115: creat d0/f22 x:0 0 0 2024-05-17T21:52:44.145 INFO:tasks.workunit.client.0.smithi107.stdout:3/95: rename d6/d8/d10/l16 to d6/d8/l22 0 2024-05-17T21:52:44.146 INFO:tasks.workunit.client.0.smithi107.stdout:1/98: rename d4 to d4/df/d14/d15 22 2024-05-17T21:52:44.147 INFO:tasks.workunit.client.0.smithi107.stdout:6/97: dread - d9/d16/f18 zero size 2024-05-17T21:52:44.148 INFO:tasks.workunit.client.0.smithi107.stdout:1/99: chown d4/fa 1184 1 2024-05-17T21:52:44.150 INFO:tasks.workunit.client.0.smithi107.stdout:5/100: dwrite d5/f8 [0,4194304] 0 2024-05-17T21:52:44.151 INFO:tasks.workunit.client.0.smithi107.stdout:2/90: fdatasync d0/d9/d11/f10 0 2024-05-17T21:52:44.156 INFO:tasks.workunit.client.0.smithi107.stdout:3/96: dwrite d6/d8/f9 [0,4194304] 0 2024-05-17T21:52:44.156 INFO:tasks.workunit.client.0.smithi107.stdout:0/122: link d3/f1e d3/dc/d22/f2f 0 2024-05-17T21:52:44.157 INFO:tasks.workunit.client.0.smithi107.stdout:5/101: dread d5/f19 [0,4194304] 0 2024-05-17T21:52:44.157 INFO:tasks.workunit.client.0.smithi107.stdout:3/97: chown f3 26041 1 2024-05-17T21:52:44.158 INFO:tasks.workunit.client.0.smithi107.stdout:3/98: fdatasync f5 0 2024-05-17T21:52:44.158 INFO:tasks.workunit.client.0.smithi107.stdout:0/123: write d3/f21 [123753,96455] 0 2024-05-17T21:52:44.159 INFO:tasks.workunit.client.0.smithi107.stdout:8/97: dwrite dc/f18 [0,4194304] 0 2024-05-17T21:52:44.159 INFO:tasks.workunit.client.0.smithi107.stdout:1/100: truncate f3 1021417 0 2024-05-17T21:52:44.160 INFO:tasks.workunit.client.0.smithi107.stdout:5/102: write d5/f19 [733702,8780] 0 2024-05-17T21:52:44.162 INFO:tasks.workunit.client.0.smithi107.stdout:8/98: chown dc/d21/d27 21467567 1 2024-05-17T21:52:44.162 INFO:tasks.workunit.client.0.smithi107.stdout:9/116: creat d0/dd/d1f/f23 x:0 0 0 2024-05-17T21:52:44.165 INFO:tasks.workunit.client.0.smithi107.stdout:1/101: rmdir d4/df 39 2024-05-17T21:52:44.166 INFO:tasks.workunit.client.0.smithi107.stdout:1/102: write d4/f5 [6176466,78324] 0 2024-05-17T21:52:44.167 INFO:tasks.workunit.client.0.smithi107.stdout:9/117: dread d0/dd/f1e [4194304,4194304] 0 2024-05-17T21:52:44.167 INFO:tasks.workunit.client.0.smithi107.stdout:9/118: stat d0/f14 0 2024-05-17T21:52:44.168 INFO:tasks.workunit.client.1.smithi138.stdout:3/16: sync 2024-05-17T21:52:44.168 INFO:tasks.workunit.client.1.smithi138.stdout:3/17: write - no filename 2024-05-17T21:52:44.168 INFO:tasks.workunit.client.1.smithi138.stdout:2/3: sync 2024-05-17T21:52:44.170 INFO:tasks.workunit.client.0.smithi107.stdout:5/103: rmdir d5/db/dc 39 2024-05-17T21:52:44.171 INFO:tasks.workunit.client.0.smithi107.stdout:5/104: readlink d5/l11 0 2024-05-17T21:52:44.173 INFO:tasks.workunit.client.0.smithi107.stdout:2/91: chown d0/d9/le 119391863 1 2024-05-17T21:52:44.176 INFO:tasks.workunit.client.1.smithi138.stdout:2/4: write f2 [802446,100678] 0 2024-05-17T21:52:44.179 INFO:tasks.workunit.client.0.smithi107.stdout:6/98: creat d9/f1d x:0 0 0 2024-05-17T21:52:44.179 INFO:tasks.workunit.client.0.smithi107.stdout:8/99: creat dc/df/f2d x:0 0 0 2024-05-17T21:52:44.180 INFO:tasks.workunit.client.0.smithi107.stdout:8/100: truncate dc/df/f20 425719 0 2024-05-17T21:52:44.182 INFO:tasks.workunit.client.1.smithi138.stdout:3/18: creat d1/f2 x:0 0 0 2024-05-17T21:52:44.183 INFO:tasks.workunit.client.1.smithi138.stdout:3/19: stat d1/f2 0 2024-05-17T21:52:44.184 INFO:tasks.workunit.client.0.smithi107.stdout:6/99: dwrite d9/f1d [0,4194304] 0 2024-05-17T21:52:44.187 INFO:tasks.workunit.client.0.smithi107.stdout:3/99: link f3 d6/d8/d10/f23 0 2024-05-17T21:52:44.189 INFO:tasks.workunit.client.0.smithi107.stdout:3/100: chown d6/d8/f12 7 1 2024-05-17T21:52:44.192 INFO:tasks.workunit.client.0.smithi107.stdout:5/105: chown d5/db/dc/lf 6201829 1 2024-05-17T21:52:44.193 INFO:tasks.workunit.client.0.smithi107.stdout:8/101: rmdir dc/df 39 2024-05-17T21:52:44.196 INFO:tasks.workunit.client.0.smithi107.stdout:2/92: mknod d0/d9/d11/d18/c1a 0 2024-05-17T21:52:44.197 INFO:tasks.workunit.client.0.smithi107.stdout:3/101: symlink d6/d1a/l24 0 2024-05-17T21:52:44.197 INFO:tasks.workunit.client.0.smithi107.stdout:6/100: dwrite d9/f1d [0,4194304] 0 2024-05-17T21:52:44.199 INFO:tasks.workunit.client.0.smithi107.stdout:5/106: creat d5/db/dc/f20 x:0 0 0 2024-05-17T21:52:44.200 INFO:tasks.workunit.client.0.smithi107.stdout:8/102: fdatasync dc/df/f11 0 2024-05-17T21:52:44.201 INFO:tasks.workunit.client.0.smithi107.stdout:9/119: dwrite d0/f10 [0,4194304] 0 2024-05-17T21:52:44.202 INFO:tasks.workunit.client.0.smithi107.stdout:0/124: dwrite d3/dc/f12 [0,4194304] 0 2024-05-17T21:52:44.202 INFO:tasks.workunit.client.0.smithi107.stdout:1/103: link d4/fd d4/df/d14/f16 0 2024-05-17T21:52:44.203 INFO:tasks.workunit.client.0.smithi107.stdout:3/102: write d6/d8/f17 [1288799,115996] 0 2024-05-17T21:52:44.205 INFO:tasks.workunit.client.0.smithi107.stdout:6/101: creat d9/d16/f1e x:0 0 0 2024-05-17T21:52:44.205 INFO:tasks.workunit.client.0.smithi107.stdout:1/104: write d4/f10 [4949041,128425] 0 2024-05-17T21:52:44.207 INFO:tasks.workunit.client.0.smithi107.stdout:9/120: write d0/f17 [1067175,12967] 0 2024-05-17T21:52:44.209 INFO:tasks.workunit.client.0.smithi107.stdout:6/102: readlink d9/d16/l17 0 2024-05-17T21:52:44.210 INFO:tasks.workunit.client.0.smithi107.stdout:8/103: creat dc/df/f2e x:0 0 0 2024-05-17T21:52:44.211 INFO:tasks.workunit.client.0.smithi107.stdout:9/121: dread - d0/dd/d1f/f23 zero size 2024-05-17T21:52:44.213 INFO:tasks.workunit.client.0.smithi107.stdout:6/103: truncate d9/f1d 5124042 0 2024-05-17T21:52:44.214 INFO:tasks.workunit.client.0.smithi107.stdout:3/103: mkdir d6/d8/d10/d25 0 2024-05-17T21:52:44.215 INFO:tasks.workunit.client.0.smithi107.stdout:6/104: read - d9/f15 zero size 2024-05-17T21:52:44.215 INFO:tasks.workunit.client.0.smithi107.stdout:0/125: dwrite d3/f15 [0,4194304] 0 2024-05-17T21:52:44.223 INFO:tasks.workunit.client.0.smithi107.stdout:6/105: unlink d9/l13 0 2024-05-17T21:52:44.224 INFO:tasks.workunit.client.0.smithi107.stdout:0/126: symlink d3/dc/d22/l30 0 2024-05-17T21:52:44.225 INFO:tasks.workunit.client.0.smithi107.stdout:1/105: symlink d4/df/d14/l17 0 2024-05-17T21:52:44.225 INFO:tasks.workunit.client.0.smithi107.stdout:2/93: link d0/c3 d0/d9/d11/df/c1b 0 2024-05-17T21:52:44.228 INFO:tasks.workunit.client.0.smithi107.stdout:3/104: dwrite d6/f1c [0,4194304] 0 2024-05-17T21:52:44.228 INFO:tasks.workunit.client.0.smithi107.stdout:9/122: rename d0/dd/c18 to d0/dd/d1f/c24 0 2024-05-17T21:52:44.231 INFO:tasks.workunit.client.0.smithi107.stdout:0/127: mkdir d3/dc/d31 0 2024-05-17T21:52:44.233 INFO:tasks.workunit.client.0.smithi107.stdout:1/106: mknod d4/df/c18 0 2024-05-17T21:52:44.234 INFO:tasks.workunit.client.0.smithi107.stdout:7/98: sync 2024-05-17T21:52:44.234 INFO:tasks.workunit.client.0.smithi107.stdout:0/128: write d3/d24/f2a [33607,59169] 0 2024-05-17T21:52:44.235 INFO:tasks.workunit.client.0.smithi107.stdout:7/99: stat d5/ca 0 2024-05-17T21:52:44.238 INFO:tasks.workunit.client.0.smithi107.stdout:3/105: dwrite d6/f1b [0,4194304] 0 2024-05-17T21:52:44.240 INFO:tasks.workunit.client.0.smithi107.stdout:8/104: rename dc/d10/d17/d25 to dc/d21/d27/d2f 0 2024-05-17T21:52:44.241 INFO:tasks.workunit.client.0.smithi107.stdout:9/123: fsync d0/f4 0 2024-05-17T21:52:44.243 INFO:tasks.workunit.client.0.smithi107.stdout:5/107: truncate f3 2608317 0 2024-05-17T21:52:44.244 INFO:tasks.workunit.client.0.smithi107.stdout:0/129: dwrite d3/dc/f14 [0,4194304] 0 2024-05-17T21:52:44.244 INFO:tasks.workunit.client.0.smithi107.stdout:5/108: chown d5/le 48 1 2024-05-17T21:52:44.245 INFO:tasks.workunit.client.0.smithi107.stdout:2/94: mkdir d0/d9/d1c 0 2024-05-17T21:52:44.252 INFO:tasks.workunit.client.0.smithi107.stdout:1/107: rename d4/fd to d4/de/f19 0 2024-05-17T21:52:44.255 INFO:tasks.workunit.client.0.smithi107.stdout:8/105: creat dc/d21/d27/d2f/f30 x:0 0 0 2024-05-17T21:52:44.257 INFO:tasks.workunit.client.0.smithi107.stdout:8/106: truncate dc/df/f11 902397 0 2024-05-17T21:52:44.257 INFO:tasks.workunit.client.0.smithi107.stdout:9/124: symlink d0/dd/l25 0 2024-05-17T21:52:44.258 INFO:tasks.workunit.client.0.smithi107.stdout:6/106: rename d9/f14 to d9/f1f 0 2024-05-17T21:52:44.259 INFO:tasks.workunit.client.0.smithi107.stdout:1/108: dwrite d4/fc [4194304,4194304] 0 2024-05-17T21:52:44.260 INFO:tasks.workunit.client.0.smithi107.stdout:0/130: creat d3/dc/d22/f32 x:0 0 0 2024-05-17T21:52:44.260 INFO:tasks.workunit.client.0.smithi107.stdout:1/109: readlink l2 0 2024-05-17T21:52:44.265 INFO:tasks.workunit.client.0.smithi107.stdout:7/100: unlink d5/c17 0 2024-05-17T21:52:44.266 INFO:tasks.workunit.client.0.smithi107.stdout:6/107: dwrite d9/d16/f18 [0,4194304] 0 2024-05-17T21:52:44.269 INFO:tasks.workunit.client.0.smithi107.stdout:4/111: sync 2024-05-17T21:52:44.270 INFO:tasks.workunit.client.0.smithi107.stdout:8/107: mknod dc/d10/c31 0 2024-05-17T21:52:44.271 INFO:tasks.workunit.client.0.smithi107.stdout:4/112: write d3/db/f14 [457682,25432] 0 2024-05-17T21:52:44.272 INFO:tasks.workunit.client.0.smithi107.stdout:9/125: mknod d0/dd/d1b/c26 0 2024-05-17T21:52:44.274 INFO:tasks.workunit.client.0.smithi107.stdout:0/131: creat d3/f33 x:0 0 0 2024-05-17T21:52:44.276 INFO:tasks.workunit.client.0.smithi107.stdout:1/110: rmdir d4/df/d14 39 2024-05-17T21:52:44.277 INFO:tasks.workunit.client.0.smithi107.stdout:1/111: dread - d4/de/f12 zero size 2024-05-17T21:52:44.279 INFO:tasks.workunit.client.0.smithi107.stdout:2/95: symlink d0/d9/d11/l1d 0 2024-05-17T21:52:44.280 INFO:tasks.workunit.client.0.smithi107.stdout:0/132: dwrite d3/f18 [0,4194304] 0 2024-05-17T21:52:44.280 INFO:tasks.workunit.client.0.smithi107.stdout:2/96: fdatasync d0/d9/d11/f10 0 2024-05-17T21:52:44.286 INFO:tasks.workunit.client.0.smithi107.stdout:8/108: creat dc/d21/d27/d2f/f32 x:0 0 0 2024-05-17T21:52:44.287 INFO:tasks.workunit.client.0.smithi107.stdout:4/113: mkdir d3/d6/d26 0 2024-05-17T21:52:44.290 INFO:tasks.workunit.client.0.smithi107.stdout:5/109: link d5/f17 d5/db/d16/f21 0 2024-05-17T21:52:44.293 INFO:tasks.workunit.client.0.smithi107.stdout:0/133: creat d3/f34 x:0 0 0 2024-05-17T21:52:44.297 INFO:tasks.workunit.client.0.smithi107.stdout:7/101: chown d5/df/d11/d1d/c29 1178 1 2024-05-17T21:52:44.298 INFO:tasks.workunit.client.0.smithi107.stdout:3/106: getdents d6/d1a 0 2024-05-17T21:52:44.303 INFO:tasks.workunit.client.0.smithi107.stdout:8/109: dread dc/d10/f26 [0,4194304] 0 2024-05-17T21:52:44.304 INFO:tasks.workunit.client.0.smithi107.stdout:1/112: truncate f3 864470 0 2024-05-17T21:52:44.304 INFO:tasks.workunit.client.0.smithi107.stdout:8/110: read dc/f18 [2918941,23591] 0 2024-05-17T21:52:44.306 INFO:tasks.workunit.client.0.smithi107.stdout:8/111: chown dc/d10/c29 277 1 2024-05-17T21:52:44.307 INFO:tasks.workunit.client.0.smithi107.stdout:0/134: creat d3/f35 x:0 0 0 2024-05-17T21:52:44.308 INFO:tasks.workunit.client.0.smithi107.stdout:2/97: dwrite d0/f6 [0,4194304] 0 2024-05-17T21:52:44.309 INFO:tasks.workunit.client.0.smithi107.stdout:0/135: write d3/f15 [2978669,126193] 0 2024-05-17T21:52:44.309 INFO:tasks.workunit.client.0.smithi107.stdout:1/113: dread d4/de/f19 [0,4194304] 0 2024-05-17T21:52:44.310 INFO:tasks.workunit.client.0.smithi107.stdout:2/98: stat d0/d9/d11/df 0 2024-05-17T21:52:44.311 INFO:tasks.workunit.client.0.smithi107.stdout:6/108: rmdir d9/d16/d19 0 2024-05-17T21:52:44.312 INFO:tasks.workunit.client.0.smithi107.stdout:0/136: write d3/f17 [3747768,94308] 0 2024-05-17T21:52:44.314 INFO:tasks.workunit.client.0.smithi107.stdout:6/109: chown d9/d16 346771 1 2024-05-17T21:52:44.317 INFO:tasks.workunit.client.0.smithi107.stdout:5/110: dwrite d5/fa [4194304,4194304] 0 2024-05-17T21:52:44.319 INFO:tasks.workunit.client.0.smithi107.stdout:8/112: mknod dc/df/c33 0 2024-05-17T21:52:44.319 INFO:tasks.workunit.client.0.smithi107.stdout:8/113: chown dc/d10/f15 1 1 2024-05-17T21:52:44.332 INFO:tasks.workunit.client.0.smithi107.stdout:9/126: getdents d0 0 2024-05-17T21:52:44.333 INFO:tasks.workunit.client.0.smithi107.stdout:0/137: symlink d3/dc/l36 0 2024-05-17T21:52:44.333 INFO:tasks.workunit.client.0.smithi107.stdout:0/138: chown d3 393 1 2024-05-17T21:52:44.333 INFO:tasks.workunit.client.0.smithi107.stdout:9/127: dread - d0/dd/d1f/f23 zero size 2024-05-17T21:52:44.335 INFO:tasks.workunit.client.0.smithi107.stdout:5/111: mknod d5/db/dc/c22 0 2024-05-17T21:52:44.336 INFO:tasks.workunit.client.0.smithi107.stdout:8/114: mkdir dc/d21/d27/d34 0 2024-05-17T21:52:44.338 INFO:tasks.workunit.client.0.smithi107.stdout:1/114: mkdir d4/df/d1a 0 2024-05-17T21:52:44.340 INFO:tasks.workunit.client.0.smithi107.stdout:3/107: dwrite d6/f11 [0,4194304] 0 2024-05-17T21:52:44.340 INFO:tasks.workunit.client.0.smithi107.stdout:4/114: getdents d3 0 2024-05-17T21:52:44.342 INFO:tasks.workunit.client.0.smithi107.stdout:6/110: mknod d9/c20 0 2024-05-17T21:52:44.342 INFO:tasks.workunit.client.0.smithi107.stdout:0/139: creat d3/dc/f37 x:0 0 0 2024-05-17T21:52:44.342 INFO:tasks.workunit.client.0.smithi107.stdout:7/102: getdents d5/df 0 2024-05-17T21:52:44.344 INFO:tasks.workunit.client.0.smithi107.stdout:6/111: chown d9/fa 4754 1 2024-05-17T21:52:44.344 INFO:tasks.workunit.client.0.smithi107.stdout:8/115: chown dc/d10/c14 4732 1 2024-05-17T21:52:44.345 INFO:tasks.workunit.client.0.smithi107.stdout:8/116: chown dc 22 1 2024-05-17T21:52:44.346 INFO:tasks.workunit.client.0.smithi107.stdout:1/115: dread d4/df/d14/f16 [0,4194304] 0 2024-05-17T21:52:44.347 INFO:tasks.workunit.client.0.smithi107.stdout:9/128: link d0/dd/d1f/f23 d0/dd/d21/f27 0 2024-05-17T21:52:44.347 INFO:tasks.workunit.client.0.smithi107.stdout:3/108: dread d6/d8/f12 [0,4194304] 0 2024-05-17T21:52:44.348 INFO:tasks.workunit.client.0.smithi107.stdout:9/129: dread - d0/dd/d21/f27 zero size 2024-05-17T21:52:44.349 INFO:tasks.workunit.client.0.smithi107.stdout:7/103: rmdir d5/df/d11 39 2024-05-17T21:52:44.350 INFO:tasks.workunit.client.0.smithi107.stdout:9/130: fdatasync d0/f22 0 2024-05-17T21:52:44.354 INFO:tasks.workunit.client.0.smithi107.stdout:8/117: dwrite dc/df/f22 [0,4194304] 0 2024-05-17T21:52:44.354 INFO:tasks.workunit.client.0.smithi107.stdout:0/140: unlink d3/dc/d22/d23/c28 0 2024-05-17T21:52:44.355 INFO:tasks.workunit.client.0.smithi107.stdout:8/118: chown dc/d21/d27/d34 19 1 2024-05-17T21:52:44.355 INFO:tasks.workunit.client.0.smithi107.stdout:5/112: creat d5/f23 x:0 0 0 2024-05-17T21:52:44.356 INFO:tasks.workunit.client.0.smithi107.stdout:4/115: truncate d3/fa 652129 0 2024-05-17T21:52:44.357 INFO:tasks.workunit.client.0.smithi107.stdout:4/116: stat d3/d6 0 2024-05-17T21:52:44.358 INFO:tasks.workunit.client.0.smithi107.stdout:9/131: dwrite d0/fe [0,4194304] 0 2024-05-17T21:52:44.358 INFO:tasks.workunit.client.0.smithi107.stdout:4/117: fdatasync d3/db/d16/f19 0 2024-05-17T21:52:44.366 INFO:tasks.workunit.client.0.smithi107.stdout:8/119: rename dc/df to dc/d21/d35 0 2024-05-17T21:52:44.367 INFO:tasks.workunit.client.0.smithi107.stdout:5/113: creat d5/db/dc/f24 x:0 0 0 2024-05-17T21:52:44.371 INFO:tasks.workunit.client.0.smithi107.stdout:3/109: symlink d6/d8/d10/d25/l26 0 2024-05-17T21:52:44.373 INFO:tasks.workunit.client.0.smithi107.stdout:9/132: creat d0/dd/d1b/f28 x:0 0 0 2024-05-17T21:52:44.375 INFO:tasks.workunit.client.0.smithi107.stdout:6/112: rename d9/c20 to d9/c21 0 2024-05-17T21:52:44.376 INFO:tasks.workunit.client.0.smithi107.stdout:1/116: dwrite d4/df/d14/f16 [0,4194304] 0 2024-05-17T21:52:44.379 INFO:tasks.workunit.client.1.smithi138.stdout:4/30: sync 2024-05-17T21:52:44.379 INFO:tasks.workunit.client.1.smithi138.stdout:6/97: sync 2024-05-17T21:52:44.379 INFO:tasks.workunit.client.1.smithi138.stdout:9/3: sync 2024-05-17T21:52:44.379 INFO:tasks.workunit.client.1.smithi138.stdout:1/24: sync 2024-05-17T21:52:44.379 INFO:tasks.workunit.client.1.smithi138.stdout:0/98: sync 2024-05-17T21:52:44.379 INFO:tasks.workunit.client.1.smithi138.stdout:8/12: sync 2024-05-17T21:52:44.379 INFO:tasks.workunit.client.1.smithi138.stdout:7/1: sync 2024-05-17T21:52:44.379 INFO:tasks.workunit.client.1.smithi138.stdout:5/8: sync 2024-05-17T21:52:44.379 INFO:tasks.workunit.client.1.smithi138.stdout:5/9: dwrite - no filename 2024-05-17T21:52:44.380 INFO:tasks.workunit.client.1.smithi138.stdout:7/2: chown . 82 1 2024-05-17T21:52:44.380 INFO:tasks.workunit.client.1.smithi138.stdout:7/3: truncate - no filename 2024-05-17T21:52:44.380 INFO:tasks.workunit.client.1.smithi138.stdout:7/4: write - no filename 2024-05-17T21:52:44.380 INFO:tasks.workunit.client.1.smithi138.stdout:7/5: dread - no filename 2024-05-17T21:52:44.380 INFO:tasks.workunit.client.1.smithi138.stdout:7/6: rename - no filename 2024-05-17T21:52:44.380 INFO:tasks.workunit.client.1.smithi138.stdout:7/7: dread - no filename 2024-05-17T21:52:44.380 INFO:tasks.workunit.client.1.smithi138.stdout:7/8: dwrite - no filename 2024-05-17T21:52:44.382 INFO:tasks.workunit.client.0.smithi107.stdout:6/113: mkdir d9/d16/d22 0 2024-05-17T21:52:44.385 INFO:tasks.workunit.client.1.smithi138.stdout:8/13: write d0/f2 [712446,69527] 0 2024-05-17T21:52:44.387 INFO:tasks.workunit.client.0.smithi107.stdout:7/104: rename d5/df/c19 to d5/df/d11/d2a/c2d 0 2024-05-17T21:52:44.388 INFO:tasks.workunit.client.1.smithi138.stdout:8/14: write d0/f2 [265166,376] 0 2024-05-17T21:52:44.388 INFO:tasks.workunit.client.0.smithi107.stdout:7/105: read d5/f18 [1335505,39121] 0 2024-05-17T21:52:44.389 INFO:tasks.workunit.client.1.smithi138.stdout:9/4: creat f2 x:0 0 0 2024-05-17T21:52:44.389 INFO:tasks.workunit.client.1.smithi138.stdout:9/5: readlink - no filename 2024-05-17T21:52:44.390 INFO:tasks.workunit.client.0.smithi107.stdout:9/133: mkdir d0/dd/d21/d29 0 2024-05-17T21:52:44.391 INFO:tasks.workunit.client.1.smithi138.stdout:5/10: mkdir d1 0 2024-05-17T21:52:44.391 INFO:tasks.workunit.client.1.smithi138.stdout:5/11: truncate - no filename 2024-05-17T21:52:44.391 INFO:tasks.workunit.client.1.smithi138.stdout:5/12: readlink - no filename 2024-05-17T21:52:44.391 INFO:tasks.workunit.client.1.smithi138.stdout:5/13: dwrite - no filename 2024-05-17T21:52:44.391 INFO:tasks.workunit.client.1.smithi138.stdout:5/14: dwrite - no filename 2024-05-17T21:52:44.391 INFO:tasks.workunit.client.1.smithi138.stdout:5/15: dwrite - no filename 2024-05-17T21:52:44.391 INFO:tasks.workunit.client.1.smithi138.stdout:5/16: dread - no filename 2024-05-17T21:52:44.392 INFO:tasks.workunit.client.0.smithi107.stdout:0/141: getdents d3/d24 0 2024-05-17T21:52:44.392 INFO:tasks.workunit.client.1.smithi138.stdout:6/98: dread d7/d8/ff [0,4194304] 0 2024-05-17T21:52:44.392 INFO:tasks.workunit.client.1.smithi138.stdout:5/17: chown d1 109 1 2024-05-17T21:52:44.392 INFO:tasks.workunit.client.1.smithi138.stdout:5/18: write - no filename 2024-05-17T21:52:44.392 INFO:tasks.workunit.client.1.smithi138.stdout:5/19: write - no filename 2024-05-17T21:52:44.392 INFO:tasks.workunit.client.1.smithi138.stdout:5/20: write - no filename 2024-05-17T21:52:44.393 INFO:tasks.workunit.client.1.smithi138.stdout:0/99: dwrite d3/d10/f20 [0,4194304] 0 2024-05-17T21:52:44.393 INFO:tasks.workunit.client.1.smithi138.stdout:8/15: dread - d0/f1 zero size 2024-05-17T21:52:44.394 INFO:tasks.workunit.client.1.smithi138.stdout:4/31: dread d0/f7 [4194304,4194304] 0 2024-05-17T21:52:44.395 INFO:tasks.workunit.client.1.smithi138.stdout:9/6: write f1 [699605,88979] 0 2024-05-17T21:52:44.397 INFO:tasks.workunit.client.1.smithi138.stdout:9/7: write f1 [67184,66424] 0 2024-05-17T21:52:44.397 INFO:tasks.workunit.client.0.smithi107.stdout:5/114: link d5/db/dc/f1f d5/db/f25 0 2024-05-17T21:52:44.399 INFO:tasks.workunit.client.0.smithi107.stdout:3/110: creat d6/d8/f27 x:0 0 0 2024-05-17T21:52:44.402 INFO:tasks.workunit.client.1.smithi138.stdout:1/25: mknod d4/c7 0 2024-05-17T21:52:44.405 INFO:tasks.workunit.client.0.smithi107.stdout:4/118: truncate d3/db/f14 1183990 0 2024-05-17T21:52:44.407 INFO:tasks.workunit.client.1.smithi138.stdout:7/9: mkdir d0 0 2024-05-17T21:52:44.407 INFO:tasks.workunit.client.1.smithi138.stdout:7/10: write - no filename 2024-05-17T21:52:44.412 INFO:tasks.workunit.client.0.smithi107.stdout:7/106: rename d5/df/d11/f15 to d5/df/d11/d1c/f2e 0 2024-05-17T21:52:44.412 INFO:tasks.workunit.client.0.smithi107.stdout:7/107: fsync d5/d16/f2b 0 2024-05-17T21:52:44.413 INFO:tasks.workunit.client.1.smithi138.stdout:6/99: mknod d7/c1b 0 2024-05-17T21:52:44.414 INFO:tasks.workunit.client.0.smithi107.stdout:9/134: write d0/dd/d1f/f23 [1016668,65701] 0 2024-05-17T21:52:44.415 INFO:tasks.workunit.client.0.smithi107.stdout:9/135: chown d0/f22 1580915 1 2024-05-17T21:52:44.417 INFO:tasks.workunit.client.0.smithi107.stdout:5/115: mknod d5/db/dc/c26 0 2024-05-17T21:52:44.418 INFO:tasks.workunit.client.0.smithi107.stdout:5/116: write d5/db/dc/f24 [976750,11198] 0 2024-05-17T21:52:44.421 INFO:tasks.workunit.client.1.smithi138.stdout:6/100: dwrite f1 [8388608,4194304] 0 2024-05-17T21:52:44.424 INFO:tasks.workunit.client.1.smithi138.stdout:8/16: creat d0/f4 x:0 0 0 2024-05-17T21:52:44.427 INFO:tasks.workunit.client.1.smithi138.stdout:8/17: chown d0/f4 13123500 1 2024-05-17T21:52:44.427 INFO:tasks.workunit.client.0.smithi107.stdout:1/117: creat d4/f1b x:0 0 0 2024-05-17T21:52:44.429 INFO:tasks.workunit.client.0.smithi107.stdout:9/136: dread d0/f4 [0,4194304] 0 2024-05-17T21:52:44.430 INFO:tasks.workunit.client.0.smithi107.stdout:1/118: dread - d4/de/f12 zero size 2024-05-17T21:52:44.430 INFO:tasks.workunit.client.0.smithi107.stdout:6/114: dwrite d9/f10 [0,4194304] 0 2024-05-17T21:52:44.430 INFO:tasks.workunit.client.0.smithi107.stdout:4/119: mknod d3/db/dd/c27 0 2024-05-17T21:52:44.431 INFO:tasks.workunit.client.1.smithi138.stdout:8/18: dread - d0/f1 zero size 2024-05-17T21:52:44.431 INFO:tasks.workunit.client.1.smithi138.stdout:9/8: creat f3 x:0 0 0 2024-05-17T21:52:44.432 INFO:tasks.workunit.client.1.smithi138.stdout:1/26: mkdir d4/d8 0 2024-05-17T21:52:44.432 INFO:tasks.workunit.client.0.smithi107.stdout:6/115: write d9/d16/f18 [4493836,34831] 0 2024-05-17T21:52:44.432 INFO:tasks.workunit.client.0.smithi107.stdout:4/120: write d3/db/dd/f12 [2963620,60778] 0 2024-05-17T21:52:44.433 INFO:tasks.workunit.client.1.smithi138.stdout:4/32: dwrite d0/f7 [4194304,4194304] 0 2024-05-17T21:52:44.433 INFO:tasks.workunit.client.0.smithi107.stdout:6/116: read d9/f10 [2626183,94055] 0 2024-05-17T21:52:44.434 INFO:tasks.workunit.client.1.smithi138.stdout:8/19: write d0/f4 [798503,41321] 0 2024-05-17T21:52:44.434 INFO:tasks.workunit.client.1.smithi138.stdout:9/9: truncate f3 739089 0 2024-05-17T21:52:44.436 INFO:tasks.workunit.client.1.smithi138.stdout:8/20: chown d0/f4 2511227 1 2024-05-17T21:52:44.437 INFO:tasks.workunit.client.1.smithi138.stdout:6/101: dwrite d7/fc [0,4194304] 0 2024-05-17T21:52:44.441 INFO:tasks.workunit.client.1.smithi138.stdout:9/10: dread f3 [0,4194304] 0 2024-05-17T21:52:44.441 INFO:tasks.workunit.client.1.smithi138.stdout:8/21: rename d0 to d0/d5 22 2024-05-17T21:52:44.443 INFO:tasks.workunit.client.1.smithi138.stdout:0/100: unlink d3/lb 0 2024-05-17T21:52:44.445 INFO:tasks.workunit.client.0.smithi107.stdout:7/108: fdatasync d5/df/f26 0 2024-05-17T21:52:44.448 INFO:tasks.workunit.client.1.smithi138.stdout:5/21: symlink d1/l2 0 2024-05-17T21:52:44.449 INFO:tasks.workunit.client.1.smithi138.stdout:5/22: dwrite - no filename 2024-05-17T21:52:44.449 INFO:tasks.workunit.client.1.smithi138.stdout:7/11: mknod d0/c1 0 2024-05-17T21:52:44.449 INFO:tasks.workunit.client.1.smithi138.stdout:6/102: chown d7/l16 10283 1 2024-05-17T21:52:44.451 INFO:tasks.workunit.client.0.smithi107.stdout:9/137: mknod d0/dd/d1b/c2a 0 2024-05-17T21:52:44.452 INFO:tasks.workunit.client.1.smithi138.stdout:5/23: rename d1 to d1/d3 22 2024-05-17T21:52:44.452 INFO:tasks.workunit.client.1.smithi138.stdout:7/12: rename d0 to d0/d2 22 2024-05-17T21:52:44.452 INFO:tasks.workunit.client.1.smithi138.stdout:1/27: dread f0 [0,4194304] 0 2024-05-17T21:52:44.453 INFO:tasks.workunit.client.1.smithi138.stdout:5/24: chown c0 6 1 2024-05-17T21:52:44.453 INFO:tasks.workunit.client.1.smithi138.stdout:5/25: write - no filename 2024-05-17T21:52:44.454 INFO:tasks.workunit.client.1.smithi138.stdout:7/13: rename d0 to d0/d3 22 2024-05-17T21:52:44.456 INFO:tasks.workunit.client.1.smithi138.stdout:5/26: readlink d1/l2 0 2024-05-17T21:52:44.456 INFO:tasks.workunit.client.1.smithi138.stdout:5/27: truncate - no filename 2024-05-17T21:52:44.456 INFO:tasks.workunit.client.1.smithi138.stdout:5/28: write - no filename 2024-05-17T21:52:44.456 INFO:tasks.workunit.client.1.smithi138.stdout:5/29: write - no filename 2024-05-17T21:52:44.456 INFO:tasks.workunit.client.1.smithi138.stdout:5/30: dwrite - no filename 2024-05-17T21:52:44.456 INFO:tasks.workunit.client.0.smithi107.stdout:4/121: write d3/db/d1e/f20 [458726,75704] 0 2024-05-17T21:52:44.456 INFO:tasks.workunit.client.0.smithi107.stdout:4/122: readlink l1 0 2024-05-17T21:52:44.457 INFO:tasks.workunit.client.0.smithi107.stdout:4/123: fsync d3/db/fc 0 2024-05-17T21:52:44.457 INFO:tasks.workunit.client.1.smithi138.stdout:6/103: stat d7/db/d12/l17 0 2024-05-17T21:52:44.457 INFO:tasks.workunit.client.1.smithi138.stdout:1/28: dread - f3 zero size 2024-05-17T21:52:44.458 INFO:tasks.workunit.client.0.smithi107.stdout:4/124: chown d3/db/dd/f12 71 1 2024-05-17T21:52:44.459 INFO:tasks.workunit.client.1.smithi138.stdout:5/31: readlink d1/l2 0 2024-05-17T21:52:44.459 INFO:tasks.workunit.client.0.smithi107.stdout:0/142: rename d3/l20 to d3/dc/d22/l38 0 2024-05-17T21:52:44.459 INFO:tasks.workunit.client.1.smithi138.stdout:9/11: dwrite f2 [0,4194304] 0 2024-05-17T21:52:44.460 INFO:tasks.workunit.client.0.smithi107.stdout:7/109: stat d5/df/l1b 0 2024-05-17T21:52:44.460 INFO:tasks.workunit.client.1.smithi138.stdout:1/29: write d4/f6 [782258,123765] 0 2024-05-17T21:52:44.460 INFO:tasks.workunit.client.1.smithi138.stdout:1/30: readlink - no filename 2024-05-17T21:52:44.461 INFO:tasks.workunit.client.0.smithi107.stdout:7/110: dread - d5/df/f26 zero size 2024-05-17T21:52:44.463 INFO:tasks.workunit.client.0.smithi107.stdout:5/117: truncate d5/f9 587769 0 2024-05-17T21:52:44.463 INFO:tasks.workunit.client.0.smithi107.stdout:4/125: dwrite d3/db/d16/f17 [0,4194304] 0 2024-05-17T21:52:44.464 INFO:tasks.workunit.client.0.smithi107.stdout:4/126: write d3/db/dd/f10 [3470150,4826] 0 2024-05-17T21:52:44.466 INFO:tasks.workunit.client.0.smithi107.stdout:1/119: mknod d4/c1c 0 2024-05-17T21:52:44.467 INFO:tasks.workunit.client.1.smithi138.stdout:9/12: unlink c0 0 2024-05-17T21:52:44.467 INFO:tasks.workunit.client.1.smithi138.stdout:9/13: chown f2 0 1 2024-05-17T21:52:44.469 INFO:tasks.workunit.client.0.smithi107.stdout:9/138: symlink d0/dd/d1f/l2b 0 2024-05-17T21:52:44.472 INFO:tasks.workunit.client.0.smithi107.stdout:6/117: creat d9/d16/d22/f23 x:0 0 0 2024-05-17T21:52:44.473 INFO:tasks.workunit.client.0.smithi107.stdout:6/118: chown d9/d16/l17 235 1 2024-05-17T21:52:44.473 INFO:tasks.workunit.client.0.smithi107.stdout:6/119: readlink d9/d16/l17 0 2024-05-17T21:52:44.475 INFO:tasks.workunit.client.0.smithi107.stdout:6/120: dread d9/f1d [4194304,4194304] 0 2024-05-17T21:52:44.476 INFO:tasks.workunit.client.0.smithi107.stdout:6/121: chown d9/f15 739609 1 2024-05-17T21:52:44.476 INFO:tasks.workunit.client.1.smithi138.stdout:6/104: rename d7/db/d12/c1a to d7/db/c1c 0 2024-05-17T21:52:44.476 INFO:tasks.workunit.client.0.smithi107.stdout:6/122: dread - d9/d16/f1e zero size 2024-05-17T21:52:44.479 INFO:tasks.workunit.client.0.smithi107.stdout:6/123: dread d9/d16/f18 [0,4194304] 0 2024-05-17T21:52:44.480 INFO:tasks.workunit.client.1.smithi138.stdout:0/101: getdents d3/d10 0 2024-05-17T21:52:44.480 INFO:tasks.workunit.client.0.smithi107.stdout:6/124: truncate d9/f1c 454442 0 2024-05-17T21:52:44.480 INFO:tasks.workunit.client.0.smithi107.stdout:0/143: mkdir d3/d39 0 2024-05-17T21:52:44.481 INFO:tasks.workunit.client.0.smithi107.stdout:0/144: fdatasync d3/f35 0 2024-05-17T21:52:44.481 INFO:tasks.workunit.client.1.smithi138.stdout:9/14: mknod c4 0 2024-05-17T21:52:44.482 INFO:tasks.workunit.client.0.smithi107.stdout:0/145: write d3/dc/f26 [849320,122169] 0 2024-05-17T21:52:44.483 INFO:tasks.workunit.client.0.smithi107.stdout:3/111: rename d6/d8/l22 to d6/d8/l28 0 2024-05-17T21:52:44.486 INFO:tasks.workunit.client.0.smithi107.stdout:7/111: mkdir d5/df/d11/d1c/d2f 0 2024-05-17T21:52:44.487 INFO:tasks.workunit.client.0.smithi107.stdout:2/99: sync 2024-05-17T21:52:44.488 INFO:tasks.workunit.client.1.smithi138.stdout:4/33: truncate d0/f7 3165133 0 2024-05-17T21:52:44.489 INFO:tasks.workunit.client.0.smithi107.stdout:3/112: dwrite d6/d1a/f1f [0,4194304] 0 2024-05-17T21:52:44.490 INFO:tasks.workunit.client.0.smithi107.stdout:4/127: symlink d3/d6/l28 0 2024-05-17T21:52:44.490 INFO:tasks.workunit.client.0.smithi107.stdout:4/128: stat l1 0 2024-05-17T21:52:44.491 INFO:tasks.workunit.client.0.smithi107.stdout:1/120: mkdir d4/df/d14/d1d 0 2024-05-17T21:52:44.492 INFO:tasks.workunit.client.0.smithi107.stdout:1/121: write d4/de/f19 [4275641,58142] 0 2024-05-17T21:52:44.493 INFO:tasks.workunit.client.0.smithi107.stdout:1/122: write d4/f8 [813502,15345] 0 2024-05-17T21:52:44.496 INFO:tasks.workunit.client.0.smithi107.stdout:9/139: mkdir d0/dd/d1f/d2c 0 2024-05-17T21:52:44.497 INFO:tasks.workunit.client.0.smithi107.stdout:9/140: write d0/f10 [346397,97870] 0 2024-05-17T21:52:44.501 INFO:tasks.workunit.client.1.smithi138.stdout:9/15: dread f3 [0,4194304] 0 2024-05-17T21:52:44.502 INFO:tasks.workunit.client.1.smithi138.stdout:5/32: link d1/l2 d1/l4 0 2024-05-17T21:52:44.502 INFO:tasks.workunit.client.1.smithi138.stdout:5/33: fsync - no filename 2024-05-17T21:52:44.504 INFO:tasks.workunit.client.0.smithi107.stdout:3/113: dread d6/d8/f17 [0,4194304] 0 2024-05-17T21:52:44.505 INFO:tasks.workunit.client.0.smithi107.stdout:6/125: unlink l3 0 2024-05-17T21:52:44.508 INFO:tasks.workunit.client.1.smithi138.stdout:9/16: dwrite f1 [0,4194304] 0 2024-05-17T21:52:44.509 INFO:tasks.workunit.client.0.smithi107.stdout:0/146: rename d3/dc/d22/f2f to d3/dc/d31/f3a 0 2024-05-17T21:52:44.509 INFO:tasks.workunit.client.0.smithi107.stdout:0/147: chown d3/dc/f14 52 1 2024-05-17T21:52:44.510 INFO:tasks.workunit.client.1.smithi138.stdout:7/14: getdents d0 0 2024-05-17T21:52:44.510 INFO:tasks.workunit.client.1.smithi138.stdout:7/15: chown d0 61194 1 2024-05-17T21:52:44.510 INFO:tasks.workunit.client.1.smithi138.stdout:7/16: fsync - no filename 2024-05-17T21:52:44.511 INFO:tasks.workunit.client.1.smithi138.stdout:7/17: chown d0/c1 81 1 2024-05-17T21:52:44.517 INFO:tasks.workunit.client.1.smithi138.stdout:4/34: read d0/f7 [3084762,78451] 0 2024-05-17T21:52:44.519 INFO:tasks.workunit.client.0.smithi107.stdout:2/100: rmdir d0/d9 39 2024-05-17T21:52:44.521 INFO:tasks.workunit.client.0.smithi107.stdout:8/120: sync 2024-05-17T21:52:44.522 INFO:tasks.workunit.client.1.smithi138.stdout:5/34: rmdir d1 39 2024-05-17T21:52:44.525 INFO:tasks.workunit.client.1.smithi138.stdout:1/31: getdents d4 0 2024-05-17T21:52:44.526 INFO:tasks.workunit.client.0.smithi107.stdout:8/121: dwrite f4 [0,4194304] 0 2024-05-17T21:52:44.528 INFO:tasks.workunit.client.1.smithi138.stdout:9/17: rename f1 to f5 0 2024-05-17T21:52:44.529 INFO:tasks.workunit.client.0.smithi107.stdout:1/123: creat d4/de/f1e x:0 0 0 2024-05-17T21:52:44.529 INFO:tasks.workunit.client.1.smithi138.stdout:9/18: read f3 [376010,4188] 0 2024-05-17T21:52:44.531 INFO:tasks.workunit.client.1.smithi138.stdout:1/32: dwrite f2 [0,4194304] 0 2024-05-17T21:52:44.536 INFO:tasks.workunit.client.1.smithi138.stdout:6/105: truncate d7/fc 196626 0 2024-05-17T21:52:44.538 INFO:tasks.workunit.client.1.smithi138.stdout:0/102: creat d3/f27 x:0 0 0 2024-05-17T21:52:44.544 INFO:tasks.workunit.client.0.smithi107.stdout:3/114: write d6/d8/d10/f23 [1561887,1311] 0 2024-05-17T21:52:44.545 INFO:tasks.workunit.client.0.smithi107.stdout:6/126: fsync d9/f1f 0 2024-05-17T21:52:44.547 INFO:tasks.workunit.client.1.smithi138.stdout:1/33: fsync f0 0 2024-05-17T21:52:44.547 INFO:tasks.workunit.client.0.smithi107.stdout:3/115: dread d6/f1c [0,4194304] 0 2024-05-17T21:52:44.548 INFO:tasks.workunit.client.1.smithi138.stdout:1/34: dread - f3 zero size 2024-05-17T21:52:44.551 INFO:tasks.workunit.client.1.smithi138.stdout:4/35: symlink d0/d3/db/ld 0 2024-05-17T21:52:44.552 INFO:tasks.workunit.client.0.smithi107.stdout:0/148: symlink d3/dc/l3b 0 2024-05-17T21:52:44.552 INFO:tasks.workunit.client.0.smithi107.stdout:0/149: write d3/f11 [377910,50421] 0 2024-05-17T21:52:44.553 INFO:tasks.workunit.client.1.smithi138.stdout:6/106: creat d7/db/d12/f1d x:0 0 0 2024-05-17T21:52:44.554 INFO:tasks.workunit.client.0.smithi107.stdout:7/112: unlink d5/fd 0 2024-05-17T21:52:44.555 INFO:tasks.workunit.client.0.smithi107.stdout:7/113: dread - d5/d16/f23 zero size 2024-05-17T21:52:44.556 INFO:tasks.workunit.client.1.smithi138.stdout:0/103: creat d3/d10/f28 x:0 0 0 2024-05-17T21:52:44.556 INFO:tasks.workunit.client.1.smithi138.stdout:0/104: chown d3/fd 0 1 2024-05-17T21:52:44.556 INFO:tasks.workunit.client.1.smithi138.stdout:0/105: readlink d3/d10/l13 0 2024-05-17T21:52:44.557 INFO:tasks.workunit.client.1.smithi138.stdout:0/106: dread - d3/d10/f14 zero size 2024-05-17T21:52:44.557 INFO:tasks.workunit.client.1.smithi138.stdout:5/35: chown d1/l4 1 1 2024-05-17T21:52:44.558 INFO:tasks.workunit.client.0.smithi107.stdout:0/150: dwrite d3/d24/f2a [0,4194304] 0 2024-05-17T21:52:44.558 INFO:tasks.workunit.client.1.smithi138.stdout:7/18: link d0/c1 d0/c4 0 2024-05-17T21:52:44.559 INFO:tasks.workunit.client.0.smithi107.stdout:0/151: truncate d3/f34 270175 0 2024-05-17T21:52:44.562 INFO:tasks.workunit.client.0.smithi107.stdout:4/129: symlink d3/d6/d26/l29 0 2024-05-17T21:52:44.566 INFO:tasks.workunit.client.0.smithi107.stdout:8/122: creat dc/d21/d35/f36 x:0 0 0 2024-05-17T21:52:44.566 INFO:tasks.workunit.client.0.smithi107.stdout:8/123: truncate dc/d21/d35/f36 676219 0 2024-05-17T21:52:44.567 INFO:tasks.workunit.client.0.smithi107.stdout:8/124: read dc/d10/f26 [2053253,90870] 0 2024-05-17T21:52:44.568 INFO:tasks.workunit.client.1.smithi138.stdout:5/36: rmdir d1 39 2024-05-17T21:52:44.568 INFO:tasks.workunit.client.1.smithi138.stdout:5/37: fsync - no filename 2024-05-17T21:52:44.568 INFO:tasks.workunit.client.1.smithi138.stdout:5/38: dwrite - no filename 2024-05-17T21:52:44.568 INFO:tasks.workunit.client.1.smithi138.stdout:5/39: read - no filename 2024-05-17T21:52:44.568 INFO:tasks.workunit.client.1.smithi138.stdout:5/40: fdatasync - no filename 2024-05-17T21:52:44.569 INFO:tasks.workunit.client.0.smithi107.stdout:9/141: symlink d0/dd/d21/d29/l2d 0 2024-05-17T21:52:44.569 INFO:tasks.workunit.client.0.smithi107.stdout:9/142: write d0/fc [2206198,117027] 0 2024-05-17T21:52:44.569 INFO:tasks.workunit.client.1.smithi138.stdout:7/19: rmdir d0 39 2024-05-17T21:52:44.570 INFO:tasks.workunit.client.1.smithi138.stdout:4/36: dwrite d0/f7 [0,4194304] 0 2024-05-17T21:52:44.571 INFO:tasks.workunit.client.0.smithi107.stdout:6/127: creat d9/d16/f24 x:0 0 0 2024-05-17T21:52:44.573 INFO:tasks.workunit.client.0.smithi107.stdout:3/116: mknod d6/d8/d10/c29 0 2024-05-17T21:52:44.579 INFO:tasks.workunit.client.0.smithi107.stdout:5/118: getdents d5/db/d16 0 2024-05-17T21:52:44.579 INFO:tasks.workunit.client.0.smithi107.stdout:5/119: chown d5/le 337632 1 2024-05-17T21:52:44.580 INFO:tasks.workunit.client.0.smithi107.stdout:5/120: write d5/db/dc/f20 [297518,19607] 0 2024-05-17T21:52:44.581 INFO:tasks.workunit.client.0.smithi107.stdout:0/152: symlink d3/dc/d31/l3c 0 2024-05-17T21:52:44.583 INFO:tasks.workunit.client.1.smithi138.stdout:4/37: mknod d0/d3/db/ce 0 2024-05-17T21:52:44.584 INFO:tasks.workunit.client.0.smithi107.stdout:2/101: unlink d0/d9/le 0 2024-05-17T21:52:44.586 INFO:tasks.workunit.client.1.smithi138.stdout:1/35: rmdir d4/d8 0 2024-05-17T21:52:44.587 INFO:tasks.workunit.client.1.smithi138.stdout:0/107: creat d3/f29 x:0 0 0 2024-05-17T21:52:44.589 INFO:tasks.workunit.client.1.smithi138.stdout:5/41: creat d1/f5 x:0 0 0 2024-05-17T21:52:44.591 INFO:tasks.workunit.client.0.smithi107.stdout:1/124: rename d4/df/d14/f16 to d4/df/d1a/f1f 0 2024-05-17T21:52:44.592 INFO:tasks.workunit.client.1.smithi138.stdout:7/20: creat d0/f5 x:0 0 0 2024-05-17T21:52:44.592 INFO:tasks.workunit.client.0.smithi107.stdout:1/125: chown d4/de/f19 0 1 2024-05-17T21:52:44.592 INFO:tasks.workunit.client.0.smithi107.stdout:6/128: fsync d9/d16/f24 0 2024-05-17T21:52:44.593 INFO:tasks.workunit.client.0.smithi107.stdout:8/125: creat dc/d21/d27/f37 x:0 0 0 2024-05-17T21:52:44.594 INFO:tasks.workunit.client.0.smithi107.stdout:9/143: mknod d0/c2e 0 2024-05-17T21:52:44.595 INFO:tasks.workunit.client.0.smithi107.stdout:7/114: write d5/f22 [303746,52465] 0 2024-05-17T21:52:44.596 INFO:tasks.workunit.client.1.smithi138.stdout:5/42: mkdir d1/d6 0 2024-05-17T21:52:44.597 INFO:tasks.workunit.client.1.smithi138.stdout:5/43: chown d1/f5 65 1 2024-05-17T21:52:44.597 INFO:tasks.workunit.client.1.smithi138.stdout:7/21: creat d0/f6 x:0 0 0 2024-05-17T21:52:44.599 INFO:tasks.workunit.client.1.smithi138.stdout:4/38: symlink d0/lf 0 2024-05-17T21:52:44.600 INFO:tasks.workunit.client.1.smithi138.stdout:4/39: write d0/f7 [1031941,111664] 0 2024-05-17T21:52:44.600 INFO:tasks.workunit.client.1.smithi138.stdout:0/108: mknod d3/c2a 0 2024-05-17T21:52:44.603 INFO:tasks.workunit.client.1.smithi138.stdout:5/44: creat d1/f7 x:0 0 0 2024-05-17T21:52:44.604 INFO:tasks.workunit.client.0.smithi107.stdout:0/153: mkdir d3/dc/d22/d3d 0 2024-05-17T21:52:44.605 INFO:tasks.workunit.client.1.smithi138.stdout:7/22: symlink d0/l7 0 2024-05-17T21:52:44.605 INFO:tasks.workunit.client.0.smithi107.stdout:0/154: write d3/f15 [538165,7019] 0 2024-05-17T21:52:44.606 INFO:tasks.workunit.client.1.smithi138.stdout:7/23: dread - d0/f5 zero size 2024-05-17T21:52:44.610 INFO:tasks.workunit.client.0.smithi107.stdout:4/130: rename l1 to d3/db/d1e/l2a 0 2024-05-17T21:52:44.613 INFO:tasks.workunit.client.0.smithi107.stdout:4/131: dread d3/db/dd/f10 [0,4194304] 0 2024-05-17T21:52:44.615 INFO:tasks.workunit.client.0.smithi107.stdout:6/129: fsync d9/f12 0 2024-05-17T21:52:44.616 INFO:tasks.workunit.client.0.smithi107.stdout:6/130: chown f1 221771028 1 2024-05-17T21:52:44.619 INFO:tasks.workunit.client.1.smithi138.stdout:4/40: rmdir d0/d3 39 2024-05-17T21:52:44.620 INFO:tasks.workunit.client.0.smithi107.stdout:7/115: chown d5/df/l14 1523007 1 2024-05-17T21:52:44.621 INFO:tasks.workunit.client.1.smithi138.stdout:5/45: fsync d1/f7 0 2024-05-17T21:52:44.622 INFO:tasks.workunit.client.1.smithi138.stdout:5/46: write d1/f7 [256252,17840] 0 2024-05-17T21:52:44.629 INFO:tasks.workunit.client.0.smithi107.stdout:0/155: creat d3/dc/d22/f3e x:0 0 0 2024-05-17T21:52:44.630 INFO:tasks.workunit.client.0.smithi107.stdout:5/121: truncate d5/f7 1996345 0 2024-05-17T21:52:44.632 INFO:tasks.workunit.client.0.smithi107.stdout:1/126: symlink d4/l20 0 2024-05-17T21:52:44.633 INFO:tasks.workunit.client.0.smithi107.stdout:4/132: creat d3/db/d16/f2b x:0 0 0 2024-05-17T21:52:44.634 INFO:tasks.workunit.client.0.smithi107.stdout:6/131: mknod d9/d16/c25 0 2024-05-17T21:52:44.634 INFO:tasks.workunit.client.0.smithi107.stdout:0/156: dwrite d3/f18 [0,4194304] 0 2024-05-17T21:52:44.635 INFO:tasks.workunit.client.0.smithi107.stdout:8/126: creat dc/d21/d27/d34/f38 x:0 0 0 2024-05-17T21:52:44.640 INFO:tasks.workunit.client.0.smithi107.stdout:0/157: truncate d3/f15 4930674 0 2024-05-17T21:52:44.640 INFO:tasks.workunit.client.0.smithi107.stdout:7/116: write d5/f18 [338810,32486] 0 2024-05-17T21:52:44.640 INFO:tasks.workunit.client.0.smithi107.stdout:8/127: chown dc/d21/d27/d34 264447 1 2024-05-17T21:52:44.642 INFO:tasks.workunit.client.0.smithi107.stdout:7/117: chown d5/df/d11/d2a 970993447 1 2024-05-17T21:52:44.643 INFO:tasks.workunit.client.0.smithi107.stdout:1/127: dwrite d4/f10 [0,4194304] 0 2024-05-17T21:52:44.644 INFO:tasks.workunit.client.0.smithi107.stdout:7/118: read d5/df/d11/f21 [2044105,108262] 0 2024-05-17T21:52:44.645 INFO:tasks.workunit.client.0.smithi107.stdout:7/119: read - d5/df/f26 zero size 2024-05-17T21:52:44.653 INFO:tasks.workunit.client.0.smithi107.stdout:4/133: read d3/db/f1f [253141,30121] 0 2024-05-17T21:52:44.661 INFO:tasks.workunit.client.0.smithi107.stdout:3/117: getdents d6/d8/d10/d25 0 2024-05-17T21:52:44.666 INFO:tasks.workunit.client.0.smithi107.stdout:0/158: mknod d3/d24/c3f 0 2024-05-17T21:52:44.670 INFO:tasks.workunit.client.0.smithi107.stdout:2/102: link d0/c3 d0/d9/d11/df/c1e 0 2024-05-17T21:52:44.673 INFO:tasks.workunit.client.0.smithi107.stdout:2/103: dread d0/d9/d11/f14 [0,4194304] 0 2024-05-17T21:52:44.675 INFO:tasks.workunit.client.0.smithi107.stdout:1/128: unlink d4/df/d1a/f1f 0 2024-05-17T21:52:44.676 INFO:tasks.workunit.client.0.smithi107.stdout:2/104: dread d0/fa [0,4194304] 0 2024-05-17T21:52:44.677 INFO:tasks.workunit.client.0.smithi107.stdout:7/120: chown d5/df/d11/d2a/c2d 267318545 1 2024-05-17T21:52:44.678 INFO:tasks.workunit.client.0.smithi107.stdout:4/134: creat d3/f2c x:0 0 0 2024-05-17T21:52:44.678 INFO:tasks.workunit.client.0.smithi107.stdout:4/135: dread - d3/f2c zero size 2024-05-17T21:52:44.679 INFO:tasks.workunit.client.0.smithi107.stdout:4/136: write d3/db/d16/f19 [2132005,10105] 0 2024-05-17T21:52:44.682 INFO:tasks.workunit.client.0.smithi107.stdout:3/118: write d6/f1c [4322935,68273] 0 2024-05-17T21:52:44.683 INFO:tasks.workunit.client.0.smithi107.stdout:5/122: write d5/f17 [1082941,84146] 0 2024-05-17T21:52:44.686 INFO:tasks.workunit.client.0.smithi107.stdout:6/132: dwrite d9/f1d [4194304,4194304] 0 2024-05-17T21:52:44.686 INFO:tasks.workunit.client.0.smithi107.stdout:0/159: write d3/dc/d22/d23/f29 [3801872,65676] 0 2024-05-17T21:52:44.689 INFO:tasks.workunit.client.0.smithi107.stdout:6/133: write d9/f12 [807699,100937] 0 2024-05-17T21:52:44.689 INFO:tasks.workunit.client.0.smithi107.stdout:0/160: chown d3/dc/f12 0 1 2024-05-17T21:52:44.689 INFO:tasks.workunit.client.0.smithi107.stdout:9/144: dwrite d0/f4 [0,4194304] 0 2024-05-17T21:52:44.692 INFO:tasks.workunit.client.1.smithi138.stdout:2/5: sync 2024-05-17T21:52:44.692 INFO:tasks.workunit.client.1.smithi138.stdout:2/6: readlink - no filename 2024-05-17T21:52:44.693 INFO:tasks.workunit.client.0.smithi107.stdout:3/119: dread d6/f1b [0,4194304] 0 2024-05-17T21:52:44.696 INFO:tasks.workunit.client.0.smithi107.stdout:2/105: write d0/fa [63663,33893] 0 2024-05-17T21:52:44.702 INFO:tasks.workunit.client.0.smithi107.stdout:8/128: truncate dc/d10/f15 284595 0 2024-05-17T21:52:44.712 INFO:tasks.workunit.client.0.smithi107.stdout:5/123: dwrite d5/f19 [0,4194304] 0 2024-05-17T21:52:44.713 INFO:tasks.workunit.client.0.smithi107.stdout:0/161: creat d3/dc/d22/f40 x:0 0 0 2024-05-17T21:52:44.717 INFO:tasks.workunit.client.0.smithi107.stdout:4/137: write d3/db/dd/f10 [3737929,59133] 0 2024-05-17T21:52:44.725 INFO:tasks.workunit.client.0.smithi107.stdout:8/129: mknod dc/d21/c39 0 2024-05-17T21:52:44.728 INFO:tasks.workunit.client.0.smithi107.stdout:6/134: symlink d9/l26 0 2024-05-17T21:52:44.729 INFO:tasks.workunit.client.0.smithi107.stdout:6/135: dread - d9/f1f zero size 2024-05-17T21:52:44.731 INFO:tasks.workunit.client.0.smithi107.stdout:9/145: mkdir d0/dd/d2f 0 2024-05-17T21:52:44.734 INFO:tasks.workunit.client.0.smithi107.stdout:6/136: dwrite d9/ff [0,4194304] 0 2024-05-17T21:52:44.740 INFO:tasks.workunit.client.0.smithi107.stdout:4/138: rename d3/d6/l28 to d3/db/d1e/l2d 0 2024-05-17T21:52:44.741 INFO:tasks.workunit.client.0.smithi107.stdout:4/139: write d3/db/f11 [4317668,89534] 0 2024-05-17T21:52:44.741 INFO:tasks.workunit.client.0.smithi107.stdout:1/129: creat d4/f21 x:0 0 0 2024-05-17T21:52:44.742 INFO:tasks.workunit.client.0.smithi107.stdout:4/140: write d3/db/d1e/f20 [3479021,24186] 0 2024-05-17T21:52:44.743 INFO:tasks.workunit.client.0.smithi107.stdout:2/106: symlink d0/d9/d1c/l1f 0 2024-05-17T21:52:44.744 INFO:tasks.workunit.client.0.smithi107.stdout:2/107: chown d0/f2 2 1 2024-05-17T21:52:44.748 INFO:tasks.workunit.client.0.smithi107.stdout:3/120: truncate d6/fc 782916 0 2024-05-17T21:52:44.751 INFO:tasks.workunit.client.0.smithi107.stdout:5/124: creat d5/db/dc/d18/f27 x:0 0 0 2024-05-17T21:52:44.751 INFO:tasks.workunit.client.0.smithi107.stdout:5/125: readlink d5/db/d16/l1d 0 2024-05-17T21:52:44.752 INFO:tasks.workunit.client.0.smithi107.stdout:5/126: read d5/db/dc/f24 [39597,77214] 0 2024-05-17T21:52:44.753 INFO:tasks.workunit.client.0.smithi107.stdout:0/162: creat d3/d39/f41 x:0 0 0 2024-05-17T21:52:44.754 INFO:tasks.workunit.client.0.smithi107.stdout:0/163: read d3/f17 [3620022,95940] 0 2024-05-17T21:52:44.757 INFO:tasks.workunit.client.0.smithi107.stdout:8/130: rename dc/d10/c31 to dc/d21/d27/d34/c3a 0 2024-05-17T21:52:44.757 INFO:tasks.workunit.client.0.smithi107.stdout:6/137: dread d9/d16/f18 [0,4194304] 0 2024-05-17T21:52:44.758 INFO:tasks.workunit.client.0.smithi107.stdout:6/138: dread d9/f1c [0,4194304] 0 2024-05-17T21:52:44.764 INFO:tasks.workunit.client.0.smithi107.stdout:4/141: symlink d3/db/dd/l2e 0 2024-05-17T21:52:44.768 INFO:tasks.workunit.client.0.smithi107.stdout:7/121: link d5/lc d5/df/l30 0 2024-05-17T21:52:44.770 INFO:tasks.workunit.client.0.smithi107.stdout:3/121: dread d6/d8/f17 [0,4194304] 0 2024-05-17T21:52:44.771 INFO:tasks.workunit.client.0.smithi107.stdout:9/146: creat d0/dd/d2f/f30 x:0 0 0 2024-05-17T21:52:44.771 INFO:tasks.workunit.client.0.smithi107.stdout:3/122: write d6/d8/f14 [1028064,1013] 0 2024-05-17T21:52:44.778 INFO:tasks.workunit.client.0.smithi107.stdout:0/164: mknod d3/dc/c42 0 2024-05-17T21:52:44.778 INFO:tasks.workunit.client.0.smithi107.stdout:0/165: chown f1 869256 1 2024-05-17T21:52:44.779 INFO:tasks.workunit.client.0.smithi107.stdout:8/131: creat dc/d10/d17/f3b x:0 0 0 2024-05-17T21:52:44.782 INFO:tasks.workunit.client.0.smithi107.stdout:6/139: creat d9/d16/d22/f27 x:0 0 0 2024-05-17T21:52:44.783 INFO:tasks.workunit.client.0.smithi107.stdout:6/140: chown d9/d16/f24 2018238802 1 2024-05-17T21:52:44.786 INFO:tasks.workunit.client.0.smithi107.stdout:2/108: truncate d0/d9/d11/f10 1975647 0 2024-05-17T21:52:44.789 INFO:tasks.workunit.client.0.smithi107.stdout:9/147: stat d0/f14 0 2024-05-17T21:52:44.789 INFO:tasks.workunit.client.0.smithi107.stdout:9/148: chown d0/dd/d1f 328720 1 2024-05-17T21:52:44.790 INFO:tasks.workunit.client.0.smithi107.stdout:7/122: dread d5/df/f1a [0,4194304] 0 2024-05-17T21:52:44.791 INFO:tasks.workunit.client.0.smithi107.stdout:7/123: stat d5/df/d11/d1c/d2f 0 2024-05-17T21:52:44.792 INFO:tasks.workunit.client.0.smithi107.stdout:3/123: mkdir d6/d2a 0 2024-05-17T21:52:44.798 INFO:tasks.workunit.client.0.smithi107.stdout:8/132: mknod dc/d21/d35/c3c 0 2024-05-17T21:52:44.801 INFO:tasks.workunit.client.0.smithi107.stdout:1/130: creat d4/df/f22 x:0 0 0 2024-05-17T21:52:44.802 INFO:tasks.workunit.client.0.smithi107.stdout:1/131: read d4/f10 [296932,25131] 0 2024-05-17T21:52:44.802 INFO:tasks.workunit.client.0.smithi107.stdout:1/132: read - d4/df/f22 zero size 2024-05-17T21:52:44.803 INFO:tasks.workunit.client.0.smithi107.stdout:6/141: rename d9/ff to d9/d16/d22/f28 0 2024-05-17T21:52:44.805 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:44 smithi138 ceph-mon[35948]: pgmap v30: 65 pgs: 65 active+clean; 283 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.1 MiB/s rd, 12 MiB/s wr, 337 op/s 2024-05-17T21:52:44.805 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:44 smithi138 ceph-mon[35948]: from='mgr.14636 ' entity='mgr.smithi138.napnll' 2024-05-17T21:52:44.805 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:44 smithi138 ceph-mon[35948]: from='mgr.14636 ' entity='mgr.smithi138.napnll' 2024-05-17T21:52:44.806 INFO:tasks.workunit.client.0.smithi107.stdout:9/149: creat d0/dd/d2f/f31 x:0 0 0 2024-05-17T21:52:44.808 INFO:tasks.workunit.client.0.smithi107.stdout:5/127: creat d5/f28 x:0 0 0 2024-05-17T21:52:44.811 INFO:tasks.workunit.client.0.smithi107.stdout:1/133: dwrite d4/fa [0,4194304] 0 2024-05-17T21:52:44.813 INFO:tasks.workunit.client.0.smithi107.stdout:2/109: dwrite d0/d9/d11/f14 [4194304,4194304] 0 2024-05-17T21:52:44.820 INFO:tasks.workunit.client.0.smithi107.stdout:2/110: dwrite d0/d9/d11/f14 [4194304,4194304] 0 2024-05-17T21:52:44.822 INFO:tasks.workunit.client.0.smithi107.stdout:2/111: dread d0/fa [0,4194304] 0 2024-05-17T21:52:44.823 INFO:tasks.workunit.client.0.smithi107.stdout:2/112: write d0/fa [386523,50929] 0 2024-05-17T21:52:44.834 INFO:tasks.workunit.client.0.smithi107.stdout:5/128: rename d5/db/l1c to d5/db/dc/l29 0 2024-05-17T21:52:44.841 INFO:tasks.workunit.client.0.smithi107.stdout:1/134: dread d4/fc [0,4194304] 0 2024-05-17T21:52:44.848 INFO:tasks.workunit.client.0.smithi107.stdout:3/124: link d6/d8/ld d6/d2a/l2b 0 2024-05-17T21:52:44.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:44 smithi107 ceph-mon[28434]: pgmap v30: 65 pgs: 65 active+clean; 283 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.1 MiB/s rd, 12 MiB/s wr, 337 op/s 2024-05-17T21:52:44.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:44 smithi107 ceph-mon[28434]: from='mgr.14636 ' entity='mgr.smithi138.napnll' 2024-05-17T21:52:44.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:44 smithi107 ceph-mon[28434]: from='mgr.14636 ' entity='mgr.smithi138.napnll' 2024-05-17T21:52:44.854 INFO:tasks.workunit.client.0.smithi107.stdout:0/166: getdents d3/dc/d31 0 2024-05-17T21:52:44.856 INFO:tasks.workunit.client.0.smithi107.stdout:2/113: creat d0/d9/d11/d18/f20 x:0 0 0 2024-05-17T21:52:44.865 INFO:tasks.workunit.client.0.smithi107.stdout:1/135: creat d4/df/d14/f23 x:0 0 0 2024-05-17T21:52:44.867 INFO:tasks.workunit.client.0.smithi107.stdout:3/125: readlink d6/d8/l28 0 2024-05-17T21:52:44.867 INFO:tasks.workunit.client.0.smithi107.stdout:6/142: write d9/f15 [610270,67641] 0 2024-05-17T21:52:44.868 INFO:tasks.workunit.client.1.smithi138.stdout:3/20: sync 2024-05-17T21:52:44.869 INFO:tasks.workunit.client.1.smithi138.stdout:3/21: write d1/f2 [764282,72432] 0 2024-05-17T21:52:44.870 INFO:tasks.workunit.client.1.smithi138.stdout:3/22: write d1/f2 [1513348,115870] 0 2024-05-17T21:52:44.871 INFO:tasks.workunit.client.0.smithi107.stdout:7/124: truncate f4 4134230 0 2024-05-17T21:52:44.872 INFO:tasks.workunit.client.0.smithi107.stdout:7/125: dread - d5/d16/f28 zero size 2024-05-17T21:52:44.873 INFO:tasks.workunit.client.0.smithi107.stdout:3/126: dwrite f5 [0,4194304] 0 2024-05-17T21:52:44.873 INFO:tasks.workunit.client.0.smithi107.stdout:0/167: creat d3/f43 x:0 0 0 2024-05-17T21:52:44.874 INFO:tasks.workunit.client.0.smithi107.stdout:0/168: dread - d3/dc/d22/f3e zero size 2024-05-17T21:52:44.877 INFO:tasks.workunit.client.1.smithi138.stdout:3/23: dwrite d1/f2 [0,4194304] 0 2024-05-17T21:52:44.887 INFO:tasks.workunit.client.0.smithi107.stdout:6/143: write d9/f10 [2148428,31355] 0 2024-05-17T21:52:44.892 INFO:tasks.workunit.client.0.smithi107.stdout:3/127: creat d6/f2c x:0 0 0 2024-05-17T21:52:44.894 INFO:tasks.workunit.client.0.smithi107.stdout:7/126: dread d5/df/f10 [0,4194304] 0 2024-05-17T21:52:44.895 INFO:tasks.workunit.client.0.smithi107.stdout:2/114: mknod d0/d9/d11/d15/c21 0 2024-05-17T21:52:44.895 INFO:tasks.workunit.client.0.smithi107.stdout:7/127: chown d5/df/d11/d2a/c2d 875 1 2024-05-17T21:52:44.898 INFO:tasks.workunit.client.0.smithi107.stdout:1/136: mknod d4/df/c24 0 2024-05-17T21:52:44.898 INFO:tasks.workunit.client.0.smithi107.stdout:1/137: readlink l2 0 2024-05-17T21:52:44.899 INFO:tasks.workunit.client.0.smithi107.stdout:6/144: creat d9/d16/f29 x:0 0 0 2024-05-17T21:52:44.901 INFO:tasks.workunit.client.0.smithi107.stdout:9/150: truncate d0/dd/f1e 5232112 0 2024-05-17T21:52:44.901 INFO:tasks.workunit.client.0.smithi107.stdout:9/151: stat d0 0 2024-05-17T21:52:44.902 INFO:tasks.workunit.client.0.smithi107.stdout:9/152: dread - d0/dd/f13 zero size 2024-05-17T21:52:44.905 INFO:tasks.workunit.client.1.smithi138.stdout:8/22: sync 2024-05-17T21:52:44.905 INFO:tasks.workunit.client.1.smithi138.stdout:1/36: sync 2024-05-17T21:52:44.905 INFO:tasks.workunit.client.1.smithi138.stdout:6/107: sync 2024-05-17T21:52:44.905 INFO:tasks.workunit.client.1.smithi138.stdout:9/19: sync 2024-05-17T21:52:44.906 INFO:tasks.workunit.client.1.smithi138.stdout:8/23: chown d0/f2 14 1 2024-05-17T21:52:44.906 INFO:tasks.workunit.client.0.smithi107.stdout:2/115: mkdir d0/d9/d1c/d22 0 2024-05-17T21:52:44.910 INFO:tasks.workunit.client.1.smithi138.stdout:1/37: dread f2 [0,4194304] 0 2024-05-17T21:52:44.913 INFO:tasks.workunit.client.0.smithi107.stdout:7/128: dwrite d5/df/d11/f21 [0,4194304] 0 2024-05-17T21:52:44.915 INFO:tasks.workunit.client.1.smithi138.stdout:8/24: dwrite d0/f1 [0,4194304] 0 2024-05-17T21:52:44.916 INFO:tasks.workunit.client.0.smithi107.stdout:9/153: read d0/fe [3307517,6811] 0 2024-05-17T21:52:44.918 INFO:tasks.workunit.client.0.smithi107.stdout:3/128: mkdir d6/d2a/d2d 0 2024-05-17T21:52:44.920 INFO:tasks.workunit.client.0.smithi107.stdout:5/129: write d5/f9 [1258671,38393] 0 2024-05-17T21:52:44.921 INFO:tasks.workunit.client.0.smithi107.stdout:5/130: fsync d5/f17 0 2024-05-17T21:52:44.923 INFO:tasks.workunit.client.0.smithi107.stdout:3/129: dwrite f5 [0,4194304] 0 2024-05-17T21:52:44.926 INFO:tasks.workunit.client.0.smithi107.stdout:2/116: symlink d0/d9/d11/d18/l23 0 2024-05-17T21:52:44.926 INFO:tasks.workunit.client.0.smithi107.stdout:2/117: chown d0/fa 39947245 1 2024-05-17T21:52:44.935 INFO:tasks.workunit.client.0.smithi107.stdout:1/138: rmdir d4/df 39 2024-05-17T21:52:44.937 INFO:tasks.workunit.client.0.smithi107.stdout:5/131: chown f3 0 1 2024-05-17T21:52:44.938 INFO:tasks.workunit.client.0.smithi107.stdout:3/130: mknod d6/c2e 0 2024-05-17T21:52:44.939 INFO:tasks.workunit.client.0.smithi107.stdout:3/131: truncate f2 5817916 0 2024-05-17T21:52:44.940 INFO:tasks.workunit.client.0.smithi107.stdout:3/132: readlink d6/d1a/l24 0 2024-05-17T21:52:44.956 INFO:tasks.workunit.client.1.smithi138.stdout:7/24: creat d0/f8 x:0 0 0 2024-05-17T21:52:44.956 INFO:tasks.workunit.client.1.smithi138.stdout:7/25: dread - d0/f6 zero size 2024-05-17T21:52:44.956 INFO:tasks.workunit.client.1.smithi138.stdout:7/26: dread - d0/f5 zero size 2024-05-17T21:52:44.957 INFO:tasks.workunit.client.1.smithi138.stdout:7/27: read - d0/f6 zero size 2024-05-17T21:52:44.957 INFO:tasks.workunit.client.1.smithi138.stdout:5/47: symlink d1/l8 0 2024-05-17T21:52:44.958 INFO:tasks.workunit.client.1.smithi138.stdout:7/28: write d0/f5 [504696,20709] 0 2024-05-17T21:52:44.959 INFO:tasks.workunit.client.1.smithi138.stdout:2/7: unlink c1 0 2024-05-17T21:52:44.961 INFO:tasks.workunit.client.1.smithi138.stdout:5/48: dread d1/f7 [0,4194304] 0 2024-05-17T21:52:44.962 INFO:tasks.workunit.client.1.smithi138.stdout:5/49: truncate d1/f5 988130 0 2024-05-17T21:52:44.962 INFO:tasks.workunit.client.1.smithi138.stdout:3/24: mkdir d1/d3 0 2024-05-17T21:52:44.963 INFO:tasks.workunit.client.1.smithi138.stdout:6/108: mknod d7/d8/c1e 0 2024-05-17T21:52:44.966 INFO:tasks.workunit.client.1.smithi138.stdout:0/109: creat d3/f2b x:0 0 0 2024-05-17T21:52:44.968 INFO:tasks.workunit.client.1.smithi138.stdout:7/29: creat d0/f9 x:0 0 0 2024-05-17T21:52:44.969 INFO:tasks.workunit.client.1.smithi138.stdout:2/8: mkdir d3 0 2024-05-17T21:52:44.971 INFO:tasks.workunit.client.1.smithi138.stdout:3/25: creat d1/f4 x:0 0 0 2024-05-17T21:52:44.972 INFO:tasks.workunit.client.1.smithi138.stdout:3/26: chown d1 11961622 1 2024-05-17T21:52:44.976 INFO:tasks.workunit.client.1.smithi138.stdout:7/30: dwrite d0/f8 [0,4194304] 0 2024-05-17T21:52:44.977 INFO:tasks.workunit.client.0.smithi107.stdout:4/142: sync 2024-05-17T21:52:44.978 INFO:tasks.workunit.client.1.smithi138.stdout:5/50: symlink d1/d6/l9 0 2024-05-17T21:52:44.981 INFO:tasks.workunit.client.0.smithi107.stdout:9/154: rename d0/dd/f1e to d0/dd/d1b/f32 0 2024-05-17T21:52:44.981 INFO:tasks.workunit.client.0.smithi107.stdout:5/132: dwrite d5/f9 [0,4194304] 0 2024-05-17T21:52:44.981 INFO:tasks.workunit.client.1.smithi138.stdout:5/51: truncate d1/f5 1147885 0 2024-05-17T21:52:44.982 INFO:tasks.workunit.client.0.smithi107.stdout:1/139: stat d4/df/d14/d1d 0 2024-05-17T21:52:44.984 INFO:tasks.workunit.client.1.smithi138.stdout:3/27: mknod d1/c5 0 2024-05-17T21:52:44.984 INFO:tasks.workunit.client.0.smithi107.stdout:1/140: write d4/de/f1e [1013573,69188] 0 2024-05-17T21:52:44.985 INFO:tasks.workunit.client.0.smithi107.stdout:1/141: chown d4/df/c24 355643474 1 2024-05-17T21:52:44.986 INFO:tasks.workunit.client.1.smithi138.stdout:3/28: fdatasync d1/f4 0 2024-05-17T21:52:44.987 INFO:tasks.workunit.client.1.smithi138.stdout:7/31: dwrite d0/f8 [0,4194304] 0 2024-05-17T21:52:44.988 INFO:tasks.workunit.client.1.smithi138.stdout:3/29: fdatasync d1/f4 0 2024-05-17T21:52:44.989 INFO:tasks.workunit.client.1.smithi138.stdout:3/30: dread - d1/f4 zero size 2024-05-17T21:52:44.989 INFO:tasks.workunit.client.1.smithi138.stdout:7/32: dread - d0/f9 zero size 2024-05-17T21:52:44.990 INFO:tasks.workunit.client.1.smithi138.stdout:7/33: fsync d0/f6 0 2024-05-17T21:52:44.992 INFO:tasks.workunit.client.1.smithi138.stdout:0/110: link d3/f21 d3/d10/f2c 0 2024-05-17T21:52:45.001 INFO:tasks.workunit.client.1.smithi138.stdout:3/31: dwrite d1/f4 [0,4194304] 0 2024-05-17T21:52:45.005 INFO:tasks.workunit.client.0.smithi107.stdout:4/143: unlink d3/db/dd/l1b 0 2024-05-17T21:52:45.009 INFO:tasks.workunit.client.1.smithi138.stdout:5/52: creat d1/d6/fa x:0 0 0 2024-05-17T21:52:45.011 INFO:tasks.workunit.client.0.smithi107.stdout:5/133: mkdir d5/db/d16/d2a 0 2024-05-17T21:52:45.012 INFO:tasks.workunit.client.0.smithi107.stdout:4/144: dwrite d3/db/d16/f2b [0,4194304] 0 2024-05-17T21:52:45.013 INFO:tasks.workunit.client.0.smithi107.stdout:4/145: write d3/f22 [457271,104366] 0 2024-05-17T21:52:45.014 INFO:tasks.workunit.client.1.smithi138.stdout:7/34: creat d0/fa x:0 0 0 2024-05-17T21:52:45.015 INFO:tasks.workunit.client.1.smithi138.stdout:4/41: truncate d0/f7 665426 0 2024-05-17T21:52:45.022 INFO:tasks.workunit.client.0.smithi107.stdout:3/133: link d6/d8/c15 d6/d8/d10/d25/c2f 0 2024-05-17T21:52:45.023 INFO:tasks.workunit.client.1.smithi138.stdout:9/20: truncate f2 4157205 0 2024-05-17T21:52:45.027 INFO:tasks.workunit.client.1.smithi138.stdout:5/53: unlink d1/d6/l9 0 2024-05-17T21:52:45.029 INFO:tasks.workunit.client.1.smithi138.stdout:6/109: getdents d7/db 0 2024-05-17T21:52:45.033 INFO:tasks.workunit.client.0.smithi107.stdout:9/155: dwrite d0/fe [4194304,4194304] 0 2024-05-17T21:52:45.037 INFO:tasks.workunit.client.1.smithi138.stdout:2/9: truncate f2 785124 0 2024-05-17T21:52:45.041 INFO:tasks.workunit.client.1.smithi138.stdout:0/111: link d3/d10/f22 d3/d10/f2d 0 2024-05-17T21:52:45.044 INFO:tasks.workunit.client.0.smithi107.stdout:5/134: mkdir d5/db/d16/d2a/d2b 0 2024-05-17T21:52:45.046 INFO:tasks.workunit.client.1.smithi138.stdout:5/54: creat d1/d6/fb x:0 0 0 2024-05-17T21:52:45.046 INFO:tasks.workunit.client.1.smithi138.stdout:5/55: chown d1/d6 68 1 2024-05-17T21:52:45.051 INFO:tasks.workunit.client.1.smithi138.stdout:6/110: dread f3 [0,4194304] 0 2024-05-17T21:52:45.051 INFO:tasks.workunit.client.0.smithi107.stdout:9/156: rename d0/c2e to d0/dd/d1f/d2c/c33 0 2024-05-17T21:52:45.056 INFO:tasks.workunit.client.1.smithi138.stdout:7/35: unlink d0/f8 0 2024-05-17T21:52:45.057 INFO:tasks.workunit.client.1.smithi138.stdout:7/36: write d0/f5 [1329714,37137] 0 2024-05-17T21:52:45.058 INFO:tasks.workunit.client.0.smithi107.stdout:3/134: link d6/f2c d6/d8/d10/f30 0 2024-05-17T21:52:45.058 INFO:tasks.workunit.client.1.smithi138.stdout:3/32: dwrite d1/f2 [4194304,4194304] 0 2024-05-17T21:52:45.059 INFO:tasks.workunit.client.1.smithi138.stdout:3/33: fsync d1/f4 0 2024-05-17T21:52:45.059 INFO:tasks.workunit.client.1.smithi138.stdout:3/34: readlink - no filename 2024-05-17T21:52:45.063 INFO:tasks.workunit.client.0.smithi107.stdout:1/142: getdents d4/df/d14 0 2024-05-17T21:52:45.072 INFO:tasks.workunit.client.0.smithi107.stdout:7/129: dwrite f4 [0,4194304] 0 2024-05-17T21:52:45.074 INFO:tasks.workunit.client.0.smithi107.stdout:1/143: symlink d4/df/d14/l25 0 2024-05-17T21:52:45.076 INFO:tasks.workunit.client.1.smithi138.stdout:0/112: fsync d3/d10/f2d 0 2024-05-17T21:52:45.076 INFO:tasks.workunit.client.0.smithi107.stdout:7/130: dread f4 [0,4194304] 0 2024-05-17T21:52:45.077 INFO:tasks.workunit.client.0.smithi107.stdout:4/146: getdents d3/d6/d26 0 2024-05-17T21:52:45.078 INFO:tasks.workunit.client.1.smithi138.stdout:7/37: mknod d0/cb 0 2024-05-17T21:52:45.079 INFO:tasks.workunit.client.0.smithi107.stdout:8/133: sync 2024-05-17T21:52:45.080 INFO:tasks.workunit.client.0.smithi107.stdout:8/134: fdatasync dc/d21/d35/f11 0 2024-05-17T21:52:45.083 INFO:tasks.workunit.client.1.smithi138.stdout:4/42: write d0/f7 [1103534,63486] 0 2024-05-17T21:52:45.085 INFO:tasks.workunit.client.0.smithi107.stdout:9/157: dread d0/dd/d1b/f32 [0,4194304] 0 2024-05-17T21:52:45.085 INFO:tasks.workunit.client.0.smithi107.stdout:1/144: mkdir d4/df/d1a/d26 0 2024-05-17T21:52:45.087 INFO:tasks.workunit.client.0.smithi107.stdout:7/131: mkdir d5/df/d11/d1c/d31 0 2024-05-17T21:52:45.089 INFO:tasks.workunit.client.0.smithi107.stdout:8/135: dwrite dc/d21/d27/d34/f38 [0,4194304] 0 2024-05-17T21:52:45.090 INFO:tasks.workunit.client.0.smithi107.stdout:7/132: stat d5/df/d11/f25 0 2024-05-17T21:52:45.092 INFO:tasks.workunit.client.0.smithi107.stdout:8/136: write dc/d21/d35/f2e [118470,40907] 0 2024-05-17T21:52:45.093 INFO:tasks.workunit.client.1.smithi138.stdout:6/111: rename d7/db/c1c to d7/db/d12/c1f 0 2024-05-17T21:52:45.094 INFO:tasks.workunit.client.0.smithi107.stdout:5/135: dwrite f3 [0,4194304] 0 2024-05-17T21:52:45.098 INFO:tasks.workunit.client.0.smithi107.stdout:3/135: truncate d6/f1b 2111089 0 2024-05-17T21:52:45.098 INFO:tasks.workunit.client.1.smithi138.stdout:0/113: mknod d3/d10/c2e 0 2024-05-17T21:52:45.099 INFO:tasks.workunit.client.1.smithi138.stdout:0/114: stat d3/d10/l13 0 2024-05-17T21:52:45.104 INFO:tasks.workunit.client.0.smithi107.stdout:5/136: symlink d5/db/d16/d2a/l2c 0 2024-05-17T21:52:45.105 INFO:tasks.workunit.client.0.smithi107.stdout:5/137: fdatasync d5/db/dc/f20 0 2024-05-17T21:52:45.107 INFO:tasks.workunit.client.0.smithi107.stdout:5/138: chown d5/db/d16/d2a 1 1 2024-05-17T21:52:45.107 INFO:tasks.workunit.client.0.smithi107.stdout:4/147: dwrite d3/db/f1f [0,4194304] 0 2024-05-17T21:52:45.108 INFO:tasks.workunit.client.1.smithi138.stdout:9/21: getdents . 0 2024-05-17T21:52:45.108 INFO:tasks.workunit.client.1.smithi138.stdout:9/22: readlink - no filename 2024-05-17T21:52:45.108 INFO:tasks.workunit.client.0.smithi107.stdout:9/158: rename d0/f22 to d0/dd/d1f/d2c/f34 0 2024-05-17T21:52:45.109 INFO:tasks.workunit.client.0.smithi107.stdout:3/136: mknod d6/c31 0 2024-05-17T21:52:45.111 INFO:tasks.workunit.client.0.smithi107.stdout:1/145: creat d4/df/d14/d1d/f27 x:0 0 0 2024-05-17T21:52:45.112 INFO:tasks.workunit.client.0.smithi107.stdout:1/146: stat d4/de 0 2024-05-17T21:52:45.114 INFO:tasks.workunit.client.0.smithi107.stdout:1/147: dread - d4/df/d14/d1d/f27 zero size 2024-05-17T21:52:45.116 INFO:tasks.workunit.client.0.smithi107.stdout:3/137: dwrite f5 [0,4194304] 0 2024-05-17T21:52:45.120 INFO:tasks.workunit.client.1.smithi138.stdout:7/38: link d0/f5 d0/fc 0 2024-05-17T21:52:45.122 INFO:tasks.workunit.client.1.smithi138.stdout:4/43: mkdir d0/d3/d10 0 2024-05-17T21:52:45.123 INFO:tasks.workunit.client.1.smithi138.stdout:4/44: truncate d0/f7 1310934 0 2024-05-17T21:52:45.124 INFO:tasks.workunit.client.0.smithi107.stdout:5/139: mkdir d5/db/d16/d2d 0 2024-05-17T21:52:45.124 INFO:tasks.workunit.client.1.smithi138.stdout:4/45: write d0/f7 [1444774,121478] 0 2024-05-17T21:52:45.128 INFO:tasks.workunit.client.1.smithi138.stdout:9/23: mkdir d6 0 2024-05-17T21:52:45.128 INFO:tasks.workunit.client.0.smithi107.stdout:9/159: write d0/f17 [528873,10552] 0 2024-05-17T21:52:45.129 INFO:tasks.workunit.client.1.smithi138.stdout:5/56: getdents d1/d6 0 2024-05-17T21:52:45.130 INFO:tasks.workunit.client.0.smithi107.stdout:0/169: sync 2024-05-17T21:52:45.130 INFO:tasks.workunit.client.0.smithi107.stdout:7/133: rename d5/l6 to d5/d16/l32 0 2024-05-17T21:52:45.130 INFO:tasks.workunit.client.1.smithi138.stdout:5/57: read d1/f5 [478037,95638] 0 2024-05-17T21:52:45.131 INFO:tasks.workunit.client.0.smithi107.stdout:7/134: readlink d5/df/l30 0 2024-05-17T21:52:45.132 INFO:tasks.workunit.client.1.smithi138.stdout:4/46: dread d0/f7 [0,4194304] 0 2024-05-17T21:52:45.132 INFO:tasks.workunit.client.0.smithi107.stdout:5/140: dwrite d5/f23 [0,4194304] 0 2024-05-17T21:52:45.132 INFO:tasks.workunit.client.0.smithi107.stdout:0/170: truncate d3/f11 1296908 0 2024-05-17T21:52:45.134 INFO:tasks.workunit.client.0.smithi107.stdout:8/137: link dc/d21/d35/c33 dc/d21/d27/d34/c3d 0 2024-05-17T21:52:45.134 INFO:tasks.workunit.client.1.smithi138.stdout:2/10: truncate f2 933636 0 2024-05-17T21:52:45.135 INFO:tasks.workunit.client.0.smithi107.stdout:1/148: write d4/fc [1461878,47037] 0 2024-05-17T21:52:45.137 INFO:tasks.workunit.client.1.smithi138.stdout:0/115: mknod d3/c2f 0 2024-05-17T21:52:45.138 INFO:tasks.workunit.client.1.smithi138.stdout:6/112: truncate d7/fa 3200820 0 2024-05-17T21:52:45.138 INFO:tasks.workunit.client.0.smithi107.stdout:0/171: dread d3/dc/d22/f2d [0,4194304] 0 2024-05-17T21:52:45.139 INFO:tasks.workunit.client.1.smithi138.stdout:6/113: chown l5 2 1 2024-05-17T21:52:45.140 INFO:tasks.workunit.client.1.smithi138.stdout:0/116: dread f2 [4194304,4194304] 0 2024-05-17T21:52:45.140 INFO:tasks.workunit.client.1.smithi138.stdout:0/117: dread - d3/d10/f25 zero size 2024-05-17T21:52:45.143 INFO:tasks.workunit.client.1.smithi138.stdout:9/24: fdatasync f3 0 2024-05-17T21:52:45.147 INFO:tasks.workunit.client.0.smithi107.stdout:9/160: creat d0/dd/f35 x:0 0 0 2024-05-17T21:52:45.148 INFO:tasks.workunit.client.0.smithi107.stdout:4/148: truncate d3/db/d1e/f20 1187707 0 2024-05-17T21:52:45.148 INFO:tasks.workunit.client.1.smithi138.stdout:5/58: fsync d1/f7 0 2024-05-17T21:52:45.150 INFO:tasks.workunit.client.1.smithi138.stdout:4/47: rmdir d0/d3 39 2024-05-17T21:52:45.154 INFO:tasks.workunit.client.0.smithi107.stdout:5/141: rename d5/db/d16/c14 to d5/db/d16/d2a/d2b/c2e 0 2024-05-17T21:52:45.156 INFO:tasks.workunit.client.1.smithi138.stdout:6/114: read d7/fc [103888,41648] 0 2024-05-17T21:52:45.157 INFO:tasks.workunit.client.0.smithi107.stdout:3/138: symlink d6/d8/l32 0 2024-05-17T21:52:45.157 INFO:tasks.workunit.client.0.smithi107.stdout:3/139: chown d6/d8/f12 154 1 2024-05-17T21:52:45.160 INFO:tasks.workunit.client.0.smithi107.stdout:1/149: creat d4/df/d1a/f28 x:0 0 0 2024-05-17T21:52:45.163 INFO:tasks.workunit.client.0.smithi107.stdout:0/172: creat d3/dc/d22/d23/f44 x:0 0 0 2024-05-17T21:52:45.164 INFO:tasks.workunit.client.0.smithi107.stdout:9/161: readlink d0/l1a 0 2024-05-17T21:52:45.164 INFO:tasks.workunit.client.0.smithi107.stdout:9/162: read d0/fe [4359070,104567] 0 2024-05-17T21:52:45.165 INFO:tasks.workunit.client.0.smithi107.stdout:2/118: sync 2024-05-17T21:52:45.165 INFO:tasks.workunit.client.0.smithi107.stdout:6/145: sync 2024-05-17T21:52:45.165 INFO:tasks.workunit.client.0.smithi107.stdout:9/163: write d0/fe [5375149,67734] 0 2024-05-17T21:52:45.165 INFO:tasks.workunit.client.0.smithi107.stdout:4/149: creat d3/d6/d26/f2f x:0 0 0 2024-05-17T21:52:45.167 INFO:tasks.workunit.client.0.smithi107.stdout:2/119: chown d0/d9/d11/d18/l23 11834 1 2024-05-17T21:52:45.169 INFO:tasks.workunit.client.0.smithi107.stdout:6/146: dread d9/f1a [0,4194304] 0 2024-05-17T21:52:45.171 INFO:tasks.workunit.client.0.smithi107.stdout:3/140: creat d6/f33 x:0 0 0 2024-05-17T21:52:45.171 INFO:tasks.workunit.client.0.smithi107.stdout:3/141: chown d6/d2a 3941 1 2024-05-17T21:52:45.171 INFO:tasks.workunit.client.0.smithi107.stdout:3/142: dread - d6/f2c zero size 2024-05-17T21:52:45.172 INFO:tasks.workunit.client.1.smithi138.stdout:2/11: creat d3/f4 x:0 0 0 2024-05-17T21:52:45.172 INFO:tasks.workunit.client.0.smithi107.stdout:0/173: symlink d3/dc/d31/l45 0 2024-05-17T21:52:45.173 INFO:tasks.workunit.client.1.smithi138.stdout:0/118: dwrite d3/d10/f2c [0,4194304] 0 2024-05-17T21:52:45.175 INFO:tasks.workunit.client.0.smithi107.stdout:0/174: dread d3/dc/d31/f3a [0,4194304] 0 2024-05-17T21:52:45.178 INFO:tasks.workunit.client.1.smithi138.stdout:6/115: creat d7/d8/f20 x:0 0 0 2024-05-17T21:52:45.179 INFO:tasks.workunit.client.0.smithi107.stdout:9/164: mknod d0/dd/c36 0 2024-05-17T21:52:45.179 INFO:tasks.workunit.client.1.smithi138.stdout:5/59: dwrite d1/f7 [0,4194304] 0 2024-05-17T21:52:45.182 INFO:tasks.workunit.client.0.smithi107.stdout:4/150: creat d3/db/d16/f30 x:0 0 0 2024-05-17T21:52:45.183 INFO:tasks.workunit.client.1.smithi138.stdout:2/12: dwrite d3/f4 [0,4194304] 0 2024-05-17T21:52:45.184 INFO:tasks.workunit.client.1.smithi138.stdout:9/25: mkdir d6/d7 0 2024-05-17T21:52:45.187 INFO:tasks.workunit.client.0.smithi107.stdout:5/142: truncate d5/db/f1a 897164 0 2024-05-17T21:52:45.191 INFO:tasks.workunit.client.1.smithi138.stdout:0/119: write d3/fd [407472,57073] 0 2024-05-17T21:52:45.193 INFO:tasks.workunit.client.1.smithi138.stdout:7/39: getdents d0 0 2024-05-17T21:52:45.196 INFO:tasks.workunit.client.1.smithi138.stdout:5/60: write d1/f7 [446655,31494] 0 2024-05-17T21:52:45.196 INFO:tasks.workunit.client.1.smithi138.stdout:5/61: chown d1 1260094606 1 2024-05-17T21:52:45.198 INFO:tasks.workunit.client.1.smithi138.stdout:6/116: creat d7/db/d12/f21 x:0 0 0 2024-05-17T21:52:45.200 INFO:tasks.workunit.client.0.smithi107.stdout:0/175: truncate d3/f9 1698457 0 2024-05-17T21:52:45.203 INFO:tasks.workunit.client.1.smithi138.stdout:4/48: mknod d0/d3/d10/c11 0 2024-05-17T21:52:45.203 INFO:tasks.workunit.client.1.smithi138.stdout:5/62: dwrite d1/f7 [0,4194304] 0 2024-05-17T21:52:45.204 INFO:tasks.workunit.client.0.smithi107.stdout:7/135: getdents d5/df 0 2024-05-17T21:52:45.204 INFO:tasks.workunit.client.1.smithi138.stdout:5/63: write d1/d6/fb [810842,31771] 0 2024-05-17T21:52:45.206 INFO:tasks.workunit.client.1.smithi138.stdout:5/64: write d1/d6/fa [184493,99865] 0 2024-05-17T21:52:45.208 INFO:tasks.workunit.client.0.smithi107.stdout:9/165: rename d0/dd/f13 to d0/dd/d21/f37 0 2024-05-17T21:52:45.209 INFO:tasks.workunit.client.1.smithi138.stdout:7/40: rmdir d0 39 2024-05-17T21:52:45.211 INFO:tasks.workunit.client.0.smithi107.stdout:4/151: creat d3/d6/d26/f31 x:0 0 0 2024-05-17T21:52:45.213 INFO:tasks.workunit.client.1.smithi138.stdout:5/65: fsync d1/d6/fa 0 2024-05-17T21:52:45.216 INFO:tasks.workunit.client.0.smithi107.stdout:5/143: creat d5/db/d16/d2a/d2b/f2f x:0 0 0 2024-05-17T21:52:45.217 INFO:tasks.workunit.client.0.smithi107.stdout:4/152: dwrite d3/fa [0,4194304] 0 2024-05-17T21:52:45.219 INFO:tasks.workunit.client.0.smithi107.stdout:8/138: getdents dc/d21/d35 0 2024-05-17T21:52:45.221 INFO:tasks.workunit.client.0.smithi107.stdout:5/144: dread d5/fa [4194304,4194304] 0 2024-05-17T21:52:45.221 INFO:tasks.workunit.client.1.smithi138.stdout:4/49: creat d0/d3/f12 x:0 0 0 2024-05-17T21:52:45.225 INFO:tasks.workunit.client.1.smithi138.stdout:0/120: rename d3/l4 to d3/d10/l30 0 2024-05-17T21:52:45.226 INFO:tasks.workunit.client.1.smithi138.stdout:8/25: sync 2024-05-17T21:52:45.226 INFO:tasks.workunit.client.1.smithi138.stdout:8/26: readlink - no filename 2024-05-17T21:52:45.226 INFO:tasks.workunit.client.1.smithi138.stdout:1/38: sync 2024-05-17T21:52:45.227 INFO:tasks.workunit.client.0.smithi107.stdout:0/176: mknod d3/dc/d22/c46 0 2024-05-17T21:52:45.227 INFO:tasks.workunit.client.0.smithi107.stdout:0/177: dread - d3/d39/f41 zero size 2024-05-17T21:52:45.228 INFO:tasks.workunit.client.0.smithi107.stdout:0/178: write d3/dc/d22/d23/f2c [415695,29529] 0 2024-05-17T21:52:45.230 INFO:tasks.workunit.client.1.smithi138.stdout:1/39: dread d4/f6 [0,4194304] 0 2024-05-17T21:52:45.232 INFO:tasks.workunit.client.1.smithi138.stdout:1/40: dread d4/f6 [0,4194304] 0 2024-05-17T21:52:45.232 INFO:tasks.workunit.client.1.smithi138.stdout:5/66: rmdir d1/d6 39 2024-05-17T21:52:45.234 INFO:tasks.workunit.client.0.smithi107.stdout:9/166: dread - d0/f1c zero size 2024-05-17T21:52:45.234 INFO:tasks.workunit.client.1.smithi138.stdout:6/117: truncate d7/fa 3516700 0 2024-05-17T21:52:45.235 INFO:tasks.workunit.client.1.smithi138.stdout:2/13: write f2 [551681,49345] 0 2024-05-17T21:52:45.235 INFO:tasks.workunit.client.0.smithi107.stdout:7/136: dwrite d5/df/f1a [0,4194304] 0 2024-05-17T21:52:45.235 INFO:tasks.workunit.client.0.smithi107.stdout:9/167: dread - d0/f1c zero size 2024-05-17T21:52:45.238 INFO:tasks.workunit.client.1.smithi138.stdout:9/26: write f2 [4603759,115630] 0 2024-05-17T21:52:45.238 INFO:tasks.workunit.client.0.smithi107.stdout:2/120: creat d0/d9/f24 x:0 0 0 2024-05-17T21:52:45.239 INFO:tasks.workunit.client.0.smithi107.stdout:3/143: write d6/f11 [777744,11708] 0 2024-05-17T21:52:45.240 INFO:tasks.workunit.client.1.smithi138.stdout:4/50: fsync d0/f7 0 2024-05-17T21:52:45.243 INFO:tasks.workunit.client.1.smithi138.stdout:0/121: creat d3/d10/f31 x:0 0 0 2024-05-17T21:52:45.243 INFO:tasks.workunit.client.0.smithi107.stdout:6/147: creat d9/f2a x:0 0 0 2024-05-17T21:52:45.244 INFO:tasks.workunit.client.0.smithi107.stdout:9/168: dwrite d0/f4 [0,4194304] 0 2024-05-17T21:52:45.246 INFO:tasks.workunit.client.1.smithi138.stdout:0/122: dread d3/f7 [0,4194304] 0 2024-05-17T21:52:45.246 INFO:tasks.workunit.client.1.smithi138.stdout:0/123: stat d3/f29 0 2024-05-17T21:52:45.247 INFO:tasks.workunit.client.1.smithi138.stdout:8/27: symlink d0/l6 0 2024-05-17T21:52:45.248 INFO:tasks.workunit.client.1.smithi138.stdout:8/28: chown d0/l6 1800354 1 2024-05-17T21:52:45.249 INFO:tasks.workunit.client.0.smithi107.stdout:0/179: rmdir d3/d24 39 2024-05-17T21:52:45.253 INFO:tasks.workunit.client.1.smithi138.stdout:6/118: creat d7/f22 x:0 0 0 2024-05-17T21:52:45.255 INFO:tasks.workunit.client.1.smithi138.stdout:2/14: rmdir d3 39 2024-05-17T21:52:45.257 INFO:tasks.workunit.client.0.smithi107.stdout:1/150: sync 2024-05-17T21:52:45.258 INFO:tasks.workunit.client.0.smithi107.stdout:5/145: dwrite d5/db/dc/f24 [0,4194304] 0 2024-05-17T21:52:45.259 INFO:tasks.workunit.client.0.smithi107.stdout:1/151: write d4/f5 [5650905,42198] 0 2024-05-17T21:52:45.260 INFO:tasks.workunit.client.0.smithi107.stdout:5/146: write d5/db/d16/d2a/d2b/f2f [814960,3452] 0 2024-05-17T21:52:45.260 INFO:tasks.workunit.client.0.smithi107.stdout:2/121: unlink d0/f2 0 2024-05-17T21:52:45.262 INFO:tasks.workunit.client.0.smithi107.stdout:5/147: chown d5/db/dc/c26 25 1 2024-05-17T21:52:45.263 INFO:tasks.workunit.client.0.smithi107.stdout:3/144: mknod d6/d8/d10/d25/c34 0 2024-05-17T21:52:45.264 INFO:tasks.workunit.client.0.smithi107.stdout:2/122: write d0/d9/f24 [862878,32941] 0 2024-05-17T21:52:45.265 INFO:tasks.workunit.client.1.smithi138.stdout:1/41: dwrite f2 [4194304,4194304] 0 2024-05-17T21:52:45.265 INFO:tasks.workunit.client.1.smithi138.stdout:1/42: dread - f3 zero size 2024-05-17T21:52:45.266 INFO:tasks.workunit.client.0.smithi107.stdout:3/145: read d6/d8/f17 [209537,102119] 0 2024-05-17T21:52:45.268 INFO:tasks.workunit.client.0.smithi107.stdout:4/153: rename d3/db/d1e/l2d to d3/d6/d26/l32 0 2024-05-17T21:52:45.268 INFO:tasks.workunit.client.0.smithi107.stdout:1/152: dwrite d4/f8 [0,4194304] 0 2024-05-17T21:52:45.269 INFO:tasks.workunit.client.0.smithi107.stdout:6/148: truncate d9/f15 426407 0 2024-05-17T21:52:45.271 INFO:tasks.workunit.client.1.smithi138.stdout:0/124: creat d3/d10/f32 x:0 0 0 2024-05-17T21:52:45.271 INFO:tasks.workunit.client.1.smithi138.stdout:0/125: stat d3/d10/f14 0 2024-05-17T21:52:45.273 INFO:tasks.workunit.client.1.smithi138.stdout:8/29: creat d0/f7 x:0 0 0 2024-05-17T21:52:45.274 INFO:tasks.workunit.client.0.smithi107.stdout:5/148: dwrite f3 [0,4194304] 0 2024-05-17T21:52:45.276 INFO:tasks.workunit.client.1.smithi138.stdout:9/27: getdents d6/d7 0 2024-05-17T21:52:45.281 INFO:tasks.workunit.client.1.smithi138.stdout:7/41: link d0/l7 d0/ld 0 2024-05-17T21:52:45.283 INFO:tasks.workunit.client.0.smithi107.stdout:5/149: dwrite d5/f28 [0,4194304] 0 2024-05-17T21:52:45.283 INFO:tasks.workunit.client.1.smithi138.stdout:0/126: mknod d3/d10/c33 0 2024-05-17T21:52:45.284 INFO:tasks.workunit.client.0.smithi107.stdout:5/150: write d5/f17 [4057404,106250] 0 2024-05-17T21:52:45.285 INFO:tasks.workunit.client.1.smithi138.stdout:0/127: chown d3/d10/f1a 10 1 2024-05-17T21:52:45.285 INFO:tasks.workunit.client.1.smithi138.stdout:8/30: unlink d0/f1 0 2024-05-17T21:52:45.294 INFO:tasks.workunit.client.1.smithi138.stdout:8/31: dwrite d0/f4 [0,4194304] 0 2024-05-17T21:52:45.312 INFO:tasks.workunit.client.1.smithi138.stdout:8/32: fdatasync d0/f4 0 2024-05-17T21:52:45.312 INFO:tasks.workunit.client.1.smithi138.stdout:8/33: chown d0 8898 1 2024-05-17T21:52:45.313 INFO:tasks.workunit.client.1.smithi138.stdout:8/34: write d0/f2 [669388,88644] 0 2024-05-17T21:52:45.379 INFO:tasks.workunit.client.1.smithi138.stdout:3/35: sync 2024-05-17T21:52:45.382 INFO:tasks.workunit.client.1.smithi138.stdout:2/15: creat d3/f5 x:0 0 0 2024-05-17T21:52:45.383 INFO:tasks.workunit.client.0.smithi107.stdout:6/149: stat d9/c11 0 2024-05-17T21:52:45.383 INFO:tasks.workunit.client.1.smithi138.stdout:9/28: write f5 [3759545,27119] 0 2024-05-17T21:52:45.384 INFO:tasks.workunit.client.1.smithi138.stdout:9/29: chown c4 31305 1 2024-05-17T21:52:45.386 INFO:tasks.workunit.client.1.smithi138.stdout:1/43: link f3 d4/f9 0 2024-05-17T21:52:45.388 INFO:tasks.workunit.client.0.smithi107.stdout:8/139: rename dc/d21/d35/c33 to dc/d10/c3e 0 2024-05-17T21:52:45.389 INFO:tasks.workunit.client.0.smithi107.stdout:8/140: dread dc/d21/d35/f20 [0,4194304] 0 2024-05-17T21:52:45.394 INFO:tasks.workunit.client.0.smithi107.stdout:0/180: write d3/f1e [605825,11987] 0 2024-05-17T21:52:45.395 INFO:tasks.workunit.client.0.smithi107.stdout:0/181: read d3/f15 [3534082,9436] 0 2024-05-17T21:52:45.396 INFO:tasks.workunit.client.0.smithi107.stdout:7/137: mkdir d5/df/d11/d1d/d33 0 2024-05-17T21:52:45.396 INFO:tasks.workunit.client.0.smithi107.stdout:0/182: dread - d3/dc/d22/f32 zero size 2024-05-17T21:52:45.399 INFO:tasks.workunit.client.1.smithi138.stdout:4/51: truncate d0/f7 220792 0 2024-05-17T21:52:45.399 INFO:tasks.workunit.client.1.smithi138.stdout:4/52: dread - d0/d3/f12 zero size 2024-05-17T21:52:45.403 INFO:tasks.workunit.client.0.smithi107.stdout:5/151: mkdir d5/db/d16/d2a/d30 0 2024-05-17T21:52:45.405 INFO:tasks.workunit.client.1.smithi138.stdout:2/16: creat d3/f6 x:0 0 0 2024-05-17T21:52:45.405 INFO:tasks.workunit.client.1.smithi138.stdout:8/35: rmdir d0 39 2024-05-17T21:52:45.405 INFO:tasks.workunit.client.1.smithi138.stdout:2/17: read - d3/f5 zero size 2024-05-17T21:52:45.406 INFO:tasks.workunit.client.0.smithi107.stdout:3/146: write d6/f2c [361829,51291] 0 2024-05-17T21:52:45.406 INFO:tasks.workunit.client.1.smithi138.stdout:2/18: write d3/f5 [742256,65946] 0 2024-05-17T21:52:45.407 INFO:tasks.workunit.client.0.smithi107.stdout:2/123: dwrite d0/fa [0,4194304] 0 2024-05-17T21:52:45.408 INFO:tasks.workunit.client.0.smithi107.stdout:2/124: chown d0/d9/d1c 14285 1 2024-05-17T21:52:45.409 INFO:tasks.workunit.client.0.smithi107.stdout:9/169: link d0/dd/l20 d0/dd/d1f/l38 0 2024-05-17T21:52:45.410 INFO:tasks.workunit.client.1.smithi138.stdout:1/44: unlink d4/f6 0 2024-05-17T21:52:45.412 INFO:tasks.workunit.client.0.smithi107.stdout:8/141: rmdir dc/d21/d35 39 2024-05-17T21:52:45.419 INFO:tasks.workunit.client.1.smithi138.stdout:7/42: dwrite d0/fc [0,4194304] 0 2024-05-17T21:52:45.420 INFO:tasks.workunit.client.1.smithi138.stdout:3/36: truncate d1/f4 4150204 0 2024-05-17T21:52:45.425 INFO:tasks.workunit.client.0.smithi107.stdout:7/138: chown d5/d16/f1e 1023 1 2024-05-17T21:52:45.428 INFO:tasks.workunit.client.0.smithi107.stdout:0/183: symlink d3/dc/d22/l47 0 2024-05-17T21:52:45.430 INFO:tasks.workunit.client.1.smithi138.stdout:8/36: dwrite d0/f4 [0,4194304] 0 2024-05-17T21:52:45.432 INFO:tasks.workunit.client.0.smithi107.stdout:7/139: dwrite d5/f22 [0,4194304] 0 2024-05-17T21:52:45.432 INFO:tasks.workunit.client.0.smithi107.stdout:5/152: rename d5/fa to d5/db/d16/d2a/d30/f31 0 2024-05-17T21:52:45.433 INFO:tasks.workunit.client.0.smithi107.stdout:5/153: chown d5/db/f25 5475 1 2024-05-17T21:52:45.433 INFO:tasks.workunit.client.0.smithi107.stdout:4/154: truncate d3/db/f1f 778406 0 2024-05-17T21:52:45.433 INFO:tasks.workunit.client.1.smithi138.stdout:2/19: creat d3/f7 x:0 0 0 2024-05-17T21:52:45.441 INFO:tasks.workunit.client.0.smithi107.stdout:5/154: dwrite d5/db/dc/d18/f27 [0,4194304] 0 2024-05-17T21:52:45.442 INFO:tasks.workunit.client.1.smithi138.stdout:9/30: truncate f5 64788 0 2024-05-17T21:52:45.444 INFO:tasks.workunit.client.1.smithi138.stdout:2/20: dwrite d3/f6 [0,4194304] 0 2024-05-17T21:52:45.447 INFO:tasks.workunit.client.0.smithi107.stdout:9/170: mknod d0/dd/d21/c39 0 2024-05-17T21:52:45.449 INFO:tasks.workunit.client.1.smithi138.stdout:7/43: symlink d0/le 0 2024-05-17T21:52:45.453 INFO:tasks.workunit.client.0.smithi107.stdout:6/150: truncate f1 2469638 0 2024-05-17T21:52:45.453 INFO:tasks.workunit.client.0.smithi107.stdout:8/142: chown ca 404554810 1 2024-05-17T21:52:45.453 INFO:tasks.workunit.client.0.smithi107.stdout:6/151: chown d9 956 1 2024-05-17T21:52:45.453 INFO:tasks.workunit.client.1.smithi138.stdout:8/37: mknod d0/c8 0 2024-05-17T21:52:45.458 INFO:tasks.workunit.client.1.smithi138.stdout:1/45: link f2 d4/fa 0 2024-05-17T21:52:45.461 INFO:tasks.workunit.client.0.smithi107.stdout:1/153: creat d4/df/f29 x:0 0 0 2024-05-17T21:52:45.461 INFO:tasks.workunit.client.1.smithi138.stdout:8/38: dwrite d0/f7 [0,4194304] 0 2024-05-17T21:52:45.464 INFO:tasks.workunit.client.1.smithi138.stdout:8/39: write d0/f4 [1352203,118662] 0 2024-05-17T21:52:45.465 INFO:tasks.workunit.client.1.smithi138.stdout:1/46: dread d4/fa [4194304,4194304] 0 2024-05-17T21:52:45.467 INFO:tasks.workunit.client.1.smithi138.stdout:0/128: rename d3/le to d3/l34 0 2024-05-17T21:52:45.468 INFO:tasks.workunit.client.1.smithi138.stdout:0/129: write d3/d10/f11 [4022392,23066] 0 2024-05-17T21:52:45.472 INFO:tasks.workunit.client.0.smithi107.stdout:0/184: creat d3/dc/d22/d23/f48 x:0 0 0 2024-05-17T21:52:45.473 INFO:tasks.workunit.client.1.smithi138.stdout:4/53: write d0/f7 [1146061,98630] 0 2024-05-17T21:52:45.474 INFO:tasks.workunit.client.1.smithi138.stdout:9/31: mknod d6/c8 0 2024-05-17T21:52:45.474 INFO:tasks.workunit.client.1.smithi138.stdout:9/32: fsync f2 0 2024-05-17T21:52:45.476 INFO:tasks.workunit.client.1.smithi138.stdout:2/21: mknod d3/c8 0 2024-05-17T21:52:45.477 INFO:tasks.workunit.client.1.smithi138.stdout:2/22: chown d3/f5 194238 1 2024-05-17T21:52:45.477 INFO:tasks.workunit.client.1.smithi138.stdout:2/23: readlink - no filename 2024-05-17T21:52:45.478 INFO:tasks.workunit.client.1.smithi138.stdout:8/40: fsync d0/f4 0 2024-05-17T21:52:45.478 INFO:tasks.workunit.client.0.smithi107.stdout:4/155: rename d3/db/d16/f2b to d3/db/f33 0 2024-05-17T21:52:45.479 INFO:tasks.workunit.client.1.smithi138.stdout:8/41: write d0/f4 [1330188,34985] 0 2024-05-17T21:52:45.479 INFO:tasks.workunit.client.0.smithi107.stdout:0/185: dread d3/dc/f16 [0,4194304] 0 2024-05-17T21:52:45.480 INFO:tasks.workunit.client.1.smithi138.stdout:8/42: write d0/f2 [1814367,117184] 0 2024-05-17T21:52:45.481 INFO:tasks.workunit.client.1.smithi138.stdout:7/44: unlink d0/f6 0 2024-05-17T21:52:45.483 INFO:tasks.workunit.client.1.smithi138.stdout:8/43: write d0/f2 [1283882,86891] 0 2024-05-17T21:52:45.483 INFO:tasks.workunit.client.1.smithi138.stdout:2/24: dread d3/f6 [0,4194304] 0 2024-05-17T21:52:45.487 INFO:tasks.workunit.client.1.smithi138.stdout:8/44: dread d0/f7 [0,4194304] 0 2024-05-17T21:52:45.488 INFO:tasks.workunit.client.1.smithi138.stdout:8/45: chown d0/c8 6 1 2024-05-17T21:52:45.488 INFO:tasks.workunit.client.0.smithi107.stdout:6/152: creat d9/d16/d22/f2b x:0 0 0 2024-05-17T21:52:45.490 INFO:tasks.workunit.client.0.smithi107.stdout:8/143: truncate dc/f18 4004049 0 2024-05-17T21:52:45.490 INFO:tasks.workunit.client.0.smithi107.stdout:8/144: fdatasync f4 0 2024-05-17T21:52:45.492 INFO:tasks.workunit.client.0.smithi107.stdout:1/154: unlink d4/f5 0 2024-05-17T21:52:45.493 INFO:tasks.workunit.client.0.smithi107.stdout:1/155: truncate d4/de/f12 671607 0 2024-05-17T21:52:45.493 INFO:tasks.workunit.client.1.smithi138.stdout:1/47: mkdir d4/db 0 2024-05-17T21:52:45.496 INFO:tasks.workunit.client.0.smithi107.stdout:7/140: fdatasync d5/fe 0 2024-05-17T21:52:45.496 INFO:tasks.workunit.client.0.smithi107.stdout:7/141: dread - d5/d16/f2b zero size 2024-05-17T21:52:45.498 INFO:tasks.workunit.client.1.smithi138.stdout:0/130: creat d3/d10/f35 x:0 0 0 2024-05-17T21:52:45.501 INFO:tasks.workunit.client.0.smithi107.stdout:5/155: rename d5/db/d16/d2a/d30 to d5/db/d16/d2d/d32 0 2024-05-17T21:52:45.503 INFO:tasks.workunit.client.0.smithi107.stdout:4/156: chown d3/db/d1e/l2a 5 1 2024-05-17T21:52:45.505 INFO:tasks.workunit.client.0.smithi107.stdout:0/186: creat d3/d39/f49 x:0 0 0 2024-05-17T21:52:45.507 INFO:tasks.workunit.client.0.smithi107.stdout:2/125: link d0/f6 d0/f25 0 2024-05-17T21:52:45.508 INFO:tasks.workunit.client.0.smithi107.stdout:4/157: dwrite d3/db/f11 [0,4194304] 0 2024-05-17T21:52:45.514 INFO:tasks.workunit.client.0.smithi107.stdout:3/147: sync 2024-05-17T21:52:45.515 INFO:tasks.workunit.client.0.smithi107.stdout:8/145: creat dc/d10/d17/f3f x:0 0 0 2024-05-17T21:52:45.515 INFO:tasks.workunit.client.0.smithi107.stdout:3/148: write d6/d1a/f1f [392296,48806] 0 2024-05-17T21:52:45.516 INFO:tasks.workunit.client.0.smithi107.stdout:3/149: chown d6/f1c 173442 1 2024-05-17T21:52:45.516 INFO:tasks.workunit.client.1.smithi138.stdout:7/45: symlink d0/lf 0 2024-05-17T21:52:45.516 INFO:tasks.workunit.client.0.smithi107.stdout:8/146: write dc/d21/d27/d2f/f32 [150340,87157] 0 2024-05-17T21:52:45.520 INFO:tasks.workunit.client.1.smithi138.stdout:2/25: creat d3/f9 x:0 0 0 2024-05-17T21:52:45.521 INFO:tasks.workunit.client.1.smithi138.stdout:3/37: link d1/c5 d1/d3/c6 0 2024-05-17T21:52:45.521 INFO:tasks.workunit.client.1.smithi138.stdout:3/38: stat d1/d3/c6 0 2024-05-17T21:52:45.524 INFO:tasks.workunit.client.0.smithi107.stdout:9/171: rename d0/dd/l20 to d0/dd/d1f/l3a 0 2024-05-17T21:52:45.525 INFO:tasks.workunit.client.1.smithi138.stdout:3/39: dread d1/f2 [4194304,4194304] 0 2024-05-17T21:52:45.526 INFO:tasks.workunit.client.1.smithi138.stdout:0/131: chown d3/d10/l30 7729694 1 2024-05-17T21:52:45.528 INFO:tasks.workunit.client.1.smithi138.stdout:4/54: mknod d0/c13 0 2024-05-17T21:52:45.533 INFO:tasks.workunit.client.1.smithi138.stdout:7/46: creat d0/f10 x:0 0 0 2024-05-17T21:52:45.534 INFO:tasks.workunit.client.1.smithi138.stdout:4/55: dwrite d0/f7 [0,4194304] 0 2024-05-17T21:52:45.535 INFO:tasks.workunit.client.1.smithi138.stdout:9/33: write f5 [870949,47575] 0 2024-05-17T21:52:45.535 INFO:tasks.workunit.client.0.smithi107.stdout:4/158: dwrite d3/db/f33 [4194304,4194304] 0 2024-05-17T21:52:45.536 INFO:tasks.workunit.client.1.smithi138.stdout:7/47: write d0/fa [341866,103776] 0 2024-05-17T21:52:45.537 INFO:tasks.workunit.client.1.smithi138.stdout:2/26: creat d3/fa x:0 0 0 2024-05-17T21:52:45.537 INFO:tasks.workunit.client.0.smithi107.stdout:3/150: readlink d6/d2a/l2b 0 2024-05-17T21:52:45.538 INFO:tasks.workunit.client.1.smithi138.stdout:2/27: fdatasync f2 0 2024-05-17T21:52:45.538 INFO:tasks.workunit.client.0.smithi107.stdout:3/151: chown d6/f33 0 1 2024-05-17T21:52:45.538 INFO:tasks.workunit.client.1.smithi138.stdout:7/48: write d0/f9 [457687,119948] 0 2024-05-17T21:52:45.540 INFO:tasks.workunit.client.1.smithi138.stdout:4/56: write d0/d3/f12 [671526,121912] 0 2024-05-17T21:52:45.541 INFO:tasks.workunit.client.1.smithi138.stdout:2/28: read - d3/f9 zero size 2024-05-17T21:52:45.541 INFO:tasks.workunit.client.0.smithi107.stdout:8/147: symlink dc/d21/d27/l40 0 2024-05-17T21:52:45.543 INFO:tasks.workunit.client.1.smithi138.stdout:4/57: rename d0/d3 to d0/d3/d14 22 2024-05-17T21:52:45.546 INFO:tasks.workunit.client.1.smithi138.stdout:7/49: dread d0/f9 [0,4194304] 0 2024-05-17T21:52:45.548 INFO:tasks.workunit.client.0.smithi107.stdout:9/172: unlink d0/f1c 0 2024-05-17T21:52:45.554 INFO:tasks.workunit.client.1.smithi138.stdout:4/58: dread d0/d3/f12 [0,4194304] 0 2024-05-17T21:52:45.554 INFO:tasks.workunit.client.1.smithi138.stdout:9/34: dwrite f5 [0,4194304] 0 2024-05-17T21:52:45.554 INFO:tasks.workunit.client.1.smithi138.stdout:8/46: truncate d0/f2 522951 0 2024-05-17T21:52:45.555 INFO:tasks.workunit.client.0.smithi107.stdout:1/156: rename d4/df/f22 to d4/df/d14/d1d/f2a 0 2024-05-17T21:52:45.556 INFO:tasks.workunit.client.0.smithi107.stdout:1/157: dread d4/fc [0,4194304] 0 2024-05-17T21:52:45.557 INFO:tasks.workunit.client.1.smithi138.stdout:3/40: mknod d1/c7 0 2024-05-17T21:52:45.557 INFO:tasks.workunit.client.0.smithi107.stdout:1/158: readlink d4/l20 0 2024-05-17T21:52:45.558 INFO:tasks.workunit.client.1.smithi138.stdout:0/132: creat d3/d10/f36 x:0 0 0 2024-05-17T21:52:45.559 INFO:tasks.workunit.client.1.smithi138.stdout:7/50: dwrite d0/fc [0,4194304] 0 2024-05-17T21:52:45.559 INFO:tasks.workunit.client.0.smithi107.stdout:2/126: mkdir d0/d26 0 2024-05-17T21:52:45.560 INFO:tasks.workunit.client.0.smithi107.stdout:2/127: write d0/d9/d11/f14 [7474465,32802] 0 2024-05-17T21:52:45.563 INFO:tasks.workunit.client.1.smithi138.stdout:7/51: dread d0/fa [0,4194304] 0 2024-05-17T21:52:45.563 INFO:tasks.workunit.client.0.smithi107.stdout:4/159: symlink d3/db/d16/l34 0 2024-05-17T21:52:45.564 INFO:tasks.workunit.client.0.smithi107.stdout:3/152: mkdir d6/d8/d10/d35 0 2024-05-17T21:52:45.565 INFO:tasks.workunit.client.0.smithi107.stdout:3/153: chown d6/d8/d10/f23 0 1 2024-05-17T21:52:45.566 INFO:tasks.workunit.client.0.smithi107.stdout:3/154: write d6/f33 [863417,111595] 0 2024-05-17T21:52:45.567 INFO:tasks.workunit.client.0.smithi107.stdout:3/155: chown d6/f1c 949316 1 2024-05-17T21:52:45.567 INFO:tasks.workunit.client.1.smithi138.stdout:2/29: unlink d3/f7 0 2024-05-17T21:52:45.568 INFO:tasks.workunit.client.0.smithi107.stdout:3/156: chown d6/d8/d10/c21 4884 1 2024-05-17T21:52:45.569 INFO:tasks.workunit.client.0.smithi107.stdout:3/157: read d6/d8/f17 [101734,112445] 0 2024-05-17T21:52:45.574 INFO:tasks.workunit.client.0.smithi107.stdout:3/158: dwrite d6/d8/d10/f1e [0,4194304] 0 2024-05-17T21:52:45.577 INFO:tasks.workunit.client.1.smithi138.stdout:1/48: link d4/fa d4/fc 0 2024-05-17T21:52:45.577 INFO:tasks.workunit.client.1.smithi138.stdout:1/49: chown d4 2682 1 2024-05-17T21:52:45.578 INFO:tasks.workunit.client.0.smithi107.stdout:0/187: creat d3/f4a x:0 0 0 2024-05-17T21:52:45.579 INFO:tasks.workunit.client.0.smithi107.stdout:0/188: fdatasync d3/dc/f26 0 2024-05-17T21:52:45.580 INFO:tasks.workunit.client.0.smithi107.stdout:3/159: dread d6/d8/d10/f23 [4194304,4194304] 0 2024-05-17T21:52:45.580 INFO:tasks.workunit.client.0.smithi107.stdout:0/189: truncate d3/dc/d22/f3e 603368 0 2024-05-17T21:52:45.583 INFO:tasks.workunit.client.0.smithi107.stdout:1/159: unlink l2 0 2024-05-17T21:52:45.583 INFO:tasks.workunit.client.0.smithi107.stdout:1/160: write d4/df/f13 [966701,32969] 0 2024-05-17T21:52:45.584 INFO:tasks.workunit.client.1.smithi138.stdout:4/59: mkdir d0/d3/db/d15 0 2024-05-17T21:52:45.587 INFO:tasks.workunit.client.0.smithi107.stdout:6/153: link d9/c21 d9/d16/c2c 0 2024-05-17T21:52:45.588 INFO:tasks.workunit.client.0.smithi107.stdout:6/154: stat d9/fa 0 2024-05-17T21:52:45.588 INFO:tasks.workunit.client.1.smithi138.stdout:0/133: mknod d3/d10/c37 0 2024-05-17T21:52:45.590 INFO:tasks.workunit.client.0.smithi107.stdout:4/160: mkdir d3/db/d35 0 2024-05-17T21:52:45.590 INFO:tasks.workunit.client.1.smithi138.stdout:0/134: write d3/d10/f1a [270001,117684] 0 2024-05-17T21:52:45.590 INFO:tasks.workunit.client.0.smithi107.stdout:4/161: fsync d3/f8 0 2024-05-17T21:52:45.591 INFO:tasks.workunit.client.1.smithi138.stdout:0/135: truncate d3/d10/f32 117437 0 2024-05-17T21:52:45.592 INFO:tasks.workunit.client.1.smithi138.stdout:0/136: write d3/f27 [589891,80247] 0 2024-05-17T21:52:45.593 INFO:tasks.workunit.client.1.smithi138.stdout:0/137: fsync d3/d10/f36 0 2024-05-17T21:52:45.593 INFO:tasks.workunit.client.0.smithi107.stdout:6/155: dwrite d9/fa [0,4194304] 0 2024-05-17T21:52:45.595 INFO:tasks.workunit.client.1.smithi138.stdout:9/35: write f3 [588188,4588] 0 2024-05-17T21:52:45.596 INFO:tasks.workunit.client.1.smithi138.stdout:9/36: fsync f2 0 2024-05-17T21:52:45.596 INFO:tasks.workunit.client.1.smithi138.stdout:2/30: symlink d3/lb 0 2024-05-17T21:52:45.598 INFO:tasks.workunit.client.0.smithi107.stdout:5/156: rename d5/f7 to d5/db/d16/d2d/d32/f33 0 2024-05-17T21:52:45.603 INFO:tasks.workunit.client.1.smithi138.stdout:1/50: mkdir d4/dd 0 2024-05-17T21:52:45.605 INFO:tasks.workunit.client.1.smithi138.stdout:9/37: dwrite f2 [0,4194304] 0 2024-05-17T21:52:45.608 INFO:tasks.workunit.client.1.smithi138.stdout:4/60: write d0/d3/f12 [1523715,8031] 0 2024-05-17T21:52:45.610 INFO:tasks.workunit.client.1.smithi138.stdout:1/51: dread d4/fc [0,4194304] 0 2024-05-17T21:52:45.613 INFO:tasks.workunit.client.1.smithi138.stdout:9/38: dread f2 [0,4194304] 0 2024-05-17T21:52:45.614 INFO:tasks.workunit.client.0.smithi107.stdout:2/128: mkdir d0/d9/d11/d27 0 2024-05-17T21:52:45.614 INFO:tasks.workunit.client.0.smithi107.stdout:2/129: chown d0/c17 5233669 1 2024-05-17T21:52:45.615 INFO:tasks.workunit.client.1.smithi138.stdout:7/52: rename d0/f9 to d0/f11 0 2024-05-17T21:52:45.615 INFO:tasks.workunit.client.0.smithi107.stdout:2/130: fsync d0/d9/d11/d18/f20 0 2024-05-17T21:52:45.616 INFO:tasks.workunit.client.1.smithi138.stdout:7/53: write d0/fc [2420790,34836] 0 2024-05-17T21:52:45.616 INFO:tasks.workunit.client.1.smithi138.stdout:4/61: dread d0/d3/f12 [0,4194304] 0 2024-05-17T21:52:45.616 INFO:tasks.workunit.client.1.smithi138.stdout:7/54: chown d0 2 1 2024-05-17T21:52:45.623 INFO:tasks.workunit.client.0.smithi107.stdout:6/156: mkdir d9/d16/d22/d2d 0 2024-05-17T21:52:45.629 INFO:tasks.workunit.client.0.smithi107.stdout:0/190: symlink d3/l4b 0 2024-05-17T21:52:45.631 INFO:tasks.workunit.client.0.smithi107.stdout:1/161: symlink d4/l2b 0 2024-05-17T21:52:45.632 INFO:tasks.workunit.client.0.smithi107.stdout:1/162: truncate d4/df/d14/f23 310484 0 2024-05-17T21:52:45.632 INFO:tasks.workunit.client.0.smithi107.stdout:2/131: dread d0/f25 [0,4194304] 0 2024-05-17T21:52:45.633 INFO:tasks.workunit.client.1.smithi138.stdout:2/31: creat d3/fc x:0 0 0 2024-05-17T21:52:45.633 INFO:tasks.workunit.client.1.smithi138.stdout:8/47: truncate d0/f7 2092941 0 2024-05-17T21:52:45.636 INFO:tasks.workunit.client.0.smithi107.stdout:8/148: link dc/d10/c1e dc/d21/d35/c41 0 2024-05-17T21:52:45.636 INFO:tasks.workunit.client.1.smithi138.stdout:3/41: write d1/f4 [3008828,110074] 0 2024-05-17T21:52:45.636 INFO:tasks.workunit.client.1.smithi138.stdout:7/55: creat d0/f12 x:0 0 0 2024-05-17T21:52:45.636 INFO:tasks.workunit.client.1.smithi138.stdout:4/62: creat d0/d3/f16 x:0 0 0 2024-05-17T21:52:45.636 INFO:tasks.workunit.client.1.smithi138.stdout:2/32: truncate d3/f4 5137471 0 2024-05-17T21:52:45.637 INFO:tasks.workunit.client.1.smithi138.stdout:2/33: write d3/f4 [5086071,83588] 0 2024-05-17T21:52:45.639 INFO:tasks.workunit.client.1.smithi138.stdout:7/56: dread d0/f5 [0,4194304] 0 2024-05-17T21:52:45.640 INFO:tasks.workunit.client.1.smithi138.stdout:7/57: write d0/f5 [2550843,95535] 0 2024-05-17T21:52:45.643 INFO:tasks.workunit.client.0.smithi107.stdout:7/142: rename d5/df/d11/d1d/l24 to d5/df/l34 0 2024-05-17T21:52:45.643 INFO:tasks.workunit.client.0.smithi107.stdout:9/173: rename d0/dd/d2f to d0/dd/d2f/d3b 22 2024-05-17T21:52:45.644 INFO:tasks.workunit.client.0.smithi107.stdout:7/143: write d5/f22 [1883964,99458] 0 2024-05-17T21:52:45.645 INFO:tasks.workunit.client.1.smithi138.stdout:5/67: sync 2024-05-17T21:52:45.645 INFO:tasks.workunit.client.1.smithi138.stdout:6/119: sync 2024-05-17T21:52:45.645 INFO:tasks.workunit.client.1.smithi138.stdout:5/68: chown d1/l8 0 1 2024-05-17T21:52:45.649 INFO:tasks.workunit.client.0.smithi107.stdout:1/163: unlink d4/fc 0 2024-05-17T21:52:45.652 INFO:tasks.workunit.client.1.smithi138.stdout:5/69: dwrite d1/f7 [4194304,4194304] 0 2024-05-17T21:52:45.658 INFO:tasks.workunit.client.0.smithi107.stdout:6/157: rename d9/l26 to d9/d16/d22/d2d/l2e 0 2024-05-17T21:52:45.658 INFO:tasks.workunit.client.0.smithi107.stdout:6/158: dread - d9/d16/f29 zero size 2024-05-17T21:52:45.659 INFO:tasks.workunit.client.0.smithi107.stdout:3/160: getdents d6/d8/d10 0 2024-05-17T21:52:45.660 INFO:tasks.workunit.client.0.smithi107.stdout:0/191: mknod d3/dc/d22/d3d/c4c 0 2024-05-17T21:52:45.663 INFO:tasks.workunit.client.0.smithi107.stdout:8/149: rename ca to dc/d21/d35/c42 0 2024-05-17T21:52:45.665 INFO:tasks.workunit.client.0.smithi107.stdout:0/192: creat d3/dc/d22/d3d/f4d x:0 0 0 2024-05-17T21:52:45.667 INFO:tasks.workunit.client.0.smithi107.stdout:3/161: mkdir d6/d8/d10/d35/d36 0 2024-05-17T21:52:45.669 INFO:tasks.workunit.client.0.smithi107.stdout:0/193: rename d3/dc/l3b to d3/dc/l4e 0 2024-05-17T21:52:45.670 INFO:tasks.workunit.client.0.smithi107.stdout:0/194: write d3/ff [702919,5262] 0 2024-05-17T21:52:45.670 INFO:tasks.workunit.client.0.smithi107.stdout:3/162: dread d6/d1a/f1f [0,4194304] 0 2024-05-17T21:52:45.671 INFO:tasks.workunit.client.0.smithi107.stdout:9/174: rename d0/dd/d1f/c24 to d0/dd/d21/c3c 0 2024-05-17T21:52:45.673 INFO:tasks.workunit.client.0.smithi107.stdout:0/195: mknod d3/dc/d22/d3d/c4f 0 2024-05-17T21:52:45.675 INFO:tasks.workunit.client.0.smithi107.stdout:6/159: rename d9/d16/d22/f2b to d9/d16/d22/d2d/f2f 0 2024-05-17T21:52:45.676 INFO:tasks.workunit.client.0.smithi107.stdout:9/175: mkdir d0/dd/d3d 0 2024-05-17T21:52:45.678 INFO:tasks.workunit.client.0.smithi107.stdout:6/160: dread d9/d16/f18 [0,4194304] 0 2024-05-17T21:52:45.679 INFO:tasks.workunit.client.0.smithi107.stdout:0/196: creat d3/dc/d22/f50 x:0 0 0 2024-05-17T21:52:45.679 INFO:tasks.workunit.client.0.smithi107.stdout:0/197: dread - d3/dc/d22/d3d/f4d zero size 2024-05-17T21:52:45.680 INFO:tasks.workunit.client.0.smithi107.stdout:3/163: mkdir d6/d2a/d2d/d37 0 2024-05-17T21:52:45.681 INFO:tasks.workunit.client.0.smithi107.stdout:9/176: mkdir d0/dd/d21/d29/d3e 0 2024-05-17T21:52:45.683 INFO:tasks.workunit.client.0.smithi107.stdout:6/161: creat d9/d16/f30 x:0 0 0 2024-05-17T21:52:45.685 INFO:tasks.workunit.client.0.smithi107.stdout:9/177: mknod d0/dd/d2f/c3f 0 2024-05-17T21:52:45.685 INFO:tasks.workunit.client.0.smithi107.stdout:8/150: link dc/d10/f15 dc/f43 0 2024-05-17T21:52:45.686 INFO:tasks.workunit.client.0.smithi107.stdout:8/151: write dc/d10/d17/f3f [720660,41168] 0 2024-05-17T21:52:45.687 INFO:tasks.workunit.client.0.smithi107.stdout:6/162: dread d9/f15 [0,4194304] 0 2024-05-17T21:52:45.688 INFO:tasks.workunit.client.0.smithi107.stdout:9/178: creat d0/dd/d1f/f40 x:0 0 0 2024-05-17T21:52:45.692 INFO:tasks.workunit.client.0.smithi107.stdout:0/198: rename d3/dc/d22/l27 to d3/l51 0 2024-05-17T21:52:45.692 INFO:tasks.workunit.client.0.smithi107.stdout:9/179: dread - d0/dd/d21/f37 zero size 2024-05-17T21:52:45.692 INFO:tasks.workunit.client.0.smithi107.stdout:5/157: sync 2024-05-17T21:52:45.693 INFO:tasks.workunit.client.0.smithi107.stdout:8/152: mkdir dc/d44 0 2024-05-17T21:52:45.694 INFO:tasks.workunit.client.0.smithi107.stdout:9/180: write d0/dd/d21/f27 [1671820,48531] 0 2024-05-17T21:52:45.695 INFO:tasks.workunit.client.0.smithi107.stdout:6/163: dread d9/fa [0,4194304] 0 2024-05-17T21:52:45.696 INFO:tasks.workunit.client.0.smithi107.stdout:0/199: creat d3/dc/d22/d3d/f52 x:0 0 0 2024-05-17T21:52:45.697 INFO:tasks.workunit.client.0.smithi107.stdout:6/164: chown d9/d16/d22/d2d/f2f 336963 1 2024-05-17T21:52:45.698 INFO:tasks.workunit.client.0.smithi107.stdout:6/165: read - d9/d16/d22/d2d/f2f zero size 2024-05-17T21:52:45.699 INFO:tasks.workunit.client.0.smithi107.stdout:8/153: creat dc/d10/d17/f45 x:0 0 0 2024-05-17T21:52:45.699 INFO:tasks.workunit.client.0.smithi107.stdout:5/158: dwrite d5/f19 [0,4194304] 0 2024-05-17T21:52:45.702 INFO:tasks.workunit.client.0.smithi107.stdout:0/200: symlink d3/l53 0 2024-05-17T21:52:45.703 INFO:tasks.workunit.client.0.smithi107.stdout:8/154: stat dc/d21/d27/d34/c3d 0 2024-05-17T21:52:45.704 INFO:tasks.workunit.client.0.smithi107.stdout:8/155: truncate dc/d21/d35/f2d 454301 0 2024-05-17T21:52:45.705 INFO:tasks.workunit.client.0.smithi107.stdout:5/159: mknod d5/db/dc/c34 0 2024-05-17T21:52:45.710 INFO:tasks.workunit.client.0.smithi107.stdout:8/156: dwrite dc/d21/d35/f36 [0,4194304] 0 2024-05-17T21:52:45.711 INFO:tasks.workunit.client.0.smithi107.stdout:8/157: dread - dc/d10/d17/f3b zero size 2024-05-17T21:52:45.711 INFO:tasks.workunit.client.0.smithi107.stdout:0/201: dwrite d3/dc/f14 [0,4194304] 0 2024-05-17T21:52:45.712 INFO:tasks.workunit.client.0.smithi107.stdout:8/158: fdatasync dc/d21/d35/f2e 0 2024-05-17T21:52:45.722 INFO:tasks.workunit.client.0.smithi107.stdout:0/202: dwrite d3/dc/d22/d23/f48 [0,4194304] 0 2024-05-17T21:52:45.724 INFO:tasks.workunit.client.0.smithi107.stdout:0/203: symlink d3/d39/l54 0 2024-05-17T21:52:45.731 INFO:tasks.workunit.client.0.smithi107.stdout:0/204: link d3/dc/d22/d3d/c4f d3/d24/c55 0 2024-05-17T21:52:45.737 INFO:tasks.workunit.client.0.smithi107.stdout:0/205: dwrite d3/dc/f16 [0,4194304] 0 2024-05-17T21:52:45.737 INFO:tasks.workunit.client.0.smithi107.stdout:0/206: dread - d3/dc/d22/d3d/f4d zero size 2024-05-17T21:52:45.758 INFO:tasks.workunit.client.0.smithi107.stdout:5/160: read d5/f8 [1267869,98556] 0 2024-05-17T21:52:45.758 INFO:tasks.workunit.client.0.smithi107.stdout:5/161: chown d5/db/d16/d2d/d32 46 1 2024-05-17T21:52:45.763 INFO:tasks.workunit.client.0.smithi107.stdout:5/162: unlink d5/db/dc/c26 0 2024-05-17T21:52:45.764 INFO:tasks.workunit.client.0.smithi107.stdout:5/163: truncate d5/db/dc/f24 4358282 0 2024-05-17T21:52:45.767 INFO:tasks.workunit.client.0.smithi107.stdout:4/162: truncate d3/db/f11 3520385 0 2024-05-17T21:52:45.769 INFO:tasks.workunit.client.0.smithi107.stdout:5/164: fdatasync d5/db/dc/f1f 0 2024-05-17T21:52:45.772 INFO:tasks.workunit.client.0.smithi107.stdout:4/163: mkdir d3/d6/d26/d36 0 2024-05-17T21:52:45.775 INFO:tasks.workunit.client.0.smithi107.stdout:3/164: sync 2024-05-17T21:52:45.776 INFO:tasks.workunit.client.0.smithi107.stdout:5/165: fdatasync d5/db/dc/f1f 0 2024-05-17T21:52:45.778 INFO:tasks.workunit.client.0.smithi107.stdout:7/144: readlink d5/df/l30 0 2024-05-17T21:52:45.778 INFO:tasks.workunit.client.0.smithi107.stdout:2/132: truncate d0/f25 3550541 0 2024-05-17T21:52:45.780 INFO:tasks.workunit.client.0.smithi107.stdout:4/164: chown d3/d6/d26/l32 943375 1 2024-05-17T21:52:45.785 INFO:tasks.workunit.client.0.smithi107.stdout:2/133: unlink d0/d9/f24 0 2024-05-17T21:52:45.786 INFO:tasks.workunit.client.0.smithi107.stdout:2/134: write d0/fa [1266518,59492] 0 2024-05-17T21:52:45.790 INFO:tasks.workunit.client.0.smithi107.stdout:2/135: dread d0/d9/d11/f14 [0,4194304] 0 2024-05-17T21:52:45.791 INFO:tasks.workunit.client.0.smithi107.stdout:1/164: write d4/f10 [971699,16686] 0 2024-05-17T21:52:45.792 INFO:tasks.workunit.client.0.smithi107.stdout:1/165: chown d4/df/d1a/d26 143 1 2024-05-17T21:52:45.794 INFO:tasks.workunit.client.0.smithi107.stdout:4/165: dread d3/f22 [0,4194304] 0 2024-05-17T21:52:45.796 INFO:tasks.workunit.client.0.smithi107.stdout:1/166: dread d4/de/f1e [0,4194304] 0 2024-05-17T21:52:45.799 INFO:tasks.workunit.client.0.smithi107.stdout:5/166: dread d5/db/d16/d2d/d32/f31 [0,4194304] 0 2024-05-17T21:52:45.806 INFO:tasks.workunit.client.0.smithi107.stdout:1/167: unlink d4/l2b 0 2024-05-17T21:52:45.807 INFO:tasks.workunit.client.0.smithi107.stdout:1/168: truncate d4/df/f13 1977228 0 2024-05-17T21:52:45.811 INFO:tasks.workunit.client.0.smithi107.stdout:5/167: stat d5/db/d16/d2a/d2b/c2e 0 2024-05-17T21:52:45.814 INFO:tasks.workunit.client.0.smithi107.stdout:4/166: truncate d3/db/f14 1012640 0 2024-05-17T21:52:45.814 INFO:tasks.workunit.client.0.smithi107.stdout:4/167: write d3/f2c [30363,98415] 0 2024-05-17T21:52:45.819 INFO:tasks.workunit.client.0.smithi107.stdout:5/168: rename d5/db/dc/c34 to d5/db/d16/d2d/d32/c35 0 2024-05-17T21:52:45.820 INFO:tasks.workunit.client.0.smithi107.stdout:5/169: write d5/f28 [3201473,91334] 0 2024-05-17T21:52:45.822 INFO:tasks.workunit.client.0.smithi107.stdout:7/145: link d5/ca d5/df/d11/d1d/c35 0 2024-05-17T21:52:45.824 INFO:tasks.workunit.client.0.smithi107.stdout:1/169: link d4/fa d4/df/d14/d1d/f2c 0 2024-05-17T21:52:45.825 INFO:tasks.workunit.client.0.smithi107.stdout:1/170: dread - d4/df/d14/d1d/f27 zero size 2024-05-17T21:52:45.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:45 smithi138 ceph-mon[35948]: Reconfiguring node-exporter.smithi138 (dependencies changed)... 2024-05-17T21:52:45.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:45 smithi138 ceph-mon[35948]: Reconfiguring daemon node-exporter.smithi138 on smithi138 2024-05-17T21:52:45.827 INFO:tasks.workunit.client.0.smithi107.stdout:1/171: chown d4/df/d14/d1d/f2c 4 1 2024-05-17T21:52:45.827 INFO:tasks.workunit.client.0.smithi107.stdout:9/181: rmdir d0/dd/d2f 39 2024-05-17T21:52:45.828 INFO:tasks.workunit.client.0.smithi107.stdout:4/168: dread d3/db/d16/f19 [0,4194304] 0 2024-05-17T21:52:45.832 INFO:tasks.workunit.client.0.smithi107.stdout:5/170: dwrite d5/f8 [0,4194304] 0 2024-05-17T21:52:45.832 INFO:tasks.workunit.client.0.smithi107.stdout:5/171: readlink l1 0 2024-05-17T21:52:45.841 INFO:tasks.workunit.client.0.smithi107.stdout:4/169: dwrite d3/db/d16/f19 [0,4194304] 0 2024-05-17T21:52:45.845 INFO:tasks.workunit.client.0.smithi107.stdout:0/207: unlink d3/l51 0 2024-05-17T21:52:45.845 INFO:tasks.workunit.client.0.smithi107.stdout:0/208: readlink d3/dc/d22/l47 0 2024-05-17T21:52:45.847 INFO:tasks.workunit.client.0.smithi107.stdout:5/172: symlink d5/db/l36 0 2024-05-17T21:52:45.848 INFO:tasks.workunit.client.1.smithi138.stdout:0/138: sync 2024-05-17T21:52:45.849 INFO:tasks.workunit.client.0.smithi107.stdout:3/165: sync 2024-05-17T21:52:45.849 INFO:tasks.workunit.client.1.smithi138.stdout:0/139: readlink d3/d10/l1f 0 2024-05-17T21:52:45.850 INFO:tasks.workunit.client.1.smithi138.stdout:0/140: write d3/d10/f14 [708544,105899] 0 2024-05-17T21:52:45.850 INFO:tasks.workunit.client.0.smithi107.stdout:1/172: rename d4/c9 to d4/df/c2d 0 2024-05-17T21:52:45.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:45 smithi107 ceph-mon[28434]: Reconfiguring node-exporter.smithi138 (dependencies changed)... 2024-05-17T21:52:45.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:45 smithi107 ceph-mon[28434]: Reconfiguring daemon node-exporter.smithi138 on smithi138 2024-05-17T21:52:45.857 INFO:tasks.workunit.client.0.smithi107.stdout:1/173: dwrite d4/df/d1a/f28 [0,4194304] 0 2024-05-17T21:52:45.857 INFO:tasks.workunit.client.0.smithi107.stdout:0/209: mkdir d3/dc/d22/d3d/d56 0 2024-05-17T21:52:45.857 INFO:tasks.workunit.client.1.smithi138.stdout:0/141: dwrite d3/f21 [0,4194304] 0 2024-05-17T21:52:45.858 INFO:tasks.workunit.client.0.smithi107.stdout:0/210: fsync d3/dc/d22/d3d/f52 0 2024-05-17T21:52:45.858 INFO:tasks.workunit.client.1.smithi138.stdout:0/142: stat d3/d10/f14 0 2024-05-17T21:52:45.859 INFO:tasks.workunit.client.1.smithi138.stdout:0/143: dread d3/d10/f15 [0,4194304] 0 2024-05-17T21:52:45.862 INFO:tasks.workunit.client.0.smithi107.stdout:3/166: creat d6/d8/d10/f38 x:0 0 0 2024-05-17T21:52:45.866 INFO:tasks.workunit.client.0.smithi107.stdout:9/182: rename d0/dd/l25 to d0/dd/d2f/l41 0 2024-05-17T21:52:45.866 INFO:tasks.workunit.client.0.smithi107.stdout:9/183: chown d0/dd 0 1 2024-05-17T21:52:45.866 INFO:tasks.workunit.client.0.smithi107.stdout:6/166: truncate d9/f10 2790520 0 2024-05-17T21:52:45.867 INFO:tasks.workunit.client.0.smithi107.stdout:6/167: dread - d9/d16/f30 zero size 2024-05-17T21:52:45.868 INFO:tasks.workunit.client.0.smithi107.stdout:1/174: dwrite d4/f10 [4194304,4194304] 0 2024-05-17T21:52:45.875 INFO:tasks.workunit.client.0.smithi107.stdout:0/211: creat d3/dc/d31/f57 x:0 0 0 2024-05-17T21:52:45.881 INFO:tasks.workunit.client.0.smithi107.stdout:0/212: dwrite d3/dc/f26 [0,4194304] 0 2024-05-17T21:52:45.882 INFO:tasks.workunit.client.0.smithi107.stdout:9/184: mkdir d0/dd/d2f/d42 0 2024-05-17T21:52:45.887 INFO:tasks.workunit.client.0.smithi107.stdout:1/175: rmdir d4/de 39 2024-05-17T21:52:45.894 INFO:tasks.workunit.client.0.smithi107.stdout:3/167: link d6/f33 d6/d2a/d2d/d37/f39 0 2024-05-17T21:52:45.898 INFO:tasks.workunit.client.0.smithi107.stdout:4/170: getdents d3/db/dd 0 2024-05-17T21:52:45.899 INFO:tasks.workunit.client.0.smithi107.stdout:1/176: mkdir d4/df/d14/d2e 0 2024-05-17T21:52:45.900 INFO:tasks.workunit.client.0.smithi107.stdout:1/177: readlink d4/df/d14/l17 0 2024-05-17T21:52:45.904 INFO:tasks.workunit.client.0.smithi107.stdout:0/213: mkdir d3/d58 0 2024-05-17T21:52:45.904 INFO:tasks.workunit.client.0.smithi107.stdout:0/214: readlink d3/dc/d31/l3c 0 2024-05-17T21:52:45.904 INFO:tasks.workunit.client.0.smithi107.stdout:8/159: truncate dc/d21/d35/f36 586598 0 2024-05-17T21:52:45.909 INFO:tasks.workunit.client.0.smithi107.stdout:1/178: creat d4/df/d14/d1d/f2f x:0 0 0 2024-05-17T21:52:45.916 INFO:tasks.workunit.client.0.smithi107.stdout:9/185: rename d0/dd/d1f/f23 to d0/f43 0 2024-05-17T21:52:45.918 INFO:tasks.workunit.client.0.smithi107.stdout:4/171: fsync d3/fa 0 2024-05-17T21:52:45.919 INFO:tasks.workunit.client.0.smithi107.stdout:3/168: dread d6/d8/f9 [0,4194304] 0 2024-05-17T21:52:45.919 INFO:tasks.workunit.client.0.smithi107.stdout:3/169: chown d6/d2a/d2d 499952827 1 2024-05-17T21:52:45.920 INFO:tasks.workunit.client.0.smithi107.stdout:1/179: symlink d4/df/d14/d1d/l30 0 2024-05-17T21:52:45.921 INFO:tasks.workunit.client.0.smithi107.stdout:1/180: write d4/df/d14/f23 [797248,5182] 0 2024-05-17T21:52:45.924 INFO:tasks.workunit.client.0.smithi107.stdout:9/186: write d0/fe [5716645,62761] 0 2024-05-17T21:52:45.927 INFO:tasks.workunit.client.0.smithi107.stdout:3/170: dwrite d6/d8/f14 [0,4194304] 0 2024-05-17T21:52:45.933 INFO:tasks.workunit.client.0.smithi107.stdout:8/160: dread dc/f18 [0,4194304] 0 2024-05-17T21:52:45.935 INFO:tasks.workunit.client.0.smithi107.stdout:4/172: mkdir d3/d6/d26/d36/d37 0 2024-05-17T21:52:45.942 INFO:tasks.workunit.client.0.smithi107.stdout:9/187: rename d0/c11 to d0/dd/d3d/c44 0 2024-05-17T21:52:45.945 INFO:tasks.workunit.client.0.smithi107.stdout:3/171: creat d6/d2a/d2d/d37/f3a x:0 0 0 2024-05-17T21:52:45.945 INFO:tasks.workunit.client.0.smithi107.stdout:3/172: readlink d6/d8/l32 0 2024-05-17T21:52:45.949 INFO:tasks.workunit.client.0.smithi107.stdout:8/161: mknod dc/c46 0 2024-05-17T21:52:45.950 INFO:tasks.workunit.client.0.smithi107.stdout:8/162: write dc/d21/d27/d2f/f30 [482587,90806] 0 2024-05-17T21:52:45.950 INFO:tasks.workunit.client.0.smithi107.stdout:4/173: dread d3/f9 [0,4194304] 0 2024-05-17T21:52:45.951 INFO:tasks.workunit.client.0.smithi107.stdout:4/174: write d3/db/fc [3090143,61051] 0 2024-05-17T21:52:45.954 INFO:tasks.workunit.client.0.smithi107.stdout:3/173: chown d6/d8/c15 152679 1 2024-05-17T21:52:45.954 INFO:tasks.workunit.client.0.smithi107.stdout:3/174: fsync d6/fc 0 2024-05-17T21:52:45.955 INFO:tasks.workunit.client.0.smithi107.stdout:3/175: chown d6/d8/d10/c21 55 1 2024-05-17T21:52:45.958 INFO:tasks.workunit.client.0.smithi107.stdout:8/163: creat dc/d10/d17/f47 x:0 0 0 2024-05-17T21:52:45.958 INFO:tasks.workunit.client.0.smithi107.stdout:3/176: write d6/fc [1564434,43624] 0 2024-05-17T21:52:45.958 INFO:tasks.workunit.client.0.smithi107.stdout:8/164: chown dc 1154223 1 2024-05-17T21:52:45.958 INFO:tasks.workunit.client.0.smithi107.stdout:9/188: mkdir d0/dd/d21/d29/d3e/d45 0 2024-05-17T21:52:45.965 INFO:tasks.workunit.client.0.smithi107.stdout:8/165: unlink dc/d10/l1f 0 2024-05-17T21:52:45.968 INFO:tasks.workunit.client.0.smithi107.stdout:3/177: dwrite d6/f33 [0,4194304] 0 2024-05-17T21:52:45.971 INFO:tasks.workunit.client.0.smithi107.stdout:8/166: symlink dc/d10/l48 0 2024-05-17T21:52:45.972 INFO:tasks.workunit.client.0.smithi107.stdout:8/167: write dc/d21/d35/f2d [578050,792] 0 2024-05-17T21:52:45.975 INFO:tasks.workunit.client.0.smithi107.stdout:8/168: dread dc/d21/d35/f2e [0,4194304] 0 2024-05-17T21:52:45.975 INFO:tasks.workunit.client.0.smithi107.stdout:9/189: unlink d0/c3 0 2024-05-17T21:52:45.981 INFO:tasks.workunit.client.0.smithi107.stdout:9/190: dwrite d0/f14 [0,4194304] 0 2024-05-17T21:52:45.981 INFO:tasks.workunit.client.0.smithi107.stdout:4/175: rmdir d3/db/dd/d21 0 2024-05-17T21:52:45.982 INFO:tasks.workunit.client.0.smithi107.stdout:9/191: fsync d0/dd/d2f/f31 0 2024-05-17T21:52:45.985 INFO:tasks.workunit.client.0.smithi107.stdout:8/169: mknod dc/d21/d35/c49 0 2024-05-17T21:52:45.985 INFO:tasks.workunit.client.0.smithi107.stdout:8/170: readlink dc/d21/d27/l40 0 2024-05-17T21:52:45.986 INFO:tasks.workunit.client.0.smithi107.stdout:8/171: dread dc/d21/d35/f2e [0,4194304] 0 2024-05-17T21:52:45.989 INFO:tasks.workunit.client.0.smithi107.stdout:4/176: mknod d3/db/d1e/c38 0 2024-05-17T21:52:46.001 INFO:tasks.workunit.client.0.smithi107.stdout:9/192: creat d0/dd/f46 x:0 0 0 2024-05-17T21:52:46.004 INFO:tasks.workunit.client.0.smithi107.stdout:9/193: creat d0/dd/d1f/d2c/f47 x:0 0 0 2024-05-17T21:52:46.005 INFO:tasks.workunit.client.0.smithi107.stdout:9/194: chown d0/dd/d1b/f28 3901484 1 2024-05-17T21:52:46.006 INFO:tasks.workunit.client.0.smithi107.stdout:9/195: truncate d0/dd/f46 633960 0 2024-05-17T21:52:46.008 INFO:tasks.workunit.client.0.smithi107.stdout:9/196: stat d0/dd/d1f/l3a 0 2024-05-17T21:52:46.032 INFO:tasks.workunit.client.0.smithi107.stdout:7/146: write d5/fe [8925423,80890] 0 2024-05-17T21:52:46.032 INFO:tasks.workunit.client.0.smithi107.stdout:0/215: sync 2024-05-17T21:52:46.032 INFO:tasks.workunit.client.0.smithi107.stdout:6/168: sync 2024-05-17T21:52:46.033 INFO:tasks.workunit.client.0.smithi107.stdout:7/147: stat d5/df/d11/d1c 0 2024-05-17T21:52:46.034 INFO:tasks.workunit.client.0.smithi107.stdout:0/216: dread d3/f34 [0,4194304] 0 2024-05-17T21:52:46.036 INFO:tasks.workunit.client.0.smithi107.stdout:7/148: read d5/fe [948149,84026] 0 2024-05-17T21:52:46.039 INFO:tasks.workunit.client.0.smithi107.stdout:6/169: creat d9/d16/d22/d2d/f31 x:0 0 0 2024-05-17T21:52:46.040 INFO:tasks.workunit.client.0.smithi107.stdout:6/170: chown d9/d16/d22/d2d/f31 188 1 2024-05-17T21:52:46.041 INFO:tasks.workunit.client.0.smithi107.stdout:0/217: dwrite d3/dc/d22/d23/f29 [4194304,4194304] 0 2024-05-17T21:52:46.042 INFO:tasks.workunit.client.0.smithi107.stdout:0/218: chown d3/dc/d22/d3d/f4d 20 1 2024-05-17T21:52:46.042 INFO:tasks.workunit.client.0.smithi107.stdout:0/219: write d3/ff [1403846,56018] 0 2024-05-17T21:52:46.051 INFO:tasks.workunit.client.0.smithi107.stdout:6/171: creat d9/d16/d22/f32 x:0 0 0 2024-05-17T21:52:46.052 INFO:tasks.workunit.client.0.smithi107.stdout:3/178: rmdir d6/d8 39 2024-05-17T21:52:46.052 INFO:tasks.workunit.client.0.smithi107.stdout:6/172: truncate d9/d16/d22/d2d/f2f 590742 0 2024-05-17T21:52:46.053 INFO:tasks.workunit.client.0.smithi107.stdout:6/173: truncate d9/d16/d22/f32 303544 0 2024-05-17T21:52:46.054 INFO:tasks.workunit.client.1.smithi138.stdout:1/52: getdents d4/db 0 2024-05-17T21:52:46.059 INFO:tasks.workunit.client.0.smithi107.stdout:6/174: dwrite d9/d16/d22/f27 [0,4194304] 0 2024-05-17T21:52:46.059 INFO:tasks.workunit.client.0.smithi107.stdout:6/175: dread - d9/f2a zero size 2024-05-17T21:52:46.060 INFO:tasks.workunit.client.0.smithi107.stdout:7/149: unlink d5/df/d11/d1d/c29 0 2024-05-17T21:52:46.060 INFO:tasks.workunit.client.0.smithi107.stdout:6/176: dread - d9/d16/d22/f23 zero size 2024-05-17T21:52:46.062 INFO:tasks.workunit.client.1.smithi138.stdout:9/39: creat d6/d7/f9 x:0 0 0 2024-05-17T21:52:46.063 INFO:tasks.workunit.client.1.smithi138.stdout:9/40: truncate d6/d7/f9 533412 0 2024-05-17T21:52:46.063 INFO:tasks.workunit.client.1.smithi138.stdout:9/41: truncate f3 937712 0 2024-05-17T21:52:46.065 INFO:tasks.workunit.client.0.smithi107.stdout:6/177: dwrite d9/f1d [0,4194304] 0 2024-05-17T21:52:46.066 INFO:tasks.workunit.client.0.smithi107.stdout:0/220: symlink d3/dc/d22/d3d/d56/l59 0 2024-05-17T21:52:46.069 INFO:tasks.workunit.client.0.smithi107.stdout:5/173: write d5/db/d16/d2d/d32/f33 [141002,81713] 0 2024-05-17T21:52:46.072 INFO:tasks.workunit.client.1.smithi138.stdout:4/63: mkdir d0/d3/d17 0 2024-05-17T21:52:46.073 INFO:tasks.workunit.client.0.smithi107.stdout:0/221: dwrite d3/dc/d22/d23/f44 [0,4194304] 0 2024-05-17T21:52:46.077 INFO:tasks.workunit.client.1.smithi138.stdout:2/34: rmdir d3 39 2024-05-17T21:52:46.081 INFO:tasks.workunit.client.1.smithi138.stdout:7/58: creat d0/f13 x:0 0 0 2024-05-17T21:52:46.082 INFO:tasks.workunit.client.1.smithi138.stdout:7/59: truncate d0/f12 145425 0 2024-05-17T21:52:46.082 INFO:tasks.workunit.client.1.smithi138.stdout:7/60: stat d0/fa 0 2024-05-17T21:52:46.083 INFO:tasks.workunit.client.1.smithi138.stdout:7/61: read - d0/f10 zero size 2024-05-17T21:52:46.083 INFO:tasks.workunit.client.1.smithi138.stdout:6/120: symlink d7/db/l23 0 2024-05-17T21:52:46.087 INFO:tasks.workunit.client.0.smithi107.stdout:5/174: symlink d5/db/d16/d2d/l37 0 2024-05-17T21:52:46.089 INFO:tasks.workunit.client.1.smithi138.stdout:7/62: dwrite d0/f10 [0,4194304] 0 2024-05-17T21:52:46.090 INFO:tasks.workunit.client.1.smithi138.stdout:7/63: write d0/f13 [456710,95150] 0 2024-05-17T21:52:46.095 INFO:tasks.workunit.client.1.smithi138.stdout:0/144: mkdir d3/d10/d38 0 2024-05-17T21:52:46.095 INFO:tasks.workunit.client.0.smithi107.stdout:0/222: dwrite f1 [0,4194304] 0 2024-05-17T21:52:46.096 INFO:tasks.workunit.client.1.smithi138.stdout:0/145: fdatasync d3/d10/f11 0 2024-05-17T21:52:46.096 INFO:tasks.workunit.client.0.smithi107.stdout:0/223: write d3/dc/f16 [2774402,120726] 0 2024-05-17T21:52:46.102 INFO:tasks.workunit.client.0.smithi107.stdout:3/179: getdents d6/d2a 0 2024-05-17T21:52:46.106 INFO:tasks.workunit.client.1.smithi138.stdout:9/42: symlink d6/d7/la 0 2024-05-17T21:52:46.106 INFO:tasks.workunit.client.0.smithi107.stdout:0/224: symlink d3/dc/d22/d3d/d56/l5a 0 2024-05-17T21:52:46.107 INFO:tasks.workunit.client.1.smithi138.stdout:3/42: mkdir d1/d3/d8 0 2024-05-17T21:52:46.111 INFO:tasks.workunit.client.1.smithi138.stdout:2/35: chown d3/f9 95 1 2024-05-17T21:52:46.113 INFO:tasks.workunit.client.1.smithi138.stdout:6/121: creat d7/d8/f24 x:0 0 0 2024-05-17T21:52:46.113 INFO:tasks.workunit.client.1.smithi138.stdout:6/122: chown d7/db/d12/f21 86 1 2024-05-17T21:52:46.116 INFO:tasks.workunit.client.0.smithi107.stdout:3/180: mknod d6/d8/d10/d25/c3b 0 2024-05-17T21:52:46.119 INFO:tasks.workunit.client.1.smithi138.stdout:7/64: symlink d0/l14 0 2024-05-17T21:52:46.119 INFO:tasks.workunit.client.1.smithi138.stdout:7/65: chown d0/lf 26143705 1 2024-05-17T21:52:46.123 INFO:tasks.workunit.client.1.smithi138.stdout:0/146: readlink d3/l34 0 2024-05-17T21:52:46.125 INFO:tasks.workunit.client.0.smithi107.stdout:0/225: getdents d3/dc/d22/d3d 0 2024-05-17T21:52:46.127 INFO:tasks.workunit.client.1.smithi138.stdout:9/43: creat d6/d7/fb x:0 0 0 2024-05-17T21:52:46.127 INFO:tasks.workunit.client.1.smithi138.stdout:9/44: chown d6/d7/fb 1 1 2024-05-17T21:52:46.128 INFO:tasks.workunit.client.1.smithi138.stdout:3/43: mknod d1/c9 0 2024-05-17T21:52:46.128 INFO:tasks.workunit.client.1.smithi138.stdout:9/45: truncate f5 5127593 0 2024-05-17T21:52:46.129 INFO:tasks.workunit.client.0.smithi107.stdout:0/226: creat d3/dc/d22/d3d/d56/f5b x:0 0 0 2024-05-17T21:52:46.130 INFO:tasks.workunit.client.0.smithi107.stdout:0/227: fdatasync d3/f35 0 2024-05-17T21:52:46.130 INFO:tasks.workunit.client.1.smithi138.stdout:4/64: creat d0/d3/db/d15/f18 x:0 0 0 2024-05-17T21:52:46.131 INFO:tasks.workunit.client.1.smithi138.stdout:2/36: write d3/f6 [204547,112106] 0 2024-05-17T21:52:46.132 INFO:tasks.workunit.client.0.smithi107.stdout:1/181: dwrite f3 [0,4194304] 0 2024-05-17T21:52:46.133 INFO:tasks.workunit.client.1.smithi138.stdout:6/123: unlink f3 0 2024-05-17T21:52:46.134 INFO:tasks.workunit.client.0.smithi107.stdout:1/182: write d4/df/d14/d1d/f27 [647442,26255] 0 2024-05-17T21:52:46.135 INFO:tasks.workunit.client.1.smithi138.stdout:3/44: dwrite d1/f4 [0,4194304] 0 2024-05-17T21:52:46.136 INFO:tasks.workunit.client.1.smithi138.stdout:5/70: link d1/f7 d1/d6/fc 0 2024-05-17T21:52:46.141 INFO:tasks.workunit.client.0.smithi107.stdout:0/228: mknod d3/d24/c5c 0 2024-05-17T21:52:46.142 INFO:tasks.workunit.client.1.smithi138.stdout:0/147: mkdir d3/d10/d39 0 2024-05-17T21:52:46.144 INFO:tasks.workunit.client.1.smithi138.stdout:8/48: link d0/f2 d0/f9 0 2024-05-17T21:52:46.145 INFO:tasks.workunit.client.0.smithi107.stdout:0/229: readlink d3/dc/l4e 0 2024-05-17T21:52:46.147 INFO:tasks.workunit.client.0.smithi107.stdout:1/183: mknod d4/c31 0 2024-05-17T21:52:46.149 INFO:tasks.workunit.client.1.smithi138.stdout:9/46: rename f3 to d6/d7/fc 0 2024-05-17T21:52:46.150 INFO:tasks.workunit.client.1.smithi138.stdout:4/65: mknod d0/d3/db/c19 0 2024-05-17T21:52:46.153 INFO:tasks.workunit.client.1.smithi138.stdout:2/37: creat d3/fd x:0 0 0 2024-05-17T21:52:46.154 INFO:tasks.workunit.client.0.smithi107.stdout:1/184: dread d4/df/f13 [0,4194304] 0 2024-05-17T21:52:46.154 INFO:tasks.workunit.client.1.smithi138.stdout:2/38: readlink d3/lb 0 2024-05-17T21:52:46.154 INFO:tasks.workunit.client.1.smithi138.stdout:2/39: write d3/f5 [544912,73371] 0 2024-05-17T21:52:46.155 INFO:tasks.workunit.client.1.smithi138.stdout:2/40: read - d3/fc zero size 2024-05-17T21:52:46.157 INFO:tasks.workunit.client.1.smithi138.stdout:6/124: unlink d7/f18 0 2024-05-17T21:52:46.158 INFO:tasks.workunit.client.1.smithi138.stdout:3/45: mknod d1/ca 0 2024-05-17T21:52:46.158 INFO:tasks.workunit.client.1.smithi138.stdout:6/125: write f1 [6298464,51515] 0 2024-05-17T21:52:46.163 INFO:tasks.workunit.client.0.smithi107.stdout:1/185: mkdir d4/df/d14/d2e/d32 0 2024-05-17T21:52:46.164 INFO:tasks.workunit.client.1.smithi138.stdout:2/41: dwrite d3/f5 [0,4194304] 0 2024-05-17T21:52:46.165 INFO:tasks.workunit.client.1.smithi138.stdout:2/42: dread - d3/fd zero size 2024-05-17T21:52:46.167 INFO:tasks.workunit.client.1.smithi138.stdout:2/43: fsync d3/f4 0 2024-05-17T21:52:46.167 INFO:tasks.workunit.client.1.smithi138.stdout:8/49: creat d0/fa x:0 0 0 2024-05-17T21:52:46.170 INFO:tasks.workunit.client.1.smithi138.stdout:5/71: dwrite d1/f7 [4194304,4194304] 0 2024-05-17T21:52:46.171 INFO:tasks.workunit.client.1.smithi138.stdout:5/72: truncate d1/d6/fc 8714872 0 2024-05-17T21:52:46.172 INFO:tasks.workunit.client.0.smithi107.stdout:1/186: rename d4/df/d14/d2e/d32 to d4/df/d14/d33 0 2024-05-17T21:52:46.173 INFO:tasks.workunit.client.0.smithi107.stdout:8/172: rmdir dc/d10 39 2024-05-17T21:52:46.173 INFO:tasks.workunit.client.0.smithi107.stdout:1/187: stat d4/df/d14/d1d/f2c 0 2024-05-17T21:52:46.174 INFO:tasks.workunit.client.0.smithi107.stdout:1/188: write d4/df/d1a/f28 [1855716,30799] 0 2024-05-17T21:52:46.174 INFO:tasks.workunit.client.0.smithi107.stdout:8/173: truncate dc/d21/d27/f37 814453 0 2024-05-17T21:52:46.175 INFO:tasks.workunit.client.1.smithi138.stdout:9/47: mknod d6/cd 0 2024-05-17T21:52:46.175 INFO:tasks.workunit.client.0.smithi107.stdout:1/189: write d4/df/d14/f23 [961218,20851] 0 2024-05-17T21:52:46.180 INFO:tasks.workunit.client.0.smithi107.stdout:8/174: chown dc/d10/f26 778 1 2024-05-17T21:52:46.181 INFO:tasks.workunit.client.0.smithi107.stdout:8/175: stat dc/l24 0 2024-05-17T21:52:46.184 INFO:tasks.workunit.client.0.smithi107.stdout:1/190: symlink d4/df/d14/l34 0 2024-05-17T21:52:46.185 INFO:tasks.workunit.client.0.smithi107.stdout:1/191: rename d4/df/d1a to d4/df/d1a/d35 22 2024-05-17T21:52:46.188 INFO:tasks.workunit.client.1.smithi138.stdout:6/126: write d7/d8/ff [2997613,92181] 0 2024-05-17T21:52:46.190 INFO:tasks.workunit.client.1.smithi138.stdout:6/127: dread d7/fc [0,4194304] 0 2024-05-17T21:52:46.192 INFO:tasks.workunit.client.1.smithi138.stdout:3/46: rename d1/c7 to d1/d3/cb 0 2024-05-17T21:52:46.194 INFO:tasks.workunit.client.1.smithi138.stdout:0/148: creat d3/d10/d38/f3a x:0 0 0 2024-05-17T21:52:46.201 INFO:tasks.workunit.client.0.smithi107.stdout:2/136: write d0/f25 [3237367,92013] 0 2024-05-17T21:52:46.202 INFO:tasks.workunit.client.0.smithi107.stdout:4/177: unlink d3/db/f11 0 2024-05-17T21:52:46.202 INFO:tasks.workunit.client.1.smithi138.stdout:0/149: dwrite d3/d10/f17 [0,4194304] 0 2024-05-17T21:52:46.203 INFO:tasks.workunit.client.1.smithi138.stdout:0/150: fdatasync f2 0 2024-05-17T21:52:46.203 INFO:tasks.workunit.client.1.smithi138.stdout:8/50: creat d0/fb x:0 0 0 2024-05-17T21:52:46.204 INFO:tasks.workunit.client.0.smithi107.stdout:1/192: dread d4/df/d14/d1d/f2c [0,4194304] 0 2024-05-17T21:52:46.204 INFO:tasks.workunit.client.0.smithi107.stdout:8/176: link dc/d10/d17/f1b dc/d21/d27/d2f/f4a 0 2024-05-17T21:52:46.204 INFO:tasks.workunit.client.0.smithi107.stdout:1/193: read - d4/df/f29 zero size 2024-05-17T21:52:46.207 INFO:tasks.workunit.client.1.smithi138.stdout:9/48: symlink d6/le 0 2024-05-17T21:52:46.209 INFO:tasks.workunit.client.1.smithi138.stdout:4/66: creat d0/d3/d17/f1a x:0 0 0 2024-05-17T21:52:46.212 INFO:tasks.workunit.client.1.smithi138.stdout:6/128: creat d7/db/f25 x:0 0 0 2024-05-17T21:52:46.215 INFO:tasks.workunit.client.1.smithi138.stdout:3/47: unlink d1/c5 0 2024-05-17T21:52:46.215 INFO:tasks.workunit.client.0.smithi107.stdout:9/197: getdents d0/dd/d1f/d2c 0 2024-05-17T21:52:46.218 INFO:tasks.workunit.client.1.smithi138.stdout:4/67: dwrite d0/d3/db/d15/f18 [0,4194304] 0 2024-05-17T21:52:46.219 INFO:tasks.workunit.client.1.smithi138.stdout:4/68: chown d0/l9 13 1 2024-05-17T21:52:46.220 INFO:tasks.workunit.client.1.smithi138.stdout:8/51: mkdir d0/dc 0 2024-05-17T21:52:46.221 INFO:tasks.workunit.client.1.smithi138.stdout:3/48: dread d1/f4 [0,4194304] 0 2024-05-17T21:52:46.224 INFO:tasks.workunit.client.1.smithi138.stdout:5/73: symlink d1/ld 0 2024-05-17T21:52:46.227 INFO:tasks.workunit.client.1.smithi138.stdout:9/49: write d6/d7/fc [893970,5021] 0 2024-05-17T21:52:46.228 INFO:tasks.workunit.client.0.smithi107.stdout:9/198: write d0/dd/d21/f27 [212156,26249] 0 2024-05-17T21:52:46.229 INFO:tasks.workunit.client.1.smithi138.stdout:6/129: creat d7/db/d12/f26 x:0 0 0 2024-05-17T21:52:46.231 INFO:tasks.workunit.client.0.smithi107.stdout:1/194: creat d4/de/f36 x:0 0 0 2024-05-17T21:52:46.236 INFO:tasks.workunit.client.0.smithi107.stdout:9/199: rename d0/dd/d21/f27 to d0/dd/d1b/f48 0 2024-05-17T21:52:46.237 INFO:tasks.workunit.client.0.smithi107.stdout:9/200: write d0/dd/d2f/f30 [84557,54915] 0 2024-05-17T21:52:46.239 INFO:tasks.workunit.client.1.smithi138.stdout:0/151: symlink d3/d10/d39/l3b 0 2024-05-17T21:52:46.241 INFO:tasks.workunit.client.1.smithi138.stdout:3/49: truncate d1/f2 6885920 0 2024-05-17T21:52:46.241 INFO:tasks.workunit.client.0.smithi107.stdout:9/201: rmdir d0/dd/d3d 39 2024-05-17T21:52:46.242 INFO:tasks.workunit.client.1.smithi138.stdout:5/74: creat d1/d6/fe x:0 0 0 2024-05-17T21:52:46.244 INFO:tasks.workunit.client.0.smithi107.stdout:1/195: rename d4/de/l11 to d4/df/d14/l37 0 2024-05-17T21:52:46.245 INFO:tasks.workunit.client.1.smithi138.stdout:9/50: symlink d6/lf 0 2024-05-17T21:52:46.246 INFO:tasks.workunit.client.0.smithi107.stdout:9/202: write d0/dd/d1f/d2c/f34 [164117,95288] 0 2024-05-17T21:52:46.246 INFO:tasks.workunit.client.1.smithi138.stdout:6/130: creat d7/db/d12/f27 x:0 0 0 2024-05-17T21:52:46.249 INFO:tasks.workunit.client.0.smithi107.stdout:9/203: creat d0/dd/d21/d29/d3e/f49 x:0 0 0 2024-05-17T21:52:46.250 INFO:tasks.workunit.client.0.smithi107.stdout:9/204: write d0/f17 [1954453,16221] 0 2024-05-17T21:52:46.250 INFO:tasks.workunit.client.1.smithi138.stdout:9/51: dwrite f5 [0,4194304] 0 2024-05-17T21:52:46.250 INFO:tasks.workunit.client.1.smithi138.stdout:9/52: readlink d6/d7/la 0 2024-05-17T21:52:46.253 INFO:tasks.workunit.client.0.smithi107.stdout:1/196: creat d4/f38 x:0 0 0 2024-05-17T21:52:46.255 INFO:tasks.workunit.client.1.smithi138.stdout:8/52: dread d0/f2 [0,4194304] 0 2024-05-17T21:52:46.256 INFO:tasks.workunit.client.0.smithi107.stdout:9/205: mkdir d0/dd/d2f/d4a 0 2024-05-17T21:52:46.257 INFO:tasks.workunit.client.1.smithi138.stdout:5/75: unlink d1/f5 0 2024-05-17T21:52:46.258 INFO:tasks.workunit.client.0.smithi107.stdout:1/197: symlink d4/df/d14/l39 0 2024-05-17T21:52:46.260 INFO:tasks.workunit.client.0.smithi107.stdout:9/206: symlink d0/dd/d1b/l4b 0 2024-05-17T21:52:46.260 INFO:tasks.workunit.client.0.smithi107.stdout:9/207: read d0/f4 [1128310,64120] 0 2024-05-17T21:52:46.261 INFO:tasks.workunit.client.1.smithi138.stdout:6/131: stat d7/db/d12/c1f 0 2024-05-17T21:52:46.261 INFO:tasks.workunit.client.0.smithi107.stdout:9/208: stat d0/f14 0 2024-05-17T21:52:46.262 INFO:tasks.workunit.client.0.smithi107.stdout:9/209: write d0/dd/f46 [533143,82880] 0 2024-05-17T21:52:46.262 INFO:tasks.workunit.client.0.smithi107.stdout:1/198: stat d4/df/c2d 0 2024-05-17T21:52:46.264 INFO:tasks.workunit.client.0.smithi107.stdout:9/210: dread d0/dd/d1f/d2c/f34 [0,4194304] 0 2024-05-17T21:52:46.271 INFO:tasks.workunit.client.0.smithi107.stdout:9/211: rename d0/f14 to d0/dd/d2f/d4a/f4c 0 2024-05-17T21:52:46.273 INFO:tasks.workunit.client.0.smithi107.stdout:1/199: getdents d4/df/d1a/d26 0 2024-05-17T21:52:46.273 INFO:tasks.workunit.client.0.smithi107.stdout:1/200: chown d4/de 619048 1 2024-05-17T21:52:46.276 INFO:tasks.workunit.client.0.smithi107.stdout:7/150: write d5/df/d11/d1c/f2e [777649,118292] 0 2024-05-17T21:52:46.277 INFO:tasks.workunit.client.0.smithi107.stdout:9/212: rename d0/dd/d1f/d2c/c33 to d0/dd/d2f/d4a/c4d 0 2024-05-17T21:52:46.278 INFO:tasks.workunit.client.0.smithi107.stdout:1/201: mkdir d4/de/d3a 0 2024-05-17T21:52:46.279 INFO:tasks.workunit.client.0.smithi107.stdout:1/202: dread - d4/de/f36 zero size 2024-05-17T21:52:46.283 INFO:tasks.workunit.client.0.smithi107.stdout:1/203: mkdir d4/df/d14/d3b 0 2024-05-17T21:52:46.284 INFO:tasks.workunit.client.0.smithi107.stdout:1/204: chown d4/de/f19 3548 1 2024-05-17T21:52:46.285 INFO:tasks.workunit.client.0.smithi107.stdout:1/205: creat d4/de/f3c x:0 0 0 2024-05-17T21:52:46.285 INFO:tasks.workunit.client.0.smithi107.stdout:5/175: write d5/db/d16/d2d/d32/f31 [3345581,57916] 0 2024-05-17T21:52:46.287 INFO:tasks.workunit.client.0.smithi107.stdout:5/176: creat d5/db/d16/d2d/d32/f38 x:0 0 0 2024-05-17T21:52:46.288 INFO:tasks.workunit.client.0.smithi107.stdout:5/177: write f3 [2279442,113115] 0 2024-05-17T21:52:46.289 INFO:tasks.workunit.client.0.smithi107.stdout:1/206: rename d4/df/c18 to d4/df/d14/d33/c3d 0 2024-05-17T21:52:46.289 INFO:tasks.workunit.client.0.smithi107.stdout:1/207: chown d4/df/d14/d2e 2 1 2024-05-17T21:52:46.291 INFO:tasks.workunit.client.0.smithi107.stdout:1/208: symlink d4/de/l3e 0 2024-05-17T21:52:46.292 INFO:tasks.workunit.client.0.smithi107.stdout:5/178: mkdir d5/db/d39 0 2024-05-17T21:52:46.294 INFO:tasks.workunit.client.0.smithi107.stdout:0/230: fsync d3/dc/d22/d3d/d56/f5b 0 2024-05-17T21:52:46.294 INFO:tasks.workunit.client.0.smithi107.stdout:5/179: truncate d5/db/f25 950401 0 2024-05-17T21:52:46.295 INFO:tasks.workunit.client.0.smithi107.stdout:3/181: dwrite d6/d8/d10/f23 [0,4194304] 0 2024-05-17T21:52:46.295 INFO:tasks.workunit.client.0.smithi107.stdout:1/209: link d4/c31 d4/df/d1a/c3f 0 2024-05-17T21:52:46.296 INFO:tasks.workunit.client.0.smithi107.stdout:1/210: readlink d4/df/d14/l17 0 2024-05-17T21:52:46.298 INFO:tasks.workunit.client.0.smithi107.stdout:0/231: mkdir d3/dc/d22/d5d 0 2024-05-17T21:52:46.300 INFO:tasks.workunit.client.0.smithi107.stdout:5/180: dread d5/f23 [0,4194304] 0 2024-05-17T21:52:46.301 INFO:tasks.workunit.client.0.smithi107.stdout:3/182: rename d6/d8/f27 to d6/d8/d10/d35/f3c 0 2024-05-17T21:52:46.303 INFO:tasks.workunit.client.0.smithi107.stdout:0/232: unlink d3/dc/d22/d3d/d56/l5a 0 2024-05-17T21:52:46.303 INFO:tasks.workunit.client.0.smithi107.stdout:1/211: dwrite d4/f10 [0,4194304] 0 2024-05-17T21:52:46.306 INFO:tasks.workunit.client.0.smithi107.stdout:5/181: mkdir d5/db/dc/d1e/d3a 0 2024-05-17T21:52:46.307 INFO:tasks.workunit.client.0.smithi107.stdout:3/183: creat d6/d8/d10/d35/d36/f3d x:0 0 0 2024-05-17T21:52:46.310 INFO:tasks.workunit.client.0.smithi107.stdout:3/184: dread f2 [0,4194304] 0 2024-05-17T21:52:46.314 INFO:tasks.workunit.client.0.smithi107.stdout:5/182: rmdir d5/db/dc/d1e/d3a 0 2024-05-17T21:52:46.319 INFO:tasks.workunit.client.1.smithi138.stdout:6/132: read d7/d8/ff [3249161,3714] 0 2024-05-17T21:52:46.321 INFO:tasks.workunit.client.1.smithi138.stdout:9/53: creat d6/d7/f10 x:0 0 0 2024-05-17T21:52:46.328 INFO:tasks.workunit.client.1.smithi138.stdout:1/53: write f3 [540239,31215] 0 2024-05-17T21:52:46.333 INFO:tasks.workunit.client.1.smithi138.stdout:4/69: rename d0/d3/db/d15 to d0/d1b 0 2024-05-17T21:52:46.333 INFO:tasks.workunit.client.1.smithi138.stdout:6/133: dwrite d7/d8/f20 [0,4194304] 0 2024-05-17T21:52:46.336 INFO:tasks.workunit.client.1.smithi138.stdout:2/44: rmdir d3 39 2024-05-17T21:52:46.336 INFO:tasks.workunit.client.1.smithi138.stdout:7/66: write d0/f11 [148183,88352] 0 2024-05-17T21:52:46.337 INFO:tasks.workunit.client.1.smithi138.stdout:9/54: dread f5 [0,4194304] 0 2024-05-17T21:52:46.340 INFO:tasks.workunit.client.1.smithi138.stdout:5/76: symlink d1/lf 0 2024-05-17T21:52:46.340 INFO:tasks.workunit.client.1.smithi138.stdout:1/54: unlink d4/c5 0 2024-05-17T21:52:46.341 INFO:tasks.workunit.client.1.smithi138.stdout:6/134: creat d7/f28 x:0 0 0 2024-05-17T21:52:46.342 INFO:tasks.workunit.client.1.smithi138.stdout:6/135: fdatasync f1 0 2024-05-17T21:52:46.343 INFO:tasks.workunit.client.1.smithi138.stdout:6/136: dread - d7/d8/f24 zero size 2024-05-17T21:52:46.344 INFO:tasks.workunit.client.1.smithi138.stdout:6/137: stat d7/cd 0 2024-05-17T21:52:46.345 INFO:tasks.workunit.client.1.smithi138.stdout:7/67: mknod d0/c15 0 2024-05-17T21:52:46.345 INFO:tasks.workunit.client.1.smithi138.stdout:9/55: dread d6/d7/fc [0,4194304] 0 2024-05-17T21:52:46.346 INFO:tasks.workunit.client.1.smithi138.stdout:7/68: write d0/f12 [300261,34525] 0 2024-05-17T21:52:46.346 INFO:tasks.workunit.client.1.smithi138.stdout:9/56: dread - d6/d7/fb zero size 2024-05-17T21:52:46.351 INFO:tasks.workunit.client.1.smithi138.stdout:3/50: dread d1/f2 [0,4194304] 0 2024-05-17T21:52:46.352 INFO:tasks.workunit.client.0.smithi107.stdout:4/178: truncate d3/db/f33 4635527 0 2024-05-17T21:52:46.352 INFO:tasks.workunit.client.0.smithi107.stdout:2/137: write d0/d9/d11/f14 [8830068,13106] 0 2024-05-17T21:52:46.353 INFO:tasks.workunit.client.0.smithi107.stdout:8/177: write dc/d10/d17/f1b [52880,95662] 0 2024-05-17T21:52:46.354 INFO:tasks.workunit.client.1.smithi138.stdout:5/77: symlink d1/d6/l10 0 2024-05-17T21:52:46.355 INFO:tasks.workunit.client.1.smithi138.stdout:5/78: chown d1/d6/l10 4246 1 2024-05-17T21:52:46.355 INFO:tasks.workunit.client.0.smithi107.stdout:3/185: dwrite d6/d8/d10/f23 [0,4194304] 0 2024-05-17T21:52:46.356 INFO:tasks.workunit.client.0.smithi107.stdout:5/183: mknod d5/db/d16/d2d/d32/c3b 0 2024-05-17T21:52:46.356 INFO:tasks.workunit.client.0.smithi107.stdout:4/179: dread d3/fa [0,4194304] 0 2024-05-17T21:52:46.357 INFO:tasks.workunit.client.0.smithi107.stdout:2/138: truncate d0/d9/d11/d18/f20 790623 0 2024-05-17T21:52:46.357 INFO:tasks.workunit.client.0.smithi107.stdout:5/184: chown d5/db/d16/d2a/l2c 1368 1 2024-05-17T21:52:46.360 INFO:tasks.workunit.client.0.smithi107.stdout:4/180: read d3/db/dd/f12 [2654578,73050] 0 2024-05-17T21:52:46.360 INFO:tasks.workunit.client.1.smithi138.stdout:2/45: creat d3/fe x:0 0 0 2024-05-17T21:52:46.361 INFO:tasks.workunit.client.1.smithi138.stdout:2/46: write d3/f9 [584130,15338] 0 2024-05-17T21:52:46.362 INFO:tasks.workunit.client.0.smithi107.stdout:8/178: mkdir dc/d44/d4b 0 2024-05-17T21:52:46.363 INFO:tasks.workunit.client.1.smithi138.stdout:6/138: symlink d7/d8/l29 0 2024-05-17T21:52:46.364 INFO:tasks.workunit.client.0.smithi107.stdout:5/185: dwrite d5/f17 [0,4194304] 0 2024-05-17T21:52:46.365 INFO:tasks.workunit.client.0.smithi107.stdout:5/186: chown d5/db/f1a 949147 1 2024-05-17T21:52:46.365 INFO:tasks.workunit.client.0.smithi107.stdout:4/181: dread d3/f8 [0,4194304] 0 2024-05-17T21:52:46.366 INFO:tasks.workunit.client.0.smithi107.stdout:4/182: chown d3/d6/d26/d36/d37 1212 1 2024-05-17T21:52:46.369 INFO:tasks.workunit.client.0.smithi107.stdout:4/183: write d3/db/d16/f19 [664516,67487] 0 2024-05-17T21:52:46.370 INFO:tasks.workunit.client.1.smithi138.stdout:8/53: write d0/f2 [299390,36882] 0 2024-05-17T21:52:46.372 INFO:tasks.workunit.client.0.smithi107.stdout:3/186: mkdir d6/d8/d3e 0 2024-05-17T21:52:46.375 INFO:tasks.workunit.client.0.smithi107.stdout:8/179: mkdir dc/d21/d27/d34/d4c 0 2024-05-17T21:52:46.381 INFO:tasks.workunit.client.1.smithi138.stdout:3/51: creat d1/fc x:0 0 0 2024-05-17T21:52:46.381 INFO:tasks.workunit.client.0.smithi107.stdout:9/213: getdents d0/dd/d2f/d4a 0 2024-05-17T21:52:46.383 INFO:tasks.workunit.client.1.smithi138.stdout:1/55: symlink d4/dd/le 0 2024-05-17T21:52:46.386 INFO:tasks.workunit.client.0.smithi107.stdout:2/139: truncate d0/d9/d11/f10 2396517 0 2024-05-17T21:52:46.387 INFO:tasks.workunit.client.0.smithi107.stdout:1/212: getdents d4/df/d14 0 2024-05-17T21:52:46.389 INFO:tasks.workunit.client.0.smithi107.stdout:4/184: rename d3/c1d to d3/d6/d26/d36/d37/c39 0 2024-05-17T21:52:46.390 INFO:tasks.workunit.client.1.smithi138.stdout:6/139: mknod d7/d8/c2a 0 2024-05-17T21:52:46.390 INFO:tasks.workunit.client.1.smithi138.stdout:6/140: chown d7/c1b 45795 1 2024-05-17T21:52:46.392 INFO:tasks.workunit.client.1.smithi138.stdout:6/141: write d7/db/d12/f27 [298058,35748] 0 2024-05-17T21:52:46.395 INFO:tasks.workunit.client.0.smithi107.stdout:5/187: creat d5/db/d39/f3c x:0 0 0 2024-05-17T21:52:46.396 INFO:tasks.workunit.client.0.smithi107.stdout:5/188: write d5/f17 [1007207,111271] 0 2024-05-17T21:52:46.400 INFO:tasks.workunit.client.0.smithi107.stdout:5/189: dread d5/db/dc/d18/f27 [0,4194304] 0 2024-05-17T21:52:46.400 INFO:tasks.workunit.client.1.smithi138.stdout:4/70: getdents d0/d3/db 0 2024-05-17T21:52:46.403 INFO:tasks.workunit.client.0.smithi107.stdout:1/213: stat d4/df/d14/d33/c3d 0 2024-05-17T21:52:46.404 INFO:tasks.workunit.client.1.smithi138.stdout:7/69: link d0/l14 d0/l16 0 2024-05-17T21:52:46.408 INFO:tasks.workunit.client.0.smithi107.stdout:4/185: mkdir d3/db/dd/d3a 0 2024-05-17T21:52:46.408 INFO:tasks.workunit.client.0.smithi107.stdout:4/186: chown d3/db/d16/f19 1231854018 1 2024-05-17T21:52:46.409 INFO:tasks.workunit.client.0.smithi107.stdout:4/187: chown d3/db/fc 122026 1 2024-05-17T21:52:46.409 INFO:tasks.workunit.client.0.smithi107.stdout:4/188: fsync d3/db/d16/f19 0 2024-05-17T21:52:46.410 INFO:tasks.workunit.client.0.smithi107.stdout:3/187: mkdir d6/d8/d3e/d3f 0 2024-05-17T21:52:46.410 INFO:tasks.workunit.client.0.smithi107.stdout:4/189: fdatasync d3/db/fc 0 2024-05-17T21:52:46.411 INFO:tasks.workunit.client.0.smithi107.stdout:3/188: write d6/fc [1651685,89367] 0 2024-05-17T21:52:46.411 INFO:tasks.workunit.client.0.smithi107.stdout:3/189: readlink d6/d2a/l2b 0 2024-05-17T21:52:46.412 INFO:tasks.workunit.client.0.smithi107.stdout:3/190: readlink d6/d8/d10/d25/l26 0 2024-05-17T21:52:46.416 INFO:tasks.workunit.client.1.smithi138.stdout:6/142: getdents d7/db 0 2024-05-17T21:52:46.416 INFO:tasks.workunit.client.1.smithi138.stdout:9/57: truncate f5 2501956 0 2024-05-17T21:52:46.417 INFO:tasks.workunit.client.1.smithi138.stdout:9/58: dread - d6/d7/fb zero size 2024-05-17T21:52:46.418 INFO:tasks.workunit.client.1.smithi138.stdout:6/143: creat d7/d8/f2b x:0 0 0 2024-05-17T21:52:46.419 INFO:tasks.workunit.client.1.smithi138.stdout:2/47: getdents d3 0 2024-05-17T21:52:46.421 INFO:tasks.workunit.client.1.smithi138.stdout:9/59: fsync f2 0 2024-05-17T21:52:46.422 INFO:tasks.workunit.client.1.smithi138.stdout:9/60: chown f2 27 1 2024-05-17T21:52:46.423 INFO:tasks.workunit.client.0.smithi107.stdout:9/214: mknod d0/dd/d21/d29/d3e/d45/c4e 0 2024-05-17T21:52:46.424 INFO:tasks.workunit.client.1.smithi138.stdout:6/144: write d7/db/f11 [693375,52848] 0 2024-05-17T21:52:46.425 INFO:tasks.workunit.client.0.smithi107.stdout:7/151: symlink d5/df/d11/d2a/l36 0 2024-05-17T21:52:46.428 INFO:tasks.workunit.client.0.smithi107.stdout:5/190: rename d5/db/d16/d2d/d32/f31 to d5/db/d16/d2a/d2b/f3d 0 2024-05-17T21:52:46.428 INFO:tasks.workunit.client.0.smithi107.stdout:5/191: chown d5/db 0 1 2024-05-17T21:52:46.428 INFO:tasks.workunit.client.0.smithi107.stdout:9/215: dwrite d0/dd/f35 [0,4194304] 0 2024-05-17T21:52:46.429 INFO:tasks.workunit.client.1.smithi138.stdout:9/61: creat d6/f11 x:0 0 0 2024-05-17T21:52:46.429 INFO:tasks.workunit.client.0.smithi107.stdout:5/192: chown d5/db/dc/d18 94831181 1 2024-05-17T21:52:46.430 INFO:tasks.workunit.client.1.smithi138.stdout:6/145: symlink d7/db/d12/l2c 0 2024-05-17T21:52:46.432 INFO:tasks.workunit.client.0.smithi107.stdout:6/178: write d9/f10 [609656,64721] 0 2024-05-17T21:52:46.433 INFO:tasks.workunit.client.1.smithi138.stdout:2/48: link d3/fe d3/ff 0 2024-05-17T21:52:46.435 INFO:tasks.workunit.client.0.smithi107.stdout:5/193: dread d5/f1b [0,4194304] 0 2024-05-17T21:52:46.436 INFO:tasks.workunit.client.0.smithi107.stdout:5/194: chown d5/db/dc/d18/f27 11626487 1 2024-05-17T21:52:46.438 INFO:tasks.workunit.client.1.smithi138.stdout:6/146: mkdir d7/d2d 0 2024-05-17T21:52:46.439 INFO:tasks.workunit.client.1.smithi138.stdout:6/147: stat d7/f22 0 2024-05-17T21:52:46.439 INFO:tasks.workunit.client.0.smithi107.stdout:5/195: dread d5/db/d16/d2a/d2b/f3d [4194304,4194304] 0 2024-05-17T21:52:46.440 INFO:tasks.workunit.client.0.smithi107.stdout:4/190: mkdir d3/d6/d26/d3b 0 2024-05-17T21:52:46.440 INFO:tasks.workunit.client.0.smithi107.stdout:4/191: write d3/f2c [208663,101584] 0 2024-05-17T21:52:46.441 INFO:tasks.workunit.client.0.smithi107.stdout:4/192: chown d3/db/d1e/c38 0 1 2024-05-17T21:52:46.443 INFO:tasks.workunit.client.0.smithi107.stdout:3/191: mkdir d6/d8/d10/d25/d40 0 2024-05-17T21:52:46.444 INFO:tasks.workunit.client.0.smithi107.stdout:3/192: write d6/d8/d10/f38 [316018,35239] 0 2024-05-17T21:52:46.445 INFO:tasks.workunit.client.1.smithi138.stdout:3/52: write d1/f4 [1270401,65849] 0 2024-05-17T21:52:46.446 INFO:tasks.workunit.client.1.smithi138.stdout:6/148: creat d7/f2e x:0 0 0 2024-05-17T21:52:46.447 INFO:tasks.workunit.client.1.smithi138.stdout:6/149: chown l5 38646523 1 2024-05-17T21:52:46.448 INFO:tasks.workunit.client.0.smithi107.stdout:4/193: dwrite d3/f2c [0,4194304] 0 2024-05-17T21:52:46.448 INFO:tasks.workunit.client.1.smithi138.stdout:6/150: truncate d7/db/d12/f26 517888 0 2024-05-17T21:52:46.449 INFO:tasks.workunit.client.1.smithi138.stdout:6/151: write f1 [9971198,99556] 0 2024-05-17T21:52:46.450 INFO:tasks.workunit.client.1.smithi138.stdout:2/49: link f2 d3/f10 0 2024-05-17T21:52:46.451 INFO:tasks.workunit.client.1.smithi138.stdout:2/50: write d3/f4 [4913187,46974] 0 2024-05-17T21:52:46.453 INFO:tasks.workunit.client.0.smithi107.stdout:8/180: truncate dc/d10/d17/f3f 665576 0 2024-05-17T21:52:46.456 INFO:tasks.workunit.client.1.smithi138.stdout:2/51: mknod d3/c11 0 2024-05-17T21:52:46.458 INFO:tasks.workunit.client.1.smithi138.stdout:6/152: dwrite d7/d8/f24 [0,4194304] 0 2024-05-17T21:52:46.458 INFO:tasks.workunit.client.1.smithi138.stdout:2/52: chown d3/f10 488 1 2024-05-17T21:52:46.458 INFO:tasks.workunit.client.0.smithi107.stdout:1/214: link d4/f1b d4/df/d14/d33/f40 0 2024-05-17T21:52:46.458 INFO:tasks.workunit.client.1.smithi138.stdout:9/62: dread f5 [0,4194304] 0 2024-05-17T21:52:46.459 INFO:tasks.workunit.client.0.smithi107.stdout:1/215: truncate d4/f1b 586313 0 2024-05-17T21:52:46.465 INFO:tasks.workunit.client.1.smithi138.stdout:6/153: mknod d7/db/d12/c2f 0 2024-05-17T21:52:46.466 INFO:tasks.workunit.client.0.smithi107.stdout:0/233: sync 2024-05-17T21:52:46.466 INFO:tasks.workunit.client.1.smithi138.stdout:6/154: write d7/d8/f24 [2172672,55578] 0 2024-05-17T21:52:46.466 INFO:tasks.workunit.client.0.smithi107.stdout:0/234: read - d3/dc/d22/d3d/f52 zero size 2024-05-17T21:52:46.467 INFO:tasks.workunit.client.0.smithi107.stdout:6/179: rename d9/d16/d22/f23 to d9/d16/d22/f33 0 2024-05-17T21:52:46.467 INFO:tasks.workunit.client.0.smithi107.stdout:6/180: chown d9/d16/f18 193399146 1 2024-05-17T21:52:46.468 INFO:tasks.workunit.client.1.smithi138.stdout:2/53: symlink d3/l12 0 2024-05-17T21:52:46.474 INFO:tasks.workunit.client.1.smithi138.stdout:9/63: rename d6/le to d6/d7/l12 0 2024-05-17T21:52:46.477 INFO:tasks.workunit.client.1.smithi138.stdout:9/64: dread f2 [0,4194304] 0 2024-05-17T21:52:46.479 INFO:tasks.workunit.client.1.smithi138.stdout:6/155: unlink d7/db/l23 0 2024-05-17T21:52:46.480 INFO:tasks.workunit.client.1.smithi138.stdout:6/156: chown d7/d8/f2b 16 1 2024-05-17T21:52:46.482 INFO:tasks.workunit.client.1.smithi138.stdout:2/54: creat d3/f13 x:0 0 0 2024-05-17T21:52:46.483 INFO:tasks.workunit.client.0.smithi107.stdout:7/152: truncate d5/df/f26 556990 0 2024-05-17T21:52:46.483 INFO:tasks.workunit.client.0.smithi107.stdout:7/153: read - d5/d16/f2b zero size 2024-05-17T21:52:46.484 INFO:tasks.workunit.client.1.smithi138.stdout:3/53: getdents d1 0 2024-05-17T21:52:46.486 INFO:tasks.workunit.client.1.smithi138.stdout:9/65: symlink d6/l13 0 2024-05-17T21:52:46.487 INFO:tasks.workunit.client.1.smithi138.stdout:9/66: write d6/f11 [670396,31862] 0 2024-05-17T21:52:46.489 INFO:tasks.workunit.client.1.smithi138.stdout:6/157: unlink d7/d8/f20 0 2024-05-17T21:52:46.490 INFO:tasks.workunit.client.1.smithi138.stdout:2/55: creat d3/f14 x:0 0 0 2024-05-17T21:52:46.495 INFO:tasks.workunit.client.0.smithi107.stdout:0/235: mknod d3/dc/d31/c5e 0 2024-05-17T21:52:46.496 INFO:tasks.workunit.client.0.smithi107.stdout:0/236: dread - d3/f35 zero size 2024-05-17T21:52:46.496 INFO:tasks.workunit.client.1.smithi138.stdout:2/56: dwrite d3/f9 [0,4194304] 0 2024-05-17T21:52:46.497 INFO:tasks.workunit.client.0.smithi107.stdout:6/181: write d9/fa [201446,64257] 0 2024-05-17T21:52:46.497 INFO:tasks.workunit.client.1.smithi138.stdout:2/57: read - d3/fa zero size 2024-05-17T21:52:46.497 INFO:tasks.workunit.client.0.smithi107.stdout:6/182: chown d9/d16/f18 99864558 1 2024-05-17T21:52:46.501 INFO:tasks.workunit.client.0.smithi107.stdout:5/196: link f3 d5/db/d39/f3e 0 2024-05-17T21:52:46.505 INFO:tasks.workunit.client.1.smithi138.stdout:6/158: dread d7/d8/ff [0,4194304] 0 2024-05-17T21:52:46.507 INFO:tasks.workunit.client.1.smithi138.stdout:2/58: dwrite d3/f6 [4194304,4194304] 0 2024-05-17T21:52:46.508 INFO:tasks.workunit.client.0.smithi107.stdout:7/154: symlink d5/df/d11/l37 0 2024-05-17T21:52:46.509 INFO:tasks.workunit.client.0.smithi107.stdout:7/155: fsync d5/d16/f2b 0 2024-05-17T21:52:46.510 INFO:tasks.workunit.client.0.smithi107.stdout:4/194: fdatasync d3/db/f14 0 2024-05-17T21:52:46.511 INFO:tasks.workunit.client.0.smithi107.stdout:4/195: truncate d3/db/d16/f30 704742 0 2024-05-17T21:52:46.513 INFO:tasks.workunit.client.0.smithi107.stdout:8/181: mknod dc/d21/d27/d34/d4c/c4d 0 2024-05-17T21:52:46.514 INFO:tasks.workunit.client.0.smithi107.stdout:8/182: dread dc/d10/d17/f1b [0,4194304] 0 2024-05-17T21:52:46.514 INFO:tasks.workunit.client.1.smithi138.stdout:6/159: rename d7/c10 to d7/d2d/c30 0 2024-05-17T21:52:46.515 INFO:tasks.workunit.client.1.smithi138.stdout:9/67: link d6/c8 d6/c14 0 2024-05-17T21:52:46.516 INFO:tasks.workunit.client.0.smithi107.stdout:0/237: mknod d3/dc/c5f 0 2024-05-17T21:52:46.517 INFO:tasks.workunit.client.1.smithi138.stdout:2/59: creat d3/f15 x:0 0 0 2024-05-17T21:52:46.518 INFO:tasks.workunit.client.1.smithi138.stdout:2/60: write d3/f15 [107855,74402] 0 2024-05-17T21:52:46.518 INFO:tasks.workunit.client.1.smithi138.stdout:3/54: link d1/f2 d1/d3/fd 0 2024-05-17T21:52:46.520 INFO:tasks.workunit.client.0.smithi107.stdout:5/197: write d5/db/d16/d2a/d2b/f3d [3999470,37020] 0 2024-05-17T21:52:46.521 INFO:tasks.workunit.client.0.smithi107.stdout:3/193: rmdir d6/d8/d10/d25/d40 0 2024-05-17T21:52:46.522 INFO:tasks.workunit.client.0.smithi107.stdout:0/238: dwrite d3/ff [0,4194304] 0 2024-05-17T21:52:46.522 INFO:tasks.workunit.client.1.smithi138.stdout:6/160: mknod d7/db/c31 0 2024-05-17T21:52:46.525 INFO:tasks.workunit.client.0.smithi107.stdout:7/156: symlink d5/d16/l38 0 2024-05-17T21:52:46.526 INFO:tasks.workunit.client.1.smithi138.stdout:9/68: dread d6/d7/fc [0,4194304] 0 2024-05-17T21:52:46.526 INFO:tasks.workunit.client.0.smithi107.stdout:2/140: sync 2024-05-17T21:52:46.527 INFO:tasks.workunit.client.1.smithi138.stdout:2/61: symlink d3/l16 0 2024-05-17T21:52:46.531 INFO:tasks.workunit.client.1.smithi138.stdout:3/55: unlink d1/fc 0 2024-05-17T21:52:46.532 INFO:tasks.workunit.client.1.smithi138.stdout:2/62: dread d3/f4 [0,4194304] 0 2024-05-17T21:52:46.533 INFO:tasks.workunit.client.0.smithi107.stdout:4/196: mkdir d3/db/dd/d3c 0 2024-05-17T21:52:46.535 INFO:tasks.workunit.client.1.smithi138.stdout:9/69: creat d6/d7/f15 x:0 0 0 2024-05-17T21:52:46.535 INFO:tasks.workunit.client.1.smithi138.stdout:9/70: chown d6/lf 2 1 2024-05-17T21:52:46.537 INFO:tasks.workunit.client.0.smithi107.stdout:9/216: getdents d0/dd/d21 0 2024-05-17T21:52:46.539 INFO:tasks.workunit.client.1.smithi138.stdout:2/63: mknod d3/c17 0 2024-05-17T21:52:46.539 INFO:tasks.workunit.client.1.smithi138.stdout:6/161: truncate d7/fa 3686209 0 2024-05-17T21:52:46.542 INFO:tasks.workunit.client.0.smithi107.stdout:5/198: dread d5/db/dc/d18/f27 [0,4194304] 0 2024-05-17T21:52:46.542 INFO:tasks.workunit.client.0.smithi107.stdout:3/194: unlink d6/d8/d10/d35/f3c 0 2024-05-17T21:52:46.542 INFO:tasks.workunit.client.1.smithi138.stdout:9/71: mkdir d6/d7/d16 0 2024-05-17T21:52:46.543 INFO:tasks.workunit.client.0.smithi107.stdout:3/195: chown d6/d8/d10/d25/l26 440 1 2024-05-17T21:52:46.543 INFO:tasks.workunit.client.0.smithi107.stdout:3/196: chown d6/d8/f14 562993080 1 2024-05-17T21:52:46.544 INFO:tasks.workunit.client.0.smithi107.stdout:3/197: fdatasync d6/d8/d10/d35/d36/f3d 0 2024-05-17T21:52:46.545 INFO:tasks.workunit.client.1.smithi138.stdout:2/64: mknod d3/c18 0 2024-05-17T21:52:46.546 INFO:tasks.workunit.client.1.smithi138.stdout:2/65: chown d3/f15 2301309 1 2024-05-17T21:52:46.547 INFO:tasks.workunit.client.1.smithi138.stdout:3/56: mkdir d1/d3/d8/de 0 2024-05-17T21:52:46.547 INFO:tasks.workunit.client.1.smithi138.stdout:2/66: dread - d3/fe zero size 2024-05-17T21:52:46.549 INFO:tasks.workunit.client.0.smithi107.stdout:6/183: creat d9/f34 x:0 0 0 2024-05-17T21:52:46.550 INFO:tasks.workunit.client.1.smithi138.stdout:2/67: write d3/fc [73945,58522] 0 2024-05-17T21:52:46.550 INFO:tasks.workunit.client.0.smithi107.stdout:6/184: write d9/d16/f30 [589832,62590] 0 2024-05-17T21:52:46.551 INFO:tasks.workunit.client.1.smithi138.stdout:9/72: dwrite d6/d7/fb [0,4194304] 0 2024-05-17T21:52:46.551 INFO:tasks.workunit.client.1.smithi138.stdout:2/68: dread - d3/fa zero size 2024-05-17T21:52:46.551 INFO:tasks.workunit.client.0.smithi107.stdout:7/157: mkdir d5/df/d11/d1c/d31/d39 0 2024-05-17T21:52:46.555 INFO:tasks.workunit.client.1.smithi138.stdout:3/57: symlink d1/lf 0 2024-05-17T21:52:46.555 INFO:tasks.workunit.client.0.smithi107.stdout:4/197: mknod d3/db/d35/c3d 0 2024-05-17T21:52:46.556 INFO:tasks.workunit.client.1.smithi138.stdout:6/162: link d7/l16 d7/d8/l32 0 2024-05-17T21:52:46.561 INFO:tasks.workunit.client.0.smithi107.stdout:9/217: dread d0/fc [0,4194304] 0 2024-05-17T21:52:46.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:46 smithi138 ceph-mon[35948]: pgmap v31: 65 pgs: 65 active+clean; 423 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 3.4 MiB/s rd, 31 MiB/s wr, 398 op/s 2024-05-17T21:52:46.590 INFO:tasks.workunit.client.0.smithi107.stdout:1/216: sync 2024-05-17T21:52:46.616 INFO:tasks.workunit.client.0.smithi107.stdout:0/239: sync 2024-05-17T21:52:46.684 INFO:tasks.workunit.client.0.smithi107.stdout:8/183: link dc/d10/d17/f3b dc/f4e 0 2024-05-17T21:52:46.685 INFO:tasks.workunit.client.1.smithi138.stdout:9/73: creat d6/d7/f17 x:0 0 0 2024-05-17T21:52:46.691 INFO:tasks.workunit.client.0.smithi107.stdout:7/158: creat d5/d16/f3a x:0 0 0 2024-05-17T21:52:46.695 INFO:tasks.workunit.client.0.smithi107.stdout:9/218: rmdir d0/dd/d2f/d4a 39 2024-05-17T21:52:46.698 INFO:tasks.workunit.client.0.smithi107.stdout:1/217: rename d4/de/d3a to d4/df/d1a/d41 0 2024-05-17T21:52:46.700 INFO:tasks.workunit.client.0.smithi107.stdout:0/240: creat d3/dc/f60 x:0 0 0 2024-05-17T21:52:46.700 INFO:tasks.workunit.client.0.smithi107.stdout:0/241: chown d3/f19 2 1 2024-05-17T21:52:46.710 INFO:tasks.workunit.client.0.smithi107.stdout:6/185: symlink d9/l35 0 2024-05-17T21:52:46.711 INFO:tasks.workunit.client.0.smithi107.stdout:2/141: dwrite d0/d9/d11/f10 [0,4194304] 0 2024-05-17T21:52:46.713 INFO:tasks.workunit.client.0.smithi107.stdout:5/199: link d5/f1b d5/db/d16/d2a/d2b/f3f 0 2024-05-17T21:52:46.716 INFO:tasks.workunit.client.0.smithi107.stdout:1/218: dread d4/de/f19 [0,4194304] 0 2024-05-17T21:52:46.716 INFO:tasks.workunit.client.1.smithi138.stdout:2/69: dwrite d3/f10 [0,4194304] 0 2024-05-17T21:52:46.717 INFO:tasks.workunit.client.0.smithi107.stdout:4/198: creat d3/db/dd/d3c/f3e x:0 0 0 2024-05-17T21:52:46.718 INFO:tasks.workunit.client.0.smithi107.stdout:7/159: dwrite f4 [0,4194304] 0 2024-05-17T21:52:46.719 INFO:tasks.workunit.client.1.smithi138.stdout:2/70: read d3/f5 [3749409,119929] 0 2024-05-17T21:52:46.720 INFO:tasks.workunit.client.1.smithi138.stdout:2/71: chown d3/f6 1307105523 1 2024-05-17T21:52:46.721 INFO:tasks.workunit.client.1.smithi138.stdout:2/72: stat d3/l12 0 2024-05-17T21:52:46.722 INFO:tasks.workunit.client.0.smithi107.stdout:1/219: dread d4/df/d14/d1d/f27 [0,4194304] 0 2024-05-17T21:52:46.722 INFO:tasks.workunit.client.1.smithi138.stdout:2/73: stat d3/fc 0 2024-05-17T21:52:46.724 INFO:tasks.workunit.client.1.smithi138.stdout:6/163: dwrite d7/fc [0,4194304] 0 2024-05-17T21:52:46.724 INFO:tasks.workunit.client.1.smithi138.stdout:6/164: dread - d7/f28 zero size 2024-05-17T21:52:46.725 INFO:tasks.workunit.client.0.smithi107.stdout:7/160: dwrite d5/f22 [0,4194304] 0 2024-05-17T21:52:46.731 INFO:tasks.workunit.client.0.smithi107.stdout:7/161: dwrite d5/d16/f1e [0,4194304] 0 2024-05-17T21:52:46.738 INFO:tasks.workunit.client.0.smithi107.stdout:3/198: truncate d6/d8/f12 562525 0 2024-05-17T21:52:46.741 INFO:tasks.workunit.client.0.smithi107.stdout:2/142: creat d0/d9/d11/df/f28 x:0 0 0 2024-05-17T21:52:46.743 INFO:tasks.workunit.client.0.smithi107.stdout:5/200: mkdir d5/db/d16/d2d/d40 0 2024-05-17T21:52:46.743 INFO:tasks.workunit.client.0.smithi107.stdout:2/143: chown d0/d9/d11/c5 96886 1 2024-05-17T21:52:46.747 INFO:tasks.workunit.client.0.smithi107.stdout:8/184: dwrite dc/d21/d35/f20 [0,4194304] 0 2024-05-17T21:52:46.750 INFO:tasks.workunit.client.0.smithi107.stdout:5/201: dread d5/f8 [0,4194304] 0 2024-05-17T21:52:46.750 INFO:tasks.workunit.client.0.smithi107.stdout:0/242: mkdir d3/d58/d61 0 2024-05-17T21:52:46.751 INFO:tasks.workunit.client.0.smithi107.stdout:0/243: fdatasync d3/f1d 0 2024-05-17T21:52:46.752 INFO:tasks.workunit.client.0.smithi107.stdout:7/162: mkdir d5/d16/d3b 0 2024-05-17T21:52:46.754 INFO:tasks.workunit.client.0.smithi107.stdout:5/202: truncate d5/db/d16/d2d/d32/f38 992733 0 2024-05-17T21:52:46.758 INFO:tasks.workunit.client.0.smithi107.stdout:0/244: dwrite d3/dc/d22/f50 [0,4194304] 0 2024-05-17T21:52:46.762 INFO:tasks.workunit.client.0.smithi107.stdout:9/219: rmdir d0/dd/d2f/d42 0 2024-05-17T21:52:46.769 INFO:tasks.workunit.client.0.smithi107.stdout:7/163: symlink d5/d16/l3c 0 2024-05-17T21:52:46.774 INFO:tasks.workunit.client.0.smithi107.stdout:5/203: symlink d5/db/dc/d18/l41 0 2024-05-17T21:52:46.778 INFO:tasks.workunit.client.0.smithi107.stdout:3/199: dread d6/d8/f12 [0,4194304] 0 2024-05-17T21:52:46.779 INFO:tasks.workunit.client.0.smithi107.stdout:3/200: readlink l0 0 2024-05-17T21:52:46.779 INFO:tasks.workunit.client.0.smithi107.stdout:3/201: fdatasync d6/d8/f14 0 2024-05-17T21:52:46.781 INFO:tasks.workunit.client.0.smithi107.stdout:4/199: link d3/f9 d3/db/d25/f3f 0 2024-05-17T21:52:46.782 INFO:tasks.workunit.client.0.smithi107.stdout:4/200: chown d3/db/dd/d3c/f3e 162023714 1 2024-05-17T21:52:46.784 INFO:tasks.workunit.client.0.smithi107.stdout:9/220: unlink d0/dd/c12 0 2024-05-17T21:52:46.784 INFO:tasks.workunit.client.0.smithi107.stdout:9/221: truncate d0/f17 2607063 0 2024-05-17T21:52:46.789 INFO:tasks.workunit.client.0.smithi107.stdout:6/186: getdents d9/d16/d22/d2d 0 2024-05-17T21:52:46.789 INFO:tasks.workunit.client.0.smithi107.stdout:9/222: dread d0/f17 [0,4194304] 0 2024-05-17T21:52:46.791 INFO:tasks.workunit.client.0.smithi107.stdout:2/144: creat d0/f29 x:0 0 0 2024-05-17T21:52:46.792 INFO:tasks.workunit.client.0.smithi107.stdout:1/220: write d4/df/f13 [2259314,125723] 0 2024-05-17T21:52:46.796 INFO:tasks.workunit.client.0.smithi107.stdout:1/221: dread d4/df/d14/d1d/f2c [0,4194304] 0 2024-05-17T21:52:46.796 INFO:tasks.workunit.client.0.smithi107.stdout:0/245: getdents d3/d58/d61 0 2024-05-17T21:52:46.801 INFO:tasks.workunit.client.0.smithi107.stdout:4/201: truncate d3/fa 973723 0 2024-05-17T21:52:46.804 INFO:tasks.workunit.client.0.smithi107.stdout:4/202: dread d3/f22 [0,4194304] 0 2024-05-17T21:52:46.804 INFO:tasks.workunit.client.0.smithi107.stdout:4/203: chown d3/d6/d26/d36 431180 1 2024-05-17T21:52:46.805 INFO:tasks.workunit.client.0.smithi107.stdout:6/187: unlink c7 0 2024-05-17T21:52:46.807 INFO:tasks.workunit.client.0.smithi107.stdout:7/164: symlink d5/df/l3d 0 2024-05-17T21:52:46.810 INFO:tasks.workunit.client.0.smithi107.stdout:7/165: dread d5/d16/f1e [0,4194304] 0 2024-05-17T21:52:46.814 INFO:tasks.workunit.client.0.smithi107.stdout:1/222: rmdir d4/df/d14/d1d 39 2024-05-17T21:52:46.815 INFO:tasks.workunit.client.0.smithi107.stdout:1/223: write d4/f21 [354381,13736] 0 2024-05-17T21:52:46.821 INFO:tasks.workunit.client.0.smithi107.stdout:4/204: rename d3/db/d16/f17 to d3/db/dd/d3a/f40 0 2024-05-17T21:52:46.822 INFO:tasks.workunit.client.1.smithi138.stdout:9/74: fsync f5 0 2024-05-17T21:52:46.828 INFO:tasks.workunit.client.0.smithi107.stdout:7/166: creat d5/d16/f3e x:0 0 0 2024-05-17T21:52:46.828 INFO:tasks.workunit.client.0.smithi107.stdout:8/185: truncate dc/d21/d27/d2f/f32 126014 0 2024-05-17T21:52:46.829 INFO:tasks.workunit.client.0.smithi107.stdout:8/186: chown dc/d21 144 1 2024-05-17T21:52:46.829 INFO:tasks.workunit.client.0.smithi107.stdout:3/202: dwrite d6/d8/d10/f23 [0,4194304] 0 2024-05-17T21:52:46.834 INFO:tasks.workunit.client.1.smithi138.stdout:2/74: rmdir d3 39 2024-05-17T21:52:46.844 INFO:tasks.workunit.client.1.smithi138.stdout:2/75: truncate d3/f10 4287064 0 2024-05-17T21:52:46.844 INFO:tasks.workunit.client.1.smithi138.stdout:2/76: chown d3 33 1 2024-05-17T21:52:46.845 INFO:tasks.workunit.client.1.smithi138.stdout:9/75: symlink d6/d7/d16/l18 0 2024-05-17T21:52:46.848 INFO:tasks.workunit.client.1.smithi138.stdout:3/58: rename d1/d3/fd to d1/d3/f10 0 2024-05-17T21:52:46.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:46 smithi107 ceph-mon[28434]: pgmap v31: 65 pgs: 65 active+clean; 423 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 3.4 MiB/s rd, 31 MiB/s wr, 398 op/s 2024-05-17T21:52:46.853 INFO:tasks.workunit.client.0.smithi107.stdout:9/223: dwrite d0/fc [0,4194304] 0 2024-05-17T21:52:46.855 INFO:tasks.workunit.client.0.smithi107.stdout:3/203: creat d6/d2a/d2d/f41 x:0 0 0 2024-05-17T21:52:46.855 INFO:tasks.workunit.client.0.smithi107.stdout:1/224: mknod d4/df/d14/d3b/c42 0 2024-05-17T21:52:46.857 INFO:tasks.workunit.client.0.smithi107.stdout:0/246: link d3/dc/d22/d3d/c4c d3/dc/d22/d23/c62 0 2024-05-17T21:52:46.860 INFO:tasks.workunit.client.0.smithi107.stdout:4/205: fsync d3/db/d25/f3f 0 2024-05-17T21:52:46.860 INFO:tasks.workunit.client.0.smithi107.stdout:5/204: dwrite d5/db/f1a [0,4194304] 0 2024-05-17T21:52:46.860 INFO:tasks.workunit.client.0.smithi107.stdout:0/247: stat d3/dc/d22/d3d/d56 0 2024-05-17T21:52:46.862 INFO:tasks.workunit.client.0.smithi107.stdout:6/188: write d9/f1c [1444619,38304] 0 2024-05-17T21:52:46.863 INFO:tasks.workunit.client.1.smithi138.stdout:3/59: dread d1/f2 [4194304,4194304] 0 2024-05-17T21:52:46.865 INFO:tasks.workunit.client.0.smithi107.stdout:4/206: stat d3/db/d16/f30 0 2024-05-17T21:52:46.866 INFO:tasks.workunit.client.0.smithi107.stdout:2/145: link d0/d9/d11/d18/l23 d0/l2a 0 2024-05-17T21:52:46.870 INFO:tasks.workunit.client.0.smithi107.stdout:3/204: dwrite d6/d8/d10/f30 [0,4194304] 0 2024-05-17T21:52:46.871 INFO:tasks.workunit.client.0.smithi107.stdout:5/205: dread d5/db/dc/f20 [0,4194304] 0 2024-05-17T21:52:46.871 INFO:tasks.workunit.client.0.smithi107.stdout:9/224: dread d0/dd/d1b/f48 [0,4194304] 0 2024-05-17T21:52:46.872 INFO:tasks.workunit.client.0.smithi107.stdout:7/167: link d5/lb d5/l3f 0 2024-05-17T21:52:46.874 INFO:tasks.workunit.client.0.smithi107.stdout:8/187: creat dc/f4f x:0 0 0 2024-05-17T21:52:46.875 INFO:tasks.workunit.client.0.smithi107.stdout:1/225: dread d4/df/d14/d1d/f2c [0,4194304] 0 2024-05-17T21:52:46.876 INFO:tasks.workunit.client.0.smithi107.stdout:6/189: symlink d9/d16/d22/d2d/l36 0 2024-05-17T21:52:46.879 INFO:tasks.workunit.client.0.smithi107.stdout:5/206: dread d5/f17 [0,4194304] 0 2024-05-17T21:52:46.879 INFO:tasks.workunit.client.0.smithi107.stdout:4/207: mkdir d3/db/dd/d3a/d41 0 2024-05-17T21:52:46.879 INFO:tasks.workunit.client.1.smithi138.stdout:2/77: write d3/ff [113773,66380] 0 2024-05-17T21:52:46.880 INFO:tasks.workunit.client.0.smithi107.stdout:5/207: stat d5/db/d16/d2a/d2b/c2e 0 2024-05-17T21:52:46.880 INFO:tasks.workunit.client.0.smithi107.stdout:4/208: write d3/db/d16/f19 [3061621,60418] 0 2024-05-17T21:52:46.880 INFO:tasks.workunit.client.1.smithi138.stdout:2/78: write d3/f15 [533051,55018] 0 2024-05-17T21:52:46.881 INFO:tasks.workunit.client.0.smithi107.stdout:2/146: creat d0/d9/d11/d15/f2b x:0 0 0 2024-05-17T21:52:46.881 INFO:tasks.workunit.client.1.smithi138.stdout:2/79: dread - d3/f13 zero size 2024-05-17T21:52:46.881 INFO:tasks.workunit.client.1.smithi138.stdout:3/60: dread d1/d3/f10 [0,4194304] 0 2024-05-17T21:52:46.882 INFO:tasks.workunit.client.1.smithi138.stdout:9/76: write d6/d7/fc [1818880,121671] 0 2024-05-17T21:52:46.887 INFO:tasks.workunit.client.1.smithi138.stdout:2/80: creat d3/f19 x:0 0 0 2024-05-17T21:52:46.889 INFO:tasks.workunit.client.1.smithi138.stdout:2/81: write d3/fe [1203513,93836] 0 2024-05-17T21:52:46.890 INFO:tasks.workunit.client.0.smithi107.stdout:7/168: creat d5/df/d11/d1c/d31/f40 x:0 0 0 2024-05-17T21:52:46.891 INFO:tasks.workunit.client.1.smithi138.stdout:9/77: mknod d6/d7/d16/c19 0 2024-05-17T21:52:46.891 INFO:tasks.workunit.client.1.smithi138.stdout:9/78: chown d6/d7/f9 249816466 1 2024-05-17T21:52:46.892 INFO:tasks.workunit.client.1.smithi138.stdout:9/79: fdatasync d6/d7/fc 0 2024-05-17T21:52:46.893 INFO:tasks.workunit.client.0.smithi107.stdout:8/188: creat dc/d21/d27/d34/d4c/f50 x:0 0 0 2024-05-17T21:52:46.894 INFO:tasks.workunit.client.1.smithi138.stdout:9/80: dread d6/d7/f9 [0,4194304] 0 2024-05-17T21:52:46.894 INFO:tasks.workunit.client.0.smithi107.stdout:0/248: write d3/ff [4929789,127006] 0 2024-05-17T21:52:46.896 INFO:tasks.workunit.client.0.smithi107.stdout:1/226: mkdir d4/df/d14/d2e/d43 0 2024-05-17T21:52:46.897 INFO:tasks.workunit.client.1.smithi138.stdout:2/82: creat d3/f1a x:0 0 0 2024-05-17T21:52:46.899 INFO:tasks.workunit.client.0.smithi107.stdout:0/249: dread d3/dc/d22/f50 [0,4194304] 0 2024-05-17T21:52:46.900 INFO:tasks.workunit.client.0.smithi107.stdout:0/250: write d3/f11 [1874832,58924] 0 2024-05-17T21:52:46.900 INFO:tasks.workunit.client.0.smithi107.stdout:4/209: symlink d3/d6/d26/l42 0 2024-05-17T21:52:46.901 INFO:tasks.workunit.client.0.smithi107.stdout:3/205: dread d6/d8/d10/f1e [0,4194304] 0 2024-05-17T21:52:46.902 INFO:tasks.workunit.client.0.smithi107.stdout:2/147: unlink d0/d9/d11/d15/c21 0 2024-05-17T21:52:46.902 INFO:tasks.workunit.client.0.smithi107.stdout:3/206: readlink d6/l1d 0 2024-05-17T21:52:46.903 INFO:tasks.workunit.client.1.smithi138.stdout:3/61: creat d1/d3/f11 x:0 0 0 2024-05-17T21:52:46.904 INFO:tasks.workunit.client.1.smithi138.stdout:9/81: creat d6/d7/d16/f1a x:0 0 0 2024-05-17T21:52:46.905 INFO:tasks.workunit.client.0.smithi107.stdout:3/207: dread d6/d8/f9 [0,4194304] 0 2024-05-17T21:52:46.905 INFO:tasks.workunit.client.0.smithi107.stdout:2/148: dread d0/d9/d11/f14 [0,4194304] 0 2024-05-17T21:52:46.906 INFO:tasks.workunit.client.0.smithi107.stdout:9/225: creat d0/dd/d3d/f4f x:0 0 0 2024-05-17T21:52:46.907 INFO:tasks.workunit.client.0.smithi107.stdout:3/208: write d6/d8/d10/f30 [3590829,30639] 0 2024-05-17T21:52:46.907 INFO:tasks.workunit.client.0.smithi107.stdout:9/226: write d0/fe [985587,82133] 0 2024-05-17T21:52:46.908 INFO:tasks.workunit.client.1.smithi138.stdout:3/62: creat d1/f12 x:0 0 0 2024-05-17T21:52:46.908 INFO:tasks.workunit.client.0.smithi107.stdout:9/227: stat d0/dd/d1f/d2c 0 2024-05-17T21:52:46.908 INFO:tasks.workunit.client.1.smithi138.stdout:9/82: dread f2 [0,4194304] 0 2024-05-17T21:52:46.909 INFO:tasks.workunit.client.1.smithi138.stdout:9/83: dread - d6/d7/d16/f1a zero size 2024-05-17T21:52:46.910 INFO:tasks.workunit.client.0.smithi107.stdout:7/169: unlink d5/df/d11/d1c/d31/f40 0 2024-05-17T21:52:46.911 INFO:tasks.workunit.client.0.smithi107.stdout:3/209: dread d6/d8/f9 [0,4194304] 0 2024-05-17T21:52:46.911 INFO:tasks.workunit.client.0.smithi107.stdout:8/189: symlink dc/d21/d27/d2f/l51 0 2024-05-17T21:52:46.912 INFO:tasks.workunit.client.1.smithi138.stdout:3/63: creat d1/d3/d8/f13 x:0 0 0 2024-05-17T21:52:46.913 INFO:tasks.workunit.client.0.smithi107.stdout:6/190: rename f1 to d9/d16/d22/d2d/f37 0 2024-05-17T21:52:46.914 INFO:tasks.workunit.client.1.smithi138.stdout:9/84: unlink c4 0 2024-05-17T21:52:46.915 INFO:tasks.workunit.client.1.smithi138.stdout:3/64: dread d1/d3/f10 [0,4194304] 0 2024-05-17T21:52:46.915 INFO:tasks.workunit.client.1.smithi138.stdout:2/83: link d3/c18 d3/c1b 0 2024-05-17T21:52:46.917 INFO:tasks.workunit.client.0.smithi107.stdout:1/227: mknod d4/df/d1a/c44 0 2024-05-17T21:52:46.918 INFO:tasks.workunit.client.1.smithi138.stdout:9/85: creat d6/f1b x:0 0 0 2024-05-17T21:52:46.919 INFO:tasks.workunit.client.1.smithi138.stdout:9/86: stat d6/d7/fc 0 2024-05-17T21:52:46.924 INFO:tasks.workunit.client.1.smithi138.stdout:3/65: mknod d1/d3/c14 0 2024-05-17T21:52:46.924 INFO:tasks.workunit.client.1.smithi138.stdout:2/84: dwrite d3/fe [0,4194304] 0 2024-05-17T21:52:46.927 INFO:tasks.workunit.client.0.smithi107.stdout:4/210: mkdir d3/db/d16/d43 0 2024-05-17T21:52:46.929 INFO:tasks.workunit.client.1.smithi138.stdout:9/87: dwrite d6/d7/fc [0,4194304] 0 2024-05-17T21:52:46.932 INFO:tasks.workunit.client.1.smithi138.stdout:2/85: creat d3/f1c x:0 0 0 2024-05-17T21:52:46.935 INFO:tasks.workunit.client.1.smithi138.stdout:3/66: fdatasync d1/d3/f10 0 2024-05-17T21:52:46.936 INFO:tasks.workunit.client.1.smithi138.stdout:3/67: chown d1/d3/f10 254060 1 2024-05-17T21:52:46.940 INFO:tasks.workunit.client.1.smithi138.stdout:9/88: symlink d6/l1c 0 2024-05-17T21:52:46.941 INFO:tasks.workunit.client.1.smithi138.stdout:2/86: creat d3/f1d x:0 0 0 2024-05-17T21:52:46.943 INFO:tasks.workunit.client.0.smithi107.stdout:7/170: mkdir d5/d41 0 2024-05-17T21:52:46.944 INFO:tasks.workunit.client.0.smithi107.stdout:7/171: write f4 [630342,51688] 0 2024-05-17T21:52:46.946 INFO:tasks.workunit.client.0.smithi107.stdout:3/210: creat d6/d2a/f42 x:0 0 0 2024-05-17T21:52:46.946 INFO:tasks.workunit.client.0.smithi107.stdout:3/211: chown d6/c18 67 1 2024-05-17T21:52:46.947 INFO:tasks.workunit.client.0.smithi107.stdout:3/212: write d6/d2a/d2d/d37/f3a [718606,75043] 0 2024-05-17T21:52:46.948 INFO:tasks.workunit.client.1.smithi138.stdout:3/68: rename d1/d3/c14 to d1/d3/d8/c15 0 2024-05-17T21:52:46.950 INFO:tasks.workunit.client.0.smithi107.stdout:8/190: readlink dc/d21/l2b 0 2024-05-17T21:52:46.952 INFO:tasks.workunit.client.1.smithi138.stdout:2/87: unlink d3/f1d 0 2024-05-17T21:52:46.955 INFO:tasks.workunit.client.1.smithi138.stdout:9/89: rename d6/l1c to d6/d7/d16/l1d 0 2024-05-17T21:52:46.956 INFO:tasks.workunit.client.0.smithi107.stdout:5/208: truncate d5/f28 1317722 0 2024-05-17T21:52:46.958 INFO:tasks.workunit.client.0.smithi107.stdout:1/228: dread d4/df/d14/d1d/f27 [0,4194304] 0 2024-05-17T21:52:46.959 INFO:tasks.workunit.client.0.smithi107.stdout:0/251: unlink d3/dc/d22/f50 0 2024-05-17T21:52:46.962 INFO:tasks.workunit.client.1.smithi138.stdout:9/90: mknod d6/d7/d16/c1e 0 2024-05-17T21:52:46.964 INFO:tasks.workunit.client.0.smithi107.stdout:0/252: dread d3/f11 [0,4194304] 0 2024-05-17T21:52:46.964 INFO:tasks.workunit.client.0.smithi107.stdout:2/149: getdents d0/d9/d11/d27 0 2024-05-17T21:52:46.965 INFO:tasks.workunit.client.0.smithi107.stdout:0/253: write d3/f18 [962714,6174] 0 2024-05-17T21:52:46.967 INFO:tasks.workunit.client.0.smithi107.stdout:7/172: creat d5/f42 x:0 0 0 2024-05-17T21:52:46.968 INFO:tasks.workunit.client.1.smithi138.stdout:9/91: mkdir d6/d7/d1f 0 2024-05-17T21:52:46.969 INFO:tasks.workunit.client.1.smithi138.stdout:9/92: chown d6/cd 328 1 2024-05-17T21:52:46.970 INFO:tasks.workunit.client.0.smithi107.stdout:3/213: rename d6/d8/d10/f1e to d6/d2a/d2d/f43 0 2024-05-17T21:52:46.970 INFO:tasks.workunit.client.0.smithi107.stdout:3/214: chown d6/f11 0 1 2024-05-17T21:52:46.981 INFO:tasks.workunit.client.1.smithi138.stdout:2/88: write d3/f5 [3389791,112456] 0 2024-05-17T21:52:46.982 INFO:tasks.workunit.client.1.smithi138.stdout:2/89: write d3/fc [281008,87538] 0 2024-05-17T21:52:46.983 INFO:tasks.workunit.client.1.smithi138.stdout:2/90: truncate d3/f1c 207382 0 2024-05-17T21:52:46.984 INFO:tasks.workunit.client.0.smithi107.stdout:0/254: rmdir d3/d39 39 2024-05-17T21:52:46.984 INFO:tasks.workunit.client.1.smithi138.stdout:2/91: write d3/f10 [4087089,67628] 0 2024-05-17T21:52:46.985 INFO:tasks.workunit.client.1.smithi138.stdout:3/69: truncate d1/f4 338057 0 2024-05-17T21:52:46.986 INFO:tasks.workunit.client.1.smithi138.stdout:2/92: write d3/f1a [451350,69] 0 2024-05-17T21:52:46.989 INFO:tasks.workunit.client.1.smithi138.stdout:9/93: rmdir d6/d7 39 2024-05-17T21:52:46.991 INFO:tasks.workunit.client.0.smithi107.stdout:6/191: creat d9/f38 x:0 0 0 2024-05-17T21:52:46.992 INFO:tasks.workunit.client.0.smithi107.stdout:8/191: write dc/f4e [39958,111101] 0 2024-05-17T21:52:46.992 INFO:tasks.workunit.client.0.smithi107.stdout:5/209: write d5/db/d39/f3e [5050459,34611] 0 2024-05-17T21:52:46.993 INFO:tasks.workunit.client.1.smithi138.stdout:2/93: rename d3/c17 to d3/c1e 0 2024-05-17T21:52:46.996 INFO:tasks.workunit.client.0.smithi107.stdout:4/211: write d3/db/f1f [1527136,49788] 0 2024-05-17T21:52:46.996 INFO:tasks.workunit.client.1.smithi138.stdout:9/94: write d6/d7/fb [2895681,115671] 0 2024-05-17T21:52:46.997 INFO:tasks.workunit.client.1.smithi138.stdout:3/70: fdatasync d1/f4 0 2024-05-17T21:52:47.001 INFO:tasks.workunit.client.1.smithi138.stdout:2/94: symlink d3/l1f 0 2024-05-17T21:52:47.002 INFO:tasks.workunit.client.0.smithi107.stdout:0/255: creat d3/dc/d31/f63 x:0 0 0 2024-05-17T21:52:47.002 INFO:tasks.workunit.client.1.smithi138.stdout:2/95: write d3/f1c [433861,107107] 0 2024-05-17T21:52:47.003 INFO:tasks.workunit.client.0.smithi107.stdout:0/256: fsync d3/dc/f16 0 2024-05-17T21:52:47.004 INFO:tasks.workunit.client.0.smithi107.stdout:7/173: dwrite d5/d16/f1e [0,4194304] 0 2024-05-17T21:52:47.005 INFO:tasks.workunit.client.1.smithi138.stdout:9/95: creat d6/d7/d16/f20 x:0 0 0 2024-05-17T21:52:47.007 INFO:tasks.workunit.client.0.smithi107.stdout:6/192: rename d9/d16/f29 to d9/d16/d22/d2d/f39 0 2024-05-17T21:52:47.008 INFO:tasks.workunit.client.1.smithi138.stdout:3/71: unlink d1/c9 0 2024-05-17T21:52:47.008 INFO:tasks.workunit.client.1.smithi138.stdout:3/72: stat d1/d3/f11 0 2024-05-17T21:52:47.009 INFO:tasks.workunit.client.0.smithi107.stdout:7/174: dread d5/df/d11/d1c/f2e [0,4194304] 0 2024-05-17T21:52:47.010 INFO:tasks.workunit.client.0.smithi107.stdout:8/192: unlink dc/d21/d35/f20 0 2024-05-17T21:52:47.011 INFO:tasks.workunit.client.0.smithi107.stdout:5/210: symlink d5/db/d16/d2d/l42 0 2024-05-17T21:52:47.012 INFO:tasks.workunit.client.1.smithi138.stdout:9/96: creat d6/d7/d16/f21 x:0 0 0 2024-05-17T21:52:47.013 INFO:tasks.workunit.client.1.smithi138.stdout:9/97: chown d6/d7/f9 367133395 1 2024-05-17T21:52:47.014 INFO:tasks.workunit.client.0.smithi107.stdout:4/212: creat d3/db/d16/f44 x:0 0 0 2024-05-17T21:52:47.014 INFO:tasks.workunit.client.1.smithi138.stdout:3/73: mkdir d1/d3/d16 0 2024-05-17T21:52:47.016 INFO:tasks.workunit.client.0.smithi107.stdout:0/257: mkdir d3/dc/d22/d3d/d56/d64 0 2024-05-17T21:52:47.017 INFO:tasks.workunit.client.0.smithi107.stdout:0/258: rename d3/d58 to d3/d58/d65 22 2024-05-17T21:52:47.018 INFO:tasks.workunit.client.0.smithi107.stdout:8/193: dwrite dc/d10/d17/f45 [0,4194304] 0 2024-05-17T21:52:47.019 INFO:tasks.workunit.client.1.smithi138.stdout:3/74: symlink d1/d3/l17 0 2024-05-17T21:52:47.019 INFO:tasks.workunit.client.1.smithi138.stdout:3/75: write d1/d3/f11 [77554,63451] 0 2024-05-17T21:52:47.020 INFO:tasks.workunit.client.0.smithi107.stdout:5/211: dwrite f3 [0,4194304] 0 2024-05-17T21:52:47.023 INFO:tasks.workunit.client.0.smithi107.stdout:5/212: chown d5/db/dc/f20 366152602 1 2024-05-17T21:52:47.023 INFO:tasks.workunit.client.1.smithi138.stdout:3/76: dread d1/d3/f11 [0,4194304] 0 2024-05-17T21:52:47.024 INFO:tasks.workunit.client.0.smithi107.stdout:6/193: rmdir d9/d16/d22 39 2024-05-17T21:52:47.025 INFO:tasks.workunit.client.0.smithi107.stdout:6/194: fsync d9/f2a 0 2024-05-17T21:52:47.025 INFO:tasks.workunit.client.0.smithi107.stdout:1/229: getdents d4/df/d14/d2e 0 2024-05-17T21:52:47.025 INFO:tasks.workunit.client.0.smithi107.stdout:6/195: stat d9/f15 0 2024-05-17T21:52:47.029 INFO:tasks.workunit.client.0.smithi107.stdout:1/230: dread d4/de/f19 [0,4194304] 0 2024-05-17T21:52:47.030 INFO:tasks.workunit.client.1.smithi138.stdout:3/77: unlink d1/f2 0 2024-05-17T21:52:47.031 INFO:tasks.workunit.client.0.smithi107.stdout:1/231: write d4/df/d14/f23 [565941,53363] 0 2024-05-17T21:52:47.031 INFO:tasks.workunit.client.0.smithi107.stdout:1/232: readlink d4/df/d14/l25 0 2024-05-17T21:52:47.032 INFO:tasks.workunit.client.0.smithi107.stdout:1/233: write d4/df/f29 [695418,35714] 0 2024-05-17T21:52:47.033 INFO:tasks.workunit.client.0.smithi107.stdout:4/213: fsync d3/db/dd/f12 0 2024-05-17T21:52:47.035 INFO:tasks.workunit.client.0.smithi107.stdout:9/228: sync 2024-05-17T21:52:47.036 INFO:tasks.workunit.client.0.smithi107.stdout:0/259: readlink d3/d39/l54 0 2024-05-17T21:52:47.037 INFO:tasks.workunit.client.0.smithi107.stdout:0/260: read d3/f1d [1155113,18296] 0 2024-05-17T21:52:47.039 INFO:tasks.workunit.client.0.smithi107.stdout:0/261: write d3/dc/d22/f32 [699851,55373] 0 2024-05-17T21:52:47.040 INFO:tasks.workunit.client.0.smithi107.stdout:1/234: dwrite d4/f21 [0,4194304] 0 2024-05-17T21:52:47.042 INFO:tasks.workunit.client.0.smithi107.stdout:0/262: write d3/dc/d22/f3e [1371307,58068] 0 2024-05-17T21:52:47.042 INFO:tasks.workunit.client.0.smithi107.stdout:1/235: readlink d4/df/d14/l37 0 2024-05-17T21:52:47.044 INFO:tasks.workunit.client.0.smithi107.stdout:0/263: write d3/dc/f60 [317001,101173] 0 2024-05-17T21:52:47.051 INFO:tasks.workunit.client.0.smithi107.stdout:1/236: dwrite d4/de/f36 [0,4194304] 0 2024-05-17T21:52:47.052 INFO:tasks.workunit.client.0.smithi107.stdout:0/264: dwrite f1 [0,4194304] 0 2024-05-17T21:52:47.054 INFO:tasks.workunit.client.0.smithi107.stdout:1/237: dread d4/df/d14/d33/f40 [0,4194304] 0 2024-05-17T21:52:47.054 INFO:tasks.workunit.client.0.smithi107.stdout:0/265: dread - d3/dc/d22/d3d/d56/f5b zero size 2024-05-17T21:52:47.055 INFO:tasks.workunit.client.0.smithi107.stdout:1/238: write d4/de/f36 [3241785,119576] 0 2024-05-17T21:52:47.056 INFO:tasks.workunit.client.0.smithi107.stdout:1/239: dread - d4/df/d14/d1d/f2f zero size 2024-05-17T21:52:47.100 INFO:tasks.workunit.client.0.smithi107.stdout:2/150: sync 2024-05-17T21:52:47.100 INFO:tasks.workunit.client.0.smithi107.stdout:2/151: truncate d0/f29 346715 0 2024-05-17T21:52:47.150 INFO:tasks.workunit.client.0.smithi107.stdout:3/215: sync 2024-05-17T21:52:47.150 INFO:tasks.workunit.client.0.smithi107.stdout:3/216: write f3 [65086,81951] 0 2024-05-17T21:52:47.151 INFO:tasks.workunit.client.0.smithi107.stdout:3/217: write d6/d2a/d2d/d37/f39 [13377,95485] 0 2024-05-17T21:52:47.203 INFO:tasks.workunit.client.1.smithi138.stdout:3/78: symlink d1/d3/l18 0 2024-05-17T21:52:47.204 INFO:tasks.workunit.client.0.smithi107.stdout:5/213: creat d5/db/d39/f43 x:0 0 0 2024-05-17T21:52:47.205 INFO:tasks.workunit.client.0.smithi107.stdout:7/175: write d5/fe [7969774,25944] 0 2024-05-17T21:52:47.205 INFO:tasks.workunit.client.0.smithi107.stdout:6/196: read - d9/d16/d22/d2d/f39 zero size 2024-05-17T21:52:47.209 INFO:tasks.workunit.client.0.smithi107.stdout:5/214: dread d5/db/d16/d2a/d2b/f3d [4194304,4194304] 0 2024-05-17T21:52:47.210 INFO:tasks.workunit.client.1.smithi138.stdout:3/79: creat d1/d3/d16/f19 x:0 0 0 2024-05-17T21:52:47.216 INFO:tasks.workunit.client.0.smithi107.stdout:4/214: rename d3/db/f14 to d3/db/d16/f45 0 2024-05-17T21:52:47.231 INFO:tasks.workunit.client.0.smithi107.stdout:3/218: mknod d6/d2a/d2d/c44 0 2024-05-17T21:52:47.232 INFO:tasks.workunit.client.0.smithi107.stdout:1/240: dwrite d4/df/d14/d1d/f2c [0,4194304] 0 2024-05-17T21:52:47.233 INFO:tasks.workunit.client.1.smithi138.stdout:3/80: write d1/d3/f10 [4398857,97611] 0 2024-05-17T21:52:47.233 INFO:tasks.workunit.client.0.smithi107.stdout:1/241: readlink d4/df/d14/l37 0 2024-05-17T21:52:47.233 INFO:tasks.workunit.client.0.smithi107.stdout:3/219: truncate d6/d8/f9 4768608 0 2024-05-17T21:52:47.235 INFO:tasks.workunit.client.0.smithi107.stdout:0/266: rename d3/c13 to d3/d58/d61/c66 0 2024-05-17T21:52:47.236 INFO:tasks.workunit.client.0.smithi107.stdout:0/267: write d3/dc/f26 [481693,9288] 0 2024-05-17T21:52:47.238 INFO:tasks.workunit.client.0.smithi107.stdout:9/229: mknod d0/dd/d2f/d4a/c50 0 2024-05-17T21:52:47.238 INFO:tasks.workunit.client.0.smithi107.stdout:1/242: dread f3 [0,4194304] 0 2024-05-17T21:52:47.238 INFO:tasks.workunit.client.0.smithi107.stdout:1/243: stat d4/df/d14/l39 0 2024-05-17T21:52:47.242 INFO:tasks.workunit.client.0.smithi107.stdout:2/152: mknod d0/d9/c2c 0 2024-05-17T21:52:47.247 INFO:tasks.workunit.client.0.smithi107.stdout:7/176: symlink d5/d41/l43 0 2024-05-17T21:52:47.247 INFO:tasks.workunit.client.0.smithi107.stdout:7/177: read d5/d16/f1e [753983,56015] 0 2024-05-17T21:52:47.248 INFO:tasks.workunit.client.0.smithi107.stdout:7/178: fsync d5/f22 0 2024-05-17T21:52:47.248 INFO:tasks.workunit.client.1.smithi138.stdout:3/81: mknod d1/d3/d8/de/c1a 0 2024-05-17T21:52:47.249 INFO:tasks.workunit.client.0.smithi107.stdout:7/179: chown d5/f18 36 1 2024-05-17T21:52:47.251 INFO:tasks.workunit.client.1.smithi138.stdout:3/82: fsync d1/d3/f10 0 2024-05-17T21:52:47.252 INFO:tasks.workunit.client.0.smithi107.stdout:8/194: getdents dc/d44 0 2024-05-17T21:52:47.252 INFO:tasks.workunit.client.0.smithi107.stdout:8/195: write dc/f4f [856211,22836] 0 2024-05-17T21:52:47.254 INFO:tasks.workunit.client.0.smithi107.stdout:1/244: mknod d4/de/c45 0 2024-05-17T21:52:47.256 INFO:tasks.workunit.client.1.smithi138.stdout:3/83: creat d1/d3/f1b x:0 0 0 2024-05-17T21:52:47.257 INFO:tasks.workunit.client.0.smithi107.stdout:0/268: creat d3/dc/d22/d5d/f67 x:0 0 0 2024-05-17T21:52:47.260 INFO:tasks.workunit.client.0.smithi107.stdout:0/269: dread d3/dc/f60 [0,4194304] 0 2024-05-17T21:52:47.261 INFO:tasks.workunit.client.0.smithi107.stdout:1/245: rmdir d4/df/d14/d1d 39 2024-05-17T21:52:47.261 INFO:tasks.workunit.client.0.smithi107.stdout:7/180: getdents d5/df/d11/d1d/d33 0 2024-05-17T21:52:47.262 INFO:tasks.workunit.client.0.smithi107.stdout:6/197: rename d9/d16/d22/f32 to d9/d16/f3a 0 2024-05-17T21:52:47.263 INFO:tasks.workunit.client.0.smithi107.stdout:8/196: mkdir dc/d44/d4b/d52 0 2024-05-17T21:52:47.264 INFO:tasks.workunit.client.0.smithi107.stdout:0/270: creat d3/dc/d22/f68 x:0 0 0 2024-05-17T21:52:47.264 INFO:tasks.workunit.client.0.smithi107.stdout:9/230: dread d0/f10 [0,4194304] 0 2024-05-17T21:52:47.265 INFO:tasks.workunit.client.0.smithi107.stdout:0/271: dread - d3/dc/d22/d3d/f52 zero size 2024-05-17T21:52:47.267 INFO:tasks.workunit.client.0.smithi107.stdout:8/197: mknod dc/d21/d27/d34/c53 0 2024-05-17T21:52:47.267 INFO:tasks.workunit.client.0.smithi107.stdout:8/198: readlink dc/d10/l48 0 2024-05-17T21:52:47.268 INFO:tasks.workunit.client.0.smithi107.stdout:7/181: mkdir d5/df/d11/d1d/d44 0 2024-05-17T21:52:47.268 INFO:tasks.workunit.client.0.smithi107.stdout:8/199: chown dc/d21/d35/f22 155 1 2024-05-17T21:52:47.268 INFO:tasks.workunit.client.0.smithi107.stdout:7/182: stat d5/d16/f3a 0 2024-05-17T21:52:47.268 INFO:tasks.workunit.client.0.smithi107.stdout:9/231: creat d0/dd/d21/d29/d3e/f51 x:0 0 0 2024-05-17T21:52:47.270 INFO:tasks.workunit.client.0.smithi107.stdout:0/272: creat d3/d39/f69 x:0 0 0 2024-05-17T21:52:47.270 INFO:tasks.workunit.client.0.smithi107.stdout:7/183: dread d5/df/f26 [0,4194304] 0 2024-05-17T21:52:47.271 INFO:tasks.workunit.client.0.smithi107.stdout:0/273: write d3/dc/d22/f68 [252676,33506] 0 2024-05-17T21:52:47.271 INFO:tasks.workunit.client.0.smithi107.stdout:7/184: write d5/df/f26 [288236,55433] 0 2024-05-17T21:52:47.271 INFO:tasks.workunit.client.0.smithi107.stdout:8/200: symlink dc/d21/d27/d2f/l54 0 2024-05-17T21:52:47.272 INFO:tasks.workunit.client.0.smithi107.stdout:0/274: chown d3/d24/l2b 5 1 2024-05-17T21:52:47.273 INFO:tasks.workunit.client.0.smithi107.stdout:6/198: creat d9/d16/f3b x:0 0 0 2024-05-17T21:52:47.273 INFO:tasks.workunit.client.0.smithi107.stdout:1/246: link d4/df/d14/d33/c3d d4/df/d14/d1d/c46 0 2024-05-17T21:52:47.273 INFO:tasks.workunit.client.0.smithi107.stdout:8/201: rename dc/d21/d27 to dc/d21/d27/d34/d4c/d55 22 2024-05-17T21:52:47.274 INFO:tasks.workunit.client.0.smithi107.stdout:1/247: chown d4/df/c2d 7 1 2024-05-17T21:52:47.274 INFO:tasks.workunit.client.0.smithi107.stdout:8/202: readlink dc/d21/d27/l40 0 2024-05-17T21:52:47.276 INFO:tasks.workunit.client.0.smithi107.stdout:0/275: dread d3/f34 [0,4194304] 0 2024-05-17T21:52:47.277 INFO:tasks.workunit.client.0.smithi107.stdout:1/248: mkdir d4/df/d14/d2e/d43/d47 0 2024-05-17T21:52:47.278 INFO:tasks.workunit.client.0.smithi107.stdout:6/199: rename d9/c11 to d9/c3c 0 2024-05-17T21:52:47.279 INFO:tasks.workunit.client.0.smithi107.stdout:0/276: symlink d3/dc/d22/d3d/d56/l6a 0 2024-05-17T21:52:47.281 INFO:tasks.workunit.client.0.smithi107.stdout:6/200: write d9/d16/f1e [816143,86379] 0 2024-05-17T21:52:47.281 INFO:tasks.workunit.client.0.smithi107.stdout:1/249: creat d4/df/d1a/d41/f48 x:0 0 0 2024-05-17T21:52:47.282 INFO:tasks.workunit.client.0.smithi107.stdout:0/277: mkdir d3/dc/d6b 0 2024-05-17T21:52:47.283 INFO:tasks.workunit.client.0.smithi107.stdout:7/185: dwrite d5/df/f26 [0,4194304] 0 2024-05-17T21:52:47.284 INFO:tasks.workunit.client.0.smithi107.stdout:8/203: dwrite dc/d10/d17/f45 [0,4194304] 0 2024-05-17T21:52:47.290 INFO:tasks.workunit.client.0.smithi107.stdout:8/204: symlink dc/d21/d27/d34/d4c/l56 0 2024-05-17T21:52:47.291 INFO:tasks.workunit.client.0.smithi107.stdout:1/250: link d4/df/d14/d3b/c42 d4/c49 0 2024-05-17T21:52:47.292 INFO:tasks.workunit.client.0.smithi107.stdout:6/201: getdents d9/d16/d22/d2d 0 2024-05-17T21:52:47.293 INFO:tasks.workunit.client.0.smithi107.stdout:6/202: dread - d9/d16/d22/d2d/f31 zero size 2024-05-17T21:52:47.294 INFO:tasks.workunit.client.0.smithi107.stdout:1/251: symlink d4/de/l4a 0 2024-05-17T21:52:47.294 INFO:tasks.workunit.client.0.smithi107.stdout:6/203: chown d9/d16/d22/d2d/l36 246480674 1 2024-05-17T21:52:47.295 INFO:tasks.workunit.client.0.smithi107.stdout:6/204: dread - d9/d16/d22/f33 zero size 2024-05-17T21:52:47.297 INFO:tasks.workunit.client.0.smithi107.stdout:1/252: symlink d4/df/d14/d2e/d43/d47/l4b 0 2024-05-17T21:52:47.297 INFO:tasks.workunit.client.0.smithi107.stdout:1/253: chown d4/df/d14 1497229 1 2024-05-17T21:52:47.298 INFO:tasks.workunit.client.0.smithi107.stdout:6/205: creat d9/d16/d22/d2d/f3d x:0 0 0 2024-05-17T21:52:47.299 INFO:tasks.workunit.client.0.smithi107.stdout:1/254: write d4/df/d14/d1d/f2a [582721,20834] 0 2024-05-17T21:52:47.304 INFO:tasks.workunit.client.0.smithi107.stdout:1/255: creat d4/df/d14/f4c x:0 0 0 2024-05-17T21:52:47.305 INFO:tasks.workunit.client.0.smithi107.stdout:6/206: dwrite d9/d16/f24 [0,4194304] 0 2024-05-17T21:52:47.305 INFO:tasks.workunit.client.0.smithi107.stdout:1/256: write d4/df/f13 [1013437,26329] 0 2024-05-17T21:52:47.306 INFO:tasks.workunit.client.0.smithi107.stdout:0/278: dread d3/d24/f2a [0,4194304] 0 2024-05-17T21:52:47.306 INFO:tasks.workunit.client.0.smithi107.stdout:6/207: chown d9/d16/c2c 3 1 2024-05-17T21:52:47.307 INFO:tasks.workunit.client.0.smithi107.stdout:0/279: read - d3/dc/d31/f57 zero size 2024-05-17T21:52:47.308 INFO:tasks.workunit.client.0.smithi107.stdout:0/280: dread - d3/f35 zero size 2024-05-17T21:52:47.309 INFO:tasks.workunit.client.0.smithi107.stdout:1/257: fdatasync d4/df/d14/d33/f40 0 2024-05-17T21:52:47.309 INFO:tasks.workunit.client.0.smithi107.stdout:5/215: sync 2024-05-17T21:52:47.311 INFO:tasks.workunit.client.0.smithi107.stdout:5/216: dread d5/db/d16/d2d/d32/f38 [0,4194304] 0 2024-05-17T21:52:47.312 INFO:tasks.workunit.client.0.smithi107.stdout:5/217: chown d5/db/dc/d18/l41 170624 1 2024-05-17T21:52:47.316 INFO:tasks.workunit.client.0.smithi107.stdout:5/218: symlink d5/db/dc/l44 0 2024-05-17T21:52:47.317 INFO:tasks.workunit.client.0.smithi107.stdout:0/281: getdents d3/d24 0 2024-05-17T21:52:47.317 INFO:tasks.workunit.client.0.smithi107.stdout:5/219: dread d5/db/dc/f20 [0,4194304] 0 2024-05-17T21:52:47.319 INFO:tasks.workunit.client.0.smithi107.stdout:5/220: mknod d5/db/d16/c45 0 2024-05-17T21:52:47.320 INFO:tasks.workunit.client.0.smithi107.stdout:5/221: fsync d5/f23 0 2024-05-17T21:52:47.322 INFO:tasks.workunit.client.0.smithi107.stdout:0/282: dwrite d3/f35 [0,4194304] 0 2024-05-17T21:52:47.323 INFO:tasks.workunit.client.0.smithi107.stdout:5/222: symlink d5/db/dc/d1e/l46 0 2024-05-17T21:52:47.325 INFO:tasks.workunit.client.0.smithi107.stdout:0/283: mkdir d3/dc/d22/d23/d6c 0 2024-05-17T21:52:47.330 INFO:tasks.workunit.client.1.smithi138.stdout:0/152: sync 2024-05-17T21:52:47.333 INFO:tasks.workunit.client.1.smithi138.stdout:0/153: symlink d3/d10/d39/l3c 0 2024-05-17T21:52:47.338 INFO:tasks.workunit.client.0.smithi107.stdout:9/232: getdents d0/dd/d2f/d4a 0 2024-05-17T21:52:47.339 INFO:tasks.workunit.client.0.smithi107.stdout:9/233: chown d0/dd/d1b/f48 780292624 1 2024-05-17T21:52:47.339 INFO:tasks.workunit.client.0.smithi107.stdout:9/234: readlink d0/l1a 0 2024-05-17T21:52:47.340 INFO:tasks.workunit.client.0.smithi107.stdout:9/235: truncate d0/dd/d21/f37 640379 0 2024-05-17T21:52:47.341 INFO:tasks.workunit.client.0.smithi107.stdout:9/236: write d0/dd/d1f/d2c/f47 [631507,98747] 0 2024-05-17T21:52:47.341 INFO:tasks.workunit.client.0.smithi107.stdout:7/186: rename d5/d41 to d5/d16/d3b/d45 0 2024-05-17T21:52:47.341 INFO:tasks.workunit.client.1.smithi138.stdout:0/154: dwrite f2 [4194304,4194304] 0 2024-05-17T21:52:47.343 INFO:tasks.workunit.client.0.smithi107.stdout:9/237: read d0/dd/f46 [334456,50723] 0 2024-05-17T21:52:47.346 INFO:tasks.workunit.client.0.smithi107.stdout:8/205: rename dc/d21/d27/d34/d4c/l56 to dc/d10/l57 0 2024-05-17T21:52:47.346 INFO:tasks.workunit.client.0.smithi107.stdout:3/220: dread d6/f1b [0,4194304] 0 2024-05-17T21:52:47.347 INFO:tasks.workunit.client.0.smithi107.stdout:4/215: dwrite d3/db/f33 [4194304,4194304] 0 2024-05-17T21:52:47.348 INFO:tasks.workunit.client.0.smithi107.stdout:7/187: fdatasync d5/d16/f1e 0 2024-05-17T21:52:47.350 INFO:tasks.workunit.client.0.smithi107.stdout:9/238: creat d0/dd/d1f/f52 x:0 0 0 2024-05-17T21:52:47.351 INFO:tasks.workunit.client.1.smithi138.stdout:0/155: dwrite d3/d10/f15 [0,4194304] 0 2024-05-17T21:52:47.353 INFO:tasks.workunit.client.0.smithi107.stdout:8/206: symlink dc/d44/d4b/l58 0 2024-05-17T21:52:47.354 INFO:tasks.workunit.client.0.smithi107.stdout:4/216: dwrite d3/f8 [0,4194304] 0 2024-05-17T21:52:47.356 INFO:tasks.workunit.client.0.smithi107.stdout:2/153: truncate d0/f25 2274281 0 2024-05-17T21:52:47.356 INFO:tasks.workunit.client.0.smithi107.stdout:3/221: symlink d6/d2a/d2d/d37/l45 0 2024-05-17T21:52:47.356 INFO:tasks.workunit.client.1.smithi138.stdout:3/84: write d1/d3/f11 [94171,127972] 0 2024-05-17T21:52:47.359 INFO:tasks.workunit.client.0.smithi107.stdout:7/188: write d5/df/f20 [291478,51458] 0 2024-05-17T21:52:47.360 INFO:tasks.workunit.client.0.smithi107.stdout:3/222: stat d6/d2a/d2d 0 2024-05-17T21:52:47.361 INFO:tasks.workunit.client.0.smithi107.stdout:3/223: write d6/f2c [3598134,105916] 0 2024-05-17T21:52:47.362 INFO:tasks.workunit.client.0.smithi107.stdout:9/239: rename d0/dd/d3d/f4f to d0/dd/d1b/f53 0 2024-05-17T21:52:47.362 INFO:tasks.workunit.client.0.smithi107.stdout:3/224: read d6/f1b [59026,40134] 0 2024-05-17T21:52:47.366 INFO:tasks.workunit.client.1.smithi138.stdout:0/156: unlink d3/ca 0 2024-05-17T21:52:47.367 INFO:tasks.workunit.client.1.smithi138.stdout:0/157: chown d3/d10/l13 7441 1 2024-05-17T21:52:47.368 INFO:tasks.workunit.client.1.smithi138.stdout:0/158: fdatasync d3/d10/f28 0 2024-05-17T21:52:47.368 INFO:tasks.workunit.client.0.smithi107.stdout:8/207: creat dc/d21/d27/d2f/f59 x:0 0 0 2024-05-17T21:52:47.371 INFO:tasks.workunit.client.0.smithi107.stdout:2/154: readlink d0/d9/d11/d18/l23 0 2024-05-17T21:52:47.372 INFO:tasks.workunit.client.1.smithi138.stdout:3/85: creat d1/d3/f1c x:0 0 0 2024-05-17T21:52:47.373 INFO:tasks.workunit.client.1.smithi138.stdout:3/86: write d1/f12 [248659,68340] 0 2024-05-17T21:52:47.374 INFO:tasks.workunit.client.1.smithi138.stdout:0/159: mkdir d3/d3d 0 2024-05-17T21:52:47.379 INFO:tasks.workunit.client.0.smithi107.stdout:3/225: unlink f2 0 2024-05-17T21:52:47.380 INFO:tasks.workunit.client.1.smithi138.stdout:3/87: symlink d1/d3/l1d 0 2024-05-17T21:52:47.381 INFO:tasks.workunit.client.0.smithi107.stdout:9/240: symlink d0/dd/d2f/d4a/l54 0 2024-05-17T21:52:47.381 INFO:tasks.workunit.client.1.smithi138.stdout:0/160: dwrite d3/d10/f32 [0,4194304] 0 2024-05-17T21:52:47.382 INFO:tasks.workunit.client.0.smithi107.stdout:1/258: getdents d4/df/d1a/d41 0 2024-05-17T21:52:47.382 INFO:tasks.workunit.client.0.smithi107.stdout:9/241: write d0/f7 [2133042,7821] 0 2024-05-17T21:52:47.387 INFO:tasks.workunit.client.1.smithi138.stdout:3/88: dread d1/d3/f10 [0,4194304] 0 2024-05-17T21:52:47.387 INFO:tasks.workunit.client.1.smithi138.stdout:0/161: mkdir d3/d10/d38/d3e 0 2024-05-17T21:52:47.388 INFO:tasks.workunit.client.1.smithi138.stdout:0/162: write d3/d10/f1a [1328464,76402] 0 2024-05-17T21:52:47.390 INFO:tasks.workunit.client.0.smithi107.stdout:8/208: rename dc/d21/d35/l19 to dc/d21/d27/d34/d4c/l5a 0 2024-05-17T21:52:47.391 INFO:tasks.workunit.client.0.smithi107.stdout:2/155: rmdir d0/d9/d11/d18 39 2024-05-17T21:52:47.392 INFO:tasks.workunit.client.0.smithi107.stdout:2/156: read d0/f29 [64021,87307] 0 2024-05-17T21:52:47.392 INFO:tasks.workunit.client.0.smithi107.stdout:2/157: chown d0/f29 960 1 2024-05-17T21:52:47.393 INFO:tasks.workunit.client.1.smithi138.stdout:3/89: symlink d1/l1e 0 2024-05-17T21:52:47.394 INFO:tasks.workunit.client.0.smithi107.stdout:7/189: creat d5/df/d11/d1d/d33/f46 x:0 0 0 2024-05-17T21:52:47.398 INFO:tasks.workunit.client.0.smithi107.stdout:6/208: write d9/d16/d22/d2d/f37 [3387862,118557] 0 2024-05-17T21:52:47.398 INFO:tasks.workunit.client.1.smithi138.stdout:3/90: chown d1/d3/c6 106390 1 2024-05-17T21:52:47.399 INFO:tasks.workunit.client.1.smithi138.stdout:3/91: fdatasync d1/d3/f10 0 2024-05-17T21:52:47.400 INFO:tasks.workunit.client.0.smithi107.stdout:3/226: dread f5 [0,4194304] 0 2024-05-17T21:52:47.400 INFO:tasks.workunit.client.1.smithi138.stdout:0/163: creat d3/d10/d38/d3e/f3f x:0 0 0 2024-05-17T21:52:47.403 INFO:tasks.workunit.client.1.smithi138.stdout:3/92: mknod d1/d3/d8/c1f 0 2024-05-17T21:52:47.405 INFO:tasks.workunit.client.1.smithi138.stdout:0/164: creat d3/d10/d38/f40 x:0 0 0 2024-05-17T21:52:47.406 INFO:tasks.workunit.client.0.smithi107.stdout:0/284: rmdir d3/dc/d22 39 2024-05-17T21:52:47.407 INFO:tasks.workunit.client.0.smithi107.stdout:0/285: write d3/f43 [294387,118397] 0 2024-05-17T21:52:47.408 INFO:tasks.workunit.client.0.smithi107.stdout:5/223: write d5/f28 [1856219,25620] 0 2024-05-17T21:52:47.408 INFO:tasks.workunit.client.0.smithi107.stdout:0/286: fdatasync d3/dc/f26 0 2024-05-17T21:52:47.409 INFO:tasks.workunit.client.1.smithi138.stdout:0/165: dread f2 [0,4194304] 0 2024-05-17T21:52:47.414 INFO:tasks.workunit.client.0.smithi107.stdout:7/190: mknod d5/df/d11/c47 0 2024-05-17T21:52:47.414 INFO:tasks.workunit.client.0.smithi107.stdout:7/191: write f4 [2408478,20700] 0 2024-05-17T21:52:47.415 INFO:tasks.workunit.client.0.smithi107.stdout:7/192: write d5/fe [610653,46036] 0 2024-05-17T21:52:47.416 INFO:tasks.workunit.client.1.smithi138.stdout:0/166: rename d3/d10/d39/l3c to d3/d3d/l41 0 2024-05-17T21:52:47.418 INFO:tasks.workunit.client.1.smithi138.stdout:3/93: creat d1/d3/f20 x:0 0 0 2024-05-17T21:52:47.425 INFO:tasks.workunit.client.0.smithi107.stdout:4/217: getdents d3/db/dd/d3c 0 2024-05-17T21:52:47.426 INFO:tasks.workunit.client.1.smithi138.stdout:0/167: symlink d3/l42 0 2024-05-17T21:52:47.426 INFO:tasks.workunit.client.0.smithi107.stdout:5/224: dread d5/f9 [0,4194304] 0 2024-05-17T21:52:47.428 INFO:tasks.workunit.client.0.smithi107.stdout:8/209: getdents dc/d44/d4b/d52 0 2024-05-17T21:52:47.432 INFO:tasks.workunit.client.1.smithi138.stdout:3/94: link d1/d3/f20 d1/d3/f21 0 2024-05-17T21:52:47.435 INFO:tasks.workunit.client.1.smithi138.stdout:3/95: mknod d1/d3/d8/de/c22 0 2024-05-17T21:52:47.437 INFO:tasks.workunit.client.1.smithi138.stdout:0/168: creat d3/f43 x:0 0 0 2024-05-17T21:52:47.438 INFO:tasks.workunit.client.0.smithi107.stdout:3/227: dread f3 [0,4194304] 0 2024-05-17T21:52:47.440 INFO:tasks.workunit.client.0.smithi107.stdout:2/158: mknod d0/d26/c2d 0 2024-05-17T21:52:47.443 INFO:tasks.workunit.client.0.smithi107.stdout:9/242: dwrite d0/f17 [0,4194304] 0 2024-05-17T21:52:47.445 INFO:tasks.workunit.client.1.smithi138.stdout:0/169: dwrite d3/f29 [0,4194304] 0 2024-05-17T21:52:47.446 INFO:tasks.workunit.client.0.smithi107.stdout:9/243: dread d0/dd/d1b/f48 [0,4194304] 0 2024-05-17T21:52:47.447 INFO:tasks.workunit.client.1.smithi138.stdout:3/96: creat d1/d3/f23 x:0 0 0 2024-05-17T21:52:47.450 INFO:tasks.workunit.client.0.smithi107.stdout:1/259: rename d4/de/f12 to d4/df/f4d 0 2024-05-17T21:52:47.452 INFO:tasks.workunit.client.1.smithi138.stdout:3/97: readlink d1/d3/l18 0 2024-05-17T21:52:47.452 INFO:tasks.workunit.client.1.smithi138.stdout:3/98: dread - d1/d3/f20 zero size 2024-05-17T21:52:47.453 INFO:tasks.workunit.client.1.smithi138.stdout:3/99: readlink d1/d3/l1d 0 2024-05-17T21:52:47.454 INFO:tasks.workunit.client.0.smithi107.stdout:8/210: fdatasync dc/d21/d27/d2f/f4a 0 2024-05-17T21:52:47.454 INFO:tasks.workunit.client.0.smithi107.stdout:8/211: chown f4 67 1 2024-05-17T21:52:47.456 INFO:tasks.workunit.client.1.smithi138.stdout:0/170: link d3/d10/f1a d3/d3d/f44 0 2024-05-17T21:52:47.462 INFO:tasks.workunit.client.1.smithi138.stdout:3/100: dwrite d1/d3/f23 [0,4194304] 0 2024-05-17T21:52:47.464 INFO:tasks.workunit.client.1.smithi138.stdout:3/101: write d1/d3/f1c [802862,16577] 0 2024-05-17T21:52:47.465 INFO:tasks.workunit.client.1.smithi138.stdout:3/102: fsync d1/d3/f11 0 2024-05-17T21:52:47.467 INFO:tasks.workunit.client.0.smithi107.stdout:6/209: creat d9/d16/f3e x:0 0 0 2024-05-17T21:52:47.468 INFO:tasks.workunit.client.0.smithi107.stdout:6/210: truncate d9/f2a 213760 0 2024-05-17T21:52:47.469 INFO:tasks.workunit.client.0.smithi107.stdout:9/244: truncate d0/f4 4792427 0 2024-05-17T21:52:47.470 INFO:tasks.workunit.client.0.smithi107.stdout:4/218: dwrite d3/db/d25/f3f [0,4194304] 0 2024-05-17T21:52:47.473 INFO:tasks.workunit.client.1.smithi138.stdout:3/103: rename d1/d3/f21 to d1/d3/d8/f24 0 2024-05-17T21:52:47.474 INFO:tasks.workunit.client.1.smithi138.stdout:3/104: stat d1/d3/d8/c1f 0 2024-05-17T21:52:47.475 INFO:tasks.workunit.client.1.smithi138.stdout:3/105: truncate d1/d3/d8/f13 999742 0 2024-05-17T21:52:47.478 INFO:tasks.workunit.client.0.smithi107.stdout:8/212: creat dc/d21/d27/f5b x:0 0 0 2024-05-17T21:52:47.479 INFO:tasks.workunit.client.0.smithi107.stdout:6/211: dread d9/f1a [0,4194304] 0 2024-05-17T21:52:47.481 INFO:tasks.workunit.client.0.smithi107.stdout:0/287: creat d3/dc/f6d x:0 0 0 2024-05-17T21:52:47.484 INFO:tasks.workunit.client.0.smithi107.stdout:3/228: creat d6/d8/d10/d25/f46 x:0 0 0 2024-05-17T21:52:47.485 INFO:tasks.workunit.client.0.smithi107.stdout:3/229: dread - d6/d8/d10/d35/d36/f3d zero size 2024-05-17T21:52:47.485 INFO:tasks.workunit.client.0.smithi107.stdout:2/159: mknod d0/d9/c2e 0 2024-05-17T21:52:47.486 INFO:tasks.workunit.client.1.smithi138.stdout:5/79: sync 2024-05-17T21:52:47.486 INFO:tasks.workunit.client.1.smithi138.stdout:1/56: sync 2024-05-17T21:52:47.486 INFO:tasks.workunit.client.1.smithi138.stdout:8/54: sync 2024-05-17T21:52:47.488 INFO:tasks.workunit.client.1.smithi138.stdout:8/55: write d0/fa [180958,51446] 0 2024-05-17T21:52:47.488 INFO:tasks.workunit.client.1.smithi138.stdout:3/106: link d1/d3/d16/f19 d1/d3/f25 0 2024-05-17T21:52:47.491 INFO:tasks.workunit.client.0.smithi107.stdout:9/245: creat d0/dd/d3d/f55 x:0 0 0 2024-05-17T21:52:47.491 INFO:tasks.workunit.client.0.smithi107.stdout:2/160: dwrite d0/d9/d11/d15/f2b [0,4194304] 0 2024-05-17T21:52:47.492 INFO:tasks.workunit.client.1.smithi138.stdout:1/57: dread d4/fc [0,4194304] 0 2024-05-17T21:52:47.492 INFO:tasks.workunit.client.0.smithi107.stdout:2/161: read - d0/d9/d11/df/f28 zero size 2024-05-17T21:52:47.493 INFO:tasks.workunit.client.0.smithi107.stdout:4/219: rename d3/f8 to d3/db/f46 0 2024-05-17T21:52:47.494 INFO:tasks.workunit.client.1.smithi138.stdout:3/107: mknod d1/c26 0 2024-05-17T21:52:47.495 INFO:tasks.workunit.client.0.smithi107.stdout:1/260: mkdir d4/df/d4e 0 2024-05-17T21:52:47.498 INFO:tasks.workunit.client.1.smithi138.stdout:5/80: link d1/f7 d1/d6/f11 0 2024-05-17T21:52:47.500 INFO:tasks.workunit.client.1.smithi138.stdout:1/58: mkdir d4/df 0 2024-05-17T21:52:47.500 INFO:tasks.workunit.client.0.smithi107.stdout:8/213: chown dc/f43 81670202 1 2024-05-17T21:52:47.501 INFO:tasks.workunit.client.0.smithi107.stdout:8/214: stat dc/d21/d27/d34 0 2024-05-17T21:52:47.501 INFO:tasks.workunit.client.0.smithi107.stdout:8/215: stat dc/d21/d27/d2f 0 2024-05-17T21:52:47.503 INFO:tasks.workunit.client.1.smithi138.stdout:1/59: dread d4/f9 [0,4194304] 0 2024-05-17T21:52:47.506 INFO:tasks.workunit.client.0.smithi107.stdout:8/216: dwrite dc/d21/d27/f37 [0,4194304] 0 2024-05-17T21:52:47.508 INFO:tasks.workunit.client.0.smithi107.stdout:0/288: fsync d3/dc/d22/d3d/f4d 0 2024-05-17T21:52:47.509 INFO:tasks.workunit.client.0.smithi107.stdout:0/289: truncate d3/d39/f69 592293 0 2024-05-17T21:52:47.510 INFO:tasks.workunit.client.1.smithi138.stdout:1/60: mknod d4/dd/c10 0 2024-05-17T21:52:47.511 INFO:tasks.workunit.client.0.smithi107.stdout:7/193: getdents d5/df/d11/d2a 0 2024-05-17T21:52:47.512 INFO:tasks.workunit.client.1.smithi138.stdout:3/108: write d1/d3/d16/f19 [628637,36372] 0 2024-05-17T21:52:47.512 INFO:tasks.workunit.client.0.smithi107.stdout:7/194: fsync d5/df/d11/f25 0 2024-05-17T21:52:47.512 INFO:tasks.workunit.client.1.smithi138.stdout:5/81: symlink d1/l12 0 2024-05-17T21:52:47.513 INFO:tasks.workunit.client.1.smithi138.stdout:8/56: truncate d0/f4 186161 0 2024-05-17T21:52:47.521 INFO:tasks.workunit.client.0.smithi107.stdout:9/246: dread d0/dd/d1f/d2c/f34 [0,4194304] 0 2024-05-17T21:52:47.522 INFO:tasks.workunit.client.1.smithi138.stdout:1/61: creat d4/dd/f11 x:0 0 0 2024-05-17T21:52:47.524 INFO:tasks.workunit.client.0.smithi107.stdout:2/162: dread d0/f29 [0,4194304] 0 2024-05-17T21:52:47.524 INFO:tasks.workunit.client.1.smithi138.stdout:3/109: creat d1/d3/d8/de/f27 x:0 0 0 2024-05-17T21:52:47.525 INFO:tasks.workunit.client.1.smithi138.stdout:3/110: write d1/d3/f10 [4599596,27] 0 2024-05-17T21:52:47.526 INFO:tasks.workunit.client.0.smithi107.stdout:1/261: mknod d4/df/d14/d2e/d43/c4f 0 2024-05-17T21:52:47.528 INFO:tasks.workunit.client.0.smithi107.stdout:5/225: getdents d5/db 0 2024-05-17T21:52:47.528 INFO:tasks.workunit.client.0.smithi107.stdout:9/247: dwrite d0/f7 [4194304,4194304] 0 2024-05-17T21:52:47.530 INFO:tasks.workunit.client.0.smithi107.stdout:6/212: mknod d9/c3f 0 2024-05-17T21:52:47.532 INFO:tasks.workunit.client.0.smithi107.stdout:9/248: write d0/dd/f35 [4136380,87403] 0 2024-05-17T21:52:47.533 INFO:tasks.workunit.client.0.smithi107.stdout:2/163: dwrite d0/d9/d11/f10 [0,4194304] 0 2024-05-17T21:52:47.534 INFO:tasks.workunit.client.0.smithi107.stdout:4/220: dwrite d3/f22 [0,4194304] 0 2024-05-17T21:52:47.535 INFO:tasks.workunit.client.0.smithi107.stdout:9/249: write d0/dd/d2f/f30 [151867,117326] 0 2024-05-17T21:52:47.536 INFO:tasks.workunit.client.0.smithi107.stdout:9/250: stat d0/dd/d1b/c2a 0 2024-05-17T21:52:47.537 INFO:tasks.workunit.client.0.smithi107.stdout:0/290: chown d3/dc/d22/d23/c62 23281 1 2024-05-17T21:52:47.538 INFO:tasks.workunit.client.0.smithi107.stdout:7/195: rmdir d5/df/d11/d1c/d31 39 2024-05-17T21:52:47.541 INFO:tasks.workunit.client.0.smithi107.stdout:3/230: fdatasync d6/d8/d10/f23 0 2024-05-17T21:52:47.543 INFO:tasks.workunit.client.0.smithi107.stdout:3/231: write d6/fc [2578581,56930] 0 2024-05-17T21:52:47.548 INFO:tasks.workunit.client.0.smithi107.stdout:5/226: creat d5/db/dc/d18/f47 x:0 0 0 2024-05-17T21:52:47.548 INFO:tasks.workunit.client.0.smithi107.stdout:0/291: dread d3/d24/f2a [0,4194304] 0 2024-05-17T21:52:47.548 INFO:tasks.workunit.client.0.smithi107.stdout:5/227: dread d5/db/dc/f20 [0,4194304] 0 2024-05-17T21:52:47.548 INFO:tasks.workunit.client.0.smithi107.stdout:5/228: write d5/f28 [2635551,64612] 0 2024-05-17T21:52:47.549 INFO:tasks.workunit.client.0.smithi107.stdout:8/217: rename dc/d21/d35/c42 to dc/d10/d17/c5c 0 2024-05-17T21:52:47.550 INFO:tasks.workunit.client.0.smithi107.stdout:0/292: read d3/dc/f16 [1085695,79054] 0 2024-05-17T21:52:47.552 INFO:tasks.workunit.client.0.smithi107.stdout:0/293: read d3/dc/d22/d23/f2c [384208,38066] 0 2024-05-17T21:52:47.556 INFO:tasks.workunit.client.0.smithi107.stdout:6/213: symlink d9/l40 0 2024-05-17T21:52:47.557 INFO:tasks.workunit.client.0.smithi107.stdout:5/229: mkdir d5/db/dc/d18/d48 0 2024-05-17T21:52:47.557 INFO:tasks.workunit.client.0.smithi107.stdout:6/214: dread d9/d16/d22/d2d/f2f [0,4194304] 0 2024-05-17T21:52:47.559 INFO:tasks.workunit.client.0.smithi107.stdout:1/262: rename d4/df/d14/d3b to d4/df/d1a/d26/d50 0 2024-05-17T21:52:47.570 INFO:tasks.workunit.client.0.smithi107.stdout:2/164: fsync d0/f6 0 2024-05-17T21:52:47.570 INFO:tasks.workunit.client.0.smithi107.stdout:4/221: mkdir d3/db/d25/d47 0 2024-05-17T21:52:47.571 INFO:tasks.workunit.client.0.smithi107.stdout:3/232: rename d6/f33 to d6/d8/d10/d25/f47 0 2024-05-17T21:52:47.571 INFO:tasks.workunit.client.0.smithi107.stdout:1/263: rmdir d4/df/d14/d2e/d43/d47 39 2024-05-17T21:52:47.572 INFO:tasks.workunit.client.0.smithi107.stdout:9/251: creat d0/dd/f56 x:0 0 0 2024-05-17T21:52:47.572 INFO:tasks.workunit.client.0.smithi107.stdout:3/233: dread - d6/d8/d10/d25/f46 zero size 2024-05-17T21:52:47.572 INFO:tasks.workunit.client.0.smithi107.stdout:1/264: chown d4/df/d14/l39 30947199 1 2024-05-17T21:52:47.573 INFO:tasks.workunit.client.0.smithi107.stdout:7/196: link d5/d16/f28 d5/df/d11/d1c/d31/f48 0 2024-05-17T21:52:47.586 INFO:tasks.workunit.client.0.smithi107.stdout:4/222: creat d3/db/f48 x:0 0 0 2024-05-17T21:52:47.587 INFO:tasks.workunit.client.0.smithi107.stdout:5/230: symlink d5/l49 0 2024-05-17T21:52:47.588 INFO:tasks.workunit.client.0.smithi107.stdout:0/294: creat d3/f6e x:0 0 0 2024-05-17T21:52:47.589 INFO:tasks.workunit.client.0.smithi107.stdout:1/265: write d4/de/f36 [5206340,53274] 0 2024-05-17T21:52:47.590 INFO:tasks.workunit.client.0.smithi107.stdout:6/215: truncate d9/f10 1571059 0 2024-05-17T21:52:47.591 INFO:tasks.workunit.client.0.smithi107.stdout:3/234: mkdir d6/d2a/d2d/d48 0 2024-05-17T21:52:47.591 INFO:tasks.workunit.client.0.smithi107.stdout:4/223: mknod d3/db/d25/c49 0 2024-05-17T21:52:47.592 INFO:tasks.workunit.client.0.smithi107.stdout:0/295: mkdir d3/d24/d6f 0 2024-05-17T21:52:47.593 INFO:tasks.workunit.client.0.smithi107.stdout:5/231: rename f3 to d5/db/d16/d2d/f4a 0 2024-05-17T21:52:47.595 INFO:tasks.workunit.client.0.smithi107.stdout:4/224: dread - d3/d6/d26/f2f zero size 2024-05-17T21:52:47.595 INFO:tasks.workunit.client.0.smithi107.stdout:1/266: chown d4/c31 11792 1 2024-05-17T21:52:47.596 INFO:tasks.workunit.client.0.smithi107.stdout:5/232: readlink d5/db/dc/lf 0 2024-05-17T21:52:47.596 INFO:tasks.workunit.client.0.smithi107.stdout:2/165: dread d0/f6 [0,4194304] 0 2024-05-17T21:52:47.596 INFO:tasks.workunit.client.0.smithi107.stdout:3/235: creat d6/d8/d10/d25/f49 x:0 0 0 2024-05-17T21:52:47.598 INFO:tasks.workunit.client.0.smithi107.stdout:9/252: dwrite d0/dd/d2f/f31 [0,4194304] 0 2024-05-17T21:52:47.600 INFO:tasks.workunit.client.0.smithi107.stdout:5/233: chown d5/le 2127640911 1 2024-05-17T21:52:47.601 INFO:tasks.workunit.client.0.smithi107.stdout:9/253: fdatasync d0/f7 0 2024-05-17T21:52:47.601 INFO:tasks.workunit.client.0.smithi107.stdout:2/166: fdatasync d0/f6 0 2024-05-17T21:52:47.602 INFO:tasks.workunit.client.0.smithi107.stdout:3/236: rmdir d6/d8 39 2024-05-17T21:52:47.604 INFO:tasks.workunit.client.0.smithi107.stdout:7/197: dwrite d5/d16/f2b [0,4194304] 0 2024-05-17T21:52:47.606 INFO:tasks.workunit.client.1.smithi138.stdout:1/62: creat d4/df/f12 x:0 0 0 2024-05-17T21:52:47.606 INFO:tasks.workunit.client.1.smithi138.stdout:5/82: rename d1/d6/fb to d1/f13 0 2024-05-17T21:52:47.607 INFO:tasks.workunit.client.1.smithi138.stdout:3/111: dwrite d1/d3/f25 [0,4194304] 0 2024-05-17T21:52:47.607 INFO:tasks.workunit.client.0.smithi107.stdout:1/267: write d4/de/f19 [543731,14394] 0 2024-05-17T21:52:47.608 INFO:tasks.workunit.client.0.smithi107.stdout:5/234: creat d5/db/d16/f4b x:0 0 0 2024-05-17T21:52:47.608 INFO:tasks.workunit.client.0.smithi107.stdout:3/237: symlink d6/d1a/l4a 0 2024-05-17T21:52:47.610 INFO:tasks.workunit.client.1.smithi138.stdout:5/83: read d1/f13 [24108,43454] 0 2024-05-17T21:52:47.610 INFO:tasks.workunit.client.0.smithi107.stdout:9/254: mkdir d0/dd/d21/d29/d57 0 2024-05-17T21:52:47.615 INFO:tasks.workunit.client.1.smithi138.stdout:1/63: dread d4/fc [0,4194304] 0 2024-05-17T21:52:47.616 INFO:tasks.workunit.client.0.smithi107.stdout:0/296: dwrite d3/dc/d31/f57 [0,4194304] 0 2024-05-17T21:52:47.617 INFO:tasks.workunit.client.1.smithi138.stdout:1/64: stat f3 0 2024-05-17T21:52:47.618 INFO:tasks.workunit.client.0.smithi107.stdout:4/225: dwrite d3/db/f48 [0,4194304] 0 2024-05-17T21:52:47.618 INFO:tasks.workunit.client.1.smithi138.stdout:3/112: dwrite d1/d3/f25 [0,4194304] 0 2024-05-17T21:52:47.620 INFO:tasks.workunit.client.0.smithi107.stdout:9/255: truncate d0/dd/d2f/f30 987066 0 2024-05-17T21:52:47.623 INFO:tasks.workunit.client.0.smithi107.stdout:9/256: truncate d0/dd/d21/f37 1568387 0 2024-05-17T21:52:47.624 INFO:tasks.workunit.client.1.smithi138.stdout:1/65: dread f3 [0,4194304] 0 2024-05-17T21:52:47.624 INFO:tasks.workunit.client.0.smithi107.stdout:2/167: rename d0/d9/d11/df/c1b to d0/d9/d11/d27/c2f 0 2024-05-17T21:52:47.626 INFO:tasks.workunit.client.0.smithi107.stdout:1/268: dwrite d4/df/d14/f4c [0,4194304] 0 2024-05-17T21:52:47.627 INFO:tasks.workunit.client.0.smithi107.stdout:2/168: write d0/f25 [1360667,120377] 0 2024-05-17T21:52:47.630 INFO:tasks.workunit.client.1.smithi138.stdout:1/66: dread f2 [0,4194304] 0 2024-05-17T21:52:47.630 INFO:tasks.workunit.client.1.smithi138.stdout:3/113: chown d1/d3/l17 117376375 1 2024-05-17T21:52:47.631 INFO:tasks.workunit.client.1.smithi138.stdout:1/67: readlink d4/dd/le 0 2024-05-17T21:52:47.632 INFO:tasks.workunit.client.1.smithi138.stdout:1/68: chown d4/dd/le 15 1 2024-05-17T21:52:47.633 INFO:tasks.workunit.client.0.smithi107.stdout:3/238: creat d6/d8/d10/f4b x:0 0 0 2024-05-17T21:52:47.634 INFO:tasks.workunit.client.1.smithi138.stdout:5/84: link d1/l8 d1/l14 0 2024-05-17T21:52:47.634 INFO:tasks.workunit.client.0.smithi107.stdout:3/239: fsync d6/d8/d10/d25/f46 0 2024-05-17T21:52:47.634 INFO:tasks.workunit.client.1.smithi138.stdout:5/85: chown d1/f13 674907 1 2024-05-17T21:52:47.635 INFO:tasks.workunit.client.1.smithi138.stdout:3/114: creat d1/d3/d8/f28 x:0 0 0 2024-05-17T21:52:47.635 INFO:tasks.workunit.client.0.smithi107.stdout:3/240: write d6/d2a/f42 [442471,102444] 0 2024-05-17T21:52:47.638 INFO:tasks.workunit.client.0.smithi107.stdout:9/257: creat d0/dd/d21/f58 x:0 0 0 2024-05-17T21:52:47.641 INFO:tasks.workunit.client.0.smithi107.stdout:5/235: rename d5/f19 to d5/db/d39/f4c 0 2024-05-17T21:52:47.646 INFO:tasks.workunit.client.1.smithi138.stdout:5/86: rename d1/l8 to d1/d6/l15 0 2024-05-17T21:52:47.647 INFO:tasks.workunit.client.0.smithi107.stdout:7/198: getdents d5 0 2024-05-17T21:52:47.647 INFO:tasks.workunit.client.0.smithi107.stdout:2/169: fdatasync d0/d9/d11/d18/f20 0 2024-05-17T21:52:47.648 INFO:tasks.workunit.client.0.smithi107.stdout:9/258: dwrite d0/fe [4194304,4194304] 0 2024-05-17T21:52:47.650 INFO:tasks.workunit.client.1.smithi138.stdout:5/87: fsync d1/f13 0 2024-05-17T21:52:47.650 INFO:tasks.workunit.client.0.smithi107.stdout:3/241: mkdir d6/d8/d10/d35/d36/d4c 0 2024-05-17T21:52:47.651 INFO:tasks.workunit.client.1.smithi138.stdout:5/88: read d1/d6/f11 [8246369,32484] 0 2024-05-17T21:52:47.651 INFO:tasks.workunit.client.0.smithi107.stdout:2/170: chown d0/d9/d11/l1d 43874647 1 2024-05-17T21:52:47.652 INFO:tasks.workunit.client.0.smithi107.stdout:0/297: dwrite d3/f11 [0,4194304] 0 2024-05-17T21:52:47.653 INFO:tasks.workunit.client.0.smithi107.stdout:4/226: rename d3/db/d25 to d3/d4a 0 2024-05-17T21:52:47.654 INFO:tasks.workunit.client.1.smithi138.stdout:5/89: mknod d1/d6/c16 0 2024-05-17T21:52:47.657 INFO:tasks.workunit.client.0.smithi107.stdout:5/236: mkdir d5/db/d16/d2d/d4d 0 2024-05-17T21:52:47.660 INFO:tasks.workunit.client.0.smithi107.stdout:3/242: dwrite d6/d8/d10/d25/f46 [0,4194304] 0 2024-05-17T21:52:47.661 INFO:tasks.workunit.client.0.smithi107.stdout:9/259: creat d0/dd/d21/d29/d3e/d45/f59 x:0 0 0 2024-05-17T21:52:47.661 INFO:tasks.workunit.client.0.smithi107.stdout:9/260: stat d0/dd/d1f/l3a 0 2024-05-17T21:52:47.666 INFO:tasks.workunit.client.1.smithi138.stdout:5/90: link d1/lf d1/d6/l17 0 2024-05-17T21:52:47.667 INFO:tasks.workunit.client.1.smithi138.stdout:5/91: chown d1/d6/c16 30926 1 2024-05-17T21:52:47.669 INFO:tasks.workunit.client.0.smithi107.stdout:2/171: dread d0/d9/d11/f14 [0,4194304] 0 2024-05-17T21:52:47.669 INFO:tasks.workunit.client.1.smithi138.stdout:1/69: write d4/f9 [1334661,74415] 0 2024-05-17T21:52:47.672 INFO:tasks.workunit.client.0.smithi107.stdout:5/237: creat d5/db/dc/f4e x:0 0 0 2024-05-17T21:52:47.672 INFO:tasks.workunit.client.0.smithi107.stdout:5/238: chown d5/db/dc/c10 20536503 1 2024-05-17T21:52:47.674 INFO:tasks.workunit.client.0.smithi107.stdout:9/261: chown d0/dd/d2f/l41 1 1 2024-05-17T21:52:47.675 INFO:tasks.workunit.client.0.smithi107.stdout:1/269: write d4/df/d14/d33/f40 [976990,16272] 0 2024-05-17T21:52:47.675 INFO:tasks.workunit.client.0.smithi107.stdout:9/262: dread - d0/dd/d1b/f28 zero size 2024-05-17T21:52:47.675 INFO:tasks.workunit.client.1.smithi138.stdout:5/92: dwrite d1/f13 [0,4194304] 0 2024-05-17T21:52:47.676 INFO:tasks.workunit.client.1.smithi138.stdout:1/70: creat d4/dd/f13 x:0 0 0 2024-05-17T21:52:47.677 INFO:tasks.workunit.client.1.smithi138.stdout:5/93: fsync d1/d6/fa 0 2024-05-17T21:52:47.677 INFO:tasks.workunit.client.0.smithi107.stdout:1/270: write d4/df/f13 [3126481,68954] 0 2024-05-17T21:52:47.677 INFO:tasks.workunit.client.1.smithi138.stdout:1/71: fdatasync d4/dd/f11 0 2024-05-17T21:52:47.680 INFO:tasks.workunit.client.0.smithi107.stdout:7/199: rename d5/df/d11/d1d/c35 to d5/df/d11/d1c/d2f/c49 0 2024-05-17T21:52:47.680 INFO:tasks.workunit.client.1.smithi138.stdout:3/115: truncate d1/d3/f23 2488447 0 2024-05-17T21:52:47.681 INFO:tasks.workunit.client.1.smithi138.stdout:3/116: dread - d1/d3/f20 zero size 2024-05-17T21:52:47.681 INFO:tasks.workunit.client.1.smithi138.stdout:5/94: dread d1/f7 [0,4194304] 0 2024-05-17T21:52:47.685 INFO:tasks.workunit.client.1.smithi138.stdout:3/117: chown d1/d3/d8/f24 3409548 1 2024-05-17T21:52:47.687 INFO:tasks.workunit.client.1.smithi138.stdout:3/118: write d1/d3/d8/de/f27 [406943,94949] 0 2024-05-17T21:52:47.687 INFO:tasks.workunit.client.0.smithi107.stdout:4/227: creat d3/d4a/d47/f4b x:0 0 0 2024-05-17T21:52:47.687 INFO:tasks.workunit.client.1.smithi138.stdout:5/95: read d1/d6/fa [128549,56081] 0 2024-05-17T21:52:47.688 INFO:tasks.workunit.client.0.smithi107.stdout:3/243: dread d6/d8/d10/d25/f47 [0,4194304] 0 2024-05-17T21:52:47.689 INFO:tasks.workunit.client.1.smithi138.stdout:7/70: sync 2024-05-17T21:52:47.689 INFO:tasks.workunit.client.1.smithi138.stdout:6/165: sync 2024-05-17T21:52:47.690 INFO:tasks.workunit.client.1.smithi138.stdout:9/98: sync 2024-05-17T21:52:47.690 INFO:tasks.workunit.client.1.smithi138.stdout:4/71: sync 2024-05-17T21:52:47.690 INFO:tasks.workunit.client.1.smithi138.stdout:2/96: sync 2024-05-17T21:52:47.690 INFO:tasks.workunit.client.0.smithi107.stdout:2/172: mknod d0/d9/d11/df/c30 0 2024-05-17T21:52:47.690 INFO:tasks.workunit.client.0.smithi107.stdout:2/173: chown d0/d9/d1c 23677425 1 2024-05-17T21:52:47.691 INFO:tasks.workunit.client.0.smithi107.stdout:4/228: dread d3/f9 [0,4194304] 0 2024-05-17T21:52:47.692 INFO:tasks.workunit.client.0.smithi107.stdout:2/174: dread - d0/d9/d11/df/f28 zero size 2024-05-17T21:52:47.693 INFO:tasks.workunit.client.0.smithi107.stdout:8/218: sync 2024-05-17T21:52:47.693 INFO:tasks.workunit.client.0.smithi107.stdout:5/239: creat d5/db/d16/d2d/f4f x:0 0 0 2024-05-17T21:52:47.694 INFO:tasks.workunit.client.1.smithi138.stdout:9/99: write d6/d7/d16/f21 [329381,75043] 0 2024-05-17T21:52:47.694 INFO:tasks.workunit.client.0.smithi107.stdout:3/244: dwrite d6/d8/f14 [0,4194304] 0 2024-05-17T21:52:47.698 INFO:tasks.workunit.client.0.smithi107.stdout:9/263: creat d0/dd/d21/d29/f5a x:0 0 0 2024-05-17T21:52:47.698 INFO:tasks.workunit.client.0.smithi107.stdout:4/229: dwrite d3/f22 [0,4194304] 0 2024-05-17T21:52:47.703 INFO:tasks.workunit.client.0.smithi107.stdout:9/264: read d0/f7 [3368476,112181] 0 2024-05-17T21:52:47.703 INFO:tasks.workunit.client.1.smithi138.stdout:3/119: dwrite d1/d3/f1c [0,4194304] 0 2024-05-17T21:52:47.704 INFO:tasks.workunit.client.0.smithi107.stdout:9/265: write d0/fe [3182295,57577] 0 2024-05-17T21:52:47.704 INFO:tasks.workunit.client.1.smithi138.stdout:5/96: dwrite d1/d6/fa [0,4194304] 0 2024-05-17T21:52:47.708 INFO:tasks.workunit.client.0.smithi107.stdout:7/200: dread d5/f18 [0,4194304] 0 2024-05-17T21:52:47.709 INFO:tasks.workunit.client.1.smithi138.stdout:1/72: link d4/dd/f13 d4/df/f14 0 2024-05-17T21:52:47.713 INFO:tasks.workunit.client.1.smithi138.stdout:7/71: creat d0/f17 x:0 0 0 2024-05-17T21:52:47.717 INFO:tasks.workunit.client.1.smithi138.stdout:2/97: creat d3/f20 x:0 0 0 2024-05-17T21:52:47.722 INFO:tasks.workunit.client.1.smithi138.stdout:9/100: rename d6/f11 to d6/d7/d1f/f22 0 2024-05-17T21:52:47.722 INFO:tasks.workunit.client.0.smithi107.stdout:3/245: creat d6/d2a/f4d x:0 0 0 2024-05-17T21:52:47.724 INFO:tasks.workunit.client.1.smithi138.stdout:3/120: unlink d1/d3/d16/f19 0 2024-05-17T21:52:47.725 INFO:tasks.workunit.client.1.smithi138.stdout:9/101: dread d6/d7/d16/f21 [0,4194304] 0 2024-05-17T21:52:47.725 INFO:tasks.workunit.client.1.smithi138.stdout:9/102: fsync d6/d7/f10 0 2024-05-17T21:52:47.726 INFO:tasks.workunit.client.0.smithi107.stdout:4/230: mknod d3/d6/c4c 0 2024-05-17T21:52:47.728 INFO:tasks.workunit.client.1.smithi138.stdout:1/73: creat d4/dd/f15 x:0 0 0 2024-05-17T21:52:47.730 INFO:tasks.workunit.client.1.smithi138.stdout:7/72: mkdir d0/d18 0 2024-05-17T21:52:47.732 INFO:tasks.workunit.client.0.smithi107.stdout:0/298: getdents d3 0 2024-05-17T21:52:47.733 INFO:tasks.workunit.client.1.smithi138.stdout:2/98: mkdir d3/d21 0 2024-05-17T21:52:47.733 INFO:tasks.workunit.client.1.smithi138.stdout:9/103: dwrite f2 [0,4194304] 0 2024-05-17T21:52:47.734 INFO:tasks.workunit.client.1.smithi138.stdout:2/99: read d3/fe [172226,64088] 0 2024-05-17T21:52:47.735 INFO:tasks.workunit.client.0.smithi107.stdout:6/216: sync 2024-05-17T21:52:47.735 INFO:tasks.workunit.client.0.smithi107.stdout:7/201: rename d5/df/d11/c47 to d5/d16/d3b/d45/c4a 0 2024-05-17T21:52:47.736 INFO:tasks.workunit.client.1.smithi138.stdout:9/104: truncate d6/f1b 136112 0 2024-05-17T21:52:47.739 INFO:tasks.workunit.client.1.smithi138.stdout:7/73: dwrite d0/f17 [0,4194304] 0 2024-05-17T21:52:47.740 INFO:tasks.workunit.client.1.smithi138.stdout:7/74: write d0/f13 [1007921,84856] 0 2024-05-17T21:52:47.742 INFO:tasks.workunit.client.0.smithi107.stdout:2/175: link d0/fa d0/d9/d1c/d22/f31 0 2024-05-17T21:52:47.750 INFO:tasks.workunit.client.0.smithi107.stdout:5/240: mkdir d5/db/d16/d2d/d4d/d50 0 2024-05-17T21:52:47.750 INFO:tasks.workunit.client.0.smithi107.stdout:5/241: chown d5/db/dc/lf 0 1 2024-05-17T21:52:47.751 INFO:tasks.workunit.client.1.smithi138.stdout:4/72: dwrite d0/d3/f12 [0,4194304] 0 2024-05-17T21:52:47.753 INFO:tasks.workunit.client.0.smithi107.stdout:3/246: unlink d6/f11 0 2024-05-17T21:52:47.756 INFO:tasks.workunit.client.1.smithi138.stdout:3/121: creat d1/d3/d16/f29 x:0 0 0 2024-05-17T21:52:47.760 INFO:tasks.workunit.client.0.smithi107.stdout:4/231: rmdir d3 39 2024-05-17T21:52:47.763 INFO:tasks.workunit.client.0.smithi107.stdout:1/271: creat d4/df/d14/d2e/d43/f51 x:0 0 0 2024-05-17T21:52:47.767 INFO:tasks.workunit.client.0.smithi107.stdout:0/299: rmdir d3/dc/d22/d3d/d56 39 2024-05-17T21:52:47.770 INFO:tasks.workunit.client.0.smithi107.stdout:6/217: rmdir d9 39 2024-05-17T21:52:47.771 INFO:tasks.workunit.client.1.smithi138.stdout:0/171: sync 2024-05-17T21:52:47.773 INFO:tasks.workunit.client.0.smithi107.stdout:8/219: write dc/d21/d35/f36 [380415,75328] 0 2024-05-17T21:52:47.775 INFO:tasks.workunit.client.1.smithi138.stdout:6/166: link d7/c1b d7/d8/c33 0 2024-05-17T21:52:47.778 INFO:tasks.workunit.client.1.smithi138.stdout:7/75: readlink d0/l16 0 2024-05-17T21:52:47.783 INFO:tasks.workunit.client.0.smithi107.stdout:3/247: creat d6/d1a/f4e x:0 0 0 2024-05-17T21:52:47.785 INFO:tasks.workunit.client.1.smithi138.stdout:2/100: write d3/f4 [2590500,73381] 0 2024-05-17T21:52:47.786 INFO:tasks.workunit.client.1.smithi138.stdout:7/76: dread d0/f17 [0,4194304] 0 2024-05-17T21:52:47.786 INFO:tasks.workunit.client.0.smithi107.stdout:2/176: write d0/d9/d1c/d22/f31 [3360008,53529] 0 2024-05-17T21:52:47.787 INFO:tasks.workunit.client.1.smithi138.stdout:4/73: mknod d0/d3/d17/c1c 0 2024-05-17T21:52:47.787 INFO:tasks.workunit.client.1.smithi138.stdout:2/101: fsync d3/f14 0 2024-05-17T21:52:47.787 INFO:tasks.workunit.client.0.smithi107.stdout:1/272: mknod d4/df/d1a/c52 0 2024-05-17T21:52:47.788 INFO:tasks.workunit.client.1.smithi138.stdout:1/74: dwrite f0 [0,4194304] 0 2024-05-17T21:52:47.791 INFO:tasks.workunit.client.0.smithi107.stdout:5/242: write d5/db/d16/d2a/d2b/f3d [8804252,46207] 0 2024-05-17T21:52:47.792 INFO:tasks.workunit.client.0.smithi107.stdout:5/243: chown d5/db/d16/d2d/d32/f38 756 1 2024-05-17T21:52:47.795 INFO:tasks.workunit.client.0.smithi107.stdout:9/266: creat d0/dd/d2f/f5b x:0 0 0 2024-05-17T21:52:47.796 INFO:tasks.workunit.client.1.smithi138.stdout:3/122: symlink d1/d3/d8/de/l2a 0 2024-05-17T21:52:47.797 INFO:tasks.workunit.client.0.smithi107.stdout:0/300: fdatasync d3/f9 0 2024-05-17T21:52:47.798 INFO:tasks.workunit.client.1.smithi138.stdout:5/97: link d1/l12 d1/d6/l18 0 2024-05-17T21:52:47.800 INFO:tasks.workunit.client.0.smithi107.stdout:6/218: chown d9/f38 1497560967 1 2024-05-17T21:52:47.802 INFO:tasks.workunit.client.0.smithi107.stdout:7/202: creat d5/df/d11/d1d/d44/f4b x:0 0 0 2024-05-17T21:52:47.803 INFO:tasks.workunit.client.0.smithi107.stdout:6/219: dread d9/d16/f1e [0,4194304] 0 2024-05-17T21:52:47.803 INFO:tasks.workunit.client.0.smithi107.stdout:7/203: read d5/df/f26 [2023748,117971] 0 2024-05-17T21:52:47.804 INFO:tasks.workunit.client.0.smithi107.stdout:6/220: write d9/d16/f24 [1640217,91156] 0 2024-05-17T21:52:47.805 INFO:tasks.workunit.client.0.smithi107.stdout:3/248: write d6/d8/d10/d25/f47 [1462148,75504] 0 2024-05-17T21:52:47.806 INFO:tasks.workunit.client.1.smithi138.stdout:6/167: chown d7/l16 85628 1 2024-05-17T21:52:47.806 INFO:tasks.workunit.client.1.smithi138.stdout:6/168: chown d7/f2e 2103 1 2024-05-17T21:52:47.806 INFO:tasks.workunit.client.0.smithi107.stdout:3/249: truncate d6/d8/d10/f38 492062 0 2024-05-17T21:52:47.807 INFO:tasks.workunit.client.0.smithi107.stdout:4/232: rename d3/db/d1e/f20 to d3/db/f4d 0 2024-05-17T21:52:47.808 INFO:tasks.workunit.client.0.smithi107.stdout:2/177: mknod d0/d9/d11/d27/c32 0 2024-05-17T21:52:47.811 INFO:tasks.workunit.client.1.smithi138.stdout:7/77: read d0/fc [2668710,29887] 0 2024-05-17T21:52:47.811 INFO:tasks.workunit.client.0.smithi107.stdout:5/244: mkdir d5/db/d16/d2d/d51 0 2024-05-17T21:52:47.811 INFO:tasks.workunit.client.1.smithi138.stdout:7/78: read d0/fc [3489246,14249] 0 2024-05-17T21:52:47.812 INFO:tasks.workunit.client.0.smithi107.stdout:5/245: chown d5/db/d16/d2d/d32 49 1 2024-05-17T21:52:47.812 INFO:tasks.workunit.client.0.smithi107.stdout:5/246: fsync d5/db/d16/d2d/d32/f33 0 2024-05-17T21:52:47.813 INFO:tasks.workunit.client.1.smithi138.stdout:4/74: mkdir d0/d3/d1d 0 2024-05-17T21:52:47.814 INFO:tasks.workunit.client.1.smithi138.stdout:1/75: truncate d4/dd/f13 940558 0 2024-05-17T21:52:47.815 INFO:tasks.workunit.client.1.smithi138.stdout:1/76: chown d4/dd/f15 33372223 1 2024-05-17T21:52:47.815 INFO:tasks.workunit.client.1.smithi138.stdout:7/79: dread d0/fc [0,4194304] 0 2024-05-17T21:52:47.816 INFO:tasks.workunit.client.1.smithi138.stdout:7/80: stat d0/fa 0 2024-05-17T21:52:47.817 INFO:tasks.workunit.client.1.smithi138.stdout:3/123: read d1/d3/d8/f13 [974648,64689] 0 2024-05-17T21:52:47.818 INFO:tasks.workunit.client.1.smithi138.stdout:5/98: mkdir d1/d6/d19 0 2024-05-17T21:52:47.822 INFO:tasks.workunit.client.1.smithi138.stdout:6/169: mkdir d7/db/d34 0 2024-05-17T21:52:47.823 INFO:tasks.workunit.client.1.smithi138.stdout:6/170: write d7/db/f11 [1081096,39929] 0 2024-05-17T21:52:47.824 INFO:tasks.workunit.client.0.smithi107.stdout:3/250: fdatasync d6/f1b 0 2024-05-17T21:52:47.827 INFO:tasks.workunit.client.0.smithi107.stdout:0/301: rename d3/f17 to d3/dc/d22/d23/d6c/f70 0 2024-05-17T21:52:47.828 INFO:tasks.workunit.client.0.smithi107.stdout:8/220: write dc/d10/d17/f3f [992304,3983] 0 2024-05-17T21:52:47.828 INFO:tasks.workunit.client.0.smithi107.stdout:3/251: dread d6/d2a/d2d/d37/f39 [0,4194304] 0 2024-05-17T21:52:47.832 INFO:tasks.workunit.client.0.smithi107.stdout:1/273: symlink d4/df/d14/d2e/d43/l53 0 2024-05-17T21:52:47.833 INFO:tasks.workunit.client.1.smithi138.stdout:7/81: write d0/fa [871921,113504] 0 2024-05-17T21:52:47.833 INFO:tasks.workunit.client.1.smithi138.stdout:3/124: stat d1/d3/l18 0 2024-05-17T21:52:47.834 INFO:tasks.workunit.client.1.smithi138.stdout:7/82: chown d0/c1 0 1 2024-05-17T21:52:47.835 INFO:tasks.workunit.client.1.smithi138.stdout:0/172: creat d3/f45 x:0 0 0 2024-05-17T21:52:47.836 INFO:tasks.workunit.client.1.smithi138.stdout:5/99: truncate d1/d6/f11 1228203 0 2024-05-17T21:52:47.838 INFO:tasks.workunit.client.1.smithi138.stdout:9/105: getdents d6/d7/d16 0 2024-05-17T21:52:47.838 INFO:tasks.workunit.client.1.smithi138.stdout:9/106: write d6/f1b [1131454,21908] 0 2024-05-17T21:52:47.839 INFO:tasks.workunit.client.1.smithi138.stdout:9/107: fsync f2 0 2024-05-17T21:52:47.840 INFO:tasks.workunit.client.1.smithi138.stdout:6/171: creat d7/d2d/f35 x:0 0 0 2024-05-17T21:52:47.841 INFO:tasks.workunit.client.1.smithi138.stdout:4/75: getdents d0/d3/db 0 2024-05-17T21:52:47.841 INFO:tasks.workunit.client.1.smithi138.stdout:6/172: chown l5 21086279 1 2024-05-17T21:52:47.842 INFO:tasks.workunit.client.0.smithi107.stdout:6/221: truncate d9/f10 1592712 0 2024-05-17T21:52:47.842 INFO:tasks.workunit.client.0.smithi107.stdout:6/222: write d9/d16/f3b [677715,107841] 0 2024-05-17T21:52:47.843 INFO:tasks.workunit.client.1.smithi138.stdout:1/77: link d4/dd/f15 d4/f16 0 2024-05-17T21:52:47.847 INFO:tasks.workunit.client.1.smithi138.stdout:7/83: readlink d0/ld 0 2024-05-17T21:52:47.848 INFO:tasks.workunit.client.1.smithi138.stdout:0/173: creat d3/d10/d38/f46 x:0 0 0 2024-05-17T21:52:47.848 INFO:tasks.workunit.client.1.smithi138.stdout:4/76: dwrite d0/d3/f16 [0,4194304] 0 2024-05-17T21:52:47.856 INFO:tasks.workunit.client.0.smithi107.stdout:1/274: mknod d4/df/d1a/d41/c54 0 2024-05-17T21:52:47.858 INFO:tasks.workunit.client.0.smithi107.stdout:5/247: truncate d5/db/f25 1254793 0 2024-05-17T21:52:47.858 INFO:tasks.workunit.client.0.smithi107.stdout:5/248: chown d5/db/dc/l12 4 1 2024-05-17T21:52:47.859 INFO:tasks.workunit.client.1.smithi138.stdout:9/108: rename d6/d7/fb to d6/d7/d1f/f23 0 2024-05-17T21:52:47.862 INFO:tasks.workunit.client.1.smithi138.stdout:1/78: mknod d4/df/c17 0 2024-05-17T21:52:47.866 INFO:tasks.workunit.client.0.smithi107.stdout:4/233: write d3/d4a/f3f [2670440,130257] 0 2024-05-17T21:52:47.868 INFO:tasks.workunit.client.0.smithi107.stdout:4/234: write d3/db/dd/d3c/f3e [414835,30548] 0 2024-05-17T21:52:47.868 INFO:tasks.workunit.client.1.smithi138.stdout:9/109: dwrite d6/d7/f10 [0,4194304] 0 2024-05-17T21:52:47.868 INFO:tasks.workunit.client.1.smithi138.stdout:3/125: unlink d1/d3/d8/c15 0 2024-05-17T21:52:47.869 INFO:tasks.workunit.client.0.smithi107.stdout:9/267: sync 2024-05-17T21:52:47.870 INFO:tasks.workunit.client.0.smithi107.stdout:4/235: read d3/db/fc [779715,10403] 0 2024-05-17T21:52:47.871 INFO:tasks.workunit.client.1.smithi138.stdout:7/84: creat d0/f19 x:0 0 0 2024-05-17T21:52:47.873 INFO:tasks.workunit.client.1.smithi138.stdout:4/77: symlink d0/d3/l1e 0 2024-05-17T21:52:47.876 INFO:tasks.workunit.client.0.smithi107.stdout:3/252: mkdir d6/d2a/d2d/d48/d4f 0 2024-05-17T21:52:47.877 INFO:tasks.workunit.client.1.smithi138.stdout:4/78: dread d0/f7 [0,4194304] 0 2024-05-17T21:52:47.877 INFO:tasks.workunit.client.0.smithi107.stdout:0/302: dwrite d3/ff [0,4194304] 0 2024-05-17T21:52:47.878 INFO:tasks.workunit.client.1.smithi138.stdout:4/79: write d0/d1b/f18 [506578,62834] 0 2024-05-17T21:52:47.879 INFO:tasks.workunit.client.0.smithi107.stdout:2/178: creat d0/d9/f33 x:0 0 0 2024-05-17T21:52:47.880 INFO:tasks.workunit.client.1.smithi138.stdout:4/80: write d0/d3/d17/f1a [1034009,14035] 0 2024-05-17T21:52:47.880 INFO:tasks.workunit.client.1.smithi138.stdout:4/81: chown d0/d3/db/c19 116 1 2024-05-17T21:52:47.881 INFO:tasks.workunit.client.1.smithi138.stdout:4/82: chown d0/f7 25658 1 2024-05-17T21:52:47.882 INFO:tasks.workunit.client.1.smithi138.stdout:6/173: rename d7/db/d12/f1d to d7/f36 0 2024-05-17T21:52:47.884 INFO:tasks.workunit.client.1.smithi138.stdout:1/79: mknod d4/df/c18 0 2024-05-17T21:52:47.885 INFO:tasks.workunit.client.0.smithi107.stdout:5/249: creat d5/db/d16/d2d/d32/f52 x:0 0 0 2024-05-17T21:52:47.887 INFO:tasks.workunit.client.0.smithi107.stdout:7/204: getdents d5/d16 0 2024-05-17T21:52:47.891 INFO:tasks.workunit.client.0.smithi107.stdout:9/268: unlink d0/dd/c36 0 2024-05-17T21:52:47.892 INFO:tasks.workunit.client.0.smithi107.stdout:9/269: write d0/dd/d2f/f31 [330314,111628] 0 2024-05-17T21:52:47.893 INFO:tasks.workunit.client.1.smithi138.stdout:0/174: symlink d3/l47 0 2024-05-17T21:52:47.894 INFO:tasks.workunit.client.0.smithi107.stdout:4/236: mknod d3/db/dd/c4e 0 2024-05-17T21:52:47.894 INFO:tasks.workunit.client.0.smithi107.stdout:4/237: chown d3/d6 6124096 1 2024-05-17T21:52:47.896 INFO:tasks.workunit.client.0.smithi107.stdout:9/270: dread d0/f17 [0,4194304] 0 2024-05-17T21:52:47.898 INFO:tasks.workunit.client.0.smithi107.stdout:4/238: dread d3/db/f46 [0,4194304] 0 2024-05-17T21:52:47.898 INFO:tasks.workunit.client.0.smithi107.stdout:4/239: fsync d3/db/f4d 0 2024-05-17T21:52:47.899 INFO:tasks.workunit.client.1.smithi138.stdout:5/100: dread d1/d6/f11 [0,4194304] 0 2024-05-17T21:52:47.899 INFO:tasks.workunit.client.0.smithi107.stdout:8/221: creat dc/d21/f5d x:0 0 0 2024-05-17T21:52:47.900 INFO:tasks.workunit.client.1.smithi138.stdout:0/175: dwrite d3/f29 [0,4194304] 0 2024-05-17T21:52:47.905 INFO:tasks.workunit.client.1.smithi138.stdout:4/83: mkdir d0/d3/d10/d1f 0 2024-05-17T21:52:47.907 INFO:tasks.workunit.client.1.smithi138.stdout:5/101: dwrite d1/d6/fe [0,4194304] 0 2024-05-17T21:52:47.909 INFO:tasks.workunit.client.1.smithi138.stdout:3/126: rename d1/d3/d8/de/c22 to d1/d3/d8/c2b 0 2024-05-17T21:52:47.909 INFO:tasks.workunit.client.1.smithi138.stdout:7/85: write d0/fc [3289225,50245] 0 2024-05-17T21:52:47.912 INFO:tasks.workunit.client.0.smithi107.stdout:0/303: dwrite d3/dc/f16 [0,4194304] 0 2024-05-17T21:52:47.913 INFO:tasks.workunit.client.1.smithi138.stdout:6/174: rename d7/f36 to d7/f37 0 2024-05-17T21:52:47.919 INFO:tasks.workunit.client.0.smithi107.stdout:5/250: creat d5/db/d16/d2a/d2b/f53 x:0 0 0 2024-05-17T21:52:47.920 INFO:tasks.workunit.client.1.smithi138.stdout:7/86: dwrite d0/f5 [0,4194304] 0 2024-05-17T21:52:47.920 INFO:tasks.workunit.client.0.smithi107.stdout:7/205: fsync d5/f18 0 2024-05-17T21:52:47.922 INFO:tasks.workunit.client.1.smithi138.stdout:7/87: write d0/f13 [1579928,41707] 0 2024-05-17T21:52:47.922 INFO:tasks.workunit.client.1.smithi138.stdout:7/88: chown d0 11 1 2024-05-17T21:52:47.925 INFO:tasks.workunit.client.1.smithi138.stdout:0/176: read f0 [418735,93912] 0 2024-05-17T21:52:47.926 INFO:tasks.workunit.client.0.smithi107.stdout:9/271: creat d0/dd/d1f/f5c x:0 0 0 2024-05-17T21:52:47.928 INFO:tasks.workunit.client.1.smithi138.stdout:4/84: creat d0/d3/db/f20 x:0 0 0 2024-05-17T21:52:47.929 INFO:tasks.workunit.client.1.smithi138.stdout:0/177: dread f2 [4194304,4194304] 0 2024-05-17T21:52:47.929 INFO:tasks.workunit.client.1.smithi138.stdout:5/102: rmdir d1 39 2024-05-17T21:52:47.930 INFO:tasks.workunit.client.1.smithi138.stdout:4/85: write d0/d3/f16 [2998043,120792] 0 2024-05-17T21:52:47.930 INFO:tasks.workunit.client.1.smithi138.stdout:4/86: readlink d0/d3/l1e 0 2024-05-17T21:52:47.930 INFO:tasks.workunit.client.1.smithi138.stdout:0/178: chown d3/d10/f2c 41 1 2024-05-17T21:52:47.931 INFO:tasks.workunit.client.1.smithi138.stdout:0/179: stat d3/d10/c26 0 2024-05-17T21:52:47.932 INFO:tasks.workunit.client.0.smithi107.stdout:4/240: mkdir d3/db/d35/d4f 0 2024-05-17T21:52:47.932 INFO:tasks.workunit.client.0.smithi107.stdout:4/241: read d3/db/d16/f30 [215216,96937] 0 2024-05-17T21:52:47.934 INFO:tasks.workunit.client.0.smithi107.stdout:8/222: chown dc/d21/d27/d34/c3a 2 1 2024-05-17T21:52:47.935 INFO:tasks.workunit.client.0.smithi107.stdout:1/275: truncate d4/df/d14/f23 248373 0 2024-05-17T21:52:47.935 INFO:tasks.workunit.client.0.smithi107.stdout:1/276: rename d4/df/d1a to d4/df/d1a/d55 22 2024-05-17T21:52:47.936 INFO:tasks.workunit.client.1.smithi138.stdout:1/80: write d4/fc [14092,14331] 0 2024-05-17T21:52:47.936 INFO:tasks.workunit.client.0.smithi107.stdout:1/277: write d4/df/d14/d1d/f2c [1010376,115920] 0 2024-05-17T21:52:47.937 INFO:tasks.workunit.client.0.smithi107.stdout:3/253: truncate f3 1429221 0 2024-05-17T21:52:47.940 INFO:tasks.workunit.client.1.smithi138.stdout:9/110: creat d6/f24 x:0 0 0 2024-05-17T21:52:47.941 INFO:tasks.workunit.client.1.smithi138.stdout:9/111: write d6/f24 [838386,87251] 0 2024-05-17T21:52:47.942 INFO:tasks.workunit.client.0.smithi107.stdout:5/251: mkdir d5/db/d16/d54 0 2024-05-17T21:52:47.945 INFO:tasks.workunit.client.0.smithi107.stdout:6/223: getdents d9/d16/d22/d2d 0 2024-05-17T21:52:47.946 INFO:tasks.workunit.client.0.smithi107.stdout:6/224: write d9/f38 [212496,6158] 0 2024-05-17T21:52:47.947 INFO:tasks.workunit.client.1.smithi138.stdout:0/180: fsync d3/d10/f25 0 2024-05-17T21:52:47.948 INFO:tasks.workunit.client.0.smithi107.stdout:9/272: fdatasync d0/dd/d2f/d4a/f4c 0 2024-05-17T21:52:47.948 INFO:tasks.workunit.client.0.smithi107.stdout:9/273: read d0/dd/d1b/f48 [1320060,112859] 0 2024-05-17T21:52:47.950 INFO:tasks.workunit.client.1.smithi138.stdout:5/103: dread d1/d6/fe [0,4194304] 0 2024-05-17T21:52:47.955 INFO:tasks.workunit.client.0.smithi107.stdout:4/242: mknod d3/db/c50 0 2024-05-17T21:52:47.956 INFO:tasks.workunit.client.0.smithi107.stdout:4/243: write d3/d4a/d47/f4b [442867,106434] 0 2024-05-17T21:52:47.957 INFO:tasks.workunit.client.0.smithi107.stdout:8/223: unlink dc/d10/c29 0 2024-05-17T21:52:47.959 INFO:tasks.workunit.client.1.smithi138.stdout:8/57: sync 2024-05-17T21:52:47.962 INFO:tasks.workunit.client.0.smithi107.stdout:7/206: unlink d5/d16/d3b/d45/c4a 0 2024-05-17T21:52:47.963 INFO:tasks.workunit.client.0.smithi107.stdout:7/207: write d5/fe [3447840,43299] 0 2024-05-17T21:52:47.967 INFO:tasks.workunit.client.0.smithi107.stdout:9/274: unlink d0/dd/d3d/f55 0 2024-05-17T21:52:47.970 INFO:tasks.workunit.client.0.smithi107.stdout:8/224: symlink dc/d10/l5e 0 2024-05-17T21:52:47.972 INFO:tasks.workunit.client.0.smithi107.stdout:3/254: symlink d6/d8/l50 0 2024-05-17T21:52:47.972 INFO:tasks.workunit.client.0.smithi107.stdout:2/179: link d0/d26/c2d d0/c34 0 2024-05-17T21:52:47.974 INFO:tasks.workunit.client.0.smithi107.stdout:0/304: link d3/dc/f26 d3/d58/f71 0 2024-05-17T21:52:47.974 INFO:tasks.workunit.client.0.smithi107.stdout:0/305: readlink d3/dc/l4e 0 2024-05-17T21:52:47.975 INFO:tasks.workunit.client.0.smithi107.stdout:5/252: mkdir d5/db/d16/d2d/d40/d55 0 2024-05-17T21:52:47.975 INFO:tasks.workunit.client.0.smithi107.stdout:5/253: chown d5/db/d16/d2d/l37 9 1 2024-05-17T21:52:47.976 INFO:tasks.workunit.client.1.smithi138.stdout:3/127: rename d1/d3/l1d to d1/l2c 0 2024-05-17T21:52:47.977 INFO:tasks.workunit.client.1.smithi138.stdout:1/81: creat d4/f19 x:0 0 0 2024-05-17T21:52:47.978 INFO:tasks.workunit.client.0.smithi107.stdout:9/275: creat d0/dd/d21/d29/d3e/f5d x:0 0 0 2024-05-17T21:52:47.978 INFO:tasks.workunit.client.1.smithi138.stdout:7/89: link d0/f13 d0/f1a 0 2024-05-17T21:52:47.979 INFO:tasks.workunit.client.1.smithi138.stdout:3/128: truncate d1/f12 833901 0 2024-05-17T21:52:47.979 INFO:tasks.workunit.client.0.smithi107.stdout:4/244: creat d3/db/dd/d3a/d41/f51 x:0 0 0 2024-05-17T21:52:47.980 INFO:tasks.workunit.client.1.smithi138.stdout:7/90: stat d0 0 2024-05-17T21:52:47.981 INFO:tasks.workunit.client.0.smithi107.stdout:8/225: symlink dc/d44/l5f 0 2024-05-17T21:52:47.982 INFO:tasks.workunit.client.0.smithi107.stdout:8/226: stat dc/d21/d35/c12 0 2024-05-17T21:52:47.983 INFO:tasks.workunit.client.1.smithi138.stdout:7/91: dread d0/f11 [0,4194304] 0 2024-05-17T21:52:47.984 INFO:tasks.workunit.client.0.smithi107.stdout:1/278: link f3 d4/df/d1a/f56 0 2024-05-17T21:52:47.984 INFO:tasks.workunit.client.1.smithi138.stdout:6/175: rename d7/d8/l32 to d7/db/l38 0 2024-05-17T21:52:47.985 INFO:tasks.workunit.client.1.smithi138.stdout:5/104: dread d1/d6/fe [0,4194304] 0 2024-05-17T21:52:47.986 INFO:tasks.workunit.client.1.smithi138.stdout:1/82: symlink d4/df/l1a 0 2024-05-17T21:52:47.988 INFO:tasks.workunit.client.0.smithi107.stdout:3/255: unlink d6/d8/d10/d25/c34 0 2024-05-17T21:52:47.988 INFO:tasks.workunit.client.1.smithi138.stdout:9/112: dwrite d6/d7/d1f/f22 [0,4194304] 0 2024-05-17T21:52:47.991 INFO:tasks.workunit.client.0.smithi107.stdout:2/180: unlink d0/d9/d11/df/f28 0 2024-05-17T21:52:47.991 INFO:tasks.workunit.client.1.smithi138.stdout:6/176: truncate d7/d8/f24 4926225 0 2024-05-17T21:52:47.992 INFO:tasks.workunit.client.0.smithi107.stdout:0/306: creat d3/dc/d22/d5d/f72 x:0 0 0 2024-05-17T21:52:47.993 INFO:tasks.workunit.client.1.smithi138.stdout:6/177: readlink l5 0 2024-05-17T21:52:47.993 INFO:tasks.workunit.client.1.smithi138.stdout:9/113: chown d6/d7/d16/c1e 215 1 2024-05-17T21:52:47.993 INFO:tasks.workunit.client.1.smithi138.stdout:3/129: dwrite d1/d3/d8/de/f27 [0,4194304] 0 2024-05-17T21:52:47.995 INFO:tasks.workunit.client.1.smithi138.stdout:6/178: stat d7/db/c15 0 2024-05-17T21:52:47.996 INFO:tasks.workunit.client.1.smithi138.stdout:8/58: creat d0/dc/fd x:0 0 0 2024-05-17T21:52:47.998 INFO:tasks.workunit.client.0.smithi107.stdout:6/225: rename d9/f38 to d9/d16/f41 0 2024-05-17T21:52:48.000 INFO:tasks.workunit.client.0.smithi107.stdout:8/227: dread dc/f18 [0,4194304] 0 2024-05-17T21:52:48.001 INFO:tasks.workunit.client.0.smithi107.stdout:5/254: creat d5/db/dc/f56 x:0 0 0 2024-05-17T21:52:48.001 INFO:tasks.workunit.client.0.smithi107.stdout:8/228: write dc/d21/d27/d2f/f30 [356447,57308] 0 2024-05-17T21:52:48.002 INFO:tasks.workunit.client.0.smithi107.stdout:8/229: write dc/d10/d17/f3b [208621,120714] 0 2024-05-17T21:52:48.008 INFO:tasks.workunit.client.0.smithi107.stdout:3/256: mknod d6/d8/d10/d25/c51 0 2024-05-17T21:52:48.009 INFO:tasks.workunit.client.1.smithi138.stdout:1/83: creat d4/df/f1b x:0 0 0 2024-05-17T21:52:48.009 INFO:tasks.workunit.client.0.smithi107.stdout:9/276: fsync d0/dd/d21/d29/d3e/f5d 0 2024-05-17T21:52:48.011 INFO:tasks.workunit.client.0.smithi107.stdout:0/307: chown d3/f1d 423561860 1 2024-05-17T21:52:48.011 INFO:tasks.workunit.client.1.smithi138.stdout:9/114: rmdir d6/d7/d16 39 2024-05-17T21:52:48.014 INFO:tasks.workunit.client.1.smithi138.stdout:6/179: fdatasync d7/f37 0 2024-05-17T21:52:48.015 INFO:tasks.workunit.client.1.smithi138.stdout:6/180: write d7/f28 [668191,53575] 0 2024-05-17T21:52:48.015 INFO:tasks.workunit.client.1.smithi138.stdout:6/181: stat d7/db/d34 0 2024-05-17T21:52:48.017 INFO:tasks.workunit.client.0.smithi107.stdout:7/208: rename d5/d16/f3e to d5/df/d11/f4c 0 2024-05-17T21:52:48.018 INFO:tasks.workunit.client.0.smithi107.stdout:7/209: write d5/f18 [3254540,112244] 0 2024-05-17T21:52:48.020 INFO:tasks.workunit.client.1.smithi138.stdout:0/181: truncate d3/d10/f17 2715788 0 2024-05-17T21:52:48.020 INFO:tasks.workunit.client.1.smithi138.stdout:0/182: chown d3/f29 8770 1 2024-05-17T21:52:48.021 INFO:tasks.workunit.client.1.smithi138.stdout:0/183: write d3/f29 [2388952,35723] 0 2024-05-17T21:52:48.022 INFO:tasks.workunit.client.1.smithi138.stdout:7/92: mknod d0/d18/c1b 0 2024-05-17T21:52:48.025 INFO:tasks.workunit.client.1.smithi138.stdout:5/105: mkdir d1/d1a 0 2024-05-17T21:52:48.027 INFO:tasks.workunit.client.0.smithi107.stdout:8/230: read dc/d21/d27/d2f/f32 [30470,27553] 0 2024-05-17T21:52:48.028 INFO:tasks.workunit.client.0.smithi107.stdout:4/245: symlink d3/d6/d26/d3b/l52 0 2024-05-17T21:52:48.029 INFO:tasks.workunit.client.0.smithi107.stdout:8/231: write dc/d21/d27/f5b [437926,92342] 0 2024-05-17T21:52:48.029 INFO:tasks.workunit.client.0.smithi107.stdout:4/246: stat d3/db/dd/l2e 0 2024-05-17T21:52:48.029 INFO:tasks.workunit.client.0.smithi107.stdout:8/232: chown dc/l24 1125241 1 2024-05-17T21:52:48.030 INFO:tasks.workunit.client.1.smithi138.stdout:3/130: mkdir d1/d3/d16/d2d 0 2024-05-17T21:52:48.030 INFO:tasks.workunit.client.0.smithi107.stdout:1/279: mkdir d4/d57 0 2024-05-17T21:52:48.031 INFO:tasks.workunit.client.1.smithi138.stdout:9/115: readlink d6/d7/l12 0 2024-05-17T21:52:48.032 INFO:tasks.workunit.client.0.smithi107.stdout:3/257: creat d6/d2a/f52 x:0 0 0 2024-05-17T21:52:48.033 INFO:tasks.workunit.client.1.smithi138.stdout:6/182: creat d7/d2d/f39 x:0 0 0 2024-05-17T21:52:48.034 INFO:tasks.workunit.client.0.smithi107.stdout:6/226: mkdir d9/d16/d42 0 2024-05-17T21:52:48.035 INFO:tasks.workunit.client.1.smithi138.stdout:0/184: mkdir d3/d3d/d48 0 2024-05-17T21:52:48.036 INFO:tasks.workunit.client.1.smithi138.stdout:0/185: chown d3/d10/d38/f40 1765 1 2024-05-17T21:52:48.036 INFO:tasks.workunit.client.1.smithi138.stdout:0/186: dread - d3/d10/f25 zero size 2024-05-17T21:52:48.037 INFO:tasks.workunit.client.0.smithi107.stdout:2/181: rename d0/d9/d11/d15/f2b to d0/d9/d11/df/f35 0 2024-05-17T21:52:48.039 INFO:tasks.workunit.client.0.smithi107.stdout:2/182: read d0/f25 [1475528,50440] 0 2024-05-17T21:52:48.041 INFO:tasks.workunit.client.1.smithi138.stdout:5/106: mkdir d1/d6/d1b 0 2024-05-17T21:52:48.043 INFO:tasks.workunit.client.0.smithi107.stdout:2/183: dread d0/d9/d11/f14 [0,4194304] 0 2024-05-17T21:52:48.046 INFO:tasks.workunit.client.1.smithi138.stdout:8/59: link d0/c8 d0/dc/ce 0 2024-05-17T21:52:48.048 INFO:tasks.workunit.client.0.smithi107.stdout:1/280: unlink d4/df/d14/l25 0 2024-05-17T21:52:48.049 INFO:tasks.workunit.client.1.smithi138.stdout:6/183: mknod d7/db/d12/c3a 0 2024-05-17T21:52:48.051 INFO:tasks.workunit.client.0.smithi107.stdout:3/258: symlink d6/d2a/d2d/l53 0 2024-05-17T21:52:48.051 INFO:tasks.workunit.client.0.smithi107.stdout:3/259: fdatasync d6/d8/d10/f38 0 2024-05-17T21:52:48.052 INFO:tasks.workunit.client.1.smithi138.stdout:0/187: unlink d3/l34 0 2024-05-17T21:52:48.053 INFO:tasks.workunit.client.1.smithi138.stdout:6/184: dread f1 [8388608,4194304] 0 2024-05-17T21:52:48.053 INFO:tasks.workunit.client.1.smithi138.stdout:0/188: chown d3/f43 190818407 1 2024-05-17T21:52:48.055 INFO:tasks.workunit.client.0.smithi107.stdout:0/308: mknod d3/c73 0 2024-05-17T21:52:48.058 INFO:tasks.workunit.client.0.smithi107.stdout:6/227: mkdir d9/d16/d22/d43 0 2024-05-17T21:52:48.058 INFO:tasks.workunit.client.0.smithi107.stdout:6/228: readlink d9/l40 0 2024-05-17T21:52:48.059 INFO:tasks.workunit.client.1.smithi138.stdout:1/84: link d4/dd/le d4/dd/l1c 0 2024-05-17T21:52:48.061 INFO:tasks.workunit.client.1.smithi138.stdout:6/185: dwrite d7/db/f25 [0,4194304] 0 2024-05-17T21:52:48.061 INFO:tasks.workunit.client.1.smithi138.stdout:1/85: chown d4/dd/f15 2522466 1 2024-05-17T21:52:48.063 INFO:tasks.workunit.client.0.smithi107.stdout:2/184: dread d0/fa [0,4194304] 0 2024-05-17T21:52:48.065 INFO:tasks.workunit.client.0.smithi107.stdout:5/255: rename d5/db/dc/d18/l41 to d5/db/dc/l57 0 2024-05-17T21:52:48.070 INFO:tasks.workunit.client.1.smithi138.stdout:1/86: dwrite d4/f9 [0,4194304] 0 2024-05-17T21:52:48.072 INFO:tasks.workunit.client.1.smithi138.stdout:1/87: write f2 [6504745,23781] 0 2024-05-17T21:52:48.074 INFO:tasks.workunit.client.0.smithi107.stdout:4/247: creat d3/db/d16/d43/f53 x:0 0 0 2024-05-17T21:52:48.076 INFO:tasks.workunit.client.1.smithi138.stdout:0/189: mknod d3/d10/d38/d3e/c49 0 2024-05-17T21:52:48.081 INFO:tasks.workunit.client.1.smithi138.stdout:6/186: mknod d7/db/c3b 0 2024-05-17T21:52:48.082 INFO:tasks.workunit.client.1.smithi138.stdout:6/187: fsync d7/f28 0 2024-05-17T21:52:48.084 INFO:tasks.workunit.client.1.smithi138.stdout:5/107: mknod d1/d6/d1b/c1c 0 2024-05-17T21:52:48.087 INFO:tasks.workunit.client.0.smithi107.stdout:9/277: rename d0/dd/d1b/f28 to d0/dd/d21/d29/d3e/f5e 0 2024-05-17T21:52:48.089 INFO:tasks.workunit.client.1.smithi138.stdout:8/60: link d0/f9 d0/ff 0 2024-05-17T21:52:48.089 INFO:tasks.workunit.client.0.smithi107.stdout:9/278: write d0/dd/d2f/d4a/f4c [1573100,85685] 0 2024-05-17T21:52:48.090 INFO:tasks.workunit.client.0.smithi107.stdout:6/229: dwrite d9/d16/d22/d2d/f2f [0,4194304] 0 2024-05-17T21:52:48.091 INFO:tasks.workunit.client.0.smithi107.stdout:5/256: rmdir d5/db/d39 39 2024-05-17T21:52:48.092 INFO:tasks.workunit.client.0.smithi107.stdout:7/210: write d5/df/d11/d1c/d31/f48 [632755,116127] 0 2024-05-17T21:52:48.093 INFO:tasks.workunit.client.0.smithi107.stdout:7/211: write d5/d16/f1e [2095405,2951] 0 2024-05-17T21:52:48.097 INFO:tasks.workunit.client.0.smithi107.stdout:8/233: write dc/f18 [2539733,17905] 0 2024-05-17T21:52:48.097 INFO:tasks.workunit.client.1.smithi138.stdout:8/61: dwrite d0/dc/fd [0,4194304] 0 2024-05-17T21:52:48.099 INFO:tasks.workunit.client.1.smithi138.stdout:8/62: dread d0/ff [0,4194304] 0 2024-05-17T21:52:48.099 INFO:tasks.workunit.client.0.smithi107.stdout:4/248: symlink d3/d6/d26/d3b/l54 0 2024-05-17T21:52:48.104 INFO:tasks.workunit.client.0.smithi107.stdout:8/234: dwrite dc/f18 [0,4194304] 0 2024-05-17T21:52:48.104 INFO:tasks.workunit.client.1.smithi138.stdout:7/93: getdents d0 0 2024-05-17T21:52:48.109 INFO:tasks.workunit.client.1.smithi138.stdout:8/63: dwrite d0/fb [0,4194304] 0 2024-05-17T21:52:48.113 INFO:tasks.workunit.client.1.smithi138.stdout:3/131: truncate d1/d3/d8/de/f27 260453 0 2024-05-17T21:52:48.116 INFO:tasks.workunit.client.1.smithi138.stdout:5/108: rename d1/d6/fc to d1/d6/d1b/f1d 0 2024-05-17T21:52:48.122 INFO:tasks.workunit.client.1.smithi138.stdout:1/88: symlink d4/db/l1d 0 2024-05-17T21:52:48.125 INFO:tasks.workunit.client.1.smithi138.stdout:0/190: symlink d3/d3d/d48/l4a 0 2024-05-17T21:52:48.126 INFO:tasks.workunit.client.0.smithi107.stdout:9/279: fdatasync d0/dd/d1f/d2c/f34 0 2024-05-17T21:52:48.126 INFO:tasks.workunit.client.1.smithi138.stdout:9/116: dwrite d6/d7/d16/f21 [0,4194304] 0 2024-05-17T21:52:48.129 INFO:tasks.workunit.client.1.smithi138.stdout:9/117: write d6/d7/fc [2059014,120613] 0 2024-05-17T21:52:48.130 INFO:tasks.workunit.client.1.smithi138.stdout:1/89: dread f2 [0,4194304] 0 2024-05-17T21:52:48.130 INFO:tasks.workunit.client.0.smithi107.stdout:3/260: write d6/d1a/f1f [168149,106976] 0 2024-05-17T21:52:48.132 INFO:tasks.workunit.client.1.smithi138.stdout:7/94: creat d0/f1c x:0 0 0 2024-05-17T21:52:48.135 INFO:tasks.workunit.client.0.smithi107.stdout:7/212: mkdir d5/df/d11/d2a/d4d 0 2024-05-17T21:52:48.136 INFO:tasks.workunit.client.0.smithi107.stdout:7/213: write d5/d16/f23 [205974,97751] 0 2024-05-17T21:52:48.136 INFO:tasks.workunit.client.1.smithi138.stdout:0/191: dwrite d3/f29 [4194304,4194304] 0 2024-05-17T21:52:48.140 INFO:tasks.workunit.client.1.smithi138.stdout:8/64: chown d0/c8 0 1 2024-05-17T21:52:48.140 INFO:tasks.workunit.client.0.smithi107.stdout:4/249: mkdir d3/db/d35/d55 0 2024-05-17T21:52:48.143 INFO:tasks.workunit.client.1.smithi138.stdout:7/95: dwrite d0/f12 [0,4194304] 0 2024-05-17T21:52:48.143 INFO:tasks.workunit.client.1.smithi138.stdout:3/132: write d1/d3/f1b [834794,106219] 0 2024-05-17T21:52:48.145 INFO:tasks.workunit.client.0.smithi107.stdout:8/235: rename dc/d10/c14 to dc/d44/d4b/c60 0 2024-05-17T21:52:48.147 INFO:tasks.workunit.client.1.smithi138.stdout:5/109: unlink d1/l2 0 2024-05-17T21:52:48.151 INFO:tasks.workunit.client.0.smithi107.stdout:2/185: creat d0/f36 x:0 0 0 2024-05-17T21:52:48.151 INFO:tasks.workunit.client.1.smithi138.stdout:6/188: write d7/d8/ff [4158120,91087] 0 2024-05-17T21:52:48.152 INFO:tasks.workunit.client.1.smithi138.stdout:6/189: write d7/f28 [124165,21248] 0 2024-05-17T21:52:48.158 INFO:tasks.workunit.client.1.smithi138.stdout:1/90: symlink d4/db/l1e 0 2024-05-17T21:52:48.159 INFO:tasks.workunit.client.0.smithi107.stdout:5/257: fdatasync d5/db/d39/f4c 0 2024-05-17T21:52:48.160 INFO:tasks.workunit.client.0.smithi107.stdout:5/258: chown d5/db/dc/c10 62 1 2024-05-17T21:52:48.161 INFO:tasks.workunit.client.0.smithi107.stdout:6/230: dwrite d9/d16/f18 [0,4194304] 0 2024-05-17T21:52:48.161 INFO:tasks.workunit.client.1.smithi138.stdout:9/118: write d6/d7/d1f/f23 [3961598,57979] 0 2024-05-17T21:52:48.161 INFO:tasks.workunit.client.0.smithi107.stdout:3/261: symlink d6/d2a/d2d/d48/l54 0 2024-05-17T21:52:48.162 INFO:tasks.workunit.client.1.smithi138.stdout:9/119: dread - d6/d7/f17 zero size 2024-05-17T21:52:48.162 INFO:tasks.workunit.client.0.smithi107.stdout:5/259: truncate d5/db/d39/f4c 4948478 0 2024-05-17T21:52:48.163 INFO:tasks.workunit.client.0.smithi107.stdout:6/231: write d9/d16/d22/d2d/f2f [1630769,54091] 0 2024-05-17T21:52:48.163 INFO:tasks.workunit.client.0.smithi107.stdout:7/214: stat d5/lb 0 2024-05-17T21:52:48.165 INFO:tasks.workunit.client.0.smithi107.stdout:5/260: truncate d5/db/d39/f43 174062 0 2024-05-17T21:52:48.166 INFO:tasks.workunit.client.0.smithi107.stdout:3/262: read d6/d8/d10/f30 [4307,48958] 0 2024-05-17T21:52:48.166 INFO:tasks.workunit.client.0.smithi107.stdout:4/250: creat d3/db/f56 x:0 0 0 2024-05-17T21:52:48.167 INFO:tasks.workunit.client.1.smithi138.stdout:8/65: creat d0/dc/f10 x:0 0 0 2024-05-17T21:52:48.167 INFO:tasks.workunit.client.0.smithi107.stdout:3/263: chown d6/d2a/d2d/d48/d4f 0 1 2024-05-17T21:52:48.168 INFO:tasks.workunit.client.0.smithi107.stdout:4/251: write d3/f2c [655992,90052] 0 2024-05-17T21:52:48.168 INFO:tasks.workunit.client.0.smithi107.stdout:1/281: getdents d4/df/d1a/d26 0 2024-05-17T21:52:48.170 INFO:tasks.workunit.client.1.smithi138.stdout:7/96: creat d0/d18/f1d x:0 0 0 2024-05-17T21:52:48.170 INFO:tasks.workunit.client.0.smithi107.stdout:7/215: dread d5/df/d11/d1c/f2e [0,4194304] 0 2024-05-17T21:52:48.171 INFO:tasks.workunit.client.0.smithi107.stdout:8/236: creat dc/d21/d35/f61 x:0 0 0 2024-05-17T21:52:48.177 INFO:tasks.workunit.client.0.smithi107.stdout:2/186: mkdir d0/d9/d1c/d37 0 2024-05-17T21:52:48.177 INFO:tasks.workunit.client.0.smithi107.stdout:2/187: chown d0/d9 119 1 2024-05-17T21:52:48.178 INFO:tasks.workunit.client.1.smithi138.stdout:2/102: sync 2024-05-17T21:52:48.178 INFO:tasks.workunit.client.1.smithi138.stdout:4/87: sync 2024-05-17T21:52:48.178 INFO:tasks.workunit.client.0.smithi107.stdout:9/280: mkdir d0/dd/d21/d29/d57/d5f 0 2024-05-17T21:52:48.182 INFO:tasks.workunit.client.1.smithi138.stdout:2/103: dread d3/f9 [0,4194304] 0 2024-05-17T21:52:48.183 INFO:tasks.workunit.client.0.smithi107.stdout:9/281: dwrite d0/fc [0,4194304] 0 2024-05-17T21:52:48.183 INFO:tasks.workunit.client.1.smithi138.stdout:4/88: dread d0/d3/f16 [0,4194304] 0 2024-05-17T21:52:48.183 INFO:tasks.workunit.client.1.smithi138.stdout:1/91: creat d4/df/f1f x:0 0 0 2024-05-17T21:52:48.184 INFO:tasks.workunit.client.1.smithi138.stdout:4/89: fsync d0/d3/db/f20 0 2024-05-17T21:52:48.184 INFO:tasks.workunit.client.1.smithi138.stdout:1/92: stat d4/df 0 2024-05-17T21:52:48.186 INFO:tasks.workunit.client.0.smithi107.stdout:9/282: dread d0/fc [0,4194304] 0 2024-05-17T21:52:48.188 INFO:tasks.workunit.client.1.smithi138.stdout:0/192: mknod d3/c4b 0 2024-05-17T21:52:48.192 INFO:tasks.workunit.client.1.smithi138.stdout:2/104: dwrite d3/f14 [0,4194304] 0 2024-05-17T21:52:48.192 INFO:tasks.workunit.client.1.smithi138.stdout:1/93: dread f3 [0,4194304] 0 2024-05-17T21:52:48.193 INFO:tasks.workunit.client.1.smithi138.stdout:8/66: mkdir d0/d11 0 2024-05-17T21:52:48.196 INFO:tasks.workunit.client.1.smithi138.stdout:7/97: creat d0/f1e x:0 0 0 2024-05-17T21:52:48.197 INFO:tasks.workunit.client.1.smithi138.stdout:7/98: chown d0/f13 66128178 1 2024-05-17T21:52:48.200 INFO:tasks.workunit.client.0.smithi107.stdout:4/252: symlink d3/d6/d26/l57 0 2024-05-17T21:52:48.205 INFO:tasks.workunit.client.0.smithi107.stdout:4/253: dwrite d3/db/d16/d43/f53 [0,4194304] 0 2024-05-17T21:52:48.206 INFO:tasks.workunit.client.0.smithi107.stdout:4/254: fdatasync d3/db/dd/d3c/f3e 0 2024-05-17T21:52:48.209 INFO:tasks.workunit.client.0.smithi107.stdout:1/282: creat d4/df/d1a/f58 x:0 0 0 2024-05-17T21:52:48.210 INFO:tasks.workunit.client.0.smithi107.stdout:1/283: stat d4/df/d1a/c52 0 2024-05-17T21:52:48.210 INFO:tasks.workunit.client.0.smithi107.stdout:1/284: truncate d4/df/f4d 1569212 0 2024-05-17T21:52:48.210 INFO:tasks.workunit.client.0.smithi107.stdout:3/264: rename d6/d2a/d2d/d37/f39 to d6/d8/d10/d25/f55 0 2024-05-17T21:52:48.211 INFO:tasks.workunit.client.0.smithi107.stdout:5/261: write d5/f17 [879924,7051] 0 2024-05-17T21:52:48.214 INFO:tasks.workunit.client.0.smithi107.stdout:0/309: link d3/d58/d61/c66 d3/dc/d22/d3d/d56/c74 0 2024-05-17T21:52:48.219 INFO:tasks.workunit.client.1.smithi138.stdout:4/90: mkdir d0/d3/d17/d21 0 2024-05-17T21:52:48.220 INFO:tasks.workunit.client.1.smithi138.stdout:5/110: truncate d1/d6/f11 921922 0 2024-05-17T21:52:48.220 INFO:tasks.workunit.client.1.smithi138.stdout:4/91: chown d0/f7 20875786 1 2024-05-17T21:52:48.221 INFO:tasks.workunit.client.1.smithi138.stdout:4/92: write d0/d1b/f18 [2367518,127703] 0 2024-05-17T21:52:48.222 INFO:tasks.workunit.client.1.smithi138.stdout:4/93: chown d0/d1b/f18 1 1 2024-05-17T21:52:48.225 INFO:tasks.workunit.client.1.smithi138.stdout:0/193: rename d3/d10/l30 to d3/d10/l4c 0 2024-05-17T21:52:48.227 INFO:tasks.workunit.client.1.smithi138.stdout:6/190: dwrite d7/fa [0,4194304] 0 2024-05-17T21:52:48.227 INFO:tasks.workunit.client.1.smithi138.stdout:2/105: unlink d3/f13 0 2024-05-17T21:52:48.229 INFO:tasks.workunit.client.1.smithi138.stdout:2/106: write d3/f14 [751737,84474] 0 2024-05-17T21:52:48.229 INFO:tasks.workunit.client.1.smithi138.stdout:6/191: write d7/db/d12/f26 [1093886,19565] 0 2024-05-17T21:52:48.234 INFO:tasks.workunit.client.1.smithi138.stdout:1/94: rename d4/dd/f11 to d4/dd/f20 0 2024-05-17T21:52:48.237 INFO:tasks.workunit.client.1.smithi138.stdout:7/99: rename d0/fc to d0/f1f 0 2024-05-17T21:52:48.239 INFO:tasks.workunit.client.1.smithi138.stdout:8/67: write d0/f2 [906517,78746] 0 2024-05-17T21:52:48.241 INFO:tasks.workunit.client.0.smithi107.stdout:4/255: rmdir d3 39 2024-05-17T21:52:48.244 INFO:tasks.workunit.client.0.smithi107.stdout:3/265: mknod d6/d8/d10/d35/d36/c56 0 2024-05-17T21:52:48.246 INFO:tasks.workunit.client.1.smithi138.stdout:9/120: creat d6/f25 x:0 0 0 2024-05-17T21:52:48.246 INFO:tasks.workunit.client.1.smithi138.stdout:8/68: dread d0/f9 [0,4194304] 0 2024-05-17T21:52:48.248 INFO:tasks.workunit.client.0.smithi107.stdout:0/310: unlink d3/dc/d22/d3d/f52 0 2024-05-17T21:52:48.249 INFO:tasks.workunit.client.0.smithi107.stdout:2/188: symlink d0/d9/d1c/d37/l38 0 2024-05-17T21:52:48.249 INFO:tasks.workunit.client.0.smithi107.stdout:2/189: readlink d0/d9/d11/d18/l23 0 2024-05-17T21:52:48.251 INFO:tasks.workunit.client.0.smithi107.stdout:6/232: rmdir d9/d16/d42 0 2024-05-17T21:52:48.254 INFO:tasks.workunit.client.0.smithi107.stdout:7/216: link d5/f22 d5/df/d11/d1c/f4e 0 2024-05-17T21:52:48.255 INFO:tasks.workunit.client.0.smithi107.stdout:7/217: truncate d5/df/f20 713477 0 2024-05-17T21:52:48.255 INFO:tasks.workunit.client.0.smithi107.stdout:5/262: getdents d5/db/dc/d18/d48 0 2024-05-17T21:52:48.257 INFO:tasks.workunit.client.0.smithi107.stdout:8/237: creat dc/f62 x:0 0 0 2024-05-17T21:52:48.258 INFO:tasks.workunit.client.0.smithi107.stdout:8/238: write dc/d21/d27/d34/f38 [463825,46698] 0 2024-05-17T21:52:48.261 INFO:tasks.workunit.client.1.smithi138.stdout:1/95: creat d4/db/f21 x:0 0 0 2024-05-17T21:52:48.264 INFO:tasks.workunit.client.1.smithi138.stdout:7/100: rename d0/f17 to d0/f20 0 2024-05-17T21:52:48.265 INFO:tasks.workunit.client.1.smithi138.stdout:7/101: fsync d0/f10 0 2024-05-17T21:52:48.272 INFO:tasks.workunit.client.1.smithi138.stdout:4/94: getdents d0/d3/d17/d21 0 2024-05-17T21:52:48.273 INFO:tasks.workunit.client.1.smithi138.stdout:3/133: dwrite d1/d3/d8/de/f27 [0,4194304] 0 2024-05-17T21:52:48.273 INFO:tasks.workunit.client.0.smithi107.stdout:4/256: stat d3/d6/d26/d36/d37/c39 0 2024-05-17T21:52:48.273 INFO:tasks.workunit.client.0.smithi107.stdout:9/283: rename d0/dd/d1b/f32 to d0/dd/f60 0 2024-05-17T21:52:48.274 INFO:tasks.workunit.client.0.smithi107.stdout:5/263: dwrite d5/f28 [0,4194304] 0 2024-05-17T21:52:48.274 INFO:tasks.workunit.client.1.smithi138.stdout:4/95: stat d0/d3/cc 0 2024-05-17T21:52:48.274 INFO:tasks.workunit.client.1.smithi138.stdout:3/134: read - d1/d3/d8/f24 zero size 2024-05-17T21:52:48.275 INFO:tasks.workunit.client.1.smithi138.stdout:3/135: chown d1/l1e 229846040 1 2024-05-17T21:52:48.276 INFO:tasks.workunit.client.1.smithi138.stdout:2/107: symlink d3/d21/l22 0 2024-05-17T21:52:48.277 INFO:tasks.workunit.client.0.smithi107.stdout:5/264: write d5/db/dc/f56 [616672,27530] 0 2024-05-17T21:52:48.279 INFO:tasks.workunit.client.0.smithi107.stdout:8/239: dwrite dc/d21/d27/d2f/f59 [0,4194304] 0 2024-05-17T21:52:48.280 INFO:tasks.workunit.client.0.smithi107.stdout:0/311: mkdir d3/d75 0 2024-05-17T21:52:48.280 INFO:tasks.workunit.client.1.smithi138.stdout:4/96: dread d0/d3/f12 [0,4194304] 0 2024-05-17T21:52:48.282 INFO:tasks.workunit.client.0.smithi107.stdout:7/218: dread d5/d16/f1e [0,4194304] 0 2024-05-17T21:52:48.283 INFO:tasks.workunit.client.1.smithi138.stdout:4/97: write d0/d3/db/f20 [745913,124309] 0 2024-05-17T21:52:48.284 INFO:tasks.workunit.client.1.smithi138.stdout:1/96: creat d4/db/f22 x:0 0 0 2024-05-17T21:52:48.286 INFO:tasks.workunit.client.0.smithi107.stdout:9/284: dwrite d0/dd/d1f/f40 [0,4194304] 0 2024-05-17T21:52:48.288 INFO:tasks.workunit.client.1.smithi138.stdout:2/108: dwrite d3/f10 [0,4194304] 0 2024-05-17T21:52:48.288 INFO:tasks.workunit.client.0.smithi107.stdout:4/257: mkdir d3/db/dd/d3c/d58 0 2024-05-17T21:52:48.288 INFO:tasks.workunit.client.1.smithi138.stdout:2/109: dread - d3/fd zero size 2024-05-17T21:52:48.289 INFO:tasks.workunit.client.0.smithi107.stdout:4/258: fsync d3/db/f1f 0 2024-05-17T21:52:48.290 INFO:tasks.workunit.client.0.smithi107.stdout:9/285: dread d0/dd/d1f/d2c/f47 [0,4194304] 0 2024-05-17T21:52:48.292 INFO:tasks.workunit.client.0.smithi107.stdout:0/312: readlink d3/dc/l1c 0 2024-05-17T21:52:48.295 INFO:tasks.workunit.client.0.smithi107.stdout:8/240: mkdir dc/d10/d63 0 2024-05-17T21:52:48.298 INFO:tasks.workunit.client.0.smithi107.stdout:7/219: fsync d5/df/f1a 0 2024-05-17T21:52:48.299 INFO:tasks.workunit.client.1.smithi138.stdout:8/69: creat d0/d11/f12 x:0 0 0 2024-05-17T21:52:48.302 INFO:tasks.workunit.client.0.smithi107.stdout:2/190: getdents d0/d9/d11/df 0 2024-05-17T21:52:48.304 INFO:tasks.workunit.client.1.smithi138.stdout:2/110: dwrite d3/f10 [0,4194304] 0 2024-05-17T21:52:48.305 INFO:tasks.workunit.client.1.smithi138.stdout:3/136: mkdir d1/d2e 0 2024-05-17T21:52:48.305 INFO:tasks.workunit.client.1.smithi138.stdout:7/102: dwrite d0/f1f [4194304,4194304] 0 2024-05-17T21:52:48.308 INFO:tasks.workunit.client.1.smithi138.stdout:2/111: write d3/f15 [1262865,58754] 0 2024-05-17T21:52:48.309 INFO:tasks.workunit.client.1.smithi138.stdout:2/112: fdatasync d3/f20 0 2024-05-17T21:52:48.312 INFO:tasks.workunit.client.1.smithi138.stdout:4/98: creat d0/d3/d17/f22 x:0 0 0 2024-05-17T21:52:48.313 INFO:tasks.workunit.client.0.smithi107.stdout:9/286: creat d0/dd/d21/d29/d3e/d45/f61 x:0 0 0 2024-05-17T21:52:48.314 INFO:tasks.workunit.client.1.smithi138.stdout:4/99: write d0/d1b/f18 [2679131,130503] 0 2024-05-17T21:52:48.318 INFO:tasks.workunit.client.1.smithi138.stdout:9/121: creat d6/d7/f26 x:0 0 0 2024-05-17T21:52:48.319 INFO:tasks.workunit.client.1.smithi138.stdout:1/97: dread d4/fc [4194304,4194304] 0 2024-05-17T21:52:48.320 INFO:tasks.workunit.client.1.smithi138.stdout:1/98: fdatasync f0 0 2024-05-17T21:52:48.322 INFO:tasks.workunit.client.1.smithi138.stdout:1/99: chown d4/df/f1f 818988 1 2024-05-17T21:52:48.322 INFO:tasks.workunit.client.1.smithi138.stdout:7/103: dread d0/f1a [0,4194304] 0 2024-05-17T21:52:48.323 INFO:tasks.workunit.client.1.smithi138.stdout:1/100: readlink d4/db/l1e 0 2024-05-17T21:52:48.324 INFO:tasks.workunit.client.1.smithi138.stdout:0/194: link d3/d10/l4c d3/d10/l4d 0 2024-05-17T21:52:48.325 INFO:tasks.workunit.client.1.smithi138.stdout:7/104: dread d0/f1a [0,4194304] 0 2024-05-17T21:52:48.330 INFO:tasks.workunit.client.1.smithi138.stdout:5/111: rename d1/l12 to d1/d6/d1b/l1e 0 2024-05-17T21:52:48.330 INFO:tasks.workunit.client.1.smithi138.stdout:9/122: dwrite d6/d7/f26 [0,4194304] 0 2024-05-17T21:52:48.330 INFO:tasks.workunit.client.0.smithi107.stdout:4/259: truncate d3/db/dd/f10 1064020 0 2024-05-17T21:52:48.331 INFO:tasks.workunit.client.1.smithi138.stdout:5/112: read d1/d6/fe [3416341,110723] 0 2024-05-17T21:52:48.332 INFO:tasks.workunit.client.1.smithi138.stdout:9/123: chown d6/d7/d1f 257768 1 2024-05-17T21:52:48.332 INFO:tasks.workunit.client.0.smithi107.stdout:5/265: rmdir d5/db/d16/d2d/d4d/d50 0 2024-05-17T21:52:48.333 INFO:tasks.workunit.client.1.smithi138.stdout:9/124: fdatasync d6/f25 0 2024-05-17T21:52:48.333 INFO:tasks.workunit.client.0.smithi107.stdout:5/266: write d5/db/d39/f4c [818223,67692] 0 2024-05-17T21:52:48.334 INFO:tasks.workunit.client.1.smithi138.stdout:3/137: write d1/d3/f11 [873504,79292] 0 2024-05-17T21:52:48.337 INFO:tasks.workunit.client.0.smithi107.stdout:0/313: rename d3/f34 to d3/dc/d31/f76 0 2024-05-17T21:52:48.337 INFO:tasks.workunit.client.0.smithi107.stdout:2/191: rename d0/d9/d11 to d0/d9/d11/df/d39 22 2024-05-17T21:52:48.339 INFO:tasks.workunit.client.0.smithi107.stdout:9/287: mkdir d0/dd/d1f/d62 0 2024-05-17T21:52:48.339 INFO:tasks.workunit.client.0.smithi107.stdout:9/288: stat d0/dd/d21/d29/d3e/d45/f61 0 2024-05-17T21:52:48.340 INFO:tasks.workunit.client.1.smithi138.stdout:3/138: dread d1/d3/d8/de/f27 [0,4194304] 0 2024-05-17T21:52:48.341 INFO:tasks.workunit.client.0.smithi107.stdout:8/241: symlink dc/d21/l64 0 2024-05-17T21:52:48.341 INFO:tasks.workunit.client.1.smithi138.stdout:6/192: getdents d7/d2d 0 2024-05-17T21:52:48.342 INFO:tasks.workunit.client.0.smithi107.stdout:8/242: write dc/d21/d27/f37 [2982575,96378] 0 2024-05-17T21:52:48.342 INFO:tasks.workunit.client.0.smithi107.stdout:7/220: getdents d5/df/d11/d2a/d4d 0 2024-05-17T21:52:48.343 INFO:tasks.workunit.client.1.smithi138.stdout:2/113: symlink d3/d21/l23 0 2024-05-17T21:52:48.343 INFO:tasks.workunit.client.0.smithi107.stdout:8/243: chown dc/d21/d27/d2f/f32 15 1 2024-05-17T21:52:48.344 INFO:tasks.workunit.client.0.smithi107.stdout:8/244: truncate dc/f4e 474882 0 2024-05-17T21:52:48.344 INFO:tasks.workunit.client.0.smithi107.stdout:4/260: creat d3/db/dd/f59 x:0 0 0 2024-05-17T21:52:48.345 INFO:tasks.workunit.client.0.smithi107.stdout:8/245: chown dc/d21/d27/d2f/f59 18 1 2024-05-17T21:52:48.345 INFO:tasks.workunit.client.0.smithi107.stdout:4/261: write d3/db/dd/d3c/f3e [529715,67509] 0 2024-05-17T21:52:48.348 INFO:tasks.workunit.client.0.smithi107.stdout:5/267: creat d5/db/dc/d18/f58 x:0 0 0 2024-05-17T21:52:48.349 INFO:tasks.workunit.client.0.smithi107.stdout:8/246: dread dc/d21/d35/f2e [0,4194304] 0 2024-05-17T21:52:48.352 INFO:tasks.workunit.client.0.smithi107.stdout:1/285: sync 2024-05-17T21:52:48.354 INFO:tasks.workunit.client.0.smithi107.stdout:2/192: dread d0/f6 [0,4194304] 0 2024-05-17T21:52:48.356 INFO:tasks.workunit.client.0.smithi107.stdout:5/268: dwrite d5/db/d16/f4b [0,4194304] 0 2024-05-17T21:52:48.357 INFO:tasks.workunit.client.1.smithi138.stdout:1/101: chown d4/dd/l1c 112498 1 2024-05-17T21:52:48.359 INFO:tasks.workunit.client.1.smithi138.stdout:0/195: symlink d3/d10/l4e 0 2024-05-17T21:52:48.363 INFO:tasks.workunit.client.0.smithi107.stdout:4/262: dread d3/db/d16/f19 [0,4194304] 0 2024-05-17T21:52:48.365 INFO:tasks.workunit.client.0.smithi107.stdout:5/269: dwrite d5/db/dc/f20 [0,4194304] 0 2024-05-17T21:52:48.365 INFO:tasks.workunit.client.1.smithi138.stdout:7/105: mknod d0/d18/c21 0 2024-05-17T21:52:48.365 INFO:tasks.workunit.client.1.smithi138.stdout:8/70: write d0/f7 [1796932,32051] 0 2024-05-17T21:52:48.366 INFO:tasks.workunit.client.0.smithi107.stdout:5/270: stat d5/db/dc/d18/d48 0 2024-05-17T21:52:48.366 INFO:tasks.workunit.client.1.smithi138.stdout:7/106: chown d0/f5 1001453821 1 2024-05-17T21:52:48.367 INFO:tasks.workunit.client.1.smithi138.stdout:8/71: read d0/dc/fd [3643272,11480] 0 2024-05-17T21:52:48.367 INFO:tasks.workunit.client.0.smithi107.stdout:5/271: write d5/db/d39/f4c [1786712,111024] 0 2024-05-17T21:52:48.367 INFO:tasks.workunit.client.0.smithi107.stdout:5/272: readlink d5/db/d16/l1d 0 2024-05-17T21:52:48.370 INFO:tasks.workunit.client.0.smithi107.stdout:0/314: mkdir d3/dc/d6b/d77 0 2024-05-17T21:52:48.374 INFO:tasks.workunit.client.0.smithi107.stdout:2/193: creat d0/d9/d11/d18/f3a x:0 0 0 2024-05-17T21:52:48.375 INFO:tasks.workunit.client.1.smithi138.stdout:3/139: readlink d1/d3/l17 0 2024-05-17T21:52:48.378 INFO:tasks.workunit.client.0.smithi107.stdout:9/289: dread d0/f4 [0,4194304] 0 2024-05-17T21:52:48.378 INFO:tasks.workunit.client.1.smithi138.stdout:6/193: mkdir d7/d2d/d3c 0 2024-05-17T21:52:48.378 INFO:tasks.workunit.client.0.smithi107.stdout:9/290: chown d0/dd/d21/d29/l2d 2981131 1 2024-05-17T21:52:48.381 INFO:tasks.workunit.client.0.smithi107.stdout:8/247: mknod dc/d10/d63/c65 0 2024-05-17T21:52:48.381 INFO:tasks.workunit.client.1.smithi138.stdout:1/102: fdatasync d4/f9 0 2024-05-17T21:52:48.382 INFO:tasks.workunit.client.1.smithi138.stdout:6/194: dread d7/fa [0,4194304] 0 2024-05-17T21:52:48.382 INFO:tasks.workunit.client.0.smithi107.stdout:8/248: chown dc/d21/d27/d34/c3d 76 1 2024-05-17T21:52:48.386 INFO:tasks.workunit.client.0.smithi107.stdout:4/263: dread d3/db/d16/f30 [0,4194304] 0 2024-05-17T21:52:48.386 INFO:tasks.workunit.client.0.smithi107.stdout:9/291: dwrite d0/dd/d2f/f30 [0,4194304] 0 2024-05-17T21:52:48.386 INFO:tasks.workunit.client.0.smithi107.stdout:7/221: write d5/df/f10 [226537,21405] 0 2024-05-17T21:52:48.386 INFO:tasks.workunit.client.1.smithi138.stdout:1/103: truncate d4/db/f21 981960 0 2024-05-17T21:52:48.386 INFO:tasks.workunit.client.1.smithi138.stdout:1/104: dread - d4/db/f22 zero size 2024-05-17T21:52:48.386 INFO:tasks.workunit.client.1.smithi138.stdout:1/105: dread - d4/dd/f15 zero size 2024-05-17T21:52:48.387 INFO:tasks.workunit.client.1.smithi138.stdout:8/72: mknod d0/d11/c13 0 2024-05-17T21:52:48.387 INFO:tasks.workunit.client.0.smithi107.stdout:9/292: chown d0/dd/d21/d29/l2d 1 1 2024-05-17T21:52:48.387 INFO:tasks.workunit.client.0.smithi107.stdout:7/222: chown d5/d16/l32 370 1 2024-05-17T21:52:48.388 INFO:tasks.workunit.client.0.smithi107.stdout:0/315: write d3/f11 [5027844,64748] 0 2024-05-17T21:52:48.388 INFO:tasks.workunit.client.1.smithi138.stdout:5/113: truncate d1/d6/f11 397345 0 2024-05-17T21:52:48.389 INFO:tasks.workunit.client.0.smithi107.stdout:0/316: chown d3/dc/d22/d3d/d56/l6a 713055 1 2024-05-17T21:52:48.390 INFO:tasks.workunit.client.0.smithi107.stdout:1/286: mkdir d4/df/d4e/d59 0 2024-05-17T21:52:48.392 INFO:tasks.workunit.client.0.smithi107.stdout:2/194: rmdir d0/d9/d11/d27 39 2024-05-17T21:52:48.398 INFO:tasks.workunit.client.0.smithi107.stdout:5/273: mkdir d5/db/d16/d54/d59 0 2024-05-17T21:52:48.398 INFO:tasks.workunit.client.1.smithi138.stdout:9/125: truncate d6/d7/d1f/f23 1317908 0 2024-05-17T21:52:48.399 INFO:tasks.workunit.client.1.smithi138.stdout:9/126: dread - d6/d7/d16/f20 zero size 2024-05-17T21:52:48.400 INFO:tasks.workunit.client.0.smithi107.stdout:4/264: creat d3/db/dd/d3a/d41/f5a x:0 0 0 2024-05-17T21:52:48.400 INFO:tasks.workunit.client.0.smithi107.stdout:4/265: stat d3/db 0 2024-05-17T21:52:48.400 INFO:tasks.workunit.client.0.smithi107.stdout:9/293: read d0/fc [1413789,112575] 0 2024-05-17T21:52:48.401 INFO:tasks.workunit.client.0.smithi107.stdout:7/223: mkdir d5/d16/d3b/d4f 0 2024-05-17T21:52:48.403 INFO:tasks.workunit.client.1.smithi138.stdout:6/195: symlink d7/d2d/l3d 0 2024-05-17T21:52:48.404 INFO:tasks.workunit.client.1.smithi138.stdout:9/127: dread d6/d7/d16/f21 [0,4194304] 0 2024-05-17T21:52:48.405 INFO:tasks.workunit.client.1.smithi138.stdout:6/196: read d7/f28 [585334,53398] 0 2024-05-17T21:52:48.405 INFO:tasks.workunit.client.1.smithi138.stdout:6/197: chown d7/db/d34 1037436 1 2024-05-17T21:52:48.409 INFO:tasks.workunit.client.0.smithi107.stdout:6/233: sync 2024-05-17T21:52:48.409 INFO:tasks.workunit.client.0.smithi107.stdout:3/266: sync 2024-05-17T21:52:48.413 INFO:tasks.workunit.client.0.smithi107.stdout:8/249: creat dc/d44/d4b/d52/f66 x:0 0 0 2024-05-17T21:52:48.414 INFO:tasks.workunit.client.1.smithi138.stdout:8/73: symlink d0/dc/l14 0 2024-05-17T21:52:48.414 INFO:tasks.workunit.client.1.smithi138.stdout:8/74: read d0/dc/fd [3749775,86651] 0 2024-05-17T21:52:48.415 INFO:tasks.workunit.client.0.smithi107.stdout:5/274: rename d5/db/d16/d2d/d40/d55 to d5/db/d16/d2d/d40/d5a 0 2024-05-17T21:52:48.416 INFO:tasks.workunit.client.1.smithi138.stdout:5/114: symlink d1/d6/d1b/l1f 0 2024-05-17T21:52:48.417 INFO:tasks.workunit.client.1.smithi138.stdout:3/140: symlink d1/d2e/l2f 0 2024-05-17T21:52:48.419 INFO:tasks.workunit.client.0.smithi107.stdout:7/224: creat d5/df/d11/d1d/d44/f50 x:0 0 0 2024-05-17T21:52:48.420 INFO:tasks.workunit.client.1.smithi138.stdout:4/100: getdents d0/d3/d10 0 2024-05-17T21:52:48.420 INFO:tasks.workunit.client.0.smithi107.stdout:5/275: dwrite d5/db/d16/f4b [0,4194304] 0 2024-05-17T21:52:48.420 INFO:tasks.workunit.client.0.smithi107.stdout:1/287: creat d4/df/d4e/d59/f5a x:0 0 0 2024-05-17T21:52:48.422 INFO:tasks.workunit.client.0.smithi107.stdout:5/276: truncate d5/db/dc/f56 1688084 0 2024-05-17T21:52:48.422 INFO:tasks.workunit.client.1.smithi138.stdout:1/106: dwrite d4/fc [0,4194304] 0 2024-05-17T21:52:48.424 INFO:tasks.workunit.client.0.smithi107.stdout:3/267: mkdir d6/d1a/d57 0 2024-05-17T21:52:48.425 INFO:tasks.workunit.client.1.smithi138.stdout:0/196: rename d3/c4b to d3/c4f 0 2024-05-17T21:52:48.427 INFO:tasks.workunit.client.1.smithi138.stdout:0/197: write d3/d10/f36 [344377,59086] 0 2024-05-17T21:52:48.427 INFO:tasks.workunit.client.0.smithi107.stdout:3/268: truncate d6/d8/d10/f38 873487 0 2024-05-17T21:52:48.427 INFO:tasks.workunit.client.0.smithi107.stdout:6/234: write d9/d16/f3a [882153,12624] 0 2024-05-17T21:52:48.429 INFO:tasks.workunit.client.1.smithi138.stdout:1/107: dwrite d4/f9 [0,4194304] 0 2024-05-17T21:52:48.431 INFO:tasks.workunit.client.0.smithi107.stdout:1/288: dwrite d4/f8 [0,4194304] 0 2024-05-17T21:52:48.431 INFO:tasks.workunit.client.1.smithi138.stdout:9/128: read d6/d7/f9 [97144,61301] 0 2024-05-17T21:52:48.432 INFO:tasks.workunit.client.1.smithi138.stdout:9/129: stat d6/d7/d16/l1d 0 2024-05-17T21:52:48.436 INFO:tasks.workunit.client.0.smithi107.stdout:8/250: rmdir dc 39 2024-05-17T21:52:48.438 INFO:tasks.workunit.client.0.smithi107.stdout:6/235: dread f5 [0,4194304] 0 2024-05-17T21:52:48.438 INFO:tasks.workunit.client.1.smithi138.stdout:6/198: creat d7/db/d12/f3e x:0 0 0 2024-05-17T21:52:48.440 INFO:tasks.workunit.client.0.smithi107.stdout:1/289: dread d4/df/d14/d1d/f2a [0,4194304] 0 2024-05-17T21:52:48.442 INFO:tasks.workunit.client.0.smithi107.stdout:6/236: fsync d9/d16/f3a 0 2024-05-17T21:52:48.443 INFO:tasks.workunit.client.1.smithi138.stdout:8/75: symlink d0/d11/l15 0 2024-05-17T21:52:48.444 INFO:tasks.workunit.client.0.smithi107.stdout:4/266: truncate d3/db/d16/f45 640463 0 2024-05-17T21:52:48.446 INFO:tasks.workunit.client.1.smithi138.stdout:6/199: dwrite d7/d2d/f35 [0,4194304] 0 2024-05-17T21:52:48.448 INFO:tasks.workunit.client.0.smithi107.stdout:6/237: dwrite d9/d16/f18 [0,4194304] 0 2024-05-17T21:52:48.449 INFO:tasks.workunit.client.1.smithi138.stdout:2/114: getdents d3/d21 0 2024-05-17T21:52:48.450 INFO:tasks.workunit.client.0.smithi107.stdout:9/294: truncate d0/dd/f35 1027706 0 2024-05-17T21:52:48.451 INFO:tasks.workunit.client.1.smithi138.stdout:8/76: dwrite d0/f7 [0,4194304] 0 2024-05-17T21:52:48.451 INFO:tasks.workunit.client.1.smithi138.stdout:2/115: truncate d3/fd 227390 0 2024-05-17T21:52:48.453 INFO:tasks.workunit.client.1.smithi138.stdout:4/101: symlink d0/d3/d10/l23 0 2024-05-17T21:52:48.457 INFO:tasks.workunit.client.0.smithi107.stdout:5/277: rename d5/db/d16/d2a/d2b to d5/db/d16/d5b 0 2024-05-17T21:52:48.459 INFO:tasks.workunit.client.0.smithi107.stdout:5/278: write d5/db/d16/d5b/f3f [1844202,96509] 0 2024-05-17T21:52:48.459 INFO:tasks.workunit.client.0.smithi107.stdout:3/269: unlink d6/d8/d10/d35/d36/c56 0 2024-05-17T21:52:48.461 INFO:tasks.workunit.client.0.smithi107.stdout:5/279: write d5/db/d16/d5b/f53 [376709,41617] 0 2024-05-17T21:52:48.461 INFO:tasks.workunit.client.0.smithi107.stdout:5/280: readlink d5/db/l36 0 2024-05-17T21:52:48.471 INFO:tasks.workunit.client.1.smithi138.stdout:9/130: symlink d6/d7/d16/l27 0 2024-05-17T21:52:48.473 INFO:tasks.workunit.client.1.smithi138.stdout:5/115: creat d1/d6/d19/f20 x:0 0 0 2024-05-17T21:52:48.477 INFO:tasks.workunit.client.1.smithi138.stdout:3/141: rename d1/d3/f23 to d1/d3/d16/f30 0 2024-05-17T21:52:48.477 INFO:tasks.workunit.client.0.smithi107.stdout:8/251: dwrite dc/d21/d35/f2d [0,4194304] 0 2024-05-17T21:52:48.478 INFO:tasks.workunit.client.1.smithi138.stdout:3/142: write d1/d3/f11 [1316270,18345] 0 2024-05-17T21:52:48.484 INFO:tasks.workunit.client.0.smithi107.stdout:4/267: rename d3/db/dd/f59 to d3/db/dd/f5b 0 2024-05-17T21:52:48.487 INFO:tasks.workunit.client.1.smithi138.stdout:8/77: symlink d0/l16 0 2024-05-17T21:52:48.487 INFO:tasks.workunit.client.0.smithi107.stdout:0/317: link d3/l4b d3/dc/d6b/d77/l78 0 2024-05-17T21:52:48.488 INFO:tasks.workunit.client.1.smithi138.stdout:2/116: dwrite d3/fe [0,4194304] 0 2024-05-17T21:52:48.490 INFO:tasks.workunit.client.1.smithi138.stdout:4/102: mkdir d0/d3/db/d24 0 2024-05-17T21:52:48.491 INFO:tasks.workunit.client.0.smithi107.stdout:0/318: dread d3/dc/d31/f57 [0,4194304] 0 2024-05-17T21:52:48.491 INFO:tasks.workunit.client.0.smithi107.stdout:7/225: mkdir d5/df/d11/d1c/d31/d39/d51 0 2024-05-17T21:52:48.494 INFO:tasks.workunit.client.1.smithi138.stdout:1/108: truncate d4/dd/f13 1089371 0 2024-05-17T21:52:48.498 INFO:tasks.workunit.client.0.smithi107.stdout:9/295: rmdir d0 39 2024-05-17T21:52:48.499 INFO:tasks.workunit.client.1.smithi138.stdout:0/198: dread d3/fc [0,4194304] 0 2024-05-17T21:52:48.499 INFO:tasks.workunit.client.1.smithi138.stdout:9/131: symlink d6/d7/d16/l28 0 2024-05-17T21:52:48.500 INFO:tasks.workunit.client.1.smithi138.stdout:9/132: chown d6/d7/d1f 0 1 2024-05-17T21:52:48.502 INFO:tasks.workunit.client.0.smithi107.stdout:6/238: rename d9/d16/f30 to d9/d16/d22/d2d/f44 0 2024-05-17T21:52:48.503 INFO:tasks.workunit.client.1.smithi138.stdout:5/116: creat d1/d6/d1b/f21 x:0 0 0 2024-05-17T21:52:48.507 INFO:tasks.workunit.client.1.smithi138.stdout:9/133: dwrite d6/d7/d16/f20 [0,4194304] 0 2024-05-17T21:52:48.508 INFO:tasks.workunit.client.0.smithi107.stdout:2/195: link d0/d9/d11/l1d d0/d9/d11/l3b 0 2024-05-17T21:52:48.510 INFO:tasks.workunit.client.1.smithi138.stdout:8/78: fdatasync d0/f9 0 2024-05-17T21:52:48.514 INFO:tasks.workunit.client.1.smithi138.stdout:2/117: symlink d3/l24 0 2024-05-17T21:52:48.514 INFO:tasks.workunit.client.1.smithi138.stdout:2/118: chown d3/fe 4 1 2024-05-17T21:52:48.517 INFO:tasks.workunit.client.1.smithi138.stdout:1/109: fsync d4/f16 0 2024-05-17T21:52:48.519 INFO:tasks.workunit.client.0.smithi107.stdout:8/252: rmdir dc/d21/d27/d34/d4c 39 2024-05-17T21:52:48.520 INFO:tasks.workunit.client.0.smithi107.stdout:8/253: write dc/f4f [1911883,28425] 0 2024-05-17T21:52:48.521 INFO:tasks.workunit.client.1.smithi138.stdout:0/199: symlink d3/d3d/d48/l50 0 2024-05-17T21:52:48.521 INFO:tasks.workunit.client.0.smithi107.stdout:8/254: readlink dc/d21/l64 0 2024-05-17T21:52:48.524 INFO:tasks.workunit.client.0.smithi107.stdout:7/226: dread d5/df/d11/f25 [0,4194304] 0 2024-05-17T21:52:48.525 INFO:tasks.workunit.client.1.smithi138.stdout:3/143: mknod d1/d3/c31 0 2024-05-17T21:52:48.526 INFO:tasks.workunit.client.0.smithi107.stdout:5/281: dwrite d5/db/dc/d18/f27 [0,4194304] 0 2024-05-17T21:52:48.527 INFO:tasks.workunit.client.1.smithi138.stdout:9/134: mkdir d6/d7/d16/d29 0 2024-05-17T21:52:48.529 INFO:tasks.workunit.client.1.smithi138.stdout:9/135: stat d6/d7/d16/l28 0 2024-05-17T21:52:48.529 INFO:tasks.workunit.client.1.smithi138.stdout:8/79: fdatasync d0/ff 0 2024-05-17T21:52:48.530 INFO:tasks.workunit.client.0.smithi107.stdout:0/319: symlink d3/dc/d6b/l79 0 2024-05-17T21:52:48.530 INFO:tasks.workunit.client.0.smithi107.stdout:0/320: readlink d3/dc/l36 0 2024-05-17T21:52:48.531 INFO:tasks.workunit.client.1.smithi138.stdout:4/103: dwrite d0/d3/f12 [0,4194304] 0 2024-05-17T21:52:48.533 INFO:tasks.workunit.client.1.smithi138.stdout:2/119: mknod d3/c25 0 2024-05-17T21:52:48.535 INFO:tasks.workunit.client.0.smithi107.stdout:5/282: dwrite d5/db/d16/d2d/f4f [0,4194304] 0 2024-05-17T21:52:48.535 INFO:tasks.workunit.client.0.smithi107.stdout:9/296: readlink d0/dd/d1b/l4b 0 2024-05-17T21:52:48.538 INFO:tasks.workunit.client.0.smithi107.stdout:4/268: dwrite d3/db/dd/f12 [0,4194304] 0 2024-05-17T21:52:48.541 INFO:tasks.workunit.client.1.smithi138.stdout:2/120: dwrite d3/fd [0,4194304] 0 2024-05-17T21:52:48.541 INFO:tasks.workunit.client.0.smithi107.stdout:3/270: symlink d6/d2a/d2d/d48/d4f/l58 0 2024-05-17T21:52:48.542 INFO:tasks.workunit.client.1.smithi138.stdout:0/200: mkdir d3/d3d/d51 0 2024-05-17T21:52:48.543 INFO:tasks.workunit.client.0.smithi107.stdout:2/196: symlink d0/d9/d1c/l3c 0 2024-05-17T21:52:48.545 INFO:tasks.workunit.client.1.smithi138.stdout:3/144: read d1/d3/f10 [2258749,59945] 0 2024-05-17T21:52:48.545 INFO:tasks.workunit.client.0.smithi107.stdout:2/197: write d0/d9/d11/d18/f20 [518501,44741] 0 2024-05-17T21:52:48.546 INFO:tasks.workunit.client.1.smithi138.stdout:3/145: chown d1/d3/d8/de/f27 1054752349 1 2024-05-17T21:52:48.548 INFO:tasks.workunit.client.1.smithi138.stdout:8/80: rename d0/f7 to d0/f17 0 2024-05-17T21:52:48.553 INFO:tasks.workunit.client.1.smithi138.stdout:2/121: symlink d3/l26 0 2024-05-17T21:52:48.554 INFO:tasks.workunit.client.1.smithi138.stdout:2/122: dread - d3/f19 zero size 2024-05-17T21:52:48.557 INFO:tasks.workunit.client.1.smithi138.stdout:0/201: unlink d3/d10/d38/f40 0 2024-05-17T21:52:48.558 INFO:tasks.workunit.client.1.smithi138.stdout:0/202: write d3/d10/f36 [952079,31660] 0 2024-05-17T21:52:48.559 INFO:tasks.workunit.client.1.smithi138.stdout:0/203: chown l1 8284 1 2024-05-17T21:52:48.560 INFO:tasks.workunit.client.1.smithi138.stdout:2/123: dread d3/f15 [0,4194304] 0 2024-05-17T21:52:48.561 INFO:tasks.workunit.client.1.smithi138.stdout:5/117: creat d1/f22 x:0 0 0 2024-05-17T21:52:48.565 INFO:tasks.workunit.client.1.smithi138.stdout:3/146: symlink d1/d3/d8/de/l32 0 2024-05-17T21:52:48.566 INFO:tasks.workunit.client.1.smithi138.stdout:3/147: readlink d1/d2e/l2f 0 2024-05-17T21:52:48.566 INFO:tasks.workunit.client.1.smithi138.stdout:0/204: dwrite d3/d10/f11 [0,4194304] 0 2024-05-17T21:52:48.567 INFO:tasks.workunit.client.1.smithi138.stdout:3/148: readlink d1/d3/d8/de/l2a 0 2024-05-17T21:52:48.567 INFO:tasks.workunit.client.0.smithi107.stdout:7/227: mknod d5/df/d11/d1c/d31/c52 0 2024-05-17T21:52:48.568 INFO:tasks.workunit.client.0.smithi107.stdout:2/198: dread - d0/d9/f33 zero size 2024-05-17T21:52:48.568 INFO:tasks.workunit.client.0.smithi107.stdout:3/271: symlink d6/d2a/d2d/l59 0 2024-05-17T21:52:48.569 INFO:tasks.workunit.client.1.smithi138.stdout:2/124: rmdir d3/d21 39 2024-05-17T21:52:48.569 INFO:tasks.workunit.client.0.smithi107.stdout:1/290: getdents d4/df/d1a 0 2024-05-17T21:52:48.570 INFO:tasks.workunit.client.1.smithi138.stdout:2/125: chown d3/l24 1040 1 2024-05-17T21:52:48.570 INFO:tasks.workunit.client.1.smithi138.stdout:5/118: fdatasync d1/f13 0 2024-05-17T21:52:48.571 INFO:tasks.workunit.client.0.smithi107.stdout:4/269: dwrite d3/db/d16/f44 [0,4194304] 0 2024-05-17T21:52:48.571 INFO:tasks.workunit.client.1.smithi138.stdout:5/119: stat d1/d6/fe 0 2024-05-17T21:52:48.571 INFO:tasks.workunit.client.1.smithi138.stdout:0/205: dread d3/d10/f32 [0,4194304] 0 2024-05-17T21:52:48.573 INFO:tasks.workunit.client.0.smithi107.stdout:8/255: creat dc/f67 x:0 0 0 2024-05-17T21:52:48.574 INFO:tasks.workunit.client.1.smithi138.stdout:9/136: link d6/d7/d16/f1a d6/f2a 0 2024-05-17T21:52:48.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:48 smithi138 ceph-mon[35948]: pgmap v32: 65 pgs: 65 active+clean; 624 MiB data, 3.4 GiB used, 533 GiB / 536 GiB avail; 5.1 MiB/s rd, 61 MiB/s wr, 353 op/s 2024-05-17T21:52:48.576 INFO:tasks.workunit.client.1.smithi138.stdout:1/110: getdents d4/dd 0 2024-05-17T21:52:48.576 INFO:tasks.workunit.client.0.smithi107.stdout:7/228: dread d5/df/f26 [0,4194304] 0 2024-05-17T21:52:48.576 INFO:tasks.workunit.client.1.smithi138.stdout:1/111: chown d4/dd/f20 15 1 2024-05-17T21:52:48.578 INFO:tasks.workunit.client.0.smithi107.stdout:0/321: rename d3/dc/l1b to d3/dc/l7a 0 2024-05-17T21:52:48.580 INFO:tasks.workunit.client.0.smithi107.stdout:7/229: chown d5/df/d11/f25 2 1 2024-05-17T21:52:48.580 INFO:tasks.workunit.client.1.smithi138.stdout:3/149: dread - d1/d3/f20 zero size 2024-05-17T21:52:48.581 INFO:tasks.workunit.client.0.smithi107.stdout:4/270: creat d3/d6/f5c x:0 0 0 2024-05-17T21:52:48.583 INFO:tasks.workunit.client.0.smithi107.stdout:3/272: mknod d6/c5a 0 2024-05-17T21:52:48.583 INFO:tasks.workunit.client.1.smithi138.stdout:2/126: chown d3/d21/l22 1515181761 1 2024-05-17T21:52:48.584 INFO:tasks.workunit.client.1.smithi138.stdout:2/127: write d3/f1c [1556262,35433] 0 2024-05-17T21:52:48.585 INFO:tasks.workunit.client.0.smithi107.stdout:6/239: dwrite d9/f1a [0,4194304] 0 2024-05-17T21:52:48.587 INFO:tasks.workunit.client.0.smithi107.stdout:4/271: write d3/d6/d26/f31 [189029,104244] 0 2024-05-17T21:52:48.587 INFO:tasks.workunit.client.1.smithi138.stdout:0/206: symlink d3/d10/d38/l52 0 2024-05-17T21:52:48.589 INFO:tasks.workunit.client.0.smithi107.stdout:1/291: symlink d4/df/d14/d2e/d43/d47/l5b 0 2024-05-17T21:52:48.590 INFO:tasks.workunit.client.1.smithi138.stdout:0/207: truncate d3/d10/d38/d3e/f3f 854840 0 2024-05-17T21:52:48.590 INFO:tasks.workunit.client.0.smithi107.stdout:4/272: stat d3/d4a/d47/f4b 0 2024-05-17T21:52:48.590 INFO:tasks.workunit.client.0.smithi107.stdout:8/256: dwrite dc/f62 [0,4194304] 0 2024-05-17T21:52:48.594 INFO:tasks.workunit.client.1.smithi138.stdout:4/104: getdents d0/d3/db 0 2024-05-17T21:52:48.596 INFO:tasks.workunit.client.0.smithi107.stdout:0/322: dread d3/dc/d31/f3a [0,4194304] 0 2024-05-17T21:52:48.596 INFO:tasks.workunit.client.0.smithi107.stdout:7/230: dwrite d5/df/f20 [0,4194304] 0 2024-05-17T21:52:48.597 INFO:tasks.workunit.client.0.smithi107.stdout:0/323: chown d3/dc/l4e 46 1 2024-05-17T21:52:48.600 INFO:tasks.workunit.client.0.smithi107.stdout:3/273: unlink d6/d8/l28 0 2024-05-17T21:52:48.602 INFO:tasks.workunit.client.1.smithi138.stdout:3/150: mknod d1/d2e/c33 0 2024-05-17T21:52:48.605 INFO:tasks.workunit.client.0.smithi107.stdout:6/240: dread d9/d16/f1e [0,4194304] 0 2024-05-17T21:52:48.607 INFO:tasks.workunit.client.0.smithi107.stdout:3/274: dwrite d6/d2a/f42 [0,4194304] 0 2024-05-17T21:52:48.609 INFO:tasks.workunit.client.0.smithi107.stdout:4/273: creat d3/db/dd/d3a/d41/f5d x:0 0 0 2024-05-17T21:52:48.612 INFO:tasks.workunit.client.1.smithi138.stdout:0/208: creat d3/d3d/f53 x:0 0 0 2024-05-17T21:52:48.614 INFO:tasks.workunit.client.0.smithi107.stdout:7/231: mkdir d5/df/d11/d1c/d2f/d53 0 2024-05-17T21:52:48.617 INFO:tasks.workunit.client.1.smithi138.stdout:0/209: dread d3/d10/f36 [0,4194304] 0 2024-05-17T21:52:48.618 INFO:tasks.workunit.client.0.smithi107.stdout:0/324: write d3/d39/f49 [88408,89593] 0 2024-05-17T21:52:48.621 INFO:tasks.workunit.client.1.smithi138.stdout:4/105: creat d0/d3/d10/f25 x:0 0 0 2024-05-17T21:52:48.621 INFO:tasks.workunit.client.0.smithi107.stdout:2/199: write d0/f6 [3069659,52708] 0 2024-05-17T21:52:48.622 INFO:tasks.workunit.client.0.smithi107.stdout:8/257: chown dc/d21/d27/d34/d4c/l5a 2312 1 2024-05-17T21:52:48.624 INFO:tasks.workunit.client.1.smithi138.stdout:0/210: dwrite d3/d10/f20 [4194304,4194304] 0 2024-05-17T21:52:48.624 INFO:tasks.workunit.client.0.smithi107.stdout:6/241: symlink d9/d16/d22/l45 0 2024-05-17T21:52:48.625 INFO:tasks.workunit.client.0.smithi107.stdout:6/242: write d9/d16/f18 [4176478,73111] 0 2024-05-17T21:52:48.626 INFO:tasks.workunit.client.0.smithi107.stdout:3/275: creat d6/d2a/f5b x:0 0 0 2024-05-17T21:52:48.630 INFO:tasks.workunit.client.0.smithi107.stdout:4/274: mkdir d3/db/d35/d5e 0 2024-05-17T21:52:48.631 INFO:tasks.workunit.client.0.smithi107.stdout:4/275: truncate d3/db/dd/d3c/f3e 1049478 0 2024-05-17T21:52:48.632 INFO:tasks.workunit.client.1.smithi138.stdout:9/137: creat d6/f2b x:0 0 0 2024-05-17T21:52:48.634 INFO:tasks.workunit.client.0.smithi107.stdout:7/232: unlink d5/df/d11/c13 0 2024-05-17T21:52:48.637 INFO:tasks.workunit.client.0.smithi107.stdout:5/283: sync 2024-05-17T21:52:48.637 INFO:tasks.workunit.client.0.smithi107.stdout:2/200: creat d0/d9/d1c/f3d x:0 0 0 2024-05-17T21:52:48.638 INFO:tasks.workunit.client.0.smithi107.stdout:1/292: write d4/df/d14/d1d/f27 [1099281,124731] 0 2024-05-17T21:52:48.638 INFO:tasks.workunit.client.0.smithi107.stdout:5/284: stat d5/db/d16 0 2024-05-17T21:52:48.639 INFO:tasks.workunit.client.0.smithi107.stdout:2/201: dread - d0/d9/d11/d18/f3a zero size 2024-05-17T21:52:48.641 INFO:tasks.workunit.client.1.smithi138.stdout:5/120: dwrite d1/d6/f11 [0,4194304] 0 2024-05-17T21:52:48.642 INFO:tasks.workunit.client.1.smithi138.stdout:0/211: unlink d3/f43 0 2024-05-17T21:52:48.642 INFO:tasks.workunit.client.1.smithi138.stdout:4/106: rmdir d0/d3/d10 39 2024-05-17T21:52:48.644 INFO:tasks.workunit.client.1.smithi138.stdout:2/128: creat d3/f27 x:0 0 0 2024-05-17T21:52:48.645 INFO:tasks.workunit.client.1.smithi138.stdout:1/112: truncate d4/fa 5435290 0 2024-05-17T21:52:48.646 INFO:tasks.workunit.client.1.smithi138.stdout:2/129: write d3/fc [816090,9024] 0 2024-05-17T21:52:48.647 INFO:tasks.workunit.client.1.smithi138.stdout:2/130: dread - d3/f27 zero size 2024-05-17T21:52:48.648 INFO:tasks.workunit.client.1.smithi138.stdout:2/131: write d3/f20 [475917,63303] 0 2024-05-17T21:52:48.650 INFO:tasks.workunit.client.0.smithi107.stdout:3/276: read d6/d8/d10/f30 [3574772,96450] 0 2024-05-17T21:52:48.651 INFO:tasks.workunit.client.0.smithi107.stdout:4/276: creat d3/d6/d26/d36/d37/f5f x:0 0 0 2024-05-17T21:52:48.651 INFO:tasks.workunit.client.0.smithi107.stdout:3/277: dread - d6/d2a/f4d zero size 2024-05-17T21:52:48.655 INFO:tasks.workunit.client.0.smithi107.stdout:8/258: mkdir dc/d21/d68 0 2024-05-17T21:52:48.657 INFO:tasks.workunit.client.0.smithi107.stdout:1/293: mkdir d4/df/d1a/d26/d50/d5c 0 2024-05-17T21:52:48.659 INFO:tasks.workunit.client.0.smithi107.stdout:7/233: dread f4 [0,4194304] 0 2024-05-17T21:52:48.664 INFO:tasks.workunit.client.1.smithi138.stdout:1/113: symlink d4/df/l23 0 2024-05-17T21:52:48.664 INFO:tasks.workunit.client.1.smithi138.stdout:1/114: chown d4/db/l1e 425819 1 2024-05-17T21:52:48.665 INFO:tasks.workunit.client.1.smithi138.stdout:1/115: write d4/df/f1b [81958,36411] 0 2024-05-17T21:52:48.665 INFO:tasks.workunit.client.0.smithi107.stdout:7/234: dwrite d5/d16/f2b [4194304,4194304] 0 2024-05-17T21:52:48.666 INFO:tasks.workunit.client.1.smithi138.stdout:1/116: chown d4 124430 1 2024-05-17T21:52:48.666 INFO:tasks.workunit.client.0.smithi107.stdout:3/278: mknod d6/d2a/d2d/d48/c5c 0 2024-05-17T21:52:48.666 INFO:tasks.workunit.client.0.smithi107.stdout:7/235: fdatasync d5/df/f10 0 2024-05-17T21:52:48.667 INFO:tasks.workunit.client.1.smithi138.stdout:0/212: write d3/d10/f1a [1117104,105055] 0 2024-05-17T21:52:48.668 INFO:tasks.workunit.client.1.smithi138.stdout:3/151: getdents d1/d3/d8 0 2024-05-17T21:52:48.669 INFO:tasks.workunit.client.1.smithi138.stdout:0/213: chown d3/d10/f1a 291925883 1 2024-05-17T21:52:48.679 INFO:tasks.workunit.client.0.smithi107.stdout:0/325: rename d3/dc/f12 to d3/f7b 0 2024-05-17T21:52:48.681 INFO:tasks.workunit.client.0.smithi107.stdout:6/243: truncate d9/d16/d22/d2d/f44 183020 0 2024-05-17T21:52:48.682 INFO:tasks.workunit.client.0.smithi107.stdout:0/326: truncate d3/dc/d22/d3d/f4d 309410 0 2024-05-17T21:52:48.684 INFO:tasks.workunit.client.0.smithi107.stdout:6/244: write d9/d16/f3e [813308,92938] 0 2024-05-17T21:52:48.685 INFO:tasks.workunit.client.0.smithi107.stdout:2/202: dwrite d0/d9/d1c/d22/f31 [0,4194304] 0 2024-05-17T21:52:48.686 INFO:tasks.workunit.client.1.smithi138.stdout:2/132: symlink d3/l28 0 2024-05-17T21:52:48.686 INFO:tasks.workunit.client.0.smithi107.stdout:2/203: write d0/f36 [319428,64069] 0 2024-05-17T21:52:48.687 INFO:tasks.workunit.client.1.smithi138.stdout:1/117: mknod d4/dd/c24 0 2024-05-17T21:52:48.689 INFO:tasks.workunit.client.0.smithi107.stdout:5/285: truncate d5/db/d39/f3e 1428115 0 2024-05-17T21:52:48.690 INFO:tasks.workunit.client.1.smithi138.stdout:9/138: dwrite d6/d7/d1f/f23 [0,4194304] 0 2024-05-17T21:52:48.691 INFO:tasks.workunit.client.1.smithi138.stdout:9/139: stat d6/d7/d16/c19 0 2024-05-17T21:52:48.692 INFO:tasks.workunit.client.0.smithi107.stdout:7/236: write d5/f22 [2323747,84451] 0 2024-05-17T21:52:48.695 INFO:tasks.workunit.client.0.smithi107.stdout:2/204: dwrite d0/fa [4194304,4194304] 0 2024-05-17T21:52:48.697 INFO:tasks.workunit.client.0.smithi107.stdout:7/237: fsync d5/df/d11/d1c/f4e 0 2024-05-17T21:52:48.698 INFO:tasks.workunit.client.0.smithi107.stdout:8/259: mkdir dc/d21/d69 0 2024-05-17T21:52:48.701 INFO:tasks.workunit.client.0.smithi107.stdout:9/297: sync 2024-05-17T21:52:48.705 INFO:tasks.workunit.client.0.smithi107.stdout:4/277: rename d3/db/dd to d3/d4a/d47/d60 0 2024-05-17T21:52:48.709 INFO:tasks.workunit.client.0.smithi107.stdout:6/245: unlink d9/d16/f18 0 2024-05-17T21:52:48.712 INFO:tasks.workunit.client.0.smithi107.stdout:0/327: creat d3/dc/d22/d23/d6c/f7c x:0 0 0 2024-05-17T21:52:48.715 INFO:tasks.workunit.client.0.smithi107.stdout:3/279: mknod d6/d8/c5d 0 2024-05-17T21:52:48.716 INFO:tasks.workunit.client.0.smithi107.stdout:2/205: mkdir d0/d9/d1c/d3e 0 2024-05-17T21:52:48.717 INFO:tasks.workunit.client.0.smithi107.stdout:7/238: unlink d5/df/l3d 0 2024-05-17T21:52:48.721 INFO:tasks.workunit.client.0.smithi107.stdout:1/294: rename d4/de/l3e to d4/df/d4e/l5d 0 2024-05-17T21:52:48.723 INFO:tasks.workunit.client.0.smithi107.stdout:1/295: dread - d4/df/d14/d2e/d43/f51 zero size 2024-05-17T21:52:48.723 INFO:tasks.workunit.client.0.smithi107.stdout:0/328: creat d3/dc/d22/f7d x:0 0 0 2024-05-17T21:52:48.724 INFO:tasks.workunit.client.0.smithi107.stdout:7/239: dwrite d5/df/d11/d1d/d44/f50 [0,4194304] 0 2024-05-17T21:52:48.725 INFO:tasks.workunit.client.0.smithi107.stdout:1/296: dread - d4/df/d14/d2e/d43/f51 zero size 2024-05-17T21:52:48.726 INFO:tasks.workunit.client.0.smithi107.stdout:2/206: truncate d0/d9/d11/f14 9638652 0 2024-05-17T21:52:48.727 INFO:tasks.workunit.client.0.smithi107.stdout:1/297: write d4/f10 [2706794,47582] 0 2024-05-17T21:52:48.729 INFO:tasks.workunit.client.0.smithi107.stdout:9/298: symlink d0/dd/d2f/l63 0 2024-05-17T21:52:48.729 INFO:tasks.workunit.client.0.smithi107.stdout:0/329: dread d3/dc/f16 [0,4194304] 0 2024-05-17T21:52:48.731 INFO:tasks.workunit.client.0.smithi107.stdout:9/299: truncate d0/dd/d1f/f52 462260 0 2024-05-17T21:52:48.732 INFO:tasks.workunit.client.0.smithi107.stdout:6/246: dread d9/f1d [0,4194304] 0 2024-05-17T21:52:48.732 INFO:tasks.workunit.client.0.smithi107.stdout:4/278: rename d3/d4a/d47/d60/d3a/d41/f5d to d3/db/d16/f61 0 2024-05-17T21:52:48.733 INFO:tasks.workunit.client.0.smithi107.stdout:9/300: dread d0/f43 [0,4194304] 0 2024-05-17T21:52:48.736 INFO:tasks.workunit.client.0.smithi107.stdout:5/286: creat d5/db/f5c x:0 0 0 2024-05-17T21:52:48.738 INFO:tasks.workunit.client.0.smithi107.stdout:3/280: link d6/d2a/d2d/f43 d6/d1a/d57/f5e 0 2024-05-17T21:52:48.739 INFO:tasks.workunit.client.0.smithi107.stdout:8/260: link dc/c1a dc/d10/d63/c6a 0 2024-05-17T21:52:48.739 INFO:tasks.workunit.client.0.smithi107.stdout:1/298: rmdir d4/df/d14/d2e/d43/d47 39 2024-05-17T21:52:48.741 INFO:tasks.workunit.client.0.smithi107.stdout:0/330: symlink d3/d39/l7e 0 2024-05-17T21:52:48.743 INFO:tasks.workunit.client.0.smithi107.stdout:4/279: creat d3/db/d16/d43/f62 x:0 0 0 2024-05-17T21:52:48.743 INFO:tasks.workunit.client.0.smithi107.stdout:4/280: chown d3/db/d35/d5e 13 1 2024-05-17T21:52:48.744 INFO:tasks.workunit.client.0.smithi107.stdout:4/281: chown d3/d6/d26/f2f 390 1 2024-05-17T21:52:48.748 INFO:tasks.workunit.client.0.smithi107.stdout:2/207: creat d0/d9/d1c/d3e/f3f x:0 0 0 2024-05-17T21:52:48.749 INFO:tasks.workunit.client.0.smithi107.stdout:8/261: symlink dc/d44/d4b/l6b 0 2024-05-17T21:52:48.752 INFO:tasks.workunit.client.0.smithi107.stdout:6/247: mknod d9/d16/c46 0 2024-05-17T21:52:48.753 INFO:tasks.workunit.client.0.smithi107.stdout:0/331: creat d3/d39/f7f x:0 0 0 2024-05-17T21:52:48.754 INFO:tasks.workunit.client.0.smithi107.stdout:1/299: dread d4/f21 [0,4194304] 0 2024-05-17T21:52:48.756 INFO:tasks.workunit.client.0.smithi107.stdout:7/240: creat d5/df/d11/d1c/f54 x:0 0 0 2024-05-17T21:52:48.758 INFO:tasks.workunit.client.0.smithi107.stdout:8/262: symlink dc/d21/d27/d34/d4c/l6c 0 2024-05-17T21:52:48.759 INFO:tasks.workunit.client.0.smithi107.stdout:3/281: write d6/d8/f17 [475375,119755] 0 2024-05-17T21:52:48.760 INFO:tasks.workunit.client.0.smithi107.stdout:6/248: mknod d9/d16/d22/c47 0 2024-05-17T21:52:48.760 INFO:tasks.workunit.client.0.smithi107.stdout:0/332: dwrite d3/dc/d22/f7d [0,4194304] 0 2024-05-17T21:52:48.761 INFO:tasks.workunit.client.0.smithi107.stdout:8/263: fdatasync dc/d21/d27/d2f/f2c 0 2024-05-17T21:52:48.761 INFO:tasks.workunit.client.0.smithi107.stdout:4/282: dread d3/db/fc [0,4194304] 0 2024-05-17T21:52:48.763 INFO:tasks.workunit.client.0.smithi107.stdout:9/301: rmdir d0/dd/d1f/d62 0 2024-05-17T21:52:48.766 INFO:tasks.workunit.client.0.smithi107.stdout:8/264: dread - dc/d21/d27/d34/d4c/f50 zero size 2024-05-17T21:52:48.767 INFO:tasks.workunit.client.0.smithi107.stdout:5/287: truncate d5/db/d16/d2d/f4a 2334154 0 2024-05-17T21:52:48.768 INFO:tasks.workunit.client.0.smithi107.stdout:1/300: symlink d4/df/d14/l5e 0 2024-05-17T21:52:48.769 INFO:tasks.workunit.client.0.smithi107.stdout:9/302: unlink d0/dd/d2f/d4a/f4c 0 2024-05-17T21:52:48.769 INFO:tasks.workunit.client.0.smithi107.stdout:0/333: rename d3/dc/d22/d3d/d56/l59 to d3/dc/d22/d23/d6c/l80 0 2024-05-17T21:52:48.769 INFO:tasks.workunit.client.0.smithi107.stdout:3/282: read d6/d8/d10/d25/f55 [511316,90685] 0 2024-05-17T21:52:48.770 INFO:tasks.workunit.client.0.smithi107.stdout:5/288: fsync d5/f9 0 2024-05-17T21:52:48.771 INFO:tasks.workunit.client.0.smithi107.stdout:0/334: readlink d3/l4b 0 2024-05-17T21:52:48.771 INFO:tasks.workunit.client.0.smithi107.stdout:2/208: creat d0/f40 x:0 0 0 2024-05-17T21:52:48.773 INFO:tasks.workunit.client.0.smithi107.stdout:5/289: write d5/f9 [1711862,74297] 0 2024-05-17T21:52:48.773 INFO:tasks.workunit.client.0.smithi107.stdout:8/265: rename dc/d21/c39 to dc/d44/d4b/d52/c6d 0 2024-05-17T21:52:48.774 INFO:tasks.workunit.client.0.smithi107.stdout:6/249: dwrite d9/d16/d22/d2d/f2f [0,4194304] 0 2024-05-17T21:52:48.774 INFO:tasks.workunit.client.0.smithi107.stdout:0/335: fdatasync d3/d39/f49 0 2024-05-17T21:52:48.775 INFO:tasks.workunit.client.0.smithi107.stdout:3/283: read d6/d1a/f1f [1590854,116103] 0 2024-05-17T21:52:48.777 INFO:tasks.workunit.client.0.smithi107.stdout:7/241: write d5/df/d11/d1c/f2e [3203135,96517] 0 2024-05-17T21:52:48.779 INFO:tasks.workunit.client.0.smithi107.stdout:3/284: chown d6/d2a/f5b 2 1 2024-05-17T21:52:48.779 INFO:tasks.workunit.client.0.smithi107.stdout:4/283: link d3/db/d16/f19 d3/db/d35/d5e/f63 0 2024-05-17T21:52:48.780 INFO:tasks.workunit.client.0.smithi107.stdout:2/209: rmdir d0/d9/d11 39 2024-05-17T21:52:48.783 INFO:tasks.workunit.client.0.smithi107.stdout:9/303: dwrite d0/dd/d21/d29/d3e/d45/f59 [0,4194304] 0 2024-05-17T21:52:48.792 INFO:tasks.workunit.client.0.smithi107.stdout:2/210: mknod d0/d9/d1c/d37/c41 0 2024-05-17T21:52:48.794 INFO:tasks.workunit.client.0.smithi107.stdout:4/284: dwrite d3/fa [0,4194304] 0 2024-05-17T21:52:48.795 INFO:tasks.workunit.client.0.smithi107.stdout:1/301: creat d4/df/d14/d2e/d43/f5f x:0 0 0 2024-05-17T21:52:48.795 INFO:tasks.workunit.client.0.smithi107.stdout:3/285: dwrite d6/d2a/d2d/f41 [0,4194304] 0 2024-05-17T21:52:48.796 INFO:tasks.workunit.client.0.smithi107.stdout:1/302: dread - d4/de/f3c zero size 2024-05-17T21:52:48.797 INFO:tasks.workunit.client.0.smithi107.stdout:1/303: fdatasync d4/de/f19 0 2024-05-17T21:52:48.798 INFO:tasks.workunit.client.0.smithi107.stdout:5/290: link d5/db/dc/l12 d5/db/d16/d2d/l5d 0 2024-05-17T21:52:48.801 INFO:tasks.workunit.client.0.smithi107.stdout:7/242: creat d5/df/d11/d1d/f55 x:0 0 0 2024-05-17T21:52:48.803 INFO:tasks.workunit.client.0.smithi107.stdout:1/304: dread d4/f21 [0,4194304] 0 2024-05-17T21:52:48.807 INFO:tasks.workunit.client.0.smithi107.stdout:9/304: creat d0/dd/d2f/f64 x:0 0 0 2024-05-17T21:52:48.808 INFO:tasks.workunit.client.0.smithi107.stdout:5/291: mknod d5/db/d16/d2d/d4d/c5e 0 2024-05-17T21:52:48.811 INFO:tasks.workunit.client.0.smithi107.stdout:2/211: mknod d0/d9/d11/c42 0 2024-05-17T21:52:48.812 INFO:tasks.workunit.client.0.smithi107.stdout:7/243: dwrite d5/df/d11/d1d/d44/f4b [0,4194304] 0 2024-05-17T21:52:48.817 INFO:tasks.workunit.client.0.smithi107.stdout:1/305: creat d4/df/d1a/d26/f60 x:0 0 0 2024-05-17T21:52:48.818 INFO:tasks.workunit.client.0.smithi107.stdout:9/305: creat d0/dd/d1b/f65 x:0 0 0 2024-05-17T21:52:48.819 INFO:tasks.workunit.client.0.smithi107.stdout:7/244: mkdir d5/df/d11/d1c/d31/d56 0 2024-05-17T21:52:48.821 INFO:tasks.workunit.client.0.smithi107.stdout:9/306: stat d0/dd/d1b/c26 0 2024-05-17T21:52:48.822 INFO:tasks.workunit.client.0.smithi107.stdout:0/336: dwrite d3/dc/f16 [4194304,4194304] 0 2024-05-17T21:52:48.825 INFO:tasks.workunit.client.0.smithi107.stdout:9/307: fdatasync d0/dd/d2f/f64 0 2024-05-17T21:52:48.832 INFO:tasks.workunit.client.0.smithi107.stdout:7/245: getdents d5/df/d11/d1c/d2f 0 2024-05-17T21:52:48.833 INFO:tasks.workunit.client.0.smithi107.stdout:6/250: dwrite d9/f10 [0,4194304] 0 2024-05-17T21:52:48.835 INFO:tasks.workunit.client.0.smithi107.stdout:9/308: link d0/f4 d0/dd/f66 0 2024-05-17T21:52:48.839 INFO:tasks.workunit.client.0.smithi107.stdout:3/286: dwrite d6/fc [0,4194304] 0 2024-05-17T21:52:48.839 INFO:tasks.workunit.client.0.smithi107.stdout:7/246: read d5/df/f10 [131622,99396] 0 2024-05-17T21:52:48.839 INFO:tasks.workunit.client.0.smithi107.stdout:5/292: dread d5/db/dc/f24 [0,4194304] 0 2024-05-17T21:52:48.841 INFO:tasks.workunit.client.0.smithi107.stdout:9/309: truncate d0/dd/d21/d29/d3e/f5e 282243 0 2024-05-17T21:52:48.842 INFO:tasks.workunit.client.0.smithi107.stdout:6/251: dread d9/d16/d22/d2d/f44 [0,4194304] 0 2024-05-17T21:52:48.852 INFO:tasks.workunit.client.0.smithi107.stdout:1/306: write d4/de/f1e [2092801,69017] 0 2024-05-17T21:52:48.852 INFO:tasks.workunit.client.0.smithi107.stdout:9/310: read - d0/dd/d2f/f5b zero size 2024-05-17T21:52:48.852 INFO:tasks.workunit.client.0.smithi107.stdout:6/252: stat d9/fa 0 2024-05-17T21:52:48.852 INFO:tasks.workunit.client.0.smithi107.stdout:7/247: symlink d5/df/d11/d1c/d2f/l57 0 2024-05-17T21:52:48.852 INFO:tasks.workunit.client.0.smithi107.stdout:3/287: mknod d6/d2a/d2d/d48/c5f 0 2024-05-17T21:52:48.852 INFO:tasks.workunit.client.0.smithi107.stdout:7/248: chown d5/d16/f1e 36 1 2024-05-17T21:52:48.853 INFO:tasks.workunit.client.1.smithi138.stdout:4/107: truncate d0/d3/d17/f1a 441025 0 2024-05-17T21:52:48.853 INFO:tasks.workunit.client.1.smithi138.stdout:3/152: creat d1/d3/d8/f34 x:0 0 0 2024-05-17T21:52:48.853 INFO:tasks.workunit.client.1.smithi138.stdout:5/121: symlink d1/d1a/l23 0 2024-05-17T21:52:48.853 INFO:tasks.workunit.client.1.smithi138.stdout:2/133: unlink d3/f14 0 2024-05-17T21:52:48.853 INFO:tasks.workunit.client.1.smithi138.stdout:1/118: symlink d4/df/l25 0 2024-05-17T21:52:48.853 INFO:tasks.workunit.client.1.smithi138.stdout:9/140: chown d6/d7/d16/l18 2276206 1 2024-05-17T21:52:48.855 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:48 smithi107 ceph-mon[28434]: pgmap v32: 65 pgs: 65 active+clean; 624 MiB data, 3.4 GiB used, 533 GiB / 536 GiB avail; 5.1 MiB/s rd, 61 MiB/s wr, 353 op/s 2024-05-17T21:52:48.855 INFO:tasks.workunit.client.1.smithi138.stdout:3/153: write d1/d3/d8/f28 [72082,19663] 0 2024-05-17T21:52:48.855 INFO:tasks.workunit.client.1.smithi138.stdout:1/119: dread - d4/df/f12 zero size 2024-05-17T21:52:48.856 INFO:tasks.workunit.client.1.smithi138.stdout:3/154: write d1/d3/d8/f34 [452134,18453] 0 2024-05-17T21:52:48.856 INFO:tasks.workunit.client.0.smithi107.stdout:1/307: creat d4/df/d1a/d26/f61 x:0 0 0 2024-05-17T21:52:48.856 INFO:tasks.workunit.client.0.smithi107.stdout:9/311: creat d0/dd/d21/d29/d3e/d45/f67 x:0 0 0 2024-05-17T21:52:48.856 INFO:tasks.workunit.client.0.smithi107.stdout:6/253: dwrite d9/f34 [0,4194304] 0 2024-05-17T21:52:48.856 INFO:tasks.workunit.client.0.smithi107.stdout:2/212: dread d0/d9/d11/f10 [0,4194304] 0 2024-05-17T21:52:48.857 INFO:tasks.workunit.client.0.smithi107.stdout:0/337: dread d3/f7b [0,4194304] 0 2024-05-17T21:52:48.857 INFO:tasks.workunit.client.1.smithi138.stdout:4/108: rename d0/c1 to d0/d3/d17/c26 0 2024-05-17T21:52:48.857 INFO:tasks.workunit.client.1.smithi138.stdout:9/141: dread d6/d7/d16/f20 [0,4194304] 0 2024-05-17T21:52:48.858 INFO:tasks.workunit.client.0.smithi107.stdout:0/338: readlink d3/dc/d6b/d77/l78 0 2024-05-17T21:52:48.858 INFO:tasks.workunit.client.1.smithi138.stdout:0/214: truncate d3/d10/f17 3115941 0 2024-05-17T21:52:48.861 INFO:tasks.workunit.client.1.smithi138.stdout:9/142: dread d6/d7/f26 [0,4194304] 0 2024-05-17T21:52:48.864 INFO:tasks.workunit.client.0.smithi107.stdout:7/249: creat d5/df/d11/d1c/d31/d39/f58 x:0 0 0 2024-05-17T21:52:48.865 INFO:tasks.workunit.client.0.smithi107.stdout:2/213: dwrite d0/d9/d1c/d3e/f3f [0,4194304] 0 2024-05-17T21:52:48.867 INFO:tasks.workunit.client.1.smithi138.stdout:3/155: rmdir d1/d3 39 2024-05-17T21:52:48.870 INFO:tasks.workunit.client.0.smithi107.stdout:9/312: mkdir d0/dd/d21/d29/d3e/d45/d68 0 2024-05-17T21:52:48.870 INFO:tasks.workunit.client.0.smithi107.stdout:5/293: truncate d5/db/dc/f20 1059887 0 2024-05-17T21:52:48.872 INFO:tasks.workunit.client.1.smithi138.stdout:4/109: rename d0/d3/c5 to d0/d3/d17/c27 0 2024-05-17T21:52:48.873 INFO:tasks.workunit.client.0.smithi107.stdout:0/339: fdatasync d3/dc/d31/f57 0 2024-05-17T21:52:48.874 INFO:tasks.workunit.client.0.smithi107.stdout:5/294: truncate d5/db/d16/d5b/f3f 5133713 0 2024-05-17T21:52:48.874 INFO:tasks.workunit.client.0.smithi107.stdout:2/214: dwrite d0/f6 [0,4194304] 0 2024-05-17T21:52:48.876 INFO:tasks.workunit.client.0.smithi107.stdout:5/295: chown d5/db/d39/f3c 79600804 1 2024-05-17T21:52:48.876 INFO:tasks.workunit.client.1.smithi138.stdout:0/215: unlink d3/c2f 0 2024-05-17T21:52:48.878 INFO:tasks.workunit.client.1.smithi138.stdout:5/122: truncate d1/f13 3992000 0 2024-05-17T21:52:48.881 INFO:tasks.workunit.client.0.smithi107.stdout:1/308: rename d4/df/d14/d2e/d43/d47/l4b to d4/df/d4e/l62 0 2024-05-17T21:52:48.882 INFO:tasks.workunit.client.0.smithi107.stdout:3/288: dread d6/d8/d10/f23 [0,4194304] 0 2024-05-17T21:52:48.882 INFO:tasks.workunit.client.0.smithi107.stdout:1/309: readlink d4/df/d14/d1d/l30 0 2024-05-17T21:52:48.883 INFO:tasks.workunit.client.1.smithi138.stdout:2/134: creat d3/f29 x:0 0 0 2024-05-17T21:52:48.887 INFO:tasks.workunit.client.0.smithi107.stdout:3/289: dread d6/d2a/d2d/d37/f3a [0,4194304] 0 2024-05-17T21:52:48.887 INFO:tasks.workunit.client.0.smithi107.stdout:4/285: sync 2024-05-17T21:52:48.887 INFO:tasks.workunit.client.0.smithi107.stdout:8/266: sync 2024-05-17T21:52:48.887 INFO:tasks.workunit.client.0.smithi107.stdout:7/250: symlink d5/d16/d3b/d4f/l59 0 2024-05-17T21:52:48.888 INFO:tasks.workunit.client.0.smithi107.stdout:8/267: chown dc/d21/l2a 2569037 1 2024-05-17T21:52:48.888 INFO:tasks.workunit.client.0.smithi107.stdout:6/254: creat d9/f48 x:0 0 0 2024-05-17T21:52:48.889 INFO:tasks.workunit.client.0.smithi107.stdout:7/251: chown d5/df/f1a 41280 1 2024-05-17T21:52:48.891 INFO:tasks.workunit.client.0.smithi107.stdout:9/313: mkdir d0/dd/d69 0 2024-05-17T21:52:48.891 INFO:tasks.workunit.client.0.smithi107.stdout:0/340: dread d3/dc/f60 [0,4194304] 0 2024-05-17T21:52:48.891 INFO:tasks.workunit.client.0.smithi107.stdout:8/268: write dc/d21/d27/d2f/f59 [342051,52299] 0 2024-05-17T21:52:48.894 INFO:tasks.workunit.client.0.smithi107.stdout:8/269: dread dc/d21/d27/d2f/f32 [0,4194304] 0 2024-05-17T21:52:48.894 INFO:tasks.workunit.client.0.smithi107.stdout:6/255: dread d9/d16/f3e [0,4194304] 0 2024-05-17T21:52:48.895 INFO:tasks.workunit.client.0.smithi107.stdout:1/310: creat d4/df/d4e/f63 x:0 0 0 2024-05-17T21:52:48.895 INFO:tasks.workunit.client.0.smithi107.stdout:1/311: write d4/f38 [797872,112620] 0 2024-05-17T21:52:48.896 INFO:tasks.workunit.client.0.smithi107.stdout:1/312: fsync d4/df/d14/f4c 0 2024-05-17T21:52:48.898 INFO:tasks.workunit.client.0.smithi107.stdout:2/215: rename d0/d9/d11/d18/f3a to d0/d9/d11/d27/f43 0 2024-05-17T21:52:48.900 INFO:tasks.workunit.client.0.smithi107.stdout:5/296: mkdir d5/d5f 0 2024-05-17T21:52:48.901 INFO:tasks.workunit.client.0.smithi107.stdout:5/297: write d5/db/f5c [122524,5204] 0 2024-05-17T21:52:48.901 INFO:tasks.workunit.client.1.smithi138.stdout:3/156: mknod d1/d3/d8/c35 0 2024-05-17T21:52:48.902 INFO:tasks.workunit.client.1.smithi138.stdout:3/157: chown d1/d3/d16/d2d 3 1 2024-05-17T21:52:48.903 INFO:tasks.workunit.client.1.smithi138.stdout:3/158: readlink d1/d3/l18 0 2024-05-17T21:52:48.903 INFO:tasks.workunit.client.0.smithi107.stdout:3/290: symlink d6/d8/d10/d35/d36/l60 0 2024-05-17T21:52:48.904 INFO:tasks.workunit.client.0.smithi107.stdout:3/291: read d6/d8/f17 [575692,78137] 0 2024-05-17T21:52:48.905 INFO:tasks.workunit.client.1.smithi138.stdout:1/120: dwrite f2 [0,4194304] 0 2024-05-17T21:52:48.906 INFO:tasks.workunit.client.1.smithi138.stdout:1/121: chown d4/db/f22 3734623 1 2024-05-17T21:52:48.909 INFO:tasks.workunit.client.1.smithi138.stdout:0/216: link d3/d3d/f53 d3/d3d/d51/f54 0 2024-05-17T21:52:48.909 INFO:tasks.workunit.client.1.smithi138.stdout:1/122: dread f3 [0,4194304] 0 2024-05-17T21:52:48.910 INFO:tasks.workunit.client.1.smithi138.stdout:0/217: chown l1 159581 1 2024-05-17T21:52:48.910 INFO:tasks.workunit.client.1.smithi138.stdout:1/123: readlink d4/df/l23 0 2024-05-17T21:52:48.910 INFO:tasks.workunit.client.1.smithi138.stdout:1/124: write f3 [1497684,29571] 0 2024-05-17T21:52:48.913 INFO:tasks.workunit.client.0.smithi107.stdout:7/252: rmdir d5/df 39 2024-05-17T21:52:48.913 INFO:tasks.workunit.client.1.smithi138.stdout:5/123: mkdir d1/d24 0 2024-05-17T21:52:48.915 INFO:tasks.workunit.client.1.smithi138.stdout:2/135: symlink d3/l2a 0 2024-05-17T21:52:48.918 INFO:tasks.workunit.client.1.smithi138.stdout:1/125: dwrite d4/f19 [0,4194304] 0 2024-05-17T21:52:48.919 INFO:tasks.workunit.client.0.smithi107.stdout:0/341: dread d3/f43 [0,4194304] 0 2024-05-17T21:52:48.921 INFO:tasks.workunit.client.1.smithi138.stdout:9/143: rename f2 to d6/d7/f2c 0 2024-05-17T21:52:48.924 INFO:tasks.workunit.client.1.smithi138.stdout:9/144: fdatasync d6/d7/d1f/f22 0 2024-05-17T21:52:48.925 INFO:tasks.workunit.client.1.smithi138.stdout:3/159: mknod d1/d3/d16/c36 0 2024-05-17T21:52:48.927 INFO:tasks.workunit.client.1.smithi138.stdout:1/126: dwrite d4/df/f12 [0,4194304] 0 2024-05-17T21:52:48.929 INFO:tasks.workunit.client.1.smithi138.stdout:4/110: rmdir d0/d3/db/d24 0 2024-05-17T21:52:48.930 INFO:tasks.workunit.client.0.smithi107.stdout:6/256: creat d9/d16/d22/d2d/f49 x:0 0 0 2024-05-17T21:52:48.931 INFO:tasks.workunit.client.1.smithi138.stdout:5/124: readlink d1/d6/l18 0 2024-05-17T21:52:48.960 INFO:tasks.workunit.client.1.smithi138.stdout:2/136: creat d3/d21/f2b x:0 0 0 2024-05-17T21:52:48.965 INFO:tasks.workunit.client.1.smithi138.stdout:2/137: dread d3/fd [0,4194304] 0 2024-05-17T21:52:48.965 INFO:tasks.workunit.client.1.smithi138.stdout:2/138: chown d3/l28 30653224 1 2024-05-17T21:52:48.969 INFO:tasks.workunit.client.0.smithi107.stdout:4/286: mknod d3/d4a/d47/d60/d3c/d58/c64 0 2024-05-17T21:52:48.971 INFO:tasks.workunit.client.0.smithi107.stdout:3/292: symlink d6/d2a/d2d/d48/l61 0 2024-05-17T21:52:48.972 INFO:tasks.workunit.client.1.smithi138.stdout:2/139: dwrite f2 [0,4194304] 0 2024-05-17T21:52:48.972 INFO:tasks.workunit.client.1.smithi138.stdout:2/140: read - d3/fa zero size 2024-05-17T21:52:48.972 INFO:tasks.workunit.client.1.smithi138.stdout:2/141: stat d3/f19 0 2024-05-17T21:52:48.972 INFO:tasks.workunit.client.1.smithi138.stdout:3/160: mknod d1/d2e/c37 0 2024-05-17T21:52:48.972 INFO:tasks.workunit.client.1.smithi138.stdout:3/161: fdatasync d1/f12 0 2024-05-17T21:52:48.978 INFO:tasks.workunit.client.0.smithi107.stdout:9/314: sync 2024-05-17T21:52:48.979 INFO:tasks.workunit.client.1.smithi138.stdout:9/145: truncate d6/d7/f26 2308687 0 2024-05-17T21:52:48.979 INFO:tasks.workunit.client.1.smithi138.stdout:2/142: dwrite d3/f6 [0,4194304] 0 2024-05-17T21:52:48.981 INFO:tasks.workunit.client.0.smithi107.stdout:7/253: fdatasync d5/df/d11/f25 0 2024-05-17T21:52:48.983 INFO:tasks.workunit.client.0.smithi107.stdout:7/254: chown d5/df/d11/d1c/d31/d39/f58 48395666 1 2024-05-17T21:52:48.990 INFO:tasks.workunit.client.0.smithi107.stdout:7/255: dread d5/df/d11/d1c/f2e [0,4194304] 0 2024-05-17T21:52:48.998 INFO:tasks.workunit.client.1.smithi138.stdout:3/162: creat d1/d3/d16/f38 x:0 0 0 2024-05-17T21:52:49.002 INFO:tasks.workunit.client.0.smithi107.stdout:3/293: dread d6/d1a/d57/f5e [0,4194304] 0 2024-05-17T21:52:49.006 INFO:tasks.workunit.client.1.smithi138.stdout:4/111: symlink d0/d3/d10/l28 0 2024-05-17T21:52:49.008 INFO:tasks.workunit.client.0.smithi107.stdout:8/270: creat dc/d21/d27/d34/f6e x:0 0 0 2024-05-17T21:52:49.014 INFO:tasks.workunit.client.1.smithi138.stdout:9/146: getdents d6/d7/d16/d29 0 2024-05-17T21:52:49.018 INFO:tasks.workunit.client.0.smithi107.stdout:1/313: mknod d4/c64 0 2024-05-17T21:52:49.018 INFO:tasks.workunit.client.1.smithi138.stdout:9/147: dread d6/d7/d16/f20 [0,4194304] 0 2024-05-17T21:52:49.020 INFO:tasks.workunit.client.1.smithi138.stdout:1/127: link d4/df/c18 d4/db/c26 0 2024-05-17T21:52:49.022 INFO:tasks.workunit.client.1.smithi138.stdout:4/112: dread d0/d1b/f18 [0,4194304] 0 2024-05-17T21:52:49.023 INFO:tasks.workunit.client.1.smithi138.stdout:4/113: write d0/d3/db/f20 [389670,964] 0 2024-05-17T21:52:49.025 INFO:tasks.workunit.client.0.smithi107.stdout:0/342: symlink d3/d24/d6f/l81 0 2024-05-17T21:52:49.029 INFO:tasks.workunit.client.1.smithi138.stdout:2/143: rename d3/l12 to d3/l2c 0 2024-05-17T21:52:49.030 INFO:tasks.workunit.client.1.smithi138.stdout:2/144: fdatasync d3/ff 0 2024-05-17T21:52:49.031 INFO:tasks.workunit.client.0.smithi107.stdout:7/256: symlink d5/df/d11/d1c/d2f/d53/l5a 0 2024-05-17T21:52:49.031 INFO:tasks.workunit.client.0.smithi107.stdout:0/343: dwrite d3/dc/d22/f2d [0,4194304] 0 2024-05-17T21:52:49.032 INFO:tasks.workunit.client.0.smithi107.stdout:0/344: chown d3/dc/d22/d23/f44 10552399 1 2024-05-17T21:52:49.032 INFO:tasks.workunit.client.1.smithi138.stdout:1/128: rename d4/dd/l1c to d4/dd/l27 0 2024-05-17T21:52:49.033 INFO:tasks.workunit.client.0.smithi107.stdout:2/216: getdents d0/d26 0 2024-05-17T21:52:49.034 INFO:tasks.workunit.client.0.smithi107.stdout:2/217: chown d0/d9/d11/d18/c1a 354439 1 2024-05-17T21:52:49.035 INFO:tasks.workunit.client.1.smithi138.stdout:2/145: dwrite d3/f1c [0,4194304] 0 2024-05-17T21:52:49.035 INFO:tasks.workunit.client.1.smithi138.stdout:2/146: chown d3/f4 344975 1 2024-05-17T21:52:49.036 INFO:tasks.workunit.client.0.smithi107.stdout:1/314: mknod d4/df/d1a/d41/c65 0 2024-05-17T21:52:49.038 INFO:tasks.workunit.client.0.smithi107.stdout:3/294: fdatasync f5 0 2024-05-17T21:52:49.042 INFO:tasks.workunit.client.0.smithi107.stdout:7/257: fsync d5/df/d11/d1c/f2e 0 2024-05-17T21:52:49.046 INFO:tasks.workunit.client.0.smithi107.stdout:0/345: rmdir d3/dc 39 2024-05-17T21:52:49.050 INFO:tasks.workunit.client.0.smithi107.stdout:3/295: write d6/d8/f9 [1425052,1181] 0 2024-05-17T21:52:49.051 INFO:tasks.workunit.client.0.smithi107.stdout:3/296: fsync d6/d1a/f4e 0 2024-05-17T21:52:49.052 INFO:tasks.workunit.client.1.smithi138.stdout:2/147: unlink d3/f19 0 2024-05-17T21:52:49.053 INFO:tasks.workunit.client.0.smithi107.stdout:3/297: write d6/d8/d10/f4b [395995,127782] 0 2024-05-17T21:52:49.053 INFO:tasks.workunit.client.1.smithi138.stdout:2/148: write d3/f6 [5546182,36120] 0 2024-05-17T21:52:49.053 INFO:tasks.workunit.client.1.smithi138.stdout:2/149: fsync d3/fa 0 2024-05-17T21:52:49.054 INFO:tasks.workunit.client.0.smithi107.stdout:8/271: creat dc/d21/d27/d34/f6f x:0 0 0 2024-05-17T21:52:49.054 INFO:tasks.workunit.client.0.smithi107.stdout:4/287: symlink d3/l65 0 2024-05-17T21:52:49.055 INFO:tasks.workunit.client.0.smithi107.stdout:8/272: write dc/d21/d27/d2f/f59 [1701807,27282] 0 2024-05-17T21:52:49.056 INFO:tasks.workunit.client.0.smithi107.stdout:8/273: dread - dc/d21/d27/d2f/f2c zero size 2024-05-17T21:52:49.056 INFO:tasks.workunit.client.0.smithi107.stdout:6/257: write d9/d16/f1e [1478959,55053] 0 2024-05-17T21:52:49.057 INFO:tasks.workunit.client.1.smithi138.stdout:9/148: creat d6/f2d x:0 0 0 2024-05-17T21:52:49.057 INFO:tasks.workunit.client.1.smithi138.stdout:1/129: truncate d4/df/f14 820307 0 2024-05-17T21:52:49.062 INFO:tasks.workunit.client.1.smithi138.stdout:7/107: sync 2024-05-17T21:52:49.062 INFO:tasks.workunit.client.0.smithi107.stdout:3/298: unlink d6/d8/d10/c21 0 2024-05-17T21:52:49.063 INFO:tasks.workunit.client.1.smithi138.stdout:7/108: write d0/f1c [372705,56499] 0 2024-05-17T21:52:49.063 INFO:tasks.workunit.client.1.smithi138.stdout:2/150: mknod d3/d21/c2d 0 2024-05-17T21:52:49.066 INFO:tasks.workunit.client.1.smithi138.stdout:9/149: unlink d6/lf 0 2024-05-17T21:52:49.068 INFO:tasks.workunit.client.1.smithi138.stdout:9/150: chown d6/d7/d16/c19 25719 1 2024-05-17T21:52:49.069 INFO:tasks.workunit.client.1.smithi138.stdout:9/151: chown d6/d7/d16/c19 6540833 1 2024-05-17T21:52:49.070 INFO:tasks.workunit.client.1.smithi138.stdout:1/130: symlink d4/df/l28 0 2024-05-17T21:52:49.070 INFO:tasks.workunit.client.1.smithi138.stdout:7/109: dwrite d0/fa [0,4194304] 0 2024-05-17T21:52:49.070 INFO:tasks.workunit.client.1.smithi138.stdout:9/152: write d6/d7/fc [224955,17674] 0 2024-05-17T21:52:49.072 INFO:tasks.workunit.client.0.smithi107.stdout:8/274: mkdir dc/d44/d4b/d52/d70 0 2024-05-17T21:52:49.073 INFO:tasks.workunit.client.0.smithi107.stdout:5/298: mknod d5/c60 0 2024-05-17T21:52:49.073 INFO:tasks.workunit.client.0.smithi107.stdout:5/299: fsync d5/f9 0 2024-05-17T21:52:49.074 INFO:tasks.workunit.client.1.smithi138.stdout:2/151: unlink d3/c11 0 2024-05-17T21:52:49.075 INFO:tasks.workunit.client.0.smithi107.stdout:0/346: symlink d3/dc/d6b/l82 0 2024-05-17T21:52:49.077 INFO:tasks.workunit.client.0.smithi107.stdout:1/315: creat d4/f66 x:0 0 0 2024-05-17T21:52:49.079 INFO:tasks.workunit.client.1.smithi138.stdout:1/131: mkdir d4/db/d29 0 2024-05-17T21:52:49.079 INFO:tasks.workunit.client.1.smithi138.stdout:2/152: dwrite d3/fe [0,4194304] 0 2024-05-17T21:52:49.081 INFO:tasks.workunit.client.0.smithi107.stdout:3/299: rmdir d6/d2a 39 2024-05-17T21:52:49.084 INFO:tasks.workunit.client.1.smithi138.stdout:2/153: dread d3/fe [0,4194304] 0 2024-05-17T21:52:49.085 INFO:tasks.workunit.client.1.smithi138.stdout:2/154: write d3/f1a [315662,20138] 0 2024-05-17T21:52:49.086 INFO:tasks.workunit.client.0.smithi107.stdout:2/218: getdents d0/d9/d11/d18 0 2024-05-17T21:52:49.086 INFO:tasks.workunit.client.1.smithi138.stdout:2/155: read d3/f1c [3048596,117473] 0 2024-05-17T21:52:49.087 INFO:tasks.workunit.client.1.smithi138.stdout:2/156: write d3/f1a [935584,1319] 0 2024-05-17T21:52:49.087 INFO:tasks.workunit.client.1.smithi138.stdout:1/132: creat d4/db/f2a x:0 0 0 2024-05-17T21:52:49.088 INFO:tasks.workunit.client.1.smithi138.stdout:1/133: fsync d4/f19 0 2024-05-17T21:52:49.088 INFO:tasks.workunit.client.1.smithi138.stdout:2/157: dread - d3/d21/f2b zero size 2024-05-17T21:52:49.089 INFO:tasks.workunit.client.0.smithi107.stdout:2/219: dread d0/f36 [0,4194304] 0 2024-05-17T21:52:49.089 INFO:tasks.workunit.client.1.smithi138.stdout:1/134: fdatasync d4/db/f22 0 2024-05-17T21:52:49.090 INFO:tasks.workunit.client.1.smithi138.stdout:5/125: truncate d1/d6/f11 2783740 0 2024-05-17T21:52:49.091 INFO:tasks.workunit.client.0.smithi107.stdout:8/275: mkdir dc/d10/d63/d71 0 2024-05-17T21:52:49.094 INFO:tasks.workunit.client.1.smithi138.stdout:2/158: dwrite d3/f5 [0,4194304] 0 2024-05-17T21:52:49.095 INFO:tasks.workunit.client.1.smithi138.stdout:0/218: rename d3/f27 to d3/d3d/d51/f55 0 2024-05-17T21:52:49.100 INFO:tasks.workunit.client.0.smithi107.stdout:0/347: creat d3/dc/d22/d5d/f83 x:0 0 0 2024-05-17T21:52:49.100 INFO:tasks.workunit.client.0.smithi107.stdout:7/258: rmdir d5 39 2024-05-17T21:52:49.102 INFO:tasks.workunit.client.0.smithi107.stdout:0/348: read d3/dc/d22/d23/f44 [3376289,18103] 0 2024-05-17T21:52:49.102 INFO:tasks.workunit.client.0.smithi107.stdout:1/316: creat d4/de/f67 x:0 0 0 2024-05-17T21:52:49.103 INFO:tasks.workunit.client.0.smithi107.stdout:1/317: readlink d4/df/d14/l17 0 2024-05-17T21:52:49.103 INFO:tasks.workunit.client.0.smithi107.stdout:9/315: truncate d0/f17 2521865 0 2024-05-17T21:52:49.104 INFO:tasks.workunit.client.0.smithi107.stdout:1/318: chown d4/df/d14/l37 317846768 1 2024-05-17T21:52:49.105 INFO:tasks.workunit.client.0.smithi107.stdout:1/319: fdatasync d4/de/f36 0 2024-05-17T21:52:49.106 INFO:tasks.workunit.client.0.smithi107.stdout:6/258: creat d9/d16/f4a x:0 0 0 2024-05-17T21:52:49.109 INFO:tasks.workunit.client.0.smithi107.stdout:8/276: rename dc/d44/l5f to dc/d44/d4b/l72 0 2024-05-17T21:52:49.110 INFO:tasks.workunit.client.1.smithi138.stdout:2/159: creat d3/d21/f2e x:0 0 0 2024-05-17T21:52:49.114 INFO:tasks.workunit.client.1.smithi138.stdout:3/163: dwrite d1/d3/d16/f30 [0,4194304] 0 2024-05-17T21:52:49.114 INFO:tasks.workunit.client.0.smithi107.stdout:8/277: read dc/d21/d27/f37 [432249,42755] 0 2024-05-17T21:52:49.120 INFO:tasks.workunit.client.1.smithi138.stdout:2/160: dread d3/f15 [0,4194304] 0 2024-05-17T21:52:49.121 INFO:tasks.workunit.client.1.smithi138.stdout:2/161: write d3/f1a [1080788,76479] 0 2024-05-17T21:52:49.122 INFO:tasks.workunit.client.0.smithi107.stdout:0/349: creat d3/d58/f84 x:0 0 0 2024-05-17T21:52:49.122 INFO:tasks.workunit.client.1.smithi138.stdout:0/219: symlink d3/l56 0 2024-05-17T21:52:49.124 INFO:tasks.workunit.client.1.smithi138.stdout:3/164: fsync d1/d3/f1c 0 2024-05-17T21:52:49.125 INFO:tasks.workunit.client.1.smithi138.stdout:3/165: write d1/d3/d16/f30 [153249,9156] 0 2024-05-17T21:52:49.131 INFO:tasks.workunit.client.1.smithi138.stdout:2/162: chown d3/c1e 710 1 2024-05-17T21:52:49.133 INFO:tasks.workunit.client.0.smithi107.stdout:9/316: rename d0/dd/d21/d29/d3e/f51 to d0/dd/d21/f6a 0 2024-05-17T21:52:49.139 INFO:tasks.workunit.client.1.smithi138.stdout:5/126: getdents d1 0 2024-05-17T21:52:49.140 INFO:tasks.workunit.client.1.smithi138.stdout:4/114: truncate d0/d1b/f18 1725766 0 2024-05-17T21:52:49.141 INFO:tasks.workunit.client.1.smithi138.stdout:3/166: symlink d1/d2e/l39 0 2024-05-17T21:52:49.141 INFO:tasks.workunit.client.1.smithi138.stdout:3/167: chown d1/lf 2228924 1 2024-05-17T21:52:49.145 INFO:tasks.workunit.client.0.smithi107.stdout:3/300: link d6/d8/ld d6/d1a/d57/l62 0 2024-05-17T21:52:49.146 INFO:tasks.workunit.client.1.smithi138.stdout:2/163: chown d3/c1b 14556746 1 2024-05-17T21:52:49.146 INFO:tasks.workunit.client.0.smithi107.stdout:3/301: write d6/d8/d10/d25/f46 [4404036,6620] 0 2024-05-17T21:52:49.147 INFO:tasks.workunit.client.0.smithi107.stdout:0/350: dwrite d3/f7b [0,4194304] 0 2024-05-17T21:52:49.147 INFO:tasks.workunit.client.0.smithi107.stdout:1/320: symlink d4/l68 0 2024-05-17T21:52:49.149 INFO:tasks.workunit.client.0.smithi107.stdout:3/302: chown d6/d8/d10/d35/d36/f3d 65 1 2024-05-17T21:52:49.149 INFO:tasks.workunit.client.0.smithi107.stdout:0/351: fsync d3/dc/f37 0 2024-05-17T21:52:49.149 INFO:tasks.workunit.client.1.smithi138.stdout:0/220: link d3/f45 d3/d10/d38/d3e/f57 0 2024-05-17T21:52:49.150 INFO:tasks.workunit.client.0.smithi107.stdout:2/220: link d0/d9/d1c/l1f d0/d9/d11/d27/l44 0 2024-05-17T21:52:49.153 INFO:tasks.workunit.client.0.smithi107.stdout:6/259: symlink d9/d16/d22/d43/l4b 0 2024-05-17T21:52:49.153 INFO:tasks.workunit.client.0.smithi107.stdout:1/321: dread d4/df/d14/d1d/f27 [0,4194304] 0 2024-05-17T21:52:49.153 INFO:tasks.workunit.client.0.smithi107.stdout:4/288: sync 2024-05-17T21:52:49.155 INFO:tasks.workunit.client.0.smithi107.stdout:1/322: readlink d4/df/d14/l17 0 2024-05-17T21:52:49.155 INFO:tasks.workunit.client.1.smithi138.stdout:5/127: chown d1/d6/l15 308 1 2024-05-17T21:52:49.155 INFO:tasks.workunit.client.0.smithi107.stdout:2/221: dread d0/d9/d1c/d22/f31 [4194304,4194304] 0 2024-05-17T21:52:49.156 INFO:tasks.workunit.client.1.smithi138.stdout:5/128: write d1/d6/d19/f20 [35885,64231] 0 2024-05-17T21:52:49.156 INFO:tasks.workunit.client.0.smithi107.stdout:1/323: write d4/fa [5253236,87157] 0 2024-05-17T21:52:49.156 INFO:tasks.workunit.client.0.smithi107.stdout:2/222: chown d0/d9/d11/df 1378391671 1 2024-05-17T21:52:49.156 INFO:tasks.workunit.client.0.smithi107.stdout:1/324: stat d4/l20 0 2024-05-17T21:52:49.157 INFO:tasks.workunit.client.1.smithi138.stdout:4/115: mknod d0/d3/d10/c29 0 2024-05-17T21:52:49.157 INFO:tasks.workunit.client.0.smithi107.stdout:2/223: fsync d0/d9/d1c/d22/f31 0 2024-05-17T21:52:49.157 INFO:tasks.workunit.client.0.smithi107.stdout:2/224: dread - d0/d9/f33 zero size 2024-05-17T21:52:49.159 INFO:tasks.workunit.client.0.smithi107.stdout:2/225: truncate d0/d9/d1c/f3d 512677 0 2024-05-17T21:52:49.160 INFO:tasks.workunit.client.1.smithi138.stdout:3/168: mkdir d1/d3/d16/d3a 0 2024-05-17T21:52:49.161 INFO:tasks.workunit.client.1.smithi138.stdout:3/169: write d1/d3/f1b [137201,93636] 0 2024-05-17T21:52:49.161 INFO:tasks.workunit.client.1.smithi138.stdout:3/170: dread - d1/d3/d16/f38 zero size 2024-05-17T21:52:49.162 INFO:tasks.workunit.client.0.smithi107.stdout:9/317: rename d0/dd/d2f/f30 to d0/dd/d21/d29/d3e/d45/f6b 0 2024-05-17T21:52:49.163 INFO:tasks.workunit.client.1.smithi138.stdout:1/135: link d4/dd/l27 d4/db/d29/l2b 0 2024-05-17T21:52:49.164 INFO:tasks.workunit.client.1.smithi138.stdout:3/171: chown d1/d3/d8/de/l32 8 1 2024-05-17T21:52:49.164 INFO:tasks.workunit.client.0.smithi107.stdout:9/318: fdatasync d0/dd/d1f/f52 0 2024-05-17T21:52:49.165 INFO:tasks.workunit.client.0.smithi107.stdout:8/278: symlink dc/d21/d68/l73 0 2024-05-17T21:52:49.165 INFO:tasks.workunit.client.1.smithi138.stdout:1/136: chown d4/f19 18380 1 2024-05-17T21:52:49.165 INFO:tasks.workunit.client.1.smithi138.stdout:7/110: write d0/f11 [213184,54935] 0 2024-05-17T21:52:49.166 INFO:tasks.workunit.client.1.smithi138.stdout:2/164: mkdir d3/d21/d2f 0 2024-05-17T21:52:49.168 INFO:tasks.workunit.client.0.smithi107.stdout:7/259: link d5/df/d11/d1c/f4e d5/df/d11/d1c/d31/d39/f5b 0 2024-05-17T21:52:49.170 INFO:tasks.workunit.client.1.smithi138.stdout:0/221: creat d3/d3d/f58 x:0 0 0 2024-05-17T21:52:49.172 INFO:tasks.workunit.client.1.smithi138.stdout:3/172: dwrite d1/d3/d16/f38 [0,4194304] 0 2024-05-17T21:52:49.176 INFO:tasks.workunit.client.1.smithi138.stdout:5/129: unlink d1/d1a/l23 0 2024-05-17T21:52:49.176 INFO:tasks.workunit.client.1.smithi138.stdout:7/111: dwrite d0/f1c [0,4194304] 0 2024-05-17T21:52:49.180 INFO:tasks.workunit.client.0.smithi107.stdout:0/352: mkdir d3/dc/d6b/d85 0 2024-05-17T21:52:49.188 INFO:tasks.workunit.client.1.smithi138.stdout:2/165: creat d3/d21/f30 x:0 0 0 2024-05-17T21:52:49.189 INFO:tasks.workunit.client.0.smithi107.stdout:4/289: mkdir d3/db/d1e/d66 0 2024-05-17T21:52:49.189 INFO:tasks.workunit.client.0.smithi107.stdout:4/290: stat d3/db/d16/f30 0 2024-05-17T21:52:49.193 INFO:tasks.workunit.client.1.smithi138.stdout:0/222: mkdir d3/d10/d38/d3e/d59 0 2024-05-17T21:52:49.193 INFO:tasks.workunit.client.1.smithi138.stdout:2/166: dwrite f2 [4194304,4194304] 0 2024-05-17T21:52:49.194 INFO:tasks.workunit.client.0.smithi107.stdout:5/300: truncate d5/db/d39/f4c 545172 0 2024-05-17T21:52:49.196 INFO:tasks.workunit.client.0.smithi107.stdout:5/301: fsync d5/db/d16/d5b/f3f 0 2024-05-17T21:52:49.196 INFO:tasks.workunit.client.0.smithi107.stdout:4/291: read d3/db/f33 [4437238,13042] 0 2024-05-17T21:52:49.197 INFO:tasks.workunit.client.1.smithi138.stdout:3/173: creat d1/d3/d8/de/f3b x:0 0 0 2024-05-17T21:52:49.199 INFO:tasks.workunit.client.1.smithi138.stdout:7/112: mknod d0/c22 0 2024-05-17T21:52:49.201 INFO:tasks.workunit.client.1.smithi138.stdout:5/130: mknod d1/d6/d19/c25 0 2024-05-17T21:52:49.201 INFO:tasks.workunit.client.1.smithi138.stdout:3/174: dread d1/f12 [0,4194304] 0 2024-05-17T21:52:49.202 INFO:tasks.workunit.client.1.smithi138.stdout:5/131: readlink d1/d6/l18 0 2024-05-17T21:52:49.202 INFO:tasks.workunit.client.1.smithi138.stdout:5/132: stat d1/f13 0 2024-05-17T21:52:49.203 INFO:tasks.workunit.client.0.smithi107.stdout:5/302: dwrite d5/f28 [4194304,4194304] 0 2024-05-17T21:52:49.203 INFO:tasks.workunit.client.1.smithi138.stdout:3/175: truncate d1/d3/d8/f34 588162 0 2024-05-17T21:52:49.203 INFO:tasks.workunit.client.1.smithi138.stdout:5/133: truncate d1/d6/d19/f20 246383 0 2024-05-17T21:52:49.204 INFO:tasks.workunit.client.0.smithi107.stdout:2/226: dwrite d0/d9/d11/df/f35 [0,4194304] 0 2024-05-17T21:52:49.207 INFO:tasks.workunit.client.0.smithi107.stdout:8/279: rmdir dc/d21/d35 39 2024-05-17T21:52:49.209 INFO:tasks.workunit.client.0.smithi107.stdout:7/260: rmdir d5/df/d11/d1d/d44 39 2024-05-17T21:52:49.209 INFO:tasks.workunit.client.0.smithi107.stdout:7/261: stat d5/fe 0 2024-05-17T21:52:49.210 INFO:tasks.workunit.client.1.smithi138.stdout:2/167: fdatasync d3/f10 0 2024-05-17T21:52:49.212 INFO:tasks.workunit.client.1.smithi138.stdout:0/223: mkdir d3/d3d/d5a 0 2024-05-17T21:52:49.216 INFO:tasks.workunit.client.1.smithi138.stdout:7/113: symlink d0/l23 0 2024-05-17T21:52:49.217 INFO:tasks.workunit.client.1.smithi138.stdout:7/114: truncate d0/f19 1043140 0 2024-05-17T21:52:49.219 INFO:tasks.workunit.client.1.smithi138.stdout:4/116: creat d0/f2a x:0 0 0 2024-05-17T21:52:49.225 INFO:tasks.workunit.client.1.smithi138.stdout:5/134: creat d1/d1a/f26 x:0 0 0 2024-05-17T21:52:49.225 INFO:tasks.workunit.client.1.smithi138.stdout:0/224: dread d3/d3d/f44 [0,4194304] 0 2024-05-17T21:52:49.227 INFO:tasks.workunit.client.1.smithi138.stdout:4/117: dwrite d0/d3/d10/f25 [0,4194304] 0 2024-05-17T21:52:49.227 INFO:tasks.workunit.client.1.smithi138.stdout:5/135: write d1/f22 [14776,63221] 0 2024-05-17T21:52:49.227 INFO:tasks.workunit.client.1.smithi138.stdout:1/137: link d4/f16 d4/df/f2c 0 2024-05-17T21:52:49.229 INFO:tasks.workunit.client.1.smithi138.stdout:4/118: write d0/d3/d10/f25 [3468943,67528] 0 2024-05-17T21:52:49.231 INFO:tasks.workunit.client.1.smithi138.stdout:2/168: creat d3/d21/f31 x:0 0 0 2024-05-17T21:52:49.236 INFO:tasks.workunit.client.1.smithi138.stdout:7/115: creat d0/f24 x:0 0 0 2024-05-17T21:52:49.240 INFO:tasks.workunit.client.0.smithi107.stdout:9/319: creat d0/dd/d21/d29/d3e/d45/d68/f6c x:0 0 0 2024-05-17T21:52:49.241 INFO:tasks.workunit.client.0.smithi107.stdout:2/227: stat d0/c34 0 2024-05-17T21:52:49.242 INFO:tasks.workunit.client.0.smithi107.stdout:2/228: fdatasync d0/f40 0 2024-05-17T21:52:49.244 INFO:tasks.workunit.client.0.smithi107.stdout:8/280: symlink dc/d10/d17/l74 0 2024-05-17T21:52:49.246 INFO:tasks.workunit.client.0.smithi107.stdout:6/260: write d9/d16/f3e [1871672,12429] 0 2024-05-17T21:52:49.249 INFO:tasks.workunit.client.1.smithi138.stdout:4/119: mkdir d0/d1b/d2b 0 2024-05-17T21:52:49.252 INFO:tasks.workunit.client.0.smithi107.stdout:4/292: fsync d3/db/d16/f45 0 2024-05-17T21:52:49.254 INFO:tasks.workunit.client.0.smithi107.stdout:5/303: symlink d5/l61 0 2024-05-17T21:52:49.257 INFO:tasks.workunit.client.0.smithi107.stdout:2/229: mknod d0/d9/d11/d27/c45 0 2024-05-17T21:52:49.260 INFO:tasks.workunit.client.0.smithi107.stdout:8/281: truncate dc/d21/d35/f11 1199084 0 2024-05-17T21:52:49.263 INFO:tasks.workunit.client.0.smithi107.stdout:0/353: rename d3/d24/c3f to d3/c86 0 2024-05-17T21:52:49.264 INFO:tasks.workunit.client.0.smithi107.stdout:0/354: write d3/f1d [4870005,86318] 0 2024-05-17T21:52:49.264 INFO:tasks.workunit.client.1.smithi138.stdout:4/120: mknod d0/c2c 0 2024-05-17T21:52:49.266 INFO:tasks.workunit.client.1.smithi138.stdout:2/169: link d3/d21/l23 d3/d21/l32 0 2024-05-17T21:52:49.266 INFO:tasks.workunit.client.0.smithi107.stdout:6/261: symlink d9/d16/d22/d2d/l4c 0 2024-05-17T21:52:49.267 INFO:tasks.workunit.client.0.smithi107.stdout:0/355: dread d3/dc/d22/f68 [0,4194304] 0 2024-05-17T21:52:49.269 INFO:tasks.workunit.client.0.smithi107.stdout:3/303: link d6/d8/d10/d25/c51 d6/d8/c63 0 2024-05-17T21:52:49.269 INFO:tasks.workunit.client.0.smithi107.stdout:3/304: fsync d6/f1c 0 2024-05-17T21:52:49.269 INFO:tasks.workunit.client.0.smithi107.stdout:1/325: write f3 [1575210,44316] 0 2024-05-17T21:52:49.270 INFO:tasks.workunit.client.1.smithi138.stdout:4/121: rmdir d0/d3 39 2024-05-17T21:52:49.272 INFO:tasks.workunit.client.0.smithi107.stdout:4/293: mknod d3/d4a/c67 0 2024-05-17T21:52:49.273 INFO:tasks.workunit.client.0.smithi107.stdout:4/294: chown d3/d4a/d47/d60/d3c 23068321 1 2024-05-17T21:52:49.273 INFO:tasks.workunit.client.0.smithi107.stdout:1/326: dread d4/de/f36 [4194304,4194304] 0 2024-05-17T21:52:49.275 INFO:tasks.workunit.client.0.smithi107.stdout:5/304: creat d5/db/d16/d5b/f62 x:0 0 0 2024-05-17T21:52:49.276 INFO:tasks.workunit.client.0.smithi107.stdout:9/320: symlink d0/dd/d2f/l6d 0 2024-05-17T21:52:49.277 INFO:tasks.workunit.client.1.smithi138.stdout:4/122: dread d0/d3/d10/f25 [0,4194304] 0 2024-05-17T21:52:49.277 INFO:tasks.workunit.client.0.smithi107.stdout:9/321: read d0/f10 [2360532,56142] 0 2024-05-17T21:52:49.278 INFO:tasks.workunit.client.1.smithi138.stdout:3/176: write d1/d3/f25 [4392543,56967] 0 2024-05-17T21:52:49.280 INFO:tasks.workunit.client.1.smithi138.stdout:1/138: rename d4/df/f14 to d4/dd/f2d 0 2024-05-17T21:52:49.283 INFO:tasks.workunit.client.0.smithi107.stdout:7/262: rmdir d5/df/d11/d2a/d4d 0 2024-05-17T21:52:49.283 INFO:tasks.workunit.client.0.smithi107.stdout:7/263: truncate d5/d16/f23 1140740 0 2024-05-17T21:52:49.285 INFO:tasks.workunit.client.1.smithi138.stdout:4/123: mknod d0/d3/db/c2d 0 2024-05-17T21:52:49.285 INFO:tasks.workunit.client.1.smithi138.stdout:1/139: dwrite d4/f19 [4194304,4194304] 0 2024-05-17T21:52:49.289 INFO:tasks.workunit.client.1.smithi138.stdout:2/170: rename d3/ff to d3/d21/d2f/f33 0 2024-05-17T21:52:49.290 INFO:tasks.workunit.client.1.smithi138.stdout:2/171: chown d3/f27 214 1 2024-05-17T21:52:49.293 INFO:tasks.workunit.client.0.smithi107.stdout:0/356: dread d3/f19 [0,4194304] 0 2024-05-17T21:52:49.295 INFO:tasks.workunit.client.0.smithi107.stdout:0/357: dread - d3/dc/d22/d5d/f67 zero size 2024-05-17T21:52:49.295 INFO:tasks.workunit.client.1.smithi138.stdout:3/177: symlink d1/l3c 0 2024-05-17T21:52:49.295 INFO:tasks.workunit.client.1.smithi138.stdout:0/225: truncate d3/d10/f1a 471666 0 2024-05-17T21:52:49.296 INFO:tasks.workunit.client.0.smithi107.stdout:3/305: mknod d6/d1a/c64 0 2024-05-17T21:52:49.299 INFO:tasks.workunit.client.0.smithi107.stdout:4/295: creat d3/f68 x:0 0 0 2024-05-17T21:52:49.301 INFO:tasks.workunit.client.1.smithi138.stdout:2/172: creat d3/d21/d2f/f34 x:0 0 0 2024-05-17T21:52:49.302 INFO:tasks.workunit.client.1.smithi138.stdout:4/124: creat d0/d3/d1d/f2e x:0 0 0 2024-05-17T21:52:49.302 INFO:tasks.workunit.client.0.smithi107.stdout:3/306: dwrite d6/d8/f14 [0,4194304] 0 2024-05-17T21:52:49.303 INFO:tasks.workunit.client.0.smithi107.stdout:3/307: write d6/d8/d10/d35/d36/f3d [877288,130487] 0 2024-05-17T21:52:49.304 INFO:tasks.workunit.client.1.smithi138.stdout:3/178: rmdir d1/d3/d16 39 2024-05-17T21:52:49.306 INFO:tasks.workunit.client.1.smithi138.stdout:0/226: rmdir d3/d10/d38 39 2024-05-17T21:52:49.307 INFO:tasks.workunit.client.1.smithi138.stdout:0/227: write d3/d10/f2c [3198874,29321] 0 2024-05-17T21:52:49.308 INFO:tasks.workunit.client.1.smithi138.stdout:1/140: fsync d4/f16 0 2024-05-17T21:52:49.310 INFO:tasks.workunit.client.1.smithi138.stdout:1/141: dread d4/db/f21 [0,4194304] 0 2024-05-17T21:52:49.311 INFO:tasks.workunit.client.0.smithi107.stdout:9/322: dwrite d0/dd/d21/d29/d3e/f5e [0,4194304] 0 2024-05-17T21:52:49.315 INFO:tasks.workunit.client.0.smithi107.stdout:6/262: mkdir d9/d16/d4d 0 2024-05-17T21:52:49.317 INFO:tasks.workunit.client.1.smithi138.stdout:3/179: mknod d1/d2e/c3d 0 2024-05-17T21:52:49.319 INFO:tasks.workunit.client.1.smithi138.stdout:5/136: write d1/d6/f11 [1495519,42939] 0 2024-05-17T21:52:49.326 INFO:tasks.workunit.client.0.smithi107.stdout:0/358: dwrite d3/f21 [0,4194304] 0 2024-05-17T21:52:49.326 INFO:tasks.workunit.client.0.smithi107.stdout:1/327: mkdir d4/df/d69 0 2024-05-17T21:52:49.327 INFO:tasks.workunit.client.0.smithi107.stdout:0/359: dread - d3/dc/d22/d3d/d56/f5b zero size 2024-05-17T21:52:49.328 INFO:tasks.workunit.client.1.smithi138.stdout:1/142: rmdir d4 39 2024-05-17T21:52:49.329 INFO:tasks.workunit.client.0.smithi107.stdout:0/360: read d3/d39/f69 [563239,70019] 0 2024-05-17T21:52:49.329 INFO:tasks.workunit.client.1.smithi138.stdout:1/143: chown f3 27233 1 2024-05-17T21:52:49.329 INFO:tasks.workunit.client.0.smithi107.stdout:5/305: symlink d5/db/d16/d2d/d51/l63 0 2024-05-17T21:52:49.330 INFO:tasks.workunit.client.0.smithi107.stdout:5/306: chown d5/db/d39/f43 61557821 1 2024-05-17T21:52:49.331 INFO:tasks.workunit.client.0.smithi107.stdout:2/230: creat d0/d9/f46 x:0 0 0 2024-05-17T21:52:49.335 INFO:tasks.workunit.client.1.smithi138.stdout:1/144: dwrite f2 [4194304,4194304] 0 2024-05-17T21:52:49.339 INFO:tasks.workunit.client.1.smithi138.stdout:6/200: sync 2024-05-17T21:52:49.340 INFO:tasks.workunit.client.1.smithi138.stdout:8/81: sync 2024-05-17T21:52:49.341 INFO:tasks.workunit.client.1.smithi138.stdout:4/125: mkdir d0/d2f 0 2024-05-17T21:52:49.344 INFO:tasks.workunit.client.1.smithi138.stdout:5/137: dread d1/d6/fa [0,4194304] 0 2024-05-17T21:52:49.345 INFO:tasks.workunit.client.1.smithi138.stdout:3/180: write d1/d3/d8/de/f27 [1501363,78298] 0 2024-05-17T21:52:49.352 INFO:tasks.workunit.client.1.smithi138.stdout:4/126: dread d0/d3/f12 [0,4194304] 0 2024-05-17T21:52:49.352 INFO:tasks.workunit.client.1.smithi138.stdout:6/201: dwrite d7/f2e [0,4194304] 0 2024-05-17T21:52:49.352 INFO:tasks.workunit.client.1.smithi138.stdout:8/82: dwrite d0/fa [0,4194304] 0 2024-05-17T21:52:49.353 INFO:tasks.workunit.client.0.smithi107.stdout:6/263: unlink d9/d16/f3a 0 2024-05-17T21:52:49.353 INFO:tasks.workunit.client.1.smithi138.stdout:8/83: dread - d0/dc/f10 zero size 2024-05-17T21:52:49.354 INFO:tasks.workunit.client.1.smithi138.stdout:6/202: write d7/d8/f2b [178440,16120] 0 2024-05-17T21:52:49.355 INFO:tasks.workunit.client.1.smithi138.stdout:6/203: write d7/d2d/f35 [2203982,67064] 0 2024-05-17T21:52:49.356 INFO:tasks.workunit.client.1.smithi138.stdout:2/173: creat d3/f35 x:0 0 0 2024-05-17T21:52:49.365 INFO:tasks.workunit.client.0.smithi107.stdout:5/307: unlink d5/db/dc/c22 0 2024-05-17T21:52:49.368 INFO:tasks.workunit.client.0.smithi107.stdout:2/231: creat d0/d9/d11/df/f47 x:0 0 0 2024-05-17T21:52:49.369 INFO:tasks.workunit.client.1.smithi138.stdout:5/138: mknod d1/d1a/c27 0 2024-05-17T21:52:49.369 INFO:tasks.workunit.client.1.smithi138.stdout:5/139: readlink d1/d6/l15 0 2024-05-17T21:52:49.370 INFO:tasks.workunit.client.0.smithi107.stdout:8/282: getdents dc/d44/d4b 0 2024-05-17T21:52:49.370 INFO:tasks.workunit.client.1.smithi138.stdout:0/228: read d3/d10/f1a [198752,41760] 0 2024-05-17T21:52:49.371 INFO:tasks.workunit.client.0.smithi107.stdout:8/283: write dc/d21/d27/d2f/f30 [837084,94611] 0 2024-05-17T21:52:49.372 INFO:tasks.workunit.client.0.smithi107.stdout:8/284: readlink dc/d10/d17/l74 0 2024-05-17T21:52:49.372 INFO:tasks.workunit.client.0.smithi107.stdout:7/264: rmdir d5/df/d11/d1c/d31/d39/d51 0 2024-05-17T21:52:49.372 INFO:tasks.workunit.client.0.smithi107.stdout:8/285: read - dc/d21/d27/d34/f6f zero size 2024-05-17T21:52:49.374 INFO:tasks.workunit.client.0.smithi107.stdout:9/323: creat d0/dd/d69/f6e x:0 0 0 2024-05-17T21:52:49.375 INFO:tasks.workunit.client.0.smithi107.stdout:0/361: write d3/dc/d22/d23/f2c [162373,35030] 0 2024-05-17T21:52:49.380 INFO:tasks.workunit.client.0.smithi107.stdout:8/286: dwrite dc/d21/d27/d2f/f59 [4194304,4194304] 0 2024-05-17T21:52:49.380 INFO:tasks.workunit.client.0.smithi107.stdout:1/328: mkdir d4/d6a 0 2024-05-17T21:52:49.381 INFO:tasks.workunit.client.0.smithi107.stdout:1/329: fdatasync d4/f8 0 2024-05-17T21:52:49.382 INFO:tasks.workunit.client.1.smithi138.stdout:8/84: rename d0/d11/f12 to d0/dc/f18 0 2024-05-17T21:52:49.382 INFO:tasks.workunit.client.0.smithi107.stdout:1/330: write d4/de/f19 [1427055,95165] 0 2024-05-17T21:52:49.385 INFO:tasks.workunit.client.1.smithi138.stdout:6/204: mkdir d7/d2d/d3f 0 2024-05-17T21:52:49.390 INFO:tasks.workunit.client.1.smithi138.stdout:2/174: truncate d3/f15 1033702 0 2024-05-17T21:52:49.392 INFO:tasks.workunit.client.1.smithi138.stdout:1/145: mkdir d4/d2e 0 2024-05-17T21:52:49.393 INFO:tasks.workunit.client.0.smithi107.stdout:2/232: mknod d0/d26/c48 0 2024-05-17T21:52:49.394 INFO:tasks.workunit.client.1.smithi138.stdout:3/181: mkdir d1/d3/d16/d3e 0 2024-05-17T21:52:49.396 INFO:tasks.workunit.client.0.smithi107.stdout:7/265: creat d5/df/d11/f5c x:0 0 0 2024-05-17T21:52:49.397 INFO:tasks.workunit.client.1.smithi138.stdout:5/140: fsync d1/f13 0 2024-05-17T21:52:49.400 INFO:tasks.workunit.client.1.smithi138.stdout:1/146: dwrite d4/df/f1f [0,4194304] 0 2024-05-17T21:52:49.402 INFO:tasks.workunit.client.0.smithi107.stdout:0/362: rmdir d3/dc/d6b/d77 39 2024-05-17T21:52:49.404 INFO:tasks.workunit.client.0.smithi107.stdout:5/308: dread d5/db/dc/f20 [0,4194304] 0 2024-05-17T21:52:49.405 INFO:tasks.workunit.client.1.smithi138.stdout:8/85: rename d0/dc/l14 to d0/l19 0 2024-05-17T21:52:49.406 INFO:tasks.workunit.client.1.smithi138.stdout:1/147: dread d4/fa [4194304,4194304] 0 2024-05-17T21:52:49.406 INFO:tasks.workunit.client.1.smithi138.stdout:8/86: chown d0/fa 203 1 2024-05-17T21:52:49.407 INFO:tasks.workunit.client.0.smithi107.stdout:4/296: getdents d3/d6/d26/d36 0 2024-05-17T21:52:49.408 INFO:tasks.workunit.client.1.smithi138.stdout:6/205: unlink d7/db/d12/l17 0 2024-05-17T21:52:49.409 INFO:tasks.workunit.client.0.smithi107.stdout:8/287: mknod dc/d21/d27/d34/d4c/c75 0 2024-05-17T21:52:49.410 INFO:tasks.workunit.client.0.smithi107.stdout:8/288: chown dc/d21/d27/d34 1 1 2024-05-17T21:52:49.410 INFO:tasks.workunit.client.0.smithi107.stdout:8/289: stat dc/d44/d4b/d52/d70 0 2024-05-17T21:52:49.415 INFO:tasks.workunit.client.1.smithi138.stdout:0/229: write f0 [1592110,76967] 0 2024-05-17T21:52:49.415 INFO:tasks.workunit.client.1.smithi138.stdout:0/230: chown d3/d3d/d51/f54 3 1 2024-05-17T21:52:49.416 INFO:tasks.workunit.client.0.smithi107.stdout:7/266: readlink d5/df/l30 0 2024-05-17T21:52:49.417 INFO:tasks.workunit.client.1.smithi138.stdout:5/141: rename d1/d6/c16 to d1/d6/c28 0 2024-05-17T21:52:49.417 INFO:tasks.workunit.client.1.smithi138.stdout:5/142: stat d1/d6/f11 0 2024-05-17T21:52:49.419 INFO:tasks.workunit.client.0.smithi107.stdout:9/324: mkdir d0/dd/d6f 0 2024-05-17T21:52:49.421 INFO:tasks.workunit.client.0.smithi107.stdout:0/363: readlink d3/dc/d6b/d77/l78 0 2024-05-17T21:52:49.422 INFO:tasks.workunit.client.0.smithi107.stdout:0/364: readlink d3/dc/d22/d3d/d56/l6a 0 2024-05-17T21:52:49.425 INFO:tasks.workunit.client.1.smithi138.stdout:1/148: rename d4/f19 to d4/dd/f2f 0 2024-05-17T21:52:49.426 INFO:tasks.workunit.client.0.smithi107.stdout:1/331: symlink d4/df/d14/l6b 0 2024-05-17T21:52:49.427 INFO:tasks.workunit.client.1.smithi138.stdout:3/182: dwrite d1/d3/d8/f13 [0,4194304] 0 2024-05-17T21:52:49.427 INFO:tasks.workunit.client.1.smithi138.stdout:1/149: truncate d4/db/f2a 450158 0 2024-05-17T21:52:49.428 INFO:tasks.workunit.client.1.smithi138.stdout:8/87: mkdir d0/dc/d1a 0 2024-05-17T21:52:49.431 INFO:tasks.workunit.client.0.smithi107.stdout:7/267: symlink d5/d16/l5d 0 2024-05-17T21:52:49.433 INFO:tasks.workunit.client.0.smithi107.stdout:9/325: creat d0/dd/d21/f70 x:0 0 0 2024-05-17T21:52:49.433 INFO:tasks.workunit.client.1.smithi138.stdout:6/206: dread f1 [8388608,4194304] 0 2024-05-17T21:52:49.434 INFO:tasks.workunit.client.0.smithi107.stdout:0/365: symlink d3/d58/d61/l87 0 2024-05-17T21:52:49.435 INFO:tasks.workunit.client.0.smithi107.stdout:8/290: mkdir dc/d44/d4b/d52/d70/d76 0 2024-05-17T21:52:49.436 INFO:tasks.workunit.client.1.smithi138.stdout:0/231: truncate d3/fc 3838449 0 2024-05-17T21:52:49.438 INFO:tasks.workunit.client.1.smithi138.stdout:5/143: mkdir d1/d6/d1b/d29 0 2024-05-17T21:52:49.442 INFO:tasks.workunit.client.0.smithi107.stdout:7/268: creat d5/df/d11/d1c/d31/d39/f5e x:0 0 0 2024-05-17T21:52:49.442 INFO:tasks.workunit.client.0.smithi107.stdout:8/291: dwrite dc/d10/d17/f47 [0,4194304] 0 2024-05-17T21:52:49.444 INFO:tasks.workunit.client.0.smithi107.stdout:9/326: rename d0/dd/d2f/d4a/c50 to d0/dd/d1f/c71 0 2024-05-17T21:52:49.444 INFO:tasks.workunit.client.1.smithi138.stdout:5/144: dwrite d1/d6/f11 [0,4194304] 0 2024-05-17T21:52:49.445 INFO:tasks.workunit.client.0.smithi107.stdout:7/269: dread d5/df/d11/d1c/f4e [0,4194304] 0 2024-05-17T21:52:49.445 INFO:tasks.workunit.client.0.smithi107.stdout:0/366: mkdir d3/d58/d88 0 2024-05-17T21:52:49.451 INFO:tasks.workunit.client.0.smithi107.stdout:1/332: symlink d4/df/d14/l6c 0 2024-05-17T21:52:49.452 INFO:tasks.workunit.client.0.smithi107.stdout:7/270: dwrite d5/d16/f28 [0,4194304] 0 2024-05-17T21:52:49.454 INFO:tasks.workunit.client.0.smithi107.stdout:2/233: getdents d0/d9/d11/d15 0 2024-05-17T21:52:49.455 INFO:tasks.workunit.client.1.smithi138.stdout:3/183: creat d1/d3/d8/de/f3f x:0 0 0 2024-05-17T21:52:49.457 INFO:tasks.workunit.client.1.smithi138.stdout:3/184: write d1/d3/f1b [1818446,26093] 0 2024-05-17T21:52:49.460 INFO:tasks.workunit.client.1.smithi138.stdout:8/88: creat d0/f1b x:0 0 0 2024-05-17T21:52:49.464 INFO:tasks.workunit.client.1.smithi138.stdout:0/232: unlink d3/d10/f28 0 2024-05-17T21:52:49.467 INFO:tasks.workunit.client.1.smithi138.stdout:8/89: dwrite d0/f2 [0,4194304] 0 2024-05-17T21:52:49.468 INFO:tasks.workunit.client.1.smithi138.stdout:4/127: link d0/l6 d0/d3/l30 0 2024-05-17T21:52:49.469 INFO:tasks.workunit.client.0.smithi107.stdout:1/333: mknod d4/df/d4e/d59/c6d 0 2024-05-17T21:52:49.470 INFO:tasks.workunit.client.1.smithi138.stdout:5/145: symlink d1/d6/d19/l2a 0 2024-05-17T21:52:49.471 INFO:tasks.workunit.client.0.smithi107.stdout:7/271: fsync d5/df/d11/d1c/f4e 0 2024-05-17T21:52:49.474 INFO:tasks.workunit.client.0.smithi107.stdout:7/272: fdatasync d5/df/d11/f21 0 2024-05-17T21:52:49.474 INFO:tasks.workunit.client.0.smithi107.stdout:7/273: truncate d5/f42 881068 0 2024-05-17T21:52:49.475 INFO:tasks.workunit.client.0.smithi107.stdout:2/234: unlink d0/d9/d11/c42 0 2024-05-17T21:52:49.476 INFO:tasks.workunit.client.1.smithi138.stdout:3/185: symlink d1/d2e/l40 0 2024-05-17T21:52:49.478 INFO:tasks.workunit.client.1.smithi138.stdout:5/146: dwrite d1/d6/d1b/f21 [0,4194304] 0 2024-05-17T21:52:49.478 INFO:tasks.workunit.client.1.smithi138.stdout:1/150: getdents d4/d2e 0 2024-05-17T21:52:49.480 INFO:tasks.workunit.client.0.smithi107.stdout:9/327: symlink d0/dd/d2f/l72 0 2024-05-17T21:52:49.482 INFO:tasks.workunit.client.1.smithi138.stdout:2/175: getdents d3/d21/d2f 0 2024-05-17T21:52:49.486 INFO:tasks.workunit.client.1.smithi138.stdout:6/207: mknod d7/d2d/d3f/c40 0 2024-05-17T21:52:49.487 INFO:tasks.workunit.client.0.smithi107.stdout:0/367: write d3/d39/f69 [479488,86352] 0 2024-05-17T21:52:49.488 INFO:tasks.workunit.client.0.smithi107.stdout:0/368: stat d3/d39/l7e 0 2024-05-17T21:52:49.489 INFO:tasks.workunit.client.0.smithi107.stdout:0/369: dread - d3/dc/d22/d23/d6c/f7c zero size 2024-05-17T21:52:49.490 INFO:tasks.workunit.client.0.smithi107.stdout:0/370: dread - d3/dc/d22/d23/d6c/f7c zero size 2024-05-17T21:52:49.492 INFO:tasks.workunit.client.0.smithi107.stdout:7/274: mknod d5/d16/d3b/d4f/c5f 0 2024-05-17T21:52:49.494 INFO:tasks.workunit.client.1.smithi138.stdout:4/128: mknod d0/d1b/c31 0 2024-05-17T21:52:49.494 INFO:tasks.workunit.client.0.smithi107.stdout:7/275: truncate d5/d16/f2b 9023945 0 2024-05-17T21:52:49.498 INFO:tasks.workunit.client.0.smithi107.stdout:1/334: creat d4/df/d69/f6e x:0 0 0 2024-05-17T21:52:49.499 INFO:tasks.workunit.client.0.smithi107.stdout:1/335: chown f3 44 1 2024-05-17T21:52:49.499 INFO:tasks.workunit.client.1.smithi138.stdout:1/151: mknod d4/df/c30 0 2024-05-17T21:52:49.501 INFO:tasks.workunit.client.1.smithi138.stdout:5/147: mkdir d1/d6/d1b/d2b 0 2024-05-17T21:52:49.501 INFO:tasks.workunit.client.1.smithi138.stdout:5/148: fsync d1/d6/d19/f20 0 2024-05-17T21:52:49.502 INFO:tasks.workunit.client.0.smithi107.stdout:0/371: creat d3/dc/d22/d23/d6c/f89 x:0 0 0 2024-05-17T21:52:49.503 INFO:tasks.workunit.client.1.smithi138.stdout:5/149: write d1/d6/d1b/f1d [228203,125117] 0 2024-05-17T21:52:49.503 INFO:tasks.workunit.client.1.smithi138.stdout:2/176: rename d3/fc to d3/d21/d2f/f36 0 2024-05-17T21:52:49.504 INFO:tasks.workunit.client.0.smithi107.stdout:2/235: dwrite d0/f29 [0,4194304] 0 2024-05-17T21:52:49.505 INFO:tasks.workunit.client.1.smithi138.stdout:6/208: creat d7/d2d/d3f/f41 x:0 0 0 2024-05-17T21:52:49.507 INFO:tasks.workunit.client.1.smithi138.stdout:0/233: creat d3/d10/d38/d3e/f5b x:0 0 0 2024-05-17T21:52:49.507 INFO:tasks.workunit.client.0.smithi107.stdout:8/292: getdents dc/d44 0 2024-05-17T21:52:49.508 INFO:tasks.workunit.client.1.smithi138.stdout:3/186: symlink d1/d3/d16/d3a/l41 0 2024-05-17T21:52:49.509 INFO:tasks.workunit.client.1.smithi138.stdout:3/187: fsync d1/d3/d8/de/f3b 0 2024-05-17T21:52:49.509 INFO:tasks.workunit.client.0.smithi107.stdout:3/308: sync 2024-05-17T21:52:49.510 INFO:tasks.workunit.client.1.smithi138.stdout:3/188: chown d1/d3 38 1 2024-05-17T21:52:49.510 INFO:tasks.workunit.client.0.smithi107.stdout:1/336: symlink d4/df/d69/l6f 0 2024-05-17T21:52:49.510 INFO:tasks.workunit.client.0.smithi107.stdout:7/276: dwrite d5/df/d11/d1d/d44/f50 [4194304,4194304] 0 2024-05-17T21:52:49.511 INFO:tasks.workunit.client.1.smithi138.stdout:3/189: read - d1/d3/f20 zero size 2024-05-17T21:52:49.511 INFO:tasks.workunit.client.0.smithi107.stdout:3/309: fdatasync d6/d8/d10/d35/d36/f3d 0 2024-05-17T21:52:49.512 INFO:tasks.workunit.client.1.smithi138.stdout:3/190: write d1/f12 [607026,130168] 0 2024-05-17T21:52:49.513 INFO:tasks.workunit.client.0.smithi107.stdout:3/310: read d6/fc [375224,52696] 0 2024-05-17T21:52:49.513 INFO:tasks.workunit.client.0.smithi107.stdout:7/277: write d5/df/d11/d1c/f54 [276079,26242] 0 2024-05-17T21:52:49.517 INFO:tasks.workunit.client.0.smithi107.stdout:2/236: rename d0/d9/d11/d18/l23 to d0/d9/d1c/d37/l49 0 2024-05-17T21:52:49.518 INFO:tasks.workunit.client.0.smithi107.stdout:2/237: readlink d0/d9/d1c/d37/l38 0 2024-05-17T21:52:49.520 INFO:tasks.workunit.client.1.smithi138.stdout:6/209: creat d7/d8/f42 x:0 0 0 2024-05-17T21:52:49.521 INFO:tasks.workunit.client.1.smithi138.stdout:6/210: chown d7/db/d34 113203 1 2024-05-17T21:52:49.522 INFO:tasks.workunit.client.0.smithi107.stdout:0/372: dwrite d3/f4a [0,4194304] 0 2024-05-17T21:52:49.522 INFO:tasks.workunit.client.0.smithi107.stdout:9/328: rmdir d0/dd/d6f 0 2024-05-17T21:52:49.522 INFO:tasks.workunit.client.1.smithi138.stdout:6/211: readlink d7/db/d12/l2c 0 2024-05-17T21:52:49.523 INFO:tasks.workunit.client.1.smithi138.stdout:4/129: mkdir d0/d3/d10/d1f/d32 0 2024-05-17T21:52:49.524 INFO:tasks.workunit.client.0.smithi107.stdout:2/238: dread d0/f29 [0,4194304] 0 2024-05-17T21:52:49.524 INFO:tasks.workunit.client.1.smithi138.stdout:5/150: dwrite d1/d6/fe [0,4194304] 0 2024-05-17T21:52:49.525 INFO:tasks.workunit.client.1.smithi138.stdout:4/130: truncate d0/d3/d1d/f2e 727681 0 2024-05-17T21:52:49.529 INFO:tasks.workunit.client.1.smithi138.stdout:1/152: mknod d4/db/d29/c31 0 2024-05-17T21:52:49.533 INFO:tasks.workunit.client.1.smithi138.stdout:3/191: symlink d1/d2e/l42 0 2024-05-17T21:52:49.534 INFO:tasks.workunit.client.1.smithi138.stdout:2/177: symlink d3/l37 0 2024-05-17T21:52:49.537 INFO:tasks.workunit.client.1.smithi138.stdout:6/212: rename d7/d2d to d7/db/d43 0 2024-05-17T21:52:49.537 INFO:tasks.workunit.client.1.smithi138.stdout:6/213: chown d7/db/d12/f26 1159907800 1 2024-05-17T21:52:49.539 INFO:tasks.workunit.client.0.smithi107.stdout:8/293: creat dc/d21/d69/f77 x:0 0 0 2024-05-17T21:52:49.540 INFO:tasks.workunit.client.1.smithi138.stdout:4/131: mknod d0/d3/d1d/c33 0 2024-05-17T21:52:49.545 INFO:tasks.workunit.client.0.smithi107.stdout:8/294: dwrite dc/d10/d17/f47 [0,4194304] 0 2024-05-17T21:52:49.546 INFO:tasks.workunit.client.0.smithi107.stdout:8/295: dread - dc/d21/d27/d2f/f2c zero size 2024-05-17T21:52:49.547 INFO:tasks.workunit.client.1.smithi138.stdout:9/153: sync 2024-05-17T21:52:49.548 INFO:tasks.workunit.client.1.smithi138.stdout:9/154: read - d6/f2d zero size 2024-05-17T21:52:49.548 INFO:tasks.workunit.client.1.smithi138.stdout:0/234: dwrite d3/f7 [0,4194304] 0 2024-05-17T21:52:49.555 INFO:tasks.workunit.client.0.smithi107.stdout:6/264: sync 2024-05-17T21:52:49.586 INFO:tasks.workunit.client.0.smithi107.stdout:7/278: mknod d5/df/d11/d1c/c60 0 2024-05-17T21:52:49.587 INFO:tasks.workunit.client.0.smithi107.stdout:1/337: truncate f3 3539161 0 2024-05-17T21:52:49.588 INFO:tasks.workunit.client.1.smithi138.stdout:3/192: creat d1/d3/d16/d3e/f43 x:0 0 0 2024-05-17T21:52:49.588 INFO:tasks.workunit.client.1.smithi138.stdout:3/193: chown d1 1804 1 2024-05-17T21:52:49.589 INFO:tasks.workunit.client.1.smithi138.stdout:5/151: truncate d1/d6/f11 2693877 0 2024-05-17T21:52:49.592 INFO:tasks.workunit.client.0.smithi107.stdout:0/373: mknod d3/d58/d88/c8a 0 2024-05-17T21:52:49.593 INFO:tasks.workunit.client.1.smithi138.stdout:6/214: mknod d7/db/d43/d3c/c44 0 2024-05-17T21:52:49.598 INFO:tasks.workunit.client.0.smithi107.stdout:8/296: unlink dc/d10/d17/f3f 0 2024-05-17T21:52:49.599 INFO:tasks.workunit.client.1.smithi138.stdout:6/215: dwrite d7/db/d43/f39 [0,4194304] 0 2024-05-17T21:52:49.602 INFO:tasks.workunit.client.1.smithi138.stdout:0/235: write d3/d10/d38/d3e/f57 [11554,105348] 0 2024-05-17T21:52:49.603 INFO:tasks.workunit.client.0.smithi107.stdout:5/309: sync 2024-05-17T21:52:49.604 INFO:tasks.workunit.client.0.smithi107.stdout:4/297: sync 2024-05-17T21:52:49.604 INFO:tasks.workunit.client.0.smithi107.stdout:4/298: dread - d3/db/d16/d43/f62 zero size 2024-05-17T21:52:49.605 INFO:tasks.workunit.client.0.smithi107.stdout:5/310: write d5/db/dc/f56 [871508,124364] 0 2024-05-17T21:52:49.606 INFO:tasks.workunit.client.0.smithi107.stdout:7/279: symlink d5/d16/d3b/d45/l61 0 2024-05-17T21:52:49.609 INFO:tasks.workunit.client.0.smithi107.stdout:3/311: truncate d6/fc 1518972 0 2024-05-17T21:52:49.610 INFO:tasks.workunit.client.1.smithi138.stdout:4/132: creat d0/f34 x:0 0 0 2024-05-17T21:52:49.611 INFO:tasks.workunit.client.1.smithi138.stdout:4/133: dread - d0/f2a zero size 2024-05-17T21:52:49.612 INFO:tasks.workunit.client.0.smithi107.stdout:0/374: fsync d3/f33 0 2024-05-17T21:52:49.614 INFO:tasks.workunit.client.0.smithi107.stdout:5/311: dwrite d5/db/d16/d2d/d32/f52 [0,4194304] 0 2024-05-17T21:52:49.614 INFO:tasks.workunit.client.0.smithi107.stdout:8/297: chown c7 64468 1 2024-05-17T21:52:49.615 INFO:tasks.workunit.client.0.smithi107.stdout:8/298: fdatasync dc/d21/f5d 0 2024-05-17T21:52:49.616 INFO:tasks.workunit.client.0.smithi107.stdout:8/299: readlink dc/d10/l5e 0 2024-05-17T21:52:49.617 INFO:tasks.workunit.client.1.smithi138.stdout:2/178: getdents d3 0 2024-05-17T21:52:49.619 INFO:tasks.workunit.client.0.smithi107.stdout:4/299: creat d3/db/f69 x:0 0 0 2024-05-17T21:52:49.620 INFO:tasks.workunit.client.0.smithi107.stdout:4/300: chown d3/db/d35/c3d 566524 1 2024-05-17T21:52:49.620 INFO:tasks.workunit.client.1.smithi138.stdout:9/155: link d6/d7/f10 d6/d7/f2e 0 2024-05-17T21:52:49.621 INFO:tasks.workunit.client.1.smithi138.stdout:9/156: readlink d6/l13 0 2024-05-17T21:52:49.623 INFO:tasks.workunit.client.1.smithi138.stdout:6/216: dread d7/d8/ff [0,4194304] 0 2024-05-17T21:52:49.624 INFO:tasks.workunit.client.1.smithi138.stdout:6/217: write d7/f37 [592271,62226] 0 2024-05-17T21:52:49.625 INFO:tasks.workunit.client.1.smithi138.stdout:6/218: stat d7/db/d43/d3f/f41 0 2024-05-17T21:52:49.626 INFO:tasks.workunit.client.1.smithi138.stdout:5/152: rmdir d1/d6/d1b/d2b 0 2024-05-17T21:52:49.628 INFO:tasks.workunit.client.1.smithi138.stdout:4/134: creat d0/d3/db/f35 x:0 0 0 2024-05-17T21:52:49.629 INFO:tasks.workunit.client.0.smithi107.stdout:1/338: symlink d4/d6a/l70 0 2024-05-17T21:52:49.629 INFO:tasks.workunit.client.1.smithi138.stdout:4/135: chown d0/d3/l30 23690 1 2024-05-17T21:52:49.630 INFO:tasks.workunit.client.1.smithi138.stdout:4/136: stat d0/l4 0 2024-05-17T21:52:49.630 INFO:tasks.workunit.client.1.smithi138.stdout:3/194: link d1/d2e/l2f d1/d3/d8/l44 0 2024-05-17T21:52:49.630 INFO:tasks.workunit.client.1.smithi138.stdout:4/137: chown d0/d3/d10/c11 7008398 1 2024-05-17T21:52:49.631 INFO:tasks.workunit.client.1.smithi138.stdout:3/195: write d1/d3/d16/d3e/f43 [225779,128078] 0 2024-05-17T21:52:49.634 INFO:tasks.workunit.client.0.smithi107.stdout:9/329: getdents d0/dd/d21/d29/d3e 0 2024-05-17T21:52:49.636 INFO:tasks.workunit.client.1.smithi138.stdout:2/179: rename d3/d21/d2f/f34 to d3/d21/f38 0 2024-05-17T21:52:49.637 INFO:tasks.workunit.client.0.smithi107.stdout:2/239: getdents d0/d26 0 2024-05-17T21:52:49.637 INFO:tasks.workunit.client.1.smithi138.stdout:0/236: dwrite d3/d3d/d51/f54 [0,4194304] 0 2024-05-17T21:52:49.638 INFO:tasks.workunit.client.0.smithi107.stdout:7/280: write d5/df/f20 [4799979,80895] 0 2024-05-17T21:52:49.638 INFO:tasks.workunit.client.1.smithi138.stdout:9/157: mknod d6/d7/d1f/c2f 0 2024-05-17T21:52:49.640 INFO:tasks.workunit.client.0.smithi107.stdout:0/375: creat d3/d24/d6f/f8b x:0 0 0 2024-05-17T21:52:49.642 INFO:tasks.workunit.client.0.smithi107.stdout:0/376: fsync d3/dc/d22/d23/d6c/f89 0 2024-05-17T21:52:49.646 INFO:tasks.workunit.client.0.smithi107.stdout:6/265: creat d9/f4e x:0 0 0 2024-05-17T21:52:49.646 INFO:tasks.workunit.client.0.smithi107.stdout:2/240: dwrite d0/d9/d11/d18/f20 [0,4194304] 0 2024-05-17T21:52:49.646 INFO:tasks.workunit.client.0.smithi107.stdout:6/266: chown d9/c3f 187132 1 2024-05-17T21:52:49.646 INFO:tasks.workunit.client.0.smithi107.stdout:2/241: read - d0/d9/f46 zero size 2024-05-17T21:52:49.647 INFO:tasks.workunit.client.1.smithi138.stdout:5/153: mkdir d1/d2c 0 2024-05-17T21:52:49.647 INFO:tasks.workunit.client.0.smithi107.stdout:6/267: truncate d9/f4e 156185 0 2024-05-17T21:52:49.647 INFO:tasks.workunit.client.0.smithi107.stdout:5/312: rename d5/db/dc/d18/f58 to d5/db/d16/d2a/f64 0 2024-05-17T21:52:49.649 INFO:tasks.workunit.client.1.smithi138.stdout:4/138: mkdir d0/d1b/d36 0 2024-05-17T21:52:49.651 INFO:tasks.workunit.client.1.smithi138.stdout:3/196: mknod d1/d3/d8/de/c45 0 2024-05-17T21:52:49.651 INFO:tasks.workunit.client.0.smithi107.stdout:2/242: dread d0/d9/d11/df/f35 [0,4194304] 0 2024-05-17T21:52:49.653 INFO:tasks.workunit.client.1.smithi138.stdout:3/197: dread d1/f12 [0,4194304] 0 2024-05-17T21:52:49.654 INFO:tasks.workunit.client.0.smithi107.stdout:2/243: write d0/d9/d1c/d3e/f3f [1817340,79472] 0 2024-05-17T21:52:49.656 INFO:tasks.workunit.client.1.smithi138.stdout:2/180: creat d3/d21/d2f/f39 x:0 0 0 2024-05-17T21:52:49.656 INFO:tasks.workunit.client.1.smithi138.stdout:2/181: dread - d3/d21/f30 zero size 2024-05-17T21:52:49.658 INFO:tasks.workunit.client.1.smithi138.stdout:0/237: mknod d3/d3d/d48/c5c 0 2024-05-17T21:52:49.659 INFO:tasks.workunit.client.0.smithi107.stdout:9/330: unlink d0/fe 0 2024-05-17T21:52:49.661 INFO:tasks.workunit.client.0.smithi107.stdout:7/281: mknod d5/df/d11/d1d/d33/c62 0 2024-05-17T21:52:49.661 INFO:tasks.workunit.client.0.smithi107.stdout:7/282: dread - d5/d16/f3a zero size 2024-05-17T21:52:49.663 INFO:tasks.workunit.client.1.smithi138.stdout:6/219: link d7/fc d7/db/d12/f45 0 2024-05-17T21:52:49.666 INFO:tasks.workunit.client.1.smithi138.stdout:4/139: mknod d0/d3/db/c37 0 2024-05-17T21:52:49.670 INFO:tasks.workunit.client.1.smithi138.stdout:2/182: symlink d3/d21/d2f/l3a 0 2024-05-17T21:52:49.670 INFO:tasks.workunit.client.0.smithi107.stdout:8/300: creat dc/d44/d4b/d52/d70/d76/f78 x:0 0 0 2024-05-17T21:52:49.670 INFO:tasks.workunit.client.0.smithi107.stdout:4/301: dwrite d3/db/d16/f30 [0,4194304] 0 2024-05-17T21:52:49.673 INFO:tasks.workunit.client.1.smithi138.stdout:6/220: creat d7/db/d43/d3f/f46 x:0 0 0 2024-05-17T21:52:49.675 INFO:tasks.workunit.client.1.smithi138.stdout:4/140: mkdir d0/d3/d1d/d38 0 2024-05-17T21:52:49.677 INFO:tasks.workunit.client.1.smithi138.stdout:0/238: dread d3/f29 [4194304,4194304] 0 2024-05-17T21:52:49.679 INFO:tasks.workunit.client.0.smithi107.stdout:3/312: unlink d6/fc 0 2024-05-17T21:52:49.681 INFO:tasks.workunit.client.0.smithi107.stdout:9/331: symlink d0/dd/d21/d29/d57/l73 0 2024-05-17T21:52:49.681 INFO:tasks.workunit.client.1.smithi138.stdout:5/154: dwrite d1/d6/fe [0,4194304] 0 2024-05-17T21:52:49.684 INFO:tasks.workunit.client.1.smithi138.stdout:2/183: truncate d3/f15 1563759 0 2024-05-17T21:52:49.684 INFO:tasks.workunit.client.0.smithi107.stdout:0/377: dread d3/f18 [0,4194304] 0 2024-05-17T21:52:49.685 INFO:tasks.workunit.client.1.smithi138.stdout:9/158: link d6/d7/f17 d6/d7/f30 0 2024-05-17T21:52:49.689 INFO:tasks.workunit.client.1.smithi138.stdout:4/141: unlink d0/d3/d1d/c33 0 2024-05-17T21:52:49.691 INFO:tasks.workunit.client.0.smithi107.stdout:6/268: mknod d9/d16/c4f 0 2024-05-17T21:52:49.691 INFO:tasks.workunit.client.1.smithi138.stdout:0/239: dwrite d3/f7 [4194304,4194304] 0 2024-05-17T21:52:49.692 INFO:tasks.workunit.client.1.smithi138.stdout:4/142: truncate d0/f34 747923 0 2024-05-17T21:52:49.693 INFO:tasks.workunit.client.1.smithi138.stdout:9/159: read d6/d7/f2e [2300561,26027] 0 2024-05-17T21:52:49.693 INFO:tasks.workunit.client.1.smithi138.stdout:6/221: dread d7/d8/f24 [0,4194304] 0 2024-05-17T21:52:49.696 INFO:tasks.workunit.client.0.smithi107.stdout:8/301: rename dc/d10/d17/f47 to dc/d10/d63/f79 0 2024-05-17T21:52:49.699 INFO:tasks.workunit.client.1.smithi138.stdout:5/155: mknod d1/d6/c2d 0 2024-05-17T21:52:49.699 INFO:tasks.workunit.client.1.smithi138.stdout:5/156: chown d1/d6/d1b/d29 1 1 2024-05-17T21:52:49.701 INFO:tasks.workunit.client.0.smithi107.stdout:9/332: creat d0/dd/d1b/f74 x:0 0 0 2024-05-17T21:52:49.705 INFO:tasks.workunit.client.1.smithi138.stdout:2/184: rename d3/l16 to d3/d21/d2f/l3b 0 2024-05-17T21:52:49.706 INFO:tasks.workunit.client.1.smithi138.stdout:3/198: getdents d1/d3/d8 0 2024-05-17T21:52:49.709 INFO:tasks.workunit.client.1.smithi138.stdout:0/240: creat d3/d10/d39/f5d x:0 0 0 2024-05-17T21:52:49.709 INFO:tasks.workunit.client.0.smithi107.stdout:1/339: write d4/df/d14/f23 [446873,122175] 0 2024-05-17T21:52:49.710 INFO:tasks.workunit.client.0.smithi107.stdout:4/302: rename d3/d4a/d47/d60/c27 to d3/db/d1e/d66/c6a 0 2024-05-17T21:52:49.711 INFO:tasks.workunit.client.0.smithi107.stdout:4/303: readlink d3/d4a/d47/d60/l2e 0 2024-05-17T21:52:49.712 INFO:tasks.workunit.client.0.smithi107.stdout:8/302: rmdir dc/d21/d35 39 2024-05-17T21:52:49.712 INFO:tasks.workunit.client.0.smithi107.stdout:8/303: readlink l6 0 2024-05-17T21:52:49.713 INFO:tasks.workunit.client.0.smithi107.stdout:8/304: read dc/f43 [207938,115965] 0 2024-05-17T21:52:49.713 INFO:tasks.workunit.client.1.smithi138.stdout:9/160: write d6/d7/d16/f21 [1546731,64456] 0 2024-05-17T21:52:49.714 INFO:tasks.workunit.client.0.smithi107.stdout:3/313: creat d6/d8/d10/d35/d36/d4c/f65 x:0 0 0 2024-05-17T21:52:49.715 INFO:tasks.workunit.client.0.smithi107.stdout:3/314: write f5 [1093278,121328] 0 2024-05-17T21:52:49.715 INFO:tasks.workunit.client.1.smithi138.stdout:4/143: creat d0/d3/d10/d1f/f39 x:0 0 0 2024-05-17T21:52:49.716 INFO:tasks.workunit.client.0.smithi107.stdout:3/315: read d6/d8/d10/f38 [54332,33663] 0 2024-05-17T21:52:49.716 INFO:tasks.workunit.client.1.smithi138.stdout:4/144: read d0/d3/d1d/f2e [38153,28667] 0 2024-05-17T21:52:49.718 INFO:tasks.workunit.client.0.smithi107.stdout:6/269: symlink d9/d16/d4d/l50 0 2024-05-17T21:52:49.719 INFO:tasks.workunit.client.0.smithi107.stdout:6/270: write d9/f15 [4696275,70928] 0 2024-05-17T21:52:49.721 INFO:tasks.workunit.client.0.smithi107.stdout:1/340: symlink d4/df/d14/d1d/l71 0 2024-05-17T21:52:49.721 INFO:tasks.workunit.client.0.smithi107.stdout:6/271: fdatasync d9/f10 0 2024-05-17T21:52:49.721 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:49 smithi138 ceph-mon[35948]: from='mgr.14636 172.21.15.138:0/3120106723' entity='mgr.smithi138.napnll' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:52:49.722 INFO:tasks.workunit.client.1.smithi138.stdout:3/199: rename d1/f12 to d1/d3/d16/d3e/f46 0 2024-05-17T21:52:49.723 INFO:tasks.workunit.client.0.smithi107.stdout:9/333: dwrite d0/dd/d1b/f48 [0,4194304] 0 2024-05-17T21:52:49.725 INFO:tasks.workunit.client.0.smithi107.stdout:0/378: rename d3/dc/ld to d3/d24/l8c 0 2024-05-17T21:52:49.725 INFO:tasks.workunit.client.1.smithi138.stdout:0/241: write d3/d3d/f44 [1076934,127835] 0 2024-05-17T21:52:49.726 INFO:tasks.workunit.client.1.smithi138.stdout:0/242: write d3/fd [114379,118567] 0 2024-05-17T21:52:49.727 INFO:tasks.workunit.client.0.smithi107.stdout:8/305: symlink dc/d21/d27/d34/d4c/l7a 0 2024-05-17T21:52:49.729 INFO:tasks.workunit.client.0.smithi107.stdout:7/283: truncate f4 358665 0 2024-05-17T21:52:49.734 INFO:tasks.workunit.client.1.smithi138.stdout:6/222: mknod d7/db/d34/c47 0 2024-05-17T21:52:49.736 INFO:tasks.workunit.client.0.smithi107.stdout:3/316: dwrite d6/d2a/f5b [0,4194304] 0 2024-05-17T21:52:49.738 INFO:tasks.workunit.client.0.smithi107.stdout:5/313: sync 2024-05-17T21:52:49.738 INFO:tasks.workunit.client.0.smithi107.stdout:2/244: sync 2024-05-17T21:52:49.740 INFO:tasks.workunit.client.1.smithi138.stdout:3/200: mknod d1/d3/d8/c47 0 2024-05-17T21:52:49.741 INFO:tasks.workunit.client.0.smithi107.stdout:2/245: stat d0/d9/d11/d27/c45 0 2024-05-17T21:52:49.746 INFO:tasks.workunit.client.0.smithi107.stdout:4/304: rename d3/db/f56 to d3/db/d35/f6b 0 2024-05-17T21:52:49.746 INFO:tasks.workunit.client.0.smithi107.stdout:3/317: dwrite d6/d2a/d2d/f41 [4194304,4194304] 0 2024-05-17T21:52:49.750 INFO:tasks.workunit.client.0.smithi107.stdout:0/379: mknod d3/dc/d22/d23/c8d 0 2024-05-17T21:52:49.750 INFO:tasks.workunit.client.1.smithi138.stdout:9/161: mkdir d6/d7/d31 0 2024-05-17T21:52:49.751 INFO:tasks.workunit.client.0.smithi107.stdout:5/314: dwrite d5/db/d16/d5b/f3d [0,4194304] 0 2024-05-17T21:52:49.752 INFO:tasks.workunit.client.0.smithi107.stdout:8/306: rmdir dc 39 2024-05-17T21:52:49.754 INFO:tasks.workunit.client.0.smithi107.stdout:9/334: dwrite d0/dd/d21/d29/d3e/d45/f6b [0,4194304] 0 2024-05-17T21:52:49.754 INFO:tasks.workunit.client.0.smithi107.stdout:0/380: dread d3/f19 [0,4194304] 0 2024-05-17T21:52:49.755 INFO:tasks.workunit.client.1.smithi138.stdout:2/185: dread - d3/d21/f2e zero size 2024-05-17T21:52:49.755 INFO:tasks.workunit.client.0.smithi107.stdout:2/246: dwrite d0/f6 [0,4194304] 0 2024-05-17T21:52:49.756 INFO:tasks.workunit.client.1.smithi138.stdout:2/186: chown d3/d21/d2f/f36 6 1 2024-05-17T21:52:49.757 INFO:tasks.workunit.client.0.smithi107.stdout:2/247: fdatasync d0/d9/d11/df/f47 0 2024-05-17T21:52:49.759 INFO:tasks.workunit.client.0.smithi107.stdout:7/284: mknod d5/d16/d3b/c63 0 2024-05-17T21:52:49.761 INFO:tasks.workunit.client.1.smithi138.stdout:6/223: symlink d7/db/d34/l48 0 2024-05-17T21:52:49.766 INFO:tasks.workunit.client.1.smithi138.stdout:4/145: mknod d0/c3a 0 2024-05-17T21:52:49.766 INFO:tasks.workunit.client.1.smithi138.stdout:5/157: write d1/f13 [1595530,22810] 0 2024-05-17T21:52:49.767 INFO:tasks.workunit.client.1.smithi138.stdout:4/146: stat d0/d3/d10/l23 0 2024-05-17T21:52:49.771 INFO:tasks.workunit.client.1.smithi138.stdout:3/201: rename d1/d2e/c33 to d1/d3/d8/de/c48 0 2024-05-17T21:52:49.775 INFO:tasks.workunit.client.1.smithi138.stdout:2/187: creat d3/d21/d2f/f3c x:0 0 0 2024-05-17T21:52:49.776 INFO:tasks.workunit.client.1.smithi138.stdout:6/224: symlink d7/l49 0 2024-05-17T21:52:49.777 INFO:tasks.workunit.client.1.smithi138.stdout:6/225: stat d7/d8 0 2024-05-17T21:52:49.778 INFO:tasks.workunit.client.1.smithi138.stdout:4/147: mknod d0/d3/d10/d1f/c3b 0 2024-05-17T21:52:49.779 INFO:tasks.workunit.client.0.smithi107.stdout:3/318: mkdir d6/d8/d10/d35/d36/d66 0 2024-05-17T21:52:49.780 INFO:tasks.workunit.client.1.smithi138.stdout:0/243: creat d3/f5e x:0 0 0 2024-05-17T21:52:49.780 INFO:tasks.workunit.client.1.smithi138.stdout:0/244: readlink d3/d10/l13 0 2024-05-17T21:52:49.781 INFO:tasks.workunit.client.1.smithi138.stdout:9/162: symlink d6/d7/l32 0 2024-05-17T21:52:49.783 INFO:tasks.workunit.client.0.smithi107.stdout:5/315: rename d5/db/dc/d18 to d5/db/d16/d54/d59/d65 0 2024-05-17T21:52:49.783 INFO:tasks.workunit.client.1.smithi138.stdout:5/158: mknod d1/d2c/c2e 0 2024-05-17T21:52:49.784 INFO:tasks.workunit.client.1.smithi138.stdout:6/226: write d7/d8/ff [973662,18261] 0 2024-05-17T21:52:49.785 INFO:tasks.workunit.client.0.smithi107.stdout:0/381: creat d3/dc/d22/d23/d6c/f8e x:0 0 0 2024-05-17T21:52:49.788 INFO:tasks.workunit.client.1.smithi138.stdout:3/202: mkdir d1/d3/d49 0 2024-05-17T21:52:49.790 INFO:tasks.workunit.client.0.smithi107.stdout:7/285: creat d5/df/d11/d1d/d33/f64 x:0 0 0 2024-05-17T21:52:49.791 INFO:tasks.workunit.client.1.smithi138.stdout:0/245: creat d3/d3d/d48/f5f x:0 0 0 2024-05-17T21:52:49.792 INFO:tasks.workunit.client.1.smithi138.stdout:0/246: write d3/d10/d38/f46 [251945,77334] 0 2024-05-17T21:52:49.792 INFO:tasks.workunit.client.0.smithi107.stdout:1/341: rmdir d4/d57 0 2024-05-17T21:52:49.793 INFO:tasks.workunit.client.0.smithi107.stdout:5/316: dread d5/f9 [0,4194304] 0 2024-05-17T21:52:49.793 INFO:tasks.workunit.client.0.smithi107.stdout:1/342: fsync d4/f38 0 2024-05-17T21:52:49.794 INFO:tasks.workunit.client.1.smithi138.stdout:9/163: fdatasync d6/d7/d16/f1a 0 2024-05-17T21:52:49.795 INFO:tasks.workunit.client.0.smithi107.stdout:6/272: truncate d9/f15 3351482 0 2024-05-17T21:52:49.796 INFO:tasks.workunit.client.0.smithi107.stdout:6/273: truncate d9/f34 4410632 0 2024-05-17T21:52:49.797 INFO:tasks.workunit.client.0.smithi107.stdout:6/274: chown d9/d16/d22/d43 233192 1 2024-05-17T21:52:49.798 INFO:tasks.workunit.client.0.smithi107.stdout:6/275: write d9/d16/d22/d2d/f3d [101372,118095] 0 2024-05-17T21:52:49.798 INFO:tasks.workunit.client.0.smithi107.stdout:3/319: creat d6/d2a/d2d/d48/d4f/f67 x:0 0 0 2024-05-17T21:52:49.799 INFO:tasks.workunit.client.1.smithi138.stdout:3/203: creat d1/d3/d16/d3e/f4a x:0 0 0 2024-05-17T21:52:49.801 INFO:tasks.workunit.client.0.smithi107.stdout:3/320: truncate d6/d1a/f4e 860889 0 2024-05-17T21:52:49.801 INFO:tasks.workunit.client.0.smithi107.stdout:8/307: chown dc/d10/c1e 9075017 1 2024-05-17T21:52:49.806 INFO:tasks.workunit.client.0.smithi107.stdout:0/382: creat d3/dc/d22/d23/f8f x:0 0 0 2024-05-17T21:52:49.806 INFO:tasks.workunit.client.0.smithi107.stdout:0/383: chown d3/d24/d6f/l81 342222630 1 2024-05-17T21:52:49.807 INFO:tasks.workunit.client.1.smithi138.stdout:6/227: link d7/db/d43/f35 d7/db/d43/d3f/f4a 0 2024-05-17T21:52:49.807 INFO:tasks.workunit.client.0.smithi107.stdout:2/248: write d0/fa [8068579,19111] 0 2024-05-17T21:52:49.808 INFO:tasks.workunit.client.0.smithi107.stdout:2/249: write d0/d9/f33 [372219,82990] 0 2024-05-17T21:52:49.808 INFO:tasks.workunit.client.1.smithi138.stdout:6/228: truncate d7/db/d43/d3f/f46 354421 0 2024-05-17T21:52:49.810 INFO:tasks.workunit.client.0.smithi107.stdout:7/286: mkdir d5/df/d11/d1c/d2f/d65 0 2024-05-17T21:52:49.812 INFO:tasks.workunit.client.0.smithi107.stdout:2/250: dread d0/f6 [0,4194304] 0 2024-05-17T21:52:49.812 INFO:tasks.workunit.client.1.smithi138.stdout:3/204: mkdir d1/d3/d49/d4b 0 2024-05-17T21:52:49.813 INFO:tasks.workunit.client.0.smithi107.stdout:5/317: symlink d5/db/d16/d2d/d32/l66 0 2024-05-17T21:52:49.815 INFO:tasks.workunit.client.1.smithi138.stdout:6/229: symlink d7/db/l4b 0 2024-05-17T21:52:49.816 INFO:tasks.workunit.client.0.smithi107.stdout:1/343: mkdir d4/df/d1a/d26/d72 0 2024-05-17T21:52:49.817 INFO:tasks.workunit.client.0.smithi107.stdout:1/344: read - d4/df/d14/d2e/d43/f5f zero size 2024-05-17T21:52:49.820 INFO:tasks.workunit.client.0.smithi107.stdout:6/276: mknod d9/d16/d22/d2d/c51 0 2024-05-17T21:52:49.822 INFO:tasks.workunit.client.1.smithi138.stdout:3/205: dwrite d1/d3/d16/f29 [0,4194304] 0 2024-05-17T21:52:49.822 INFO:tasks.workunit.client.0.smithi107.stdout:3/321: unlink d6/f1b 0 2024-05-17T21:52:49.823 INFO:tasks.workunit.client.1.smithi138.stdout:0/247: link d3/d3d/d48/l50 d3/l60 0 2024-05-17T21:52:49.823 INFO:tasks.workunit.client.0.smithi107.stdout:1/345: dwrite d4/df/d1a/d26/f61 [0,4194304] 0 2024-05-17T21:52:49.824 INFO:tasks.workunit.client.1.smithi138.stdout:4/148: dwrite d0/d1b/f18 [0,4194304] 0 2024-05-17T21:52:49.824 INFO:tasks.workunit.client.1.smithi138.stdout:6/230: dread d7/db/d12/f26 [0,4194304] 0 2024-05-17T21:52:49.825 INFO:tasks.workunit.client.1.smithi138.stdout:3/206: write d1/d3/d8/de/f27 [2069705,18583] 0 2024-05-17T21:52:49.831 INFO:tasks.workunit.client.0.smithi107.stdout:0/384: write d3/d24/f2a [2851539,81194] 0 2024-05-17T21:52:49.832 INFO:tasks.workunit.client.0.smithi107.stdout:0/385: fdatasync d3/dc/d22/d3d/d56/f5b 0 2024-05-17T21:52:49.836 INFO:tasks.workunit.client.1.smithi138.stdout:6/231: dwrite d7/db/d43/d3f/f46 [0,4194304] 0 2024-05-17T21:52:49.839 INFO:tasks.workunit.client.1.smithi138.stdout:0/248: mknod d3/d10/d39/c61 0 2024-05-17T21:52:49.840 INFO:tasks.workunit.client.1.smithi138.stdout:0/249: chown d3/d3d/f58 14442139 1 2024-05-17T21:52:49.845 INFO:tasks.workunit.client.1.smithi138.stdout:4/149: creat d0/d3/d10/f3c x:0 0 0 2024-05-17T21:52:49.849 INFO:tasks.workunit.client.0.smithi107.stdout:5/318: symlink d5/db/d16/d2d/d40/l67 0 2024-05-17T21:52:49.850 INFO:tasks.workunit.client.0.smithi107.stdout:4/305: getdents d3/d6/d26 0 2024-05-17T21:52:49.851 INFO:tasks.workunit.client.0.smithi107.stdout:4/306: write d3/d6/d26/f2f [244893,25598] 0 2024-05-17T21:52:49.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:49 smithi107 ceph-mon[28434]: from='mgr.14636 172.21.15.138:0/3120106723' entity='mgr.smithi138.napnll' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:52:49.853 INFO:tasks.workunit.client.0.smithi107.stdout:8/308: write dc/d21/d27/f37 [3821997,56253] 0 2024-05-17T21:52:49.854 INFO:tasks.workunit.client.1.smithi138.stdout:4/150: rmdir d0/d3 39 2024-05-17T21:52:49.855 INFO:tasks.workunit.client.0.smithi107.stdout:1/346: creat d4/df/d4e/d59/f73 x:0 0 0 2024-05-17T21:52:49.857 INFO:tasks.workunit.client.0.smithi107.stdout:9/335: getdents d0/dd 0 2024-05-17T21:52:49.857 INFO:tasks.workunit.client.0.smithi107.stdout:0/386: symlink d3/d39/l90 0 2024-05-17T21:52:49.859 INFO:tasks.workunit.client.1.smithi138.stdout:3/207: link d1/lf d1/d2e/l4c 0 2024-05-17T21:52:49.860 INFO:tasks.workunit.client.1.smithi138.stdout:3/208: readlink d1/d3/d8/de/l2a 0 2024-05-17T21:52:49.861 INFO:tasks.workunit.client.1.smithi138.stdout:3/209: write d1/d3/d8/f13 [4361413,130503] 0 2024-05-17T21:52:49.861 INFO:tasks.workunit.client.0.smithi107.stdout:7/287: rename d5/df/l34 to d5/d16/d3b/d45/l66 0 2024-05-17T21:52:49.862 INFO:tasks.workunit.client.1.smithi138.stdout:0/250: creat d3/d10/f62 x:0 0 0 2024-05-17T21:52:49.863 INFO:tasks.workunit.client.0.smithi107.stdout:2/251: symlink d0/d9/l4a 0 2024-05-17T21:52:49.865 INFO:tasks.workunit.client.0.smithi107.stdout:7/288: dread d5/df/d11/d1c/f4e [0,4194304] 0 2024-05-17T21:52:49.867 INFO:tasks.workunit.client.0.smithi107.stdout:4/307: creat d3/db/d1e/f6c x:0 0 0 2024-05-17T21:52:49.867 INFO:tasks.workunit.client.1.smithi138.stdout:6/232: write d7/fc [1585412,21903] 0 2024-05-17T21:52:49.870 INFO:tasks.workunit.client.0.smithi107.stdout:6/277: link d9/d16/f1e d9/d16/d4d/f52 0 2024-05-17T21:52:49.870 INFO:tasks.workunit.client.1.smithi138.stdout:0/251: symlink d3/d10/d38/l63 0 2024-05-17T21:52:49.871 INFO:tasks.workunit.client.1.smithi138.stdout:0/252: write d3/d3d/d48/f5f [218366,124496] 0 2024-05-17T21:52:49.873 INFO:tasks.workunit.client.1.smithi138.stdout:3/210: mkdir d1/d4d 0 2024-05-17T21:52:49.875 INFO:tasks.workunit.client.0.smithi107.stdout:9/336: mkdir d0/dd/d21/d29/d3e/d45/d68/d75 0 2024-05-17T21:52:49.875 INFO:tasks.workunit.client.1.smithi138.stdout:3/211: truncate d1/d3/d16/d3e/f4a 681616 0 2024-05-17T21:52:49.875 INFO:tasks.workunit.client.0.smithi107.stdout:7/289: dwrite d5/df/d11/d1d/d44/f4b [4194304,4194304] 0 2024-05-17T21:52:49.876 INFO:tasks.workunit.client.1.smithi138.stdout:6/233: creat d7/db/d43/d3f/f4c x:0 0 0 2024-05-17T21:52:49.876 INFO:tasks.workunit.client.0.smithi107.stdout:4/308: dwrite d3/f2c [0,4194304] 0 2024-05-17T21:52:49.877 INFO:tasks.workunit.client.0.smithi107.stdout:9/337: write d0/dd/d1b/f65 [1015046,60311] 0 2024-05-17T21:52:49.878 INFO:tasks.workunit.client.0.smithi107.stdout:4/309: dread - d3/db/d16/f61 zero size 2024-05-17T21:52:49.878 INFO:tasks.workunit.client.0.smithi107.stdout:0/387: mkdir d3/dc/d6b/d77/d91 0 2024-05-17T21:52:49.880 INFO:tasks.workunit.client.0.smithi107.stdout:0/388: chown d3/dc/d31/l45 80366098 1 2024-05-17T21:52:49.883 INFO:tasks.workunit.client.0.smithi107.stdout:1/347: rename d4/df/d4e/d59/c6d to d4/df/d4e/c74 0 2024-05-17T21:52:49.884 INFO:tasks.workunit.client.1.smithi138.stdout:3/212: creat d1/d3/d16/d3e/f4e x:0 0 0 2024-05-17T21:52:49.884 INFO:tasks.workunit.client.1.smithi138.stdout:6/234: dwrite d7/db/d43/f39 [0,4194304] 0 2024-05-17T21:52:49.884 INFO:tasks.workunit.client.0.smithi107.stdout:2/252: dread d0/fa [0,4194304] 0 2024-05-17T21:52:49.886 INFO:tasks.workunit.client.1.smithi138.stdout:4/151: dread d0/d3/f16 [0,4194304] 0 2024-05-17T21:52:49.886 INFO:tasks.workunit.client.1.smithi138.stdout:6/235: truncate d7/d8/f42 279776 0 2024-05-17T21:52:49.887 INFO:tasks.workunit.client.0.smithi107.stdout:1/348: dread d4/f38 [0,4194304] 0 2024-05-17T21:52:49.887 INFO:tasks.workunit.client.1.smithi138.stdout:6/236: chown d7/l49 227810 1 2024-05-17T21:52:49.889 INFO:tasks.workunit.client.0.smithi107.stdout:1/349: write d4/df/d1a/f28 [244082,80897] 0 2024-05-17T21:52:49.890 INFO:tasks.workunit.client.0.smithi107.stdout:5/319: mknod d5/db/c68 0 2024-05-17T21:52:49.892 INFO:tasks.workunit.client.0.smithi107.stdout:8/309: write dc/d21/d27/d2f/f32 [424008,51016] 0 2024-05-17T21:52:49.892 INFO:tasks.workunit.client.0.smithi107.stdout:8/310: chown dc/d44 99193 1 2024-05-17T21:52:49.894 INFO:tasks.workunit.client.1.smithi138.stdout:4/152: creat d0/d1b/f3d x:0 0 0 2024-05-17T21:52:49.895 INFO:tasks.workunit.client.1.smithi138.stdout:0/253: link d3/d10/f2d d3/d3d/d51/f64 0 2024-05-17T21:52:49.896 INFO:tasks.workunit.client.0.smithi107.stdout:6/278: write d9/d16/f41 [713740,78012] 0 2024-05-17T21:52:49.898 INFO:tasks.workunit.client.0.smithi107.stdout:9/338: write d0/dd/d1b/f53 [453208,4655] 0 2024-05-17T21:52:49.899 INFO:tasks.workunit.client.1.smithi138.stdout:6/237: link d7/db/d12/f26 d7/db/f4d 0 2024-05-17T21:52:49.901 INFO:tasks.workunit.client.0.smithi107.stdout:4/310: unlink d3/d4a/d47/d60/d3c/d58/c64 0 2024-05-17T21:52:49.904 INFO:tasks.workunit.client.0.smithi107.stdout:7/290: dwrite d5/df/f26 [0,4194304] 0 2024-05-17T21:52:49.904 INFO:tasks.workunit.client.1.smithi138.stdout:0/254: creat d3/d10/d38/d3e/f65 x:0 0 0 2024-05-17T21:52:49.904 INFO:tasks.workunit.client.1.smithi138.stdout:4/153: dread d0/d3/db/f20 [0,4194304] 0 2024-05-17T21:52:49.907 INFO:tasks.workunit.client.1.smithi138.stdout:3/213: dread d1/d3/d8/de/f27 [0,4194304] 0 2024-05-17T21:52:49.907 INFO:tasks.workunit.client.0.smithi107.stdout:2/253: truncate d0/f6 4109202 0 2024-05-17T21:52:49.908 INFO:tasks.workunit.client.1.smithi138.stdout:3/214: truncate d1/d3/d8/f34 814130 0 2024-05-17T21:52:49.908 INFO:tasks.workunit.client.0.smithi107.stdout:4/311: dwrite d3/d6/d26/f2f [0,4194304] 0 2024-05-17T21:52:49.911 INFO:tasks.workunit.client.1.smithi138.stdout:4/154: rename d0/d3/d17/f22 to d0/d1b/f3e 0 2024-05-17T21:52:49.912 INFO:tasks.workunit.client.1.smithi138.stdout:4/155: chown d0/d3/db/f35 7270221 1 2024-05-17T21:52:49.912 INFO:tasks.workunit.client.0.smithi107.stdout:4/312: dread d3/db/fc [0,4194304] 0 2024-05-17T21:52:49.912 INFO:tasks.workunit.client.0.smithi107.stdout:1/350: symlink d4/df/d69/l75 0 2024-05-17T21:52:49.913 INFO:tasks.workunit.client.0.smithi107.stdout:4/313: write d3/f2c [1276529,56377] 0 2024-05-17T21:52:49.914 INFO:tasks.workunit.client.0.smithi107.stdout:8/311: mknod dc/d21/d27/d34/d4c/c7b 0 2024-05-17T21:52:49.914 INFO:tasks.workunit.client.0.smithi107.stdout:4/314: fdatasync d3/f22 0 2024-05-17T21:52:49.918 INFO:tasks.workunit.client.1.smithi138.stdout:4/156: rename d0/c3a to d0/d3/db/c3f 0 2024-05-17T21:52:49.919 INFO:tasks.workunit.client.1.smithi138.stdout:4/157: write d0/f2a [536939,83124] 0 2024-05-17T21:52:49.920 INFO:tasks.workunit.client.0.smithi107.stdout:7/291: mkdir d5/df/d11/d1c/d2f/d67 0 2024-05-17T21:52:49.921 INFO:tasks.workunit.client.0.smithi107.stdout:7/292: write d5/df/f1a [4662231,126844] 0 2024-05-17T21:52:49.924 INFO:tasks.workunit.client.1.smithi138.stdout:3/215: getdents d1/d3/d8 0 2024-05-17T21:52:49.925 INFO:tasks.workunit.client.0.smithi107.stdout:7/293: dread d5/d16/f1e [0,4194304] 0 2024-05-17T21:52:49.926 INFO:tasks.workunit.client.1.smithi138.stdout:4/158: dwrite d0/d1b/f3d [0,4194304] 0 2024-05-17T21:52:49.927 INFO:tasks.workunit.client.0.smithi107.stdout:9/339: symlink d0/dd/d2f/l76 0 2024-05-17T21:52:49.928 INFO:tasks.workunit.client.1.smithi138.stdout:3/216: mkdir d1/d4f 0 2024-05-17T21:52:49.930 INFO:tasks.workunit.client.1.smithi138.stdout:3/217: write d1/d3/d8/f13 [2353103,66415] 0 2024-05-17T21:52:49.931 INFO:tasks.workunit.client.0.smithi107.stdout:4/315: rename d3/d6/d26/d3b/l52 to d3/d4a/d47/d60/d3a/l6d 0 2024-05-17T21:52:49.932 INFO:tasks.workunit.client.1.smithi138.stdout:3/218: dread - d1/d3/d16/d3e/f4e zero size 2024-05-17T21:52:49.934 INFO:tasks.workunit.client.0.smithi107.stdout:8/312: dwrite dc/d10/d63/f79 [4194304,4194304] 0 2024-05-17T21:52:49.935 INFO:tasks.workunit.client.0.smithi107.stdout:3/322: sync 2024-05-17T21:52:49.936 INFO:tasks.workunit.client.1.smithi138.stdout:1/153: sync 2024-05-17T21:52:49.936 INFO:tasks.workunit.client.1.smithi138.stdout:7/116: sync 2024-05-17T21:52:49.936 INFO:tasks.workunit.client.1.smithi138.stdout:8/90: sync 2024-05-17T21:52:49.937 INFO:tasks.workunit.client.0.smithi107.stdout:9/340: dwrite d0/dd/d2f/f31 [0,4194304] 0 2024-05-17T21:52:49.938 INFO:tasks.workunit.client.1.smithi138.stdout:3/219: mkdir d1/d4f/d50 0 2024-05-17T21:52:49.938 INFO:tasks.workunit.client.0.smithi107.stdout:7/294: mknod d5/df/d11/d1c/d2f/d53/c68 0 2024-05-17T21:52:49.939 INFO:tasks.workunit.client.1.smithi138.stdout:1/154: symlink d4/db/l32 0 2024-05-17T21:52:49.940 INFO:tasks.workunit.client.0.smithi107.stdout:3/323: write d6/d2a/f4d [848493,82520] 0 2024-05-17T21:52:49.940 INFO:tasks.workunit.client.1.smithi138.stdout:1/155: read - d4/dd/f15 zero size 2024-05-17T21:52:49.942 INFO:tasks.workunit.client.0.smithi107.stdout:0/389: getdents d3/dc/d22/d3d/d56 0 2024-05-17T21:52:49.943 INFO:tasks.workunit.client.0.smithi107.stdout:3/324: rename d6/d8/d3e to d6/d8/d3e/d3f/d68 22 2024-05-17T21:52:49.944 INFO:tasks.workunit.client.0.smithi107.stdout:2/254: link d0/d9/d11/c13 d0/d9/d11/d15/c4b 0 2024-05-17T21:52:49.945 INFO:tasks.workunit.client.0.smithi107.stdout:3/325: dread d6/d8/f12 [0,4194304] 0 2024-05-17T21:52:49.951 INFO:tasks.workunit.client.1.smithi138.stdout:0/255: dread d3/d10/f2c [0,4194304] 0 2024-05-17T21:52:49.952 INFO:tasks.workunit.client.1.smithi138.stdout:1/156: symlink d4/db/l33 0 2024-05-17T21:52:49.952 INFO:tasks.workunit.client.1.smithi138.stdout:1/157: chown d4/db/l33 26 1 2024-05-17T21:52:49.953 INFO:tasks.workunit.client.1.smithi138.stdout:6/238: write d7/f28 [1731378,101815] 0 2024-05-17T21:52:49.953 INFO:tasks.workunit.client.0.smithi107.stdout:8/313: chown dc/d21/d35/f2d 2 1 2024-05-17T21:52:49.954 INFO:tasks.workunit.client.0.smithi107.stdout:8/314: write dc/d21/d27/d34/f6f [534947,91658] 0 2024-05-17T21:52:49.955 INFO:tasks.workunit.client.0.smithi107.stdout:9/341: symlink d0/dd/d21/d29/d57/l77 0 2024-05-17T21:52:49.956 INFO:tasks.workunit.client.0.smithi107.stdout:8/315: write dc/f62 [882056,27790] 0 2024-05-17T21:52:49.960 INFO:tasks.workunit.client.0.smithi107.stdout:5/320: write d5/db/d16/d2d/d32/f38 [161443,41129] 0 2024-05-17T21:52:49.960 INFO:tasks.workunit.client.0.smithi107.stdout:0/390: truncate d3/f1e 869752 0 2024-05-17T21:52:49.961 INFO:tasks.workunit.client.1.smithi138.stdout:1/158: fdatasync d4/dd/f20 0 2024-05-17T21:52:49.961 INFO:tasks.workunit.client.1.smithi138.stdout:6/239: dwrite d7/f28 [0,4194304] 0 2024-05-17T21:52:49.961 INFO:tasks.workunit.client.0.smithi107.stdout:0/391: write d3/f1d [3604076,23861] 0 2024-05-17T21:52:49.962 INFO:tasks.workunit.client.1.smithi138.stdout:1/159: read - d4/dd/f20 zero size 2024-05-17T21:52:49.962 INFO:tasks.workunit.client.0.smithi107.stdout:5/321: write d5/db/d16/d5b/f53 [1083462,97247] 0 2024-05-17T21:52:49.963 INFO:tasks.workunit.client.1.smithi138.stdout:1/160: chown d4/db/l1e 6911 1 2024-05-17T21:52:49.964 INFO:tasks.workunit.client.0.smithi107.stdout:0/392: write d3/dc/f16 [4342809,37936] 0 2024-05-17T21:52:49.964 INFO:tasks.workunit.client.1.smithi138.stdout:1/161: truncate d4/db/f22 962343 0 2024-05-17T21:52:49.965 INFO:tasks.workunit.client.0.smithi107.stdout:0/393: stat d3/dc/d22 0 2024-05-17T21:52:49.966 INFO:tasks.workunit.client.0.smithi107.stdout:3/326: rmdir d6/d8/d10 39 2024-05-17T21:52:49.969 INFO:tasks.workunit.client.0.smithi107.stdout:1/351: getdents d4/df/d4e 0 2024-05-17T21:52:49.970 INFO:tasks.workunit.client.1.smithi138.stdout:6/240: symlink d7/db/d34/l4e 0 2024-05-17T21:52:49.971 INFO:tasks.workunit.client.0.smithi107.stdout:6/279: truncate d9/f12 3532253 0 2024-05-17T21:52:49.972 INFO:tasks.workunit.client.1.smithi138.stdout:8/91: dread d0/fb [0,4194304] 0 2024-05-17T21:52:49.972 INFO:tasks.workunit.client.1.smithi138.stdout:6/241: truncate d7/d8/f2b 777811 0 2024-05-17T21:52:49.973 INFO:tasks.workunit.client.0.smithi107.stdout:6/280: truncate d9/d16/d22/d2d/f31 1041471 0 2024-05-17T21:52:49.973 INFO:tasks.workunit.client.0.smithi107.stdout:6/281: stat d9/f34 0 2024-05-17T21:52:49.973 INFO:tasks.workunit.client.0.smithi107.stdout:6/282: chown d9/fa 7282858 1 2024-05-17T21:52:49.974 INFO:tasks.workunit.client.1.smithi138.stdout:1/162: unlink d4/f16 0 2024-05-17T21:52:49.975 INFO:tasks.workunit.client.1.smithi138.stdout:6/242: dread - d7/db/d12/f3e zero size 2024-05-17T21:52:49.975 INFO:tasks.workunit.client.0.smithi107.stdout:7/295: creat d5/df/d11/d1c/d2f/d65/f69 x:0 0 0 2024-05-17T21:52:49.976 INFO:tasks.workunit.client.1.smithi138.stdout:7/117: getdents d0/d18 0 2024-05-17T21:52:49.977 INFO:tasks.workunit.client.0.smithi107.stdout:8/316: readlink dc/d10/l57 0 2024-05-17T21:52:49.978 INFO:tasks.workunit.client.1.smithi138.stdout:4/159: dwrite d0/d1b/f3d [4194304,4194304] 0 2024-05-17T21:52:49.979 INFO:tasks.workunit.client.0.smithi107.stdout:6/283: dwrite d9/f34 [0,4194304] 0 2024-05-17T21:52:49.982 INFO:tasks.workunit.client.0.smithi107.stdout:9/342: rename d0/dd/d1b/f48 to d0/dd/f78 0 2024-05-17T21:52:49.984 INFO:tasks.workunit.client.1.smithi138.stdout:8/92: dwrite d0/fb [0,4194304] 0 2024-05-17T21:52:49.984 INFO:tasks.workunit.client.1.smithi138.stdout:0/256: dread f2 [0,4194304] 0 2024-05-17T21:52:49.995 INFO:tasks.workunit.client.0.smithi107.stdout:8/317: dread dc/d21/d27/d2f/f32 [0,4194304] 0 2024-05-17T21:52:49.996 INFO:tasks.workunit.client.0.smithi107.stdout:8/318: chown dc/d21/d27/d34/d4c/l6c 250 1 2024-05-17T21:52:50.057 INFO:tasks.workunit.client.0.smithi107.stdout:4/316: sync 2024-05-17T21:52:50.058 INFO:tasks.workunit.client.0.smithi107.stdout:4/317: write d3/d4a/d47/d60/d3a/d41/f5a [795077,34237] 0 2024-05-17T21:52:50.084 INFO:tasks.workunit.client.1.smithi138.stdout:7/118: mknod d0/d18/c25 0 2024-05-17T21:52:50.087 INFO:tasks.workunit.client.1.smithi138.stdout:0/257: rmdir d3/d10/d39 39 2024-05-17T21:52:50.088 INFO:tasks.workunit.client.1.smithi138.stdout:3/220: write d1/d3/d8/f24 [247149,15672] 0 2024-05-17T21:52:50.089 INFO:tasks.workunit.client.1.smithi138.stdout:8/93: fsync d0/dc/f18 0 2024-05-17T21:52:50.090 INFO:tasks.workunit.client.0.smithi107.stdout:5/322: write d5/db/d16/d2a/f64 [545980,27320] 0 2024-05-17T21:52:50.092 INFO:tasks.workunit.client.0.smithi107.stdout:0/394: rmdir d3/dc/d22/d3d/d56 39 2024-05-17T21:52:50.095 INFO:tasks.workunit.client.0.smithi107.stdout:3/327: mkdir d6/d2a/d2d/d69 0 2024-05-17T21:52:50.099 INFO:tasks.workunit.client.1.smithi138.stdout:3/221: rename c0 to d1/d3/d16/d3a/c51 0 2024-05-17T21:52:50.100 INFO:tasks.workunit.client.0.smithi107.stdout:7/296: creat d5/df/d11/d1d/d33/f6a x:0 0 0 2024-05-17T21:52:50.107 INFO:tasks.workunit.client.1.smithi138.stdout:4/160: creat d0/d3/f40 x:0 0 0 2024-05-17T21:52:50.108 INFO:tasks.workunit.client.1.smithi138.stdout:4/161: write d0/d1b/f3d [7883869,23679] 0 2024-05-17T21:52:50.111 INFO:tasks.workunit.client.1.smithi138.stdout:1/163: getdents d4/dd 0 2024-05-17T21:52:50.113 INFO:tasks.workunit.client.0.smithi107.stdout:4/318: creat d3/d6/d26/d36/f6e x:0 0 0 2024-05-17T21:52:50.115 INFO:tasks.workunit.client.1.smithi138.stdout:4/162: dwrite d0/f2a [0,4194304] 0 2024-05-17T21:52:50.115 INFO:tasks.workunit.client.1.smithi138.stdout:8/94: rename d0/f4 to d0/d11/f1c 0 2024-05-17T21:52:50.116 INFO:tasks.workunit.client.1.smithi138.stdout:4/163: fsync d0/f34 0 2024-05-17T21:52:50.117 INFO:tasks.workunit.client.1.smithi138.stdout:7/119: write d0/f1a [1957824,87620] 0 2024-05-17T21:52:50.122 INFO:tasks.workunit.client.0.smithi107.stdout:3/328: creat d6/d2a/f6a x:0 0 0 2024-05-17T21:52:50.122 INFO:tasks.workunit.client.0.smithi107.stdout:3/329: chown d6/d2a 80836798 1 2024-05-17T21:52:50.123 INFO:tasks.workunit.client.0.smithi107.stdout:3/330: chown d6/d1a/d57/f5e 534 1 2024-05-17T21:52:50.124 INFO:tasks.workunit.client.1.smithi138.stdout:7/120: dwrite d0/f13 [0,4194304] 0 2024-05-17T21:52:50.127 INFO:tasks.workunit.client.1.smithi138.stdout:8/95: rmdir d0/dc 39 2024-05-17T21:52:50.128 INFO:tasks.workunit.client.1.smithi138.stdout:8/96: chown d0/f9 26997 1 2024-05-17T21:52:50.129 INFO:tasks.workunit.client.0.smithi107.stdout:9/343: read d0/dd/d21/d29/d3e/f5e [3021964,16834] 0 2024-05-17T21:52:50.130 INFO:tasks.workunit.client.1.smithi138.stdout:4/164: unlink d0/d3/db/ce 0 2024-05-17T21:52:50.131 INFO:tasks.workunit.client.1.smithi138.stdout:7/121: dwrite d0/f24 [0,4194304] 0 2024-05-17T21:52:50.132 INFO:tasks.workunit.client.1.smithi138.stdout:7/122: chown d0/le 99163 1 2024-05-17T21:52:50.134 INFO:tasks.workunit.client.1.smithi138.stdout:7/123: write d0/f1f [5287449,81051] 0 2024-05-17T21:52:50.135 INFO:tasks.workunit.client.0.smithi107.stdout:9/344: dread d0/f43 [0,4194304] 0 2024-05-17T21:52:50.135 INFO:tasks.workunit.client.0.smithi107.stdout:9/345: chown d0/dd/d21/d29/d3e/d45/c4e 23512 1 2024-05-17T21:52:50.135 INFO:tasks.workunit.client.0.smithi107.stdout:4/319: unlink d3/db/d16/f30 0 2024-05-17T21:52:50.136 INFO:tasks.workunit.client.1.smithi138.stdout:7/124: truncate d0/d18/f1d 622479 0 2024-05-17T21:52:50.136 INFO:tasks.workunit.client.0.smithi107.stdout:4/320: dread - d3/db/d16/d43/f62 zero size 2024-05-17T21:52:50.136 INFO:tasks.workunit.client.1.smithi138.stdout:1/164: mknod d4/d2e/c34 0 2024-05-17T21:52:50.139 INFO:tasks.workunit.client.0.smithi107.stdout:5/323: mknod d5/db/d16/d54/d59/d65/d48/c69 0 2024-05-17T21:52:50.142 INFO:tasks.workunit.client.1.smithi138.stdout:3/222: truncate d1/d3/f11 868915 0 2024-05-17T21:52:50.142 INFO:tasks.workunit.client.0.smithi107.stdout:0/395: creat d3/dc/d6b/d77/d91/f92 x:0 0 0 2024-05-17T21:52:50.143 INFO:tasks.workunit.client.1.smithi138.stdout:4/165: write d0/f7 [1077743,120452] 0 2024-05-17T21:52:50.145 INFO:tasks.workunit.client.0.smithi107.stdout:4/321: dwrite d3/f68 [0,4194304] 0 2024-05-17T21:52:50.148 INFO:tasks.workunit.client.0.smithi107.stdout:3/331: chown d6/d8/d10/d35/d36/d66 7891 1 2024-05-17T21:52:50.149 INFO:tasks.workunit.client.1.smithi138.stdout:1/165: mkdir d4/df/d35 0 2024-05-17T21:52:50.149 INFO:tasks.workunit.client.0.smithi107.stdout:5/324: dwrite d5/db/d16/d5b/f3d [4194304,4194304] 0 2024-05-17T21:52:50.153 INFO:tasks.workunit.client.0.smithi107.stdout:1/352: rename d4/c6 to d4/df/d14/d2e/d43/c76 0 2024-05-17T21:52:50.156 INFO:tasks.workunit.client.0.smithi107.stdout:1/353: fdatasync d4/df/d14/d1d/f2c 0 2024-05-17T21:52:50.157 INFO:tasks.workunit.client.0.smithi107.stdout:3/332: dread d6/d8/f14 [0,4194304] 0 2024-05-17T21:52:50.159 INFO:tasks.workunit.client.1.smithi138.stdout:8/97: creat d0/dc/d1a/f1d x:0 0 0 2024-05-17T21:52:50.160 INFO:tasks.workunit.client.1.smithi138.stdout:4/166: dread d0/d3/f12 [0,4194304] 0 2024-05-17T21:52:50.160 INFO:tasks.workunit.client.0.smithi107.stdout:2/255: dwrite d0/f25 [0,4194304] 0 2024-05-17T21:52:50.161 INFO:tasks.workunit.client.0.smithi107.stdout:0/396: rmdir d3/dc/d31 39 2024-05-17T21:52:50.161 INFO:tasks.workunit.client.1.smithi138.stdout:3/223: mknod d1/d3/d49/d4b/c52 0 2024-05-17T21:52:50.162 INFO:tasks.workunit.client.0.smithi107.stdout:2/256: fdatasync d0/f40 0 2024-05-17T21:52:50.163 INFO:tasks.workunit.client.0.smithi107.stdout:4/322: mknod d3/d4a/d47/d60/d3a/d41/c6f 0 2024-05-17T21:52:50.163 INFO:tasks.workunit.client.0.smithi107.stdout:0/397: fsync d3/dc/d22/d23/d6c/f7c 0 2024-05-17T21:52:50.164 INFO:tasks.workunit.client.0.smithi107.stdout:6/284: getdents d9/d16/d4d 0 2024-05-17T21:52:50.165 INFO:tasks.workunit.client.1.smithi138.stdout:8/98: unlink d0/d11/l15 0 2024-05-17T21:52:50.169 INFO:tasks.workunit.client.1.smithi138.stdout:4/167: rename d0/d3/d10/d1f/f39 to d0/d3/d10/f41 0 2024-05-17T21:52:50.174 INFO:tasks.workunit.client.1.smithi138.stdout:7/125: getdents d0 0 2024-05-17T21:52:50.175 INFO:tasks.workunit.client.0.smithi107.stdout:8/319: rename dc/d10/d17/f1b to dc/d21/d68/f7c 0 2024-05-17T21:52:50.176 INFO:tasks.workunit.client.1.smithi138.stdout:8/99: write d0/dc/fd [1756452,22593] 0 2024-05-17T21:52:50.178 INFO:tasks.workunit.client.1.smithi138.stdout:4/168: dwrite d0/d3/d10/f3c [0,4194304] 0 2024-05-17T21:52:50.179 INFO:tasks.workunit.client.0.smithi107.stdout:8/320: read dc/f43 [179976,104706] 0 2024-05-17T21:52:50.184 INFO:tasks.workunit.client.1.smithi138.stdout:7/126: symlink d0/l26 0 2024-05-17T21:52:50.184 INFO:tasks.workunit.client.1.smithi138.stdout:1/166: write d4/db/f21 [1475604,108810] 0 2024-05-17T21:52:50.186 INFO:tasks.workunit.client.0.smithi107.stdout:2/257: mknod d0/d26/c4c 0 2024-05-17T21:52:50.186 INFO:tasks.workunit.client.0.smithi107.stdout:1/354: dread d4/fa [0,4194304] 0 2024-05-17T21:52:50.186 INFO:tasks.workunit.client.1.smithi138.stdout:3/224: dread d1/d3/f1c [0,4194304] 0 2024-05-17T21:52:50.186 INFO:tasks.workunit.client.1.smithi138.stdout:7/127: write d0/f1e [109049,37472] 0 2024-05-17T21:52:50.187 INFO:tasks.workunit.client.1.smithi138.stdout:3/225: chown d1/d3/d8 999 1 2024-05-17T21:52:50.188 INFO:tasks.workunit.client.1.smithi138.stdout:3/226: chown d1/d3/d8 185045858 1 2024-05-17T21:52:50.190 INFO:tasks.workunit.client.1.smithi138.stdout:4/169: fsync d0/d3/d10/f25 0 2024-05-17T21:52:50.191 INFO:tasks.workunit.client.1.smithi138.stdout:4/170: write d0/f7 [4464932,94785] 0 2024-05-17T21:52:50.194 INFO:tasks.workunit.client.0.smithi107.stdout:0/398: rmdir d3/dc/d22/d3d 39 2024-05-17T21:52:50.195 INFO:tasks.workunit.client.0.smithi107.stdout:7/297: sync 2024-05-17T21:52:50.195 INFO:tasks.workunit.client.0.smithi107.stdout:7/298: fdatasync d5/d16/f28 0 2024-05-17T21:52:50.197 INFO:tasks.workunit.client.0.smithi107.stdout:7/299: read d5/d16/f23 [701272,71282] 0 2024-05-17T21:52:50.202 INFO:tasks.workunit.client.0.smithi107.stdout:8/321: mknod dc/d10/d17/c7d 0 2024-05-17T21:52:50.203 INFO:tasks.workunit.client.1.smithi138.stdout:1/167: dread d4/df/f12 [0,4194304] 0 2024-05-17T21:52:50.204 INFO:tasks.workunit.client.1.smithi138.stdout:4/171: mkdir d0/d1b/d42 0 2024-05-17T21:52:50.204 INFO:tasks.workunit.client.1.smithi138.stdout:4/172: chown d0/c13 105 1 2024-05-17T21:52:50.204 INFO:tasks.workunit.client.0.smithi107.stdout:4/323: write d3/db/d16/f45 [1443336,87985] 0 2024-05-17T21:52:50.206 INFO:tasks.workunit.client.0.smithi107.stdout:4/324: write d3/fa [3129790,52666] 0 2024-05-17T21:52:50.206 INFO:tasks.workunit.client.0.smithi107.stdout:3/333: dwrite d6/d8/f12 [0,4194304] 0 2024-05-17T21:52:50.208 INFO:tasks.workunit.client.0.smithi107.stdout:0/399: creat d3/d58/d61/f93 x:0 0 0 2024-05-17T21:52:50.209 INFO:tasks.workunit.client.1.smithi138.stdout:3/227: unlink d1/d3/d16/d3e/f46 0 2024-05-17T21:52:50.212 INFO:tasks.workunit.client.1.smithi138.stdout:4/173: unlink d0/d1b/f3d 0 2024-05-17T21:52:50.212 INFO:tasks.workunit.client.1.smithi138.stdout:8/100: getdents d0/d11 0 2024-05-17T21:52:50.213 INFO:tasks.workunit.client.0.smithi107.stdout:4/325: dread d3/db/fc [0,4194304] 0 2024-05-17T21:52:50.215 INFO:tasks.workunit.client.0.smithi107.stdout:2/258: dwrite d0/f29 [0,4194304] 0 2024-05-17T21:52:50.216 INFO:tasks.workunit.client.0.smithi107.stdout:7/300: dread d5/f18 [0,4194304] 0 2024-05-17T21:52:50.218 INFO:tasks.workunit.client.0.smithi107.stdout:4/326: dread d3/f68 [0,4194304] 0 2024-05-17T21:52:50.219 INFO:tasks.workunit.client.0.smithi107.stdout:4/327: write d3/db/d16/d43/f53 [621003,63708] 0 2024-05-17T21:52:50.220 INFO:tasks.workunit.client.1.smithi138.stdout:7/128: getdents d0/d18 0 2024-05-17T21:52:50.221 INFO:tasks.workunit.client.1.smithi138.stdout:7/129: chown d0/lf 62559832 1 2024-05-17T21:52:50.222 INFO:tasks.workunit.client.1.smithi138.stdout:4/174: creat d0/d3/d10/d1f/d32/f43 x:0 0 0 2024-05-17T21:52:50.225 INFO:tasks.workunit.client.0.smithi107.stdout:0/400: mkdir d3/d58/d61/d94 0 2024-05-17T21:52:50.227 INFO:tasks.workunit.client.0.smithi107.stdout:3/334: truncate d6/d8/d10/f30 3848085 0 2024-05-17T21:52:50.230 INFO:tasks.workunit.client.1.smithi138.stdout:7/130: dwrite d0/d18/f1d [0,4194304] 0 2024-05-17T21:52:50.230 INFO:tasks.workunit.client.0.smithi107.stdout:0/401: dwrite d3/f4a [0,4194304] 0 2024-05-17T21:52:50.231 INFO:tasks.workunit.client.1.smithi138.stdout:7/131: stat d0/le 0 2024-05-17T21:52:50.231 INFO:tasks.workunit.client.0.smithi107.stdout:6/285: rename d9/f34 to d9/f53 0 2024-05-17T21:52:50.232 INFO:tasks.workunit.client.0.smithi107.stdout:0/402: write d3/f1d [1198849,45201] 0 2024-05-17T21:52:50.234 INFO:tasks.workunit.client.1.smithi138.stdout:4/175: mknod d0/d3/d10/d1f/d32/c44 0 2024-05-17T21:52:50.234 INFO:tasks.workunit.client.1.smithi138.stdout:4/176: readlink d0/l6 0 2024-05-17T21:52:50.235 INFO:tasks.workunit.client.1.smithi138.stdout:4/177: chown d0/d3/db/ld 10853040 1 2024-05-17T21:52:50.236 INFO:tasks.workunit.client.1.smithi138.stdout:4/178: write d0/f7 [4037806,113592] 0 2024-05-17T21:52:50.239 INFO:tasks.workunit.client.0.smithi107.stdout:5/325: getdents d5/db/d16/d54/d59 0 2024-05-17T21:52:50.240 INFO:tasks.workunit.client.1.smithi138.stdout:7/132: creat d0/f27 x:0 0 0 2024-05-17T21:52:50.241 INFO:tasks.workunit.client.1.smithi138.stdout:7/133: write d0/f11 [1443657,66750] 0 2024-05-17T21:52:50.242 INFO:tasks.workunit.client.0.smithi107.stdout:8/322: dwrite dc/f43 [0,4194304] 0 2024-05-17T21:52:50.243 INFO:tasks.workunit.client.1.smithi138.stdout:8/101: getdents d0 0 2024-05-17T21:52:50.244 INFO:tasks.workunit.client.0.smithi107.stdout:8/323: readlink dc/d21/l2b 0 2024-05-17T21:52:50.244 INFO:tasks.workunit.client.1.smithi138.stdout:8/102: write d0/fb [2765507,101358] 0 2024-05-17T21:52:50.251 INFO:tasks.workunit.client.0.smithi107.stdout:2/259: dwrite d0/d9/d11/f10 [0,4194304] 0 2024-05-17T21:52:50.252 INFO:tasks.workunit.client.1.smithi138.stdout:4/179: mkdir d0/d1b/d42/d45 0 2024-05-17T21:52:50.253 INFO:tasks.workunit.client.1.smithi138.stdout:4/180: write d0/f7 [3474103,40805] 0 2024-05-17T21:52:50.253 INFO:tasks.workunit.client.0.smithi107.stdout:3/335: mknod d6/d2a/d2d/d48/d4f/c6b 0 2024-05-17T21:52:50.260 INFO:tasks.workunit.client.1.smithi138.stdout:4/181: dwrite d0/d3/f40 [0,4194304] 0 2024-05-17T21:52:50.261 INFO:tasks.workunit.client.0.smithi107.stdout:9/346: link d0/dd/d1b/l4b d0/dd/d2f/l79 0 2024-05-17T21:52:50.262 INFO:tasks.workunit.client.0.smithi107.stdout:9/347: dread - d0/dd/d2f/f64 zero size 2024-05-17T21:52:50.262 INFO:tasks.workunit.client.1.smithi138.stdout:4/182: read d0/d3/f12 [2507041,71300] 0 2024-05-17T21:52:50.263 INFO:tasks.workunit.client.0.smithi107.stdout:8/324: dread dc/d21/d27/d2f/f32 [0,4194304] 0 2024-05-17T21:52:50.263 INFO:tasks.workunit.client.0.smithi107.stdout:7/301: symlink d5/df/d11/d1c/l6b 0 2024-05-17T21:52:50.265 INFO:tasks.workunit.client.0.smithi107.stdout:8/325: write dc/d21/d35/f36 [433869,117920] 0 2024-05-17T21:52:50.267 INFO:tasks.workunit.client.0.smithi107.stdout:4/328: link d3/f9 d3/db/d1e/d66/f70 0 2024-05-17T21:52:50.267 INFO:tasks.workunit.client.0.smithi107.stdout:7/302: fsync d5/df/d11/d1c/d2f/d65/f69 0 2024-05-17T21:52:50.269 INFO:tasks.workunit.client.1.smithi138.stdout:4/183: mknod d0/d1b/d42/c46 0 2024-05-17T21:52:50.270 INFO:tasks.workunit.client.0.smithi107.stdout:4/329: write d3/d4a/d47/f4b [1592445,1208] 0 2024-05-17T21:52:50.272 INFO:tasks.workunit.client.0.smithi107.stdout:3/336: creat d6/d2a/d2d/f6c x:0 0 0 2024-05-17T21:52:50.274 INFO:tasks.workunit.client.0.smithi107.stdout:9/348: dwrite d0/dd/d1b/f65 [0,4194304] 0 2024-05-17T21:52:50.275 INFO:tasks.workunit.client.1.smithi138.stdout:4/184: fdatasync d0/d3/d17/f1a 0 2024-05-17T21:52:50.276 INFO:tasks.workunit.client.0.smithi107.stdout:0/403: creat d3/d58/d61/d94/f95 x:0 0 0 2024-05-17T21:52:50.277 INFO:tasks.workunit.client.0.smithi107.stdout:9/349: readlink d0/dd/d2f/d4a/l54 0 2024-05-17T21:52:50.277 INFO:tasks.workunit.client.0.smithi107.stdout:4/330: chown d3/d4a/d47/d60/d3a/d41/f5a 351335 1 2024-05-17T21:52:50.278 INFO:tasks.workunit.client.0.smithi107.stdout:5/326: truncate d5/db/f1a 3887216 0 2024-05-17T21:52:50.282 INFO:tasks.workunit.client.1.smithi138.stdout:8/103: truncate d0/f9 35493 0 2024-05-17T21:52:50.284 INFO:tasks.workunit.client.1.smithi138.stdout:4/185: creat d0/d1b/d36/f47 x:0 0 0 2024-05-17T21:52:50.284 INFO:tasks.workunit.client.1.smithi138.stdout:7/134: dread d0/f5 [0,4194304] 0 2024-05-17T21:52:50.285 INFO:tasks.workunit.client.1.smithi138.stdout:7/135: chown d0/l14 1184994 1 2024-05-17T21:52:50.285 INFO:tasks.workunit.client.0.smithi107.stdout:8/326: rmdir dc/d44/d4b/d52 39 2024-05-17T21:52:50.286 INFO:tasks.workunit.client.1.smithi138.stdout:8/104: write d0/f17 [4107151,100462] 0 2024-05-17T21:52:50.286 INFO:tasks.workunit.client.0.smithi107.stdout:6/286: dread d9/f1a [0,4194304] 0 2024-05-17T21:52:50.287 INFO:tasks.workunit.client.1.smithi138.stdout:8/105: chown d0/l19 2 1 2024-05-17T21:52:50.288 INFO:tasks.workunit.client.1.smithi138.stdout:4/186: chown d0/d3/db/c3f 20 1 2024-05-17T21:52:50.289 INFO:tasks.workunit.client.0.smithi107.stdout:7/303: creat d5/d16/d3b/d45/f6c x:0 0 0 2024-05-17T21:52:50.290 INFO:tasks.workunit.client.1.smithi138.stdout:7/136: rmdir d0/d18 39 2024-05-17T21:52:50.291 INFO:tasks.workunit.client.1.smithi138.stdout:7/137: truncate d0/f1a 4198160 0 2024-05-17T21:52:50.292 INFO:tasks.workunit.client.1.smithi138.stdout:7/138: write d0/f13 [2407172,25268] 0 2024-05-17T21:52:50.292 INFO:tasks.workunit.client.1.smithi138.stdout:4/187: dread d0/d3/f40 [0,4194304] 0 2024-05-17T21:52:50.293 INFO:tasks.workunit.client.1.smithi138.stdout:7/139: readlink d0/l7 0 2024-05-17T21:52:50.293 INFO:tasks.workunit.client.1.smithi138.stdout:4/188: read d0/d1b/f18 [1290212,30825] 0 2024-05-17T21:52:50.294 INFO:tasks.workunit.client.1.smithi138.stdout:7/140: chown d0/c1 2 1 2024-05-17T21:52:50.294 INFO:tasks.workunit.client.1.smithi138.stdout:4/189: readlink d0/d3/d10/l28 0 2024-05-17T21:52:50.295 INFO:tasks.workunit.client.1.smithi138.stdout:4/190: chown d0/d3/f12 78 1 2024-05-17T21:52:50.299 INFO:tasks.workunit.client.1.smithi138.stdout:5/159: sync 2024-05-17T21:52:50.299 INFO:tasks.workunit.client.1.smithi138.stdout:2/188: sync 2024-05-17T21:52:50.302 INFO:tasks.workunit.client.0.smithi107.stdout:4/331: creat d3/d4a/d47/d60/d3a/d41/f71 x:0 0 0 2024-05-17T21:52:50.303 INFO:tasks.workunit.client.1.smithi138.stdout:5/160: read d1/f13 [3650760,96472] 0 2024-05-17T21:52:50.307 INFO:tasks.workunit.client.1.smithi138.stdout:8/106: dread d0/dc/fd [0,4194304] 0 2024-05-17T21:52:50.310 INFO:tasks.workunit.client.0.smithi107.stdout:9/350: creat d0/dd/d21/d29/d3e/d45/d68/f7a x:0 0 0 2024-05-17T21:52:50.310 INFO:tasks.workunit.client.0.smithi107.stdout:4/332: dwrite d3/db/f1f [4194304,4194304] 0 2024-05-17T21:52:50.310 INFO:tasks.workunit.client.1.smithi138.stdout:7/141: dread d0/f12 [0,4194304] 0 2024-05-17T21:52:50.311 INFO:tasks.workunit.client.1.smithi138.stdout:8/107: dread d0/fb [0,4194304] 0 2024-05-17T21:52:50.311 INFO:tasks.workunit.client.0.smithi107.stdout:4/333: write d3/d6/d26/f31 [1187465,22301] 0 2024-05-17T21:52:50.313 INFO:tasks.workunit.client.0.smithi107.stdout:8/327: rename dc/d10/d17 to dc/d21/d27/d34/d4c/d7e 0 2024-05-17T21:52:50.314 INFO:tasks.workunit.client.0.smithi107.stdout:8/328: dread - dc/f67 zero size 2024-05-17T21:52:50.315 INFO:tasks.workunit.client.0.smithi107.stdout:0/404: write d3/d58/f71 [466649,21242] 0 2024-05-17T21:52:50.316 INFO:tasks.workunit.client.1.smithi138.stdout:2/189: symlink d3/d21/l3d 0 2024-05-17T21:52:50.316 INFO:tasks.workunit.client.0.smithi107.stdout:8/329: write dc/d21/d35/f61 [427706,76313] 0 2024-05-17T21:52:50.317 INFO:tasks.workunit.client.0.smithi107.stdout:0/405: readlink d3/d58/d61/l87 0 2024-05-17T21:52:50.317 INFO:tasks.workunit.client.0.smithi107.stdout:8/330: write dc/d21/d35/f2d [3430290,40992] 0 2024-05-17T21:52:50.318 INFO:tasks.workunit.client.0.smithi107.stdout:3/337: creat d6/d2a/d2d/d69/f6d x:0 0 0 2024-05-17T21:52:50.319 INFO:tasks.workunit.client.0.smithi107.stdout:3/338: write d6/d2a/f5b [4462148,87903] 0 2024-05-17T21:52:50.322 INFO:tasks.workunit.client.0.smithi107.stdout:5/327: getdents d5/d5f 0 2024-05-17T21:52:50.323 INFO:tasks.workunit.client.1.smithi138.stdout:4/191: creat d0/d1b/d42/d45/f48 x:0 0 0 2024-05-17T21:52:50.324 INFO:tasks.workunit.client.0.smithi107.stdout:2/260: getdents d0/d9/d11 0 2024-05-17T21:52:50.324 INFO:tasks.workunit.client.0.smithi107.stdout:0/406: dwrite d3/d39/f7f [0,4194304] 0 2024-05-17T21:52:50.325 INFO:tasks.workunit.client.0.smithi107.stdout:9/351: creat d0/dd/d1b/f7b x:0 0 0 2024-05-17T21:52:50.326 INFO:tasks.workunit.client.0.smithi107.stdout:0/407: write d3/d39/f7f [3194898,681] 0 2024-05-17T21:52:50.326 INFO:tasks.workunit.client.1.smithi138.stdout:8/108: mknod d0/d11/c1e 0 2024-05-17T21:52:50.326 INFO:tasks.workunit.client.0.smithi107.stdout:2/261: write d0/d9/d1c/d3e/f3f [111291,62968] 0 2024-05-17T21:52:50.326 INFO:tasks.workunit.client.0.smithi107.stdout:0/408: chown d3/dc/d6b/d85 192 1 2024-05-17T21:52:50.328 INFO:tasks.workunit.client.0.smithi107.stdout:4/334: rmdir d3/db/d1e/d66 39 2024-05-17T21:52:50.330 INFO:tasks.workunit.client.0.smithi107.stdout:9/352: dread d0/f43 [0,4194304] 0 2024-05-17T21:52:50.336 INFO:tasks.workunit.client.1.smithi138.stdout:2/190: dwrite d3/f35 [0,4194304] 0 2024-05-17T21:52:50.336 INFO:tasks.workunit.client.0.smithi107.stdout:8/331: rename dc/d21/d27/d2f/f32 to dc/d21/d27/d2f/f7f 0 2024-05-17T21:52:50.338 INFO:tasks.workunit.client.0.smithi107.stdout:9/353: dwrite d0/dd/d21/f70 [0,4194304] 0 2024-05-17T21:52:50.339 INFO:tasks.workunit.client.0.smithi107.stdout:9/354: fdatasync d0/dd/d21/d29/f5a 0 2024-05-17T21:52:50.343 INFO:tasks.workunit.client.1.smithi138.stdout:5/161: mkdir d1/d24/d2f 0 2024-05-17T21:52:50.344 INFO:tasks.workunit.client.1.smithi138.stdout:8/109: dread d0/fb [0,4194304] 0 2024-05-17T21:52:50.344 INFO:tasks.workunit.client.1.smithi138.stdout:5/162: chown d1/d6/d19/c25 288788344 1 2024-05-17T21:52:50.344 INFO:tasks.workunit.client.0.smithi107.stdout:8/332: dwrite dc/d21/d69/f77 [0,4194304] 0 2024-05-17T21:52:50.345 INFO:tasks.workunit.client.0.smithi107.stdout:1/355: sync 2024-05-17T21:52:50.347 INFO:tasks.workunit.client.0.smithi107.stdout:5/328: unlink d5/f28 0 2024-05-17T21:52:50.349 INFO:tasks.workunit.client.1.smithi138.stdout:8/110: dread d0/dc/fd [0,4194304] 0 2024-05-17T21:52:50.350 INFO:tasks.workunit.client.1.smithi138.stdout:4/192: mkdir d0/d2f/d49 0 2024-05-17T21:52:50.352 INFO:tasks.workunit.client.1.smithi138.stdout:2/191: mkdir d3/d21/d2f/d3e 0 2024-05-17T21:52:50.353 INFO:tasks.workunit.client.1.smithi138.stdout:2/192: readlink d3/d21/l22 0 2024-05-17T21:52:50.353 INFO:tasks.workunit.client.1.smithi138.stdout:2/193: chown d3/d21/d2f/f3c 0 1 2024-05-17T21:52:50.355 INFO:tasks.workunit.client.0.smithi107.stdout:2/262: mknod d0/d9/d11/d27/c4d 0 2024-05-17T21:52:50.356 INFO:tasks.workunit.client.1.smithi138.stdout:5/163: stat d1/d1a/c27 0 2024-05-17T21:52:50.356 INFO:tasks.workunit.client.0.smithi107.stdout:8/333: dwrite dc/d21/d35/f2e [0,4194304] 0 2024-05-17T21:52:50.357 INFO:tasks.workunit.client.0.smithi107.stdout:1/356: dwrite d4/f66 [0,4194304] 0 2024-05-17T21:52:50.358 INFO:tasks.workunit.client.1.smithi138.stdout:8/111: unlink d0/d11/c13 0 2024-05-17T21:52:50.359 INFO:tasks.workunit.client.0.smithi107.stdout:4/335: creat d3/db/f72 x:0 0 0 2024-05-17T21:52:50.359 INFO:tasks.workunit.client.0.smithi107.stdout:8/334: write dc/d21/d27/d34/f38 [4402158,84619] 0 2024-05-17T21:52:50.359 INFO:tasks.workunit.client.1.smithi138.stdout:8/112: chown d0/l16 149191 1 2024-05-17T21:52:50.361 INFO:tasks.workunit.client.0.smithi107.stdout:8/335: write dc/d21/d69/f77 [2470919,53685] 0 2024-05-17T21:52:50.361 INFO:tasks.workunit.client.0.smithi107.stdout:7/304: link d5/df/d11/f5c d5/f6d 0 2024-05-17T21:52:50.361 INFO:tasks.workunit.client.0.smithi107.stdout:3/339: dread d6/d8/d10/f30 [0,4194304] 0 2024-05-17T21:52:50.364 INFO:tasks.workunit.client.0.smithi107.stdout:8/336: stat dc/d21/d27/d34 0 2024-05-17T21:52:50.364 INFO:tasks.workunit.client.1.smithi138.stdout:4/193: rename d0/d3/db/c37 to d0/d3/d17/d21/c4a 0 2024-05-17T21:52:50.366 INFO:tasks.workunit.client.0.smithi107.stdout:7/305: write d5/df/d11/d1d/d33/f64 [408928,82731] 0 2024-05-17T21:52:50.367 INFO:tasks.workunit.client.1.smithi138.stdout:7/142: link d0/lf d0/l28 0 2024-05-17T21:52:50.371 INFO:tasks.workunit.client.0.smithi107.stdout:4/336: dwrite d3/d6/d26/f2f [0,4194304] 0 2024-05-17T21:52:50.375 INFO:tasks.workunit.client.0.smithi107.stdout:3/340: dwrite d6/d2a/f52 [0,4194304] 0 2024-05-17T21:52:50.377 INFO:tasks.workunit.client.0.smithi107.stdout:5/329: creat d5/db/d16/d2d/d4d/f6a x:0 0 0 2024-05-17T21:52:50.380 INFO:tasks.workunit.client.0.smithi107.stdout:7/306: dwrite d5/df/d11/d1d/f55 [0,4194304] 0 2024-05-17T21:52:50.380 INFO:tasks.workunit.client.1.smithi138.stdout:8/113: dwrite d0/dc/fd [4194304,4194304] 0 2024-05-17T21:52:50.382 INFO:tasks.workunit.client.1.smithi138.stdout:8/114: chown d0/dc/ce 70924764 1 2024-05-17T21:52:50.387 INFO:tasks.workunit.client.0.smithi107.stdout:3/341: dwrite d6/d2a/d2d/d69/f6d [0,4194304] 0 2024-05-17T21:52:50.392 INFO:tasks.workunit.client.1.smithi138.stdout:8/115: creat d0/dc/f1f x:0 0 0 2024-05-17T21:52:50.392 INFO:tasks.workunit.client.1.smithi138.stdout:8/116: chown d0/c3 320 1 2024-05-17T21:52:50.398 INFO:tasks.workunit.client.0.smithi107.stdout:4/337: creat d3/d4a/f73 x:0 0 0 2024-05-17T21:52:50.398 INFO:tasks.workunit.client.1.smithi138.stdout:8/117: dread d0/fb [0,4194304] 0 2024-05-17T21:52:50.402 INFO:tasks.workunit.client.0.smithi107.stdout:4/338: truncate d3/d6/f5c 217890 0 2024-05-17T21:52:50.402 INFO:tasks.workunit.client.0.smithi107.stdout:9/355: creat d0/dd/d21/d29/d3e/d45/d68/d75/f7c x:0 0 0 2024-05-17T21:52:50.402 INFO:tasks.workunit.client.1.smithi138.stdout:2/194: truncate d3/d21/d2f/f33 1403530 0 2024-05-17T21:52:50.404 INFO:tasks.workunit.client.0.smithi107.stdout:5/330: chown d5/db/dc/l57 1 1 2024-05-17T21:52:50.406 INFO:tasks.workunit.client.0.smithi107.stdout:7/307: symlink d5/d16/d3b/l6e 0 2024-05-17T21:52:50.406 INFO:tasks.workunit.client.1.smithi138.stdout:8/118: dread d0/f9 [0,4194304] 0 2024-05-17T21:52:50.408 INFO:tasks.workunit.client.0.smithi107.stdout:0/409: creat d3/dc/d6b/f96 x:0 0 0 2024-05-17T21:52:50.408 INFO:tasks.workunit.client.1.smithi138.stdout:7/143: dread d0/f20 [0,4194304] 0 2024-05-17T21:52:50.409 INFO:tasks.workunit.client.1.smithi138.stdout:5/164: dwrite d1/f7 [0,4194304] 0 2024-05-17T21:52:50.410 INFO:tasks.workunit.client.1.smithi138.stdout:5/165: fdatasync d1/d6/fe 0 2024-05-17T21:52:50.410 INFO:tasks.workunit.client.0.smithi107.stdout:1/357: mknod d4/df/d1a/d26/d72/c77 0 2024-05-17T21:52:50.411 INFO:tasks.workunit.client.1.smithi138.stdout:8/119: mknod d0/c20 0 2024-05-17T21:52:50.411 INFO:tasks.workunit.client.1.smithi138.stdout:8/120: readlink d0/l19 0 2024-05-17T21:52:50.412 INFO:tasks.workunit.client.0.smithi107.stdout:3/342: symlink d6/d1a/l6e 0 2024-05-17T21:52:50.412 INFO:tasks.workunit.client.0.smithi107.stdout:1/358: read d4/df/d14/f23 [362211,544] 0 2024-05-17T21:52:50.414 INFO:tasks.workunit.client.0.smithi107.stdout:1/359: dread - d4/df/d4e/d59/f5a zero size 2024-05-17T21:52:50.415 INFO:tasks.workunit.client.1.smithi138.stdout:5/166: symlink d1/d2c/l30 0 2024-05-17T21:52:50.418 INFO:tasks.workunit.client.1.smithi138.stdout:8/121: symlink d0/dc/l21 0 2024-05-17T21:52:50.419 INFO:tasks.workunit.client.1.smithi138.stdout:8/122: write d0/dc/f10 [627353,67089] 0 2024-05-17T21:52:50.420 INFO:tasks.workunit.client.0.smithi107.stdout:9/356: dread d0/dd/d1f/d2c/f34 [0,4194304] 0 2024-05-17T21:52:50.422 INFO:tasks.workunit.client.0.smithi107.stdout:1/360: dwrite d4/df/d1a/d41/f48 [0,4194304] 0 2024-05-17T21:52:50.422 INFO:tasks.workunit.client.0.smithi107.stdout:9/357: read - d0/dd/d1b/f7b zero size 2024-05-17T21:52:50.422 INFO:tasks.workunit.client.0.smithi107.stdout:7/308: readlink d5/d16/d3b/d45/l66 0 2024-05-17T21:52:50.423 INFO:tasks.workunit.client.0.smithi107.stdout:1/361: fsync d4/df/d1a/f28 0 2024-05-17T21:52:50.424 INFO:tasks.workunit.client.1.smithi138.stdout:5/167: dwrite d1/d6/d1b/f1d [0,4194304] 0 2024-05-17T21:52:50.424 INFO:tasks.workunit.client.0.smithi107.stdout:7/309: write d5/df/f20 [3998009,120746] 0 2024-05-17T21:52:50.426 INFO:tasks.workunit.client.1.smithi138.stdout:8/123: dread d0/fb [0,4194304] 0 2024-05-17T21:52:50.427 INFO:tasks.workunit.client.0.smithi107.stdout:8/337: creat dc/d21/f80 x:0 0 0 2024-05-17T21:52:50.431 INFO:tasks.workunit.client.1.smithi138.stdout:8/124: dread d0/dc/fd [0,4194304] 0 2024-05-17T21:52:50.437 INFO:tasks.workunit.client.0.smithi107.stdout:0/410: rename d3/dc/d6b to d3/dc/d22/d23/d97 0 2024-05-17T21:52:50.438 INFO:tasks.workunit.client.1.smithi138.stdout:9/164: sync 2024-05-17T21:52:50.438 INFO:tasks.workunit.client.1.smithi138.stdout:6/243: sync 2024-05-17T21:52:50.439 INFO:tasks.workunit.client.1.smithi138.stdout:8/125: unlink d0/dc/f1f 0 2024-05-17T21:52:50.444 INFO:tasks.workunit.client.0.smithi107.stdout:5/331: write d5/f8 [3855439,119105] 0 2024-05-17T21:52:50.445 INFO:tasks.workunit.client.0.smithi107.stdout:9/358: mknod d0/dd/d1f/c7d 0 2024-05-17T21:52:50.446 INFO:tasks.workunit.client.1.smithi138.stdout:2/195: dread d3/f9 [0,4194304] 0 2024-05-17T21:52:50.446 INFO:tasks.workunit.client.0.smithi107.stdout:9/359: write d0/dd/d21/d29/d3e/f49 [570245,35462] 0 2024-05-17T21:52:50.447 INFO:tasks.workunit.client.1.smithi138.stdout:6/244: creat d7/db/d43/f4f x:0 0 0 2024-05-17T21:52:50.454 INFO:tasks.workunit.client.1.smithi138.stdout:6/245: dwrite d7/db/f11 [0,4194304] 0 2024-05-17T21:52:50.456 INFO:tasks.workunit.client.0.smithi107.stdout:1/362: dwrite d4/f21 [4194304,4194304] 0 2024-05-17T21:52:50.457 INFO:tasks.workunit.client.1.smithi138.stdout:2/196: creat d3/d21/d2f/f3f x:0 0 0 2024-05-17T21:52:50.457 INFO:tasks.workunit.client.0.smithi107.stdout:1/363: read - d4/df/d1a/d26/f60 zero size 2024-05-17T21:52:50.458 INFO:tasks.workunit.client.0.smithi107.stdout:7/310: mknod d5/d16/d3b/c6f 0 2024-05-17T21:52:50.459 INFO:tasks.workunit.client.0.smithi107.stdout:1/364: write d4/de/f19 [724867,37383] 0 2024-05-17T21:52:50.459 INFO:tasks.workunit.client.0.smithi107.stdout:7/311: dread - d5/df/d11/d1c/d31/d39/f58 zero size 2024-05-17T21:52:50.462 INFO:tasks.workunit.client.1.smithi138.stdout:8/126: write d0/ff [62689,61558] 0 2024-05-17T21:52:50.463 INFO:tasks.workunit.client.0.smithi107.stdout:3/343: getdents d6/d8/d10/d35/d36/d66 0 2024-05-17T21:52:50.465 INFO:tasks.workunit.client.1.smithi138.stdout:9/165: rename d6/f2b to d6/f33 0 2024-05-17T21:52:50.466 INFO:tasks.workunit.client.1.smithi138.stdout:6/246: creat d7/d8/f50 x:0 0 0 2024-05-17T21:52:50.467 INFO:tasks.workunit.client.1.smithi138.stdout:9/166: read f5 [514136,36644] 0 2024-05-17T21:52:50.467 INFO:tasks.workunit.client.1.smithi138.stdout:6/247: dread - d7/db/d43/d3f/f41 zero size 2024-05-17T21:52:50.468 INFO:tasks.workunit.client.0.smithi107.stdout:5/332: symlink d5/db/d16/d2d/d51/l6b 0 2024-05-17T21:52:50.469 INFO:tasks.workunit.client.1.smithi138.stdout:6/248: truncate d7/d8/f42 942451 0 2024-05-17T21:52:50.469 INFO:tasks.workunit.client.1.smithi138.stdout:2/197: mkdir d3/d21/d2f/d40 0 2024-05-17T21:52:50.473 INFO:tasks.workunit.client.1.smithi138.stdout:2/198: dread d3/f35 [0,4194304] 0 2024-05-17T21:52:50.473 INFO:tasks.workunit.client.0.smithi107.stdout:4/339: getdents d3/d4a/d47 0 2024-05-17T21:52:50.475 INFO:tasks.workunit.client.0.smithi107.stdout:8/338: mkdir dc/d10/d63/d71/d81 0 2024-05-17T21:52:50.476 INFO:tasks.workunit.client.1.smithi138.stdout:6/249: symlink d7/l51 0 2024-05-17T21:52:50.476 INFO:tasks.workunit.client.0.smithi107.stdout:8/339: write dc/f67 [705274,89851] 0 2024-05-17T21:52:50.479 INFO:tasks.workunit.client.0.smithi107.stdout:8/340: read dc/d21/d27/d34/d4c/d7e/f3b [392207,115516] 0 2024-05-17T21:52:50.481 INFO:tasks.workunit.client.0.smithi107.stdout:4/340: dwrite d3/db/f1f [0,4194304] 0 2024-05-17T21:52:50.481 INFO:tasks.workunit.client.0.smithi107.stdout:8/341: dread - dc/d21/d27/d2f/f2c zero size 2024-05-17T21:52:50.482 INFO:tasks.workunit.client.0.smithi107.stdout:4/341: dread - d3/db/d16/f61 zero size 2024-05-17T21:52:50.483 INFO:tasks.workunit.client.0.smithi107.stdout:1/365: mkdir d4/d78 0 2024-05-17T21:52:50.484 INFO:tasks.workunit.client.1.smithi138.stdout:8/127: link d0/fb d0/dc/f22 0 2024-05-17T21:52:50.484 INFO:tasks.workunit.client.1.smithi138.stdout:8/128: readlink d0/l16 0 2024-05-17T21:52:50.486 INFO:tasks.workunit.client.0.smithi107.stdout:1/366: dread - d4/df/d69/f6e zero size 2024-05-17T21:52:50.486 INFO:tasks.workunit.client.1.smithi138.stdout:8/129: dread d0/f9 [0,4194304] 0 2024-05-17T21:52:50.487 INFO:tasks.workunit.client.0.smithi107.stdout:9/360: dwrite d0/dd/f78 [0,4194304] 0 2024-05-17T21:52:50.488 INFO:tasks.workunit.client.0.smithi107.stdout:0/411: creat d3/dc/d22/f98 x:0 0 0 2024-05-17T21:52:50.489 INFO:tasks.workunit.client.0.smithi107.stdout:1/367: chown d4/df/d1a/c44 14800404 1 2024-05-17T21:52:50.494 INFO:tasks.workunit.client.1.smithi138.stdout:9/167: write d6/d7/f26 [729726,23505] 0 2024-05-17T21:52:50.497 INFO:tasks.workunit.client.1.smithi138.stdout:6/250: link d7/db/d43/d3f/f41 d7/db/f52 0 2024-05-17T21:52:50.498 INFO:tasks.workunit.client.0.smithi107.stdout:7/312: creat d5/f70 x:0 0 0 2024-05-17T21:52:50.499 INFO:tasks.workunit.client.0.smithi107.stdout:7/313: fdatasync d5/df/f26 0 2024-05-17T21:52:50.500 INFO:tasks.workunit.client.1.smithi138.stdout:8/130: mknod d0/dc/d1a/c23 0 2024-05-17T21:52:50.501 INFO:tasks.workunit.client.0.smithi107.stdout:0/412: mknod d3/dc/d22/d23/c99 0 2024-05-17T21:52:50.501 INFO:tasks.workunit.client.1.smithi138.stdout:9/168: dwrite d6/f1b [0,4194304] 0 2024-05-17T21:52:50.503 INFO:tasks.workunit.client.0.smithi107.stdout:1/368: unlink d4/df/d14/d1d/l30 0 2024-05-17T21:52:50.507 INFO:tasks.workunit.client.1.smithi138.stdout:9/169: dwrite d6/d7/d1f/f22 [0,4194304] 0 2024-05-17T21:52:50.508 INFO:tasks.workunit.client.0.smithi107.stdout:8/342: unlink dc/d21/d68/f7c 0 2024-05-17T21:52:50.508 INFO:tasks.workunit.client.0.smithi107.stdout:0/413: dwrite f1 [0,4194304] 0 2024-05-17T21:52:50.509 INFO:tasks.workunit.client.1.smithi138.stdout:6/251: link d7/f2e d7/db/d34/f53 0 2024-05-17T21:52:50.512 INFO:tasks.workunit.client.0.smithi107.stdout:4/342: dread d3/db/f46 [0,4194304] 0 2024-05-17T21:52:50.513 INFO:tasks.workunit.client.0.smithi107.stdout:4/343: chown d3/db/f33 142 1 2024-05-17T21:52:50.515 INFO:tasks.workunit.client.1.smithi138.stdout:2/199: write d3/d21/d2f/f33 [1413530,104909] 0 2024-05-17T21:52:50.516 INFO:tasks.workunit.client.1.smithi138.stdout:9/170: dwrite d6/d7/d1f/f22 [0,4194304] 0 2024-05-17T21:52:50.518 INFO:tasks.workunit.client.1.smithi138.stdout:9/171: chown d6/d7/d16/f20 142 1 2024-05-17T21:52:50.519 INFO:tasks.workunit.client.1.smithi138.stdout:9/172: write d6/d7/d1f/f22 [1570775,100446] 0 2024-05-17T21:52:50.520 INFO:tasks.workunit.client.0.smithi107.stdout:1/369: symlink d4/df/d1a/d26/l79 0 2024-05-17T21:52:50.521 INFO:tasks.workunit.client.0.smithi107.stdout:5/333: dwrite d5/db/f25 [0,4194304] 0 2024-05-17T21:52:50.523 INFO:tasks.workunit.client.0.smithi107.stdout:0/414: mkdir d3/d58/d61/d94/d9a 0 2024-05-17T21:52:50.524 INFO:tasks.workunit.client.0.smithi107.stdout:4/344: dwrite d3/d4a/d47/d60/d3a/d41/f51 [0,4194304] 0 2024-05-17T21:52:50.524 INFO:tasks.workunit.client.1.smithi138.stdout:6/252: truncate d7/db/d43/d3f/f4a 2980094 0 2024-05-17T21:52:50.525 INFO:tasks.workunit.client.1.smithi138.stdout:8/131: rmdir d0/dc/d1a 39 2024-05-17T21:52:50.528 INFO:tasks.workunit.client.1.smithi138.stdout:8/132: readlink d0/dc/l21 0 2024-05-17T21:52:50.534 INFO:tasks.workunit.client.1.smithi138.stdout:9/173: dread d6/f24 [0,4194304] 0 2024-05-17T21:52:50.535 INFO:tasks.workunit.client.1.smithi138.stdout:6/253: dwrite d7/d8/f50 [0,4194304] 0 2024-05-17T21:52:50.538 INFO:tasks.workunit.client.1.smithi138.stdout:6/254: read d7/f2e [1919990,125177] 0 2024-05-17T21:52:50.538 INFO:tasks.workunit.client.0.smithi107.stdout:1/370: mkdir d4/df/d1a/d26/d7a 0 2024-05-17T21:52:50.539 INFO:tasks.workunit.client.1.smithi138.stdout:6/255: stat d7/db/d34/c47 0 2024-05-17T21:52:50.539 INFO:tasks.workunit.client.0.smithi107.stdout:0/415: creat d3/d58/d61/f9b x:0 0 0 2024-05-17T21:52:50.539 INFO:tasks.workunit.client.0.smithi107.stdout:1/371: dread - d4/df/d14/d1d/f2f zero size 2024-05-17T21:52:50.540 INFO:tasks.workunit.client.0.smithi107.stdout:0/416: chown d3/dc/d22/d23/c62 1321899212 1 2024-05-17T21:52:50.542 INFO:tasks.workunit.client.1.smithi138.stdout:9/174: creat d6/d7/d1f/f34 x:0 0 0 2024-05-17T21:52:50.542 INFO:tasks.workunit.client.0.smithi107.stdout:1/372: write d4/df/d1a/d26/f60 [982455,78591] 0 2024-05-17T21:52:50.543 INFO:tasks.workunit.client.0.smithi107.stdout:0/417: stat d3/dc/d22/d23/d97/d77 0 2024-05-17T21:52:50.543 INFO:tasks.workunit.client.0.smithi107.stdout:7/314: rmdir d5/df/d11/d1c/d2f/d67 0 2024-05-17T21:52:50.544 INFO:tasks.workunit.client.1.smithi138.stdout:6/256: rmdir d7/db/d12 39 2024-05-17T21:52:50.546 INFO:tasks.workunit.client.0.smithi107.stdout:5/334: rename d5/db/d16/d2d/d32/f33 to d5/db/d16/d2d/d40/d5a/f6c 0 2024-05-17T21:52:50.546 INFO:tasks.workunit.client.1.smithi138.stdout:2/200: link d3/l2a d3/d21/d2f/d3e/l41 0 2024-05-17T21:52:50.548 INFO:tasks.workunit.client.0.smithi107.stdout:5/335: write d5/db/d16/d2d/d4d/f6a [122681,69389] 0 2024-05-17T21:52:50.550 INFO:tasks.workunit.client.0.smithi107.stdout:0/418: write d3/f18 [4131564,115787] 0 2024-05-17T21:52:50.550 INFO:tasks.workunit.client.1.smithi138.stdout:2/201: symlink d3/d21/d2f/l42 0 2024-05-17T21:52:50.551 INFO:tasks.workunit.client.1.smithi138.stdout:9/175: mknod d6/d7/d16/d29/c35 0 2024-05-17T21:52:50.552 INFO:tasks.workunit.client.0.smithi107.stdout:0/419: write d3/dc/d22/d5d/f83 [825349,75438] 0 2024-05-17T21:52:50.554 INFO:tasks.workunit.client.1.smithi138.stdout:2/202: readlink d3/d21/d2f/l3b 0 2024-05-17T21:52:50.555 INFO:tasks.workunit.client.0.smithi107.stdout:5/336: mkdir d5/db/d16/d2d/d4d/d6d 0 2024-05-17T21:52:50.556 INFO:tasks.workunit.client.1.smithi138.stdout:2/203: fdatasync d3/f6 0 2024-05-17T21:52:50.557 INFO:tasks.workunit.client.1.smithi138.stdout:2/204: chown d3/d21/d2f/d3e 0 1 2024-05-17T21:52:50.558 INFO:tasks.workunit.client.0.smithi107.stdout:4/345: getdents d3/d4a/d47/d60/d3a/d41 0 2024-05-17T21:52:50.559 INFO:tasks.workunit.client.1.smithi138.stdout:2/205: write d3/f10 [3024930,68901] 0 2024-05-17T21:52:50.559 INFO:tasks.workunit.client.0.smithi107.stdout:5/337: fdatasync d5/db/dc/f20 0 2024-05-17T21:52:50.561 INFO:tasks.workunit.client.0.smithi107.stdout:4/346: symlink d3/d4a/d47/d60/d3a/d41/l74 0 2024-05-17T21:52:50.562 INFO:tasks.workunit.client.1.smithi138.stdout:2/206: mknod d3/d21/c43 0 2024-05-17T21:52:50.563 INFO:tasks.workunit.client.0.smithi107.stdout:4/347: dread - d3/d4a/d47/d60/f5b zero size 2024-05-17T21:52:50.564 INFO:tasks.workunit.client.0.smithi107.stdout:7/315: getdents d5/df/d11/d1d 0 2024-05-17T21:52:50.566 INFO:tasks.workunit.client.0.smithi107.stdout:5/338: dread d5/db/d16/d5b/f53 [0,4194304] 0 2024-05-17T21:52:50.567 INFO:tasks.workunit.client.0.smithi107.stdout:5/339: write d5/f23 [3863749,124162] 0 2024-05-17T21:52:50.569 INFO:tasks.workunit.client.0.smithi107.stdout:4/348: dwrite d3/db/f69 [0,4194304] 0 2024-05-17T21:52:50.573 INFO:tasks.workunit.client.0.smithi107.stdout:6/287: sync 2024-05-17T21:52:50.574 INFO:tasks.workunit.client.0.smithi107.stdout:7/316: getdents d5/d16/d3b/d45 0 2024-05-17T21:52:50.577 INFO:tasks.workunit.client.0.smithi107.stdout:6/288: creat d9/d16/d22/d2d/f54 x:0 0 0 2024-05-17T21:52:50.579 INFO:tasks.workunit.client.0.smithi107.stdout:4/349: dread d3/d4a/d47/d60/d3c/f3e [0,4194304] 0 2024-05-17T21:52:50.582 INFO:tasks.workunit.client.0.smithi107.stdout:7/317: dwrite d5/df/d11/d1d/d44/f4b [4194304,4194304] 0 2024-05-17T21:52:50.583 INFO:tasks.workunit.client.0.smithi107.stdout:9/361: write d0/f17 [848110,107780] 0 2024-05-17T21:52:50.584 INFO:tasks.workunit.client.0.smithi107.stdout:4/350: truncate d3/d4a/d47/d60/d3a/d41/f71 421990 0 2024-05-17T21:52:50.586 INFO:tasks.workunit.client.0.smithi107.stdout:4/351: fsync d3/d4a/d47/d60/d3a/d41/f51 0 2024-05-17T21:52:50.591 INFO:tasks.workunit.client.0.smithi107.stdout:0/420: dread d3/f1d [0,4194304] 0 2024-05-17T21:52:50.595 INFO:tasks.workunit.client.1.smithi138.stdout:8/133: dwrite d0/f2 [0,4194304] 0 2024-05-17T21:52:50.595 INFO:tasks.workunit.client.0.smithi107.stdout:9/362: dwrite d0/dd/d21/d29/d3e/d45/f6b [4194304,4194304] 0 2024-05-17T21:52:50.599 INFO:tasks.workunit.client.1.smithi138.stdout:8/134: dread d0/ff [0,4194304] 0 2024-05-17T21:52:50.603 INFO:tasks.workunit.client.0.smithi107.stdout:0/421: dwrite d3/ff [0,4194304] 0 2024-05-17T21:52:50.604 INFO:tasks.workunit.client.0.smithi107.stdout:9/363: mknod d0/dd/d2f/d4a/c7e 0 2024-05-17T21:52:50.606 INFO:tasks.workunit.client.0.smithi107.stdout:9/364: fsync d0/dd/d2f/f31 0 2024-05-17T21:52:50.608 INFO:tasks.workunit.client.1.smithi138.stdout:9/176: write f5 [2746986,8250] 0 2024-05-17T21:52:50.608 INFO:tasks.workunit.client.1.smithi138.stdout:8/135: dwrite d0/dc/f10 [0,4194304] 0 2024-05-17T21:52:50.609 INFO:tasks.workunit.client.1.smithi138.stdout:8/136: read - d0/dc/f18 zero size 2024-05-17T21:52:50.612 INFO:tasks.workunit.client.0.smithi107.stdout:1/373: dwrite f3 [0,4194304] 0 2024-05-17T21:52:50.613 INFO:tasks.workunit.client.0.smithi107.stdout:1/374: read - d4/de/f3c zero size 2024-05-17T21:52:50.614 INFO:tasks.workunit.client.1.smithi138.stdout:9/177: mknod d6/d7/d16/c36 0 2024-05-17T21:52:50.616 INFO:tasks.workunit.client.1.smithi138.stdout:8/137: stat d0/dc/d1a/c23 0 2024-05-17T21:52:50.620 INFO:tasks.workunit.client.0.smithi107.stdout:4/352: dread d3/db/d16/d43/f53 [0,4194304] 0 2024-05-17T21:52:50.620 INFO:tasks.workunit.client.0.smithi107.stdout:9/365: rmdir d0/dd/d3d 39 2024-05-17T21:52:50.622 INFO:tasks.workunit.client.1.smithi138.stdout:2/207: truncate d3/f4 868656 0 2024-05-17T21:52:50.622 INFO:tasks.workunit.client.0.smithi107.stdout:1/375: dwrite d4/df/d1a/d26/f61 [0,4194304] 0 2024-05-17T21:52:50.625 INFO:tasks.workunit.client.0.smithi107.stdout:0/422: creat d3/f9c x:0 0 0 2024-05-17T21:52:50.629 INFO:tasks.workunit.client.0.smithi107.stdout:0/423: dread - d3/dc/d22/d23/d97/f96 zero size 2024-05-17T21:52:50.632 INFO:tasks.workunit.client.0.smithi107.stdout:9/366: dwrite d0/f17 [0,4194304] 0 2024-05-17T21:52:50.632 INFO:tasks.workunit.client.1.smithi138.stdout:2/208: write d3/f20 [589256,99308] 0 2024-05-17T21:52:50.633 INFO:tasks.workunit.client.0.smithi107.stdout:3/344: sync 2024-05-17T21:52:50.633 INFO:tasks.workunit.client.0.smithi107.stdout:2/263: sync 2024-05-17T21:52:50.636 INFO:tasks.workunit.client.0.smithi107.stdout:6/289: write d9/f15 [2082760,92903] 0 2024-05-17T21:52:50.637 INFO:tasks.workunit.client.1.smithi138.stdout:8/138: link d0/f17 d0/dc/d1a/f24 0 2024-05-17T21:52:50.639 INFO:tasks.workunit.client.1.smithi138.stdout:6/257: write d7/db/d43/d3f/f4a [1787706,56024] 0 2024-05-17T21:52:50.640 INFO:tasks.workunit.client.1.smithi138.stdout:6/258: write d7/db/d43/d3f/f4c [989436,40483] 0 2024-05-17T21:52:50.641 INFO:tasks.workunit.client.0.smithi107.stdout:5/340: dwrite d5/db/f1a [4194304,4194304] 0 2024-05-17T21:52:50.642 INFO:tasks.workunit.client.0.smithi107.stdout:1/376: creat d4/df/d1a/f7b x:0 0 0 2024-05-17T21:52:50.643 INFO:tasks.workunit.client.1.smithi138.stdout:9/178: rmdir d6/d7/d31 0 2024-05-17T21:52:50.643 INFO:tasks.workunit.client.0.smithi107.stdout:3/345: read d6/d1a/d57/f5e [957065,97367] 0 2024-05-17T21:52:50.644 INFO:tasks.workunit.client.0.smithi107.stdout:7/318: truncate d5/df/f1a 1620264 0 2024-05-17T21:52:50.645 INFO:tasks.workunit.client.1.smithi138.stdout:0/258: sync 2024-05-17T21:52:50.649 INFO:tasks.workunit.client.0.smithi107.stdout:0/424: dwrite d3/d58/d61/d94/f95 [0,4194304] 0 2024-05-17T21:52:50.649 INFO:tasks.workunit.client.1.smithi138.stdout:2/209: dread - d3/f29 zero size 2024-05-17T21:52:50.654 INFO:tasks.workunit.client.1.smithi138.stdout:6/259: mknod d7/d8/c54 0 2024-05-17T21:52:50.658 INFO:tasks.workunit.client.0.smithi107.stdout:6/290: mknod d9/d16/d22/d2d/c55 0 2024-05-17T21:52:50.660 INFO:tasks.workunit.client.1.smithi138.stdout:9/179: creat d6/d7/d16/d29/f37 x:0 0 0 2024-05-17T21:52:50.666 INFO:tasks.workunit.client.1.smithi138.stdout:8/139: symlink d0/dc/l25 0 2024-05-17T21:52:50.666 INFO:tasks.workunit.client.1.smithi138.stdout:8/140: chown d0/dc/d1a/f24 28 1 2024-05-17T21:52:50.666 INFO:tasks.workunit.client.1.smithi138.stdout:6/260: creat d7/db/d43/d3c/f55 x:0 0 0 2024-05-17T21:52:50.666 INFO:tasks.workunit.client.1.smithi138.stdout:8/141: dread - d0/f1b zero size 2024-05-17T21:52:50.666 INFO:tasks.workunit.client.1.smithi138.stdout:9/180: truncate d6/f2a 124986 0 2024-05-17T21:52:50.666 INFO:tasks.workunit.client.1.smithi138.stdout:6/261: fsync d7/d8/f2b 0 2024-05-17T21:52:50.667 INFO:tasks.workunit.client.1.smithi138.stdout:9/181: chown d6/d7 316828023 1 2024-05-17T21:52:50.667 INFO:tasks.workunit.client.0.smithi107.stdout:3/346: rename d6/d8/d10/d35/d36/l60 to d6/d8/d10/l6f 0 2024-05-17T21:52:50.668 INFO:tasks.workunit.client.1.smithi138.stdout:6/262: write d7/f28 [1640169,30703] 0 2024-05-17T21:52:50.668 INFO:tasks.workunit.client.1.smithi138.stdout:0/259: symlink d3/l66 0 2024-05-17T21:52:50.669 INFO:tasks.workunit.client.1.smithi138.stdout:0/260: fsync d3/f7 0 2024-05-17T21:52:50.670 INFO:tasks.workunit.client.1.smithi138.stdout:0/261: chown d3/d3d/f58 83106331 1 2024-05-17T21:52:50.671 INFO:tasks.workunit.client.0.smithi107.stdout:4/353: link d3/db/d35/d5e/f63 d3/d4a/f75 0 2024-05-17T21:52:50.672 INFO:tasks.workunit.client.0.smithi107.stdout:4/354: readlink d3/d6/d26/l29 0 2024-05-17T21:52:50.672 INFO:tasks.workunit.client.0.smithi107.stdout:4/355: readlink d3/d6/d26/l57 0 2024-05-17T21:52:50.673 INFO:tasks.workunit.client.0.smithi107.stdout:8/343: sync 2024-05-17T21:52:50.673 INFO:tasks.workunit.client.0.smithi107.stdout:4/356: fdatasync d3/d4a/d47/d60/f12 0 2024-05-17T21:52:50.674 INFO:tasks.workunit.client.0.smithi107.stdout:8/344: fdatasync dc/d21/d35/f11 0 2024-05-17T21:52:50.674 INFO:tasks.workunit.client.0.smithi107.stdout:8/345: chown dc/f4f 143301 1 2024-05-17T21:52:50.679 INFO:tasks.workunit.client.0.smithi107.stdout:8/346: dread dc/d21/d27/f5b [0,4194304] 0 2024-05-17T21:52:50.679 INFO:tasks.workunit.client.0.smithi107.stdout:5/341: dread d5/f17 [0,4194304] 0 2024-05-17T21:52:50.680 INFO:tasks.workunit.client.0.smithi107.stdout:5/342: chown d5/db/dc/d1e 49 1 2024-05-17T21:52:50.680 INFO:tasks.workunit.client.0.smithi107.stdout:8/347: chown dc/d21/d35/c49 878011 1 2024-05-17T21:52:50.681 INFO:tasks.workunit.client.0.smithi107.stdout:8/348: stat dc/f4f 0 2024-05-17T21:52:50.681 INFO:tasks.workunit.client.0.smithi107.stdout:5/343: write d5/db/d16/d5b/f3f [7651432,62632] 0 2024-05-17T21:52:50.682 INFO:tasks.workunit.client.1.smithi138.stdout:9/182: mkdir d6/d7/d16/d29/d38 0 2024-05-17T21:52:50.683 INFO:tasks.workunit.client.1.smithi138.stdout:9/183: read d6/d7/f9 [210557,49522] 0 2024-05-17T21:52:50.684 INFO:tasks.workunit.client.1.smithi138.stdout:9/184: chown d6/d7/l32 2891044 1 2024-05-17T21:52:50.690 INFO:tasks.workunit.client.0.smithi107.stdout:6/291: stat d9/c3c 0 2024-05-17T21:52:50.691 INFO:tasks.workunit.client.0.smithi107.stdout:6/292: stat d9/d16/d22 0 2024-05-17T21:52:50.691 INFO:tasks.workunit.client.1.smithi138.stdout:0/262: creat d3/d10/d38/d3e/f67 x:0 0 0 2024-05-17T21:52:50.691 INFO:tasks.workunit.client.1.smithi138.stdout:9/185: dwrite d6/d7/f26 [0,4194304] 0 2024-05-17T21:52:50.692 INFO:tasks.workunit.client.1.smithi138.stdout:2/210: rmdir d3/d21/d2f/d40 0 2024-05-17T21:52:50.692 INFO:tasks.workunit.client.1.smithi138.stdout:9/186: dread - d6/f33 zero size 2024-05-17T21:52:50.693 INFO:tasks.workunit.client.1.smithi138.stdout:9/187: truncate d6/d7/f15 1016290 0 2024-05-17T21:52:50.701 INFO:tasks.workunit.client.0.smithi107.stdout:1/377: link d4/df/f13 d4/df/d1a/d26/f7c 0 2024-05-17T21:52:50.701 INFO:tasks.workunit.client.1.smithi138.stdout:9/188: dwrite d6/d7/d1f/f34 [0,4194304] 0 2024-05-17T21:52:50.702 INFO:tasks.workunit.client.1.smithi138.stdout:8/142: write d0/dc/d1a/f24 [2257804,24114] 0 2024-05-17T21:52:50.705 INFO:tasks.workunit.client.0.smithi107.stdout:3/347: rename d6/d8/l50 to d6/d8/d10/d25/l70 0 2024-05-17T21:52:50.708 INFO:tasks.workunit.client.0.smithi107.stdout:1/378: dwrite d4/df/d1a/f7b [0,4194304] 0 2024-05-17T21:52:50.709 INFO:tasks.workunit.client.0.smithi107.stdout:0/425: link d3/d24/f2a d3/d24/d6f/f9d 0 2024-05-17T21:52:50.717 INFO:tasks.workunit.client.1.smithi138.stdout:3/228: sync 2024-05-17T21:52:50.717 INFO:tasks.workunit.client.1.smithi138.stdout:1/168: sync 2024-05-17T21:52:50.717 INFO:tasks.workunit.client.1.smithi138.stdout:4/194: sync 2024-05-17T21:52:50.717 INFO:tasks.workunit.client.1.smithi138.stdout:7/144: sync 2024-05-17T21:52:50.717 INFO:tasks.workunit.client.1.smithi138.stdout:5/168: sync 2024-05-17T21:52:50.718 INFO:tasks.workunit.client.1.smithi138.stdout:5/169: readlink d1/d6/l15 0 2024-05-17T21:52:50.723 INFO:tasks.workunit.client.0.smithi107.stdout:8/349: mknod dc/d21/d68/c82 0 2024-05-17T21:52:50.724 INFO:tasks.workunit.client.0.smithi107.stdout:8/350: chown dc/f62 0 1 2024-05-17T21:52:50.724 INFO:tasks.workunit.client.1.smithi138.stdout:7/145: dwrite d0/f1e [0,4194304] 0 2024-05-17T21:52:50.724 INFO:tasks.workunit.client.1.smithi138.stdout:6/263: truncate d7/db/d43/d3f/f46 286774 0 2024-05-17T21:52:50.725 INFO:tasks.workunit.client.1.smithi138.stdout:0/263: fsync d3/d10/f31 0 2024-05-17T21:52:50.727 INFO:tasks.workunit.client.0.smithi107.stdout:5/344: mkdir d5/db/dc/d1e/d6e 0 2024-05-17T21:52:50.727 INFO:tasks.workunit.client.0.smithi107.stdout:5/345: readlink d5/db/d16/l1d 0 2024-05-17T21:52:50.731 INFO:tasks.workunit.client.1.smithi138.stdout:7/146: dwrite d0/f1a [0,4194304] 0 2024-05-17T21:52:50.732 INFO:tasks.workunit.client.0.smithi107.stdout:9/367: link d0/dd/d1b/c26 d0/dd/d21/c7f 0 2024-05-17T21:52:50.733 INFO:tasks.workunit.client.1.smithi138.stdout:8/143: rmdir d0 39 2024-05-17T21:52:50.735 INFO:tasks.workunit.client.1.smithi138.stdout:9/189: stat d6/d7/d16/f1a 0 2024-05-17T21:52:50.736 INFO:tasks.workunit.client.1.smithi138.stdout:9/190: dread - d6/d7/d16/d29/f37 zero size 2024-05-17T21:52:50.740 INFO:tasks.workunit.client.1.smithi138.stdout:1/169: fdatasync d4/df/f12 0 2024-05-17T21:52:50.743 INFO:tasks.workunit.client.0.smithi107.stdout:3/348: rmdir d6/d2a 39 2024-05-17T21:52:50.743 INFO:tasks.workunit.client.0.smithi107.stdout:3/349: stat d6/d8/d10/d35/d36/f3d 0 2024-05-17T21:52:50.744 INFO:tasks.workunit.client.1.smithi138.stdout:3/229: symlink d1/d3/d49/l53 0 2024-05-17T21:52:50.746 INFO:tasks.workunit.client.1.smithi138.stdout:3/230: write d1/d3/d16/d3e/f4a [647129,60447] 0 2024-05-17T21:52:50.747 INFO:tasks.workunit.client.0.smithi107.stdout:1/379: rename d4/c31 to d4/df/d14/d2e/c7d 0 2024-05-17T21:52:50.749 INFO:tasks.workunit.client.1.smithi138.stdout:5/170: read d1/f22 [66609,59472] 0 2024-05-17T21:52:50.753 INFO:tasks.workunit.client.0.smithi107.stdout:8/351: mkdir dc/d21/d27/d83 0 2024-05-17T21:52:50.754 INFO:tasks.workunit.client.1.smithi138.stdout:3/231: dwrite d1/d3/d8/f13 [0,4194304] 0 2024-05-17T21:52:50.756 INFO:tasks.workunit.client.0.smithi107.stdout:2/264: getdents d0/d9/d1c/d3e 0 2024-05-17T21:52:50.756 INFO:tasks.workunit.client.0.smithi107.stdout:4/357: write d3/db/d35/d5e/f63 [2875744,35991] 0 2024-05-17T21:52:50.757 INFO:tasks.workunit.client.1.smithi138.stdout:6/264: dread d7/d8/f24 [0,4194304] 0 2024-05-17T21:52:50.758 INFO:tasks.workunit.client.0.smithi107.stdout:4/358: write d3/d4a/d47/d60/f12 [3290947,83694] 0 2024-05-17T21:52:50.760 INFO:tasks.workunit.client.0.smithi107.stdout:4/359: write d3/db/d16/f19 [1660580,39967] 0 2024-05-17T21:52:50.761 INFO:tasks.workunit.client.1.smithi138.stdout:2/211: truncate d3/f15 1794 0 2024-05-17T21:52:50.765 INFO:tasks.workunit.client.1.smithi138.stdout:9/191: dread d6/d7/d16/f21 [0,4194304] 0 2024-05-17T21:52:50.765 INFO:tasks.workunit.client.0.smithi107.stdout:3/350: symlink d6/d8/d3e/l71 0 2024-05-17T21:52:50.765 INFO:tasks.workunit.client.0.smithi107.stdout:7/319: getdents d5/df 0 2024-05-17T21:52:50.767 INFO:tasks.workunit.client.0.smithi107.stdout:0/426: rename d3/dc/d22/d5d to d3/d75/d9e 0 2024-05-17T21:52:50.768 INFO:tasks.workunit.client.1.smithi138.stdout:8/144: readlink d0/dc/l21 0 2024-05-17T21:52:50.769 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:50 smithi138 ceph-mon[35948]: pgmap v33: 65 pgs: 65 active+clean; 624 MiB data, 3.4 GiB used, 533 GiB / 536 GiB avail; 4.8 MiB/s rd, 60 MiB/s wr, 231 op/s 2024-05-17T21:52:50.769 INFO:tasks.workunit.client.1.smithi138.stdout:1/170: mkdir d4/d2e/d36 0 2024-05-17T21:52:50.772 INFO:tasks.workunit.client.0.smithi107.stdout:6/293: write d9/f12 [1941960,79761] 0 2024-05-17T21:52:50.773 INFO:tasks.workunit.client.1.smithi138.stdout:2/212: dwrite d3/d21/d2f/f33 [0,4194304] 0 2024-05-17T21:52:50.773 INFO:tasks.workunit.client.0.smithi107.stdout:7/320: dwrite d5/df/d11/d1d/f55 [0,4194304] 0 2024-05-17T21:52:50.776 INFO:tasks.workunit.client.0.smithi107.stdout:8/352: creat dc/d44/f84 x:0 0 0 2024-05-17T21:52:50.777 INFO:tasks.workunit.client.0.smithi107.stdout:8/353: write dc/f43 [373693,107068] 0 2024-05-17T21:52:50.777 INFO:tasks.workunit.client.1.smithi138.stdout:7/147: dwrite d0/f20 [4194304,4194304] 0 2024-05-17T21:52:50.780 INFO:tasks.workunit.client.0.smithi107.stdout:2/265: creat d0/d9/d11/d15/f4e x:0 0 0 2024-05-17T21:52:50.781 INFO:tasks.workunit.client.0.smithi107.stdout:2/266: write d0/d9/d11/f10 [4655206,75492] 0 2024-05-17T21:52:50.782 INFO:tasks.workunit.client.0.smithi107.stdout:2/267: dread - d0/d9/f46 zero size 2024-05-17T21:52:50.782 INFO:tasks.workunit.client.1.smithi138.stdout:7/148: readlink d0/l26 0 2024-05-17T21:52:50.785 INFO:tasks.workunit.client.1.smithi138.stdout:7/149: write d0/fa [1033307,125837] 0 2024-05-17T21:52:50.785 INFO:tasks.workunit.client.1.smithi138.stdout:0/264: dwrite d3/d3d/d51/f55 [0,4194304] 0 2024-05-17T21:52:50.787 INFO:tasks.workunit.client.1.smithi138.stdout:5/171: creat d1/d6/f31 x:0 0 0 2024-05-17T21:52:50.790 INFO:tasks.workunit.client.0.smithi107.stdout:0/427: dread d3/f18 [0,4194304] 0 2024-05-17T21:52:50.791 INFO:tasks.workunit.client.1.smithi138.stdout:0/265: dread d3/d10/d38/d3e/f57 [0,4194304] 0 2024-05-17T21:52:50.791 INFO:tasks.workunit.client.0.smithi107.stdout:1/380: dwrite d4/df/d1a/d26/f7c [0,4194304] 0 2024-05-17T21:52:50.792 INFO:tasks.workunit.client.0.smithi107.stdout:9/368: mkdir d0/d80 0 2024-05-17T21:52:50.793 INFO:tasks.workunit.client.1.smithi138.stdout:0/266: write d3/d10/d38/d3e/f5b [783967,106456] 0 2024-05-17T21:52:50.794 INFO:tasks.workunit.client.0.smithi107.stdout:9/369: write d0/dd/d21/d29/d3e/d45/f6b [4029601,121694] 0 2024-05-17T21:52:50.799 INFO:tasks.workunit.client.1.smithi138.stdout:0/267: write d3/d10/f11 [787410,74512] 0 2024-05-17T21:52:50.799 INFO:tasks.workunit.client.0.smithi107.stdout:5/346: rename d5/db/d16/d2d/d32/f52 to d5/db/d16/d54/d59/d65/f6f 0 2024-05-17T21:52:50.802 INFO:tasks.workunit.client.1.smithi138.stdout:9/192: rename d6/cd to d6/d7/d16/d29/d38/c39 0 2024-05-17T21:52:50.804 INFO:tasks.workunit.client.1.smithi138.stdout:9/193: dread d6/f2a [0,4194304] 0 2024-05-17T21:52:50.805 INFO:tasks.workunit.client.0.smithi107.stdout:6/294: creat d9/d16/d22/f56 x:0 0 0 2024-05-17T21:52:50.806 INFO:tasks.workunit.client.0.smithi107.stdout:5/347: dwrite d5/db/d39/f43 [0,4194304] 0 2024-05-17T21:52:50.809 INFO:tasks.workunit.client.1.smithi138.stdout:4/195: link d0/c13 d0/d3/d10/d1f/d32/c4b 0 2024-05-17T21:52:50.812 INFO:tasks.workunit.client.0.smithi107.stdout:2/268: mkdir d0/d9/d1c/d3e/d4f 0 2024-05-17T21:52:50.815 INFO:tasks.workunit.client.1.smithi138.stdout:4/196: dwrite d0/f7 [4194304,4194304] 0 2024-05-17T21:52:50.817 INFO:tasks.workunit.client.0.smithi107.stdout:3/351: dread d6/d8/f9 [0,4194304] 0 2024-05-17T21:52:50.818 INFO:tasks.workunit.client.1.smithi138.stdout:5/172: creat d1/d6/d1b/f32 x:0 0 0 2024-05-17T21:52:50.823 INFO:tasks.workunit.client.1.smithi138.stdout:3/232: creat d1/d4f/d50/f54 x:0 0 0 2024-05-17T21:52:50.824 INFO:tasks.workunit.client.1.smithi138.stdout:3/233: write d1/d3/f25 [662092,92983] 0 2024-05-17T21:52:50.826 INFO:tasks.workunit.client.0.smithi107.stdout:1/381: dread d4/df/d14/f4c [0,4194304] 0 2024-05-17T21:52:50.826 INFO:tasks.workunit.client.0.smithi107.stdout:1/382: fdatasync d4/df/d1a/f58 0 2024-05-17T21:52:50.827 INFO:tasks.workunit.client.1.smithi138.stdout:6/265: link d7/db/d43/d3f/f4a d7/db/d43/d3f/f56 0 2024-05-17T21:52:50.827 INFO:tasks.workunit.client.0.smithi107.stdout:1/383: truncate d4/df/d14/d2e/d43/f51 616981 0 2024-05-17T21:52:50.828 INFO:tasks.workunit.client.0.smithi107.stdout:1/384: stat d4/d6a/l70 0 2024-05-17T21:52:50.829 INFO:tasks.workunit.client.1.smithi138.stdout:3/234: dread d1/d3/d8/f13 [0,4194304] 0 2024-05-17T21:52:50.833 INFO:tasks.workunit.client.0.smithi107.stdout:7/321: write d5/df/d11/f5c [822060,16380] 0 2024-05-17T21:52:50.837 INFO:tasks.workunit.client.1.smithi138.stdout:1/171: creat d4/df/d35/f37 x:0 0 0 2024-05-17T21:52:50.837 INFO:tasks.workunit.client.0.smithi107.stdout:6/295: fdatasync d9/f53 0 2024-05-17T21:52:50.837 INFO:tasks.workunit.client.0.smithi107.stdout:6/296: read - d9/d16/f4a zero size 2024-05-17T21:52:50.838 INFO:tasks.workunit.client.0.smithi107.stdout:6/297: chown d9/d16/d22/f27 14985 1 2024-05-17T21:52:50.839 INFO:tasks.workunit.client.0.smithi107.stdout:6/298: write d9/d16/d22/d2d/f3d [559443,103255] 0 2024-05-17T21:52:50.839 INFO:tasks.workunit.client.1.smithi138.stdout:8/145: rename d0/l6 to d0/d11/l26 0 2024-05-17T21:52:50.842 INFO:tasks.workunit.client.0.smithi107.stdout:8/354: mknod dc/d44/d4b/d52/d70/c85 0 2024-05-17T21:52:50.847 INFO:tasks.workunit.client.0.smithi107.stdout:3/352: chown d6/d2a/d2d/d69 75482 1 2024-05-17T21:52:50.847 INFO:tasks.workunit.client.1.smithi138.stdout:4/197: mkdir d0/d3/d1d/d4c 0 2024-05-17T21:52:50.848 INFO:tasks.workunit.client.0.smithi107.stdout:3/353: stat d6/d2a/d2d/d48/d4f 0 2024-05-17T21:52:50.848 INFO:tasks.workunit.client.0.smithi107.stdout:0/428: dwrite d3/dc/d22/d23/f44 [0,4194304] 0 2024-05-17T21:52:50.849 INFO:tasks.workunit.client.0.smithi107.stdout:4/360: sync 2024-05-17T21:52:50.849 INFO:tasks.workunit.client.0.smithi107.stdout:9/370: mkdir d0/dd/d2f/d81 0 2024-05-17T21:52:50.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:50 smithi107 ceph-mon[28434]: pgmap v33: 65 pgs: 65 active+clean; 624 MiB data, 3.4 GiB used, 533 GiB / 536 GiB avail; 4.8 MiB/s rd, 60 MiB/s wr, 231 op/s 2024-05-17T21:52:50.857 INFO:tasks.workunit.client.0.smithi107.stdout:5/348: rename d5/db/d39/f4c to d5/db/d39/f70 0 2024-05-17T21:52:50.859 INFO:tasks.workunit.client.1.smithi138.stdout:6/266: mknod d7/db/d43/d3f/c57 0 2024-05-17T21:52:50.860 INFO:tasks.workunit.client.1.smithi138.stdout:6/267: write d7/db/f11 [3342153,90818] 0 2024-05-17T21:52:50.861 INFO:tasks.workunit.client.1.smithi138.stdout:3/235: mknod d1/d3/d8/c55 0 2024-05-17T21:52:50.862 INFO:tasks.workunit.client.0.smithi107.stdout:4/361: dread d3/db/d16/f19 [0,4194304] 0 2024-05-17T21:52:50.863 INFO:tasks.workunit.client.0.smithi107.stdout:6/299: rmdir d9/d16/d22/d2d 39 2024-05-17T21:52:50.864 INFO:tasks.workunit.client.0.smithi107.stdout:6/300: fsync d9/f4e 0 2024-05-17T21:52:50.865 INFO:tasks.workunit.client.1.smithi138.stdout:0/268: creat d3/d10/d38/d3e/d59/f68 x:0 0 0 2024-05-17T21:52:50.866 INFO:tasks.workunit.client.0.smithi107.stdout:8/355: rmdir dc/d10/d63/d71 39 2024-05-17T21:52:50.869 INFO:tasks.workunit.client.0.smithi107.stdout:8/356: chown dc/d21/d27/d34 6065 1 2024-05-17T21:52:50.870 INFO:tasks.workunit.client.0.smithi107.stdout:2/269: dwrite d0/d9/d11/f14 [4194304,4194304] 0 2024-05-17T21:52:50.870 INFO:tasks.workunit.client.1.smithi138.stdout:1/172: symlink d4/df/d35/l38 0 2024-05-17T21:52:50.870 INFO:tasks.workunit.client.1.smithi138.stdout:1/173: chown d4/d2e 2 1 2024-05-17T21:52:50.871 INFO:tasks.workunit.client.0.smithi107.stdout:0/429: mkdir d3/d24/d6f/d9f 0 2024-05-17T21:52:50.871 INFO:tasks.workunit.client.1.smithi138.stdout:5/173: write d1/f22 [912250,56130] 0 2024-05-17T21:52:50.872 INFO:tasks.workunit.client.1.smithi138.stdout:0/269: dwrite d3/d10/d38/d3e/f3f [0,4194304] 0 2024-05-17T21:52:50.874 INFO:tasks.workunit.client.0.smithi107.stdout:0/430: dread d3/d39/f49 [0,4194304] 0 2024-05-17T21:52:50.875 INFO:tasks.workunit.client.1.smithi138.stdout:8/146: symlink d0/d11/l27 0 2024-05-17T21:52:50.876 INFO:tasks.workunit.client.1.smithi138.stdout:8/147: fdatasync d0/f1b 0 2024-05-17T21:52:50.878 INFO:tasks.workunit.client.0.smithi107.stdout:9/371: creat d0/dd/d1f/f82 x:0 0 0 2024-05-17T21:52:50.879 INFO:tasks.workunit.client.0.smithi107.stdout:9/372: fdatasync d0/dd/d1f/f40 0 2024-05-17T21:52:50.881 INFO:tasks.workunit.client.0.smithi107.stdout:1/385: symlink d4/df/d1a/d26/d7a/l7e 0 2024-05-17T21:52:50.881 INFO:tasks.workunit.client.1.smithi138.stdout:7/150: getdents d0/d18 0 2024-05-17T21:52:50.881 INFO:tasks.workunit.client.0.smithi107.stdout:1/386: chown d4/df 482 1 2024-05-17T21:52:50.884 INFO:tasks.workunit.client.1.smithi138.stdout:4/198: creat d0/d3/d10/d1f/f4d x:0 0 0 2024-05-17T21:52:50.885 INFO:tasks.workunit.client.1.smithi138.stdout:4/199: read d0/d3/f12 [1977148,2534] 0 2024-05-17T21:52:50.886 INFO:tasks.workunit.client.0.smithi107.stdout:9/373: dwrite d0/dd/d1b/f53 [0,4194304] 0 2024-05-17T21:52:50.886 INFO:tasks.workunit.client.0.smithi107.stdout:4/362: symlink d3/d6/d26/d3b/l76 0 2024-05-17T21:52:50.887 INFO:tasks.workunit.client.1.smithi138.stdout:9/194: truncate d6/f1b 874135 0 2024-05-17T21:52:50.887 INFO:tasks.workunit.client.1.smithi138.stdout:9/195: stat d6/d7/fc 0 2024-05-17T21:52:50.889 INFO:tasks.workunit.client.1.smithi138.stdout:9/196: write d6/f25 [645990,99507] 0 2024-05-17T21:52:50.890 INFO:tasks.workunit.client.1.smithi138.stdout:9/197: dread - d6/d7/f17 zero size 2024-05-17T21:52:50.892 INFO:tasks.workunit.client.0.smithi107.stdout:6/301: rmdir d9/d16 39 2024-05-17T21:52:50.896 INFO:tasks.workunit.client.0.smithi107.stdout:2/270: creat d0/d9/d1c/d37/f50 x:0 0 0 2024-05-17T21:52:50.896 INFO:tasks.workunit.client.0.smithi107.stdout:2/271: rename d0/d9 to d0/d9/d1c/d37/d51 22 2024-05-17T21:52:50.898 INFO:tasks.workunit.client.1.smithi138.stdout:2/213: rename d3/l1f to d3/d21/d2f/l44 0 2024-05-17T21:52:50.900 INFO:tasks.workunit.client.1.smithi138.stdout:1/174: creat d4/db/f39 x:0 0 0 2024-05-17T21:52:50.906 INFO:tasks.workunit.client.1.smithi138.stdout:5/174: creat d1/d6/d1b/f33 x:0 0 0 2024-05-17T21:52:50.910 INFO:tasks.workunit.client.1.smithi138.stdout:8/148: creat d0/d11/f28 x:0 0 0 2024-05-17T21:52:50.912 INFO:tasks.workunit.client.0.smithi107.stdout:1/387: dread d4/df/d14/d1d/f27 [0,4194304] 0 2024-05-17T21:52:50.913 INFO:tasks.workunit.client.1.smithi138.stdout:7/151: dread d0/fa [0,4194304] 0 2024-05-17T21:52:50.913 INFO:tasks.workunit.client.0.smithi107.stdout:8/357: write dc/d21/d35/f22 [2994709,39932] 0 2024-05-17T21:52:50.913 INFO:tasks.workunit.client.0.smithi107.stdout:8/358: dread - dc/d44/f84 zero size 2024-05-17T21:52:50.915 INFO:tasks.workunit.client.1.smithi138.stdout:5/175: dwrite d1/d1a/f26 [0,4194304] 0 2024-05-17T21:52:50.915 INFO:tasks.workunit.client.1.smithi138.stdout:7/152: chown d0/d18/f1d 18280 1 2024-05-17T21:52:50.922 INFO:tasks.workunit.client.0.smithi107.stdout:0/431: creat d3/d24/d6f/d9f/fa0 x:0 0 0 2024-05-17T21:52:50.922 INFO:tasks.workunit.client.1.smithi138.stdout:6/268: symlink d7/db/l58 0 2024-05-17T21:52:50.924 INFO:tasks.workunit.client.0.smithi107.stdout:7/322: getdents d5/d16/d3b/d4f 0 2024-05-17T21:52:50.926 INFO:tasks.workunit.client.0.smithi107.stdout:9/374: creat d0/dd/d69/f83 x:0 0 0 2024-05-17T21:52:50.926 INFO:tasks.workunit.client.0.smithi107.stdout:9/375: dread - d0/dd/d21/d29/d3e/d45/d68/f7a zero size 2024-05-17T21:52:50.927 INFO:tasks.workunit.client.0.smithi107.stdout:9/376: write d0/dd/d2f/f64 [19355,36128] 0 2024-05-17T21:52:50.929 INFO:tasks.workunit.client.1.smithi138.stdout:1/175: mkdir d4/df/d3a 0 2024-05-17T21:52:50.931 INFO:tasks.workunit.client.1.smithi138.stdout:0/270: truncate d3/fc 3706022 0 2024-05-17T21:52:50.932 INFO:tasks.workunit.client.1.smithi138.stdout:0/271: write d3/d10/f1a [1966971,116088] 0 2024-05-17T21:52:50.934 INFO:tasks.workunit.client.1.smithi138.stdout:9/198: dwrite d6/d7/f30 [0,4194304] 0 2024-05-17T21:52:50.936 INFO:tasks.workunit.client.1.smithi138.stdout:4/200: link d0/f34 d0/d3/d1d/d4c/f4e 0 2024-05-17T21:52:50.937 INFO:tasks.workunit.client.1.smithi138.stdout:0/272: dread d3/d10/d38/f46 [0,4194304] 0 2024-05-17T21:52:50.937 INFO:tasks.workunit.client.1.smithi138.stdout:0/273: readlink d3/l56 0 2024-05-17T21:52:50.940 INFO:tasks.workunit.client.1.smithi138.stdout:5/176: symlink d1/d6/d19/l34 0 2024-05-17T21:52:50.942 INFO:tasks.workunit.client.0.smithi107.stdout:4/363: creat d3/db/d1e/f77 x:0 0 0 2024-05-17T21:52:50.943 INFO:tasks.workunit.client.0.smithi107.stdout:4/364: stat d3/db/f33 0 2024-05-17T21:52:50.943 INFO:tasks.workunit.client.0.smithi107.stdout:2/272: truncate d0/d9/d1c/d3e/f3f 1053127 0 2024-05-17T21:52:50.944 INFO:tasks.workunit.client.0.smithi107.stdout:2/273: stat d0/d9/d1c/d37/l38 0 2024-05-17T21:52:50.944 INFO:tasks.workunit.client.0.smithi107.stdout:2/274: chown d0/d9/d11/f14 413484 1 2024-05-17T21:52:50.946 INFO:tasks.workunit.client.1.smithi138.stdout:3/236: rename d1/d2e/l4c to d1/d4f/d50/l56 0 2024-05-17T21:52:50.954 INFO:tasks.workunit.client.1.smithi138.stdout:1/176: mkdir d4/dd/d3b 0 2024-05-17T21:52:50.954 INFO:tasks.workunit.client.0.smithi107.stdout:4/365: creat d3/d4a/f78 x:0 0 0 2024-05-17T21:52:50.955 INFO:tasks.workunit.client.0.smithi107.stdout:4/366: chown d3/db/d16/f19 111948635 1 2024-05-17T21:52:50.956 INFO:tasks.workunit.client.0.smithi107.stdout:9/377: creat d0/dd/d2f/d81/f84 x:0 0 0 2024-05-17T21:52:50.959 INFO:tasks.workunit.client.0.smithi107.stdout:8/359: link dc/d21/d35/f2e dc/f86 0 2024-05-17T21:52:50.962 INFO:tasks.workunit.client.0.smithi107.stdout:2/275: creat d0/d9/d11/f52 x:0 0 0 2024-05-17T21:52:50.968 INFO:tasks.workunit.client.0.smithi107.stdout:0/432: link d3/dc/f16 d3/fa1 0 2024-05-17T21:52:50.969 INFO:tasks.workunit.client.1.smithi138.stdout:5/177: creat d1/d2c/f35 x:0 0 0 2024-05-17T21:52:50.972 INFO:tasks.workunit.client.0.smithi107.stdout:7/323: rename d5/lb to d5/df/d11/d1c/l71 0 2024-05-17T21:52:50.972 INFO:tasks.workunit.client.1.smithi138.stdout:8/149: rename d0/dc to d0/dc/d29 22 2024-05-17T21:52:50.974 INFO:tasks.workunit.client.0.smithi107.stdout:4/367: creat d3/d4a/d47/d60/f79 x:0 0 0 2024-05-17T21:52:50.975 INFO:tasks.workunit.client.1.smithi138.stdout:3/237: unlink d1/d3/d8/c1f 0 2024-05-17T21:52:50.976 INFO:tasks.workunit.client.1.smithi138.stdout:3/238: read - d1/d4f/d50/f54 zero size 2024-05-17T21:52:50.976 INFO:tasks.workunit.client.0.smithi107.stdout:3/354: sync 2024-05-17T21:52:50.977 INFO:tasks.workunit.client.0.smithi107.stdout:3/355: chown f5 119695 1 2024-05-17T21:52:50.979 INFO:tasks.workunit.client.0.smithi107.stdout:8/360: mknod dc/d21/d35/c87 0 2024-05-17T21:52:50.981 INFO:tasks.workunit.client.1.smithi138.stdout:4/201: write d0/d3/f40 [3236081,63359] 0 2024-05-17T21:52:50.982 INFO:tasks.workunit.client.1.smithi138.stdout:1/177: rmdir d4/df 39 2024-05-17T21:52:50.982 INFO:tasks.workunit.client.0.smithi107.stdout:8/361: dread dc/d21/d35/f61 [0,4194304] 0 2024-05-17T21:52:50.983 INFO:tasks.workunit.client.1.smithi138.stdout:5/178: dread d1/d6/fe [0,4194304] 0 2024-05-17T21:52:50.984 INFO:tasks.workunit.client.1.smithi138.stdout:5/179: chown d1/d24 0 1 2024-05-17T21:52:50.984 INFO:tasks.workunit.client.0.smithi107.stdout:6/302: getdents d9/d16 0 2024-05-17T21:52:50.985 INFO:tasks.workunit.client.1.smithi138.stdout:5/180: write d1/d6/f11 [1583478,104840] 0 2024-05-17T21:52:50.986 INFO:tasks.workunit.client.1.smithi138.stdout:0/274: write d3/d10/f35 [82320,55124] 0 2024-05-17T21:52:50.987 INFO:tasks.workunit.client.1.smithi138.stdout:0/275: dread - d3/d3d/f58 zero size 2024-05-17T21:52:50.990 INFO:tasks.workunit.client.0.smithi107.stdout:4/368: rename d3/d6/d26/d3b/l54 to d3/db/d16/d43/l7a 0 2024-05-17T21:52:50.991 INFO:tasks.workunit.client.0.smithi107.stdout:6/303: dread f5 [0,4194304] 0 2024-05-17T21:52:50.992 INFO:tasks.workunit.client.1.smithi138.stdout:7/153: creat d0/f29 x:0 0 0 2024-05-17T21:52:50.994 INFO:tasks.workunit.client.0.smithi107.stdout:3/356: dread d6/d2a/f42 [0,4194304] 0 2024-05-17T21:52:50.994 INFO:tasks.workunit.client.1.smithi138.stdout:7/154: chown d0/f1f 7 1 2024-05-17T21:52:50.995 INFO:tasks.workunit.client.1.smithi138.stdout:5/181: dwrite d1/d6/d1b/f33 [0,4194304] 0 2024-05-17T21:52:50.995 INFO:tasks.workunit.client.1.smithi138.stdout:2/214: write d3/f1c [4565844,58868] 0 2024-05-17T21:52:50.997 INFO:tasks.workunit.client.1.smithi138.stdout:6/269: rename d7/db/f4d to d7/db/d43/f59 0 2024-05-17T21:52:50.998 INFO:tasks.workunit.client.1.smithi138.stdout:0/276: dwrite d3/d10/d38/d3e/f3f [0,4194304] 0 2024-05-17T21:52:50.999 INFO:tasks.workunit.client.1.smithi138.stdout:0/277: readlink d3/l56 0 2024-05-17T21:52:51.001 INFO:tasks.workunit.client.1.smithi138.stdout:8/150: creat d0/dc/d1a/f2a x:0 0 0 2024-05-17T21:52:51.003 INFO:tasks.workunit.client.0.smithi107.stdout:8/362: symlink dc/d21/d68/l88 0 2024-05-17T21:52:51.005 INFO:tasks.workunit.client.0.smithi107.stdout:9/378: truncate d0/f43 83135 0 2024-05-17T21:52:51.008 INFO:tasks.workunit.client.1.smithi138.stdout:4/202: rmdir d0/d1b 39 2024-05-17T21:52:51.009 INFO:tasks.workunit.client.1.smithi138.stdout:4/203: dread d0/f34 [0,4194304] 0 2024-05-17T21:52:51.013 INFO:tasks.workunit.client.0.smithi107.stdout:4/369: creat d3/db/f7b x:0 0 0 2024-05-17T21:52:51.014 INFO:tasks.workunit.client.0.smithi107.stdout:4/370: write d3/d6/f5c [10819,120212] 0 2024-05-17T21:52:51.016 INFO:tasks.workunit.client.0.smithi107.stdout:6/304: stat d9/d16/d22/d2d/f39 0 2024-05-17T21:52:51.017 INFO:tasks.workunit.client.1.smithi138.stdout:2/215: readlink d3/l24 0 2024-05-17T21:52:51.018 INFO:tasks.workunit.client.0.smithi107.stdout:3/357: stat d6/d8/c63 0 2024-05-17T21:52:51.021 INFO:tasks.workunit.client.1.smithi138.stdout:9/199: rename d6/d7/f2e to d6/d7/d16/d29/f3a 0 2024-05-17T21:52:51.021 INFO:tasks.workunit.client.0.smithi107.stdout:6/305: dwrite d9/d16/f3e [0,4194304] 0 2024-05-17T21:52:51.024 INFO:tasks.workunit.client.0.smithi107.stdout:8/363: creat dc/d44/d4b/d52/f89 x:0 0 0 2024-05-17T21:52:51.025 INFO:tasks.workunit.client.1.smithi138.stdout:6/270: creat d7/db/d43/d3f/f5a x:0 0 0 2024-05-17T21:52:51.028 INFO:tasks.workunit.client.0.smithi107.stdout:8/364: dread dc/f43 [0,4194304] 0 2024-05-17T21:52:51.028 INFO:tasks.workunit.client.1.smithi138.stdout:3/239: symlink d1/l57 0 2024-05-17T21:52:51.030 INFO:tasks.workunit.client.1.smithi138.stdout:9/200: dwrite d6/d7/d16/d29/f37 [0,4194304] 0 2024-05-17T21:52:51.031 INFO:tasks.workunit.client.1.smithi138.stdout:1/178: unlink d4/df/l28 0 2024-05-17T21:52:51.032 INFO:tasks.workunit.client.0.smithi107.stdout:5/349: sync 2024-05-17T21:52:51.033 INFO:tasks.workunit.client.0.smithi107.stdout:4/371: symlink d3/d4a/d47/d60/d3a/l7c 0 2024-05-17T21:52:51.033 INFO:tasks.workunit.client.0.smithi107.stdout:5/350: chown d5/db/d39/f3c 175698 1 2024-05-17T21:52:51.034 INFO:tasks.workunit.client.0.smithi107.stdout:4/372: chown d3/db/d35/f6b 0 1 2024-05-17T21:52:51.040 INFO:tasks.workunit.client.1.smithi138.stdout:2/216: stat d3/d21/f30 0 2024-05-17T21:52:51.041 INFO:tasks.workunit.client.0.smithi107.stdout:0/433: write d3/dc/d31/f76 [34713,3008] 0 2024-05-17T21:52:51.042 INFO:tasks.workunit.client.0.smithi107.stdout:0/434: write d3/d39/f49 [948296,118041] 0 2024-05-17T21:52:51.044 INFO:tasks.workunit.client.1.smithi138.stdout:0/278: mknod d3/d10/c69 0 2024-05-17T21:52:51.045 INFO:tasks.workunit.client.1.smithi138.stdout:0/279: chown d3/d10/d38/l63 119427020 1 2024-05-17T21:52:51.047 INFO:tasks.workunit.client.1.smithi138.stdout:0/280: truncate d3/d10/d38/d3e/f67 1025340 0 2024-05-17T21:52:51.048 INFO:tasks.workunit.client.1.smithi138.stdout:6/271: creat d7/db/d43/d3f/f5b x:0 0 0 2024-05-17T21:52:51.048 INFO:tasks.workunit.client.0.smithi107.stdout:6/306: dread d9/d16/d22/d2d/f37 [0,4194304] 0 2024-05-17T21:52:51.049 INFO:tasks.workunit.client.0.smithi107.stdout:8/365: symlink dc/d44/d4b/d52/d70/d76/l8a 0 2024-05-17T21:52:51.049 INFO:tasks.workunit.client.1.smithi138.stdout:3/240: creat d1/d4f/f58 x:0 0 0 2024-05-17T21:52:51.050 INFO:tasks.workunit.client.0.smithi107.stdout:8/366: stat dc/d10/c3e 0 2024-05-17T21:52:51.050 INFO:tasks.workunit.client.0.smithi107.stdout:6/307: chown d9/d16/d22/d2d/c51 160024 1 2024-05-17T21:52:51.050 INFO:tasks.workunit.client.1.smithi138.stdout:3/241: write d1/d3/d16/f38 [2284427,105240] 0 2024-05-17T21:52:51.050 INFO:tasks.workunit.client.0.smithi107.stdout:6/308: fdatasync d9/d16/f24 0 2024-05-17T21:52:51.051 INFO:tasks.workunit.client.1.smithi138.stdout:9/201: mkdir d6/d7/d1f/d3b 0 2024-05-17T21:52:51.053 INFO:tasks.workunit.client.0.smithi107.stdout:5/351: unlink d5/f23 0 2024-05-17T21:52:51.056 INFO:tasks.workunit.client.1.smithi138.stdout:4/204: creat d0/d1b/d42/f4f x:0 0 0 2024-05-17T21:52:51.057 INFO:tasks.workunit.client.0.smithi107.stdout:3/358: symlink d6/l72 0 2024-05-17T21:52:51.058 INFO:tasks.workunit.client.1.smithi138.stdout:3/242: dwrite d1/f4 [0,4194304] 0 2024-05-17T21:52:51.058 INFO:tasks.workunit.client.0.smithi107.stdout:2/276: truncate d0/d9/d11/f14 2185690 0 2024-05-17T21:52:51.059 INFO:tasks.workunit.client.1.smithi138.stdout:3/243: chown d1/d3/f25 15 1 2024-05-17T21:52:51.060 INFO:tasks.workunit.client.1.smithi138.stdout:3/244: fdatasync d1/d3/d16/f30 0 2024-05-17T21:52:51.060 INFO:tasks.workunit.client.1.smithi138.stdout:7/155: link d0/f24 d0/f2a 0 2024-05-17T21:52:51.061 INFO:tasks.workunit.client.0.smithi107.stdout:9/379: truncate d0/dd/d21/f37 1047833 0 2024-05-17T21:52:51.062 INFO:tasks.workunit.client.1.smithi138.stdout:3/245: write d1/d3/d16/f29 [5095814,124950] 0 2024-05-17T21:52:51.062 INFO:tasks.workunit.client.1.smithi138.stdout:2/217: creat d3/d21/d2f/f45 x:0 0 0 2024-05-17T21:52:51.063 INFO:tasks.workunit.client.0.smithi107.stdout:9/380: write d0/dd/d1f/f5c [664746,41685] 0 2024-05-17T21:52:51.063 INFO:tasks.workunit.client.0.smithi107.stdout:0/435: rmdir d3/dc 39 2024-05-17T21:52:51.063 INFO:tasks.workunit.client.1.smithi138.stdout:2/218: readlink d3/d21/d2f/l42 0 2024-05-17T21:52:51.064 INFO:tasks.workunit.client.1.smithi138.stdout:2/219: chown d3/f1c 1223464087 1 2024-05-17T21:52:51.066 INFO:tasks.workunit.client.1.smithi138.stdout:8/151: creat d0/dc/f2b x:0 0 0 2024-05-17T21:52:51.066 INFO:tasks.workunit.client.1.smithi138.stdout:9/202: dread d6/d7/fc [0,4194304] 0 2024-05-17T21:52:51.067 INFO:tasks.workunit.client.0.smithi107.stdout:8/367: fsync dc/d21/d27/f5b 0 2024-05-17T21:52:51.067 INFO:tasks.workunit.client.1.smithi138.stdout:9/203: chown d6/d7/d16/l27 528547 1 2024-05-17T21:52:51.068 INFO:tasks.workunit.client.0.smithi107.stdout:8/368: readlink dc/d21/d68/l88 0 2024-05-17T21:52:51.068 INFO:tasks.workunit.client.1.smithi138.stdout:9/204: chown d6/d7/f2c 127739 1 2024-05-17T21:52:51.068 INFO:tasks.workunit.client.1.smithi138.stdout:9/205: readlink d6/d7/la 0 2024-05-17T21:52:51.073 INFO:tasks.workunit.client.1.smithi138.stdout:1/179: mkdir d4/df/d3a/d3c 0 2024-05-17T21:52:51.073 INFO:tasks.workunit.client.0.smithi107.stdout:5/352: read d5/db/d16/d5b/f2f [43450,85103] 0 2024-05-17T21:52:51.073 INFO:tasks.workunit.client.0.smithi107.stdout:3/359: dread d6/d8/d10/f23 [0,4194304] 0 2024-05-17T21:52:51.074 INFO:tasks.workunit.client.1.smithi138.stdout:1/180: readlink d4/db/l1e 0 2024-05-17T21:52:51.074 INFO:tasks.workunit.client.0.smithi107.stdout:1/388: sync 2024-05-17T21:52:51.074 INFO:tasks.workunit.client.0.smithi107.stdout:7/324: sync 2024-05-17T21:52:51.075 INFO:tasks.workunit.client.1.smithi138.stdout:9/206: dwrite d6/d7/d1f/f23 [0,4194304] 0 2024-05-17T21:52:51.075 INFO:tasks.workunit.client.1.smithi138.stdout:9/207: readlink d6/d7/d16/l27 0 2024-05-17T21:52:51.075 INFO:tasks.workunit.client.0.smithi107.stdout:3/360: fsync d6/d8/d10/d35/d36/d4c/f65 0 2024-05-17T21:52:51.076 INFO:tasks.workunit.client.0.smithi107.stdout:2/277: fdatasync d0/d9/d11/df/f47 0 2024-05-17T21:52:51.078 INFO:tasks.workunit.client.0.smithi107.stdout:9/381: read d0/dd/f78 [31396,117085] 0 2024-05-17T21:52:51.078 INFO:tasks.workunit.client.1.smithi138.stdout:7/156: fsync d0/f5 0 2024-05-17T21:52:51.079 INFO:tasks.workunit.client.0.smithi107.stdout:0/436: rmdir d3/d75/d9e 39 2024-05-17T21:52:51.083 INFO:tasks.workunit.client.1.smithi138.stdout:5/182: getdents d1/d6 0 2024-05-17T21:52:51.084 INFO:tasks.workunit.client.1.smithi138.stdout:7/157: dwrite d0/f13 [0,4194304] 0 2024-05-17T21:52:51.085 INFO:tasks.workunit.client.0.smithi107.stdout:9/382: dwrite d0/f17 [4194304,4194304] 0 2024-05-17T21:52:51.087 INFO:tasks.workunit.client.0.smithi107.stdout:8/369: rmdir dc 39 2024-05-17T21:52:51.092 INFO:tasks.workunit.client.1.smithi138.stdout:5/183: dread d1/d6/fe [0,4194304] 0 2024-05-17T21:52:51.093 INFO:tasks.workunit.client.1.smithi138.stdout:5/184: chown d1/f13 88576 1 2024-05-17T21:52:51.094 INFO:tasks.workunit.client.1.smithi138.stdout:7/158: dread d0/f12 [0,4194304] 0 2024-05-17T21:52:51.094 INFO:tasks.workunit.client.1.smithi138.stdout:3/246: rename d1/d3/d8/c47 to d1/d3/d49/c59 0 2024-05-17T21:52:51.096 INFO:tasks.workunit.client.0.smithi107.stdout:6/309: symlink d9/d16/d22/l57 0 2024-05-17T21:52:51.097 INFO:tasks.workunit.client.1.smithi138.stdout:3/247: dread d1/d3/f20 [0,4194304] 0 2024-05-17T21:52:51.098 INFO:tasks.workunit.client.0.smithi107.stdout:6/310: write d9/d16/d22/d2d/f31 [26972,7700] 0 2024-05-17T21:52:51.098 INFO:tasks.workunit.client.1.smithi138.stdout:2/220: symlink d3/d21/l46 0 2024-05-17T21:52:51.100 INFO:tasks.workunit.client.1.smithi138.stdout:8/152: fsync d0/f2 0 2024-05-17T21:52:51.102 INFO:tasks.workunit.client.0.smithi107.stdout:4/373: rename d3/db/d1e/d66/c6a to d3/db/d35/d4f/c7d 0 2024-05-17T21:52:51.104 INFO:tasks.workunit.client.0.smithi107.stdout:5/353: creat d5/db/d16/d2d/d40/f71 x:0 0 0 2024-05-17T21:52:51.108 INFO:tasks.workunit.client.0.smithi107.stdout:1/389: unlink d4/df/d1a/d26/f61 0 2024-05-17T21:52:51.109 INFO:tasks.workunit.client.0.smithi107.stdout:1/390: chown d4/c1c 1238 1 2024-05-17T21:52:51.109 INFO:tasks.workunit.client.0.smithi107.stdout:7/325: fsync d5/df/d11/d1c/d31/d39/f5b 0 2024-05-17T21:52:51.113 INFO:tasks.workunit.client.0.smithi107.stdout:2/278: rmdir d0/d9/d1c/d37 39 2024-05-17T21:52:51.117 INFO:tasks.workunit.client.0.smithi107.stdout:0/437: readlink d3/dc/d31/l3c 0 2024-05-17T21:52:51.118 INFO:tasks.workunit.client.1.smithi138.stdout:9/208: mkdir d6/d7/d16/d29/d38/d3c 0 2024-05-17T21:52:51.119 INFO:tasks.workunit.client.0.smithi107.stdout:9/383: mknod d0/dd/d21/d29/d57/c85 0 2024-05-17T21:52:51.128 INFO:tasks.workunit.client.0.smithi107.stdout:4/374: fdatasync d3/db/d16/d43/f53 0 2024-05-17T21:52:51.129 INFO:tasks.workunit.client.0.smithi107.stdout:4/375: chown d3/fa 906311 1 2024-05-17T21:52:51.129 INFO:tasks.workunit.client.1.smithi138.stdout:3/248: mknod d1/d3/d16/d3e/c5a 0 2024-05-17T21:52:51.131 INFO:tasks.workunit.client.0.smithi107.stdout:5/354: dread d5/db/d16/f4b [0,4194304] 0 2024-05-17T21:52:51.132 INFO:tasks.workunit.client.1.smithi138.stdout:3/249: write d1/d3/d8/f28 [127581,109536] 0 2024-05-17T21:52:51.132 INFO:tasks.workunit.client.1.smithi138.stdout:7/159: dwrite d0/f12 [4194304,4194304] 0 2024-05-17T21:52:51.135 INFO:tasks.workunit.client.0.smithi107.stdout:1/391: mknod d4/df/d1a/c7f 0 2024-05-17T21:52:51.136 INFO:tasks.workunit.client.1.smithi138.stdout:0/281: rmdir d3/d3d/d5a 0 2024-05-17T21:52:51.137 INFO:tasks.workunit.client.1.smithi138.stdout:0/282: truncate d3/f24 151278 0 2024-05-17T21:52:51.139 INFO:tasks.workunit.client.0.smithi107.stdout:7/326: rename d5/d16/l32 to d5/df/d11/d1d/d33/l72 0 2024-05-17T21:52:51.142 INFO:tasks.workunit.client.0.smithi107.stdout:2/279: read - d0/d9/d11/d27/f43 zero size 2024-05-17T21:52:51.144 INFO:tasks.workunit.client.1.smithi138.stdout:9/209: creat d6/d7/d16/d29/d38/f3d x:0 0 0 2024-05-17T21:52:51.145 INFO:tasks.workunit.client.1.smithi138.stdout:5/185: mkdir d1/d6/d1b/d29/d36 0 2024-05-17T21:52:51.146 INFO:tasks.workunit.client.0.smithi107.stdout:3/361: write d6/d8/f14 [4436848,59588] 0 2024-05-17T21:52:51.150 INFO:tasks.workunit.client.1.smithi138.stdout:4/205: truncate d0/d3/d10/f3c 2245692 0 2024-05-17T21:52:51.151 INFO:tasks.workunit.client.0.smithi107.stdout:5/355: dread d5/db/d16/d5b/f3d [0,4194304] 0 2024-05-17T21:52:51.151 INFO:tasks.workunit.client.1.smithi138.stdout:4/206: dread - d0/d1b/d42/f4f zero size 2024-05-17T21:52:51.152 INFO:tasks.workunit.client.0.smithi107.stdout:5/356: chown d5/db/dc/f4e 14 1 2024-05-17T21:52:51.153 INFO:tasks.workunit.client.0.smithi107.stdout:4/376: mkdir d3/d6/d26/d36/d7e 0 2024-05-17T21:52:51.155 INFO:tasks.workunit.client.0.smithi107.stdout:5/357: dread d5/f17 [0,4194304] 0 2024-05-17T21:52:51.156 INFO:tasks.workunit.client.0.smithi107.stdout:5/358: fdatasync d5/db/dc/f4e 0 2024-05-17T21:52:51.156 INFO:tasks.workunit.client.1.smithi138.stdout:7/160: rmdir d0/d18 39 2024-05-17T21:52:51.157 INFO:tasks.workunit.client.1.smithi138.stdout:0/283: dread d3/d3d/f53 [0,4194304] 0 2024-05-17T21:52:51.157 INFO:tasks.workunit.client.1.smithi138.stdout:0/284: chown d3/d3d/d51 1587 1 2024-05-17T21:52:51.158 INFO:tasks.workunit.client.1.smithi138.stdout:0/285: fdatasync d3/d10/f1a 0 2024-05-17T21:52:51.159 INFO:tasks.workunit.client.1.smithi138.stdout:3/250: symlink d1/d4f/l5b 0 2024-05-17T21:52:51.162 INFO:tasks.workunit.client.1.smithi138.stdout:6/272: link d7/db/d12/l2c d7/db/d34/l5c 0 2024-05-17T21:52:51.162 INFO:tasks.workunit.client.1.smithi138.stdout:3/251: dread d1/d3/f1c [0,4194304] 0 2024-05-17T21:52:51.163 INFO:tasks.workunit.client.1.smithi138.stdout:6/273: write d7/d8/f2b [1409619,6151] 0 2024-05-17T21:52:51.163 INFO:tasks.workunit.client.0.smithi107.stdout:0/438: truncate d3/dc/f16 463781 0 2024-05-17T21:52:51.165 INFO:tasks.workunit.client.1.smithi138.stdout:1/181: link d4/df/f1b d4/f3d 0 2024-05-17T21:52:51.169 INFO:tasks.workunit.client.1.smithi138.stdout:9/210: dread d6/f1b [0,4194304] 0 2024-05-17T21:52:51.171 INFO:tasks.workunit.client.0.smithi107.stdout:4/377: truncate d3/db/d16/f19 765609 0 2024-05-17T21:52:51.175 INFO:tasks.workunit.client.1.smithi138.stdout:6/274: dread d7/db/d12/f45 [0,4194304] 0 2024-05-17T21:52:51.177 INFO:tasks.workunit.client.0.smithi107.stdout:7/327: mknod d5/c73 0 2024-05-17T21:52:51.179 INFO:tasks.workunit.client.0.smithi107.stdout:9/384: dwrite d0/dd/d1f/d2c/f47 [0,4194304] 0 2024-05-17T21:52:51.181 INFO:tasks.workunit.client.1.smithi138.stdout:0/286: dread - d3/f2b zero size 2024-05-17T21:52:51.181 INFO:tasks.workunit.client.0.smithi107.stdout:2/280: mknod d0/d9/d1c/d37/c53 0 2024-05-17T21:52:51.182 INFO:tasks.workunit.client.1.smithi138.stdout:0/287: write d3/d10/f15 [1160072,50599] 0 2024-05-17T21:52:51.182 INFO:tasks.workunit.client.0.smithi107.stdout:2/281: fdatasync d0/d9/d11/df/f47 0 2024-05-17T21:52:51.186 INFO:tasks.workunit.client.1.smithi138.stdout:3/252: mkdir d1/d3/d8/d5c 0 2024-05-17T21:52:51.187 INFO:tasks.workunit.client.0.smithi107.stdout:4/378: mknod d3/d6/d26/d3b/c7f 0 2024-05-17T21:52:51.187 INFO:tasks.workunit.client.1.smithi138.stdout:3/253: truncate d1/d3/d16/d3e/f43 438536 0 2024-05-17T21:52:51.188 INFO:tasks.workunit.client.1.smithi138.stdout:3/254: fsync d1/d3/f25 0 2024-05-17T21:52:51.188 INFO:tasks.workunit.client.0.smithi107.stdout:9/385: dwrite d0/dd/d1f/f82 [0,4194304] 0 2024-05-17T21:52:51.191 INFO:tasks.workunit.client.0.smithi107.stdout:5/359: truncate d5/db/d16/d2d/f4a 923564 0 2024-05-17T21:52:51.192 INFO:tasks.workunit.client.0.smithi107.stdout:6/311: sync 2024-05-17T21:52:51.193 INFO:tasks.workunit.client.0.smithi107.stdout:3/362: dwrite d6/d8/f17 [0,4194304] 0 2024-05-17T21:52:51.193 INFO:tasks.workunit.client.0.smithi107.stdout:6/312: stat d9/d16/f4a 0 2024-05-17T21:52:51.193 INFO:tasks.workunit.client.1.smithi138.stdout:1/182: dread - d4/dd/f15 zero size 2024-05-17T21:52:51.194 INFO:tasks.workunit.client.0.smithi107.stdout:5/360: write d5/db/d16/d54/d59/d65/f27 [1581033,39794] 0 2024-05-17T21:52:51.195 INFO:tasks.workunit.client.1.smithi138.stdout:1/183: write f0 [670424,122067] 0 2024-05-17T21:52:51.197 INFO:tasks.workunit.client.1.smithi138.stdout:1/184: write d4/db/f2a [1319751,48413] 0 2024-05-17T21:52:51.199 INFO:tasks.workunit.client.0.smithi107.stdout:8/370: rename dc/d44/d4b/d52/f66 to dc/f8b 0 2024-05-17T21:52:51.201 INFO:tasks.workunit.client.0.smithi107.stdout:3/363: dwrite d6/d8/d10/d25/f46 [0,4194304] 0 2024-05-17T21:52:51.202 INFO:tasks.workunit.client.0.smithi107.stdout:7/328: symlink d5/df/d11/d1d/l74 0 2024-05-17T21:52:51.211 INFO:tasks.workunit.client.0.smithi107.stdout:2/282: creat d0/d9/d11/df/f54 x:0 0 0 2024-05-17T21:52:51.211 INFO:tasks.workunit.client.1.smithi138.stdout:4/207: link d0/d3/db/f20 d0/d3/d1d/d38/f50 0 2024-05-17T21:52:51.215 INFO:tasks.workunit.client.1.smithi138.stdout:2/221: link d3/d21/d2f/d3e/l41 d3/d21/l47 0 2024-05-17T21:52:51.217 INFO:tasks.workunit.client.1.smithi138.stdout:8/153: getdents d0/dc/d1a 0 2024-05-17T21:52:51.219 INFO:tasks.workunit.client.1.smithi138.stdout:6/275: chown d7/d8/c14 7395 1 2024-05-17T21:52:51.219 INFO:tasks.workunit.client.1.smithi138.stdout:5/186: truncate d1/f7 2395507 0 2024-05-17T21:52:51.220 INFO:tasks.workunit.client.1.smithi138.stdout:5/187: readlink d1/d2c/l30 0 2024-05-17T21:52:51.221 INFO:tasks.workunit.client.0.smithi107.stdout:0/439: truncate d3/dc/f14 1632256 0 2024-05-17T21:52:51.225 INFO:tasks.workunit.client.1.smithi138.stdout:0/288: rename d3/d10/f36 to d3/d3d/f6a 0 2024-05-17T21:52:51.226 INFO:tasks.workunit.client.0.smithi107.stdout:0/440: dwrite d3/dc/d22/f32 [0,4194304] 0 2024-05-17T21:52:51.227 INFO:tasks.workunit.client.0.smithi107.stdout:1/392: rename d4/df/d1a/d26/d72/c77 to d4/d6a/c80 0 2024-05-17T21:52:51.230 INFO:tasks.workunit.client.0.smithi107.stdout:1/393: dread d4/df/d14/d1d/f27 [0,4194304] 0 2024-05-17T21:52:51.238 INFO:tasks.workunit.client.1.smithi138.stdout:1/185: creat d4/d2e/f3e x:0 0 0 2024-05-17T21:52:51.239 INFO:tasks.workunit.client.1.smithi138.stdout:9/211: mkdir d6/d7/d16/d29/d38/d3c/d3e 0 2024-05-17T21:52:51.240 INFO:tasks.workunit.client.1.smithi138.stdout:4/208: creat d0/d3/d10/f51 x:0 0 0 2024-05-17T21:52:51.245 INFO:tasks.workunit.client.1.smithi138.stdout:4/209: dwrite d0/d3/f40 [0,4194304] 0 2024-05-17T21:52:51.246 INFO:tasks.workunit.client.0.smithi107.stdout:4/379: rename d3/db/d16 to d3/d4a/d47/d60/d3a/d41/d80 0 2024-05-17T21:52:51.250 INFO:tasks.workunit.client.1.smithi138.stdout:8/154: rename d0/dc/fd to d0/f2c 0 2024-05-17T21:52:51.251 INFO:tasks.workunit.client.1.smithi138.stdout:8/155: dread - d0/dc/d1a/f2a zero size 2024-05-17T21:52:51.252 INFO:tasks.workunit.client.0.smithi107.stdout:8/371: write dc/d21/d27/d2f/f59 [8320981,124523] 0 2024-05-17T21:52:51.253 INFO:tasks.workunit.client.1.smithi138.stdout:0/289: mkdir d3/d3d/d6b 0 2024-05-17T21:52:51.255 INFO:tasks.workunit.client.0.smithi107.stdout:3/364: creat d6/d1a/d57/f73 x:0 0 0 2024-05-17T21:52:51.255 INFO:tasks.workunit.client.1.smithi138.stdout:3/255: truncate d1/d3/f11 1539940 0 2024-05-17T21:52:51.257 INFO:tasks.workunit.client.0.smithi107.stdout:1/394: mkdir d4/df/d1a/d26/d7a/d81 0 2024-05-17T21:52:51.259 INFO:tasks.workunit.client.1.smithi138.stdout:3/256: read d1/d3/d16/f38 [221490,23782] 0 2024-05-17T21:52:51.259 INFO:tasks.workunit.client.1.smithi138.stdout:9/212: symlink d6/d7/d1f/l3f 0 2024-05-17T21:52:51.260 INFO:tasks.workunit.client.0.smithi107.stdout:9/386: dwrite d0/dd/f35 [0,4194304] 0 2024-05-17T21:52:51.261 INFO:tasks.workunit.client.1.smithi138.stdout:2/222: mknod d3/c48 0 2024-05-17T21:52:51.264 INFO:tasks.workunit.client.0.smithi107.stdout:4/380: creat d3/db/d35/f81 x:0 0 0 2024-05-17T21:52:51.264 INFO:tasks.workunit.client.0.smithi107.stdout:4/381: dread - d3/db/f7b zero size 2024-05-17T21:52:51.265 INFO:tasks.workunit.client.1.smithi138.stdout:5/188: creat d1/d6/d1b/d29/d36/f37 x:0 0 0 2024-05-17T21:52:51.266 INFO:tasks.workunit.client.1.smithi138.stdout:3/257: dwrite d1/d3/d8/de/f3f [0,4194304] 0 2024-05-17T21:52:51.268 INFO:tasks.workunit.client.1.smithi138.stdout:7/161: getdents d0 0 2024-05-17T21:52:51.270 INFO:tasks.workunit.client.1.smithi138.stdout:5/189: write d1/d6/fe [4648706,49585] 0 2024-05-17T21:52:51.270 INFO:tasks.workunit.client.1.smithi138.stdout:8/156: mkdir d0/d11/d2d 0 2024-05-17T21:52:51.271 INFO:tasks.workunit.client.0.smithi107.stdout:8/372: readlink dc/d21/d27/d2f/l28 0 2024-05-17T21:52:51.272 INFO:tasks.workunit.client.1.smithi138.stdout:5/190: chown d1/d6/d1b/f32 133107896 1 2024-05-17T21:52:51.275 INFO:tasks.workunit.client.0.smithi107.stdout:7/329: creat d5/df/d11/d1c/f75 x:0 0 0 2024-05-17T21:52:51.276 INFO:tasks.workunit.client.1.smithi138.stdout:1/186: fdatasync d4/dd/f2d 0 2024-05-17T21:52:51.278 INFO:tasks.workunit.client.1.smithi138.stdout:9/213: mkdir d6/d7/d16/d29/d38/d40 0 2024-05-17T21:52:51.279 INFO:tasks.workunit.client.0.smithi107.stdout:8/373: dwrite dc/d21/d27/d2f/f59 [0,4194304] 0 2024-05-17T21:52:51.283 INFO:tasks.workunit.client.1.smithi138.stdout:4/210: getdents d0/d1b/d2b 0 2024-05-17T21:52:51.283 INFO:tasks.workunit.client.1.smithi138.stdout:2/223: dread d3/f9 [0,4194304] 0 2024-05-17T21:52:51.284 INFO:tasks.workunit.client.1.smithi138.stdout:4/211: stat d0/d1b/d36/f47 0 2024-05-17T21:52:51.285 INFO:tasks.workunit.client.1.smithi138.stdout:2/224: dread - d3/d21/d2f/f3c zero size 2024-05-17T21:52:51.285 INFO:tasks.workunit.client.0.smithi107.stdout:5/361: getdents d5/db/d39 0 2024-05-17T21:52:51.288 INFO:tasks.workunit.client.1.smithi138.stdout:7/162: unlink d0/f1c 0 2024-05-17T21:52:51.289 INFO:tasks.workunit.client.0.smithi107.stdout:4/382: symlink d3/d4a/d47/d60/d3c/l82 0 2024-05-17T21:52:51.290 INFO:tasks.workunit.client.1.smithi138.stdout:3/258: mkdir d1/d3/d16/d5d 0 2024-05-17T21:52:51.291 INFO:tasks.workunit.client.1.smithi138.stdout:3/259: fdatasync d1/d3/d16/f29 0 2024-05-17T21:52:51.296 INFO:tasks.workunit.client.0.smithi107.stdout:2/283: getdents d0/d9 0 2024-05-17T21:52:51.297 INFO:tasks.workunit.client.1.smithi138.stdout:1/187: truncate d4/df/f2c 499491 0 2024-05-17T21:52:51.298 INFO:tasks.workunit.client.0.smithi107.stdout:5/362: mknod d5/db/d16/d2d/d4d/c72 0 2024-05-17T21:52:51.298 INFO:tasks.workunit.client.1.smithi138.stdout:1/188: truncate d4/db/f21 1652239 0 2024-05-17T21:52:51.299 INFO:tasks.workunit.client.1.smithi138.stdout:1/189: chown d4/db/f21 4821 1 2024-05-17T21:52:51.300 INFO:tasks.workunit.client.0.smithi107.stdout:6/313: rename d9/d16/d22/d43 to d9/d58 0 2024-05-17T21:52:51.300 INFO:tasks.workunit.client.0.smithi107.stdout:3/365: dwrite d6/d1a/d57/f5e [4194304,4194304] 0 2024-05-17T21:52:51.301 INFO:tasks.workunit.client.1.smithi138.stdout:6/276: getdents d7 0 2024-05-17T21:52:51.301 INFO:tasks.workunit.client.0.smithi107.stdout:5/363: truncate d5/db/dc/f4e 683779 0 2024-05-17T21:52:51.303 INFO:tasks.workunit.client.0.smithi107.stdout:0/441: getdents d3/d58 0 2024-05-17T21:52:51.303 INFO:tasks.workunit.client.0.smithi107.stdout:6/314: write d9/f1c [1945906,127897] 0 2024-05-17T21:52:51.305 INFO:tasks.workunit.client.1.smithi138.stdout:9/214: dwrite d6/f33 [0,4194304] 0 2024-05-17T21:52:51.305 INFO:tasks.workunit.client.0.smithi107.stdout:1/395: write d4/df/d14/d1d/f27 [669098,22669] 0 2024-05-17T21:52:51.306 INFO:tasks.workunit.client.1.smithi138.stdout:9/215: chown d6/d7/d1f/f34 2870847 1 2024-05-17T21:52:51.306 INFO:tasks.workunit.client.0.smithi107.stdout:6/315: dread d9/f4e [0,4194304] 0 2024-05-17T21:52:51.306 INFO:tasks.workunit.client.0.smithi107.stdout:1/396: fdatasync d4/df/d4e/d59/f5a 0 2024-05-17T21:52:51.312 INFO:tasks.workunit.client.0.smithi107.stdout:5/364: dwrite d5/db/d16/d2d/d4d/f6a [0,4194304] 0 2024-05-17T21:52:51.315 INFO:tasks.workunit.client.1.smithi138.stdout:8/157: getdents d0/dc/d1a 0 2024-05-17T21:52:51.318 INFO:tasks.workunit.client.1.smithi138.stdout:0/290: dwrite d3/d3d/d51/f64 [0,4194304] 0 2024-05-17T21:52:51.319 INFO:tasks.workunit.client.0.smithi107.stdout:4/383: truncate d3/db/d35/d5e/f63 205693 0 2024-05-17T21:52:51.321 INFO:tasks.workunit.client.0.smithi107.stdout:9/387: dwrite d0/dd/f60 [0,4194304] 0 2024-05-17T21:52:51.325 INFO:tasks.workunit.client.0.smithi107.stdout:0/442: unlink d3/d39/l54 0 2024-05-17T21:52:51.327 INFO:tasks.workunit.client.1.smithi138.stdout:0/291: dwrite d3/f7 [0,4194304] 0 2024-05-17T21:52:51.329 INFO:tasks.workunit.client.1.smithi138.stdout:0/292: chown d3/d3d/d51/f54 5 1 2024-05-17T21:52:51.329 INFO:tasks.workunit.client.0.smithi107.stdout:7/330: mknod d5/df/d11/d1d/c76 0 2024-05-17T21:52:51.335 INFO:tasks.workunit.client.0.smithi107.stdout:1/397: write d4/df/f4d [2522339,129593] 0 2024-05-17T21:52:51.337 INFO:tasks.workunit.client.0.smithi107.stdout:6/316: creat d9/d16/d4d/f59 x:0 0 0 2024-05-17T21:52:51.337 INFO:tasks.workunit.client.1.smithi138.stdout:2/225: mknod d3/d21/d2f/d3e/c49 0 2024-05-17T21:52:51.340 INFO:tasks.workunit.client.0.smithi107.stdout:7/331: dwrite d5/df/f10 [0,4194304] 0 2024-05-17T21:52:51.341 INFO:tasks.workunit.client.0.smithi107.stdout:5/365: dread d5/db/d16/d2d/f4f [0,4194304] 0 2024-05-17T21:52:51.342 INFO:tasks.workunit.client.1.smithi138.stdout:3/260: symlink d1/d3/d16/d5d/l5e 0 2024-05-17T21:52:51.342 INFO:tasks.workunit.client.0.smithi107.stdout:5/366: chown d5/db/dc/c10 864017347 1 2024-05-17T21:52:51.345 INFO:tasks.workunit.client.1.smithi138.stdout:7/163: dread d0/f1a [0,4194304] 0 2024-05-17T21:52:51.345 INFO:tasks.workunit.client.1.smithi138.stdout:6/277: dread d7/db/d43/d3f/f56 [0,4194304] 0 2024-05-17T21:52:51.347 INFO:tasks.workunit.client.1.smithi138.stdout:8/158: symlink d0/d11/l2e 0 2024-05-17T21:52:51.348 INFO:tasks.workunit.client.0.smithi107.stdout:0/443: mknod d3/d58/d61/ca2 0 2024-05-17T21:52:51.350 INFO:tasks.workunit.client.0.smithi107.stdout:3/366: write d6/d8/d10/d25/f55 [1604140,15909] 0 2024-05-17T21:52:51.353 INFO:tasks.workunit.client.0.smithi107.stdout:8/374: rename dc/d44/d4b/l6b to dc/d21/l8c 0 2024-05-17T21:52:51.353 INFO:tasks.workunit.client.1.smithi138.stdout:5/191: write d1/d6/d1b/f1d [928105,53469] 0 2024-05-17T21:52:51.354 INFO:tasks.workunit.client.1.smithi138.stdout:4/212: link d0/d3/la d0/d2f/l52 0 2024-05-17T21:52:51.354 INFO:tasks.workunit.client.0.smithi107.stdout:8/375: chown dc/d21/d27/d2f/l28 17400 1 2024-05-17T21:52:51.355 INFO:tasks.workunit.client.1.smithi138.stdout:5/192: read - d1/d6/f31 zero size 2024-05-17T21:52:51.355 INFO:tasks.workunit.client.0.smithi107.stdout:8/376: write dc/f67 [1071100,85270] 0 2024-05-17T21:52:51.356 INFO:tasks.workunit.client.0.smithi107.stdout:6/317: mknod d9/d16/c5a 0 2024-05-17T21:52:51.358 INFO:tasks.workunit.client.1.smithi138.stdout:0/293: write d3/d3d/d51/f54 [949830,87528] 0 2024-05-17T21:52:51.360 INFO:tasks.workunit.client.1.smithi138.stdout:2/226: fsync d3/d21/f38 0 2024-05-17T21:52:51.362 INFO:tasks.workunit.client.0.smithi107.stdout:7/332: symlink d5/df/d11/d1c/d2f/l77 0 2024-05-17T21:52:51.362 INFO:tasks.workunit.client.1.smithi138.stdout:1/190: dwrite d4/dd/f2d [0,4194304] 0 2024-05-17T21:52:51.367 INFO:tasks.workunit.client.0.smithi107.stdout:5/367: mkdir d5/db/d16/d2a/d73 0 2024-05-17T21:52:51.368 INFO:tasks.workunit.client.1.smithi138.stdout:3/261: rmdir d1/d4f/d50 39 2024-05-17T21:52:51.374 INFO:tasks.workunit.client.1.smithi138.stdout:6/278: creat d7/d8/f5d x:0 0 0 2024-05-17T21:52:51.375 INFO:tasks.workunit.client.0.smithi107.stdout:3/367: mknod d6/d8/d10/c74 0 2024-05-17T21:52:51.375 INFO:tasks.workunit.client.1.smithi138.stdout:6/279: write d7/d8/f42 [785331,112562] 0 2024-05-17T21:52:51.377 INFO:tasks.workunit.client.1.smithi138.stdout:8/159: mkdir d0/d2f 0 2024-05-17T21:52:51.380 INFO:tasks.workunit.client.1.smithi138.stdout:6/280: dread d7/db/d43/d3f/f4c [0,4194304] 0 2024-05-17T21:52:51.380 INFO:tasks.workunit.client.0.smithi107.stdout:2/284: rename d0/d9/d11/df/f54 to d0/d26/f55 0 2024-05-17T21:52:51.381 INFO:tasks.workunit.client.1.smithi138.stdout:4/213: creat d0/d3/d10/d1f/f53 x:0 0 0 2024-05-17T21:52:51.381 INFO:tasks.workunit.client.1.smithi138.stdout:9/216: write d6/d7/d16/d29/f3a [4804521,34220] 0 2024-05-17T21:52:51.384 INFO:tasks.workunit.client.1.smithi138.stdout:4/214: write d0/d3/db/f35 [699325,124503] 0 2024-05-17T21:52:51.384 INFO:tasks.workunit.client.1.smithi138.stdout:9/217: chown d6/d7/f15 867513 1 2024-05-17T21:52:51.384 INFO:tasks.workunit.client.1.smithi138.stdout:0/294: chown d3/l60 1582 1 2024-05-17T21:52:51.483 INFO:tasks.workunit.client.1.smithi138.stdout:4/215: dread - d0/d1b/d36/f47 zero size 2024-05-17T21:52:51.484 INFO:tasks.workunit.client.0.smithi107.stdout:8/377: sync 2024-05-17T21:52:51.485 INFO:tasks.workunit.client.1.smithi138.stdout:6/281: dwrite d7/db/d12/f3e [0,4194304] 0 2024-05-17T21:52:51.487 INFO:tasks.workunit.client.1.smithi138.stdout:6/282: write d7/d8/f50 [2634279,129707] 0 2024-05-17T21:52:51.488 INFO:tasks.workunit.client.0.smithi107.stdout:5/368: symlink d5/db/d16/d2d/d4d/l74 0 2024-05-17T21:52:51.489 INFO:tasks.workunit.client.0.smithi107.stdout:9/388: dwrite d0/dd/d21/f37 [0,4194304] 0 2024-05-17T21:52:51.490 INFO:tasks.workunit.client.0.smithi107.stdout:0/444: write d3/f18 [3621285,30469] 0 2024-05-17T21:52:51.491 INFO:tasks.workunit.client.0.smithi107.stdout:5/369: truncate d5/db/d16/d2d/d32/f38 1589139 0 2024-05-17T21:52:51.491 INFO:tasks.workunit.client.0.smithi107.stdout:4/384: link d3/d4a/d47/d60/l2e d3/d4a/d47/d60/d3a/d41/l83 0 2024-05-17T21:52:51.505 INFO:tasks.workunit.client.1.smithi138.stdout:0/295: dread f2 [0,4194304] 0 2024-05-17T21:52:51.506 INFO:tasks.workunit.client.1.smithi138.stdout:0/296: write d3/d10/d38/d3e/d59/f68 [10726,49469] 0 2024-05-17T21:52:51.520 INFO:tasks.workunit.client.1.smithi138.stdout:3/262: fdatasync d1/d3/d8/f13 0 2024-05-17T21:52:51.523 INFO:tasks.workunit.client.0.smithi107.stdout:3/368: mkdir d6/d2a/d2d/d37/d75 0 2024-05-17T21:52:51.526 INFO:tasks.workunit.client.0.smithi107.stdout:3/369: dread d6/d8/f17 [0,4194304] 0 2024-05-17T21:52:51.527 INFO:tasks.workunit.client.0.smithi107.stdout:6/318: read d9/d16/d22/d2d/f2f [2942754,16835] 0 2024-05-17T21:52:51.528 INFO:tasks.workunit.client.1.smithi138.stdout:9/218: mknod d6/d7/d16/d29/d38/d3c/c41 0 2024-05-17T21:52:51.528 INFO:tasks.workunit.client.0.smithi107.stdout:7/333: write d5/df/d11/d1d/d44/f4b [8771339,73707] 0 2024-05-17T21:52:51.529 INFO:tasks.workunit.client.0.smithi107.stdout:8/378: creat dc/d10/f8d x:0 0 0 2024-05-17T21:52:51.533 INFO:tasks.workunit.client.0.smithi107.stdout:7/334: dwrite d5/f6d [0,4194304] 0 2024-05-17T21:52:51.535 INFO:tasks.workunit.client.0.smithi107.stdout:7/335: chown d5/d16/f3a 2188 1 2024-05-17T21:52:51.535 INFO:tasks.workunit.client.0.smithi107.stdout:4/385: mknod d3/d6/d26/d36/d37/c84 0 2024-05-17T21:52:51.535 INFO:tasks.workunit.client.0.smithi107.stdout:7/336: chown d5/d16/f2b 78180912 1 2024-05-17T21:52:51.536 INFO:tasks.workunit.client.0.smithi107.stdout:7/337: chown d5/df/f20 2471421 1 2024-05-17T21:52:51.537 INFO:tasks.workunit.client.0.smithi107.stdout:7/338: chown d5/d16/l2c 6986 1 2024-05-17T21:52:51.537 INFO:tasks.workunit.client.1.smithi138.stdout:6/283: mknod d7/db/d43/d3c/c5e 0 2024-05-17T21:52:51.537 INFO:tasks.workunit.client.1.smithi138.stdout:6/284: dread - d7/d8/f5d zero size 2024-05-17T21:52:51.538 INFO:tasks.workunit.client.0.smithi107.stdout:1/398: getdents d4/df/d69 0 2024-05-17T21:52:51.539 INFO:tasks.workunit.client.1.smithi138.stdout:4/216: write d0/d1b/f18 [4761431,29420] 0 2024-05-17T21:52:51.540 INFO:tasks.workunit.client.1.smithi138.stdout:4/217: readlink d0/d3/d10/l23 0 2024-05-17T21:52:51.541 INFO:tasks.workunit.client.1.smithi138.stdout:4/218: write d0/d3/d10/f25 [1791393,127105] 0 2024-05-17T21:52:51.546 INFO:tasks.workunit.client.1.smithi138.stdout:3/263: truncate d1/d3/d8/f24 176848 0 2024-05-17T21:52:51.547 INFO:tasks.workunit.client.1.smithi138.stdout:3/264: write d1/d3/d16/d3e/f4a [1346960,107006] 0 2024-05-17T21:52:51.548 INFO:tasks.workunit.client.1.smithi138.stdout:3/265: fdatasync d1/d3/d16/f29 0 2024-05-17T21:52:51.549 INFO:tasks.workunit.client.1.smithi138.stdout:5/193: link c0 d1/d1a/c38 0 2024-05-17T21:52:51.551 INFO:tasks.workunit.client.0.smithi107.stdout:8/379: rename dc/d21/d35/f2d to dc/d21/d69/f8e 0 2024-05-17T21:52:51.551 INFO:tasks.workunit.client.1.smithi138.stdout:7/164: truncate d0/f11 335956 0 2024-05-17T21:52:51.552 INFO:tasks.workunit.client.0.smithi107.stdout:8/380: read - dc/d44/d4b/d52/f89 zero size 2024-05-17T21:52:51.552 INFO:tasks.workunit.client.0.smithi107.stdout:5/370: mkdir d5/d75 0 2024-05-17T21:52:51.553 INFO:tasks.workunit.client.0.smithi107.stdout:4/386: mkdir d3/d6/d26/d36/d37/d85 0 2024-05-17T21:52:51.554 INFO:tasks.workunit.client.1.smithi138.stdout:2/227: link d3/d21/d2f/f3c d3/f4a 0 2024-05-17T21:52:51.557 INFO:tasks.workunit.client.1.smithi138.stdout:6/285: read d7/db/d43/d3f/f46 [1201,50324] 0 2024-05-17T21:52:51.558 INFO:tasks.workunit.client.1.smithi138.stdout:1/191: link f0 d4/f3f 0 2024-05-17T21:52:51.560 INFO:tasks.workunit.client.1.smithi138.stdout:0/297: mknod d3/c6c 0 2024-05-17T21:52:51.561 INFO:tasks.workunit.client.0.smithi107.stdout:5/371: dwrite d5/db/d16/d2d/d32/f38 [0,4194304] 0 2024-05-17T21:52:51.561 INFO:tasks.workunit.client.0.smithi107.stdout:4/387: dwrite d3/db/f7b [0,4194304] 0 2024-05-17T21:52:51.562 INFO:tasks.workunit.client.0.smithi107.stdout:4/388: stat d3/db/f72 0 2024-05-17T21:52:51.570 INFO:tasks.workunit.client.0.smithi107.stdout:5/372: dwrite d5/db/d16/d2d/d4d/f6a [0,4194304] 0 2024-05-17T21:52:51.571 INFO:tasks.workunit.client.1.smithi138.stdout:5/194: mkdir d1/d1a/d39 0 2024-05-17T21:52:51.572 INFO:tasks.workunit.client.0.smithi107.stdout:7/339: write d5/df/f1a [805935,104570] 0 2024-05-17T21:52:51.573 INFO:tasks.workunit.client.1.smithi138.stdout:6/286: dread d7/db/d43/f39 [0,4194304] 0 2024-05-17T21:52:51.574 INFO:tasks.workunit.client.1.smithi138.stdout:3/266: rename d1/d3/d16/c36 to d1/d3/d16/d2d/c5f 0 2024-05-17T21:52:51.575 INFO:tasks.workunit.client.1.smithi138.stdout:3/267: fdatasync d1/d3/d16/d3e/f43 0 2024-05-17T21:52:51.576 INFO:tasks.workunit.client.0.smithi107.stdout:4/389: dwrite d3/d4a/d47/d60/d3a/d41/d80/f45 [0,4194304] 0 2024-05-17T21:52:51.577 INFO:tasks.workunit.client.0.smithi107.stdout:8/381: rename dc/d21/d27/d2f/l28 to dc/d21/d35/l8f 0 2024-05-17T21:52:51.578 INFO:tasks.workunit.client.1.smithi138.stdout:9/219: getdents d6/d7/d16/d29/d38/d40 0 2024-05-17T21:52:51.581 INFO:tasks.workunit.client.1.smithi138.stdout:3/268: dread d1/f4 [0,4194304] 0 2024-05-17T21:52:51.584 INFO:tasks.workunit.client.0.smithi107.stdout:1/399: mknod d4/df/d14/c82 0 2024-05-17T21:52:51.587 INFO:tasks.workunit.client.0.smithi107.stdout:2/285: getdents d0/d9/d11/d27 0 2024-05-17T21:52:51.588 INFO:tasks.workunit.client.1.smithi138.stdout:4/219: mknod d0/d2f/d49/c54 0 2024-05-17T21:52:51.589 INFO:tasks.workunit.client.1.smithi138.stdout:5/195: unlink c0 0 2024-05-17T21:52:51.590 INFO:tasks.workunit.client.0.smithi107.stdout:3/370: creat d6/d8/f76 x:0 0 0 2024-05-17T21:52:51.591 INFO:tasks.workunit.client.0.smithi107.stdout:3/371: write d6/f1c [408135,18998] 0 2024-05-17T21:52:51.593 INFO:tasks.workunit.client.0.smithi107.stdout:5/373: mknod d5/db/d16/d2a/c76 0 2024-05-17T21:52:51.594 INFO:tasks.workunit.client.0.smithi107.stdout:7/340: chown d5/df/d11/d1d/d33/l72 29244 1 2024-05-17T21:52:51.596 INFO:tasks.workunit.client.1.smithi138.stdout:5/196: dwrite d1/f7 [0,4194304] 0 2024-05-17T21:52:51.596 INFO:tasks.workunit.client.0.smithi107.stdout:4/390: symlink d3/d4a/l86 0 2024-05-17T21:52:51.598 INFO:tasks.workunit.client.1.smithi138.stdout:9/220: mknod d6/d7/d16/d29/d38/d3c/c42 0 2024-05-17T21:52:51.598 INFO:tasks.workunit.client.0.smithi107.stdout:8/382: chown dc/d21/d27/d2f/l51 0 1 2024-05-17T21:52:51.598 INFO:tasks.workunit.client.1.smithi138.stdout:9/221: stat d6/d7/d1f/c2f 0 2024-05-17T21:52:51.600 INFO:tasks.workunit.client.0.smithi107.stdout:1/400: creat d4/df/d1a/d26/d72/f83 x:0 0 0 2024-05-17T21:52:51.601 INFO:tasks.workunit.client.0.smithi107.stdout:1/401: truncate d4/df/d69/f6e 992806 0 2024-05-17T21:52:51.602 INFO:tasks.workunit.client.0.smithi107.stdout:1/402: fsync d4/df/d1a/f7b 0 2024-05-17T21:52:51.602 INFO:tasks.workunit.client.0.smithi107.stdout:1/403: fsync d4/f38 0 2024-05-17T21:52:51.603 INFO:tasks.workunit.client.0.smithi107.stdout:2/286: rename d0/d9/l4a to d0/d9/d11/d27/l56 0 2024-05-17T21:52:51.603 INFO:tasks.workunit.client.0.smithi107.stdout:1/404: readlink d4/df/d69/l75 0 2024-05-17T21:52:51.605 INFO:tasks.workunit.client.0.smithi107.stdout:6/319: getdents d9 0 2024-05-17T21:52:51.606 INFO:tasks.workunit.client.0.smithi107.stdout:3/372: rmdir d6/d2a/d2d/d48 39 2024-05-17T21:52:51.608 INFO:tasks.workunit.client.0.smithi107.stdout:5/374: write d5/db/d16/d54/d59/d65/f6f [631369,111222] 0 2024-05-17T21:52:51.610 INFO:tasks.workunit.client.0.smithi107.stdout:7/341: mkdir d5/df/d78 0 2024-05-17T21:52:51.611 INFO:tasks.workunit.client.1.smithi138.stdout:1/192: mkdir d4/dd/d3b/d40 0 2024-05-17T21:52:51.611 INFO:tasks.workunit.client.0.smithi107.stdout:5/375: dread d5/f1b [4194304,4194304] 0 2024-05-17T21:52:51.612 INFO:tasks.workunit.client.0.smithi107.stdout:5/376: write d5/db/d16/d54/d59/d65/f27 [4938288,26714] 0 2024-05-17T21:52:51.613 INFO:tasks.workunit.client.1.smithi138.stdout:0/298: mkdir d3/d10/d6d 0 2024-05-17T21:52:51.616 INFO:tasks.workunit.client.0.smithi107.stdout:4/391: rename d3/db/f48 to d3/d6/d26/d36/d7e/f87 0 2024-05-17T21:52:51.617 INFO:tasks.workunit.client.1.smithi138.stdout:6/287: write d7/db/d12/f45 [4990048,113058] 0 2024-05-17T21:52:51.617 INFO:tasks.workunit.client.0.smithi107.stdout:9/389: sync 2024-05-17T21:52:51.617 INFO:tasks.workunit.client.0.smithi107.stdout:0/445: sync 2024-05-17T21:52:51.622 INFO:tasks.workunit.client.1.smithi138.stdout:5/197: creat d1/d6/d19/f3a x:0 0 0 2024-05-17T21:52:51.623 INFO:tasks.workunit.client.0.smithi107.stdout:3/373: symlink d6/d8/d10/d35/l77 0 2024-05-17T21:52:51.626 INFO:tasks.workunit.client.1.smithi138.stdout:7/165: creat d0/f2b x:0 0 0 2024-05-17T21:52:51.628 INFO:tasks.workunit.client.1.smithi138.stdout:3/269: fsync d1/d3/f11 0 2024-05-17T21:52:51.629 INFO:tasks.workunit.client.0.smithi107.stdout:8/383: mknod dc/c90 0 2024-05-17T21:52:51.633 INFO:tasks.workunit.client.1.smithi138.stdout:1/193: rmdir d4/db/d29 39 2024-05-17T21:52:51.633 INFO:tasks.workunit.client.0.smithi107.stdout:8/384: dread dc/f4f [0,4194304] 0 2024-05-17T21:52:51.634 INFO:tasks.workunit.client.1.smithi138.stdout:3/270: dwrite d1/d3/d8/f34 [0,4194304] 0 2024-05-17T21:52:51.637 INFO:tasks.workunit.client.1.smithi138.stdout:4/220: mkdir d0/d1b/d2b/d55 0 2024-05-17T21:52:51.640 INFO:tasks.workunit.client.1.smithi138.stdout:6/288: mkdir d7/db/d43/d5f 0 2024-05-17T21:52:51.645 INFO:tasks.workunit.client.1.smithi138.stdout:5/198: creat d1/d6/d1b/d29/f3b x:0 0 0 2024-05-17T21:52:51.646 INFO:tasks.workunit.client.1.smithi138.stdout:5/199: fsync d1/d6/d1b/f21 0 2024-05-17T21:52:51.646 INFO:tasks.workunit.client.0.smithi107.stdout:1/405: symlink d4/df/l84 0 2024-05-17T21:52:51.646 INFO:tasks.workunit.client.0.smithi107.stdout:6/320: creat d9/d16/d4d/f5b x:0 0 0 2024-05-17T21:52:51.648 INFO:tasks.workunit.client.0.smithi107.stdout:3/374: creat d6/d2a/f78 x:0 0 0 2024-05-17T21:52:51.651 INFO:tasks.workunit.client.0.smithi107.stdout:1/406: dwrite d4/f21 [4194304,4194304] 0 2024-05-17T21:52:51.652 INFO:tasks.workunit.client.1.smithi138.stdout:2/228: link d3/l37 d3/d21/d2f/l4b 0 2024-05-17T21:52:51.653 INFO:tasks.workunit.client.0.smithi107.stdout:7/342: rename d5/df/d11/d1d/c76 to d5/df/d11/d2a/c79 0 2024-05-17T21:52:51.654 INFO:tasks.workunit.client.1.smithi138.stdout:0/299: getdents d3/d3d/d6b 0 2024-05-17T21:52:51.655 INFO:tasks.workunit.client.1.smithi138.stdout:3/271: mknod d1/d3/d16/c60 0 2024-05-17T21:52:51.657 INFO:tasks.workunit.client.1.smithi138.stdout:4/221: rename d0/d3/d10/c11 to d0/d3/d1d/c56 0 2024-05-17T21:52:51.659 INFO:tasks.workunit.client.0.smithi107.stdout:5/377: creat d5/db/d16/d2d/d4d/d6d/f77 x:0 0 0 2024-05-17T21:52:51.659 INFO:tasks.workunit.client.0.smithi107.stdout:7/343: chown d5/d16/d3b/d4f 370924 1 2024-05-17T21:52:51.660 INFO:tasks.workunit.client.0.smithi107.stdout:9/390: symlink d0/dd/d21/d29/d57/d5f/l86 0 2024-05-17T21:52:51.662 INFO:tasks.workunit.client.0.smithi107.stdout:9/391: fdatasync d0/dd/d1b/f7b 0 2024-05-17T21:52:51.662 INFO:tasks.workunit.client.0.smithi107.stdout:7/344: readlink d5/df/d11/d1c/d2f/d53/l5a 0 2024-05-17T21:52:51.662 INFO:tasks.workunit.client.0.smithi107.stdout:6/321: creat d9/d16/d22/d2d/f5c x:0 0 0 2024-05-17T21:52:51.663 INFO:tasks.workunit.client.0.smithi107.stdout:1/407: dwrite d4/f38 [0,4194304] 0 2024-05-17T21:52:51.666 INFO:tasks.workunit.client.1.smithi138.stdout:2/229: creat d3/d21/d2f/d3e/f4c x:0 0 0 2024-05-17T21:52:51.672 INFO:tasks.workunit.client.1.smithi138.stdout:0/300: rename d3/d10/d38/d3e/f57 to d3/d3d/d51/f6e 0 2024-05-17T21:52:51.673 INFO:tasks.workunit.client.0.smithi107.stdout:3/375: dread d6/f1c [0,4194304] 0 2024-05-17T21:52:51.675 INFO:tasks.workunit.client.1.smithi138.stdout:3/272: dwrite d1/d3/f11 [0,4194304] 0 2024-05-17T21:52:51.675 INFO:tasks.workunit.client.1.smithi138.stdout:6/289: mkdir d7/db/d60 0 2024-05-17T21:52:51.677 INFO:tasks.workunit.client.1.smithi138.stdout:6/290: read d7/db/d43/d3f/f56 [2632381,39689] 0 2024-05-17T21:52:51.679 INFO:tasks.workunit.client.1.smithi138.stdout:3/273: truncate d1/d3/d8/de/f3b 1005333 0 2024-05-17T21:52:51.680 INFO:tasks.workunit.client.1.smithi138.stdout:4/222: truncate d0/d3/d1d/f2e 1587720 0 2024-05-17T21:52:51.682 INFO:tasks.workunit.client.1.smithi138.stdout:3/274: fsync d1/d3/d16/f30 0 2024-05-17T21:52:51.683 INFO:tasks.workunit.client.1.smithi138.stdout:0/301: dwrite d3/d3d/d51/f54 [0,4194304] 0 2024-05-17T21:52:51.683 INFO:tasks.workunit.client.1.smithi138.stdout:9/222: getdents d6/d7/d16/d29 0 2024-05-17T21:52:51.685 INFO:tasks.workunit.client.1.smithi138.stdout:3/275: chown d1/d3/d16/d3a 1146 1 2024-05-17T21:52:51.685 INFO:tasks.workunit.client.0.smithi107.stdout:8/385: creat dc/d21/d27/d83/f91 x:0 0 0 2024-05-17T21:52:51.687 INFO:tasks.workunit.client.0.smithi107.stdout:1/408: dread d4/de/f36 [0,4194304] 0 2024-05-17T21:52:51.690 INFO:tasks.workunit.client.1.smithi138.stdout:7/166: getdents d0/d18 0 2024-05-17T21:52:51.695 INFO:tasks.workunit.client.1.smithi138.stdout:2/230: chown d3/c1b 1722768394 1 2024-05-17T21:52:51.697 INFO:tasks.workunit.client.1.smithi138.stdout:0/302: dwrite d3/d3d/f58 [0,4194304] 0 2024-05-17T21:52:51.700 INFO:tasks.workunit.client.0.smithi107.stdout:7/345: creat d5/df/d11/f7a x:0 0 0 2024-05-17T21:52:51.700 INFO:tasks.workunit.client.0.smithi107.stdout:6/322: read d9/fa [2682229,47577] 0 2024-05-17T21:52:51.702 INFO:tasks.workunit.client.0.smithi107.stdout:2/287: rename d0/f6 to d0/d9/d11/d15/f57 0 2024-05-17T21:52:51.707 INFO:tasks.workunit.client.1.smithi138.stdout:6/291: dread d7/f28 [0,4194304] 0 2024-05-17T21:52:51.707 INFO:tasks.workunit.client.1.smithi138.stdout:5/200: symlink d1/d1a/l3c 0 2024-05-17T21:52:51.707 INFO:tasks.workunit.client.0.smithi107.stdout:0/446: link d3/dc/d22/d3d/c4c d3/d58/d61/d94/d9a/ca3 0 2024-05-17T21:52:51.708 INFO:tasks.workunit.client.0.smithi107.stdout:8/386: dread dc/d10/f15 [0,4194304] 0 2024-05-17T21:52:51.709 INFO:tasks.workunit.client.0.smithi107.stdout:8/387: dread - dc/d21/d27/d83/f91 zero size 2024-05-17T21:52:51.710 INFO:tasks.workunit.client.0.smithi107.stdout:8/388: write dc/d21/f80 [92969,78066] 0 2024-05-17T21:52:51.711 INFO:tasks.workunit.client.0.smithi107.stdout:5/378: write d5/db/d16/d5b/f3f [5429392,84594] 0 2024-05-17T21:52:51.712 INFO:tasks.workunit.client.0.smithi107.stdout:9/392: mkdir d0/dd/d2f/d87 0 2024-05-17T21:52:51.713 INFO:tasks.workunit.client.0.smithi107.stdout:9/393: write d0/dd/f60 [3680982,75124] 0 2024-05-17T21:52:51.715 INFO:tasks.workunit.client.0.smithi107.stdout:3/376: rename d6/d8/d10/d25/l26 to d6/d8/d10/d35/l79 0 2024-05-17T21:52:51.717 INFO:tasks.workunit.client.0.smithi107.stdout:0/447: truncate d3/dc/d22/f40 507851 0 2024-05-17T21:52:51.717 INFO:tasks.workunit.client.0.smithi107.stdout:0/448: dread - d3/d24/d6f/f8b zero size 2024-05-17T21:52:51.718 INFO:tasks.workunit.client.1.smithi138.stdout:1/194: getdents d4/df 0 2024-05-17T21:52:51.721 INFO:tasks.workunit.client.0.smithi107.stdout:5/379: dread - d5/db/d39/f3c zero size 2024-05-17T21:52:51.723 INFO:tasks.workunit.client.1.smithi138.stdout:1/195: dread d4/db/f21 [0,4194304] 0 2024-05-17T21:52:51.724 INFO:tasks.workunit.client.0.smithi107.stdout:7/346: symlink d5/l7b 0 2024-05-17T21:52:51.725 INFO:tasks.workunit.client.0.smithi107.stdout:7/347: write d5/fe [4301026,92073] 0 2024-05-17T21:52:51.727 INFO:tasks.workunit.client.0.smithi107.stdout:7/348: stat d5/df/d11/d1d/d44 0 2024-05-17T21:52:51.729 INFO:tasks.workunit.client.1.smithi138.stdout:6/292: rename d7/l51 to d7/db/d43/d3c/l61 0 2024-05-17T21:52:51.730 INFO:tasks.workunit.client.1.smithi138.stdout:5/201: mknod d1/d6/d1b/d29/c3d 0 2024-05-17T21:52:51.731 INFO:tasks.workunit.client.0.smithi107.stdout:5/380: mknod d5/db/d16/d54/c78 0 2024-05-17T21:52:51.731 INFO:tasks.workunit.client.0.smithi107.stdout:3/377: dread d6/d8/d10/d25/f47 [0,4194304] 0 2024-05-17T21:52:51.732 INFO:tasks.workunit.client.1.smithi138.stdout:5/202: write d1/d6/d1b/d29/d36/f37 [854418,4458] 0 2024-05-17T21:52:51.732 INFO:tasks.workunit.client.0.smithi107.stdout:5/381: write d5/db/dc/f56 [2392729,14865] 0 2024-05-17T21:52:51.733 INFO:tasks.workunit.client.0.smithi107.stdout:9/394: mknod d0/dd/d2f/d87/c88 0 2024-05-17T21:52:51.734 INFO:tasks.workunit.client.0.smithi107.stdout:9/395: fsync d0/dd/d1b/f7b 0 2024-05-17T21:52:51.735 INFO:tasks.workunit.client.0.smithi107.stdout:9/396: write d0/dd/d1f/d2c/f47 [1322350,1878] 0 2024-05-17T21:52:51.736 INFO:tasks.workunit.client.0.smithi107.stdout:9/397: truncate d0/dd/d1b/f7b 574928 0 2024-05-17T21:52:51.737 INFO:tasks.workunit.client.0.smithi107.stdout:7/349: creat d5/df/d11/d1c/d2f/f7c x:0 0 0 2024-05-17T21:52:51.740 INFO:tasks.workunit.client.0.smithi107.stdout:1/409: getdents d4/df/d14/d2e/d43/d47 0 2024-05-17T21:52:51.741 INFO:tasks.workunit.client.0.smithi107.stdout:1/410: read d4/df/d14/f4c [3691982,55765] 0 2024-05-17T21:52:51.741 INFO:tasks.workunit.client.1.smithi138.stdout:4/223: creat d0/d3/f57 x:0 0 0 2024-05-17T21:52:51.744 INFO:tasks.workunit.client.1.smithi138.stdout:9/223: truncate d6/d7/d16/d29/f3a 2801145 0 2024-05-17T21:52:51.746 INFO:tasks.workunit.client.1.smithi138.stdout:0/303: rename d3/d10/d39/c61 to d3/d10/d38/d3e/d59/c6f 0 2024-05-17T21:52:51.746 INFO:tasks.workunit.client.0.smithi107.stdout:1/411: dwrite d4/de/f67 [0,4194304] 0 2024-05-17T21:52:51.747 INFO:tasks.workunit.client.0.smithi107.stdout:5/382: mknod d5/db/dc/d1e/c79 0 2024-05-17T21:52:51.747 INFO:tasks.workunit.client.0.smithi107.stdout:1/412: readlink d4/df/d1a/d26/d7a/l7e 0 2024-05-17T21:52:51.748 INFO:tasks.workunit.client.1.smithi138.stdout:4/224: dwrite d0/d1b/d42/d45/f48 [0,4194304] 0 2024-05-17T21:52:51.749 INFO:tasks.workunit.client.0.smithi107.stdout:1/413: read d4/de/f19 [3847654,53497] 0 2024-05-17T21:52:51.768 INFO:tasks.workunit.client.0.smithi107.stdout:4/392: sync 2024-05-17T21:52:51.769 INFO:tasks.workunit.client.0.smithi107.stdout:4/393: write d3/d4a/f73 [410615,7552] 0 2024-05-17T21:52:51.828 INFO:tasks.workunit.client.0.smithi107.stdout:8/389: sync 2024-05-17T21:52:51.829 INFO:tasks.workunit.client.0.smithi107.stdout:2/288: sync 2024-05-17T21:52:51.829 INFO:tasks.workunit.client.0.smithi107.stdout:2/289: write d0/f29 [1559550,79964] 0 2024-05-17T21:52:51.829 INFO:tasks.workunit.client.0.smithi107.stdout:2/290: chown d0/f25 28113924 1 2024-05-17T21:52:51.831 INFO:tasks.workunit.client.0.smithi107.stdout:8/390: dread dc/f43 [0,4194304] 0 2024-05-17T21:52:51.831 INFO:tasks.workunit.client.0.smithi107.stdout:8/391: fdatasync dc/d21/d35/f36 0 2024-05-17T21:52:51.884 INFO:tasks.workunit.client.0.smithi107.stdout:6/323: rename d9/d16/d22/d2d/f3d to d9/d16/f5d 0 2024-05-17T21:52:51.885 INFO:tasks.workunit.client.0.smithi107.stdout:6/324: readlink d9/d16/d22/l57 0 2024-05-17T21:52:51.885 INFO:tasks.workunit.client.0.smithi107.stdout:6/325: stat d9/fa 0 2024-05-17T21:52:51.885 INFO:tasks.workunit.client.1.smithi138.stdout:7/167: creat d0/f2c x:0 0 0 2024-05-17T21:52:51.886 INFO:tasks.workunit.client.1.smithi138.stdout:1/196: write f0 [4522414,101046] 0 2024-05-17T21:52:51.889 INFO:tasks.workunit.client.1.smithi138.stdout:3/276: link d1/d3/cb d1/d3/d16/d2d/c61 0 2024-05-17T21:52:51.890 INFO:tasks.workunit.client.1.smithi138.stdout:2/231: dwrite d3/f35 [0,4194304] 0 2024-05-17T21:52:51.891 INFO:tasks.workunit.client.1.smithi138.stdout:2/232: read - d3/d21/d2f/f3c zero size 2024-05-17T21:52:51.893 INFO:tasks.workunit.client.1.smithi138.stdout:2/233: dread - d3/d21/f31 zero size 2024-05-17T21:52:51.893 INFO:tasks.workunit.client.1.smithi138.stdout:9/224: mknod d6/d7/d16/c43 0 2024-05-17T21:52:51.897 INFO:tasks.workunit.client.1.smithi138.stdout:6/293: rename d7/db/d43/d3c/c44 to d7/db/d12/c62 0 2024-05-17T21:52:51.898 INFO:tasks.workunit.client.1.smithi138.stdout:6/294: write d7/db/d43/f4f [702278,112721] 0 2024-05-17T21:52:51.901 INFO:tasks.workunit.client.0.smithi107.stdout:1/414: mkdir d4/df/d1a/d26/d72/d85 0 2024-05-17T21:52:51.904 INFO:tasks.workunit.client.1.smithi138.stdout:5/203: stat d1/d1a/c27 0 2024-05-17T21:52:51.908 INFO:tasks.workunit.client.1.smithi138.stdout:7/168: stat d0/c22 0 2024-05-17T21:52:51.908 INFO:tasks.workunit.client.0.smithi107.stdout:4/394: dwrite d3/d4a/d47/d60/f5b [0,4194304] 0 2024-05-17T21:52:51.910 INFO:tasks.workunit.client.0.smithi107.stdout:4/395: dread d3/d4a/d47/d60/d3a/d41/f71 [0,4194304] 0 2024-05-17T21:52:51.916 INFO:tasks.workunit.client.0.smithi107.stdout:9/398: unlink d0/f4 0 2024-05-17T21:52:51.916 INFO:tasks.workunit.client.0.smithi107.stdout:9/399: dread - d0/dd/d69/f83 zero size 2024-05-17T21:52:51.918 INFO:tasks.workunit.client.0.smithi107.stdout:3/378: rename d6/l72 to d6/l7a 0 2024-05-17T21:52:51.919 INFO:tasks.workunit.client.0.smithi107.stdout:3/379: write d6/d8/d10/d35/d36/d4c/f65 [493431,38672] 0 2024-05-17T21:52:51.920 INFO:tasks.workunit.client.1.smithi138.stdout:2/234: fsync d3/d21/d2f/f3c 0 2024-05-17T21:52:51.923 INFO:tasks.workunit.client.1.smithi138.stdout:4/225: dwrite d0/d3/d1d/d4c/f4e [0,4194304] 0 2024-05-17T21:52:51.923 INFO:tasks.workunit.client.1.smithi138.stdout:6/295: mkdir d7/db/d43/d63 0 2024-05-17T21:52:51.929 INFO:tasks.workunit.client.1.smithi138.stdout:5/204: mknod d1/d24/c3e 0 2024-05-17T21:52:51.930 INFO:tasks.workunit.client.0.smithi107.stdout:5/383: truncate d5/db/f1a 3710843 0 2024-05-17T21:52:51.931 INFO:tasks.workunit.client.1.smithi138.stdout:5/205: write d1/d6/fe [5530367,9752] 0 2024-05-17T21:52:51.933 INFO:tasks.workunit.client.1.smithi138.stdout:7/169: chown d0/cb 0 1 2024-05-17T21:52:51.935 INFO:tasks.workunit.client.0.smithi107.stdout:9/400: mkdir d0/dd/d2f/d87/d89 0 2024-05-17T21:52:51.937 INFO:tasks.workunit.client.1.smithi138.stdout:3/277: mkdir d1/d62 0 2024-05-17T21:52:51.939 INFO:tasks.workunit.client.1.smithi138.stdout:9/225: mkdir d6/d7/d16/d29/d38/d40/d44 0 2024-05-17T21:52:51.944 INFO:tasks.workunit.client.0.smithi107.stdout:5/384: symlink d5/db/d16/d5b/l7a 0 2024-05-17T21:52:51.945 INFO:tasks.workunit.client.1.smithi138.stdout:2/235: symlink d3/d21/d2f/d3e/l4d 0 2024-05-17T21:52:51.948 INFO:tasks.workunit.client.1.smithi138.stdout:4/226: mknod d0/d1b/d36/c58 0 2024-05-17T21:52:51.950 INFO:tasks.workunit.client.0.smithi107.stdout:0/449: rename d3/l4b to d3/d75/la4 0 2024-05-17T21:52:51.952 INFO:tasks.workunit.client.0.smithi107.stdout:6/326: rename d9/d16/f5d to d9/d16/f5e 0 2024-05-17T21:52:51.952 INFO:tasks.workunit.client.0.smithi107.stdout:6/327: dread - d9/f48 zero size 2024-05-17T21:52:51.953 INFO:tasks.workunit.client.0.smithi107.stdout:6/328: write d9/d16/d22/d2d/f5c [78678,100114] 0 2024-05-17T21:52:51.954 INFO:tasks.workunit.client.1.smithi138.stdout:0/304: rmdir d3/d3d/d6b 0 2024-05-17T21:52:51.955 INFO:tasks.workunit.client.0.smithi107.stdout:9/401: fsync d0/f10 0 2024-05-17T21:52:51.959 INFO:tasks.workunit.client.0.smithi107.stdout:3/380: link d6/d1a/d57/f73 d6/d8/d10/d35/d36/d66/f7b 0 2024-05-17T21:52:51.959 INFO:tasks.workunit.client.0.smithi107.stdout:3/381: write d6/d2a/f78 [292858,45970] 0 2024-05-17T21:52:51.960 INFO:tasks.workunit.client.1.smithi138.stdout:5/206: mknod d1/d6/d19/c3f 0 2024-05-17T21:52:51.961 INFO:tasks.workunit.client.1.smithi138.stdout:4/227: dread d0/d3/d10/f25 [0,4194304] 0 2024-05-17T21:52:51.961 INFO:tasks.workunit.client.0.smithi107.stdout:1/415: link d4/l20 d4/df/d1a/d26/d50/l86 0 2024-05-17T21:52:51.962 INFO:tasks.workunit.client.1.smithi138.stdout:4/228: dread - d0/d3/d10/f41 zero size 2024-05-17T21:52:51.962 INFO:tasks.workunit.client.1.smithi138.stdout:7/170: readlink d0/l26 0 2024-05-17T21:52:51.964 INFO:tasks.workunit.client.0.smithi107.stdout:5/385: unlink d5/f17 0 2024-05-17T21:52:51.965 INFO:tasks.workunit.client.0.smithi107.stdout:5/386: write d5/db/d16/d54/d59/d65/f6f [5201053,77862] 0 2024-05-17T21:52:51.965 INFO:tasks.workunit.client.0.smithi107.stdout:7/350: rename d5/d16/l38 to d5/df/d11/d2a/l7d 0 2024-05-17T21:52:51.966 INFO:tasks.workunit.client.0.smithi107.stdout:7/351: write d5/d16/f1e [2800137,66453] 0 2024-05-17T21:52:51.967 INFO:tasks.workunit.client.1.smithi138.stdout:9/226: mkdir d6/d7/d16/d29/d45 0 2024-05-17T21:52:51.971 INFO:tasks.workunit.client.0.smithi107.stdout:3/382: creat d6/d2a/d2d/d37/f7c x:0 0 0 2024-05-17T21:52:51.972 INFO:tasks.workunit.client.1.smithi138.stdout:2/236: dwrite d3/f1a [0,4194304] 0 2024-05-17T21:52:51.974 INFO:tasks.workunit.client.1.smithi138.stdout:0/305: dread d3/f45 [0,4194304] 0 2024-05-17T21:52:51.974 INFO:tasks.workunit.client.0.smithi107.stdout:1/416: fdatasync d4/df/d14/f23 0 2024-05-17T21:52:51.975 INFO:tasks.workunit.client.1.smithi138.stdout:5/207: symlink d1/d6/d1b/d29/l40 0 2024-05-17T21:52:51.975 INFO:tasks.workunit.client.1.smithi138.stdout:5/208: readlink d1/d6/l15 0 2024-05-17T21:52:51.978 INFO:tasks.workunit.client.1.smithi138.stdout:7/171: mknod d0/d18/c2d 0 2024-05-17T21:52:51.980 INFO:tasks.workunit.client.0.smithi107.stdout:0/450: mkdir d3/d58/d61/d94/d9a/da5 0 2024-05-17T21:52:51.981 INFO:tasks.workunit.client.1.smithi138.stdout:1/197: getdents d4/df 0 2024-05-17T21:52:51.982 INFO:tasks.workunit.client.1.smithi138.stdout:5/209: dwrite d1/f7 [0,4194304] 0 2024-05-17T21:52:51.983 INFO:tasks.workunit.client.0.smithi107.stdout:5/387: mknod d5/db/d16/d2d/d4d/c7b 0 2024-05-17T21:52:51.984 INFO:tasks.workunit.client.1.smithi138.stdout:5/210: write d1/d6/d1b/d29/d36/f37 [631048,24726] 0 2024-05-17T21:52:51.985 INFO:tasks.workunit.client.1.smithi138.stdout:5/211: readlink d1/d1a/l3c 0 2024-05-17T21:52:51.987 INFO:tasks.workunit.client.0.smithi107.stdout:4/396: sync 2024-05-17T21:52:51.987 INFO:tasks.workunit.client.1.smithi138.stdout:5/212: write d1/d6/f31 [420997,84746] 0 2024-05-17T21:52:51.989 INFO:tasks.workunit.client.0.smithi107.stdout:7/352: readlink d5/df/l1b 0 2024-05-17T21:52:51.990 INFO:tasks.workunit.client.0.smithi107.stdout:5/388: dwrite d5/db/d16/d2d/d32/f38 [0,4194304] 0 2024-05-17T21:52:51.991 INFO:tasks.workunit.client.0.smithi107.stdout:7/353: chown d5/df/d11/d1d/d33/f64 16275 1 2024-05-17T21:52:51.991 INFO:tasks.workunit.client.1.smithi138.stdout:9/227: mknod d6/d7/d16/d29/d38/c46 0 2024-05-17T21:52:51.991 INFO:tasks.workunit.client.1.smithi138.stdout:9/228: fdatasync d6/f25 0 2024-05-17T21:52:51.993 INFO:tasks.workunit.client.1.smithi138.stdout:9/229: truncate d6/d7/d16/d29/d38/f3d 618690 0 2024-05-17T21:52:51.995 INFO:tasks.workunit.client.0.smithi107.stdout:6/329: truncate d9/f53 1839856 0 2024-05-17T21:52:51.998 INFO:tasks.workunit.client.0.smithi107.stdout:6/330: truncate d9/d16/f5e 666202 0 2024-05-17T21:52:51.998 INFO:tasks.workunit.client.0.smithi107.stdout:5/389: dwrite d5/db/d16/d2a/f64 [0,4194304] 0 2024-05-17T21:52:51.999 INFO:tasks.workunit.client.0.smithi107.stdout:3/383: read d6/d8/f12 [515180,7818] 0 2024-05-17T21:52:52.001 INFO:tasks.workunit.client.1.smithi138.stdout:2/237: symlink d3/d21/d2f/d3e/l4e 0 2024-05-17T21:52:52.002 INFO:tasks.workunit.client.1.smithi138.stdout:2/238: write d3/d21/f2b [343344,63271] 0 2024-05-17T21:52:52.003 INFO:tasks.workunit.client.0.smithi107.stdout:0/451: symlink d3/d58/d61/d94/la6 0 2024-05-17T21:52:52.004 INFO:tasks.workunit.client.1.smithi138.stdout:0/306: read d3/d10/f32 [1297111,119825] 0 2024-05-17T21:52:52.006 INFO:tasks.workunit.client.0.smithi107.stdout:4/397: creat d3/d4a/d47/d60/d3a/d41/f88 x:0 0 0 2024-05-17T21:52:52.011 INFO:tasks.workunit.client.0.smithi107.stdout:7/354: write d5/df/d11/d1d/f55 [4227273,6410] 0 2024-05-17T21:52:52.011 INFO:tasks.workunit.client.0.smithi107.stdout:7/355: stat d5/df/d11/d1c/d2f/d53 0 2024-05-17T21:52:52.017 INFO:tasks.workunit.client.0.smithi107.stdout:9/402: creat d0/dd/d21/d29/d57/f8a x:0 0 0 2024-05-17T21:52:52.017 INFO:tasks.workunit.client.1.smithi138.stdout:3/278: dwrite d1/d3/d8/f24 [0,4194304] 0 2024-05-17T21:52:52.018 INFO:tasks.workunit.client.1.smithi138.stdout:9/230: unlink d6/d7/d1f/c2f 0 2024-05-17T21:52:52.019 INFO:tasks.workunit.client.0.smithi107.stdout:4/398: dread d3/d4a/d47/d60/f10 [0,4194304] 0 2024-05-17T21:52:52.021 INFO:tasks.workunit.client.1.smithi138.stdout:6/296: getdents d7/d8 0 2024-05-17T21:52:52.023 INFO:tasks.workunit.client.1.smithi138.stdout:2/239: mknod d3/d21/d2f/c4f 0 2024-05-17T21:52:52.026 INFO:tasks.workunit.client.0.smithi107.stdout:5/390: unlink d5/db/dc/l57 0 2024-05-17T21:52:52.027 INFO:tasks.workunit.client.1.smithi138.stdout:0/307: rmdir d3/d10/d38/d3e 39 2024-05-17T21:52:52.031 INFO:tasks.workunit.client.0.smithi107.stdout:8/392: rename dc/d21/d35/f11 to dc/d21/f92 0 2024-05-17T21:52:52.032 INFO:tasks.workunit.client.1.smithi138.stdout:1/198: rename d4/db/c26 to d4/df/d3a/c41 0 2024-05-17T21:52:52.033 INFO:tasks.workunit.client.1.smithi138.stdout:1/199: readlink d4/db/l33 0 2024-05-17T21:52:52.036 INFO:tasks.workunit.client.0.smithi107.stdout:5/391: dread d5/db/dc/f56 [0,4194304] 0 2024-05-17T21:52:52.039 INFO:tasks.workunit.client.0.smithi107.stdout:7/356: mknod d5/df/d11/d1c/d2f/d65/c7e 0 2024-05-17T21:52:52.040 INFO:tasks.workunit.client.0.smithi107.stdout:7/357: chown d5/d16/d3b/d45/f6c 1147201 1 2024-05-17T21:52:52.041 INFO:tasks.workunit.client.1.smithi138.stdout:9/231: mknod d6/d7/d16/d29/d38/c47 0 2024-05-17T21:52:52.041 INFO:tasks.workunit.client.1.smithi138.stdout:9/232: chown d6/d7/f9 1545964 1 2024-05-17T21:52:52.043 INFO:tasks.workunit.client.0.smithi107.stdout:7/358: dread d5/df/d11/d1d/d33/f64 [0,4194304] 0 2024-05-17T21:52:52.044 INFO:tasks.workunit.client.0.smithi107.stdout:9/403: rmdir d0 39 2024-05-17T21:52:52.044 INFO:tasks.workunit.client.0.smithi107.stdout:7/359: readlink d5/df/d11/d1c/d2f/l77 0 2024-05-17T21:52:52.045 INFO:tasks.workunit.client.1.smithi138.stdout:6/297: creat d7/db/d43/f64 x:0 0 0 2024-05-17T21:52:52.046 INFO:tasks.workunit.client.1.smithi138.stdout:6/298: chown d7/d8/c54 0 1 2024-05-17T21:52:52.047 INFO:tasks.workunit.client.0.smithi107.stdout:6/331: mkdir d9/d16/d22/d5f 0 2024-05-17T21:52:52.047 INFO:tasks.workunit.client.1.smithi138.stdout:6/299: chown d7/db/d34/l48 77647596 1 2024-05-17T21:52:52.047 INFO:tasks.workunit.client.1.smithi138.stdout:2/240: readlink d3/l26 0 2024-05-17T21:52:52.048 INFO:tasks.workunit.client.1.smithi138.stdout:2/241: write f2 [4122025,118551] 0 2024-05-17T21:52:52.048 INFO:tasks.workunit.client.0.smithi107.stdout:4/399: creat d3/d6/d26/d36/d7e/f89 x:0 0 0 2024-05-17T21:52:52.049 INFO:tasks.workunit.client.0.smithi107.stdout:4/400: chown d3/db/d35 32 1 2024-05-17T21:52:52.050 INFO:tasks.workunit.client.0.smithi107.stdout:4/401: write d3/d4a/d47/d60/d3a/d41/d80/f45 [3549701,76305] 0 2024-05-17T21:52:52.051 INFO:tasks.workunit.client.1.smithi138.stdout:4/229: getdents d0/d1b/d42/d45 0 2024-05-17T21:52:52.051 INFO:tasks.workunit.client.1.smithi138.stdout:4/230: dread - d0/d3/d10/f41 zero size 2024-05-17T21:52:52.053 INFO:tasks.workunit.client.0.smithi107.stdout:3/384: symlink d6/l7d 0 2024-05-17T21:52:52.053 INFO:tasks.workunit.client.0.smithi107.stdout:3/385: stat d6/d8/d10/f4b 0 2024-05-17T21:52:52.055 INFO:tasks.workunit.client.1.smithi138.stdout:7/172: truncate d0/f1f 5271110 0 2024-05-17T21:52:52.058 INFO:tasks.workunit.client.1.smithi138.stdout:5/213: rename d1/d6/d1b/f32 to d1/d6/f41 0 2024-05-17T21:52:52.059 INFO:tasks.workunit.client.1.smithi138.stdout:0/308: dwrite d3/d10/d38/d3e/f5b [0,4194304] 0 2024-05-17T21:52:52.060 INFO:tasks.workunit.client.1.smithi138.stdout:1/200: mknod d4/df/d3a/c42 0 2024-05-17T21:52:52.063 INFO:tasks.workunit.client.0.smithi107.stdout:2/291: rename d0/c19 to d0/d9/d1c/d3e/d4f/c58 0 2024-05-17T21:52:52.064 INFO:tasks.workunit.client.1.smithi138.stdout:3/279: mknod d1/d4f/c63 0 2024-05-17T21:52:52.064 INFO:tasks.workunit.client.1.smithi138.stdout:6/300: dread d7/d8/ff [0,4194304] 0 2024-05-17T21:52:52.068 INFO:tasks.workunit.client.0.smithi107.stdout:3/386: dread d6/d2a/d2d/d69/f6d [0,4194304] 0 2024-05-17T21:52:52.071 INFO:tasks.workunit.client.1.smithi138.stdout:3/280: dwrite d1/d3/d8/f28 [0,4194304] 0 2024-05-17T21:52:52.078 INFO:tasks.workunit.client.1.smithi138.stdout:9/233: rename d6/d7/d1f to d6/d7/d16/d29/d48 0 2024-05-17T21:52:52.081 INFO:tasks.workunit.client.1.smithi138.stdout:9/234: truncate d6/f2d 661003 0 2024-05-17T21:52:52.081 INFO:tasks.workunit.client.1.smithi138.stdout:0/309: fdatasync d3/f29 0 2024-05-17T21:52:52.082 INFO:tasks.workunit.client.1.smithi138.stdout:1/201: creat d4/df/d35/f43 x:0 0 0 2024-05-17T21:52:52.083 INFO:tasks.workunit.client.0.smithi107.stdout:1/417: link d4/c64 d4/df/d14/c87 0 2024-05-17T21:52:52.083 INFO:tasks.workunit.client.1.smithi138.stdout:3/281: dwrite d1/d3/d16/d3e/f4e [0,4194304] 0 2024-05-17T21:52:52.084 INFO:tasks.workunit.client.1.smithi138.stdout:9/235: fsync d6/d7/d16/d29/d48/f22 0 2024-05-17T21:52:52.086 INFO:tasks.workunit.client.1.smithi138.stdout:2/242: link d3/f1c d3/d21/d2f/f50 0 2024-05-17T21:52:52.088 INFO:tasks.workunit.client.1.smithi138.stdout:2/243: chown d3/d21/d2f/l3a 8914 1 2024-05-17T21:52:52.091 INFO:tasks.workunit.client.1.smithi138.stdout:2/244: write d3/d21/d2f/f39 [682127,95225] 0 2024-05-17T21:52:52.091 INFO:tasks.workunit.client.1.smithi138.stdout:5/214: dwrite d1/d1a/f26 [4194304,4194304] 0 2024-05-17T21:52:52.092 INFO:tasks.workunit.client.1.smithi138.stdout:2/245: readlink d3/d21/l46 0 2024-05-17T21:52:52.092 INFO:tasks.workunit.client.0.smithi107.stdout:8/393: mknod dc/d21/d27/d2f/c93 0 2024-05-17T21:52:52.097 INFO:tasks.workunit.client.1.smithi138.stdout:1/202: creat d4/df/f44 x:0 0 0 2024-05-17T21:52:52.098 INFO:tasks.workunit.client.1.smithi138.stdout:1/203: stat d4/dd/f2f 0 2024-05-17T21:52:52.102 INFO:tasks.workunit.client.1.smithi138.stdout:0/310: fdatasync d3/d10/f32 0 2024-05-17T21:52:52.106 INFO:tasks.workunit.client.1.smithi138.stdout:3/282: truncate d1/d3/f10 6516421 0 2024-05-17T21:52:52.111 INFO:tasks.workunit.client.0.smithi107.stdout:4/402: mkdir d3/db/d35/d55/d8a 0 2024-05-17T21:52:52.114 INFO:tasks.workunit.client.1.smithi138.stdout:7/173: rename d0/c4 to d0/d18/c2e 0 2024-05-17T21:52:52.118 INFO:tasks.workunit.client.1.smithi138.stdout:4/231: dwrite d0/d3/d17/f1a [0,4194304] 0 2024-05-17T21:52:52.119 INFO:tasks.workunit.client.1.smithi138.stdout:4/232: read d0/d1b/d42/d45/f48 [931511,1363] 0 2024-05-17T21:52:52.121 INFO:tasks.workunit.client.0.smithi107.stdout:1/418: symlink d4/df/d69/l88 0 2024-05-17T21:52:52.123 INFO:tasks.workunit.client.1.smithi138.stdout:1/204: symlink d4/df/d35/l45 0 2024-05-17T21:52:52.125 INFO:tasks.workunit.client.0.smithi107.stdout:2/292: symlink d0/d9/l59 0 2024-05-17T21:52:52.125 INFO:tasks.workunit.client.0.smithi107.stdout:1/419: dread d4/df/d1a/d26/f7c [0,4194304] 0 2024-05-17T21:52:52.127 INFO:tasks.workunit.client.1.smithi138.stdout:4/233: dwrite d0/d1b/d42/d45/f48 [0,4194304] 0 2024-05-17T21:52:52.127 INFO:tasks.workunit.client.1.smithi138.stdout:4/234: chown d0 477 1 2024-05-17T21:52:52.130 INFO:tasks.workunit.client.0.smithi107.stdout:0/452: rmdir d3/dc/d22/d3d/d56/d64 0 2024-05-17T21:52:52.133 INFO:tasks.workunit.client.0.smithi107.stdout:5/392: mkdir d5/db/d16/d2d/d40/d5a/d7c 0 2024-05-17T21:52:52.135 INFO:tasks.workunit.client.1.smithi138.stdout:3/283: write d1/d3/d8/de/f27 [4142412,84450] 0 2024-05-17T21:52:52.135 INFO:tasks.workunit.client.0.smithi107.stdout:3/387: mknod d6/d8/d3e/d3f/c7e 0 2024-05-17T21:52:52.137 INFO:tasks.workunit.client.1.smithi138.stdout:2/246: symlink d3/l51 0 2024-05-17T21:52:52.138 INFO:tasks.workunit.client.0.smithi107.stdout:9/404: mknod d0/dd/d2f/d4a/c8b 0 2024-05-17T21:52:52.141 INFO:tasks.workunit.client.0.smithi107.stdout:4/403: mkdir d3/d6/d26/d36/d8b 0 2024-05-17T21:52:52.142 INFO:tasks.workunit.client.1.smithi138.stdout:3/284: dread d1/d3/d16/d3e/f43 [0,4194304] 0 2024-05-17T21:52:52.146 INFO:tasks.workunit.client.1.smithi138.stdout:7/174: dwrite d0/f27 [0,4194304] 0 2024-05-17T21:52:52.147 INFO:tasks.workunit.client.1.smithi138.stdout:7/175: dread - d0/f2b zero size 2024-05-17T21:52:52.148 INFO:tasks.workunit.client.0.smithi107.stdout:2/293: dread d0/f36 [0,4194304] 0 2024-05-17T21:52:52.150 INFO:tasks.workunit.client.0.smithi107.stdout:8/394: creat dc/d10/d63/d71/f94 x:0 0 0 2024-05-17T21:52:52.152 INFO:tasks.workunit.client.1.smithi138.stdout:4/235: write d0/d1b/f3e [691417,73814] 0 2024-05-17T21:52:52.152 INFO:tasks.workunit.client.0.smithi107.stdout:8/395: read dc/f67 [641714,55481] 0 2024-05-17T21:52:52.153 INFO:tasks.workunit.client.0.smithi107.stdout:7/360: truncate d5/f6d 2311889 0 2024-05-17T21:52:52.155 INFO:tasks.workunit.client.0.smithi107.stdout:4/404: dread d3/fa [0,4194304] 0 2024-05-17T21:52:52.156 INFO:tasks.workunit.client.1.smithi138.stdout:9/236: creat d6/f49 x:0 0 0 2024-05-17T21:52:52.157 INFO:tasks.workunit.client.1.smithi138.stdout:2/247: rmdir d3/d21/d2f/d3e 39 2024-05-17T21:52:52.159 INFO:tasks.workunit.client.0.smithi107.stdout:6/332: getdents d9/d16/d4d 0 2024-05-17T21:52:52.169 INFO:tasks.workunit.client.1.smithi138.stdout:1/205: readlink d4/db/d29/l2b 0 2024-05-17T21:52:52.170 INFO:tasks.workunit.client.1.smithi138.stdout:0/311: dwrite d3/f2b [0,4194304] 0 2024-05-17T21:52:52.171 INFO:tasks.workunit.client.1.smithi138.stdout:0/312: write d3/d3d/f53 [4479919,73548] 0 2024-05-17T21:52:52.172 INFO:tasks.workunit.client.1.smithi138.stdout:4/236: symlink d0/d2f/l59 0 2024-05-17T21:52:52.176 INFO:tasks.workunit.client.0.smithi107.stdout:0/453: symlink d3/dc/d22/d3d/la7 0 2024-05-17T21:52:52.176 INFO:tasks.workunit.client.1.smithi138.stdout:4/237: dread d0/d3/d1d/d4c/f4e [0,4194304] 0 2024-05-17T21:52:52.177 INFO:tasks.workunit.client.0.smithi107.stdout:0/454: chown d3/d58/d61/d94/la6 3267 1 2024-05-17T21:52:52.177 INFO:tasks.workunit.client.1.smithi138.stdout:2/248: chown d3/l2a 40121 1 2024-05-17T21:52:52.177 INFO:tasks.workunit.client.0.smithi107.stdout:1/420: write d4/df/d14/f23 [1103283,129782] 0 2024-05-17T21:52:52.178 INFO:tasks.workunit.client.1.smithi138.stdout:2/249: chown d3/d21/l3d 46 1 2024-05-17T21:52:52.178 INFO:tasks.workunit.client.0.smithi107.stdout:7/361: mknod d5/df/d11/d1d/d33/c7f 0 2024-05-17T21:52:52.179 INFO:tasks.workunit.client.0.smithi107.stdout:0/455: chown d3/dc/d22/d3d/d56/l6a 5380438 1 2024-05-17T21:52:52.179 INFO:tasks.workunit.client.0.smithi107.stdout:7/362: stat d5/df/d11/d1c/l6b 0 2024-05-17T21:52:52.180 INFO:tasks.workunit.client.1.smithi138.stdout:5/215: getdents d1/d6/d1b 0 2024-05-17T21:52:52.180 INFO:tasks.workunit.client.0.smithi107.stdout:7/363: chown d5/df/d11/f25 15 1 2024-05-17T21:52:52.181 INFO:tasks.workunit.client.0.smithi107.stdout:5/393: fsync d5/db/d16/d5b/f2f 0 2024-05-17T21:52:52.181 INFO:tasks.workunit.client.1.smithi138.stdout:5/216: dread - d1/d2c/f35 zero size 2024-05-17T21:52:52.183 INFO:tasks.workunit.client.1.smithi138.stdout:5/217: chown d1/d6/f41 177847 1 2024-05-17T21:52:52.184 INFO:tasks.workunit.client.1.smithi138.stdout:5/218: chown d1/d6/fa 162 1 2024-05-17T21:52:52.184 INFO:tasks.workunit.client.1.smithi138.stdout:4/238: dwrite d0/d1b/f18 [0,4194304] 0 2024-05-17T21:52:52.185 INFO:tasks.workunit.client.1.smithi138.stdout:4/239: rename d0 to d0/d1b/d42/d5a 22 2024-05-17T21:52:52.186 INFO:tasks.workunit.client.1.smithi138.stdout:5/219: chown d1/d1a/l3c 1116812 1 2024-05-17T21:52:52.188 INFO:tasks.workunit.client.0.smithi107.stdout:3/388: symlink d6/d2a/d2d/d48/d4f/l7f 0 2024-05-17T21:52:52.188 INFO:tasks.workunit.client.0.smithi107.stdout:3/389: stat d6/d8/d3e/l71 0 2024-05-17T21:52:52.189 INFO:tasks.workunit.client.0.smithi107.stdout:3/390: chown d6/d2a/d2d 550362184 1 2024-05-17T21:52:52.189 INFO:tasks.workunit.client.0.smithi107.stdout:5/394: dread d5/db/d39/f70 [0,4194304] 0 2024-05-17T21:52:52.190 INFO:tasks.workunit.client.1.smithi138.stdout:7/176: symlink d0/l2f 0 2024-05-17T21:52:52.191 INFO:tasks.workunit.client.0.smithi107.stdout:6/333: creat d9/d16/d4d/f60 x:0 0 0 2024-05-17T21:52:52.194 INFO:tasks.workunit.client.1.smithi138.stdout:1/206: symlink d4/df/d35/l46 0 2024-05-17T21:52:52.195 INFO:tasks.workunit.client.0.smithi107.stdout:8/396: write dc/f43 [729486,24554] 0 2024-05-17T21:52:52.195 INFO:tasks.workunit.client.0.smithi107.stdout:8/397: stat dc/d21/d27/d2f/c93 0 2024-05-17T21:52:52.196 INFO:tasks.workunit.client.0.smithi107.stdout:2/294: mkdir d0/d9/d11/df/d5a 0 2024-05-17T21:52:52.202 INFO:tasks.workunit.client.0.smithi107.stdout:1/421: unlink d4/df/d1a/f56 0 2024-05-17T21:52:52.204 INFO:tasks.workunit.client.1.smithi138.stdout:3/285: truncate d1/d3/d16/d3e/f4a 1122811 0 2024-05-17T21:52:52.204 INFO:tasks.workunit.client.1.smithi138.stdout:9/237: mknod d6/d7/d16/d29/d45/c4a 0 2024-05-17T21:52:52.207 INFO:tasks.workunit.client.1.smithi138.stdout:7/177: dread d0/d18/f1d [0,4194304] 0 2024-05-17T21:52:52.208 INFO:tasks.workunit.client.0.smithi107.stdout:4/405: dwrite d3/db/f4d [0,4194304] 0 2024-05-17T21:52:52.209 INFO:tasks.workunit.client.1.smithi138.stdout:3/286: dread d1/d3/d8/de/f3f [0,4194304] 0 2024-05-17T21:52:52.209 INFO:tasks.workunit.client.1.smithi138.stdout:2/250: write d3/f5 [4592622,124856] 0 2024-05-17T21:52:52.210 INFO:tasks.workunit.client.1.smithi138.stdout:3/287: readlink d1/d2e/l39 0 2024-05-17T21:52:52.211 INFO:tasks.workunit.client.1.smithi138.stdout:3/288: chown d1/d3/f20 1203452 1 2024-05-17T21:52:52.212 INFO:tasks.workunit.client.1.smithi138.stdout:4/240: rmdir d0/d3/d17 39 2024-05-17T21:52:52.212 INFO:tasks.workunit.client.0.smithi107.stdout:6/334: dread d9/f15 [0,4194304] 0 2024-05-17T21:52:52.212 INFO:tasks.workunit.client.1.smithi138.stdout:3/289: read d1/f4 [867518,37728] 0 2024-05-17T21:52:52.216 INFO:tasks.workunit.client.1.smithi138.stdout:5/220: symlink d1/d6/l42 0 2024-05-17T21:52:52.220 INFO:tasks.workunit.client.0.smithi107.stdout:9/405: sync 2024-05-17T21:52:52.221 INFO:tasks.workunit.client.1.smithi138.stdout:1/207: mkdir d4/db/d29/d47 0 2024-05-17T21:52:52.223 INFO:tasks.workunit.client.1.smithi138.stdout:0/313: mkdir d3/d10/d38/d70 0 2024-05-17T21:52:52.227 INFO:tasks.workunit.client.0.smithi107.stdout:1/422: symlink d4/df/d14/d2e/d43/d47/l89 0 2024-05-17T21:52:52.227 INFO:tasks.workunit.client.0.smithi107.stdout:3/391: dwrite d6/d2a/d2d/d69/f6d [0,4194304] 0 2024-05-17T21:52:52.231 INFO:tasks.workunit.client.1.smithi138.stdout:7/178: truncate d0/f24 4792453 0 2024-05-17T21:52:52.232 INFO:tasks.workunit.client.1.smithi138.stdout:9/238: dwrite d6/d7/d16/f20 [4194304,4194304] 0 2024-05-17T21:52:52.233 INFO:tasks.workunit.client.0.smithi107.stdout:1/423: dread d4/de/f36 [0,4194304] 0 2024-05-17T21:52:52.235 INFO:tasks.workunit.client.1.smithi138.stdout:4/241: mkdir d0/d3/d1d/d5b 0 2024-05-17T21:52:52.235 INFO:tasks.workunit.client.1.smithi138.stdout:4/242: stat d0/f34 0 2024-05-17T21:52:52.236 INFO:tasks.workunit.client.0.smithi107.stdout:3/392: dwrite d6/d8/d10/d25/f49 [0,4194304] 0 2024-05-17T21:52:52.238 INFO:tasks.workunit.client.0.smithi107.stdout:6/335: symlink d9/l61 0 2024-05-17T21:52:52.243 INFO:tasks.workunit.client.0.smithi107.stdout:9/406: creat d0/dd/d21/d29/d3e/d45/d68/f8c x:0 0 0 2024-05-17T21:52:52.244 INFO:tasks.workunit.client.1.smithi138.stdout:7/179: unlink d0/d18/c1b 0 2024-05-17T21:52:52.245 INFO:tasks.workunit.client.0.smithi107.stdout:8/398: symlink dc/d21/d27/d2f/l95 0 2024-05-17T21:52:52.248 INFO:tasks.workunit.client.0.smithi107.stdout:0/456: link d3/dc/f6d d3/dc/d22/d3d/fa8 0 2024-05-17T21:52:52.249 INFO:tasks.workunit.client.0.smithi107.stdout:7/364: rename d5/d16/l2c to d5/df/d11/d1d/l80 0 2024-05-17T21:52:52.250 INFO:tasks.workunit.client.1.smithi138.stdout:9/239: write d6/d7/f2c [4732000,15248] 0 2024-05-17T21:52:52.253 INFO:tasks.workunit.client.0.smithi107.stdout:3/393: mkdir d6/d8/d10/d25/d80 0 2024-05-17T21:52:52.253 INFO:tasks.workunit.client.0.smithi107.stdout:8/399: dread dc/d21/d27/d34/f6f [0,4194304] 0 2024-05-17T21:52:52.254 INFO:tasks.workunit.client.0.smithi107.stdout:4/406: truncate d3/d4a/f75 1253115 0 2024-05-17T21:52:52.255 INFO:tasks.workunit.client.1.smithi138.stdout:7/180: stat d0/ld 0 2024-05-17T21:52:52.256 INFO:tasks.workunit.client.1.smithi138.stdout:9/240: fdatasync d6/d7/d16/f1a 0 2024-05-17T21:52:52.257 INFO:tasks.workunit.client.0.smithi107.stdout:3/394: dread d6/d2a/f42 [0,4194304] 0 2024-05-17T21:52:52.257 INFO:tasks.workunit.client.1.smithi138.stdout:0/314: dread d3/fc [0,4194304] 0 2024-05-17T21:52:52.258 INFO:tasks.workunit.client.0.smithi107.stdout:9/407: creat d0/dd/d21/d29/f8d x:0 0 0 2024-05-17T21:52:52.260 INFO:tasks.workunit.client.0.smithi107.stdout:2/295: creat d0/d9/d1c/d22/f5b x:0 0 0 2024-05-17T21:52:52.260 INFO:tasks.workunit.client.1.smithi138.stdout:5/221: creat d1/f43 x:0 0 0 2024-05-17T21:52:52.262 INFO:tasks.workunit.client.0.smithi107.stdout:7/365: symlink d5/df/d11/d1c/d31/l81 0 2024-05-17T21:52:52.262 INFO:tasks.workunit.client.0.smithi107.stdout:7/366: chown d5/df/d11 64953580 1 2024-05-17T21:52:52.265 INFO:tasks.workunit.client.1.smithi138.stdout:9/241: rmdir d6/d7/d16/d29/d38 39 2024-05-17T21:52:52.267 INFO:tasks.workunit.client.0.smithi107.stdout:8/400: creat dc/d44/d4b/d52/f96 x:0 0 0 2024-05-17T21:52:52.269 INFO:tasks.workunit.client.0.smithi107.stdout:4/407: symlink d3/d4a/d47/d60/d3a/d41/d80/d43/l8c 0 2024-05-17T21:52:52.269 INFO:tasks.workunit.client.0.smithi107.stdout:4/408: readlink d3/db/d1e/l2a 0 2024-05-17T21:52:52.270 INFO:tasks.workunit.client.0.smithi107.stdout:4/409: chown d3/d6/c24 57545646 1 2024-05-17T21:52:52.270 INFO:tasks.workunit.client.1.smithi138.stdout:0/315: unlink d3/c2a 0 2024-05-17T21:52:52.271 INFO:tasks.workunit.client.0.smithi107.stdout:4/410: read d3/d4a/f73 [233992,124189] 0 2024-05-17T21:52:52.272 INFO:tasks.workunit.client.0.smithi107.stdout:5/395: getdents d5/db/d16/d54/d59 0 2024-05-17T21:52:52.273 INFO:tasks.workunit.client.1.smithi138.stdout:3/290: getdents d1/d2e 0 2024-05-17T21:52:52.275 INFO:tasks.workunit.client.0.smithi107.stdout:9/408: write d0/dd/d1f/f52 [580225,57110] 0 2024-05-17T21:52:52.277 INFO:tasks.workunit.client.1.smithi138.stdout:5/222: rmdir d1/d6/d1b 39 2024-05-17T21:52:52.278 INFO:tasks.workunit.client.0.smithi107.stdout:4/411: dwrite d3/d4a/d47/d60/d3a/d41/d80/d43/f53 [4194304,4194304] 0 2024-05-17T21:52:52.280 INFO:tasks.workunit.client.1.smithi138.stdout:9/242: mkdir d6/d7/d16/d29/d45/d4b 0 2024-05-17T21:52:52.282 INFO:tasks.workunit.client.0.smithi107.stdout:0/457: mkdir d3/dc/d22/d3d/da9 0 2024-05-17T21:52:52.283 INFO:tasks.workunit.client.1.smithi138.stdout:9/243: write d6/d7/d16/d29/d48/f22 [711481,51915] 0 2024-05-17T21:52:52.283 INFO:tasks.workunit.client.1.smithi138.stdout:2/251: dwrite d3/f15 [0,4194304] 0 2024-05-17T21:52:52.283 INFO:tasks.workunit.client.1.smithi138.stdout:4/243: link d0/c13 d0/d1b/d36/c5c 0 2024-05-17T21:52:52.284 INFO:tasks.workunit.client.1.smithi138.stdout:9/244: write f5 [1351133,93835] 0 2024-05-17T21:52:52.285 INFO:tasks.workunit.client.0.smithi107.stdout:9/409: dwrite d0/dd/d21/d29/d3e/d45/d68/f6c [0,4194304] 0 2024-05-17T21:52:52.287 INFO:tasks.workunit.client.0.smithi107.stdout:4/412: dwrite d3/db/f4d [0,4194304] 0 2024-05-17T21:52:52.291 INFO:tasks.workunit.client.0.smithi107.stdout:7/367: read d5/df/f20 [2111835,125573] 0 2024-05-17T21:52:52.292 INFO:tasks.workunit.client.0.smithi107.stdout:7/368: chown d5/f42 970922 1 2024-05-17T21:52:52.292 INFO:tasks.workunit.client.0.smithi107.stdout:7/369: readlink d5/l7b 0 2024-05-17T21:52:52.292 INFO:tasks.workunit.client.0.smithi107.stdout:1/424: rename d4/df/d14/d2e/d43/l53 to d4/df/d14/d2e/d43/l8a 0 2024-05-17T21:52:52.295 INFO:tasks.workunit.client.1.smithi138.stdout:3/291: creat d1/d3/d8/f64 x:0 0 0 2024-05-17T21:52:52.300 INFO:tasks.workunit.client.1.smithi138.stdout:7/181: creat d0/f30 x:0 0 0 2024-05-17T21:52:52.301 INFO:tasks.workunit.client.1.smithi138.stdout:7/182: write d0/f2b [894144,9922] 0 2024-05-17T21:52:52.305 INFO:tasks.workunit.client.1.smithi138.stdout:5/223: symlink d1/d24/l44 0 2024-05-17T21:52:52.307 INFO:tasks.workunit.client.1.smithi138.stdout:4/244: chown d0/d3/d10/f3c 119 1 2024-05-17T21:52:52.310 INFO:tasks.workunit.client.1.smithi138.stdout:9/245: creat d6/d7/d16/d29/d48/f4c x:0 0 0 2024-05-17T21:52:52.314 INFO:tasks.workunit.client.1.smithi138.stdout:7/183: write d0/f1e [2562645,90165] 0 2024-05-17T21:52:52.316 INFO:tasks.workunit.client.0.smithi107.stdout:0/458: dread d3/dc/d22/f68 [0,4194304] 0 2024-05-17T21:52:52.316 INFO:tasks.workunit.client.1.smithi138.stdout:2/252: symlink d3/d21/d2f/d3e/l52 0 2024-05-17T21:52:52.317 INFO:tasks.workunit.client.0.smithi107.stdout:0/459: write d3/d24/d6f/f8b [767542,30312] 0 2024-05-17T21:52:52.317 INFO:tasks.workunit.client.0.smithi107.stdout:5/396: dread d5/db/d16/d2d/f4a [0,4194304] 0 2024-05-17T21:52:52.318 INFO:tasks.workunit.client.0.smithi107.stdout:7/370: mkdir d5/df/d11/d1d/d33/d82 0 2024-05-17T21:52:52.319 INFO:tasks.workunit.client.0.smithi107.stdout:7/371: dread - d5/f70 zero size 2024-05-17T21:52:52.320 INFO:tasks.workunit.client.1.smithi138.stdout:4/245: dread d0/d1b/d42/d45/f48 [0,4194304] 0 2024-05-17T21:52:52.321 INFO:tasks.workunit.client.1.smithi138.stdout:0/316: rename d3/d10/d38/d3e/f65 to d3/d10/f71 0 2024-05-17T21:52:52.334 INFO:tasks.workunit.client.1.smithi138.stdout:9/246: symlink d6/d7/d16/d29/d45/l4d 0 2024-05-17T21:52:52.335 INFO:tasks.workunit.client.1.smithi138.stdout:4/246: mknod d0/d1b/d42/c5d 0 2024-05-17T21:52:52.336 INFO:tasks.workunit.client.1.smithi138.stdout:2/253: read d3/f10 [657209,59185] 0 2024-05-17T21:52:52.337 INFO:tasks.workunit.client.1.smithi138.stdout:3/292: creat d1/f65 x:0 0 0 2024-05-17T21:52:52.337 INFO:tasks.workunit.client.0.smithi107.stdout:2/296: link d0/d9/d1c/f3d d0/d9/d11/f5c 0 2024-05-17T21:52:52.338 INFO:tasks.workunit.client.1.smithi138.stdout:3/293: fdatasync d1/d3/f1b 0 2024-05-17T21:52:52.338 INFO:tasks.workunit.client.0.smithi107.stdout:4/413: chown d3/d4a/f75 6 1 2024-05-17T21:52:52.338 INFO:tasks.workunit.client.1.smithi138.stdout:0/317: rename d3/d10/d39/f5d to d3/d10/d39/f72 0 2024-05-17T21:52:52.339 INFO:tasks.workunit.client.1.smithi138.stdout:0/318: fsync d3/f7 0 2024-05-17T21:52:52.340 INFO:tasks.workunit.client.1.smithi138.stdout:5/224: creat d1/f45 x:0 0 0 2024-05-17T21:52:52.345 INFO:tasks.workunit.client.1.smithi138.stdout:3/294: dread d1/d3/f1b [0,4194304] 0 2024-05-17T21:52:52.346 INFO:tasks.workunit.client.1.smithi138.stdout:0/319: dwrite d3/d10/f25 [0,4194304] 0 2024-05-17T21:52:52.346 INFO:tasks.workunit.client.1.smithi138.stdout:3/295: dread - d1/d4f/f58 zero size 2024-05-17T21:52:52.346 INFO:tasks.workunit.client.0.smithi107.stdout:8/401: rename dc/d10 to dc/d21/d97 0 2024-05-17T21:52:52.347 INFO:tasks.workunit.client.1.smithi138.stdout:3/296: readlink d1/d3/l18 0 2024-05-17T21:52:52.348 INFO:tasks.workunit.client.1.smithi138.stdout:3/297: readlink d1/d4f/l5b 0 2024-05-17T21:52:52.348 INFO:tasks.workunit.client.0.smithi107.stdout:6/336: dwrite d9/f53 [0,4194304] 0 2024-05-17T21:52:52.350 INFO:tasks.workunit.client.1.smithi138.stdout:3/298: dread d1/d3/d8/de/f3b [0,4194304] 0 2024-05-17T21:52:52.351 INFO:tasks.workunit.client.1.smithi138.stdout:3/299: stat d1/c26 0 2024-05-17T21:52:52.352 INFO:tasks.workunit.client.1.smithi138.stdout:3/300: write d1/d3/d8/de/f27 [2072083,123709] 0 2024-05-17T21:52:52.353 INFO:tasks.workunit.client.0.smithi107.stdout:5/397: dread d5/f1b [0,4194304] 0 2024-05-17T21:52:52.355 INFO:tasks.workunit.client.0.smithi107.stdout:4/414: creat d3/d4a/d47/f8d x:0 0 0 2024-05-17T21:52:52.355 INFO:tasks.workunit.client.0.smithi107.stdout:8/402: dwrite dc/d21/d35/f36 [0,4194304] 0 2024-05-17T21:52:52.357 INFO:tasks.workunit.client.0.smithi107.stdout:7/372: fsync d5/df/d11/f5c 0 2024-05-17T21:52:52.359 INFO:tasks.workunit.client.1.smithi138.stdout:7/184: write d0/f1f [865197,16418] 0 2024-05-17T21:52:52.359 INFO:tasks.workunit.client.0.smithi107.stdout:7/373: read - d5/d16/d3b/d45/f6c zero size 2024-05-17T21:52:52.360 INFO:tasks.workunit.client.1.smithi138.stdout:2/254: mknod d3/d21/c53 0 2024-05-17T21:52:52.361 INFO:tasks.workunit.client.1.smithi138.stdout:2/255: stat d3/d21/d2f/f36 0 2024-05-17T21:52:52.362 INFO:tasks.workunit.client.0.smithi107.stdout:7/374: dread - d5/df/d11/d1d/d33/f46 zero size 2024-05-17T21:52:52.363 INFO:tasks.workunit.client.0.smithi107.stdout:7/375: chown d5/d16/d3b 348020770 1 2024-05-17T21:52:52.367 INFO:tasks.workunit.client.0.smithi107.stdout:4/415: dwrite d3/d6/f5c [0,4194304] 0 2024-05-17T21:52:52.369 INFO:tasks.workunit.client.0.smithi107.stdout:8/403: mknod dc/d21/d27/d83/c98 0 2024-05-17T21:52:52.370 INFO:tasks.workunit.client.1.smithi138.stdout:0/320: unlink d3/l66 0 2024-05-17T21:52:52.374 INFO:tasks.workunit.client.0.smithi107.stdout:9/410: rename d0/dd/f56 to d0/dd/d3d/f8e 0 2024-05-17T21:52:52.376 INFO:tasks.workunit.client.1.smithi138.stdout:9/247: creat d6/d7/d16/d29/d38/d3c/f4e x:0 0 0 2024-05-17T21:52:52.377 INFO:tasks.workunit.client.1.smithi138.stdout:9/248: stat d6/d7/d16/f21 0 2024-05-17T21:52:52.379 INFO:tasks.workunit.client.1.smithi138.stdout:4/247: mkdir d0/d3/d17/d5e 0 2024-05-17T21:52:52.380 INFO:tasks.workunit.client.0.smithi107.stdout:4/416: symlink d3/db/d35/l8e 0 2024-05-17T21:52:52.380 INFO:tasks.workunit.client.1.smithi138.stdout:4/248: readlink d0/d3/d10/l28 0 2024-05-17T21:52:52.381 INFO:tasks.workunit.client.1.smithi138.stdout:3/301: read d1/d3/d16/f38 [3282895,128533] 0 2024-05-17T21:52:52.382 INFO:tasks.workunit.client.1.smithi138.stdout:4/249: write d0/d3/d10/d1f/d32/f43 [60724,66012] 0 2024-05-17T21:52:52.383 INFO:tasks.workunit.client.0.smithi107.stdout:8/404: read dc/d21/d27/d2f/f4a [101966,99890] 0 2024-05-17T21:52:52.383 INFO:tasks.workunit.client.0.smithi107.stdout:8/405: read - dc/d21/d97/d63/d71/f94 zero size 2024-05-17T21:52:52.385 INFO:tasks.workunit.client.1.smithi138.stdout:5/225: mkdir d1/d46 0 2024-05-17T21:52:52.385 INFO:tasks.workunit.client.1.smithi138.stdout:5/226: readlink d1/d2c/l30 0 2024-05-17T21:52:52.387 INFO:tasks.workunit.client.0.smithi107.stdout:0/460: rename d3/d75/d9e/f67 to d3/d58/faa 0 2024-05-17T21:52:52.387 INFO:tasks.workunit.client.0.smithi107.stdout:3/395: sync 2024-05-17T21:52:52.388 INFO:tasks.workunit.client.0.smithi107.stdout:9/411: mknod d0/dd/d3d/c8f 0 2024-05-17T21:52:52.391 INFO:tasks.workunit.client.1.smithi138.stdout:3/302: unlink d1/d4f/f58 0 2024-05-17T21:52:52.392 INFO:tasks.workunit.client.1.smithi138.stdout:3/303: chown d1/d4f/c63 85906587 1 2024-05-17T21:52:52.393 INFO:tasks.workunit.client.1.smithi138.stdout:2/256: symlink d3/l54 0 2024-05-17T21:52:52.395 INFO:tasks.workunit.client.0.smithi107.stdout:0/461: dwrite d3/d58/d61/f9b [0,4194304] 0 2024-05-17T21:52:52.397 INFO:tasks.workunit.client.1.smithi138.stdout:4/250: chown d0/d1b/d36/c5c 96251390 1 2024-05-17T21:52:52.399 INFO:tasks.workunit.client.0.smithi107.stdout:8/406: dread dc/d21/d69/f77 [0,4194304] 0 2024-05-17T21:52:52.400 INFO:tasks.workunit.client.0.smithi107.stdout:6/337: dread d9/fa [0,4194304] 0 2024-05-17T21:52:52.402 INFO:tasks.workunit.client.1.smithi138.stdout:2/257: dwrite d3/d21/f2b [0,4194304] 0 2024-05-17T21:52:52.403 INFO:tasks.workunit.client.1.smithi138.stdout:2/258: fsync d3/f35 0 2024-05-17T21:52:52.405 INFO:tasks.workunit.client.1.smithi138.stdout:2/259: write d3/f5 [226494,124224] 0 2024-05-17T21:52:52.405 INFO:tasks.workunit.client.1.smithi138.stdout:7/185: dread d0/f10 [0,4194304] 0 2024-05-17T21:52:52.406 INFO:tasks.workunit.client.0.smithi107.stdout:8/407: dwrite dc/d21/d35/f22 [4194304,4194304] 0 2024-05-17T21:52:52.407 INFO:tasks.workunit.client.1.smithi138.stdout:2/260: read d3/f1a [3986793,51238] 0 2024-05-17T21:52:52.407 INFO:tasks.workunit.client.0.smithi107.stdout:7/376: creat d5/f83 x:0 0 0 2024-05-17T21:52:52.408 INFO:tasks.workunit.client.1.smithi138.stdout:3/304: symlink d1/d3/d16/d3a/l66 0 2024-05-17T21:52:52.409 INFO:tasks.workunit.client.0.smithi107.stdout:8/408: write dc/d21/d35/f22 [4883311,125561] 0 2024-05-17T21:52:52.413 INFO:tasks.workunit.client.1.smithi138.stdout:7/186: dwrite d0/f27 [0,4194304] 0 2024-05-17T21:52:52.414 INFO:tasks.workunit.client.0.smithi107.stdout:9/412: creat d0/dd/d21/d29/f90 x:0 0 0 2024-05-17T21:52:52.414 INFO:tasks.workunit.client.0.smithi107.stdout:5/398: getdents d5/db/d16/d2d/d40/d5a 0 2024-05-17T21:52:52.416 INFO:tasks.workunit.client.1.smithi138.stdout:4/251: dread d0/d3/d1d/f2e [0,4194304] 0 2024-05-17T21:52:52.421 INFO:tasks.workunit.client.0.smithi107.stdout:0/462: unlink d3/d24/c25 0 2024-05-17T21:52:52.424 INFO:tasks.workunit.client.1.smithi138.stdout:9/249: dwrite d6/d7/f9 [0,4194304] 0 2024-05-17T21:52:52.427 INFO:tasks.workunit.client.0.smithi107.stdout:2/297: rename d0/d9/d11/d27/c2f to d0/d9/d11/df/d5a/c5d 0 2024-05-17T21:52:52.428 INFO:tasks.workunit.client.1.smithi138.stdout:4/252: dwrite d0/d3/d10/f51 [0,4194304] 0 2024-05-17T21:52:52.428 INFO:tasks.workunit.client.0.smithi107.stdout:2/298: read - d0/d9/d11/df/f47 zero size 2024-05-17T21:52:52.430 INFO:tasks.workunit.client.1.smithi138.stdout:4/253: truncate d0/f2a 4444407 0 2024-05-17T21:52:52.436 INFO:tasks.workunit.client.1.smithi138.stdout:0/321: truncate d3/f21 199604 0 2024-05-17T21:52:52.438 INFO:tasks.workunit.client.0.smithi107.stdout:1/425: sync 2024-05-17T21:52:52.440 INFO:tasks.workunit.client.0.smithi107.stdout:5/399: symlink d5/db/d16/d5b/l7d 0 2024-05-17T21:52:52.441 INFO:tasks.workunit.client.1.smithi138.stdout:9/250: symlink d6/d7/d16/d29/d38/d40/l4f 0 2024-05-17T21:52:52.442 INFO:tasks.workunit.client.0.smithi107.stdout:5/400: chown d5/db/d16/d54/d59/d65/f47 38061754 1 2024-05-17T21:52:52.443 INFO:tasks.workunit.client.1.smithi138.stdout:3/305: rename d1/l2c to d1/d3/d8/l67 0 2024-05-17T21:52:52.444 INFO:tasks.workunit.client.1.smithi138.stdout:3/306: write d1/d3/d8/f64 [137127,83771] 0 2024-05-17T21:52:52.448 INFO:tasks.workunit.client.0.smithi107.stdout:4/417: rename d3/d4a/d47/d60/d3c/f3e to d3/d4a/d47/d60/d3a/d41/d80/f8f 0 2024-05-17T21:52:52.449 INFO:tasks.workunit.client.0.smithi107.stdout:4/418: stat d3/d6/c24 0 2024-05-17T21:52:52.451 INFO:tasks.workunit.client.0.smithi107.stdout:6/338: creat d9/d58/f62 x:0 0 0 2024-05-17T21:52:52.452 INFO:tasks.workunit.client.0.smithi107.stdout:0/463: dread d3/f4a [0,4194304] 0 2024-05-17T21:52:52.460 INFO:tasks.workunit.client.1.smithi138.stdout:4/254: dread d0/f2a [0,4194304] 0 2024-05-17T21:52:52.461 INFO:tasks.workunit.client.1.smithi138.stdout:7/187: link d0/f24 d0/f31 0 2024-05-17T21:52:52.462 INFO:tasks.workunit.client.1.smithi138.stdout:2/261: dwrite d3/f4 [0,4194304] 0 2024-05-17T21:52:52.463 INFO:tasks.workunit.client.1.smithi138.stdout:2/262: chown d3/d21/d2f/f39 1718126 1 2024-05-17T21:52:52.467 INFO:tasks.workunit.client.1.smithi138.stdout:7/188: dwrite d0/f2b [0,4194304] 0 2024-05-17T21:52:52.470 INFO:tasks.workunit.client.1.smithi138.stdout:3/307: rename d1/d3/c6 to d1/d3/d8/de/c68 0 2024-05-17T21:52:52.471 INFO:tasks.workunit.client.1.smithi138.stdout:3/308: read d1/d3/d16/d3e/f43 [135517,28852] 0 2024-05-17T21:52:52.473 INFO:tasks.workunit.client.0.smithi107.stdout:4/419: dread d3/db/f46 [0,4194304] 0 2024-05-17T21:52:52.476 INFO:tasks.workunit.client.0.smithi107.stdout:9/413: truncate d0/dd/d21/f70 1382488 0 2024-05-17T21:52:52.476 INFO:tasks.workunit.client.1.smithi138.stdout:0/322: link d3/d10/d38/l52 d3/d10/d38/d3e/d59/l73 0 2024-05-17T21:52:52.477 INFO:tasks.workunit.client.1.smithi138.stdout:0/323: stat d3/d10/l4e 0 2024-05-17T21:52:52.477 INFO:tasks.workunit.client.0.smithi107.stdout:9/414: read - d0/dd/d21/d29/d3e/d45/f67 zero size 2024-05-17T21:52:52.478 INFO:tasks.workunit.client.0.smithi107.stdout:3/396: getdents d6/d2a 0 2024-05-17T21:52:52.478 INFO:tasks.workunit.client.1.smithi138.stdout:2/263: truncate d3/f29 688997 0 2024-05-17T21:52:52.479 INFO:tasks.workunit.client.1.smithi138.stdout:2/264: truncate d3/f4a 746934 0 2024-05-17T21:52:52.479 INFO:tasks.workunit.client.0.smithi107.stdout:9/415: readlink d0/dd/d21/d29/d57/d5f/l86 0 2024-05-17T21:52:52.480 INFO:tasks.workunit.client.1.smithi138.stdout:2/265: write d3/f5 [4263564,130790] 0 2024-05-17T21:52:52.482 INFO:tasks.workunit.client.0.smithi107.stdout:7/377: link d5/df/d11/d1c/d31/c52 d5/df/d11/d1c/d2f/d53/c84 0 2024-05-17T21:52:52.485 INFO:tasks.workunit.client.0.smithi107.stdout:8/409: creat dc/d21/d97/d63/f99 x:0 0 0 2024-05-17T21:52:52.485 INFO:tasks.workunit.client.0.smithi107.stdout:8/410: readlink dc/d44/d4b/l58 0 2024-05-17T21:52:52.486 INFO:tasks.workunit.client.1.smithi138.stdout:2/266: dwrite d3/d21/f30 [0,4194304] 0 2024-05-17T21:52:52.487 INFO:tasks.workunit.client.0.smithi107.stdout:1/426: mknod d4/df/d1a/d26/d7a/d81/c8b 0 2024-05-17T21:52:52.487 INFO:tasks.workunit.client.0.smithi107.stdout:1/427: chown d4/df/l84 25582263 1 2024-05-17T21:52:52.488 INFO:tasks.workunit.client.0.smithi107.stdout:5/401: mknod d5/db/dc/d1e/d6e/c7e 0 2024-05-17T21:52:52.489 INFO:tasks.workunit.client.1.smithi138.stdout:3/309: dread d1/d3/d8/de/f3b [0,4194304] 0 2024-05-17T21:52:52.490 INFO:tasks.workunit.client.0.smithi107.stdout:4/420: mknod d3/d4a/d47/d60/d3c/d58/c90 0 2024-05-17T21:52:52.492 INFO:tasks.workunit.client.1.smithi138.stdout:9/251: link d6/d7/d16/l28 d6/d7/d16/d29/l50 0 2024-05-17T21:52:52.493 INFO:tasks.workunit.client.1.smithi138.stdout:9/252: write d6/d7/f26 [3447224,11436] 0 2024-05-17T21:52:52.495 INFO:tasks.workunit.client.1.smithi138.stdout:7/189: rename d0/d18/c2e to d0/d18/c32 0 2024-05-17T21:52:52.496 INFO:tasks.workunit.client.0.smithi107.stdout:9/416: creat d0/dd/d21/d29/d3e/f91 x:0 0 0 2024-05-17T21:52:52.498 INFO:tasks.workunit.client.0.smithi107.stdout:2/299: rename d0/l2a to d0/d9/d11/d18/l5e 0 2024-05-17T21:52:52.499 INFO:tasks.workunit.client.1.smithi138.stdout:0/324: creat d3/d10/d6d/f74 x:0 0 0 2024-05-17T21:52:52.500 INFO:tasks.workunit.client.1.smithi138.stdout:3/310: mknod d1/d3/d8/c69 0 2024-05-17T21:52:52.501 INFO:tasks.workunit.client.0.smithi107.stdout:5/402: creat d5/db/d16/d54/d59/f7f x:0 0 0 2024-05-17T21:52:52.502 INFO:tasks.workunit.client.0.smithi107.stdout:8/411: dread dc/d21/d27/d34/d4c/d7e/f3b [0,4194304] 0 2024-05-17T21:52:52.503 INFO:tasks.workunit.client.0.smithi107.stdout:4/421: mkdir d3/d6/d26/d3b/d91 0 2024-05-17T21:52:52.505 INFO:tasks.workunit.client.1.smithi138.stdout:2/267: dwrite d3/fa [0,4194304] 0 2024-05-17T21:52:52.506 INFO:tasks.workunit.client.0.smithi107.stdout:5/403: dread d5/db/d16/d2d/d32/f38 [0,4194304] 0 2024-05-17T21:52:52.509 INFO:tasks.workunit.client.1.smithi138.stdout:0/325: readlink d3/d10/l4d 0 2024-05-17T21:52:52.513 INFO:tasks.workunit.client.0.smithi107.stdout:6/339: rename d9/fa to d9/d16/d22/d2d/f63 0 2024-05-17T21:52:52.513 INFO:tasks.workunit.client.1.smithi138.stdout:7/190: dwrite d0/f20 [0,4194304] 0 2024-05-17T21:52:52.516 INFO:tasks.workunit.client.0.smithi107.stdout:2/300: fsync d0/d9/d1c/d22/f31 0 2024-05-17T21:52:52.518 INFO:tasks.workunit.client.1.smithi138.stdout:7/191: dread d0/f10 [0,4194304] 0 2024-05-17T21:52:52.519 INFO:tasks.workunit.client.1.smithi138.stdout:9/253: symlink d6/l51 0 2024-05-17T21:52:52.520 INFO:tasks.workunit.client.1.smithi138.stdout:9/254: readlink d6/d7/l32 0 2024-05-17T21:52:52.525 INFO:tasks.workunit.client.0.smithi107.stdout:8/412: creat dc/d21/d27/d34/d4c/f9a x:0 0 0 2024-05-17T21:52:52.526 INFO:tasks.workunit.client.1.smithi138.stdout:3/311: symlink d1/d4d/l6a 0 2024-05-17T21:52:52.528 INFO:tasks.workunit.client.0.smithi107.stdout:0/464: getdents d3/dc/d22/d3d 0 2024-05-17T21:52:52.528 INFO:tasks.workunit.client.1.smithi138.stdout:0/326: truncate d3/fc 4367129 0 2024-05-17T21:52:52.531 INFO:tasks.workunit.client.0.smithi107.stdout:1/428: dread d4/de/f19 [0,4194304] 0 2024-05-17T21:52:52.531 INFO:tasks.workunit.client.0.smithi107.stdout:3/397: rmdir d6/d2a/d2d/d37/d75 0 2024-05-17T21:52:52.532 INFO:tasks.workunit.client.1.smithi138.stdout:3/312: dread - d1/d4f/d50/f54 zero size 2024-05-17T21:52:52.534 INFO:tasks.workunit.client.1.smithi138.stdout:0/327: creat d3/d10/d6d/f75 x:0 0 0 2024-05-17T21:52:52.536 INFO:tasks.workunit.client.0.smithi107.stdout:0/465: dwrite d3/dc/d22/d3d/f4d [0,4194304] 0 2024-05-17T21:52:52.536 INFO:tasks.workunit.client.1.smithi138.stdout:7/192: symlink d0/l33 0 2024-05-17T21:52:52.538 INFO:tasks.workunit.client.1.smithi138.stdout:7/193: write d0/f10 [1803717,15694] 0 2024-05-17T21:52:52.543 INFO:tasks.workunit.client.0.smithi107.stdout:4/422: dread d3/d6/d26/d36/d7e/f87 [0,4194304] 0 2024-05-17T21:52:52.543 INFO:tasks.workunit.client.0.smithi107.stdout:0/466: dwrite d3/dc/d22/d23/d6c/f7c [0,4194304] 0 2024-05-17T21:52:52.544 INFO:tasks.workunit.client.0.smithi107.stdout:4/423: truncate d3/d4a/f78 187018 0 2024-05-17T21:52:52.545 INFO:tasks.workunit.client.0.smithi107.stdout:4/424: truncate d3/d4a/f78 823865 0 2024-05-17T21:52:52.547 INFO:tasks.workunit.client.0.smithi107.stdout:4/425: write d3/d4a/d47/d60/d3a/d41/d80/f44 [2278375,82459] 0 2024-05-17T21:52:52.551 INFO:tasks.workunit.client.0.smithi107.stdout:5/404: symlink d5/db/d16/l80 0 2024-05-17T21:52:52.552 INFO:tasks.workunit.client.1.smithi138.stdout:2/268: dread f2 [4194304,4194304] 0 2024-05-17T21:52:52.554 INFO:tasks.workunit.client.1.smithi138.stdout:7/194: dwrite d0/f29 [0,4194304] 0 2024-05-17T21:52:52.557 INFO:tasks.workunit.client.1.smithi138.stdout:3/313: dread d1/d3/f1c [0,4194304] 0 2024-05-17T21:52:52.558 INFO:tasks.workunit.client.0.smithi107.stdout:1/429: mknod d4/df/d1a/d26/c8c 0 2024-05-17T21:52:52.559 INFO:tasks.workunit.client.0.smithi107.stdout:3/398: mknod d6/d8/d10/d35/d36/d4c/c81 0 2024-05-17T21:52:52.561 INFO:tasks.workunit.client.1.smithi138.stdout:2/269: mkdir d3/d21/d2f/d3e/d55 0 2024-05-17T21:52:52.562 INFO:tasks.workunit.client.1.smithi138.stdout:2/270: write d3/d21/f38 [447086,109316] 0 2024-05-17T21:52:52.566 INFO:tasks.workunit.client.0.smithi107.stdout:7/378: rename d5/df/f26 to d5/df/d11/d1c/d31/d39/f85 0 2024-05-17T21:52:52.567 INFO:tasks.workunit.client.1.smithi138.stdout:0/328: dread d3/d10/f17 [0,4194304] 0 2024-05-17T21:52:52.567 INFO:tasks.workunit.client.1.smithi138.stdout:3/314: creat d1/d4f/d50/f6b x:0 0 0 2024-05-17T21:52:52.568 INFO:tasks.workunit.client.1.smithi138.stdout:0/329: write d3/d3d/f58 [7946,76268] 0 2024-05-17T21:52:52.569 INFO:tasks.workunit.client.1.smithi138.stdout:3/315: chown d1/d3/d8/c35 2087 1 2024-05-17T21:52:52.571 INFO:tasks.workunit.client.0.smithi107.stdout:8/413: dwrite dc/d21/d35/f2e [0,4194304] 0 2024-05-17T21:52:52.574 INFO:tasks.workunit.client.0.smithi107.stdout:0/467: unlink d3/dc/c5f 0 2024-05-17T21:52:52.574 INFO:tasks.workunit.client.0.smithi107.stdout:0/468: write d3/ff [4226805,12443] 0 2024-05-17T21:52:52.576 INFO:tasks.workunit.client.1.smithi138.stdout:0/330: mknod d3/d10/d6d/c76 0 2024-05-17T21:52:52.578 INFO:tasks.workunit.client.0.smithi107.stdout:4/426: write d3/db/f46 [2217570,3452] 0 2024-05-17T21:52:52.580 INFO:tasks.workunit.client.1.smithi138.stdout:2/271: mknod d3/d21/d2f/c56 0 2024-05-17T21:52:52.582 INFO:tasks.workunit.client.0.smithi107.stdout:1/430: creat d4/df/d1a/f8d x:0 0 0 2024-05-17T21:52:52.583 INFO:tasks.workunit.client.0.smithi107.stdout:1/431: read d4/de/f19 [4036623,34110] 0 2024-05-17T21:52:52.584 INFO:tasks.workunit.client.1.smithi138.stdout:7/195: getdents d0 0 2024-05-17T21:52:52.585 INFO:tasks.workunit.client.1.smithi138.stdout:0/331: dread d3/fd [0,4194304] 0 2024-05-17T21:52:52.585 INFO:tasks.workunit.client.0.smithi107.stdout:4/427: dwrite d3/db/d35/f81 [0,4194304] 0 2024-05-17T21:52:52.590 INFO:tasks.workunit.client.0.smithi107.stdout:4/428: dread d3/d6/f5c [0,4194304] 0 2024-05-17T21:52:52.591 INFO:tasks.workunit.client.0.smithi107.stdout:4/429: dread - d3/d4a/d47/d60/d3a/d41/d80/f61 zero size 2024-05-17T21:52:52.591 INFO:tasks.workunit.client.0.smithi107.stdout:4/430: chown d3/db/d35/d4f 231 1 2024-05-17T21:52:52.592 INFO:tasks.workunit.client.1.smithi138.stdout:2/272: stat d3/f20 0 2024-05-17T21:52:52.593 INFO:tasks.workunit.client.0.smithi107.stdout:8/414: truncate dc/f4f 1824410 0 2024-05-17T21:52:52.593 INFO:tasks.workunit.client.1.smithi138.stdout:0/332: dread - d3/d10/d38/f3a zero size 2024-05-17T21:52:52.598 INFO:tasks.workunit.client.1.smithi138.stdout:3/316: truncate d1/d3/f11 742140 0 2024-05-17T21:52:52.599 INFO:tasks.workunit.client.0.smithi107.stdout:6/340: write d9/f1f [372141,103665] 0 2024-05-17T21:52:52.600 INFO:tasks.workunit.client.0.smithi107.stdout:1/432: stat d4/d6a/c80 0 2024-05-17T21:52:52.601 INFO:tasks.workunit.client.1.smithi138.stdout:0/333: fsync d3/d10/f2c 0 2024-05-17T21:52:52.602 INFO:tasks.workunit.client.1.smithi138.stdout:3/317: dread d1/d3/f1c [0,4194304] 0 2024-05-17T21:52:52.603 INFO:tasks.workunit.client.1.smithi138.stdout:3/318: chown d1/c26 13110 1 2024-05-17T21:52:52.605 INFO:tasks.workunit.client.0.smithi107.stdout:7/379: symlink d5/df/d78/l86 0 2024-05-17T21:52:52.606 INFO:tasks.workunit.client.0.smithi107.stdout:7/380: truncate d5/d16/d3b/d45/f6c 68760 0 2024-05-17T21:52:52.609 INFO:tasks.workunit.client.1.smithi138.stdout:3/319: unlink d1/l57 0 2024-05-17T21:52:52.609 INFO:tasks.workunit.client.1.smithi138.stdout:2/273: dwrite d3/f1c [0,4194304] 0 2024-05-17T21:52:52.610 INFO:tasks.workunit.client.0.smithi107.stdout:4/431: truncate d3/d4a/d47/d60/f10 411912 0 2024-05-17T21:52:52.613 INFO:tasks.workunit.client.1.smithi138.stdout:0/334: unlink d3/d10/d38/d3e/d59/c6f 0 2024-05-17T21:52:52.614 INFO:tasks.workunit.client.0.smithi107.stdout:8/415: rmdir dc/d21/d97 39 2024-05-17T21:52:52.615 INFO:tasks.workunit.client.0.smithi107.stdout:2/301: getdents d0/d9/d11/d27 0 2024-05-17T21:52:52.617 INFO:tasks.workunit.client.0.smithi107.stdout:2/302: truncate d0/d9/d11/d15/f4e 394079 0 2024-05-17T21:52:52.617 INFO:tasks.workunit.client.1.smithi138.stdout:3/320: rename d1/d2e/l42 to d1/d4d/l6c 0 2024-05-17T21:52:52.620 INFO:tasks.workunit.client.0.smithi107.stdout:1/433: mknod d4/df/d1a/d26/d72/c8e 0 2024-05-17T21:52:52.621 INFO:tasks.workunit.client.0.smithi107.stdout:9/417: sync 2024-05-17T21:52:52.621 INFO:tasks.workunit.client.1.smithi138.stdout:2/274: creat d3/f57 x:0 0 0 2024-05-17T21:52:52.623 INFO:tasks.workunit.client.0.smithi107.stdout:5/405: rename d5/db/d16/d5b/f62 to d5/db/d16/f81 0 2024-05-17T21:52:52.623 INFO:tasks.workunit.client.0.smithi107.stdout:2/303: dwrite d0/d9/d11/df/f47 [0,4194304] 0 2024-05-17T21:52:52.624 INFO:tasks.workunit.client.0.smithi107.stdout:2/304: chown d0/c34 4 1 2024-05-17T21:52:52.625 INFO:tasks.workunit.client.1.smithi138.stdout:3/321: mknod d1/d3/d8/d5c/c6d 0 2024-05-17T21:52:52.625 INFO:tasks.workunit.client.0.smithi107.stdout:5/406: write d5/f8 [2752156,78824] 0 2024-05-17T21:52:52.625 INFO:tasks.workunit.client.0.smithi107.stdout:7/381: creat d5/df/d11/d1d/d33/f87 x:0 0 0 2024-05-17T21:52:52.626 INFO:tasks.workunit.client.1.smithi138.stdout:2/275: mkdir d3/d21/d2f/d3e/d58 0 2024-05-17T21:52:52.626 INFO:tasks.workunit.client.0.smithi107.stdout:4/432: creat d3/d4a/d47/f92 x:0 0 0 2024-05-17T21:52:52.628 INFO:tasks.workunit.client.1.smithi138.stdout:0/335: rename d3/d10/d39/l3b to d3/d10/d38/l77 0 2024-05-17T21:52:52.630 INFO:tasks.workunit.client.1.smithi138.stdout:3/322: write d1/f4 [3754601,91463] 0 2024-05-17T21:52:52.630 INFO:tasks.workunit.client.1.smithi138.stdout:3/323: chown d1/d2e/l39 49564601 1 2024-05-17T21:52:52.631 INFO:tasks.workunit.client.0.smithi107.stdout:0/469: rmdir d3/dc/d22/d3d/da9 0 2024-05-17T21:52:52.632 INFO:tasks.workunit.client.1.smithi138.stdout:2/276: read - d3/f27 zero size 2024-05-17T21:52:52.632 INFO:tasks.workunit.client.0.smithi107.stdout:0/470: write d3/dc/f26 [5048719,2034] 0 2024-05-17T21:52:52.633 INFO:tasks.workunit.client.1.smithi138.stdout:2/277: fdatasync d3/d21/d2f/f39 0 2024-05-17T21:52:52.634 INFO:tasks.workunit.client.1.smithi138.stdout:2/278: chown d3/d21/d2f/f50 0 1 2024-05-17T21:52:52.634 INFO:tasks.workunit.client.0.smithi107.stdout:0/471: write d3/dc/d22/d23/f2c [958310,126598] 0 2024-05-17T21:52:52.636 INFO:tasks.workunit.client.1.smithi138.stdout:3/324: creat d1/d3/d49/d4b/f6e x:0 0 0 2024-05-17T21:52:52.636 INFO:tasks.workunit.client.0.smithi107.stdout:0/472: chown d3/d24/d6f/f9d 581344 1 2024-05-17T21:52:52.636 INFO:tasks.workunit.client.1.smithi138.stdout:3/325: chown d1/d3/d8/de 29062 1 2024-05-17T21:52:52.639 INFO:tasks.workunit.client.0.smithi107.stdout:3/399: rename d6/d8/d10/d25/c2f to d6/d2a/d2d/d69/c82 0 2024-05-17T21:52:52.639 INFO:tasks.workunit.client.1.smithi138.stdout:2/279: creat d3/d21/f59 x:0 0 0 2024-05-17T21:52:52.641 INFO:tasks.workunit.client.1.smithi138.stdout:3/326: mknod d1/d62/c6f 0 2024-05-17T21:52:52.643 INFO:tasks.workunit.client.0.smithi107.stdout:4/433: mkdir d3/db/d35/d4f/d93 0 2024-05-17T21:52:52.644 INFO:tasks.workunit.client.1.smithi138.stdout:2/280: getdents d3/d21/d2f/d3e/d55 0 2024-05-17T21:52:52.646 INFO:tasks.workunit.client.0.smithi107.stdout:9/418: dwrite d0/dd/d21/f6a [0,4194304] 0 2024-05-17T21:52:52.646 INFO:tasks.workunit.client.0.smithi107.stdout:0/473: dwrite d3/dc/d31/f57 [0,4194304] 0 2024-05-17T21:52:52.647 INFO:tasks.workunit.client.0.smithi107.stdout:7/382: dread d5/df/d11/f21 [0,4194304] 0 2024-05-17T21:52:52.649 INFO:tasks.workunit.client.1.smithi138.stdout:3/327: symlink d1/d3/l70 0 2024-05-17T21:52:52.650 INFO:tasks.workunit.client.1.smithi138.stdout:2/281: rmdir d3/d21/d2f/d3e 39 2024-05-17T21:52:52.656 INFO:tasks.workunit.client.1.smithi138.stdout:3/328: mkdir d1/d3/d16/d3e/d71 0 2024-05-17T21:52:52.656 INFO:tasks.workunit.client.0.smithi107.stdout:7/383: dwrite d5/df/d11/d1c/d31/d39/f5e [0,4194304] 0 2024-05-17T21:52:52.656 INFO:tasks.workunit.client.1.smithi138.stdout:0/336: dread d3/d10/f11 [0,4194304] 0 2024-05-17T21:52:52.661 INFO:tasks.workunit.client.0.smithi107.stdout:1/434: rename d4/df/d1a/d26/d50/l86 to d4/df/d1a/d41/l8f 0 2024-05-17T21:52:52.662 INFO:tasks.workunit.client.0.smithi107.stdout:1/435: chown d4/df/d14/d2e/d43/f51 24132971 1 2024-05-17T21:52:52.664 INFO:tasks.workunit.client.1.smithi138.stdout:3/329: mkdir d1/d3/d8/d72 0 2024-05-17T21:52:52.721 INFO:tasks.workunit.client.1.smithi138.stdout:8/160: sync 2024-05-17T21:52:52.721 INFO:tasks.workunit.client.1.smithi138.stdout:8/161: fsync d0/f17 0 2024-05-17T21:52:52.724 INFO:tasks.workunit.client.1.smithi138.stdout:2/282: creat d3/d21/d2f/d3e/f5a x:0 0 0 2024-05-17T21:52:52.726 INFO:tasks.workunit.client.0.smithi107.stdout:8/416: creat dc/d21/d97/d63/d71/d81/f9b x:0 0 0 2024-05-17T21:52:52.727 INFO:tasks.workunit.client.0.smithi107.stdout:8/417: write dc/d21/d27/d34/d4c/f50 [15916,2127] 0 2024-05-17T21:52:52.734 INFO:tasks.workunit.client.0.smithi107.stdout:9/419: creat d0/dd/d3d/f92 x:0 0 0 2024-05-17T21:52:52.736 INFO:tasks.workunit.client.1.smithi138.stdout:2/283: truncate d3/f10 280872 0 2024-05-17T21:52:52.736 INFO:tasks.workunit.client.0.smithi107.stdout:5/407: write d5/db/d39/f3c [176334,4680] 0 2024-05-17T21:52:52.737 INFO:tasks.workunit.client.1.smithi138.stdout:0/337: link d3/d10/d38/f46 d3/d3d/f78 0 2024-05-17T21:52:52.738 INFO:tasks.workunit.client.0.smithi107.stdout:4/434: rmdir d3 39 2024-05-17T21:52:52.740 INFO:tasks.workunit.client.0.smithi107.stdout:7/384: write d5/df/f10 [3360358,54927] 0 2024-05-17T21:52:52.741 INFO:tasks.workunit.client.0.smithi107.stdout:7/385: readlink d5/df/d11/d1d/d33/l72 0 2024-05-17T21:52:52.741 INFO:tasks.workunit.client.1.smithi138.stdout:8/162: truncate d0/ff 3828587 0 2024-05-17T21:52:52.743 INFO:tasks.workunit.client.1.smithi138.stdout:2/284: unlink d3/f1a 0 2024-05-17T21:52:52.746 INFO:tasks.workunit.client.0.smithi107.stdout:3/400: write d6/d8/d10/f23 [1930194,109324] 0 2024-05-17T21:52:52.747 INFO:tasks.workunit.client.0.smithi107.stdout:3/401: write d6/d2a/f78 [497106,89890] 0 2024-05-17T21:52:52.752 INFO:tasks.workunit.client.1.smithi138.stdout:2/285: creat d3/d21/d2f/f5b x:0 0 0 2024-05-17T21:52:52.760 INFO:tasks.workunit.client.0.smithi107.stdout:0/474: rename d3/dc/d22/d23/d6c to d3/d24/d6f/d9f/dab 0 2024-05-17T21:52:52.761 INFO:tasks.workunit.client.0.smithi107.stdout:1/436: symlink d4/df/d4e/l90 0 2024-05-17T21:52:52.762 INFO:tasks.workunit.client.0.smithi107.stdout:2/305: creat d0/d9/d11/f5f x:0 0 0 2024-05-17T21:52:52.764 INFO:tasks.workunit.client.1.smithi138.stdout:2/286: rename d3/c18 to d3/c5c 0 2024-05-17T21:52:52.764 INFO:tasks.workunit.client.1.smithi138.stdout:2/287: stat d3/l26 0 2024-05-17T21:52:52.767 INFO:tasks.workunit.client.0.smithi107.stdout:6/341: link d9/d16/f4a d9/d16/f64 0 2024-05-17T21:52:52.769 INFO:tasks.workunit.client.1.smithi138.stdout:2/288: rename d3/d21/d2f/d3e/f4c to d3/d21/d2f/d3e/d58/f5d 0 2024-05-17T21:52:52.769 INFO:tasks.workunit.client.0.smithi107.stdout:5/408: truncate d5/db/f1a 1774044 0 2024-05-17T21:52:52.771 INFO:tasks.workunit.client.1.smithi138.stdout:2/289: write d3/d21/d2f/d3e/d58/f5d [678864,126329] 0 2024-05-17T21:52:52.773 INFO:tasks.workunit.client.0.smithi107.stdout:7/386: mknod d5/df/d11/c88 0 2024-05-17T21:52:52.775 INFO:tasks.workunit.client.1.smithi138.stdout:2/290: dread d3/f35 [0,4194304] 0 2024-05-17T21:52:52.775 INFO:tasks.workunit.client.0.smithi107.stdout:3/402: mkdir d6/d8/d10/d35/d36/d4c/d83 0 2024-05-17T21:52:52.775 INFO:tasks.workunit.client.0.smithi107.stdout:7/387: write d5/df/d11/d1c/d31/d39/f5e [824142,21579] 0 2024-05-17T21:52:52.776 INFO:tasks.workunit.client.1.smithi138.stdout:2/291: dread - d3/d21/d2f/f45 zero size 2024-05-17T21:52:52.776 INFO:tasks.workunit.client.0.smithi107.stdout:5/409: dwrite d5/db/d16/d54/d59/d65/f6f [0,4194304] 0 2024-05-17T21:52:52.776 INFO:tasks.workunit.client.0.smithi107.stdout:0/475: mknod d3/d58/cac 0 2024-05-17T21:52:52.778 INFO:tasks.workunit.client.0.smithi107.stdout:3/403: chown d6/d2a/d2d/d69/f6d 32257896 1 2024-05-17T21:52:52.778 INFO:tasks.workunit.client.0.smithi107.stdout:1/437: symlink d4/de/l91 0 2024-05-17T21:52:52.778 INFO:tasks.workunit.client.0.smithi107.stdout:5/410: chown d5/db/d16/d2d/d4d/d6d 75869 1 2024-05-17T21:52:52.779 INFO:tasks.workunit.client.0.smithi107.stdout:1/438: chown d4/f10 22 1 2024-05-17T21:52:52.780 INFO:tasks.workunit.client.0.smithi107.stdout:3/404: write d6/d8/f76 [619576,5821] 0 2024-05-17T21:52:52.780 INFO:tasks.workunit.client.0.smithi107.stdout:8/418: symlink dc/d21/d97/d63/l9c 0 2024-05-17T21:52:52.780 INFO:tasks.workunit.client.0.smithi107.stdout:5/411: dread - d5/db/d16/d2d/d4d/d6d/f77 zero size 2024-05-17T21:52:52.781 INFO:tasks.workunit.client.0.smithi107.stdout:8/419: chown dc/d21/l2a 91962 1 2024-05-17T21:52:52.781 INFO:tasks.workunit.client.0.smithi107.stdout:1/439: chown d4/df/d14/d2e/d43/f5f 605 1 2024-05-17T21:52:52.782 INFO:tasks.workunit.client.0.smithi107.stdout:6/342: fsync d9/d16/d22/d2d/f37 0 2024-05-17T21:52:52.784 INFO:tasks.workunit.client.0.smithi107.stdout:1/440: dread - d4/df/d14/d2e/d43/f5f zero size 2024-05-17T21:52:52.785 INFO:tasks.workunit.client.0.smithi107.stdout:4/435: rmdir d3/d6/d26/d36 39 2024-05-17T21:52:52.788 INFO:tasks.workunit.client.0.smithi107.stdout:5/412: dread d5/db/dc/f20 [0,4194304] 0 2024-05-17T21:52:52.789 INFO:tasks.workunit.client.0.smithi107.stdout:5/413: stat d5/db/d16 0 2024-05-17T21:52:52.789 INFO:tasks.workunit.client.0.smithi107.stdout:9/420: link d0/dd/d2f/d4a/c4d d0/dd/d21/d29/d57/c93 0 2024-05-17T21:52:52.791 INFO:tasks.workunit.client.0.smithi107.stdout:5/414: read d5/db/d39/f70 [533318,1638] 0 2024-05-17T21:52:52.792 INFO:tasks.workunit.client.0.smithi107.stdout:6/343: write d9/f12 [1178723,11958] 0 2024-05-17T21:52:52.792 INFO:tasks.workunit.client.0.smithi107.stdout:9/421: readlink d0/dd/d1f/l2b 0 2024-05-17T21:52:52.795 INFO:tasks.workunit.client.0.smithi107.stdout:4/436: rename d3/f22 to d3/d4a/d47/d60/d3a/d41/d80/f94 0 2024-05-17T21:52:52.796 INFO:tasks.workunit.client.0.smithi107.stdout:2/306: creat d0/d9/f60 x:0 0 0 2024-05-17T21:52:52.797 INFO:tasks.workunit.client.0.smithi107.stdout:0/476: dwrite d3/d58/faa [0,4194304] 0 2024-05-17T21:52:52.798 INFO:tasks.workunit.client.0.smithi107.stdout:4/437: chown d3/d4a/d47/d60/d3a/d41 7 1 2024-05-17T21:52:52.804 INFO:tasks.workunit.client.0.smithi107.stdout:5/415: mknod d5/db/d16/d2d/d32/c82 0 2024-05-17T21:52:52.805 INFO:tasks.workunit.client.0.smithi107.stdout:5/416: chown d5/db/d39/f70 0 1 2024-05-17T21:52:52.805 INFO:tasks.workunit.client.0.smithi107.stdout:7/388: link d5/l7b d5/df/d11/d1c/d31/d39/l89 0 2024-05-17T21:52:52.806 INFO:tasks.workunit.client.0.smithi107.stdout:5/417: dread - d5/db/d16/f81 zero size 2024-05-17T21:52:52.808 INFO:tasks.workunit.client.0.smithi107.stdout:2/307: creat d0/d9/d1c/d3e/d4f/f61 x:0 0 0 2024-05-17T21:52:52.809 INFO:tasks.workunit.client.0.smithi107.stdout:8/420: link dc/d21/d97/l57 dc/d21/d27/l9d 0 2024-05-17T21:52:52.810 INFO:tasks.workunit.client.0.smithi107.stdout:0/477: rename d3/d58/d61/d94/d9a to d3/d24/d6f/d9f/dad 0 2024-05-17T21:52:52.811 INFO:tasks.workunit.client.0.smithi107.stdout:1/441: dwrite d4/de/f19 [0,4194304] 0 2024-05-17T21:52:52.813 INFO:tasks.workunit.client.0.smithi107.stdout:3/405: truncate d6/d8/d10/d25/f49 232242 0 2024-05-17T21:52:52.814 INFO:tasks.workunit.client.0.smithi107.stdout:6/344: symlink d9/d16/d22/d5f/l65 0 2024-05-17T21:52:52.816 INFO:tasks.workunit.client.0.smithi107.stdout:5/418: stat d5/db/d16/d2a/l2c 0 2024-05-17T21:52:52.816 INFO:tasks.workunit.client.0.smithi107.stdout:8/421: rmdir dc/d21/d97/d63/d71 39 2024-05-17T21:52:52.817 INFO:tasks.workunit.client.0.smithi107.stdout:9/422: write d0/dd/f78 [613455,11456] 0 2024-05-17T21:52:52.820 INFO:tasks.workunit.client.0.smithi107.stdout:2/308: rename d0/f29 to d0/d9/d1c/d3e/f62 0 2024-05-17T21:52:52.822 INFO:tasks.workunit.client.0.smithi107.stdout:3/406: mknod d6/d1a/c84 0 2024-05-17T21:52:52.822 INFO:tasks.workunit.client.0.smithi107.stdout:4/438: mkdir d3/d6/d26/d36/d8b/d95 0 2024-05-17T21:52:52.824 INFO:tasks.workunit.client.0.smithi107.stdout:5/419: mknod d5/db/d16/d2d/d4d/d6d/c83 0 2024-05-17T21:52:52.825 INFO:tasks.workunit.client.0.smithi107.stdout:1/442: dwrite d4/df/d1a/d26/f60 [0,4194304] 0 2024-05-17T21:52:52.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:52 smithi138 ceph-mon[35948]: pgmap v34: 65 pgs: 65 active+clean; 1.2 GiB data, 5.1 GiB used, 531 GiB / 536 GiB avail; 26 MiB/s rd, 153 MiB/s wr, 407 op/s 2024-05-17T21:52:52.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:52 smithi138 ceph-mon[35948]: from='mgr.14636 ' entity='mgr.smithi138.napnll' 2024-05-17T21:52:52.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:52 smithi138 ceph-mon[35948]: from='mgr.14636 ' entity='mgr.smithi138.napnll' 2024-05-17T21:52:52.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:52 smithi138 ceph-mon[35948]: from='mgr.14636 172.21.15.138:0/3120106723' entity='mgr.smithi138.napnll' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-05-17T21:52:52.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:52 smithi138 ceph-mon[35948]: from='mgr.14636 172.21.15.138:0/3120106723' entity='mgr.smithi138.napnll' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-05-17T21:52:52.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:52 smithi138 ceph-mon[35948]: from='mgr.14636 172.21.15.138:0/3120106723' entity='mgr.smithi138.napnll' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-17T21:52:52.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:52 smithi138 ceph-mon[35948]: from='mgr.14636 172.21.15.138:0/3120106723' entity='mgr.smithi138.napnll' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:52:52.830 INFO:tasks.workunit.client.0.smithi107.stdout:6/345: mkdir d9/d58/d66 0 2024-05-17T21:52:52.833 INFO:tasks.workunit.client.0.smithi107.stdout:4/439: chown d3/db/f33 9585307 1 2024-05-17T21:52:52.837 INFO:tasks.workunit.client.0.smithi107.stdout:7/389: dread d5/f22 [0,4194304] 0 2024-05-17T21:52:52.842 INFO:tasks.workunit.client.0.smithi107.stdout:0/478: getdents d3/d58/d61/d94 0 2024-05-17T21:52:52.844 INFO:tasks.workunit.client.0.smithi107.stdout:4/440: dread - d3/d6/d26/d36/d7e/f89 zero size 2024-05-17T21:52:52.844 INFO:tasks.workunit.client.0.smithi107.stdout:3/407: creat d6/d8/f85 x:0 0 0 2024-05-17T21:52:52.844 INFO:tasks.workunit.client.0.smithi107.stdout:2/309: dwrite d0/d9/f60 [0,4194304] 0 2024-05-17T21:52:52.846 INFO:tasks.workunit.client.0.smithi107.stdout:7/390: mknod d5/d16/c8a 0 2024-05-17T21:52:52.846 INFO:tasks.workunit.client.0.smithi107.stdout:9/423: dwrite d0/dd/d2f/f5b [0,4194304] 0 2024-05-17T21:52:52.847 INFO:tasks.workunit.client.0.smithi107.stdout:1/443: creat d4/df/d1a/d26/d72/d85/f92 x:0 0 0 2024-05-17T21:52:52.848 INFO:tasks.workunit.client.0.smithi107.stdout:6/346: write d9/d16/f24 [4934835,14645] 0 2024-05-17T21:52:52.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:52 smithi107 ceph-mon[28434]: pgmap v34: 65 pgs: 65 active+clean; 1.2 GiB data, 5.1 GiB used, 531 GiB / 536 GiB avail; 26 MiB/s rd, 153 MiB/s wr, 407 op/s 2024-05-17T21:52:52.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:52 smithi107 ceph-mon[28434]: from='mgr.14636 ' entity='mgr.smithi138.napnll' 2024-05-17T21:52:52.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:52 smithi107 ceph-mon[28434]: from='mgr.14636 ' entity='mgr.smithi138.napnll' 2024-05-17T21:52:52.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:52 smithi107 ceph-mon[28434]: from='mgr.14636 172.21.15.138:0/3120106723' entity='mgr.smithi138.napnll' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-05-17T21:52:52.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:52 smithi107 ceph-mon[28434]: from='mgr.14636 172.21.15.138:0/3120106723' entity='mgr.smithi138.napnll' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-05-17T21:52:52.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:52 smithi107 ceph-mon[28434]: from='mgr.14636 172.21.15.138:0/3120106723' entity='mgr.smithi138.napnll' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-17T21:52:52.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:52 smithi107 ceph-mon[28434]: from='mgr.14636 172.21.15.138:0/3120106723' entity='mgr.smithi138.napnll' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:52:52.853 INFO:tasks.workunit.client.0.smithi107.stdout:7/391: mknod d5/d16/c8b 0 2024-05-17T21:52:52.853 INFO:tasks.workunit.client.0.smithi107.stdout:9/424: dread - d0/dd/d21/d29/f90 zero size 2024-05-17T21:52:52.853 INFO:tasks.workunit.client.0.smithi107.stdout:3/408: chown d6/d8/d10/d25/c51 72 1 2024-05-17T21:52:52.853 INFO:tasks.workunit.client.0.smithi107.stdout:4/441: fsync d3/db/d35/f6b 0 2024-05-17T21:52:52.853 INFO:tasks.workunit.client.0.smithi107.stdout:2/310: dread d0/d9/d11/d15/f4e [0,4194304] 0 2024-05-17T21:52:52.854 INFO:tasks.workunit.client.0.smithi107.stdout:6/347: mkdir d9/d16/d4d/d67 0 2024-05-17T21:52:52.854 INFO:tasks.workunit.client.0.smithi107.stdout:3/409: stat d6/d2a/d2d/d37/l45 0 2024-05-17T21:52:52.857 INFO:tasks.workunit.client.0.smithi107.stdout:3/410: chown d6/c18 35 1 2024-05-17T21:52:52.858 INFO:tasks.workunit.client.0.smithi107.stdout:6/348: chown d9/d16/d4d/f59 997 1 2024-05-17T21:52:52.860 INFO:tasks.workunit.client.0.smithi107.stdout:8/422: dread dc/d21/d27/d2f/f59 [0,4194304] 0 2024-05-17T21:52:52.861 INFO:tasks.workunit.client.0.smithi107.stdout:4/442: rename d3/f68 to d3/db/d35/d5e/f96 0 2024-05-17T21:52:52.862 INFO:tasks.workunit.client.0.smithi107.stdout:1/444: truncate d4/df/d14/f4c 3605203 0 2024-05-17T21:52:52.863 INFO:tasks.workunit.client.0.smithi107.stdout:9/425: rmdir d0/dd/d1b 39 2024-05-17T21:52:52.863 INFO:tasks.workunit.client.0.smithi107.stdout:4/443: fsync d3/db/f4d 0 2024-05-17T21:52:52.864 INFO:tasks.workunit.client.0.smithi107.stdout:3/411: rmdir d6/d2a/d2d/d48 39 2024-05-17T21:52:52.865 INFO:tasks.workunit.client.0.smithi107.stdout:0/479: dwrite d3/dc/f6d [0,4194304] 0 2024-05-17T21:52:52.866 INFO:tasks.workunit.client.0.smithi107.stdout:8/423: rmdir dc/d21/d69 39 2024-05-17T21:52:52.866 INFO:tasks.workunit.client.0.smithi107.stdout:6/349: chown d9/d16/l17 133523232 1 2024-05-17T21:52:52.866 INFO:tasks.workunit.client.0.smithi107.stdout:7/392: creat d5/df/d11/d1d/f8c x:0 0 0 2024-05-17T21:52:52.867 INFO:tasks.workunit.client.0.smithi107.stdout:3/412: chown d6/d8/d10/d25/f55 7 1 2024-05-17T21:52:52.867 INFO:tasks.workunit.client.0.smithi107.stdout:4/444: mknod d3/db/c97 0 2024-05-17T21:52:52.868 INFO:tasks.workunit.client.0.smithi107.stdout:7/393: stat d5/df/d11/d2a 0 2024-05-17T21:52:52.869 INFO:tasks.workunit.client.0.smithi107.stdout:4/445: readlink d3/d6/d26/l32 0 2024-05-17T21:52:52.873 INFO:tasks.workunit.client.0.smithi107.stdout:2/311: getdents d0/d9/d1c/d22 0 2024-05-17T21:52:52.873 INFO:tasks.workunit.client.0.smithi107.stdout:0/480: mkdir d3/dc/dae 0 2024-05-17T21:52:52.875 INFO:tasks.workunit.client.0.smithi107.stdout:3/413: mknod d6/d8/d10/d35/d36/d4c/c86 0 2024-05-17T21:52:52.876 INFO:tasks.workunit.client.0.smithi107.stdout:6/350: symlink d9/d16/d22/l68 0 2024-05-17T21:52:52.876 INFO:tasks.workunit.client.0.smithi107.stdout:8/424: mknod dc/d21/d27/d2f/c9e 0 2024-05-17T21:52:52.878 INFO:tasks.workunit.client.0.smithi107.stdout:7/394: rmdir d5/df/d11/d1c 39 2024-05-17T21:52:52.880 INFO:tasks.workunit.client.0.smithi107.stdout:2/312: unlink d0/c17 0 2024-05-17T21:52:52.880 INFO:tasks.workunit.client.0.smithi107.stdout:6/351: read - d9/d16/d22/d2d/f49 zero size 2024-05-17T21:52:52.880 INFO:tasks.workunit.client.0.smithi107.stdout:0/481: mknod d3/d24/d6f/d9f/dad/caf 0 2024-05-17T21:52:52.882 INFO:tasks.workunit.client.0.smithi107.stdout:5/420: dread d5/f8 [0,4194304] 0 2024-05-17T21:52:52.882 INFO:tasks.workunit.client.0.smithi107.stdout:4/446: dwrite d3/fa [0,4194304] 0 2024-05-17T21:52:52.883 INFO:tasks.workunit.client.0.smithi107.stdout:1/445: getdents d4/df/d14/d2e 0 2024-05-17T21:52:52.883 INFO:tasks.workunit.client.0.smithi107.stdout:0/482: dread - d3/dc/f37 zero size 2024-05-17T21:52:52.886 INFO:tasks.workunit.client.0.smithi107.stdout:6/352: symlink d9/d58/l69 0 2024-05-17T21:52:52.887 INFO:tasks.workunit.client.0.smithi107.stdout:3/414: rmdir d6/d2a/d2d/d48/d4f 39 2024-05-17T21:52:52.889 INFO:tasks.workunit.client.0.smithi107.stdout:1/446: creat d4/df/d1a/d26/d7a/f93 x:0 0 0 2024-05-17T21:52:52.890 INFO:tasks.workunit.client.0.smithi107.stdout:4/447: mkdir d3/d6/d26/d36/d7e/d98 0 2024-05-17T21:52:52.894 INFO:tasks.workunit.client.0.smithi107.stdout:6/353: rmdir d9/d16/d4d 39 2024-05-17T21:52:52.894 INFO:tasks.workunit.client.0.smithi107.stdout:5/421: chown d5/l49 1000 0 2024-05-17T21:52:52.897 INFO:tasks.workunit.client.0.smithi107.stdout:4/448: write d3/d4a/d47/f4b [1118342,125968] 0 2024-05-17T21:52:52.903 INFO:tasks.workunit.client.0.smithi107.stdout:3/415: link d6/d2a/d2d/f43 d6/d8/d3e/d3f/f87 0 2024-05-17T21:52:52.903 INFO:tasks.workunit.client.0.smithi107.stdout:8/425: dwrite dc/d21/d97/d63/d71/d81/f9b [0,4194304] 0 2024-05-17T21:52:52.903 INFO:tasks.workunit.client.0.smithi107.stdout:4/449: readlink d3/d6/d26/d3b/l76 0 2024-05-17T21:52:52.905 INFO:tasks.workunit.client.0.smithi107.stdout:7/395: dwrite d5/df/d11/d1c/f4e [0,4194304] 0 2024-05-17T21:52:52.909 INFO:tasks.workunit.client.0.smithi107.stdout:1/447: dread d4/de/f1e [0,4194304] 0 2024-05-17T21:52:52.910 INFO:tasks.workunit.client.0.smithi107.stdout:8/426: creat dc/d44/d4b/d52/f9f x:0 0 0 2024-05-17T21:52:52.913 INFO:tasks.workunit.client.0.smithi107.stdout:3/416: chown d6/d8/d10/d35/l79 77 1 2024-05-17T21:52:52.916 INFO:tasks.workunit.client.0.smithi107.stdout:6/354: dwrite d9/f10 [0,4194304] 0 2024-05-17T21:52:52.924 INFO:tasks.workunit.client.0.smithi107.stdout:7/396: write d5/df/d11/d1d/d33/f6a [922551,69722] 0 2024-05-17T21:52:52.924 INFO:tasks.workunit.client.0.smithi107.stdout:5/422: dwrite d5/db/d16/d5b/f3f [0,4194304] 0 2024-05-17T21:52:52.927 INFO:tasks.workunit.client.0.smithi107.stdout:3/417: mkdir d6/d8/d10/d35/d36/d66/d88 0 2024-05-17T21:52:52.927 INFO:tasks.workunit.client.0.smithi107.stdout:0/483: dread d3/d58/f71 [0,4194304] 0 2024-05-17T21:52:52.928 INFO:tasks.workunit.client.0.smithi107.stdout:6/355: symlink d9/d16/l6a 0 2024-05-17T21:52:52.929 INFO:tasks.workunit.client.0.smithi107.stdout:7/397: write d5/f22 [3629126,95033] 0 2024-05-17T21:52:52.930 INFO:tasks.workunit.client.0.smithi107.stdout:3/418: fdatasync d6/d2a/f52 0 2024-05-17T21:52:52.931 INFO:tasks.workunit.client.1.smithi138.stdout:5/227: sync 2024-05-17T21:52:52.932 INFO:tasks.workunit.client.1.smithi138.stdout:6/301: sync 2024-05-17T21:52:52.932 INFO:tasks.workunit.client.1.smithi138.stdout:1/208: sync 2024-05-17T21:52:52.932 INFO:tasks.workunit.client.1.smithi138.stdout:5/228: stat d1/d1a/l3c 0 2024-05-17T21:52:52.934 INFO:tasks.workunit.client.0.smithi107.stdout:6/356: mknod d9/d16/c6b 0 2024-05-17T21:52:52.936 INFO:tasks.workunit.client.0.smithi107.stdout:5/423: creat d5/f84 x:0 0 0 2024-05-17T21:52:52.937 INFO:tasks.workunit.client.0.smithi107.stdout:1/448: dwrite d4/f1b [0,4194304] 0 2024-05-17T21:52:52.940 INFO:tasks.workunit.client.1.smithi138.stdout:6/302: dwrite d7/d8/f42 [0,4194304] 0 2024-05-17T21:52:52.941 INFO:tasks.workunit.client.0.smithi107.stdout:6/357: chown d9/d16/d22/l57 7588896 1 2024-05-17T21:52:52.942 INFO:tasks.workunit.client.1.smithi138.stdout:1/209: mknod d4/dd/c48 0 2024-05-17T21:52:52.943 INFO:tasks.workunit.client.0.smithi107.stdout:8/427: dwrite dc/d21/d35/f2e [0,4194304] 0 2024-05-17T21:52:52.943 INFO:tasks.workunit.client.0.smithi107.stdout:6/358: readlink d9/d16/l6a 0 2024-05-17T21:52:52.944 INFO:tasks.workunit.client.0.smithi107.stdout:8/428: stat dc/d21/d35/c87 0 2024-05-17T21:52:52.946 INFO:tasks.workunit.client.0.smithi107.stdout:4/450: dread d3/db/f7b [0,4194304] 0 2024-05-17T21:52:52.947 INFO:tasks.workunit.client.1.smithi138.stdout:5/229: creat d1/d6/d1b/f47 x:0 0 0 2024-05-17T21:52:52.949 INFO:tasks.workunit.client.1.smithi138.stdout:5/230: write d1/d6/d19/f3a [1024006,109428] 0 2024-05-17T21:52:52.950 INFO:tasks.workunit.client.1.smithi138.stdout:5/231: stat d1/d46 0 2024-05-17T21:52:52.951 INFO:tasks.workunit.client.0.smithi107.stdout:7/398: creat d5/df/d11/d1c/d31/d39/f8d x:0 0 0 2024-05-17T21:52:52.953 INFO:tasks.workunit.client.0.smithi107.stdout:7/399: write d5/df/d11/f21 [4702659,20485] 0 2024-05-17T21:52:52.953 INFO:tasks.workunit.client.1.smithi138.stdout:1/210: creat d4/dd/d3b/f49 x:0 0 0 2024-05-17T21:52:52.954 INFO:tasks.workunit.client.0.smithi107.stdout:7/400: dread - d5/df/d11/d1d/f8c zero size 2024-05-17T21:52:52.956 INFO:tasks.workunit.client.1.smithi138.stdout:6/303: rename d7/db/d43/d3c/l61 to d7/db/d60/l65 0 2024-05-17T21:52:52.957 INFO:tasks.workunit.client.0.smithi107.stdout:5/424: creat d5/db/d16/d5b/f85 x:0 0 0 2024-05-17T21:52:52.959 INFO:tasks.workunit.client.1.smithi138.stdout:5/232: unlink d1/f7 0 2024-05-17T21:52:52.960 INFO:tasks.workunit.client.1.smithi138.stdout:5/233: chown d1/d6/f31 9950378 1 2024-05-17T21:52:52.961 INFO:tasks.workunit.client.0.smithi107.stdout:0/484: rename d3/dc/d22/f2d to d3/d39/fb0 0 2024-05-17T21:52:52.962 INFO:tasks.workunit.client.1.smithi138.stdout:1/211: mknod d4/df/d35/c4a 0 2024-05-17T21:52:52.962 INFO:tasks.workunit.client.0.smithi107.stdout:0/485: write d3/f9c [670703,47643] 0 2024-05-17T21:52:52.963 INFO:tasks.workunit.client.1.smithi138.stdout:1/212: dread - d4/dd/f20 zero size 2024-05-17T21:52:52.963 INFO:tasks.workunit.client.0.smithi107.stdout:0/486: fsync d3/f11 0 2024-05-17T21:52:52.963 INFO:tasks.workunit.client.1.smithi138.stdout:1/213: chown d4/dd/d3b/d40 14383217 1 2024-05-17T21:52:52.964 INFO:tasks.workunit.client.0.smithi107.stdout:8/429: mkdir dc/d21/d97/da0 0 2024-05-17T21:52:52.965 INFO:tasks.workunit.client.0.smithi107.stdout:8/430: truncate dc/d21/d27/d34/f6e 189860 0 2024-05-17T21:52:52.965 INFO:tasks.workunit.client.0.smithi107.stdout:3/419: getdents d6/d2a 0 2024-05-17T21:52:52.966 INFO:tasks.workunit.client.0.smithi107.stdout:8/431: fsync dc/d21/d27/d34/d4c/f50 0 2024-05-17T21:52:52.967 INFO:tasks.workunit.client.1.smithi138.stdout:5/234: symlink d1/d6/l48 0 2024-05-17T21:52:52.968 INFO:tasks.workunit.client.0.smithi107.stdout:6/359: write d9/f1d [8182371,33555] 0 2024-05-17T21:52:52.972 INFO:tasks.workunit.client.1.smithi138.stdout:1/214: creat d4/db/f4b x:0 0 0 2024-05-17T21:52:52.972 INFO:tasks.workunit.client.1.smithi138.stdout:1/215: fdatasync d4/dd/d3b/f49 0 2024-05-17T21:52:52.973 INFO:tasks.workunit.client.0.smithi107.stdout:4/451: mkdir d3/db/d1e/d99 0 2024-05-17T21:52:52.973 INFO:tasks.workunit.client.1.smithi138.stdout:1/216: write d4/db/f4b [167862,6045] 0 2024-05-17T21:52:52.975 INFO:tasks.workunit.client.0.smithi107.stdout:5/425: dwrite d5/db/dc/f56 [0,4194304] 0 2024-05-17T21:52:52.975 INFO:tasks.workunit.client.1.smithi138.stdout:5/235: mknod d1/d6/d1b/d29/c49 0 2024-05-17T21:52:52.977 INFO:tasks.workunit.client.0.smithi107.stdout:3/420: unlink d6/d8/c5d 0 2024-05-17T21:52:52.979 INFO:tasks.workunit.client.1.smithi138.stdout:1/217: write d4/fc [7919912,69634] 0 2024-05-17T21:52:52.981 INFO:tasks.workunit.client.0.smithi107.stdout:7/401: symlink d5/df/d11/l8e 0 2024-05-17T21:52:52.981 INFO:tasks.workunit.client.0.smithi107.stdout:0/487: dread d3/dc/d22/f32 [0,4194304] 0 2024-05-17T21:52:52.985 INFO:tasks.workunit.client.1.smithi138.stdout:6/304: dread d7/fc [0,4194304] 0 2024-05-17T21:52:52.985 INFO:tasks.workunit.client.0.smithi107.stdout:1/449: write d4/df/f13 [928517,71855] 0 2024-05-17T21:52:52.986 INFO:tasks.workunit.client.1.smithi138.stdout:5/236: dwrite d1/d6/d1b/f47 [0,4194304] 0 2024-05-17T21:52:52.992 INFO:tasks.workunit.client.1.smithi138.stdout:1/218: dwrite d4/d2e/f3e [0,4194304] 0 2024-05-17T21:52:52.995 INFO:tasks.workunit.client.0.smithi107.stdout:3/421: write d6/d8/d10/d25/f47 [2339832,49999] 0 2024-05-17T21:52:52.996 INFO:tasks.workunit.client.1.smithi138.stdout:6/305: dwrite d7/db/d43/d3c/f55 [0,4194304] 0 2024-05-17T21:52:52.999 INFO:tasks.workunit.client.1.smithi138.stdout:5/237: rmdir d1/d6/d1b/d29/d36 39 2024-05-17T21:52:53.007 INFO:tasks.workunit.client.1.smithi138.stdout:1/219: rename d4/d2e/f3e to d4/df/d3a/f4c 0 2024-05-17T21:52:53.008 INFO:tasks.workunit.client.0.smithi107.stdout:1/450: rmdir d4/df/d1a/d41 39 2024-05-17T21:52:53.008 INFO:tasks.workunit.client.0.smithi107.stdout:1/451: fdatasync d4/de/f36 0 2024-05-17T21:52:53.009 INFO:tasks.workunit.client.1.smithi138.stdout:6/306: write d7/db/d43/f35 [3810655,38505] 0 2024-05-17T21:52:53.011 INFO:tasks.workunit.client.0.smithi107.stdout:6/360: rename d9/f2a to d9/d58/d66/f6c 0 2024-05-17T21:52:53.012 INFO:tasks.workunit.client.1.smithi138.stdout:6/307: dread d7/fc [4194304,4194304] 0 2024-05-17T21:52:53.014 INFO:tasks.workunit.client.0.smithi107.stdout:5/426: symlink d5/db/l86 0 2024-05-17T21:52:53.016 INFO:tasks.workunit.client.0.smithi107.stdout:3/422: creat d6/d8/d10/d25/f89 x:0 0 0 2024-05-17T21:52:53.016 INFO:tasks.workunit.client.1.smithi138.stdout:6/308: readlink d7/l16 0 2024-05-17T21:52:53.017 INFO:tasks.workunit.client.0.smithi107.stdout:3/423: dread - d6/d8/d10/d25/f89 zero size 2024-05-17T21:52:53.017 INFO:tasks.workunit.client.0.smithi107.stdout:0/488: symlink d3/dc/d22/d23/d97/d85/lb1 0 2024-05-17T21:52:53.018 INFO:tasks.workunit.client.1.smithi138.stdout:5/238: mknod d1/c4a 0 2024-05-17T21:52:53.019 INFO:tasks.workunit.client.1.smithi138.stdout:5/239: write d1/f45 [738538,125957] 0 2024-05-17T21:52:53.020 INFO:tasks.workunit.client.0.smithi107.stdout:8/432: creat dc/d21/d27/d2f/fa1 x:0 0 0 2024-05-17T21:52:53.021 INFO:tasks.workunit.client.0.smithi107.stdout:1/452: creat d4/df/d14/d2e/d43/d47/f94 x:0 0 0 2024-05-17T21:52:53.022 INFO:tasks.workunit.client.1.smithi138.stdout:6/309: mknod d7/db/d43/d5f/c66 0 2024-05-17T21:52:53.024 INFO:tasks.workunit.client.0.smithi107.stdout:4/452: creat d3/db/d1e/f9a x:0 0 0 2024-05-17T21:52:53.025 INFO:tasks.workunit.client.1.smithi138.stdout:5/240: dread d1/d6/d19/f20 [0,4194304] 0 2024-05-17T21:52:53.028 INFO:tasks.workunit.client.1.smithi138.stdout:6/310: rmdir d7/d8 39 2024-05-17T21:52:53.028 INFO:tasks.workunit.client.0.smithi107.stdout:9/426: sync 2024-05-17T21:52:53.032 INFO:tasks.workunit.client.1.smithi138.stdout:1/220: dread d4/dd/f13 [0,4194304] 0 2024-05-17T21:52:53.033 INFO:tasks.workunit.client.0.smithi107.stdout:7/402: link d5/l9 d5/df/d11/d1c/d31/d39/l8f 0 2024-05-17T21:52:53.035 INFO:tasks.workunit.client.0.smithi107.stdout:8/433: symlink dc/d21/d27/la2 0 2024-05-17T21:52:53.036 INFO:tasks.workunit.client.0.smithi107.stdout:5/427: dread d5/db/dc/f1f [0,4194304] 0 2024-05-17T21:52:53.036 INFO:tasks.workunit.client.0.smithi107.stdout:5/428: fsync d5/f1b 0 2024-05-17T21:52:53.038 INFO:tasks.workunit.client.0.smithi107.stdout:5/429: chown d5/db/dc/d1e/d6e/c7e 30173329 1 2024-05-17T21:52:53.038 INFO:tasks.workunit.client.0.smithi107.stdout:5/430: chown d5/d75 2595 1 2024-05-17T21:52:53.039 INFO:tasks.workunit.client.0.smithi107.stdout:4/453: mknod d3/db/d35/c9b 0 2024-05-17T21:52:53.040 INFO:tasks.workunit.client.0.smithi107.stdout:1/453: dread d4/df/d14/d1d/f2a [0,4194304] 0 2024-05-17T21:52:53.042 INFO:tasks.workunit.client.0.smithi107.stdout:9/427: chown d0/dd/d21/d29/d57/c93 9225 1 2024-05-17T21:52:53.043 INFO:tasks.workunit.client.1.smithi138.stdout:5/241: truncate d1/d6/fa 1023996 0 2024-05-17T21:52:53.044 INFO:tasks.workunit.client.1.smithi138.stdout:1/221: creat d4/db/f4d x:0 0 0 2024-05-17T21:52:53.045 INFO:tasks.workunit.client.0.smithi107.stdout:3/424: symlink d6/l8a 0 2024-05-17T21:52:53.046 INFO:tasks.workunit.client.1.smithi138.stdout:6/311: truncate d7/fa 1893025 0 2024-05-17T21:52:53.049 INFO:tasks.workunit.client.1.smithi138.stdout:1/222: mkdir d4/df/d4e 0 2024-05-17T21:52:53.051 INFO:tasks.workunit.client.0.smithi107.stdout:8/434: dread dc/d21/d97/d63/f79 [4194304,4194304] 0 2024-05-17T21:52:53.051 INFO:tasks.workunit.client.1.smithi138.stdout:6/312: fdatasync d7/db/d43/d3f/f4c 0 2024-05-17T21:52:53.053 INFO:tasks.workunit.client.1.smithi138.stdout:5/242: dread d1/d6/d1b/f47 [0,4194304] 0 2024-05-17T21:52:53.053 INFO:tasks.workunit.client.0.smithi107.stdout:5/431: rmdir d5/db/d16/d2d 39 2024-05-17T21:52:53.054 INFO:tasks.workunit.client.1.smithi138.stdout:5/243: read - d1/d6/f41 zero size 2024-05-17T21:52:53.060 INFO:tasks.workunit.client.0.smithi107.stdout:1/454: rename d4/df/d14/f23 to d4/df/d1a/d26/d50/f95 0 2024-05-17T21:52:53.062 INFO:tasks.workunit.client.0.smithi107.stdout:6/361: write d9/d16/f3b [1077614,129212] 0 2024-05-17T21:52:53.063 INFO:tasks.workunit.client.0.smithi107.stdout:9/428: mkdir d0/dd/d21/d29/d3e/d94 0 2024-05-17T21:52:53.064 INFO:tasks.workunit.client.0.smithi107.stdout:4/454: dwrite d3/d4a/d47/d60/d3a/d41/d80/f61 [0,4194304] 0 2024-05-17T21:52:53.065 INFO:tasks.workunit.client.0.smithi107.stdout:4/455: fsync d3/d4a/f3f 0 2024-05-17T21:52:53.065 INFO:tasks.workunit.client.1.smithi138.stdout:1/223: creat d4/d2e/d36/f4f x:0 0 0 2024-05-17T21:52:53.065 INFO:tasks.workunit.client.0.smithi107.stdout:9/429: read - d0/dd/d21/d29/d3e/d45/d68/f7a zero size 2024-05-17T21:52:53.066 INFO:tasks.workunit.client.1.smithi138.stdout:1/224: truncate d4/f9 4327009 0 2024-05-17T21:52:53.066 INFO:tasks.workunit.client.1.smithi138.stdout:5/244: dwrite d1/d6/d1b/f21 [0,4194304] 0 2024-05-17T21:52:53.067 INFO:tasks.workunit.client.0.smithi107.stdout:3/425: creat d6/d8/d10/d35/d36/f8b x:0 0 0 2024-05-17T21:52:53.067 INFO:tasks.workunit.client.0.smithi107.stdout:9/430: chown d0/dd/d21/d29 1439096 1 2024-05-17T21:52:53.069 INFO:tasks.workunit.client.1.smithi138.stdout:6/313: symlink d7/db/d43/d63/l67 0 2024-05-17T21:52:53.072 INFO:tasks.workunit.client.0.smithi107.stdout:8/435: mknod dc/d44/d4b/d52/d70/ca3 0 2024-05-17T21:52:53.073 INFO:tasks.workunit.client.0.smithi107.stdout:5/432: creat d5/db/d16/d54/d59/f87 x:0 0 0 2024-05-17T21:52:53.074 INFO:tasks.workunit.client.1.smithi138.stdout:1/225: rename d4/dd/c48 to d4/db/d29/c50 0 2024-05-17T21:52:53.074 INFO:tasks.workunit.client.0.smithi107.stdout:5/433: stat d5/db/d16/d54/d59/d65/f27 0 2024-05-17T21:52:53.075 INFO:tasks.workunit.client.0.smithi107.stdout:5/434: dread - d5/db/d16/d5b/f85 zero size 2024-05-17T21:52:53.076 INFO:tasks.workunit.client.0.smithi107.stdout:1/455: symlink d4/df/d4e/l96 0 2024-05-17T21:52:53.076 INFO:tasks.workunit.client.0.smithi107.stdout:1/456: readlink d4/df/d14/l34 0 2024-05-17T21:52:53.089 INFO:tasks.workunit.client.0.smithi107.stdout:3/426: dread d6/d2a/d2d/d69/f6d [0,4194304] 0 2024-05-17T21:52:53.090 INFO:tasks.workunit.client.0.smithi107.stdout:3/427: readlink d6/d2a/d2d/d48/l54 0 2024-05-17T21:52:53.090 INFO:tasks.workunit.client.0.smithi107.stdout:3/428: chown d6/d8 119894167 1 2024-05-17T21:52:53.092 INFO:tasks.workunit.client.1.smithi138.stdout:4/255: sync 2024-05-17T21:52:53.092 INFO:tasks.workunit.client.1.smithi138.stdout:9/255: sync 2024-05-17T21:52:53.130 INFO:tasks.workunit.client.1.smithi138.stdout:1/226: readlink d4/db/d29/l2b 0 2024-05-17T21:52:53.131 INFO:tasks.workunit.client.1.smithi138.stdout:1/227: chown d4/df/f44 0 1 2024-05-17T21:52:53.131 INFO:tasks.workunit.client.0.smithi107.stdout:4/456: rename d3/d4a/d47/f92 to d3/d4a/d47/d60/d3c/f9c 0 2024-05-17T21:52:53.134 INFO:tasks.workunit.client.1.smithi138.stdout:5/245: creat d1/d6/d1b/d29/d36/f4b x:0 0 0 2024-05-17T21:52:53.138 INFO:tasks.workunit.client.0.smithi107.stdout:2/313: sync 2024-05-17T21:52:53.138 INFO:tasks.workunit.client.1.smithi138.stdout:1/228: dwrite d4/dd/f20 [0,4194304] 0 2024-05-17T21:52:53.139 INFO:tasks.workunit.client.0.smithi107.stdout:2/314: chown d0/d9/d11/d15/c16 33208 1 2024-05-17T21:52:53.139 INFO:tasks.workunit.client.1.smithi138.stdout:1/229: chown d4/d2e/c34 310 1 2024-05-17T21:52:53.141 INFO:tasks.workunit.client.1.smithi138.stdout:9/256: unlink d6/l13 0 2024-05-17T21:52:53.143 INFO:tasks.workunit.client.1.smithi138.stdout:1/230: read d4/db/f4b [23882,100374] 0 2024-05-17T21:52:53.144 INFO:tasks.workunit.client.1.smithi138.stdout:1/231: write d4/db/f4b [108990,44141] 0 2024-05-17T21:52:53.149 INFO:tasks.workunit.client.1.smithi138.stdout:5/246: dread d1/d6/d1b/f1d [0,4194304] 0 2024-05-17T21:52:53.150 INFO:tasks.workunit.client.1.smithi138.stdout:5/247: write d1/f43 [150551,62425] 0 2024-05-17T21:52:53.151 INFO:tasks.workunit.client.1.smithi138.stdout:5/248: chown d1/d24/l44 57194812 1 2024-05-17T21:52:53.151 INFO:tasks.workunit.client.1.smithi138.stdout:4/256: mknod d0/d3/d1d/d38/c5f 0 2024-05-17T21:52:53.154 INFO:tasks.workunit.client.0.smithi107.stdout:7/403: truncate d5/fe 3217535 0 2024-05-17T21:52:53.155 INFO:tasks.workunit.client.0.smithi107.stdout:7/404: write d5/df/d11/d1d/d44/f50 [2708705,22862] 0 2024-05-17T21:52:53.156 INFO:tasks.workunit.client.0.smithi107.stdout:4/457: mkdir d3/d4a/d47/d60/d3c/d9d 0 2024-05-17T21:52:53.158 INFO:tasks.workunit.client.1.smithi138.stdout:5/249: dwrite d1/f43 [0,4194304] 0 2024-05-17T21:52:53.158 INFO:tasks.workunit.client.0.smithi107.stdout:9/431: getdents d0/d80 0 2024-05-17T21:52:53.159 INFO:tasks.workunit.client.0.smithi107.stdout:8/436: rmdir dc/d44 39 2024-05-17T21:52:53.159 INFO:tasks.workunit.client.1.smithi138.stdout:5/250: write d1/d6/d1b/f21 [611656,38597] 0 2024-05-17T21:52:53.161 INFO:tasks.workunit.client.0.smithi107.stdout:7/405: dwrite d5/df/d11/d1d/d33/f46 [0,4194304] 0 2024-05-17T21:52:53.162 INFO:tasks.workunit.client.0.smithi107.stdout:7/406: write d5/f70 [676829,48159] 0 2024-05-17T21:52:53.164 INFO:tasks.workunit.client.0.smithi107.stdout:5/435: rmdir d5/db/d16/d2d/d4d/d6d 39 2024-05-17T21:52:53.167 INFO:tasks.workunit.client.1.smithi138.stdout:4/257: rename d0/d3/d10 to d0/d2f/d49/d60 0 2024-05-17T21:52:53.169 INFO:tasks.workunit.client.0.smithi107.stdout:2/315: symlink d0/d9/d1c/d3e/d4f/l63 0 2024-05-17T21:52:53.170 INFO:tasks.workunit.client.1.smithi138.stdout:6/314: dwrite d7/db/d12/f27 [0,4194304] 0 2024-05-17T21:52:53.172 INFO:tasks.workunit.client.0.smithi107.stdout:6/362: write d9/d16/d22/d2d/f39 [342621,122127] 0 2024-05-17T21:52:53.177 INFO:tasks.workunit.client.1.smithi138.stdout:1/232: mknod d4/c51 0 2024-05-17T21:52:53.178 INFO:tasks.workunit.client.1.smithi138.stdout:1/233: chown d4/dd/f2d 1020 1 2024-05-17T21:52:53.178 INFO:tasks.workunit.client.0.smithi107.stdout:4/458: creat d3/d4a/d47/d60/d3a/d41/d80/d43/f9e x:0 0 0 2024-05-17T21:52:53.179 INFO:tasks.workunit.client.0.smithi107.stdout:4/459: write d3/f2c [2510314,65979] 0 2024-05-17T21:52:53.182 INFO:tasks.workunit.client.0.smithi107.stdout:8/437: truncate dc/d44/f84 764647 0 2024-05-17T21:52:53.185 INFO:tasks.workunit.client.1.smithi138.stdout:5/251: dwrite d1/d6/f41 [0,4194304] 0 2024-05-17T21:52:53.186 INFO:tasks.workunit.client.1.smithi138.stdout:5/252: chown d1/d6/l10 10072 1 2024-05-17T21:52:53.187 INFO:tasks.workunit.client.0.smithi107.stdout:3/429: write d6/d8/d10/f30 [1320361,12400] 0 2024-05-17T21:52:53.188 INFO:tasks.workunit.client.0.smithi107.stdout:2/316: mkdir d0/d26/d64 0 2024-05-17T21:52:53.190 INFO:tasks.workunit.client.0.smithi107.stdout:1/457: link d4/df/d4e/l5d d4/df/d4e/l97 0 2024-05-17T21:52:53.191 INFO:tasks.workunit.client.1.smithi138.stdout:4/258: symlink d0/d1b/d2b/d55/l61 0 2024-05-17T21:52:53.198 INFO:tasks.workunit.client.1.smithi138.stdout:4/259: dwrite d0/d1b/d42/f4f [0,4194304] 0 2024-05-17T21:52:53.199 INFO:tasks.workunit.client.1.smithi138.stdout:9/257: write d6/d7/f10 [2449861,69646] 0 2024-05-17T21:52:53.200 INFO:tasks.workunit.client.0.smithi107.stdout:8/438: rmdir dc/d21/d27 39 2024-05-17T21:52:53.201 INFO:tasks.workunit.client.1.smithi138.stdout:1/234: mknod d4/c52 0 2024-05-17T21:52:53.206 INFO:tasks.workunit.client.0.smithi107.stdout:3/430: mkdir d6/d2a/d2d/d37/d8c 0 2024-05-17T21:52:53.207 INFO:tasks.workunit.client.1.smithi138.stdout:4/260: rename d0/d1b/d42/c46 to d0/d2f/d49/c62 0 2024-05-17T21:52:53.210 INFO:tasks.workunit.client.1.smithi138.stdout:4/261: truncate d0/d2f/d49/d60/d1f/d32/f43 987534 0 2024-05-17T21:52:53.211 INFO:tasks.workunit.client.0.smithi107.stdout:6/363: symlink d9/d58/d66/l6d 0 2024-05-17T21:52:53.212 INFO:tasks.workunit.client.1.smithi138.stdout:6/315: link d7/d8/f50 d7/f68 0 2024-05-17T21:52:53.213 INFO:tasks.workunit.client.0.smithi107.stdout:9/432: dwrite d0/dd/d21/d29/d3e/f5e [0,4194304] 0 2024-05-17T21:52:53.213 INFO:tasks.workunit.client.0.smithi107.stdout:4/460: mknod d3/d6/d26/d36/d8b/d95/c9f 0 2024-05-17T21:52:53.213 INFO:tasks.workunit.client.1.smithi138.stdout:6/316: fdatasync d7/db/d12/f3e 0 2024-05-17T21:52:53.214 INFO:tasks.workunit.client.1.smithi138.stdout:5/253: creat d1/d1a/d39/f4c x:0 0 0 2024-05-17T21:52:53.215 INFO:tasks.workunit.client.0.smithi107.stdout:5/436: dwrite d5/db/d16/d2d/f4a [0,4194304] 0 2024-05-17T21:52:53.215 INFO:tasks.workunit.client.0.smithi107.stdout:2/317: dwrite d0/d26/f55 [0,4194304] 0 2024-05-17T21:52:53.216 INFO:tasks.workunit.client.1.smithi138.stdout:5/254: write d1/f45 [598992,105380] 0 2024-05-17T21:52:53.217 INFO:tasks.workunit.client.0.smithi107.stdout:2/318: readlink d0/d9/d1c/d3e/d4f/l63 0 2024-05-17T21:52:53.218 INFO:tasks.workunit.client.1.smithi138.stdout:4/262: creat d0/d3/d17/f63 x:0 0 0 2024-05-17T21:52:53.219 INFO:tasks.workunit.client.1.smithi138.stdout:4/263: chown d0/d3/d17/f63 634583403 1 2024-05-17T21:52:53.224 INFO:tasks.workunit.client.1.smithi138.stdout:5/255: symlink d1/d6/d1b/d29/d36/l4d 0 2024-05-17T21:52:53.225 INFO:tasks.workunit.client.1.smithi138.stdout:5/256: chown d1/d6/d1b/d29/f3b 12454371 1 2024-05-17T21:52:53.226 INFO:tasks.workunit.client.1.smithi138.stdout:6/317: mkdir d7/d69 0 2024-05-17T21:52:53.231 INFO:tasks.workunit.client.1.smithi138.stdout:6/318: mkdir d7/db/d43/d5f/d6a 0 2024-05-17T21:52:53.232 INFO:tasks.workunit.client.0.smithi107.stdout:3/431: rename d6/d2a/d2d/d48/d4f/c6b to d6/d8/d10/d35/c8d 0 2024-05-17T21:52:53.235 INFO:tasks.workunit.client.0.smithi107.stdout:9/433: unlink d0/dd/d1b/f7b 0 2024-05-17T21:52:53.236 INFO:tasks.workunit.client.0.smithi107.stdout:1/458: link d4/l68 d4/df/d14/d2e/d43/l98 0 2024-05-17T21:52:53.237 INFO:tasks.workunit.client.0.smithi107.stdout:1/459: fsync d4/f66 0 2024-05-17T21:52:53.239 INFO:tasks.workunit.client.1.smithi138.stdout:6/319: link l5 d7/db/l6b 0 2024-05-17T21:52:53.240 INFO:tasks.workunit.client.0.smithi107.stdout:5/437: truncate d5/db/d16/d2d/d40/d5a/f6c 637215 0 2024-05-17T21:52:53.245 INFO:tasks.workunit.client.0.smithi107.stdout:6/364: getdents d9/d16/d22 0 2024-05-17T21:52:53.246 INFO:tasks.workunit.client.1.smithi138.stdout:9/258: dread d6/d7/d16/d29/f3a [0,4194304] 0 2024-05-17T21:52:53.246 INFO:tasks.workunit.client.0.smithi107.stdout:5/438: mknod d5/db/dc/d1e/d6e/c88 0 2024-05-17T21:52:53.247 INFO:tasks.workunit.client.0.smithi107.stdout:1/460: dread d4/df/d14/d1d/f27 [0,4194304] 0 2024-05-17T21:52:53.247 INFO:tasks.workunit.client.1.smithi138.stdout:6/320: dwrite d7/db/d43/f64 [0,4194304] 0 2024-05-17T21:52:53.248 INFO:tasks.workunit.client.1.smithi138.stdout:5/257: dread d1/d6/d1b/f33 [0,4194304] 0 2024-05-17T21:52:53.249 INFO:tasks.workunit.client.1.smithi138.stdout:6/321: write d7/db/d12/f27 [3100468,82362] 0 2024-05-17T21:52:53.250 INFO:tasks.workunit.client.1.smithi138.stdout:4/264: dread d0/d3/d1d/d38/f50 [0,4194304] 0 2024-05-17T21:52:53.250 INFO:tasks.workunit.client.0.smithi107.stdout:9/434: dwrite d0/dd/f35 [4194304,4194304] 0 2024-05-17T21:52:53.251 INFO:tasks.workunit.client.1.smithi138.stdout:5/258: write d1/d1a/d39/f4c [243198,99013] 0 2024-05-17T21:52:53.251 INFO:tasks.workunit.client.1.smithi138.stdout:4/265: chown d0/l9 638 1 2024-05-17T21:52:53.259 INFO:tasks.workunit.client.0.smithi107.stdout:6/365: mkdir d9/d16/d22/d5f/d6e 0 2024-05-17T21:52:53.260 INFO:tasks.workunit.client.0.smithi107.stdout:4/461: dread d3/d4a/d47/d60/d3a/d41/f51 [0,4194304] 0 2024-05-17T21:52:53.260 INFO:tasks.workunit.client.1.smithi138.stdout:5/259: mkdir d1/d1a/d39/d4e 0 2024-05-17T21:52:53.261 INFO:tasks.workunit.client.0.smithi107.stdout:5/439: fdatasync d5/db/dc/f20 0 2024-05-17T21:52:53.263 INFO:tasks.workunit.client.0.smithi107.stdout:0/489: sync 2024-05-17T21:52:53.264 INFO:tasks.workunit.client.0.smithi107.stdout:1/461: creat d4/df/d1a/d26/f99 x:0 0 0 2024-05-17T21:52:53.265 INFO:tasks.workunit.client.1.smithi138.stdout:5/260: dread d1/d6/d1b/f33 [0,4194304] 0 2024-05-17T21:52:53.266 INFO:tasks.workunit.client.0.smithi107.stdout:9/435: symlink d0/dd/d21/d29/d3e/d45/d68/l95 0 2024-05-17T21:52:53.267 INFO:tasks.workunit.client.0.smithi107.stdout:3/432: getdents d6/d8/d10/d35/d36/d66 0 2024-05-17T21:52:53.269 INFO:tasks.workunit.client.0.smithi107.stdout:3/433: chown d6/d2a/d2d/d48/l61 63815911 1 2024-05-17T21:52:53.272 INFO:tasks.workunit.client.0.smithi107.stdout:9/436: dwrite d0/dd/d3d/f92 [0,4194304] 0 2024-05-17T21:52:53.273 INFO:tasks.workunit.client.0.smithi107.stdout:9/437: write d0/f10 [2140024,81853] 0 2024-05-17T21:52:53.275 INFO:tasks.workunit.client.1.smithi138.stdout:4/266: getdents d0/d1b/d2b/d55 0 2024-05-17T21:52:53.276 INFO:tasks.workunit.client.1.smithi138.stdout:4/267: chown d0/c2c 31 1 2024-05-17T21:52:53.277 INFO:tasks.workunit.client.0.smithi107.stdout:5/440: mkdir d5/db/d16/d54/d59/d65/d48/d89 0 2024-05-17T21:52:53.278 INFO:tasks.workunit.client.1.smithi138.stdout:9/259: dread d6/f33 [0,4194304] 0 2024-05-17T21:52:53.279 INFO:tasks.workunit.client.0.smithi107.stdout:5/441: write d5/db/d39/f3c [643626,81430] 0 2024-05-17T21:52:53.280 INFO:tasks.workunit.client.1.smithi138.stdout:5/261: creat d1/f4f x:0 0 0 2024-05-17T21:52:53.280 INFO:tasks.workunit.client.0.smithi107.stdout:8/439: dwrite dc/d21/d27/d2f/f7f [0,4194304] 0 2024-05-17T21:52:53.281 INFO:tasks.workunit.client.0.smithi107.stdout:5/442: chown d5/db/d16/d54/d59/d65/d48/c69 14 1 2024-05-17T21:52:53.285 INFO:tasks.workunit.client.0.smithi107.stdout:1/462: unlink d4/df/d14/d2e/d43/f5f 0 2024-05-17T21:52:53.287 INFO:tasks.workunit.client.0.smithi107.stdout:1/463: write d4/df/d4e/f63 [983797,114137] 0 2024-05-17T21:52:53.287 INFO:tasks.workunit.client.0.smithi107.stdout:1/464: fdatasync d4/df/d1a/f7b 0 2024-05-17T21:52:53.288 INFO:tasks.workunit.client.1.smithi138.stdout:9/260: symlink d6/d7/d16/l52 0 2024-05-17T21:52:53.288 INFO:tasks.workunit.client.0.smithi107.stdout:1/465: stat d4/de/f19 0 2024-05-17T21:52:53.291 INFO:tasks.workunit.client.1.smithi138.stdout:5/262: mknod d1/d24/c50 0 2024-05-17T21:52:53.292 INFO:tasks.workunit.client.0.smithi107.stdout:3/434: creat d6/d8/d10/d35/d36/f8e x:0 0 0 2024-05-17T21:52:53.293 INFO:tasks.workunit.client.0.smithi107.stdout:3/435: read - d6/d8/d10/d35/d36/f8b zero size 2024-05-17T21:52:53.294 INFO:tasks.workunit.client.0.smithi107.stdout:4/462: link d3/db/d1e/f77 d3/db/d35/d5e/fa0 0 2024-05-17T21:52:53.294 INFO:tasks.workunit.client.0.smithi107.stdout:6/366: dread d9/f15 [0,4194304] 0 2024-05-17T21:52:53.295 INFO:tasks.workunit.client.0.smithi107.stdout:4/463: chown d3/d6/d26/d36/d37/c39 5995065 1 2024-05-17T21:52:53.295 INFO:tasks.workunit.client.1.smithi138.stdout:9/261: unlink d6/f2d 0 2024-05-17T21:52:53.296 INFO:tasks.workunit.client.0.smithi107.stdout:4/464: fsync d3/d4a/d47/f4b 0 2024-05-17T21:52:53.297 INFO:tasks.workunit.client.0.smithi107.stdout:4/465: chown d3/d4a/d47/d60/d3a/d41/d80/f44 6411698 1 2024-05-17T21:52:53.297 INFO:tasks.workunit.client.0.smithi107.stdout:9/438: rename d0/dd/d21/d29/d3e/f49 to d0/dd/d2f/d87/d89/f96 0 2024-05-17T21:52:53.299 INFO:tasks.workunit.client.0.smithi107.stdout:3/436: dread d6/d8/d10/d35/d36/d4c/f65 [0,4194304] 0 2024-05-17T21:52:53.300 INFO:tasks.workunit.client.1.smithi138.stdout:4/268: link d0/d1b/d42/d45/f48 d0/d2f/d49/d60/d1f/d32/f64 0 2024-05-17T21:52:53.300 INFO:tasks.workunit.client.0.smithi107.stdout:3/437: dread - d6/d8/f85 zero size 2024-05-17T21:52:53.300 INFO:tasks.workunit.client.0.smithi107.stdout:3/438: stat d6/d8/f85 0 2024-05-17T21:52:53.303 INFO:tasks.workunit.client.0.smithi107.stdout:0/490: symlink d3/d75/lb2 0 2024-05-17T21:52:53.307 INFO:tasks.workunit.client.0.smithi107.stdout:8/440: creat dc/d44/d4b/d52/d70/fa4 x:0 0 0 2024-05-17T21:52:53.308 INFO:tasks.workunit.client.0.smithi107.stdout:5/443: creat d5/db/d16/d54/d59/d65/d48/f8a x:0 0 0 2024-05-17T21:52:53.309 INFO:tasks.workunit.client.1.smithi138.stdout:4/269: creat d0/d1b/d2b/f65 x:0 0 0 2024-05-17T21:52:53.310 INFO:tasks.workunit.client.1.smithi138.stdout:4/270: chown d0/d2f/l59 1222007470 1 2024-05-17T21:52:53.312 INFO:tasks.workunit.client.1.smithi138.stdout:9/262: unlink d6/c8 0 2024-05-17T21:52:53.313 INFO:tasks.workunit.client.1.smithi138.stdout:9/263: truncate d6/d7/d16/d29/d38/f3d 840819 0 2024-05-17T21:52:53.318 INFO:tasks.workunit.client.1.smithi138.stdout:9/264: creat d6/d7/d16/d29/d45/f53 x:0 0 0 2024-05-17T21:52:53.318 INFO:tasks.workunit.client.1.smithi138.stdout:9/265: write d6/d7/f9 [454953,116065] 0 2024-05-17T21:52:53.320 INFO:tasks.workunit.client.1.smithi138.stdout:5/263: truncate d1/f22 253633 0 2024-05-17T21:52:53.320 INFO:tasks.workunit.client.0.smithi107.stdout:6/367: dwrite d9/f10 [4194304,4194304] 0 2024-05-17T21:52:53.325 INFO:tasks.workunit.client.0.smithi107.stdout:9/439: creat d0/dd/d2f/d87/d89/f97 x:0 0 0 2024-05-17T21:52:53.325 INFO:tasks.workunit.client.0.smithi107.stdout:9/440: fdatasync d0/dd/d21/f37 0 2024-05-17T21:52:53.326 INFO:tasks.workunit.client.0.smithi107.stdout:3/439: mknod d6/d2a/c8f 0 2024-05-17T21:52:53.326 INFO:tasks.workunit.client.1.smithi138.stdout:5/264: creat d1/d1a/d39/d4e/f51 x:0 0 0 2024-05-17T21:52:53.326 INFO:tasks.workunit.client.1.smithi138.stdout:4/271: truncate d0/d3/d1d/d4c/f4e 152141 0 2024-05-17T21:52:53.326 INFO:tasks.workunit.client.1.smithi138.stdout:4/272: stat d0/d3/d1d/d4c 0 2024-05-17T21:52:53.327 INFO:tasks.workunit.client.1.smithi138.stdout:4/273: chown d0/d1b/d42/c5d 270 1 2024-05-17T21:52:53.328 INFO:tasks.workunit.client.1.smithi138.stdout:4/274: stat d0/d3/f57 0 2024-05-17T21:52:53.328 INFO:tasks.workunit.client.1.smithi138.stdout:4/275: dread - d0/d3/f57 zero size 2024-05-17T21:52:53.329 INFO:tasks.workunit.client.1.smithi138.stdout:5/265: creat d1/d2c/f52 x:0 0 0 2024-05-17T21:52:53.330 INFO:tasks.workunit.client.1.smithi138.stdout:5/266: fsync d1/f45 0 2024-05-17T21:52:53.330 INFO:tasks.workunit.client.0.smithi107.stdout:9/441: dwrite d0/dd/d1f/f40 [0,4194304] 0 2024-05-17T21:52:53.331 INFO:tasks.workunit.client.1.smithi138.stdout:4/276: write d0/d2f/d49/d60/f51 [4109411,63646] 0 2024-05-17T21:52:53.340 INFO:tasks.workunit.client.0.smithi107.stdout:4/466: dwrite d3/d4a/f73 [0,4194304] 0 2024-05-17T21:52:53.342 INFO:tasks.workunit.client.1.smithi138.stdout:5/267: fsync d1/f13 0 2024-05-17T21:52:53.342 INFO:tasks.workunit.client.1.smithi138.stdout:7/196: sync 2024-05-17T21:52:53.343 INFO:tasks.workunit.client.1.smithi138.stdout:7/197: chown d0/l14 149 1 2024-05-17T21:52:53.344 INFO:tasks.workunit.client.1.smithi138.stdout:4/277: mknod d0/d3/d17/c66 0 2024-05-17T21:52:53.349 INFO:tasks.workunit.client.0.smithi107.stdout:4/467: dwrite d3/d6/d26/d36/d37/f5f [0,4194304] 0 2024-05-17T21:52:53.352 INFO:tasks.workunit.client.1.smithi138.stdout:7/198: symlink d0/l34 0 2024-05-17T21:52:53.353 INFO:tasks.workunit.client.0.smithi107.stdout:3/440: dread d6/f2c [0,4194304] 0 2024-05-17T21:52:53.353 INFO:tasks.workunit.client.0.smithi107.stdout:6/368: write d9/f1f [484590,67842] 0 2024-05-17T21:52:53.356 INFO:tasks.workunit.client.0.smithi107.stdout:7/407: sync 2024-05-17T21:52:53.357 INFO:tasks.workunit.client.0.smithi107.stdout:7/408: chown d5 174 1 2024-05-17T21:52:53.357 INFO:tasks.workunit.client.0.smithi107.stdout:0/491: rename d3/f15 to d3/dc/d22/d23/d97/d85/fb3 0 2024-05-17T21:52:53.357 INFO:tasks.workunit.client.1.smithi138.stdout:5/268: mkdir d1/d53 0 2024-05-17T21:52:53.358 INFO:tasks.workunit.client.0.smithi107.stdout:0/492: truncate d3/dc/d22/f98 253796 0 2024-05-17T21:52:53.358 INFO:tasks.workunit.client.0.smithi107.stdout:9/442: creat d0/dd/d2f/d87/f98 x:0 0 0 2024-05-17T21:52:53.360 INFO:tasks.workunit.client.0.smithi107.stdout:8/441: unlink dc/d21/d35/c12 0 2024-05-17T21:52:53.362 INFO:tasks.workunit.client.0.smithi107.stdout:5/444: mknod d5/c8b 0 2024-05-17T21:52:53.364 INFO:tasks.workunit.client.1.smithi138.stdout:5/269: fsync d1/d6/fa 0 2024-05-17T21:52:53.366 INFO:tasks.workunit.client.0.smithi107.stdout:4/468: creat d3/d4a/d47/d60/fa1 x:0 0 0 2024-05-17T21:52:53.366 INFO:tasks.workunit.client.1.smithi138.stdout:4/278: getdents d0/d2f/d49 0 2024-05-17T21:52:53.368 INFO:tasks.workunit.client.0.smithi107.stdout:6/369: mkdir d9/d16/d22/d5f/d6f 0 2024-05-17T21:52:53.368 INFO:tasks.workunit.client.1.smithi138.stdout:5/270: truncate d1/d6/f11 2508631 0 2024-05-17T21:52:53.370 INFO:tasks.workunit.client.1.smithi138.stdout:4/279: symlink d0/d3/d17/d21/l67 0 2024-05-17T21:52:53.371 INFO:tasks.workunit.client.0.smithi107.stdout:3/441: mkdir d6/d8/d3e/d3f/d90 0 2024-05-17T21:52:53.371 INFO:tasks.workunit.client.0.smithi107.stdout:4/469: dwrite d3/d4a/d47/d60/d3a/d41/f5a [0,4194304] 0 2024-05-17T21:52:53.373 INFO:tasks.workunit.client.0.smithi107.stdout:7/409: creat d5/df/d11/d1c/d2f/d53/f90 x:0 0 0 2024-05-17T21:52:53.373 INFO:tasks.workunit.client.1.smithi138.stdout:5/271: dread d1/d6/f31 [0,4194304] 0 2024-05-17T21:52:53.377 INFO:tasks.workunit.client.0.smithi107.stdout:1/466: getdents d4/df/d1a/d26/d7a/d81 0 2024-05-17T21:52:53.380 INFO:tasks.workunit.client.0.smithi107.stdout:6/370: creat d9/d16/d22/d2d/f70 x:0 0 0 2024-05-17T21:52:53.382 INFO:tasks.workunit.client.0.smithi107.stdout:3/442: mkdir d6/d8/d10/d35/d91 0 2024-05-17T21:52:53.383 INFO:tasks.workunit.client.0.smithi107.stdout:3/443: write d6/d8/d10/d25/f46 [1340540,65444] 0 2024-05-17T21:52:53.384 INFO:tasks.workunit.client.0.smithi107.stdout:3/444: stat d6/d2a/d2d/d48/l61 0 2024-05-17T21:52:53.388 INFO:tasks.workunit.client.1.smithi138.stdout:5/272: unlink d1/f22 0 2024-05-17T21:52:53.389 INFO:tasks.workunit.client.1.smithi138.stdout:7/199: mknod d0/c35 0 2024-05-17T21:52:53.393 INFO:tasks.workunit.client.0.smithi107.stdout:1/467: mkdir d4/df/d14/d2e/d43/d47/d9a 0 2024-05-17T21:52:53.394 INFO:tasks.workunit.client.1.smithi138.stdout:7/200: write d0/f10 [4306785,200] 0 2024-05-17T21:52:53.399 INFO:tasks.workunit.client.0.smithi107.stdout:4/470: fsync d3/d4a/d47/d60/d3a/d41/d80/f8f 0 2024-05-17T21:52:53.402 INFO:tasks.workunit.client.1.smithi138.stdout:7/201: readlink d0/ld 0 2024-05-17T21:52:53.403 INFO:tasks.workunit.client.0.smithi107.stdout:4/471: dread d3/d4a/d47/d60/d3a/d41/f5a [0,4194304] 0 2024-05-17T21:52:53.403 INFO:tasks.workunit.client.0.smithi107.stdout:4/472: chown d3/db/d1e/f6c 0 1 2024-05-17T21:52:53.406 INFO:tasks.workunit.client.1.smithi138.stdout:5/273: creat d1/d6/d1b/f54 x:0 0 0 2024-05-17T21:52:53.409 INFO:tasks.workunit.client.0.smithi107.stdout:9/443: dwrite d0/dd/d1b/f53 [4194304,4194304] 0 2024-05-17T21:52:53.410 INFO:tasks.workunit.client.0.smithi107.stdout:0/493: write d3/dc/d22/f40 [1043818,111532] 0 2024-05-17T21:52:53.412 INFO:tasks.workunit.client.0.smithi107.stdout:0/494: readlink d3/dc/d22/l30 0 2024-05-17T21:52:53.412 INFO:tasks.workunit.client.1.smithi138.stdout:5/274: dwrite d1/d2c/f52 [0,4194304] 0 2024-05-17T21:52:53.413 INFO:tasks.workunit.client.0.smithi107.stdout:8/442: write dc/f4f [2027081,72685] 0 2024-05-17T21:52:53.414 INFO:tasks.workunit.client.0.smithi107.stdout:0/495: write d3/f33 [15887,39181] 0 2024-05-17T21:52:53.415 INFO:tasks.workunit.client.0.smithi107.stdout:6/371: write d9/d16/d22/f27 [4909144,46380] 0 2024-05-17T21:52:53.418 INFO:tasks.workunit.client.0.smithi107.stdout:3/445: creat d6/d8/d10/d35/d36/d4c/d83/f92 x:0 0 0 2024-05-17T21:52:53.419 INFO:tasks.workunit.client.0.smithi107.stdout:3/446: chown d6/d8/f76 6953 1 2024-05-17T21:52:53.420 INFO:tasks.workunit.client.0.smithi107.stdout:3/447: stat d6/d8/d10/f30 0 2024-05-17T21:52:53.422 INFO:tasks.workunit.client.1.smithi138.stdout:4/280: dwrite d0/d3/d1d/d4c/f4e [0,4194304] 0 2024-05-17T21:52:53.423 INFO:tasks.workunit.client.0.smithi107.stdout:4/473: dread d3/db/f1f [4194304,4194304] 0 2024-05-17T21:52:53.423 INFO:tasks.workunit.client.1.smithi138.stdout:4/281: fdatasync d0/d1b/d42/f4f 0 2024-05-17T21:52:53.423 INFO:tasks.workunit.client.1.smithi138.stdout:4/282: readlink d0/d2f/l59 0 2024-05-17T21:52:53.424 INFO:tasks.workunit.client.0.smithi107.stdout:4/474: write d3/db/f69 [4007372,60183] 0 2024-05-17T21:52:53.430 INFO:tasks.workunit.client.1.smithi138.stdout:7/202: getdents d0/d18 0 2024-05-17T21:52:53.431 INFO:tasks.workunit.client.0.smithi107.stdout:9/444: mknod d0/dd/d21/d29/d57/d5f/c99 0 2024-05-17T21:52:53.431 INFO:tasks.workunit.client.1.smithi138.stdout:5/275: link d1/d6/c2d d1/d1a/d39/c55 0 2024-05-17T21:52:53.432 INFO:tasks.workunit.client.0.smithi107.stdout:6/372: fdatasync d9/f4e 0 2024-05-17T21:52:53.433 INFO:tasks.workunit.client.1.smithi138.stdout:4/283: fsync d0/d2f/d49/d60/f25 0 2024-05-17T21:52:53.435 INFO:tasks.workunit.client.0.smithi107.stdout:8/443: symlink dc/d21/d97/d63/d71/d81/la5 0 2024-05-17T21:52:53.439 INFO:tasks.workunit.client.0.smithi107.stdout:7/410: getdents d5/df/d11/d1c/d2f/d53 0 2024-05-17T21:52:53.441 INFO:tasks.workunit.client.1.smithi138.stdout:7/203: dwrite d0/d18/f1d [0,4194304] 0 2024-05-17T21:52:53.442 INFO:tasks.workunit.client.1.smithi138.stdout:4/284: symlink d0/d2f/l68 0 2024-05-17T21:52:53.442 INFO:tasks.workunit.client.1.smithi138.stdout:4/285: chown d0/d3/f40 529889327 1 2024-05-17T21:52:53.445 INFO:tasks.workunit.client.1.smithi138.stdout:5/276: mknod d1/d24/d2f/c56 0 2024-05-17T21:52:53.449 INFO:tasks.workunit.client.1.smithi138.stdout:7/204: dread - d0/f30 zero size 2024-05-17T21:52:53.449 INFO:tasks.workunit.client.1.smithi138.stdout:4/286: dread d0/d3/d1d/d4c/f4e [0,4194304] 0 2024-05-17T21:52:53.449 INFO:tasks.workunit.client.0.smithi107.stdout:4/475: fdatasync d3/fa 0 2024-05-17T21:52:53.453 INFO:tasks.workunit.client.0.smithi107.stdout:0/496: fsync d3/dc/d22/d3d/f4d 0 2024-05-17T21:52:53.454 INFO:tasks.workunit.client.0.smithi107.stdout:8/444: rmdir dc/d44 39 2024-05-17T21:52:53.456 INFO:tasks.workunit.client.0.smithi107.stdout:7/411: rmdir d5/df/d11/d1c 39 2024-05-17T21:52:53.457 INFO:tasks.workunit.client.0.smithi107.stdout:7/412: write d5/df/d11/f25 [4961046,98435] 0 2024-05-17T21:52:53.458 INFO:tasks.workunit.client.1.smithi138.stdout:5/277: rename d1/d6/f11 to d1/d53/f57 0 2024-05-17T21:52:53.458 INFO:tasks.workunit.client.0.smithi107.stdout:1/468: getdents d4/df/d69 0 2024-05-17T21:52:53.462 INFO:tasks.workunit.client.0.smithi107.stdout:0/497: creat d3/d24/d6f/d9f/dab/fb4 x:0 0 0 2024-05-17T21:52:53.464 INFO:tasks.workunit.client.0.smithi107.stdout:0/498: write d3/dc/d22/d23/f2c [108920,123891] 0 2024-05-17T21:52:53.465 INFO:tasks.workunit.client.0.smithi107.stdout:1/469: dwrite d4/df/d69/f6e [0,4194304] 0 2024-05-17T21:52:53.465 INFO:tasks.workunit.client.1.smithi138.stdout:4/287: write d0/d2f/d49/d60/d1f/d32/f64 [1217392,78257] 0 2024-05-17T21:52:53.466 INFO:tasks.workunit.client.1.smithi138.stdout:4/288: dread - d0/d1b/d36/f47 zero size 2024-05-17T21:52:53.467 INFO:tasks.workunit.client.1.smithi138.stdout:7/205: link d0/d18/c32 d0/d18/c36 0 2024-05-17T21:52:53.469 INFO:tasks.workunit.client.0.smithi107.stdout:7/413: mkdir d5/d16/d3b/d45/d91 0 2024-05-17T21:52:53.469 INFO:tasks.workunit.client.1.smithi138.stdout:4/289: creat d0/d3/db/f69 x:0 0 0 2024-05-17T21:52:53.477 INFO:tasks.workunit.client.0.smithi107.stdout:0/499: rename d3/d24/d6f/d9f/dad to d3/d24/d6f/db5 0 2024-05-17T21:52:53.477 INFO:tasks.workunit.client.1.smithi138.stdout:4/290: getdents d0/d3/d17/d5e 0 2024-05-17T21:52:53.478 INFO:tasks.workunit.client.0.smithi107.stdout:1/470: mknod d4/df/d1a/d26/d50/c9b 0 2024-05-17T21:52:53.479 INFO:tasks.workunit.client.0.smithi107.stdout:3/448: link d6/d1a/d57/f5e d6/d2a/d2d/d48/f93 0 2024-05-17T21:52:53.481 INFO:tasks.workunit.client.1.smithi138.stdout:4/291: dread d0/d1b/f18 [0,4194304] 0 2024-05-17T21:52:53.483 INFO:tasks.workunit.client.0.smithi107.stdout:0/500: dwrite d3/dc/d22/f7d [4194304,4194304] 0 2024-05-17T21:52:53.484 INFO:tasks.workunit.client.1.smithi138.stdout:4/292: dread d0/d1b/f3e [0,4194304] 0 2024-05-17T21:52:53.485 INFO:tasks.workunit.client.1.smithi138.stdout:4/293: readlink d0/d2f/d49/d60/l23 0 2024-05-17T21:52:53.487 INFO:tasks.workunit.client.0.smithi107.stdout:2/319: sync 2024-05-17T21:52:53.488 INFO:tasks.workunit.client.1.smithi138.stdout:4/294: creat d0/d2f/f6a x:0 0 0 2024-05-17T21:52:53.490 INFO:tasks.workunit.client.1.smithi138.stdout:4/295: write d0/d2f/d49/d60/d1f/d32/f64 [3184821,31456] 0 2024-05-17T21:52:53.491 INFO:tasks.workunit.client.0.smithi107.stdout:7/414: mkdir d5/df/d11/d1c/d31/d39/d92 0 2024-05-17T21:52:53.494 INFO:tasks.workunit.client.1.smithi138.stdout:7/206: dread d0/f31 [0,4194304] 0 2024-05-17T21:52:53.495 INFO:tasks.workunit.client.1.smithi138.stdout:7/207: dread - d0/f30 zero size 2024-05-17T21:52:53.495 INFO:tasks.workunit.client.0.smithi107.stdout:0/501: creat d3/dc/d22/d23/d97/d77/d91/fb6 x:0 0 0 2024-05-17T21:52:53.495 INFO:tasks.workunit.client.0.smithi107.stdout:2/320: dwrite d0/d9/d11/f5f [0,4194304] 0 2024-05-17T21:52:53.497 INFO:tasks.workunit.client.0.smithi107.stdout:2/321: chown d0/d9/d11/d18/c1a 14 1 2024-05-17T21:52:53.497 INFO:tasks.workunit.client.0.smithi107.stdout:8/445: symlink dc/d44/d4b/d52/d70/d76/la6 0 2024-05-17T21:52:53.498 INFO:tasks.workunit.client.0.smithi107.stdout:0/502: dread - d3/d24/d6f/d9f/fa0 zero size 2024-05-17T21:52:53.498 INFO:tasks.workunit.client.1.smithi138.stdout:4/296: unlink d0/d3/db/f35 0 2024-05-17T21:52:53.500 INFO:tasks.workunit.client.0.smithi107.stdout:1/471: mknod d4/c9c 0 2024-05-17T21:52:53.500 INFO:tasks.workunit.client.1.smithi138.stdout:7/208: symlink d0/d18/l37 0 2024-05-17T21:52:53.501 INFO:tasks.workunit.client.0.smithi107.stdout:3/449: dread d6/d2a/f42 [0,4194304] 0 2024-05-17T21:52:53.504 INFO:tasks.workunit.client.1.smithi138.stdout:4/297: mknod d0/d1b/d2b/d55/c6b 0 2024-05-17T21:52:53.504 INFO:tasks.workunit.client.0.smithi107.stdout:0/503: creat d3/dc/d22/d23/fb7 x:0 0 0 2024-05-17T21:52:53.505 INFO:tasks.workunit.client.1.smithi138.stdout:7/209: chown d0/f2b 1579195 1 2024-05-17T21:52:53.508 INFO:tasks.workunit.client.0.smithi107.stdout:1/472: rename d4/f10 to d4/df/d14/d33/f9d 0 2024-05-17T21:52:53.513 INFO:tasks.workunit.client.1.smithi138.stdout:4/298: link d0/d1b/d42/f4f d0/d3/d17/d21/f6c 0 2024-05-17T21:52:53.513 INFO:tasks.workunit.client.1.smithi138.stdout:4/299: stat d0 0 2024-05-17T21:52:53.513 INFO:tasks.workunit.client.1.smithi138.stdout:4/300: dread - d0/d1b/d2b/f65 zero size 2024-05-17T21:52:53.513 INFO:tasks.workunit.client.0.smithi107.stdout:0/504: symlink d3/d58/d61/d94/lb8 0 2024-05-17T21:52:53.514 INFO:tasks.workunit.client.1.smithi138.stdout:4/301: chown d0/c13 70 1 2024-05-17T21:52:53.515 INFO:tasks.workunit.client.1.smithi138.stdout:7/210: unlink d0/c22 0 2024-05-17T21:52:53.516 INFO:tasks.workunit.client.1.smithi138.stdout:0/338: sync 2024-05-17T21:52:53.516 INFO:tasks.workunit.client.1.smithi138.stdout:8/163: sync 2024-05-17T21:52:53.516 INFO:tasks.workunit.client.1.smithi138.stdout:3/330: sync 2024-05-17T21:52:53.516 INFO:tasks.workunit.client.1.smithi138.stdout:3/331: stat d1/d62 0 2024-05-17T21:52:53.520 INFO:tasks.workunit.client.0.smithi107.stdout:3/450: link d6/d2a/f42 d6/d2a/d2d/d48/d4f/f94 0 2024-05-17T21:52:53.523 INFO:tasks.workunit.client.1.smithi138.stdout:0/339: dwrite d3/f7 [0,4194304] 0 2024-05-17T21:52:53.524 INFO:tasks.workunit.client.0.smithi107.stdout:0/505: rmdir d3/d24 39 2024-05-17T21:52:53.529 INFO:tasks.workunit.client.0.smithi107.stdout:8/446: link dc/d21/d27/d34/c3a dc/d21/d27/d83/ca7 0 2024-05-17T21:52:53.532 INFO:tasks.workunit.client.0.smithi107.stdout:3/451: chown d6/d8/c15 669 1 2024-05-17T21:52:53.533 INFO:tasks.workunit.client.0.smithi107.stdout:2/322: getdents d0/d9 0 2024-05-17T21:52:53.533 INFO:tasks.workunit.client.0.smithi107.stdout:0/506: rmdir d3/d58/d61/d94 39 2024-05-17T21:52:53.534 INFO:tasks.workunit.client.0.smithi107.stdout:2/323: write d0/d9/f33 [1203903,92479] 0 2024-05-17T21:52:53.536 INFO:tasks.workunit.client.0.smithi107.stdout:1/473: link d4/df/d14/f4c d4/df/d14/d2e/f9e 0 2024-05-17T21:52:53.536 INFO:tasks.workunit.client.0.smithi107.stdout:2/324: fsync d0/d9/d11/f5f 0 2024-05-17T21:52:53.538 INFO:tasks.workunit.client.0.smithi107.stdout:3/452: rmdir d6/d2a/d2d/d37/d8c 0 2024-05-17T21:52:53.539 INFO:tasks.workunit.client.0.smithi107.stdout:0/507: creat d3/dc/d22/fb9 x:0 0 0 2024-05-17T21:52:53.541 INFO:tasks.workunit.client.0.smithi107.stdout:0/508: fdatasync d3/f1e 0 2024-05-17T21:52:53.542 INFO:tasks.workunit.client.0.smithi107.stdout:8/447: dwrite dc/f8b [0,4194304] 0 2024-05-17T21:52:53.545 INFO:tasks.workunit.client.0.smithi107.stdout:1/474: link d4/de/f67 d4/df/d1a/d26/d7a/d81/f9f 0 2024-05-17T21:52:53.545 INFO:tasks.workunit.client.0.smithi107.stdout:8/448: mknod dc/d21/d27/d83/ca8 0 2024-05-17T21:52:53.545 INFO:tasks.workunit.client.0.smithi107.stdout:0/509: symlink d3/d24/d6f/d9f/lba 0 2024-05-17T21:52:53.547 INFO:tasks.workunit.client.0.smithi107.stdout:1/475: creat d4/df/d1a/d26/d50/fa0 x:0 0 0 2024-05-17T21:52:53.548 INFO:tasks.workunit.client.0.smithi107.stdout:0/510: fsync d3/d24/f2a 0 2024-05-17T21:52:53.548 INFO:tasks.workunit.client.0.smithi107.stdout:8/449: truncate dc/d21/d35/f61 387572 0 2024-05-17T21:52:53.549 INFO:tasks.workunit.client.0.smithi107.stdout:0/511: chown d3/f35 127786 1 2024-05-17T21:52:53.550 INFO:tasks.workunit.client.0.smithi107.stdout:1/476: truncate d4/df/d14/d1d/f27 1336451 0 2024-05-17T21:52:53.551 INFO:tasks.workunit.client.0.smithi107.stdout:0/512: creat d3/dc/dae/fbb x:0 0 0 2024-05-17T21:52:53.553 INFO:tasks.workunit.client.0.smithi107.stdout:8/450: dread dc/d21/d27/f5b [0,4194304] 0 2024-05-17T21:52:53.554 INFO:tasks.workunit.client.0.smithi107.stdout:0/513: creat d3/d75/fbc x:0 0 0 2024-05-17T21:52:53.557 INFO:tasks.workunit.client.0.smithi107.stdout:0/514: rename d3/dc/d31/f57 to d3/d24/d6f/d9f/fbd 0 2024-05-17T21:52:53.559 INFO:tasks.workunit.client.0.smithi107.stdout:8/451: link dc/d21/d97/l57 dc/d21/d27/d2f/la9 0 2024-05-17T21:52:53.560 INFO:tasks.workunit.client.0.smithi107.stdout:0/515: mknod d3/dc/d22/cbe 0 2024-05-17T21:52:53.560 INFO:tasks.workunit.client.0.smithi107.stdout:0/516: readlink d3/dc/l1c 0 2024-05-17T21:52:53.560 INFO:tasks.workunit.client.0.smithi107.stdout:8/452: mkdir dc/d21/daa 0 2024-05-17T21:52:53.562 INFO:tasks.workunit.client.0.smithi107.stdout:0/517: truncate d3/dc/f16 896567 0 2024-05-17T21:52:53.562 INFO:tasks.workunit.client.0.smithi107.stdout:8/453: write dc/d21/d69/f77 [5174161,31062] 0 2024-05-17T21:52:53.563 INFO:tasks.workunit.client.0.smithi107.stdout:0/518: mknod d3/d24/cbf 0 2024-05-17T21:52:53.564 INFO:tasks.workunit.client.0.smithi107.stdout:8/454: chown dc/d21/d97/d63/f99 195 1 2024-05-17T21:52:53.568 INFO:tasks.workunit.client.1.smithi138.stdout:7/211: dwrite d0/f12 [4194304,4194304] 0 2024-05-17T21:52:53.568 INFO:tasks.workunit.client.1.smithi138.stdout:0/340: write d3/d10/f25 [4191295,77749] 0 2024-05-17T21:52:53.569 INFO:tasks.workunit.client.1.smithi138.stdout:4/302: truncate d0/d3/d1d/d4c/f4e 2622937 0 2024-05-17T21:52:53.570 INFO:tasks.workunit.client.0.smithi107.stdout:8/455: dwrite dc/d44/d4b/d52/f89 [0,4194304] 0 2024-05-17T21:52:53.571 INFO:tasks.workunit.client.1.smithi138.stdout:0/341: chown d3/d3d/f44 436396474 1 2024-05-17T21:52:53.576 INFO:tasks.workunit.client.1.smithi138.stdout:3/332: link d1/d4f/d50/f54 d1/d3/d16/d3e/f73 0 2024-05-17T21:52:53.581 INFO:tasks.workunit.client.1.smithi138.stdout:8/164: write d0/d11/f1c [539601,89718] 0 2024-05-17T21:52:53.582 INFO:tasks.workunit.client.0.smithi107.stdout:8/456: chown dc/d21/d27/d2f/f30 1760707115 1 2024-05-17T21:52:53.583 INFO:tasks.workunit.client.0.smithi107.stdout:8/457: readlink dc/d21/l2a 0 2024-05-17T21:52:53.583 INFO:tasks.workunit.client.0.smithi107.stdout:8/458: readlink dc/d21/d27/l40 0 2024-05-17T21:52:53.584 INFO:tasks.workunit.client.1.smithi138.stdout:4/303: mknod d0/d3/d1d/c6d 0 2024-05-17T21:52:53.586 INFO:tasks.workunit.client.1.smithi138.stdout:0/342: write d3/d10/f71 [232869,65632] 0 2024-05-17T21:52:53.587 INFO:tasks.workunit.client.1.smithi138.stdout:3/333: creat d1/d3/d16/d3e/f74 x:0 0 0 2024-05-17T21:52:53.587 INFO:tasks.workunit.client.0.smithi107.stdout:8/459: mkdir dc/d21/d35/dab 0 2024-05-17T21:52:53.591 INFO:tasks.workunit.client.0.smithi107.stdout:8/460: mknod dc/cac 0 2024-05-17T21:52:53.592 INFO:tasks.workunit.client.0.smithi107.stdout:8/461: chown dc/d21/d97/d63/d71 190861 1 2024-05-17T21:52:53.592 INFO:tasks.workunit.client.1.smithi138.stdout:8/165: dwrite d0/f17 [4194304,4194304] 0 2024-05-17T21:52:53.592 INFO:tasks.workunit.client.1.smithi138.stdout:0/343: write d3/d10/d38/d3e/d59/f68 [927385,96225] 0 2024-05-17T21:52:53.595 INFO:tasks.workunit.client.1.smithi138.stdout:8/166: symlink d0/d11/l30 0 2024-05-17T21:52:53.601 INFO:tasks.workunit.client.1.smithi138.stdout:3/334: link d1/d3/d16/d3a/l66 d1/d4f/d50/l75 0 2024-05-17T21:52:53.603 INFO:tasks.workunit.client.1.smithi138.stdout:8/167: dwrite d0/dc/d1a/f24 [0,4194304] 0 2024-05-17T21:52:53.604 INFO:tasks.workunit.client.1.smithi138.stdout:3/335: dread d1/d3/d16/d3e/f43 [0,4194304] 0 2024-05-17T21:52:53.607 INFO:tasks.workunit.client.0.smithi107.stdout:2/325: dwrite d0/d9/d11/f14 [0,4194304] 0 2024-05-17T21:52:53.609 INFO:tasks.workunit.client.1.smithi138.stdout:8/168: mkdir d0/d31 0 2024-05-17T21:52:53.609 INFO:tasks.workunit.client.1.smithi138.stdout:7/212: dread d0/f13 [0,4194304] 0 2024-05-17T21:52:53.610 INFO:tasks.workunit.client.1.smithi138.stdout:0/344: getdents d3/d10/d38/d3e 0 2024-05-17T21:52:53.613 INFO:tasks.workunit.client.1.smithi138.stdout:3/336: mknod d1/d3/d16/d3a/c76 0 2024-05-17T21:52:53.613 INFO:tasks.workunit.client.0.smithi107.stdout:3/453: dread d6/d8/d3e/d3f/f87 [0,4194304] 0 2024-05-17T21:52:53.616 INFO:tasks.workunit.client.0.smithi107.stdout:1/477: truncate d4/de/f1e 2015611 0 2024-05-17T21:52:53.618 INFO:tasks.workunit.client.1.smithi138.stdout:7/213: chown d0/f1e 38 1 2024-05-17T21:52:53.618 INFO:tasks.workunit.client.0.smithi107.stdout:3/454: chown d6/d1a/d57/l62 1892171 1 2024-05-17T21:52:53.619 INFO:tasks.workunit.client.0.smithi107.stdout:3/455: chown d6/c18 6 1 2024-05-17T21:52:53.620 INFO:tasks.workunit.client.1.smithi138.stdout:4/304: dread d0/d2f/d49/d60/f25 [0,4194304] 0 2024-05-17T21:52:53.621 INFO:tasks.workunit.client.1.smithi138.stdout:4/305: write d0/d2f/d49/d60/f51 [3680986,85220] 0 2024-05-17T21:52:53.622 INFO:tasks.workunit.client.1.smithi138.stdout:4/306: fdatasync d0/d2f/d49/d60/d1f/d32/f43 0 2024-05-17T21:52:53.623 INFO:tasks.workunit.client.0.smithi107.stdout:2/326: link d0/d9/d11/d18/f20 d0/d9/d1c/d22/f65 0 2024-05-17T21:52:53.624 INFO:tasks.workunit.client.0.smithi107.stdout:2/327: write d0/d9/d11/f5f [3110607,96071] 0 2024-05-17T21:52:53.625 INFO:tasks.workunit.client.1.smithi138.stdout:3/337: unlink d1/d3/d8/f13 0 2024-05-17T21:52:53.626 INFO:tasks.workunit.client.1.smithi138.stdout:3/338: chown d1/d3/d16/d3e/f4e 129762066 1 2024-05-17T21:52:53.626 INFO:tasks.workunit.client.0.smithi107.stdout:0/519: getdents d3/dc/d22 0 2024-05-17T21:52:53.627 INFO:tasks.workunit.client.0.smithi107.stdout:0/520: stat d3/dc/f6d 0 2024-05-17T21:52:53.628 INFO:tasks.workunit.client.0.smithi107.stdout:1/478: unlink d4/de/f67 0 2024-05-17T21:52:53.628 INFO:tasks.workunit.client.0.smithi107.stdout:5/445: sync 2024-05-17T21:52:53.628 INFO:tasks.workunit.client.0.smithi107.stdout:5/446: readlink l1 0 2024-05-17T21:52:53.629 INFO:tasks.workunit.client.0.smithi107.stdout:1/479: write d4/df/d1a/d26/f60 [233197,18492] 0 2024-05-17T21:52:53.631 INFO:tasks.workunit.client.0.smithi107.stdout:2/328: truncate d0/d9/d11/d15/f4e 966097 0 2024-05-17T21:52:53.632 INFO:tasks.workunit.client.1.smithi138.stdout:7/214: write d0/f19 [1017558,2163] 0 2024-05-17T21:52:53.632 INFO:tasks.workunit.client.0.smithi107.stdout:0/521: mkdir d3/dc/dae/dc0 0 2024-05-17T21:52:53.633 INFO:tasks.workunit.client.1.smithi138.stdout:7/215: truncate d0/f2b 5086807 0 2024-05-17T21:52:53.641 INFO:tasks.workunit.client.0.smithi107.stdout:2/329: readlink d0/d9/d11/d18/l5e 0 2024-05-17T21:52:53.645 INFO:tasks.workunit.client.0.smithi107.stdout:3/456: dread d6/f1c [0,4194304] 0 2024-05-17T21:52:53.645 INFO:tasks.workunit.client.1.smithi138.stdout:7/216: dread - d0/f2c zero size 2024-05-17T21:52:53.649 INFO:tasks.workunit.client.1.smithi138.stdout:0/345: write d3/fd [322494,2212] 0 2024-05-17T21:52:53.651 INFO:tasks.workunit.client.1.smithi138.stdout:0/346: write d3/d10/f1a [1186021,41442] 0 2024-05-17T21:52:53.651 INFO:tasks.workunit.client.0.smithi107.stdout:2/330: write d0/d9/d11/d15/f57 [2258175,31346] 0 2024-05-17T21:52:53.652 INFO:tasks.workunit.client.1.smithi138.stdout:7/217: dwrite d0/f1e [0,4194304] 0 2024-05-17T21:52:53.657 INFO:tasks.workunit.client.0.smithi107.stdout:5/447: rename d5/db/d16/d2d/d32/l66 to d5/db/l8c 0 2024-05-17T21:52:53.662 INFO:tasks.workunit.client.1.smithi138.stdout:8/169: dwrite d0/dc/f22 [4194304,4194304] 0 2024-05-17T21:52:53.663 INFO:tasks.workunit.client.1.smithi138.stdout:3/339: link d1/d3/d8/de/f3f d1/d4f/f77 0 2024-05-17T21:52:53.664 INFO:tasks.workunit.client.1.smithi138.stdout:3/340: fdatasync d1/f65 0 2024-05-17T21:52:53.666 INFO:tasks.workunit.client.0.smithi107.stdout:3/457: creat d6/d8/d10/d25/f95 x:0 0 0 2024-05-17T21:52:53.666 INFO:tasks.workunit.client.0.smithi107.stdout:1/480: rmdir d4/df/d14/d2e/d43/d47/d9a 0 2024-05-17T21:52:53.670 INFO:tasks.workunit.client.1.smithi138.stdout:4/307: getdents d0/d2f 0 2024-05-17T21:52:53.671 INFO:tasks.workunit.client.0.smithi107.stdout:0/522: write d3/f6e [796177,121924] 0 2024-05-17T21:52:53.672 INFO:tasks.workunit.client.0.smithi107.stdout:1/481: dread d4/df/d14/d33/f40 [0,4194304] 0 2024-05-17T21:52:53.672 INFO:tasks.workunit.client.1.smithi138.stdout:0/347: symlink d3/l79 0 2024-05-17T21:52:53.673 INFO:tasks.workunit.client.0.smithi107.stdout:3/458: dwrite d6/d8/d10/d35/d36/f8e [0,4194304] 0 2024-05-17T21:52:53.674 INFO:tasks.workunit.client.1.smithi138.stdout:3/341: dwrite d1/d3/f25 [0,4194304] 0 2024-05-17T21:52:53.676 INFO:tasks.workunit.client.1.smithi138.stdout:9/266: sync 2024-05-17T21:52:53.676 INFO:tasks.workunit.client.1.smithi138.stdout:2/292: sync 2024-05-17T21:52:53.677 INFO:tasks.workunit.client.1.smithi138.stdout:7/218: dwrite d0/f31 [0,4194304] 0 2024-05-17T21:52:53.677 INFO:tasks.workunit.client.1.smithi138.stdout:3/342: write d1/d3/f1c [1403236,55657] 0 2024-05-17T21:52:53.694 INFO:tasks.workunit.client.1.smithi138.stdout:3/343: creat d1/d3/d16/d3a/f78 x:0 0 0 2024-05-17T21:52:53.695 INFO:tasks.workunit.client.1.smithi138.stdout:3/344: fdatasync d1/d3/d16/d3e/f43 0 2024-05-17T21:52:53.698 INFO:tasks.workunit.client.1.smithi138.stdout:8/170: unlink d0/d11/l26 0 2024-05-17T21:52:53.698 INFO:tasks.workunit.client.1.smithi138.stdout:3/345: readlink d1/d3/d8/l44 0 2024-05-17T21:52:53.699 INFO:tasks.workunit.client.1.smithi138.stdout:8/171: write d0/d11/f28 [529028,50131] 0 2024-05-17T21:52:53.701 INFO:tasks.workunit.client.0.smithi107.stdout:1/482: write d4/df/d1a/d26/d50/f95 [1464769,30988] 0 2024-05-17T21:52:53.702 INFO:tasks.workunit.client.0.smithi107.stdout:1/483: chown d4/f21 31 1 2024-05-17T21:52:53.703 INFO:tasks.workunit.client.1.smithi138.stdout:2/293: getdents d3/d21/d2f/d3e/d55 0 2024-05-17T21:52:53.705 INFO:tasks.workunit.client.1.smithi138.stdout:2/294: fsync d3/d21/d2f/d3e/f5a 0 2024-05-17T21:52:53.707 INFO:tasks.workunit.client.1.smithi138.stdout:7/219: dwrite d0/fa [0,4194304] 0 2024-05-17T21:52:53.707 INFO:tasks.workunit.client.1.smithi138.stdout:2/295: dread d3/f29 [0,4194304] 0 2024-05-17T21:52:53.709 INFO:tasks.workunit.client.1.smithi138.stdout:2/296: chown d3/l24 188423 1 2024-05-17T21:52:53.709 INFO:tasks.workunit.client.0.smithi107.stdout:5/448: getdents d5/db/d16/d54 0 2024-05-17T21:52:53.710 INFO:tasks.workunit.client.1.smithi138.stdout:4/308: write d0/d1b/d42/f4f [3295551,27263] 0 2024-05-17T21:52:53.712 INFO:tasks.workunit.client.0.smithi107.stdout:0/523: getdents d3/dc 0 2024-05-17T21:52:53.713 INFO:tasks.workunit.client.1.smithi138.stdout:8/172: rename d0/dc/ce to d0/c32 0 2024-05-17T21:52:53.714 INFO:tasks.workunit.client.1.smithi138.stdout:8/173: truncate d0/fb 9291722 0 2024-05-17T21:52:53.716 INFO:tasks.workunit.client.1.smithi138.stdout:9/267: dwrite d6/f33 [0,4194304] 0 2024-05-17T21:52:53.717 INFO:tasks.workunit.client.0.smithi107.stdout:5/449: mknod d5/db/d16/d54/d59/d65/d48/d89/c8d 0 2024-05-17T21:52:53.718 INFO:tasks.workunit.client.0.smithi107.stdout:5/450: dread - d5/f84 zero size 2024-05-17T21:52:53.718 INFO:tasks.workunit.client.1.smithi138.stdout:5/278: sync 2024-05-17T21:52:53.718 INFO:tasks.workunit.client.1.smithi138.stdout:6/322: sync 2024-05-17T21:52:53.718 INFO:tasks.workunit.client.1.smithi138.stdout:1/235: sync 2024-05-17T21:52:53.719 INFO:tasks.workunit.client.1.smithi138.stdout:3/346: fdatasync d1/d3/d16/d3e/f4a 0 2024-05-17T21:52:53.719 INFO:tasks.workunit.client.0.smithi107.stdout:5/451: stat d5/db/dc/d1e/c79 0 2024-05-17T21:52:53.720 INFO:tasks.workunit.client.0.smithi107.stdout:0/524: dwrite d3/dc/d22/d23/f44 [0,4194304] 0 2024-05-17T21:52:53.722 INFO:tasks.workunit.client.0.smithi107.stdout:5/452: dread - d5/db/d16/d54/d59/d65/d48/f8a zero size 2024-05-17T21:52:53.722 INFO:tasks.workunit.client.1.smithi138.stdout:2/297: rmdir d3/d21/d2f/d3e 39 2024-05-17T21:52:53.723 INFO:tasks.workunit.client.0.smithi107.stdout:0/525: truncate d3/d58/faa 4957537 0 2024-05-17T21:52:53.724 INFO:tasks.workunit.client.0.smithi107.stdout:0/526: dread - d3/dc/d22/d23/f8f zero size 2024-05-17T21:52:53.724 INFO:tasks.workunit.client.1.smithi138.stdout:4/309: mknod d0/d1b/c6e 0 2024-05-17T21:52:53.725 INFO:tasks.workunit.client.0.smithi107.stdout:3/459: write d6/d2a/f42 [3158050,93583] 0 2024-05-17T21:52:53.727 INFO:tasks.workunit.client.0.smithi107.stdout:5/453: dread d5/db/d39/f3e [0,4194304] 0 2024-05-17T21:52:53.729 INFO:tasks.workunit.client.0.smithi107.stdout:5/454: write d5/db/d16/d54/d59/d65/f6f [922724,75106] 0 2024-05-17T21:52:53.729 INFO:tasks.workunit.client.1.smithi138.stdout:0/348: dread d3/d10/f15 [0,4194304] 0 2024-05-17T21:52:53.732 INFO:tasks.workunit.client.1.smithi138.stdout:0/349: truncate d3/d10/d39/f72 421069 0 2024-05-17T21:52:53.733 INFO:tasks.workunit.client.0.smithi107.stdout:0/527: mkdir d3/d75/dc1 0 2024-05-17T21:52:53.734 INFO:tasks.workunit.client.1.smithi138.stdout:1/236: creat d4/dd/d3b/f53 x:0 0 0 2024-05-17T21:52:53.734 INFO:tasks.workunit.client.1.smithi138.stdout:1/237: readlink d4/df/d35/l46 0 2024-05-17T21:52:53.737 INFO:tasks.workunit.client.1.smithi138.stdout:6/323: creat d7/db/d43/d3c/f6c x:0 0 0 2024-05-17T21:52:53.738 INFO:tasks.workunit.client.0.smithi107.stdout:5/455: truncate d5/db/d16/f21 4496349 0 2024-05-17T21:52:53.738 INFO:tasks.workunit.client.0.smithi107.stdout:1/484: dread d4/df/d14/d33/f9d [4194304,4194304] 0 2024-05-17T21:52:53.742 INFO:tasks.workunit.client.0.smithi107.stdout:1/485: creat d4/df/d14/d33/fa1 x:0 0 0 2024-05-17T21:52:53.743 INFO:tasks.workunit.client.1.smithi138.stdout:4/310: rmdir d0/d2f/d49/d60 39 2024-05-17T21:52:53.744 INFO:tasks.workunit.client.1.smithi138.stdout:8/174: truncate d0/ff 3105199 0 2024-05-17T21:52:53.745 INFO:tasks.workunit.client.1.smithi138.stdout:8/175: read d0/fb [5890791,121079] 0 2024-05-17T21:52:53.746 INFO:tasks.workunit.client.1.smithi138.stdout:9/268: write d6/d7/d16/d29/f3a [2642662,22139] 0 2024-05-17T21:52:53.747 INFO:tasks.workunit.client.0.smithi107.stdout:1/486: mknod d4/df/d4e/ca2 0 2024-05-17T21:52:53.748 INFO:tasks.workunit.client.0.smithi107.stdout:5/456: mkdir d5/db/d16/d2d/d40/d5a/d7c/d8e 0 2024-05-17T21:52:53.749 INFO:tasks.workunit.client.0.smithi107.stdout:1/487: chown d4/df/d1a/d26/d72/d85/f92 18584 1 2024-05-17T21:52:53.749 INFO:tasks.workunit.client.0.smithi107.stdout:3/460: dwrite d6/d1a/d57/f73 [0,4194304] 0 2024-05-17T21:52:53.751 INFO:tasks.workunit.client.1.smithi138.stdout:7/220: dwrite d0/f11 [0,4194304] 0 2024-05-17T21:52:53.752 INFO:tasks.workunit.client.0.smithi107.stdout:0/528: write d3/dc/d22/f68 [309739,53797] 0 2024-05-17T21:52:53.753 INFO:tasks.workunit.client.0.smithi107.stdout:5/457: mkdir d5/db/dc/d8f 0 2024-05-17T21:52:53.754 INFO:tasks.workunit.client.1.smithi138.stdout:7/221: read d0/f31 [4460348,75454] 0 2024-05-17T21:52:53.754 INFO:tasks.workunit.client.1.smithi138.stdout:7/222: read d0/f2b [25178,53588] 0 2024-05-17T21:52:53.754 INFO:tasks.workunit.client.1.smithi138.stdout:7/223: rename d0 to d0/d18/d38 22 2024-05-17T21:52:53.755 INFO:tasks.workunit.client.0.smithi107.stdout:1/488: creat d4/df/d14/d2e/d43/d47/fa3 x:0 0 0 2024-05-17T21:52:53.756 INFO:tasks.workunit.client.0.smithi107.stdout:1/489: truncate d4/df/d14/d1d/f2f 198977 0 2024-05-17T21:52:53.759 INFO:tasks.workunit.client.1.smithi138.stdout:0/350: dread d3/d10/f14 [0,4194304] 0 2024-05-17T21:52:53.761 INFO:tasks.workunit.client.1.smithi138.stdout:6/324: creat d7/db/d43/d3c/f6d x:0 0 0 2024-05-17T21:52:53.763 INFO:tasks.workunit.client.0.smithi107.stdout:1/490: dwrite d4/fa [0,4194304] 0 2024-05-17T21:52:53.764 INFO:tasks.workunit.client.1.smithi138.stdout:5/279: truncate d1/f43 3706513 0 2024-05-17T21:52:53.765 INFO:tasks.workunit.client.0.smithi107.stdout:1/491: dread - d4/df/d1a/d26/d72/f83 zero size 2024-05-17T21:52:53.766 INFO:tasks.workunit.client.1.smithi138.stdout:1/238: dwrite d4/dd/f2d [0,4194304] 0 2024-05-17T21:52:53.767 INFO:tasks.workunit.client.1.smithi138.stdout:5/280: write d1/d6/d19/f3a [1189980,128658] 0 2024-05-17T21:52:53.768 INFO:tasks.workunit.client.1.smithi138.stdout:3/347: truncate d1/d3/d16/f29 2240836 0 2024-05-17T21:52:53.768 INFO:tasks.workunit.client.0.smithi107.stdout:5/458: dread d5/db/d16/d2a/f64 [0,4194304] 0 2024-05-17T21:52:53.771 INFO:tasks.workunit.client.1.smithi138.stdout:2/298: mknod d3/d21/d2f/d3e/d58/c5e 0 2024-05-17T21:52:53.772 INFO:tasks.workunit.client.1.smithi138.stdout:2/299: fsync d3/d21/d2f/f3f 0 2024-05-17T21:52:53.775 INFO:tasks.workunit.client.1.smithi138.stdout:4/311: creat d0/d2f/f6f x:0 0 0 2024-05-17T21:52:53.775 INFO:tasks.workunit.client.1.smithi138.stdout:8/176: mkdir d0/dc/d1a/d33 0 2024-05-17T21:52:53.776 INFO:tasks.workunit.client.0.smithi107.stdout:0/529: link d3/dc/l7a d3/dc/d31/lc2 0 2024-05-17T21:52:53.777 INFO:tasks.workunit.client.0.smithi107.stdout:0/530: chown d3/f7b 4403 1 2024-05-17T21:52:53.777 INFO:tasks.workunit.client.1.smithi138.stdout:8/177: read - d0/dc/d1a/f2a zero size 2024-05-17T21:52:53.779 INFO:tasks.workunit.client.0.smithi107.stdout:1/492: symlink d4/df/d14/la4 0 2024-05-17T21:52:53.779 INFO:tasks.workunit.client.1.smithi138.stdout:1/239: dwrite d4/df/d35/f37 [0,4194304] 0 2024-05-17T21:52:53.780 INFO:tasks.workunit.client.0.smithi107.stdout:1/493: write d4/df/d1a/d26/d7a/f93 [370725,5558] 0 2024-05-17T21:52:53.780 INFO:tasks.workunit.client.1.smithi138.stdout:8/178: write d0/d11/f1c [34955,91582] 0 2024-05-17T21:52:53.781 INFO:tasks.workunit.client.0.smithi107.stdout:0/531: creat d3/dc/fc3 x:0 0 0 2024-05-17T21:52:53.781 INFO:tasks.workunit.client.0.smithi107.stdout:3/461: write d6/d2a/d2d/d48/f93 [7343640,95273] 0 2024-05-17T21:52:53.781 INFO:tasks.workunit.client.0.smithi107.stdout:1/494: chown d4/df/d4e/l96 154578713 1 2024-05-17T21:52:53.782 INFO:tasks.workunit.client.1.smithi138.stdout:1/240: chown d4/dd/d3b/f53 1684482 1 2024-05-17T21:52:53.790 INFO:tasks.workunit.client.0.smithi107.stdout:3/462: rename d6/d1a/f1f to d6/d1a/f96 0 2024-05-17T21:52:53.790 INFO:tasks.workunit.client.0.smithi107.stdout:9/445: sync 2024-05-17T21:52:53.791 INFO:tasks.workunit.client.1.smithi138.stdout:7/224: unlink d0/d18/c25 0 2024-05-17T21:52:53.792 INFO:tasks.workunit.client.0.smithi107.stdout:1/495: creat d4/df/d1a/d26/d7a/fa5 x:0 0 0 2024-05-17T21:52:53.792 INFO:tasks.workunit.client.1.smithi138.stdout:4/312: read d0/d3/d17/f1a [4099751,117250] 0 2024-05-17T21:52:53.793 INFO:tasks.workunit.client.0.smithi107.stdout:3/463: creat d6/d8/d10/d35/d36/d4c/f97 x:0 0 0 2024-05-17T21:52:53.795 INFO:tasks.workunit.client.0.smithi107.stdout:3/464: truncate d6/d8/d10/f4b 1058831 0 2024-05-17T21:52:53.799 INFO:tasks.workunit.client.0.smithi107.stdout:9/446: mkdir d0/dd/d21/d29/d3e/d94/d9a 0 2024-05-17T21:52:53.800 INFO:tasks.workunit.client.1.smithi138.stdout:3/348: dread d1/d3/d8/f34 [0,4194304] 0 2024-05-17T21:52:53.801 INFO:tasks.workunit.client.1.smithi138.stdout:5/281: mkdir d1/d1a/d39/d4e/d58 0 2024-05-17T21:52:53.801 INFO:tasks.workunit.client.1.smithi138.stdout:9/269: write d6/d7/f30 [4601491,122644] 0 2024-05-17T21:52:53.801 INFO:tasks.workunit.client.1.smithi138.stdout:3/349: dread - d1/d3/d49/d4b/f6e zero size 2024-05-17T21:52:53.802 INFO:tasks.workunit.client.0.smithi107.stdout:3/465: symlink d6/d2a/d2d/d69/l98 0 2024-05-17T21:52:53.802 INFO:tasks.workunit.client.0.smithi107.stdout:1/496: dwrite d4/df/d14/d33/f9d [4194304,4194304] 0 2024-05-17T21:52:53.805 INFO:tasks.workunit.client.0.smithi107.stdout:3/466: chown d6/d1a/l24 172436580 1 2024-05-17T21:52:53.807 INFO:tasks.workunit.client.0.smithi107.stdout:3/467: write d6/d2a/f5b [3961246,119136] 0 2024-05-17T21:52:53.809 INFO:tasks.workunit.client.1.smithi138.stdout:9/270: dwrite d6/d7/f17 [0,4194304] 0 2024-05-17T21:52:53.809 INFO:tasks.workunit.client.0.smithi107.stdout:9/447: dwrite d0/dd/d1f/f82 [0,4194304] 0 2024-05-17T21:52:53.809 INFO:tasks.workunit.client.0.smithi107.stdout:5/459: dread d5/db/d16/d2d/f4f [0,4194304] 0 2024-05-17T21:52:53.810 INFO:tasks.workunit.client.0.smithi107.stdout:5/460: fdatasync d5/db/f5c 0 2024-05-17T21:52:53.811 INFO:tasks.workunit.client.0.smithi107.stdout:3/468: dread - d6/d8/d10/d35/d36/d4c/d83/f92 zero size 2024-05-17T21:52:53.812 INFO:tasks.workunit.client.0.smithi107.stdout:5/461: fdatasync d5/db/d16/d5b/f85 0 2024-05-17T21:52:53.812 INFO:tasks.workunit.client.0.smithi107.stdout:0/532: chown d3/dc/d22/f68 26281821 1 2024-05-17T21:52:53.814 INFO:tasks.workunit.client.0.smithi107.stdout:1/497: fdatasync d4/df/d14/d2e/f9e 0 2024-05-17T21:52:53.814 INFO:tasks.workunit.client.0.smithi107.stdout:9/448: dread - d0/dd/d21/f58 zero size 2024-05-17T21:52:53.821 INFO:tasks.workunit.client.1.smithi138.stdout:1/241: creat d4/d2e/f54 x:0 0 0 2024-05-17T21:52:53.823 INFO:tasks.workunit.client.1.smithi138.stdout:1/242: write d4/fc [4984945,89641] 0 2024-05-17T21:52:53.825 INFO:tasks.workunit.client.0.smithi107.stdout:1/498: chown d4/df/d4e/l5d 49 1 2024-05-17T21:52:53.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:53 smithi138 ceph-mon[35948]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-05-17T21:52:53.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:53 smithi138 ceph-mon[35948]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-05-17T21:52:53.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:53 smithi138 ceph-mon[35948]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-17T21:52:53.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:53 smithi138 ceph-mon[35948]: Upgrade: Need to upgrade myself (mgr.smithi138.napnll) 2024-05-17T21:52:53.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:53 smithi138 ceph-mon[35948]: Upgrade: Need to upgrade myself (mgr.smithi138.napnll) 2024-05-17T21:52:53.828 INFO:tasks.workunit.client.0.smithi107.stdout:9/449: fsync d0/fc 0 2024-05-17T21:52:53.829 INFO:tasks.workunit.client.0.smithi107.stdout:9/450: dread - d0/dd/d21/d29/d3e/f91 zero size 2024-05-17T21:52:53.830 INFO:tasks.workunit.client.0.smithi107.stdout:9/451: dread - d0/dd/d21/d29/d3e/d45/d68/f7a zero size 2024-05-17T21:52:53.831 INFO:tasks.workunit.client.0.smithi107.stdout:0/533: creat d3/d75/dc1/fc4 x:0 0 0 2024-05-17T21:52:53.831 INFO:tasks.workunit.client.1.smithi138.stdout:4/313: fsync d0/d2f/d49/d60/d1f/d32/f64 0 2024-05-17T21:52:53.831 INFO:tasks.workunit.client.0.smithi107.stdout:9/452: write d0/dd/d1f/f40 [2272807,59992] 0 2024-05-17T21:52:53.832 INFO:tasks.workunit.client.1.smithi138.stdout:1/243: dwrite d4/db/f4d [0,4194304] 0 2024-05-17T21:52:53.834 INFO:tasks.workunit.client.0.smithi107.stdout:5/462: creat d5/db/d16/d2d/d40/d5a/d7c/d8e/f90 x:0 0 0 2024-05-17T21:52:53.834 INFO:tasks.workunit.client.1.smithi138.stdout:6/325: mknod d7/c6e 0 2024-05-17T21:52:53.835 INFO:tasks.workunit.client.1.smithi138.stdout:6/326: readlink d7/db/d12/l19 0 2024-05-17T21:52:53.840 INFO:tasks.workunit.client.1.smithi138.stdout:5/282: chown d1/lf 13 1 2024-05-17T21:52:53.841 INFO:tasks.workunit.client.1.smithi138.stdout:6/327: dread d7/f37 [0,4194304] 0 2024-05-17T21:52:53.842 INFO:tasks.workunit.client.1.smithi138.stdout:6/328: dread - d7/db/d43/d3c/f6c zero size 2024-05-17T21:52:53.842 INFO:tasks.workunit.client.0.smithi107.stdout:0/534: readlink d3/d24/d6f/d9f/dab/l80 0 2024-05-17T21:52:53.844 INFO:tasks.workunit.client.0.smithi107.stdout:9/453: mkdir d0/dd/d2f/d87/d89/d9b 0 2024-05-17T21:52:53.846 INFO:tasks.workunit.client.0.smithi107.stdout:5/463: fdatasync d5/db/d16/d5b/f2f 0 2024-05-17T21:52:53.849 INFO:tasks.workunit.client.0.smithi107.stdout:1/499: symlink d4/df/d14/la6 0 2024-05-17T21:52:53.849 INFO:tasks.workunit.client.0.smithi107.stdout:1/500: write d4/df/f4d [1369929,69909] 0 2024-05-17T21:52:53.850 INFO:tasks.workunit.client.0.smithi107.stdout:0/535: creat d3/dc/d22/fc5 x:0 0 0 2024-05-17T21:52:53.851 INFO:tasks.workunit.client.0.smithi107.stdout:1/501: write d4/df/d1a/f58 [1006146,14407] 0 2024-05-17T21:52:53.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:53 smithi107 ceph-mon[28434]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-05-17T21:52:53.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:53 smithi107 ceph-mon[28434]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-05-17T21:52:53.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:53 smithi107 ceph-mon[28434]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-17T21:52:53.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:53 smithi107 ceph-mon[28434]: Upgrade: Need to upgrade myself (mgr.smithi138.napnll) 2024-05-17T21:52:53.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:53 smithi107 ceph-mon[28434]: Upgrade: Need to upgrade myself (mgr.smithi138.napnll) 2024-05-17T21:52:53.855 INFO:tasks.workunit.client.0.smithi107.stdout:5/464: write d5/db/dc/f1f [2386867,22117] 0 2024-05-17T21:52:53.858 INFO:tasks.workunit.client.0.smithi107.stdout:3/469: dread d6/d2a/f52 [0,4194304] 0 2024-05-17T21:52:53.859 INFO:tasks.workunit.client.0.smithi107.stdout:0/536: mknod d3/d75/d9e/cc6 0 2024-05-17T21:52:53.860 INFO:tasks.workunit.client.1.smithi138.stdout:7/225: mkdir d0/d39 0 2024-05-17T21:52:53.862 INFO:tasks.workunit.client.0.smithi107.stdout:5/465: fdatasync d5/db/d16/d2d/d4d/d6d/f77 0 2024-05-17T21:52:53.865 INFO:tasks.workunit.client.1.smithi138.stdout:4/314: creat d0/d2f/d49/d60/d1f/f70 x:0 0 0 2024-05-17T21:52:53.865 INFO:tasks.workunit.client.0.smithi107.stdout:1/502: mkdir d4/da7 0 2024-05-17T21:52:53.866 INFO:tasks.workunit.client.1.smithi138.stdout:4/315: chown d0/d2f/d49/d60/f25 96950 1 2024-05-17T21:52:53.866 INFO:tasks.workunit.client.1.smithi138.stdout:2/300: dwrite f2 [0,4194304] 0 2024-05-17T21:52:53.868 INFO:tasks.workunit.client.1.smithi138.stdout:2/301: write d3/f35 [1880014,48740] 0 2024-05-17T21:52:53.868 INFO:tasks.workunit.client.0.smithi107.stdout:5/466: creat d5/db/d16/d2d/d32/f91 x:0 0 0 2024-05-17T21:52:53.869 INFO:tasks.workunit.client.0.smithi107.stdout:5/467: fdatasync d5/db/d16/d2d/d32/f91 0 2024-05-17T21:52:53.870 INFO:tasks.workunit.client.0.smithi107.stdout:9/454: truncate d0/dd/d2f/d87/d89/f96 351335 0 2024-05-17T21:52:53.871 INFO:tasks.workunit.client.0.smithi107.stdout:1/503: dread d4/fa [4194304,4194304] 0 2024-05-17T21:52:53.873 INFO:tasks.workunit.client.0.smithi107.stdout:1/504: truncate d4/df/d1a/d26/d7a/f93 1181517 0 2024-05-17T21:52:53.873 INFO:tasks.workunit.client.0.smithi107.stdout:1/505: fsync d4/f38 0 2024-05-17T21:52:53.875 INFO:tasks.workunit.client.0.smithi107.stdout:1/506: write d4/df/d14/d1d/f2f [745789,46778] 0 2024-05-17T21:52:53.876 INFO:tasks.workunit.client.0.smithi107.stdout:9/455: mknod d0/dd/d1f/c9c 0 2024-05-17T21:52:53.877 INFO:tasks.workunit.client.1.smithi138.stdout:6/329: symlink d7/d8/l6f 0 2024-05-17T21:52:53.878 INFO:tasks.workunit.client.0.smithi107.stdout:9/456: truncate d0/dd/d21/d29/f5a 430870 0 2024-05-17T21:52:53.878 INFO:tasks.workunit.client.0.smithi107.stdout:5/468: mkdir d5/db/d16/d54/d59/d92 0 2024-05-17T21:52:53.878 INFO:tasks.workunit.client.1.smithi138.stdout:3/350: symlink d1/d3/d16/d3e/d71/l79 0 2024-05-17T21:52:53.879 INFO:tasks.workunit.client.0.smithi107.stdout:5/469: chown d5/db/dc/d1e/c79 4672 1 2024-05-17T21:52:53.880 INFO:tasks.workunit.client.1.smithi138.stdout:9/271: symlink d6/d7/d16/d29/d48/d3b/l54 0 2024-05-17T21:52:53.881 INFO:tasks.workunit.client.0.smithi107.stdout:1/507: dwrite d4/df/d1a/d26/f60 [0,4194304] 0 2024-05-17T21:52:53.882 INFO:tasks.workunit.client.1.smithi138.stdout:8/179: rmdir d0/d11/d2d 0 2024-05-17T21:52:53.886 INFO:tasks.workunit.client.1.smithi138.stdout:3/351: dwrite d1/d3/d16/d3a/f78 [0,4194304] 0 2024-05-17T21:52:53.886 INFO:tasks.workunit.client.0.smithi107.stdout:9/457: mkdir d0/dd/d3d/d9d 0 2024-05-17T21:52:53.888 INFO:tasks.workunit.client.1.smithi138.stdout:1/244: creat d4/df/d4e/f55 x:0 0 0 2024-05-17T21:52:53.891 INFO:tasks.workunit.client.1.smithi138.stdout:4/316: mkdir d0/d2f/d49/d60/d1f/d32/d71 0 2024-05-17T21:52:53.892 INFO:tasks.workunit.client.0.smithi107.stdout:0/537: mkdir d3/dc/d22/d3d/dc7 0 2024-05-17T21:52:53.899 INFO:tasks.workunit.client.1.smithi138.stdout:6/330: creat d7/d8/f70 x:0 0 0 2024-05-17T21:52:53.900 INFO:tasks.workunit.client.1.smithi138.stdout:9/272: mknod d6/d7/d16/d29/d38/c55 0 2024-05-17T21:52:53.902 INFO:tasks.workunit.client.0.smithi107.stdout:9/458: creat d0/dd/d69/f9e x:0 0 0 2024-05-17T21:52:53.906 INFO:tasks.workunit.client.1.smithi138.stdout:7/226: unlink d0/f5 0 2024-05-17T21:52:53.907 INFO:tasks.workunit.client.1.smithi138.stdout:7/227: write d0/f19 [770057,80133] 0 2024-05-17T21:52:53.907 INFO:tasks.workunit.client.0.smithi107.stdout:0/538: symlink d3/dc/d31/lc8 0 2024-05-17T21:52:53.908 INFO:tasks.workunit.client.1.smithi138.stdout:3/352: symlink d1/d3/d16/d3e/d71/l7a 0 2024-05-17T21:52:53.909 INFO:tasks.workunit.client.1.smithi138.stdout:3/353: dread - d1/d4f/d50/f6b zero size 2024-05-17T21:52:53.909 INFO:tasks.workunit.client.0.smithi107.stdout:1/508: creat d4/df/fa8 x:0 0 0 2024-05-17T21:52:53.910 INFO:tasks.workunit.client.0.smithi107.stdout:1/509: truncate d4/df/fa8 626725 0 2024-05-17T21:52:53.910 INFO:tasks.workunit.client.1.smithi138.stdout:1/245: mknod d4/dd/c56 0 2024-05-17T21:52:53.911 INFO:tasks.workunit.client.0.smithi107.stdout:1/510: readlink d4/df/d14/la4 0 2024-05-17T21:52:53.911 INFO:tasks.workunit.client.0.smithi107.stdout:1/511: readlink d4/de/l4a 0 2024-05-17T21:52:53.911 INFO:tasks.workunit.client.0.smithi107.stdout:9/459: mknod d0/dd/d2f/d4a/c9f 0 2024-05-17T21:52:53.912 INFO:tasks.workunit.client.1.smithi138.stdout:4/317: symlink d0/d3/d1d/d38/l72 0 2024-05-17T21:52:53.913 INFO:tasks.workunit.client.1.smithi138.stdout:4/318: write d0/d3/d17/f63 [146719,126850] 0 2024-05-17T21:52:53.914 INFO:tasks.workunit.client.1.smithi138.stdout:2/302: unlink d3/d21/l32 0 2024-05-17T21:52:53.916 INFO:tasks.workunit.client.1.smithi138.stdout:5/283: rename d1/d6/l48 to d1/d6/d1b/d29/l59 0 2024-05-17T21:52:53.919 INFO:tasks.workunit.client.1.smithi138.stdout:8/180: write d0/fb [3231690,20976] 0 2024-05-17T21:52:53.920 INFO:tasks.workunit.client.0.smithi107.stdout:0/539: symlink d3/dc/d22/d23/d97/d77/lc9 0 2024-05-17T21:52:53.922 INFO:tasks.workunit.client.0.smithi107.stdout:5/470: dwrite d5/f1b [0,4194304] 0 2024-05-17T21:52:53.924 INFO:tasks.workunit.client.1.smithi138.stdout:7/228: stat d0/cb 0 2024-05-17T21:52:53.925 INFO:tasks.workunit.client.0.smithi107.stdout:1/512: dwrite d4/df/d14/d1d/f27 [0,4194304] 0 2024-05-17T21:52:53.925 INFO:tasks.workunit.client.1.smithi138.stdout:8/181: dwrite d0/dc/f2b [0,4194304] 0 2024-05-17T21:52:53.926 INFO:tasks.workunit.client.0.smithi107.stdout:0/540: chown d3/d58/d61/d94 976 1 2024-05-17T21:52:53.926 INFO:tasks.workunit.client.0.smithi107.stdout:5/471: write d5/db/d16/d2d/d4d/d6d/f77 [874385,76262] 0 2024-05-17T21:52:53.927 INFO:tasks.workunit.client.1.smithi138.stdout:3/354: creat d1/d3/d16/d3e/d71/f7b x:0 0 0 2024-05-17T21:52:53.929 INFO:tasks.workunit.client.1.smithi138.stdout:1/246: symlink d4/d2e/d36/l57 0 2024-05-17T21:52:53.933 INFO:tasks.workunit.client.0.smithi107.stdout:9/460: getdents d0/dd/d21/d29 0 2024-05-17T21:52:53.953 INFO:tasks.workunit.client.0.smithi107.stdout:6/373: sync 2024-05-17T21:52:53.977 INFO:tasks.workunit.client.0.smithi107.stdout:1/513: truncate d4/fa 4978683 0 2024-05-17T21:52:53.977 INFO:tasks.workunit.client.0.smithi107.stdout:1/514: chown d4 252911 1 2024-05-17T21:52:53.981 INFO:tasks.workunit.client.0.smithi107.stdout:9/461: rename d0/dd/d21/d29/d3e/d45/f59 to d0/dd/d2f/d87/d89/d9b/fa0 0 2024-05-17T21:52:53.982 INFO:tasks.workunit.client.1.smithi138.stdout:5/284: symlink d1/d53/l5a 0 2024-05-17T21:52:53.982 INFO:tasks.workunit.client.0.smithi107.stdout:9/462: write d0/dd/d3d/f92 [2802106,4201] 0 2024-05-17T21:52:53.983 INFO:tasks.workunit.client.1.smithi138.stdout:5/285: dread - d1/d6/d1b/f54 zero size 2024-05-17T21:52:53.985 INFO:tasks.workunit.client.1.smithi138.stdout:6/331: creat d7/d69/f71 x:0 0 0 2024-05-17T21:52:53.988 INFO:tasks.workunit.client.0.smithi107.stdout:1/515: stat d4/df/d14/d2e/d43/l8a 0 2024-05-17T21:52:53.989 INFO:tasks.workunit.client.1.smithi138.stdout:5/286: dwrite d1/d2c/f52 [0,4194304] 0 2024-05-17T21:52:53.989 INFO:tasks.workunit.client.0.smithi107.stdout:1/516: truncate d4/de/f36 6073207 0 2024-05-17T21:52:53.994 INFO:tasks.workunit.client.1.smithi138.stdout:7/229: symlink d0/d18/l3a 0 2024-05-17T21:52:53.997 INFO:tasks.workunit.client.0.smithi107.stdout:9/463: creat d0/dd/d21/d29/d3e/d45/d68/fa1 x:0 0 0 2024-05-17T21:52:53.998 INFO:tasks.workunit.client.0.smithi107.stdout:6/374: dread d9/f1d [4194304,4194304] 0 2024-05-17T21:52:54.000 INFO:tasks.workunit.client.0.smithi107.stdout:1/517: symlink d4/df/d1a/d26/la9 0 2024-05-17T21:52:54.001 INFO:tasks.workunit.client.0.smithi107.stdout:1/518: write d4/df/d14/d33/fa1 [993127,16078] 0 2024-05-17T21:52:54.002 INFO:tasks.workunit.client.1.smithi138.stdout:1/247: rename d4/db/l1d to d4/dd/d3b/d40/l58 0 2024-05-17T21:52:54.004 INFO:tasks.workunit.client.1.smithi138.stdout:4/319: getdents d0/d2f/d49/d60/d1f/d32/d71 0 2024-05-17T21:52:54.006 INFO:tasks.workunit.client.0.smithi107.stdout:9/464: symlink d0/dd/d2f/d87/d89/d9b/la2 0 2024-05-17T21:52:54.013 INFO:tasks.workunit.client.0.smithi107.stdout:9/465: dwrite d0/dd/d21/d29/f90 [0,4194304] 0 2024-05-17T21:52:54.014 INFO:tasks.workunit.client.1.smithi138.stdout:8/182: creat d0/dc/d1a/d33/f34 x:0 0 0 2024-05-17T21:52:54.015 INFO:tasks.workunit.client.0.smithi107.stdout:9/466: dread d0/dd/d21/d29/f5a [0,4194304] 0 2024-05-17T21:52:54.016 INFO:tasks.workunit.client.1.smithi138.stdout:1/248: chown d4/df/d3a/c41 0 1 2024-05-17T21:52:54.017 INFO:tasks.workunit.client.1.smithi138.stdout:1/249: truncate d4/df/d35/f43 863011 0 2024-05-17T21:52:54.017 INFO:tasks.workunit.client.1.smithi138.stdout:1/250: chown d4/db/f4b 112612 1 2024-05-17T21:52:54.019 INFO:tasks.workunit.client.0.smithi107.stdout:9/467: fsync d0/dd/f60 0 2024-05-17T21:52:54.021 INFO:tasks.workunit.client.1.smithi138.stdout:2/303: creat d3/d21/d2f/f5f x:0 0 0 2024-05-17T21:52:54.022 INFO:tasks.workunit.client.1.smithi138.stdout:2/304: write d3/f20 [450072,124835] 0 2024-05-17T21:52:54.022 INFO:tasks.workunit.client.1.smithi138.stdout:1/251: dread d4/dd/f13 [0,4194304] 0 2024-05-17T21:52:54.022 INFO:tasks.workunit.client.1.smithi138.stdout:5/287: mkdir d1/d1a/d39/d4e/d58/d5b 0 2024-05-17T21:52:54.025 INFO:tasks.workunit.client.1.smithi138.stdout:1/252: read d4/df/d35/f37 [1153706,89384] 0 2024-05-17T21:52:54.025 INFO:tasks.workunit.client.0.smithi107.stdout:9/468: dwrite d0/dd/d21/d29/d3e/d45/f6b [4194304,4194304] 0 2024-05-17T21:52:54.026 INFO:tasks.workunit.client.0.smithi107.stdout:9/469: chown d0/dd/d2f/l76 13956466 1 2024-05-17T21:52:54.030 INFO:tasks.workunit.client.0.smithi107.stdout:4/476: sync 2024-05-17T21:52:54.030 INFO:tasks.workunit.client.0.smithi107.stdout:7/415: sync 2024-05-17T21:52:54.030 INFO:tasks.workunit.client.0.smithi107.stdout:8/462: sync 2024-05-17T21:52:54.031 INFO:tasks.workunit.client.1.smithi138.stdout:5/288: creat d1/d24/f5c x:0 0 0 2024-05-17T21:52:54.032 INFO:tasks.workunit.client.0.smithi107.stdout:9/470: creat d0/dd/d21/d29/d57/fa3 x:0 0 0 2024-05-17T21:52:54.032 INFO:tasks.workunit.client.0.smithi107.stdout:4/477: chown d3/d4a/d47/d60/d3a/d41/d80/l34 118557 1 2024-05-17T21:52:54.033 INFO:tasks.workunit.client.0.smithi107.stdout:7/416: dread d5/f42 [0,4194304] 0 2024-05-17T21:52:54.033 INFO:tasks.workunit.client.1.smithi138.stdout:2/305: rmdir d3/d21/d2f/d3e/d58 39 2024-05-17T21:52:54.034 INFO:tasks.workunit.client.0.smithi107.stdout:9/471: chown d0/dd/d1b/c26 1601258 1 2024-05-17T21:52:54.034 INFO:tasks.workunit.client.0.smithi107.stdout:4/478: dread - d3/d4a/d47/d60/f79 zero size 2024-05-17T21:52:54.035 INFO:tasks.workunit.client.0.smithi107.stdout:4/479: chown d3/d4a/d47/d60/d3c 36 1 2024-05-17T21:52:54.035 INFO:tasks.workunit.client.0.smithi107.stdout:7/417: fdatasync d5/df/d11/d1c/d2f/f7c 0 2024-05-17T21:52:54.039 INFO:tasks.workunit.client.1.smithi138.stdout:4/320: dread d0/f2a [0,4194304] 0 2024-05-17T21:52:54.040 INFO:tasks.workunit.client.1.smithi138.stdout:4/321: write d0/d1b/d2b/f65 [604985,9987] 0 2024-05-17T21:52:54.041 INFO:tasks.workunit.client.1.smithi138.stdout:2/306: dwrite d3/d21/d2f/f45 [0,4194304] 0 2024-05-17T21:52:54.041 INFO:tasks.workunit.client.1.smithi138.stdout:1/253: rename d4/df/l25 to d4/d2e/l59 0 2024-05-17T21:52:54.042 INFO:tasks.workunit.client.1.smithi138.stdout:4/322: chown d0/d3/l8 3 1 2024-05-17T21:52:54.042 INFO:tasks.workunit.client.1.smithi138.stdout:1/254: fsync d4/db/f22 0 2024-05-17T21:52:54.044 INFO:tasks.workunit.client.0.smithi107.stdout:9/472: symlink d0/dd/d69/la4 0 2024-05-17T21:52:54.050 INFO:tasks.workunit.client.1.smithi138.stdout:4/323: dwrite d0/d3/f57 [0,4194304] 0 2024-05-17T21:52:54.051 INFO:tasks.workunit.client.1.smithi138.stdout:4/324: chown d0/d2f/f6f 61 1 2024-05-17T21:52:54.051 INFO:tasks.workunit.client.0.smithi107.stdout:7/418: chown d5/l7b 119395869 1 2024-05-17T21:52:54.052 INFO:tasks.workunit.client.0.smithi107.stdout:7/419: read - d5/f83 zero size 2024-05-17T21:52:54.057 INFO:tasks.workunit.client.0.smithi107.stdout:7/420: unlink d5/df/d11/f25 0 2024-05-17T21:52:54.066 INFO:tasks.workunit.client.1.smithi138.stdout:2/307: symlink d3/d21/d2f/d3e/d55/l60 0 2024-05-17T21:52:54.068 INFO:tasks.workunit.client.1.smithi138.stdout:1/255: mknod d4/db/d29/d47/c5a 0 2024-05-17T21:52:54.069 INFO:tasks.workunit.client.1.smithi138.stdout:5/289: creat d1/d6/d1b/f5d x:0 0 0 2024-05-17T21:52:54.070 INFO:tasks.workunit.client.0.smithi107.stdout:9/473: dread d0/dd/d21/f37 [0,4194304] 0 2024-05-17T21:52:54.071 INFO:tasks.workunit.client.0.smithi107.stdout:9/474: readlink d0/dd/d69/la4 0 2024-05-17T21:52:54.071 INFO:tasks.workunit.client.0.smithi107.stdout:9/475: stat d0/dd/d2f/f5b 0 2024-05-17T21:52:54.072 INFO:tasks.workunit.client.1.smithi138.stdout:5/290: mkdir d1/d24/d2f/d5e 0 2024-05-17T21:52:54.073 INFO:tasks.workunit.client.1.smithi138.stdout:5/291: mknod d1/d53/c5f 0 2024-05-17T21:52:54.128 INFO:tasks.workunit.client.1.smithi138.stdout:0/351: sync 2024-05-17T21:52:54.131 INFO:tasks.workunit.client.1.smithi138.stdout:9/273: truncate d6/d7/f30 4014947 0 2024-05-17T21:52:54.131 INFO:tasks.workunit.client.1.smithi138.stdout:0/352: read d3/d10/d38/f46 [192055,90712] 0 2024-05-17T21:52:54.137 INFO:tasks.workunit.client.1.smithi138.stdout:9/274: link d6/d7/d16/d29/f3a d6/d7/d16/d29/d38/d3c/d3e/f56 0 2024-05-17T21:52:54.141 INFO:tasks.workunit.client.1.smithi138.stdout:9/275: write d6/d7/d16/d29/d45/f53 [751855,123721] 0 2024-05-17T21:52:54.146 INFO:tasks.workunit.client.0.smithi107.stdout:3/470: sync 2024-05-17T21:52:54.147 INFO:tasks.workunit.client.0.smithi107.stdout:2/331: sync 2024-05-17T21:52:54.147 INFO:tasks.workunit.client.0.smithi107.stdout:0/541: sync 2024-05-17T21:52:54.147 INFO:tasks.workunit.client.0.smithi107.stdout:5/472: sync 2024-05-17T21:52:54.148 INFO:tasks.workunit.client.0.smithi107.stdout:3/471: fdatasync d6/d1a/d57/f73 0 2024-05-17T21:52:54.148 INFO:tasks.workunit.client.1.smithi138.stdout:3/355: dwrite d1/d3/d16/d3e/f4a [0,4194304] 0 2024-05-17T21:52:54.150 INFO:tasks.workunit.client.0.smithi107.stdout:8/463: rmdir dc/d21/d27/d83 39 2024-05-17T21:52:54.155 INFO:tasks.workunit.client.0.smithi107.stdout:5/473: mkdir d5/db/dc/d1e/d6e/d93 0 2024-05-17T21:52:54.155 INFO:tasks.workunit.client.0.smithi107.stdout:0/542: dwrite d3/d24/d6f/d9f/dab/f89 [0,4194304] 0 2024-05-17T21:52:54.156 INFO:tasks.workunit.client.0.smithi107.stdout:3/472: creat d6/d8/d10/d25/f99 x:0 0 0 2024-05-17T21:52:54.157 INFO:tasks.workunit.client.0.smithi107.stdout:2/332: symlink d0/d9/d1c/l66 0 2024-05-17T21:52:54.158 INFO:tasks.workunit.client.0.smithi107.stdout:5/474: chown d5/db/d16/d2d/d4d/l74 29 1 2024-05-17T21:52:54.158 INFO:tasks.workunit.client.0.smithi107.stdout:3/473: chown d6/d1a/l4a 1031363805 1 2024-05-17T21:52:54.160 INFO:tasks.workunit.client.1.smithi138.stdout:3/356: dread d1/d3/f11 [0,4194304] 0 2024-05-17T21:52:54.161 INFO:tasks.workunit.client.0.smithi107.stdout:3/474: write d6/d2a/d2d/f41 [229117,44406] 0 2024-05-17T21:52:54.161 INFO:tasks.workunit.client.0.smithi107.stdout:8/464: getdents dc/d21/d35/dab 0 2024-05-17T21:52:54.165 INFO:tasks.workunit.client.0.smithi107.stdout:8/465: readlink dc/d21/l8c 0 2024-05-17T21:52:54.165 INFO:tasks.workunit.client.0.smithi107.stdout:2/333: dread d0/d26/f55 [0,4194304] 0 2024-05-17T21:52:54.166 INFO:tasks.workunit.client.1.smithi138.stdout:3/357: symlink d1/d3/d16/d2d/l7c 0 2024-05-17T21:52:54.166 INFO:tasks.workunit.client.1.smithi138.stdout:3/358: fsync d1/f4 0 2024-05-17T21:52:54.167 INFO:tasks.workunit.client.0.smithi107.stdout:2/334: chown d0/d9/d1c/d3e/d4f/f61 1072375547 1 2024-05-17T21:52:54.170 INFO:tasks.workunit.client.0.smithi107.stdout:4/480: sync 2024-05-17T21:52:54.171 INFO:tasks.workunit.client.0.smithi107.stdout:3/475: mknod d6/d8/d10/d35/d36/d66/d88/c9a 0 2024-05-17T21:52:54.171 INFO:tasks.workunit.client.0.smithi107.stdout:3/476: write f5 [4472234,28127] 0 2024-05-17T21:52:54.172 INFO:tasks.workunit.client.0.smithi107.stdout:2/335: write d0/d9/d1c/d3e/f62 [5030472,82815] 0 2024-05-17T21:52:54.179 INFO:tasks.workunit.client.1.smithi138.stdout:3/359: getdents d1 0 2024-05-17T21:52:54.180 INFO:tasks.workunit.client.0.smithi107.stdout:3/477: symlink d6/d8/d3e/d3f/d90/l9b 0 2024-05-17T21:52:54.181 INFO:tasks.workunit.client.1.smithi138.stdout:3/360: readlink d1/d4d/l6c 0 2024-05-17T21:52:54.184 INFO:tasks.workunit.client.0.smithi107.stdout:0/543: dread d3/f11 [0,4194304] 0 2024-05-17T21:52:54.185 INFO:tasks.workunit.client.1.smithi138.stdout:3/361: dread d1/d3/d16/d3a/f78 [0,4194304] 0 2024-05-17T21:52:54.186 INFO:tasks.workunit.client.1.smithi138.stdout:3/362: truncate d1/f65 917943 0 2024-05-17T21:52:54.187 INFO:tasks.workunit.client.0.smithi107.stdout:3/478: getdents d6/d8/d10/d35/d36 0 2024-05-17T21:52:54.188 INFO:tasks.workunit.client.1.smithi138.stdout:3/363: chown d1/d3/f10 124322 1 2024-05-17T21:52:54.190 INFO:tasks.workunit.client.1.smithi138.stdout:1/256: dread d4/df/f12 [0,4194304] 0 2024-05-17T21:52:54.193 INFO:tasks.workunit.client.1.smithi138.stdout:3/364: link d1/d3/d8/f34 d1/d3/d49/f7d 0 2024-05-17T21:52:54.197 INFO:tasks.workunit.client.0.smithi107.stdout:0/544: dread d3/d24/f2a [0,4194304] 0 2024-05-17T21:52:54.198 INFO:tasks.workunit.client.1.smithi138.stdout:4/325: write d0/d3/db/f20 [704680,60869] 0 2024-05-17T21:52:54.200 INFO:tasks.workunit.client.0.smithi107.stdout:0/545: rmdir d3/d24/d6f 39 2024-05-17T21:52:54.200 INFO:tasks.workunit.client.1.smithi138.stdout:3/365: dwrite d1/d4f/d50/f6b [0,4194304] 0 2024-05-17T21:52:54.202 INFO:tasks.workunit.client.0.smithi107.stdout:7/421: dwrite d5/df/d11/d1d/d33/f64 [0,4194304] 0 2024-05-17T21:52:54.206 INFO:tasks.workunit.client.0.smithi107.stdout:3/479: dread d6/d8/d10/d25/f55 [0,4194304] 0 2024-05-17T21:52:54.207 INFO:tasks.workunit.client.1.smithi138.stdout:3/366: dread d1/d3/d16/d3e/f4a [0,4194304] 0 2024-05-17T21:52:54.209 INFO:tasks.workunit.client.0.smithi107.stdout:7/422: symlink d5/d16/d3b/d45/l93 0 2024-05-17T21:52:54.211 INFO:tasks.workunit.client.0.smithi107.stdout:0/546: fdatasync d3/fa1 0 2024-05-17T21:52:54.213 INFO:tasks.workunit.client.0.smithi107.stdout:3/480: truncate d6/d8/d10/d35/d36/d4c/f65 73455 0 2024-05-17T21:52:54.213 INFO:tasks.workunit.client.0.smithi107.stdout:3/481: stat d6/d1a/l6e 0 2024-05-17T21:52:54.214 INFO:tasks.workunit.client.0.smithi107.stdout:3/482: chown d6/d8/d10/d35/d36 148297269 1 2024-05-17T21:52:54.214 INFO:tasks.workunit.client.1.smithi138.stdout:3/367: creat d1/d4d/f7e x:0 0 0 2024-05-17T21:52:54.216 INFO:tasks.workunit.client.0.smithi107.stdout:7/423: read d5/d16/f23 [877889,71911] 0 2024-05-17T21:52:54.217 INFO:tasks.workunit.client.1.smithi138.stdout:9/276: mknod d6/d7/c57 0 2024-05-17T21:52:54.217 INFO:tasks.workunit.client.1.smithi138.stdout:3/368: symlink d1/l7f 0 2024-05-17T21:52:54.219 INFO:tasks.workunit.client.0.smithi107.stdout:3/483: mknod d6/d8/d10/c9c 0 2024-05-17T21:52:54.222 INFO:tasks.workunit.client.0.smithi107.stdout:7/424: creat d5/df/d11/d1c/d31/d39/f94 x:0 0 0 2024-05-17T21:52:54.223 INFO:tasks.workunit.client.0.smithi107.stdout:7/425: chown d5/df/d11/d1c/d2f/d53/f90 1122561415 1 2024-05-17T21:52:54.223 INFO:tasks.workunit.client.1.smithi138.stdout:2/308: unlink d3/l26 0 2024-05-17T21:52:54.227 INFO:tasks.workunit.client.1.smithi138.stdout:5/292: rename d1/d24 to d1/d60 0 2024-05-17T21:52:54.229 INFO:tasks.workunit.client.0.smithi107.stdout:3/484: mknod d6/d8/d10/d35/d36/d4c/c9d 0 2024-05-17T21:52:54.231 INFO:tasks.workunit.client.1.smithi138.stdout:9/277: dread d6/d7/d16/d29/d48/f22 [0,4194304] 0 2024-05-17T21:52:54.232 INFO:tasks.workunit.client.0.smithi107.stdout:7/426: creat d5/df/d78/f95 x:0 0 0 2024-05-17T21:52:54.237 INFO:tasks.workunit.client.1.smithi138.stdout:2/309: creat d3/f61 x:0 0 0 2024-05-17T21:52:54.238 INFO:tasks.workunit.client.0.smithi107.stdout:0/547: dread d3/dc/f6d [0,4194304] 0 2024-05-17T21:52:54.238 INFO:tasks.workunit.client.1.smithi138.stdout:2/310: write d3/f57 [129712,27021] 0 2024-05-17T21:52:54.239 INFO:tasks.workunit.client.0.smithi107.stdout:7/427: dwrite d5/df/d78/f95 [0,4194304] 0 2024-05-17T21:52:54.240 INFO:tasks.workunit.client.1.smithi138.stdout:9/278: mknod d6/d7/d16/d29/d48/d3b/c58 0 2024-05-17T21:52:54.240 INFO:tasks.workunit.client.1.smithi138.stdout:2/311: truncate d3/d21/d2f/f5b 283636 0 2024-05-17T21:52:54.240 INFO:tasks.workunit.client.1.smithi138.stdout:2/312: chown d3/d21/d2f/f5b 1890692 1 2024-05-17T21:52:54.241 INFO:tasks.workunit.client.1.smithi138.stdout:2/313: dread - d3/d21/f31 zero size 2024-05-17T21:52:54.242 INFO:tasks.workunit.client.1.smithi138.stdout:2/314: chown d3/d21/d2f/f3c 133108 1 2024-05-17T21:52:54.243 INFO:tasks.workunit.client.0.smithi107.stdout:1/519: rename d4/df/d14/d33 to d4/df/d1a/d26/d72/daa 0 2024-05-17T21:52:54.244 INFO:tasks.workunit.client.1.smithi138.stdout:5/293: rename d1/d6/d1b/l1e to d1/d1a/d39/d4e/d58/l61 0 2024-05-17T21:52:54.245 INFO:tasks.workunit.client.0.smithi107.stdout:1/520: truncate d4/df/d14/d2e/d43/d47/f94 61936 0 2024-05-17T21:52:54.247 INFO:tasks.workunit.client.1.smithi138.stdout:2/315: mkdir d3/d21/d2f/d3e/d55/d62 0 2024-05-17T21:52:54.248 INFO:tasks.workunit.client.0.smithi107.stdout:0/548: creat d3/dc/d22/d23/d97/d77/fca x:0 0 0 2024-05-17T21:52:54.249 INFO:tasks.workunit.client.0.smithi107.stdout:0/549: readlink d3/d58/d61/d94/lb8 0 2024-05-17T21:52:54.249 INFO:tasks.workunit.client.1.smithi138.stdout:5/294: mkdir d1/d1a/d39/d62 0 2024-05-17T21:52:54.249 INFO:tasks.workunit.client.0.smithi107.stdout:0/550: truncate d3/dc/fc3 305231 0 2024-05-17T21:52:54.250 INFO:tasks.workunit.client.0.smithi107.stdout:6/375: rename d9/d16/d4d/l50 to d9/d58/l71 0 2024-05-17T21:52:54.251 INFO:tasks.workunit.client.1.smithi138.stdout:9/279: mkdir d6/d59 0 2024-05-17T21:52:54.252 INFO:tasks.workunit.client.0.smithi107.stdout:3/485: getdents d6/d2a/d2d/d69 0 2024-05-17T21:52:54.253 INFO:tasks.workunit.client.1.smithi138.stdout:2/316: write d3/fe [304993,110541] 0 2024-05-17T21:52:54.257 INFO:tasks.workunit.client.1.smithi138.stdout:9/280: write d6/f25 [491187,94519] 0 2024-05-17T21:52:54.257 INFO:tasks.workunit.client.0.smithi107.stdout:3/486: dwrite d6/d2a/f5b [4194304,4194304] 0 2024-05-17T21:52:54.258 INFO:tasks.workunit.client.0.smithi107.stdout:0/551: stat d3/dc/d22/d3d/d56/c74 0 2024-05-17T21:52:54.259 INFO:tasks.workunit.client.0.smithi107.stdout:0/552: chown d3/dc/d22/d23/f8f 299236611 1 2024-05-17T21:52:54.260 INFO:tasks.workunit.client.1.smithi138.stdout:2/317: write d3/d21/d2f/d3e/d58/f5d [904103,30915] 0 2024-05-17T21:52:54.261 INFO:tasks.workunit.client.0.smithi107.stdout:0/553: write d3/d39/f69 [1595938,17867] 0 2024-05-17T21:52:54.262 INFO:tasks.workunit.client.0.smithi107.stdout:6/376: write d9/f1d [3353287,31737] 0 2024-05-17T21:52:54.264 INFO:tasks.workunit.client.1.smithi138.stdout:9/281: unlink d6/d7/d16/d29/d38/f3d 0 2024-05-17T21:52:54.266 INFO:tasks.workunit.client.0.smithi107.stdout:7/428: creat d5/df/d11/d1d/f96 x:0 0 0 2024-05-17T21:52:54.266 INFO:tasks.workunit.client.0.smithi107.stdout:8/466: dwrite dc/d21/d27/d2f/f4a [0,4194304] 0 2024-05-17T21:52:54.267 INFO:tasks.workunit.client.0.smithi107.stdout:1/521: creat d4/d78/fab x:0 0 0 2024-05-17T21:52:54.267 INFO:tasks.workunit.client.0.smithi107.stdout:2/336: write d0/d9/d11/d18/f20 [4497951,45351] 0 2024-05-17T21:52:54.270 INFO:tasks.workunit.client.0.smithi107.stdout:3/487: mknod d6/d8/d10/d35/c9e 0 2024-05-17T21:52:54.272 INFO:tasks.workunit.client.0.smithi107.stdout:3/488: readlink d6/d1a/l6e 0 2024-05-17T21:52:54.273 INFO:tasks.workunit.client.0.smithi107.stdout:2/337: truncate d0/d9/d1c/d22/f5b 700694 0 2024-05-17T21:52:54.273 INFO:tasks.workunit.client.0.smithi107.stdout:4/481: dwrite d3/db/d35/d5e/fa0 [0,4194304] 0 2024-05-17T21:52:54.275 INFO:tasks.workunit.client.1.smithi138.stdout:2/318: mkdir d3/d21/d2f/d3e/d63 0 2024-05-17T21:52:54.277 INFO:tasks.workunit.client.1.smithi138.stdout:9/282: mkdir d6/d7/d16/d29/d38/d3c/d3e/d5a 0 2024-05-17T21:52:54.278 INFO:tasks.workunit.client.1.smithi138.stdout:9/283: stat d6/d7 0 2024-05-17T21:52:54.279 INFO:tasks.workunit.client.0.smithi107.stdout:6/377: creat d9/d16/d22/f72 x:0 0 0 2024-05-17T21:52:54.280 INFO:tasks.workunit.client.1.smithi138.stdout:1/257: dwrite d4/df/f2c [0,4194304] 0 2024-05-17T21:52:54.284 INFO:tasks.workunit.client.1.smithi138.stdout:4/326: dwrite d0/d2f/d49/d60/f3c [0,4194304] 0 2024-05-17T21:52:54.285 INFO:tasks.workunit.client.0.smithi107.stdout:7/429: rmdir d5/df 39 2024-05-17T21:52:54.288 INFO:tasks.workunit.client.1.smithi138.stdout:0/353: sync 2024-05-17T21:52:54.288 INFO:tasks.workunit.client.1.smithi138.stdout:6/332: sync 2024-05-17T21:52:54.288 INFO:tasks.workunit.client.1.smithi138.stdout:8/183: sync 2024-05-17T21:52:54.288 INFO:tasks.workunit.client.1.smithi138.stdout:6/333: readlink d7/l16 0 2024-05-17T21:52:54.289 INFO:tasks.workunit.client.1.smithi138.stdout:8/184: read d0/f17 [5320670,75626] 0 2024-05-17T21:52:54.291 INFO:tasks.workunit.client.1.smithi138.stdout:2/319: symlink d3/d21/d2f/d3e/l64 0 2024-05-17T21:52:54.292 INFO:tasks.workunit.client.1.smithi138.stdout:0/354: chown d3/d3d/d48/f5f 3 1 2024-05-17T21:52:54.292 INFO:tasks.workunit.client.0.smithi107.stdout:8/467: read - dc/d21/d27/d83/f91 zero size 2024-05-17T21:52:54.293 INFO:tasks.workunit.client.1.smithi138.stdout:8/185: write d0/dc/f22 [6602628,117206] 0 2024-05-17T21:52:54.294 INFO:tasks.workunit.client.1.smithi138.stdout:1/258: dwrite d4/df/f12 [0,4194304] 0 2024-05-17T21:52:54.294 INFO:tasks.workunit.client.1.smithi138.stdout:6/334: read d7/db/d12/f27 [3501393,27941] 0 2024-05-17T21:52:54.294 INFO:tasks.workunit.client.1.smithi138.stdout:4/327: creat d0/d2f/d49/d60/d1f/f73 x:0 0 0 2024-05-17T21:52:54.294 INFO:tasks.workunit.client.1.smithi138.stdout:2/320: read d3/f10 [1576803,123628] 0 2024-05-17T21:52:54.295 INFO:tasks.workunit.client.0.smithi107.stdout:1/522: creat d4/df/d4e/d59/fac x:0 0 0 2024-05-17T21:52:54.300 INFO:tasks.workunit.client.1.smithi138.stdout:6/335: write d7/db/d43/f64 [2868902,46093] 0 2024-05-17T21:52:54.300 INFO:tasks.workunit.client.1.smithi138.stdout:8/186: write d0/dc/d1a/f1d [518155,1901] 0 2024-05-17T21:52:54.302 INFO:tasks.workunit.client.0.smithi107.stdout:9/476: rename d0/dd/d21/d29/d57/fa3 to d0/fa5 0 2024-05-17T21:52:54.307 INFO:tasks.workunit.client.0.smithi107.stdout:3/489: creat d6/d8/d3e/d3f/d90/f9f x:0 0 0 2024-05-17T21:52:54.310 INFO:tasks.workunit.client.0.smithi107.stdout:2/338: dwrite d0/d26/f55 [0,4194304] 0 2024-05-17T21:52:54.312 INFO:tasks.workunit.client.0.smithi107.stdout:0/554: unlink d3/dc/c2e 0 2024-05-17T21:52:54.318 INFO:tasks.workunit.client.0.smithi107.stdout:3/490: dwrite d6/d8/d10/d25/f95 [0,4194304] 0 2024-05-17T21:52:54.319 INFO:tasks.workunit.client.0.smithi107.stdout:6/378: mkdir d9/d16/d4d/d73 0 2024-05-17T21:52:54.320 INFO:tasks.workunit.client.0.smithi107.stdout:6/379: read d9/d16/f3b [426664,68765] 0 2024-05-17T21:52:54.322 INFO:tasks.workunit.client.0.smithi107.stdout:7/430: read - d5/df/d11/d1d/f8c zero size 2024-05-17T21:52:54.326 INFO:tasks.workunit.client.1.smithi138.stdout:5/295: write d1/d6/fa [1108487,38220] 0 2024-05-17T21:52:54.326 INFO:tasks.workunit.client.0.smithi107.stdout:1/523: dread d4/df/d1a/f28 [0,4194304] 0 2024-05-17T21:52:54.326 INFO:tasks.workunit.client.1.smithi138.stdout:1/259: creat d4/db/d29/f5b x:0 0 0 2024-05-17T21:52:54.327 INFO:tasks.workunit.client.1.smithi138.stdout:5/296: fsync d1/d6/d1b/d29/d36/f37 0 2024-05-17T21:52:54.329 INFO:tasks.workunit.client.1.smithi138.stdout:4/328: symlink d0/d2f/d49/d60/l74 0 2024-05-17T21:52:54.329 INFO:tasks.workunit.client.1.smithi138.stdout:4/329: readlink d0/d2f/d49/d60/l28 0 2024-05-17T21:52:54.331 INFO:tasks.workunit.client.1.smithi138.stdout:1/260: dread d4/df/f2c [0,4194304] 0 2024-05-17T21:52:54.331 INFO:tasks.workunit.client.0.smithi107.stdout:5/475: rename d5/f84 to d5/db/d16/d2a/f94 0 2024-05-17T21:52:54.331 INFO:tasks.workunit.client.1.smithi138.stdout:4/330: truncate d0/d2f/d49/d60/d1f/f73 948001 0 2024-05-17T21:52:54.333 INFO:tasks.workunit.client.1.smithi138.stdout:1/261: dread d4/f3d [0,4194304] 0 2024-05-17T21:52:54.334 INFO:tasks.workunit.client.0.smithi107.stdout:9/477: symlink d0/dd/d2f/d81/la6 0 2024-05-17T21:52:54.334 INFO:tasks.workunit.client.1.smithi138.stdout:1/262: stat d4/f9 0 2024-05-17T21:52:54.334 INFO:tasks.workunit.client.1.smithi138.stdout:1/263: stat d4/db/f39 0 2024-05-17T21:52:54.335 INFO:tasks.workunit.client.0.smithi107.stdout:9/478: fsync d0/dd/d21/d29/d3e/d45/d68/f7a 0 2024-05-17T21:52:54.335 INFO:tasks.workunit.client.1.smithi138.stdout:8/187: write d0/dc/d1a/f24 [1395683,42265] 0 2024-05-17T21:52:54.340 INFO:tasks.workunit.client.1.smithi138.stdout:3/369: stat d1/d4f/d50/l56 0 2024-05-17T21:52:54.343 INFO:tasks.workunit.client.1.smithi138.stdout:8/188: dwrite d0/dc/f10 [0,4194304] 0 2024-05-17T21:52:54.344 INFO:tasks.workunit.client.1.smithi138.stdout:7/230: fdatasync d0/f12 0 2024-05-17T21:52:54.346 INFO:tasks.workunit.client.0.smithi107.stdout:2/339: dread d0/d9/d1c/d22/f65 [0,4194304] 0 2024-05-17T21:52:54.349 INFO:tasks.workunit.client.0.smithi107.stdout:7/431: symlink d5/d16/d3b/l97 0 2024-05-17T21:52:54.350 INFO:tasks.workunit.client.1.smithi138.stdout:4/331: dread d0/f7 [0,4194304] 0 2024-05-17T21:52:54.353 INFO:tasks.workunit.client.1.smithi138.stdout:9/284: dwrite d6/d7/d16/d29/f3a [0,4194304] 0 2024-05-17T21:52:54.355 INFO:tasks.workunit.client.0.smithi107.stdout:4/482: dwrite d3/db/d35/d5e/fa0 [4194304,4194304] 0 2024-05-17T21:52:54.356 INFO:tasks.workunit.client.1.smithi138.stdout:9/285: chown d6/d7/d16/l52 5548 1 2024-05-17T21:52:54.361 INFO:tasks.workunit.client.1.smithi138.stdout:4/332: dread d0/d2f/d49/d60/d1f/d32/f43 [0,4194304] 0 2024-05-17T21:52:54.364 INFO:tasks.workunit.client.1.smithi138.stdout:8/189: dread d0/fa [0,4194304] 0 2024-05-17T21:52:54.365 INFO:tasks.workunit.client.0.smithi107.stdout:5/476: mknod d5/db/dc/d1e/d6e/c95 0 2024-05-17T21:52:54.366 INFO:tasks.workunit.client.0.smithi107.stdout:5/477: fdatasync d5/db/d39/f3c 0 2024-05-17T21:52:54.366 INFO:tasks.workunit.client.0.smithi107.stdout:5/478: chown d5/db/d16/d2d/d51 2926 1 2024-05-17T21:52:54.368 INFO:tasks.workunit.client.1.smithi138.stdout:4/333: dread d0/f2a [0,4194304] 0 2024-05-17T21:52:54.369 INFO:tasks.workunit.client.0.smithi107.stdout:9/479: creat d0/dd/d1f/fa7 x:0 0 0 2024-05-17T21:52:54.372 INFO:tasks.workunit.client.0.smithi107.stdout:5/479: read d5/db/d16/d5b/f3d [1878507,89695] 0 2024-05-17T21:52:54.372 INFO:tasks.workunit.client.1.smithi138.stdout:5/297: dread d1/d6/d1b/f1d [0,4194304] 0 2024-05-17T21:52:54.372 INFO:tasks.workunit.client.1.smithi138.stdout:2/321: mkdir d3/d21/d2f/d3e/d63/d65 0 2024-05-17T21:52:54.373 INFO:tasks.workunit.client.1.smithi138.stdout:2/322: stat d3/f27 0 2024-05-17T21:52:54.374 INFO:tasks.workunit.client.1.smithi138.stdout:2/323: fsync d3/d21/d2f/f39 0 2024-05-17T21:52:54.374 INFO:tasks.workunit.client.1.smithi138.stdout:8/190: dwrite d0/dc/f22 [4194304,4194304] 0 2024-05-17T21:52:54.376 INFO:tasks.workunit.client.1.smithi138.stdout:2/324: write d3/d21/d2f/f5f [693579,16219] 0 2024-05-17T21:52:54.381 INFO:tasks.workunit.client.1.smithi138.stdout:1/264: unlink d4/df/d35/f43 0 2024-05-17T21:52:54.383 INFO:tasks.workunit.client.1.smithi138.stdout:2/325: dwrite d3/fe [0,4194304] 0 2024-05-17T21:52:54.387 INFO:tasks.workunit.client.0.smithi107.stdout:8/468: truncate dc/d44/f84 585972 0 2024-05-17T21:52:54.387 INFO:tasks.workunit.client.0.smithi107.stdout:8/469: stat dc/f4f 0 2024-05-17T21:52:54.389 INFO:tasks.workunit.client.0.smithi107.stdout:4/483: symlink d3/db/d35/d4f/la2 0 2024-05-17T21:52:54.391 INFO:tasks.workunit.client.0.smithi107.stdout:1/524: write d4/f1b [606574,113252] 0 2024-05-17T21:52:54.394 INFO:tasks.workunit.client.0.smithi107.stdout:9/480: symlink d0/dd/d69/la8 0 2024-05-17T21:52:54.395 INFO:tasks.workunit.client.0.smithi107.stdout:9/481: rename d0/dd to d0/dd/d21/da9 22 2024-05-17T21:52:54.396 INFO:tasks.workunit.client.1.smithi138.stdout:0/355: link d3/d10/f31 d3/f7a 0 2024-05-17T21:52:54.398 INFO:tasks.workunit.client.1.smithi138.stdout:9/286: symlink d6/d7/d16/d29/d38/d3c/d3e/l5b 0 2024-05-17T21:52:54.399 INFO:tasks.workunit.client.1.smithi138.stdout:9/287: write d6/d7/f26 [659529,111880] 0 2024-05-17T21:52:54.404 INFO:tasks.workunit.client.1.smithi138.stdout:9/288: dwrite d6/d7/f26 [0,4194304] 0 2024-05-17T21:52:54.405 INFO:tasks.workunit.client.1.smithi138.stdout:4/334: creat d0/d3/d17/d21/f75 x:0 0 0 2024-05-17T21:52:54.407 INFO:tasks.workunit.client.0.smithi107.stdout:7/432: creat d5/df/d11/d1c/d31/d39/d92/f98 x:0 0 0 2024-05-17T21:52:54.409 INFO:tasks.workunit.client.1.smithi138.stdout:5/298: mkdir d1/d2c/d63 0 2024-05-17T21:52:54.412 INFO:tasks.workunit.client.0.smithi107.stdout:4/484: mkdir d3/d4a/d47/da3 0 2024-05-17T21:52:54.414 INFO:tasks.workunit.client.0.smithi107.stdout:1/525: rmdir d4/df/d1a/d26/d7a 39 2024-05-17T21:52:54.414 INFO:tasks.workunit.client.0.smithi107.stdout:1/526: fsync d4/f1b 0 2024-05-17T21:52:54.415 INFO:tasks.workunit.client.0.smithi107.stdout:1/527: chown d4/df/d14/la4 16610 1 2024-05-17T21:52:54.416 INFO:tasks.workunit.client.0.smithi107.stdout:9/482: creat d0/dd/d21/faa x:0 0 0 2024-05-17T21:52:54.417 INFO:tasks.workunit.client.0.smithi107.stdout:0/555: write d3/dc/d22/f3e [1746382,115311] 0 2024-05-17T21:52:54.419 INFO:tasks.workunit.client.1.smithi138.stdout:1/265: chown d4/db/d29/c50 12223 1 2024-05-17T21:52:54.422 INFO:tasks.workunit.client.1.smithi138.stdout:6/336: creat d7/f72 x:0 0 0 2024-05-17T21:52:54.422 INFO:tasks.workunit.client.0.smithi107.stdout:9/483: dwrite d0/dd/d21/d29/d3e/d45/f6b [4194304,4194304] 0 2024-05-17T21:52:54.424 INFO:tasks.workunit.client.1.smithi138.stdout:3/370: mkdir d1/d4f/d80 0 2024-05-17T21:52:54.427 INFO:tasks.workunit.client.1.smithi138.stdout:9/289: dread d6/d7/d16/d29/f37 [0,4194304] 0 2024-05-17T21:52:54.429 INFO:tasks.workunit.client.0.smithi107.stdout:8/470: symlink dc/d21/d27/d2f/lad 0 2024-05-17T21:52:54.434 INFO:tasks.workunit.client.0.smithi107.stdout:4/485: mknod d3/d6/d26/d36/d7e/ca4 0 2024-05-17T21:52:54.434 INFO:tasks.workunit.client.1.smithi138.stdout:9/290: dread d6/d7/f15 [0,4194304] 0 2024-05-17T21:52:54.434 INFO:tasks.workunit.client.1.smithi138.stdout:0/356: mknod d3/d10/d39/c7b 0 2024-05-17T21:52:54.435 INFO:tasks.workunit.client.0.smithi107.stdout:8/471: dwrite dc/d21/d97/f8d [0,4194304] 0 2024-05-17T21:52:54.435 INFO:tasks.workunit.client.1.smithi138.stdout:0/357: write d3/f7 [359456,60124] 0 2024-05-17T21:52:54.436 INFO:tasks.workunit.client.0.smithi107.stdout:8/472: fdatasync dc/d44/d4b/d52/f96 0 2024-05-17T21:52:54.439 INFO:tasks.workunit.client.0.smithi107.stdout:0/556: fdatasync d3/d24/f2a 0 2024-05-17T21:52:54.442 INFO:tasks.workunit.client.1.smithi138.stdout:5/299: creat d1/d60/f64 x:0 0 0 2024-05-17T21:52:54.444 INFO:tasks.workunit.client.1.smithi138.stdout:8/191: symlink d0/d31/l35 0 2024-05-17T21:52:54.444 INFO:tasks.workunit.client.0.smithi107.stdout:6/380: link d9/f15 d9/d16/d22/f74 0 2024-05-17T21:52:54.445 INFO:tasks.workunit.client.1.smithi138.stdout:8/192: truncate d0/f1b 449151 0 2024-05-17T21:52:54.446 INFO:tasks.workunit.client.0.smithi107.stdout:2/340: getdents d0/d9/d11/d15 0 2024-05-17T21:52:54.447 INFO:tasks.workunit.client.1.smithi138.stdout:1/266: rmdir d4/db/d29/d47 39 2024-05-17T21:52:54.447 INFO:tasks.workunit.client.0.smithi107.stdout:9/484: dread d0/dd/d3d/f92 [0,4194304] 0 2024-05-17T21:52:54.451 INFO:tasks.workunit.client.0.smithi107.stdout:7/433: link d5/df/d11/d1c/d31/d39/f8d d5/df/d11/d1d/d33/f99 0 2024-05-17T21:52:54.455 INFO:tasks.workunit.client.0.smithi107.stdout:4/486: creat d3/d4a/d47/d60/d3a/d41/d80/fa5 x:0 0 0 2024-05-17T21:52:54.456 INFO:tasks.workunit.client.0.smithi107.stdout:7/434: dwrite d5/df/d78/f95 [0,4194304] 0 2024-05-17T21:52:54.459 INFO:tasks.workunit.client.0.smithi107.stdout:8/473: creat dc/d21/d68/fae x:0 0 0 2024-05-17T21:52:54.463 INFO:tasks.workunit.client.1.smithi138.stdout:6/337: write d7/db/d43/d3f/f41 [270398,66653] 0 2024-05-17T21:52:54.464 INFO:tasks.workunit.client.1.smithi138.stdout:1/267: dread d4/fa [4194304,4194304] 0 2024-05-17T21:52:54.466 INFO:tasks.workunit.client.1.smithi138.stdout:4/335: symlink d0/d2f/d49/d60/d1f/d32/d71/l76 0 2024-05-17T21:52:54.467 INFO:tasks.workunit.client.1.smithi138.stdout:2/326: creat d3/f66 x:0 0 0 2024-05-17T21:52:54.468 INFO:tasks.workunit.client.1.smithi138.stdout:8/193: mknod d0/d31/c36 0 2024-05-17T21:52:54.469 INFO:tasks.workunit.client.1.smithi138.stdout:8/194: write d0/f17 [1703898,107237] 0 2024-05-17T21:52:54.470 INFO:tasks.workunit.client.1.smithi138.stdout:2/327: read d3/d21/d2f/f36 [90110,88788] 0 2024-05-17T21:52:54.474 INFO:tasks.workunit.client.1.smithi138.stdout:7/231: creat d0/f3b x:0 0 0 2024-05-17T21:52:54.475 INFO:tasks.workunit.client.1.smithi138.stdout:9/291: mkdir d6/d7/d16/d29/d38/d40/d44/d5c 0 2024-05-17T21:52:54.475 INFO:tasks.workunit.client.1.smithi138.stdout:0/358: symlink d3/d10/d38/l7c 0 2024-05-17T21:52:54.477 INFO:tasks.workunit.client.1.smithi138.stdout:1/268: creat d4/df/d4e/f5c x:0 0 0 2024-05-17T21:52:54.477 INFO:tasks.workunit.client.1.smithi138.stdout:1/269: chown d4/db/f4d 22948 1 2024-05-17T21:52:54.477 INFO:tasks.workunit.client.1.smithi138.stdout:4/336: mknod d0/d3/d1d/c77 0 2024-05-17T21:52:54.479 INFO:tasks.workunit.client.1.smithi138.stdout:8/195: unlink d0/d11/l30 0 2024-05-17T21:52:54.483 INFO:tasks.workunit.client.1.smithi138.stdout:1/270: unlink d4/df/f44 0 2024-05-17T21:52:54.484 INFO:tasks.workunit.client.1.smithi138.stdout:1/271: chown f0 0 1 2024-05-17T21:52:54.484 INFO:tasks.workunit.client.1.smithi138.stdout:4/337: mkdir d0/d3/d17/d21/d78 0 2024-05-17T21:52:54.484 INFO:tasks.workunit.client.1.smithi138.stdout:1/272: chown d4/fa 12744 1 2024-05-17T21:52:54.485 INFO:tasks.workunit.client.1.smithi138.stdout:8/196: mknod d0/c37 0 2024-05-17T21:52:54.485 INFO:tasks.workunit.client.1.smithi138.stdout:3/371: link d1/d3/d49/f7d d1/d3/d8/d5c/f81 0 2024-05-17T21:52:54.492 INFO:tasks.workunit.client.1.smithi138.stdout:0/359: rename d3/f24 to d3/d10/f7d 0 2024-05-17T21:52:54.493 INFO:tasks.workunit.client.1.smithi138.stdout:1/273: mkdir d4/db/d29/d5d 0 2024-05-17T21:52:54.495 INFO:tasks.workunit.client.1.smithi138.stdout:1/274: mkdir d4/df/d35/d5e 0 2024-05-17T21:52:54.496 INFO:tasks.workunit.client.1.smithi138.stdout:3/372: creat d1/d3/f82 x:0 0 0 2024-05-17T21:52:54.496 INFO:tasks.workunit.client.1.smithi138.stdout:1/275: chown d4/db/f4d 10 1 2024-05-17T21:52:54.497 INFO:tasks.workunit.client.1.smithi138.stdout:4/338: dread d0/d2f/d49/d60/f51 [0,4194304] 0 2024-05-17T21:52:54.497 INFO:tasks.workunit.client.1.smithi138.stdout:3/373: chown d1/d62 14588507 1 2024-05-17T21:52:54.498 INFO:tasks.workunit.client.1.smithi138.stdout:1/276: mkdir d4/df/d5f 0 2024-05-17T21:52:54.500 INFO:tasks.workunit.client.1.smithi138.stdout:1/277: chown d4/db/l1e 1321872306 1 2024-05-17T21:52:54.500 INFO:tasks.workunit.client.1.smithi138.stdout:4/339: mkdir d0/d1b/d2b/d55/d79 0 2024-05-17T21:52:54.501 INFO:tasks.workunit.client.1.smithi138.stdout:4/340: fsync d0/d2f/d49/d60/d1f/d32/f43 0 2024-05-17T21:52:54.502 INFO:tasks.workunit.client.1.smithi138.stdout:4/341: chown d0/d3/d1d/d38/f50 8555574 1 2024-05-17T21:52:54.506 INFO:tasks.workunit.client.1.smithi138.stdout:1/278: dwrite d4/dd/f15 [0,4194304] 0 2024-05-17T21:52:54.507 INFO:tasks.workunit.client.1.smithi138.stdout:4/342: rename d0/d3/d1d/d4c/f4e to d0/d2f/d49/d60/d1f/f7a 0 2024-05-17T21:52:54.509 INFO:tasks.workunit.client.1.smithi138.stdout:1/279: read d4/db/f4b [120017,92221] 0 2024-05-17T21:52:54.519 INFO:tasks.workunit.client.0.smithi107.stdout:3/491: sync 2024-05-17T21:52:54.520 INFO:tasks.workunit.client.0.smithi107.stdout:5/480: link d5/db/d16/d2d/l5d d5/d75/l96 0 2024-05-17T21:52:54.520 INFO:tasks.workunit.client.1.smithi138.stdout:1/280: dread d4/df/f1f [0,4194304] 0 2024-05-17T21:52:54.521 INFO:tasks.workunit.client.0.smithi107.stdout:3/492: chown f5 515003794 1 2024-05-17T21:52:54.521 INFO:tasks.workunit.client.0.smithi107.stdout:5/481: chown d5/c8b 1 1 2024-05-17T21:52:54.522 INFO:tasks.workunit.client.1.smithi138.stdout:1/281: rmdir d4/dd/d3b 39 2024-05-17T21:52:54.523 INFO:tasks.workunit.client.1.smithi138.stdout:1/282: chown d4/db 0 1 2024-05-17T21:52:54.523 INFO:tasks.workunit.client.0.smithi107.stdout:5/482: stat d5/db/d16/d2a/d73 0 2024-05-17T21:52:54.524 INFO:tasks.workunit.client.0.smithi107.stdout:4/487: mknod d3/db/ca6 0 2024-05-17T21:52:54.524 INFO:tasks.workunit.client.0.smithi107.stdout:7/435: mkdir d5/df/d11/d1c/d31/d39/d92/d9a 0 2024-05-17T21:52:54.525 INFO:tasks.workunit.client.0.smithi107.stdout:8/474: rename dc/d21/l2a to dc/d21/daa/laf 0 2024-05-17T21:52:54.527 INFO:tasks.workunit.client.0.smithi107.stdout:4/488: write d3/db/f46 [3839337,119316] 0 2024-05-17T21:52:54.527 INFO:tasks.workunit.client.0.smithi107.stdout:3/493: symlink d6/d8/d10/d35/d36/d66/la0 0 2024-05-17T21:52:54.527 INFO:tasks.workunit.client.1.smithi138.stdout:1/283: getdents d4/df/d3a/d3c 0 2024-05-17T21:52:54.529 INFO:tasks.workunit.client.0.smithi107.stdout:5/483: fdatasync d5/db/d16/d2d/d32/f91 0 2024-05-17T21:52:54.529 INFO:tasks.workunit.client.0.smithi107.stdout:6/381: creat d9/d16/d22/d5f/f75 x:0 0 0 2024-05-17T21:52:54.530 INFO:tasks.workunit.client.1.smithi138.stdout:3/374: dread d1/f4 [0,4194304] 0 2024-05-17T21:52:54.530 INFO:tasks.workunit.client.0.smithi107.stdout:4/489: stat d3/db/d35/d5e/fa0 0 2024-05-17T21:52:54.532 INFO:tasks.workunit.client.1.smithi138.stdout:3/375: write d1/d3/d8/de/f27 [4972727,20872] 0 2024-05-17T21:52:54.535 INFO:tasks.workunit.client.0.smithi107.stdout:5/484: dread d5/db/d39/f3e [0,4194304] 0 2024-05-17T21:52:54.536 INFO:tasks.workunit.client.0.smithi107.stdout:5/485: read d5/db/dc/f20 [963621,59957] 0 2024-05-17T21:52:54.538 INFO:tasks.workunit.client.0.smithi107.stdout:1/528: link d4/df/d1a/d26/d72/daa/f40 d4/df/d1a/d26/d7a/d81/fad 0 2024-05-17T21:52:54.539 INFO:tasks.workunit.client.0.smithi107.stdout:1/529: stat d4/df/d1a/d26/f60 0 2024-05-17T21:52:54.539 INFO:tasks.workunit.client.1.smithi138.stdout:3/376: mknod d1/d4f/d50/c83 0 2024-05-17T21:52:54.542 INFO:tasks.workunit.client.1.smithi138.stdout:1/284: rmdir d4/df/d35/d5e 0 2024-05-17T21:52:54.544 INFO:tasks.workunit.client.0.smithi107.stdout:6/382: write d9/d16/d22/d2d/f5c [1041774,124825] 0 2024-05-17T21:52:54.546 INFO:tasks.workunit.client.1.smithi138.stdout:3/377: mknod d1/d62/c84 0 2024-05-17T21:52:54.546 INFO:tasks.workunit.client.1.smithi138.stdout:3/378: chown d1/d3/d8 19 1 2024-05-17T21:52:54.547 INFO:tasks.workunit.client.1.smithi138.stdout:3/379: stat d1/d3/d8/c2b 0 2024-05-17T21:52:54.547 INFO:tasks.workunit.client.0.smithi107.stdout:4/490: mknod d3/db/d35/ca7 0 2024-05-17T21:52:54.548 INFO:tasks.workunit.client.1.smithi138.stdout:3/380: chown d1/d3/d8/f28 0 1 2024-05-17T21:52:54.548 INFO:tasks.workunit.client.0.smithi107.stdout:4/491: fdatasync d3/db/d35/f81 0 2024-05-17T21:52:54.549 INFO:tasks.workunit.client.0.smithi107.stdout:7/436: symlink d5/df/d11/d1d/d44/l9b 0 2024-05-17T21:52:54.554 INFO:tasks.workunit.client.0.smithi107.stdout:9/485: getdents d0/dd/d2f/d81 0 2024-05-17T21:52:54.554 INFO:tasks.workunit.client.0.smithi107.stdout:1/530: dread d4/f21 [4194304,4194304] 0 2024-05-17T21:52:54.554 INFO:tasks.workunit.client.1.smithi138.stdout:1/285: mkdir d4/db/d60 0 2024-05-17T21:52:54.554 INFO:tasks.workunit.client.1.smithi138.stdout:3/381: creat d1/d3/d49/d4b/f85 x:0 0 0 2024-05-17T21:52:54.554 INFO:tasks.workunit.client.1.smithi138.stdout:6/338: truncate d7/db/d43/d3f/f56 764801 0 2024-05-17T21:52:54.555 INFO:tasks.workunit.client.0.smithi107.stdout:5/486: fsync d5/db/dc/f20 0 2024-05-17T21:52:54.555 INFO:tasks.workunit.client.1.smithi138.stdout:3/382: dread d1/d3/f11 [0,4194304] 0 2024-05-17T21:52:54.556 INFO:tasks.workunit.client.1.smithi138.stdout:7/232: mknod d0/c3c 0 2024-05-17T21:52:54.559 INFO:tasks.workunit.client.1.smithi138.stdout:2/328: truncate d3/d21/d2f/f45 1139081 0 2024-05-17T21:52:54.560 INFO:tasks.workunit.client.0.smithi107.stdout:1/531: dwrite d4/df/d1a/d26/d7a/fa5 [0,4194304] 0 2024-05-17T21:52:54.561 INFO:tasks.workunit.client.0.smithi107.stdout:1/532: truncate d4/d78/fab 393904 0 2024-05-17T21:52:54.561 INFO:tasks.workunit.client.1.smithi138.stdout:9/292: truncate d6/f24 426670 0 2024-05-17T21:52:54.563 INFO:tasks.workunit.client.0.smithi107.stdout:1/533: read d4/df/d1a/f7b [337066,37050] 0 2024-05-17T21:52:54.567 INFO:tasks.workunit.client.1.smithi138.stdout:7/233: fdatasync d0/f29 0 2024-05-17T21:52:54.568 INFO:tasks.workunit.client.1.smithi138.stdout:8/197: truncate d0/f17 5501549 0 2024-05-17T21:52:54.569 INFO:tasks.workunit.client.1.smithi138.stdout:3/383: dread d1/d3/d8/f28 [0,4194304] 0 2024-05-17T21:52:54.570 INFO:tasks.workunit.client.0.smithi107.stdout:6/383: rename d9/d58/f62 to d9/d16/d22/d5f/d6f/f76 0 2024-05-17T21:52:54.571 INFO:tasks.workunit.client.0.smithi107.stdout:6/384: readlink d9/d16/d22/d2d/l36 0 2024-05-17T21:52:54.572 INFO:tasks.workunit.client.0.smithi107.stdout:6/385: write d9/d16/d4d/f60 [339486,25478] 0 2024-05-17T21:52:54.574 INFO:tasks.workunit.client.1.smithi138.stdout:2/329: rename d3/f57 to d3/d21/f67 0 2024-05-17T21:52:54.574 INFO:tasks.workunit.client.1.smithi138.stdout:2/330: fdatasync d3/f4 0 2024-05-17T21:52:54.575 INFO:tasks.workunit.client.0.smithi107.stdout:4/492: symlink d3/d4a/d47/la8 0 2024-05-17T21:52:54.576 INFO:tasks.workunit.client.0.smithi107.stdout:4/493: stat d3/db/c97 0 2024-05-17T21:52:54.579 INFO:tasks.workunit.client.0.smithi107.stdout:9/486: mkdir d0/dd/d1f/dab 0 2024-05-17T21:52:54.580 INFO:tasks.workunit.client.0.smithi107.stdout:9/487: truncate d0/dd/d21/d29/d3e/f91 624262 0 2024-05-17T21:52:54.582 INFO:tasks.workunit.client.0.smithi107.stdout:5/487: creat d5/db/d16/d5b/f97 x:0 0 0 2024-05-17T21:52:54.586 INFO:tasks.workunit.client.0.smithi107.stdout:1/534: symlink d4/df/d4e/d59/lae 0 2024-05-17T21:52:54.586 INFO:tasks.workunit.client.0.smithi107.stdout:5/488: read - d5/db/d16/d2d/d40/d5a/d7c/d8e/f90 zero size 2024-05-17T21:52:54.590 INFO:tasks.workunit.client.0.smithi107.stdout:8/475: dwrite dc/d21/d27/f5b [0,4194304] 0 2024-05-17T21:52:54.592 INFO:tasks.workunit.client.0.smithi107.stdout:0/557: dwrite d3/d24/d6f/d9f/dab/f70 [0,4194304] 0 2024-05-17T21:52:54.595 INFO:tasks.workunit.client.0.smithi107.stdout:5/489: read d5/db/d16/d5b/f2f [71260,111871] 0 2024-05-17T21:52:54.595 INFO:tasks.workunit.client.0.smithi107.stdout:7/437: truncate d5/fe 1390122 0 2024-05-17T21:52:54.596 INFO:tasks.workunit.client.0.smithi107.stdout:0/558: fsync d3/f11 0 2024-05-17T21:52:54.598 INFO:tasks.workunit.client.1.smithi138.stdout:7/234: rename d0/f29 to d0/d39/f3d 0 2024-05-17T21:52:54.598 INFO:tasks.workunit.client.1.smithi138.stdout:1/286: dwrite d4/db/f4b [0,4194304] 0 2024-05-17T21:52:54.599 INFO:tasks.workunit.client.1.smithi138.stdout:1/287: dread - d4/df/d4e/f5c zero size 2024-05-17T21:52:54.600 INFO:tasks.workunit.client.0.smithi107.stdout:9/488: mkdir d0/dd/d21/d29/d57/d5f/dac 0 2024-05-17T21:52:54.600 INFO:tasks.workunit.client.0.smithi107.stdout:3/494: dwrite d6/d8/d10/f30 [0,4194304] 0 2024-05-17T21:52:54.601 INFO:tasks.workunit.client.0.smithi107.stdout:3/495: dread - d6/d2a/f6a zero size 2024-05-17T21:52:54.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:54 smithi107 ceph-mon[28434]: pgmap v35: 65 pgs: 65 active+clean; 1.2 GiB data, 5.1 GiB used, 531 GiB / 536 GiB avail; 25 MiB/s rd, 143 MiB/s wr, 274 op/s 2024-05-17T21:52:54.605 INFO:tasks.workunit.client.1.smithi138.stdout:1/288: dread d4/df/f2c [0,4194304] 0 2024-05-17T21:52:54.606 INFO:tasks.workunit.client.1.smithi138.stdout:3/384: creat d1/d4f/d80/f86 x:0 0 0 2024-05-17T21:52:54.608 INFO:tasks.workunit.client.1.smithi138.stdout:3/385: write d1/d3/d16/d3e/f4e [1871995,71154] 0 2024-05-17T21:52:54.609 INFO:tasks.workunit.client.1.smithi138.stdout:2/331: mknod d3/d21/d2f/d3e/d63/d65/c68 0 2024-05-17T21:52:54.609 INFO:tasks.workunit.client.1.smithi138.stdout:3/386: chown d1/d3/l70 243271 1 2024-05-17T21:52:54.611 INFO:tasks.workunit.client.1.smithi138.stdout:3/387: chown d1/d3/d16/d3e/f4e 0 1 2024-05-17T21:52:54.612 INFO:tasks.workunit.client.0.smithi107.stdout:9/489: dwrite d0/dd/d2f/f5b [0,4194304] 0 2024-05-17T21:52:54.614 INFO:tasks.workunit.client.1.smithi138.stdout:6/339: getdents d7/db/d43 0 2024-05-17T21:52:54.614 INFO:tasks.workunit.client.0.smithi107.stdout:6/386: mknod d9/d16/d22/c77 0 2024-05-17T21:52:54.615 INFO:tasks.workunit.client.1.smithi138.stdout:6/340: fsync d7/db/f52 0 2024-05-17T21:52:54.615 INFO:tasks.workunit.client.1.smithi138.stdout:9/293: link d6/d7/d16/d29/c35 d6/d7/d16/d29/d48/d3b/c5d 0 2024-05-17T21:52:54.616 INFO:tasks.workunit.client.1.smithi138.stdout:6/341: readlink d7/db/l38 0 2024-05-17T21:52:54.617 INFO:tasks.workunit.client.0.smithi107.stdout:6/387: truncate d9/d16/d22/d2d/f39 1390300 0 2024-05-17T21:52:54.617 INFO:tasks.workunit.client.0.smithi107.stdout:3/496: dwrite d6/d8/d3e/d3f/f87 [0,4194304] 0 2024-05-17T21:52:54.617 INFO:tasks.workunit.client.1.smithi138.stdout:9/294: chown d6/d7/d16/d29/d48/f23 1 1 2024-05-17T21:52:54.618 INFO:tasks.workunit.client.1.smithi138.stdout:9/295: fdatasync f5 0 2024-05-17T21:52:54.619 INFO:tasks.workunit.client.0.smithi107.stdout:5/490: rmdir d5/db/d16 39 2024-05-17T21:52:54.657 INFO:tasks.workunit.client.0.smithi107.stdout:1/535: mkdir d4/da7/daf 0 2024-05-17T21:52:54.658 INFO:tasks.workunit.client.1.smithi138.stdout:1/289: creat d4/df/d35/f61 x:0 0 0 2024-05-17T21:52:54.658 INFO:tasks.workunit.client.0.smithi107.stdout:1/536: fdatasync d4/df/d1a/d26/f60 0 2024-05-17T21:52:54.659 INFO:tasks.workunit.client.0.smithi107.stdout:9/490: write d0/dd/d3d/f8e [392183,28466] 0 2024-05-17T21:52:54.662 INFO:tasks.workunit.client.0.smithi107.stdout:8/476: symlink dc/d21/d35/dab/lb0 0 2024-05-17T21:52:54.662 INFO:tasks.workunit.client.0.smithi107.stdout:1/537: read d4/df/d1a/d26/d72/daa/f9d [7396040,60809] 0 2024-05-17T21:52:54.664 INFO:tasks.workunit.client.0.smithi107.stdout:4/494: dwrite d3/d4a/d47/d60/d3a/d41/f5a [0,4194304] 0 2024-05-17T21:52:54.664 INFO:tasks.workunit.client.1.smithi138.stdout:8/198: write d0/f2 [135724,834] 0 2024-05-17T21:52:54.665 INFO:tasks.workunit.client.0.smithi107.stdout:1/538: write d4/df/d1a/d26/d50/fa0 [818550,97911] 0 2024-05-17T21:52:54.665 INFO:tasks.workunit.client.0.smithi107.stdout:4/495: readlink d3/d4a/l86 0 2024-05-17T21:52:54.666 INFO:tasks.workunit.client.0.smithi107.stdout:3/497: symlink d6/d8/d10/d35/d36/d66/la1 0 2024-05-17T21:52:54.671 INFO:tasks.workunit.client.0.smithi107.stdout:7/438: creat d5/df/d11/d1c/d31/d56/f9c x:0 0 0 2024-05-17T21:52:54.673 INFO:tasks.workunit.client.0.smithi107.stdout:0/559: link d3/d75/d9e/f83 d3/dc/d22/d3d/dc7/fcb 0 2024-05-17T21:52:54.674 INFO:tasks.workunit.client.0.smithi107.stdout:7/439: chown d5/d16/d3b/d45/l43 15227 1 2024-05-17T21:52:54.675 INFO:tasks.workunit.client.1.smithi138.stdout:9/296: symlink d6/d7/d16/d29/d38/d3c/d3e/l5e 0 2024-05-17T21:52:54.677 INFO:tasks.workunit.client.0.smithi107.stdout:8/477: dwrite dc/d44/d4b/d52/f96 [0,4194304] 0 2024-05-17T21:52:54.677 INFO:tasks.workunit.client.1.smithi138.stdout:7/235: creat d0/d18/f3e x:0 0 0 2024-05-17T21:52:54.678 INFO:tasks.workunit.client.0.smithi107.stdout:8/478: chown dc/d21/d97/c1e 4623936 1 2024-05-17T21:52:54.679 INFO:tasks.workunit.client.0.smithi107.stdout:8/479: dread - dc/d44/d4b/d52/d70/fa4 zero size 2024-05-17T21:52:54.687 INFO:tasks.workunit.client.1.smithi138.stdout:2/332: mkdir d3/d21/d2f/d3e/d69 0 2024-05-17T21:52:54.689 INFO:tasks.workunit.client.1.smithi138.stdout:6/342: write d7/db/d43/f59 [1598546,16326] 0 2024-05-17T21:52:54.691 INFO:tasks.workunit.client.1.smithi138.stdout:3/388: mkdir d1/d4f/d87 0 2024-05-17T21:52:54.694 INFO:tasks.workunit.client.1.smithi138.stdout:6/343: read d7/db/d43/f64 [2374368,20628] 0 2024-05-17T21:52:54.694 INFO:tasks.workunit.client.1.smithi138.stdout:2/333: dwrite d3/fa [0,4194304] 0 2024-05-17T21:52:54.695 INFO:tasks.workunit.client.0.smithi107.stdout:6/388: dread d9/f1a [0,4194304] 0 2024-05-17T21:52:54.696 INFO:tasks.workunit.client.0.smithi107.stdout:6/389: truncate d9/f1f 1515637 0 2024-05-17T21:52:54.696 INFO:tasks.workunit.client.1.smithi138.stdout:3/389: truncate d1/d3/d16/d3e/f74 594476 0 2024-05-17T21:52:54.697 INFO:tasks.workunit.client.1.smithi138.stdout:9/297: dread d6/f2a [0,4194304] 0 2024-05-17T21:52:54.701 INFO:tasks.workunit.client.0.smithi107.stdout:3/498: mknod d6/d2a/d2d/d37/ca2 0 2024-05-17T21:52:54.702 INFO:tasks.workunit.client.0.smithi107.stdout:3/499: write d6/d8/d10/d35/d36/f8e [5117232,62238] 0 2024-05-17T21:52:54.702 INFO:tasks.workunit.client.1.smithi138.stdout:7/236: chown d0/l14 3243 1 2024-05-17T21:52:54.704 INFO:tasks.workunit.client.1.smithi138.stdout:3/390: read d1/d3/d16/d3e/f4e [4083642,46794] 0 2024-05-17T21:52:54.704 INFO:tasks.workunit.client.0.smithi107.stdout:4/496: mkdir d3/d4a/d47/d60/d3a/d41/d80/da9 0 2024-05-17T21:52:54.706 INFO:tasks.workunit.client.1.smithi138.stdout:8/199: dwrite d0/dc/d1a/f24 [0,4194304] 0 2024-05-17T21:52:54.707 INFO:tasks.workunit.client.0.smithi107.stdout:3/500: dwrite d6/d8/f14 [0,4194304] 0 2024-05-17T21:52:54.708 INFO:tasks.workunit.client.1.smithi138.stdout:1/290: creat d4/db/f62 x:0 0 0 2024-05-17T21:52:54.709 INFO:tasks.workunit.client.1.smithi138.stdout:1/291: dread - d4/df/d35/f61 zero size 2024-05-17T21:52:54.717 INFO:tasks.workunit.client.0.smithi107.stdout:9/491: mkdir d0/d80/dad 0 2024-05-17T21:52:54.718 INFO:tasks.workunit.client.1.smithi138.stdout:2/334: rmdir d3/d21/d2f/d3e/d55 39 2024-05-17T21:52:54.720 INFO:tasks.workunit.client.1.smithi138.stdout:7/237: write d0/f1e [508599,100631] 0 2024-05-17T21:52:54.721 INFO:tasks.workunit.client.0.smithi107.stdout:8/480: mkdir dc/d21/d27/d34/d4c/db1 0 2024-05-17T21:52:54.722 INFO:tasks.workunit.client.1.smithi138.stdout:9/298: symlink d6/d7/d16/d29/d48/d3b/l5f 0 2024-05-17T21:52:54.725 INFO:tasks.workunit.client.0.smithi107.stdout:6/390: unlink d9/d16/l6a 0 2024-05-17T21:52:54.725 INFO:tasks.workunit.client.1.smithi138.stdout:8/200: mknod d0/d31/c38 0 2024-05-17T21:52:54.727 INFO:tasks.workunit.client.1.smithi138.stdout:1/292: fsync d4/dd/f2d 0 2024-05-17T21:52:54.729 INFO:tasks.workunit.client.1.smithi138.stdout:1/293: chown d4/df/c30 15474291 1 2024-05-17T21:52:54.730 INFO:tasks.workunit.client.0.smithi107.stdout:4/497: creat d3/d4a/d47/d60/d3a/d41/d80/d43/faa x:0 0 0 2024-05-17T21:52:54.731 INFO:tasks.workunit.client.0.smithi107.stdout:3/501: rmdir d6/d8/d3e/d3f/d90 39 2024-05-17T21:52:54.732 INFO:tasks.workunit.client.1.smithi138.stdout:8/201: dwrite d0/dc/d1a/f24 [4194304,4194304] 0 2024-05-17T21:52:54.735 INFO:tasks.workunit.client.0.smithi107.stdout:7/440: write d5/df/f20 [4235100,23064] 0 2024-05-17T21:52:54.736 INFO:tasks.workunit.client.1.smithi138.stdout:7/238: unlink d0/d18/f3e 0 2024-05-17T21:52:54.738 INFO:tasks.workunit.client.1.smithi138.stdout:9/299: write d6/f49 [33386,77841] 0 2024-05-17T21:52:54.739 INFO:tasks.workunit.client.0.smithi107.stdout:3/502: dwrite d6/d1a/d57/f73 [0,4194304] 0 2024-05-17T21:52:54.739 INFO:tasks.workunit.client.0.smithi107.stdout:0/560: creat d3/dc/d22/d3d/dc7/fcc x:0 0 0 2024-05-17T21:52:54.740 INFO:tasks.workunit.client.1.smithi138.stdout:3/391: creat d1/d4f/d87/f88 x:0 0 0 2024-05-17T21:52:54.743 INFO:tasks.workunit.client.0.smithi107.stdout:0/561: chown d3/d58/d61/c66 1 1 2024-05-17T21:52:54.743 INFO:tasks.workunit.client.0.smithi107.stdout:9/492: rename d0/dd/d21/d29/d3e/f5d to d0/dd/d2f/d87/d89/fae 0 2024-05-17T21:52:54.744 INFO:tasks.workunit.client.0.smithi107.stdout:8/481: dread dc/d21/d35/f22 [4194304,4194304] 0 2024-05-17T21:52:54.745 INFO:tasks.workunit.client.0.smithi107.stdout:0/562: dread - d3/dc/d22/fc5 zero size 2024-05-17T21:52:54.745 INFO:tasks.workunit.client.1.smithi138.stdout:6/344: dwrite d7/db/d43/d3f/f4c [0,4194304] 0 2024-05-17T21:52:54.746 INFO:tasks.workunit.client.0.smithi107.stdout:0/563: fsync d3/dc/d22/d3d/d56/f5b 0 2024-05-17T21:52:54.746 INFO:tasks.workunit.client.0.smithi107.stdout:7/441: dwrite d5/df/d11/d1d/f55 [0,4194304] 0 2024-05-17T21:52:54.749 INFO:tasks.workunit.client.1.smithi138.stdout:9/300: dwrite d6/d7/d16/f20 [4194304,4194304] 0 2024-05-17T21:52:54.754 INFO:tasks.workunit.client.0.smithi107.stdout:7/442: dwrite d5/df/d11/d1c/d2f/d65/f69 [0,4194304] 0 2024-05-17T21:52:54.759 INFO:tasks.workunit.client.0.smithi107.stdout:4/498: symlink d3/d4a/d47/d60/lab 0 2024-05-17T21:52:54.760 INFO:tasks.workunit.client.1.smithi138.stdout:2/335: mknod d3/d21/d2f/d3e/d55/c6a 0 2024-05-17T21:52:54.763 INFO:tasks.workunit.client.0.smithi107.stdout:3/503: symlink d6/d1a/d57/la3 0 2024-05-17T21:52:54.764 INFO:tasks.workunit.client.1.smithi138.stdout:5/300: sync 2024-05-17T21:52:54.764 INFO:tasks.workunit.client.0.smithi107.stdout:1/539: truncate d4/df/d1a/d26/d7a/fa5 3681605 0 2024-05-17T21:52:54.767 INFO:tasks.workunit.client.0.smithi107.stdout:8/482: dread dc/d44/f84 [0,4194304] 0 2024-05-17T21:52:54.769 INFO:tasks.workunit.client.0.smithi107.stdout:9/493: dread d0/dd/f60 [4194304,4194304] 0 2024-05-17T21:52:54.773 INFO:tasks.workunit.client.1.smithi138.stdout:5/301: dwrite d1/d6/d19/f20 [0,4194304] 0 2024-05-17T21:52:54.773 INFO:tasks.workunit.client.0.smithi107.stdout:1/540: dread d4/df/d1a/d26/d72/daa/f40 [0,4194304] 0 2024-05-17T21:52:54.774 INFO:tasks.workunit.client.1.smithi138.stdout:3/392: rename d1/d3/d8/de/c45 to d1/d3/d8/d72/c89 0 2024-05-17T21:52:54.775 INFO:tasks.workunit.client.0.smithi107.stdout:1/541: dread d4/df/d14/d2e/d43/f51 [0,4194304] 0 2024-05-17T21:52:54.775 INFO:tasks.workunit.client.0.smithi107.stdout:6/391: truncate d9/d16/d22/f33 71838 0 2024-05-17T21:52:54.777 INFO:tasks.workunit.client.0.smithi107.stdout:0/564: dwrite d3/dc/f37 [0,4194304] 0 2024-05-17T21:52:54.778 INFO:tasks.workunit.client.1.smithi138.stdout:6/345: creat d7/db/d43/d5f/f73 x:0 0 0 2024-05-17T21:52:54.779 INFO:tasks.workunit.client.1.smithi138.stdout:6/346: read - d7/f22 zero size 2024-05-17T21:52:54.780 INFO:tasks.workunit.client.0.smithi107.stdout:4/499: symlink d3/db/lac 0 2024-05-17T21:52:54.780 INFO:tasks.workunit.client.0.smithi107.stdout:4/500: chown d3/d4a/f75 139871119 1 2024-05-17T21:52:54.781 INFO:tasks.workunit.client.1.smithi138.stdout:8/202: symlink d0/d2f/l39 0 2024-05-17T21:52:54.784 INFO:tasks.workunit.client.1.smithi138.stdout:2/336: creat d3/d21/d2f/d3e/d58/f6b x:0 0 0 2024-05-17T21:52:54.785 INFO:tasks.workunit.client.1.smithi138.stdout:2/337: fdatasync d3/f1c 0 2024-05-17T21:52:54.786 INFO:tasks.workunit.client.0.smithi107.stdout:8/483: write dc/d21/f92 [126781,110] 0 2024-05-17T21:52:54.787 INFO:tasks.workunit.client.1.smithi138.stdout:7/239: symlink d0/l3f 0 2024-05-17T21:52:54.787 INFO:tasks.workunit.client.0.smithi107.stdout:2/341: sync 2024-05-17T21:52:54.788 INFO:tasks.workunit.client.0.smithi107.stdout:1/542: dread d4/df/d1a/f58 [0,4194304] 0 2024-05-17T21:52:54.789 INFO:tasks.workunit.client.1.smithi138.stdout:1/294: dread f2 [0,4194304] 0 2024-05-17T21:52:54.789 INFO:tasks.workunit.client.1.smithi138.stdout:8/203: dread d0/dc/d1a/f1d [0,4194304] 0 2024-05-17T21:52:54.791 INFO:tasks.workunit.client.1.smithi138.stdout:3/393: symlink d1/d3/d8/d72/l8a 0 2024-05-17T21:52:54.794 INFO:tasks.workunit.client.1.smithi138.stdout:8/204: dread d0/f17 [0,4194304] 0 2024-05-17T21:52:54.796 INFO:tasks.workunit.client.0.smithi107.stdout:1/543: dwrite d4/df/d1a/d26/d72/d85/f92 [0,4194304] 0 2024-05-17T21:52:54.798 INFO:tasks.workunit.client.1.smithi138.stdout:6/347: rename d7/d8/l29 to d7/db/d34/l74 0 2024-05-17T21:52:54.799 INFO:tasks.workunit.client.1.smithi138.stdout:6/348: fsync d7/db/d43/d3c/f6c 0 2024-05-17T21:52:54.802 INFO:tasks.workunit.client.1.smithi138.stdout:8/205: dwrite d0/dc/d1a/f1d [0,4194304] 0 2024-05-17T21:52:54.803 INFO:tasks.workunit.client.0.smithi107.stdout:2/342: dread d0/d9/f33 [0,4194304] 0 2024-05-17T21:52:54.803 INFO:tasks.workunit.client.0.smithi107.stdout:2/343: chown d0/f36 96008 1 2024-05-17T21:52:54.804 INFO:tasks.workunit.client.0.smithi107.stdout:2/344: truncate d0/d9/d1c/d22/f5b 1041602 0 2024-05-17T21:52:54.805 INFO:tasks.workunit.client.0.smithi107.stdout:7/443: symlink d5/l9d 0 2024-05-17T21:52:54.805 INFO:tasks.workunit.client.0.smithi107.stdout:2/345: dread - d0/d9/d1c/d37/f50 zero size 2024-05-17T21:52:54.807 INFO:tasks.workunit.client.1.smithi138.stdout:7/240: write d0/f11 [4339767,69109] 0 2024-05-17T21:52:54.808 INFO:tasks.workunit.client.1.smithi138.stdout:8/206: dread d0/fa [0,4194304] 0 2024-05-17T21:52:54.808 INFO:tasks.workunit.client.0.smithi107.stdout:4/501: mkdir d3/d4a/d47/d60/d3a/d41/d80/dad 0 2024-05-17T21:52:54.810 INFO:tasks.workunit.client.0.smithi107.stdout:4/502: write d3/d4a/d47/f4b [1347030,103104] 0 2024-05-17T21:52:54.813 INFO:tasks.workunit.client.1.smithi138.stdout:1/295: fsync d4/fc 0 2024-05-17T21:52:54.818 INFO:tasks.workunit.client.0.smithi107.stdout:4/503: dwrite d3/f2c [0,4194304] 0 2024-05-17T21:52:54.820 INFO:tasks.workunit.client.0.smithi107.stdout:9/494: truncate d0/dd/d2f/d87/d89/fae 863569 0 2024-05-17T21:52:54.821 INFO:tasks.workunit.client.0.smithi107.stdout:9/495: stat d0/dd/d69 0 2024-05-17T21:52:54.821 INFO:tasks.workunit.client.1.smithi138.stdout:6/349: symlink d7/db/d60/l75 0 2024-05-17T21:52:54.823 INFO:tasks.workunit.client.1.smithi138.stdout:7/241: rmdir d0/d39 39 2024-05-17T21:52:54.824 INFO:tasks.workunit.client.0.smithi107.stdout:3/504: dwrite d6/d2a/d2d/d69/f6d [4194304,4194304] 0 2024-05-17T21:52:54.825 INFO:tasks.workunit.client.0.smithi107.stdout:0/565: mkdir d3/d24/dcd 0 2024-05-17T21:52:54.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:54 smithi138 ceph-mon[35948]: pgmap v35: 65 pgs: 65 active+clean; 1.2 GiB data, 5.1 GiB used, 531 GiB / 536 GiB avail; 25 MiB/s rd, 143 MiB/s wr, 274 op/s 2024-05-17T21:52:54.827 INFO:tasks.workunit.client.1.smithi138.stdout:4/343: sync 2024-05-17T21:52:54.827 INFO:tasks.workunit.client.1.smithi138.stdout:0/360: sync 2024-05-17T21:52:54.827 INFO:tasks.workunit.client.1.smithi138.stdout:0/361: readlink d3/l60 0 2024-05-17T21:52:54.827 INFO:tasks.workunit.client.0.smithi107.stdout:7/444: mkdir d5/df/d11/d2a/d9e 0 2024-05-17T21:52:54.828 INFO:tasks.workunit.client.1.smithi138.stdout:0/362: dread d3/d10/d39/f72 [0,4194304] 0 2024-05-17T21:52:54.828 INFO:tasks.workunit.client.0.smithi107.stdout:7/445: write d5/d16/f28 [4764786,112163] 0 2024-05-17T21:52:54.829 INFO:tasks.workunit.client.0.smithi107.stdout:6/392: write d9/d16/d4d/f52 [1649002,7436] 0 2024-05-17T21:52:54.829 INFO:tasks.workunit.client.1.smithi138.stdout:0/363: dread d3/d10/d39/f72 [0,4194304] 0 2024-05-17T21:52:54.836 INFO:tasks.workunit.client.1.smithi138.stdout:9/301: getdents d6/d7/d16/d29/d45 0 2024-05-17T21:52:54.840 INFO:tasks.workunit.client.1.smithi138.stdout:9/302: dread d6/d7/d16/d29/f3a [0,4194304] 0 2024-05-17T21:52:54.840 INFO:tasks.workunit.client.1.smithi138.stdout:6/350: unlink d7/d8/c1e 0 2024-05-17T21:52:54.842 INFO:tasks.workunit.client.1.smithi138.stdout:7/242: mkdir d0/d18/d40 0 2024-05-17T21:52:54.843 INFO:tasks.workunit.client.1.smithi138.stdout:7/243: chown d0/d18/f1d 1 1 2024-05-17T21:52:54.843 INFO:tasks.workunit.client.1.smithi138.stdout:5/302: getdents d1/d60/d2f 0 2024-05-17T21:52:54.844 INFO:tasks.workunit.client.1.smithi138.stdout:9/303: read d6/d7/d16/d29/d48/f23 [2898216,103588] 0 2024-05-17T21:52:54.844 INFO:tasks.workunit.client.1.smithi138.stdout:5/303: chown d1/c4a 2 1 2024-05-17T21:52:54.848 INFO:tasks.workunit.client.1.smithi138.stdout:2/338: rename d3/l24 to d3/d21/d2f/d3e/l6c 0 2024-05-17T21:52:54.849 INFO:tasks.workunit.client.1.smithi138.stdout:6/351: symlink d7/d69/l76 0 2024-05-17T21:52:54.852 INFO:tasks.workunit.client.1.smithi138.stdout:4/344: creat d0/d1b/d2b/d55/d79/f7b x:0 0 0 2024-05-17T21:52:54.856 INFO:tasks.workunit.client.1.smithi138.stdout:3/394: truncate d1/d3/f25 3416936 0 2024-05-17T21:52:54.857 INFO:tasks.workunit.client.1.smithi138.stdout:3/395: stat d1/d3/d16/d3a 0 2024-05-17T21:52:54.858 INFO:tasks.workunit.client.1.smithi138.stdout:3/396: fdatasync d1/d4f/d80/f86 0 2024-05-17T21:52:54.858 INFO:tasks.workunit.client.1.smithi138.stdout:7/244: mkdir d0/d18/d40/d41 0 2024-05-17T21:52:54.859 INFO:tasks.workunit.client.1.smithi138.stdout:1/296: dwrite d4/df/f2c [4194304,4194304] 0 2024-05-17T21:52:54.860 INFO:tasks.workunit.client.1.smithi138.stdout:9/304: symlink d6/d7/d16/d29/d38/d3c/d3e/d5a/l60 0 2024-05-17T21:52:54.861 INFO:tasks.workunit.client.1.smithi138.stdout:4/345: stat d0/d3/d17/d21/c4a 0 2024-05-17T21:52:54.862 INFO:tasks.workunit.client.1.smithi138.stdout:4/346: fsync d0/d2f/f6f 0 2024-05-17T21:52:54.863 INFO:tasks.workunit.client.1.smithi138.stdout:6/352: mkdir d7/db/d43/d5f/d6a/d77 0 2024-05-17T21:52:54.864 INFO:tasks.workunit.client.1.smithi138.stdout:6/353: chown d7/d8/f70 82834158 1 2024-05-17T21:52:54.865 INFO:tasks.workunit.client.0.smithi107.stdout:5/491: sync 2024-05-17T21:52:54.867 INFO:tasks.workunit.client.1.smithi138.stdout:3/397: truncate d1/d3/f11 579839 0 2024-05-17T21:52:54.869 INFO:tasks.workunit.client.1.smithi138.stdout:4/347: write d0/d2f/d49/d60/f51 [3830149,102521] 0 2024-05-17T21:52:54.871 INFO:tasks.workunit.client.1.smithi138.stdout:6/354: mkdir d7/d8/d78 0 2024-05-17T21:52:54.874 INFO:tasks.workunit.client.1.smithi138.stdout:7/245: mknod d0/d39/c42 0 2024-05-17T21:52:54.875 INFO:tasks.workunit.client.1.smithi138.stdout:1/297: mkdir d4/df/d3a/d3c/d63 0 2024-05-17T21:52:54.875 INFO:tasks.workunit.client.1.smithi138.stdout:6/355: stat d7/db/d34/f53 0 2024-05-17T21:52:54.876 INFO:tasks.workunit.client.1.smithi138.stdout:5/304: dwrite d1/f13 [0,4194304] 0 2024-05-17T21:52:54.880 INFO:tasks.workunit.client.1.smithi138.stdout:4/348: rename d0/d2f/f6a to d0/d1b/d42/f7c 0 2024-05-17T21:52:54.881 INFO:tasks.workunit.client.1.smithi138.stdout:2/339: link d3/d21/d2f/f45 d3/d21/d2f/d3e/f6d 0 2024-05-17T21:52:54.882 INFO:tasks.workunit.client.1.smithi138.stdout:3/398: dwrite d1/d3/d16/d3e/d71/f7b [0,4194304] 0 2024-05-17T21:52:54.885 INFO:tasks.workunit.client.0.smithi107.stdout:2/346: unlink d0/d9/d11/f14 0 2024-05-17T21:52:54.887 INFO:tasks.workunit.client.0.smithi107.stdout:9/496: mkdir d0/dd/d21/daf 0 2024-05-17T21:52:54.888 INFO:tasks.workunit.client.0.smithi107.stdout:3/505: read - d6/d8/d3e/d3f/d90/f9f zero size 2024-05-17T21:52:54.889 INFO:tasks.workunit.client.1.smithi138.stdout:1/298: dwrite d4/df/f12 [0,4194304] 0 2024-05-17T21:52:54.890 INFO:tasks.workunit.client.0.smithi107.stdout:9/497: truncate d0/dd/d21/d29/d3e/d45/d68/fa1 854145 0 2024-05-17T21:52:54.892 INFO:tasks.workunit.client.1.smithi138.stdout:3/399: dread d1/d3/d16/d3e/f4a [0,4194304] 0 2024-05-17T21:52:54.893 INFO:tasks.workunit.client.1.smithi138.stdout:3/400: readlink d1/d3/l17 0 2024-05-17T21:52:54.894 INFO:tasks.workunit.client.1.smithi138.stdout:2/340: creat d3/d21/d2f/d3e/d63/f6e x:0 0 0 2024-05-17T21:52:54.895 INFO:tasks.workunit.client.1.smithi138.stdout:2/341: read - d3/d21/f59 zero size 2024-05-17T21:52:54.896 INFO:tasks.workunit.client.0.smithi107.stdout:9/498: dwrite d0/dd/d21/d29/d3e/d45/d68/f7a [0,4194304] 0 2024-05-17T21:52:54.898 INFO:tasks.workunit.client.0.smithi107.stdout:2/347: rmdir d0 39 2024-05-17T21:52:54.900 INFO:tasks.workunit.client.1.smithi138.stdout:7/246: rename d0/f2c to d0/d18/d40/d41/f43 0 2024-05-17T21:52:54.903 INFO:tasks.workunit.client.1.smithi138.stdout:5/305: mkdir d1/d1a/d39/d4e/d58/d5b/d65 0 2024-05-17T21:52:54.904 INFO:tasks.workunit.client.0.smithi107.stdout:6/393: fsync d9/f53 0 2024-05-17T21:52:54.908 INFO:tasks.workunit.client.0.smithi107.stdout:1/544: link d4/de/l4a d4/df/d14/d2e/d43/d47/lb0 0 2024-05-17T21:52:54.911 INFO:tasks.workunit.client.1.smithi138.stdout:5/306: dwrite d1/d6/d1b/d29/f3b [0,4194304] 0 2024-05-17T21:52:54.911 INFO:tasks.workunit.client.0.smithi107.stdout:6/394: dwrite d9/d16/d4d/f52 [0,4194304] 0 2024-05-17T21:52:54.912 INFO:tasks.workunit.client.0.smithi107.stdout:7/446: creat d5/df/d11/d1c/d2f/f9f x:0 0 0 2024-05-17T21:52:54.913 INFO:tasks.workunit.client.0.smithi107.stdout:7/447: dread - d5/df/d11/d1c/d31/d56/f9c zero size 2024-05-17T21:52:54.914 INFO:tasks.workunit.client.0.smithi107.stdout:5/492: mknod d5/db/dc/c98 0 2024-05-17T21:52:54.914 INFO:tasks.workunit.client.0.smithi107.stdout:7/448: fsync d5/df/d11/d1c/d31/d39/f58 0 2024-05-17T21:52:54.917 INFO:tasks.workunit.client.0.smithi107.stdout:0/566: fsync d3/d39/fb0 0 2024-05-17T21:52:54.919 INFO:tasks.workunit.client.1.smithi138.stdout:6/356: rename d7/db/d43/d3c to d7/db/d43/d3f/d79 0 2024-05-17T21:52:54.919 INFO:tasks.workunit.client.1.smithi138.stdout:1/299: dwrite d4/df/d35/f37 [4194304,4194304] 0 2024-05-17T21:52:54.921 INFO:tasks.workunit.client.1.smithi138.stdout:6/357: write d7/d8/f5d [1010322,25156] 0 2024-05-17T21:52:54.922 INFO:tasks.workunit.client.0.smithi107.stdout:3/506: symlink d6/d8/d10/d35/d36/d66/d88/la4 0 2024-05-17T21:52:54.923 INFO:tasks.workunit.client.0.smithi107.stdout:3/507: dread - d6/d8/d3e/d3f/d90/f9f zero size 2024-05-17T21:52:54.923 INFO:tasks.workunit.client.0.smithi107.stdout:3/508: chown d6/d2a/d2d/f6c 2923 1 2024-05-17T21:52:54.925 INFO:tasks.workunit.client.1.smithi138.stdout:7/247: creat d0/d18/d40/f44 x:0 0 0 2024-05-17T21:52:54.928 INFO:tasks.workunit.client.0.smithi107.stdout:9/499: creat d0/dd/d21/d29/d3e/d94/fb0 x:0 0 0 2024-05-17T21:52:54.930 INFO:tasks.workunit.client.0.smithi107.stdout:8/484: getdents dc/d21/d27/d34/d4c/d7e 0 2024-05-17T21:52:54.933 INFO:tasks.workunit.client.1.smithi138.stdout:5/307: rename d1/d6/d1b/f5d to d1/d1a/d39/d62/f66 0 2024-05-17T21:52:54.937 INFO:tasks.workunit.client.1.smithi138.stdout:7/248: creat d0/d18/f45 x:0 0 0 2024-05-17T21:52:54.938 INFO:tasks.workunit.client.1.smithi138.stdout:2/342: creat d3/f6f x:0 0 0 2024-05-17T21:52:54.939 INFO:tasks.workunit.client.1.smithi138.stdout:2/343: dread d3/f4a [0,4194304] 0 2024-05-17T21:52:54.940 INFO:tasks.workunit.client.0.smithi107.stdout:3/509: symlink d6/d2a/la5 0 2024-05-17T21:52:54.941 INFO:tasks.workunit.client.0.smithi107.stdout:3/510: chown d6/d8/d10/d25/f47 379 1 2024-05-17T21:52:54.942 INFO:tasks.workunit.client.0.smithi107.stdout:5/493: dwrite d5/db/d16/d2d/d40/d5a/f6c [0,4194304] 0 2024-05-17T21:52:54.942 INFO:tasks.workunit.client.0.smithi107.stdout:2/348: unlink d0/d9/d1c/d37/c53 0 2024-05-17T21:52:54.943 INFO:tasks.workunit.client.0.smithi107.stdout:3/511: write d6/d8/d10/f4b [1199644,39389] 0 2024-05-17T21:52:54.944 INFO:tasks.workunit.client.1.smithi138.stdout:3/401: dread d1/d3/f25 [0,4194304] 0 2024-05-17T21:52:54.944 INFO:tasks.workunit.client.0.smithi107.stdout:9/500: symlink d0/dd/d2f/d87/d89/d9b/lb1 0 2024-05-17T21:52:54.945 INFO:tasks.workunit.client.1.smithi138.stdout:6/358: symlink d7/d8/d78/l7a 0 2024-05-17T21:52:54.946 INFO:tasks.workunit.client.0.smithi107.stdout:9/501: write d0/dd/d21/d29/f8d [223908,124263] 0 2024-05-17T21:52:54.947 INFO:tasks.workunit.client.0.smithi107.stdout:1/545: symlink d4/df/d14/d2e/d43/lb1 0 2024-05-17T21:52:54.948 INFO:tasks.workunit.client.1.smithi138.stdout:7/249: stat d0/f27 0 2024-05-17T21:52:54.949 INFO:tasks.workunit.client.1.smithi138.stdout:7/250: chown d0/d18/l37 127 1 2024-05-17T21:52:54.951 INFO:tasks.workunit.client.0.smithi107.stdout:6/395: unlink d9/d16/c25 0 2024-05-17T21:52:54.953 INFO:tasks.workunit.client.0.smithi107.stdout:6/396: dread d9/f1a [0,4194304] 0 2024-05-17T21:52:54.956 INFO:tasks.workunit.client.0.smithi107.stdout:7/449: creat d5/df/d11/d1d/d33/d82/fa0 x:0 0 0 2024-05-17T21:52:54.957 INFO:tasks.workunit.client.1.smithi138.stdout:1/300: write d4/df/f1b [950283,123386] 0 2024-05-17T21:52:54.959 INFO:tasks.workunit.client.0.smithi107.stdout:6/397: dwrite d9/d16/d22/d2d/f39 [0,4194304] 0 2024-05-17T21:52:54.962 INFO:tasks.workunit.client.0.smithi107.stdout:0/567: mknod d3/cce 0 2024-05-17T21:52:54.963 INFO:tasks.workunit.client.1.smithi138.stdout:6/359: unlink d7/db/d34/l48 0 2024-05-17T21:52:54.966 INFO:tasks.workunit.client.0.smithi107.stdout:5/494: write d5/db/d16/f81 [115799,3432] 0 2024-05-17T21:52:54.969 INFO:tasks.workunit.client.0.smithi107.stdout:0/568: dwrite d3/d75/fbc [0,4194304] 0 2024-05-17T21:52:54.969 INFO:tasks.workunit.client.0.smithi107.stdout:2/349: write d0/d9/d11/d18/f20 [1013317,48865] 0 2024-05-17T21:52:54.969 INFO:tasks.workunit.client.0.smithi107.stdout:2/350: fsync d0/d9/d11/df/f47 0 2024-05-17T21:52:54.970 INFO:tasks.workunit.client.1.smithi138.stdout:5/308: creat d1/d6/d1b/d29/f67 x:0 0 0 2024-05-17T21:52:54.972 INFO:tasks.workunit.client.1.smithi138.stdout:3/402: write d1/d3/d8/de/f3b [1726979,117019] 0 2024-05-17T21:52:54.974 INFO:tasks.workunit.client.0.smithi107.stdout:9/502: rename d0/dd/d21/d29/d57/d5f/dac to d0/dd/d69/db2 0 2024-05-17T21:52:54.976 INFO:tasks.workunit.client.1.smithi138.stdout:6/360: rmdir d7/db 39 2024-05-17T21:52:54.977 INFO:tasks.workunit.client.0.smithi107.stdout:7/450: readlink d5/df/d11/d2a/l7d 0 2024-05-17T21:52:54.978 INFO:tasks.workunit.client.0.smithi107.stdout:7/451: readlink d5/df/d11/d2a/l7d 0 2024-05-17T21:52:54.979 INFO:tasks.workunit.client.0.smithi107.stdout:7/452: truncate d5/df/d11/d1c/d31/f48 5908124 0 2024-05-17T21:52:54.979 INFO:tasks.workunit.client.0.smithi107.stdout:6/398: mknod d9/d16/d22/d5f/c78 0 2024-05-17T21:52:54.980 INFO:tasks.workunit.client.0.smithi107.stdout:6/399: stat d9/d16/d4d/d67 0 2024-05-17T21:52:54.980 INFO:tasks.workunit.client.1.smithi138.stdout:5/309: mkdir d1/d6/d1b/d29/d36/d68 0 2024-05-17T21:52:54.984 INFO:tasks.workunit.client.1.smithi138.stdout:1/301: mkdir d4/dd/d64 0 2024-05-17T21:52:54.984 INFO:tasks.workunit.client.0.smithi107.stdout:7/453: dwrite d5/df/d11/d1c/f2e [0,4194304] 0 2024-05-17T21:52:54.987 INFO:tasks.workunit.client.0.smithi107.stdout:3/512: mkdir d6/d8/d10/d35/d91/da6 0 2024-05-17T21:52:54.989 INFO:tasks.workunit.client.1.smithi138.stdout:7/251: rename d0/d18/c21 to d0/c46 0 2024-05-17T21:52:54.989 INFO:tasks.workunit.client.0.smithi107.stdout:3/513: chown d6/d8/d10/d35/d36/d4c/f97 3018 1 2024-05-17T21:52:54.990 INFO:tasks.workunit.client.1.smithi138.stdout:5/310: mkdir d1/d1a/d39/d69 0 2024-05-17T21:52:54.991 INFO:tasks.workunit.client.0.smithi107.stdout:3/514: write d6/d8/d10/d35/d36/d4c/f97 [625868,9333] 0 2024-05-17T21:52:54.991 INFO:tasks.workunit.client.1.smithi138.stdout:5/311: write d1/d6/d19/f20 [1118092,40553] 0 2024-05-17T21:52:54.992 INFO:tasks.workunit.client.1.smithi138.stdout:2/344: link d3/d21/d2f/l4b d3/l70 0 2024-05-17T21:52:54.993 INFO:tasks.workunit.client.0.smithi107.stdout:9/503: symlink d0/dd/d2f/d87/lb3 0 2024-05-17T21:52:54.995 INFO:tasks.workunit.client.1.smithi138.stdout:3/403: creat d1/d2e/f8b x:0 0 0 2024-05-17T21:52:54.996 INFO:tasks.workunit.client.1.smithi138.stdout:3/404: write d1/d3/d49/d4b/f85 [863066,48101] 0 2024-05-17T21:52:54.996 INFO:tasks.workunit.client.0.smithi107.stdout:1/546: mkdir d4/df/d1a/d41/db2 0 2024-05-17T21:52:54.998 INFO:tasks.workunit.client.1.smithi138.stdout:6/361: mknod d7/db/d43/c7b 0 2024-05-17T21:52:54.999 INFO:tasks.workunit.client.1.smithi138.stdout:3/405: dread d1/d3/f25 [0,4194304] 0 2024-05-17T21:52:55.000 INFO:tasks.workunit.client.0.smithi107.stdout:5/495: creat d5/db/d16/d2a/d73/f99 x:0 0 0 2024-05-17T21:52:55.000 INFO:tasks.workunit.client.1.smithi138.stdout:6/362: write d7/db/d43/d3f/d79/f6c [91342,97134] 0 2024-05-17T21:52:55.001 INFO:tasks.workunit.client.0.smithi107.stdout:5/496: read - d5/db/d16/d2a/d73/f99 zero size 2024-05-17T21:52:55.001 INFO:tasks.workunit.client.0.smithi107.stdout:0/569: mkdir d3/d24/dcf 0 2024-05-17T21:52:55.002 INFO:tasks.workunit.client.1.smithi138.stdout:5/312: truncate d1/d53/f57 276393 0 2024-05-17T21:52:55.002 INFO:tasks.workunit.client.0.smithi107.stdout:5/497: fdatasync d5/db/d16/d2d/d32/f91 0 2024-05-17T21:52:55.003 INFO:tasks.workunit.client.1.smithi138.stdout:7/252: read d0/f20 [5752615,49056] 0 2024-05-17T21:52:55.003 INFO:tasks.workunit.client.0.smithi107.stdout:2/351: symlink d0/d9/d11/l67 0 2024-05-17T21:52:55.004 INFO:tasks.workunit.client.1.smithi138.stdout:1/302: symlink d4/db/d60/l65 0 2024-05-17T21:52:55.005 INFO:tasks.workunit.client.1.smithi138.stdout:1/303: fsync f0 0 2024-05-17T21:52:55.006 INFO:tasks.workunit.client.0.smithi107.stdout:3/515: mkdir d6/d8/d3e/d3f/d90/da7 0 2024-05-17T21:52:55.008 INFO:tasks.workunit.client.0.smithi107.stdout:9/504: creat d0/dd/d3d/fb4 x:0 0 0 2024-05-17T21:52:55.011 INFO:tasks.workunit.client.1.smithi138.stdout:6/363: unlink d7/db/d43/f4f 0 2024-05-17T21:52:55.012 INFO:tasks.workunit.client.0.smithi107.stdout:6/400: creat d9/d16/d4d/d67/f79 x:0 0 0 2024-05-17T21:52:55.012 INFO:tasks.workunit.client.1.smithi138.stdout:5/313: creat d1/d1a/d39/f6a x:0 0 0 2024-05-17T21:52:55.014 INFO:tasks.workunit.client.1.smithi138.stdout:2/345: link d3/d21/f67 d3/d21/d2f/d3e/d58/f71 0 2024-05-17T21:52:55.015 INFO:tasks.workunit.client.0.smithi107.stdout:7/454: creat d5/df/d11/d1c/d31/d39/d92/d9a/fa1 x:0 0 0 2024-05-17T21:52:55.018 INFO:tasks.workunit.client.1.smithi138.stdout:7/253: dread d0/f19 [0,4194304] 0 2024-05-17T21:52:55.019 INFO:tasks.workunit.client.1.smithi138.stdout:3/406: truncate d1/d3/f11 1155390 0 2024-05-17T21:52:55.020 INFO:tasks.workunit.client.0.smithi107.stdout:1/547: mknod d4/df/d14/cb3 0 2024-05-17T21:52:55.021 INFO:tasks.workunit.client.0.smithi107.stdout:1/548: write d4/df/d1a/d26/d72/f83 [730795,83353] 0 2024-05-17T21:52:55.024 INFO:tasks.workunit.client.1.smithi138.stdout:6/364: fdatasync d7/d8/f50 0 2024-05-17T21:52:55.025 INFO:tasks.workunit.client.0.smithi107.stdout:0/570: mkdir d3/d75/dd0 0 2024-05-17T21:52:55.026 INFO:tasks.workunit.client.0.smithi107.stdout:0/571: chown d3/d24/d6f/d9f/dab/f8e 46099 1 2024-05-17T21:52:55.027 INFO:tasks.workunit.client.0.smithi107.stdout:0/572: stat d3/d75/dd0 0 2024-05-17T21:52:55.027 INFO:tasks.workunit.client.1.smithi138.stdout:5/314: chown d1/d6/l18 4 1 2024-05-17T21:52:55.027 INFO:tasks.workunit.client.1.smithi138.stdout:5/315: fdatasync d1/d6/fa 0 2024-05-17T21:52:55.027 INFO:tasks.workunit.client.0.smithi107.stdout:7/455: fdatasync d5/df/f1a 0 2024-05-17T21:52:55.027 INFO:tasks.workunit.client.0.smithi107.stdout:0/573: write d3/ff [1579208,100183] 0 2024-05-17T21:52:55.033 INFO:tasks.workunit.client.1.smithi138.stdout:5/316: dwrite d1/d6/d1b/d29/f67 [0,4194304] 0 2024-05-17T21:52:55.034 INFO:tasks.workunit.client.0.smithi107.stdout:3/516: getdents d6/d2a/d2d/d69 0 2024-05-17T21:52:55.035 INFO:tasks.workunit.client.1.smithi138.stdout:1/304: truncate f0 2749060 0 2024-05-17T21:52:55.039 INFO:tasks.workunit.client.1.smithi138.stdout:7/254: symlink d0/d39/l47 0 2024-05-17T21:52:55.039 INFO:tasks.workunit.client.1.smithi138.stdout:7/255: chown d0/d18 818270223 1 2024-05-17T21:52:55.040 INFO:tasks.workunit.client.0.smithi107.stdout:6/401: mknod d9/d16/d22/d5f/d6e/c7a 0 2024-05-17T21:52:55.041 INFO:tasks.workunit.client.0.smithi107.stdout:6/402: truncate d9/d16/d22/d2d/f54 104935 0 2024-05-17T21:52:55.042 INFO:tasks.workunit.client.0.smithi107.stdout:1/549: chown d4/df/d1a/d26/d7a/d81 732 1 2024-05-17T21:52:55.047 INFO:tasks.workunit.client.0.smithi107.stdout:1/550: dwrite d4/df/d14/d2e/d43/d47/f94 [0,4194304] 0 2024-05-17T21:52:55.047 INFO:tasks.workunit.client.0.smithi107.stdout:2/352: write d0/d9/d1c/d3e/f3f [738571,34008] 0 2024-05-17T21:52:55.049 INFO:tasks.workunit.client.0.smithi107.stdout:7/456: rmdir d5/df/d11/d1c/d2f/d65 39 2024-05-17T21:52:55.051 INFO:tasks.workunit.client.0.smithi107.stdout:7/457: fdatasync d5/df/d78/f95 0 2024-05-17T21:52:55.051 INFO:tasks.workunit.client.0.smithi107.stdout:5/498: link d5/f1b d5/db/f9a 0 2024-05-17T21:52:55.053 INFO:tasks.workunit.client.0.smithi107.stdout:8/485: sync 2024-05-17T21:52:55.053 INFO:tasks.workunit.client.0.smithi107.stdout:4/504: sync 2024-05-17T21:52:55.053 INFO:tasks.workunit.client.0.smithi107.stdout:9/505: link d0/dd/d69/la8 d0/dd/d21/d29/d57/d5f/lb5 0 2024-05-17T21:52:55.054 INFO:tasks.workunit.client.1.smithi138.stdout:8/207: sync 2024-05-17T21:52:55.055 INFO:tasks.workunit.client.0.smithi107.stdout:5/499: read d5/db/f9a [1412960,8005] 0 2024-05-17T21:52:55.056 INFO:tasks.workunit.client.1.smithi138.stdout:8/208: write d0/d11/f1c [1264591,23599] 0 2024-05-17T21:52:55.060 INFO:tasks.workunit.client.0.smithi107.stdout:8/486: dread dc/d21/d27/d34/f6e [0,4194304] 0 2024-05-17T21:52:55.060 INFO:tasks.workunit.client.0.smithi107.stdout:9/506: dread d0/dd/d3d/f8e [0,4194304] 0 2024-05-17T21:52:55.061 INFO:tasks.workunit.client.0.smithi107.stdout:2/353: dwrite d0/d9/d1c/d22/f65 [0,4194304] 0 2024-05-17T21:52:55.062 INFO:tasks.workunit.client.1.smithi138.stdout:6/365: truncate d7/db/d43/f35 1399763 0 2024-05-17T21:52:55.063 INFO:tasks.workunit.client.0.smithi107.stdout:1/551: creat d4/df/d1a/d26/d72/fb4 x:0 0 0 2024-05-17T21:52:55.063 INFO:tasks.workunit.client.0.smithi107.stdout:0/574: dread d3/d39/fb0 [0,4194304] 0 2024-05-17T21:52:55.065 INFO:tasks.workunit.client.1.smithi138.stdout:2/346: creat d3/f72 x:0 0 0 2024-05-17T21:52:55.065 INFO:tasks.workunit.client.0.smithi107.stdout:0/575: readlink d3/d58/d61/d94/la6 0 2024-05-17T21:52:55.065 INFO:tasks.workunit.client.1.smithi138.stdout:2/347: chown d3/d21/d2f/l3b 63 1 2024-05-17T21:52:55.068 INFO:tasks.workunit.client.1.smithi138.stdout:5/317: symlink d1/d1a/d39/d4e/d58/d5b/d65/l6b 0 2024-05-17T21:52:55.071 INFO:tasks.workunit.client.1.smithi138.stdout:1/305: mknod d4/db/d29/c66 0 2024-05-17T21:52:55.071 INFO:tasks.workunit.client.1.smithi138.stdout:1/306: readlink d4/df/d35/l46 0 2024-05-17T21:52:55.072 INFO:tasks.workunit.client.0.smithi107.stdout:7/458: creat d5/df/d78/fa2 x:0 0 0 2024-05-17T21:52:55.073 INFO:tasks.workunit.client.1.smithi138.stdout:3/407: write d1/d3/d8/d5c/f81 [3897696,43212] 0 2024-05-17T21:52:55.074 INFO:tasks.workunit.client.1.smithi138.stdout:3/408: write d1/d3/f20 [2720153,19689] 0 2024-05-17T21:52:55.078 INFO:tasks.workunit.client.1.smithi138.stdout:8/209: symlink d0/l3a 0 2024-05-17T21:52:55.081 INFO:tasks.workunit.client.1.smithi138.stdout:7/256: dread d0/f13 [0,4194304] 0 2024-05-17T21:52:55.082 INFO:tasks.workunit.client.1.smithi138.stdout:3/409: dwrite d1/d3/d8/de/f27 [0,4194304] 0 2024-05-17T21:52:55.082 INFO:tasks.workunit.client.1.smithi138.stdout:6/366: creat d7/db/d34/f7c x:0 0 0 2024-05-17T21:52:55.083 INFO:tasks.workunit.client.1.smithi138.stdout:6/367: chown d7/l49 272624534 1 2024-05-17T21:52:55.083 INFO:tasks.workunit.client.1.smithi138.stdout:6/368: stat c0 0 2024-05-17T21:52:55.085 INFO:tasks.workunit.client.1.smithi138.stdout:2/348: mkdir d3/d21/d2f/d3e/d55/d73 0 2024-05-17T21:52:55.085 INFO:tasks.workunit.client.1.smithi138.stdout:2/349: chown d3/l51 181021001 1 2024-05-17T21:52:55.088 INFO:tasks.workunit.client.1.smithi138.stdout:5/318: unlink d1/d6/d19/l34 0 2024-05-17T21:52:55.088 INFO:tasks.workunit.client.0.smithi107.stdout:3/517: dwrite d6/d8/d10/d35/d36/d4c/f65 [0,4194304] 0 2024-05-17T21:52:55.090 INFO:tasks.workunit.client.0.smithi107.stdout:3/518: read - d6/d8/d10/d25/f89 zero size 2024-05-17T21:52:55.090 INFO:tasks.workunit.client.0.smithi107.stdout:9/507: creat d0/dd/d2f/d87/fb6 x:0 0 0 2024-05-17T21:52:55.090 INFO:tasks.workunit.client.1.smithi138.stdout:1/307: mkdir d4/df/d3a/d3c/d67 0 2024-05-17T21:52:55.092 INFO:tasks.workunit.client.1.smithi138.stdout:1/308: chown d4/c52 32480 1 2024-05-17T21:52:55.093 INFO:tasks.workunit.client.0.smithi107.stdout:5/500: dread d5/db/d16/d5b/f3d [4194304,4194304] 0 2024-05-17T21:52:55.094 INFO:tasks.workunit.client.0.smithi107.stdout:1/552: creat d4/df/d1a/d26/d7a/fb5 x:0 0 0 2024-05-17T21:52:55.095 INFO:tasks.workunit.client.1.smithi138.stdout:2/350: dwrite d3/d21/d2f/f50 [4194304,4194304] 0 2024-05-17T21:52:55.098 INFO:tasks.workunit.client.0.smithi107.stdout:7/459: mknod d5/d16/ca3 0 2024-05-17T21:52:55.099 INFO:tasks.workunit.client.0.smithi107.stdout:2/354: write d0/d9/d11/f5c [735241,5995] 0 2024-05-17T21:52:55.100 INFO:tasks.workunit.client.0.smithi107.stdout:7/460: readlink d5/d16/d3b/d45/l61 0 2024-05-17T21:52:55.100 INFO:tasks.workunit.client.0.smithi107.stdout:7/461: stat d5/d16/d3b/d4f 0 2024-05-17T21:52:55.102 INFO:tasks.workunit.client.0.smithi107.stdout:4/505: creat d3/db/d35/d4f/d93/fae x:0 0 0 2024-05-17T21:52:55.104 INFO:tasks.workunit.client.1.smithi138.stdout:9/305: sync 2024-05-17T21:52:55.104 INFO:tasks.workunit.client.1.smithi138.stdout:0/364: sync 2024-05-17T21:52:55.104 INFO:tasks.workunit.client.1.smithi138.stdout:4/349: sync 2024-05-17T21:52:55.105 INFO:tasks.workunit.client.1.smithi138.stdout:6/369: creat d7/db/d43/d63/f7d x:0 0 0 2024-05-17T21:52:55.106 INFO:tasks.workunit.client.0.smithi107.stdout:8/487: mknod dc/d44/d4b/cb2 0 2024-05-17T21:52:55.107 INFO:tasks.workunit.client.1.smithi138.stdout:9/306: write d6/d7/d16/d29/d38/d3c/f4e [814531,117661] 0 2024-05-17T21:52:55.109 INFO:tasks.workunit.client.1.smithi138.stdout:1/309: unlink d4/db/f62 0 2024-05-17T21:52:55.109 INFO:tasks.workunit.client.1.smithi138.stdout:1/310: fdatasync d4/df/d35/f37 0 2024-05-17T21:52:55.110 INFO:tasks.workunit.client.1.smithi138.stdout:2/351: unlink d3/d21/l47 0 2024-05-17T21:52:55.110 INFO:tasks.workunit.client.0.smithi107.stdout:9/508: rmdir d0/dd/d21/d29 39 2024-05-17T21:52:55.111 INFO:tasks.workunit.client.1.smithi138.stdout:8/210: mkdir d0/dc/d3b 0 2024-05-17T21:52:55.113 INFO:tasks.workunit.client.1.smithi138.stdout:7/257: rename d0/fa to d0/d39/f48 0 2024-05-17T21:52:55.114 INFO:tasks.workunit.client.1.smithi138.stdout:7/258: write d0/d18/f45 [539099,9688] 0 2024-05-17T21:52:55.114 INFO:tasks.workunit.client.1.smithi138.stdout:7/259: stat d0/f20 0 2024-05-17T21:52:55.114 INFO:tasks.workunit.client.0.smithi107.stdout:1/553: mknod d4/df/d14/d1d/cb6 0 2024-05-17T21:52:55.116 INFO:tasks.workunit.client.1.smithi138.stdout:4/350: unlink d0/d1b/d36/f47 0 2024-05-17T21:52:55.117 INFO:tasks.workunit.client.1.smithi138.stdout:4/351: readlink d0/d2f/d49/d60/d1f/d32/d71/l76 0 2024-05-17T21:52:55.118 INFO:tasks.workunit.client.0.smithi107.stdout:2/355: creat d0/d9/d1c/d3e/d4f/f68 x:0 0 0 2024-05-17T21:52:55.119 INFO:tasks.workunit.client.1.smithi138.stdout:5/319: creat d1/d6/d1b/d29/d36/d68/f6c x:0 0 0 2024-05-17T21:52:55.120 INFO:tasks.workunit.client.1.smithi138.stdout:5/320: truncate d1/d1a/d39/d62/f66 801811 0 2024-05-17T21:52:55.120 INFO:tasks.workunit.client.1.smithi138.stdout:6/370: fdatasync d7/f37 0 2024-05-17T21:52:55.121 INFO:tasks.workunit.client.1.smithi138.stdout:6/371: chown d7/db/f52 65354250 1 2024-05-17T21:52:55.122 INFO:tasks.workunit.client.0.smithi107.stdout:6/403: getdents d9/d16/d22 0 2024-05-17T21:52:55.124 INFO:tasks.workunit.client.0.smithi107.stdout:8/488: fdatasync dc/f18 0 2024-05-17T21:52:55.125 INFO:tasks.workunit.client.1.smithi138.stdout:9/307: creat d6/d7/d16/d29/d48/f61 x:0 0 0 2024-05-17T21:52:55.126 INFO:tasks.workunit.client.0.smithi107.stdout:9/509: mknod d0/dd/d2f/d87/d89/cb7 0 2024-05-17T21:52:55.127 INFO:tasks.workunit.client.0.smithi107.stdout:9/510: chown d0/d80/dad 22717694 1 2024-05-17T21:52:55.127 INFO:tasks.workunit.client.1.smithi138.stdout:5/321: dwrite d1/d6/d1b/f54 [0,4194304] 0 2024-05-17T21:52:55.131 INFO:tasks.workunit.client.1.smithi138.stdout:0/365: dread d3/d3d/d51/f64 [0,4194304] 0 2024-05-17T21:52:55.132 INFO:tasks.workunit.client.0.smithi107.stdout:1/554: unlink d4/df/d1a/c52 0 2024-05-17T21:52:55.134 INFO:tasks.workunit.client.0.smithi107.stdout:1/555: write d4/df/d1a/d26/d72/daa/fa1 [1528288,91087] 0 2024-05-17T21:52:55.136 INFO:tasks.workunit.client.0.smithi107.stdout:1/556: chown d4/df/d14/d2e/d43/lb1 167244710 1 2024-05-17T21:52:55.137 INFO:tasks.workunit.client.1.smithi138.stdout:8/211: creat d0/d2f/f3c x:0 0 0 2024-05-17T21:52:55.141 INFO:tasks.workunit.client.0.smithi107.stdout:6/404: mkdir d9/d58/d7b 0 2024-05-17T21:52:55.141 INFO:tasks.workunit.client.0.smithi107.stdout:6/405: stat d9/d16/d22/d5f/f75 0 2024-05-17T21:52:55.142 INFO:tasks.workunit.client.1.smithi138.stdout:7/260: symlink d0/d18/l49 0 2024-05-17T21:52:55.143 INFO:tasks.workunit.client.1.smithi138.stdout:7/261: stat d0/d39/c42 0 2024-05-17T21:52:55.143 INFO:tasks.workunit.client.1.smithi138.stdout:7/262: stat d0/f27 0 2024-05-17T21:52:55.144 INFO:tasks.workunit.client.0.smithi107.stdout:8/489: stat dc/d21/d97/d63/c6a 0 2024-05-17T21:52:55.149 INFO:tasks.workunit.client.0.smithi107.stdout:3/519: rename d6/d8/d10/d35/d36/d4c/d83 to d6/d2a/d2d/da8 0 2024-05-17T21:52:55.150 INFO:tasks.workunit.client.0.smithi107.stdout:3/520: fdatasync d6/d2a/d2d/d37/f7c 0 2024-05-17T21:52:55.151 INFO:tasks.workunit.client.1.smithi138.stdout:6/372: creat d7/db/d34/f7e x:0 0 0 2024-05-17T21:52:55.152 INFO:tasks.workunit.client.0.smithi107.stdout:2/356: dwrite d0/d9/d11/df/f35 [0,4194304] 0 2024-05-17T21:52:55.154 INFO:tasks.workunit.client.0.smithi107.stdout:2/357: write d0/d9/d1c/d3e/d4f/f68 [591813,55441] 0 2024-05-17T21:52:55.158 INFO:tasks.workunit.client.1.smithi138.stdout:9/308: symlink d6/d7/d16/d29/d38/d3c/l62 0 2024-05-17T21:52:55.161 INFO:tasks.workunit.client.1.smithi138.stdout:1/311: rename d4/c7 to d4/dd/d3b/c68 0 2024-05-17T21:52:55.161 INFO:tasks.workunit.client.1.smithi138.stdout:1/312: read - d4/d2e/f54 zero size 2024-05-17T21:52:55.162 INFO:tasks.workunit.client.0.smithi107.stdout:9/511: write d0/dd/d3d/f8e [512532,48859] 0 2024-05-17T21:52:55.164 INFO:tasks.workunit.client.0.smithi107.stdout:8/490: mknod dc/d21/d35/cb3 0 2024-05-17T21:52:55.165 INFO:tasks.workunit.client.0.smithi107.stdout:8/491: dread - dc/d44/d4b/d52/d70/d76/f78 zero size 2024-05-17T21:52:55.166 INFO:tasks.workunit.client.1.smithi138.stdout:8/212: symlink d0/d2f/l3d 0 2024-05-17T21:52:55.166 INFO:tasks.workunit.client.1.smithi138.stdout:3/410: link d1/ca d1/d3/d16/d5d/c8c 0 2024-05-17T21:52:55.166 INFO:tasks.workunit.client.1.smithi138.stdout:8/213: readlink d0/l19 0 2024-05-17T21:52:55.167 INFO:tasks.workunit.client.0.smithi107.stdout:8/492: write dc/f43 [84664,125971] 0 2024-05-17T21:52:55.169 INFO:tasks.workunit.client.0.smithi107.stdout:5/501: rename d5/db/d16/d2d/d51/l6b to d5/db/d16/d2a/l9b 0 2024-05-17T21:52:55.169 INFO:tasks.workunit.client.1.smithi138.stdout:7/263: readlink d0/lf 0 2024-05-17T21:52:55.170 INFO:tasks.workunit.client.0.smithi107.stdout:0/576: link d3/dc/l36 d3/dc/d22/d23/d97/ld1 0 2024-05-17T21:52:55.172 INFO:tasks.workunit.client.0.smithi107.stdout:3/521: symlink d6/d2a/d2d/da8/la9 0 2024-05-17T21:52:55.173 INFO:tasks.workunit.client.0.smithi107.stdout:5/502: dread d5/db/d39/f3e [0,4194304] 0 2024-05-17T21:52:55.173 INFO:tasks.workunit.client.1.smithi138.stdout:3/411: dwrite d1/d3/f1c [0,4194304] 0 2024-05-17T21:52:55.174 INFO:tasks.workunit.client.0.smithi107.stdout:3/522: truncate d6/d2a/f78 720582 0 2024-05-17T21:52:55.175 INFO:tasks.workunit.client.0.smithi107.stdout:5/503: write d5/db/d16/d2d/d4d/d6d/f77 [180786,26936] 0 2024-05-17T21:52:55.175 INFO:tasks.workunit.client.0.smithi107.stdout:3/523: write d6/d8/d10/f30 [3080373,115585] 0 2024-05-17T21:52:55.176 INFO:tasks.workunit.client.1.smithi138.stdout:6/373: creat d7/d8/d78/f7f x:0 0 0 2024-05-17T21:52:55.176 INFO:tasks.workunit.client.0.smithi107.stdout:1/557: symlink d4/df/d14/lb7 0 2024-05-17T21:52:55.177 INFO:tasks.workunit.client.0.smithi107.stdout:5/504: write d5/db/d16/d5b/f97 [383737,67626] 0 2024-05-17T21:52:55.179 INFO:tasks.workunit.client.1.smithi138.stdout:5/322: creat d1/d2c/d63/f6d x:0 0 0 2024-05-17T21:52:55.179 INFO:tasks.workunit.client.0.smithi107.stdout:7/462: link d5/d16/d3b/d45/l93 d5/df/d11/d1c/d2f/d65/la4 0 2024-05-17T21:52:55.181 INFO:tasks.workunit.client.1.smithi138.stdout:9/309: creat d6/d7/d16/d29/d48/d3b/f63 x:0 0 0 2024-05-17T21:52:55.182 INFO:tasks.workunit.client.1.smithi138.stdout:6/374: write d7/db/d43/d3f/d79/f55 [2661774,122559] 0 2024-05-17T21:52:55.183 INFO:tasks.workunit.client.0.smithi107.stdout:9/512: creat d0/dd/d3d/fb8 x:0 0 0 2024-05-17T21:52:55.184 INFO:tasks.workunit.client.1.smithi138.stdout:2/352: creat d3/f74 x:0 0 0 2024-05-17T21:52:55.185 INFO:tasks.workunit.client.1.smithi138.stdout:0/366: symlink d3/d10/l7e 0 2024-05-17T21:52:55.185 INFO:tasks.workunit.client.1.smithi138.stdout:5/323: write d1/d6/d1b/d29/d36/f37 [633451,123588] 0 2024-05-17T21:52:55.187 INFO:tasks.workunit.client.0.smithi107.stdout:0/577: symlink d3/d75/dc1/ld2 0 2024-05-17T21:52:55.187 INFO:tasks.workunit.client.0.smithi107.stdout:1/558: dwrite d4/de/f19 [4194304,4194304] 0 2024-05-17T21:52:55.188 INFO:tasks.workunit.client.1.smithi138.stdout:8/214: mkdir d0/d11/d3e 0 2024-05-17T21:52:55.190 INFO:tasks.workunit.client.1.smithi138.stdout:3/412: dwrite d1/d3/d8/f34 [4194304,4194304] 0 2024-05-17T21:52:55.190 INFO:tasks.workunit.client.0.smithi107.stdout:8/493: dread dc/f86 [0,4194304] 0 2024-05-17T21:52:55.190 INFO:tasks.workunit.client.0.smithi107.stdout:2/358: symlink d0/l69 0 2024-05-17T21:52:55.190 INFO:tasks.workunit.client.1.smithi138.stdout:0/367: write d3/d10/f1a [1053830,117924] 0 2024-05-17T21:52:55.191 INFO:tasks.workunit.client.0.smithi107.stdout:8/494: chown dc/d21/f92 193341 1 2024-05-17T21:52:55.192 INFO:tasks.workunit.client.0.smithi107.stdout:2/359: chown d0/d9/d1c/l3c 726096 1 2024-05-17T21:52:55.192 INFO:tasks.workunit.client.1.smithi138.stdout:5/324: stat d1/d1a/d39/d4e/d58/d5b/d65/l6b 0 2024-05-17T21:52:55.193 INFO:tasks.workunit.client.1.smithi138.stdout:2/353: write d3/d21/d2f/d3e/d58/f5d [1406343,50726] 0 2024-05-17T21:52:55.194 INFO:tasks.workunit.client.0.smithi107.stdout:1/559: dread d4/df/d1a/d26/d72/daa/f40 [0,4194304] 0 2024-05-17T21:52:55.195 INFO:tasks.workunit.client.1.smithi138.stdout:1/313: dread d4/db/f21 [0,4194304] 0 2024-05-17T21:52:55.197 INFO:tasks.workunit.client.1.smithi138.stdout:2/354: write d3/d21/d2f/f50 [7745308,16278] 0 2024-05-17T21:52:55.197 INFO:tasks.workunit.client.1.smithi138.stdout:1/314: chown d4/df/d3a/d3c/d67 268105 1 2024-05-17T21:52:55.198 INFO:tasks.workunit.client.0.smithi107.stdout:3/524: dread d6/d8/d3e/d3f/f87 [4194304,4194304] 0 2024-05-17T21:52:55.201 INFO:tasks.workunit.client.0.smithi107.stdout:6/406: creat d9/f7c x:0 0 0 2024-05-17T21:52:55.201 INFO:tasks.workunit.client.1.smithi138.stdout:6/375: creat d7/d69/f80 x:0 0 0 2024-05-17T21:52:55.202 INFO:tasks.workunit.client.1.smithi138.stdout:7/264: dread d0/f11 [0,4194304] 0 2024-05-17T21:52:55.204 INFO:tasks.workunit.client.0.smithi107.stdout:7/463: symlink d5/df/d11/d1d/d33/d82/la5 0 2024-05-17T21:52:55.208 INFO:tasks.workunit.client.0.smithi107.stdout:9/513: rename d0/dd/d1b/c26 to d0/dd/d2f/d87/cb9 0 2024-05-17T21:52:55.209 INFO:tasks.workunit.client.0.smithi107.stdout:9/514: fsync d0/dd/d1f/d2c/f47 0 2024-05-17T21:52:55.211 INFO:tasks.workunit.client.0.smithi107.stdout:0/578: creat d3/dc/d22/d23/d97/d77/d91/fd3 x:0 0 0 2024-05-17T21:52:55.211 INFO:tasks.workunit.client.1.smithi138.stdout:6/376: dread d7/db/d12/f26 [0,4194304] 0 2024-05-17T21:52:55.212 INFO:tasks.workunit.client.0.smithi107.stdout:2/360: creat d0/d9/d1c/d3e/d4f/f6a x:0 0 0 2024-05-17T21:52:55.214 INFO:tasks.workunit.client.1.smithi138.stdout:5/325: creat d1/d53/f6e x:0 0 0 2024-05-17T21:52:55.214 INFO:tasks.workunit.client.1.smithi138.stdout:5/326: read - d1/d6/d1b/d29/d36/f4b zero size 2024-05-17T21:52:55.215 INFO:tasks.workunit.client.1.smithi138.stdout:5/327: fdatasync d1/d1a/f26 0 2024-05-17T21:52:55.215 INFO:tasks.workunit.client.0.smithi107.stdout:9/515: dread d0/f43 [0,4194304] 0 2024-05-17T21:52:55.216 INFO:tasks.workunit.client.1.smithi138.stdout:0/368: unlink d3/d10/d38/f3a 0 2024-05-17T21:52:55.217 INFO:tasks.workunit.client.0.smithi107.stdout:1/560: symlink d4/d78/lb8 0 2024-05-17T21:52:55.218 INFO:tasks.workunit.client.1.smithi138.stdout:4/352: getdents d0/d2f/d49/d60/d1f/d32/d71 0 2024-05-17T21:52:55.219 INFO:tasks.workunit.client.0.smithi107.stdout:1/561: write d4/df/d4e/f63 [1288886,9970] 0 2024-05-17T21:52:55.221 INFO:tasks.workunit.client.0.smithi107.stdout:5/505: mknod d5/d5f/c9c 0 2024-05-17T21:52:55.221 INFO:tasks.workunit.client.1.smithi138.stdout:9/310: truncate d6/d7/f17 3659569 0 2024-05-17T21:52:55.223 INFO:tasks.workunit.client.1.smithi138.stdout:9/311: write d6/d7/d16/d29/d48/f34 [4092179,79696] 0 2024-05-17T21:52:55.227 INFO:tasks.workunit.client.0.smithi107.stdout:4/506: sync 2024-05-17T21:52:55.227 INFO:tasks.workunit.client.0.smithi107.stdout:3/525: unlink d6/d2a/d2d/d48/c5f 0 2024-05-17T21:52:55.228 INFO:tasks.workunit.client.0.smithi107.stdout:7/464: mknod d5/df/d11/d2a/ca6 0 2024-05-17T21:52:55.229 INFO:tasks.workunit.client.0.smithi107.stdout:9/516: dwrite d0/dd/d2f/d87/d89/f97 [0,4194304] 0 2024-05-17T21:52:55.230 INFO:tasks.workunit.client.0.smithi107.stdout:7/465: write d5/df/d11/f7a [220500,93226] 0 2024-05-17T21:52:55.233 INFO:tasks.workunit.client.1.smithi138.stdout:3/413: mkdir d1/d3/d8d 0 2024-05-17T21:52:55.237 INFO:tasks.workunit.client.0.smithi107.stdout:5/506: dwrite d5/db/d16/d5b/f85 [0,4194304] 0 2024-05-17T21:52:55.241 INFO:tasks.workunit.client.1.smithi138.stdout:3/414: dwrite d1/d3/d49/d4b/f85 [0,4194304] 0 2024-05-17T21:52:55.242 INFO:tasks.workunit.client.0.smithi107.stdout:1/562: chown d4/c64 11010 1 2024-05-17T21:52:55.244 INFO:tasks.workunit.client.1.smithi138.stdout:3/415: read d1/d3/d16/d3a/f78 [2355640,82923] 0 2024-05-17T21:52:55.244 INFO:tasks.workunit.client.0.smithi107.stdout:8/495: write dc/d21/d27/d34/d4c/d7e/f3b [614796,10863] 0 2024-05-17T21:52:55.246 INFO:tasks.workunit.client.1.smithi138.stdout:8/215: dwrite d0/dc/d1a/f24 [0,4194304] 0 2024-05-17T21:52:55.250 INFO:tasks.workunit.client.1.smithi138.stdout:4/353: unlink d0/d2f/d49/d60/d1f/d32/c44 0 2024-05-17T21:52:55.251 INFO:tasks.workunit.client.1.smithi138.stdout:4/354: readlink d0/d2f/l68 0 2024-05-17T21:52:55.251 INFO:tasks.workunit.client.1.smithi138.stdout:4/355: stat d0/l4 0 2024-05-17T21:52:55.252 INFO:tasks.workunit.client.0.smithi107.stdout:6/407: symlink d9/d16/d22/d2d/l7d 0 2024-05-17T21:52:55.255 INFO:tasks.workunit.client.0.smithi107.stdout:9/517: mkdir d0/dd/d2f/d87/dba 0 2024-05-17T21:52:55.256 INFO:tasks.workunit.client.1.smithi138.stdout:2/355: symlink d3/d21/d2f/d3e/d55/d73/l75 0 2024-05-17T21:52:55.259 INFO:tasks.workunit.client.1.smithi138.stdout:9/312: creat d6/d7/d16/d29/d38/d3c/d3e/d5a/f64 x:0 0 0 2024-05-17T21:52:55.261 INFO:tasks.workunit.client.0.smithi107.stdout:7/466: mkdir d5/df/d11/d1c/d2f/d65/da7 0 2024-05-17T21:52:55.263 INFO:tasks.workunit.client.1.smithi138.stdout:6/377: write d7/d8/f24 [909580,59112] 0 2024-05-17T21:52:55.266 INFO:tasks.workunit.client.0.smithi107.stdout:2/361: dwrite d0/d9/f33 [0,4194304] 0 2024-05-17T21:52:55.268 INFO:tasks.workunit.client.0.smithi107.stdout:8/496: mkdir dc/d21/daa/db4 0 2024-05-17T21:52:55.268 INFO:tasks.workunit.client.0.smithi107.stdout:2/362: write d0/f25 [2119135,36312] 0 2024-05-17T21:52:55.274 INFO:tasks.workunit.client.1.smithi138.stdout:3/416: chown d1/d3/d16/f29 1835 1 2024-05-17T21:52:55.276 INFO:tasks.workunit.client.0.smithi107.stdout:3/526: dwrite d6/d2a/d2d/d37/f3a [0,4194304] 0 2024-05-17T21:52:55.277 INFO:tasks.workunit.client.1.smithi138.stdout:8/216: rename d0/d11/f1c to d0/dc/d1a/f3f 0 2024-05-17T21:52:55.277 INFO:tasks.workunit.client.0.smithi107.stdout:2/363: dwrite d0/d9/d1c/d3e/d4f/f61 [0,4194304] 0 2024-05-17T21:52:55.278 INFO:tasks.workunit.client.0.smithi107.stdout:9/518: dread d0/f43 [0,4194304] 0 2024-05-17T21:52:55.279 INFO:tasks.workunit.client.1.smithi138.stdout:8/217: truncate d0/dc/d1a/d33/f34 121229 0 2024-05-17T21:52:55.280 INFO:tasks.workunit.client.0.smithi107.stdout:7/467: fdatasync d5/d16/f3a 0 2024-05-17T21:52:55.280 INFO:tasks.workunit.client.0.smithi107.stdout:2/364: readlink d0/d9/l59 0 2024-05-17T21:52:55.281 INFO:tasks.workunit.client.1.smithi138.stdout:4/356: creat d0/d1b/d2b/d55/f7d x:0 0 0 2024-05-17T21:52:55.281 INFO:tasks.workunit.client.1.smithi138.stdout:3/417: dwrite d1/d3/d8/d5c/f81 [0,4194304] 0 2024-05-17T21:52:55.282 INFO:tasks.workunit.client.1.smithi138.stdout:3/418: dread - d1/d3/d16/d3e/f73 zero size 2024-05-17T21:52:55.284 INFO:tasks.workunit.client.1.smithi138.stdout:2/356: unlink d3/f72 0 2024-05-17T21:52:55.286 INFO:tasks.workunit.client.1.smithi138.stdout:2/357: fsync d3/f20 0 2024-05-17T21:52:55.289 INFO:tasks.workunit.client.1.smithi138.stdout:1/315: link d4/df/l23 d4/df/d5f/l69 0 2024-05-17T21:52:55.290 INFO:tasks.workunit.client.1.smithi138.stdout:1/316: chown d4/f9 233312 1 2024-05-17T21:52:55.290 INFO:tasks.workunit.client.1.smithi138.stdout:2/358: read d3/d21/d2f/f36 [101789,74022] 0 2024-05-17T21:52:55.295 INFO:tasks.workunit.client.0.smithi107.stdout:8/497: fsync dc/d21/d69/f8e 0 2024-05-17T21:52:55.295 INFO:tasks.workunit.client.1.smithi138.stdout:8/218: dwrite d0/dc/f10 [0,4194304] 0 2024-05-17T21:52:55.295 INFO:tasks.workunit.client.1.smithi138.stdout:7/265: creat d0/f4a x:0 0 0 2024-05-17T21:52:55.296 INFO:tasks.workunit.client.1.smithi138.stdout:8/219: dread - d0/dc/d1a/f2a zero size 2024-05-17T21:52:55.296 INFO:tasks.workunit.client.0.smithi107.stdout:8/498: readlink dc/d21/d97/d63/l9c 0 2024-05-17T21:52:55.296 INFO:tasks.workunit.client.0.smithi107.stdout:1/563: dwrite d4/df/f29 [0,4194304] 0 2024-05-17T21:52:55.297 INFO:tasks.workunit.client.0.smithi107.stdout:8/499: fdatasync dc/f4e 0 2024-05-17T21:52:55.297 INFO:tasks.workunit.client.1.smithi138.stdout:6/378: creat d7/d8/d78/f81 x:0 0 0 2024-05-17T21:52:55.298 INFO:tasks.workunit.client.0.smithi107.stdout:6/408: rename d9/d16/d22/d5f/d6f/f76 to d9/d58/f7e 0 2024-05-17T21:52:55.299 INFO:tasks.workunit.client.1.smithi138.stdout:6/379: chown d7/d8/f70 33513171 1 2024-05-17T21:52:55.299 INFO:tasks.workunit.client.0.smithi107.stdout:6/409: write d9/d16/d22/d2d/f70 [335039,127884] 0 2024-05-17T21:52:55.302 INFO:tasks.workunit.client.1.smithi138.stdout:0/369: link d3/d10/f17 d3/d3d/f7f 0 2024-05-17T21:52:55.302 INFO:tasks.workunit.client.0.smithi107.stdout:6/410: write d9/d16/d22/d2d/f39 [2239424,98903] 0 2024-05-17T21:52:55.305 INFO:tasks.workunit.client.1.smithi138.stdout:0/370: read d3/d10/d39/f72 [136569,85061] 0 2024-05-17T21:52:55.306 INFO:tasks.workunit.client.1.smithi138.stdout:8/220: dread d0/d11/f28 [0,4194304] 0 2024-05-17T21:52:55.307 INFO:tasks.workunit.client.1.smithi138.stdout:5/328: dwrite d1/d6/d1b/f1d [0,4194304] 0 2024-05-17T21:52:55.307 INFO:tasks.workunit.client.1.smithi138.stdout:8/221: readlink d0/dc/l21 0 2024-05-17T21:52:55.309 INFO:tasks.workunit.client.1.smithi138.stdout:5/329: write d1/d6/d1b/d29/f67 [4900006,46527] 0 2024-05-17T21:52:55.316 INFO:tasks.workunit.client.0.smithi107.stdout:3/527: mkdir d6/d2a/d2d/da8/daa 0 2024-05-17T21:52:55.317 INFO:tasks.workunit.client.1.smithi138.stdout:8/222: dwrite d0/dc/f18 [0,4194304] 0 2024-05-17T21:52:55.318 INFO:tasks.workunit.client.0.smithi107.stdout:3/528: truncate d6/d2a/d2d/d48/d4f/f67 388881 0 2024-05-17T21:52:55.319 INFO:tasks.workunit.client.0.smithi107.stdout:3/529: stat d6/d1a/l6e 0 2024-05-17T21:52:55.321 INFO:tasks.workunit.client.1.smithi138.stdout:9/313: dwrite d6/d7/fc [0,4194304] 0 2024-05-17T21:52:55.322 INFO:tasks.workunit.client.0.smithi107.stdout:5/507: creat d5/db/d16/f9d x:0 0 0 2024-05-17T21:52:55.323 INFO:tasks.workunit.client.0.smithi107.stdout:5/508: readlink d5/db/dc/l44 0 2024-05-17T21:52:55.326 INFO:tasks.workunit.client.0.smithi107.stdout:2/365: rmdir d0/d9/d1c/d22 39 2024-05-17T21:52:55.326 INFO:tasks.workunit.client.0.smithi107.stdout:3/530: dwrite d6/d2a/d2d/f6c [0,4194304] 0 2024-05-17T21:52:55.328 INFO:tasks.workunit.client.1.smithi138.stdout:4/357: mkdir d0/d2f/d49/d60/d1f/d7e 0 2024-05-17T21:52:55.331 INFO:tasks.workunit.client.1.smithi138.stdout:4/358: truncate d0/d2f/d49/d60/d1f/f70 93127 0 2024-05-17T21:52:55.331 INFO:tasks.workunit.client.0.smithi107.stdout:2/366: dread d0/d9/d1c/f3d [0,4194304] 0 2024-05-17T21:52:55.332 INFO:tasks.workunit.client.1.smithi138.stdout:3/419: creat d1/d3/d8/de/f8e x:0 0 0 2024-05-17T21:52:55.333 INFO:tasks.workunit.client.1.smithi138.stdout:3/420: chown d1/d2e/f8b 152 1 2024-05-17T21:52:55.336 INFO:tasks.workunit.client.0.smithi107.stdout:1/564: creat d4/df/d1a/d26/d7a/fb9 x:0 0 0 2024-05-17T21:52:55.339 INFO:tasks.workunit.client.1.smithi138.stdout:2/359: rmdir d3/d21/d2f/d3e/d55/d73 39 2024-05-17T21:52:55.344 INFO:tasks.workunit.client.1.smithi138.stdout:7/266: write d0/f20 [6982057,92375] 0 2024-05-17T21:52:55.346 INFO:tasks.workunit.client.0.smithi107.stdout:5/509: rename d5/db/dc/f20 to d5/db/d16/d2a/d73/f9e 0 2024-05-17T21:52:55.349 INFO:tasks.workunit.client.1.smithi138.stdout:7/267: dwrite d0/f12 [0,4194304] 0 2024-05-17T21:52:55.349 INFO:tasks.workunit.client.0.smithi107.stdout:3/531: rmdir d6 39 2024-05-17T21:52:55.354 INFO:tasks.workunit.client.1.smithi138.stdout:5/330: rmdir d1/d2c/d63 39 2024-05-17T21:52:55.355 INFO:tasks.workunit.client.1.smithi138.stdout:5/331: stat d1/f45 0 2024-05-17T21:52:55.358 INFO:tasks.workunit.client.0.smithi107.stdout:2/367: creat d0/d9/d1c/d3e/f6b x:0 0 0 2024-05-17T21:52:55.359 INFO:tasks.workunit.client.1.smithi138.stdout:8/223: creat d0/f40 x:0 0 0 2024-05-17T21:52:55.359 INFO:tasks.workunit.client.0.smithi107.stdout:7/468: truncate d5/d16/f3a 702968 0 2024-05-17T21:52:55.360 INFO:tasks.workunit.client.0.smithi107.stdout:1/565: symlink d4/df/d69/lba 0 2024-05-17T21:52:55.360 INFO:tasks.workunit.client.0.smithi107.stdout:7/469: truncate d5/df/d11/f4c 1039914 0 2024-05-17T21:52:55.361 INFO:tasks.workunit.client.0.smithi107.stdout:1/566: chown d4/df/d4e/d59/f73 0 1 2024-05-17T21:52:55.364 INFO:tasks.workunit.client.1.smithi138.stdout:3/421: rmdir d1/d3/d8/d72 39 2024-05-17T21:52:55.365 INFO:tasks.workunit.client.1.smithi138.stdout:8/224: dwrite d0/dc/f2b [0,4194304] 0 2024-05-17T21:52:55.366 INFO:tasks.workunit.client.1.smithi138.stdout:8/225: truncate d0/f1b 1057635 0 2024-05-17T21:52:55.369 INFO:tasks.workunit.client.0.smithi107.stdout:2/368: truncate d0/f36 1202879 0 2024-05-17T21:52:55.370 INFO:tasks.workunit.client.0.smithi107.stdout:7/470: creat d5/df/d11/d1c/d31/d56/fa8 x:0 0 0 2024-05-17T21:52:55.370 INFO:tasks.workunit.client.0.smithi107.stdout:8/500: link dc/d21/d27/d34/d4c/d7e/c7d dc/d21/d27/d34/d4c/db1/cb5 0 2024-05-17T21:52:55.370 INFO:tasks.workunit.client.1.smithi138.stdout:7/268: rmdir d0/d18/d40 39 2024-05-17T21:52:55.370 INFO:tasks.workunit.client.1.smithi138.stdout:7/269: dread - d0/f3b zero size 2024-05-17T21:52:55.370 INFO:tasks.workunit.client.1.smithi138.stdout:0/371: mkdir d3/d10/d38/d70/d80 0 2024-05-17T21:52:55.372 INFO:tasks.workunit.client.0.smithi107.stdout:6/411: creat d9/d16/f7f x:0 0 0 2024-05-17T21:52:55.373 INFO:tasks.workunit.client.0.smithi107.stdout:9/519: getdents d0/dd/d2f/d4a 0 2024-05-17T21:52:55.377 INFO:tasks.workunit.client.0.smithi107.stdout:3/532: unlink d6/d8/d10/d35/l79 0 2024-05-17T21:52:55.377 INFO:tasks.workunit.client.1.smithi138.stdout:0/372: dwrite d3/d10/f1a [0,4194304] 0 2024-05-17T21:52:55.377 INFO:tasks.workunit.client.1.smithi138.stdout:0/373: readlink d3/d10/l13 0 2024-05-17T21:52:55.379 INFO:tasks.workunit.client.1.smithi138.stdout:9/314: unlink d6/l51 0 2024-05-17T21:52:55.382 INFO:tasks.workunit.client.0.smithi107.stdout:2/369: fdatasync d0/d9/d1c/d22/f5b 0 2024-05-17T21:52:55.383 INFO:tasks.workunit.client.0.smithi107.stdout:2/370: chown d0/d9/d11/d27/c4d 246 1 2024-05-17T21:52:55.383 INFO:tasks.workunit.client.1.smithi138.stdout:4/359: fsync d0/f34 0 2024-05-17T21:52:55.383 INFO:tasks.workunit.client.0.smithi107.stdout:2/371: chown d0/d9/d11/d18/c1a 11 1 2024-05-17T21:52:55.384 INFO:tasks.workunit.client.0.smithi107.stdout:1/567: symlink d4/df/lbb 0 2024-05-17T21:52:55.385 INFO:tasks.workunit.client.0.smithi107.stdout:1/568: fsync d4/df/d4e/d59/f5a 0 2024-05-17T21:52:55.385 INFO:tasks.workunit.client.1.smithi138.stdout:1/317: creat d4/db/f6a x:0 0 0 2024-05-17T21:52:55.386 INFO:tasks.workunit.client.1.smithi138.stdout:1/318: read - d4/df/d4e/f5c zero size 2024-05-17T21:52:55.387 INFO:tasks.workunit.client.0.smithi107.stdout:7/471: creat d5/df/d11/d1c/d2f/d53/fa9 x:0 0 0 2024-05-17T21:52:55.387 INFO:tasks.workunit.client.0.smithi107.stdout:1/569: truncate d4/df/d1a/d26/d7a/fb5 921232 0 2024-05-17T21:52:55.387 INFO:tasks.workunit.client.0.smithi107.stdout:1/570: readlink d4/df/d4e/d59/lae 0 2024-05-17T21:52:55.388 INFO:tasks.workunit.client.0.smithi107.stdout:7/472: stat d5/df/d11/d1c/d2f/d65 0 2024-05-17T21:52:55.389 INFO:tasks.workunit.client.1.smithi138.stdout:2/360: symlink d3/d21/d2f/d3e/d69/l76 0 2024-05-17T21:52:55.389 INFO:tasks.workunit.client.1.smithi138.stdout:2/361: readlink d3/l51 0 2024-05-17T21:52:55.390 INFO:tasks.workunit.client.1.smithi138.stdout:2/362: dread - d3/d21/d2f/d3e/d58/f6b zero size 2024-05-17T21:52:55.391 INFO:tasks.workunit.client.0.smithi107.stdout:7/473: dread d5/df/d11/f21 [4194304,4194304] 0 2024-05-17T21:52:55.392 INFO:tasks.workunit.client.1.smithi138.stdout:8/226: creat d0/d2f/f41 x:0 0 0 2024-05-17T21:52:55.392 INFO:tasks.workunit.client.0.smithi107.stdout:9/520: fsync d0/f43 0 2024-05-17T21:52:55.393 INFO:tasks.workunit.client.1.smithi138.stdout:6/380: creat d7/db/f82 x:0 0 0 2024-05-17T21:52:55.394 INFO:tasks.workunit.client.0.smithi107.stdout:9/521: chown d0/dd/d1f/l3a 0 1 2024-05-17T21:52:55.394 INFO:tasks.workunit.client.1.smithi138.stdout:7/270: fdatasync d0/f10 0 2024-05-17T21:52:55.395 INFO:tasks.workunit.client.1.smithi138.stdout:6/381: chown d7/db/d12/c1f 42749483 1 2024-05-17T21:52:55.396 INFO:tasks.workunit.client.0.smithi107.stdout:6/412: dwrite d9/f4e [0,4194304] 0 2024-05-17T21:52:55.397 INFO:tasks.workunit.client.1.smithi138.stdout:5/332: fsync d1/f43 0 2024-05-17T21:52:55.397 INFO:tasks.workunit.client.1.smithi138.stdout:6/382: write d7/db/d43/d3f/d79/f6c [1071192,80619] 0 2024-05-17T21:52:55.399 INFO:tasks.workunit.client.0.smithi107.stdout:2/372: creat d0/d26/f6c x:0 0 0 2024-05-17T21:52:55.400 INFO:tasks.workunit.client.1.smithi138.stdout:8/227: dwrite d0/ff [0,4194304] 0 2024-05-17T21:52:55.402 INFO:tasks.workunit.client.0.smithi107.stdout:3/533: dwrite d6/d1a/d57/f5e [4194304,4194304] 0 2024-05-17T21:52:55.403 INFO:tasks.workunit.client.1.smithi138.stdout:4/360: fsync d0/f34 0 2024-05-17T21:52:55.406 INFO:tasks.workunit.client.1.smithi138.stdout:1/319: chown d4/d2e/l59 29 1 2024-05-17T21:52:55.409 INFO:tasks.workunit.client.0.smithi107.stdout:7/474: unlink d5/df/f1a 0 2024-05-17T21:52:55.409 INFO:tasks.workunit.client.1.smithi138.stdout:8/228: dwrite d0/f40 [0,4194304] 0 2024-05-17T21:52:55.410 INFO:tasks.workunit.client.1.smithi138.stdout:3/422: symlink d1/d3/d8/l8f 0 2024-05-17T21:52:55.412 INFO:tasks.workunit.client.0.smithi107.stdout:3/534: dread d6/d8/d10/f4b [0,4194304] 0 2024-05-17T21:52:55.415 INFO:tasks.workunit.client.1.smithi138.stdout:2/363: mkdir d3/d21/d2f/d3e/d58/d77 0 2024-05-17T21:52:55.417 INFO:tasks.workunit.client.1.smithi138.stdout:8/229: dwrite d0/dc/d1a/f24 [4194304,4194304] 0 2024-05-17T21:52:55.418 INFO:tasks.workunit.client.1.smithi138.stdout:2/364: chown d3/d21/d2f/d3e/d55/d62 35158 1 2024-05-17T21:52:55.418 INFO:tasks.workunit.client.0.smithi107.stdout:7/475: dwrite d5/df/d11/d1c/d31/d56/fa8 [0,4194304] 0 2024-05-17T21:52:55.419 INFO:tasks.workunit.client.1.smithi138.stdout:2/365: chown d3/d21/f67 52062414 1 2024-05-17T21:52:55.419 INFO:tasks.workunit.client.1.smithi138.stdout:8/230: chown d0/dc/f2b 52686 1 2024-05-17T21:52:55.421 INFO:tasks.workunit.client.0.smithi107.stdout:7/476: write d5/df/d11/d1c/d31/d39/f5e [2549396,57259] 0 2024-05-17T21:52:55.422 INFO:tasks.workunit.client.0.smithi107.stdout:7/477: dread - d5/df/d11/d1d/f96 zero size 2024-05-17T21:52:55.423 INFO:tasks.workunit.client.0.smithi107.stdout:2/373: mkdir d0/d9/d11/df/d5a/d6d 0 2024-05-17T21:52:55.424 INFO:tasks.workunit.client.0.smithi107.stdout:2/374: write d0/d26/f6c [765655,70521] 0 2024-05-17T21:52:55.429 INFO:tasks.workunit.client.0.smithi107.stdout:7/478: dwrite d5/df/d11/d1d/f55 [0,4194304] 0 2024-05-17T21:52:55.432 INFO:tasks.workunit.client.0.smithi107.stdout:8/501: truncate dc/d21/d35/f2e 558758 0 2024-05-17T21:52:55.433 INFO:tasks.workunit.client.1.smithi138.stdout:2/366: dwrite d3/d21/d2f/d3e/d58/f5d [0,4194304] 0 2024-05-17T21:52:55.433 INFO:tasks.workunit.client.0.smithi107.stdout:1/571: truncate d4/df/d1a/f7b 2073004 0 2024-05-17T21:52:55.434 INFO:tasks.workunit.client.1.smithi138.stdout:0/374: dwrite d3/d10/f20 [0,4194304] 0 2024-05-17T21:52:55.441 INFO:tasks.workunit.client.1.smithi138.stdout:9/315: dwrite d6/d7/f15 [0,4194304] 0 2024-05-17T21:52:55.445 INFO:tasks.workunit.client.1.smithi138.stdout:6/383: symlink d7/db/d43/d5f/d6a/l83 0 2024-05-17T21:52:55.447 INFO:tasks.workunit.client.0.smithi107.stdout:9/522: creat d0/dd/d21/d29/d3e/d45/fbb x:0 0 0 2024-05-17T21:52:55.447 INFO:tasks.workunit.client.1.smithi138.stdout:6/384: readlink d7/d8/d78/l7a 0 2024-05-17T21:52:55.448 INFO:tasks.workunit.client.1.smithi138.stdout:4/361: dread d0/d3/f57 [0,4194304] 0 2024-05-17T21:52:55.449 INFO:tasks.workunit.client.0.smithi107.stdout:6/413: mknod d9/d16/c80 0 2024-05-17T21:52:55.451 INFO:tasks.workunit.client.1.smithi138.stdout:1/320: mkdir d4/dd/d3b/d6b 0 2024-05-17T21:52:55.453 INFO:tasks.workunit.client.1.smithi138.stdout:1/321: truncate d4/df/d4e/f55 813649 0 2024-05-17T21:52:55.454 INFO:tasks.workunit.client.1.smithi138.stdout:1/322: read - d4/dd/d3b/f49 zero size 2024-05-17T21:52:55.454 INFO:tasks.workunit.client.1.smithi138.stdout:3/423: write d1/d3/d16/d3e/f4a [3358771,104809] 0 2024-05-17T21:52:55.455 INFO:tasks.workunit.client.1.smithi138.stdout:1/323: dread - d4/db/f39 zero size 2024-05-17T21:52:55.455 INFO:tasks.workunit.client.0.smithi107.stdout:7/479: symlink d5/df/d11/d1d/d33/d82/laa 0 2024-05-17T21:52:55.455 INFO:tasks.workunit.client.1.smithi138.stdout:1/324: stat d4/d2e/d36 0 2024-05-17T21:52:55.456 INFO:tasks.workunit.client.0.smithi107.stdout:6/414: dwrite d9/d16/d22/d2d/f39 [4194304,4194304] 0 2024-05-17T21:52:55.457 INFO:tasks.workunit.client.1.smithi138.stdout:3/424: truncate d1/d3/d49/d4b/f85 4405602 0 2024-05-17T21:52:55.458 INFO:tasks.workunit.client.0.smithi107.stdout:8/502: mkdir dc/d21/d35/db6 0 2024-05-17T21:52:55.460 INFO:tasks.workunit.client.0.smithi107.stdout:1/572: rename d4/df/d1a/d26/d7a/d81 to d4/df/d1a/d26/d72/dbc 0 2024-05-17T21:52:55.465 INFO:tasks.workunit.client.0.smithi107.stdout:9/523: mknod d0/dd/d21/d29/d57/d5f/cbc 0 2024-05-17T21:52:55.466 INFO:tasks.workunit.client.1.smithi138.stdout:0/375: mkdir d3/d10/d6d/d81 0 2024-05-17T21:52:55.470 INFO:tasks.workunit.client.1.smithi138.stdout:2/367: symlink d3/d21/d2f/d3e/d63/l78 0 2024-05-17T21:52:55.471 INFO:tasks.workunit.client.1.smithi138.stdout:2/368: write d3/d21/f2b [3905673,65234] 0 2024-05-17T21:52:55.471 INFO:tasks.workunit.client.0.smithi107.stdout:7/480: mknod d5/df/d11/d1d/d44/cab 0 2024-05-17T21:52:55.472 INFO:tasks.workunit.client.1.smithi138.stdout:2/369: chown d3/d21/d2f/d3e/d58 84937 1 2024-05-17T21:52:55.475 INFO:tasks.workunit.client.1.smithi138.stdout:0/376: dread d3/d3d/d51/f6e [0,4194304] 0 2024-05-17T21:52:55.477 INFO:tasks.workunit.client.1.smithi138.stdout:6/385: symlink d7/db/d43/d5f/d6a/l84 0 2024-05-17T21:52:55.479 INFO:tasks.workunit.client.0.smithi107.stdout:9/524: rename d0/dd/d21/d29/d3e/d45/fbb to d0/dd/d69/db2/fbd 0 2024-05-17T21:52:55.481 INFO:tasks.workunit.client.0.smithi107.stdout:6/415: mkdir d9/d81 0 2024-05-17T21:52:55.482 INFO:tasks.workunit.client.1.smithi138.stdout:1/325: unlink d4/dd/l27 0 2024-05-17T21:52:55.488 INFO:tasks.workunit.client.1.smithi138.stdout:8/231: write d0/d11/f28 [1316096,92528] 0 2024-05-17T21:52:55.490 INFO:tasks.workunit.client.0.smithi107.stdout:9/525: write d0/fa5 [738623,92684] 0 2024-05-17T21:52:55.490 INFO:tasks.workunit.client.1.smithi138.stdout:7/271: write d0/f24 [3883927,63562] 0 2024-05-17T21:52:55.491 INFO:tasks.workunit.client.0.smithi107.stdout:9/526: chown d0/dd/d21/d29/d57/d5f/l86 27687118 1 2024-05-17T21:52:55.491 INFO:tasks.workunit.client.1.smithi138.stdout:7/272: write d0/f12 [6061624,6854] 0 2024-05-17T21:52:55.493 INFO:tasks.workunit.client.0.smithi107.stdout:7/481: mkdir d5/d16/d3b/d45/d91/dac 0 2024-05-17T21:52:55.493 INFO:tasks.workunit.client.0.smithi107.stdout:3/535: truncate d6/d8/d10/f30 4047525 0 2024-05-17T21:52:55.497 INFO:tasks.workunit.client.1.smithi138.stdout:2/370: creat d3/d21/d2f/d3e/d63/d65/f79 x:0 0 0 2024-05-17T21:52:55.498 INFO:tasks.workunit.client.1.smithi138.stdout:2/371: dread - d3/d21/d2f/d3e/d58/f6b zero size 2024-05-17T21:52:55.498 INFO:tasks.workunit.client.0.smithi107.stdout:1/573: creat d4/df/d1a/fbd x:0 0 0 2024-05-17T21:52:55.500 INFO:tasks.workunit.client.1.smithi138.stdout:0/377: rename d3/l79 to d3/d10/d39/l82 0 2024-05-17T21:52:55.501 INFO:tasks.workunit.client.1.smithi138.stdout:0/378: truncate d3/d10/f1a 5028233 0 2024-05-17T21:52:55.502 INFO:tasks.workunit.client.0.smithi107.stdout:6/416: creat d9/d81/f82 x:0 0 0 2024-05-17T21:52:55.503 INFO:tasks.workunit.client.0.smithi107.stdout:9/527: mknod d0/dd/d1b/cbe 0 2024-05-17T21:52:55.504 INFO:tasks.workunit.client.0.smithi107.stdout:9/528: dread - d0/dd/d1b/f74 zero size 2024-05-17T21:52:55.504 INFO:tasks.workunit.client.1.smithi138.stdout:4/362: mkdir d0/d3/d17/d5e/d7f 0 2024-05-17T21:52:55.508 INFO:tasks.workunit.client.0.smithi107.stdout:3/536: creat d6/d2a/d2d/d37/fab x:0 0 0 2024-05-17T21:52:55.509 INFO:tasks.workunit.client.1.smithi138.stdout:5/333: getdents d1/d6/d1b/d29/d36 0 2024-05-17T21:52:55.510 INFO:tasks.workunit.client.0.smithi107.stdout:8/503: getdents dc/d21/d97/d63 0 2024-05-17T21:52:55.511 INFO:tasks.workunit.client.1.smithi138.stdout:8/232: creat d0/d11/f42 x:0 0 0 2024-05-17T21:52:55.513 INFO:tasks.workunit.client.1.smithi138.stdout:8/233: chown d0/d11/d3e 0 1 2024-05-17T21:52:55.514 INFO:tasks.workunit.client.1.smithi138.stdout:3/425: dwrite d1/d3/d49/d4b/f85 [4194304,4194304] 0 2024-05-17T21:52:55.516 INFO:tasks.workunit.client.1.smithi138.stdout:8/234: chown d0/dc/f18 0 1 2024-05-17T21:52:55.520 INFO:tasks.workunit.client.0.smithi107.stdout:9/529: chown d0/l19 7913316 1 2024-05-17T21:52:55.521 INFO:tasks.workunit.client.1.smithi138.stdout:1/326: dread d4/f3f [0,4194304] 0 2024-05-17T21:52:55.522 INFO:tasks.workunit.client.1.smithi138.stdout:5/334: dwrite d1/d1a/d39/d4e/f51 [0,4194304] 0 2024-05-17T21:52:55.522 INFO:tasks.workunit.client.1.smithi138.stdout:2/372: mknod d3/d21/d2f/d3e/d63/d65/c7a 0 2024-05-17T21:52:55.525 INFO:tasks.workunit.client.0.smithi107.stdout:7/482: fdatasync f4 0 2024-05-17T21:52:55.526 INFO:tasks.workunit.client.0.smithi107.stdout:7/483: write d5/df/d11/d1d/d44/f50 [9245807,81908] 0 2024-05-17T21:52:55.527 INFO:tasks.workunit.client.1.smithi138.stdout:2/373: write d3/d21/d2f/f3f [662294,72587] 0 2024-05-17T21:52:55.528 INFO:tasks.workunit.client.1.smithi138.stdout:9/316: dwrite d6/f24 [0,4194304] 0 2024-05-17T21:52:55.530 INFO:tasks.workunit.client.1.smithi138.stdout:6/386: creat d7/db/d43/d5f/d6a/d77/f85 x:0 0 0 2024-05-17T21:52:55.530 INFO:tasks.workunit.client.0.smithi107.stdout:3/537: read d6/d8/d10/d25/f49 [137033,49564] 0 2024-05-17T21:52:55.531 INFO:tasks.workunit.client.1.smithi138.stdout:0/379: dread d3/d10/f25 [0,4194304] 0 2024-05-17T21:52:55.532 INFO:tasks.workunit.client.0.smithi107.stdout:8/504: chown dc/d21/d27/d83/ca7 7615700 1 2024-05-17T21:52:55.534 INFO:tasks.workunit.client.0.smithi107.stdout:1/574: write d4/df/d1a/d26/d72/daa/f9d [8400278,11351] 0 2024-05-17T21:52:55.535 INFO:tasks.workunit.client.1.smithi138.stdout:4/363: mknod d0/d2f/d49/d60/d1f/d32/d71/c80 0 2024-05-17T21:52:55.535 INFO:tasks.workunit.client.1.smithi138.stdout:0/380: write d3/d3d/f53 [2791442,69632] 0 2024-05-17T21:52:55.536 INFO:tasks.workunit.client.0.smithi107.stdout:1/575: chown d4/df/d14/l6b 463288 1 2024-05-17T21:52:55.536 INFO:tasks.workunit.client.1.smithi138.stdout:2/374: write d3/d21/d2f/f50 [7600138,103985] 0 2024-05-17T21:52:55.539 INFO:tasks.workunit.client.0.smithi107.stdout:1/576: truncate d4/df/d1a/fbd 61589 0 2024-05-17T21:52:55.539 INFO:tasks.workunit.client.0.smithi107.stdout:3/538: dwrite d6/d2a/d2d/d37/f3a [0,4194304] 0 2024-05-17T21:52:55.539 INFO:tasks.workunit.client.0.smithi107.stdout:9/530: symlink d0/dd/d21/d29/lbf 0 2024-05-17T21:52:55.540 INFO:tasks.workunit.client.1.smithi138.stdout:5/335: dwrite d1/d6/d19/f20 [0,4194304] 0 2024-05-17T21:52:55.546 INFO:tasks.workunit.client.0.smithi107.stdout:0/579: sync 2024-05-17T21:52:55.548 INFO:tasks.workunit.client.0.smithi107.stdout:0/580: dread d3/f1d [0,4194304] 0 2024-05-17T21:52:55.548 INFO:tasks.workunit.client.1.smithi138.stdout:1/327: truncate d4/fa 7603958 0 2024-05-17T21:52:55.550 INFO:tasks.workunit.client.1.smithi138.stdout:0/381: dwrite d3/f5e [0,4194304] 0 2024-05-17T21:52:55.552 INFO:tasks.workunit.client.0.smithi107.stdout:8/505: mkdir dc/d21/d97/db7 0 2024-05-17T21:52:55.555 INFO:tasks.workunit.client.0.smithi107.stdout:0/581: dwrite d3/f33 [0,4194304] 0 2024-05-17T21:52:55.556 INFO:tasks.workunit.client.0.smithi107.stdout:6/417: rmdir d9/d16/d4d/d73 0 2024-05-17T21:52:55.557 INFO:tasks.workunit.client.1.smithi138.stdout:6/387: mknod d7/db/d43/c86 0 2024-05-17T21:52:55.557 INFO:tasks.workunit.client.0.smithi107.stdout:6/418: chown d9/d58/d66/l6d 75503699 1 2024-05-17T21:52:55.559 INFO:tasks.workunit.client.0.smithi107.stdout:1/577: unlink d4/df/d4e/d59/fac 0 2024-05-17T21:52:55.559 INFO:tasks.workunit.client.0.smithi107.stdout:1/578: readlink d4/df/d14/d1d/l71 0 2024-05-17T21:52:55.566 INFO:tasks.workunit.client.0.smithi107.stdout:3/539: rename d6/d1a/l4a to d6/d8/d10/d35/d36/d66/lac 0 2024-05-17T21:52:55.566 INFO:tasks.workunit.client.1.smithi138.stdout:9/317: write d6/f2a [993308,30395] 0 2024-05-17T21:52:55.569 INFO:tasks.workunit.client.1.smithi138.stdout:7/273: creat d0/f4b x:0 0 0 2024-05-17T21:52:55.570 INFO:tasks.workunit.client.1.smithi138.stdout:8/235: creat d0/dc/d3b/f43 x:0 0 0 2024-05-17T21:52:55.573 INFO:tasks.workunit.client.1.smithi138.stdout:5/336: creat d1/d6/d1b/d29/d36/d68/f6f x:0 0 0 2024-05-17T21:52:55.573 INFO:tasks.workunit.client.1.smithi138.stdout:8/236: dread d0/d11/f28 [0,4194304] 0 2024-05-17T21:52:55.575 INFO:tasks.workunit.client.1.smithi138.stdout:5/337: write d1/d6/d19/f20 [4437810,17649] 0 2024-05-17T21:52:55.575 INFO:tasks.workunit.client.1.smithi138.stdout:8/237: write d0/f1b [1110504,35282] 0 2024-05-17T21:52:55.575 INFO:tasks.workunit.client.0.smithi107.stdout:7/484: creat d5/df/d11/d1c/d2f/d65/da7/fad x:0 0 0 2024-05-17T21:52:55.576 INFO:tasks.workunit.client.0.smithi107.stdout:6/419: fsync d9/f1c 0 2024-05-17T21:52:55.577 INFO:tasks.workunit.client.0.smithi107.stdout:7/485: chown d5/df/d11/d1c/c60 292 1 2024-05-17T21:52:55.580 INFO:tasks.workunit.client.1.smithi138.stdout:0/382: write d3/d10/f11 [2381017,41690] 0 2024-05-17T21:52:55.580 INFO:tasks.workunit.client.0.smithi107.stdout:1/579: creat d4/df/d1a/d26/d72/fbe x:0 0 0 2024-05-17T21:52:55.581 INFO:tasks.workunit.client.1.smithi138.stdout:4/364: write d0/f2a [3024352,54274] 0 2024-05-17T21:52:55.582 INFO:tasks.workunit.client.0.smithi107.stdout:1/580: write d4/df/d4e/f63 [2265167,120975] 0 2024-05-17T21:52:55.582 INFO:tasks.workunit.client.1.smithi138.stdout:6/388: creat d7/db/d43/d63/f87 x:0 0 0 2024-05-17T21:52:55.583 INFO:tasks.workunit.client.1.smithi138.stdout:6/389: write d7/db/d43/d3f/f5b [964610,43582] 0 2024-05-17T21:52:55.584 INFO:tasks.workunit.client.0.smithi107.stdout:1/581: dread d4/df/d1a/d26/d7a/fb5 [0,4194304] 0 2024-05-17T21:52:55.585 INFO:tasks.workunit.client.0.smithi107.stdout:3/540: mkdir d6/d2a/d2d/d69/dad 0 2024-05-17T21:52:55.585 INFO:tasks.workunit.client.1.smithi138.stdout:2/375: write f2 [2237082,38864] 0 2024-05-17T21:52:55.589 INFO:tasks.workunit.client.0.smithi107.stdout:6/420: creat d9/d16/d22/d2d/f83 x:0 0 0 2024-05-17T21:52:55.590 INFO:tasks.workunit.client.0.smithi107.stdout:6/421: chown d9/f4e 143781 1 2024-05-17T21:52:55.590 INFO:tasks.workunit.client.0.smithi107.stdout:6/422: chown d9/f4e 1 1 2024-05-17T21:52:55.590 INFO:tasks.workunit.client.0.smithi107.stdout:6/423: chown d9/d16/f64 3 1 2024-05-17T21:52:55.593 INFO:tasks.workunit.client.1.smithi138.stdout:3/426: rename d1/d3/d8/d72/c89 to d1/d3/d49/d4b/c90 0 2024-05-17T21:52:55.593 INFO:tasks.workunit.client.0.smithi107.stdout:9/531: truncate d0/dd/d1b/f65 3609227 0 2024-05-17T21:52:55.594 INFO:tasks.workunit.client.0.smithi107.stdout:9/532: readlink d0/dd/d2f/l6d 0 2024-05-17T21:52:55.594 INFO:tasks.workunit.client.1.smithi138.stdout:3/427: read - d1/d3/d8/de/f8e zero size 2024-05-17T21:52:55.597 INFO:tasks.workunit.client.1.smithi138.stdout:2/376: dread d3/f5 [0,4194304] 0 2024-05-17T21:52:55.598 INFO:tasks.workunit.client.0.smithi107.stdout:3/541: symlink d6/d8/d10/lae 0 2024-05-17T21:52:55.598 INFO:tasks.workunit.client.0.smithi107.stdout:0/582: write d3/dc/d22/d23/d97/d85/fb3 [576397,107378] 0 2024-05-17T21:52:55.598 INFO:tasks.workunit.client.1.smithi138.stdout:5/338: rmdir d1 39 2024-05-17T21:52:55.599 INFO:tasks.workunit.client.1.smithi138.stdout:2/377: chown d3/d21/d2f/d3e/d58/f71 1108557 1 2024-05-17T21:52:55.599 INFO:tasks.workunit.client.0.smithi107.stdout:0/583: fsync f1 0 2024-05-17T21:52:55.599 INFO:tasks.workunit.client.1.smithi138.stdout:2/378: chown d3/d21/d2f/f5f 457459229 1 2024-05-17T21:52:55.600 INFO:tasks.workunit.client.1.smithi138.stdout:8/238: mkdir d0/dc/d1a/d44 0 2024-05-17T21:52:55.600 INFO:tasks.workunit.client.0.smithi107.stdout:6/424: symlink d9/d58/l84 0 2024-05-17T21:52:55.601 INFO:tasks.workunit.client.0.smithi107.stdout:3/542: read d6/d2a/d2d/f41 [8319559,7513] 0 2024-05-17T21:52:55.601 INFO:tasks.workunit.client.0.smithi107.stdout:9/533: dwrite d0/dd/d2f/d87/f98 [0,4194304] 0 2024-05-17T21:52:55.602 INFO:tasks.workunit.client.0.smithi107.stdout:0/584: write d3/dc/d22/d23/d97/f96 [794451,6932] 0 2024-05-17T21:52:55.603 INFO:tasks.workunit.client.0.smithi107.stdout:6/425: chown d9/d16/d22/d2d/f83 390 1 2024-05-17T21:52:55.604 INFO:tasks.workunit.client.0.smithi107.stdout:6/426: dread - d9/d16/f7f zero size 2024-05-17T21:52:55.604 INFO:tasks.workunit.client.0.smithi107.stdout:1/582: link d4/df/d1a/d41/f48 d4/df/d69/fbf 0 2024-05-17T21:52:55.605 INFO:tasks.workunit.client.1.smithi138.stdout:4/365: symlink d0/d3/d17/d5e/l81 0 2024-05-17T21:52:55.609 INFO:tasks.workunit.client.1.smithi138.stdout:7/274: truncate d0/f1f 4527262 0 2024-05-17T21:52:55.610 INFO:tasks.workunit.client.0.smithi107.stdout:1/583: dwrite d4/df/d1a/d26/d72/daa/f9d [0,4194304] 0 2024-05-17T21:52:55.618 INFO:tasks.workunit.client.0.smithi107.stdout:3/543: write d6/d8/d10/d25/f55 [3128919,64940] 0 2024-05-17T21:52:55.618 INFO:tasks.workunit.client.0.smithi107.stdout:1/584: dwrite d4/de/f36 [0,4194304] 0 2024-05-17T21:52:55.619 INFO:tasks.workunit.client.1.smithi138.stdout:3/428: symlink d1/d4f/d50/l91 0 2024-05-17T21:52:55.623 INFO:tasks.workunit.client.1.smithi138.stdout:5/339: fsync d1/d53/f6e 0 2024-05-17T21:52:55.624 INFO:tasks.workunit.client.0.smithi107.stdout:6/427: mkdir d9/d16/d4d/d85 0 2024-05-17T21:52:55.627 INFO:tasks.workunit.client.1.smithi138.stdout:8/239: creat d0/d2f/f45 x:0 0 0 2024-05-17T21:52:55.628 INFO:tasks.workunit.client.1.smithi138.stdout:8/240: truncate d0/d2f/f3c 543849 0 2024-05-17T21:52:55.629 INFO:tasks.workunit.client.1.smithi138.stdout:2/379: truncate d3/f29 511736 0 2024-05-17T21:52:55.633 INFO:tasks.workunit.client.0.smithi107.stdout:9/534: write d0/f7 [401931,118884] 0 2024-05-17T21:52:55.639 INFO:tasks.workunit.client.0.smithi107.stdout:4/507: sync 2024-05-17T21:52:55.639 INFO:tasks.workunit.client.0.smithi107.stdout:2/375: sync 2024-05-17T21:52:55.639 INFO:tasks.workunit.client.0.smithi107.stdout:5/510: sync 2024-05-17T21:52:55.639 INFO:tasks.workunit.client.1.smithi138.stdout:1/328: rename d4/db/d29/d47 to d4/dd/d3b/d6c 0 2024-05-17T21:52:55.640 INFO:tasks.workunit.client.0.smithi107.stdout:0/585: dread - d3/dc/d31/f63 zero size 2024-05-17T21:52:55.641 INFO:tasks.workunit.client.0.smithi107.stdout:2/376: write d0/d9/d11/f5f [2844189,98672] 0 2024-05-17T21:52:55.641 INFO:tasks.workunit.client.0.smithi107.stdout:0/586: stat d3/f35 0 2024-05-17T21:52:55.642 INFO:tasks.workunit.client.0.smithi107.stdout:4/508: read d3/db/d1e/d66/f70 [2061981,72831] 0 2024-05-17T21:52:55.642 INFO:tasks.workunit.client.1.smithi138.stdout:1/329: dread f0 [0,4194304] 0 2024-05-17T21:52:55.643 INFO:tasks.workunit.client.0.smithi107.stdout:6/428: creat d9/d16/d4d/d67/f86 x:0 0 0 2024-05-17T21:52:55.645 INFO:tasks.workunit.client.0.smithi107.stdout:6/429: write d9/f7c [160436,44634] 0 2024-05-17T21:52:55.645 INFO:tasks.workunit.client.1.smithi138.stdout:8/241: symlink d0/d2f/l46 0 2024-05-17T21:52:55.648 INFO:tasks.workunit.client.1.smithi138.stdout:2/380: creat d3/d21/d2f/d3e/d63/f7b x:0 0 0 2024-05-17T21:52:55.650 INFO:tasks.workunit.client.0.smithi107.stdout:3/544: dread d6/d2a/f42 [0,4194304] 0 2024-05-17T21:52:55.651 INFO:tasks.workunit.client.0.smithi107.stdout:2/377: dwrite d0/d9/d11/df/f47 [4194304,4194304] 0 2024-05-17T21:52:55.651 INFO:tasks.workunit.client.1.smithi138.stdout:4/366: link d0/d2f/d49/d60/d1f/f4d d0/d2f/f82 0 2024-05-17T21:52:55.652 INFO:tasks.workunit.client.1.smithi138.stdout:4/367: stat d0/d2f/d49/d60/f41 0 2024-05-17T21:52:55.652 INFO:tasks.workunit.client.0.smithi107.stdout:9/535: dread d0/f17 [0,4194304] 0 2024-05-17T21:52:55.652 INFO:tasks.workunit.client.0.smithi107.stdout:2/378: chown d0/d9/d11/d27/c32 29308073 1 2024-05-17T21:52:55.654 INFO:tasks.workunit.client.1.smithi138.stdout:7/275: mkdir d0/d4c 0 2024-05-17T21:52:55.655 INFO:tasks.workunit.client.0.smithi107.stdout:5/511: truncate d5/db/d16/d54/d59/d65/f47 476361 0 2024-05-17T21:52:55.658 INFO:tasks.workunit.client.1.smithi138.stdout:3/429: mkdir d1/d3/d8d/d92 0 2024-05-17T21:52:55.658 INFO:tasks.workunit.client.0.smithi107.stdout:4/509: write d3/db/d35/d5e/f96 [2534572,11176] 0 2024-05-17T21:52:55.659 INFO:tasks.workunit.client.0.smithi107.stdout:0/587: mkdir d3/dc/d22/d23/d97/d85/dd4 0 2024-05-17T21:52:55.660 INFO:tasks.workunit.client.0.smithi107.stdout:0/588: chown d3/dc/d22/d3d/dc7/fcc 1 1 2024-05-17T21:52:55.661 INFO:tasks.workunit.client.0.smithi107.stdout:0/589: chown d3/dc/d22/d23/d97/d85 213404915 1 2024-05-17T21:52:55.662 INFO:tasks.workunit.client.1.smithi138.stdout:5/340: symlink d1/d6/d1b/l70 0 2024-05-17T21:52:55.663 INFO:tasks.workunit.client.0.smithi107.stdout:6/430: creat d9/d16/d22/d5f/d6e/f87 x:0 0 0 2024-05-17T21:52:55.664 INFO:tasks.workunit.client.1.smithi138.stdout:8/242: mknod d0/dc/d1a/c47 0 2024-05-17T21:52:55.667 INFO:tasks.workunit.client.0.smithi107.stdout:5/512: dread d5/db/d16/d5b/f53 [0,4194304] 0 2024-05-17T21:52:55.668 INFO:tasks.workunit.client.1.smithi138.stdout:0/383: getdents d3/d10/d38/d70 0 2024-05-17T21:52:55.668 INFO:tasks.workunit.client.1.smithi138.stdout:1/330: dwrite d4/f9 [4194304,4194304] 0 2024-05-17T21:52:55.669 INFO:tasks.workunit.client.0.smithi107.stdout:3/545: symlink d6/d8/d10/d35/d36/d4c/laf 0 2024-05-17T21:52:55.669 INFO:tasks.workunit.client.1.smithi138.stdout:2/381: dread d3/f10 [0,4194304] 0 2024-05-17T21:52:55.673 INFO:tasks.workunit.client.1.smithi138.stdout:5/341: dwrite d1/f45 [0,4194304] 0 2024-05-17T21:52:55.676 INFO:tasks.workunit.client.1.smithi138.stdout:1/331: dread d4/db/f2a [0,4194304] 0 2024-05-17T21:52:55.676 INFO:tasks.workunit.client.0.smithi107.stdout:3/546: chown d6/d2a/f5b 51944635 1 2024-05-17T21:52:55.676 INFO:tasks.workunit.client.1.smithi138.stdout:1/332: stat d4/df/d35/c4a 0 2024-05-17T21:52:55.677 INFO:tasks.workunit.client.1.smithi138.stdout:9/318: rename d6/d7/d16/d29/c35 to d6/d7/d16/d29/d38/d3c/d3e/d5a/c65 0 2024-05-17T21:52:55.679 INFO:tasks.workunit.client.1.smithi138.stdout:3/430: mknod d1/d3/d49/d4b/c93 0 2024-05-17T21:52:55.680 INFO:tasks.workunit.client.0.smithi107.stdout:3/547: stat d6/d8/d10/d25/c51 0 2024-05-17T21:52:55.683 INFO:tasks.workunit.client.0.smithi107.stdout:1/585: link d4/df/d14/d1d/f2c d4/df/d1a/d26/d72/d85/fc0 0 2024-05-17T21:52:55.685 INFO:tasks.workunit.client.0.smithi107.stdout:2/379: dread d0/d26/f55 [0,4194304] 0 2024-05-17T21:52:55.686 INFO:tasks.workunit.client.0.smithi107.stdout:5/513: mkdir d5/db/d16/d2d/d40/d9f 0 2024-05-17T21:52:55.686 INFO:tasks.workunit.client.1.smithi138.stdout:3/431: dwrite d1/d3/d8/d5c/f81 [0,4194304] 0 2024-05-17T21:52:55.688 INFO:tasks.workunit.client.1.smithi138.stdout:3/432: write d1/d3/d49/f7d [6263969,45398] 0 2024-05-17T21:52:55.690 INFO:tasks.workunit.client.0.smithi107.stdout:1/586: write d4/df/f4d [3589495,110617] 0 2024-05-17T21:52:55.690 INFO:tasks.workunit.client.0.smithi107.stdout:2/380: dread - d0/d9/d11/f52 zero size 2024-05-17T21:52:55.692 INFO:tasks.workunit.client.1.smithi138.stdout:8/243: dwrite d0/fa [0,4194304] 0 2024-05-17T21:52:55.694 INFO:tasks.workunit.client.1.smithi138.stdout:8/244: write d0/dc/f10 [4370519,54713] 0 2024-05-17T21:52:55.694 INFO:tasks.workunit.client.1.smithi138.stdout:4/368: creat d0/d2f/d49/d60/d1f/d7e/f83 x:0 0 0 2024-05-17T21:52:55.696 INFO:tasks.workunit.client.0.smithi107.stdout:0/590: dwrite d3/dc/d22/f98 [0,4194304] 0 2024-05-17T21:52:55.697 INFO:tasks.workunit.client.0.smithi107.stdout:6/431: dwrite d9/d16/f5e [0,4194304] 0 2024-05-17T21:52:55.698 INFO:tasks.workunit.client.1.smithi138.stdout:4/369: read d0/d2f/d49/d60/d1f/f70 [53203,25538] 0 2024-05-17T21:52:55.698 INFO:tasks.workunit.client.0.smithi107.stdout:1/587: write d4/df/d1a/d26/f7c [2913038,130165] 0 2024-05-17T21:52:55.701 INFO:tasks.workunit.client.0.smithi107.stdout:4/510: dread d3/f9 [0,4194304] 0 2024-05-17T21:52:55.704 INFO:tasks.workunit.client.0.smithi107.stdout:4/511: write d3/db/d1e/d66/f70 [3958903,121328] 0 2024-05-17T21:52:55.705 INFO:tasks.workunit.client.1.smithi138.stdout:5/342: rename d1/d1a/d39/d62/f66 to d1/d46/f71 0 2024-05-17T21:52:55.706 INFO:tasks.workunit.client.0.smithi107.stdout:0/591: readlink d3/dc/l36 0 2024-05-17T21:52:55.706 INFO:tasks.workunit.client.0.smithi107.stdout:3/548: dwrite d6/d8/d10/d35/d36/f8e [0,4194304] 0 2024-05-17T21:52:55.708 INFO:tasks.workunit.client.0.smithi107.stdout:4/512: write d3/d4a/d47/d60/d3a/d41/d80/f61 [3127722,77868] 0 2024-05-17T21:52:55.708 INFO:tasks.workunit.client.1.smithi138.stdout:2/382: dread d3/f5 [0,4194304] 0 2024-05-17T21:52:55.710 INFO:tasks.workunit.client.1.smithi138.stdout:5/343: write d1/d6/d1b/d29/f67 [535773,67517] 0 2024-05-17T21:52:55.713 INFO:tasks.workunit.client.0.smithi107.stdout:4/513: dread d3/f2c [0,4194304] 0 2024-05-17T21:52:55.714 INFO:tasks.workunit.client.1.smithi138.stdout:5/344: truncate d1/d6/d1b/d29/f67 5251272 0 2024-05-17T21:52:55.717 INFO:tasks.workunit.client.1.smithi138.stdout:7/276: creat d0/f4d x:0 0 0 2024-05-17T21:52:55.719 INFO:tasks.workunit.client.1.smithi138.stdout:8/245: mknod d0/d2f/c48 0 2024-05-17T21:52:55.720 INFO:tasks.workunit.client.1.smithi138.stdout:8/246: truncate d0/dc/d1a/f2a 836586 0 2024-05-17T21:52:55.723 INFO:tasks.workunit.client.1.smithi138.stdout:4/370: fdatasync d0/d1b/d42/f7c 0 2024-05-17T21:52:55.727 INFO:tasks.workunit.client.0.smithi107.stdout:9/536: mknod d0/dd/d2f/cc0 0 2024-05-17T21:52:55.728 INFO:tasks.workunit.client.1.smithi138.stdout:1/333: dread d4/dd/f15 [0,4194304] 0 2024-05-17T21:52:55.729 INFO:tasks.workunit.client.1.smithi138.stdout:1/334: readlink d4/df/l23 0 2024-05-17T21:52:55.729 INFO:tasks.workunit.client.1.smithi138.stdout:1/335: chown d4/df/d35/f37 443 1 2024-05-17T21:52:55.733 INFO:tasks.workunit.client.1.smithi138.stdout:0/384: creat d3/f83 x:0 0 0 2024-05-17T21:52:55.738 INFO:tasks.workunit.client.0.smithi107.stdout:5/514: dread d5/f8 [0,4194304] 0 2024-05-17T21:52:55.740 INFO:tasks.workunit.client.0.smithi107.stdout:8/506: sync 2024-05-17T21:52:55.740 INFO:tasks.workunit.client.0.smithi107.stdout:7/486: sync 2024-05-17T21:52:55.740 INFO:tasks.workunit.client.0.smithi107.stdout:2/381: symlink d0/d9/d1c/d22/l6e 0 2024-05-17T21:52:55.741 INFO:tasks.workunit.client.0.smithi107.stdout:7/487: chown d5/d16/d3b/d45/d91 21172942 1 2024-05-17T21:52:55.747 INFO:tasks.workunit.client.1.smithi138.stdout:4/371: dread d0/d2f/d49/d60/d1f/d32/f64 [0,4194304] 0 2024-05-17T21:52:55.749 INFO:tasks.workunit.client.0.smithi107.stdout:3/549: creat d6/d8/d10/d35/fb0 x:0 0 0 2024-05-17T21:52:55.750 INFO:tasks.workunit.client.1.smithi138.stdout:2/383: mknod d3/d21/d2f/d3e/d55/c7c 0 2024-05-17T21:52:55.750 INFO:tasks.workunit.client.0.smithi107.stdout:3/550: dread - d6/d8/d3e/d3f/d90/f9f zero size 2024-05-17T21:52:55.751 INFO:tasks.workunit.client.1.smithi138.stdout:3/433: write d1/d4f/d50/f54 [755092,36796] 0 2024-05-17T21:52:55.753 INFO:tasks.workunit.client.1.smithi138.stdout:7/277: creat d0/d18/d40/f4e x:0 0 0 2024-05-17T21:52:55.755 INFO:tasks.workunit.client.1.smithi138.stdout:8/247: mknod d0/dc/d1a/d44/c49 0 2024-05-17T21:52:55.756 INFO:tasks.workunit.client.0.smithi107.stdout:8/507: chown dc/d44/d4b/l72 18401 1 2024-05-17T21:52:55.756 INFO:tasks.workunit.client.1.smithi138.stdout:1/336: mkdir d4/df/d3a/d3c/d63/d6d 0 2024-05-17T21:52:55.757 INFO:tasks.workunit.client.0.smithi107.stdout:0/592: truncate d3/d39/f7f 1158991 0 2024-05-17T21:52:55.758 INFO:tasks.workunit.client.1.smithi138.stdout:4/372: mkdir d0/d1b/d42/d45/d84 0 2024-05-17T21:52:55.760 INFO:tasks.workunit.client.0.smithi107.stdout:7/488: read d5/d16/f3a [358460,86222] 0 2024-05-17T21:52:55.761 INFO:tasks.workunit.client.1.smithi138.stdout:2/384: symlink d3/d21/d2f/d3e/d58/l7d 0 2024-05-17T21:52:55.762 INFO:tasks.workunit.client.1.smithi138.stdout:2/385: write d3/f4 [1719784,6848] 0 2024-05-17T21:52:55.762 INFO:tasks.workunit.client.1.smithi138.stdout:5/345: creat d1/d6/f72 x:0 0 0 2024-05-17T21:52:55.763 INFO:tasks.workunit.client.1.smithi138.stdout:9/319: truncate d6/d7/d16/f20 1459874 0 2024-05-17T21:52:55.763 INFO:tasks.workunit.client.1.smithi138.stdout:5/346: write d1/d6/d1b/f54 [5064089,60185] 0 2024-05-17T21:52:55.764 INFO:tasks.workunit.client.0.smithi107.stdout:1/588: mkdir d4/df/dc1 0 2024-05-17T21:52:55.764 INFO:tasks.workunit.client.1.smithi138.stdout:5/347: fdatasync d1/d6/d1b/f21 0 2024-05-17T21:52:55.765 INFO:tasks.workunit.client.1.smithi138.stdout:3/434: rmdir d1/d3/d16/d5d 39 2024-05-17T21:52:55.769 INFO:tasks.workunit.client.1.smithi138.stdout:8/248: mkdir d0/dc/d1a/d33/d4a 0 2024-05-17T21:52:55.770 INFO:tasks.workunit.client.1.smithi138.stdout:8/249: readlink d0/d2f/l3d 0 2024-05-17T21:52:55.770 INFO:tasks.workunit.client.1.smithi138.stdout:8/250: dread - d0/d2f/f45 zero size 2024-05-17T21:52:55.777 INFO:tasks.workunit.client.0.smithi107.stdout:0/593: dread - d3/d58/f84 zero size 2024-05-17T21:52:55.777 INFO:tasks.workunit.client.1.smithi138.stdout:4/373: dwrite d0/d2f/d49/d60/d1f/f70 [0,4194304] 0 2024-05-17T21:52:55.777 INFO:tasks.workunit.client.1.smithi138.stdout:4/374: dread - d0/d1b/d42/f7c zero size 2024-05-17T21:52:55.779 INFO:tasks.workunit.client.0.smithi107.stdout:6/432: write d9/d16/d22/f74 [2850817,102147] 0 2024-05-17T21:52:55.779 INFO:tasks.workunit.client.1.smithi138.stdout:4/375: write d0/d2f/d49/d60/f51 [2068436,89204] 0 2024-05-17T21:52:55.780 INFO:tasks.workunit.client.0.smithi107.stdout:6/433: chown d9/d16/d22/d5f/c78 12899414 1 2024-05-17T21:52:55.781 INFO:tasks.workunit.client.1.smithi138.stdout:0/385: write d3/d3d/f7f [383749,124984] 0 2024-05-17T21:52:55.782 INFO:tasks.workunit.client.1.smithi138.stdout:3/435: symlink d1/d3/d49/d4b/l94 0 2024-05-17T21:52:55.785 INFO:tasks.workunit.client.1.smithi138.stdout:7/278: unlink d0/c46 0 2024-05-17T21:52:55.788 INFO:tasks.workunit.client.1.smithi138.stdout:8/251: rmdir d0/d31 39 2024-05-17T21:52:55.789 INFO:tasks.workunit.client.1.smithi138.stdout:8/252: fdatasync d0/dc/f2b 0 2024-05-17T21:52:55.790 INFO:tasks.workunit.client.0.smithi107.stdout:8/508: mkdir dc/d21/db8 0 2024-05-17T21:52:55.790 INFO:tasks.workunit.client.0.smithi107.stdout:4/514: dwrite d3/d4a/d47/d60/d3a/d41/d80/f19 [0,4194304] 0 2024-05-17T21:52:55.791 INFO:tasks.workunit.client.0.smithi107.stdout:8/509: chown dc/d21/d68/l88 1592269 1 2024-05-17T21:52:55.792 INFO:tasks.workunit.client.0.smithi107.stdout:4/515: dread - d3/d4a/d47/d60/d3a/d41/d80/d43/f9e zero size 2024-05-17T21:52:55.793 INFO:tasks.workunit.client.1.smithi138.stdout:8/253: dread d0/fa [0,4194304] 0 2024-05-17T21:52:55.794 INFO:tasks.workunit.client.1.smithi138.stdout:8/254: fsync d0/dc/f22 0 2024-05-17T21:52:55.795 INFO:tasks.workunit.client.1.smithi138.stdout:8/255: write d0/f17 [346235,74368] 0 2024-05-17T21:52:55.796 INFO:tasks.workunit.client.1.smithi138.stdout:8/256: dread d0/dc/d1a/d33/f34 [0,4194304] 0 2024-05-17T21:52:55.798 INFO:tasks.workunit.client.1.smithi138.stdout:8/257: write d0/d11/f42 [431836,82049] 0 2024-05-17T21:52:55.798 INFO:tasks.workunit.client.1.smithi138.stdout:9/320: mknod d6/c66 0 2024-05-17T21:52:55.799 INFO:tasks.workunit.client.1.smithi138.stdout:9/321: chown d6/d7/f15 387234 1 2024-05-17T21:52:55.801 INFO:tasks.workunit.client.0.smithi107.stdout:6/434: read d9/d16/d22/d2d/f2f [3171746,116429] 0 2024-05-17T21:52:55.802 INFO:tasks.workunit.client.0.smithi107.stdout:7/489: mknod d5/df/d11/d1c/cae 0 2024-05-17T21:52:55.802 INFO:tasks.workunit.client.0.smithi107.stdout:9/537: truncate d0/dd/d2f/d87/d89/f97 2193725 0 2024-05-17T21:52:55.803 INFO:tasks.workunit.client.1.smithi138.stdout:5/348: mkdir d1/d6/d73 0 2024-05-17T21:52:55.806 INFO:tasks.workunit.client.0.smithi107.stdout:5/515: truncate d5/db/d16/d2d/d4d/d6d/f77 265858 0 2024-05-17T21:52:55.810 INFO:tasks.workunit.client.1.smithi138.stdout:7/279: mknod d0/d18/d40/d41/c4f 0 2024-05-17T21:52:55.810 INFO:tasks.workunit.client.0.smithi107.stdout:3/551: write d6/f2c [2230912,63047] 0 2024-05-17T21:52:55.812 INFO:tasks.workunit.client.0.smithi107.stdout:9/538: dwrite d0/dd/d3d/fb4 [0,4194304] 0 2024-05-17T21:52:55.812 INFO:tasks.workunit.client.0.smithi107.stdout:7/490: dwrite d5/df/d11/d1c/d2f/d53/fa9 [0,4194304] 0 2024-05-17T21:52:55.813 INFO:tasks.workunit.client.0.smithi107.stdout:2/382: getdents d0/d26 0 2024-05-17T21:52:55.816 INFO:tasks.workunit.client.0.smithi107.stdout:4/516: unlink d3/db/c97 0 2024-05-17T21:52:55.817 INFO:tasks.workunit.client.0.smithi107.stdout:3/552: fdatasync d6/d2a/d2d/d69/f6d 0 2024-05-17T21:52:55.818 INFO:tasks.workunit.client.0.smithi107.stdout:7/491: chown d5/df/d11/d1c/d31/d39/d92/f98 57 1 2024-05-17T21:52:55.819 INFO:tasks.workunit.client.0.smithi107.stdout:3/553: fsync d6/d2a/f6a 0 2024-05-17T21:52:55.819 INFO:tasks.workunit.client.0.smithi107.stdout:4/517: chown d3/d6/d26/d36/d7e/d98 1 1 2024-05-17T21:52:55.824 INFO:tasks.workunit.client.0.smithi107.stdout:1/589: write d4/df/d1a/d26/d72/d85/fc0 [4793056,45498] 0 2024-05-17T21:52:55.825 INFO:tasks.workunit.client.0.smithi107.stdout:5/516: stat d5/db/l8c 0 2024-05-17T21:52:55.826 INFO:tasks.workunit.client.0.smithi107.stdout:1/590: stat d4/df/d1a/d26/d7a/fb9 0 2024-05-17T21:52:55.830 INFO:tasks.workunit.client.1.smithi138.stdout:8/258: creat d0/dc/d1a/f4b x:0 0 0 2024-05-17T21:52:55.832 INFO:tasks.workunit.client.1.smithi138.stdout:9/322: rename d6/d7/d16/d29/d45/d4b to d6/d7/d16/d29/d48/d67 0 2024-05-17T21:52:55.833 INFO:tasks.workunit.client.0.smithi107.stdout:1/591: dread d4/df/d1a/d26/d72/daa/f9d [0,4194304] 0 2024-05-17T21:52:55.833 INFO:tasks.workunit.client.1.smithi138.stdout:9/323: stat d6/f49 0 2024-05-17T21:52:55.833 INFO:tasks.workunit.client.1.smithi138.stdout:9/324: stat d6/d7/l32 0 2024-05-17T21:52:55.834 INFO:tasks.workunit.client.0.smithi107.stdout:8/510: dwrite dc/d21/d27/d34/d4c/d7e/f45 [0,4194304] 0 2024-05-17T21:52:55.836 INFO:tasks.workunit.client.1.smithi138.stdout:4/376: symlink d0/d3/d17/d5e/d7f/l85 0 2024-05-17T21:52:55.838 INFO:tasks.workunit.client.0.smithi107.stdout:6/435: mkdir d9/d16/d4d/d85/d88 0 2024-05-17T21:52:55.839 INFO:tasks.workunit.client.1.smithi138.stdout:7/280: mkdir d0/d18/d40/d41/d50 0 2024-05-17T21:52:55.843 INFO:tasks.workunit.client.0.smithi107.stdout:9/539: rename d0/dd/d21/d29/f90 to d0/dd/d69/fc1 0 2024-05-17T21:52:55.844 INFO:tasks.workunit.client.0.smithi107.stdout:9/540: chown d0/dd/d21/d29/d57/d5f/cbc 308321 1 2024-05-17T21:52:55.845 INFO:tasks.workunit.client.1.smithi138.stdout:1/337: getdents d4/db 0 2024-05-17T21:52:55.846 INFO:tasks.workunit.client.0.smithi107.stdout:3/554: mkdir d6/d1a/db1 0 2024-05-17T21:52:55.848 INFO:tasks.workunit.client.0.smithi107.stdout:7/492: creat d5/d16/d3b/d45/faf x:0 0 0 2024-05-17T21:52:55.850 INFO:tasks.workunit.client.1.smithi138.stdout:2/386: getdents d3/d21/d2f/d3e/d63 0 2024-05-17T21:52:55.851 INFO:tasks.workunit.client.1.smithi138.stdout:2/387: fsync d3/d21/d2f/f5b 0 2024-05-17T21:52:55.852 INFO:tasks.workunit.client.0.smithi107.stdout:9/541: dread d0/dd/d1f/f52 [0,4194304] 0 2024-05-17T21:52:55.852 INFO:tasks.workunit.client.1.smithi138.stdout:8/259: dread d0/dc/d1a/d33/f34 [0,4194304] 0 2024-05-17T21:52:55.853 INFO:tasks.workunit.client.0.smithi107.stdout:8/511: symlink dc/d21/d97/d63/lb9 0 2024-05-17T21:52:55.854 INFO:tasks.workunit.client.1.smithi138.stdout:9/325: symlink d6/d7/d16/d29/d38/d40/l68 0 2024-05-17T21:52:55.858 INFO:tasks.workunit.client.1.smithi138.stdout:0/386: creat d3/f84 x:0 0 0 2024-05-17T21:52:55.861 INFO:tasks.workunit.client.1.smithi138.stdout:3/436: creat d1/d3/d16/f95 x:0 0 0 2024-05-17T21:52:55.861 INFO:tasks.workunit.client.1.smithi138.stdout:9/326: dread d6/d7/d16/d29/d48/f22 [0,4194304] 0 2024-05-17T21:52:55.861 INFO:tasks.workunit.client.0.smithi107.stdout:6/436: dread d9/f15 [0,4194304] 0 2024-05-17T21:52:55.864 INFO:tasks.workunit.client.1.smithi138.stdout:4/377: rename d0/c13 to d0/d2f/d49/d60/d1f/c86 0 2024-05-17T21:52:55.865 INFO:tasks.workunit.client.1.smithi138.stdout:7/281: stat d0/d18/c36 0 2024-05-17T21:52:55.868 INFO:tasks.workunit.client.0.smithi107.stdout:3/555: creat d6/d1a/d57/fb2 x:0 0 0 2024-05-17T21:52:55.871 INFO:tasks.workunit.client.0.smithi107.stdout:4/518: dwrite d3/f2c [0,4194304] 0 2024-05-17T21:52:55.872 INFO:tasks.workunit.client.0.smithi107.stdout:4/519: chown d3/db/d35/c3d 10889648 1 2024-05-17T21:52:55.873 INFO:tasks.workunit.client.0.smithi107.stdout:0/594: getdents d3/dc/dae 0 2024-05-17T21:52:55.875 INFO:tasks.workunit.client.0.smithi107.stdout:1/592: symlink d4/df/d1a/d26/d72/dbc/lc2 0 2024-05-17T21:52:55.878 INFO:tasks.workunit.client.1.smithi138.stdout:7/282: mkdir d0/d18/d40/d41/d51 0 2024-05-17T21:52:55.879 INFO:tasks.workunit.client.0.smithi107.stdout:2/383: creat d0/f6f x:0 0 0 2024-05-17T21:52:55.879 INFO:tasks.workunit.client.1.smithi138.stdout:1/338: creat d4/db/d29/d5d/f6e x:0 0 0 2024-05-17T21:52:55.880 INFO:tasks.workunit.client.1.smithi138.stdout:2/388: symlink d3/d21/d2f/d3e/d55/d73/l7e 0 2024-05-17T21:52:55.881 INFO:tasks.workunit.client.0.smithi107.stdout:6/437: truncate f5 2039631 0 2024-05-17T21:52:55.881 INFO:tasks.workunit.client.1.smithi138.stdout:2/389: write d3/f61 [232948,36737] 0 2024-05-17T21:52:55.882 INFO:tasks.workunit.client.1.smithi138.stdout:8/260: mknod d0/c4c 0 2024-05-17T21:52:55.882 INFO:tasks.workunit.client.0.smithi107.stdout:9/542: dwrite d0/dd/d2f/d87/d89/d9b/fa0 [0,4194304] 0 2024-05-17T21:52:55.882 INFO:tasks.workunit.client.1.smithi138.stdout:5/349: link d1/d60/c50 d1/d2c/d63/c74 0 2024-05-17T21:52:55.884 INFO:tasks.workunit.client.1.smithi138.stdout:9/327: symlink d6/d7/d16/d29/d38/d40/d44/d5c/l69 0 2024-05-17T21:52:55.888 INFO:tasks.workunit.client.0.smithi107.stdout:3/556: dread d6/d8/d10/f38 [0,4194304] 0 2024-05-17T21:52:55.888 INFO:tasks.workunit.client.1.smithi138.stdout:8/261: mkdir d0/dc/d3b/d4d 0 2024-05-17T21:52:55.890 INFO:tasks.workunit.client.0.smithi107.stdout:6/438: chown d9/l40 41970 1 2024-05-17T21:52:55.891 INFO:tasks.workunit.client.0.smithi107.stdout:9/543: dwrite d0/dd/f35 [4194304,4194304] 0 2024-05-17T21:52:55.891 INFO:tasks.workunit.client.1.smithi138.stdout:7/283: getdents d0/d18/d40/d41/d51 0 2024-05-17T21:52:55.894 INFO:tasks.workunit.client.0.smithi107.stdout:0/595: getdents d3/d24/dcd 0 2024-05-17T21:52:55.894 INFO:tasks.workunit.client.0.smithi107.stdout:0/596: chown d3/d24/d6f 484385 1 2024-05-17T21:52:55.895 INFO:tasks.workunit.client.1.smithi138.stdout:5/350: truncate d1/d1a/f26 6875215 0 2024-05-17T21:52:55.896 INFO:tasks.workunit.client.1.smithi138.stdout:5/351: write d1/f13 [4408802,19806] 0 2024-05-17T21:52:55.900 INFO:tasks.workunit.client.1.smithi138.stdout:7/284: rename d0/l14 to d0/d18/d40/l52 0 2024-05-17T21:52:55.903 INFO:tasks.workunit.client.1.smithi138.stdout:5/352: read d1/d6/d1b/f21 [1628210,123940] 0 2024-05-17T21:52:55.903 INFO:tasks.workunit.client.1.smithi138.stdout:8/262: truncate d0/f2c 6837899 0 2024-05-17T21:52:55.904 INFO:tasks.workunit.client.1.smithi138.stdout:3/437: write d1/d3/d8/de/f3f [3988223,84902] 0 2024-05-17T21:52:55.905 INFO:tasks.workunit.client.1.smithi138.stdout:5/353: fdatasync d1/d6/d1b/d29/d36/d68/f6f 0 2024-05-17T21:52:55.906 INFO:tasks.workunit.client.1.smithi138.stdout:4/378: write d0/d2f/f82 [491134,81526] 0 2024-05-17T21:52:55.907 INFO:tasks.workunit.client.0.smithi107.stdout:9/544: rename d0/dd/d2f/l72 to d0/dd/d21/d29/d57/lc2 0 2024-05-17T21:52:55.908 INFO:tasks.workunit.client.0.smithi107.stdout:8/512: write dc/d21/d27/d34/f6e [362774,68390] 0 2024-05-17T21:52:55.910 INFO:tasks.workunit.client.0.smithi107.stdout:3/557: mknod d6/d8/cb3 0 2024-05-17T21:52:55.912 INFO:tasks.workunit.client.0.smithi107.stdout:6/439: creat d9/d16/d4d/d85/d88/f89 x:0 0 0 2024-05-17T21:52:55.913 INFO:tasks.workunit.client.0.smithi107.stdout:6/440: write d9/d16/d22/d5f/d6e/f87 [191177,49515] 0 2024-05-17T21:52:55.914 INFO:tasks.workunit.client.1.smithi138.stdout:1/339: dread d4/df/d3a/f4c [0,4194304] 0 2024-05-17T21:52:55.915 INFO:tasks.workunit.client.0.smithi107.stdout:6/441: write d9/d16/f5e [2168923,7264] 0 2024-05-17T21:52:55.915 INFO:tasks.workunit.client.0.smithi107.stdout:0/597: rename d3/dc/d22/d3d/d56/l6a to d3/d24/dcd/ld5 0 2024-05-17T21:52:55.916 INFO:tasks.workunit.client.0.smithi107.stdout:7/493: write d5/fe [2092206,78051] 0 2024-05-17T21:52:55.919 INFO:tasks.workunit.client.0.smithi107.stdout:2/384: getdents d0/d9/d1c/d3e/d4f 0 2024-05-17T21:52:55.920 INFO:tasks.workunit.client.1.smithi138.stdout:4/379: truncate d0/d3/f12 4080967 0 2024-05-17T21:52:55.921 INFO:tasks.workunit.client.0.smithi107.stdout:2/385: readlink d0/d9/d11/d18/l5e 0 2024-05-17T21:52:55.922 INFO:tasks.workunit.client.1.smithi138.stdout:1/340: dwrite d4/d2e/d36/f4f [0,4194304] 0 2024-05-17T21:52:55.923 INFO:tasks.workunit.client.0.smithi107.stdout:0/598: dwrite d3/dc/d22/fc5 [0,4194304] 0 2024-05-17T21:52:55.924 INFO:tasks.workunit.client.0.smithi107.stdout:4/520: truncate d3/db/d1e/f77 7314418 0 2024-05-17T21:52:55.924 INFO:tasks.workunit.client.0.smithi107.stdout:0/599: read - d3/dc/dae/fbb zero size 2024-05-17T21:52:55.928 INFO:tasks.workunit.client.1.smithi138.stdout:2/390: dwrite d3/f4a [0,4194304] 0 2024-05-17T21:52:55.928 INFO:tasks.workunit.client.1.smithi138.stdout:9/328: getdents d6/d7/d16 0 2024-05-17T21:52:55.929 INFO:tasks.workunit.client.0.smithi107.stdout:3/558: dread d6/d8/d10/d25/f46 [0,4194304] 0 2024-05-17T21:52:55.930 INFO:tasks.workunit.client.1.smithi138.stdout:7/285: creat d0/d18/d40/d41/d50/f53 x:0 0 0 2024-05-17T21:52:55.931 INFO:tasks.workunit.client.1.smithi138.stdout:8/263: mkdir d0/dc/d1a/d33/d4a/d4e 0 2024-05-17T21:52:55.935 INFO:tasks.workunit.client.1.smithi138.stdout:1/341: dread f0 [0,4194304] 0 2024-05-17T21:52:55.935 INFO:tasks.workunit.client.1.smithi138.stdout:7/286: chown d0/f13 114860 1 2024-05-17T21:52:55.936 INFO:tasks.workunit.client.1.smithi138.stdout:7/287: chown d0/d4c 20 1 2024-05-17T21:52:55.938 INFO:tasks.workunit.client.1.smithi138.stdout:4/380: dwrite d0/d2f/d49/d60/f3c [4194304,4194304] 0 2024-05-17T21:52:55.940 INFO:tasks.workunit.client.1.smithi138.stdout:3/438: creat d1/d3/d8d/d92/f96 x:0 0 0 2024-05-17T21:52:55.944 INFO:tasks.workunit.client.1.smithi138.stdout:5/354: symlink d1/d6/l75 0 2024-05-17T21:52:55.946 INFO:tasks.workunit.client.0.smithi107.stdout:6/442: rmdir d9/d81 39 2024-05-17T21:52:55.950 INFO:tasks.workunit.client.0.smithi107.stdout:7/494: unlink d5/df/d11/f21 0 2024-05-17T21:52:55.954 INFO:tasks.workunit.client.0.smithi107.stdout:9/545: dwrite d0/dd/d2f/f31 [4194304,4194304] 0 2024-05-17T21:52:55.955 INFO:tasks.workunit.client.0.smithi107.stdout:9/546: readlink d0/dd/d2f/l76 0 2024-05-17T21:52:55.956 INFO:tasks.workunit.client.0.smithi107.stdout:8/513: mkdir dc/d21/d27/d2f/dba 0 2024-05-17T21:52:55.961 INFO:tasks.workunit.client.0.smithi107.stdout:9/547: dwrite d0/dd/d1b/f53 [4194304,4194304] 0 2024-05-17T21:52:55.961 INFO:tasks.workunit.client.1.smithi138.stdout:9/329: symlink d6/d7/d16/d29/d38/d3c/l6a 0 2024-05-17T21:52:55.964 INFO:tasks.workunit.client.1.smithi138.stdout:1/342: creat d4/db/d29/d5d/f6f x:0 0 0 2024-05-17T21:52:55.965 INFO:tasks.workunit.client.0.smithi107.stdout:0/600: creat d3/d39/fd6 x:0 0 0 2024-05-17T21:52:55.966 INFO:tasks.workunit.client.1.smithi138.stdout:7/288: write d0/f11 [3632589,7709] 0 2024-05-17T21:52:55.968 INFO:tasks.workunit.client.0.smithi107.stdout:3/559: rename d6/d8/d10/c74 to d6/d8/d3e/cb4 0 2024-05-17T21:52:55.969 INFO:tasks.workunit.client.0.smithi107.stdout:4/521: dwrite d3/db/f7b [0,4194304] 0 2024-05-17T21:52:55.969 INFO:tasks.workunit.client.1.smithi138.stdout:4/381: unlink d0/d3/d17/f1a 0 2024-05-17T21:52:55.970 INFO:tasks.workunit.client.0.smithi107.stdout:4/522: write d3/d4a/d47/d60/d3a/d41/f51 [2524539,122988] 0 2024-05-17T21:52:55.971 INFO:tasks.workunit.client.1.smithi138.stdout:4/382: write d0/d2f/f6f [605494,58708] 0 2024-05-17T21:52:55.972 INFO:tasks.workunit.client.0.smithi107.stdout:7/495: symlink d5/df/d11/d1d/d33/d82/lb0 0 2024-05-17T21:52:55.973 INFO:tasks.workunit.client.0.smithi107.stdout:7/496: chown d5/df/d11/d1c 37 1 2024-05-17T21:52:55.978 INFO:tasks.workunit.client.0.smithi107.stdout:2/386: mknod d0/d26/d64/c70 0 2024-05-17T21:52:55.981 INFO:tasks.workunit.client.0.smithi107.stdout:2/387: fdatasync d0/d9/d1c/d37/f50 0 2024-05-17T21:52:55.981 INFO:tasks.workunit.client.0.smithi107.stdout:2/388: write d0/d9/d1c/d3e/f62 [6062502,30215] 0 2024-05-17T21:52:55.982 INFO:tasks.workunit.client.1.smithi138.stdout:8/264: rename d0/d31/l35 to d0/d11/d3e/l4f 0 2024-05-17T21:52:55.982 INFO:tasks.workunit.client.1.smithi138.stdout:8/265: stat d0/dc/f10 0 2024-05-17T21:52:55.984 INFO:tasks.workunit.client.1.smithi138.stdout:9/330: creat d6/d7/d16/d29/d38/d40/f6b x:0 0 0 2024-05-17T21:52:55.986 INFO:tasks.workunit.client.0.smithi107.stdout:3/560: creat d6/d8/d3e/d3f/d90/fb5 x:0 0 0 2024-05-17T21:52:55.987 INFO:tasks.workunit.client.0.smithi107.stdout:3/561: write d6/d2a/d2d/f6c [2288081,100511] 0 2024-05-17T21:52:55.988 INFO:tasks.workunit.client.1.smithi138.stdout:8/266: dwrite d0/f9 [0,4194304] 0 2024-05-17T21:52:55.989 INFO:tasks.workunit.client.1.smithi138.stdout:8/267: chown d0/dc/d3b 1911304 1 2024-05-17T21:52:55.991 INFO:tasks.workunit.client.0.smithi107.stdout:4/523: creat d3/d6/d26/d36/d7e/faf x:0 0 0 2024-05-17T21:52:55.992 INFO:tasks.workunit.client.1.smithi138.stdout:2/391: dread d3/f6 [4194304,4194304] 0 2024-05-17T21:52:55.993 INFO:tasks.workunit.client.1.smithi138.stdout:5/355: symlink d1/d6/d1b/l76 0 2024-05-17T21:52:55.995 INFO:tasks.workunit.client.1.smithi138.stdout:3/439: dread d1/d3/d16/d3e/f4e [0,4194304] 0 2024-05-17T21:52:55.997 INFO:tasks.workunit.client.1.smithi138.stdout:1/343: rename d4/db/l33 to d4/dd/d3b/l70 0 2024-05-17T21:52:56.000 INFO:tasks.workunit.client.1.smithi138.stdout:1/344: chown d4/df/d3a/c41 0 1 2024-05-17T21:52:56.001 INFO:tasks.workunit.client.1.smithi138.stdout:2/392: dwrite d3/d21/f59 [0,4194304] 0 2024-05-17T21:52:56.003 INFO:tasks.workunit.client.1.smithi138.stdout:7/289: symlink d0/l54 0 2024-05-17T21:52:56.004 INFO:tasks.workunit.client.1.smithi138.stdout:5/356: dwrite d1/d6/d1b/f21 [0,4194304] 0 2024-05-17T21:52:56.005 INFO:tasks.workunit.client.1.smithi138.stdout:7/290: chown d0/d18/d40/d41/f43 1217455 1 2024-05-17T21:52:56.005 INFO:tasks.workunit.client.1.smithi138.stdout:5/357: dread - d1/f4f zero size 2024-05-17T21:52:56.006 INFO:tasks.workunit.client.1.smithi138.stdout:9/331: rmdir d6/d7/d16/d29/d38/d40/d44 39 2024-05-17T21:52:56.008 INFO:tasks.workunit.client.1.smithi138.stdout:8/268: symlink d0/dc/d1a/d33/d4a/l50 0 2024-05-17T21:52:56.015 INFO:tasks.workunit.client.1.smithi138.stdout:7/291: dwrite d0/f10 [4194304,4194304] 0 2024-05-17T21:52:56.017 INFO:tasks.workunit.client.1.smithi138.stdout:4/383: rename d0/d3/d17/d21/f6c to d0/d2f/d49/d60/d1f/d32/f87 0 2024-05-17T21:52:56.019 INFO:tasks.workunit.client.1.smithi138.stdout:1/345: symlink d4/d2e/d36/l71 0 2024-05-17T21:52:56.021 INFO:tasks.workunit.client.1.smithi138.stdout:2/393: mkdir d3/d21/d2f/d3e/d63/d7f 0 2024-05-17T21:52:56.024 INFO:tasks.workunit.client.0.smithi107.stdout:4/524: creat d3/d4a/d47/d60/d3c/fb0 x:0 0 0 2024-05-17T21:52:56.024 INFO:tasks.workunit.client.0.smithi107.stdout:8/514: truncate dc/d21/d27/d2f/f4a 813066 0 2024-05-17T21:52:56.025 INFO:tasks.workunit.client.1.smithi138.stdout:2/394: read d3/fe [3668715,23531] 0 2024-05-17T21:52:56.027 INFO:tasks.workunit.client.0.smithi107.stdout:7/497: link d5/df/d11/d1c/d2f/f7c d5/df/d11/d1d/d33/d82/fb1 0 2024-05-17T21:52:56.029 INFO:tasks.workunit.client.0.smithi107.stdout:7/498: write d5/df/d78/f95 [1241884,21420] 0 2024-05-17T21:52:56.030 INFO:tasks.workunit.client.0.smithi107.stdout:9/548: rmdir d0/dd/d1f/dab 0 2024-05-17T21:52:56.030 INFO:tasks.workunit.client.0.smithi107.stdout:4/525: dwrite d3/d4a/f78 [0,4194304] 0 2024-05-17T21:52:56.032 INFO:tasks.workunit.client.1.smithi138.stdout:8/269: mkdir d0/dc/d1a/d51 0 2024-05-17T21:52:56.033 INFO:tasks.workunit.client.1.smithi138.stdout:5/358: dwrite d1/d6/f31 [0,4194304] 0 2024-05-17T21:52:56.035 INFO:tasks.workunit.client.1.smithi138.stdout:8/270: dread - d0/d2f/f41 zero size 2024-05-17T21:52:56.035 INFO:tasks.workunit.client.1.smithi138.stdout:5/359: chown d1/d1a/d39/f4c 0 1 2024-05-17T21:52:56.038 INFO:tasks.workunit.client.0.smithi107.stdout:3/562: mkdir d6/d8/db6 0 2024-05-17T21:52:56.038 INFO:tasks.workunit.client.1.smithi138.stdout:8/271: write d0/f17 [1855283,89476] 0 2024-05-17T21:52:56.038 INFO:tasks.workunit.client.1.smithi138.stdout:3/440: dwrite d1/d3/f25 [0,4194304] 0 2024-05-17T21:52:56.039 INFO:tasks.workunit.client.1.smithi138.stdout:8/272: fdatasync d0/f40 0 2024-05-17T21:52:56.042 INFO:tasks.workunit.client.1.smithi138.stdout:1/346: write d4/dd/f15 [1112904,87553] 0 2024-05-17T21:52:56.044 INFO:tasks.workunit.client.1.smithi138.stdout:5/360: truncate d1/d6/d1b/d29/f3b 4331699 0 2024-05-17T21:52:56.045 INFO:tasks.workunit.client.0.smithi107.stdout:7/499: creat d5/df/d11/d1c/d2f/d53/fb2 x:0 0 0 2024-05-17T21:52:56.045 INFO:tasks.workunit.client.1.smithi138.stdout:5/361: dread - d1/d6/d1b/d29/d36/f4b zero size 2024-05-17T21:52:56.046 INFO:tasks.workunit.client.1.smithi138.stdout:2/395: rename d3/f5 to d3/d21/d2f/d3e/d63/f80 0 2024-05-17T21:52:56.048 INFO:tasks.workunit.client.1.smithi138.stdout:9/332: rmdir d6/d7/d16/d29/d38/d40/d44 39 2024-05-17T21:52:56.050 INFO:tasks.workunit.client.1.smithi138.stdout:5/362: dread d1/f45 [0,4194304] 0 2024-05-17T21:52:56.051 INFO:tasks.workunit.client.1.smithi138.stdout:5/363: write d1/d60/f5c [17813,1472] 0 2024-05-17T21:52:56.051 INFO:tasks.workunit.client.0.smithi107.stdout:4/526: unlink d3/d4a/d47/d60/d3a/d41/d80/f19 0 2024-05-17T21:52:56.052 INFO:tasks.workunit.client.0.smithi107.stdout:0/601: getdents d3/dc/d22/d23/d97/d85 0 2024-05-17T21:52:56.054 INFO:tasks.workunit.client.0.smithi107.stdout:3/563: write d6/d8/f17 [3585386,891] 0 2024-05-17T21:52:56.054 INFO:tasks.workunit.client.1.smithi138.stdout:3/441: creat d1/d4f/d50/f97 x:0 0 0 2024-05-17T21:52:56.055 INFO:tasks.workunit.client.0.smithi107.stdout:3/564: stat d6/d2a/d2d/d37/l45 0 2024-05-17T21:52:56.056 INFO:tasks.workunit.client.0.smithi107.stdout:7/500: mkdir d5/d16/d3b/d45/d91/db3 0 2024-05-17T21:52:56.058 INFO:tasks.workunit.client.0.smithi107.stdout:2/389: getdents d0/d9/d11/df 0 2024-05-17T21:52:56.061 INFO:tasks.workunit.client.0.smithi107.stdout:3/565: dwrite d6/d2a/f78 [0,4194304] 0 2024-05-17T21:52:56.062 INFO:tasks.workunit.client.1.smithi138.stdout:2/396: fsync d3/d21/d2f/f33 0 2024-05-17T21:52:56.063 INFO:tasks.workunit.client.1.smithi138.stdout:2/397: write d3/f4 [5027142,17004] 0 2024-05-17T21:52:56.064 INFO:tasks.workunit.client.0.smithi107.stdout:8/515: creat dc/d21/d97/d63/d71/fbb x:0 0 0 2024-05-17T21:52:56.065 INFO:tasks.workunit.client.1.smithi138.stdout:9/333: unlink d6/d7/d16/d29/d48/f61 0 2024-05-17T21:52:56.065 INFO:tasks.workunit.client.1.smithi138.stdout:2/398: write d3/d21/d2f/f5b [335719,26206] 0 2024-05-17T21:52:56.066 INFO:tasks.workunit.client.1.smithi138.stdout:2/399: truncate d3/f6f 1035570 0 2024-05-17T21:52:56.070 INFO:tasks.workunit.client.0.smithi107.stdout:9/549: rmdir d0/dd/d2f/d87/dba 0 2024-05-17T21:52:56.072 INFO:tasks.workunit.client.1.smithi138.stdout:3/442: creat d1/d2e/f98 x:0 0 0 2024-05-17T21:52:56.072 INFO:tasks.workunit.client.0.smithi107.stdout:0/602: dread d3/d39/f7f [0,4194304] 0 2024-05-17T21:52:56.074 INFO:tasks.workunit.client.0.smithi107.stdout:0/603: write d3/dc/d22/fc5 [988845,108924] 0 2024-05-17T21:52:56.074 INFO:tasks.workunit.client.0.smithi107.stdout:7/501: creat d5/df/d11/d1c/d2f/d65/fb4 x:0 0 0 2024-05-17T21:52:56.074 INFO:tasks.workunit.client.0.smithi107.stdout:0/604: chown d3/f33 1 1 2024-05-17T21:52:56.075 INFO:tasks.workunit.client.0.smithi107.stdout:7/502: chown d5/d16/f28 1307 1 2024-05-17T21:52:56.076 INFO:tasks.workunit.client.0.smithi107.stdout:2/390: fdatasync d0/d9/d11/f5c 0 2024-05-17T21:52:56.077 INFO:tasks.workunit.client.1.smithi138.stdout:8/273: mknod d0/dc/d3b/d4d/c52 0 2024-05-17T21:52:56.079 INFO:tasks.workunit.client.1.smithi138.stdout:1/347: chown d4/db/f4d 48001 1 2024-05-17T21:52:56.080 INFO:tasks.workunit.client.0.smithi107.stdout:7/503: dread d5/df/d11/d1d/d33/f6a [0,4194304] 0 2024-05-17T21:52:56.084 INFO:tasks.workunit.client.0.smithi107.stdout:0/605: mkdir d3/d24/d6f/db5/dd7 0 2024-05-17T21:52:56.088 INFO:tasks.workunit.client.1.smithi138.stdout:4/384: truncate d0/d2f/d49/d60/d1f/f73 488732 0 2024-05-17T21:52:56.090 INFO:tasks.workunit.client.0.smithi107.stdout:2/391: creat d0/d9/d11/df/f71 x:0 0 0 2024-05-17T21:52:56.091 INFO:tasks.workunit.client.1.smithi138.stdout:3/443: mknod d1/d4f/d50/c99 0 2024-05-17T21:52:56.091 INFO:tasks.workunit.client.0.smithi107.stdout:2/392: chown d0/d9/l59 2417790 1 2024-05-17T21:52:56.092 INFO:tasks.workunit.client.1.smithi138.stdout:7/292: link d0/l28 d0/d4c/l55 0 2024-05-17T21:52:56.094 INFO:tasks.workunit.client.0.smithi107.stdout:8/516: symlink dc/d21/d27/d34/d4c/db1/lbc 0 2024-05-17T21:52:56.095 INFO:tasks.workunit.client.1.smithi138.stdout:8/274: mknod d0/dc/d1a/c53 0 2024-05-17T21:52:56.097 INFO:tasks.workunit.client.1.smithi138.stdout:1/348: readlink d4/dd/le 0 2024-05-17T21:52:56.098 INFO:tasks.workunit.client.0.smithi107.stdout:7/504: symlink d5/df/d11/d1c/d31/d39/d92/lb5 0 2024-05-17T21:52:56.099 INFO:tasks.workunit.client.1.smithi138.stdout:9/334: mkdir d6/d7/d16/d29/d6c 0 2024-05-17T21:52:56.100 INFO:tasks.workunit.client.0.smithi107.stdout:0/606: stat d3/dc/d22/fb9 0 2024-05-17T21:52:56.101 INFO:tasks.workunit.client.1.smithi138.stdout:2/400: mkdir d3/d81 0 2024-05-17T21:52:56.103 INFO:tasks.workunit.client.1.smithi138.stdout:4/385: mkdir d0/d1b/d88 0 2024-05-17T21:52:56.106 INFO:tasks.workunit.client.1.smithi138.stdout:5/364: creat d1/d6/d1b/f77 x:0 0 0 2024-05-17T21:52:56.107 INFO:tasks.workunit.client.1.smithi138.stdout:2/401: dwrite d3/d21/f31 [0,4194304] 0 2024-05-17T21:52:56.108 INFO:tasks.workunit.client.0.smithi107.stdout:2/393: symlink d0/d9/d11/d18/l72 0 2024-05-17T21:52:56.108 INFO:tasks.workunit.client.1.smithi138.stdout:3/444: creat d1/d3/d16/d3e/d71/f9a x:0 0 0 2024-05-17T21:52:56.109 INFO:tasks.workunit.client.0.smithi107.stdout:4/527: link d3/d4a/d47/d60/l2e d3/db/d1e/lb1 0 2024-05-17T21:52:56.111 INFO:tasks.workunit.client.1.smithi138.stdout:7/293: mknod d0/d18/d40/d41/c56 0 2024-05-17T21:52:56.115 INFO:tasks.workunit.client.1.smithi138.stdout:8/275: symlink d0/d11/l54 0 2024-05-17T21:52:56.116 INFO:tasks.workunit.client.1.smithi138.stdout:8/276: write d0/d2f/f3c [882709,60997] 0 2024-05-17T21:52:56.119 INFO:tasks.workunit.client.1.smithi138.stdout:9/335: creat d6/d7/d16/f6d x:0 0 0 2024-05-17T21:52:56.122 INFO:tasks.workunit.client.0.smithi107.stdout:0/607: stat d3/dc/d22/c46 0 2024-05-17T21:52:56.123 INFO:tasks.workunit.client.0.smithi107.stdout:0/608: write d3/dc/d22/d3d/d56/f5b [653768,6975] 0 2024-05-17T21:52:56.124 INFO:tasks.workunit.client.1.smithi138.stdout:4/386: creat d0/d2f/d49/d60/f89 x:0 0 0 2024-05-17T21:52:56.126 INFO:tasks.workunit.client.0.smithi107.stdout:4/528: rmdir d3/d4a/d47/d60/d3c 39 2024-05-17T21:52:56.128 INFO:tasks.workunit.client.1.smithi138.stdout:2/402: unlink d3/c48 0 2024-05-17T21:52:56.129 INFO:tasks.workunit.client.0.smithi107.stdout:8/517: fsync dc/d21/d97/d63/f79 0 2024-05-17T21:52:56.131 INFO:tasks.workunit.client.1.smithi138.stdout:3/445: mkdir d1/d62/d9b 0 2024-05-17T21:52:56.131 INFO:tasks.workunit.client.1.smithi138.stdout:3/446: fdatasync d1/d4f/d50/f6b 0 2024-05-17T21:52:56.132 INFO:tasks.workunit.client.1.smithi138.stdout:3/447: fdatasync d1/f4 0 2024-05-17T21:52:56.133 INFO:tasks.workunit.client.0.smithi107.stdout:9/550: getdents d0 0 2024-05-17T21:52:56.135 INFO:tasks.workunit.client.1.smithi138.stdout:5/365: dread d1/d1a/f26 [4194304,4194304] 0 2024-05-17T21:52:56.135 INFO:tasks.workunit.client.0.smithi107.stdout:7/505: mknod d5/cb6 0 2024-05-17T21:52:56.136 INFO:tasks.workunit.client.1.smithi138.stdout:1/349: write d4/f3f [3654434,71451] 0 2024-05-17T21:52:56.136 INFO:tasks.workunit.client.0.smithi107.stdout:7/506: readlink d5/d16/d3b/l97 0 2024-05-17T21:52:56.139 INFO:tasks.workunit.client.1.smithi138.stdout:3/448: dwrite d1/d3/d16/f95 [0,4194304] 0 2024-05-17T21:52:56.140 INFO:tasks.workunit.client.1.smithi138.stdout:8/277: mknod d0/d2f/c55 0 2024-05-17T21:52:56.144 INFO:tasks.workunit.client.1.smithi138.stdout:4/387: mkdir d0/d3/d1d/d38/d8a 0 2024-05-17T21:52:56.146 INFO:tasks.workunit.client.0.smithi107.stdout:9/551: truncate d0/dd/d1f/f52 645117 0 2024-05-17T21:52:56.148 INFO:tasks.workunit.client.0.smithi107.stdout:7/507: symlink d5/df/d11/d1c/d31/d56/lb7 0 2024-05-17T21:52:56.149 INFO:tasks.workunit.client.0.smithi107.stdout:0/609: fdatasync d3/dc/f14 0 2024-05-17T21:52:56.150 INFO:tasks.workunit.client.0.smithi107.stdout:0/610: chown d3/d39/fb0 0 1 2024-05-17T21:52:56.152 INFO:tasks.workunit.client.0.smithi107.stdout:2/394: link d0/d26/f6c d0/d9/d1c/f73 0 2024-05-17T21:52:56.152 INFO:tasks.workunit.client.0.smithi107.stdout:2/395: readlink d0/d9/d1c/d3e/d4f/l63 0 2024-05-17T21:52:56.153 INFO:tasks.workunit.client.1.smithi138.stdout:5/366: creat d1/d46/f78 x:0 0 0 2024-05-17T21:52:56.154 INFO:tasks.workunit.client.0.smithi107.stdout:4/529: getdents d3/d6/d26/d36/d7e/d98 0 2024-05-17T21:52:56.154 INFO:tasks.workunit.client.1.smithi138.stdout:5/367: readlink d1/d60/l44 0 2024-05-17T21:52:56.155 INFO:tasks.workunit.client.0.smithi107.stdout:2/396: read d0/d9/d1c/f73 [157243,106823] 0 2024-05-17T21:52:56.155 INFO:tasks.workunit.client.0.smithi107.stdout:8/518: creat dc/d21/d27/d2f/dba/fbd x:0 0 0 2024-05-17T21:52:56.156 INFO:tasks.workunit.client.0.smithi107.stdout:9/552: mkdir d0/dd/d2f/d81/dc3 0 2024-05-17T21:52:56.157 INFO:tasks.workunit.client.0.smithi107.stdout:9/553: write d0/dd/d2f/d87/fb6 [644066,93609] 0 2024-05-17T21:52:56.158 INFO:tasks.workunit.client.1.smithi138.stdout:3/449: mkdir d1/d3/d8/d5c/d9c 0 2024-05-17T21:52:56.159 INFO:tasks.workunit.client.0.smithi107.stdout:9/554: chown d0/dd/d21/d29/d57/c93 0 1 2024-05-17T21:52:56.160 INFO:tasks.workunit.client.0.smithi107.stdout:9/555: truncate d0/dd/d1f/f5c 1714172 0 2024-05-17T21:52:56.164 INFO:tasks.workunit.client.1.smithi138.stdout:2/403: write d3/d21/d2f/d3e/d58/f71 [1181710,6832] 0 2024-05-17T21:52:56.164 INFO:tasks.workunit.client.1.smithi138.stdout:3/450: dwrite d1/d3/d8d/d92/f96 [0,4194304] 0 2024-05-17T21:52:56.165 INFO:tasks.workunit.client.1.smithi138.stdout:1/350: symlink d4/dd/d64/l72 0 2024-05-17T21:52:56.166 INFO:tasks.workunit.client.0.smithi107.stdout:7/508: dread d5/df/d78/f95 [0,4194304] 0 2024-05-17T21:52:56.166 INFO:tasks.workunit.client.0.smithi107.stdout:2/397: chown d0/d9/d1c/l1f 480 1 2024-05-17T21:52:56.166 INFO:tasks.workunit.client.1.smithi138.stdout:1/351: write f0 [3773523,68198] 0 2024-05-17T21:52:56.166 INFO:tasks.workunit.client.1.smithi138.stdout:2/404: stat d3/d21/d2f/l3b 0 2024-05-17T21:52:56.167 INFO:tasks.workunit.client.0.smithi107.stdout:4/530: dread d3/fa [0,4194304] 0 2024-05-17T21:52:56.167 INFO:tasks.workunit.client.0.smithi107.stdout:5/517: sync 2024-05-17T21:52:56.171 INFO:tasks.workunit.client.1.smithi138.stdout:4/388: creat d0/d1b/d88/f8b x:0 0 0 2024-05-17T21:52:56.173 INFO:tasks.workunit.client.0.smithi107.stdout:0/611: creat d3/d24/d6f/db5/da5/fd8 x:0 0 0 2024-05-17T21:52:56.175 INFO:tasks.workunit.client.1.smithi138.stdout:7/294: rename d0/f2b to d0/d18/f57 0 2024-05-17T21:52:56.175 INFO:tasks.workunit.client.0.smithi107.stdout:2/398: dwrite d0/d9/d1c/d3e/f6b [0,4194304] 0 2024-05-17T21:52:56.176 INFO:tasks.workunit.client.1.smithi138.stdout:3/451: dread d1/d3/f1b [0,4194304] 0 2024-05-17T21:52:56.176 INFO:tasks.workunit.client.1.smithi138.stdout:7/295: chown d0/d39/f3d 1118 1 2024-05-17T21:52:56.176 INFO:tasks.workunit.client.0.smithi107.stdout:9/556: creat d0/dd/d21/d29/d3e/fc4 x:0 0 0 2024-05-17T21:52:56.178 INFO:tasks.workunit.client.0.smithi107.stdout:9/557: fdatasync d0/dd/d2f/d87/d89/d9b/fa0 0 2024-05-17T21:52:56.180 INFO:tasks.workunit.client.1.smithi138.stdout:5/368: creat d1/d60/d2f/d5e/f79 x:0 0 0 2024-05-17T21:52:56.185 INFO:tasks.workunit.client.1.smithi138.stdout:1/352: stat d4/fa 0 2024-05-17T21:52:56.186 INFO:tasks.workunit.client.0.smithi107.stdout:4/531: read d3/db/f33 [4597560,53696] 0 2024-05-17T21:52:56.186 INFO:tasks.workunit.client.0.smithi107.stdout:5/518: mkdir d5/db/d16/d54/da0 0 2024-05-17T21:52:56.187 INFO:tasks.workunit.client.0.smithi107.stdout:5/519: chown d5/db/d16/f81 402 1 2024-05-17T21:52:56.187 INFO:tasks.workunit.client.1.smithi138.stdout:2/405: chown d3/l37 390903 1 2024-05-17T21:52:56.188 INFO:tasks.workunit.client.0.smithi107.stdout:0/612: dread - d3/d39/f41 zero size 2024-05-17T21:52:56.191 INFO:tasks.workunit.client.0.smithi107.stdout:8/519: dread dc/f18 [0,4194304] 0 2024-05-17T21:52:56.192 INFO:tasks.workunit.client.1.smithi138.stdout:4/389: dread d0/d1b/f3e [0,4194304] 0 2024-05-17T21:52:56.192 INFO:tasks.workunit.client.0.smithi107.stdout:2/399: rename d0/f25 to d0/d9/d11/df/d5a/d6d/f74 0 2024-05-17T21:52:56.192 INFO:tasks.workunit.client.0.smithi107.stdout:8/520: chown dc/d44/f84 40625 1 2024-05-17T21:52:56.192 INFO:tasks.workunit.client.1.smithi138.stdout:8/278: write d0/f2c [6293515,61196] 0 2024-05-17T21:52:56.194 INFO:tasks.workunit.client.1.smithi138.stdout:9/336: rename d6/d7/d16/d29/d38/d3c/d3e to d6/d7/d16/d29/d48/d6e 0 2024-05-17T21:52:56.194 INFO:tasks.workunit.client.0.smithi107.stdout:8/521: truncate dc/d21/d97/d63/d71/f94 558168 0 2024-05-17T21:52:56.196 INFO:tasks.workunit.client.1.smithi138.stdout:3/452: symlink d1/d3/d8/d72/l9d 0 2024-05-17T21:52:56.196 INFO:tasks.workunit.client.1.smithi138.stdout:6/390: sync 2024-05-17T21:52:56.197 INFO:tasks.workunit.client.1.smithi138.stdout:0/387: sync 2024-05-17T21:52:56.198 INFO:tasks.workunit.client.1.smithi138.stdout:7/296: creat d0/d18/d40/f58 x:0 0 0 2024-05-17T21:52:56.198 INFO:tasks.workunit.client.0.smithi107.stdout:8/522: dwrite dc/d21/d97/d63/f79 [4194304,4194304] 0 2024-05-17T21:52:56.200 INFO:tasks.workunit.client.0.smithi107.stdout:9/558: creat d0/dd/d2f/d87/d89/d9b/fc5 x:0 0 0 2024-05-17T21:52:56.203 INFO:tasks.workunit.client.1.smithi138.stdout:5/369: creat d1/d6/d1b/d29/d36/f7a x:0 0 0 2024-05-17T21:52:56.203 INFO:tasks.workunit.client.1.smithi138.stdout:0/388: dwrite d3/f83 [0,4194304] 0 2024-05-17T21:52:56.205 INFO:tasks.workunit.client.1.smithi138.stdout:5/370: chown d1/d1a/d39/d4e/d58/d5b/d65 33854 1 2024-05-17T21:52:56.206 INFO:tasks.workunit.client.1.smithi138.stdout:1/353: creat d4/dd/d3b/f73 x:0 0 0 2024-05-17T21:52:56.206 INFO:tasks.workunit.client.1.smithi138.stdout:5/371: write d1/d53/f6e [625172,56845] 0 2024-05-17T21:52:56.208 INFO:tasks.workunit.client.1.smithi138.stdout:5/372: read d1/d60/f5c [6507,7458] 0 2024-05-17T21:52:56.209 INFO:tasks.workunit.client.1.smithi138.stdout:5/373: dread - d1/d60/f64 zero size 2024-05-17T21:52:56.210 INFO:tasks.workunit.client.0.smithi107.stdout:5/520: creat d5/db/d16/d2d/fa1 x:0 0 0 2024-05-17T21:52:56.211 INFO:tasks.workunit.client.1.smithi138.stdout:2/406: creat d3/d21/d2f/d3e/d63/d65/f82 x:0 0 0 2024-05-17T21:52:56.216 INFO:tasks.workunit.client.1.smithi138.stdout:4/390: rmdir d0/d1b/d2b/d55/d79 39 2024-05-17T21:52:56.221 INFO:tasks.workunit.client.1.smithi138.stdout:6/391: symlink d7/db/d43/d5f/l88 0 2024-05-17T21:52:56.223 INFO:tasks.workunit.client.1.smithi138.stdout:7/297: creat d0/d18/d40/d41/d50/f59 x:0 0 0 2024-05-17T21:52:56.227 INFO:tasks.workunit.client.1.smithi138.stdout:8/279: mknod d0/c56 0 2024-05-17T21:52:56.228 INFO:tasks.workunit.client.1.smithi138.stdout:8/280: chown d0/c56 6 1 2024-05-17T21:52:56.229 INFO:tasks.workunit.client.0.smithi107.stdout:0/613: creat d3/dc/d22/d23/d97/d85/fd9 x:0 0 0 2024-05-17T21:52:56.229 INFO:tasks.workunit.client.0.smithi107.stdout:7/509: creat d5/df/d11/d1d/fb8 x:0 0 0 2024-05-17T21:52:56.229 INFO:tasks.workunit.client.0.smithi107.stdout:8/523: creat dc/d44/fbe x:0 0 0 2024-05-17T21:52:56.230 INFO:tasks.workunit.client.0.smithi107.stdout:4/532: symlink d3/d4a/d47/da3/lb2 0 2024-05-17T21:52:56.230 INFO:tasks.workunit.client.1.smithi138.stdout:4/391: unlink d0/d3/f57 0 2024-05-17T21:52:56.230 INFO:tasks.workunit.client.1.smithi138.stdout:1/354: dwrite d4/db/f22 [0,4194304] 0 2024-05-17T21:52:56.230 INFO:tasks.workunit.client.1.smithi138.stdout:8/281: write d0/f9 [3280710,45308] 0 2024-05-17T21:52:56.233 INFO:tasks.workunit.client.0.smithi107.stdout:5/521: write d5/db/d16/d5b/f3d [5689964,26225] 0 2024-05-17T21:52:56.234 INFO:tasks.workunit.client.1.smithi138.stdout:5/374: dread d1/f13 [0,4194304] 0 2024-05-17T21:52:56.234 INFO:tasks.workunit.client.0.smithi107.stdout:5/522: chown d5/db/d16/d5b/c2e 818939 1 2024-05-17T21:52:56.235 INFO:tasks.workunit.client.1.smithi138.stdout:0/389: dread d3/f7 [0,4194304] 0 2024-05-17T21:52:56.236 INFO:tasks.workunit.client.1.smithi138.stdout:5/375: chown d1/d6/d1b/f77 120996 1 2024-05-17T21:52:56.237 INFO:tasks.workunit.client.0.smithi107.stdout:8/524: dwrite dc/d21/d27/d34/d4c/f9a [0,4194304] 0 2024-05-17T21:52:56.237 INFO:tasks.workunit.client.1.smithi138.stdout:5/376: write d1/d60/f64 [321476,100064] 0 2024-05-17T21:52:56.239 INFO:tasks.workunit.client.1.smithi138.stdout:6/392: chown d7/db/l6b 27 1 2024-05-17T21:52:56.244 INFO:tasks.workunit.client.0.smithi107.stdout:5/523: dwrite d5/db/d16/d2d/d40/d5a/d7c/d8e/f90 [0,4194304] 0 2024-05-17T21:52:56.247 INFO:tasks.workunit.client.0.smithi107.stdout:5/524: dread d5/db/d16/d5b/f53 [0,4194304] 0 2024-05-17T21:52:56.248 INFO:tasks.workunit.client.1.smithi138.stdout:5/377: dwrite d1/d6/fa [0,4194304] 0 2024-05-17T21:52:56.249 INFO:tasks.workunit.client.0.smithi107.stdout:7/510: mknod d5/df/d11/d1c/d2f/cb9 0 2024-05-17T21:52:56.250 INFO:tasks.workunit.client.1.smithi138.stdout:5/378: dread - d1/d6/d1b/f77 zero size 2024-05-17T21:52:56.256 INFO:tasks.workunit.client.0.smithi107.stdout:0/614: rename d3/dc/l1c to d3/dc/d31/lda 0 2024-05-17T21:52:56.256 INFO:tasks.workunit.client.0.smithi107.stdout:0/615: readlink d3/d75/dc1/ld2 0 2024-05-17T21:52:56.262 INFO:tasks.workunit.client.0.smithi107.stdout:9/559: dread d0/dd/d21/f70 [0,4194304] 0 2024-05-17T21:52:56.263 INFO:tasks.workunit.client.1.smithi138.stdout:3/453: truncate d1/d3/d8/f28 2661092 0 2024-05-17T21:52:56.264 INFO:tasks.workunit.client.1.smithi138.stdout:1/355: dread d4/fa [4194304,4194304] 0 2024-05-17T21:52:56.264 INFO:tasks.workunit.client.0.smithi107.stdout:4/533: write d3/d4a/d47/d60/d3a/d41/d80/d43/f62 [380518,1078] 0 2024-05-17T21:52:56.265 INFO:tasks.workunit.client.0.smithi107.stdout:8/525: mknod dc/cbf 0 2024-05-17T21:52:56.267 INFO:tasks.workunit.client.1.smithi138.stdout:8/282: dread d0/fa [0,4194304] 0 2024-05-17T21:52:56.268 INFO:tasks.workunit.client.1.smithi138.stdout:8/283: write d0/d11/f42 [198954,76697] 0 2024-05-17T21:52:56.269 INFO:tasks.workunit.client.0.smithi107.stdout:0/616: creat d3/d24/d6f/d9f/dab/fdb x:0 0 0 2024-05-17T21:52:56.272 INFO:tasks.workunit.client.0.smithi107.stdout:5/525: mknod d5/ca2 0 2024-05-17T21:52:56.272 INFO:tasks.workunit.client.0.smithi107.stdout:7/511: dwrite d5/df/d11/f4c [0,4194304] 0 2024-05-17T21:52:56.274 INFO:tasks.workunit.client.0.smithi107.stdout:4/534: creat d3/d6/d26/d36/d8b/d95/fb3 x:0 0 0 2024-05-17T21:52:56.276 INFO:tasks.workunit.client.0.smithi107.stdout:4/535: chown d3/d6/d26/d36/d37/c39 780494 1 2024-05-17T21:52:56.278 INFO:tasks.workunit.client.1.smithi138.stdout:2/407: dwrite d3/d21/d2f/f45 [0,4194304] 0 2024-05-17T21:52:56.281 INFO:tasks.workunit.client.1.smithi138.stdout:4/392: mknod d0/d1b/d2b/d55/c8c 0 2024-05-17T21:52:56.283 INFO:tasks.workunit.client.0.smithi107.stdout:0/617: mkdir d3/dc/d31/ddc 0 2024-05-17T21:52:56.284 INFO:tasks.workunit.client.1.smithi138.stdout:4/393: read d0/d3/d1d/f2e [206978,103289] 0 2024-05-17T21:52:56.285 INFO:tasks.workunit.client.1.smithi138.stdout:3/454: chown d1/d4d/f7e 565638 1 2024-05-17T21:52:56.286 INFO:tasks.workunit.client.0.smithi107.stdout:1/593: sync 2024-05-17T21:52:56.286 INFO:tasks.workunit.client.0.smithi107.stdout:6/443: sync 2024-05-17T21:52:56.286 INFO:tasks.workunit.client.0.smithi107.stdout:3/566: sync 2024-05-17T21:52:56.289 INFO:tasks.workunit.client.1.smithi138.stdout:1/356: unlink d4/dd/d3b/d6c/c5a 0 2024-05-17T21:52:56.289 INFO:tasks.workunit.client.1.smithi138.stdout:6/393: dread d7/db/d12/f27 [0,4194304] 0 2024-05-17T21:52:56.289 INFO:tasks.workunit.client.0.smithi107.stdout:1/594: dread d4/d78/fab [0,4194304] 0 2024-05-17T21:52:56.290 INFO:tasks.workunit.client.1.smithi138.stdout:0/390: dwrite d3/d10/f25 [4194304,4194304] 0 2024-05-17T21:52:56.291 INFO:tasks.workunit.client.1.smithi138.stdout:9/337: getdents d6/d7/d16 0 2024-05-17T21:52:56.292 INFO:tasks.workunit.client.0.smithi107.stdout:7/512: creat d5/df/d11/d1c/d2f/d65/da7/fba x:0 0 0 2024-05-17T21:52:56.293 INFO:tasks.workunit.client.0.smithi107.stdout:3/567: dwrite d6/d8/f12 [4194304,4194304] 0 2024-05-17T21:52:56.295 INFO:tasks.workunit.client.0.smithi107.stdout:3/568: chown d6/d8/d10/f4b 15201 1 2024-05-17T21:52:56.297 INFO:tasks.workunit.client.0.smithi107.stdout:4/536: unlink d3/d4a/d47/d60/f5b 0 2024-05-17T21:52:56.305 INFO:tasks.workunit.client.0.smithi107.stdout:6/444: rename d9/l61 to d9/d58/d66/l8a 0 2024-05-17T21:52:56.306 INFO:tasks.workunit.client.0.smithi107.stdout:6/445: chown d9/d16/f4a 619852385 1 2024-05-17T21:52:56.308 INFO:tasks.workunit.client.0.smithi107.stdout:5/526: mknod d5/db/dc/d8f/ca3 0 2024-05-17T21:52:56.310 INFO:tasks.workunit.client.0.smithi107.stdout:1/595: creat d4/df/d14/d1d/fc3 x:0 0 0 2024-05-17T21:52:56.310 INFO:tasks.workunit.client.0.smithi107.stdout:8/526: link dc/c1a dc/d21/d35/cc0 0 2024-05-17T21:52:56.311 INFO:tasks.workunit.client.0.smithi107.stdout:8/527: write dc/d21/f80 [870177,71471] 0 2024-05-17T21:52:56.312 INFO:tasks.workunit.client.0.smithi107.stdout:0/618: dread d3/dc/d22/d23/f44 [0,4194304] 0 2024-05-17T21:52:56.313 INFO:tasks.workunit.client.0.smithi107.stdout:0/619: write d3/f11 [556599,89260] 0 2024-05-17T21:52:56.315 INFO:tasks.workunit.client.0.smithi107.stdout:9/560: getdents d0/dd/d21/d29/d57/d5f 0 2024-05-17T21:52:56.317 INFO:tasks.workunit.client.0.smithi107.stdout:6/446: chown d9/d16/f41 0 1 2024-05-17T21:52:56.319 INFO:tasks.workunit.client.0.smithi107.stdout:5/527: write d5/db/d16/d2d/f4f [694717,95822] 0 2024-05-17T21:52:56.319 INFO:tasks.workunit.client.0.smithi107.stdout:7/513: mknod d5/d16/d3b/d45/d91/dac/cbb 0 2024-05-17T21:52:56.322 INFO:tasks.workunit.client.0.smithi107.stdout:9/561: dwrite d0/dd/d21/d29/d3e/f5e [0,4194304] 0 2024-05-17T21:52:56.323 INFO:tasks.workunit.client.0.smithi107.stdout:3/569: mknod d6/d2a/d2d/d69/dad/cb7 0 2024-05-17T21:52:56.323 INFO:tasks.workunit.client.0.smithi107.stdout:7/514: truncate d5/df/d11/d1c/d2f/d65/da7/fba 496925 0 2024-05-17T21:52:56.324 INFO:tasks.workunit.client.0.smithi107.stdout:9/562: chown d0/dd/d21/d29/d3e/d45/d68/f6c 10 1 2024-05-17T21:52:56.324 INFO:tasks.workunit.client.0.smithi107.stdout:3/570: chown d6/d8/d3e/d3f/d90 0 1 2024-05-17T21:52:56.324 INFO:tasks.workunit.client.0.smithi107.stdout:8/528: creat dc/d44/d4b/d52/d70/fc1 x:0 0 0 2024-05-17T21:52:56.328 INFO:tasks.workunit.client.0.smithi107.stdout:6/447: creat d9/d16/d4d/f8b x:0 0 0 2024-05-17T21:52:56.330 INFO:tasks.workunit.client.0.smithi107.stdout:6/448: dread - d9/d16/d4d/f59 zero size 2024-05-17T21:52:56.332 INFO:tasks.workunit.client.0.smithi107.stdout:9/563: dread - d0/dd/d69/f6e zero size 2024-05-17T21:52:56.333 INFO:tasks.workunit.client.0.smithi107.stdout:8/529: mknod dc/d21/d35/cc2 0 2024-05-17T21:52:56.334 INFO:tasks.workunit.client.0.smithi107.stdout:0/620: write d3/fa1 [929908,60724] 0 2024-05-17T21:52:56.336 INFO:tasks.workunit.client.0.smithi107.stdout:4/537: truncate d3/d4a/f3f 2185574 0 2024-05-17T21:52:56.337 INFO:tasks.workunit.client.0.smithi107.stdout:0/621: chown d3/dc/d22/d23/d97/l82 84365 1 2024-05-17T21:52:56.337 INFO:tasks.workunit.client.0.smithi107.stdout:7/515: symlink d5/lbc 0 2024-05-17T21:52:56.338 INFO:tasks.workunit.client.0.smithi107.stdout:0/622: readlink d3/dc/d22/d23/d97/l82 0 2024-05-17T21:52:56.338 INFO:tasks.workunit.client.0.smithi107.stdout:8/530: creat dc/d21/daa/fc3 x:0 0 0 2024-05-17T21:52:56.339 INFO:tasks.workunit.client.0.smithi107.stdout:1/596: getdents d4/d6a 0 2024-05-17T21:52:56.342 INFO:tasks.workunit.client.0.smithi107.stdout:8/531: fsync dc/d21/f5d 0 2024-05-17T21:52:56.342 INFO:tasks.workunit.client.0.smithi107.stdout:0/623: creat d3/d24/d6f/db5/fdd x:0 0 0 2024-05-17T21:52:56.343 INFO:tasks.workunit.client.0.smithi107.stdout:7/516: truncate d5/df/d11/d1d/d33/f6a 1231384 0 2024-05-17T21:52:56.344 INFO:tasks.workunit.client.0.smithi107.stdout:8/532: chown dc/cac 49327392 1 2024-05-17T21:52:56.344 INFO:tasks.workunit.client.0.smithi107.stdout:4/538: truncate d3/db/d1e/f77 6587934 0 2024-05-17T21:52:56.345 INFO:tasks.workunit.client.0.smithi107.stdout:0/624: chown d3/d75/d9e/f72 8 1 2024-05-17T21:52:56.346 INFO:tasks.workunit.client.0.smithi107.stdout:7/517: mknod d5/d16/d3b/d45/cbd 0 2024-05-17T21:52:56.346 INFO:tasks.workunit.client.0.smithi107.stdout:8/533: read dc/d21/d27/f5b [3428230,35348] 0 2024-05-17T21:52:56.347 INFO:tasks.workunit.client.0.smithi107.stdout:8/534: write dc/d21/d27/d2f/fa1 [466814,106577] 0 2024-05-17T21:52:56.348 INFO:tasks.workunit.client.0.smithi107.stdout:1/597: link d4/df/f4d d4/df/d1a/fc4 0 2024-05-17T21:52:56.349 INFO:tasks.workunit.client.0.smithi107.stdout:3/571: dread d6/d8/d10/d25/f55 [0,4194304] 0 2024-05-17T21:52:56.349 INFO:tasks.workunit.client.0.smithi107.stdout:8/535: chown dc/d21/d27/d83/c98 4 1 2024-05-17T21:52:56.350 INFO:tasks.workunit.client.0.smithi107.stdout:4/539: dread - d3/d6/d26/d36/f6e zero size 2024-05-17T21:52:56.351 INFO:tasks.workunit.client.0.smithi107.stdout:5/528: write d5/db/d16/d54/d59/d65/f47 [1246031,9186] 0 2024-05-17T21:52:56.353 INFO:tasks.workunit.client.0.smithi107.stdout:6/449: write d9/f1a [3722128,25112] 0 2024-05-17T21:52:56.353 INFO:tasks.workunit.client.0.smithi107.stdout:9/564: write d0/dd/d1f/f52 [560613,108802] 0 2024-05-17T21:52:56.354 INFO:tasks.workunit.client.0.smithi107.stdout:9/565: fdatasync d0/dd/d21/d29/d3e/d45/d68/fa1 0 2024-05-17T21:52:56.355 INFO:tasks.workunit.client.0.smithi107.stdout:1/598: rename d4/df/d1a/d26/d72/f83 to d4/da7/fc5 0 2024-05-17T21:52:56.356 INFO:tasks.workunit.client.0.smithi107.stdout:0/625: symlink d3/dc/d31/ddc/lde 0 2024-05-17T21:52:56.357 INFO:tasks.workunit.client.0.smithi107.stdout:9/566: dread d0/dd/d21/d29/d3e/f91 [0,4194304] 0 2024-05-17T21:52:56.357 INFO:tasks.workunit.client.0.smithi107.stdout:1/599: write d4/df/d1a/d26/d72/fb4 [809545,124836] 0 2024-05-17T21:52:56.358 INFO:tasks.workunit.client.0.smithi107.stdout:7/518: mknod d5/cbe 0 2024-05-17T21:52:56.358 INFO:tasks.workunit.client.0.smithi107.stdout:1/600: chown d4/df/d4e/c74 393 1 2024-05-17T21:52:56.362 INFO:tasks.workunit.client.0.smithi107.stdout:8/536: creat dc/d21/db8/fc4 x:0 0 0 2024-05-17T21:52:56.364 INFO:tasks.workunit.client.0.smithi107.stdout:9/567: dwrite d0/dd/d2f/d87/d89/d9b/fa0 [0,4194304] 0 2024-05-17T21:52:56.365 INFO:tasks.workunit.client.0.smithi107.stdout:9/568: truncate d0/dd/d2f/f31 9186525 0 2024-05-17T21:52:56.368 INFO:tasks.workunit.client.0.smithi107.stdout:0/626: truncate d3/d24/f2a 4239485 0 2024-05-17T21:52:56.368 INFO:tasks.workunit.client.0.smithi107.stdout:9/569: truncate d0/dd/d2f/d81/f84 673703 0 2024-05-17T21:52:56.368 INFO:tasks.workunit.client.0.smithi107.stdout:5/529: mknod d5/db/d16/d2d/d4d/ca4 0 2024-05-17T21:52:56.372 INFO:tasks.workunit.client.0.smithi107.stdout:8/537: mkdir dc/d21/daa/dc5 0 2024-05-17T21:52:56.373 INFO:tasks.workunit.client.0.smithi107.stdout:6/450: creat d9/d58/d7b/f8c x:0 0 0 2024-05-17T21:52:56.374 INFO:tasks.workunit.client.0.smithi107.stdout:6/451: chown d9/d16/d22/l68 3767935 1 2024-05-17T21:52:56.375 INFO:tasks.workunit.client.0.smithi107.stdout:9/570: dwrite d0/dd/f35 [4194304,4194304] 0 2024-05-17T21:52:56.378 INFO:tasks.workunit.client.0.smithi107.stdout:0/627: write d3/dc/d22/d23/f44 [1967784,116677] 0 2024-05-17T21:52:56.379 INFO:tasks.workunit.client.0.smithi107.stdout:5/530: mknod d5/db/d16/d2d/d40/d5a/d7c/ca5 0 2024-05-17T21:52:56.381 INFO:tasks.workunit.client.0.smithi107.stdout:5/531: dread - d5/db/d16/d2a/d73/f99 zero size 2024-05-17T21:52:56.383 INFO:tasks.workunit.client.0.smithi107.stdout:2/400: sync 2024-05-17T21:52:56.383 INFO:tasks.workunit.client.0.smithi107.stdout:3/572: truncate d6/d2a/d2d/d48/f93 5706338 0 2024-05-17T21:52:56.391 INFO:tasks.workunit.client.0.smithi107.stdout:7/519: link d5/d16/f28 d5/df/d11/d1c/fbf 0 2024-05-17T21:52:56.391 INFO:tasks.workunit.client.0.smithi107.stdout:1/601: creat d4/df/d14/d2e/d43/fc6 x:0 0 0 2024-05-17T21:52:56.391 INFO:tasks.workunit.client.0.smithi107.stdout:1/602: chown d4/de/f36 0 1 2024-05-17T21:52:56.391 INFO:tasks.workunit.client.0.smithi107.stdout:5/532: mknod d5/db/dc/d1e/ca6 0 2024-05-17T21:52:56.391 INFO:tasks.workunit.client.0.smithi107.stdout:1/603: chown d4/df/d1a/d26/d7a/fb9 1990943 1 2024-05-17T21:52:56.396 INFO:tasks.workunit.client.0.smithi107.stdout:2/401: symlink d0/l75 0 2024-05-17T21:52:56.398 INFO:tasks.workunit.client.0.smithi107.stdout:7/520: dwrite d5/df/d11/d1c/d31/d39/f5e [0,4194304] 0 2024-05-17T21:52:56.399 INFO:tasks.workunit.client.0.smithi107.stdout:6/452: creat d9/f8d x:0 0 0 2024-05-17T21:52:56.403 INFO:tasks.workunit.client.0.smithi107.stdout:8/538: write dc/f86 [73683,36275] 0 2024-05-17T21:52:56.403 INFO:tasks.workunit.client.0.smithi107.stdout:5/533: dwrite d5/db/d39/f3c [0,4194304] 0 2024-05-17T21:52:56.404 INFO:tasks.workunit.client.0.smithi107.stdout:4/540: truncate d3/db/d1e/f77 5988169 0 2024-05-17T21:52:56.408 INFO:tasks.workunit.client.0.smithi107.stdout:0/628: link d3/dc/d22/d3d/c4f d3/d75/dc1/cdf 0 2024-05-17T21:52:56.408 INFO:tasks.workunit.client.0.smithi107.stdout:1/604: write d4/df/d1a/f58 [69498,37780] 0 2024-05-17T21:52:56.409 INFO:tasks.workunit.client.0.smithi107.stdout:9/571: dwrite d0/dd/d21/d29/f5a [0,4194304] 0 2024-05-17T21:52:56.409 INFO:tasks.workunit.client.0.smithi107.stdout:2/402: symlink d0/d26/d64/l76 0 2024-05-17T21:52:56.412 INFO:tasks.workunit.client.0.smithi107.stdout:6/453: truncate d9/d16/d4d/d67/f79 747674 0 2024-05-17T21:52:56.416 INFO:tasks.workunit.client.0.smithi107.stdout:4/541: unlink d3/d4a/d47/d60/lab 0 2024-05-17T21:52:56.419 INFO:tasks.workunit.client.0.smithi107.stdout:8/539: mknod dc/d21/cc6 0 2024-05-17T21:52:56.420 INFO:tasks.workunit.client.0.smithi107.stdout:9/572: mkdir d0/dd/d21/d29/d57/d5f/dc6 0 2024-05-17T21:52:56.424 INFO:tasks.workunit.client.0.smithi107.stdout:2/403: link d0/d9/d11/f52 d0/d9/f77 0 2024-05-17T21:52:56.424 INFO:tasks.workunit.client.0.smithi107.stdout:8/540: stat dc/d44/d4b/d52/d70/d76/f78 0 2024-05-17T21:52:56.424 INFO:tasks.workunit.client.0.smithi107.stdout:1/605: link d4/c64 d4/de/cc7 0 2024-05-17T21:52:56.424 INFO:tasks.workunit.client.0.smithi107.stdout:3/573: dwrite d6/d2a/d2d/f43 [4194304,4194304] 0 2024-05-17T21:52:56.424 INFO:tasks.workunit.client.0.smithi107.stdout:5/534: link d5/db/d16/d54/d59/d65/d48/f8a d5/db/d16/d2d/d40/d5a/fa7 0 2024-05-17T21:52:56.426 INFO:tasks.workunit.client.1.smithi138.stdout:8/284: symlink d0/dc/d1a/l57 0 2024-05-17T21:52:56.426 INFO:tasks.workunit.client.0.smithi107.stdout:7/521: dread d5/fe [0,4194304] 0 2024-05-17T21:52:56.426 INFO:tasks.workunit.client.0.smithi107.stdout:9/573: write d0/dd/d3d/fb8 [396980,51429] 0 2024-05-17T21:52:56.427 INFO:tasks.workunit.client.1.smithi138.stdout:8/285: chown d0/d31 2923086 1 2024-05-17T21:52:56.428 INFO:tasks.workunit.client.1.smithi138.stdout:1/357: unlink d4/df/d35/f61 0 2024-05-17T21:52:56.429 INFO:tasks.workunit.client.1.smithi138.stdout:3/455: write d1/d3/d16/f29 [2225237,117417] 0 2024-05-17T21:52:56.430 INFO:tasks.workunit.client.1.smithi138.stdout:6/394: creat d7/db/d43/d5f/d6a/f89 x:0 0 0 2024-05-17T21:52:56.432 INFO:tasks.workunit.client.1.smithi138.stdout:3/456: chown d1/d4f/d50/c99 122517463 1 2024-05-17T21:52:56.432 INFO:tasks.workunit.client.1.smithi138.stdout:7/298: link d0/l28 d0/d39/l5a 0 2024-05-17T21:52:56.432 INFO:tasks.workunit.client.1.smithi138.stdout:0/391: symlink d3/d10/d38/d3e/d59/l85 0 2024-05-17T21:52:56.433 INFO:tasks.workunit.client.1.smithi138.stdout:9/338: dread d6/f24 [0,4194304] 0 2024-05-17T21:52:56.434 INFO:tasks.workunit.client.1.smithi138.stdout:7/299: dread - d0/d18/d40/f4e zero size 2024-05-17T21:52:56.435 INFO:tasks.workunit.client.1.smithi138.stdout:5/379: rename d1/f43 to d1/d2c/f7b 0 2024-05-17T21:52:56.435 INFO:tasks.workunit.client.0.smithi107.stdout:4/542: dread d3/d4a/f3f [0,4194304] 0 2024-05-17T21:52:56.436 INFO:tasks.workunit.client.1.smithi138.stdout:5/380: dread - d1/d6/d1b/d29/d36/f7a zero size 2024-05-17T21:52:56.437 INFO:tasks.workunit.client.0.smithi107.stdout:6/454: dwrite d9/f10 [8388608,4194304] 0 2024-05-17T21:52:56.437 INFO:tasks.workunit.client.0.smithi107.stdout:3/574: creat d6/d2a/d2d/da8/fb8 x:0 0 0 2024-05-17T21:52:56.441 INFO:tasks.workunit.client.1.smithi138.stdout:1/358: symlink d4/df/d35/l74 0 2024-05-17T21:52:56.441 INFO:tasks.workunit.client.0.smithi107.stdout:5/535: dread d5/db/d16/f81 [0,4194304] 0 2024-05-17T21:52:56.441 INFO:tasks.workunit.client.0.smithi107.stdout:1/606: dwrite d4/df/d14/d2e/d43/d47/f94 [0,4194304] 0 2024-05-17T21:52:56.442 INFO:tasks.workunit.client.0.smithi107.stdout:3/575: readlink d6/d2a/d2d/d37/l45 0 2024-05-17T21:52:56.443 INFO:tasks.workunit.client.1.smithi138.stdout:0/392: dread d3/fd [0,4194304] 0 2024-05-17T21:52:56.445 INFO:tasks.workunit.client.1.smithi138.stdout:6/395: mknod d7/db/d43/d3f/d79/c8a 0 2024-05-17T21:52:56.446 INFO:tasks.workunit.client.0.smithi107.stdout:8/541: dwrite dc/f8b [0,4194304] 0 2024-05-17T21:52:56.446 INFO:tasks.workunit.client.0.smithi107.stdout:4/543: write d3/d4a/d47/d60/d3a/d41/d80/f44 [3219070,121782] 0 2024-05-17T21:52:56.447 INFO:tasks.workunit.client.1.smithi138.stdout:3/457: creat d1/d4f/d50/f9e x:0 0 0 2024-05-17T21:52:56.447 INFO:tasks.workunit.client.0.smithi107.stdout:7/522: mkdir d5/df/d11/d1c/d31/d39/dc0 0 2024-05-17T21:52:56.452 INFO:tasks.workunit.client.0.smithi107.stdout:8/542: readlink dc/d21/d27/d34/d4c/l6c 0 2024-05-17T21:52:56.453 INFO:tasks.workunit.client.0.smithi107.stdout:0/629: dwrite d3/d39/fb0 [0,4194304] 0 2024-05-17T21:52:56.455 INFO:tasks.workunit.client.1.smithi138.stdout:7/300: creat d0/d18/d40/d41/f5b x:0 0 0 2024-05-17T21:52:56.457 INFO:tasks.workunit.client.0.smithi107.stdout:2/404: readlink d0/d9/d11/l67 0 2024-05-17T21:52:56.458 INFO:tasks.workunit.client.1.smithi138.stdout:2/408: rename d3/d21/f59 to d3/d21/d2f/d3e/d63/f83 0 2024-05-17T21:52:56.459 INFO:tasks.workunit.client.1.smithi138.stdout:1/359: symlink d4/dd/d64/l75 0 2024-05-17T21:52:56.460 INFO:tasks.workunit.client.1.smithi138.stdout:2/409: write d3/d21/d2f/f5f [437312,46170] 0 2024-05-17T21:52:56.460 INFO:tasks.workunit.client.0.smithi107.stdout:1/607: dread - d4/df/d4e/d59/f73 zero size 2024-05-17T21:52:56.462 INFO:tasks.workunit.client.0.smithi107.stdout:5/536: creat d5/db/d16/d54/d59/d65/d48/d89/fa8 x:0 0 0 2024-05-17T21:52:56.464 INFO:tasks.workunit.client.0.smithi107.stdout:0/630: dwrite d3/d58/faa [0,4194304] 0 2024-05-17T21:52:56.465 INFO:tasks.workunit.client.0.smithi107.stdout:5/537: write d5/db/d39/f43 [2452383,81136] 0 2024-05-17T21:52:56.465 INFO:tasks.workunit.client.0.smithi107.stdout:7/523: mkdir d5/d16/d3b/d45/d91/dc1 0 2024-05-17T21:52:56.467 INFO:tasks.workunit.client.1.smithi138.stdout:2/410: dwrite d3/f61 [0,4194304] 0 2024-05-17T21:52:56.468 INFO:tasks.workunit.client.0.smithi107.stdout:7/524: chown d5/df/d11/d1c/d31/d56/f9c 202 1 2024-05-17T21:52:56.468 INFO:tasks.workunit.client.1.smithi138.stdout:3/458: creat d1/d3/d8/d5c/f9f x:0 0 0 2024-05-17T21:52:56.474 INFO:tasks.workunit.client.0.smithi107.stdout:9/574: write d0/dd/d69/f6e [100238,52143] 0 2024-05-17T21:52:56.477 INFO:tasks.workunit.client.0.smithi107.stdout:2/405: symlink d0/d9/d11/df/d5a/d6d/l78 0 2024-05-17T21:52:56.478 INFO:tasks.workunit.client.0.smithi107.stdout:6/455: dread d9/d16/d22/d2d/f2f [0,4194304] 0 2024-05-17T21:52:56.481 INFO:tasks.workunit.client.1.smithi138.stdout:8/286: rename d0/fa to d0/dc/d1a/f58 0 2024-05-17T21:52:56.481 INFO:tasks.workunit.client.1.smithi138.stdout:9/339: write d6/d7/d16/d29/d48/d6e/f56 [1442879,81842] 0 2024-05-17T21:52:56.482 INFO:tasks.workunit.client.0.smithi107.stdout:6/456: read d9/d16/d22/f74 [347907,58906] 0 2024-05-17T21:52:56.482 INFO:tasks.workunit.client.0.smithi107.stdout:0/631: dread d3/d39/f49 [0,4194304] 0 2024-05-17T21:52:56.484 INFO:tasks.workunit.client.0.smithi107.stdout:6/457: write d9/d16/f5e [3581665,14899] 0 2024-05-17T21:52:56.485 INFO:tasks.workunit.client.1.smithi138.stdout:8/287: dread d0/f9 [0,4194304] 0 2024-05-17T21:52:56.487 INFO:tasks.workunit.client.1.smithi138.stdout:8/288: truncate d0/d11/f42 616703 0 2024-05-17T21:52:56.488 INFO:tasks.workunit.client.1.smithi138.stdout:1/360: fsync d4/fa 0 2024-05-17T21:52:56.489 INFO:tasks.workunit.client.1.smithi138.stdout:1/361: chown d4/df/d3a/c41 72364 1 2024-05-17T21:52:56.489 INFO:tasks.workunit.client.0.smithi107.stdout:7/525: truncate d5/d16/f23 480784 0 2024-05-17T21:52:56.489 INFO:tasks.workunit.client.1.smithi138.stdout:6/396: write d7/db/d34/f53 [4161130,80152] 0 2024-05-17T21:52:56.490 INFO:tasks.workunit.client.0.smithi107.stdout:3/576: dwrite d6/d1a/f4e [0,4194304] 0 2024-05-17T21:52:56.490 INFO:tasks.workunit.client.1.smithi138.stdout:1/362: chown d4/df/l1a 275904357 1 2024-05-17T21:52:56.491 INFO:tasks.workunit.client.1.smithi138.stdout:0/393: mknod d3/d10/d38/c86 0 2024-05-17T21:52:56.491 INFO:tasks.workunit.client.1.smithi138.stdout:1/363: write d4/df/f2c [8155505,56178] 0 2024-05-17T21:52:56.494 INFO:tasks.workunit.client.0.smithi107.stdout:8/543: write f4 [1471746,101584] 0 2024-05-17T21:52:56.494 INFO:tasks.workunit.client.0.smithi107.stdout:9/575: creat d0/dd/d21/d29/d3e/fc7 x:0 0 0 2024-05-17T21:52:56.495 INFO:tasks.workunit.client.0.smithi107.stdout:8/544: fsync dc/d21/f80 0 2024-05-17T21:52:56.497 INFO:tasks.workunit.client.0.smithi107.stdout:4/544: dread d3/db/f33 [4194304,4194304] 0 2024-05-17T21:52:56.498 INFO:tasks.workunit.client.0.smithi107.stdout:8/545: write dc/d44/d4b/d52/d70/d76/f78 [805877,83264] 0 2024-05-17T21:52:56.499 INFO:tasks.workunit.client.0.smithi107.stdout:4/545: fdatasync d3/d4a/d47/f4b 0 2024-05-17T21:52:56.502 INFO:tasks.workunit.client.0.smithi107.stdout:2/406: read d0/d9/d1c/d22/f31 [5323307,40328] 0 2024-05-17T21:52:56.504 INFO:tasks.workunit.client.0.smithi107.stdout:0/632: fdatasync d3/dc/d22/d23/f29 0 2024-05-17T21:52:56.508 INFO:tasks.workunit.client.0.smithi107.stdout:6/458: symlink d9/d16/d4d/d85/d88/l8e 0 2024-05-17T21:52:56.509 INFO:tasks.workunit.client.1.smithi138.stdout:8/289: mkdir d0/dc/d1a/d59 0 2024-05-17T21:52:56.510 INFO:tasks.workunit.client.1.smithi138.stdout:7/301: truncate d0/f11 3386996 0 2024-05-17T21:52:56.512 INFO:tasks.workunit.client.0.smithi107.stdout:3/577: rmdir d6/d2a/d2d 39 2024-05-17T21:52:56.513 INFO:tasks.workunit.client.1.smithi138.stdout:6/397: mkdir d7/db/d43/d5f/d6a/d8b 0 2024-05-17T21:52:56.515 INFO:tasks.workunit.client.1.smithi138.stdout:9/340: write d6/d7/d16/f20 [993934,75540] 0 2024-05-17T21:52:56.515 INFO:tasks.workunit.client.1.smithi138.stdout:1/364: fsync d4/db/f6a 0 2024-05-17T21:52:56.516 INFO:tasks.workunit.client.1.smithi138.stdout:9/341: readlink d6/d7/d16/l18 0 2024-05-17T21:52:56.518 INFO:tasks.workunit.client.1.smithi138.stdout:3/459: mknod d1/d62/d9b/ca0 0 2024-05-17T21:52:56.522 INFO:tasks.workunit.client.1.smithi138.stdout:2/411: mkdir d3/d21/d2f/d3e/d63/d7f/d84 0 2024-05-17T21:52:56.522 INFO:tasks.workunit.client.1.smithi138.stdout:0/394: dwrite d3/d10/f15 [0,4194304] 0 2024-05-17T21:52:56.522 INFO:tasks.workunit.client.1.smithi138.stdout:2/412: readlink d3/d21/l3d 0 2024-05-17T21:52:56.524 INFO:tasks.workunit.client.1.smithi138.stdout:2/413: write d3/d21/d2f/f33 [1993083,32511] 0 2024-05-17T21:52:56.528 INFO:tasks.workunit.client.1.smithi138.stdout:8/290: creat d0/d31/f5a x:0 0 0 2024-05-17T21:52:56.529 INFO:tasks.workunit.client.1.smithi138.stdout:8/291: dread - d0/dc/d3b/f43 zero size 2024-05-17T21:52:56.530 INFO:tasks.workunit.client.0.smithi107.stdout:2/407: creat d0/d9/d11/d18/f79 x:0 0 0 2024-05-17T21:52:56.530 INFO:tasks.workunit.client.0.smithi107.stdout:7/526: dwrite d5/df/d11/d1d/d33/d82/fb1 [0,4194304] 0 2024-05-17T21:52:56.531 INFO:tasks.workunit.client.0.smithi107.stdout:2/408: readlink d0/d9/d1c/l1f 0 2024-05-17T21:52:56.531 INFO:tasks.workunit.client.0.smithi107.stdout:7/527: readlink d5/df/d11/l37 0 2024-05-17T21:52:56.532 INFO:tasks.workunit.client.0.smithi107.stdout:7/528: readlink d5/df/d11/d1c/d2f/l77 0 2024-05-17T21:52:56.532 INFO:tasks.workunit.client.1.smithi138.stdout:6/398: mknod d7/db/d43/d5f/d6a/c8c 0 2024-05-17T21:52:56.533 INFO:tasks.workunit.client.1.smithi138.stdout:6/399: chown d7/f22 174 1 2024-05-17T21:52:56.534 INFO:tasks.workunit.client.1.smithi138.stdout:1/365: symlink d4/df/d35/l76 0 2024-05-17T21:52:56.534 INFO:tasks.workunit.client.1.smithi138.stdout:6/400: readlink l5 0 2024-05-17T21:52:56.537 INFO:tasks.workunit.client.1.smithi138.stdout:9/342: mkdir d6/d7/d16/d29/d48/d6f 0 2024-05-17T21:52:56.538 INFO:tasks.workunit.client.0.smithi107.stdout:6/459: dwrite d9/d16/f3e [0,4194304] 0 2024-05-17T21:52:56.540 INFO:tasks.workunit.client.1.smithi138.stdout:3/460: creat d1/d2e/fa1 x:0 0 0 2024-05-17T21:52:56.542 INFO:tasks.workunit.client.1.smithi138.stdout:5/381: rename d1/d6/d1b/f21 to d1/d6/d1b/d29/f7c 0 2024-05-17T21:52:56.544 INFO:tasks.workunit.client.1.smithi138.stdout:1/366: dwrite d4/db/f6a [0,4194304] 0 2024-05-17T21:52:56.547 INFO:tasks.workunit.client.1.smithi138.stdout:4/394: sync 2024-05-17T21:52:56.547 INFO:tasks.workunit.client.1.smithi138.stdout:4/395: chown d0/d1b/d88 5024644 1 2024-05-17T21:52:56.549 INFO:tasks.workunit.client.0.smithi107.stdout:1/608: getdents d4/de 0 2024-05-17T21:52:56.551 INFO:tasks.workunit.client.0.smithi107.stdout:8/546: mknod dc/d21/d27/d34/cc7 0 2024-05-17T21:52:56.552 INFO:tasks.workunit.client.1.smithi138.stdout:7/302: mkdir d0/d5c 0 2024-05-17T21:52:56.554 INFO:tasks.workunit.client.1.smithi138.stdout:6/401: mknod d7/d69/c8d 0 2024-05-17T21:52:56.555 INFO:tasks.workunit.client.1.smithi138.stdout:6/402: fsync d7/f2e 0 2024-05-17T21:52:56.557 INFO:tasks.workunit.client.0.smithi107.stdout:9/576: truncate d0/dd/d1f/f82 2359772 0 2024-05-17T21:52:56.558 INFO:tasks.workunit.client.1.smithi138.stdout:0/395: symlink d3/d10/d6d/d81/l87 0 2024-05-17T21:52:56.560 INFO:tasks.workunit.client.1.smithi138.stdout:7/303: dwrite d0/d18/d40/d41/d50/f59 [0,4194304] 0 2024-05-17T21:52:56.561 INFO:tasks.workunit.client.1.smithi138.stdout:7/304: write d0/f4a [116255,120384] 0 2024-05-17T21:52:56.562 INFO:tasks.workunit.client.0.smithi107.stdout:0/633: mknod d3/dc/d22/ce0 0 2024-05-17T21:52:56.564 INFO:tasks.workunit.client.1.smithi138.stdout:7/305: write d0/f24 [5072653,127597] 0 2024-05-17T21:52:56.567 INFO:tasks.workunit.client.0.smithi107.stdout:7/529: creat d5/df/d11/d1c/d31/d39/d92/d9a/fc2 x:0 0 0 2024-05-17T21:52:56.568 INFO:tasks.workunit.client.1.smithi138.stdout:2/414: mkdir d3/d81/d85 0 2024-05-17T21:52:56.568 INFO:tasks.workunit.client.1.smithi138.stdout:2/415: readlink d3/d21/d2f/l42 0 2024-05-17T21:52:56.569 INFO:tasks.workunit.client.0.smithi107.stdout:5/538: getdents d5/db/d16/d2d/d40/d5a/d7c/d8e 0 2024-05-17T21:52:56.570 INFO:tasks.workunit.client.1.smithi138.stdout:2/416: write d3/d21/d2f/f3c [3137225,51100] 0 2024-05-17T21:52:56.570 INFO:tasks.workunit.client.0.smithi107.stdout:5/539: dread - d5/db/d16/d2a/f94 zero size 2024-05-17T21:52:56.570 INFO:tasks.workunit.client.1.smithi138.stdout:5/382: mknod d1/d1a/d39/d4e/d58/c7d 0 2024-05-17T21:52:56.571 INFO:tasks.workunit.client.1.smithi138.stdout:5/383: readlink d1/d6/l75 0 2024-05-17T21:52:56.574 INFO:tasks.workunit.client.1.smithi138.stdout:1/367: mknod d4/dd/d3b/d40/c77 0 2024-05-17T21:52:56.574 INFO:tasks.workunit.client.0.smithi107.stdout:7/530: dwrite d5/df/d11/d1d/d33/d82/fb1 [0,4194304] 0 2024-05-17T21:52:56.575 INFO:tasks.workunit.client.0.smithi107.stdout:7/531: chown d5/df/d11/d1d/d44/f50 969154 1 2024-05-17T21:52:56.582 INFO:tasks.workunit.client.0.smithi107.stdout:1/609: dread d4/df/d1a/d26/d72/d85/f92 [0,4194304] 0 2024-05-17T21:52:56.582 INFO:tasks.workunit.client.0.smithi107.stdout:2/409: truncate d0/f36 1912323 0 2024-05-17T21:52:56.594 INFO:tasks.workunit.client.0.smithi107.stdout:3/578: symlink d6/d2a/d2d/da8/daa/lb9 0 2024-05-17T21:52:56.594 INFO:tasks.workunit.client.0.smithi107.stdout:9/577: write d0/fc [3895477,108430] 0 2024-05-17T21:52:56.594 INFO:tasks.workunit.client.0.smithi107.stdout:9/578: chown d0/dd/d1f/c7d 69262854 1 2024-05-17T21:52:56.595 INFO:tasks.workunit.client.0.smithi107.stdout:6/460: link d9/d16/d4d/d85/d88/f89 d9/d58/d7b/f8f 0 2024-05-17T21:52:56.596 INFO:tasks.workunit.client.1.smithi138.stdout:5/384: truncate d1/d1a/f26 865230 0 2024-05-17T21:52:56.599 INFO:tasks.workunit.client.0.smithi107.stdout:8/547: symlink dc/d21/d27/d2f/lc8 0 2024-05-17T21:52:56.601 INFO:tasks.workunit.client.0.smithi107.stdout:3/579: dread d6/d2a/d2d/d48/d4f/f94 [0,4194304] 0 2024-05-17T21:52:56.601 INFO:tasks.workunit.client.1.smithi138.stdout:8/292: rmdir d0/dc/d1a/d59 0 2024-05-17T21:52:56.602 INFO:tasks.workunit.client.1.smithi138.stdout:0/396: dwrite d3/d3d/d51/f6e [0,4194304] 0 2024-05-17T21:52:56.602 INFO:tasks.workunit.client.0.smithi107.stdout:4/546: getdents d3/d6/d26/d36 0 2024-05-17T21:52:56.603 INFO:tasks.workunit.client.1.smithi138.stdout:8/293: dread - d0/d2f/f41 zero size 2024-05-17T21:52:56.603 INFO:tasks.workunit.client.1.smithi138.stdout:6/403: dread d7/db/d43/f64 [0,4194304] 0 2024-05-17T21:52:56.604 INFO:tasks.workunit.client.1.smithi138.stdout:3/461: creat d1/d3/d16/fa2 x:0 0 0 2024-05-17T21:52:56.605 INFO:tasks.workunit.client.0.smithi107.stdout:2/410: mkdir d0/d26/d64/d7a 0 2024-05-17T21:52:56.609 INFO:tasks.workunit.client.1.smithi138.stdout:9/343: rename d6/d7/d16/d29/d38/d40/l68 to d6/l70 0 2024-05-17T21:52:56.610 INFO:tasks.workunit.client.0.smithi107.stdout:0/634: mkdir d3/dc/dae/dc0/de1 0 2024-05-17T21:52:56.611 INFO:tasks.workunit.client.1.smithi138.stdout:5/385: creat d1/d1a/d39/d62/f7e x:0 0 0 2024-05-17T21:52:56.613 INFO:tasks.workunit.client.0.smithi107.stdout:5/540: write d5/db/d16/f4b [4180919,62574] 0 2024-05-17T21:52:56.614 INFO:tasks.workunit.client.1.smithi138.stdout:1/368: dread f2 [0,4194304] 0 2024-05-17T21:52:56.614 INFO:tasks.workunit.client.1.smithi138.stdout:0/397: rmdir d3/d10/d38/d3e/d59 39 2024-05-17T21:52:56.615 INFO:tasks.workunit.client.1.smithi138.stdout:6/404: creat d7/d69/f8e x:0 0 0 2024-05-17T21:52:56.616 INFO:tasks.workunit.client.1.smithi138.stdout:6/405: write d7/d69/f71 [870963,6225] 0 2024-05-17T21:52:56.617 INFO:tasks.workunit.client.1.smithi138.stdout:6/406: stat d7 0 2024-05-17T21:52:56.618 INFO:tasks.workunit.client.1.smithi138.stdout:8/294: unlink d0/dc/d1a/d33/f34 0 2024-05-17T21:52:56.618 INFO:tasks.workunit.client.0.smithi107.stdout:5/541: dwrite d5/db/d39/f3c [0,4194304] 0 2024-05-17T21:52:56.624 INFO:tasks.workunit.client.1.smithi138.stdout:4/396: rename d0/f2a to d0/d3/d1d/d4c/f8d 0 2024-05-17T21:52:56.627 INFO:tasks.workunit.client.1.smithi138.stdout:4/397: truncate d0/d2f/d49/d60/f89 192358 0 2024-05-17T21:52:56.631 INFO:tasks.workunit.client.1.smithi138.stdout:3/462: dread d1/d3/d8/de/f27 [0,4194304] 0 2024-05-17T21:52:56.632 INFO:tasks.workunit.client.1.smithi138.stdout:3/463: readlink d1/d2e/l40 0 2024-05-17T21:52:56.633 INFO:tasks.workunit.client.0.smithi107.stdout:2/411: symlink d0/d9/d11/df/l7b 0 2024-05-17T21:52:56.634 INFO:tasks.workunit.client.0.smithi107.stdout:1/610: truncate d4/df/d1a/d26/d7a/fa5 2518560 0 2024-05-17T21:52:56.635 INFO:tasks.workunit.client.1.smithi138.stdout:9/344: dread d6/d7/f9 [0,4194304] 0 2024-05-17T21:52:56.638 INFO:tasks.workunit.client.0.smithi107.stdout:0/635: read d3/f43 [304909,119810] 0 2024-05-17T21:52:56.638 INFO:tasks.workunit.client.1.smithi138.stdout:1/369: creat d4/dd/d3b/d6c/f78 x:0 0 0 2024-05-17T21:52:56.643 INFO:tasks.workunit.client.0.smithi107.stdout:7/532: truncate d5/df/f20 1046863 0 2024-05-17T21:52:56.643 INFO:tasks.workunit.client.1.smithi138.stdout:7/306: truncate d0/f2a 3840099 0 2024-05-17T21:52:56.644 INFO:tasks.workunit.client.0.smithi107.stdout:0/636: dwrite d3/dc/d22/d23/d97/d85/fd9 [0,4194304] 0 2024-05-17T21:52:56.647 INFO:tasks.workunit.client.0.smithi107.stdout:8/548: creat dc/d21/daa/db4/fc9 x:0 0 0 2024-05-17T21:52:56.648 INFO:tasks.workunit.client.1.smithi138.stdout:4/398: creat d0/d3/d17/d5e/f8e x:0 0 0 2024-05-17T21:52:56.649 INFO:tasks.workunit.client.0.smithi107.stdout:8/549: chown dc/d21/daa/db4 130893022 1 2024-05-17T21:52:56.649 INFO:tasks.workunit.client.0.smithi107.stdout:0/637: read d3/ff [4917024,12731] 0 2024-05-17T21:52:56.652 INFO:tasks.workunit.client.1.smithi138.stdout:5/386: creat d1/d1a/d39/d69/f7f x:0 0 0 2024-05-17T21:52:56.654 INFO:tasks.workunit.client.0.smithi107.stdout:6/461: truncate d9/d16/d4d/f52 129198 0 2024-05-17T21:52:56.655 INFO:tasks.workunit.client.1.smithi138.stdout:0/398: fdatasync d3/fc 0 2024-05-17T21:52:56.655 INFO:tasks.workunit.client.0.smithi107.stdout:5/542: dwrite d5/db/d16/f81 [0,4194304] 0 2024-05-17T21:52:56.656 INFO:tasks.workunit.client.1.smithi138.stdout:0/399: readlink d3/l56 0 2024-05-17T21:52:56.660 INFO:tasks.workunit.client.1.smithi138.stdout:1/370: rename d4/d2e/c34 to d4/df/c79 0 2024-05-17T21:52:56.660 INFO:tasks.workunit.client.0.smithi107.stdout:2/412: dread d0/fa [0,4194304] 0 2024-05-17T21:52:56.661 INFO:tasks.workunit.client.1.smithi138.stdout:8/295: dwrite d0/f2 [4194304,4194304] 0 2024-05-17T21:52:56.661 INFO:tasks.workunit.client.0.smithi107.stdout:4/547: dread d3/db/d35/d5e/f63 [0,4194304] 0 2024-05-17T21:52:56.661 INFO:tasks.workunit.client.1.smithi138.stdout:1/371: fdatasync d4/db/f4d 0 2024-05-17T21:52:56.662 INFO:tasks.workunit.client.0.smithi107.stdout:8/550: dwrite dc/d21/d35/f2e [0,4194304] 0 2024-05-17T21:52:56.663 INFO:tasks.workunit.client.0.smithi107.stdout:8/551: stat dc/d21/f92 0 2024-05-17T21:52:56.663 INFO:tasks.workunit.client.0.smithi107.stdout:4/548: chown d3/d4a/d47/la8 5 1 2024-05-17T21:52:56.664 INFO:tasks.workunit.client.0.smithi107.stdout:8/552: chown dc/d21/d97/f15 1941 1 2024-05-17T21:52:56.671 INFO:tasks.workunit.client.0.smithi107.stdout:3/580: creat d6/d8/d3e/d3f/fba x:0 0 0 2024-05-17T21:52:56.672 INFO:tasks.workunit.client.0.smithi107.stdout:3/581: read - d6/d8/d10/d35/fb0 zero size 2024-05-17T21:52:56.673 INFO:tasks.workunit.client.1.smithi138.stdout:1/372: dwrite d4/d2e/f54 [0,4194304] 0 2024-05-17T21:52:56.678 INFO:tasks.workunit.client.1.smithi138.stdout:7/307: symlink d0/d18/d40/d41/l5d 0 2024-05-17T21:52:56.686 INFO:tasks.workunit.client.0.smithi107.stdout:1/611: mkdir d4/df/d1a/d41/db2/dc8 0 2024-05-17T21:52:56.687 INFO:tasks.workunit.client.0.smithi107.stdout:5/543: unlink l1 0 2024-05-17T21:52:56.688 INFO:tasks.workunit.client.0.smithi107.stdout:5/544: chown d5/db/d16/d54/d59/d65/d48/d89 133 1 2024-05-17T21:52:56.690 INFO:tasks.workunit.client.0.smithi107.stdout:6/462: mknod d9/d16/d22/d5f/d6e/c90 0 2024-05-17T21:52:56.691 INFO:tasks.workunit.client.1.smithi138.stdout:9/345: creat d6/d7/d16/d29/d48/d6f/f71 x:0 0 0 2024-05-17T21:52:56.692 INFO:tasks.workunit.client.1.smithi138.stdout:9/346: dread - d6/d7/d16/d29/d38/d40/f6b zero size 2024-05-17T21:52:56.694 INFO:tasks.workunit.client.0.smithi107.stdout:2/413: symlink d0/d9/d1c/d3e/l7c 0 2024-05-17T21:52:56.695 INFO:tasks.workunit.client.0.smithi107.stdout:0/638: dread d3/f33 [0,4194304] 0 2024-05-17T21:52:56.697 INFO:tasks.workunit.client.1.smithi138.stdout:6/407: dwrite d7/db/d43/d3f/f4a [0,4194304] 0 2024-05-17T21:52:56.697 INFO:tasks.workunit.client.0.smithi107.stdout:4/549: truncate d3/db/d35/d5e/f63 2868355 0 2024-05-17T21:52:56.697 INFO:tasks.workunit.client.0.smithi107.stdout:4/550: readlink d3/db/d1e/l2a 0 2024-05-17T21:52:56.697 INFO:tasks.workunit.client.1.smithi138.stdout:6/408: readlink d7/db/d60/l75 0 2024-05-17T21:52:56.701 INFO:tasks.workunit.client.0.smithi107.stdout:3/582: creat d6/d8/d10/d35/d36/d66/d88/fbb x:0 0 0 2024-05-17T21:52:56.703 INFO:tasks.workunit.client.0.smithi107.stdout:3/583: write d6/d8/d3e/d3f/f87 [6993394,8501] 0 2024-05-17T21:52:56.704 INFO:tasks.workunit.client.1.smithi138.stdout:6/409: dwrite d7/db/d43/d3f/f5a [0,4194304] 0 2024-05-17T21:52:56.704 INFO:tasks.workunit.client.0.smithi107.stdout:3/584: fdatasync d6/d2a/d2d/d69/f6d 0 2024-05-17T21:52:56.705 INFO:tasks.workunit.client.0.smithi107.stdout:3/585: chown d6/d2a/f6a 1335025 1 2024-05-17T21:52:56.712 INFO:tasks.workunit.client.1.smithi138.stdout:7/308: rename d0/d18/l49 to d0/d5c/l5e 0 2024-05-17T21:52:56.714 INFO:tasks.workunit.client.1.smithi138.stdout:4/399: mkdir d0/d3/d8f 0 2024-05-17T21:52:56.718 INFO:tasks.workunit.client.1.smithi138.stdout:5/387: link d1/f4f d1/d53/f80 0 2024-05-17T21:52:56.719 INFO:tasks.workunit.client.1.smithi138.stdout:5/388: stat d1/d6/d1b/d29/f67 0 2024-05-17T21:52:56.720 INFO:tasks.workunit.client.1.smithi138.stdout:5/389: fsync d1/d6/d19/f20 0 2024-05-17T21:52:56.721 INFO:tasks.workunit.client.1.smithi138.stdout:4/400: dwrite d0/d2f/d49/d60/d1f/f70 [0,4194304] 0 2024-05-17T21:52:56.726 INFO:tasks.workunit.client.1.smithi138.stdout:1/373: write d4/dd/f13 [3145836,54097] 0 2024-05-17T21:52:56.729 INFO:tasks.workunit.client.1.smithi138.stdout:9/347: dwrite d6/d7/d16/d29/d48/f22 [0,4194304] 0 2024-05-17T21:52:56.730 INFO:tasks.workunit.client.1.smithi138.stdout:8/296: symlink d0/dc/d1a/d33/d4a/d4e/l5b 0 2024-05-17T21:52:56.730 INFO:tasks.workunit.client.0.smithi107.stdout:8/553: dread dc/d21/d27/d34/f38 [0,4194304] 0 2024-05-17T21:52:56.735 INFO:tasks.workunit.client.1.smithi138.stdout:6/410: creat d7/db/d43/d63/f8f x:0 0 0 2024-05-17T21:52:56.737 INFO:tasks.workunit.client.1.smithi138.stdout:8/297: dwrite d0/dc/f2b [0,4194304] 0 2024-05-17T21:52:56.753 INFO:tasks.workunit.client.0.smithi107.stdout:6/463: creat d9/d16/d22/d5f/d6f/f91 x:0 0 0 2024-05-17T21:52:56.756 INFO:tasks.workunit.client.0.smithi107.stdout:0/639: symlink d3/d24/d6f/d9f/dab/le2 0 2024-05-17T21:52:56.759 INFO:tasks.workunit.client.1.smithi138.stdout:7/309: mkdir d0/d18/d40/d5f 0 2024-05-17T21:52:56.762 INFO:tasks.workunit.client.0.smithi107.stdout:3/586: symlink d6/d8/d10/d35/d91/lbc 0 2024-05-17T21:52:56.763 INFO:tasks.workunit.client.0.smithi107.stdout:3/587: write d6/d2a/f6a [609725,105968] 0 2024-05-17T21:52:56.765 INFO:tasks.workunit.client.1.smithi138.stdout:5/390: unlink d1/d6/d1b/d29/c49 0 2024-05-17T21:52:56.767 INFO:tasks.workunit.client.1.smithi138.stdout:0/400: link d3/d10/c1c d3/d10/d39/c88 0 2024-05-17T21:52:56.769 INFO:tasks.workunit.client.1.smithi138.stdout:5/391: dread d1/d6/d1b/d29/f7c [0,4194304] 0 2024-05-17T21:52:56.770 INFO:tasks.workunit.client.1.smithi138.stdout:4/401: chown d0/d2f/d49/d60/d1f/d32/c4b 17984 1 2024-05-17T21:52:56.771 INFO:tasks.workunit.client.0.smithi107.stdout:5/545: dwrite d5/db/d16/d5b/f3f [0,4194304] 0 2024-05-17T21:52:56.772 INFO:tasks.workunit.client.1.smithi138.stdout:5/392: dread d1/d60/f64 [0,4194304] 0 2024-05-17T21:52:56.775 INFO:tasks.workunit.client.1.smithi138.stdout:6/411: rename d7/f37 to d7/d69/f90 0 2024-05-17T21:52:56.778 INFO:tasks.workunit.client.1.smithi138.stdout:7/310: mknod d0/d5c/c60 0 2024-05-17T21:52:56.780 INFO:tasks.workunit.client.0.smithi107.stdout:0/640: rename d3/dc/d31/l3c to d3/d24/d6f/le3 0 2024-05-17T21:52:56.784 INFO:tasks.workunit.client.1.smithi138.stdout:4/402: chown d0/d3/la 376 1 2024-05-17T21:52:56.785 INFO:tasks.workunit.client.0.smithi107.stdout:3/588: fdatasync d6/d8/d10/f38 0 2024-05-17T21:52:56.787 INFO:tasks.workunit.client.0.smithi107.stdout:0/641: dwrite d3/dc/dae/fbb [0,4194304] 0 2024-05-17T21:52:56.788 INFO:tasks.workunit.client.0.smithi107.stdout:8/554: symlink dc/d21/d27/d34/lca 0 2024-05-17T21:52:56.789 INFO:tasks.workunit.client.1.smithi138.stdout:2/417: sync 2024-05-17T21:52:56.789 INFO:tasks.workunit.client.1.smithi138.stdout:1/374: symlink d4/dd/l7a 0 2024-05-17T21:52:56.790 INFO:tasks.workunit.client.1.smithi138.stdout:1/375: write d4/f9 [4009169,25934] 0 2024-05-17T21:52:56.790 INFO:tasks.workunit.client.0.smithi107.stdout:1/612: creat d4/df/d14/fc9 x:0 0 0 2024-05-17T21:52:56.791 INFO:tasks.workunit.client.1.smithi138.stdout:9/348: symlink d6/l72 0 2024-05-17T21:52:56.793 INFO:tasks.workunit.client.1.smithi138.stdout:0/401: rename d3/d10/l1f to d3/d10/d6d/d81/l89 0 2024-05-17T21:52:56.795 INFO:tasks.workunit.client.1.smithi138.stdout:6/412: mknod d7/d8/d78/c91 0 2024-05-17T21:52:56.798 INFO:tasks.workunit.client.0.smithi107.stdout:2/414: creat d0/f7d x:0 0 0 2024-05-17T21:52:56.798 INFO:tasks.workunit.client.1.smithi138.stdout:9/349: read d6/d7/d16/d29/d48/d6e/f56 [1459372,90352] 0 2024-05-17T21:52:56.798 INFO:tasks.workunit.client.0.smithi107.stdout:0/642: dwrite d3/d24/d6f/d9f/dab/f7c [4194304,4194304] 0 2024-05-17T21:52:56.799 INFO:tasks.workunit.client.0.smithi107.stdout:4/551: read d3/d4a/f75 [2064807,105298] 0 2024-05-17T21:52:56.800 INFO:tasks.workunit.client.0.smithi107.stdout:0/643: read d3/dc/d22/d3d/f4d [3269192,23767] 0 2024-05-17T21:52:56.803 INFO:tasks.workunit.client.0.smithi107.stdout:8/555: mkdir dc/d44/d4b/d52/dcb 0 2024-05-17T21:52:56.803 INFO:tasks.workunit.client.0.smithi107.stdout:0/644: write d3/d24/d6f/d9f/dab/fb4 [455929,11795] 0 2024-05-17T21:52:56.803 INFO:tasks.workunit.client.1.smithi138.stdout:7/311: dwrite d0/d18/d40/d41/f43 [0,4194304] 0 2024-05-17T21:52:56.805 INFO:tasks.workunit.client.0.smithi107.stdout:8/556: chown dc/d21/d97/l5e 2172 1 2024-05-17T21:52:56.805 INFO:tasks.workunit.client.1.smithi138.stdout:5/393: mknod d1/d6/d1b/d29/c81 0 2024-05-17T21:52:56.806 INFO:tasks.workunit.client.0.smithi107.stdout:0/645: write d3/dc/d22/d23/d97/d85/fd9 [4800906,125800] 0 2024-05-17T21:52:56.807 INFO:tasks.workunit.client.1.smithi138.stdout:0/402: unlink l1 0 2024-05-17T21:52:56.808 INFO:tasks.workunit.client.0.smithi107.stdout:0/646: read d3/d39/fb0 [3309558,127870] 0 2024-05-17T21:52:56.808 INFO:tasks.workunit.client.0.smithi107.stdout:0/647: chown d3/d39 4682 1 2024-05-17T21:52:56.809 INFO:tasks.workunit.client.0.smithi107.stdout:2/415: unlink d0/d9/d11/d15/c16 0 2024-05-17T21:52:56.809 INFO:tasks.workunit.client.1.smithi138.stdout:6/413: mknod d7/db/d12/c92 0 2024-05-17T21:52:56.812 INFO:tasks.workunit.client.1.smithi138.stdout:9/350: rename d6/f24 to d6/d7/d16/d29/d38/f73 0 2024-05-17T21:52:56.813 INFO:tasks.workunit.client.1.smithi138.stdout:9/351: dread - d6/d7/d16/d29/d48/d3b/f63 zero size 2024-05-17T21:52:56.816 INFO:tasks.workunit.client.0.smithi107.stdout:1/613: symlink d4/da7/daf/lca 0 2024-05-17T21:52:56.818 INFO:tasks.workunit.client.1.smithi138.stdout:2/418: creat d3/d21/d2f/d3e/d58/d77/f86 x:0 0 0 2024-05-17T21:52:56.818 INFO:tasks.workunit.client.1.smithi138.stdout:8/298: truncate d0/dc/d1a/f3f 1142637 0 2024-05-17T21:52:56.820 INFO:tasks.workunit.client.0.smithi107.stdout:5/546: rmdir d5/db/d16/d54/d59 39 2024-05-17T21:52:56.822 INFO:tasks.workunit.client.0.smithi107.stdout:6/464: getdents d9/d58/d66 0 2024-05-17T21:52:56.823 INFO:tasks.workunit.client.1.smithi138.stdout:8/299: dwrite d0/dc/f2b [0,4194304] 0 2024-05-17T21:52:56.824 INFO:tasks.workunit.client.0.smithi107.stdout:6/465: write d9/d16/d22/d5f/d6e/f87 [790154,38748] 0 2024-05-17T21:52:56.824 INFO:tasks.workunit.client.1.smithi138.stdout:8/300: write d0/dc/f2b [1199700,113966] 0 2024-05-17T21:52:56.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:56 smithi138 ceph-mon[35948]: pgmap v36: 65 pgs: 65 active+clean; 1.5 GiB data, 5.7 GiB used, 531 GiB / 536 GiB avail; 44 MiB/s rd, 184 MiB/s wr, 335 op/s 2024-05-17T21:52:56.826 INFO:tasks.workunit.client.1.smithi138.stdout:8/301: write d0/fb [277776,116628] 0 2024-05-17T21:52:56.829 INFO:tasks.workunit.client.0.smithi107.stdout:0/648: symlink d3/dc/dae/dc0/le4 0 2024-05-17T21:52:56.831 INFO:tasks.workunit.client.1.smithi138.stdout:6/414: rmdir d7/db/d43/d5f 39 2024-05-17T21:52:56.836 INFO:tasks.workunit.client.0.smithi107.stdout:4/552: mknod d3/d4a/d47/d60/d3c/cb4 0 2024-05-17T21:52:56.836 INFO:tasks.workunit.client.0.smithi107.stdout:1/614: dread d4/df/d14/d1d/f27 [0,4194304] 0 2024-05-17T21:52:56.836 INFO:tasks.workunit.client.1.smithi138.stdout:4/403: rmdir d0/d3/d8f 0 2024-05-17T21:52:56.837 INFO:tasks.workunit.client.0.smithi107.stdout:1/615: chown d4/df/d4e/l96 89 1 2024-05-17T21:52:56.837 INFO:tasks.workunit.client.0.smithi107.stdout:1/616: chown d4/df/d14/d1d 1601 1 2024-05-17T21:52:56.839 INFO:tasks.workunit.client.0.smithi107.stdout:1/617: write d4/df/d1a/fbd [407668,55556] 0 2024-05-17T21:52:56.839 INFO:tasks.workunit.client.0.smithi107.stdout:3/589: rename d6/d2a/d2d/da8/la9 to d6/d2a/lbd 0 2024-05-17T21:52:56.839 INFO:tasks.workunit.client.0.smithi107.stdout:4/553: dread d3/db/f1f [0,4194304] 0 2024-05-17T21:52:56.843 INFO:tasks.workunit.client.1.smithi138.stdout:1/376: creat d4/db/f7b x:0 0 0 2024-05-17T21:52:56.845 INFO:tasks.workunit.client.1.smithi138.stdout:5/394: write d1/d53/f80 [638469,104017] 0 2024-05-17T21:52:56.846 INFO:tasks.workunit.client.1.smithi138.stdout:5/395: stat d1/d1a/d39/d4e/d58 0 2024-05-17T21:52:56.846 INFO:tasks.workunit.client.0.smithi107.stdout:1/618: dwrite d4/de/f36 [0,4194304] 0 2024-05-17T21:52:56.848 INFO:tasks.workunit.client.0.smithi107.stdout:8/557: creat dc/d21/d97/db7/fcc x:0 0 0 2024-05-17T21:52:56.849 INFO:tasks.workunit.client.1.smithi138.stdout:8/302: mknod d0/dc/d1a/d33/d4a/d4e/c5c 0 2024-05-17T21:52:56.851 INFO:tasks.workunit.client.0.smithi107.stdout:6/466: creat d9/d16/d22/f92 x:0 0 0 2024-05-17T21:52:56.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:56 smithi107 ceph-mon[28434]: pgmap v36: 65 pgs: 65 active+clean; 1.5 GiB data, 5.7 GiB used, 531 GiB / 536 GiB avail; 44 MiB/s rd, 184 MiB/s wr, 335 op/s 2024-05-17T21:52:56.852 INFO:tasks.workunit.client.1.smithi138.stdout:0/403: mknod d3/d10/c8a 0 2024-05-17T21:52:56.855 INFO:tasks.workunit.client.1.smithi138.stdout:6/415: creat d7/db/d43/d3f/d79/f93 x:0 0 0 2024-05-17T21:52:56.855 INFO:tasks.workunit.client.1.smithi138.stdout:3/464: sync 2024-05-17T21:52:56.858 INFO:tasks.workunit.client.1.smithi138.stdout:7/312: rename d0/f1f to d0/d18/d40/d41/d50/f61 0 2024-05-17T21:52:56.859 INFO:tasks.workunit.client.1.smithi138.stdout:4/404: fdatasync d0/d2f/d49/d60/d1f/f53 0 2024-05-17T21:52:56.861 INFO:tasks.workunit.client.0.smithi107.stdout:0/649: dwrite d3/d24/d6f/d9f/fbd [0,4194304] 0 2024-05-17T21:52:56.862 INFO:tasks.workunit.client.1.smithi138.stdout:8/303: unlink d0/d2f/f3c 0 2024-05-17T21:52:56.864 INFO:tasks.workunit.client.1.smithi138.stdout:8/304: write d0/d11/f42 [1031059,118728] 0 2024-05-17T21:52:56.865 INFO:tasks.workunit.client.1.smithi138.stdout:8/305: chown d0/dc/d1a/d33/d4a/d4e 0 1 2024-05-17T21:52:56.868 INFO:tasks.workunit.client.0.smithi107.stdout:5/547: rename d5/db/d16/d54/c78 to d5/db/d16/d54/d59/d92/ca9 0 2024-05-17T21:52:56.872 INFO:tasks.workunit.client.0.smithi107.stdout:8/558: mkdir dc/d21/d97/db7/dcd 0 2024-05-17T21:52:56.873 INFO:tasks.workunit.client.1.smithi138.stdout:7/313: fdatasync d0/f19 0 2024-05-17T21:52:56.875 INFO:tasks.workunit.client.0.smithi107.stdout:6/467: creat d9/d58/d66/f93 x:0 0 0 2024-05-17T21:52:56.877 INFO:tasks.workunit.client.1.smithi138.stdout:9/352: rename d6/d59 to d6/d7/d16/d29/d48/d6e/d74 0 2024-05-17T21:52:56.878 INFO:tasks.workunit.client.0.smithi107.stdout:2/416: link d0/d26/d64/l76 d0/d9/d11/d15/l7e 0 2024-05-17T21:52:56.879 INFO:tasks.workunit.client.0.smithi107.stdout:3/590: mkdir d6/d8/d3e/d3f/dbe 0 2024-05-17T21:52:56.883 INFO:tasks.workunit.client.1.smithi138.stdout:1/377: link d4/df/f2c d4/df/d3a/d3c/f7c 0 2024-05-17T21:52:56.883 INFO:tasks.workunit.client.1.smithi138.stdout:1/378: readlink d4/df/d35/l46 0 2024-05-17T21:52:56.885 INFO:tasks.workunit.client.0.smithi107.stdout:4/554: mkdir d3/db/d1e/db5 0 2024-05-17T21:52:56.886 INFO:tasks.workunit.client.0.smithi107.stdout:4/555: write d3/db/f69 [577296,109197] 0 2024-05-17T21:52:56.887 INFO:tasks.workunit.client.0.smithi107.stdout:5/548: mknod d5/db/d16/d2a/d73/caa 0 2024-05-17T21:52:56.889 INFO:tasks.workunit.client.0.smithi107.stdout:1/619: mknod d4/df/d1a/ccb 0 2024-05-17T21:52:56.890 INFO:tasks.workunit.client.1.smithi138.stdout:0/404: creat d3/d10/d38/d70/d80/f8b x:0 0 0 2024-05-17T21:52:56.890 INFO:tasks.workunit.client.1.smithi138.stdout:0/405: chown d3/d10 0 1 2024-05-17T21:52:56.890 INFO:tasks.workunit.client.0.smithi107.stdout:8/559: readlink dc/d21/d97/l57 0 2024-05-17T21:52:56.891 INFO:tasks.workunit.client.1.smithi138.stdout:8/306: symlink d0/dc/d1a/d44/l5d 0 2024-05-17T21:52:56.892 INFO:tasks.workunit.client.1.smithi138.stdout:8/307: truncate d0/dc/d1a/f4b 613735 0 2024-05-17T21:52:56.892 INFO:tasks.workunit.client.0.smithi107.stdout:7/533: sync 2024-05-17T21:52:56.892 INFO:tasks.workunit.client.0.smithi107.stdout:9/579: sync 2024-05-17T21:52:56.895 INFO:tasks.workunit.client.0.smithi107.stdout:3/591: creat d6/d8/d10/d35/d36/d66/fbf x:0 0 0 2024-05-17T21:52:56.895 INFO:tasks.workunit.client.0.smithi107.stdout:0/650: symlink d3/dc/d22/d23/d97/d85/dd4/le5 0 2024-05-17T21:52:56.897 INFO:tasks.workunit.client.0.smithi107.stdout:3/592: dread - d6/d8/d10/d25/f89 zero size 2024-05-17T21:52:56.897 INFO:tasks.workunit.client.0.smithi107.stdout:4/556: symlink d3/db/d35/d55/lb6 0 2024-05-17T21:52:56.898 INFO:tasks.workunit.client.1.smithi138.stdout:8/308: dwrite d0/d31/f5a [0,4194304] 0 2024-05-17T21:52:56.899 INFO:tasks.workunit.client.0.smithi107.stdout:0/651: read d3/d39/f69 [1156769,113191] 0 2024-05-17T21:52:56.901 INFO:tasks.workunit.client.1.smithi138.stdout:2/419: rename d3/d21/d2f/d3e/d63/d65/c68 to d3/d21/d2f/d3e/d58/c87 0 2024-05-17T21:52:56.902 INFO:tasks.workunit.client.1.smithi138.stdout:2/420: fsync d3/d21/f38 0 2024-05-17T21:52:56.902 INFO:tasks.workunit.client.0.smithi107.stdout:9/580: dwrite d0/fc [0,4194304] 0 2024-05-17T21:52:56.903 INFO:tasks.workunit.client.1.smithi138.stdout:2/421: write d3/f4 [1593118,53896] 0 2024-05-17T21:52:56.905 INFO:tasks.workunit.client.0.smithi107.stdout:7/534: dwrite d5/df/d11/d1c/d2f/d53/f90 [0,4194304] 0 2024-05-17T21:52:56.906 INFO:tasks.workunit.client.0.smithi107.stdout:1/620: chown d4/c49 20723 1 2024-05-17T21:52:56.907 INFO:tasks.workunit.client.0.smithi107.stdout:8/560: truncate dc/d21/d35/f22 2721412 0 2024-05-17T21:52:56.911 INFO:tasks.workunit.client.1.smithi138.stdout:3/465: truncate d1/d3/d16/f29 795327 0 2024-05-17T21:52:56.914 INFO:tasks.workunit.client.0.smithi107.stdout:9/581: write d0/dd/d21/d29/d3e/d45/d68/f6c [2557158,20576] 0 2024-05-17T21:52:56.914 INFO:tasks.workunit.client.0.smithi107.stdout:7/535: fdatasync d5/df/d11/d1c/d2f/d65/da7/fba 0 2024-05-17T21:52:56.917 INFO:tasks.workunit.client.0.smithi107.stdout:7/536: fsync d5/df/d11/f7a 0 2024-05-17T21:52:56.917 INFO:tasks.workunit.client.0.smithi107.stdout:5/549: dwrite d5/f1b [4194304,4194304] 0 2024-05-17T21:52:56.921 INFO:tasks.workunit.client.1.smithi138.stdout:7/314: mknod d0/d18/d40/d41/d51/c62 0 2024-05-17T21:52:56.921 INFO:tasks.workunit.client.0.smithi107.stdout:2/417: dwrite d0/fa [0,4194304] 0 2024-05-17T21:52:56.922 INFO:tasks.workunit.client.0.smithi107.stdout:2/418: chown d0/d9/d1c/d3e 0 1 2024-05-17T21:52:56.923 INFO:tasks.workunit.client.1.smithi138.stdout:7/315: write d0/d18/f45 [347676,39031] 0 2024-05-17T21:52:56.923 INFO:tasks.workunit.client.1.smithi138.stdout:8/309: symlink d0/dc/d1a/d33/l5e 0 2024-05-17T21:52:56.923 INFO:tasks.workunit.client.0.smithi107.stdout:7/537: truncate d5/df/d11/d1c/d31/d39/d92/d9a/fa1 351811 0 2024-05-17T21:52:56.924 INFO:tasks.workunit.client.1.smithi138.stdout:7/316: write d0/f20 [9298125,81974] 0 2024-05-17T21:52:56.925 INFO:tasks.workunit.client.1.smithi138.stdout:8/310: write d0/f1b [1523360,120701] 0 2024-05-17T21:52:56.926 INFO:tasks.workunit.client.1.smithi138.stdout:7/317: chown d0/d5c 287 1 2024-05-17T21:52:56.927 INFO:tasks.workunit.client.0.smithi107.stdout:1/621: creat d4/df/d4e/d59/fcc x:0 0 0 2024-05-17T21:52:56.932 INFO:tasks.workunit.client.0.smithi107.stdout:8/561: symlink dc/d44/lce 0 2024-05-17T21:52:56.935 INFO:tasks.workunit.client.1.smithi138.stdout:2/422: symlink d3/d21/d2f/d3e/d55/d73/l88 0 2024-05-17T21:52:56.936 INFO:tasks.workunit.client.0.smithi107.stdout:6/468: write f5 [2192481,4001] 0 2024-05-17T21:52:56.938 INFO:tasks.workunit.client.1.smithi138.stdout:9/353: unlink d6/d7/f30 0 2024-05-17T21:52:56.942 INFO:tasks.workunit.client.1.smithi138.stdout:4/405: creat d0/d2f/d49/f90 x:0 0 0 2024-05-17T21:52:56.944 INFO:tasks.workunit.client.1.smithi138.stdout:0/406: write d3/d3d/f78 [358750,52868] 0 2024-05-17T21:52:56.944 INFO:tasks.workunit.client.1.smithi138.stdout:4/406: chown d0/d1b/d36/c58 125822 1 2024-05-17T21:52:56.944 INFO:tasks.workunit.client.1.smithi138.stdout:1/379: dwrite d4/df/d3a/d3c/f7c [0,4194304] 0 2024-05-17T21:52:56.945 INFO:tasks.workunit.client.1.smithi138.stdout:0/407: chown d3/d3d/f7f 111864442 1 2024-05-17T21:52:56.946 INFO:tasks.workunit.client.1.smithi138.stdout:5/396: getdents d1/d1a/d39/d4e/d58/d5b 0 2024-05-17T21:52:56.951 INFO:tasks.workunit.client.0.smithi107.stdout:9/582: rename d0/dd/d2f/l63 to d0/dd/d1f/lc8 0 2024-05-17T21:52:56.953 INFO:tasks.workunit.client.0.smithi107.stdout:5/550: creat d5/db/d16/d2a/fab x:0 0 0 2024-05-17T21:52:56.953 INFO:tasks.workunit.client.1.smithi138.stdout:6/416: getdents d7/db/d43/d63 0 2024-05-17T21:52:56.954 INFO:tasks.workunit.client.0.smithi107.stdout:5/551: fdatasync d5/db/d16/f9d 0 2024-05-17T21:52:56.955 INFO:tasks.workunit.client.1.smithi138.stdout:6/417: truncate d7/d69/f8e 317280 0 2024-05-17T21:52:56.955 INFO:tasks.workunit.client.0.smithi107.stdout:5/552: chown d5/db/d16/d54/d59/d65/d48 27035340 1 2024-05-17T21:52:56.956 INFO:tasks.workunit.client.0.smithi107.stdout:4/557: mknod d3/db/d1e/db5/cb7 0 2024-05-17T21:52:56.957 INFO:tasks.workunit.client.0.smithi107.stdout:5/553: write d5/db/d16/d5b/f85 [3618503,89273] 0 2024-05-17T21:52:56.957 INFO:tasks.workunit.client.0.smithi107.stdout:4/558: readlink d3/db/d1e/l2a 0 2024-05-17T21:52:56.959 INFO:tasks.workunit.client.1.smithi138.stdout:7/318: dread d0/d39/f3d [0,4194304] 0 2024-05-17T21:52:56.962 INFO:tasks.workunit.client.1.smithi138.stdout:8/311: mknod d0/d31/c5f 0 2024-05-17T21:52:56.963 INFO:tasks.workunit.client.0.smithi107.stdout:1/622: rmdir d4/df 39 2024-05-17T21:52:56.964 INFO:tasks.workunit.client.0.smithi107.stdout:8/562: symlink dc/d21/d27/d34/d4c/lcf 0 2024-05-17T21:52:56.965 INFO:tasks.workunit.client.1.smithi138.stdout:9/354: fdatasync d6/d7/f9 0 2024-05-17T21:52:56.966 INFO:tasks.workunit.client.0.smithi107.stdout:7/538: dread d5/df/d11/d1c/d2f/d65/f69 [0,4194304] 0 2024-05-17T21:52:56.966 INFO:tasks.workunit.client.0.smithi107.stdout:6/469: symlink d9/d16/d4d/d85/l94 0 2024-05-17T21:52:56.967 INFO:tasks.workunit.client.1.smithi138.stdout:9/355: dread d6/f49 [0,4194304] 0 2024-05-17T21:52:56.969 INFO:tasks.workunit.client.0.smithi107.stdout:3/593: creat d6/d8/fc0 x:0 0 0 2024-05-17T21:52:56.971 INFO:tasks.workunit.client.1.smithi138.stdout:1/380: unlink d4/df/d35/l74 0 2024-05-17T21:52:56.971 INFO:tasks.workunit.client.1.smithi138.stdout:1/381: readlink d4/df/d35/l45 0 2024-05-17T21:52:56.972 INFO:tasks.workunit.client.1.smithi138.stdout:0/408: symlink d3/d3d/d48/l8c 0 2024-05-17T21:52:56.973 INFO:tasks.workunit.client.0.smithi107.stdout:0/652: truncate d3/d58/f71 4882572 0 2024-05-17T21:52:56.976 INFO:tasks.workunit.client.1.smithi138.stdout:3/466: creat d1/d3/d8/d5c/d9c/fa3 x:0 0 0 2024-05-17T21:52:56.982 INFO:tasks.workunit.client.0.smithi107.stdout:1/623: write d4/df/d1a/d26/d72/fb4 [1365177,34644] 0 2024-05-17T21:52:56.983 INFO:tasks.workunit.client.0.smithi107.stdout:1/624: chown d4/df/d1a/d41/c65 6 1 2024-05-17T21:52:56.984 INFO:tasks.workunit.client.1.smithi138.stdout:2/423: symlink d3/d21/d2f/d3e/d55/l89 0 2024-05-17T21:52:56.984 INFO:tasks.workunit.client.1.smithi138.stdout:4/407: dwrite d0/d3/f16 [0,4194304] 0 2024-05-17T21:52:56.985 INFO:tasks.workunit.client.0.smithi107.stdout:1/625: fsync d4/df/d1a/d26/f7c 0 2024-05-17T21:52:56.985 INFO:tasks.workunit.client.1.smithi138.stdout:4/408: stat d0 0 2024-05-17T21:52:56.985 INFO:tasks.workunit.client.0.smithi107.stdout:8/563: dread dc/d44/d4b/d52/f96 [0,4194304] 0 2024-05-17T21:52:56.986 INFO:tasks.workunit.client.1.smithi138.stdout:8/312: dread d0/dc/d1a/f58 [0,4194304] 0 2024-05-17T21:52:56.992 INFO:tasks.workunit.client.0.smithi107.stdout:6/470: symlink d9/d16/d4d/l95 0 2024-05-17T21:52:56.993 INFO:tasks.workunit.client.1.smithi138.stdout:8/313: dwrite d0/f2 [4194304,4194304] 0 2024-05-17T21:52:56.993 INFO:tasks.workunit.client.0.smithi107.stdout:6/471: chown d9/l40 78 1 2024-05-17T21:52:56.993 INFO:tasks.workunit.client.0.smithi107.stdout:6/472: dread - d9/d16/d4d/f59 zero size 2024-05-17T21:52:56.996 INFO:tasks.workunit.client.1.smithi138.stdout:9/356: mkdir d6/d7/d16/d29/d48/d6e/d74/d75 0 2024-05-17T21:52:56.999 INFO:tasks.workunit.client.0.smithi107.stdout:6/473: dwrite d9/d16/d4d/d67/f79 [0,4194304] 0 2024-05-17T21:52:56.999 INFO:tasks.workunit.client.1.smithi138.stdout:5/397: dwrite d1/d60/f64 [0,4194304] 0 2024-05-17T21:52:56.999 INFO:tasks.workunit.client.1.smithi138.stdout:0/409: mknod d3/d3d/c8d 0 2024-05-17T21:52:57.003 INFO:tasks.workunit.client.0.smithi107.stdout:7/539: write d5/d16/f2b [1149410,47163] 0 2024-05-17T21:52:57.004 INFO:tasks.workunit.client.0.smithi107.stdout:4/559: mknod d3/d4a/d47/d60/d3c/d9d/cb8 0 2024-05-17T21:52:57.006 INFO:tasks.workunit.client.1.smithi138.stdout:8/314: dwrite d0/f17 [4194304,4194304] 0 2024-05-17T21:52:57.007 INFO:tasks.workunit.client.1.smithi138.stdout:8/315: chown d0/d31 2 1 2024-05-17T21:52:57.008 INFO:tasks.workunit.client.1.smithi138.stdout:3/467: rename d1/d3/f82 to d1/d3/d8/d72/fa4 0 2024-05-17T21:52:57.013 INFO:tasks.workunit.client.0.smithi107.stdout:6/474: dread d9/f1c [0,4194304] 0 2024-05-17T21:52:57.014 INFO:tasks.workunit.client.0.smithi107.stdout:1/626: mknod d4/df/d1a/d26/d7a/ccd 0 2024-05-17T21:52:57.023 INFO:tasks.workunit.client.1.smithi138.stdout:2/424: mkdir d3/d21/d2f/d8a 0 2024-05-17T21:52:57.024 INFO:tasks.workunit.client.0.smithi107.stdout:4/560: truncate d3/db/d1e/d66/f70 1130359 0 2024-05-17T21:52:57.024 INFO:tasks.workunit.client.0.smithi107.stdout:2/419: link d0/d9/d11/c5 d0/d26/d64/c7f 0 2024-05-17T21:52:57.025 INFO:tasks.workunit.client.1.smithi138.stdout:9/357: mkdir d6/d7/d16/d29/d48/d6e/d74/d75/d76 0 2024-05-17T21:52:57.026 INFO:tasks.workunit.client.0.smithi107.stdout:6/475: mkdir d9/d16/d4d/d85/d96 0 2024-05-17T21:52:57.026 INFO:tasks.workunit.client.1.smithi138.stdout:9/358: chown d6/d7/d16/d29/d48/d6e/d5a 2 1 2024-05-17T21:52:57.027 INFO:tasks.workunit.client.0.smithi107.stdout:1/627: symlink d4/da7/lce 0 2024-05-17T21:52:57.028 INFO:tasks.workunit.client.0.smithi107.stdout:3/594: rename d6/d2a/d2d/d37/fab to d6/fc1 0 2024-05-17T21:52:57.030 INFO:tasks.workunit.client.0.smithi107.stdout:9/583: getdents d0/dd/d21/d29/d3e 0 2024-05-17T21:52:57.030 INFO:tasks.workunit.client.1.smithi138.stdout:7/319: link d0/cb d0/d5c/c63 0 2024-05-17T21:52:57.031 INFO:tasks.workunit.client.1.smithi138.stdout:7/320: truncate d0/f4b 439820 0 2024-05-17T21:52:57.031 INFO:tasks.workunit.client.0.smithi107.stdout:9/584: truncate d0/dd/d21/faa 204616 0 2024-05-17T21:52:57.034 INFO:tasks.workunit.client.1.smithi138.stdout:2/425: unlink d3/d21/d2f/d3e/f5a 0 2024-05-17T21:52:57.035 INFO:tasks.workunit.client.0.smithi107.stdout:7/540: symlink d5/df/d11/d1d/lc3 0 2024-05-17T21:52:57.037 INFO:tasks.workunit.client.1.smithi138.stdout:0/410: creat d3/f8e x:0 0 0 2024-05-17T21:52:57.039 INFO:tasks.workunit.client.1.smithi138.stdout:8/316: dread d0/f2c [4194304,4194304] 0 2024-05-17T21:52:57.040 INFO:tasks.workunit.client.0.smithi107.stdout:2/420: mknod d0/d9/d11/d27/c80 0 2024-05-17T21:52:57.042 INFO:tasks.workunit.client.0.smithi107.stdout:8/564: dread dc/d21/d35/f22 [0,4194304] 0 2024-05-17T21:52:57.042 INFO:tasks.workunit.client.1.smithi138.stdout:2/426: dwrite d3/d21/d2f/d3e/d63/d65/f79 [0,4194304] 0 2024-05-17T21:52:57.044 INFO:tasks.workunit.client.1.smithi138.stdout:4/409: rename d0/d1b/d42/c5d to d0/d2f/d49/c91 0 2024-05-17T21:52:57.044 INFO:tasks.workunit.client.0.smithi107.stdout:6/476: rmdir d9/d58 39 2024-05-17T21:52:57.046 INFO:tasks.workunit.client.1.smithi138.stdout:2/427: read d3/f10 [2747233,47981] 0 2024-05-17T21:52:57.052 INFO:tasks.workunit.client.0.smithi107.stdout:0/653: getdents d3/d58 0 2024-05-17T21:52:57.053 INFO:tasks.workunit.client.0.smithi107.stdout:9/585: fdatasync d0/f17 0 2024-05-17T21:52:57.054 INFO:tasks.workunit.client.0.smithi107.stdout:3/595: dread d6/d8/d10/d35/d36/f8e [0,4194304] 0 2024-05-17T21:52:57.055 INFO:tasks.workunit.client.0.smithi107.stdout:7/541: truncate d5/df/d11/f5c 1273477 0 2024-05-17T21:52:57.057 INFO:tasks.workunit.client.1.smithi138.stdout:0/411: truncate d3/d10/f62 65516 0 2024-05-17T21:52:57.059 INFO:tasks.workunit.client.0.smithi107.stdout:7/542: truncate d5/df/d11/d1c/d31/d39/f58 684161 0 2024-05-17T21:52:57.059 INFO:tasks.workunit.client.0.smithi107.stdout:9/586: dwrite d0/fc [0,4194304] 0 2024-05-17T21:52:57.060 INFO:tasks.workunit.client.1.smithi138.stdout:1/382: rename d4/df/c17 to d4/db/d29/d5d/c7d 0 2024-05-17T21:52:57.061 INFO:tasks.workunit.client.0.smithi107.stdout:2/421: creat d0/d9/d11/d15/f81 x:0 0 0 2024-05-17T21:52:57.062 INFO:tasks.workunit.client.1.smithi138.stdout:3/468: write d1/d3/d16/f29 [1299964,97652] 0 2024-05-17T21:52:57.067 INFO:tasks.workunit.client.0.smithi107.stdout:7/543: dwrite d5/df/d11/d1d/d33/d82/fa0 [0,4194304] 0 2024-05-17T21:52:57.068 INFO:tasks.workunit.client.1.smithi138.stdout:9/359: creat d6/d7/f77 x:0 0 0 2024-05-17T21:52:57.071 INFO:tasks.workunit.client.0.smithi107.stdout:0/654: mknod d3/d24/d6f/d9f/ce6 0 2024-05-17T21:52:57.072 INFO:tasks.workunit.client.1.smithi138.stdout:5/398: rename d1/d6/d1b/l76 to d1/d1a/d39/d62/l82 0 2024-05-17T21:52:57.073 INFO:tasks.workunit.client.0.smithi107.stdout:7/544: dwrite d5/d16/f2b [8388608,4194304] 0 2024-05-17T21:52:57.075 INFO:tasks.workunit.client.0.smithi107.stdout:7/545: chown d5/df/d11/d1c/d31/d56/f9c 1034121588 1 2024-05-17T21:52:57.075 INFO:tasks.workunit.client.1.smithi138.stdout:1/383: creat d4/df/d3a/f7e x:0 0 0 2024-05-17T21:52:57.077 INFO:tasks.workunit.client.0.smithi107.stdout:3/596: fsync d6/d8/d10/d25/f55 0 2024-05-17T21:52:57.078 INFO:tasks.workunit.client.1.smithi138.stdout:5/399: dread d1/f4f [0,4194304] 0 2024-05-17T21:52:57.082 INFO:tasks.workunit.client.0.smithi107.stdout:9/587: mkdir d0/d80/dc9 0 2024-05-17T21:52:57.084 INFO:tasks.workunit.client.0.smithi107.stdout:8/565: mknod dc/d44/d4b/d52/dcb/cd0 0 2024-05-17T21:52:57.084 INFO:tasks.workunit.client.0.smithi107.stdout:8/566: chown dc/d21/d97 3276 1 2024-05-17T21:52:57.085 INFO:tasks.workunit.client.0.smithi107.stdout:2/422: mkdir d0/d9/d11/df/d5a/d6d/d82 0 2024-05-17T21:52:57.087 INFO:tasks.workunit.client.1.smithi138.stdout:7/321: dwrite d0/d18/f57 [4194304,4194304] 0 2024-05-17T21:52:57.087 INFO:tasks.workunit.client.1.smithi138.stdout:0/412: mknod d3/c8f 0 2024-05-17T21:52:57.088 INFO:tasks.workunit.client.0.smithi107.stdout:6/477: creat d9/d81/f97 x:0 0 0 2024-05-17T21:52:57.091 INFO:tasks.workunit.client.1.smithi138.stdout:8/317: rename d0/dc/d1a/c23 to d0/dc/d1a/d33/c60 0 2024-05-17T21:52:57.093 INFO:tasks.workunit.client.0.smithi107.stdout:0/655: symlink d3/dc/d22/d23/d97/d85/dd4/le7 0 2024-05-17T21:52:57.093 INFO:tasks.workunit.client.1.smithi138.stdout:8/318: write d0/f2 [7031537,20715] 0 2024-05-17T21:52:57.094 INFO:tasks.workunit.client.1.smithi138.stdout:1/384: dread - d4/db/f39 zero size 2024-05-17T21:52:57.098 INFO:tasks.workunit.client.1.smithi138.stdout:5/400: symlink d1/d6/d1b/d29/d36/d68/l83 0 2024-05-17T21:52:57.100 INFO:tasks.workunit.client.1.smithi138.stdout:4/410: truncate d0/d3/d1d/d38/f50 283080 0 2024-05-17T21:52:57.101 INFO:tasks.workunit.client.0.smithi107.stdout:4/561: getdents d3/db/d35/d4f/d93 0 2024-05-17T21:52:57.101 INFO:tasks.workunit.client.1.smithi138.stdout:4/411: write d0/d2f/d49/d60/d1f/f70 [3470006,115671] 0 2024-05-17T21:52:57.101 INFO:tasks.workunit.client.0.smithi107.stdout:4/562: write d3/fa [8848555,67932] 0 2024-05-17T21:52:57.102 INFO:tasks.workunit.client.1.smithi138.stdout:3/469: write d1/d3/f10 [2369850,72704] 0 2024-05-17T21:52:57.103 INFO:tasks.workunit.client.0.smithi107.stdout:9/588: mkdir d0/dd/d21/d29/dca 0 2024-05-17T21:52:57.105 INFO:tasks.workunit.client.0.smithi107.stdout:8/567: chown dc/d21/d27/d2f/l54 49 1 2024-05-17T21:52:57.108 INFO:tasks.workunit.client.0.smithi107.stdout:4/563: dwrite d3/d4a/d47/d60/d3a/d41/d80/f44 [0,4194304] 0 2024-05-17T21:52:57.108 INFO:tasks.workunit.client.1.smithi138.stdout:2/428: rename d3/d21/d2f/d3e/d63/d65/f82 to d3/d21/d2f/d8a/f8b 0 2024-05-17T21:52:57.110 INFO:tasks.workunit.client.1.smithi138.stdout:8/319: mknod d0/dc/d3b/c61 0 2024-05-17T21:52:57.117 INFO:tasks.workunit.client.1.smithi138.stdout:1/385: dwrite d4/df/d3a/f4c [0,4194304] 0 2024-05-17T21:52:57.117 INFO:tasks.workunit.client.1.smithi138.stdout:1/386: stat d4/dd 0 2024-05-17T21:52:57.118 INFO:tasks.workunit.client.0.smithi107.stdout:9/589: creat d0/dd/d1b/fcb x:0 0 0 2024-05-17T21:52:57.119 INFO:tasks.workunit.client.0.smithi107.stdout:9/590: chown d0/dd/d21/d29/d57/c93 316646536 1 2024-05-17T21:52:57.123 INFO:tasks.workunit.client.1.smithi138.stdout:7/322: mkdir d0/d18/d64 0 2024-05-17T21:52:57.125 INFO:tasks.workunit.client.0.smithi107.stdout:8/568: rename dc/d21/d27/la2 to dc/d21/db8/ld1 0 2024-05-17T21:52:57.128 INFO:tasks.workunit.client.0.smithi107.stdout:7/546: write d5/f22 [5090305,71879] 0 2024-05-17T21:52:57.131 INFO:tasks.workunit.client.0.smithi107.stdout:3/597: link d6/d1a/l6e d6/d1a/d57/lc2 0 2024-05-17T21:52:57.131 INFO:tasks.workunit.client.0.smithi107.stdout:8/569: dwrite dc/d21/f92 [0,4194304] 0 2024-05-17T21:52:57.132 INFO:tasks.workunit.client.1.smithi138.stdout:8/320: mkdir d0/dc/d1a/d33/d4a/d62 0 2024-05-17T21:52:57.133 INFO:tasks.workunit.client.1.smithi138.stdout:3/470: write d1/d3/d16/d3e/f4e [1971718,1792] 0 2024-05-17T21:52:57.137 INFO:tasks.workunit.client.0.smithi107.stdout:6/478: rmdir d9/d16/d22/d5f 39 2024-05-17T21:52:57.140 INFO:tasks.workunit.client.0.smithi107.stdout:4/564: creat d3/d6/d26/d3b/d91/fb9 x:0 0 0 2024-05-17T21:52:57.142 INFO:tasks.workunit.client.0.smithi107.stdout:4/565: truncate d3/d4a/d47/d60/d3c/fb0 99186 0 2024-05-17T21:52:57.142 INFO:tasks.workunit.client.1.smithi138.stdout:9/360: getdents d6/d7/d16/d29/d38/d40/d44 0 2024-05-17T21:52:57.143 INFO:tasks.workunit.client.0.smithi107.stdout:9/591: mknod d0/dd/d3d/d9d/ccc 0 2024-05-17T21:52:57.144 INFO:tasks.workunit.client.0.smithi107.stdout:2/423: getdents d0/d9/d1c/d37 0 2024-05-17T21:52:57.146 INFO:tasks.workunit.client.0.smithi107.stdout:7/547: mkdir d5/df/d11/d1d/dc4 0 2024-05-17T21:52:57.147 INFO:tasks.workunit.client.1.smithi138.stdout:7/323: symlink d0/d18/l65 0 2024-05-17T21:52:57.149 INFO:tasks.workunit.client.1.smithi138.stdout:0/413: rename d3/c6c to d3/d10/c90 0 2024-05-17T21:52:57.151 INFO:tasks.workunit.client.1.smithi138.stdout:2/429: creat d3/f8c x:0 0 0 2024-05-17T21:52:57.151 INFO:tasks.workunit.client.0.smithi107.stdout:9/592: dwrite d0/dd/d21/d29/d3e/f5e [0,4194304] 0 2024-05-17T21:52:57.151 INFO:tasks.workunit.client.0.smithi107.stdout:6/479: dwrite d9/f12 [0,4194304] 0 2024-05-17T21:52:57.152 INFO:tasks.workunit.client.1.smithi138.stdout:2/430: chown d3/d21/d2f/d3e/d63/l78 2488 1 2024-05-17T21:52:57.152 INFO:tasks.workunit.client.0.smithi107.stdout:4/566: dread d3/d4a/d47/d60/d3a/d41/f71 [0,4194304] 0 2024-05-17T21:52:57.154 INFO:tasks.workunit.client.0.smithi107.stdout:4/567: write d3/d4a/d47/d60/d3a/d41/f51 [214041,33303] 0 2024-05-17T21:52:57.154 INFO:tasks.workunit.client.1.smithi138.stdout:1/387: rmdir d4/dd/d3b/d6b 0 2024-05-17T21:52:57.155 INFO:tasks.workunit.client.0.smithi107.stdout:9/593: dread d0/f43 [0,4194304] 0 2024-05-17T21:52:57.156 INFO:tasks.workunit.client.0.smithi107.stdout:9/594: dread - d0/dd/d21/d29/d3e/fc7 zero size 2024-05-17T21:52:57.157 INFO:tasks.workunit.client.1.smithi138.stdout:7/324: rmdir d0/d18/d40/d41/d51 39 2024-05-17T21:52:57.158 INFO:tasks.workunit.client.1.smithi138.stdout:7/325: write d0/f20 [6163525,109383] 0 2024-05-17T21:52:57.159 INFO:tasks.workunit.client.0.smithi107.stdout:8/570: creat dc/fd2 x:0 0 0 2024-05-17T21:52:57.163 INFO:tasks.workunit.client.0.smithi107.stdout:7/548: mkdir d5/df/d78/dc5 0 2024-05-17T21:52:57.165 INFO:tasks.workunit.client.1.smithi138.stdout:0/414: symlink d3/d3d/l91 0 2024-05-17T21:52:57.166 INFO:tasks.workunit.client.0.smithi107.stdout:3/598: link d6/d8/d10/d35/d36/d66/la0 d6/d8/d3e/lc3 0 2024-05-17T21:52:57.169 INFO:tasks.workunit.client.0.smithi107.stdout:6/480: creat d9/d16/d4d/d85/f98 x:0 0 0 2024-05-17T21:52:57.170 INFO:tasks.workunit.client.1.smithi138.stdout:3/471: link d1/d3/d16/d3a/f78 d1/fa5 0 2024-05-17T21:52:57.172 INFO:tasks.workunit.client.1.smithi138.stdout:2/431: rmdir d3/d21/d2f/d8a 39 2024-05-17T21:52:57.174 INFO:tasks.workunit.client.1.smithi138.stdout:1/388: stat d4/df/c79 0 2024-05-17T21:52:57.177 INFO:tasks.workunit.client.0.smithi107.stdout:8/571: rmdir dc/d21/d27/d34/d4c/db1 39 2024-05-17T21:52:57.181 INFO:tasks.workunit.client.0.smithi107.stdout:3/599: write d6/f1c [3511310,3564] 0 2024-05-17T21:52:57.186 INFO:tasks.workunit.client.0.smithi107.stdout:9/595: mkdir d0/dd/d21/dcd 0 2024-05-17T21:52:57.187 INFO:tasks.workunit.client.1.smithi138.stdout:5/401: rename d1/d6/d1b/f47 to d1/f84 0 2024-05-17T21:52:57.187 INFO:tasks.workunit.client.1.smithi138.stdout:5/402: stat d1/d6/d19 0 2024-05-17T21:52:57.188 INFO:tasks.workunit.client.0.smithi107.stdout:8/572: unlink dc/l24 0 2024-05-17T21:52:57.189 INFO:tasks.workunit.client.1.smithi138.stdout:0/415: symlink d3/l92 0 2024-05-17T21:52:57.189 INFO:tasks.workunit.client.1.smithi138.stdout:0/416: stat d3/d10/c26 0 2024-05-17T21:52:57.190 INFO:tasks.workunit.client.0.smithi107.stdout:3/600: dwrite d6/d2a/d2d/d48/f93 [4194304,4194304] 0 2024-05-17T21:52:57.190 INFO:tasks.workunit.client.0.smithi107.stdout:8/573: chown dc/f8b 668 1 2024-05-17T21:52:57.190 INFO:tasks.workunit.client.0.smithi107.stdout:2/424: dwrite d0/d9/d11/f52 [0,4194304] 0 2024-05-17T21:52:57.190 INFO:tasks.workunit.client.1.smithi138.stdout:3/472: creat d1/d3/d8d/d92/fa6 x:0 0 0 2024-05-17T21:52:57.191 INFO:tasks.workunit.client.1.smithi138.stdout:0/417: dread d3/d3d/f44 [4194304,4194304] 0 2024-05-17T21:52:57.194 INFO:tasks.workunit.client.1.smithi138.stdout:4/412: rename d0/d3/d1d/d38/l72 to d0/d3/d17/d5e/d7f/l92 0 2024-05-17T21:52:57.201 INFO:tasks.workunit.client.1.smithi138.stdout:0/418: dread d3/d3d/d51/f54 [4194304,4194304] 0 2024-05-17T21:52:57.201 INFO:tasks.workunit.client.1.smithi138.stdout:9/361: dwrite d6/f1b [0,4194304] 0 2024-05-17T21:52:57.203 INFO:tasks.workunit.client.1.smithi138.stdout:0/419: fdatasync d3/f2b 0 2024-05-17T21:52:57.205 INFO:tasks.workunit.client.1.smithi138.stdout:0/420: write d3/d10/f20 [188616,110655] 0 2024-05-17T21:52:57.205 INFO:tasks.workunit.client.1.smithi138.stdout:2/432: link d3/d21/c43 d3/d21/d2f/d3e/d58/d77/c8d 0 2024-05-17T21:52:57.206 INFO:tasks.workunit.client.0.smithi107.stdout:6/481: write d9/f15 [4164425,48399] 0 2024-05-17T21:52:57.206 INFO:tasks.workunit.client.1.smithi138.stdout:2/433: dread - d3/d21/d2f/d3e/d58/f6b zero size 2024-05-17T21:52:57.212 INFO:tasks.workunit.client.0.smithi107.stdout:3/601: truncate d6/d8/d10/f4b 1154099 0 2024-05-17T21:52:57.212 INFO:tasks.workunit.client.0.smithi107.stdout:3/602: chown d6/d8/d10/d35 1357868958 1 2024-05-17T21:52:57.213 INFO:tasks.workunit.client.0.smithi107.stdout:3/603: fdatasync d6/d2a/f52 0 2024-05-17T21:52:57.213 INFO:tasks.workunit.client.1.smithi138.stdout:4/413: creat d0/d2f/d49/d60/d1f/d7e/f93 x:0 0 0 2024-05-17T21:52:57.215 INFO:tasks.workunit.client.1.smithi138.stdout:6/418: sync 2024-05-17T21:52:57.215 INFO:tasks.workunit.client.0.smithi107.stdout:7/549: rmdir d5/df/d78/dc5 0 2024-05-17T21:52:57.218 INFO:tasks.workunit.client.0.smithi107.stdout:4/568: getdents d3/d4a 0 2024-05-17T21:52:57.221 INFO:tasks.workunit.client.0.smithi107.stdout:9/596: write d0/dd/d21/d29/d3e/d45/d68/d75/f7c [543762,55930] 0 2024-05-17T21:52:57.221 INFO:tasks.workunit.client.0.smithi107.stdout:3/604: dwrite d6/d8/d10/d35/d36/d66/d88/fbb [0,4194304] 0 2024-05-17T21:52:57.223 INFO:tasks.workunit.client.0.smithi107.stdout:6/482: symlink d9/d16/d4d/l99 0 2024-05-17T21:52:57.224 INFO:tasks.workunit.client.1.smithi138.stdout:3/473: creat d1/d4f/d80/fa7 x:0 0 0 2024-05-17T21:52:57.224 INFO:tasks.workunit.client.0.smithi107.stdout:3/605: write d6/d2a/d2d/d48/f93 [5830862,27250] 0 2024-05-17T21:52:57.225 INFO:tasks.workunit.client.1.smithi138.stdout:3/474: truncate d1/d3/d8/d5c/f9f 520138 0 2024-05-17T21:52:57.226 INFO:tasks.workunit.client.0.smithi107.stdout:7/550: mknod d5/df/d11/d1c/d31/d39/d92/d9a/cc6 0 2024-05-17T21:52:57.228 INFO:tasks.workunit.client.0.smithi107.stdout:3/606: chown d6/d8/d10/d35/d36/f8b 0 1 2024-05-17T21:52:57.230 INFO:tasks.workunit.client.0.smithi107.stdout:7/551: fsync d5/df/d78/fa2 0 2024-05-17T21:52:57.231 INFO:tasks.workunit.client.0.smithi107.stdout:8/574: creat dc/d21/fd3 x:0 0 0 2024-05-17T21:52:57.232 INFO:tasks.workunit.client.1.smithi138.stdout:2/434: mknod d3/d81/c8e 0 2024-05-17T21:52:57.232 INFO:tasks.workunit.client.0.smithi107.stdout:9/597: dwrite d0/dd/d1f/f40 [0,4194304] 0 2024-05-17T21:52:57.233 INFO:tasks.workunit.client.0.smithi107.stdout:2/425: dread d0/d9/d11/f10 [0,4194304] 0 2024-05-17T21:52:57.240 INFO:tasks.workunit.client.1.smithi138.stdout:4/414: mkdir d0/d3/db/d94 0 2024-05-17T21:52:57.241 INFO:tasks.workunit.client.1.smithi138.stdout:9/362: write d6/d7/f10 [168981,25324] 0 2024-05-17T21:52:57.242 INFO:tasks.workunit.client.1.smithi138.stdout:6/419: readlink d7/db/d34/l5c 0 2024-05-17T21:52:57.242 INFO:tasks.workunit.client.1.smithi138.stdout:4/415: chown d0/d2f/d49/c54 0 1 2024-05-17T21:52:57.244 INFO:tasks.workunit.client.1.smithi138.stdout:5/403: link d1/d1a/d39/d4e/d58/c7d d1/d2c/d63/c85 0 2024-05-17T21:52:57.247 INFO:tasks.workunit.client.1.smithi138.stdout:9/363: dread d6/f1b [0,4194304] 0 2024-05-17T21:52:57.248 INFO:tasks.workunit.client.1.smithi138.stdout:0/421: write d3/d10/d38/d3e/f67 [1820558,112038] 0 2024-05-17T21:52:57.250 INFO:tasks.workunit.client.0.smithi107.stdout:2/426: mknod d0/d9/d11/df/d5a/c83 0 2024-05-17T21:52:57.251 INFO:tasks.workunit.client.1.smithi138.stdout:5/404: dwrite d1/d6/d1b/f54 [0,4194304] 0 2024-05-17T21:52:57.251 INFO:tasks.workunit.client.1.smithi138.stdout:3/475: rmdir d1/d4f/d80 39 2024-05-17T21:52:57.252 INFO:tasks.workunit.client.1.smithi138.stdout:5/405: readlink d1/d6/d1b/l70 0 2024-05-17T21:52:57.252 INFO:tasks.workunit.client.0.smithi107.stdout:3/607: dread d6/d8/d10/d25/f55 [0,4194304] 0 2024-05-17T21:52:57.254 INFO:tasks.workunit.client.0.smithi107.stdout:2/427: chown d0/d9/d11/df/f47 126752552 1 2024-05-17T21:52:57.254 INFO:tasks.workunit.client.0.smithi107.stdout:4/569: dwrite d3/db/f33 [4194304,4194304] 0 2024-05-17T21:52:57.255 INFO:tasks.workunit.client.1.smithi138.stdout:2/435: creat d3/d21/d2f/d3e/d63/d7f/f8f x:0 0 0 2024-05-17T21:52:57.255 INFO:tasks.workunit.client.0.smithi107.stdout:8/575: dread dc/d21/d69/f77 [0,4194304] 0 2024-05-17T21:52:57.255 INFO:tasks.workunit.client.1.smithi138.stdout:2/436: readlink d3/d21/l3d 0 2024-05-17T21:52:57.256 INFO:tasks.workunit.client.0.smithi107.stdout:4/570: chown d3/db/f7b 622429 1 2024-05-17T21:52:57.258 INFO:tasks.workunit.client.1.smithi138.stdout:8/321: rename d0/dc/d1a/d33/d4a/d62 to d0/d63 0 2024-05-17T21:52:57.262 INFO:tasks.workunit.client.0.smithi107.stdout:6/483: creat d9/d16/d22/d2d/f9a x:0 0 0 2024-05-17T21:52:57.264 INFO:tasks.workunit.client.0.smithi107.stdout:7/552: dwrite d5/d16/f28 [4194304,4194304] 0 2024-05-17T21:52:57.264 INFO:tasks.workunit.client.0.smithi107.stdout:3/608: symlink d6/d8/d3e/d3f/d90/lc4 0 2024-05-17T21:52:57.267 INFO:tasks.workunit.client.1.smithi138.stdout:8/322: dwrite d0/d11/f42 [0,4194304] 0 2024-05-17T21:52:57.268 INFO:tasks.workunit.client.0.smithi107.stdout:2/428: chown d0/c3 38555 1 2024-05-17T21:52:57.270 INFO:tasks.workunit.client.1.smithi138.stdout:8/323: truncate d0/d2f/f45 206158 0 2024-05-17T21:52:57.271 INFO:tasks.workunit.client.1.smithi138.stdout:5/406: mknod d1/d1a/d39/d4e/d58/d5b/d65/c86 0 2024-05-17T21:52:57.272 INFO:tasks.workunit.client.1.smithi138.stdout:5/407: fsync d1/d2c/f35 0 2024-05-17T21:52:57.279 INFO:tasks.workunit.client.1.smithi138.stdout:2/437: fsync d3/f10 0 2024-05-17T21:52:57.280 INFO:tasks.workunit.client.1.smithi138.stdout:4/416: dread d0/f7 [0,4194304] 0 2024-05-17T21:52:57.281 INFO:tasks.workunit.client.1.smithi138.stdout:1/389: rename d4/df/d3a/c41 to d4/dd/d3b/d40/c7f 0 2024-05-17T21:52:57.284 INFO:tasks.workunit.client.1.smithi138.stdout:4/417: write d0/d1b/d2b/f65 [837096,93829] 0 2024-05-17T21:52:57.284 INFO:tasks.workunit.client.1.smithi138.stdout:5/408: dwrite d1/d53/f6e [0,4194304] 0 2024-05-17T21:52:57.285 INFO:tasks.workunit.client.0.smithi107.stdout:7/553: write d5/df/d11/d1c/d31/d39/f85 [1737807,64726] 0 2024-05-17T21:52:57.286 INFO:tasks.workunit.client.1.smithi138.stdout:6/420: mkdir d7/db/d43/d5f/d94 0 2024-05-17T21:52:57.287 INFO:tasks.workunit.client.0.smithi107.stdout:9/598: getdents d0/dd/d69 0 2024-05-17T21:52:57.288 INFO:tasks.workunit.client.0.smithi107.stdout:1/628: sync 2024-05-17T21:52:57.288 INFO:tasks.workunit.client.0.smithi107.stdout:5/554: sync 2024-05-17T21:52:57.292 INFO:tasks.workunit.client.1.smithi138.stdout:7/326: rename d0/d5c to d0/d18/d40/d41/d51/d66 0 2024-05-17T21:52:57.293 INFO:tasks.workunit.client.1.smithi138.stdout:7/327: write d0/f1e [5023993,96529] 0 2024-05-17T21:52:57.294 INFO:tasks.workunit.client.0.smithi107.stdout:3/609: unlink d6/d8/d3e/lc3 0 2024-05-17T21:52:57.295 INFO:tasks.workunit.client.1.smithi138.stdout:1/390: mknod d4/df/d4e/c80 0 2024-05-17T21:52:57.296 INFO:tasks.workunit.client.1.smithi138.stdout:1/391: dread - d4/dd/d3b/f53 zero size 2024-05-17T21:52:57.297 INFO:tasks.workunit.client.1.smithi138.stdout:5/409: creat d1/d60/f87 x:0 0 0 2024-05-17T21:52:57.298 INFO:tasks.workunit.client.1.smithi138.stdout:4/418: mkdir d0/d2f/d49/d60/d1f/d32/d71/d95 0 2024-05-17T21:52:57.299 INFO:tasks.workunit.client.0.smithi107.stdout:9/599: symlink d0/d80/lce 0 2024-05-17T21:52:57.300 INFO:tasks.workunit.client.1.smithi138.stdout:6/421: symlink d7/db/d43/d63/l95 0 2024-05-17T21:52:57.302 INFO:tasks.workunit.client.1.smithi138.stdout:8/324: unlink d0/c32 0 2024-05-17T21:52:57.305 INFO:tasks.workunit.client.1.smithi138.stdout:0/422: rename d3/d3d/d51/f55 to d3/d10/d38/d70/d80/f93 0 2024-05-17T21:52:57.308 INFO:tasks.workunit.client.0.smithi107.stdout:3/610: creat d6/d2a/d2d/d69/fc5 x:0 0 0 2024-05-17T21:52:57.308 INFO:tasks.workunit.client.1.smithi138.stdout:1/392: creat d4/df/d3a/f81 x:0 0 0 2024-05-17T21:52:57.312 INFO:tasks.workunit.client.0.smithi107.stdout:8/576: write dc/f67 [858265,10573] 0 2024-05-17T21:52:57.316 INFO:tasks.workunit.client.0.smithi107.stdout:9/600: creat d0/d80/fcf x:0 0 0 2024-05-17T21:52:57.317 INFO:tasks.workunit.client.1.smithi138.stdout:9/364: dwrite d6/d7/d16/d29/d48/f23 [4194304,4194304] 0 2024-05-17T21:52:57.318 INFO:tasks.workunit.client.0.smithi107.stdout:4/571: write d3/db/f4d [469553,106097] 0 2024-05-17T21:52:57.319 INFO:tasks.workunit.client.1.smithi138.stdout:6/422: creat d7/d8/f96 x:0 0 0 2024-05-17T21:52:57.320 INFO:tasks.workunit.client.1.smithi138.stdout:1/393: dwrite d4/db/d29/d5d/f6f [0,4194304] 0 2024-05-17T21:52:57.322 INFO:tasks.workunit.client.1.smithi138.stdout:5/410: dread d1/d6/d1b/f1d [0,4194304] 0 2024-05-17T21:52:57.323 INFO:tasks.workunit.client.1.smithi138.stdout:1/394: readlink d4/d2e/d36/l71 0 2024-05-17T21:52:57.330 INFO:tasks.workunit.client.1.smithi138.stdout:0/423: truncate d3/fc 4651222 0 2024-05-17T21:52:57.330 INFO:tasks.workunit.client.0.smithi107.stdout:6/484: dwrite d9/d16/f1e [0,4194304] 0 2024-05-17T21:52:57.333 INFO:tasks.workunit.client.0.smithi107.stdout:3/611: dread d6/d8/f17 [0,4194304] 0 2024-05-17T21:52:57.334 INFO:tasks.workunit.client.0.smithi107.stdout:2/429: getdents d0/d9/d11/df 0 2024-05-17T21:52:57.335 INFO:tasks.workunit.client.0.smithi107.stdout:0/656: sync 2024-05-17T21:52:57.339 INFO:tasks.workunit.client.0.smithi107.stdout:9/601: unlink d0/dd/d1f/f52 0 2024-05-17T21:52:57.340 INFO:tasks.workunit.client.0.smithi107.stdout:8/577: dread dc/d21/d27/d34/d4c/d7e/f45 [0,4194304] 0 2024-05-17T21:52:57.342 INFO:tasks.workunit.client.0.smithi107.stdout:4/572: rename d3/d6/d26/l29 to d3/db/d35/lba 0 2024-05-17T21:52:57.343 INFO:tasks.workunit.client.1.smithi138.stdout:4/419: creat d0/d1b/d2b/d55/d79/f96 x:0 0 0 2024-05-17T21:52:57.344 INFO:tasks.workunit.client.0.smithi107.stdout:1/629: creat d4/df/d1a/d26/fcf x:0 0 0 2024-05-17T21:52:57.345 INFO:tasks.workunit.client.1.smithi138.stdout:9/365: chown d6/c14 182008 1 2024-05-17T21:52:57.346 INFO:tasks.workunit.client.0.smithi107.stdout:5/555: link d5/db/d16/d5b/f2f d5/db/d16/d54/d59/d65/fac 0 2024-05-17T21:52:57.351 INFO:tasks.workunit.client.1.smithi138.stdout:5/411: mkdir d1/d53/d88 0 2024-05-17T21:52:57.352 INFO:tasks.workunit.client.1.smithi138.stdout:5/412: readlink d1/d6/l17 0 2024-05-17T21:52:57.352 INFO:tasks.workunit.client.0.smithi107.stdout:7/554: getdents d5/df/d11/d1c/d31/d56 0 2024-05-17T21:52:57.352 INFO:tasks.workunit.client.0.smithi107.stdout:7/555: stat d5/df/d78/fa2 0 2024-05-17T21:52:57.353 INFO:tasks.workunit.client.1.smithi138.stdout:7/328: dwrite d0/f1a [0,4194304] 0 2024-05-17T21:52:57.354 INFO:tasks.workunit.client.1.smithi138.stdout:7/329: chown d0/d18/d40/d41/d51/d66 2 1 2024-05-17T21:52:57.356 INFO:tasks.workunit.client.1.smithi138.stdout:3/476: rename d1/d3/d16/d2d/c61 to d1/d3/d8/d5c/ca8 0 2024-05-17T21:52:57.358 INFO:tasks.workunit.client.0.smithi107.stdout:3/612: mkdir d6/d2a/d2d/d37/dc6 0 2024-05-17T21:52:57.360 INFO:tasks.workunit.client.0.smithi107.stdout:2/430: creat d0/d9/d1c/d3e/d4f/f84 x:0 0 0 2024-05-17T21:52:57.362 INFO:tasks.workunit.client.1.smithi138.stdout:4/420: unlink d0/d3/d1d/d4c/f8d 0 2024-05-17T21:52:57.363 INFO:tasks.workunit.client.0.smithi107.stdout:0/657: rmdir d3/dc/dae/dc0 39 2024-05-17T21:52:57.365 INFO:tasks.workunit.client.1.smithi138.stdout:6/423: dwrite d7/d8/f2b [0,4194304] 0 2024-05-17T21:52:57.365 INFO:tasks.workunit.client.1.smithi138.stdout:9/366: creat d6/d7/d16/d29/d38/d40/d44/f78 x:0 0 0 2024-05-17T21:52:57.365 INFO:tasks.workunit.client.0.smithi107.stdout:7/556: dread d5/df/d11/d1c/f2e [0,4194304] 0 2024-05-17T21:52:57.366 INFO:tasks.workunit.client.1.smithi138.stdout:0/424: dwrite d3/f7 [0,4194304] 0 2024-05-17T21:52:57.366 INFO:tasks.workunit.client.0.smithi107.stdout:9/602: chown d0/dd/f66 1927241 1 2024-05-17T21:52:57.368 INFO:tasks.workunit.client.0.smithi107.stdout:8/578: unlink dc/cac 0 2024-05-17T21:52:57.369 INFO:tasks.workunit.client.1.smithi138.stdout:0/425: write d3/d10/f20 [5804798,22485] 0 2024-05-17T21:52:57.372 INFO:tasks.workunit.client.0.smithi107.stdout:4/573: creat d3/d6/d26/d36/d8b/fbb x:0 0 0 2024-05-17T21:52:57.374 INFO:tasks.workunit.client.1.smithi138.stdout:4/421: dwrite d0/d3/d17/d21/f75 [0,4194304] 0 2024-05-17T21:52:57.376 INFO:tasks.workunit.client.0.smithi107.stdout:4/574: read d3/d4a/d47/f4b [1157791,103326] 0 2024-05-17T21:52:57.376 INFO:tasks.workunit.client.1.smithi138.stdout:1/395: symlink d4/dd/l82 0 2024-05-17T21:52:57.376 INFO:tasks.workunit.client.1.smithi138.stdout:4/422: readlink d0/d3/db/ld 0 2024-05-17T21:52:57.377 INFO:tasks.workunit.client.1.smithi138.stdout:1/396: chown d4/dd/d3b/d40/c77 384783 1 2024-05-17T21:52:57.378 INFO:tasks.workunit.client.1.smithi138.stdout:1/397: dread - d4/db/f7b zero size 2024-05-17T21:52:57.382 INFO:tasks.workunit.client.1.smithi138.stdout:7/330: read d0/f11 [498767,42010] 0 2024-05-17T21:52:57.383 INFO:tasks.workunit.client.1.smithi138.stdout:2/438: rename d3/d21/d2f/d3e/d55/c6a to d3/d21/d2f/d3e/d63/c90 0 2024-05-17T21:52:57.383 INFO:tasks.workunit.client.0.smithi107.stdout:4/575: dwrite d3/d6/d26/d36/d7e/f89 [0,4194304] 0 2024-05-17T21:52:57.385 INFO:tasks.workunit.client.0.smithi107.stdout:6/485: creat d9/d58/f9b x:0 0 0 2024-05-17T21:52:57.390 INFO:tasks.workunit.client.0.smithi107.stdout:0/658: mkdir d3/d24/d6f/d9f/de8 0 2024-05-17T21:52:57.393 INFO:tasks.workunit.client.1.smithi138.stdout:6/424: unlink d7/db/d43/f39 0 2024-05-17T21:52:57.394 INFO:tasks.workunit.client.0.smithi107.stdout:9/603: rename d0/dd/d1f/f5c to d0/d80/fd0 0 2024-05-17T21:52:57.396 INFO:tasks.workunit.client.1.smithi138.stdout:8/325: getdents d0 0 2024-05-17T21:52:57.398 INFO:tasks.workunit.client.1.smithi138.stdout:8/326: write d0/dc/d1a/f2a [558983,27693] 0 2024-05-17T21:52:57.399 INFO:tasks.workunit.client.0.smithi107.stdout:1/630: mknod d4/df/d1a/cd0 0 2024-05-17T21:52:57.400 INFO:tasks.workunit.client.0.smithi107.stdout:1/631: chown d4/df/d1a/cd0 6974 1 2024-05-17T21:52:57.400 INFO:tasks.workunit.client.1.smithi138.stdout:0/426: rmdir d3/d3d/d48 39 2024-05-17T21:52:57.403 INFO:tasks.workunit.client.1.smithi138.stdout:1/398: symlink d4/df/d3a/d3c/l83 0 2024-05-17T21:52:57.406 INFO:tasks.workunit.client.1.smithi138.stdout:3/477: rename d1/d3/d16/d3a/f78 to d1/d3/d16/d3e/d71/fa9 0 2024-05-17T21:52:57.410 INFO:tasks.workunit.client.1.smithi138.stdout:2/439: rmdir d3/d21/d2f/d3e/d55 39 2024-05-17T21:52:57.410 INFO:tasks.workunit.client.1.smithi138.stdout:1/399: dwrite d4/df/f2c [0,4194304] 0 2024-05-17T21:52:57.412 INFO:tasks.workunit.client.1.smithi138.stdout:9/367: unlink d6/l70 0 2024-05-17T21:52:57.413 INFO:tasks.workunit.client.0.smithi107.stdout:2/431: mknod d0/d9/d11/c85 0 2024-05-17T21:52:57.414 INFO:tasks.workunit.client.1.smithi138.stdout:6/425: mknod d7/db/d43/d63/c97 0 2024-05-17T21:52:57.414 INFO:tasks.workunit.client.1.smithi138.stdout:2/440: dwrite d3/f4 [4194304,4194304] 0 2024-05-17T21:52:57.419 INFO:tasks.workunit.client.1.smithi138.stdout:8/327: mknod d0/dc/d1a/d44/c64 0 2024-05-17T21:52:57.420 INFO:tasks.workunit.client.0.smithi107.stdout:3/613: truncate d6/d8/f14 3568456 0 2024-05-17T21:52:57.421 INFO:tasks.workunit.client.1.smithi138.stdout:0/427: stat d3/c1d 0 2024-05-17T21:52:57.425 INFO:tasks.workunit.client.1.smithi138.stdout:8/328: dread d0/dc/d1a/f2a [0,4194304] 0 2024-05-17T21:52:57.430 INFO:tasks.workunit.client.1.smithi138.stdout:7/331: symlink d0/d18/d64/l67 0 2024-05-17T21:52:57.431 INFO:tasks.workunit.client.1.smithi138.stdout:3/478: rmdir d1/d3/d49 39 2024-05-17T21:52:57.436 INFO:tasks.workunit.client.1.smithi138.stdout:1/400: dread d4/db/f2a [0,4194304] 0 2024-05-17T21:52:57.439 INFO:tasks.workunit.client.1.smithi138.stdout:6/426: rmdir d7 39 2024-05-17T21:52:57.441 INFO:tasks.workunit.client.1.smithi138.stdout:5/413: getdents d1/d2c 0 2024-05-17T21:52:57.442 INFO:tasks.workunit.client.1.smithi138.stdout:8/329: rename d0/c56 to d0/dc/d1a/d51/c65 0 2024-05-17T21:52:57.442 INFO:tasks.workunit.client.1.smithi138.stdout:5/414: readlink d1/d6/l17 0 2024-05-17T21:52:57.442 INFO:tasks.workunit.client.1.smithi138.stdout:7/332: rename d0/d18/d40 to d0/d18/d40/d41/d51/d68 22 2024-05-17T21:52:57.443 INFO:tasks.workunit.client.1.smithi138.stdout:7/333: write d0/f1a [4102381,107540] 0 2024-05-17T21:52:57.445 INFO:tasks.workunit.client.1.smithi138.stdout:9/368: dread d6/d7/d16/d29/f3a [0,4194304] 0 2024-05-17T21:52:57.448 INFO:tasks.workunit.client.1.smithi138.stdout:2/441: creat d3/d21/d2f/d3e/d63/d7f/d84/f91 x:0 0 0 2024-05-17T21:52:57.448 INFO:tasks.workunit.client.1.smithi138.stdout:4/423: truncate d0/d3/d17/d21/f75 3709989 0 2024-05-17T21:52:57.451 INFO:tasks.workunit.client.1.smithi138.stdout:4/424: write d0/d3/d17/d5e/f8e [404317,88983] 0 2024-05-17T21:52:57.451 INFO:tasks.workunit.client.1.smithi138.stdout:3/479: dwrite d1/d3/d49/f7d [4194304,4194304] 0 2024-05-17T21:52:57.453 INFO:tasks.workunit.client.1.smithi138.stdout:4/425: stat d0/d3/d17/d5e 0 2024-05-17T21:52:57.453 INFO:tasks.workunit.client.1.smithi138.stdout:6/427: read d7/db/d43/d3f/f5a [224173,112310] 0 2024-05-17T21:52:57.453 INFO:tasks.workunit.client.0.smithi107.stdout:9/604: symlink d0/ld1 0 2024-05-17T21:52:57.454 INFO:tasks.workunit.client.0.smithi107.stdout:7/557: link d5/df/d11/d1c/d2f/f9f d5/d16/d3b/d45/d91/fc7 0 2024-05-17T21:52:57.454 INFO:tasks.workunit.client.0.smithi107.stdout:5/556: rmdir d5/db/dc/d1e/d6e/d93 0 2024-05-17T21:52:57.454 INFO:tasks.workunit.client.0.smithi107.stdout:3/614: write d6/d8/d10/d35/d36/d66/d88/fbb [2843187,67486] 0 2024-05-17T21:52:57.455 INFO:tasks.workunit.client.0.smithi107.stdout:1/632: chown d4/l20 4279661 1 2024-05-17T21:52:57.455 INFO:tasks.workunit.client.0.smithi107.stdout:4/576: creat d3/db/d35/d55/d8a/fbc x:0 0 0 2024-05-17T21:52:57.455 INFO:tasks.workunit.client.0.smithi107.stdout:2/432: truncate d0/d9/d1c/f73 1317894 0 2024-05-17T21:52:57.459 INFO:tasks.workunit.client.1.smithi138.stdout:1/401: rename d4/dd/f13 to d4/df/d4e/f84 0 2024-05-17T21:52:57.459 INFO:tasks.workunit.client.0.smithi107.stdout:1/633: dread - d4/df/d1a/d26/d72/fbe zero size 2024-05-17T21:52:57.461 INFO:tasks.workunit.client.1.smithi138.stdout:3/480: dwrite d1/d3/d16/d3e/f43 [0,4194304] 0 2024-05-17T21:52:57.466 INFO:tasks.workunit.client.0.smithi107.stdout:0/659: mkdir d3/dc/dae/dc0/de1/de9 0 2024-05-17T21:52:57.469 INFO:tasks.workunit.client.1.smithi138.stdout:7/334: rmdir d0/d18 39 2024-05-17T21:52:57.472 INFO:tasks.workunit.client.0.smithi107.stdout:8/579: rename dc/d21/d27/d2f/f59 to dc/d21/d27/d34/fd4 0 2024-05-17T21:52:57.473 INFO:tasks.workunit.client.1.smithi138.stdout:9/369: symlink d6/d7/d16/l79 0 2024-05-17T21:52:57.473 INFO:tasks.workunit.client.0.smithi107.stdout:4/577: symlink d3/db/d35/lbd 0 2024-05-17T21:52:57.476 INFO:tasks.workunit.client.1.smithi138.stdout:7/335: dwrite d0/f3b [0,4194304] 0 2024-05-17T21:52:57.482 INFO:tasks.workunit.client.0.smithi107.stdout:7/558: read d5/df/d11/d1d/d44/f4b [4153733,62037] 0 2024-05-17T21:52:57.487 INFO:tasks.workunit.client.0.smithi107.stdout:9/605: write d0/dd/d21/d29/d3e/fc4 [251728,49091] 0 2024-05-17T21:52:57.488 INFO:tasks.workunit.client.1.smithi138.stdout:1/402: rename d4/dd/f20 to d4/df/d3a/d3c/d67/f85 0 2024-05-17T21:52:57.489 INFO:tasks.workunit.client.0.smithi107.stdout:1/634: read d4/df/d1a/f7b [1377099,93766] 0 2024-05-17T21:52:57.490 INFO:tasks.workunit.client.0.smithi107.stdout:1/635: fsync d4/df/d1a/d26/d72/fb4 0 2024-05-17T21:52:57.490 INFO:tasks.workunit.client.0.smithi107.stdout:1/636: chown d4/df/d1a/d41/db2 28939 1 2024-05-17T21:52:57.491 INFO:tasks.workunit.client.1.smithi138.stdout:8/330: chown d0/dc/d1a/f1d 29 1 2024-05-17T21:52:57.492 INFO:tasks.workunit.client.0.smithi107.stdout:4/578: symlink d3/d6/d26/d36/d7e/lbe 0 2024-05-17T21:52:57.493 INFO:tasks.workunit.client.0.smithi107.stdout:3/615: getdents d6/d8/db6 0 2024-05-17T21:52:57.494 INFO:tasks.workunit.client.1.smithi138.stdout:8/331: dread d0/dc/f10 [4194304,4194304] 0 2024-05-17T21:52:57.494 INFO:tasks.workunit.client.1.smithi138.stdout:2/442: unlink d3/d21/d2f/d3e/d58/c87 0 2024-05-17T21:52:57.497 INFO:tasks.workunit.client.1.smithi138.stdout:0/428: getdents d3/d3d/d51 0 2024-05-17T21:52:57.498 INFO:tasks.workunit.client.0.smithi107.stdout:4/579: dwrite d3/d4a/d47/f8d [0,4194304] 0 2024-05-17T21:52:57.499 INFO:tasks.workunit.client.1.smithi138.stdout:8/332: dread d0/dc/f2b [0,4194304] 0 2024-05-17T21:52:57.499 INFO:tasks.workunit.client.0.smithi107.stdout:2/433: symlink d0/d9/d11/df/d5a/d6d/d82/l86 0 2024-05-17T21:52:57.501 INFO:tasks.workunit.client.0.smithi107.stdout:7/559: creat d5/d16/d3b/d45/fc8 x:0 0 0 2024-05-17T21:52:57.502 INFO:tasks.workunit.client.0.smithi107.stdout:5/557: truncate d5/db/d16/f21 408610 0 2024-05-17T21:52:57.503 INFO:tasks.workunit.client.0.smithi107.stdout:9/606: mknod d0/dd/d21/d29/d3e/d94/cd2 0 2024-05-17T21:52:57.506 INFO:tasks.workunit.client.1.smithi138.stdout:4/426: link d0/d1b/d88/f8b d0/d3/d1d/f97 0 2024-05-17T21:52:57.509 INFO:tasks.workunit.client.1.smithi138.stdout:2/443: dread d3/f4a [0,4194304] 0 2024-05-17T21:52:57.510 INFO:tasks.workunit.client.0.smithi107.stdout:8/580: mkdir dc/dd5 0 2024-05-17T21:52:57.511 INFO:tasks.workunit.client.0.smithi107.stdout:1/637: fsync d4/df/d1a/d26/d72/d85/f92 0 2024-05-17T21:52:57.512 INFO:tasks.workunit.client.1.smithi138.stdout:3/481: unlink d1/d2e/f8b 0 2024-05-17T21:52:57.513 INFO:tasks.workunit.client.0.smithi107.stdout:3/616: creat d6/d8/d10/d35/fc7 x:0 0 0 2024-05-17T21:52:57.517 INFO:tasks.workunit.client.0.smithi107.stdout:2/434: mknod d0/d9/d1c/d3e/c87 0 2024-05-17T21:52:57.517 INFO:tasks.workunit.client.0.smithi107.stdout:2/435: write d0/fa [8183239,77638] 0 2024-05-17T21:52:57.520 INFO:tasks.workunit.client.1.smithi138.stdout:0/429: mkdir d3/d10/d38/d70/d80/d94 0 2024-05-17T21:52:57.522 INFO:tasks.workunit.client.0.smithi107.stdout:4/580: dread d3/d6/d26/f31 [0,4194304] 0 2024-05-17T21:52:57.522 INFO:tasks.workunit.client.1.smithi138.stdout:8/333: write d0/ff [1844552,15934] 0 2024-05-17T21:52:57.524 INFO:tasks.workunit.client.0.smithi107.stdout:5/558: read d5/f8 [468415,118765] 0 2024-05-17T21:52:57.525 INFO:tasks.workunit.client.1.smithi138.stdout:7/336: creat d0/d18/d40/d41/d51/f69 x:0 0 0 2024-05-17T21:52:57.526 INFO:tasks.workunit.client.0.smithi107.stdout:0/660: truncate d3/dc/f37 3341447 0 2024-05-17T21:52:57.526 INFO:tasks.workunit.client.1.smithi138.stdout:7/337: write d0/f20 [10281613,114104] 0 2024-05-17T21:52:57.527 INFO:tasks.workunit.client.0.smithi107.stdout:8/581: dread - dc/d21/d27/d2f/f2c zero size 2024-05-17T21:52:57.528 INFO:tasks.workunit.client.1.smithi138.stdout:6/428: creat d7/db/d43/f98 x:0 0 0 2024-05-17T21:52:57.530 INFO:tasks.workunit.client.0.smithi107.stdout:9/607: dread d0/dd/d2f/d87/f98 [0,4194304] 0 2024-05-17T21:52:57.531 INFO:tasks.workunit.client.0.smithi107.stdout:8/582: dread dc/d21/d27/d34/d4c/f50 [0,4194304] 0 2024-05-17T21:52:57.531 INFO:tasks.workunit.client.1.smithi138.stdout:2/444: readlink d3/l2c 0 2024-05-17T21:52:57.536 INFO:tasks.workunit.client.1.smithi138.stdout:3/482: fdatasync d1/d4d/f7e 0 2024-05-17T21:52:57.536 INFO:tasks.workunit.client.0.smithi107.stdout:2/436: dread d0/d9/d11/df/d5a/d6d/f74 [0,4194304] 0 2024-05-17T21:52:57.537 INFO:tasks.workunit.client.0.smithi107.stdout:8/583: dwrite dc/f4f [0,4194304] 0 2024-05-17T21:52:57.543 INFO:tasks.workunit.client.0.smithi107.stdout:0/661: creat d3/d75/d9e/fea x:0 0 0 2024-05-17T21:52:57.545 INFO:tasks.workunit.client.0.smithi107.stdout:1/638: symlink d4/df/d1a/ld1 0 2024-05-17T21:52:57.546 INFO:tasks.workunit.client.0.smithi107.stdout:5/559: dwrite d5/db/d16/d2d/f4a [0,4194304] 0 2024-05-17T21:52:57.546 INFO:tasks.workunit.client.0.smithi107.stdout:1/639: fsync d4/df/d1a/d26/d7a/fb9 0 2024-05-17T21:52:57.547 INFO:tasks.workunit.client.1.smithi138.stdout:1/403: dwrite d4/db/f21 [0,4194304] 0 2024-05-17T21:52:57.548 INFO:tasks.workunit.client.0.smithi107.stdout:5/560: chown d5/db/d16/f81 4898 1 2024-05-17T21:52:57.548 INFO:tasks.workunit.client.0.smithi107.stdout:1/640: write d4/df/fa8 [191340,102296] 0 2024-05-17T21:52:57.549 INFO:tasks.workunit.client.0.smithi107.stdout:6/486: sync 2024-05-17T21:52:57.551 INFO:tasks.workunit.client.0.smithi107.stdout:3/617: truncate d6/d8/d10/d25/f47 2289175 0 2024-05-17T21:52:57.551 INFO:tasks.workunit.client.1.smithi138.stdout:8/334: write d0/d2f/f41 [349519,110634] 0 2024-05-17T21:52:57.552 INFO:tasks.workunit.client.0.smithi107.stdout:1/641: write d4/df/d14/fc9 [134262,2408] 0 2024-05-17T21:52:57.554 INFO:tasks.workunit.client.0.smithi107.stdout:1/642: truncate d4/df/d14/d1d/f2f 964155 0 2024-05-17T21:52:57.554 INFO:tasks.workunit.client.1.smithi138.stdout:7/338: creat d0/d18/d40/d41/d50/f6a x:0 0 0 2024-05-17T21:52:57.556 INFO:tasks.workunit.client.0.smithi107.stdout:7/560: creat d5/df/fc9 x:0 0 0 2024-05-17T21:52:57.557 INFO:tasks.workunit.client.1.smithi138.stdout:6/429: mknod d7/db/d43/d5f/d6a/c99 0 2024-05-17T21:52:57.557 INFO:tasks.workunit.client.1.smithi138.stdout:6/430: chown d7/db/d43/c86 237586 1 2024-05-17T21:52:57.559 INFO:tasks.workunit.client.1.smithi138.stdout:2/445: dread - d3/d21/d2f/d8a/f8b zero size 2024-05-17T21:52:57.562 INFO:tasks.workunit.client.0.smithi107.stdout:1/643: dwrite d4/df/d1a/d26/f60 [0,4194304] 0 2024-05-17T21:52:57.562 INFO:tasks.workunit.client.1.smithi138.stdout:3/483: creat d1/d3/d16/d2d/faa x:0 0 0 2024-05-17T21:52:57.563 INFO:tasks.workunit.client.1.smithi138.stdout:9/370: getdents d6/d7/d16/d29/d48 0 2024-05-17T21:52:57.566 INFO:tasks.workunit.client.0.smithi107.stdout:1/644: read d4/df/d1a/f28 [2099513,9898] 0 2024-05-17T21:52:57.567 INFO:tasks.workunit.client.0.smithi107.stdout:6/487: chown d9/c3c 227 1 2024-05-17T21:52:57.570 INFO:tasks.workunit.client.1.smithi138.stdout:1/404: unlink d4/df/d3a/f7e 0 2024-05-17T21:52:57.570 INFO:tasks.workunit.client.1.smithi138.stdout:1/405: dread - d4/df/d3a/f81 zero size 2024-05-17T21:52:57.572 INFO:tasks.workunit.client.1.smithi138.stdout:8/335: fdatasync d0/d11/f42 0 2024-05-17T21:52:57.573 INFO:tasks.workunit.client.1.smithi138.stdout:8/336: stat d0/dc/d1a/d33 0 2024-05-17T21:52:57.573 INFO:tasks.workunit.client.0.smithi107.stdout:2/437: truncate d0/d9/f46 909356 0 2024-05-17T21:52:57.575 INFO:tasks.workunit.client.1.smithi138.stdout:6/431: creat d7/d8/f9a x:0 0 0 2024-05-17T21:52:57.576 INFO:tasks.workunit.client.1.smithi138.stdout:6/432: read - d7/d8/f96 zero size 2024-05-17T21:52:57.580 INFO:tasks.workunit.client.1.smithi138.stdout:2/446: creat d3/d81/f92 x:0 0 0 2024-05-17T21:52:57.581 INFO:tasks.workunit.client.1.smithi138.stdout:3/484: creat d1/d3/d16/d2d/fab x:0 0 0 2024-05-17T21:52:57.582 INFO:tasks.workunit.client.0.smithi107.stdout:6/488: fsync d9/d16/d22/d5f/f75 0 2024-05-17T21:52:57.584 INFO:tasks.workunit.client.1.smithi138.stdout:9/371: creat d6/d7/d16/d29/d38/d40/d44/d5c/f7a x:0 0 0 2024-05-17T21:52:57.584 INFO:tasks.workunit.client.1.smithi138.stdout:4/427: write d0/d2f/d49/d60/d1f/f73 [311516,6364] 0 2024-05-17T21:52:57.586 INFO:tasks.workunit.client.1.smithi138.stdout:4/428: write d0/d3/db/f69 [572523,7403] 0 2024-05-17T21:52:57.587 INFO:tasks.workunit.client.0.smithi107.stdout:4/581: rename d3/db/d35/d4f/c7d to d3/db/d1e/cbf 0 2024-05-17T21:52:57.587 INFO:tasks.workunit.client.0.smithi107.stdout:6/489: dread d9/d16/d22/d2d/f70 [0,4194304] 0 2024-05-17T21:52:57.589 INFO:tasks.workunit.client.0.smithi107.stdout:2/438: mkdir d0/d9/d1c/d22/d88 0 2024-05-17T21:52:57.590 INFO:tasks.workunit.client.0.smithi107.stdout:4/582: write d3/d4a/d47/d60/d3a/d41/d80/d43/f62 [824062,7268] 0 2024-05-17T21:52:57.591 INFO:tasks.workunit.client.1.smithi138.stdout:0/430: truncate d3/d3d/f7f 100350 0 2024-05-17T21:52:57.592 INFO:tasks.workunit.client.0.smithi107.stdout:9/608: getdents d0/dd/d2f/d87 0 2024-05-17T21:52:57.594 INFO:tasks.workunit.client.0.smithi107.stdout:0/662: dwrite d3/d58/d61/f93 [0,4194304] 0 2024-05-17T21:52:57.594 INFO:tasks.workunit.client.1.smithi138.stdout:8/337: symlink d0/dc/d1a/d33/l66 0 2024-05-17T21:52:57.595 INFO:tasks.workunit.client.1.smithi138.stdout:7/339: dread d0/f31 [0,4194304] 0 2024-05-17T21:52:57.597 INFO:tasks.workunit.client.1.smithi138.stdout:1/406: dread d4/df/f2c [4194304,4194304] 0 2024-05-17T21:52:57.597 INFO:tasks.workunit.client.1.smithi138.stdout:1/407: stat d4/f3d 0 2024-05-17T21:52:57.597 INFO:tasks.workunit.client.1.smithi138.stdout:6/433: mkdir d7/db/d43/d5f/d6a/d9b 0 2024-05-17T21:52:57.598 INFO:tasks.workunit.client.0.smithi107.stdout:2/439: dread d0/d9/d1c/d22/f31 [4194304,4194304] 0 2024-05-17T21:52:57.601 INFO:tasks.workunit.client.0.smithi107.stdout:8/584: rename dc/d21/d27/d83/f91 to dc/d21/d27/d2f/dba/fd6 0 2024-05-17T21:52:57.601 INFO:tasks.workunit.client.0.smithi107.stdout:0/663: dread d3/d24/d6f/d9f/fbd [0,4194304] 0 2024-05-17T21:52:57.601 INFO:tasks.workunit.client.0.smithi107.stdout:3/618: truncate d6/d1a/f4e 1922723 0 2024-05-17T21:52:57.602 INFO:tasks.workunit.client.0.smithi107.stdout:3/619: dread - d6/d1a/d57/fb2 zero size 2024-05-17T21:52:57.602 INFO:tasks.workunit.client.1.smithi138.stdout:6/434: dread d7/db/d43/f35 [0,4194304] 0 2024-05-17T21:52:57.603 INFO:tasks.workunit.client.0.smithi107.stdout:6/490: fdatasync d9/f53 0 2024-05-17T21:52:57.605 INFO:tasks.workunit.client.1.smithi138.stdout:2/447: rename d3/d21/d2f/d3e/d63/d65 to d3/d21/d2f/d3e/d63/d7f/d93 0 2024-05-17T21:52:57.609 INFO:tasks.workunit.client.0.smithi107.stdout:3/620: dwrite d6/d8/d3e/d3f/d90/fb5 [0,4194304] 0 2024-05-17T21:52:57.614 INFO:tasks.workunit.client.1.smithi138.stdout:9/372: dwrite d6/d7/d16/d29/f3a [0,4194304] 0 2024-05-17T21:52:57.615 INFO:tasks.workunit.client.1.smithi138.stdout:9/373: stat d6/d7/d16/d29/d48 0 2024-05-17T21:52:57.617 INFO:tasks.workunit.client.1.smithi138.stdout:8/338: mknod d0/dc/d3b/c67 0 2024-05-17T21:52:57.618 INFO:tasks.workunit.client.0.smithi107.stdout:4/583: symlink d3/db/d35/d4f/lc0 0 2024-05-17T21:52:57.619 INFO:tasks.workunit.client.1.smithi138.stdout:7/340: creat d0/d39/f6b x:0 0 0 2024-05-17T21:52:57.622 INFO:tasks.workunit.client.1.smithi138.stdout:1/408: unlink f3 0 2024-05-17T21:52:57.623 INFO:tasks.workunit.client.1.smithi138.stdout:1/409: chown d4/dd/d3b/d6c/f78 12 1 2024-05-17T21:52:57.624 INFO:tasks.workunit.client.1.smithi138.stdout:1/410: read d4/f3d [534077,44003] 0 2024-05-17T21:52:57.625 INFO:tasks.workunit.client.1.smithi138.stdout:1/411: read - d4/dd/d3b/f49 zero size 2024-05-17T21:52:57.628 INFO:tasks.workunit.client.1.smithi138.stdout:3/485: creat d1/d4f/d80/fac x:0 0 0 2024-05-17T21:52:57.630 INFO:tasks.workunit.client.0.smithi107.stdout:2/440: creat d0/d9/d11/d15/f89 x:0 0 0 2024-05-17T21:52:57.631 INFO:tasks.workunit.client.1.smithi138.stdout:2/448: creat d3/d21/d2f/d3e/d63/d7f/d84/f94 x:0 0 0 2024-05-17T21:52:57.632 INFO:tasks.workunit.client.0.smithi107.stdout:2/441: truncate d0/d9/d11/d18/f79 481615 0 2024-05-17T21:52:57.633 INFO:tasks.workunit.client.0.smithi107.stdout:5/561: rename d5/db/d16/d2a/c76 to d5/db/d16/d54/d59/cad 0 2024-05-17T21:52:57.634 INFO:tasks.workunit.client.0.smithi107.stdout:5/562: dread - d5/db/d16/d2d/d40/d5a/fa7 zero size 2024-05-17T21:52:57.635 INFO:tasks.workunit.client.1.smithi138.stdout:0/431: unlink d3/d10/f62 0 2024-05-17T21:52:57.637 INFO:tasks.workunit.client.1.smithi138.stdout:9/374: rename d6/d7/la to d6/d7/d16/d29/d38/d3c/l7b 0 2024-05-17T21:52:57.638 INFO:tasks.workunit.client.0.smithi107.stdout:6/491: symlink d9/d16/d22/d5f/d6e/l9c 0 2024-05-17T21:52:57.639 INFO:tasks.workunit.client.1.smithi138.stdout:8/339: write d0/f1b [1496571,69343] 0 2024-05-17T21:52:57.639 INFO:tasks.workunit.client.0.smithi107.stdout:0/664: dwrite d3/dc/f14 [0,4194304] 0 2024-05-17T21:52:57.641 INFO:tasks.workunit.client.1.smithi138.stdout:7/341: chown d0/d18/d40/l52 41 1 2024-05-17T21:52:57.642 INFO:tasks.workunit.client.1.smithi138.stdout:7/342: chown d0/d18/d40/d41 1841382 1 2024-05-17T21:52:57.642 INFO:tasks.workunit.client.1.smithi138.stdout:7/343: dread - d0/d18/d40/f4e zero size 2024-05-17T21:52:57.644 INFO:tasks.workunit.client.0.smithi107.stdout:9/609: write d0/d80/fd0 [2248182,14745] 0 2024-05-17T21:52:57.645 INFO:tasks.workunit.client.1.smithi138.stdout:7/344: dread d0/d39/f3d [0,4194304] 0 2024-05-17T21:52:57.646 INFO:tasks.workunit.client.0.smithi107.stdout:4/584: mkdir d3/d6/d26/d36/d37/dc1 0 2024-05-17T21:52:57.647 INFO:tasks.workunit.client.0.smithi107.stdout:4/585: chown d3/d6/d26/f31 1394 1 2024-05-17T21:52:57.652 INFO:tasks.workunit.client.1.smithi138.stdout:6/435: symlink d7/db/d43/d5f/d6a/d8b/l9c 0 2024-05-17T21:52:57.654 INFO:tasks.workunit.client.0.smithi107.stdout:7/561: rename d5/df/d11/d1c/l6b to d5/d16/d3b/d45/d91/db3/lca 0 2024-05-17T21:52:57.658 INFO:tasks.workunit.client.1.smithi138.stdout:6/436: dwrite d7/d8/f5d [0,4194304] 0 2024-05-17T21:52:57.658 INFO:tasks.workunit.client.1.smithi138.stdout:4/429: link d0/d3/d17/c66 d0/d2f/d49/d60/d1f/d32/d71/c98 0 2024-05-17T21:52:57.663 INFO:tasks.workunit.client.1.smithi138.stdout:1/412: rename d4/dd/d3b/f49 to d4/db/d29/d5d/f86 0 2024-05-17T21:52:57.664 INFO:tasks.workunit.client.0.smithi107.stdout:2/442: write d0/d9/d11/f10 [1777750,8511] 0 2024-05-17T21:52:57.666 INFO:tasks.workunit.client.0.smithi107.stdout:2/443: stat d0/d9/d11/df/d5a/c5d 0 2024-05-17T21:52:57.669 INFO:tasks.workunit.client.1.smithi138.stdout:8/340: mkdir d0/dc/d1a/d33/d4a/d68 0 2024-05-17T21:52:57.670 INFO:tasks.workunit.client.0.smithi107.stdout:9/610: creat d0/dd/d2f/d87/d89/fd3 x:0 0 0 2024-05-17T21:52:57.673 INFO:tasks.workunit.client.1.smithi138.stdout:7/345: creat d0/d18/d40/f6c x:0 0 0 2024-05-17T21:52:57.676 INFO:tasks.workunit.client.0.smithi107.stdout:8/585: truncate dc/d21/d97/f8d 1283665 0 2024-05-17T21:52:57.677 INFO:tasks.workunit.client.0.smithi107.stdout:8/586: write dc/d21/d97/d63/f79 [9303331,13121] 0 2024-05-17T21:52:57.677 INFO:tasks.workunit.client.1.smithi138.stdout:2/449: mknod d3/d21/d2f/d3e/c95 0 2024-05-17T21:52:57.678 INFO:tasks.workunit.client.1.smithi138.stdout:7/346: dwrite d0/d18/f57 [0,4194304] 0 2024-05-17T21:52:57.679 INFO:tasks.workunit.client.1.smithi138.stdout:2/450: write d3/f1c [3217117,83143] 0 2024-05-17T21:52:57.681 INFO:tasks.workunit.client.0.smithi107.stdout:3/621: rename d6/d1a/d57/f5e to d6/d2a/d2d/d37/fc8 0 2024-05-17T21:52:57.687 INFO:tasks.workunit.client.0.smithi107.stdout:0/665: write d3/dc/d31/f63 [998502,111104] 0 2024-05-17T21:52:57.687 INFO:tasks.workunit.client.0.smithi107.stdout:4/586: symlink d3/db/d1e/d99/lc2 0 2024-05-17T21:52:57.688 INFO:tasks.workunit.client.1.smithi138.stdout:6/437: dwrite d7/f28 [0,4194304] 0 2024-05-17T21:52:57.689 INFO:tasks.workunit.client.0.smithi107.stdout:0/666: write d3/fa1 [1013381,110814] 0 2024-05-17T21:52:57.690 INFO:tasks.workunit.client.1.smithi138.stdout:6/438: write d7/d69/f80 [439299,105746] 0 2024-05-17T21:52:57.695 INFO:tasks.workunit.client.1.smithi138.stdout:3/486: rename d1/d3/d8 to d1/d62/dad 0 2024-05-17T21:52:57.700 INFO:tasks.workunit.client.1.smithi138.stdout:9/375: mknod d6/d7/d16/d29/c7c 0 2024-05-17T21:52:57.701 INFO:tasks.workunit.client.0.smithi107.stdout:7/562: dwrite f4 [0,4194304] 0 2024-05-17T21:52:57.703 INFO:tasks.workunit.client.1.smithi138.stdout:8/341: truncate d0/d11/f28 336655 0 2024-05-17T21:52:57.704 INFO:tasks.workunit.client.1.smithi138.stdout:3/487: dwrite d1/d3/d8d/d92/fa6 [0,4194304] 0 2024-05-17T21:52:57.706 INFO:tasks.workunit.client.1.smithi138.stdout:3/488: dread - d1/d3/d49/d4b/f6e zero size 2024-05-17T21:52:57.713 INFO:tasks.workunit.client.1.smithi138.stdout:2/451: mkdir d3/d81/d96 0 2024-05-17T21:52:57.717 INFO:tasks.workunit.client.1.smithi138.stdout:1/413: fdatasync d4/d2e/f54 0 2024-05-17T21:52:57.719 INFO:tasks.workunit.client.1.smithi138.stdout:4/430: dread d0/d3/f40 [0,4194304] 0 2024-05-17T21:52:57.720 INFO:tasks.workunit.client.1.smithi138.stdout:6/439: rename d7/db/d12/f3e to d7/d69/f9d 0 2024-05-17T21:52:57.720 INFO:tasks.workunit.client.1.smithi138.stdout:4/431: chown d0/d3/db/c3f 4148534 1 2024-05-17T21:52:57.724 INFO:tasks.workunit.client.1.smithi138.stdout:8/342: truncate d0/dc/d1a/f2a 970826 0 2024-05-17T21:52:57.725 INFO:tasks.workunit.client.1.smithi138.stdout:3/489: chown d1/d3/d49/d4b/c90 892 1 2024-05-17T21:52:57.735 INFO:tasks.workunit.client.0.smithi107.stdout:5/563: creat d5/d75/fae x:0 0 0 2024-05-17T21:52:57.736 INFO:tasks.workunit.client.0.smithi107.stdout:3/622: mkdir d6/d8/d3e/dc9 0 2024-05-17T21:52:57.738 INFO:tasks.workunit.client.0.smithi107.stdout:3/623: read d6/d2a/d2d/d37/f3a [990796,38265] 0 2024-05-17T21:52:57.742 INFO:tasks.workunit.client.0.smithi107.stdout:5/564: dwrite d5/db/d39/f43 [4194304,4194304] 0 2024-05-17T21:52:57.743 INFO:tasks.workunit.client.1.smithi138.stdout:2/452: write d3/f27 [886024,104918] 0 2024-05-17T21:52:57.745 INFO:tasks.workunit.client.0.smithi107.stdout:6/492: rename d9/d16/d22/f74 to d9/d16/d22/d5f/f9d 0 2024-05-17T21:52:57.754 INFO:tasks.workunit.client.0.smithi107.stdout:0/667: fsync d3/d24/f2a 0 2024-05-17T21:52:57.756 INFO:tasks.workunit.client.0.smithi107.stdout:3/624: mknod d6/d2a/d2d/d37/cca 0 2024-05-17T21:52:57.757 INFO:tasks.workunit.client.0.smithi107.stdout:0/668: truncate d3/d24/d6f/db5/fdd 410443 0 2024-05-17T21:52:57.758 INFO:tasks.workunit.client.1.smithi138.stdout:8/343: chown d0/d63 6999834 1 2024-05-17T21:52:57.759 INFO:tasks.workunit.client.1.smithi138.stdout:8/344: fsync d0/dc/f18 0 2024-05-17T21:52:57.760 INFO:tasks.workunit.client.1.smithi138.stdout:8/345: chown d0/d2f/c55 10846 1 2024-05-17T21:52:57.763 INFO:tasks.workunit.client.0.smithi107.stdout:9/611: rename d0/dd/d2f/d87/d89/d9b to d0/dd/d21/d29/dd4 0 2024-05-17T21:52:57.765 INFO:tasks.workunit.client.1.smithi138.stdout:8/346: dwrite d0/dc/d1a/f4b [0,4194304] 0 2024-05-17T21:52:57.767 INFO:tasks.workunit.client.1.smithi138.stdout:0/432: link d3/d10/d38/l77 d3/d3d/l95 0 2024-05-17T21:52:57.771 INFO:tasks.workunit.client.0.smithi107.stdout:8/587: truncate f4 1754345 0 2024-05-17T21:52:57.773 INFO:tasks.workunit.client.1.smithi138.stdout:5/415: sync 2024-05-17T21:52:57.773 INFO:tasks.workunit.client.0.smithi107.stdout:3/625: unlink d6/d8/f85 0 2024-05-17T21:52:57.774 INFO:tasks.workunit.client.1.smithi138.stdout:5/416: chown d1/d1a/d39/f6a 28 1 2024-05-17T21:52:57.779 INFO:tasks.workunit.client.0.smithi107.stdout:6/493: mkdir d9/d16/d9e 0 2024-05-17T21:52:57.779 INFO:tasks.workunit.client.0.smithi107.stdout:6/494: chown d9/f12 16319 1 2024-05-17T21:52:57.781 INFO:tasks.workunit.client.0.smithi107.stdout:6/495: write d9/d16/d22/d2d/f83 [523319,95916] 0 2024-05-17T21:52:57.781 INFO:tasks.workunit.client.1.smithi138.stdout:3/490: symlink d1/d3/lae 0 2024-05-17T21:52:57.782 INFO:tasks.workunit.client.0.smithi107.stdout:6/496: chown d9/d16/d4d/d85/f98 14794194 1 2024-05-17T21:52:57.784 INFO:tasks.workunit.client.1.smithi138.stdout:2/453: dread d3/d21/f30 [0,4194304] 0 2024-05-17T21:52:57.784 INFO:tasks.workunit.client.0.smithi107.stdout:8/588: fsync dc/d21/d27/d2f/f30 0 2024-05-17T21:52:57.786 INFO:tasks.workunit.client.1.smithi138.stdout:2/454: chown d3/d21/d2f/d3e/d63/f6e 108 1 2024-05-17T21:52:57.787 INFO:tasks.workunit.client.1.smithi138.stdout:7/347: getdents d0 0 2024-05-17T21:52:57.787 INFO:tasks.workunit.client.0.smithi107.stdout:6/497: dwrite d9/d16/d4d/d67/f79 [0,4194304] 0 2024-05-17T21:52:57.791 INFO:tasks.workunit.client.1.smithi138.stdout:9/376: dwrite d6/d7/f17 [0,4194304] 0 2024-05-17T21:52:57.792 INFO:tasks.workunit.client.0.smithi107.stdout:7/563: truncate d5/df/d11/d1d/d33/d82/fb1 2890695 0 2024-05-17T21:52:57.793 INFO:tasks.workunit.client.1.smithi138.stdout:8/347: creat d0/d11/f69 x:0 0 0 2024-05-17T21:52:57.794 INFO:tasks.workunit.client.1.smithi138.stdout:8/348: fsync d0/dc/d3b/f43 0 2024-05-17T21:52:57.794 INFO:tasks.workunit.client.0.smithi107.stdout:5/565: link d5/db/d16/d2d/l42 d5/db/d16/d54/da0/laf 0 2024-05-17T21:52:57.794 INFO:tasks.workunit.client.0.smithi107.stdout:3/626: dread d6/d8/d10/f4b [0,4194304] 0 2024-05-17T21:52:57.802 INFO:tasks.workunit.client.1.smithi138.stdout:6/440: creat d7/db/f9e x:0 0 0 2024-05-17T21:52:57.803 INFO:tasks.workunit.client.0.smithi107.stdout:1/645: sync 2024-05-17T21:52:57.804 INFO:tasks.workunit.client.1.smithi138.stdout:5/417: mknod d1/d1a/d39/d4e/d58/d5b/d65/c89 0 2024-05-17T21:52:57.806 INFO:tasks.workunit.client.1.smithi138.stdout:4/432: link d0/lf d0/d2f/d49/d60/d1f/d7e/l99 0 2024-05-17T21:52:57.808 INFO:tasks.workunit.client.0.smithi107.stdout:1/646: dwrite d4/df/d4e/f63 [0,4194304] 0 2024-05-17T21:52:57.809 INFO:tasks.workunit.client.1.smithi138.stdout:3/491: rename d1/d2e/c37 to d1/d62/dad/de/caf 0 2024-05-17T21:52:57.810 INFO:tasks.workunit.client.0.smithi107.stdout:1/647: write d4/df/d1a/d26/d7a/f93 [1974980,50884] 0 2024-05-17T21:52:57.811 INFO:tasks.workunit.client.1.smithi138.stdout:6/441: dwrite d7/db/d43/d3f/f4c [4194304,4194304] 0 2024-05-17T21:52:57.811 INFO:tasks.workunit.client.0.smithi107.stdout:0/669: link d3/d24/d6f/d9f/ce6 d3/d39/ceb 0 2024-05-17T21:52:57.811 INFO:tasks.workunit.client.0.smithi107.stdout:6/498: read d9/f1f [46302,63344] 0 2024-05-17T21:52:57.815 INFO:tasks.workunit.client.0.smithi107.stdout:7/564: creat d5/d16/d3b/d45/d91/db3/fcb x:0 0 0 2024-05-17T21:52:57.822 INFO:tasks.workunit.client.1.smithi138.stdout:8/349: read d0/dc/d1a/f2a [871959,21] 0 2024-05-17T21:52:57.825 INFO:tasks.workunit.client.0.smithi107.stdout:4/587: rename d3/d6/c24 to d3/d4a/d47/d60/cc3 0 2024-05-17T21:52:57.831 INFO:tasks.workunit.client.1.smithi138.stdout:4/433: rename d0/d3/f16 to d0/d3/d17/d21/d78/f9a 0 2024-05-17T21:52:57.832 INFO:tasks.workunit.client.0.smithi107.stdout:1/648: fsync d4/de/f1e 0 2024-05-17T21:52:57.834 INFO:tasks.workunit.client.1.smithi138.stdout:3/492: fsync d1/fa5 0 2024-05-17T21:52:57.836 INFO:tasks.workunit.client.1.smithi138.stdout:5/418: dwrite d1/d46/f71 [0,4194304] 0 2024-05-17T21:52:57.836 INFO:tasks.workunit.client.1.smithi138.stdout:6/442: rmdir d7/db/d43 39 2024-05-17T21:52:57.842 INFO:tasks.workunit.client.1.smithi138.stdout:8/350: chown d0/fb 739235 1 2024-05-17T21:52:57.843 INFO:tasks.workunit.client.1.smithi138.stdout:2/455: dread d3/f15 [0,4194304] 0 2024-05-17T21:52:57.844 INFO:tasks.workunit.client.1.smithi138.stdout:0/433: dwrite d3/fc [4194304,4194304] 0 2024-05-17T21:52:57.846 INFO:tasks.workunit.client.1.smithi138.stdout:1/414: getdents d4/df/d3a/d3c 0 2024-05-17T21:52:57.847 INFO:tasks.workunit.client.0.smithi107.stdout:6/499: dwrite d9/d16/d22/d5f/f9d [0,4194304] 0 2024-05-17T21:52:57.854 INFO:tasks.workunit.client.0.smithi107.stdout:9/612: rename d0/dd/d21/d29/d3e/d45/d68/fa1 to d0/dd/d2f/d4a/fd5 0 2024-05-17T21:52:57.855 INFO:tasks.workunit.client.0.smithi107.stdout:2/444: sync 2024-05-17T21:52:57.856 INFO:tasks.workunit.client.0.smithi107.stdout:4/588: write d3/d4a/d47/d60/d3a/d41/d80/f94 [4040176,72231] 0 2024-05-17T21:52:57.857 INFO:tasks.workunit.client.0.smithi107.stdout:8/589: creat dc/d21/fd7 x:0 0 0 2024-05-17T21:52:57.858 INFO:tasks.workunit.client.0.smithi107.stdout:1/649: dread d4/df/d1a/d26/d72/d85/fc0 [0,4194304] 0 2024-05-17T21:52:57.859 INFO:tasks.workunit.client.1.smithi138.stdout:6/443: dread - d7/db/d43/d3f/d79/f93 zero size 2024-05-17T21:52:57.860 INFO:tasks.workunit.client.1.smithi138.stdout:9/377: link d6/f49 d6/d7/d16/d29/d48/f7d 0 2024-05-17T21:52:57.861 INFO:tasks.workunit.client.0.smithi107.stdout:0/670: creat d3/d24/dcf/fec x:0 0 0 2024-05-17T21:52:57.862 INFO:tasks.workunit.client.0.smithi107.stdout:0/671: chown d3/d24/d6f/d9f/dab/f70 74 1 2024-05-17T21:52:57.862 INFO:tasks.workunit.client.1.smithi138.stdout:8/351: mkdir d0/d11/d6a 0 2024-05-17T21:52:57.863 INFO:tasks.workunit.client.0.smithi107.stdout:0/672: write d3/d24/d6f/d9f/dab/f7c [2135713,8087] 0 2024-05-17T21:52:57.864 INFO:tasks.workunit.client.0.smithi107.stdout:9/613: dread d0/dd/d3d/fb8 [0,4194304] 0 2024-05-17T21:52:57.865 INFO:tasks.workunit.client.1.smithi138.stdout:3/493: dwrite d1/f4 [4194304,4194304] 0 2024-05-17T21:52:57.865 INFO:tasks.workunit.client.0.smithi107.stdout:9/614: read - d0/d80/fcf zero size 2024-05-17T21:52:57.872 INFO:tasks.workunit.client.0.smithi107.stdout:5/566: rename d5/db/dc/f4e to d5/db/d16/d2d/d40/d5a/d7c/d8e/fb0 0 2024-05-17T21:52:57.873 INFO:tasks.workunit.client.1.smithi138.stdout:5/419: mkdir d1/d8a 0 2024-05-17T21:52:57.875 INFO:tasks.workunit.client.0.smithi107.stdout:3/627: truncate d6/d1a/d57/f73 1882414 0 2024-05-17T21:52:57.877 INFO:tasks.workunit.client.0.smithi107.stdout:2/445: creat d0/d26/d64/f8a x:0 0 0 2024-05-17T21:52:57.878 INFO:tasks.workunit.client.1.smithi138.stdout:9/378: mkdir d6/d7/d16/d29/d45/d7e 0 2024-05-17T21:52:57.879 INFO:tasks.workunit.client.0.smithi107.stdout:2/446: write d0/d9/d11/d18/f20 [1276790,88700] 0 2024-05-17T21:52:57.880 INFO:tasks.workunit.client.1.smithi138.stdout:8/352: symlink d0/dc/d1a/l6b 0 2024-05-17T21:52:57.881 INFO:tasks.workunit.client.0.smithi107.stdout:5/567: dwrite d5/db/d16/d2d/f4a [0,4194304] 0 2024-05-17T21:52:57.882 INFO:tasks.workunit.client.0.smithi107.stdout:2/447: dread - d0/d9/d11/df/f71 zero size 2024-05-17T21:52:57.882 INFO:tasks.workunit.client.0.smithi107.stdout:4/589: creat d3/db/d35/d55/fc4 x:0 0 0 2024-05-17T21:52:57.882 INFO:tasks.workunit.client.1.smithi138.stdout:0/434: dread d3/f83 [0,4194304] 0 2024-05-17T21:52:57.885 INFO:tasks.workunit.client.1.smithi138.stdout:9/379: dwrite d6/d7/f10 [0,4194304] 0 2024-05-17T21:52:57.886 INFO:tasks.workunit.client.0.smithi107.stdout:7/565: truncate d5/df/d11/d1c/d2f/d53/fa9 132476 0 2024-05-17T21:52:57.886 INFO:tasks.workunit.client.1.smithi138.stdout:9/380: write d6/d7/d16/d29/d48/f23 [4057321,124561] 0 2024-05-17T21:52:57.888 INFO:tasks.workunit.client.0.smithi107.stdout:8/590: creat dc/d21/d97/fd8 x:0 0 0 2024-05-17T21:52:57.889 INFO:tasks.workunit.client.1.smithi138.stdout:2/456: creat d3/d21/d2f/d3e/d55/d73/f97 x:0 0 0 2024-05-17T21:52:57.889 INFO:tasks.workunit.client.0.smithi107.stdout:8/591: readlink dc/d21/d97/l48 0 2024-05-17T21:52:57.890 INFO:tasks.workunit.client.0.smithi107.stdout:1/650: creat d4/df/d1a/d26/d72/daa/fd2 x:0 0 0 2024-05-17T21:52:57.893 INFO:tasks.workunit.client.1.smithi138.stdout:3/494: unlink d1/d62/dad/de/f3b 0 2024-05-17T21:52:57.894 INFO:tasks.workunit.client.1.smithi138.stdout:4/434: truncate d0/d1b/f18 3959510 0 2024-05-17T21:52:57.894 INFO:tasks.workunit.client.1.smithi138.stdout:9/381: dwrite f5 [0,4194304] 0 2024-05-17T21:52:57.897 INFO:tasks.workunit.client.1.smithi138.stdout:3/495: dread d1/d62/dad/d5c/f9f [0,4194304] 0 2024-05-17T21:52:57.900 INFO:tasks.workunit.client.0.smithi107.stdout:0/673: symlink d3/d58/led 0 2024-05-17T21:52:57.900 INFO:tasks.workunit.client.0.smithi107.stdout:0/674: chown d3 0 1 2024-05-17T21:52:57.902 INFO:tasks.workunit.client.1.smithi138.stdout:5/420: creat d1/d2c/f8b x:0 0 0 2024-05-17T21:52:57.904 INFO:tasks.workunit.client.1.smithi138.stdout:5/421: write d1/d6/d1b/f54 [2275591,109216] 0 2024-05-17T21:52:57.904 INFO:tasks.workunit.client.0.smithi107.stdout:6/500: fdatasync d9/d16/d22/f28 0 2024-05-17T21:52:57.905 INFO:tasks.workunit.client.1.smithi138.stdout:5/422: stat d1/d53/d88 0 2024-05-17T21:52:57.905 INFO:tasks.workunit.client.1.smithi138.stdout:1/415: dwrite d4/dd/f2f [4194304,4194304] 0 2024-05-17T21:52:57.906 INFO:tasks.workunit.client.0.smithi107.stdout:9/615: symlink d0/dd/d1f/d2c/ld6 0 2024-05-17T21:52:57.907 INFO:tasks.workunit.client.1.smithi138.stdout:1/416: fsync d4/dd/d3b/f53 0 2024-05-17T21:52:57.912 INFO:tasks.workunit.client.1.smithi138.stdout:6/444: dwrite f1 [4194304,4194304] 0 2024-05-17T21:52:57.914 INFO:tasks.workunit.client.1.smithi138.stdout:0/435: mknod d3/d3d/d51/c96 0 2024-05-17T21:52:57.918 INFO:tasks.workunit.client.0.smithi107.stdout:3/628: mknod d6/d8/d10/d35/ccb 0 2024-05-17T21:52:57.922 INFO:tasks.workunit.client.1.smithi138.stdout:3/496: mkdir d1/d62/db0 0 2024-05-17T21:52:57.929 INFO:tasks.workunit.client.1.smithi138.stdout:8/353: rename d0/f9 to d0/dc/d1a/d33/d4a/d68/f6c 0 2024-05-17T21:52:57.930 INFO:tasks.workunit.client.0.smithi107.stdout:6/501: symlink d9/d16/d4d/l9f 0 2024-05-17T21:52:57.931 INFO:tasks.workunit.client.0.smithi107.stdout:6/502: chown d9/d16/d22/d2d 31173043 1 2024-05-17T21:52:57.932 INFO:tasks.workunit.client.0.smithi107.stdout:6/503: chown d9/d16/d4d/f52 15047748 1 2024-05-17T21:52:57.933 INFO:tasks.workunit.client.0.smithi107.stdout:6/504: chown d9/d16/c80 15706804 1 2024-05-17T21:52:57.934 INFO:tasks.workunit.client.0.smithi107.stdout:6/505: write d9/d16/d4d/f8b [779354,28073] 0 2024-05-17T21:52:57.935 INFO:tasks.workunit.client.0.smithi107.stdout:6/506: dread - d9/d58/f7e zero size 2024-05-17T21:52:57.935 INFO:tasks.workunit.client.1.smithi138.stdout:2/457: creat d3/d21/d2f/d3e/d55/d62/f98 x:0 0 0 2024-05-17T21:52:57.936 INFO:tasks.workunit.client.0.smithi107.stdout:6/507: chown d9/d16/f5e 24 1 2024-05-17T21:52:57.937 INFO:tasks.workunit.client.0.smithi107.stdout:6/508: chown d9/d58/l69 4 1 2024-05-17T21:52:57.938 INFO:tasks.workunit.client.0.smithi107.stdout:6/509: write d9/f15 [4872249,1412] 0 2024-05-17T21:52:57.939 INFO:tasks.workunit.client.0.smithi107.stdout:5/568: fdatasync d5/db/d16/d2d/d4d/d6d/f77 0 2024-05-17T21:52:57.940 INFO:tasks.workunit.client.1.smithi138.stdout:9/382: truncate d6/d7/d16/d29/d48/f7d 24880 0 2024-05-17T21:52:57.941 INFO:tasks.workunit.client.1.smithi138.stdout:2/458: dread d3/d21/d2f/d3e/d63/d7f/d93/f79 [0,4194304] 0 2024-05-17T21:52:57.941 INFO:tasks.workunit.client.1.smithi138.stdout:9/383: dread - d6/d7/d16/f6d zero size 2024-05-17T21:52:57.942 INFO:tasks.workunit.client.1.smithi138.stdout:2/459: chown d3/d21/f2b 1400 1 2024-05-17T21:52:57.942 INFO:tasks.workunit.client.1.smithi138.stdout:3/497: stat d1/d3/d49/c59 0 2024-05-17T21:52:57.943 INFO:tasks.workunit.client.0.smithi107.stdout:7/566: truncate d5/f6d 453584 0 2024-05-17T21:52:57.943 INFO:tasks.workunit.client.0.smithi107.stdout:3/629: dwrite d6/d8/d3e/d3f/f87 [4194304,4194304] 0 2024-05-17T21:52:57.944 INFO:tasks.workunit.client.1.smithi138.stdout:1/417: creat d4/df/d3a/d3c/d63/d6d/f87 x:0 0 0 2024-05-17T21:52:57.949 INFO:tasks.workunit.client.0.smithi107.stdout:0/675: rename d3/d39/ceb to d3/dc/d22/d23/d97/d77/d91/cee 0 2024-05-17T21:52:57.949 INFO:tasks.workunit.client.1.smithi138.stdout:3/498: dwrite d1/d3/d16/d3e/f73 [0,4194304] 0 2024-05-17T21:52:57.951 INFO:tasks.workunit.client.0.smithi107.stdout:9/616: getdents d0/dd/d21/daf 0 2024-05-17T21:52:57.952 INFO:tasks.workunit.client.0.smithi107.stdout:8/592: write dc/d21/d27/f5b [4468057,127290] 0 2024-05-17T21:52:57.953 INFO:tasks.workunit.client.1.smithi138.stdout:4/435: rename d0/d2f/l68 to d0/d3/d1d/d5b/l9b 0 2024-05-17T21:52:57.955 INFO:tasks.workunit.client.0.smithi107.stdout:1/651: write d4/df/d1a/d41/f48 [2503410,66490] 0 2024-05-17T21:52:57.955 INFO:tasks.workunit.client.0.smithi107.stdout:2/448: creat d0/d9/d11/f8b x:0 0 0 2024-05-17T21:52:57.955 INFO:tasks.workunit.client.1.smithi138.stdout:6/445: link d7/db/d43/d5f/d6a/d77/f85 d7/db/d43/d3f/d79/f9f 0 2024-05-17T21:52:57.958 INFO:tasks.workunit.client.0.smithi107.stdout:0/676: dwrite d3/dc/d22/d3d/f4d [0,4194304] 0 2024-05-17T21:52:57.963 INFO:tasks.workunit.client.1.smithi138.stdout:9/384: unlink d6/d7/d16/d29/d38/c46 0 2024-05-17T21:52:57.965 INFO:tasks.workunit.client.0.smithi107.stdout:6/510: mknod d9/d16/d22/d5f/d6e/ca0 0 2024-05-17T21:52:57.965 INFO:tasks.workunit.client.0.smithi107.stdout:6/511: stat d9/d81/f97 0 2024-05-17T21:52:57.966 INFO:tasks.workunit.client.0.smithi107.stdout:6/512: chown d9/d16/d22/d5f/c78 13 1 2024-05-17T21:52:57.967 INFO:tasks.workunit.client.0.smithi107.stdout:6/513: write d9/d58/f9b [1016461,48784] 0 2024-05-17T21:52:57.968 INFO:tasks.workunit.client.1.smithi138.stdout:2/460: unlink d3/d21/d2f/l42 0 2024-05-17T21:52:57.968 INFO:tasks.workunit.client.1.smithi138.stdout:0/436: dwrite d3/d10/d38/d3e/f5b [0,4194304] 0 2024-05-17T21:52:57.969 INFO:tasks.workunit.client.0.smithi107.stdout:6/514: write d9/d16/f7f [952974,9279] 0 2024-05-17T21:52:57.976 INFO:tasks.workunit.client.0.smithi107.stdout:7/567: creat d5/df/d11/d1c/d31/d39/fcc x:0 0 0 2024-05-17T21:52:57.977 INFO:tasks.workunit.client.1.smithi138.stdout:2/461: dwrite d3/f74 [0,4194304] 0 2024-05-17T21:52:57.977 INFO:tasks.workunit.client.0.smithi107.stdout:7/568: readlink d5/df/d11/d1d/d33/d82/lb0 0 2024-05-17T21:52:57.981 INFO:tasks.workunit.client.0.smithi107.stdout:3/630: mkdir d6/d8/d10/d35/d36/d4c/dcc 0 2024-05-17T21:52:57.984 INFO:tasks.workunit.client.0.smithi107.stdout:9/617: write d0/f43 [27930,93901] 0 2024-05-17T21:52:57.986 INFO:tasks.workunit.client.0.smithi107.stdout:9/618: write d0/dd/d1b/f74 [965313,72893] 0 2024-05-17T21:52:57.986 INFO:tasks.workunit.client.0.smithi107.stdout:9/619: stat d0 0 2024-05-17T21:52:57.992 INFO:tasks.workunit.client.1.smithi138.stdout:8/354: read d0/f2 [1014714,69674] 0 2024-05-17T21:52:57.992 INFO:tasks.workunit.client.1.smithi138.stdout:4/436: dwrite d0/f7 [0,4194304] 0 2024-05-17T21:52:57.994 INFO:tasks.workunit.client.0.smithi107.stdout:1/652: mkdir d4/df/d14/d2e/dd3 0 2024-05-17T21:52:57.996 INFO:tasks.workunit.client.0.smithi107.stdout:6/515: creat d9/d58/fa1 x:0 0 0 2024-05-17T21:52:57.997 INFO:tasks.workunit.client.0.smithi107.stdout:4/590: link d3/d4a/d47/d60/d3a/d41/l83 d3/d4a/d47/lc5 0 2024-05-17T21:52:57.997 INFO:tasks.workunit.client.0.smithi107.stdout:6/516: fdatasync d9/f4e 0 2024-05-17T21:52:57.998 INFO:tasks.workunit.client.0.smithi107.stdout:4/591: read - d3/d6/d26/d36/f6e zero size 2024-05-17T21:52:58.000 INFO:tasks.workunit.client.0.smithi107.stdout:4/592: chown d3/d6/c23 33241881 1 2024-05-17T21:52:58.001 INFO:tasks.workunit.client.1.smithi138.stdout:2/462: unlink d3/d21/d2f/d3e/c49 0 2024-05-17T21:52:58.003 INFO:tasks.workunit.client.0.smithi107.stdout:3/631: mkdir d6/d2a/d2d/d69/dad/dcd 0 2024-05-17T21:52:58.003 INFO:tasks.workunit.client.1.smithi138.stdout:2/463: read d3/f27 [756483,83377] 0 2024-05-17T21:52:58.005 INFO:tasks.workunit.client.1.smithi138.stdout:2/464: chown d3/d21/d2f/d3e/d63/d7f/d93 905 1 2024-05-17T21:52:58.005 INFO:tasks.workunit.client.1.smithi138.stdout:6/446: mkdir d7/da0 0 2024-05-17T21:52:58.006 INFO:tasks.workunit.client.0.smithi107.stdout:9/620: readlink d0/dd/d21/d29/d57/lc2 0 2024-05-17T21:52:58.006 INFO:tasks.workunit.client.1.smithi138.stdout:6/447: truncate d7/db/d34/f7e 3039 0 2024-05-17T21:52:58.007 INFO:tasks.workunit.client.1.smithi138.stdout:6/448: chown d7/d8/c33 9808113 1 2024-05-17T21:52:58.011 INFO:tasks.workunit.client.0.smithi107.stdout:1/653: mknod d4/df/d4e/cd4 0 2024-05-17T21:52:58.012 INFO:tasks.workunit.client.1.smithi138.stdout:9/385: link d6/d7/d16/f20 d6/d7/d16/d29/d48/d6e/d74/d75/d76/f7f 0 2024-05-17T21:52:58.012 INFO:tasks.workunit.client.0.smithi107.stdout:9/621: dwrite d0/dd/d1b/f74 [0,4194304] 0 2024-05-17T21:52:58.014 INFO:tasks.workunit.client.1.smithi138.stdout:9/386: rename d6/d7 to d6/d7/d16/d29/d48/d6e/d74/d75/d80 22 2024-05-17T21:52:58.014 INFO:tasks.workunit.client.0.smithi107.stdout:9/622: write d0/dd/d21/d29/f8d [1292833,59276] 0 2024-05-17T21:52:58.015 INFO:tasks.workunit.client.1.smithi138.stdout:1/418: link d4/db/d29/d5d/f6f d4/df/d3a/d3c/f88 0 2024-05-17T21:52:58.020 INFO:tasks.workunit.client.0.smithi107.stdout:9/623: dread d0/dd/d21/d29/d3e/d45/d68/d75/f7c [0,4194304] 0 2024-05-17T21:52:58.021 INFO:tasks.workunit.client.1.smithi138.stdout:0/437: dread d3/d3d/f58 [0,4194304] 0 2024-05-17T21:52:58.025 INFO:tasks.workunit.client.0.smithi107.stdout:5/569: write d5/db/d16/d2d/d40/d5a/d7c/d8e/fb0 [1541598,39016] 0 2024-05-17T21:52:58.026 INFO:tasks.workunit.client.1.smithi138.stdout:4/437: truncate d0/d3/d1d/d38/f50 1154227 0 2024-05-17T21:52:58.027 INFO:tasks.workunit.client.0.smithi107.stdout:3/632: mkdir d6/d8/d10/dce 0 2024-05-17T21:52:58.027 INFO:tasks.workunit.client.0.smithi107.stdout:5/570: dread - d5/db/d16/d2d/fa1 zero size 2024-05-17T21:52:58.030 INFO:tasks.workunit.client.1.smithi138.stdout:9/387: creat d6/d7/d16/d29/d38/f81 x:0 0 0 2024-05-17T21:52:58.030 INFO:tasks.workunit.client.0.smithi107.stdout:7/569: dwrite d5/d16/f1e [0,4194304] 0 2024-05-17T21:52:58.036 INFO:tasks.workunit.client.1.smithi138.stdout:3/499: link d1/d4f/l5b d1/d4f/lb1 0 2024-05-17T21:52:58.040 INFO:tasks.workunit.client.1.smithi138.stdout:8/355: write d0/dc/d1a/f2a [1430262,16536] 0 2024-05-17T21:52:58.043 INFO:tasks.workunit.client.1.smithi138.stdout:6/449: mkdir d7/db/d43/da1 0 2024-05-17T21:52:58.044 INFO:tasks.workunit.client.1.smithi138.stdout:2/465: dwrite d3/f4a [0,4194304] 0 2024-05-17T21:52:58.045 INFO:tasks.workunit.client.1.smithi138.stdout:4/438: mkdir d0/d3/d1d/d38/d9c 0 2024-05-17T21:52:58.046 INFO:tasks.workunit.client.1.smithi138.stdout:2/466: write d3/d21/d2f/d3e/d58/d77/f86 [1031451,65530] 0 2024-05-17T21:52:58.049 INFO:tasks.workunit.client.0.smithi107.stdout:4/593: mkdir d3/db/d1e/dc6 0 2024-05-17T21:52:58.050 INFO:tasks.workunit.client.1.smithi138.stdout:9/388: unlink d6/d7/d16/d29/d48/f22 0 2024-05-17T21:52:58.051 INFO:tasks.workunit.client.0.smithi107.stdout:4/594: write d3/d4a/d47/d60/f79 [288779,83327] 0 2024-05-17T21:52:58.051 INFO:tasks.workunit.client.0.smithi107.stdout:8/593: getdents dc/d44/d4b/d52/d70/d76 0 2024-05-17T21:52:58.051 INFO:tasks.workunit.client.1.smithi138.stdout:9/389: write d6/d7/d16/d29/d38/d3c/f4e [49077,23448] 0 2024-05-17T21:52:58.052 INFO:tasks.workunit.client.0.smithi107.stdout:8/594: write dc/f4f [3859688,37039] 0 2024-05-17T21:52:58.054 INFO:tasks.workunit.client.0.smithi107.stdout:5/571: symlink d5/db/d16/d2d/d40/d5a/d7c/lb1 0 2024-05-17T21:52:58.054 INFO:tasks.workunit.client.1.smithi138.stdout:0/438: write d3/d10/f32 [2933474,104399] 0 2024-05-17T21:52:58.055 INFO:tasks.workunit.client.1.smithi138.stdout:3/500: mkdir d1/d3/d16/d3e/d71/db2 0 2024-05-17T21:52:58.056 INFO:tasks.workunit.client.1.smithi138.stdout:3/501: chown d1/d3/d16/d3e/f73 0 1 2024-05-17T21:52:58.057 INFO:tasks.workunit.client.0.smithi107.stdout:7/570: mknod d5/df/d11/d1c/d31/d39/d92/ccd 0 2024-05-17T21:52:58.058 INFO:tasks.workunit.client.0.smithi107.stdout:6/517: truncate d9/d16/d4d/f52 3157019 0 2024-05-17T21:52:58.058 INFO:tasks.workunit.client.0.smithi107.stdout:2/449: link d0/d9/d11/d15/c4b d0/d9/d11/d15/c8c 0 2024-05-17T21:52:58.059 INFO:tasks.workunit.client.0.smithi107.stdout:6/518: fsync d9/d16/d4d/d85/f98 0 2024-05-17T21:52:58.059 INFO:tasks.workunit.client.0.smithi107.stdout:0/677: getdents d3/d24/d6f/db5 0 2024-05-17T21:52:58.063 INFO:tasks.workunit.client.0.smithi107.stdout:9/624: truncate d0/dd/d2f/d87/d89/fae 936869 0 2024-05-17T21:52:58.064 INFO:tasks.workunit.client.1.smithi138.stdout:4/439: creat d0/d1b/d2b/f9d x:0 0 0 2024-05-17T21:52:58.064 INFO:tasks.workunit.client.1.smithi138.stdout:1/419: write d4/fc [4318712,107128] 0 2024-05-17T21:52:58.065 INFO:tasks.workunit.client.1.smithi138.stdout:4/440: dread - d0/d1b/d88/f8b zero size 2024-05-17T21:52:58.068 INFO:tasks.workunit.client.0.smithi107.stdout:8/595: mknod dc/d21/d35/dab/cd9 0 2024-05-17T21:52:58.068 INFO:tasks.workunit.client.1.smithi138.stdout:8/356: dread d0/dc/d1a/f1d [0,4194304] 0 2024-05-17T21:52:58.068 INFO:tasks.workunit.client.0.smithi107.stdout:8/596: dread - dc/fd2 zero size 2024-05-17T21:52:58.069 INFO:tasks.workunit.client.1.smithi138.stdout:8/357: stat d0/dc/d1a/d44/c64 0 2024-05-17T21:52:58.070 INFO:tasks.workunit.client.0.smithi107.stdout:5/572: creat d5/db/d16/d2a/d73/fb2 x:0 0 0 2024-05-17T21:52:58.071 INFO:tasks.workunit.client.0.smithi107.stdout:8/597: truncate dc/d21/d27/d34/d4c/f50 771466 0 2024-05-17T21:52:58.071 INFO:tasks.workunit.client.0.smithi107.stdout:5/573: chown d5/db/d16/d2d/d4d 721555 1 2024-05-17T21:52:58.074 INFO:tasks.workunit.client.0.smithi107.stdout:2/450: creat d0/d26/d64/f8d x:0 0 0 2024-05-17T21:52:58.074 INFO:tasks.workunit.client.1.smithi138.stdout:0/439: mkdir d3/d10/d39/d97 0 2024-05-17T21:52:58.076 INFO:tasks.workunit.client.0.smithi107.stdout:4/595: dread d3/d6/d26/d36/d37/f5f [0,4194304] 0 2024-05-17T21:52:58.076 INFO:tasks.workunit.client.0.smithi107.stdout:0/678: write d3/dc/f16 [2096262,119064] 0 2024-05-17T21:52:58.077 INFO:tasks.workunit.client.1.smithi138.stdout:3/502: rmdir d1/d3/d8d/d92 39 2024-05-17T21:52:58.079 INFO:tasks.workunit.client.1.smithi138.stdout:3/503: write d1/d4f/d80/fac [686081,63785] 0 2024-05-17T21:52:58.079 INFO:tasks.workunit.client.0.smithi107.stdout:6/519: rmdir d9/d16/d4d/d85 39 2024-05-17T21:52:58.081 INFO:tasks.workunit.client.1.smithi138.stdout:0/440: dwrite d3/d10/f32 [0,4194304] 0 2024-05-17T21:52:58.081 INFO:tasks.workunit.client.1.smithi138.stdout:3/504: dread - d1/d3/d16/d2d/fab zero size 2024-05-17T21:52:58.082 INFO:tasks.workunit.client.0.smithi107.stdout:9/625: mkdir d0/dd/d21/d29/d57/dd7 0 2024-05-17T21:52:58.083 INFO:tasks.workunit.client.0.smithi107.stdout:0/679: dwrite d3/dc/d22/d23/f44 [0,4194304] 0 2024-05-17T21:52:58.088 INFO:tasks.workunit.client.1.smithi138.stdout:4/441: creat d0/d2f/d49/d60/f9e x:0 0 0 2024-05-17T21:52:58.091 INFO:tasks.workunit.client.0.smithi107.stdout:0/680: dread d3/dc/f14 [0,4194304] 0 2024-05-17T21:52:58.092 INFO:tasks.workunit.client.0.smithi107.stdout:7/571: dwrite d5/df/d11/d1c/d31/d39/f8d [0,4194304] 0 2024-05-17T21:52:58.094 INFO:tasks.workunit.client.0.smithi107.stdout:4/596: symlink d3/d6/d26/d36/d8b/lc7 0 2024-05-17T21:52:58.094 INFO:tasks.workunit.client.0.smithi107.stdout:7/572: chown d5/df/d11/d1d/d33/f87 143810450 1 2024-05-17T21:52:58.096 INFO:tasks.workunit.client.1.smithi138.stdout:3/505: symlink d1/d4f/d50/lb3 0 2024-05-17T21:52:58.096 INFO:tasks.workunit.client.0.smithi107.stdout:6/520: creat d9/d58/d7b/fa2 x:0 0 0 2024-05-17T21:52:58.097 INFO:tasks.workunit.client.0.smithi107.stdout:9/626: creat d0/dd/d3d/fd8 x:0 0 0 2024-05-17T21:52:58.101 INFO:tasks.workunit.client.1.smithi138.stdout:2/467: rename d3/d21/d2f/d3e/d63/d7f to d3/d21/d2f/d3e/d55/d99 0 2024-05-17T21:52:58.103 INFO:tasks.workunit.client.1.smithi138.stdout:4/442: mkdir d0/d3/d17/d21/d78/d9f 0 2024-05-17T21:52:58.106 INFO:tasks.workunit.client.0.smithi107.stdout:7/573: dwrite d5/df/d11/d1c/d31/d39/d92/d9a/fc2 [0,4194304] 0 2024-05-17T21:52:58.106 INFO:tasks.workunit.client.1.smithi138.stdout:9/390: creat d6/d7/f82 x:0 0 0 2024-05-17T21:52:58.109 INFO:tasks.workunit.client.0.smithi107.stdout:0/681: mkdir d3/d24/d6f/db5/def 0 2024-05-17T21:52:58.109 INFO:tasks.workunit.client.1.smithi138.stdout:6/450: link d7/db/d60/l75 d7/db/d43/la2 0 2024-05-17T21:52:58.109 INFO:tasks.workunit.client.0.smithi107.stdout:4/597: fsync d3/d4a/d47/d60/d3c/f9c 0 2024-05-17T21:52:58.111 INFO:tasks.workunit.client.0.smithi107.stdout:6/521: mkdir d9/d58/d7b/da3 0 2024-05-17T21:52:58.112 INFO:tasks.workunit.client.1.smithi138.stdout:2/468: mknod d3/d21/d2f/d3e/d55/d99/d84/c9a 0 2024-05-17T21:52:58.115 INFO:tasks.workunit.client.1.smithi138.stdout:2/469: dread d3/f6f [0,4194304] 0 2024-05-17T21:52:58.115 INFO:tasks.workunit.client.1.smithi138.stdout:3/506: dwrite d1/d3/d8d/d92/fa6 [0,4194304] 0 2024-05-17T21:52:58.116 INFO:tasks.workunit.client.0.smithi107.stdout:0/682: dwrite d3/dc/fc3 [0,4194304] 0 2024-05-17T21:52:58.116 INFO:tasks.workunit.client.1.smithi138.stdout:9/391: rename d6/d7/d16/d29/d45 to d6/d7/d16/d29/d6c/d83 0 2024-05-17T21:52:58.118 INFO:tasks.workunit.client.1.smithi138.stdout:6/451: symlink d7/db/d43/d5f/la3 0 2024-05-17T21:52:58.122 INFO:tasks.workunit.client.0.smithi107.stdout:6/522: dread d9/d16/f41 [0,4194304] 0 2024-05-17T21:52:58.125 INFO:tasks.workunit.client.0.smithi107.stdout:5/574: getdents d5/db/d16/d2d/d40 0 2024-05-17T21:52:58.128 INFO:tasks.workunit.client.1.smithi138.stdout:6/452: unlink d7/db/d34/f7e 0 2024-05-17T21:52:58.131 INFO:tasks.workunit.client.0.smithi107.stdout:0/683: rename d3/dc/d22/d23/f2c to d3/d24/d6f/db5/def/ff0 0 2024-05-17T21:52:58.131 INFO:tasks.workunit.client.1.smithi138.stdout:8/358: dread d0/f17 [4194304,4194304] 0 2024-05-17T21:52:58.132 INFO:tasks.workunit.client.0.smithi107.stdout:0/684: write d3/dc/d31/f63 [1744337,44786] 0 2024-05-17T21:52:58.133 INFO:tasks.workunit.client.0.smithi107.stdout:4/598: fdatasync d3/d4a/d47/d60/f10 0 2024-05-17T21:52:58.135 INFO:tasks.workunit.client.1.smithi138.stdout:0/441: write d3/d10/d6d/f75 [55272,6984] 0 2024-05-17T21:52:58.136 INFO:tasks.workunit.client.0.smithi107.stdout:9/627: creat d0/dd/d2f/fd9 x:0 0 0 2024-05-17T21:52:58.137 INFO:tasks.workunit.client.0.smithi107.stdout:9/628: chown d0/dd/d1f 1 1 2024-05-17T21:52:58.139 INFO:tasks.workunit.client.1.smithi138.stdout:3/507: rename d1/d4f/d50/f97 to d1/d3/d16/d5d/fb4 0 2024-05-17T21:52:58.143 INFO:tasks.workunit.client.0.smithi107.stdout:7/574: link d5/df/d11/d1d/d33/d82/la5 d5/df/d11/d1c/d31/d39/d92/d9a/lce 0 2024-05-17T21:52:58.144 INFO:tasks.workunit.client.0.smithi107.stdout:7/575: write d5/df/fc9 [904206,66324] 0 2024-05-17T21:52:58.148 INFO:tasks.workunit.client.1.smithi138.stdout:1/420: dwrite d4/f9 [0,4194304] 0 2024-05-17T21:52:58.148 INFO:tasks.workunit.client.1.smithi138.stdout:2/470: creat d3/f9b x:0 0 0 2024-05-17T21:52:58.149 INFO:tasks.workunit.client.1.smithi138.stdout:2/471: stat d3/d21/d2f/f33 0 2024-05-17T21:52:58.149 INFO:tasks.workunit.client.0.smithi107.stdout:8/598: truncate dc/d21/d97/d63/d71/d81/f9b 2474670 0 2024-05-17T21:52:58.150 INFO:tasks.workunit.client.0.smithi107.stdout:8/599: fdatasync dc/d21/d97/d63/d71/f94 0 2024-05-17T21:52:58.151 INFO:tasks.workunit.client.0.smithi107.stdout:8/600: write dc/d21/fd7 [362325,55684] 0 2024-05-17T21:52:58.154 INFO:tasks.workunit.client.0.smithi107.stdout:2/451: truncate d0/d26/f55 714538 0 2024-05-17T21:52:58.155 INFO:tasks.workunit.client.0.smithi107.stdout:2/452: stat d0/d9/d11/d18/f20 0 2024-05-17T21:52:58.155 INFO:tasks.workunit.client.1.smithi138.stdout:8/359: dread d0/dc/f22 [0,4194304] 0 2024-05-17T21:52:58.157 INFO:tasks.workunit.client.1.smithi138.stdout:1/421: dwrite d4/dd/d3b/f73 [0,4194304] 0 2024-05-17T21:52:58.158 INFO:tasks.workunit.client.1.smithi138.stdout:3/508: chown d1/d62/dad/f28 55347268 1 2024-05-17T21:52:58.158 INFO:tasks.workunit.client.1.smithi138.stdout:2/472: dwrite d3/f4 [4194304,4194304] 0 2024-05-17T21:52:58.160 INFO:tasks.workunit.client.1.smithi138.stdout:3/509: write d1/d3/d16/f29 [780552,79932] 0 2024-05-17T21:52:58.161 INFO:tasks.workunit.client.1.smithi138.stdout:1/422: dread d4/db/f2a [0,4194304] 0 2024-05-17T21:52:58.162 INFO:tasks.workunit.client.0.smithi107.stdout:2/453: dwrite d0/d9/d1c/d3e/d4f/f61 [0,4194304] 0 2024-05-17T21:52:58.164 INFO:tasks.workunit.client.0.smithi107.stdout:2/454: readlink d0/d9/d11/df/d5a/d6d/d82/l86 0 2024-05-17T21:52:58.167 INFO:tasks.workunit.client.0.smithi107.stdout:9/629: chown d0/dd/d2f/d87/d89/fae 681 1 2024-05-17T21:52:58.168 INFO:tasks.workunit.client.1.smithi138.stdout:0/442: symlink d3/d10/d38/l98 0 2024-05-17T21:52:58.169 INFO:tasks.workunit.client.0.smithi107.stdout:6/523: truncate d9/f48 847835 0 2024-05-17T21:52:58.170 INFO:tasks.workunit.client.1.smithi138.stdout:4/443: write d0/d3/d1d/d38/f50 [1743633,107292] 0 2024-05-17T21:52:58.173 INFO:tasks.workunit.client.0.smithi107.stdout:5/575: creat d5/d5f/fb3 x:0 0 0 2024-05-17T21:52:58.173 INFO:tasks.workunit.client.0.smithi107.stdout:2/455: dwrite d0/d9/d1c/d3e/d4f/f61 [0,4194304] 0 2024-05-17T21:52:58.174 INFO:tasks.workunit.client.1.smithi138.stdout:8/360: chown d0/d11 4848 1 2024-05-17T21:52:58.174 INFO:tasks.workunit.client.0.smithi107.stdout:5/576: fsync d5/db/d16/d2d/fa1 0 2024-05-17T21:52:58.175 INFO:tasks.workunit.client.0.smithi107.stdout:2/456: write d0/d9/f77 [4535386,61301] 0 2024-05-17T21:52:58.176 INFO:tasks.workunit.client.1.smithi138.stdout:9/392: write d6/d7/d16/f21 [2643448,30923] 0 2024-05-17T21:52:58.177 INFO:tasks.workunit.client.0.smithi107.stdout:7/576: mknod d5/df/d11/d1d/d33/d82/ccf 0 2024-05-17T21:52:58.184 INFO:tasks.workunit.client.0.smithi107.stdout:5/577: dwrite d5/f1b [0,4194304] 0 2024-05-17T21:52:58.184 INFO:tasks.workunit.client.0.smithi107.stdout:8/601: rename dc/d21/d27/d2f/dba/fbd to dc/d21/d35/fda 0 2024-05-17T21:52:58.185 INFO:tasks.workunit.client.1.smithi138.stdout:2/473: creat d3/d21/d2f/d3e/d69/f9c x:0 0 0 2024-05-17T21:52:58.186 INFO:tasks.workunit.client.0.smithi107.stdout:8/602: write dc/f4f [4068782,101487] 0 2024-05-17T21:52:58.187 INFO:tasks.workunit.client.0.smithi107.stdout:5/578: truncate d5/db/d16/d2d/d32/f91 611870 0 2024-05-17T21:52:58.187 INFO:tasks.workunit.client.1.smithi138.stdout:3/510: creat d1/d62/dad/de/fb5 x:0 0 0 2024-05-17T21:52:58.188 INFO:tasks.workunit.client.1.smithi138.stdout:3/511: read - d1/d62/dad/de/f8e zero size 2024-05-17T21:52:58.189 INFO:tasks.workunit.client.0.smithi107.stdout:0/685: creat d3/d24/d6f/d9f/de8/ff1 x:0 0 0 2024-05-17T21:52:58.189 INFO:tasks.workunit.client.1.smithi138.stdout:3/512: chown d1/d4f/d80/fac 1120 1 2024-05-17T21:52:58.190 INFO:tasks.workunit.client.1.smithi138.stdout:3/513: chown d1/d62/dad/c55 1768555 1 2024-05-17T21:52:58.191 INFO:tasks.workunit.client.1.smithi138.stdout:4/444: mkdir d0/d3/db/da0 0 2024-05-17T21:52:58.192 INFO:tasks.workunit.client.1.smithi138.stdout:4/445: fdatasync d0/d2f/d49/d60/d1f/f70 0 2024-05-17T21:52:58.193 INFO:tasks.workunit.client.0.smithi107.stdout:9/630: write d0/dd/d2f/d87/f98 [4935592,90593] 0 2024-05-17T21:52:58.194 INFO:tasks.workunit.client.0.smithi107.stdout:9/631: dread d0/dd/f78 [0,4194304] 0 2024-05-17T21:52:58.197 INFO:tasks.workunit.client.1.smithi138.stdout:9/393: mknod d6/d7/d16/d29/d48/d6e/d74/d75/c84 0 2024-05-17T21:52:58.198 INFO:tasks.workunit.client.1.smithi138.stdout:9/394: chown d6/d7/d16/d29/d48/f34 0 1 2024-05-17T21:52:58.199 INFO:tasks.workunit.client.1.smithi138.stdout:2/474: mkdir d3/d21/d2f/d3e/d58/d9d 0 2024-05-17T21:52:58.200 INFO:tasks.workunit.client.1.smithi138.stdout:9/395: readlink d6/d7/d16/d29/d48/d6e/l5b 0 2024-05-17T21:52:58.200 INFO:tasks.workunit.client.0.smithi107.stdout:9/632: dwrite d0/f43 [0,4194304] 0 2024-05-17T21:52:58.200 INFO:tasks.workunit.client.0.smithi107.stdout:2/457: mkdir d0/d26/d8e 0 2024-05-17T21:52:58.202 INFO:tasks.workunit.client.1.smithi138.stdout:9/396: truncate d6/d7/d16/d29/d38/d40/f6b 738087 0 2024-05-17T21:52:58.205 INFO:tasks.workunit.client.0.smithi107.stdout:6/524: rename d9/d16/d22/d2d/f83 to d9/d16/d22/d5f/d6e/fa4 0 2024-05-17T21:52:58.206 INFO:tasks.workunit.client.1.smithi138.stdout:8/361: rmdir d0/d31 39 2024-05-17T21:52:58.208 INFO:tasks.workunit.client.0.smithi107.stdout:6/525: stat d9/d16/d22/d2d/l36 0 2024-05-17T21:52:58.210 INFO:tasks.workunit.client.1.smithi138.stdout:2/475: write d3/d21/d2f/d3e/d63/f83 [5197514,38777] 0 2024-05-17T21:52:58.212 INFO:tasks.workunit.client.0.smithi107.stdout:5/579: mkdir d5/db/d16/d2d/d40/d5a/db4 0 2024-05-17T21:52:58.213 INFO:tasks.workunit.client.1.smithi138.stdout:3/514: mknod d1/d62/db0/cb6 0 2024-05-17T21:52:58.214 INFO:tasks.workunit.client.0.smithi107.stdout:0/686: write d3/dc/d22/f68 [395823,9663] 0 2024-05-17T21:52:58.215 INFO:tasks.workunit.client.0.smithi107.stdout:0/687: dread - d3/d75/dc1/fc4 zero size 2024-05-17T21:52:58.217 INFO:tasks.workunit.client.0.smithi107.stdout:0/688: fdatasync d3/dc/d22/d23/d97/d85/fb3 0 2024-05-17T21:52:58.217 INFO:tasks.workunit.client.0.smithi107.stdout:0/689: fsync d3/dc/dae/fbb 0 2024-05-17T21:52:58.219 INFO:tasks.workunit.client.0.smithi107.stdout:2/458: read d0/d9/d1c/f73 [648825,24255] 0 2024-05-17T21:52:58.220 INFO:tasks.workunit.client.1.smithi138.stdout:3/515: truncate d1/f65 962537 0 2024-05-17T21:52:58.220 INFO:tasks.workunit.client.1.smithi138.stdout:9/397: getdents d6/d7/d16/d29/d48/d67 0 2024-05-17T21:52:58.221 INFO:tasks.workunit.client.0.smithi107.stdout:7/577: getdents d5/d16/d3b/d45/d91/dc1 0 2024-05-17T21:52:58.221 INFO:tasks.workunit.client.1.smithi138.stdout:3/516: fdatasync d1/d3/d16/d3e/f4a 0 2024-05-17T21:52:58.223 INFO:tasks.workunit.client.1.smithi138.stdout:2/476: mknod d3/d81/d96/c9e 0 2024-05-17T21:52:58.225 INFO:tasks.workunit.client.0.smithi107.stdout:5/580: creat d5/db/dc/d1e/fb5 x:0 0 0 2024-05-17T21:52:58.226 INFO:tasks.workunit.client.1.smithi138.stdout:2/477: dread d3/f4a [0,4194304] 0 2024-05-17T21:52:58.227 INFO:tasks.workunit.client.0.smithi107.stdout:4/599: dwrite d3/db/d35/d5e/f63 [0,4194304] 0 2024-05-17T21:52:58.228 INFO:tasks.workunit.client.0.smithi107.stdout:0/690: dwrite d3/dc/d22/f68 [0,4194304] 0 2024-05-17T21:52:58.229 INFO:tasks.workunit.client.1.smithi138.stdout:9/398: mknod d6/d7/d16/d29/d48/d67/c85 0 2024-05-17T21:52:58.230 INFO:tasks.workunit.client.0.smithi107.stdout:0/691: dread - d3/d75/dc1/fc4 zero size 2024-05-17T21:52:58.235 INFO:tasks.workunit.client.0.smithi107.stdout:5/581: dread d5/db/d39/f43 [4194304,4194304] 0 2024-05-17T21:52:58.238 INFO:tasks.workunit.client.1.smithi138.stdout:8/362: dwrite d0/f17 [0,4194304] 0 2024-05-17T21:52:58.248 INFO:tasks.workunit.client.0.smithi107.stdout:2/459: symlink d0/d9/d11/df/l8f 0 2024-05-17T21:52:58.250 INFO:tasks.workunit.client.1.smithi138.stdout:0/443: write d3/d3d/f6a [1294981,56998] 0 2024-05-17T21:52:58.251 INFO:tasks.workunit.client.1.smithi138.stdout:0/444: write d3/f8e [665424,22570] 0 2024-05-17T21:52:58.254 INFO:tasks.workunit.client.1.smithi138.stdout:3/517: creat d1/d3/d8d/fb7 x:0 0 0 2024-05-17T21:52:58.255 INFO:tasks.workunit.client.1.smithi138.stdout:3/518: stat d1/d62/dad/de 0 2024-05-17T21:52:58.258 INFO:tasks.workunit.client.1.smithi138.stdout:2/478: rmdir d3/d81/d85 0 2024-05-17T21:52:58.258 INFO:tasks.workunit.client.0.smithi107.stdout:4/600: read d3/db/d35/d5e/f96 [828123,36231] 0 2024-05-17T21:52:58.260 INFO:tasks.workunit.client.1.smithi138.stdout:8/363: mknod d0/d2f/c6d 0 2024-05-17T21:52:58.261 INFO:tasks.workunit.client.0.smithi107.stdout:4/601: truncate d3/d4a/d47/d60/d3a/d41/d80/d43/f9e 567257 0 2024-05-17T21:52:58.262 INFO:tasks.workunit.client.1.smithi138.stdout:9/399: dread d6/d7/d16/f21 [0,4194304] 0 2024-05-17T21:52:58.262 INFO:tasks.workunit.client.1.smithi138.stdout:0/445: rmdir d3/d10/d38/d70/d80 39 2024-05-17T21:52:58.263 INFO:tasks.workunit.client.1.smithi138.stdout:9/400: stat d6/d7/d16/d29/d48 0 2024-05-17T21:52:58.265 INFO:tasks.workunit.client.1.smithi138.stdout:3/519: rename d1/f4 to d1/d62/fb8 0 2024-05-17T21:52:58.266 INFO:tasks.workunit.client.1.smithi138.stdout:3/520: readlink d1/d62/dad/d72/l8a 0 2024-05-17T21:52:58.266 INFO:tasks.workunit.client.1.smithi138.stdout:3/521: chown d1/d3/d16/d3a/c76 9395 1 2024-05-17T21:52:58.267 INFO:tasks.workunit.client.1.smithi138.stdout:2/479: truncate d3/f27 1739766 0 2024-05-17T21:52:58.268 INFO:tasks.workunit.client.1.smithi138.stdout:2/480: chown d3/d21/d2f/d3e/d55/d99/d93/f79 8 1 2024-05-17T21:52:58.269 INFO:tasks.workunit.client.1.smithi138.stdout:8/364: creat d0/d2f/f6e x:0 0 0 2024-05-17T21:52:58.271 INFO:tasks.workunit.client.1.smithi138.stdout:0/446: unlink d3/l92 0 2024-05-17T21:52:58.274 INFO:tasks.workunit.client.1.smithi138.stdout:9/401: unlink d6/d7/d16/d29/d48/d3b/c58 0 2024-05-17T21:52:58.274 INFO:tasks.workunit.client.1.smithi138.stdout:2/481: dwrite d3/f9b [0,4194304] 0 2024-05-17T21:52:58.278 INFO:tasks.workunit.client.1.smithi138.stdout:3/522: mknod d1/d4f/d80/cb9 0 2024-05-17T21:52:58.283 INFO:tasks.workunit.client.1.smithi138.stdout:8/365: dread d0/dc/d1a/f2a [0,4194304] 0 2024-05-17T21:52:58.287 INFO:tasks.workunit.client.0.smithi107.stdout:7/578: truncate d5/df/d11/d1c/d31/d39/f85 1464981 0 2024-05-17T21:52:58.288 INFO:tasks.workunit.client.1.smithi138.stdout:9/402: mkdir d6/d7/d16/d29/d48/d67/d86 0 2024-05-17T21:52:58.290 INFO:tasks.workunit.client.1.smithi138.stdout:2/482: mkdir d3/d21/d2f/d3e/d55/d99/d84/d9f 0 2024-05-17T21:52:58.294 INFO:tasks.workunit.client.1.smithi138.stdout:9/403: dwrite d6/f33 [4194304,4194304] 0 2024-05-17T21:52:58.295 INFO:tasks.workunit.client.1.smithi138.stdout:9/404: chown d6/d7/d16/c19 90 1 2024-05-17T21:52:58.296 INFO:tasks.workunit.client.0.smithi107.stdout:0/692: creat d3/dc/d22/ff2 x:0 0 0 2024-05-17T21:52:58.299 INFO:tasks.workunit.client.0.smithi107.stdout:5/582: write d5/f9 [4760194,43534] 0 2024-05-17T21:52:58.300 INFO:tasks.workunit.client.0.smithi107.stdout:9/633: link d0/dd/d2f/d87/cb9 d0/dd/d2f/cda 0 2024-05-17T21:52:58.302 INFO:tasks.workunit.client.1.smithi138.stdout:0/447: truncate d3/d3d/d48/f5f 139914 0 2024-05-17T21:52:58.303 INFO:tasks.workunit.client.0.smithi107.stdout:7/579: creat d5/d16/d3b/d45/fd0 x:0 0 0 2024-05-17T21:52:58.303 INFO:tasks.workunit.client.0.smithi107.stdout:7/580: chown d5/d16/d3b/d45/f6c 1656 1 2024-05-17T21:52:58.304 INFO:tasks.workunit.client.1.smithi138.stdout:9/405: dwrite d6/d7/d16/d29/d48/d6f/f71 [0,4194304] 0 2024-05-17T21:52:58.306 INFO:tasks.workunit.client.1.smithi138.stdout:3/523: mknod d1/d62/dad/cba 0 2024-05-17T21:52:58.308 INFO:tasks.workunit.client.1.smithi138.stdout:3/524: dread - d1/d3/d16/fa2 zero size 2024-05-17T21:52:58.309 INFO:tasks.workunit.client.1.smithi138.stdout:3/525: chown d1/d62/dad/f24 271 1 2024-05-17T21:52:58.310 INFO:tasks.workunit.client.0.smithi107.stdout:8/603: truncate dc/d21/d97/d63/d71/d81/f9b 3225218 0 2024-05-17T21:52:58.311 INFO:tasks.workunit.client.0.smithi107.stdout:0/693: fdatasync d3/d39/f41 0 2024-05-17T21:52:58.312 INFO:tasks.workunit.client.0.smithi107.stdout:1/654: sync 2024-05-17T21:52:58.313 INFO:tasks.workunit.client.1.smithi138.stdout:0/448: stat d3/d10/d38/d70/d80/f8b 0 2024-05-17T21:52:58.314 INFO:tasks.workunit.client.0.smithi107.stdout:7/581: write d5/df/d78/f95 [2626881,34140] 0 2024-05-17T21:52:58.316 INFO:tasks.workunit.client.0.smithi107.stdout:2/460: getdents d0 0 2024-05-17T21:52:58.317 INFO:tasks.workunit.client.1.smithi138.stdout:9/406: write d6/f1b [1762987,21039] 0 2024-05-17T21:52:58.317 INFO:tasks.workunit.client.0.smithi107.stdout:2/461: truncate d0/f7d 309041 0 2024-05-17T21:52:58.318 INFO:tasks.workunit.client.1.smithi138.stdout:3/526: rmdir d1 39 2024-05-17T21:52:58.319 INFO:tasks.workunit.client.0.smithi107.stdout:2/462: chown d0/d9/d1c/d3e/d4f/f6a 2 1 2024-05-17T21:52:58.323 INFO:tasks.workunit.client.0.smithi107.stdout:5/583: rename d5/l49 to d5/db/dc/d1e/d6e/lb6 0 2024-05-17T21:52:58.324 INFO:tasks.workunit.client.1.smithi138.stdout:2/483: rmdir d3/d21/d2f/d3e/d58/d9d 0 2024-05-17T21:52:58.325 INFO:tasks.workunit.client.0.smithi107.stdout:9/634: symlink d0/d80/dad/ldb 0 2024-05-17T21:52:58.325 INFO:tasks.workunit.client.0.smithi107.stdout:4/602: truncate d3/db/f46 1995231 0 2024-05-17T21:52:58.328 INFO:tasks.workunit.client.0.smithi107.stdout:8/604: mkdir dc/ddb 0 2024-05-17T21:52:58.329 INFO:tasks.workunit.client.0.smithi107.stdout:8/605: write dc/d21/d97/f15 [2730611,53238] 0 2024-05-17T21:52:58.330 INFO:tasks.workunit.client.1.smithi138.stdout:0/449: symlink d3/l99 0 2024-05-17T21:52:58.330 INFO:tasks.workunit.client.1.smithi138.stdout:8/366: write d0/dc/d1a/f3f [2149242,36895] 0 2024-05-17T21:52:58.330 INFO:tasks.workunit.client.0.smithi107.stdout:2/463: mknod d0/d9/d11/d27/c90 0 2024-05-17T21:52:58.331 INFO:tasks.workunit.client.0.smithi107.stdout:8/606: read dc/d21/f80 [528955,101607] 0 2024-05-17T21:52:58.333 INFO:tasks.workunit.client.0.smithi107.stdout:0/694: symlink d3/d24/lf3 0 2024-05-17T21:52:58.333 INFO:tasks.workunit.client.0.smithi107.stdout:2/464: write d0/d9/d1c/d3e/f6b [4209860,30144] 0 2024-05-17T21:52:58.334 INFO:tasks.workunit.client.1.smithi138.stdout:8/367: dread d0/d2f/f41 [0,4194304] 0 2024-05-17T21:52:58.334 INFO:tasks.workunit.client.1.smithi138.stdout:9/407: link d6/d7/d16/d29/d38/f81 d6/d7/d16/d29/d38/d40/d44/f87 0 2024-05-17T21:52:58.335 INFO:tasks.workunit.client.1.smithi138.stdout:8/368: dread d0/d2f/f41 [0,4194304] 0 2024-05-17T21:52:58.337 INFO:tasks.workunit.client.1.smithi138.stdout:3/527: truncate d1/d3/d16/d3e/d71/fa9 1432949 0 2024-05-17T21:52:58.338 INFO:tasks.workunit.client.0.smithi107.stdout:9/635: write d0/f10 [1436327,2793] 0 2024-05-17T21:52:58.339 INFO:tasks.workunit.client.0.smithi107.stdout:9/636: readlink d0/dd/d1f/l38 0 2024-05-17T21:52:58.342 INFO:tasks.workunit.client.0.smithi107.stdout:4/603: creat d3/d4a/d47/d60/d3a/d41/fc8 x:0 0 0 2024-05-17T21:52:58.342 INFO:tasks.workunit.client.0.smithi107.stdout:7/582: dread d5/df/d78/f95 [0,4194304] 0 2024-05-17T21:52:58.343 INFO:tasks.workunit.client.1.smithi138.stdout:0/450: symlink d3/d10/d38/d70/d80/l9a 0 2024-05-17T21:52:58.344 INFO:tasks.workunit.client.1.smithi138.stdout:0/451: truncate d3/d3d/f6a 1386547 0 2024-05-17T21:52:58.346 INFO:tasks.workunit.client.1.smithi138.stdout:2/484: mkdir d3/da0 0 2024-05-17T21:52:58.347 INFO:tasks.workunit.client.1.smithi138.stdout:2/485: write d3/d21/d2f/d3e/f6d [1413234,120320] 0 2024-05-17T21:52:58.350 INFO:tasks.workunit.client.1.smithi138.stdout:8/369: symlink d0/dc/d1a/d33/l6f 0 2024-05-17T21:52:58.351 INFO:tasks.workunit.client.1.smithi138.stdout:0/452: dwrite d3/d10/f20 [4194304,4194304] 0 2024-05-17T21:52:58.354 INFO:tasks.workunit.client.0.smithi107.stdout:8/607: creat dc/d44/fdc x:0 0 0 2024-05-17T21:52:58.354 INFO:tasks.workunit.client.0.smithi107.stdout:8/608: readlink dc/d21/d97/l5e 0 2024-05-17T21:52:58.356 INFO:tasks.workunit.client.1.smithi138.stdout:3/528: chown d1/ca 1236789 1 2024-05-17T21:52:58.356 INFO:tasks.workunit.client.0.smithi107.stdout:5/584: write d5/db/d16/d54/d59/d65/f27 [3583942,101211] 0 2024-05-17T21:52:58.360 INFO:tasks.workunit.client.0.smithi107.stdout:9/637: dread d0/dd/d1f/d2c/f47 [0,4194304] 0 2024-05-17T21:52:58.361 INFO:tasks.workunit.client.0.smithi107.stdout:9/638: truncate d0/dd/d2f/fd9 493329 0 2024-05-17T21:52:58.365 INFO:tasks.workunit.client.1.smithi138.stdout:8/370: symlink d0/dc/d1a/l70 0 2024-05-17T21:52:58.368 INFO:tasks.workunit.client.1.smithi138.stdout:9/408: dwrite d6/d7/f2c [0,4194304] 0 2024-05-17T21:52:58.369 INFO:tasks.workunit.client.1.smithi138.stdout:3/529: mkdir d1/d4f/d50/dbb 0 2024-05-17T21:52:58.372 INFO:tasks.workunit.client.1.smithi138.stdout:0/453: mkdir d3/d10/d38/d3e/d59/d9b 0 2024-05-17T21:52:58.374 INFO:tasks.workunit.client.0.smithi107.stdout:5/585: dread - d5/db/d16/d54/d59/f7f zero size 2024-05-17T21:52:58.378 INFO:tasks.workunit.client.0.smithi107.stdout:0/695: truncate d3/dc/d22/d23/d97/d85/fb3 1291354 0 2024-05-17T21:52:58.382 INFO:tasks.workunit.client.0.smithi107.stdout:4/604: mknod d3/d6/d26/d36/d7e/d98/cc9 0 2024-05-17T21:52:58.383 INFO:tasks.workunit.client.0.smithi107.stdout:4/605: chown d3/d4a/d47/d60/d3a/d41/d80 9400 1 2024-05-17T21:52:58.383 INFO:tasks.workunit.client.1.smithi138.stdout:8/371: dwrite d0/d2f/f41 [0,4194304] 0 2024-05-17T21:52:58.384 INFO:tasks.workunit.client.0.smithi107.stdout:7/583: rename d5/df/d11/d1c/d2f/d65/la4 to d5/df/d11/d1d/dc4/ld1 0 2024-05-17T21:52:58.386 INFO:tasks.workunit.client.0.smithi107.stdout:8/609: mknod dc/dd5/cdd 0 2024-05-17T21:52:58.388 INFO:tasks.workunit.client.0.smithi107.stdout:2/465: rmdir d0/d9/d1c/d22/d88 0 2024-05-17T21:52:58.391 INFO:tasks.workunit.client.1.smithi138.stdout:9/409: dwrite d6/d7/d16/d29/d48/d6e/d74/d75/d76/f7f [0,4194304] 0 2024-05-17T21:52:58.391 INFO:tasks.workunit.client.0.smithi107.stdout:2/466: chown d0/d9/d1c/d3e/f62 15518 1 2024-05-17T21:52:58.392 INFO:tasks.workunit.client.0.smithi107.stdout:7/584: dwrite d5/df/d11/d1c/f2e [0,4194304] 0 2024-05-17T21:52:58.392 INFO:tasks.workunit.client.0.smithi107.stdout:5/586: creat d5/db/d16/d2d/d40/d5a/d7c/fb7 x:0 0 0 2024-05-17T21:52:58.393 INFO:tasks.workunit.client.0.smithi107.stdout:2/467: chown d0/d9/d11/d27/c32 1693 1 2024-05-17T21:52:58.396 INFO:tasks.workunit.client.0.smithi107.stdout:0/696: chown d3/dc/d22/cbe 929423 1 2024-05-17T21:52:58.398 INFO:tasks.workunit.client.1.smithi138.stdout:9/410: dread d6/d7/d16/f21 [0,4194304] 0 2024-05-17T21:52:58.399 INFO:tasks.workunit.client.0.smithi107.stdout:9/639: link d0/dd/d1b/fcb d0/dd/d21/d29/d3e/d45/d68/fdc 0 2024-05-17T21:52:58.400 INFO:tasks.workunit.client.1.smithi138.stdout:8/372: mkdir d0/d71 0 2024-05-17T21:52:58.402 INFO:tasks.workunit.client.1.smithi138.stdout:0/454: getdents d3/d10/d38 0 2024-05-17T21:52:58.403 INFO:tasks.workunit.client.0.smithi107.stdout:4/606: symlink d3/db/d1e/d99/lca 0 2024-05-17T21:52:58.404 INFO:tasks.workunit.client.1.smithi138.stdout:9/411: dwrite d6/d7/d16/d29/d48/f23 [0,4194304] 0 2024-05-17T21:52:58.406 INFO:tasks.workunit.client.0.smithi107.stdout:0/697: dwrite d3/dc/d22/d3d/f4d [0,4194304] 0 2024-05-17T21:52:58.407 INFO:tasks.workunit.client.1.smithi138.stdout:8/373: write d0/d11/f69 [544449,82648] 0 2024-05-17T21:52:58.411 INFO:tasks.workunit.client.1.smithi138.stdout:0/455: dwrite d3/d10/d38/f46 [0,4194304] 0 2024-05-17T21:52:58.412 INFO:tasks.workunit.client.1.smithi138.stdout:0/456: stat d3/d10/d38/d3e/d59/f68 0 2024-05-17T21:52:58.413 INFO:tasks.workunit.client.1.smithi138.stdout:0/457: rename d3/d10 to d3/d10/d39/d9c 22 2024-05-17T21:52:58.423 INFO:tasks.workunit.client.1.smithi138.stdout:0/458: creat d3/d10/d38/d3e/d59/f9d x:0 0 0 2024-05-17T21:52:58.426 INFO:tasks.workunit.client.0.smithi107.stdout:9/640: symlink d0/dd/d21/d29/d3e/d45/ldd 0 2024-05-17T21:52:58.428 INFO:tasks.workunit.client.1.smithi138.stdout:3/530: dwrite d1/fa5 [0,4194304] 0 2024-05-17T21:52:58.429 INFO:tasks.workunit.client.0.smithi107.stdout:2/468: dread d0/d9/d1c/d3e/f62 [0,4194304] 0 2024-05-17T21:52:58.430 INFO:tasks.workunit.client.1.smithi138.stdout:3/531: fsync d1/d3/d16/d2d/fab 0 2024-05-17T21:52:58.432 INFO:tasks.workunit.client.1.smithi138.stdout:8/374: getdents d0/dc/d3b 0 2024-05-17T21:52:58.435 INFO:tasks.workunit.client.0.smithi107.stdout:9/641: chown d0/dd/d21/d29/d57/d5f/lb5 66 1 2024-05-17T21:52:58.436 INFO:tasks.workunit.client.0.smithi107.stdout:9/642: stat d0/dd/d21/d29/d3e/d45/ldd 0 2024-05-17T21:52:58.437 INFO:tasks.workunit.client.1.smithi138.stdout:8/375: creat d0/dc/d1a/d33/d4a/d4e/f72 x:0 0 0 2024-05-17T21:52:58.437 INFO:tasks.workunit.client.0.smithi107.stdout:3/633: sync 2024-05-17T21:52:58.438 INFO:tasks.workunit.client.1.smithi138.stdout:8/376: readlink d0/dc/l25 0 2024-05-17T21:52:58.440 INFO:tasks.workunit.client.1.smithi138.stdout:3/532: dwrite d1/d62/dad/de/f27 [0,4194304] 0 2024-05-17T21:52:58.441 INFO:tasks.workunit.client.0.smithi107.stdout:3/634: fsync d6/d2a/d2d/f43 0 2024-05-17T21:52:58.441 INFO:tasks.workunit.client.0.smithi107.stdout:7/585: creat d5/df/d11/d1c/d2f/fd2 x:0 0 0 2024-05-17T21:52:58.441 INFO:tasks.workunit.client.1.smithi138.stdout:8/377: truncate d0/dc/d3b/f43 497540 0 2024-05-17T21:52:58.441 INFO:tasks.workunit.client.1.smithi138.stdout:8/378: readlink d0/d11/l27 0 2024-05-17T21:52:58.442 INFO:tasks.workunit.client.1.smithi138.stdout:3/533: truncate d1/d3/f20 4616808 0 2024-05-17T21:52:58.442 INFO:tasks.workunit.client.1.smithi138.stdout:3/534: stat d1/d3/d16/f29 0 2024-05-17T21:52:58.443 INFO:tasks.workunit.client.0.smithi107.stdout:4/607: dread d3/d4a/f73 [0,4194304] 0 2024-05-17T21:52:58.445 INFO:tasks.workunit.client.0.smithi107.stdout:9/643: dwrite d0/dd/d2f/d87/d89/fd3 [0,4194304] 0 2024-05-17T21:52:58.447 INFO:tasks.workunit.client.0.smithi107.stdout:8/610: write dc/d21/d27/d2f/f30 [1970151,90087] 0 2024-05-17T21:52:58.449 INFO:tasks.workunit.client.1.smithi138.stdout:8/379: chown d0/d31/c38 34 1 2024-05-17T21:52:58.450 INFO:tasks.workunit.client.0.smithi107.stdout:0/698: getdents d3/d58 0 2024-05-17T21:52:58.451 INFO:tasks.workunit.client.1.smithi138.stdout:9/412: write d6/d7/d16/d29/d38/d40/d44/f87 [754437,16442] 0 2024-05-17T21:52:58.455 INFO:tasks.workunit.client.0.smithi107.stdout:5/587: dwrite d5/db/d16/d2d/d4d/d6d/f77 [0,4194304] 0 2024-05-17T21:52:58.456 INFO:tasks.workunit.client.0.smithi107.stdout:7/586: mkdir d5/d16/dd3 0 2024-05-17T21:52:58.459 INFO:tasks.workunit.client.1.smithi138.stdout:3/535: unlink d1/d3/d16/d3a/l66 0 2024-05-17T21:52:58.460 INFO:tasks.workunit.client.1.smithi138.stdout:3/536: chown d1/d3/d16/d2d 9027905 1 2024-05-17T21:52:58.461 INFO:tasks.workunit.client.0.smithi107.stdout:8/611: dread dc/f67 [0,4194304] 0 2024-05-17T21:52:58.461 INFO:tasks.workunit.client.1.smithi138.stdout:0/459: write d3/f29 [8294635,73931] 0 2024-05-17T21:52:58.461 INFO:tasks.workunit.client.1.smithi138.stdout:9/413: dwrite d6/d7/d16/d29/d48/f23 [0,4194304] 0 2024-05-17T21:52:58.463 INFO:tasks.workunit.client.0.smithi107.stdout:8/612: truncate dc/d44/d4b/d52/f96 4263047 0 2024-05-17T21:52:58.464 INFO:tasks.workunit.client.0.smithi107.stdout:4/608: mknod d3/d6/ccb 0 2024-05-17T21:52:58.465 INFO:tasks.workunit.client.1.smithi138.stdout:9/414: read d6/d7/d16/d29/f37 [2305960,33382] 0 2024-05-17T21:52:58.468 INFO:tasks.workunit.client.1.smithi138.stdout:0/460: creat d3/d10/d6d/d81/f9e x:0 0 0 2024-05-17T21:52:58.471 INFO:tasks.workunit.client.1.smithi138.stdout:0/461: read d3/d10/f25 [5004841,84686] 0 2024-05-17T21:52:58.473 INFO:tasks.workunit.client.1.smithi138.stdout:0/462: readlink d3/d10/l4c 0 2024-05-17T21:52:58.473 INFO:tasks.workunit.client.1.smithi138.stdout:9/415: dwrite d6/f25 [0,4194304] 0 2024-05-17T21:52:58.474 INFO:tasks.workunit.client.1.smithi138.stdout:9/416: fsync d6/d7/f9 0 2024-05-17T21:52:58.476 INFO:tasks.workunit.client.0.smithi107.stdout:7/587: mknod d5/df/d11/d1c/d2f/d65/cd4 0 2024-05-17T21:52:58.476 INFO:tasks.workunit.client.0.smithi107.stdout:2/469: dwrite d0/d9/f46 [0,4194304] 0 2024-05-17T21:52:58.477 INFO:tasks.workunit.client.1.smithi138.stdout:9/417: dread d6/d7/d16/d29/d38/d40/f6b [0,4194304] 0 2024-05-17T21:52:58.478 INFO:tasks.workunit.client.0.smithi107.stdout:2/470: fdatasync d0/d9/d11/f5f 0 2024-05-17T21:52:58.479 INFO:tasks.workunit.client.0.smithi107.stdout:8/613: dread - dc/d21/d97/d63/f99 zero size 2024-05-17T21:52:58.480 INFO:tasks.workunit.client.1.smithi138.stdout:9/418: write d6/d7/d16/d29/d48/d6e/d74/d75/d76/f7f [2853550,43031] 0 2024-05-17T21:52:58.480 INFO:tasks.workunit.client.0.smithi107.stdout:8/614: stat dc/d21/d27/d34/f38 0 2024-05-17T21:52:58.481 INFO:tasks.workunit.client.1.smithi138.stdout:9/419: chown d6/d7/d16/d29/d48/d6e 4467484 1 2024-05-17T21:52:58.482 INFO:tasks.workunit.client.1.smithi138.stdout:3/537: dread d1/d3/f1c [0,4194304] 0 2024-05-17T21:52:58.484 INFO:tasks.workunit.client.1.smithi138.stdout:8/380: truncate d0/dc/d1a/f4b 4050136 0 2024-05-17T21:52:58.486 INFO:tasks.workunit.client.1.smithi138.stdout:9/420: truncate d6/d7/d16/d29/d6c/d83/f53 1308009 0 2024-05-17T21:52:58.486 INFO:tasks.workunit.client.0.smithi107.stdout:4/609: creat d3/d4a/d47/d60/d3c/fcc x:0 0 0 2024-05-17T21:52:58.489 INFO:tasks.workunit.client.0.smithi107.stdout:3/635: write d6/d8/f14 [1487082,19064] 0 2024-05-17T21:52:58.490 INFO:tasks.workunit.client.0.smithi107.stdout:3/636: chown d6/d2a/d2d/da8 8529 1 2024-05-17T21:52:58.494 INFO:tasks.workunit.client.1.smithi138.stdout:8/381: rmdir d0/d11 39 2024-05-17T21:52:58.495 INFO:tasks.workunit.client.1.smithi138.stdout:8/382: dread d0/d2f/f45 [0,4194304] 0 2024-05-17T21:52:58.495 INFO:tasks.workunit.client.1.smithi138.stdout:9/421: creat d6/d7/d16/d29/d48/d3b/f88 x:0 0 0 2024-05-17T21:52:58.496 INFO:tasks.workunit.client.0.smithi107.stdout:7/588: creat d5/df/d11/d1d/d33/d82/fd5 x:0 0 0 2024-05-17T21:52:58.497 INFO:tasks.workunit.client.0.smithi107.stdout:3/637: dwrite d6/d2a/f6a [0,4194304] 0 2024-05-17T21:52:58.498 INFO:tasks.workunit.client.1.smithi138.stdout:3/538: link d1/d4f/d50/f54 d1/d4f/d87/fbc 0 2024-05-17T21:52:58.498 INFO:tasks.workunit.client.1.smithi138.stdout:3/539: stat d1/d3/f10 0 2024-05-17T21:52:58.499 INFO:tasks.workunit.client.0.smithi107.stdout:2/471: write d0/d9/d11/df/d5a/d6d/f74 [92164,46745] 0 2024-05-17T21:52:58.501 INFO:tasks.workunit.client.1.smithi138.stdout:8/383: creat d0/dc/d1a/d51/f73 x:0 0 0 2024-05-17T21:52:58.504 INFO:tasks.workunit.client.0.smithi107.stdout:0/699: creat d3/dc/ff4 x:0 0 0 2024-05-17T21:52:58.506 INFO:tasks.workunit.client.0.smithi107.stdout:0/700: truncate d3/dc/d31/f3a 1741299 0 2024-05-17T21:52:58.506 INFO:tasks.workunit.client.1.smithi138.stdout:5/423: sync 2024-05-17T21:52:58.506 INFO:tasks.workunit.client.1.smithi138.stdout:7/348: sync 2024-05-17T21:52:58.509 INFO:tasks.workunit.client.0.smithi107.stdout:5/588: creat d5/fb8 x:0 0 0 2024-05-17T21:52:58.510 INFO:tasks.workunit.client.1.smithi138.stdout:0/463: dread d3/f2b [0,4194304] 0 2024-05-17T21:52:58.512 INFO:tasks.workunit.client.1.smithi138.stdout:9/422: dread d6/d7/fc [0,4194304] 0 2024-05-17T21:52:58.514 INFO:tasks.workunit.client.0.smithi107.stdout:5/589: dread d5/db/d16/d2d/d40/d5a/d7c/d8e/fb0 [0,4194304] 0 2024-05-17T21:52:58.515 INFO:tasks.workunit.client.0.smithi107.stdout:7/589: rmdir d5/d16/d3b/d45/d91/dac 39 2024-05-17T21:52:58.515 INFO:tasks.workunit.client.0.smithi107.stdout:5/590: write d5/db/dc/d1e/fb5 [455296,58287] 0 2024-05-17T21:52:58.516 INFO:tasks.workunit.client.1.smithi138.stdout:8/384: mknod d0/dc/d1a/c74 0 2024-05-17T21:52:58.516 INFO:tasks.workunit.client.1.smithi138.stdout:0/464: dwrite d3/fc [4194304,4194304] 0 2024-05-17T21:52:58.518 INFO:tasks.workunit.client.1.smithi138.stdout:0/465: write d3/f45 [2616112,18523] 0 2024-05-17T21:52:58.519 INFO:tasks.workunit.client.0.smithi107.stdout:3/638: creat d6/d2a/d2d/d48/d4f/fcf x:0 0 0 2024-05-17T21:52:58.522 INFO:tasks.workunit.client.0.smithi107.stdout:5/591: dwrite d5/db/dc/f56 [0,4194304] 0 2024-05-17T21:52:58.522 INFO:tasks.workunit.client.0.smithi107.stdout:8/615: symlink dc/d21/daa/dc5/lde 0 2024-05-17T21:52:58.523 INFO:tasks.workunit.client.0.smithi107.stdout:7/590: dwrite d5/d16/d3b/d45/fd0 [0,4194304] 0 2024-05-17T21:52:58.524 INFO:tasks.workunit.client.0.smithi107.stdout:8/616: dread - dc/d21/fd3 zero size 2024-05-17T21:52:58.525 INFO:tasks.workunit.client.0.smithi107.stdout:2/472: mknod d0/d9/d11/df/d5a/c91 0 2024-05-17T21:52:58.526 INFO:tasks.workunit.client.1.smithi138.stdout:7/349: rename d0/f3b to d0/d4c/f6d 0 2024-05-17T21:52:58.528 INFO:tasks.workunit.client.0.smithi107.stdout:0/701: rmdir d3/d24/d6f/d9f 39 2024-05-17T21:52:58.528 INFO:tasks.workunit.client.0.smithi107.stdout:0/702: chown d3/d58/d61/d94/la6 9804986 1 2024-05-17T21:52:58.529 INFO:tasks.workunit.client.1.smithi138.stdout:1/423: sync 2024-05-17T21:52:58.530 INFO:tasks.workunit.client.1.smithi138.stdout:4/446: sync 2024-05-17T21:52:58.530 INFO:tasks.workunit.client.1.smithi138.stdout:6/453: sync 2024-05-17T21:52:58.530 INFO:tasks.workunit.client.1.smithi138.stdout:2/486: sync 2024-05-17T21:52:58.535 INFO:tasks.workunit.client.0.smithi107.stdout:0/703: dwrite d3/dc/d22/d23/f8f [0,4194304] 0 2024-05-17T21:52:58.535 INFO:tasks.workunit.client.1.smithi138.stdout:9/423: symlink d6/d7/d16/d29/d38/d3c/l89 0 2024-05-17T21:52:58.536 INFO:tasks.workunit.client.1.smithi138.stdout:6/454: fsync d7/d69/f71 0 2024-05-17T21:52:58.536 INFO:tasks.workunit.client.0.smithi107.stdout:0/704: fsync d3/d75/d9e/fea 0 2024-05-17T21:52:58.541 INFO:tasks.workunit.client.1.smithi138.stdout:8/385: dread d0/dc/d1a/f2a [0,4194304] 0 2024-05-17T21:52:58.541 INFO:tasks.workunit.client.1.smithi138.stdout:0/466: mkdir d3/d10/d6d/d9f 0 2024-05-17T21:52:58.542 INFO:tasks.workunit.client.1.smithi138.stdout:3/540: dwrite d1/d62/dad/f64 [0,4194304] 0 2024-05-17T21:52:58.543 INFO:tasks.workunit.client.1.smithi138.stdout:8/386: readlink d0/dc/d1a/d33/l5e 0 2024-05-17T21:52:58.544 INFO:tasks.workunit.client.1.smithi138.stdout:5/424: mknod d1/d6/c8c 0 2024-05-17T21:52:58.544 INFO:tasks.workunit.client.0.smithi107.stdout:2/473: dread - d0/d9/d11/d27/f43 zero size 2024-05-17T21:52:58.547 INFO:tasks.workunit.client.1.smithi138.stdout:0/467: dread d3/d10/d39/f72 [0,4194304] 0 2024-05-17T21:52:58.548 INFO:tasks.workunit.client.0.smithi107.stdout:3/639: rename l0 to d6/d1a/db1/ld0 0 2024-05-17T21:52:58.549 INFO:tasks.workunit.client.1.smithi138.stdout:3/541: write d1/d62/dad/f24 [957872,17769] 0 2024-05-17T21:52:58.551 INFO:tasks.workunit.client.0.smithi107.stdout:0/705: read d3/dc/f26 [3591932,90402] 0 2024-05-17T21:52:58.553 INFO:tasks.workunit.client.0.smithi107.stdout:0/706: chown d3/dc/d22/d3d/dc7/fcc 242774 1 2024-05-17T21:52:58.554 INFO:tasks.workunit.client.1.smithi138.stdout:9/424: dwrite d6/d7/d16/d29/d48/d6e/d5a/f64 [0,4194304] 0 2024-05-17T21:52:58.556 INFO:tasks.workunit.client.0.smithi107.stdout:2/474: write d0/d9/f33 [1324434,954] 0 2024-05-17T21:52:58.559 INFO:tasks.workunit.client.1.smithi138.stdout:0/468: dwrite d3/f84 [0,4194304] 0 2024-05-17T21:52:58.563 INFO:tasks.workunit.client.1.smithi138.stdout:6/455: mkdir d7/db/d43/d5f/da4 0 2024-05-17T21:52:58.564 INFO:tasks.workunit.client.0.smithi107.stdout:3/640: dread - d6/d2a/d2d/d37/f7c zero size 2024-05-17T21:52:58.573 INFO:tasks.workunit.client.0.smithi107.stdout:2/475: creat d0/d9/d11/df/d5a/d6d/d82/f92 x:0 0 0 2024-05-17T21:52:58.573 INFO:tasks.workunit.client.0.smithi107.stdout:1/655: sync 2024-05-17T21:52:58.573 INFO:tasks.workunit.client.0.smithi107.stdout:6/526: sync 2024-05-17T21:52:58.573 INFO:tasks.workunit.client.0.smithi107.stdout:8/617: write dc/d21/d27/d34/f6f [1657870,108809] 0 2024-05-17T21:52:58.573 INFO:tasks.workunit.client.1.smithi138.stdout:5/425: mknod d1/d60/c8d 0 2024-05-17T21:52:58.575 INFO:tasks.workunit.client.1.smithi138.stdout:8/387: dwrite d0/d11/f69 [0,4194304] 0 2024-05-17T21:52:58.575 INFO:tasks.workunit.client.0.smithi107.stdout:3/641: creat d6/d8/d10/d35/d36/fd1 x:0 0 0 2024-05-17T21:52:58.576 INFO:tasks.workunit.client.0.smithi107.stdout:7/591: dwrite d5/df/d11/d1c/d2f/f9f [0,4194304] 0 2024-05-17T21:52:58.576 INFO:tasks.workunit.client.1.smithi138.stdout:8/388: chown d0/dc/d1a/f1d 8 1 2024-05-17T21:52:58.577 INFO:tasks.workunit.client.0.smithi107.stdout:8/618: chown dc/d21/d27/d34/d4c/d7e/l74 242 1 2024-05-17T21:52:58.577 INFO:tasks.workunit.client.0.smithi107.stdout:3/642: chown d6/c5a 7 1 2024-05-17T21:52:58.577 INFO:tasks.workunit.client.1.smithi138.stdout:8/389: read d0/dc/d1a/f24 [8023277,25588] 0 2024-05-17T21:52:58.579 INFO:tasks.workunit.client.0.smithi107.stdout:8/619: chown dc/d21/d35/dab/lb0 7999 1 2024-05-17T21:52:58.581 INFO:tasks.workunit.client.0.smithi107.stdout:8/620: write dc/d44/d4b/d52/f89 [4380715,2035] 0 2024-05-17T21:52:58.584 INFO:tasks.workunit.client.0.smithi107.stdout:5/592: rename d5/db/dc/f56 to d5/db/d16/d2d/d40/d5a/db4/fb9 0 2024-05-17T21:52:58.588 INFO:tasks.workunit.client.0.smithi107.stdout:2/476: mkdir d0/d93 0 2024-05-17T21:52:58.589 INFO:tasks.workunit.client.1.smithi138.stdout:9/425: creat d6/d7/d16/f8a x:0 0 0 2024-05-17T21:52:58.592 INFO:tasks.workunit.client.1.smithi138.stdout:7/350: truncate d0/d18/d40/f44 872942 0 2024-05-17T21:52:58.592 INFO:tasks.workunit.client.0.smithi107.stdout:3/643: write f3 [7235870,64619] 0 2024-05-17T21:52:58.594 INFO:tasks.workunit.client.1.smithi138.stdout:6/456: mknod d7/db/d43/d5f/ca5 0 2024-05-17T21:52:58.596 INFO:tasks.workunit.client.1.smithi138.stdout:6/457: chown d7/d69 118 1 2024-05-17T21:52:58.596 INFO:tasks.workunit.client.0.smithi107.stdout:7/592: creat d5/d16/d3b/d45/d91/db3/fd6 x:0 0 0 2024-05-17T21:52:58.599 INFO:tasks.workunit.client.1.smithi138.stdout:2/487: dwrite d3/f29 [0,4194304] 0 2024-05-17T21:52:58.600 INFO:tasks.workunit.client.0.smithi107.stdout:8/621: symlink dc/d21/d97/ldf 0 2024-05-17T21:52:58.600 INFO:tasks.workunit.client.1.smithi138.stdout:7/351: dwrite d0/f12 [4194304,4194304] 0 2024-05-17T21:52:58.603 INFO:tasks.workunit.client.1.smithi138.stdout:8/390: readlink d0/d11/d3e/l4f 0 2024-05-17T21:52:58.605 INFO:tasks.workunit.client.0.smithi107.stdout:5/593: mkdir d5/db/d16/d54/d59/d92/dba 0 2024-05-17T21:52:58.605 INFO:tasks.workunit.client.0.smithi107.stdout:6/527: dread d9/d16/d22/d2d/f39 [0,4194304] 0 2024-05-17T21:52:58.606 INFO:tasks.workunit.client.1.smithi138.stdout:3/542: symlink d1/d3/d16/lbd 0 2024-05-17T21:52:58.610 INFO:tasks.workunit.client.1.smithi138.stdout:4/447: truncate d0/d2f/d49/d60/d1f/d32/f87 2322869 0 2024-05-17T21:52:58.612 INFO:tasks.workunit.client.1.smithi138.stdout:3/543: dread d1/d3/f1c [0,4194304] 0 2024-05-17T21:52:58.612 INFO:tasks.workunit.client.1.smithi138.stdout:4/448: write d0/d1b/d2b/d55/f7d [779632,93317] 0 2024-05-17T21:52:58.616 INFO:tasks.workunit.client.0.smithi107.stdout:1/656: link d4/df/d1a/d26/d50/fa0 d4/d78/fd5 0 2024-05-17T21:52:58.617 INFO:tasks.workunit.client.0.smithi107.stdout:0/707: rename d3/dc/d22/d3d/c4f to d3/d75/dd0/cf5 0 2024-05-17T21:52:58.620 INFO:tasks.workunit.client.0.smithi107.stdout:5/594: creat d5/db/d16/d54/d59/d65/fbb x:0 0 0 2024-05-17T21:52:58.622 INFO:tasks.workunit.client.1.smithi138.stdout:7/352: creat d0/d18/d64/f6e x:0 0 0 2024-05-17T21:52:58.623 INFO:tasks.workunit.client.1.smithi138.stdout:7/353: write d0/d18/f1d [3507440,110993] 0 2024-05-17T21:52:58.623 INFO:tasks.workunit.client.0.smithi107.stdout:7/593: mkdir d5/df/dd7 0 2024-05-17T21:52:58.625 INFO:tasks.workunit.client.1.smithi138.stdout:8/391: write d0/f1b [2238102,29082] 0 2024-05-17T21:52:58.626 INFO:tasks.workunit.client.1.smithi138.stdout:1/424: getdents d4/d2e 0 2024-05-17T21:52:58.627 INFO:tasks.workunit.client.0.smithi107.stdout:6/528: dread d9/f1f [0,4194304] 0 2024-05-17T21:52:58.627 INFO:tasks.workunit.client.0.smithi107.stdout:7/594: write d5/df/d11/d1c/d31/d39/f5b [1779451,102500] 0 2024-05-17T21:52:58.627 INFO:tasks.workunit.client.0.smithi107.stdout:5/595: dwrite d5/fb8 [0,4194304] 0 2024-05-17T21:52:58.630 INFO:tasks.workunit.client.0.smithi107.stdout:0/708: mkdir d3/d75/dd0/df6 0 2024-05-17T21:52:58.631 INFO:tasks.workunit.client.1.smithi138.stdout:0/469: truncate d3/d10/d38/f46 2925210 0 2024-05-17T21:52:58.632 INFO:tasks.workunit.client.1.smithi138.stdout:0/470: chown d3/d10/c2e 470296805 1 2024-05-17T21:52:58.632 INFO:tasks.workunit.client.1.smithi138.stdout:3/544: mknod d1/d4f/d80/cbe 0 2024-05-17T21:52:58.634 INFO:tasks.workunit.client.0.smithi107.stdout:6/529: dwrite d9/d16/f5e [0,4194304] 0 2024-05-17T21:52:58.635 INFO:tasks.workunit.client.1.smithi138.stdout:4/449: mkdir d0/d2f/d49/d60/d1f/da1 0 2024-05-17T21:52:58.636 INFO:tasks.workunit.client.1.smithi138.stdout:4/450: fdatasync d0/d2f/d49/d60/d1f/d7e/f83 0 2024-05-17T21:52:58.637 INFO:tasks.workunit.client.0.smithi107.stdout:0/709: dread d3/dc/d22/d23/f44 [0,4194304] 0 2024-05-17T21:52:58.638 INFO:tasks.workunit.client.0.smithi107.stdout:0/710: dread - d3/d75/dc1/fc4 zero size 2024-05-17T21:52:58.640 INFO:tasks.workunit.client.0.smithi107.stdout:2/477: read d0/d9/f60 [3769208,111442] 0 2024-05-17T21:52:58.640 INFO:tasks.workunit.client.1.smithi138.stdout:2/488: mknod d3/ca1 0 2024-05-17T21:52:58.642 INFO:tasks.workunit.client.0.smithi107.stdout:1/657: mknod d4/cd6 0 2024-05-17T21:52:58.647 INFO:tasks.workunit.client.1.smithi138.stdout:2/489: dwrite d3/d21/d2f/d3e/d69/f9c [0,4194304] 0 2024-05-17T21:52:58.649 INFO:tasks.workunit.client.1.smithi138.stdout:2/490: write d3/d21/d2f/f45 [3137911,90099] 0 2024-05-17T21:52:58.649 INFO:tasks.workunit.client.1.smithi138.stdout:2/491: chown d3 930 1 2024-05-17T21:52:58.655 INFO:tasks.workunit.client.1.smithi138.stdout:0/471: truncate d3/d3d/f53 5443829 0 2024-05-17T21:52:58.655 INFO:tasks.workunit.client.0.smithi107.stdout:7/595: dwrite d5/fe [0,4194304] 0 2024-05-17T21:52:58.657 INFO:tasks.workunit.client.1.smithi138.stdout:6/458: write d7/fa [1455564,104497] 0 2024-05-17T21:52:58.657 INFO:tasks.workunit.client.1.smithi138.stdout:6/459: stat d7/c6e 0 2024-05-17T21:52:58.659 INFO:tasks.workunit.client.0.smithi107.stdout:6/530: unlink d9/d16/d22/d2d/f44 0 2024-05-17T21:52:58.660 INFO:tasks.workunit.client.0.smithi107.stdout:6/531: dread - d9/d58/fa1 zero size 2024-05-17T21:52:58.660 INFO:tasks.workunit.client.1.smithi138.stdout:4/451: symlink d0/d2f/la2 0 2024-05-17T21:52:58.662 INFO:tasks.workunit.client.0.smithi107.stdout:3/644: getdents d6/d1a 0 2024-05-17T21:52:58.664 INFO:tasks.workunit.client.0.smithi107.stdout:0/711: fdatasync d3/d39/f49 0 2024-05-17T21:52:58.666 INFO:tasks.workunit.client.0.smithi107.stdout:1/658: creat d4/df/d1a/d26/d72/d85/fd7 x:0 0 0 2024-05-17T21:52:58.669 INFO:tasks.workunit.client.0.smithi107.stdout:4/610: sync 2024-05-17T21:52:58.669 INFO:tasks.workunit.client.0.smithi107.stdout:9/644: sync 2024-05-17T21:52:58.669 INFO:tasks.workunit.client.0.smithi107.stdout:5/596: rename d5/db/dc/d1e/d6e/lb6 to d5/db/d16/d2a/d73/lbc 0 2024-05-17T21:52:58.673 INFO:tasks.workunit.client.1.smithi138.stdout:7/354: dwrite d0/d4c/f6d [0,4194304] 0 2024-05-17T21:52:58.676 INFO:tasks.workunit.client.0.smithi107.stdout:6/532: dread d9/d16/d22/d2d/f37 [0,4194304] 0 2024-05-17T21:52:58.677 INFO:tasks.workunit.client.0.smithi107.stdout:6/533: read - d9/d16/d22/f92 zero size 2024-05-17T21:52:58.680 INFO:tasks.workunit.client.1.smithi138.stdout:2/492: creat d3/d21/d2f/d8a/fa2 x:0 0 0 2024-05-17T21:52:58.681 INFO:tasks.workunit.client.1.smithi138.stdout:2/493: chown d3/da0 16394507 1 2024-05-17T21:52:58.681 INFO:tasks.workunit.client.1.smithi138.stdout:8/392: dwrite d0/fb [0,4194304] 0 2024-05-17T21:52:58.682 INFO:tasks.workunit.client.1.smithi138.stdout:9/426: getdents d6/d7/d16/d29/d38 0 2024-05-17T21:52:58.685 INFO:tasks.workunit.client.1.smithi138.stdout:7/355: dwrite d0/d4c/f6d [0,4194304] 0 2024-05-17T21:52:58.685 INFO:tasks.workunit.client.1.smithi138.stdout:9/427: write d6/d7/d16/f8a [401954,99630] 0 2024-05-17T21:52:58.689 INFO:tasks.workunit.client.1.smithi138.stdout:9/428: chown d6/d7/d16/l18 0 1 2024-05-17T21:52:58.690 INFO:tasks.workunit.client.1.smithi138.stdout:2/494: dread d3/f9b [0,4194304] 0 2024-05-17T21:52:58.695 INFO:tasks.workunit.client.1.smithi138.stdout:4/452: unlink d0/d3/la 0 2024-05-17T21:52:58.701 INFO:tasks.workunit.client.1.smithi138.stdout:1/425: rename d4/dd/d3b/d40 to d4/db/d29/d89 0 2024-05-17T21:52:58.704 INFO:tasks.workunit.client.1.smithi138.stdout:3/545: write d1/d3/f11 [1209600,33601] 0 2024-05-17T21:52:58.706 INFO:tasks.workunit.client.1.smithi138.stdout:7/356: readlink d0/l7 0 2024-05-17T21:52:58.706 INFO:tasks.workunit.client.0.smithi107.stdout:1/659: creat d4/df/d1a/d26/d7a/fd8 x:0 0 0 2024-05-17T21:52:58.706 INFO:tasks.workunit.client.0.smithi107.stdout:5/597: unlink d5/db/d16/d5b/f85 0 2024-05-17T21:52:58.707 INFO:tasks.workunit.client.0.smithi107.stdout:6/534: symlink d9/d58/d66/la5 0 2024-05-17T21:52:58.707 INFO:tasks.workunit.client.0.smithi107.stdout:7/596: write d5/df/d11/d1c/f54 [1023297,15609] 0 2024-05-17T21:52:58.709 INFO:tasks.workunit.client.0.smithi107.stdout:5/598: write d5/f9 [4261903,15262] 0 2024-05-17T21:52:58.711 INFO:tasks.workunit.client.0.smithi107.stdout:5/599: dread - d5/db/d16/d2a/fab zero size 2024-05-17T21:52:58.712 INFO:tasks.workunit.client.1.smithi138.stdout:2/495: mkdir d3/d21/d2f/d3e/d63/da3 0 2024-05-17T21:52:58.712 INFO:tasks.workunit.client.1.smithi138.stdout:2/496: chown d3/d21/d2f/d3e/d55/l60 6095 1 2024-05-17T21:52:58.713 INFO:tasks.workunit.client.0.smithi107.stdout:2/478: dwrite d0/f36 [0,4194304] 0 2024-05-17T21:52:58.717 INFO:tasks.workunit.client.1.smithi138.stdout:6/460: fsync d7/d69/f90 0 2024-05-17T21:52:58.717 INFO:tasks.workunit.client.1.smithi138.stdout:6/461: dread - d7/db/d43/d3f/d79/f6d zero size 2024-05-17T21:52:58.718 INFO:tasks.workunit.client.1.smithi138.stdout:4/453: rmdir d0/d2f/d49/d60/d1f 39 2024-05-17T21:52:58.719 INFO:tasks.workunit.client.1.smithi138.stdout:6/462: read d7/db/d43/d3f/f4a [2262193,2727] 0 2024-05-17T21:52:58.722 INFO:tasks.workunit.client.1.smithi138.stdout:8/393: rename d0/dc/d1a/f24 to d0/dc/d3b/f75 0 2024-05-17T21:52:58.723 INFO:tasks.workunit.client.1.smithi138.stdout:8/394: write d0/d2f/f6e [163330,79859] 0 2024-05-17T21:52:58.727 INFO:tasks.workunit.client.1.smithi138.stdout:3/546: mknod d1/d3/d49/cbf 0 2024-05-17T21:52:58.729 INFO:tasks.workunit.client.1.smithi138.stdout:9/429: dwrite d6/d7/d16/d29/d38/d40/f6b [0,4194304] 0 2024-05-17T21:52:58.732 INFO:tasks.workunit.client.1.smithi138.stdout:7/357: creat d0/d18/d40/d41/d51/d66/f6f x:0 0 0 2024-05-17T21:52:58.734 INFO:tasks.workunit.client.0.smithi107.stdout:5/600: dread d5/db/d16/f4b [0,4194304] 0 2024-05-17T21:52:58.734 INFO:tasks.workunit.client.1.smithi138.stdout:2/497: unlink d3/d21/d2f/d3e/d55/d99/d84/f94 0 2024-05-17T21:52:58.741 INFO:tasks.workunit.client.0.smithi107.stdout:5/601: dwrite d5/db/d16/d2d/d32/f91 [0,4194304] 0 2024-05-17T21:52:58.746 INFO:tasks.workunit.client.1.smithi138.stdout:0/472: read d3/d3d/f78 [2812690,100625] 0 2024-05-17T21:52:58.750 INFO:tasks.workunit.client.0.smithi107.stdout:0/712: symlink d3/d75/lf7 0 2024-05-17T21:52:58.751 INFO:tasks.workunit.client.1.smithi138.stdout:6/463: rename d7/db/d12 to d7/db/d34/da6 0 2024-05-17T21:52:58.752 INFO:tasks.workunit.client.0.smithi107.stdout:4/611: creat d3/d4a/d47/d60/d3a/d41/d80/dad/fcd x:0 0 0 2024-05-17T21:52:58.752 INFO:tasks.workunit.client.0.smithi107.stdout:4/612: fsync d3/fa 0 2024-05-17T21:52:58.754 INFO:tasks.workunit.client.1.smithi138.stdout:8/395: mknod d0/dc/d3b/c76 0 2024-05-17T21:52:58.757 INFO:tasks.workunit.client.0.smithi107.stdout:3/645: link d6/l7d d6/d8/d10/d35/d91/da6/ld2 0 2024-05-17T21:52:58.759 INFO:tasks.workunit.client.0.smithi107.stdout:1/660: creat d4/df/d1a/d41/db2/fd9 x:0 0 0 2024-05-17T21:52:58.760 INFO:tasks.workunit.client.0.smithi107.stdout:7/597: fsync d5/df/f10 0 2024-05-17T21:52:58.763 INFO:tasks.workunit.client.1.smithi138.stdout:2/498: mkdir d3/d21/d2f/d3e/d58/d77/da4 0 2024-05-17T21:52:58.766 INFO:tasks.workunit.client.0.smithi107.stdout:2/479: dread - d0/f40 zero size 2024-05-17T21:52:58.769 INFO:tasks.workunit.client.0.smithi107.stdout:8/622: sync 2024-05-17T21:52:58.769 INFO:tasks.workunit.client.1.smithi138.stdout:6/464: creat d7/d69/fa7 x:0 0 0 2024-05-17T21:52:58.771 INFO:tasks.workunit.client.0.smithi107.stdout:1/661: dread d4/df/d14/d1d/f27 [0,4194304] 0 2024-05-17T21:52:58.771 INFO:tasks.workunit.client.0.smithi107.stdout:5/602: rename d5/db/d16/d2d/d40/f71 to d5/db/d16/d54/d59/fbd 0 2024-05-17T21:52:58.773 INFO:tasks.workunit.client.0.smithi107.stdout:0/713: write d3/d24/d6f/d9f/de8/ff1 [996450,119610] 0 2024-05-17T21:52:58.774 INFO:tasks.workunit.client.1.smithi138.stdout:8/396: creat d0/dc/d3b/d4d/f77 x:0 0 0 2024-05-17T21:52:58.776 INFO:tasks.workunit.client.1.smithi138.stdout:3/547: mkdir d1/dc0 0 2024-05-17T21:52:58.777 INFO:tasks.workunit.client.1.smithi138.stdout:3/548: chown d1/d3/d49/d4b/l94 5 1 2024-05-17T21:52:58.778 INFO:tasks.workunit.client.1.smithi138.stdout:3/549: fsync d1/d3/d49/d4b/f85 0 2024-05-17T21:52:58.778 INFO:tasks.workunit.client.0.smithi107.stdout:4/613: unlink d3/d6/d26/d36/d7e/faf 0 2024-05-17T21:52:58.780 INFO:tasks.workunit.client.0.smithi107.stdout:6/535: mknod d9/d16/ca6 0 2024-05-17T21:52:58.781 INFO:tasks.workunit.client.0.smithi107.stdout:6/536: write d9/f53 [116355,24316] 0 2024-05-17T21:52:58.783 INFO:tasks.workunit.client.0.smithi107.stdout:3/646: mknod d6/d8/d3e/d3f/d90/cd3 0 2024-05-17T21:52:58.784 INFO:tasks.workunit.client.1.smithi138.stdout:6/465: dread d7/db/d43/d3f/d79/f55 [0,4194304] 0 2024-05-17T21:52:58.785 INFO:tasks.workunit.client.1.smithi138.stdout:6/466: write d7/db/f82 [516515,24740] 0 2024-05-17T21:52:58.791 INFO:tasks.workunit.client.0.smithi107.stdout:2/480: mknod d0/d9/d11/df/d5a/d6d/d82/c94 0 2024-05-17T21:52:58.792 INFO:tasks.workunit.client.1.smithi138.stdout:1/426: getdents d4/dd/d64 0 2024-05-17T21:52:58.793 INFO:tasks.workunit.client.1.smithi138.stdout:1/427: read - d4/db/d29/d5d/f86 zero size 2024-05-17T21:52:58.795 INFO:tasks.workunit.client.0.smithi107.stdout:8/623: rename dc/d21/d97/f26 to dc/d21/daa/db4/fe0 0 2024-05-17T21:52:58.796 INFO:tasks.workunit.client.1.smithi138.stdout:8/397: mkdir d0/dc/d1a/d78 0 2024-05-17T21:52:58.797 INFO:tasks.workunit.client.0.smithi107.stdout:5/603: mkdir d5/db/d16/d2d/d32/dbe 0 2024-05-17T21:52:58.800 INFO:tasks.workunit.client.1.smithi138.stdout:3/550: rmdir d1/d2e 39 2024-05-17T21:52:58.805 INFO:tasks.workunit.client.0.smithi107.stdout:4/614: symlink d3/d6/d26/d36/d7e/lce 0 2024-05-17T21:52:58.807 INFO:tasks.workunit.client.0.smithi107.stdout:6/537: truncate d9/d16/d22/d2d/f2f 3204492 0 2024-05-17T21:52:58.808 INFO:tasks.workunit.client.0.smithi107.stdout:6/538: write f5 [2654084,97233] 0 2024-05-17T21:52:58.809 INFO:tasks.workunit.client.0.smithi107.stdout:3/647: symlink d6/d8/d3e/ld4 0 2024-05-17T21:52:58.812 INFO:tasks.workunit.client.0.smithi107.stdout:2/481: creat d0/d9/d1c/d3e/d4f/f95 x:0 0 0 2024-05-17T21:52:58.813 INFO:tasks.workunit.client.1.smithi138.stdout:9/430: truncate d6/d7/d16/f1a 7942 0 2024-05-17T21:52:58.820 INFO:tasks.workunit.client.1.smithi138.stdout:1/428: dwrite d4/df/d3a/d3c/d67/f85 [0,4194304] 0 2024-05-17T21:52:58.824 INFO:tasks.workunit.client.1.smithi138.stdout:4/454: write d0/d2f/d49/d60/d1f/f4d [531751,65621] 0 2024-05-17T21:52:58.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:58 smithi138 ceph-mon[35948]: pgmap v37: 65 pgs: 65 active+clean; 1.9 GiB data, 7.0 GiB used, 529 GiB / 536 GiB avail; 67 MiB/s rd, 229 MiB/s wr, 429 op/s 2024-05-17T21:52:58.826 INFO:tasks.workunit.client.0.smithi107.stdout:0/714: unlink d3/dc/d22/d23/d97/d77/l78 0 2024-05-17T21:52:58.826 INFO:tasks.workunit.client.1.smithi138.stdout:7/358: getdents d0/d18/d64 0 2024-05-17T21:52:58.827 INFO:tasks.workunit.client.1.smithi138.stdout:1/429: dread d4/df/d3a/d3c/d67/f85 [0,4194304] 0 2024-05-17T21:52:58.828 INFO:tasks.workunit.client.1.smithi138.stdout:7/359: write d0/d18/d40/d41/d50/f59 [2816476,63704] 0 2024-05-17T21:52:58.829 INFO:tasks.workunit.client.0.smithi107.stdout:4/615: unlink d3/d4a/d47/d60/d3a/d41/d80/d43/f9e 0 2024-05-17T21:52:58.831 INFO:tasks.workunit.client.1.smithi138.stdout:0/473: link d3/d10/l7e d3/d10/d38/d70/la0 0 2024-05-17T21:52:58.832 INFO:tasks.workunit.client.1.smithi138.stdout:0/474: chown d3/d10/f71 8419561 1 2024-05-17T21:52:58.833 INFO:tasks.workunit.client.1.smithi138.stdout:0/475: chown d3/d10/d6d/f75 3711907 1 2024-05-17T21:52:58.838 INFO:tasks.workunit.client.0.smithi107.stdout:1/662: truncate d4/df/f29 867901 0 2024-05-17T21:52:58.838 INFO:tasks.workunit.client.1.smithi138.stdout:8/398: unlink d0/dc/d3b/f75 0 2024-05-17T21:52:58.839 INFO:tasks.workunit.client.1.smithi138.stdout:3/551: dread - d1/d4f/d50/f9e zero size 2024-05-17T21:52:58.842 INFO:tasks.workunit.client.0.smithi107.stdout:9/645: sync 2024-05-17T21:52:58.843 INFO:tasks.workunit.client.0.smithi107.stdout:5/604: rename d5/db/d16/d54/d59/d92/ca9 to d5/db/d16/d2d/d40/cbf 0 2024-05-17T21:52:58.843 INFO:tasks.workunit.client.0.smithi107.stdout:9/646: chown d0/dd/d21/d29/d57/d5f/l86 1 1 2024-05-17T21:52:58.846 INFO:tasks.workunit.client.1.smithi138.stdout:4/455: readlink d0/d2f/d49/d60/l74 0 2024-05-17T21:52:58.847 INFO:tasks.workunit.client.1.smithi138.stdout:1/430: chown d4/db/d60/l65 37 1 2024-05-17T21:52:58.848 INFO:tasks.workunit.client.1.smithi138.stdout:1/431: write d4/dd/d3b/d6c/f78 [880363,90623] 0 2024-05-17T21:52:58.849 INFO:tasks.workunit.client.1.smithi138.stdout:1/432: chown d4/dd/d3b/f73 0 1 2024-05-17T21:52:58.851 INFO:tasks.workunit.client.0.smithi107.stdout:3/648: dwrite d6/d8/f76 [0,4194304] 0 2024-05-17T21:52:58.852 INFO:tasks.workunit.client.1.smithi138.stdout:1/433: read d4/dd/f2d [2481489,101011] 0 2024-05-17T21:52:58.852 INFO:tasks.workunit.client.1.smithi138.stdout:7/360: symlink d0/d18/d40/l70 0 2024-05-17T21:52:58.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:58 smithi107 ceph-mon[28434]: pgmap v37: 65 pgs: 65 active+clean; 1.9 GiB data, 7.0 GiB used, 529 GiB / 536 GiB avail; 67 MiB/s rd, 229 MiB/s wr, 429 op/s 2024-05-17T21:52:58.854 INFO:tasks.workunit.client.0.smithi107.stdout:6/539: link d9/d58/f9b d9/d16/d22/d5f/d6f/fa7 0 2024-05-17T21:52:58.854 INFO:tasks.workunit.client.1.smithi138.stdout:6/467: link c0 d7/db/d43/d63/ca8 0 2024-05-17T21:52:58.855 INFO:tasks.workunit.client.0.smithi107.stdout:6/540: fsync d9/d16/d22/d5f/f75 0 2024-05-17T21:52:58.855 INFO:tasks.workunit.client.0.smithi107.stdout:3/649: read d6/d8/d10/f23 [5604011,30812] 0 2024-05-17T21:52:58.857 INFO:tasks.workunit.client.1.smithi138.stdout:0/476: fdatasync d3/d3d/f44 0 2024-05-17T21:52:58.857 INFO:tasks.workunit.client.0.smithi107.stdout:0/715: write d3/d75/d9e/f83 [1897878,44936] 0 2024-05-17T21:52:58.858 INFO:tasks.workunit.client.0.smithi107.stdout:1/663: creat d4/df/d1a/d26/d72/d85/fda x:0 0 0 2024-05-17T21:52:58.861 INFO:tasks.workunit.client.0.smithi107.stdout:8/624: creat dc/d21/d27/d34/fe1 x:0 0 0 2024-05-17T21:52:58.862 INFO:tasks.workunit.client.1.smithi138.stdout:8/399: truncate d0/dc/f2b 1237837 0 2024-05-17T21:52:58.864 INFO:tasks.workunit.client.0.smithi107.stdout:9/647: write d0/dd/f78 [2325812,72117] 0 2024-05-17T21:52:58.864 INFO:tasks.workunit.client.1.smithi138.stdout:3/552: symlink d1/d3/d8d/d92/lc1 0 2024-05-17T21:52:58.871 INFO:tasks.workunit.client.1.smithi138.stdout:6/468: symlink d7/d8/la9 0 2024-05-17T21:52:58.872 INFO:tasks.workunit.client.0.smithi107.stdout:6/541: symlink d9/d16/d22/d5f/la8 0 2024-05-17T21:52:58.873 INFO:tasks.workunit.client.0.smithi107.stdout:3/650: symlink d6/d2a/d2d/d69/dad/ld5 0 2024-05-17T21:52:58.876 INFO:tasks.workunit.client.1.smithi138.stdout:5/426: sync 2024-05-17T21:52:58.876 INFO:tasks.workunit.client.0.smithi107.stdout:2/482: creat d0/d9/d1c/f96 x:0 0 0 2024-05-17T21:52:58.877 INFO:tasks.workunit.client.0.smithi107.stdout:2/483: fsync d0/d9/d11/f52 0 2024-05-17T21:52:58.880 INFO:tasks.workunit.client.1.smithi138.stdout:4/456: mknod d0/d1b/d42/d45/d84/ca3 0 2024-05-17T21:52:58.882 INFO:tasks.workunit.client.1.smithi138.stdout:7/361: unlink d0/c3c 0 2024-05-17T21:52:58.884 INFO:tasks.workunit.client.0.smithi107.stdout:9/648: creat d0/dd/d69/fde x:0 0 0 2024-05-17T21:52:58.885 INFO:tasks.workunit.client.1.smithi138.stdout:1/434: symlink d4/db/d29/l8a 0 2024-05-17T21:52:58.887 INFO:tasks.workunit.client.0.smithi107.stdout:5/605: creat d5/db/d16/d54/da0/fc0 x:0 0 0 2024-05-17T21:52:58.890 INFO:tasks.workunit.client.0.smithi107.stdout:4/616: link d3/d6/d26/d36/d8b/fbb d3/db/d35/d5e/fcf 0 2024-05-17T21:52:58.893 INFO:tasks.workunit.client.1.smithi138.stdout:5/427: stat d1/ld 0 2024-05-17T21:52:58.894 INFO:tasks.workunit.client.1.smithi138.stdout:9/431: getdents d6/d7/d16/d29/d48/d6e/d74 0 2024-05-17T21:52:58.896 INFO:tasks.workunit.client.0.smithi107.stdout:0/716: dwrite d3/dc/d22/d23/f44 [4194304,4194304] 0 2024-05-17T21:52:58.896 INFO:tasks.workunit.client.0.smithi107.stdout:4/617: chown d3/db/d1e/db5/cb7 1 1 2024-05-17T21:52:58.896 INFO:tasks.workunit.client.0.smithi107.stdout:4/618: readlink d3/l15 0 2024-05-17T21:52:58.897 INFO:tasks.workunit.client.0.smithi107.stdout:4/619: fsync d3/db/f33 0 2024-05-17T21:52:58.899 INFO:tasks.workunit.client.0.smithi107.stdout:1/664: mknod d4/df/d1a/d26/d72/dbc/cdb 0 2024-05-17T21:52:58.901 INFO:tasks.workunit.client.0.smithi107.stdout:8/625: mknod dc/d21/d27/d34/d4c/ce2 0 2024-05-17T21:52:58.901 INFO:tasks.workunit.client.0.smithi107.stdout:1/665: chown d4/df/d14/d2e/d43/d47/l89 5791907 1 2024-05-17T21:52:58.903 INFO:tasks.workunit.client.0.smithi107.stdout:1/666: write d4/df/d1a/d26/fcf [58451,101766] 0 2024-05-17T21:52:58.904 INFO:tasks.workunit.client.0.smithi107.stdout:9/649: write d0/dd/d2f/d4a/fd5 [1642559,36813] 0 2024-05-17T21:52:58.904 INFO:tasks.workunit.client.1.smithi138.stdout:7/362: rename d0/d39 to d0/d39/d71 22 2024-05-17T21:52:58.907 INFO:tasks.workunit.client.1.smithi138.stdout:3/553: truncate d1/d3/d49/f7d 2973852 0 2024-05-17T21:52:58.909 INFO:tasks.workunit.client.1.smithi138.stdout:0/477: creat d3/fa1 x:0 0 0 2024-05-17T21:52:58.911 INFO:tasks.workunit.client.1.smithi138.stdout:9/432: mkdir d6/d7/d16/d29/d38/d8b 0 2024-05-17T21:52:58.913 INFO:tasks.workunit.client.1.smithi138.stdout:8/400: link d0/dc/d1a/d51/f73 d0/dc/f79 0 2024-05-17T21:52:58.914 INFO:tasks.workunit.client.1.smithi138.stdout:8/401: chown d0/d2f/f41 0 1 2024-05-17T21:52:58.915 INFO:tasks.workunit.client.1.smithi138.stdout:4/457: mknod d0/d3/ca4 0 2024-05-17T21:52:58.919 INFO:tasks.workunit.client.1.smithi138.stdout:3/554: symlink d1/d62/dad/d5c/d9c/lc2 0 2024-05-17T21:52:58.920 INFO:tasks.workunit.client.1.smithi138.stdout:6/469: link d7/d8/c54 d7/db/d43/d5f/d6a/d9b/caa 0 2024-05-17T21:52:58.920 INFO:tasks.workunit.client.1.smithi138.stdout:3/555: chown d1/d3/d16/d3e/f4e 7 1 2024-05-17T21:52:58.925 INFO:tasks.workunit.client.1.smithi138.stdout:9/433: mknod d6/d7/d16/c8c 0 2024-05-17T21:52:58.927 INFO:tasks.workunit.client.1.smithi138.stdout:4/458: creat d0/d3/d17/d21/d78/fa5 x:0 0 0 2024-05-17T21:52:58.928 INFO:tasks.workunit.client.1.smithi138.stdout:9/434: write d6/d7/d16/d29/f3a [280716,119267] 0 2024-05-17T21:52:58.928 INFO:tasks.workunit.client.1.smithi138.stdout:3/556: dwrite d1/d3/d49/d4b/f6e [0,4194304] 0 2024-05-17T21:52:58.931 INFO:tasks.workunit.client.1.smithi138.stdout:8/402: dread d0/dc/f18 [0,4194304] 0 2024-05-17T21:52:58.931 INFO:tasks.workunit.client.1.smithi138.stdout:5/428: write d1/f4f [357798,3698] 0 2024-05-17T21:52:58.934 INFO:tasks.workunit.client.1.smithi138.stdout:6/470: rename d7/db/d34/l5c to d7/d69/lab 0 2024-05-17T21:52:58.936 INFO:tasks.workunit.client.1.smithi138.stdout:6/471: dread - d7/db/d43/d5f/d6a/f89 zero size 2024-05-17T21:52:58.938 INFO:tasks.workunit.client.1.smithi138.stdout:8/403: dwrite d0/d2f/f41 [0,4194304] 0 2024-05-17T21:52:58.948 INFO:tasks.workunit.client.0.smithi107.stdout:7/598: sync 2024-05-17T21:52:58.953 INFO:tasks.workunit.client.0.smithi107.stdout:5/606: creat d5/db/dc/d1e/fc1 x:0 0 0 2024-05-17T21:52:58.955 INFO:tasks.workunit.client.0.smithi107.stdout:6/542: write d9/d16/f4a [887129,21600] 0 2024-05-17T21:52:58.956 INFO:tasks.workunit.client.0.smithi107.stdout:3/651: truncate d6/d2a/d2d/d48/f93 3262636 0 2024-05-17T21:52:58.958 INFO:tasks.workunit.client.0.smithi107.stdout:3/652: stat d6/d8/l32 0 2024-05-17T21:52:58.958 INFO:tasks.workunit.client.0.smithi107.stdout:4/620: creat d3/d4a/d47/d60/d3a/fd0 x:0 0 0 2024-05-17T21:52:58.959 INFO:tasks.workunit.client.0.smithi107.stdout:3/653: chown d6/d2a/la5 487548281 1 2024-05-17T21:52:58.960 INFO:tasks.workunit.client.0.smithi107.stdout:5/607: write d5/db/d16/d54/d59/d65/f27 [756365,17380] 0 2024-05-17T21:52:58.961 INFO:tasks.workunit.client.0.smithi107.stdout:4/621: chown d3/d6/ccb 36972770 1 2024-05-17T21:52:58.962 INFO:tasks.workunit.client.0.smithi107.stdout:5/608: chown d5/db/d16/d2d/fa1 132 1 2024-05-17T21:52:58.965 INFO:tasks.workunit.client.1.smithi138.stdout:0/478: mkdir d3/d3d/da2 0 2024-05-17T21:52:58.968 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:58.976+0000 7f32615ac700 1 -- 172.21.15.107:0/3284653748 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f325c0fbcb0 msgr2=0x7f325c0fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:52:58.969 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:58.976+0000 7f32615ac700 1 --2- 172.21.15.107:0/3284653748 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f325c0fbcb0 0x7f325c0fc0c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f324c004940 tx=0x7f324c009e30 comp rx=0 tx=0).stop 2024-05-17T21:52:58.969 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:58.976+0000 7f32615ac700 1 -- 172.21.15.107:0/3284653748 shutdown_connections 2024-05-17T21:52:58.969 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:58.976+0000 7f32615ac700 1 --2- 172.21.15.107:0/3284653748 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f325c0fcf60 0x7f325c0fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:52:58.969 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:58.976+0000 7f32615ac700 1 --2- 172.21.15.107:0/3284653748 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f325c0fbcb0 0x7f325c0fc0c0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:52:58.969 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:58.976+0000 7f32615ac700 1 -- 172.21.15.107:0/3284653748 >> 172.21.15.107:0/3284653748 conn(0x7f325c0f7430 msgr2=0x7f325c0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:52:58.969 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:58.976+0000 7f32615ac700 1 -- 172.21.15.107:0/3284653748 shutdown_connections 2024-05-17T21:52:58.970 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:58.976+0000 7f32615ac700 1 -- 172.21.15.107:0/3284653748 wait complete. 2024-05-17T21:52:58.970 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:58.977+0000 7f32615ac700 1 Processor -- start 2024-05-17T21:52:58.970 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:58.977+0000 7f32615ac700 1 -- start start 2024-05-17T21:52:58.970 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:58.978+0000 7f32615ac700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f325c0fbcb0 0x7f325c10e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:52:58.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:58.978+0000 7f32615ac700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f325c0fcf60 0x7f325c110aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:52:58.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:58.978+0000 7f32615ac700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f325c066420 con 0x7f325c0fcf60 2024-05-17T21:52:58.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:58.978+0000 7f32615ac700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f325c066560 con 0x7f325c0fbcb0 2024-05-17T21:52:58.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:58.978+0000 7f325a7fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f325c0fcf60 0x7f325c110aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:52:58.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:58.978+0000 7f325a7fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f325c0fcf60 0x7f325c110aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:35318/0 (socket says 172.21.15.107:35318) 2024-05-17T21:52:58.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:58.978+0000 7f325a7fc700 1 -- 172.21.15.107:0/2721777088 learned_addr learned my addr 172.21.15.107:0/2721777088 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:52:58.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:58.978+0000 7f325affd700 1 --2- 172.21.15.107:0/2721777088 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f325c0fbcb0 0x7f325c10e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:52:58.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:58.979+0000 7f325affd700 1 -- 172.21.15.107:0/2721777088 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f325c0fcf60 msgr2=0x7f325c110aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:52:58.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:58.979+0000 7f325affd700 1 --2- 172.21.15.107:0/2721777088 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f325c0fcf60 0x7f325c110aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:52:58.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:58.979+0000 7f325affd700 1 -- 172.21.15.107:0/2721777088 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f324c005040 con 0x7f325c0fbcb0 2024-05-17T21:52:58.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:58.979+0000 7f325a7fc700 1 --2- 172.21.15.107:0/2721777088 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f325c0fcf60 0x7f325c110aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T21:52:58.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:58.979+0000 7f325affd700 1 --2- 172.21.15.107:0/2721777088 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f325c0fbcb0 0x7f325c10e550 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f324c00b970 tx=0x7f324c009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:52:58.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:58.979+0000 7f32437fe700 1 -- 172.21.15.107:0/2721777088 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f324c00e040 con 0x7f325c0fbcb0 2024-05-17T21:52:58.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:58.979+0000 7f32437fe700 1 -- 172.21.15.107:0/2721777088 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f324c0093d0 con 0x7f325c0fbcb0 2024-05-17T21:52:58.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:58.979+0000 7f32437fe700 1 -- 172.21.15.107:0/2721777088 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f324c01c3f0 con 0x7f325c0fbcb0 2024-05-17T21:52:58.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:58.979+0000 7f32615ac700 1 -- 172.21.15.107:0/2721777088 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f325c1110d0 con 0x7f325c0fbcb0 2024-05-17T21:52:58.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:58.979+0000 7f32615ac700 1 -- 172.21.15.107:0/2721777088 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f325c111690 con 0x7f325c0fbcb0 2024-05-17T21:52:58.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:58.981+0000 7f32615ac700 1 -- 172.21.15.107:0/2721777088 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f325c05f5c0 con 0x7f325c0fbcb0 2024-05-17T21:52:58.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:58.981+0000 7f32437fe700 1 -- 172.21.15.107:0/2721777088 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) v1 ==== 93316+0+0 (secure 0 0 0) 0x7f324c00bc50 con 0x7f325c0fbcb0 2024-05-17T21:52:58.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:58.982+0000 7f32437fe700 1 --2- 172.21.15.107:0/2721777088 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7f3244070bb0 0x7f3244073060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:52:58.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:58.982+0000 7f32437fe700 1 -- 172.21.15.107:0/2721777088 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f324c091be0 con 0x7f325c0fbcb0 2024-05-17T21:52:58.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:58.982+0000 7f325a7fc700 1 --2- 172.21.15.107:0/2721777088 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7f3244070bb0 0x7f3244073060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:52:58.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:58.984+0000 7f325a7fc700 1 --2- 172.21.15.107:0/2721777088 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7f3244070bb0 0x7f3244073060 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3250009f60 tx=0x7f32500091a0 comp rx=0 tx=0).ready entity=mgr.14636 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:52:58.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:58.990+0000 7f32437fe700 1 -- 172.21.15.107:0/2721777088 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+190121 (secure 0 0 0) 0x7f324c05a010 con 0x7f325c0fbcb0 2024-05-17T21:52:58.997 INFO:tasks.workunit.client.0.smithi107.stdout:2/484: sync 2024-05-17T21:52:58.998 INFO:tasks.workunit.client.0.smithi107.stdout:2/485: chown d0/d9/d1c/d22/f5b 860 1 2024-05-17T21:52:58.998 INFO:tasks.workunit.client.0.smithi107.stdout:2/486: chown d0/d9/d1c/d3e/l7c 45482 1 2024-05-17T21:52:59.006 INFO:tasks.workunit.client.0.smithi107.stdout:1/667: rmdir d4/df/d14/d1d 39 2024-05-17T21:52:59.019 INFO:tasks.workunit.client.0.smithi107.stdout:8/626: mknod dc/d44/d4b/ce3 0 2024-05-17T21:52:59.020 INFO:tasks.workunit.client.0.smithi107.stdout:8/627: chown dc/d21/d27/d34/d4c/l7a 0 1 2024-05-17T21:52:59.021 INFO:tasks.workunit.client.0.smithi107.stdout:8/628: chown dc/d21/d27/l9d 53587 1 2024-05-17T21:52:59.022 INFO:tasks.workunit.client.1.smithi138.stdout:7/363: dwrite d0/f27 [0,4194304] 0 2024-05-17T21:52:59.024 INFO:tasks.workunit.client.1.smithi138.stdout:5/429: unlink d1/d6/d1b/d29/d36/d68/l83 0 2024-05-17T21:52:59.031 INFO:tasks.workunit.client.1.smithi138.stdout:2/499: sync 2024-05-17T21:52:59.033 INFO:tasks.workunit.client.0.smithi107.stdout:4/622: rmdir d3/d6/d26 39 2024-05-17T21:52:59.035 INFO:tasks.workunit.client.1.smithi138.stdout:9/435: mknod d6/c8d 0 2024-05-17T21:52:59.037 INFO:tasks.workunit.client.0.smithi107.stdout:5/609: rmdir d5/db/d16/d2a 39 2024-05-17T21:52:59.038 INFO:tasks.workunit.client.1.smithi138.stdout:8/404: dread d0/dc/f2b [0,4194304] 0 2024-05-17T21:52:59.042 INFO:tasks.workunit.client.0.smithi107.stdout:4/623: dwrite d3/db/d35/d4f/d93/fae [0,4194304] 0 2024-05-17T21:52:59.042 INFO:tasks.workunit.client.0.smithi107.stdout:2/487: rmdir d0/d9/d1c/d37 39 2024-05-17T21:52:59.043 INFO:tasks.workunit.client.1.smithi138.stdout:4/459: mkdir d0/d2f/d49/d60/d1f/d32/d71/d95/da6 0 2024-05-17T21:52:59.051 INFO:tasks.workunit.client.1.smithi138.stdout:5/430: fsync d1/f45 0 2024-05-17T21:52:59.051 INFO:tasks.workunit.client.1.smithi138.stdout:4/460: dwrite d0/d2f/d49/d60/f9e [0,4194304] 0 2024-05-17T21:52:59.052 INFO:tasks.workunit.client.1.smithi138.stdout:5/431: write d1/d1a/d39/d69/f7f [998569,3664] 0 2024-05-17T21:52:59.057 INFO:tasks.workunit.client.1.smithi138.stdout:2/500: mknod d3/d21/d2f/d3e/d55/d62/ca5 0 2024-05-17T21:52:59.058 INFO:tasks.workunit.client.1.smithi138.stdout:5/432: dread d1/d1a/d39/d69/f7f [0,4194304] 0 2024-05-17T21:52:59.059 INFO:tasks.workunit.client.0.smithi107.stdout:3/654: creat d6/d8/d3e/dc9/fd6 x:0 0 0 2024-05-17T21:52:59.059 INFO:tasks.workunit.client.1.smithi138.stdout:5/433: write d1/d6/d1b/d29/d36/f7a [636058,57326] 0 2024-05-17T21:52:59.063 INFO:tasks.workunit.client.1.smithi138.stdout:3/557: creat d1/fc3 x:0 0 0 2024-05-17T21:52:59.068 INFO:tasks.workunit.client.0.smithi107.stdout:5/610: rmdir d5/db/d16/d2d/d40 39 2024-05-17T21:52:59.077 INFO:tasks.workunit.client.1.smithi138.stdout:2/501: mknod d3/d21/d2f/d3e/d55/d73/ca6 0 2024-05-17T21:52:59.080 INFO:tasks.workunit.client.1.smithi138.stdout:9/436: mkdir d6/d7/d16/d29/d48/d6e/d74/d75/d76/d8e 0 2024-05-17T21:52:59.081 INFO:tasks.workunit.client.0.smithi107.stdout:9/650: mknod d0/dd/d2f/d87/cdf 0 2024-05-17T21:52:59.082 INFO:tasks.workunit.client.1.smithi138.stdout:9/437: write d6/d7/d16/f8a [1352937,113486] 0 2024-05-17T21:52:59.082 INFO:tasks.workunit.client.0.smithi107.stdout:3/655: rmdir d6/d2a/d2d/d69 39 2024-05-17T21:52:59.083 INFO:tasks.workunit.client.1.smithi138.stdout:4/461: dread d0/d3/db/f69 [0,4194304] 0 2024-05-17T21:52:59.085 INFO:tasks.workunit.client.1.smithi138.stdout:2/502: dwrite d3/f74 [4194304,4194304] 0 2024-05-17T21:52:59.085 INFO:tasks.workunit.client.1.smithi138.stdout:9/438: read f5 [934783,57821] 0 2024-05-17T21:52:59.089 INFO:tasks.workunit.client.0.smithi107.stdout:2/488: symlink d0/d93/l97 0 2024-05-17T21:52:59.090 INFO:tasks.workunit.client.1.smithi138.stdout:7/364: mknod d0/d18/d40/d41/c72 0 2024-05-17T21:52:59.092 INFO:tasks.workunit.client.0.smithi107.stdout:6/543: getdents d9/d16/d4d/d67 0 2024-05-17T21:52:59.105 INFO:tasks.workunit.client.1.smithi138.stdout:5/434: getdents d1/d8a 0 2024-05-17T21:52:59.109 INFO:tasks.workunit.client.1.smithi138.stdout:2/503: read d3/f27 [1525073,17203] 0 2024-05-17T21:52:59.109 INFO:tasks.workunit.client.1.smithi138.stdout:3/558: symlink d1/d3/lc4 0 2024-05-17T21:52:59.110 INFO:tasks.workunit.client.1.smithi138.stdout:7/365: readlink d0/ld 0 2024-05-17T21:52:59.115 INFO:tasks.workunit.client.1.smithi138.stdout:3/559: dwrite d1/fa5 [0,4194304] 0 2024-05-17T21:52:59.115 INFO:tasks.workunit.client.0.smithi107.stdout:6/544: dread d9/d16/d4d/f52 [0,4194304] 0 2024-05-17T21:52:59.120 INFO:tasks.workunit.client.1.smithi138.stdout:2/504: rename d3/d21/d2f/d3e/d63/f83 to d3/d81/fa7 0 2024-05-17T21:52:59.120 INFO:tasks.workunit.client.1.smithi138.stdout:2/505: fdatasync d3/d21/d2f/f39 0 2024-05-17T21:52:59.121 INFO:tasks.workunit.client.0.smithi107.stdout:6/545: symlink d9/d58/d7b/la9 0 2024-05-17T21:52:59.126 INFO:tasks.workunit.client.1.smithi138.stdout:2/506: dwrite d3/d21/d2f/d3e/d58/f5d [0,4194304] 0 2024-05-17T21:52:59.130 INFO:tasks.workunit.client.1.smithi138.stdout:4/462: getdents d0/d1b/d42 0 2024-05-17T21:52:59.132 INFO:tasks.workunit.client.1.smithi138.stdout:4/463: write d0/d3/db/f20 [1640170,44689] 0 2024-05-17T21:52:59.135 INFO:tasks.workunit.client.1.smithi138.stdout:2/507: chown d3/d21/d2f/d3e/l41 47415927 1 2024-05-17T21:52:59.137 INFO:tasks.workunit.client.1.smithi138.stdout:4/464: creat d0/d3/d17/d21/fa7 x:0 0 0 2024-05-17T21:52:59.137 INFO:tasks.workunit.client.1.smithi138.stdout:4/465: chown d0/d3/cc 31132 1 2024-05-17T21:52:59.138 INFO:tasks.workunit.client.1.smithi138.stdout:4/466: read - d0/d3/d17/d21/fa7 zero size 2024-05-17T21:52:59.144 INFO:tasks.workunit.client.1.smithi138.stdout:4/467: rmdir d0/d3/db/d94 0 2024-05-17T21:52:59.146 INFO:tasks.workunit.client.1.smithi138.stdout:4/468: rmdir d0/d3/d17/d5e/d7f 39 2024-05-17T21:52:59.147 INFO:tasks.workunit.client.1.smithi138.stdout:4/469: fdatasync d0/d2f/d49/f90 0 2024-05-17T21:52:59.150 INFO:tasks.workunit.client.1.smithi138.stdout:4/470: creat d0/d3/db/da0/fa8 x:0 0 0 2024-05-17T21:52:59.163 INFO:tasks.workunit.client.0.smithi107.stdout:0/717: dwrite d3/dc/f60 [0,4194304] 0 2024-05-17T21:52:59.165 INFO:tasks.workunit.client.0.smithi107.stdout:0/718: stat d3/d24 0 2024-05-17T21:52:59.166 INFO:tasks.workunit.client.0.smithi107.stdout:0/719: chown d3/dc/dae/fbb 2005936 1 2024-05-17T21:52:59.170 INFO:tasks.workunit.client.1.smithi138.stdout:6/472: dwrite d7/db/d43/d3f/d79/f9f [0,4194304] 0 2024-05-17T21:52:59.178 INFO:tasks.workunit.client.1.smithi138.stdout:6/473: rename d7/db/d34/da6/l19 to d7/db/d43/d5f/d94/lac 0 2024-05-17T21:52:59.179 INFO:tasks.workunit.client.1.smithi138.stdout:6/474: stat d7/db/d43/d5f/d6a/c99 0 2024-05-17T21:52:59.182 INFO:tasks.workunit.client.0.smithi107.stdout:1/668: dwrite d4/f1b [0,4194304] 0 2024-05-17T21:52:59.182 INFO:tasks.workunit.client.1.smithi138.stdout:6/475: symlink d7/db/d43/d5f/d6a/d77/lad 0 2024-05-17T21:52:59.188 INFO:tasks.workunit.client.0.smithi107.stdout:1/669: mknod d4/df/d1a/d26/d50/d5c/cdc 0 2024-05-17T21:52:59.189 INFO:tasks.workunit.client.1.smithi138.stdout:6/476: getdents d7/db/d43 0 2024-05-17T21:52:59.190 INFO:tasks.workunit.client.1.smithi138.stdout:6/477: write d7/d8/f9a [187737,64455] 0 2024-05-17T21:52:59.191 INFO:tasks.workunit.client.0.smithi107.stdout:1/670: symlink d4/df/d1a/d41/ldd 0 2024-05-17T21:52:59.193 INFO:tasks.workunit.client.1.smithi138.stdout:6/478: truncate d7/db/d43/d63/f8f 513489 0 2024-05-17T21:52:59.193 INFO:tasks.workunit.client.1.smithi138.stdout:6/479: dread - d7/db/d43/f98 zero size 2024-05-17T21:52:59.194 INFO:tasks.workunit.client.1.smithi138.stdout:8/405: write d0/dc/f10 [4125677,123194] 0 2024-05-17T21:52:59.195 INFO:tasks.workunit.client.1.smithi138.stdout:8/406: chown d0/d2f/c55 7 1 2024-05-17T21:52:59.196 INFO:tasks.workunit.client.1.smithi138.stdout:8/407: chown d0/dc/d3b 1661786 1 2024-05-17T21:52:59.197 INFO:tasks.workunit.client.0.smithi107.stdout:4/624: write d3/db/d1e/f6c [680171,110126] 0 2024-05-17T21:52:59.198 INFO:tasks.workunit.client.1.smithi138.stdout:6/480: rename d7/d8/d78/c91 to d7/db/d43/d5f/d6a/d77/cae 0 2024-05-17T21:52:59.199 INFO:tasks.workunit.client.0.smithi107.stdout:1/671: dwrite d4/df/d1a/d26/d50/f95 [0,4194304] 0 2024-05-17T21:52:59.199 INFO:tasks.workunit.client.1.smithi138.stdout:8/408: symlink d0/dc/d1a/d33/d4a/d4e/l7a 0 2024-05-17T21:52:59.200 INFO:tasks.workunit.client.1.smithi138.stdout:6/481: unlink d7/d8/f2b 0 2024-05-17T21:52:59.204 INFO:tasks.workunit.client.0.smithi107.stdout:4/625: unlink d3/d4a/d47/d60/d3c/d58/c90 0 2024-05-17T21:52:59.207 INFO:tasks.workunit.client.1.smithi138.stdout:8/409: symlink d0/d63/l7b 0 2024-05-17T21:52:59.208 INFO:tasks.workunit.client.1.smithi138.stdout:1/435: sync 2024-05-17T21:52:59.208 INFO:tasks.workunit.client.1.smithi138.stdout:6/482: dwrite d7/db/d43/d3f/f5b [0,4194304] 0 2024-05-17T21:52:59.209 INFO:tasks.workunit.client.1.smithi138.stdout:8/410: read d0/d2f/f41 [2412286,58116] 0 2024-05-17T21:52:59.211 INFO:tasks.workunit.client.1.smithi138.stdout:1/436: write d4/dd/d3b/f53 [468271,30145] 0 2024-05-17T21:52:59.211 INFO:tasks.workunit.client.0.smithi107.stdout:4/626: read d3/d4a/d47/d60/d3a/d41/d80/d43/f53 [8082802,36231] 0 2024-05-17T21:52:59.213 INFO:tasks.workunit.client.0.smithi107.stdout:8/629: dwrite dc/d21/d97/d63/d71/d81/f9b [0,4194304] 0 2024-05-17T21:52:59.216 INFO:tasks.workunit.client.0.smithi107.stdout:8/630: write dc/d21/d27/d34/f6e [575709,1437] 0 2024-05-17T21:52:59.219 INFO:tasks.workunit.client.0.smithi107.stdout:5/611: dwrite d5/db/d16/d54/d59/f87 [0,4194304] 0 2024-05-17T21:52:59.219 INFO:tasks.workunit.client.1.smithi138.stdout:1/437: write d4/df/d4e/f84 [1618046,52009] 0 2024-05-17T21:52:59.222 INFO:tasks.workunit.client.1.smithi138.stdout:8/411: rename d0/dc/d1a/d33/d4a/d4e/f72 to d0/dc/d1a/d33/d4a/d4e/f7c 0 2024-05-17T21:52:59.225 INFO:tasks.workunit.client.0.smithi107.stdout:4/627: mknod d3/d6/d26/d36/d7e/cd1 0 2024-05-17T21:52:59.226 INFO:tasks.workunit.client.1.smithi138.stdout:6/483: creat d7/db/d43/d5f/d6a/d9b/faf x:0 0 0 2024-05-17T21:52:59.228 INFO:tasks.workunit.client.0.smithi107.stdout:5/612: dwrite d5/db/d16/d54/d59/f87 [0,4194304] 0 2024-05-17T21:52:59.231 INFO:tasks.workunit.client.0.smithi107.stdout:4/628: mknod d3/d4a/d47/da3/cd2 0 2024-05-17T21:52:59.232 INFO:tasks.workunit.client.1.smithi138.stdout:6/484: mkdir d7/db/d43/d5f/da4/db0 0 2024-05-17T21:52:59.233 INFO:tasks.workunit.client.1.smithi138.stdout:6/485: chown d7/d8/la9 193438 1 2024-05-17T21:52:59.235 INFO:tasks.workunit.client.0.smithi107.stdout:5/613: mkdir d5/db/d16/d2d/d40/dc2 0 2024-05-17T21:52:59.237 INFO:tasks.workunit.client.0.smithi107.stdout:8/631: getdents dc/d21/d27/d34/d4c 0 2024-05-17T21:52:59.238 INFO:tasks.workunit.client.1.smithi138.stdout:6/486: getdents d7/db/d43/d5f/da4/db0 0 2024-05-17T21:52:59.239 INFO:tasks.workunit.client.1.smithi138.stdout:6/487: fsync d7/db/d43/d5f/f73 0 2024-05-17T21:52:59.239 INFO:tasks.workunit.client.0.smithi107.stdout:7/599: rename d5/df/l30 to d5/df/dd7/ld8 0 2024-05-17T21:52:59.240 INFO:tasks.workunit.client.1.smithi138.stdout:6/488: truncate d7/d8/d78/f81 931284 0 2024-05-17T21:52:59.241 INFO:tasks.workunit.client.0.smithi107.stdout:7/600: chown d5/df/d11/d1c/d31/d39/d92/d9a/fc2 575 1 2024-05-17T21:52:59.242 INFO:tasks.workunit.client.0.smithi107.stdout:5/614: creat d5/db/d16/d2d/d40/d9f/fc3 x:0 0 0 2024-05-17T21:52:59.242 INFO:tasks.workunit.client.0.smithi107.stdout:3/656: sync 2024-05-17T21:52:59.243 INFO:tasks.workunit.client.0.smithi107.stdout:9/651: rename d0/dd/d1f/d2c to d0/dd/d2f/d87/d89/de0 0 2024-05-17T21:52:59.244 INFO:tasks.workunit.client.0.smithi107.stdout:3/657: chown d6/d2a/d2d/da8/fb8 95886 1 2024-05-17T21:52:59.245 INFO:tasks.workunit.client.1.smithi138.stdout:6/489: symlink d7/db/d43/d5f/d6a/d8b/lb1 0 2024-05-17T21:52:59.245 INFO:tasks.workunit.client.1.smithi138.stdout:6/490: chown d7/db/d43 1 1 2024-05-17T21:52:59.246 INFO:tasks.workunit.client.1.smithi138.stdout:6/491: dread - d7/db/d43/d63/f7d zero size 2024-05-17T21:52:59.247 INFO:tasks.workunit.client.0.smithi107.stdout:5/615: read d5/db/d16/d54/d59/d65/f27 [2962899,23708] 0 2024-05-17T21:52:59.247 INFO:tasks.workunit.client.1.smithi138.stdout:5/435: truncate d1/f13 3928295 0 2024-05-17T21:52:59.252 INFO:tasks.workunit.client.1.smithi138.stdout:7/366: truncate d0/f1e 797227 0 2024-05-17T21:52:59.253 INFO:tasks.workunit.client.0.smithi107.stdout:2/489: rename d0/d26/f55 to d0/d9/d1c/d37/f98 0 2024-05-17T21:52:59.254 INFO:tasks.workunit.client.0.smithi107.stdout:3/658: dread d6/d8/d10/d35/d36/f3d [0,4194304] 0 2024-05-17T21:52:59.254 INFO:tasks.workunit.client.1.smithi138.stdout:5/436: mkdir d1/d1a/d8e 0 2024-05-17T21:52:59.255 INFO:tasks.workunit.client.1.smithi138.stdout:5/437: chown d1/d6/l10 1503172 1 2024-05-17T21:52:59.255 INFO:tasks.workunit.client.1.smithi138.stdout:5/438: write d1/d6/f72 [624583,46698] 0 2024-05-17T21:52:59.256 INFO:tasks.workunit.client.1.smithi138.stdout:7/367: dread - d0/f4d zero size 2024-05-17T21:52:59.258 INFO:tasks.workunit.client.1.smithi138.stdout:5/439: truncate d1/d6/d1b/d29/d36/f7a 1068008 0 2024-05-17T21:52:59.258 INFO:tasks.workunit.client.0.smithi107.stdout:9/652: creat d0/dd/d21/daf/fe1 x:0 0 0 2024-05-17T21:52:59.259 INFO:tasks.workunit.client.1.smithi138.stdout:5/440: chown d1/d60/c8d 7 1 2024-05-17T21:52:59.260 INFO:tasks.workunit.client.1.smithi138.stdout:2/508: truncate d3/d21/f2b 718538 0 2024-05-17T21:52:59.261 INFO:tasks.workunit.client.0.smithi107.stdout:4/629: dread d3/fa [0,4194304] 0 2024-05-17T21:52:59.261 INFO:tasks.workunit.client.0.smithi107.stdout:8/632: getdents dc/d21/d27/d2f 0 2024-05-17T21:52:59.265 INFO:tasks.workunit.client.1.smithi138.stdout:5/441: rename d1/d53/f57 to d1/d1a/d39/d4e/d58/d5b/d65/f8f 0 2024-05-17T21:52:59.265 INFO:tasks.workunit.client.1.smithi138.stdout:2/509: dwrite d3/f29 [0,4194304] 0 2024-05-17T21:52:59.267 INFO:tasks.workunit.client.1.smithi138.stdout:5/442: truncate d1/d53/f6e 4382364 0 2024-05-17T21:52:59.267 INFO:tasks.workunit.client.1.smithi138.stdout:7/368: creat d0/d18/d40/d5f/f73 x:0 0 0 2024-05-17T21:52:59.269 INFO:tasks.workunit.client.0.smithi107.stdout:2/490: mkdir d0/d9/d11/d15/d99 0 2024-05-17T21:52:59.270 INFO:tasks.workunit.client.1.smithi138.stdout:2/510: mkdir d3/d81/d96/da8 0 2024-05-17T21:52:59.272 INFO:tasks.workunit.client.0.smithi107.stdout:3/659: unlink d6/c18 0 2024-05-17T21:52:59.273 INFO:tasks.workunit.client.1.smithi138.stdout:5/443: fdatasync d1/d1a/d39/f4c 0 2024-05-17T21:52:59.276 INFO:tasks.workunit.client.1.smithi138.stdout:2/511: dwrite d3/d21/d2f/f5f [0,4194304] 0 2024-05-17T21:52:59.278 INFO:tasks.workunit.client.1.smithi138.stdout:7/369: creat d0/d18/d40/d41/d50/f74 x:0 0 0 2024-05-17T21:52:59.278 INFO:tasks.workunit.client.0.smithi107.stdout:6/546: sync 2024-05-17T21:52:59.279 INFO:tasks.workunit.client.0.smithi107.stdout:9/653: creat d0/dd/d21/d29/d57/fe2 x:0 0 0 2024-05-17T21:52:59.279 INFO:tasks.workunit.client.0.smithi107.stdout:6/547: dread - d9/d16/d22/f92 zero size 2024-05-17T21:52:59.279 INFO:tasks.workunit.client.0.smithi107.stdout:5/616: dwrite d5/db/d16/d2a/f64 [0,4194304] 0 2024-05-17T21:52:59.282 INFO:tasks.workunit.client.0.smithi107.stdout:5/617: write d5/db/d16/d5b/f97 [979567,91193] 0 2024-05-17T21:52:59.285 INFO:tasks.workunit.client.0.smithi107.stdout:5/618: chown d5/db/d16/d5b/f3d 0 1 2024-05-17T21:52:59.285 INFO:tasks.workunit.client.1.smithi138.stdout:4/471: creat d0/d3/d17/fa9 x:0 0 0 2024-05-17T21:52:59.286 INFO:tasks.workunit.client.1.smithi138.stdout:4/472: truncate d0/f34 2909036 0 2024-05-17T21:52:59.290 INFO:tasks.workunit.client.1.smithi138.stdout:7/370: fsync d0/f30 0 2024-05-17T21:52:59.292 INFO:tasks.workunit.client.0.smithi107.stdout:9/654: symlink d0/d80/dad/le3 0 2024-05-17T21:52:59.292 INFO:tasks.workunit.client.0.smithi107.stdout:5/619: dwrite d5/db/d16/d2a/fab [0,4194304] 0 2024-05-17T21:52:59.293 INFO:tasks.workunit.client.1.smithi138.stdout:2/512: link d3/d21/d2f/d3e/d58/f6b d3/d21/d2f/d3e/d55/d99/fa9 0 2024-05-17T21:52:59.294 INFO:tasks.workunit.client.1.smithi138.stdout:2/513: dread - d3/f8c zero size 2024-05-17T21:52:59.294 INFO:tasks.workunit.client.0.smithi107.stdout:4/630: symlink d3/d6/ld3 0 2024-05-17T21:52:59.295 INFO:tasks.workunit.client.1.smithi138.stdout:4/473: mknod d0/d2f/d49/d60/d1f/d7e/caa 0 2024-05-17T21:52:59.297 INFO:tasks.workunit.client.1.smithi138.stdout:2/514: dread d3/d21/d2f/f5b [0,4194304] 0 2024-05-17T21:52:59.298 INFO:tasks.workunit.client.0.smithi107.stdout:8/633: fdatasync dc/d21/d27/d2f/f4a 0 2024-05-17T21:52:59.299 INFO:tasks.workunit.client.0.smithi107.stdout:1/672: rename d4/df/d1a/d26/d72/dbc/lc2 to d4/df/lde 0 2024-05-17T21:52:59.300 INFO:tasks.workunit.client.1.smithi138.stdout:7/371: symlink d0/d18/d40/d5f/l75 0 2024-05-17T21:52:59.301 INFO:tasks.workunit.client.0.smithi107.stdout:4/631: dread d3/db/f1f [0,4194304] 0 2024-05-17T21:52:59.302 INFO:tasks.workunit.client.0.smithi107.stdout:3/660: mknod d6/d8/d3e/d3f/d90/da7/cd7 0 2024-05-17T21:52:59.303 INFO:tasks.workunit.client.0.smithi107.stdout:3/661: dread - d6/d2a/d2d/da8/f92 zero size 2024-05-17T21:52:59.304 INFO:tasks.workunit.client.0.smithi107.stdout:9/655: rmdir d0/dd/d1b 39 2024-05-17T21:52:59.306 INFO:tasks.workunit.client.0.smithi107.stdout:9/656: stat d0/dd/d21/d29/d57/l77 0 2024-05-17T21:52:59.307 INFO:tasks.workunit.client.1.smithi138.stdout:2/515: unlink d3/f29 0 2024-05-17T21:52:59.307 INFO:tasks.workunit.client.0.smithi107.stdout:9/657: dread - d0/dd/d21/daf/fe1 zero size 2024-05-17T21:52:59.308 INFO:tasks.workunit.client.0.smithi107.stdout:5/620: mknod d5/db/d39/cc4 0 2024-05-17T21:52:59.308 INFO:tasks.workunit.client.1.smithi138.stdout:7/372: symlink d0/d18/d40/d41/d51/l76 0 2024-05-17T21:52:59.309 INFO:tasks.workunit.client.1.smithi138.stdout:7/373: readlink d0/d18/d40/l70 0 2024-05-17T21:52:59.310 INFO:tasks.workunit.client.1.smithi138.stdout:5/444: dread d1/d53/f6e [0,4194304] 0 2024-05-17T21:52:59.310 INFO:tasks.workunit.client.1.smithi138.stdout:3/560: write d1/d3/d49/f7d [3854567,103185] 0 2024-05-17T21:52:59.312 INFO:tasks.workunit.client.1.smithi138.stdout:5/445: write d1/d6/d1b/d29/d36/f7a [1743856,108407] 0 2024-05-17T21:52:59.312 INFO:tasks.workunit.client.0.smithi107.stdout:0/720: dwrite d3/dc/d22/f7d [4194304,4194304] 0 2024-05-17T21:52:59.314 INFO:tasks.workunit.client.0.smithi107.stdout:0/721: chown d3/dc/d31/f3a 4796 1 2024-05-17T21:52:59.315 INFO:tasks.workunit.client.1.smithi138.stdout:4/474: creat d0/d3/d1d/d38/d8a/fab x:0 0 0 2024-05-17T21:52:59.318 INFO:tasks.workunit.client.0.smithi107.stdout:7/601: rename d5/d16/d3b/d45/fc8 to d5/d16/d3b/d45/d91/db3/fd9 0 2024-05-17T21:52:59.320 INFO:tasks.workunit.client.0.smithi107.stdout:4/632: symlink d3/d4a/d47/da3/ld4 0 2024-05-17T21:52:59.321 INFO:tasks.workunit.client.0.smithi107.stdout:7/602: write d5/fe [1834014,5563] 0 2024-05-17T21:52:59.321 INFO:tasks.workunit.client.0.smithi107.stdout:4/633: chown d3/d6/d26/d36/d37 163437199 1 2024-05-17T21:52:59.323 INFO:tasks.workunit.client.0.smithi107.stdout:7/603: write d5/df/d11/f4c [1858884,54749] 0 2024-05-17T21:52:59.323 INFO:tasks.workunit.client.1.smithi138.stdout:5/446: dread d1/d6/d1b/d29/d36/f37 [0,4194304] 0 2024-05-17T21:52:59.324 INFO:tasks.workunit.client.0.smithi107.stdout:7/604: stat d5/d16/l3c 0 2024-05-17T21:52:59.324 INFO:tasks.workunit.client.1.smithi138.stdout:7/374: write d0/f20 [3186053,108123] 0 2024-05-17T21:52:59.325 INFO:tasks.workunit.client.1.smithi138.stdout:7/375: stat d0/d18/l37 0 2024-05-17T21:52:59.326 INFO:tasks.workunit.client.0.smithi107.stdout:1/673: dread d4/f38 [0,4194304] 0 2024-05-17T21:52:59.327 INFO:tasks.workunit.client.0.smithi107.stdout:5/621: truncate d5/db/d16/f4b 4872959 0 2024-05-17T21:52:59.328 INFO:tasks.workunit.client.0.smithi107.stdout:9/658: dread d0/dd/d2f/f64 [0,4194304] 0 2024-05-17T21:52:59.330 INFO:tasks.workunit.client.0.smithi107.stdout:9/659: chown d0/dd/d3d/d9d 16266059 1 2024-05-17T21:52:59.330 INFO:tasks.workunit.client.1.smithi138.stdout:3/561: rename d1/ca to d1/d62/dad/d5c/cc5 0 2024-05-17T21:52:59.331 INFO:tasks.workunit.client.0.smithi107.stdout:9/660: write d0/dd/d21/d29/d3e/d94/fb0 [438971,99732] 0 2024-05-17T21:52:59.333 INFO:tasks.workunit.client.1.smithi138.stdout:8/412: rmdir d0/dc/d1a 39 2024-05-17T21:52:59.334 INFO:tasks.workunit.client.1.smithi138.stdout:4/475: creat d0/d3/d17/d21/d78/fac x:0 0 0 2024-05-17T21:52:59.337 INFO:tasks.workunit.client.0.smithi107.stdout:3/662: dread d6/d8/d10/d25/f55 [0,4194304] 0 2024-05-17T21:52:59.340 INFO:tasks.workunit.client.1.smithi138.stdout:5/447: dread d1/d6/d1b/d29/f67 [0,4194304] 0 2024-05-17T21:52:59.345 INFO:tasks.workunit.client.0.smithi107.stdout:4/634: unlink d3/d4a/f78 0 2024-05-17T21:52:59.345 INFO:tasks.workunit.client.0.smithi107.stdout:4/635: stat d3/d6/d26/d36/d7e/f87 0 2024-05-17T21:52:59.347 INFO:tasks.workunit.client.0.smithi107.stdout:1/674: mkdir d4/df/d14/d2e/ddf 0 2024-05-17T21:52:59.347 INFO:tasks.workunit.client.1.smithi138.stdout:3/562: dread d1/d3/d49/d4b/f85 [0,4194304] 0 2024-05-17T21:52:59.349 INFO:tasks.workunit.client.0.smithi107.stdout:1/675: read d4/df/d14/f4c [1932768,19725] 0 2024-05-17T21:52:59.349 INFO:tasks.workunit.client.1.smithi138.stdout:8/413: mkdir d0/dc/d3b/d4d/d7d 0 2024-05-17T21:52:59.352 INFO:tasks.workunit.client.1.smithi138.stdout:0/479: sync 2024-05-17T21:52:59.352 INFO:tasks.workunit.client.1.smithi138.stdout:9/439: sync 2024-05-17T21:52:59.353 INFO:tasks.workunit.client.0.smithi107.stdout:5/622: truncate d5/db/d39/f70 446700 0 2024-05-17T21:52:59.353 INFO:tasks.workunit.client.1.smithi138.stdout:6/492: write d7/f22 [680075,125010] 0 2024-05-17T21:52:59.354 INFO:tasks.workunit.client.1.smithi138.stdout:5/448: dread d1/d60/f64 [0,4194304] 0 2024-05-17T21:52:59.355 INFO:tasks.workunit.client.1.smithi138.stdout:6/493: dread - d7/db/d43/d3f/d79/f93 zero size 2024-05-17T21:52:59.356 INFO:tasks.workunit.client.1.smithi138.stdout:6/494: stat d7/d8/d78 0 2024-05-17T21:52:59.359 INFO:tasks.workunit.client.0.smithi107.stdout:9/661: write d0/dd/d69/db2/fbd [485968,129548] 0 2024-05-17T21:52:59.360 INFO:tasks.workunit.client.1.smithi138.stdout:3/563: creat d1/d4d/fc6 x:0 0 0 2024-05-17T21:52:59.364 INFO:tasks.workunit.client.1.smithi138.stdout:8/414: mknod d0/d31/c7e 0 2024-05-17T21:52:59.367 INFO:tasks.workunit.client.1.smithi138.stdout:0/480: write d3/f83 [702831,98484] 0 2024-05-17T21:52:59.367 INFO:tasks.workunit.client.0.smithi107.stdout:2/491: dread d0/d9/d1c/d37/f98 [0,4194304] 0 2024-05-17T21:52:59.369 INFO:tasks.workunit.client.0.smithi107.stdout:1/676: truncate d4/df/d14/d1d/fc3 498008 0 2024-05-17T21:52:59.370 INFO:tasks.workunit.client.1.smithi138.stdout:9/440: creat d6/d7/d16/d29/d38/d40/d44/d5c/f8f x:0 0 0 2024-05-17T21:52:59.371 INFO:tasks.workunit.client.0.smithi107.stdout:0/722: creat d3/d24/ff8 x:0 0 0 2024-05-17T21:52:59.371 INFO:tasks.workunit.client.1.smithi138.stdout:9/441: write d6/d7/d16/f20 [4872907,57655] 0 2024-05-17T21:52:59.374 INFO:tasks.workunit.client.1.smithi138.stdout:6/495: fdatasync d7/d8/f50 0 2024-05-17T21:52:59.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:59.383+0000 7f32615ac700 1 -- 172.21.15.107:0/2721777088 --> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f325c1012c0 con 0x7f3244070bb0 2024-05-17T21:52:59.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:59.385+0000 7f32437fe700 1 -- 172.21.15.107:0/2721777088 <== mgr.14636 v2:172.21.15.138:6828/1369055740 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+349 (secure 0 0 0) 0x7f325c1012c0 con 0x7f3244070bb0 2024-05-17T21:52:59.378 INFO:tasks.workunit.client.1.smithi138.stdout:7/376: rename d0/d39/f6b to d0/d18/d40/d41/f77 0 2024-05-17T21:52:59.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:59.387+0000 7f32615ac700 1 -- 172.21.15.107:0/2721777088 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7f3244070bb0 msgr2=0x7f3244073060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:52:59.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:59.387+0000 7f32615ac700 1 --2- 172.21.15.107:0/2721777088 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7f3244070bb0 0x7f3244073060 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3250009f60 tx=0x7f32500091a0 comp rx=0 tx=0).stop 2024-05-17T21:52:59.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:59.387+0000 7f32615ac700 1 -- 172.21.15.107:0/2721777088 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f325c0fbcb0 msgr2=0x7f325c10e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:52:59.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:59.387+0000 7f32615ac700 1 --2- 172.21.15.107:0/2721777088 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f325c0fbcb0 0x7f325c10e550 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f324c00b970 tx=0x7f324c009e30 comp rx=0 tx=0).stop 2024-05-17T21:52:59.380 INFO:tasks.workunit.client.0.smithi107.stdout:4/636: mkdir d3/d6/dd5 0 2024-05-17T21:52:59.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:59.387+0000 7f32615ac700 1 -- 172.21.15.107:0/2721777088 shutdown_connections 2024-05-17T21:52:59.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:59.387+0000 7f32615ac700 1 --2- 172.21.15.107:0/2721777088 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7f3244070bb0 0x7f3244073060 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:52:59.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:59.387+0000 7f32615ac700 1 --2- 172.21.15.107:0/2721777088 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f325c0fbcb0 0x7f325c10e550 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:52:59.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:59.387+0000 7f32615ac700 1 --2- 172.21.15.107:0/2721777088 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f325c0fcf60 0x7f325c110aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:52:59.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:59.387+0000 7f32615ac700 1 -- 172.21.15.107:0/2721777088 >> 172.21.15.107:0/2721777088 conn(0x7f325c0f7430 msgr2=0x7f325c0f8b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:52:59.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:59.387+0000 7f32615ac700 1 -- 172.21.15.107:0/2721777088 shutdown_connections 2024-05-17T21:52:59.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:59.387+0000 7f32615ac700 1 -- 172.21.15.107:0/2721777088 wait complete. 2024-05-17T21:52:59.383 INFO:tasks.workunit.client.1.smithi138.stdout:7/377: dwrite d0/d18/d40/d41/f43 [0,4194304] 0 2024-05-17T21:52:59.384 INFO:tasks.workunit.client.0.smithi107.stdout:0/723: unlink d3/dc/d22/d23/f8f 0 2024-05-17T21:52:59.384 INFO:tasks.workunit.client.1.smithi138.stdout:7/378: chown d0/d18/d40/d41/d51/d66/c60 399355 1 2024-05-17T21:52:59.386 INFO:tasks.workunit.client.1.smithi138.stdout:7/379: truncate d0/d18/d40/f6c 827097 0 2024-05-17T21:52:59.386 INFO:tasks.workunit.client.0.smithi107.stdout:6/548: rename d9/d16/d22/f33 to d9/d16/d4d/faa 0 2024-05-17T21:52:59.387 INFO:tasks.workunit.client.0.smithi107.stdout:6/549: readlink d9/d16/l17 0 2024-05-17T21:52:59.388 INFO:tasks.workunit.client.0.smithi107.stdout:6/550: write d9/f1c [487217,39629] 0 2024-05-17T21:52:59.392 INFO:tasks.workunit.client.1.smithi138.stdout:1/438: sync 2024-05-17T21:52:59.392 INFO:tasks.workunit.client.0.smithi107.stdout:4/637: mkdir d3/db/d1e/d99/dd6 0 2024-05-17T21:52:59.393 INFO:tasks.workunit.client.1.smithi138.stdout:1/439: chown d4/df/d35/l38 116165978 1 2024-05-17T21:52:59.394 INFO:teuthology.orchestra.run.smithi107.stdout:true 2024-05-17T21:52:59.395 INFO:tasks.workunit.client.0.smithi107.stdout:1/677: getdents d4/df/d14/d2e/dd3 0 2024-05-17T21:52:59.395 INFO:tasks.workunit.client.1.smithi138.stdout:5/449: truncate d1/d6/d19/f20 485825 0 2024-05-17T21:52:59.397 INFO:tasks.workunit.client.0.smithi107.stdout:8/634: write dc/d21/d27/d2f/f7f [4118045,35234] 0 2024-05-17T21:52:59.401 INFO:tasks.workunit.client.0.smithi107.stdout:7/605: rename d5/d16/d3b/c6f to d5/df/d11/d1c/d2f/d65/da7/cda 0 2024-05-17T21:52:59.402 INFO:tasks.workunit.client.1.smithi138.stdout:6/496: mkdir d7/db/d43/d5f/d6a/d9b/db2 0 2024-05-17T21:52:59.404 INFO:tasks.workunit.client.0.smithi107.stdout:8/635: dwrite dc/f4e [0,4194304] 0 2024-05-17T21:52:59.405 INFO:tasks.workunit.client.1.smithi138.stdout:2/516: write d3/f9 [2990650,101082] 0 2024-05-17T21:52:59.406 INFO:tasks.workunit.client.1.smithi138.stdout:2/517: write d3/d21/d2f/d8a/fa2 [285197,127968] 0 2024-05-17T21:52:59.410 INFO:tasks.workunit.client.0.smithi107.stdout:6/551: rmdir d9 39 2024-05-17T21:52:59.412 INFO:tasks.workunit.client.0.smithi107.stdout:4/638: unlink d3/d4a/d47/d60/d3c/f9c 0 2024-05-17T21:52:59.413 INFO:tasks.workunit.client.1.smithi138.stdout:8/415: creat d0/dc/d3b/d4d/d7d/f7f x:0 0 0 2024-05-17T21:52:59.414 INFO:tasks.workunit.client.0.smithi107.stdout:8/636: dwrite dc/d21/d97/d63/f79 [4194304,4194304] 0 2024-05-17T21:52:59.414 INFO:tasks.workunit.client.0.smithi107.stdout:4/639: fdatasync d3/d4a/d47/d60/d3a/d41/d80/f44 0 2024-05-17T21:52:59.415 INFO:tasks.workunit.client.0.smithi107.stdout:1/678: symlink d4/df/d4e/le0 0 2024-05-17T21:52:59.417 INFO:tasks.workunit.client.1.smithi138.stdout:4/476: getdents d0/d2f/d49/d60/d1f 0 2024-05-17T21:52:59.421 INFO:tasks.workunit.client.1.smithi138.stdout:7/380: mkdir d0/d39/d78 0 2024-05-17T21:52:59.427 INFO:tasks.workunit.client.1.smithi138.stdout:6/497: dread f1 [0,4194304] 0 2024-05-17T21:52:59.428 INFO:tasks.workunit.client.0.smithi107.stdout:5/623: rename d5/db/d16/d54/d59/d92/dba to d5/db/dc/dc5 0 2024-05-17T21:52:59.428 INFO:tasks.workunit.client.1.smithi138.stdout:6/498: write d7/db/d43/d5f/f73 [179742,48403] 0 2024-05-17T21:52:59.430 INFO:tasks.workunit.client.0.smithi107.stdout:7/606: rmdir d5/df/d11/d1c/d31/d56 39 2024-05-17T21:52:59.431 INFO:tasks.workunit.client.0.smithi107.stdout:7/607: write d5/f83 [576014,2689] 0 2024-05-17T21:52:59.431 INFO:tasks.workunit.client.1.smithi138.stdout:1/440: symlink d4/df/d4e/l8b 0 2024-05-17T21:52:59.432 INFO:tasks.workunit.client.1.smithi138.stdout:1/441: write d4/db/f39 [351262,96827] 0 2024-05-17T21:52:59.433 INFO:tasks.workunit.client.0.smithi107.stdout:7/608: write d5/df/d11/d1c/d2f/fd2 [113714,106989] 0 2024-05-17T21:52:59.433 INFO:tasks.workunit.client.1.smithi138.stdout:5/450: truncate d1/d6/d1b/d29/d36/f37 1501966 0 2024-05-17T21:52:59.439 INFO:tasks.workunit.client.1.smithi138.stdout:2/518: fdatasync d3/d21/d2f/f3c 0 2024-05-17T21:52:59.440 INFO:tasks.workunit.client.1.smithi138.stdout:2/519: dread - d3/f8c zero size 2024-05-17T21:52:59.442 INFO:tasks.workunit.client.1.smithi138.stdout:3/564: creat d1/d3/d8d/fc7 x:0 0 0 2024-05-17T21:52:59.442 INFO:tasks.workunit.client.1.smithi138.stdout:5/451: dwrite d1/d6/f72 [0,4194304] 0 2024-05-17T21:52:59.443 INFO:tasks.workunit.client.0.smithi107.stdout:3/663: write d6/d2a/d2d/f43 [2405556,11130] 0 2024-05-17T21:52:59.446 INFO:tasks.workunit.client.0.smithi107.stdout:4/640: creat d3/db/d35/d55/d8a/fd7 x:0 0 0 2024-05-17T21:52:59.447 INFO:tasks.workunit.client.0.smithi107.stdout:4/641: readlink d3/d6/d26/l42 0 2024-05-17T21:52:59.450 INFO:tasks.workunit.client.1.smithi138.stdout:5/452: dwrite d1/f4f [0,4194304] 0 2024-05-17T21:52:59.452 INFO:tasks.workunit.client.1.smithi138.stdout:4/477: fdatasync d0/d3/f40 0 2024-05-17T21:52:59.455 INFO:tasks.workunit.client.0.smithi107.stdout:4/642: dwrite d3/d4a/f3f [0,4194304] 0 2024-05-17T21:52:59.456 INFO:tasks.workunit.client.1.smithi138.stdout:7/381: mknod d0/d18/d40/d41/d51/d66/c79 0 2024-05-17T21:52:59.457 INFO:tasks.workunit.client.0.smithi107.stdout:4/643: chown d3/d4a/d47/d60/c1c 45487351 1 2024-05-17T21:52:59.458 INFO:tasks.workunit.client.1.smithi138.stdout:7/382: readlink d0/d18/d40/d5f/l75 0 2024-05-17T21:52:59.459 INFO:tasks.workunit.client.0.smithi107.stdout:2/492: rename d0/d9/d11/df/d5a/c83 to d0/d9/d11/d15/d99/c9a 0 2024-05-17T21:52:59.461 INFO:tasks.workunit.client.1.smithi138.stdout:4/478: dwrite d0/d3/d17/fa9 [0,4194304] 0 2024-05-17T21:52:59.462 INFO:tasks.workunit.client.0.smithi107.stdout:5/624: unlink d5/db/l86 0 2024-05-17T21:52:59.463 INFO:tasks.workunit.client.1.smithi138.stdout:1/442: dread f0 [0,4194304] 0 2024-05-17T21:52:59.465 INFO:tasks.workunit.client.1.smithi138.stdout:7/383: read - d0/d18/d40/d41/d50/f53 zero size 2024-05-17T21:52:59.470 INFO:tasks.workunit.client.1.smithi138.stdout:6/499: fdatasync d7/db/d43/d3f/f46 0 2024-05-17T21:52:59.474 INFO:tasks.workunit.client.0.smithi107.stdout:9/662: truncate d0/dd/d1b/f53 166971 0 2024-05-17T21:52:59.475 INFO:tasks.workunit.client.0.smithi107.stdout:9/663: stat d0/dd/d21/d29/d3e/d45/d68/l95 0 2024-05-17T21:52:59.478 INFO:tasks.workunit.client.0.smithi107.stdout:0/724: truncate d3/dc/d22/d23/f44 7421394 0 2024-05-17T21:52:59.486 INFO:tasks.workunit.client.0.smithi107.stdout:7/609: creat d5/d16/d3b/d45/d91/dc1/fdb x:0 0 0 2024-05-17T21:52:59.488 INFO:tasks.workunit.client.1.smithi138.stdout:3/565: write d1/d3/f1c [4843456,94225] 0 2024-05-17T21:52:59.490 INFO:tasks.workunit.client.1.smithi138.stdout:3/566: write d1/d3/d16/d3e/f74 [709463,99767] 0 2024-05-17T21:52:59.490 INFO:tasks.workunit.client.1.smithi138.stdout:8/416: rmdir d0/dc/d1a/d33/d4a/d4e 39 2024-05-17T21:52:59.491 INFO:tasks.workunit.client.0.smithi107.stdout:1/679: dread d4/df/d1a/d26/d72/dbc/f9f [0,4194304] 0 2024-05-17T21:52:59.491 INFO:tasks.workunit.client.1.smithi138.stdout:3/567: write d1/d3/d16/f95 [4125273,108979] 0 2024-05-17T21:52:59.493 INFO:tasks.workunit.client.0.smithi107.stdout:1/680: readlink d4/d78/lb8 0 2024-05-17T21:52:59.494 INFO:tasks.workunit.client.1.smithi138.stdout:5/453: mkdir d1/d1a/d39/d62/d90 0 2024-05-17T21:52:59.494 INFO:tasks.workunit.client.0.smithi107.stdout:1/681: chown d4/df/d1a/d41/ldd 25617 1 2024-05-17T21:52:59.498 INFO:tasks.workunit.client.0.smithi107.stdout:8/637: dwrite dc/d44/d4b/d52/f9f [0,4194304] 0 2024-05-17T21:52:59.498 INFO:tasks.workunit.client.1.smithi138.stdout:8/417: dwrite d0/dc/d3b/d4d/d7d/f7f [0,4194304] 0 2024-05-17T21:52:59.499 INFO:tasks.workunit.client.0.smithi107.stdout:8/638: stat dc/d21/d69 0 2024-05-17T21:52:59.500 INFO:tasks.workunit.client.1.smithi138.stdout:0/481: getdents d3/d10/d38/d70/d80 0 2024-05-17T21:52:59.501 INFO:tasks.workunit.client.1.smithi138.stdout:8/418: chown d0/dc/d1a 6 1 2024-05-17T21:52:59.502 INFO:tasks.workunit.client.1.smithi138.stdout:3/568: dwrite d1/d62/dad/d5c/f81 [0,4194304] 0 2024-05-17T21:52:59.504 INFO:tasks.workunit.client.0.smithi107.stdout:7/610: unlink d5/d16/d3b/l97 0 2024-05-17T21:52:59.511 INFO:tasks.workunit.client.1.smithi138.stdout:4/479: rename d0/d2f/d49/c62 to d0/d3/d1d/d38/d9c/cad 0 2024-05-17T21:52:59.511 INFO:tasks.workunit.client.0.smithi107.stdout:3/664: write d6/d8/d10/f23 [3950086,66104] 0 2024-05-17T21:52:59.512 INFO:tasks.workunit.client.1.smithi138.stdout:0/482: dwrite d3/d10/d6d/f75 [0,4194304] 0 2024-05-17T21:52:59.516 INFO:tasks.workunit.client.0.smithi107.stdout:2/493: dread d0/d9/d1c/d22/f31 [4194304,4194304] 0 2024-05-17T21:52:59.517 INFO:tasks.workunit.client.1.smithi138.stdout:1/443: mkdir d4/dd/d3b/d8c 0 2024-05-17T21:52:59.519 INFO:tasks.workunit.client.1.smithi138.stdout:9/442: getdents d6/d7/d16/d29/d48/d6f 0 2024-05-17T21:52:59.524 INFO:tasks.workunit.client.1.smithi138.stdout:6/500: symlink d7/db/d43/d5f/d6a/d9b/lb3 0 2024-05-17T21:52:59.526 INFO:tasks.workunit.client.0.smithi107.stdout:5/625: creat d5/db/fc6 x:0 0 0 2024-05-17T21:52:59.528 INFO:tasks.workunit.client.0.smithi107.stdout:8/639: symlink dc/dd5/le4 0 2024-05-17T21:52:59.528 INFO:tasks.workunit.client.1.smithi138.stdout:2/520: dwrite d3/d21/f30 [0,4194304] 0 2024-05-17T21:52:59.528 INFO:tasks.workunit.client.0.smithi107.stdout:9/664: write d0/dd/d3d/fb8 [1336033,1522] 0 2024-05-17T21:52:59.529 INFO:tasks.workunit.client.0.smithi107.stdout:0/725: write d3/d39/fb0 [2528403,6874] 0 2024-05-17T21:52:59.531 INFO:tasks.workunit.client.0.smithi107.stdout:0/726: readlink d3/dc/l36 0 2024-05-17T21:52:59.531 INFO:tasks.workunit.client.0.smithi107.stdout:7/611: creat d5/df/d11/d1c/d31/d39/d92/d9a/fdc x:0 0 0 2024-05-17T21:52:59.532 INFO:tasks.workunit.client.1.smithi138.stdout:7/384: dread d0/f4a [0,4194304] 0 2024-05-17T21:52:59.532 INFO:tasks.workunit.client.1.smithi138.stdout:7/385: readlink d0/ld 0 2024-05-17T21:52:59.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:59.540+0000 7f87482b5700 1 -- 172.21.15.107:0/865235710 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f87400fcfa0 msgr2=0x7f87400fd410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:52:59.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:59.540+0000 7f87482b5700 1 --2- 172.21.15.107:0/865235710 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f87400fcfa0 0x7f87400fd410 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f8734004880 tx=0x7f8734009e30 comp rx=0 tx=0).stop 2024-05-17T21:52:59.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:59.540+0000 7f87482b5700 1 -- 172.21.15.107:0/865235710 shutdown_connections 2024-05-17T21:52:59.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:59.540+0000 7f87482b5700 1 --2- 172.21.15.107:0/865235710 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f87400fcfa0 0x7f87400fd410 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:52:59.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:59.540+0000 7f87482b5700 1 --2- 172.21.15.107:0/865235710 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87400fbe30 0x7f87400fc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:52:59.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:59.540+0000 7f87482b5700 1 -- 172.21.15.107:0/865235710 >> 172.21.15.107:0/865235710 conn(0x7f87400f7440 msgr2=0x7f87400f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:52:59.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:59.541+0000 7f87482b5700 1 -- 172.21.15.107:0/865235710 shutdown_connections 2024-05-17T21:52:59.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:59.541+0000 7f87482b5700 1 -- 172.21.15.107:0/865235710 wait complete. 2024-05-17T21:52:59.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:59.541+0000 7f87482b5700 1 Processor -- start 2024-05-17T21:52:59.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:59.541+0000 7f87482b5700 1 -- start start 2024-05-17T21:52:59.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:59.542+0000 7f87482b5700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f87400fbe30 0x7f8740109380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:52:59.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:59.542+0000 7f87482b5700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87400fcfa0 0x7f87401098c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:52:59.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:59.542+0000 7f87482b5700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8740109f10 con 0x7f87400fbe30 2024-05-17T21:52:59.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:59.542+0000 7f87482b5700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f874010a760 con 0x7f87400fcfa0 2024-05-17T21:52:59.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:59.542+0000 7f8746051700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f87400fbe30 0x7f8740109380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:52:59.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:59.542+0000 7f8746051700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f87400fbe30 0x7f8740109380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:35346/0 (socket says 172.21.15.107:35346) 2024-05-17T21:52:59.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:59.542+0000 7f8746051700 1 -- 172.21.15.107:0/1971363552 learned_addr learned my addr 172.21.15.107:0/1971363552 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:52:59.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:59.542+0000 7f8745850700 1 --2- 172.21.15.107:0/1971363552 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87400fcfa0 0x7f87401098c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:52:59.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:59.543+0000 7f8746051700 1 -- 172.21.15.107:0/1971363552 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87400fcfa0 msgr2=0x7f87401098c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:52:59.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:59.543+0000 7f8746051700 1 --2- 172.21.15.107:0/1971363552 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87400fcfa0 0x7f87401098c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:52:59.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:59.543+0000 7f8746051700 1 -- 172.21.15.107:0/1971363552 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8734005040 con 0x7f87400fbe30 2024-05-17T21:52:59.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:59.543+0000 7f8746051700 1 --2- 172.21.15.107:0/1971363552 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f87400fbe30 0x7f8740109380 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f873c0041d0 tx=0x7f873c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:52:59.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:59.543+0000 7f8732ffd700 1 -- 172.21.15.107:0/1971363552 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f873c007cc0 con 0x7f87400fbe30 2024-05-17T21:52:59.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:59.543+0000 7f87482b5700 1 -- 172.21.15.107:0/1971363552 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f874010a9e0 con 0x7f87400fbe30 2024-05-17T21:52:59.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:59.543+0000 7f87482b5700 1 -- 172.21.15.107:0/1971363552 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f874010afa0 con 0x7f87400fbe30 2024-05-17T21:52:59.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:59.543+0000 7f8732ffd700 1 -- 172.21.15.107:0/1971363552 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f873c00f040 con 0x7f87400fbe30 2024-05-17T21:52:59.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:59.543+0000 7f8732ffd700 1 -- 172.21.15.107:0/1971363552 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f873c00ca50 con 0x7f87400fbe30 2024-05-17T21:52:59.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:59.545+0000 7f87482b5700 1 -- 172.21.15.107:0/1971363552 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87240052e0 con 0x7f87400fbe30 2024-05-17T21:52:59.537 INFO:tasks.workunit.client.0.smithi107.stdout:2/494: rmdir d0/d9/d11/d15 39 2024-05-17T21:52:59.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:59.546+0000 7f8732ffd700 1 -- 172.21.15.107:0/1971363552 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) v1 ==== 93316+0+0 (secure 0 0 0) 0x7f873c009730 con 0x7f87400fbe30 2024-05-17T21:52:59.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:59.546+0000 7f8732ffd700 1 --2- 172.21.15.107:0/1971363552 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7f872c070bb0 0x7f872c073060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:52:59.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:59.547+0000 7f8732ffd700 1 -- 172.21.15.107:0/1971363552 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f873c090400 con 0x7f87400fbe30 2024-05-17T21:52:59.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:59.547+0000 7f8745850700 1 --2- 172.21.15.107:0/1971363552 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7f872c070bb0 0x7f872c073060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:52:59.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:59.548+0000 7f8745850700 1 --2- 172.21.15.107:0/1971363552 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7f872c070bb0 0x7f872c073060 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f8734000c00 tx=0x7f8734009e30 comp rx=0 tx=0).ready entity=mgr.14636 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:52:59.541 INFO:tasks.workunit.client.0.smithi107.stdout:4/644: getdents d3/d4a/d47/d60/d3a 0 2024-05-17T21:52:59.542 INFO:tasks.workunit.client.0.smithi107.stdout:1/682: getdents d4/df/d1a/d41/db2/dc8 0 2024-05-17T21:52:59.545 INFO:tasks.workunit.client.0.smithi107.stdout:5/626: mkdir d5/d75/dc7 0 2024-05-17T21:52:59.545 INFO:tasks.workunit.client.1.smithi138.stdout:5/454: symlink d1/d1a/d39/d4e/l91 0 2024-05-17T21:52:59.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:59.553+0000 7f8732ffd700 1 -- 172.21.15.107:0/1971363552 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+190121 (secure 0 0 0) 0x7f873c0516c0 con 0x7f87400fbe30 2024-05-17T21:52:59.546 INFO:tasks.workunit.client.1.smithi138.stdout:5/455: chown d1/d60/f64 870348 1 2024-05-17T21:52:59.548 INFO:tasks.workunit.client.1.smithi138.stdout:0/483: dread d3/d10/f2d [0,4194304] 0 2024-05-17T21:52:59.548 INFO:tasks.workunit.client.0.smithi107.stdout:3/665: mknod d6/d8/db6/cd8 0 2024-05-17T21:52:59.549 INFO:tasks.workunit.client.0.smithi107.stdout:3/666: chown d6/d2a/d2d/c44 1 1 2024-05-17T21:52:59.549 INFO:tasks.workunit.client.1.smithi138.stdout:3/569: symlink d1/d3/d49/lc8 0 2024-05-17T21:52:59.551 INFO:tasks.workunit.client.1.smithi138.stdout:3/570: read d1/d3/d8d/d92/f96 [3008750,18148] 0 2024-05-17T21:52:59.552 INFO:tasks.workunit.client.1.smithi138.stdout:3/571: dread - d1/fc3 zero size 2024-05-17T21:52:59.556 INFO:tasks.workunit.client.1.smithi138.stdout:4/480: symlink d0/d2f/d49/d60/d1f/d7e/lae 0 2024-05-17T21:52:59.560 INFO:tasks.workunit.client.0.smithi107.stdout:8/640: dread dc/d21/d27/d34/d4c/d7e/f45 [0,4194304] 0 2024-05-17T21:52:59.563 INFO:tasks.workunit.client.0.smithi107.stdout:5/627: dread d5/db/d16/f81 [0,4194304] 0 2024-05-17T21:52:59.564 INFO:tasks.workunit.client.1.smithi138.stdout:6/501: creat d7/d8/fb4 x:0 0 0 2024-05-17T21:52:59.565 INFO:tasks.workunit.client.1.smithi138.stdout:8/419: dwrite d0/d11/f69 [4194304,4194304] 0 2024-05-17T21:52:59.566 INFO:tasks.workunit.client.1.smithi138.stdout:8/420: chown d0/dc/d1a/d33/l6f 396 1 2024-05-17T21:52:59.568 INFO:tasks.workunit.client.0.smithi107.stdout:3/667: unlink d6/d8/d3e/d3f/d90/fb5 0 2024-05-17T21:52:59.569 INFO:tasks.workunit.client.0.smithi107.stdout:9/665: truncate d0/dd/d3d/fb4 2643161 0 2024-05-17T21:52:59.571 INFO:tasks.workunit.client.1.smithi138.stdout:5/456: creat d1/d1a/d39/d69/f92 x:0 0 0 2024-05-17T21:52:59.572 INFO:tasks.workunit.client.0.smithi107.stdout:4/645: mknod d3/d6/d26/cd8 0 2024-05-17T21:52:59.573 INFO:tasks.workunit.client.1.smithi138.stdout:0/484: mknod d3/d10/d38/d70/d80/ca3 0 2024-05-17T21:52:59.574 INFO:tasks.workunit.client.0.smithi107.stdout:4/646: chown d3/d6/d26/d36/d7e/lce 1716964003 1 2024-05-17T21:52:59.574 INFO:tasks.workunit.client.0.smithi107.stdout:0/727: truncate d3/dc/d22/d3d/f4d 1126470 0 2024-05-17T21:52:59.574 INFO:tasks.workunit.client.1.smithi138.stdout:0/485: stat d3/d3d/d48/c5c 0 2024-05-17T21:52:59.577 INFO:tasks.workunit.client.1.smithi138.stdout:3/572: creat d1/d3/d16/d3e/d71/fc9 x:0 0 0 2024-05-17T21:52:59.578 INFO:tasks.workunit.client.1.smithi138.stdout:5/457: dwrite d1/d6/fa [4194304,4194304] 0 2024-05-17T21:52:59.578 INFO:tasks.workunit.client.0.smithi107.stdout:9/666: dwrite d0/dd/d2f/fd9 [0,4194304] 0 2024-05-17T21:52:59.579 INFO:tasks.workunit.client.1.smithi138.stdout:4/481: unlink d0/d1b/d42/d45/f48 0 2024-05-17T21:52:59.579 INFO:tasks.workunit.client.0.smithi107.stdout:9/667: write d0/f7 [66377,13817] 0 2024-05-17T21:52:59.580 INFO:tasks.workunit.client.0.smithi107.stdout:8/641: rmdir dc/d44/d4b/d52/d70/d76 39 2024-05-17T21:52:59.581 INFO:tasks.workunit.client.0.smithi107.stdout:8/642: chown dc/d21/d97/d63/d71/f94 7957 1 2024-05-17T21:52:59.581 INFO:tasks.workunit.client.0.smithi107.stdout:5/628: creat d5/db/dc/d1e/d6e/fc8 x:0 0 0 2024-05-17T21:52:59.582 INFO:tasks.workunit.client.0.smithi107.stdout:5/629: readlink d5/db/d16/d2d/l5d 0 2024-05-17T21:52:59.582 INFO:tasks.workunit.client.1.smithi138.stdout:9/443: getdents d6/d7/d16/d29/d48/d6e/d74/d75/d76/d8e 0 2024-05-17T21:52:59.583 INFO:tasks.workunit.client.0.smithi107.stdout:5/630: stat d5/db/d16/d54/d59/d65/d48/d89/fa8 0 2024-05-17T21:52:59.590 INFO:tasks.workunit.client.1.smithi138.stdout:9/444: dwrite d6/d7/f2c [0,4194304] 0 2024-05-17T21:52:59.591 INFO:tasks.workunit.client.0.smithi107.stdout:3/668: creat d6/d8/d3e/dc9/fd9 x:0 0 0 2024-05-17T21:52:59.591 INFO:tasks.workunit.client.1.smithi138.stdout:6/502: rmdir d7/db 39 2024-05-17T21:52:59.596 INFO:tasks.workunit.client.1.smithi138.stdout:2/521: mkdir d3/da0/daa 0 2024-05-17T21:52:59.596 INFO:tasks.workunit.client.0.smithi107.stdout:1/683: truncate d4/df/d1a/d26/d7a/f93 955259 0 2024-05-17T21:52:59.603 INFO:tasks.workunit.client.0.smithi107.stdout:0/728: unlink d3/d24/d6f/d9f/dab/f7c 0 2024-05-17T21:52:59.603 INFO:tasks.workunit.client.0.smithi107.stdout:6/552: sync 2024-05-17T21:52:59.606 INFO:tasks.workunit.client.0.smithi107.stdout:4/647: creat d3/d4a/d47/d60/d3c/fd9 x:0 0 0 2024-05-17T21:52:59.609 INFO:tasks.workunit.client.1.smithi138.stdout:0/486: truncate d3/d3d/f44 455704 0 2024-05-17T21:52:59.611 INFO:tasks.workunit.client.0.smithi107.stdout:8/643: stat dc/c1a 0 2024-05-17T21:52:59.613 INFO:tasks.workunit.client.0.smithi107.stdout:5/631: mkdir d5/db/d16/d2d/d40/d9f/dc9 0 2024-05-17T21:52:59.615 INFO:tasks.workunit.client.1.smithi138.stdout:3/573: symlink d1/d4f/d80/lca 0 2024-05-17T21:52:59.616 INFO:tasks.workunit.client.1.smithi138.stdout:0/487: dread d3/d10/d38/d3e/f67 [0,4194304] 0 2024-05-17T21:52:59.616 INFO:tasks.workunit.client.1.smithi138.stdout:3/574: write d1/d62/dad/de/f3f [2571012,52062] 0 2024-05-17T21:52:59.616 INFO:tasks.workunit.client.1.smithi138.stdout:8/421: write d0/dc/d1a/f2a [653739,103117] 0 2024-05-17T21:52:59.617 INFO:tasks.workunit.client.1.smithi138.stdout:0/488: fdatasync d3/d10/d38/d70/d80/f8b 0 2024-05-17T21:52:59.619 INFO:tasks.workunit.client.0.smithi107.stdout:2/495: getdents d0 0 2024-05-17T21:52:59.621 INFO:tasks.workunit.client.0.smithi107.stdout:1/684: mkdir d4/df/d1a/d26/d72/de1 0 2024-05-17T21:52:59.626 INFO:tasks.workunit.client.1.smithi138.stdout:6/503: mkdir d7/d69/db5 0 2024-05-17T21:52:59.630 INFO:tasks.workunit.client.1.smithi138.stdout:9/445: creat d6/d7/d16/d29/d38/d40/d44/f90 x:0 0 0 2024-05-17T21:52:59.630 INFO:tasks.workunit.client.0.smithi107.stdout:9/668: dwrite d0/dd/d21/d29/d3e/d45/f61 [0,4194304] 0 2024-05-17T21:52:59.631 INFO:tasks.workunit.client.1.smithi138.stdout:6/504: dread d7/d69/f71 [0,4194304] 0 2024-05-17T21:52:59.631 INFO:tasks.workunit.client.1.smithi138.stdout:9/446: write d6/f33 [6024642,108053] 0 2024-05-17T21:52:59.632 INFO:tasks.workunit.client.1.smithi138.stdout:2/522: fdatasync d3/d81/fa7 0 2024-05-17T21:52:59.634 INFO:tasks.workunit.client.0.smithi107.stdout:4/648: mknod d3/d4a/d47/d60/d3a/d41/cda 0 2024-05-17T21:52:59.634 INFO:tasks.workunit.client.0.smithi107.stdout:3/669: write d6/d1a/f96 [3828848,57143] 0 2024-05-17T21:52:59.636 INFO:tasks.workunit.client.1.smithi138.stdout:7/386: link d0/d18/c36 d0/d18/d40/d41/d51/d66/c7a 0 2024-05-17T21:52:59.639 INFO:tasks.workunit.client.1.smithi138.stdout:5/458: dwrite d1/d60/f64 [0,4194304] 0 2024-05-17T21:52:59.641 INFO:tasks.workunit.client.0.smithi107.stdout:8/644: read dc/d44/d4b/d52/d70/d76/f78 [308037,117086] 0 2024-05-17T21:52:59.642 INFO:tasks.workunit.client.0.smithi107.stdout:8/645: chown dc/d21/d97/db7/fcc 97560 1 2024-05-17T21:52:59.647 INFO:tasks.workunit.client.0.smithi107.stdout:6/553: unlink d9/d58/d7b/f8c 0 2024-05-17T21:52:59.649 INFO:tasks.workunit.client.1.smithi138.stdout:1/444: getdents d4/db/d29/d89 0 2024-05-17T21:52:59.650 INFO:tasks.workunit.client.1.smithi138.stdout:1/445: chown d4/dd/d64/l72 140177484 1 2024-05-17T21:52:59.650 INFO:tasks.workunit.client.0.smithi107.stdout:0/729: mkdir d3/dc/d22/d3d/df9 0 2024-05-17T21:52:59.654 INFO:tasks.workunit.client.0.smithi107.stdout:3/670: fdatasync d6/fc1 0 2024-05-17T21:52:59.655 INFO:tasks.workunit.client.1.smithi138.stdout:0/489: dread d3/d10/d38/d3e/d59/f68 [0,4194304] 0 2024-05-17T21:52:59.655 INFO:tasks.workunit.client.1.smithi138.stdout:9/447: rmdir d6/d7/d16/d29/d48/d6f 39 2024-05-17T21:52:59.657 INFO:tasks.workunit.client.0.smithi107.stdout:4/649: mknod d3/d6/d26/d36/cdb 0 2024-05-17T21:52:59.657 INFO:tasks.workunit.client.0.smithi107.stdout:4/650: stat d3/db/d35/l8e 0 2024-05-17T21:52:59.658 INFO:tasks.workunit.client.1.smithi138.stdout:7/387: mknod d0/d18/d40/d5f/c7b 0 2024-05-17T21:52:59.658 INFO:tasks.workunit.client.0.smithi107.stdout:9/669: dread d0/dd/f46 [0,4194304] 0 2024-05-17T21:52:59.659 INFO:tasks.workunit.client.0.smithi107.stdout:9/670: fsync d0/d80/fd0 0 2024-05-17T21:52:59.660 INFO:tasks.workunit.client.1.smithi138.stdout:7/388: truncate d0/d18/d40/d41/d51/d66/f6f 761098 0 2024-05-17T21:52:59.661 INFO:tasks.workunit.client.1.smithi138.stdout:7/389: read d0/d18/d40/d41/f43 [3772084,86733] 0 2024-05-17T21:52:59.662 INFO:tasks.workunit.client.1.smithi138.stdout:7/390: write d0/d18/d40/d41/f5b [161211,106179] 0 2024-05-17T21:52:59.667 INFO:tasks.workunit.client.1.smithi138.stdout:4/482: truncate d0/d2f/d49/d60/f3c 465300 0 2024-05-17T21:52:59.671 INFO:tasks.workunit.client.1.smithi138.stdout:3/575: truncate d1/d3/f1b 1253858 0 2024-05-17T21:52:59.671 INFO:tasks.workunit.client.0.smithi107.stdout:8/646: dread dc/d21/d27/d34/fd4 [4194304,4194304] 0 2024-05-17T21:52:59.674 INFO:tasks.workunit.client.1.smithi138.stdout:1/446: symlink d4/df/d3a/d3c/d67/l8d 0 2024-05-17T21:52:59.674 INFO:tasks.workunit.client.1.smithi138.stdout:4/483: dwrite d0/d1b/d2b/d55/f7d [0,4194304] 0 2024-05-17T21:52:59.675 INFO:tasks.workunit.client.0.smithi107.stdout:5/632: truncate d5/db/f1a 6537131 0 2024-05-17T21:52:59.675 INFO:tasks.workunit.client.0.smithi107.stdout:5/633: chown d5/db/d16/d2a/d73 645725 1 2024-05-17T21:52:59.676 INFO:tasks.workunit.client.0.smithi107.stdout:0/730: creat d3/dc/d22/d23/d97/d77/d91/ffa x:0 0 0 2024-05-17T21:52:59.680 INFO:tasks.workunit.client.0.smithi107.stdout:3/671: creat d6/d8/d10/d25/fda x:0 0 0 2024-05-17T21:52:59.681 INFO:tasks.workunit.client.1.smithi138.stdout:3/576: dwrite d1/d3/f25 [0,4194304] 0 2024-05-17T21:52:59.682 INFO:tasks.workunit.client.0.smithi107.stdout:4/651: fsync d3/d4a/f73 0 2024-05-17T21:52:59.683 INFO:tasks.workunit.client.1.smithi138.stdout:9/448: dread - d6/d7/d16/d29/d48/f4c zero size 2024-05-17T21:52:59.685 INFO:tasks.workunit.client.1.smithi138.stdout:9/449: truncate d6/d7/d16/d29/d48/d3b/f63 550583 0 2024-05-17T21:52:59.688 INFO:tasks.workunit.client.0.smithi107.stdout:9/671: unlink d0/dd/d3d/d9d/ccc 0 2024-05-17T21:52:59.691 INFO:tasks.workunit.client.0.smithi107.stdout:6/554: dwrite d9/d16/f1e [0,4194304] 0 2024-05-17T21:52:59.691 INFO:tasks.workunit.client.0.smithi107.stdout:6/555: readlink d9/d16/d4d/l99 0 2024-05-17T21:52:59.693 INFO:tasks.workunit.client.1.smithi138.stdout:9/450: dread d6/d7/d16/d29/d38/f81 [0,4194304] 0 2024-05-17T21:52:59.693 INFO:tasks.workunit.client.0.smithi107.stdout:2/496: rename d0/d9/d1c/d3e/d4f/f61 to d0/d9/d11/f9b 0 2024-05-17T21:52:59.694 INFO:tasks.workunit.client.1.smithi138.stdout:9/451: write d6/d7/d16/d29/f3a [1511901,24411] 0 2024-05-17T21:52:59.695 INFO:tasks.workunit.client.0.smithi107.stdout:2/497: write d0/d9/d11/df/f35 [1121185,70490] 0 2024-05-17T21:52:59.696 INFO:tasks.workunit.client.0.smithi107.stdout:5/634: creat d5/db/d16/d2d/d40/fca x:0 0 0 2024-05-17T21:52:59.702 INFO:tasks.workunit.client.1.smithi138.stdout:4/484: rmdir d0/d1b/d42/d45/d84 39 2024-05-17T21:52:59.704 INFO:tasks.workunit.client.1.smithi138.stdout:0/490: creat d3/d10/d38/d70/d80/d94/fa4 x:0 0 0 2024-05-17T21:52:59.705 INFO:tasks.workunit.client.1.smithi138.stdout:0/491: fsync d3/fc 0 2024-05-17T21:52:59.705 INFO:tasks.workunit.client.0.smithi107.stdout:6/556: symlink d9/d16/d22/d5f/d6e/lab 0 2024-05-17T21:52:59.706 INFO:tasks.workunit.client.1.smithi138.stdout:0/492: chown d3/d10/d6d/f75 142 1 2024-05-17T21:52:59.708 INFO:tasks.workunit.client.1.smithi138.stdout:3/577: unlink d1/d3/d16/d2d/fab 0 2024-05-17T21:52:59.712 INFO:tasks.workunit.client.0.smithi107.stdout:8/647: rename dc/d21/d27/d34/d4c/db1 to dc/d21/d97/d63/d71/d81/de5 0 2024-05-17T21:52:59.712 INFO:tasks.workunit.client.1.smithi138.stdout:5/459: write d1/d6/d1b/d29/d36/f37 [338304,91634] 0 2024-05-17T21:52:59.719 INFO:tasks.workunit.client.0.smithi107.stdout:4/652: dread d3/d4a/d47/d60/d3a/d41/d80/f94 [0,4194304] 0 2024-05-17T21:52:59.720 INFO:tasks.workunit.client.1.smithi138.stdout:1/447: truncate d4/f3f 363219 0 2024-05-17T21:52:59.721 INFO:tasks.workunit.client.1.smithi138.stdout:4/485: creat d0/d3/db/faf x:0 0 0 2024-05-17T21:52:59.731 INFO:tasks.workunit.client.0.smithi107.stdout:6/557: dwrite d9/d16/f24 [0,4194304] 0 2024-05-17T21:52:59.734 INFO:tasks.workunit.client.1.smithi138.stdout:0/493: write d3/d10/d38/d70/d80/f93 [553974,105512] 0 2024-05-17T21:52:59.735 INFO:tasks.workunit.client.1.smithi138.stdout:0/494: fsync d3/f84 0 2024-05-17T21:52:59.748 INFO:tasks.workunit.client.1.smithi138.stdout:9/452: symlink d6/l91 0 2024-05-17T21:52:59.749 INFO:tasks.workunit.client.0.smithi107.stdout:7/612: sync 2024-05-17T21:52:59.754 INFO:tasks.workunit.client.1.smithi138.stdout:7/391: getdents d0/d18 0 2024-05-17T21:52:59.754 INFO:tasks.workunit.client.0.smithi107.stdout:5/635: symlink d5/db/d16/d2d/d40/d9f/dc9/lcb 0 2024-05-17T21:52:59.756 INFO:tasks.workunit.client.1.smithi138.stdout:7/392: dread d0/d18/d40/d41/d51/d66/f6f [0,4194304] 0 2024-05-17T21:52:59.757 INFO:tasks.workunit.client.0.smithi107.stdout:3/672: rename d6/d8/d10/d35/d36/d66/d88/la4 to d6/d1a/d57/ldb 0 2024-05-17T21:52:59.758 INFO:tasks.workunit.client.0.smithi107.stdout:0/731: creat d3/dc/d22/d23/d97/ffb x:0 0 0 2024-05-17T21:52:59.760 INFO:tasks.workunit.client.1.smithi138.stdout:9/453: dread d6/d7/d16/f20 [0,4194304] 0 2024-05-17T21:52:59.760 INFO:tasks.workunit.client.0.smithi107.stdout:0/732: write d3/d24/ff8 [783686,125319] 0 2024-05-17T21:52:59.763 INFO:tasks.workunit.client.0.smithi107.stdout:0/733: write d3/d58/d61/f93 [4861889,92042] 0 2024-05-17T21:52:59.764 INFO:tasks.workunit.client.0.smithi107.stdout:3/673: dwrite d6/d2a/f78 [0,4194304] 0 2024-05-17T21:52:59.766 INFO:tasks.workunit.client.0.smithi107.stdout:4/653: mkdir d3/d4a/d47/d60/d3a/ddc 0 2024-05-17T21:52:59.766 INFO:tasks.workunit.client.0.smithi107.stdout:6/558: rmdir d9/d58/d66 39 2024-05-17T21:52:59.767 INFO:tasks.workunit.client.0.smithi107.stdout:4/654: dread - d3/d4a/d47/d60/d3a/fd0 zero size 2024-05-17T21:52:59.768 INFO:tasks.workunit.client.0.smithi107.stdout:4/655: readlink d3/d4a/d47/d60/d3a/d41/d80/l18 0 2024-05-17T21:52:59.815 INFO:tasks.workunit.client.1.smithi138.stdout:5/460: creat d1/d1a/f93 x:0 0 0 2024-05-17T21:52:59.816 INFO:tasks.workunit.client.1.smithi138.stdout:5/461: chown d1/d6/l18 40463964 1 2024-05-17T21:52:59.818 INFO:tasks.workunit.client.0.smithi107.stdout:5/636: mknod d5/db/d16/d2d/d40/d5a/ccc 0 2024-05-17T21:52:59.819 INFO:tasks.workunit.client.0.smithi107.stdout:5/637: dread - d5/db/dc/d1e/d6e/fc8 zero size 2024-05-17T21:52:59.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:59.841+0000 7f87482b5700 1 -- 172.21.15.107:0/1971363552 --> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8724000bf0 con 0x7f872c070bb0 2024-05-17T21:52:59.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:59.843+0000 7f8732ffd700 1 -- 172.21.15.107:0/1971363552 <== mgr.14636 v2:172.21.15.138:6828/1369055740 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+349 (secure 0 0 0) 0x7f8724000bf0 con 0x7f872c070bb0 2024-05-17T21:52:59.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:59.845+0000 7f87482b5700 1 -- 172.21.15.107:0/1971363552 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7f872c070bb0 msgr2=0x7f872c073060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:52:59.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:59.845+0000 7f87482b5700 1 --2- 172.21.15.107:0/1971363552 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7f872c070bb0 0x7f872c073060 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f8734000c00 tx=0x7f8734009e30 comp rx=0 tx=0).stop 2024-05-17T21:52:59.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:59.845+0000 7f87482b5700 1 -- 172.21.15.107:0/1971363552 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f87400fbe30 msgr2=0x7f8740109380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:52:59.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:59.845+0000 7f87482b5700 1 --2- 172.21.15.107:0/1971363552 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f87400fbe30 0x7f8740109380 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f873c0041d0 tx=0x7f873c00b040 comp rx=0 tx=0).stop 2024-05-17T21:52:59.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:59.845+0000 7f87482b5700 1 -- 172.21.15.107:0/1971363552 shutdown_connections 2024-05-17T21:52:59.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:59.845+0000 7f87482b5700 1 --2- 172.21.15.107:0/1971363552 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7f872c070bb0 0x7f872c073060 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:52:59.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:59.845+0000 7f87482b5700 1 --2- 172.21.15.107:0/1971363552 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f87400fbe30 0x7f8740109380 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:52:59.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:59.845+0000 7f87482b5700 1 --2- 172.21.15.107:0/1971363552 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87400fcfa0 0x7f87401098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:52:59.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:59.845+0000 7f87482b5700 1 -- 172.21.15.107:0/1971363552 >> 172.21.15.107:0/1971363552 conn(0x7f87400f7440 msgr2=0x7f87400f8ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:52:59.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:59.845+0000 7f87482b5700 1 -- 172.21.15.107:0/1971363552 shutdown_connections 2024-05-17T21:52:59.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:59.845+0000 7f87482b5700 1 -- 172.21.15.107:0/1971363552 wait complete. 2024-05-17T21:52:59.839 INFO:tasks.workunit.client.1.smithi138.stdout:9/454: creat d6/d7/d16/d29/d6c/d83/f92 x:0 0 0 2024-05-17T21:52:59.840 INFO:tasks.workunit.client.1.smithi138.stdout:1/448: link d4/df/d35/l45 d4/db/d29/d89/l8e 0 2024-05-17T21:52:59.843 INFO:tasks.workunit.client.1.smithi138.stdout:3/578: rename d1/d4f/l5b to d1/d4f/lcb 0 2024-05-17T21:52:59.844 INFO:tasks.workunit.client.0.smithi107.stdout:3/674: fsync d6/d2a/d2d/d37/f3a 0 2024-05-17T21:52:59.844 INFO:tasks.workunit.client.1.smithi138.stdout:3/579: write d1/d4f/d80/f86 [30229,16796] 0 2024-05-17T21:52:59.846 INFO:tasks.workunit.client.1.smithi138.stdout:3/580: chown d1/d3/d16/d3e/d71/fa9 19 1 2024-05-17T21:52:59.847 INFO:tasks.workunit.client.0.smithi107.stdout:6/559: creat d9/d16/d22/d5f/fac x:0 0 0 2024-05-17T21:52:59.849 INFO:tasks.workunit.client.1.smithi138.stdout:0/495: link d3/d10/f2c d3/d10/d6d/fa5 0 2024-05-17T21:52:59.851 INFO:tasks.workunit.client.0.smithi107.stdout:7/613: creat d5/d16/dd3/fdd x:0 0 0 2024-05-17T21:52:59.852 INFO:tasks.workunit.client.1.smithi138.stdout:1/449: dread d4/db/f22 [0,4194304] 0 2024-05-17T21:52:59.853 INFO:tasks.workunit.client.0.smithi107.stdout:2/498: getdents d0/d9 0 2024-05-17T21:52:59.857 INFO:tasks.workunit.client.1.smithi138.stdout:3/581: rename d1/d3/d16/d3a/l41 to d1/d4d/lcc 0 2024-05-17T21:52:59.859 INFO:tasks.workunit.client.0.smithi107.stdout:3/675: readlink d6/d2a/d2d/d69/l98 0 2024-05-17T21:52:59.859 INFO:tasks.workunit.client.0.smithi107.stdout:3/676: fsync d6/d8/fc0 0 2024-05-17T21:52:59.862 INFO:tasks.workunit.client.1.smithi138.stdout:9/455: symlink d6/d7/d16/d29/d48/l93 0 2024-05-17T21:52:59.864 INFO:tasks.workunit.client.0.smithi107.stdout:7/614: unlink d5/df/d11/d1d/d44/f50 0 2024-05-17T21:52:59.865 INFO:tasks.workunit.client.1.smithi138.stdout:1/450: truncate d4/df/f1f 4393001 0 2024-05-17T21:52:59.867 INFO:tasks.workunit.client.0.smithi107.stdout:8/648: write dc/d21/f5d [284756,71546] 0 2024-05-17T21:52:59.873 INFO:tasks.workunit.client.0.smithi107.stdout:6/560: symlink d9/d58/d66/lad 0 2024-05-17T21:52:59.874 INFO:tasks.workunit.client.1.smithi138.stdout:5/462: creat d1/d6/d1b/f94 x:0 0 0 2024-05-17T21:52:59.876 INFO:tasks.workunit.client.0.smithi107.stdout:4/656: rename d3/d4a/d47/la8 to d3/db/d1e/ldd 0 2024-05-17T21:52:59.876 INFO:tasks.workunit.client.0.smithi107.stdout:6/561: dread d9/d16/f5e [0,4194304] 0 2024-05-17T21:52:59.877 INFO:tasks.workunit.client.0.smithi107.stdout:7/615: dread d5/d16/f1e [0,4194304] 0 2024-05-17T21:52:59.881 INFO:tasks.workunit.client.1.smithi138.stdout:1/451: rmdir d4/dd/d3b 39 2024-05-17T21:52:59.883 INFO:tasks.workunit.client.0.smithi107.stdout:2/499: creat d0/d26/d8e/f9c x:0 0 0 2024-05-17T21:52:59.885 INFO:tasks.workunit.client.0.smithi107.stdout:3/677: dread d6/d2a/d2d/f41 [0,4194304] 0 2024-05-17T21:52:59.885 INFO:tasks.workunit.client.0.smithi107.stdout:3/678: readlink d6/l7d 0 2024-05-17T21:52:59.885 INFO:tasks.workunit.client.1.smithi138.stdout:5/463: rename d1/d1a/d8e to d1/d46/d95 0 2024-05-17T21:52:59.886 INFO:tasks.workunit.client.1.smithi138.stdout:1/452: rename d4/df/d5f to d4/df/d5f/d8f 22 2024-05-17T21:52:59.886 INFO:tasks.workunit.client.1.smithi138.stdout:5/464: stat d1/d1a/d39/d69 0 2024-05-17T21:52:59.890 INFO:tasks.workunit.client.0.smithi107.stdout:4/657: dread d3/f2c [0,4194304] 0 2024-05-17T21:52:59.894 INFO:tasks.workunit.client.0.smithi107.stdout:7/616: mkdir d5/d16/d3b/d45/d91/dc1/dde 0 2024-05-17T21:52:59.895 INFO:tasks.workunit.client.0.smithi107.stdout:7/617: stat d5/df/d11/d2a/c2d 0 2024-05-17T21:52:59.895 INFO:tasks.workunit.client.1.smithi138.stdout:4/486: dwrite d0/d2f/d49/d60/d1f/d32/f43 [0,4194304] 0 2024-05-17T21:52:59.897 INFO:tasks.workunit.client.1.smithi138.stdout:4/487: write d0/d3/d17/fa9 [3303966,56446] 0 2024-05-17T21:52:59.897 INFO:tasks.workunit.client.1.smithi138.stdout:5/465: mkdir d1/d53/d96 0 2024-05-17T21:52:59.898 INFO:tasks.workunit.client.1.smithi138.stdout:0/496: getdents d3/d10/d38/d70/d80/d94 0 2024-05-17T21:52:59.900 INFO:tasks.workunit.client.1.smithi138.stdout:0/497: chown d3/d10/c26 2733366 1 2024-05-17T21:52:59.900 INFO:tasks.workunit.client.0.smithi107.stdout:2/500: mknod d0/d9/d1c/d37/c9d 0 2024-05-17T21:52:59.901 INFO:tasks.workunit.client.0.smithi107.stdout:5/638: dwrite d5/db/d16/d54/d59/fbd [0,4194304] 0 2024-05-17T21:52:59.902 INFO:tasks.workunit.client.0.smithi107.stdout:5/639: chown d5/db/dc/d1e/fb5 5042 1 2024-05-17T21:52:59.904 INFO:tasks.workunit.client.1.smithi138.stdout:3/582: dwrite d1/d4f/d87/fbc [0,4194304] 0 2024-05-17T21:52:59.906 INFO:tasks.workunit.client.1.smithi138.stdout:3/583: fdatasync d1/d4f/d80/fa7 0 2024-05-17T21:52:59.906 INFO:tasks.workunit.client.0.smithi107.stdout:8/649: link dc/d44/fdc dc/d21/daa/fe6 0 2024-05-17T21:52:59.910 INFO:tasks.workunit.client.0.smithi107.stdout:4/658: dread - d3/db/f72 zero size 2024-05-17T21:52:59.911 INFO:tasks.workunit.client.1.smithi138.stdout:4/488: dread d0/d3/f40 [0,4194304] 0 2024-05-17T21:52:59.912 INFO:tasks.workunit.client.0.smithi107.stdout:4/659: write d3/db/f69 [3496564,54567] 0 2024-05-17T21:52:59.913 INFO:tasks.workunit.client.1.smithi138.stdout:5/466: symlink d1/d1a/d39/d62/l97 0 2024-05-17T21:52:59.915 INFO:tasks.workunit.client.0.smithi107.stdout:6/562: fsync d9/f48 0 2024-05-17T21:52:59.916 INFO:tasks.workunit.client.1.smithi138.stdout:9/456: getdents d6/d7/d16/d29/d48/d6e 0 2024-05-17T21:52:59.916 INFO:tasks.workunit.client.0.smithi107.stdout:6/563: write d9/d58/fa1 [749066,124634] 0 2024-05-17T21:52:59.917 INFO:tasks.workunit.client.1.smithi138.stdout:9/457: chown d6/d7/d16/d29/d48/d6e/d74 13 1 2024-05-17T21:52:59.917 INFO:tasks.workunit.client.0.smithi107.stdout:7/618: write d5/d16/d3b/d45/d91/db3/fd9 [427071,75286] 0 2024-05-17T21:52:59.919 INFO:tasks.workunit.client.1.smithi138.stdout:1/453: creat d4/dd/f90 x:0 0 0 2024-05-17T21:52:59.920 INFO:tasks.workunit.client.1.smithi138.stdout:3/584: mknod d1/d62/ccd 0 2024-05-17T21:52:59.921 INFO:tasks.workunit.client.1.smithi138.stdout:3/585: readlink d1/d62/dad/d72/l9d 0 2024-05-17T21:52:59.922 INFO:tasks.workunit.client.0.smithi107.stdout:2/501: mkdir d0/d9/d11/d18/d9e 0 2024-05-17T21:52:59.923 INFO:tasks.workunit.client.1.smithi138.stdout:3/586: write d1/d3/d8d/d92/fa6 [4689466,95712] 0 2024-05-17T21:52:59.924 INFO:tasks.workunit.client.1.smithi138.stdout:3/587: stat d1/d62/dad/de/f3f 0 2024-05-17T21:52:59.924 INFO:tasks.workunit.client.1.smithi138.stdout:5/467: creat d1/d6/d1b/d29/d36/f98 x:0 0 0 2024-05-17T21:52:59.926 INFO:tasks.workunit.client.1.smithi138.stdout:0/498: symlink d3/d10/d38/d3e/la6 0 2024-05-17T21:52:59.930 INFO:tasks.workunit.client.1.smithi138.stdout:1/454: creat d4/d2e/d36/f91 x:0 0 0 2024-05-17T21:52:59.930 INFO:tasks.workunit.client.0.smithi107.stdout:4/660: read d3/db/d35/f81 [2033659,59707] 0 2024-05-17T21:52:59.933 INFO:tasks.workunit.client.0.smithi107.stdout:6/564: creat d9/d58/d7b/fae x:0 0 0 2024-05-17T21:52:59.935 INFO:tasks.workunit.client.0.smithi107.stdout:7/619: mkdir d5/d16/d3b/ddf 0 2024-05-17T21:52:59.936 INFO:tasks.workunit.client.1.smithi138.stdout:4/489: dread d0/f34 [0,4194304] 0 2024-05-17T21:52:59.936 INFO:tasks.workunit.client.0.smithi107.stdout:7/620: stat d5/df/d11/d1c/d2f/l57 0 2024-05-17T21:52:59.938 INFO:tasks.workunit.client.1.smithi138.stdout:5/468: write d1/d6/d1b/f1d [3313359,67142] 0 2024-05-17T21:52:59.939 INFO:tasks.workunit.client.0.smithi107.stdout:2/502: mkdir d0/d9/d1c/d37/d9f 0 2024-05-17T21:52:59.940 INFO:tasks.workunit.client.1.smithi138.stdout:0/499: symlink d3/d10/d38/d3e/d59/la7 0 2024-05-17T21:52:59.941 INFO:tasks.workunit.client.0.smithi107.stdout:2/503: chown d0/d9/d11/df/d5a/d6d/d82/c94 715 1 2024-05-17T21:52:59.941 INFO:tasks.workunit.client.0.smithi107.stdout:5/640: rename d5/db/d16/l1d to d5/db/d39/lcd 0 2024-05-17T21:52:59.942 INFO:tasks.workunit.client.1.smithi138.stdout:0/500: read d3/d10/f20 [351996,49019] 0 2024-05-17T21:52:59.942 INFO:tasks.workunit.client.0.smithi107.stdout:8/650: dread dc/d21/d97/f8d [0,4194304] 0 2024-05-17T21:52:59.944 INFO:tasks.workunit.client.0.smithi107.stdout:4/661: unlink d3/d6/ccb 0 2024-05-17T21:52:59.945 INFO:tasks.workunit.client.0.smithi107.stdout:4/662: write d3/db/d35/d55/fc4 [495283,95704] 0 2024-05-17T21:52:59.947 INFO:tasks.workunit.client.1.smithi138.stdout:3/588: unlink d1/d62/dad/de/caf 0 2024-05-17T21:52:59.948 INFO:tasks.workunit.client.0.smithi107.stdout:7/621: creat d5/df/d11/d1c/d2f/d53/fe0 x:0 0 0 2024-05-17T21:52:59.949 INFO:tasks.workunit.client.1.smithi138.stdout:4/490: truncate d0/d1b/f3e 696579 0 2024-05-17T21:52:59.949 INFO:tasks.workunit.client.1.smithi138.stdout:1/455: dread d4/f3d [0,4194304] 0 2024-05-17T21:52:59.950 INFO:tasks.workunit.client.0.smithi107.stdout:3/679: getdents d6/d8/d10/d35/d91 0 2024-05-17T21:52:59.951 INFO:tasks.workunit.client.0.smithi107.stdout:2/504: creat d0/d9/d11/df/d5a/d6d/fa0 x:0 0 0 2024-05-17T21:52:59.952 INFO:tasks.workunit.client.0.smithi107.stdout:2/505: chown d0/d26/c4c 14 1 2024-05-17T21:52:59.952 INFO:tasks.workunit.client.0.smithi107.stdout:5/641: creat d5/d75/fce x:0 0 0 2024-05-17T21:52:59.953 INFO:tasks.workunit.client.0.smithi107.stdout:8/651: dread - dc/d44/d4b/d52/d70/fa4 zero size 2024-05-17T21:52:59.954 INFO:tasks.workunit.client.1.smithi138.stdout:9/458: creat d6/d7/d16/d29/f94 x:0 0 0 2024-05-17T21:52:59.955 INFO:tasks.workunit.client.0.smithi107.stdout:6/565: truncate d9/d16/d22/d5f/d6f/fa7 695896 0 2024-05-17T21:52:59.955 INFO:tasks.workunit.client.0.smithi107.stdout:5/642: write d5/db/d16/d2d/d32/f91 [1655949,103475] 0 2024-05-17T21:52:59.956 INFO:tasks.workunit.client.0.smithi107.stdout:7/622: mknod d5/df/d11/d1d/d33/d82/ce1 0 2024-05-17T21:52:59.957 INFO:tasks.workunit.client.0.smithi107.stdout:3/680: dwrite d6/d8/f76 [0,4194304] 0 2024-05-17T21:52:59.957 INFO:tasks.workunit.client.0.smithi107.stdout:6/566: write d9/d58/d7b/fae [717946,97401] 0 2024-05-17T21:52:59.959 INFO:tasks.workunit.client.0.smithi107.stdout:5/643: chown d5/db/d39/cc4 0 1 2024-05-17T21:52:59.959 INFO:tasks.workunit.client.0.smithi107.stdout:7/623: fsync d5/df/d11/d1c/f4e 0 2024-05-17T21:52:59.961 INFO:tasks.workunit.client.0.smithi107.stdout:2/506: mknod d0/d26/ca1 0 2024-05-17T21:52:59.961 INFO:tasks.workunit.client.1.smithi138.stdout:5/469: mkdir d1/d53/d88/d99 0 2024-05-17T21:52:59.965 INFO:tasks.workunit.client.1.smithi138.stdout:4/491: mknod d0/d3/d1d/d38/cb0 0 2024-05-17T21:52:59.967 INFO:tasks.workunit.client.1.smithi138.stdout:4/492: write d0/d2f/d49/f90 [664417,45638] 0 2024-05-17T21:52:59.969 INFO:tasks.workunit.client.1.smithi138.stdout:0/501: dread d3/d3d/d51/f54 [0,4194304] 0 2024-05-17T21:52:59.971 INFO:tasks.workunit.client.0.smithi107.stdout:8/652: rename dc/d44/d4b/d52/d70/d76/l8a to dc/d21/daa/dc5/le7 0 2024-05-17T21:52:59.972 INFO:tasks.workunit.client.1.smithi138.stdout:4/493: dread d0/d3/d17/fa9 [0,4194304] 0 2024-05-17T21:52:59.972 INFO:tasks.workunit.client.1.smithi138.stdout:9/459: fdatasync d6/d7/d16/f21 0 2024-05-17T21:52:59.973 INFO:tasks.workunit.client.1.smithi138.stdout:4/494: chown d0/d2f/d49/d60/d1f/f4d 111 1 2024-05-17T21:52:59.973 INFO:tasks.workunit.client.1.smithi138.stdout:4/495: readlink d0/d3/l1e 0 2024-05-17T21:52:59.977 INFO:tasks.workunit.client.0.smithi107.stdout:6/567: readlink d9/d16/d22/l68 0 2024-05-17T21:52:59.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:59.985+0000 7f52913a4700 1 -- 172.21.15.107:0/2540372228 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f528c0fd270 msgr2=0x7f528c0fd680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:52:59.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:59.985+0000 7f52913a4700 1 --2- 172.21.15.107:0/2540372228 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f528c0fd270 0x7f528c0fd680 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f527c004940 tx=0x7f527c009e30 comp rx=0 tx=0).stop 2024-05-17T21:52:59.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:59.986+0000 7f52913a4700 1 -- 172.21.15.107:0/2540372228 shutdown_connections 2024-05-17T21:52:59.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:59.986+0000 7f52913a4700 1 --2- 172.21.15.107:0/2540372228 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f528c0fdbc0 0x7f528c0fbfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:52:59.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:59.986+0000 7f52913a4700 1 --2- 172.21.15.107:0/2540372228 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f528c0fd270 0x7f528c0fd680 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:52:59.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:59.986+0000 7f52913a4700 1 -- 172.21.15.107:0/2540372228 >> 172.21.15.107:0/2540372228 conn(0x7f528c0f6b80 msgr2=0x7f528c0f8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:52:59.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:59.986+0000 7f52913a4700 1 -- 172.21.15.107:0/2540372228 shutdown_connections 2024-05-17T21:52:59.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:59.986+0000 7f52913a4700 1 -- 172.21.15.107:0/2540372228 wait complete. 2024-05-17T21:52:59.979 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:59 smithi107 ceph-mon[28434]: Upgrade: Updating mgr.smithi107.xhhyiw 2024-05-17T21:52:59.979 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:59 smithi107 ceph-mon[28434]: from='mgr.14636 ' entity='mgr.smithi138.napnll' 2024-05-17T21:52:59.979 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:59 smithi107 ceph-mon[28434]: from='mgr.14636 172.21.15.138:0/3120106723' entity='mgr.smithi138.napnll' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi107.xhhyiw", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-17T21:52:59.980 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:59 smithi107 ceph-mon[28434]: from='mgr.14636 ' entity='mgr.smithi138.napnll' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi107.xhhyiw", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-17T21:52:59.980 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:59 smithi107 ceph-mon[28434]: from='mgr.14636 172.21.15.138:0/3120106723' entity='mgr.smithi138.napnll' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-17T21:52:59.980 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:59 smithi107 ceph-mon[28434]: from='mgr.14636 172.21.15.138:0/3120106723' entity='mgr.smithi138.napnll' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:52:59.980 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:52:59 smithi107 ceph-mon[28434]: Deploying daemon mgr.smithi107.xhhyiw on smithi107 2024-05-17T21:52:59.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:59.986+0000 7f52913a4700 1 Processor -- start 2024-05-17T21:52:59.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:59.986+0000 7f52913a4700 1 -- start start 2024-05-17T21:52:59.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:59.987+0000 7f52913a4700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f528c0fd270 0x7f528c0ffd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:52:59.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:59.987+0000 7f52913a4700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f528c0fe2d0 0x7f528c0fe740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:52:59.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:59.987+0000 7f52913a4700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f528c0fec80 con 0x7f528c0fe2d0 2024-05-17T21:52:59.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:59.987+0000 7f52913a4700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f528c0fedf0 con 0x7f528c0fd270 2024-05-17T21:52:59.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:59.987+0000 7f528a7fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f528c0fe2d0 0x7f528c0fe740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:52:59.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:59.987+0000 7f528affd700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f528c0fd270 0x7f528c0ffd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:52:59.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:59.987+0000 7f528a7fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f528c0fe2d0 0x7f528c0fe740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:35368/0 (socket says 172.21.15.107:35368) 2024-05-17T21:52:59.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:59.987+0000 7f528a7fc700 1 -- 172.21.15.107:0/3840585171 learned_addr learned my addr 172.21.15.107:0/3840585171 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:52:59.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:59.987+0000 7f528a7fc700 1 -- 172.21.15.107:0/3840585171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f528c0fd270 msgr2=0x7f528c0ffd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:52:59.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:59.987+0000 7f528a7fc700 1 --2- 172.21.15.107:0/3840585171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f528c0fd270 0x7f528c0ffd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:52:59.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:59.987+0000 7f528a7fc700 1 -- 172.21.15.107:0/3840585171 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f527c005040 con 0x7f528c0fe2d0 2024-05-17T21:52:59.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:59.987+0000 7f528a7fc700 1 --2- 172.21.15.107:0/3840585171 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f528c0fe2d0 0x7f528c0fe740 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f528000eb40 tx=0x7f528000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:52:59.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:59.987+0000 7f52737fe700 1 -- 172.21.15.107:0/3840585171 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5280007cc0 con 0x7f528c0fe2d0 2024-05-17T21:52:59.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:59.987+0000 7f52913a4700 1 -- 172.21.15.107:0/3840585171 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f528c0ff070 con 0x7f528c0fe2d0 2024-05-17T21:52:59.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:59.988+0000 7f52737fe700 1 -- 172.21.15.107:0/3840585171 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f528000f040 con 0x7f528c0fe2d0 2024-05-17T21:52:59.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:59.988+0000 7f52737fe700 1 -- 172.21.15.107:0/3840585171 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f528000ca10 con 0x7f528c0fe2d0 2024-05-17T21:52:59.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:59.988+0000 7f52913a4700 1 -- 172.21.15.107:0/3840585171 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f528c115c90 con 0x7f528c0fe2d0 2024-05-17T21:52:59.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:59.989+0000 7f52913a4700 1 -- 172.21.15.107:0/3840585171 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52780052e0 con 0x7f528c0fe2d0 2024-05-17T21:52:59.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:59.989+0000 7f52737fe700 1 -- 172.21.15.107:0/3840585171 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) v1 ==== 93316+0+0 (secure 0 0 0) 0x7f528000cb70 con 0x7f528c0fe2d0 2024-05-17T21:52:59.984 INFO:tasks.workunit.client.0.smithi107.stdout:5/644: read d5/db/f9a [3980577,108612] 0 2024-05-17T21:52:59.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:59.990+0000 7f52737fe700 1 --2- 172.21.15.107:0/3840585171 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7f5274070bb0 0x7f5274073060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:52:59.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:59.990+0000 7f52737fe700 1 -- 172.21.15.107:0/3840585171 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f5280090520 con 0x7f528c0fe2d0 2024-05-17T21:52:59.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:59.990+0000 7f528affd700 1 --2- 172.21.15.107:0/3840585171 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7f5274070bb0 0x7f5274073060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:52:59.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:59.991+0000 7f528affd700 1 --2- 172.21.15.107:0/3840585171 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7f5274070bb0 0x7f5274073060 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f527c013740 tx=0x7f527c009e30 comp rx=0 tx=0).ready entity=mgr.14636 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:52:59.986 INFO:tasks.workunit.client.0.smithi107.stdout:7/624: unlink d5/d16/d3b/d45/d91/db3/fcb 0 2024-05-17T21:52:59.986 INFO:tasks.workunit.client.0.smithi107.stdout:2/507: unlink d0/d26/d8e/f9c 0 2024-05-17T21:52:59.986 INFO:tasks.workunit.client.1.smithi138.stdout:1/456: creat d4/dd/d3b/d6c/f92 x:0 0 0 2024-05-17T21:52:59.986 INFO:tasks.workunit.client.1.smithi138.stdout:5/470: truncate d1/d1a/d39/d69/f7f 1854795 0 2024-05-17T21:52:59.987 INFO:tasks.workunit.client.0.smithi107.stdout:4/663: creat d3/d6/fde x:0 0 0 2024-05-17T21:52:59.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:52:59.995+0000 7f52737fe700 1 -- 172.21.15.107:0/3840585171 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+190121 (secure 0 0 0) 0x7f52800516c0 con 0x7f528c0fe2d0 2024-05-17T21:52:59.989 INFO:tasks.workunit.client.0.smithi107.stdout:4/664: write d3/d4a/d47/d60/f79 [118137,99016] 0 2024-05-17T21:52:59.990 INFO:tasks.workunit.client.0.smithi107.stdout:8/653: creat dc/d21/d97/db7/fe8 x:0 0 0 2024-05-17T21:52:59.991 INFO:tasks.workunit.client.1.smithi138.stdout:9/460: creat d6/d7/d16/d29/d6c/d83/f95 x:0 0 0 2024-05-17T21:52:59.991 INFO:tasks.workunit.client.0.smithi107.stdout:8/654: dread - dc/d21/db8/fc4 zero size 2024-05-17T21:52:59.992 INFO:tasks.workunit.client.0.smithi107.stdout:4/665: truncate d3/d4a/d47/d60/d3a/d41/f51 4952479 0 2024-05-17T21:52:59.992 INFO:tasks.workunit.client.1.smithi138.stdout:9/461: dread - d6/d7/d16/d29/d38/d40/d44/d5c/f7a zero size 2024-05-17T21:52:59.992 INFO:tasks.workunit.client.1.smithi138.stdout:4/496: symlink d0/d1b/d2b/lb1 0 2024-05-17T21:52:59.992 INFO:tasks.workunit.client.0.smithi107.stdout:4/666: chown d3/d6/d26/d36/d37/dc1 11205 1 2024-05-17T21:52:59.995 INFO:tasks.workunit.client.1.smithi138.stdout:8/422: sync 2024-05-17T21:52:59.996 INFO:tasks.workunit.client.1.smithi138.stdout:5/471: unlink d1/d60/f64 0 2024-05-17T21:52:59.996 INFO:tasks.workunit.client.0.smithi107.stdout:4/667: dread d3/d4a/d47/d60/f79 [0,4194304] 0 2024-05-17T21:52:59.998 INFO:tasks.workunit.client.0.smithi107.stdout:8/655: dwrite dc/d44/d4b/d52/f9f [0,4194304] 0 2024-05-17T21:52:59.999 INFO:tasks.workunit.client.1.smithi138.stdout:3/589: truncate d1/d3/d16/d3e/f74 756042 0 2024-05-17T21:53:00.000 INFO:tasks.workunit.client.1.smithi138.stdout:0/502: mkdir d3/d10/d6d/d9f/da8 0 2024-05-17T21:53:00.001 INFO:tasks.workunit.client.1.smithi138.stdout:0/503: readlink d3/d10/d38/l52 0 2024-05-17T21:53:00.003 INFO:tasks.workunit.client.1.smithi138.stdout:4/497: unlink d0/d3/d1d/d38/c5f 0 2024-05-17T21:53:00.005 INFO:tasks.workunit.client.0.smithi107.stdout:2/508: unlink d0/d9/d11/df/f71 0 2024-05-17T21:53:00.005 INFO:tasks.workunit.client.0.smithi107.stdout:3/681: dwrite d6/d1a/f4e [0,4194304] 0 2024-05-17T21:53:00.006 INFO:tasks.workunit.client.0.smithi107.stdout:2/509: read - d0/f40 zero size 2024-05-17T21:53:00.009 INFO:tasks.workunit.client.1.smithi138.stdout:9/462: dread d6/d7/d16/d29/d48/d6e/f56 [0,4194304] 0 2024-05-17T21:53:00.010 INFO:tasks.workunit.client.1.smithi138.stdout:9/463: fdatasync d6/d7/f2c 0 2024-05-17T21:53:00.010 INFO:tasks.workunit.client.1.smithi138.stdout:1/457: mkdir d4/d93 0 2024-05-17T21:53:00.011 INFO:tasks.workunit.client.1.smithi138.stdout:1/458: write d4/df/d4e/f84 [2222654,480] 0 2024-05-17T21:53:00.013 INFO:tasks.workunit.client.0.smithi107.stdout:6/568: mkdir d9/d16/d22/daf 0 2024-05-17T21:53:00.014 INFO:tasks.workunit.client.0.smithi107.stdout:1/685: sync 2024-05-17T21:53:00.016 INFO:tasks.workunit.client.1.smithi138.stdout:1/459: read d4/dd/f2d [1848984,89696] 0 2024-05-17T21:53:00.019 INFO:tasks.workunit.client.1.smithi138.stdout:3/590: read d1/d3/f1b [212660,83916] 0 2024-05-17T21:53:00.020 INFO:tasks.workunit.client.0.smithi107.stdout:4/668: rename d3/d4a/d47/d60/d3a/d41/d80/da9 to d3/ddf 0 2024-05-17T21:53:00.020 INFO:tasks.workunit.client.1.smithi138.stdout:3/591: stat d1/d4d/f7e 0 2024-05-17T21:53:00.022 INFO:tasks.workunit.client.0.smithi107.stdout:8/656: write dc/d44/d4b/d52/f96 [507957,110155] 0 2024-05-17T21:53:00.025 INFO:tasks.workunit.client.0.smithi107.stdout:3/682: mkdir d6/d8/d3e/d3f/d90/da7/ddc 0 2024-05-17T21:53:00.027 INFO:tasks.workunit.client.1.smithi138.stdout:3/592: dwrite d1/d3/d8d/fc7 [0,4194304] 0 2024-05-17T21:53:00.027 INFO:tasks.workunit.client.0.smithi107.stdout:6/569: rmdir d9/d16/d22 39 2024-05-17T21:53:00.032 INFO:tasks.workunit.client.1.smithi138.stdout:5/472: mkdir d1/d8a/d9a 0 2024-05-17T21:53:00.034 INFO:tasks.workunit.client.0.smithi107.stdout:5/645: creat d5/db/fcf x:0 0 0 2024-05-17T21:53:00.034 INFO:tasks.workunit.client.0.smithi107.stdout:0/734: sync 2024-05-17T21:53:00.034 INFO:tasks.workunit.client.0.smithi107.stdout:9/672: sync 2024-05-17T21:53:00.036 INFO:tasks.workunit.client.0.smithi107.stdout:9/673: chown d0/d80/fcf 3515427 1 2024-05-17T21:53:00.040 INFO:tasks.workunit.client.0.smithi107.stdout:2/510: rename d0/d9/d11/d27/c32 to d0/d26/d8e/ca2 0 2024-05-17T21:53:00.042 INFO:tasks.workunit.client.1.smithi138.stdout:1/460: dwrite d4/db/d29/d5d/f86 [0,4194304] 0 2024-05-17T21:53:00.042 INFO:tasks.workunit.client.0.smithi107.stdout:3/683: dread f3 [4194304,4194304] 0 2024-05-17T21:53:00.042 INFO:tasks.workunit.client.0.smithi107.stdout:7/625: rmdir d5/df/d11/d2a/d9e 0 2024-05-17T21:53:00.044 INFO:tasks.workunit.client.0.smithi107.stdout:7/626: fdatasync d5/df/d11/d1c/d31/d39/d92/d9a/fa1 0 2024-05-17T21:53:00.047 INFO:tasks.workunit.client.1.smithi138.stdout:4/498: unlink d0/d3/d1d/c56 0 2024-05-17T21:53:00.049 INFO:tasks.workunit.client.0.smithi107.stdout:8/657: creat dc/d21/d27/d2f/fe9 x:0 0 0 2024-05-17T21:53:00.052 INFO:tasks.workunit.client.1.smithi138.stdout:9/464: symlink d6/d7/d16/d29/d48/d67/d86/l96 0 2024-05-17T21:53:00.053 INFO:tasks.workunit.client.1.smithi138.stdout:8/423: write d0/d2f/f45 [766774,37694] 0 2024-05-17T21:53:00.055 INFO:tasks.workunit.client.1.smithi138.stdout:8/424: truncate d0/dc/d1a/f1d 4801245 0 2024-05-17T21:53:00.057 INFO:tasks.workunit.client.0.smithi107.stdout:0/735: fsync d3/d39/f49 0 2024-05-17T21:53:00.060 INFO:tasks.workunit.client.1.smithi138.stdout:6/505: sync 2024-05-17T21:53:00.060 INFO:tasks.workunit.client.1.smithi138.stdout:2/523: sync 2024-05-17T21:53:00.060 INFO:tasks.workunit.client.1.smithi138.stdout:7/393: sync 2024-05-17T21:53:00.062 INFO:tasks.workunit.client.1.smithi138.stdout:0/504: creat d3/d10/d38/d70/fa9 x:0 0 0 2024-05-17T21:53:00.065 INFO:tasks.workunit.client.1.smithi138.stdout:0/505: readlink d3/d10/d38/l63 0 2024-05-17T21:53:00.065 INFO:tasks.workunit.client.1.smithi138.stdout:4/499: chown d0/d2f/d49/c91 1 1 2024-05-17T21:53:00.067 INFO:tasks.workunit.client.1.smithi138.stdout:7/394: dwrite d0/d18/d40/f4e [0,4194304] 0 2024-05-17T21:53:00.068 INFO:tasks.workunit.client.1.smithi138.stdout:9/465: creat d6/d7/d16/d29/d38/f97 x:0 0 0 2024-05-17T21:53:00.075 INFO:tasks.workunit.client.1.smithi138.stdout:5/473: creat d1/d53/d96/f9b x:0 0 0 2024-05-17T21:53:00.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:59 smithi138 ceph-mon[35948]: Upgrade: Updating mgr.smithi107.xhhyiw 2024-05-17T21:53:00.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:59 smithi138 ceph-mon[35948]: from='mgr.14636 ' entity='mgr.smithi138.napnll' 2024-05-17T21:53:00.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:59 smithi138 ceph-mon[35948]: from='mgr.14636 172.21.15.138:0/3120106723' entity='mgr.smithi138.napnll' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi107.xhhyiw", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-17T21:53:00.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:59 smithi138 ceph-mon[35948]: from='mgr.14636 ' entity='mgr.smithi138.napnll' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi107.xhhyiw", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-17T21:53:00.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:59 smithi138 ceph-mon[35948]: from='mgr.14636 172.21.15.138:0/3120106723' entity='mgr.smithi138.napnll' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-17T21:53:00.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:59 smithi138 ceph-mon[35948]: from='mgr.14636 172.21.15.138:0/3120106723' entity='mgr.smithi138.napnll' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:53:00.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:52:59 smithi138 ceph-mon[35948]: Deploying daemon mgr.smithi107.xhhyiw on smithi107 2024-05-17T21:53:00.079 INFO:tasks.workunit.client.1.smithi138.stdout:2/524: symlink d3/d21/d2f/d3e/d55/d99/d84/lab 0 2024-05-17T21:53:00.080 INFO:tasks.workunit.client.1.smithi138.stdout:1/461: getdents d4/dd/d64 0 2024-05-17T21:53:00.083 INFO:tasks.workunit.client.1.smithi138.stdout:7/395: mkdir d0/d18/d64/d7c 0 2024-05-17T21:53:00.084 INFO:tasks.workunit.client.1.smithi138.stdout:7/396: dread d0/d18/d40/f44 [0,4194304] 0 2024-05-17T21:53:00.086 INFO:tasks.workunit.client.1.smithi138.stdout:5/474: creat d1/d46/f9c x:0 0 0 2024-05-17T21:53:00.088 INFO:tasks.workunit.client.1.smithi138.stdout:2/525: mknod d3/d21/cac 0 2024-05-17T21:53:00.089 INFO:tasks.workunit.client.1.smithi138.stdout:0/506: unlink d3/d3d/l95 0 2024-05-17T21:53:00.090 INFO:tasks.workunit.client.1.smithi138.stdout:0/507: chown d3/d10/d38/l63 3 1 2024-05-17T21:53:00.091 INFO:tasks.workunit.client.1.smithi138.stdout:1/462: unlink d4/db/d29/c31 0 2024-05-17T21:53:00.093 INFO:tasks.workunit.client.1.smithi138.stdout:3/593: getdents d1/d3/d8d 0 2024-05-17T21:53:00.095 INFO:tasks.workunit.client.1.smithi138.stdout:7/397: dread d0/d18/d40/d41/d51/d66/f6f [0,4194304] 0 2024-05-17T21:53:00.097 INFO:tasks.workunit.client.1.smithi138.stdout:2/526: rmdir d3/da0 39 2024-05-17T21:53:00.099 INFO:tasks.workunit.client.1.smithi138.stdout:1/463: symlink d4/df/d3a/d3c/l94 0 2024-05-17T21:53:00.100 INFO:tasks.workunit.client.1.smithi138.stdout:6/506: dread d7/db/f25 [0,4194304] 0 2024-05-17T21:53:00.100 INFO:tasks.workunit.client.1.smithi138.stdout:2/527: truncate d3/d21/d2f/d3e/d55/d99/d84/f91 461022 0 2024-05-17T21:53:00.101 INFO:tasks.workunit.client.1.smithi138.stdout:6/507: write d7/d8/fb4 [737314,2747] 0 2024-05-17T21:53:00.102 INFO:tasks.workunit.client.1.smithi138.stdout:2/528: write d3/f4 [4427113,49308] 0 2024-05-17T21:53:00.103 INFO:tasks.workunit.client.1.smithi138.stdout:9/466: creat d6/d7/d16/d29/d48/f98 x:0 0 0 2024-05-17T21:53:00.104 INFO:tasks.workunit.client.1.smithi138.stdout:3/594: symlink d1/d4f/d87/lce 0 2024-05-17T21:53:00.105 INFO:tasks.workunit.client.1.smithi138.stdout:8/425: getdents d0/dc/d1a/d33/d4a 0 2024-05-17T21:53:00.108 INFO:tasks.workunit.client.0.smithi107.stdout:9/674: rmdir d0/dd/d21/d29/d3e/d45/d68/d75 39 2024-05-17T21:53:00.109 INFO:tasks.workunit.client.0.smithi107.stdout:4/669: getdents d3/d6/d26/d36/d37/d85 0 2024-05-17T21:53:00.109 INFO:tasks.workunit.client.1.smithi138.stdout:1/464: rename d4/db/d29/l8a to d4/d93/l95 0 2024-05-17T21:53:00.110 INFO:tasks.workunit.client.0.smithi107.stdout:3/684: write d6/d8/d10/d35/d36/f8e [3369199,65533] 0 2024-05-17T21:53:00.110 INFO:tasks.workunit.client.0.smithi107.stdout:5/646: dread d5/db/f1a [0,4194304] 0 2024-05-17T21:53:00.111 INFO:tasks.workunit.client.0.smithi107.stdout:6/570: mknod d9/cb0 0 2024-05-17T21:53:00.114 INFO:tasks.workunit.client.0.smithi107.stdout:5/647: fsync d5/db/d16/f9d 0 2024-05-17T21:53:00.115 INFO:tasks.workunit.client.1.smithi138.stdout:0/508: write d3/f21 [223678,40949] 0 2024-05-17T21:53:00.117 INFO:tasks.workunit.client.0.smithi107.stdout:9/675: mkdir d0/dd/d1f/de4 0 2024-05-17T21:53:00.118 INFO:tasks.workunit.client.0.smithi107.stdout:7/627: dread d5/df/d11/d1c/fbf [4194304,4194304] 0 2024-05-17T21:53:00.119 INFO:tasks.workunit.client.1.smithi138.stdout:8/426: creat d0/dc/d1a/d51/f80 x:0 0 0 2024-05-17T21:53:00.119 INFO:tasks.workunit.client.0.smithi107.stdout:9/676: stat d0/dd/d2f/f31 0 2024-05-17T21:53:00.120 INFO:tasks.workunit.client.0.smithi107.stdout:6/571: dread d9/d16/f7f [0,4194304] 0 2024-05-17T21:53:00.120 INFO:tasks.workunit.client.1.smithi138.stdout:5/475: creat d1/d2c/d63/f9d x:0 0 0 2024-05-17T21:53:00.121 INFO:tasks.workunit.client.1.smithi138.stdout:9/467: dwrite d6/d7/d16/d29/f3a [0,4194304] 0 2024-05-17T21:53:00.121 INFO:tasks.workunit.client.1.smithi138.stdout:3/595: dread d1/d3/d16/d3e/f4e [0,4194304] 0 2024-05-17T21:53:00.121 INFO:tasks.workunit.client.0.smithi107.stdout:8/658: dwrite dc/d21/d35/fda [0,4194304] 0 2024-05-17T21:53:00.122 INFO:tasks.workunit.client.0.smithi107.stdout:1/686: getdents d4/d78 0 2024-05-17T21:53:00.123 INFO:tasks.workunit.client.0.smithi107.stdout:8/659: chown dc/d21/d97/f15 112245906 1 2024-05-17T21:53:00.127 INFO:tasks.workunit.client.1.smithi138.stdout:8/427: dwrite d0/dc/d1a/f3f [0,4194304] 0 2024-05-17T21:53:00.128 INFO:tasks.workunit.client.1.smithi138.stdout:6/508: rename d7/db/d43/d5f/d6a/d8b to d7/db/d43/d3f/db6 0 2024-05-17T21:53:00.128 INFO:tasks.workunit.client.1.smithi138.stdout:8/428: fsync d0/dc/f10 0 2024-05-17T21:53:00.130 INFO:tasks.workunit.client.0.smithi107.stdout:5/648: mkdir d5/db/d16/d2a/d73/dd0 0 2024-05-17T21:53:00.132 INFO:tasks.workunit.client.0.smithi107.stdout:0/736: truncate d3/dc/d22/d3d/fa8 641211 0 2024-05-17T21:53:00.132 INFO:tasks.workunit.client.0.smithi107.stdout:5/649: chown d5/db/d16/d54/d59/d65/d48/d89/c8d 3 1 2024-05-17T21:53:00.134 INFO:tasks.workunit.client.0.smithi107.stdout:5/650: chown d5/db/d39/f43 1031439046 1 2024-05-17T21:53:00.134 INFO:tasks.workunit.client.1.smithi138.stdout:0/509: truncate d3/d3d/f58 677777 0 2024-05-17T21:53:00.135 INFO:tasks.workunit.client.0.smithi107.stdout:3/685: dread d6/d2a/f6a [0,4194304] 0 2024-05-17T21:53:00.138 INFO:tasks.workunit.client.1.smithi138.stdout:5/476: unlink d1/d1a/d39/f4c 0 2024-05-17T21:53:00.142 INFO:tasks.workunit.client.0.smithi107.stdout:1/687: symlink d4/df/d1a/d26/d72/daa/le2 0 2024-05-17T21:53:00.144 INFO:tasks.workunit.client.0.smithi107.stdout:1/688: write d4/df/d1a/d26/d72/d85/fda [19461,70261] 0 2024-05-17T21:53:00.144 INFO:tasks.workunit.client.0.smithi107.stdout:8/660: creat dc/d21/d97/d63/d71/fea x:0 0 0 2024-05-17T21:53:00.146 INFO:tasks.workunit.client.1.smithi138.stdout:5/477: dwrite d1/d1a/f93 [0,4194304] 0 2024-05-17T21:53:00.147 INFO:tasks.workunit.client.1.smithi138.stdout:5/478: write d1/d2c/f8b [560340,59214] 0 2024-05-17T21:53:00.148 INFO:tasks.workunit.client.1.smithi138.stdout:7/398: rename d0/d4c/l55 to d0/d18/d40/d41/d50/l7d 0 2024-05-17T21:53:00.149 INFO:tasks.workunit.client.1.smithi138.stdout:5/479: write d1/d6/d1b/f54 [88949,88887] 0 2024-05-17T21:53:00.150 INFO:tasks.workunit.client.0.smithi107.stdout:4/670: creat d3/d6/fe0 x:0 0 0 2024-05-17T21:53:00.151 INFO:tasks.workunit.client.1.smithi138.stdout:8/429: rmdir d0/d11/d3e 39 2024-05-17T21:53:00.151 INFO:tasks.workunit.client.1.smithi138.stdout:5/480: chown d1/d6/d1b/l70 27652 1 2024-05-17T21:53:00.151 INFO:tasks.workunit.client.1.smithi138.stdout:7/399: read d0/d18/d40/d41/d51/d66/f6f [99333,5880] 0 2024-05-17T21:53:00.153 INFO:tasks.workunit.client.1.smithi138.stdout:6/509: symlink d7/db/d43/d5f/d6a/lb7 0 2024-05-17T21:53:00.153 INFO:tasks.workunit.client.1.smithi138.stdout:7/400: stat d0/d18/d40/d41/c4f 0 2024-05-17T21:53:00.154 INFO:tasks.workunit.client.0.smithi107.stdout:5/651: stat d5/db/d16/d2a/l9b 0 2024-05-17T21:53:00.157 INFO:tasks.workunit.client.1.smithi138.stdout:2/529: link d3/f9 d3/d21/d2f/d3e/fad 0 2024-05-17T21:53:00.160 INFO:tasks.workunit.client.0.smithi107.stdout:7/628: dread d5/d16/d3b/d45/d91/fc7 [0,4194304] 0 2024-05-17T21:53:00.162 INFO:tasks.workunit.client.0.smithi107.stdout:9/677: getdents d0/dd/d21/d29/d57/dd7 0 2024-05-17T21:53:00.163 INFO:tasks.workunit.client.0.smithi107.stdout:1/689: chown d4/l68 0 1 2024-05-17T21:53:00.163 INFO:tasks.workunit.client.0.smithi107.stdout:1/690: readlink d4/df/d14/l5e 0 2024-05-17T21:53:00.165 INFO:tasks.workunit.client.0.smithi107.stdout:0/737: fdatasync d3/dc/d22/d23/f44 0 2024-05-17T21:53:00.170 INFO:tasks.workunit.client.1.smithi138.stdout:5/481: creat d1/d1a/d39/d4e/d58/d5b/f9e x:0 0 0 2024-05-17T21:53:00.172 INFO:tasks.workunit.client.1.smithi138.stdout:7/401: read d0/f13 [615158,34022] 0 2024-05-17T21:53:00.173 INFO:tasks.workunit.client.0.smithi107.stdout:3/686: symlink d6/d8/ldd 0 2024-05-17T21:53:00.174 INFO:tasks.workunit.client.1.smithi138.stdout:6/510: creat d7/d69/fb8 x:0 0 0 2024-05-17T21:53:00.175 INFO:tasks.workunit.client.1.smithi138.stdout:6/511: dread - d7/d8/f96 zero size 2024-05-17T21:53:00.176 INFO:tasks.workunit.client.1.smithi138.stdout:6/512: write d7/db/d43/d3f/d79/f9f [1953806,27293] 0 2024-05-17T21:53:00.179 INFO:tasks.workunit.client.0.smithi107.stdout:9/678: unlink d0/dd/d21/d29/d3e/d45/d68/f7a 0 2024-05-17T21:53:00.180 INFO:tasks.workunit.client.1.smithi138.stdout:0/510: mkdir d3/d10/d6d/d9f/da8/daa 0 2024-05-17T21:53:00.181 INFO:tasks.workunit.client.0.smithi107.stdout:0/738: dread d3/dc/fc3 [0,4194304] 0 2024-05-17T21:53:00.182 INFO:tasks.workunit.client.0.smithi107.stdout:8/661: link dc/d21/d97/fd8 dc/d21/d97/feb 0 2024-05-17T21:53:00.183 INFO:tasks.workunit.client.1.smithi138.stdout:1/465: dwrite d4/db/f4b [4194304,4194304] 0 2024-05-17T21:53:00.185 INFO:tasks.workunit.client.1.smithi138.stdout:6/513: dwrite d7/db/d34/f7c [0,4194304] 0 2024-05-17T21:53:00.186 INFO:tasks.workunit.client.0.smithi107.stdout:1/691: rename d4/df/d1a/d26/d72/daa/f9d to d4/df/d4e/d59/fe3 0 2024-05-17T21:53:00.187 INFO:tasks.workunit.client.1.smithi138.stdout:9/468: creat d6/d7/f99 x:0 0 0 2024-05-17T21:53:00.192 INFO:tasks.workunit.client.0.smithi107.stdout:4/671: getdents d3/db/d1e/dc6 0 2024-05-17T21:53:00.192 INFO:tasks.workunit.client.1.smithi138.stdout:6/514: dwrite d7/d69/fa7 [0,4194304] 0 2024-05-17T21:53:00.193 INFO:tasks.workunit.client.0.smithi107.stdout:4/672: read d3/db/d1e/d66/f70 [2613813,101525] 0 2024-05-17T21:53:00.197 INFO:tasks.workunit.client.0.smithi107.stdout:3/687: creat d6/d1a/fde x:0 0 0 2024-05-17T21:53:00.199 INFO:tasks.workunit.client.0.smithi107.stdout:6/572: getdents d9/d16/d4d 0 2024-05-17T21:53:00.208 INFO:tasks.workunit.client.1.smithi138.stdout:1/466: fdatasync d4/df/d3a/d3c/f88 0 2024-05-17T21:53:00.212 INFO:tasks.workunit.client.1.smithi138.stdout:9/469: symlink d6/d7/d16/d29/d48/d6e/d74/l9a 0 2024-05-17T21:53:00.212 INFO:tasks.workunit.client.1.smithi138.stdout:8/430: dwrite d0/dc/d1a/f58 [4194304,4194304] 0 2024-05-17T21:53:00.213 INFO:tasks.workunit.client.1.smithi138.stdout:8/431: readlink d0/d2f/l39 0 2024-05-17T21:53:00.214 INFO:tasks.workunit.client.1.smithi138.stdout:8/432: stat d0/dc/d1a/f1d 0 2024-05-17T21:53:00.215 INFO:tasks.workunit.client.1.smithi138.stdout:8/433: write d0/f1b [2680454,24209] 0 2024-05-17T21:53:00.216 INFO:tasks.workunit.client.1.smithi138.stdout:5/482: symlink d1/d6/l9f 0 2024-05-17T21:53:00.217 INFO:tasks.workunit.client.1.smithi138.stdout:5/483: dread - d1/d6/d1b/d29/d36/f98 zero size 2024-05-17T21:53:00.218 INFO:tasks.workunit.client.1.smithi138.stdout:5/484: chown d1/d6/l75 12929 1 2024-05-17T21:53:00.224 INFO:tasks.workunit.client.0.smithi107.stdout:3/688: creat d6/d2a/d2d/d69/dad/fdf x:0 0 0 2024-05-17T21:53:00.226 INFO:tasks.workunit.client.0.smithi107.stdout:6/573: mkdir d9/d16/d4d/d85/d88/db1 0 2024-05-17T21:53:00.226 INFO:tasks.workunit.client.1.smithi138.stdout:7/402: symlink d0/d39/d78/l7e 0 2024-05-17T21:53:00.227 INFO:tasks.workunit.client.0.smithi107.stdout:5/652: write d5/db/d16/f4b [2170051,26968] 0 2024-05-17T21:53:00.228 INFO:tasks.workunit.client.1.smithi138.stdout:7/403: write d0/f12 [6742885,31656] 0 2024-05-17T21:53:00.228 INFO:tasks.workunit.client.0.smithi107.stdout:8/662: mknod dc/d21/d97/d63/cec 0 2024-05-17T21:53:00.229 INFO:tasks.workunit.client.1.smithi138.stdout:2/530: write d3/fd [2595760,127280] 0 2024-05-17T21:53:00.229 INFO:tasks.workunit.client.0.smithi107.stdout:3/689: dread d6/d8/f76 [0,4194304] 0 2024-05-17T21:53:00.229 INFO:tasks.workunit.client.1.smithi138.stdout:3/596: getdents d1/d2e 0 2024-05-17T21:53:00.232 INFO:tasks.workunit.client.1.smithi138.stdout:0/511: truncate d3/f2b 3500819 0 2024-05-17T21:53:00.235 INFO:tasks.workunit.client.1.smithi138.stdout:1/467: creat d4/db/d29/d89/f96 x:0 0 0 2024-05-17T21:53:00.238 INFO:tasks.workunit.client.0.smithi107.stdout:6/574: symlink d9/d16/d4d/lb2 0 2024-05-17T21:53:00.240 INFO:tasks.workunit.client.1.smithi138.stdout:9/470: mknod d6/d7/d16/d29/d48/d3b/c9b 0 2024-05-17T21:53:00.242 INFO:tasks.workunit.client.0.smithi107.stdout:1/692: dwrite d4/f21 [0,4194304] 0 2024-05-17T21:53:00.243 INFO:tasks.workunit.client.0.smithi107.stdout:5/653: mknod d5/db/d16/d2d/d40/d9f/dc9/cd1 0 2024-05-17T21:53:00.245 INFO:tasks.workunit.client.0.smithi107.stdout:1/693: dread - d4/de/f3c zero size 2024-05-17T21:53:00.245 INFO:tasks.workunit.client.0.smithi107.stdout:5/654: dread - d5/db/d16/d2d/d40/d9f/fc3 zero size 2024-05-17T21:53:00.247 INFO:tasks.workunit.client.0.smithi107.stdout:1/694: readlink d4/df/d1a/d26/d72/daa/le2 0 2024-05-17T21:53:00.247 INFO:tasks.workunit.client.0.smithi107.stdout:1/695: stat d4/df/d1a 0 2024-05-17T21:53:00.249 INFO:tasks.workunit.client.0.smithi107.stdout:1/696: dread - d4/df/d4e/d59/f5a zero size 2024-05-17T21:53:00.249 INFO:tasks.workunit.client.0.smithi107.stdout:4/673: write d3/db/d35/d5e/f96 [4533519,121388] 0 2024-05-17T21:53:00.250 INFO:tasks.workunit.client.1.smithi138.stdout:3/597: dread d1/d4f/f77 [0,4194304] 0 2024-05-17T21:53:00.250 INFO:tasks.workunit.client.1.smithi138.stdout:2/531: dread d3/d21/d2f/f50 [0,4194304] 0 2024-05-17T21:53:00.250 INFO:tasks.workunit.client.0.smithi107.stdout:1/697: dread - d4/df/d4e/d59/f5a zero size 2024-05-17T21:53:00.251 INFO:tasks.workunit.client.0.smithi107.stdout:4/674: fsync d3/db/d1e/f6c 0 2024-05-17T21:53:00.251 INFO:tasks.workunit.client.1.smithi138.stdout:3/598: stat d1/d62/dad/de 0 2024-05-17T21:53:00.251 INFO:tasks.workunit.client.1.smithi138.stdout:2/532: readlink d3/l2a 0 2024-05-17T21:53:00.251 INFO:tasks.workunit.client.1.smithi138.stdout:6/515: creat d7/db/d43/d5f/d6a/d9b/db2/fb9 x:0 0 0 2024-05-17T21:53:00.253 INFO:tasks.workunit.client.1.smithi138.stdout:6/516: chown d7/db/d43/c86 7401649 1 2024-05-17T21:53:00.254 INFO:tasks.workunit.client.1.smithi138.stdout:3/599: write d1/d3/d16/d3e/d71/f7b [2408787,99726] 0 2024-05-17T21:53:00.257 INFO:tasks.workunit.client.1.smithi138.stdout:7/404: stat d0/f2a 0 2024-05-17T21:53:00.260 INFO:tasks.workunit.client.0.smithi107.stdout:9/679: getdents d0/dd 0 2024-05-17T21:53:00.262 INFO:tasks.workunit.client.1.smithi138.stdout:3/600: dwrite d1/d4d/fc6 [0,4194304] 0 2024-05-17T21:53:00.262 INFO:tasks.workunit.client.0.smithi107.stdout:8/663: symlink dc/d21/d35/db6/led 0 2024-05-17T21:53:00.264 INFO:tasks.workunit.client.0.smithi107.stdout:5/655: mknod d5/db/d16/d54/d59/cd2 0 2024-05-17T21:53:00.269 INFO:tasks.workunit.client.1.smithi138.stdout:9/471: dread d6/d7/d16/d29/f37 [0,4194304] 0 2024-05-17T21:53:00.269 INFO:tasks.workunit.client.0.smithi107.stdout:0/739: rename d3/dc/d22/d3d/f4d to d3/dc/d22/ffc 0 2024-05-17T21:53:00.270 INFO:tasks.workunit.client.1.smithi138.stdout:9/472: write d6/d7/d16/d29/d6c/d83/f92 [303208,10050] 0 2024-05-17T21:53:00.270 INFO:tasks.workunit.client.1.smithi138.stdout:1/468: rmdir d4 39 2024-05-17T21:53:00.273 INFO:tasks.workunit.client.1.smithi138.stdout:8/434: fdatasync d0/f2c 0 2024-05-17T21:53:00.275 INFO:tasks.workunit.client.1.smithi138.stdout:5/485: write d1/f45 [3277664,76154] 0 2024-05-17T21:53:00.275 INFO:tasks.workunit.client.1.smithi138.stdout:5/486: dread - d1/d2c/d63/f9d zero size 2024-05-17T21:53:00.277 INFO:tasks.workunit.client.0.smithi107.stdout:3/690: link d6/d1a/f96 d6/d2a/d2d/da8/daa/fe0 0 2024-05-17T21:53:00.281 INFO:tasks.workunit.client.1.smithi138.stdout:2/533: creat d3/d21/d2f/d3e/d58/fae x:0 0 0 2024-05-17T21:53:00.281 INFO:tasks.workunit.client.0.smithi107.stdout:6/575: mknod d9/cb3 0 2024-05-17T21:53:00.283 INFO:tasks.workunit.client.1.smithi138.stdout:6/517: creat d7/d8/fba x:0 0 0 2024-05-17T21:53:00.284 INFO:tasks.workunit.client.0.smithi107.stdout:5/656: truncate d5/db/d16/d2d/d40/d5a/f6c 1016919 0 2024-05-17T21:53:00.285 INFO:tasks.workunit.client.1.smithi138.stdout:5/487: dwrite d1/d53/d96/f9b [0,4194304] 0 2024-05-17T21:53:00.285 INFO:tasks.workunit.client.0.smithi107.stdout:5/657: dread - d5/db/fc6 zero size 2024-05-17T21:53:00.285 INFO:tasks.workunit.client.0.smithi107.stdout:5/658: dread - d5/db/dc/d1e/fc1 zero size 2024-05-17T21:53:00.287 INFO:tasks.workunit.client.1.smithi138.stdout:3/601: dread d1/d3/d8d/d92/f96 [0,4194304] 0 2024-05-17T21:53:00.288 INFO:tasks.workunit.client.0.smithi107.stdout:1/698: dread d4/df/d1a/d26/d7a/fa5 [0,4194304] 0 2024-05-17T21:53:00.289 INFO:tasks.workunit.client.0.smithi107.stdout:1/699: dread - d4/df/d4e/d59/f5a zero size 2024-05-17T21:53:00.291 INFO:tasks.workunit.client.0.smithi107.stdout:0/740: unlink d3/d24/dcf/fec 0 2024-05-17T21:53:00.292 INFO:tasks.workunit.client.1.smithi138.stdout:0/512: dwrite d3/d10/f31 [0,4194304] 0 2024-05-17T21:53:00.293 INFO:tasks.workunit.client.0.smithi107.stdout:3/691: mkdir d6/d2a/d2d/d69/dad/de1 0 2024-05-17T21:53:00.294 INFO:tasks.workunit.client.1.smithi138.stdout:7/405: dread d0/f19 [0,4194304] 0 2024-05-17T21:53:00.294 INFO:tasks.workunit.client.0.smithi107.stdout:3/692: write d6/d8/fc0 [181121,23441] 0 2024-05-17T21:53:00.295 INFO:tasks.workunit.client.1.smithi138.stdout:4/500: sync 2024-05-17T21:53:00.296 INFO:tasks.workunit.client.0.smithi107.stdout:3/693: write d6/d2a/d2d/d37/fc8 [2661639,24797] 0 2024-05-17T21:53:00.298 INFO:tasks.workunit.client.1.smithi138.stdout:4/501: read d0/d1b/d2b/d55/f7d [2880806,95435] 0 2024-05-17T21:53:00.306 INFO:tasks.workunit.client.0.smithi107.stdout:9/680: rename d0/dd/d2f/d87/d89/f96 to d0/dd/d21/dcd/fe5 0 2024-05-17T21:53:00.307 INFO:tasks.workunit.client.1.smithi138.stdout:1/469: dread d4/db/d29/d5d/f86 [0,4194304] 0 2024-05-17T21:53:00.307 INFO:tasks.workunit.client.1.smithi138.stdout:9/473: mkdir d6/d7/d16/d29/d6c/d9c 0 2024-05-17T21:53:00.308 INFO:tasks.workunit.client.1.smithi138.stdout:8/435: symlink d0/d63/l81 0 2024-05-17T21:53:00.310 INFO:tasks.workunit.client.0.smithi107.stdout:0/741: stat d3/dc/d31/lc2 0 2024-05-17T21:53:00.311 INFO:tasks.workunit.client.1.smithi138.stdout:2/534: fdatasync d3/f15 0 2024-05-17T21:53:00.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:00.319+0000 7f52913a4700 1 -- 172.21.15.107:0/3840585171 --> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f5278000bf0 con 0x7f5274070bb0 2024-05-17T21:53:00.314 INFO:tasks.workunit.client.0.smithi107.stdout:3/694: readlink d6/d1a/d57/ldb 0 2024-05-17T21:53:00.315 INFO:tasks.workunit.client.0.smithi107.stdout:3/695: truncate d6/d8/d10/d35/d36/fd1 634057 0 2024-05-17T21:53:00.315 INFO:tasks.workunit.client.0.smithi107.stdout:3/696: chown f3 219116 1 2024-05-17T21:53:00.316 INFO:tasks.workunit.client.1.smithi138.stdout:5/488: mkdir d1/d53/d96/da0 0 2024-05-17T21:53:00.317 INFO:tasks.workunit.client.1.smithi138.stdout:2/535: dwrite d3/d21/d2f/f5f [0,4194304] 0 2024-05-17T21:53:00.318 INFO:tasks.workunit.client.0.smithi107.stdout:4/675: link d3/d4a/d47/d60/d3a/l7c d3/db/d35/le1 0 2024-05-17T21:53:00.318 INFO:tasks.workunit.client.0.smithi107.stdout:4/676: stat d3/d4a/d47/da3 0 2024-05-17T21:53:00.318 INFO:tasks.workunit.client.1.smithi138.stdout:5/489: write d1/d60/d2f/d5e/f79 [488633,111448] 0 2024-05-17T21:53:00.319 INFO:tasks.workunit.client.1.smithi138.stdout:3/602: rmdir d1/d62/d9b 39 2024-05-17T21:53:00.322 INFO:tasks.workunit.client.0.smithi107.stdout:6/576: readlink d9/d16/d22/l57 0 2024-05-17T21:53:00.326 INFO:tasks.workunit.client.1.smithi138.stdout:5/490: dwrite d1/d6/d1b/f54 [4194304,4194304] 0 2024-05-17T21:53:00.328 INFO:tasks.workunit.client.1.smithi138.stdout:7/406: symlink d0/d18/d64/l7f 0 2024-05-17T21:53:00.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:00.337+0000 7f52737fe700 1 -- 172.21.15.107:0/3840585171 <== mgr.14636 v2:172.21.15.138:6828/1369055740 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3388 (secure 0 0 0) 0x7f5278000bf0 con 0x7f5274070bb0 2024-05-17T21:53:00.330 INFO:teuthology.orchestra.run.smithi107.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-17T21:53:00.330 INFO:teuthology.orchestra.run.smithi107.stdout:alertmanager.smithi107 smithi107 *:9093,9094 starting - - - - 2024-05-17T21:53:00.330 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi107 smithi107 running (9m) 59s ago 9m 6660k - 17.2.7-952-g9903f776 d15a36853796 828741c3a89c 2024-05-17T21:53:00.330 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi138 smithi138 running (8m) 60s ago 8m 6660k - 17.2.7-952-g9903f776 d15a36853796 20e9ccf1fadd 2024-05-17T21:53:00.330 INFO:teuthology.orchestra.run.smithi107.stdout:grafana.smithi107 smithi107 *:3000 starting - - - - 2024-05-17T21:53:00.330 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.drqody smithi107 running (3m) 59s ago 3m 38.1M - 17.2.7-952-g9903f776 d15a36853796 0d851bc912f0 2024-05-17T21:53:00.330 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.idmbgj smithi107 running (3m) 59s ago 3m 11.6M - 17.2.7-952-g9903f776 d15a36853796 1348b09ab29e 2024-05-17T21:53:00.330 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi138.gwbtyd smithi138 running (3m) 60s ago 3m 70.7M - 17.2.7-952-g9903f776 d15a36853796 fd2ba75354c0 2024-05-17T21:53:00.330 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi138.zckrvs smithi138 running (3m) 60s ago 3m 12.4M - 17.2.7-952-g9903f776 d15a36853796 a1cfd2fe4cda 2024-05-17T21:53:00.330 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi107.xhhyiw smithi107 *:9283 running (10m) 59s ago 10m 398M - 17.2.7-952-g9903f776 d15a36853796 b566f08938ae 2024-05-17T21:53:00.330 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi138.napnll smithi138 *:8443,9283 running (89s) 60s ago 8m 570M - 19.0.0-3794-g931809a9 8880f7613d02 3f320d02d8e2 2024-05-17T21:53:00.331 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi107 smithi107 running (10m) 59s ago 10m 51.2M 2048M 17.2.7-952-g9903f776 d15a36853796 7de9c23eee97 2024-05-17T21:53:00.331 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi138 smithi138 running (8m) 60s ago 8m 39.5M 2048M 17.2.7-952-g9903f776 d15a36853796 ee61a065778d 2024-05-17T21:53:00.331 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi107 smithi107 *:9100 starting - - - - 2024-05-17T21:53:00.331 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi138 smithi138 *:9100 starting - - - - 2024-05-17T21:53:00.331 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0 smithi107 running (7m) 59s ago 7m 57.7M 1557M 17.2.7-952-g9903f776 d15a36853796 f2d20009361d 2024-05-17T21:53:00.331 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1 smithi107 running (6m) 59s ago 6m 62.5M 1557M 17.2.7-952-g9903f776 d15a36853796 e5d3e99844a9 2024-05-17T21:53:00.331 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2 smithi107 running (6m) 59s ago 6m 54.8M 1557M 17.2.7-952-g9903f776 d15a36853796 e044d7b78088 2024-05-17T21:53:00.332 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3 smithi138 running (6m) 60s ago 6m 56.2M 2581M 17.2.7-952-g9903f776 d15a36853796 8c980cc8af87 2024-05-17T21:53:00.332 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4 smithi138 running (5m) 60s ago 5m 52.6M 2581M 17.2.7-952-g9903f776 d15a36853796 b306a36a744e 2024-05-17T21:53:00.332 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5 smithi138 running (5m) 60s ago 5m 53.9M 2581M 17.2.7-952-g9903f776 d15a36853796 7c8d8ed74974 2024-05-17T21:53:00.332 INFO:teuthology.orchestra.run.smithi107.stdout:prometheus.smithi107 smithi107 *:9095 starting - - - - 2024-05-17T21:53:00.333 INFO:tasks.workunit.client.1.smithi138.stdout:0/513: symlink d3/d10/d38/d70/d80/d94/lab 0 2024-05-17T21:53:00.333 INFO:tasks.workunit.client.0.smithi107.stdout:9/681: mkdir d0/dd/d2f/d4a/de6 0 2024-05-17T21:53:00.333 INFO:tasks.workunit.client.0.smithi107.stdout:9/682: truncate d0/dd/d21/daf/fe1 924045 0 2024-05-17T21:53:00.333 INFO:tasks.workunit.client.0.smithi107.stdout:1/700: fdatasync d4/df/f29 0 2024-05-17T21:53:00.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:00.340+0000 7f52913a4700 1 -- 172.21.15.107:0/3840585171 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7f5274070bb0 msgr2=0x7f5274073060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:53:00.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:00.340+0000 7f52913a4700 1 --2- 172.21.15.107:0/3840585171 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7f5274070bb0 0x7f5274073060 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f527c013740 tx=0x7f527c009e30 comp rx=0 tx=0).stop 2024-05-17T21:53:00.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:00.340+0000 7f52913a4700 1 -- 172.21.15.107:0/3840585171 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f528c0fe2d0 msgr2=0x7f528c0fe740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:53:00.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:00.340+0000 7f52913a4700 1 --2- 172.21.15.107:0/3840585171 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f528c0fe2d0 0x7f528c0fe740 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f528000eb40 tx=0x7f528000b040 comp rx=0 tx=0).stop 2024-05-17T21:53:00.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:00.341+0000 7f52913a4700 1 -- 172.21.15.107:0/3840585171 shutdown_connections 2024-05-17T21:53:00.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:00.341+0000 7f52913a4700 1 --2- 172.21.15.107:0/3840585171 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7f5274070bb0 0x7f5274073060 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:53:00.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:00.341+0000 7f52913a4700 1 --2- 172.21.15.107:0/3840585171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f528c0fd270 0x7f528c0ffd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:53:00.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:00.341+0000 7f52913a4700 1 --2- 172.21.15.107:0/3840585171 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f528c0fe2d0 0x7f528c0fe740 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:53:00.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:00.341+0000 7f52913a4700 1 -- 172.21.15.107:0/3840585171 >> 172.21.15.107:0/3840585171 conn(0x7f528c0f6b80 msgr2=0x7f528c0fb3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:53:00.335 INFO:tasks.workunit.client.1.smithi138.stdout:5/491: dread d1/d53/f80 [0,4194304] 0 2024-05-17T21:53:00.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:00.342+0000 7f52913a4700 1 -- 172.21.15.107:0/3840585171 shutdown_connections 2024-05-17T21:53:00.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:00.342+0000 7f52913a4700 1 -- 172.21.15.107:0/3840585171 wait complete. 2024-05-17T21:53:00.336 INFO:tasks.workunit.client.0.smithi107.stdout:0/742: rmdir d3 39 2024-05-17T21:53:00.336 INFO:tasks.workunit.client.1.smithi138.stdout:7/407: dwrite d0/f4d [0,4194304] 0 2024-05-17T21:53:00.337 INFO:tasks.workunit.client.1.smithi138.stdout:5/492: chown d1/d6/d1b/d29 7 1 2024-05-17T21:53:00.337 INFO:tasks.workunit.client.0.smithi107.stdout:5/659: write d5/db/d16/d5b/f2f [501717,117427] 0 2024-05-17T21:53:00.339 INFO:tasks.workunit.client.0.smithi107.stdout:9/683: dwrite d0/dd/d21/faa [0,4194304] 0 2024-05-17T21:53:00.344 INFO:tasks.workunit.client.1.smithi138.stdout:9/474: dread d6/f49 [0,4194304] 0 2024-05-17T21:53:00.346 INFO:tasks.workunit.client.1.smithi138.stdout:1/470: creat d4/df/f97 x:0 0 0 2024-05-17T21:53:00.347 INFO:tasks.workunit.client.1.smithi138.stdout:1/471: write d4/db/f4d [4717374,65901] 0 2024-05-17T21:53:00.349 INFO:tasks.workunit.client.1.smithi138.stdout:8/436: mkdir d0/dc/d1a/d51/d82 0 2024-05-17T21:53:00.351 INFO:tasks.workunit.client.1.smithi138.stdout:1/472: truncate d4/df/f2c 9355244 0 2024-05-17T21:53:00.354 INFO:tasks.workunit.client.1.smithi138.stdout:9/475: dwrite d6/d7/d16/d29/d6c/d83/f53 [0,4194304] 0 2024-05-17T21:53:00.356 INFO:tasks.workunit.client.0.smithi107.stdout:3/697: creat d6/d8/d10/d25/d80/fe2 x:0 0 0 2024-05-17T21:53:00.359 INFO:tasks.workunit.client.0.smithi107.stdout:5/660: mknod d5/db/d16/d54/da0/cd3 0 2024-05-17T21:53:00.362 INFO:tasks.workunit.client.0.smithi107.stdout:0/743: dwrite d3/d24/ff8 [0,4194304] 0 2024-05-17T21:53:00.363 INFO:tasks.workunit.client.0.smithi107.stdout:4/677: mknod d3/db/d1e/d99/dd6/ce2 0 2024-05-17T21:53:00.364 INFO:tasks.workunit.client.0.smithi107.stdout:6/577: dread d9/d16/d22/d5f/f9d [0,4194304] 0 2024-05-17T21:53:00.364 INFO:tasks.workunit.client.0.smithi107.stdout:4/678: stat d3/d6/fde 0 2024-05-17T21:53:00.365 INFO:tasks.workunit.client.1.smithi138.stdout:6/518: truncate d7/f28 1499063 0 2024-05-17T21:53:00.367 INFO:tasks.workunit.client.0.smithi107.stdout:6/578: write d9/d16/d22/d5f/f75 [309074,25151] 0 2024-05-17T21:53:00.368 INFO:tasks.workunit.client.1.smithi138.stdout:7/408: fdatasync d0/f4b 0 2024-05-17T21:53:00.368 INFO:tasks.workunit.client.0.smithi107.stdout:6/579: read d9/d16/f1e [1121760,31177] 0 2024-05-17T21:53:00.369 INFO:tasks.workunit.client.1.smithi138.stdout:0/514: dwrite d3/d10/f22 [4194304,4194304] 0 2024-05-17T21:53:00.369 INFO:tasks.workunit.client.0.smithi107.stdout:4/679: write d3/d4a/d47/d60/d3a/d41/d80/dad/fcd [843799,70958] 0 2024-05-17T21:53:00.369 INFO:tasks.workunit.client.0.smithi107.stdout:1/701: creat d4/df/d14/d2e/dd3/fe4 x:0 0 0 2024-05-17T21:53:00.372 INFO:tasks.workunit.client.0.smithi107.stdout:3/698: rename d6/d8/ce to d6/d1a/db1/ce3 0 2024-05-17T21:53:00.373 INFO:tasks.workunit.client.0.smithi107.stdout:3/699: chown d6/d8/d10/d35/d91/lbc 736834 1 2024-05-17T21:53:00.378 INFO:tasks.workunit.client.0.smithi107.stdout:0/744: fsync d3/d39/f69 0 2024-05-17T21:53:00.381 INFO:tasks.workunit.client.1.smithi138.stdout:8/437: mknod d0/dc/d1a/d33/c83 0 2024-05-17T21:53:00.382 INFO:tasks.workunit.client.1.smithi138.stdout:2/536: symlink d3/d21/d2f/d3e/d55/d99/d84/d9f/laf 0 2024-05-17T21:53:00.384 INFO:tasks.workunit.client.0.smithi107.stdout:4/680: creat d3/d4a/d47/d60/d3c/d58/fe3 x:0 0 0 2024-05-17T21:53:00.387 INFO:tasks.workunit.client.1.smithi138.stdout:9/476: write d6/d7/d16/d29/d48/d6f/f71 [3856030,105693] 0 2024-05-17T21:53:00.388 INFO:tasks.workunit.client.0.smithi107.stdout:6/580: dwrite d9/d16/d22/d2d/f39 [4194304,4194304] 0 2024-05-17T21:53:00.388 INFO:tasks.workunit.client.0.smithi107.stdout:4/681: dread d3/db/d35/d55/fc4 [0,4194304] 0 2024-05-17T21:53:00.388 INFO:tasks.workunit.client.0.smithi107.stdout:2/511: sync 2024-05-17T21:53:00.388 INFO:tasks.workunit.client.0.smithi107.stdout:7/629: sync 2024-05-17T21:53:00.390 INFO:tasks.workunit.client.0.smithi107.stdout:4/682: chown d3/l15 286094385 1 2024-05-17T21:53:00.390 INFO:tasks.workunit.client.0.smithi107.stdout:5/661: mkdir d5/d75/dc7/dd4 0 2024-05-17T21:53:00.390 INFO:tasks.workunit.client.0.smithi107.stdout:6/581: chown d9/d16/d22/d5f/d6e/f87 403487 1 2024-05-17T21:53:00.391 INFO:tasks.workunit.client.1.smithi138.stdout:2/537: dwrite d3/d21/d2f/f3f [0,4194304] 0 2024-05-17T21:53:00.393 INFO:tasks.workunit.client.0.smithi107.stdout:0/745: mknod d3/dc/dae/dc0/cfd 0 2024-05-17T21:53:00.394 INFO:tasks.workunit.client.0.smithi107.stdout:0/746: chown d3/d24/d6f/d9f/dab/fdb 6 1 2024-05-17T21:53:00.397 INFO:tasks.workunit.client.1.smithi138.stdout:7/409: readlink d0/l3f 0 2024-05-17T21:53:00.399 INFO:tasks.workunit.client.1.smithi138.stdout:5/493: dread d1/d6/f41 [0,4194304] 0 2024-05-17T21:53:00.405 INFO:tasks.workunit.client.1.smithi138.stdout:0/515: mknod d3/d10/d6d/cac 0 2024-05-17T21:53:00.408 INFO:tasks.workunit.client.0.smithi107.stdout:9/684: truncate d0/dd/d2f/fd9 2479101 0 2024-05-17T21:53:00.412 INFO:tasks.workunit.client.1.smithi138.stdout:8/438: symlink d0/dc/d1a/d33/d4a/l84 0 2024-05-17T21:53:00.415 INFO:tasks.workunit.client.1.smithi138.stdout:1/473: dread d4/df/f1f [0,4194304] 0 2024-05-17T21:53:00.416 INFO:tasks.workunit.client.0.smithi107.stdout:4/683: creat d3/fe4 x:0 0 0 2024-05-17T21:53:00.418 INFO:tasks.workunit.client.0.smithi107.stdout:6/582: mknod d9/d16/d22/d2d/cb4 0 2024-05-17T21:53:00.419 INFO:tasks.workunit.client.1.smithi138.stdout:3/603: link d1/d3/d16/lbd d1/d3/d8d/lcf 0 2024-05-17T21:53:00.421 INFO:tasks.workunit.client.1.smithi138.stdout:6/519: write d7/d8/f70 [231633,92951] 0 2024-05-17T21:53:00.422 INFO:tasks.workunit.client.0.smithi107.stdout:3/700: dwrite d6/d8/f12 [4194304,4194304] 0 2024-05-17T21:53:00.424 INFO:tasks.workunit.client.1.smithi138.stdout:2/538: dread d3/d81/fa7 [0,4194304] 0 2024-05-17T21:53:00.425 INFO:tasks.workunit.client.0.smithi107.stdout:9/685: mkdir d0/dd/d1b/de7 0 2024-05-17T21:53:00.426 INFO:tasks.workunit.client.0.smithi107.stdout:4/684: dwrite d3/db/d35/d4f/d93/fae [0,4194304] 0 2024-05-17T21:53:00.427 INFO:tasks.workunit.client.1.smithi138.stdout:7/410: readlink d0/l28 0 2024-05-17T21:53:00.430 INFO:tasks.workunit.client.1.smithi138.stdout:9/477: dwrite d6/d7/d16/d29/f37 [0,4194304] 0 2024-05-17T21:53:00.431 INFO:tasks.workunit.client.0.smithi107.stdout:1/702: truncate d4/df/d1a/f58 83188 0 2024-05-17T21:53:00.433 INFO:tasks.workunit.client.0.smithi107.stdout:2/512: mknod d0/d9/d11/ca3 0 2024-05-17T21:53:00.433 INFO:tasks.workunit.client.1.smithi138.stdout:9/478: fdatasync d6/d7/d16/d29/d38/d40/d44/f78 0 2024-05-17T21:53:00.434 INFO:tasks.workunit.client.1.smithi138.stdout:9/479: fsync d6/d7/f99 0 2024-05-17T21:53:00.435 INFO:tasks.workunit.client.0.smithi107.stdout:1/703: write d4/de/f36 [2109797,113344] 0 2024-05-17T21:53:00.436 INFO:tasks.workunit.client.0.smithi107.stdout:4/685: dwrite d3/db/d35/d55/d8a/fd7 [0,4194304] 0 2024-05-17T21:53:00.440 INFO:tasks.workunit.client.0.smithi107.stdout:2/513: read d0/d9/d11/f10 [2846790,78311] 0 2024-05-17T21:53:00.440 INFO:tasks.workunit.client.1.smithi138.stdout:4/502: truncate d0/d2f/f6f 416642 0 2024-05-17T21:53:00.441 INFO:tasks.workunit.client.0.smithi107.stdout:2/514: chown d0/d9/d1c/d3e/f6b 500 1 2024-05-17T21:53:00.441 INFO:tasks.workunit.client.1.smithi138.stdout:7/411: dread d0/d18/f45 [0,4194304] 0 2024-05-17T21:53:00.443 INFO:tasks.workunit.client.0.smithi107.stdout:6/583: rename d9/f53 to d9/d16/d22/d2d/fb5 0 2024-05-17T21:53:00.446 INFO:tasks.workunit.client.1.smithi138.stdout:1/474: unlink d4/df/d35/l76 0 2024-05-17T21:53:00.446 INFO:tasks.workunit.client.1.smithi138.stdout:1/475: readlink d4/d2e/l59 0 2024-05-17T21:53:00.447 INFO:tasks.workunit.client.1.smithi138.stdout:1/476: chown d4/d2e/d36/f4f 85457107 1 2024-05-17T21:53:00.450 INFO:tasks.workunit.client.1.smithi138.stdout:2/539: creat d3/d21/d2f/d3e/d69/fb0 x:0 0 0 2024-05-17T21:53:00.455 INFO:tasks.workunit.client.0.smithi107.stdout:0/747: dwrite d3/f18 [0,4194304] 0 2024-05-17T21:53:00.455 INFO:tasks.workunit.client.1.smithi138.stdout:6/520: dread d7/db/d34/da6/f26 [0,4194304] 0 2024-05-17T21:53:00.455 INFO:tasks.workunit.client.1.smithi138.stdout:5/494: creat d1/d53/d96/da0/fa1 x:0 0 0 2024-05-17T21:53:00.457 INFO:tasks.workunit.client.1.smithi138.stdout:5/495: write d1/d1a/d39/d4e/f51 [689828,118079] 0 2024-05-17T21:53:00.461 INFO:tasks.workunit.client.0.smithi107.stdout:7/630: dwrite d5/df/d11/f5c [0,4194304] 0 2024-05-17T21:53:00.463 INFO:tasks.workunit.client.0.smithi107.stdout:7/631: fsync d5/df/d11/d1c/f4e 0 2024-05-17T21:53:00.468 INFO:tasks.workunit.client.0.smithi107.stdout:2/515: mknod d0/d9/d1c/d3e/ca4 0 2024-05-17T21:53:00.472 INFO:tasks.workunit.client.1.smithi138.stdout:7/412: rename d0/d18/d40/d41/d50/f6a to d0/d39/f80 0 2024-05-17T21:53:00.477 INFO:tasks.workunit.client.1.smithi138.stdout:2/540: creat d3/d21/d2f/d3e/d58/fb1 x:0 0 0 2024-05-17T21:53:00.481 INFO:tasks.workunit.client.1.smithi138.stdout:6/521: unlink d7/db/d43/d5f/d6a/c8c 0 2024-05-17T21:53:00.482 INFO:tasks.workunit.client.0.smithi107.stdout:3/701: dwrite d6/d2a/f4d [0,4194304] 0 2024-05-17T21:53:00.484 INFO:tasks.workunit.client.0.smithi107.stdout:9/686: dwrite d0/dd/d21/d29/d3e/f91 [0,4194304] 0 2024-05-17T21:53:00.484 INFO:tasks.workunit.client.1.smithi138.stdout:0/516: truncate d3/d10/d6d/f75 3920691 0 2024-05-17T21:53:00.484 INFO:tasks.workunit.client.0.smithi107.stdout:3/702: truncate d6/d8/d3e/d3f/fba 169453 0 2024-05-17T21:53:00.487 INFO:tasks.workunit.client.0.smithi107.stdout:9/687: fdatasync d0/dd/d21/d29/d3e/d45/f61 0 2024-05-17T21:53:00.489 INFO:tasks.workunit.client.1.smithi138.stdout:8/439: fsync d0/dc/d1a/f1d 0 2024-05-17T21:53:00.491 INFO:tasks.workunit.client.1.smithi138.stdout:7/413: mkdir d0/d18/d40/d41/d51/d81 0 2024-05-17T21:53:00.492 INFO:tasks.workunit.client.1.smithi138.stdout:4/503: write d0/d3/d1d/f2e [624493,89291] 0 2024-05-17T21:53:00.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:00.501+0000 7fc8c3598700 1 -- 172.21.15.107:0/3997773272 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8bc0fbcc0 msgr2=0x7fc8bc0fc0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:53:00.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:00.501+0000 7fc8c3598700 1 --2- 172.21.15.107:0/3997773272 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8bc0fbcc0 0x7fc8bc0fc0d0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fc8b8004940 tx=0x7fc8b8009e30 comp rx=0 tx=0).stop 2024-05-17T21:53:00.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:00.501+0000 7fc8c3598700 1 -- 172.21.15.107:0/3997773272 shutdown_connections 2024-05-17T21:53:00.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:00.501+0000 7fc8c3598700 1 --2- 172.21.15.107:0/3997773272 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8bc0fcf70 0x7fc8bc0fd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:53:00.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:00.501+0000 7fc8c3598700 1 --2- 172.21.15.107:0/3997773272 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8bc0fbcc0 0x7fc8bc0fc0d0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:53:00.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:00.501+0000 7fc8c3598700 1 -- 172.21.15.107:0/3997773272 >> 172.21.15.107:0/3997773272 conn(0x7fc8bc0f7420 msgr2=0x7fc8bc0f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:53:00.494 INFO:tasks.workunit.client.0.smithi107.stdout:7/632: creat d5/d16/d3b/d4f/fe2 x:0 0 0 2024-05-17T21:53:00.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:00.502+0000 7fc8c3598700 1 -- 172.21.15.107:0/3997773272 shutdown_connections 2024-05-17T21:53:00.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:00.502+0000 7fc8c3598700 1 -- 172.21.15.107:0/3997773272 wait complete. 2024-05-17T21:53:00.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:00.502+0000 7fc8c3598700 1 Processor -- start 2024-05-17T21:53:00.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:00.502+0000 7fc8c3598700 1 -- start start 2024-05-17T21:53:00.496 INFO:tasks.workunit.client.0.smithi107.stdout:7/633: fsync d5/df/d11/d1c/f4e 0 2024-05-17T21:53:00.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:00.503+0000 7fc8c3598700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8bc0fbcc0 0x7fc8bc10fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:53:00.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:00.503+0000 7fc8c3598700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8bc0fcf70 0x7fc8bc110160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:53:00.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:00.503+0000 7fc8c3598700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8bc110760 con 0x7fc8bc0fbcc0 2024-05-17T21:53:00.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:00.503+0000 7fc8c3598700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8bc110fb0 con 0x7fc8bc0fcf70 2024-05-17T21:53:00.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:00.503+0000 7fc8c1334700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8bc0fbcc0 0x7fc8bc10fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:53:00.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:00.503+0000 7fc8c1334700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8bc0fbcc0 0x7fc8bc10fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:35388/0 (socket says 172.21.15.107:35388) 2024-05-17T21:53:00.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:00.503+0000 7fc8c1334700 1 -- 172.21.15.107:0/2116208213 learned_addr learned my addr 172.21.15.107:0/2116208213 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:53:00.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:00.503+0000 7fc8c0b33700 1 --2- 172.21.15.107:0/2116208213 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8bc0fcf70 0x7fc8bc110160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:53:00.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:00.503+0000 7fc8c1334700 1 -- 172.21.15.107:0/2116208213 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8bc0fcf70 msgr2=0x7fc8bc110160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:53:00.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:00.503+0000 7fc8c1334700 1 --2- 172.21.15.107:0/2116208213 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8bc0fcf70 0x7fc8bc110160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:53:00.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:00.503+0000 7fc8c1334700 1 -- 172.21.15.107:0/2116208213 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8b8005040 con 0x7fc8bc0fbcc0 2024-05-17T21:53:00.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:00.504+0000 7fc8c1334700 1 --2- 172.21.15.107:0/2116208213 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8bc0fbcc0 0x7fc8bc10fc20 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fc8b8009150 tx=0x7fc8b8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:53:00.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:00.504+0000 7fc8b1ffb700 1 -- 172.21.15.107:0/2116208213 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc8b800e040 con 0x7fc8bc0fbcc0 2024-05-17T21:53:00.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:00.504+0000 7fc8b1ffb700 1 -- 172.21.15.107:0/2116208213 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc8b800bbf0 con 0x7fc8bc0fbcc0 2024-05-17T21:53:00.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:00.504+0000 7fc8c3598700 1 -- 172.21.15.107:0/2116208213 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8bc111230 con 0x7fc8bc0fbcc0 2024-05-17T21:53:00.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:00.504+0000 7fc8b1ffb700 1 -- 172.21.15.107:0/2116208213 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc8b801c630 con 0x7fc8bc0fbcc0 2024-05-17T21:53:00.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:00.504+0000 7fc8c3598700 1 -- 172.21.15.107:0/2116208213 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8bc111770 con 0x7fc8bc0fbcc0 2024-05-17T21:53:00.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:00.505+0000 7fc8c3598700 1 -- 172.21.15.107:0/2116208213 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8bc101900 con 0x7fc8bc0fbcc0 2024-05-17T21:53:00.499 INFO:tasks.workunit.client.1.smithi138.stdout:2/541: rename d3/d21/d2f/d3e/d55/d62 to d3/d81/d96/db2 0 2024-05-17T21:53:00.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:00.506+0000 7fc8b1ffb700 1 -- 172.21.15.107:0/2116208213 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) v1 ==== 93316+0+0 (secure 0 0 0) 0x7fc8b800bd60 con 0x7fc8bc0fbcc0 2024-05-17T21:53:00.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:00.507+0000 7fc8b1ffb700 1 --2- 172.21.15.107:0/2116208213 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7fc8a8070bb0 0x7fc8a8073060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:53:00.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:00.507+0000 7fc8b1ffb700 1 -- 172.21.15.107:0/2116208213 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7fc8b8091f10 con 0x7fc8bc0fbcc0 2024-05-17T21:53:00.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:00.508+0000 7fc8c0b33700 1 --2- 172.21.15.107:0/2116208213 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7fc8a8070bb0 0x7fc8a8073060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:53:00.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:00.508+0000 7fc8c0b33700 1 --2- 172.21.15.107:0/2116208213 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7fc8a8070bb0 0x7fc8a8073060 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc8ac004990 tx=0x7fc8ac009ee0 comp rx=0 tx=0).ready entity=mgr.14636 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:53:00.501 INFO:tasks.workunit.client.1.smithi138.stdout:2/542: dread d3/d21/d2f/d3e/d55/d99/d84/f91 [0,4194304] 0 2024-05-17T21:53:00.502 INFO:tasks.workunit.client.1.smithi138.stdout:6/522: creat d7/db/d43/d3f/d79/fbb x:0 0 0 2024-05-17T21:53:00.502 INFO:tasks.workunit.client.0.smithi107.stdout:4/686: dread d3/d6/d26/d36/d7e/f87 [0,4194304] 0 2024-05-17T21:53:00.503 INFO:tasks.workunit.client.1.smithi138.stdout:0/517: fsync d3/d10/d39/f72 0 2024-05-17T21:53:00.504 INFO:tasks.workunit.client.0.smithi107.stdout:6/584: write d9/d58/d66/f6c [624426,77530] 0 2024-05-17T21:53:00.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:00.513+0000 7fc8b1ffb700 1 -- 172.21.15.107:0/2116208213 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+190121 (secure 0 0 0) 0x7fc8b805a370 con 0x7fc8bc0fbcc0 2024-05-17T21:53:00.507 INFO:tasks.workunit.client.0.smithi107.stdout:4/687: dread d3/db/f1f [8388608,4194304] 0 2024-05-17T21:53:00.507 INFO:tasks.workunit.client.1.smithi138.stdout:1/477: dwrite d4/dd/f2d [0,4194304] 0 2024-05-17T21:53:00.509 INFO:tasks.workunit.client.0.smithi107.stdout:0/748: symlink d3/d75/dd0/df6/lfe 0 2024-05-17T21:53:00.509 INFO:tasks.workunit.client.0.smithi107.stdout:0/749: chown d3/f4a 2982 1 2024-05-17T21:53:00.515 INFO:tasks.workunit.client.0.smithi107.stdout:3/703: unlink d6/d1a/f96 0 2024-05-17T21:53:00.515 INFO:tasks.workunit.client.1.smithi138.stdout:1/478: dwrite d4/db/f6a [4194304,4194304] 0 2024-05-17T21:53:00.516 INFO:tasks.workunit.client.0.smithi107.stdout:3/704: fdatasync d6/d8/d3e/d3f/fba 0 2024-05-17T21:53:00.516 INFO:tasks.workunit.client.0.smithi107.stdout:1/704: truncate d4/de/f19 6718404 0 2024-05-17T21:53:00.517 INFO:tasks.workunit.client.0.smithi107.stdout:1/705: fsync d4/df/d14/d2e/d43/fc6 0 2024-05-17T21:53:00.524 INFO:tasks.workunit.client.0.smithi107.stdout:9/688: mknod d0/dd/d3d/ce8 0 2024-05-17T21:53:00.525 INFO:tasks.workunit.client.1.smithi138.stdout:8/440: mknod d0/d11/c85 0 2024-05-17T21:53:00.526 INFO:tasks.workunit.client.1.smithi138.stdout:8/441: write d0/dc/d1a/f58 [6555097,56937] 0 2024-05-17T21:53:00.527 INFO:tasks.workunit.client.0.smithi107.stdout:7/634: fsync d5/df/d11/d1c/d31/d56/f9c 0 2024-05-17T21:53:00.528 INFO:tasks.workunit.client.1.smithi138.stdout:8/442: write d0/dc/d1a/f2a [782753,74241] 0 2024-05-17T21:53:00.529 INFO:tasks.workunit.client.1.smithi138.stdout:0/518: mkdir d3/d10/d38/d3e/d59/dad 0 2024-05-17T21:53:00.530 INFO:tasks.workunit.client.0.smithi107.stdout:9/689: dwrite d0/dd/f60 [4194304,4194304] 0 2024-05-17T21:53:00.531 INFO:tasks.workunit.client.0.smithi107.stdout:2/516: mknod d0/d9/d11/ca5 0 2024-05-17T21:53:00.532 INFO:tasks.workunit.client.1.smithi138.stdout:7/414: link d0/d39/f48 d0/d18/d40/f82 0 2024-05-17T21:53:00.532 INFO:tasks.workunit.client.0.smithi107.stdout:9/690: fsync d0/dd/d21/faa 0 2024-05-17T21:53:00.533 INFO:tasks.workunit.client.1.smithi138.stdout:7/415: fsync d0/d18/d40/d41/d50/f59 0 2024-05-17T21:53:00.534 INFO:tasks.workunit.client.1.smithi138.stdout:7/416: chown d0/d18/f1d 47975807 1 2024-05-17T21:53:00.542 INFO:tasks.workunit.client.1.smithi138.stdout:1/479: creat d4/df/f98 x:0 0 0 2024-05-17T21:53:00.543 INFO:tasks.workunit.client.0.smithi107.stdout:0/750: read d3/dc/d22/d3d/fa8 [469230,34847] 0 2024-05-17T21:53:00.546 INFO:tasks.workunit.client.0.smithi107.stdout:5/662: sync 2024-05-17T21:53:00.546 INFO:tasks.workunit.client.0.smithi107.stdout:8/664: sync 2024-05-17T21:53:00.548 INFO:tasks.workunit.client.0.smithi107.stdout:5/663: fsync d5/db/dc/d1e/fc1 0 2024-05-17T21:53:00.548 INFO:tasks.workunit.client.1.smithi138.stdout:6/523: symlink d7/db/d43/d5f/da4/db0/lbc 0 2024-05-17T21:53:00.549 INFO:tasks.workunit.client.1.smithi138.stdout:6/524: chown d7/d8/d78/l7a 851151495 1 2024-05-17T21:53:00.550 INFO:tasks.workunit.client.0.smithi107.stdout:7/635: rename d5/df/d11/d1c/d31/l81 to d5/d16/d3b/ddf/le3 0 2024-05-17T21:53:00.552 INFO:tasks.workunit.client.0.smithi107.stdout:2/517: rmdir d0/d9/d1c/d3e/d4f 39 2024-05-17T21:53:00.555 INFO:tasks.workunit.client.0.smithi107.stdout:9/691: dread d0/dd/d1b/f74 [0,4194304] 0 2024-05-17T21:53:00.555 INFO:tasks.workunit.client.0.smithi107.stdout:6/585: dwrite d9/d16/f5e [0,4194304] 0 2024-05-17T21:53:00.556 INFO:tasks.workunit.client.0.smithi107.stdout:9/692: stat d0/dd/d1f 0 2024-05-17T21:53:00.557 INFO:tasks.workunit.client.0.smithi107.stdout:6/586: dread - d9/d81/f97 zero size 2024-05-17T21:53:00.558 INFO:tasks.workunit.client.1.smithi138.stdout:0/519: rename d3/d10/l4d to d3/d10/d39/d97/lae 0 2024-05-17T21:53:00.559 INFO:tasks.workunit.client.1.smithi138.stdout:0/520: readlink d3/d10/d38/l63 0 2024-05-17T21:53:00.561 INFO:tasks.workunit.client.0.smithi107.stdout:2/518: dwrite d0/d9/f33 [0,4194304] 0 2024-05-17T21:53:00.561 INFO:tasks.workunit.client.1.smithi138.stdout:2/543: dwrite d3/d21/f2b [0,4194304] 0 2024-05-17T21:53:00.566 INFO:tasks.workunit.client.0.smithi107.stdout:1/706: dwrite d4/df/d1a/d26/d50/fa0 [0,4194304] 0 2024-05-17T21:53:00.570 INFO:tasks.workunit.client.0.smithi107.stdout:2/519: dwrite d0/d9/d11/df/f47 [8388608,4194304] 0 2024-05-17T21:53:00.570 INFO:tasks.workunit.client.1.smithi138.stdout:4/504: creat d0/d2f/d49/d60/d1f/d32/fb2 x:0 0 0 2024-05-17T21:53:00.571 INFO:tasks.workunit.client.1.smithi138.stdout:1/480: chown d4/db/l32 47430 1 2024-05-17T21:53:00.571 INFO:tasks.workunit.client.0.smithi107.stdout:0/751: creat d3/dc/d22/d3d/dc7/fff x:0 0 0 2024-05-17T21:53:00.572 INFO:tasks.workunit.client.0.smithi107.stdout:0/752: chown d3/dc/d22/f68 27382827 1 2024-05-17T21:53:00.581 INFO:tasks.workunit.client.1.smithi138.stdout:4/505: dwrite d0/d2f/d49/d60/d1f/d32/f43 [0,4194304] 0 2024-05-17T21:53:00.583 INFO:tasks.workunit.client.1.smithi138.stdout:0/521: creat d3/d10/d6d/d9f/faf x:0 0 0 2024-05-17T21:53:00.590 INFO:tasks.workunit.client.1.smithi138.stdout:7/417: symlink d0/d18/d40/d41/d51/d81/l83 0 2024-05-17T21:53:00.591 INFO:tasks.workunit.client.1.smithi138.stdout:7/418: fdatasync d0/d18/d40/d41/d50/f59 0 2024-05-17T21:53:00.593 INFO:tasks.workunit.client.0.smithi107.stdout:6/587: dread - d9/d16/d22/f56 zero size 2024-05-17T21:53:00.594 INFO:tasks.workunit.client.1.smithi138.stdout:1/481: creat d4/df/d3a/d3c/f99 x:0 0 0 2024-05-17T21:53:00.595 INFO:tasks.workunit.client.0.smithi107.stdout:9/693: rename d0/dd/d21/d29/d57/l77 to d0/dd/le9 0 2024-05-17T21:53:00.597 INFO:tasks.workunit.client.1.smithi138.stdout:0/522: creat d3/d3d/d48/fb0 x:0 0 0 2024-05-17T21:53:00.598 INFO:tasks.workunit.client.1.smithi138.stdout:0/523: chown d3/d3d/f78 1860293740 1 2024-05-17T21:53:00.599 INFO:tasks.workunit.client.1.smithi138.stdout:2/544: symlink d3/d21/d2f/lb3 0 2024-05-17T21:53:00.601 INFO:tasks.workunit.client.0.smithi107.stdout:2/520: mkdir d0/da6 0 2024-05-17T21:53:00.602 INFO:tasks.workunit.client.1.smithi138.stdout:1/482: rmdir d4/d2e/d36 39 2024-05-17T21:53:00.606 INFO:tasks.workunit.client.1.smithi138.stdout:4/506: rename d0/d2f/d49/d60/d1f/d32/d71/c98 to d0/d3/db/cb3 0 2024-05-17T21:53:00.608 INFO:tasks.workunit.client.0.smithi107.stdout:0/753: creat d3/dc/d22/d23/d97/d77/d91/f100 x:0 0 0 2024-05-17T21:53:00.609 INFO:tasks.workunit.client.1.smithi138.stdout:2/545: dread d3/f6f [0,4194304] 0 2024-05-17T21:53:00.611 INFO:tasks.workunit.client.1.smithi138.stdout:1/483: unlink d4/db/d29/d89/f96 0 2024-05-17T21:53:00.612 INFO:tasks.workunit.client.0.smithi107.stdout:3/705: truncate d6/d2a/d2d/d48/d4f/f94 4168290 0 2024-05-17T21:53:00.615 INFO:tasks.workunit.client.0.smithi107.stdout:6/588: dwrite d9/d16/d22/d5f/d6e/fa4 [0,4194304] 0 2024-05-17T21:53:00.615 INFO:tasks.workunit.client.1.smithi138.stdout:6/525: getdents d7/d8 0 2024-05-17T21:53:00.615 INFO:tasks.workunit.client.1.smithi138.stdout:0/524: dread d3/d10/f11 [0,4194304] 0 2024-05-17T21:53:00.615 INFO:tasks.workunit.client.0.smithi107.stdout:4/688: getdents d3/db/d1e/d99 0 2024-05-17T21:53:00.616 INFO:tasks.workunit.client.0.smithi107.stdout:6/589: stat d9/d16/c80 0 2024-05-17T21:53:00.616 INFO:tasks.workunit.client.1.smithi138.stdout:4/507: truncate d0/d1b/d88/f8b 857063 0 2024-05-17T21:53:00.620 INFO:tasks.workunit.client.0.smithi107.stdout:7/636: dwrite d5/d16/f3a [0,4194304] 0 2024-05-17T21:53:00.627 INFO:tasks.workunit.client.1.smithi138.stdout:1/484: write d4/d2e/d36/f4f [4141266,46201] 0 2024-05-17T21:53:00.628 INFO:tasks.workunit.client.0.smithi107.stdout:0/754: symlink d3/d24/d6f/d9f/l101 0 2024-05-17T21:53:00.629 INFO:tasks.workunit.client.0.smithi107.stdout:0/755: chown d3/dc/d22/d23/d97/d77 1436 1 2024-05-17T21:53:00.629 INFO:tasks.workunit.client.1.smithi138.stdout:2/546: dwrite d3/d21/d2f/f36 [0,4194304] 0 2024-05-17T21:53:00.636 INFO:tasks.workunit.client.0.smithi107.stdout:3/706: mkdir d6/d8/d10/d35/d91/de4 0 2024-05-17T21:53:00.638 INFO:tasks.workunit.client.1.smithi138.stdout:6/526: dwrite d7/d69/f9d [0,4194304] 0 2024-05-17T21:53:00.638 INFO:tasks.workunit.client.0.smithi107.stdout:8/665: truncate dc/f43 3662239 0 2024-05-17T21:53:00.639 INFO:tasks.workunit.client.0.smithi107.stdout:4/689: dread - d3/db/d35/f6b zero size 2024-05-17T21:53:00.640 INFO:tasks.workunit.client.1.smithi138.stdout:7/419: dwrite d0/d18/d40/f44 [0,4194304] 0 2024-05-17T21:53:00.644 INFO:tasks.workunit.client.0.smithi107.stdout:6/590: rename d9/d16/d22/d2d/f49 to d9/d81/fb6 0 2024-05-17T21:53:00.650 INFO:tasks.workunit.client.1.smithi138.stdout:1/485: creat d4/db/d29/d89/f9a x:0 0 0 2024-05-17T21:53:00.652 INFO:tasks.workunit.client.0.smithi107.stdout:2/521: creat d0/d9/d1c/d37/d9f/fa7 x:0 0 0 2024-05-17T21:53:00.652 INFO:tasks.workunit.client.0.smithi107.stdout:1/707: truncate d4/df/d1a/d26/d50/f95 2493944 0 2024-05-17T21:53:00.654 INFO:tasks.workunit.client.1.smithi138.stdout:2/547: creat d3/d81/d96/fb4 x:0 0 0 2024-05-17T21:53:00.657 INFO:tasks.workunit.client.0.smithi107.stdout:5/664: getdents d5/db/d16 0 2024-05-17T21:53:00.658 INFO:tasks.workunit.client.1.smithi138.stdout:1/486: dwrite d4/df/d3a/f81 [0,4194304] 0 2024-05-17T21:53:00.660 INFO:tasks.workunit.client.0.smithi107.stdout:1/708: dwrite d4/df/d4e/d59/fcc [0,4194304] 0 2024-05-17T21:53:00.661 INFO:tasks.workunit.client.1.smithi138.stdout:6/527: creat d7/d8/d78/fbd x:0 0 0 2024-05-17T21:53:00.664 INFO:tasks.workunit.client.1.smithi138.stdout:1/487: dread d4/dd/f2d [0,4194304] 0 2024-05-17T21:53:00.764 INFO:tasks.workunit.client.1.smithi138.stdout:7/420: mkdir d0/d18/d40/d41/d51/d81/d84 0 2024-05-17T21:53:00.769 INFO:tasks.workunit.client.0.smithi107.stdout:2/522: fdatasync d0/d9/d1c/f73 0 2024-05-17T21:53:00.771 INFO:tasks.workunit.client.1.smithi138.stdout:0/525: rename d3/d10/c37 to d3/d10/cb1 0 2024-05-17T21:53:00.773 INFO:tasks.workunit.client.1.smithi138.stdout:4/508: rename d0/d2f/d49/d60/f9e to d0/d3/d17/d5e/fb4 0 2024-05-17T21:53:00.780 INFO:tasks.workunit.client.1.smithi138.stdout:1/488: unlink d4/df/c30 0 2024-05-17T21:53:00.783 INFO:tasks.workunit.client.0.smithi107.stdout:1/709: rename d4/de/f36 to d4/df/d1a/d41/fe5 0 2024-05-17T21:53:00.785 INFO:tasks.workunit.client.0.smithi107.stdout:3/707: dwrite d6/d8/d10/d25/f99 [0,4194304] 0 2024-05-17T21:53:00.789 INFO:tasks.workunit.client.1.smithi138.stdout:6/528: dread d7/db/f11 [0,4194304] 0 2024-05-17T21:53:00.790 INFO:tasks.workunit.client.1.smithi138.stdout:6/529: stat d7/db/c3b 0 2024-05-17T21:53:00.792 INFO:tasks.workunit.client.0.smithi107.stdout:4/690: sync 2024-05-17T21:53:00.793 INFO:tasks.workunit.client.0.smithi107.stdout:4/691: stat d3/db/d35/d4f/lc0 0 2024-05-17T21:53:00.794 INFO:tasks.workunit.client.1.smithi138.stdout:4/509: dread d0/d3/d17/d21/d78/f9a [0,4194304] 0 2024-05-17T21:53:00.794 INFO:tasks.workunit.client.1.smithi138.stdout:3/604: sync 2024-05-17T21:53:00.794 INFO:tasks.workunit.client.0.smithi107.stdout:4/692: write d3/d4a/f75 [1174205,108023] 0 2024-05-17T21:53:00.800 INFO:tasks.workunit.client.1.smithi138.stdout:3/605: dwrite d1/d4f/d80/f86 [0,4194304] 0 2024-05-17T21:53:00.811 INFO:tasks.workunit.client.1.smithi138.stdout:7/421: creat d0/d39/f85 x:0 0 0 2024-05-17T21:53:00.811 INFO:tasks.workunit.client.1.smithi138.stdout:7/422: fdatasync d0/f4d 0 2024-05-17T21:53:00.814 INFO:tasks.workunit.client.1.smithi138.stdout:0/526: dread d3/d10/f35 [0,4194304] 0 2024-05-17T21:53:00.820 INFO:tasks.workunit.client.1.smithi138.stdout:4/510: symlink d0/d3/db/da0/lb5 0 2024-05-17T21:53:00.820 INFO:tasks.workunit.client.1.smithi138.stdout:1/489: dwrite d4/df/f12 [0,4194304] 0 2024-05-17T21:53:00.824 INFO:tasks.workunit.client.1.smithi138.stdout:3/606: mknod d1/d62/dad/d5c/cd0 0 2024-05-17T21:53:00.829 INFO:tasks.workunit.client.1.smithi138.stdout:7/423: chown d0/f1a 180628 1 2024-05-17T21:53:00.832 INFO:tasks.workunit.client.1.smithi138.stdout:6/530: mkdir d7/da0/dbe 0 2024-05-17T21:53:00.833 INFO:tasks.workunit.client.0.smithi107.stdout:2/523: mknod d0/d9/d11/d18/ca8 0 2024-05-17T21:53:00.837 INFO:tasks.workunit.client.0.smithi107.stdout:2/524: read d0/d9/d1c/d37/f98 [208200,96773] 0 2024-05-17T21:53:00.837 INFO:tasks.workunit.client.1.smithi138.stdout:3/607: chown d1/c26 29416 1 2024-05-17T21:53:00.838 INFO:tasks.workunit.client.0.smithi107.stdout:2/525: chown d0/d9/d11/c85 2424 1 2024-05-17T21:53:00.839 INFO:tasks.workunit.client.0.smithi107.stdout:5/665: symlink d5/db/d16/d2d/d4d/ld5 0 2024-05-17T21:53:00.842 INFO:tasks.workunit.client.0.smithi107.stdout:0/756: creat d3/d75/f102 x:0 0 0 2024-05-17T21:53:00.843 INFO:tasks.workunit.client.1.smithi138.stdout:0/527: mknod d3/d10/d6d/d9f/da8/daa/cb2 0 2024-05-17T21:53:00.843 INFO:tasks.workunit.client.0.smithi107.stdout:0/757: write d3/dc/f14 [620481,125352] 0 2024-05-17T21:53:00.844 INFO:tasks.workunit.client.0.smithi107.stdout:9/694: getdents d0/dd/d21/d29/d3e/d94/d9a 0 2024-05-17T21:53:00.845 INFO:tasks.workunit.client.1.smithi138.stdout:3/608: dwrite d1/f65 [0,4194304] 0 2024-05-17T21:53:00.846 INFO:tasks.workunit.client.1.smithi138.stdout:1/490: mknod d4/dd/c9b 0 2024-05-17T21:53:00.848 INFO:tasks.workunit.client.1.smithi138.stdout:1/491: write d4/dd/d3b/d6c/f92 [183060,11657] 0 2024-05-17T21:53:00.851 INFO:tasks.workunit.client.1.smithi138.stdout:3/609: dread d1/d4f/d80/fac [0,4194304] 0 2024-05-17T21:53:00.851 INFO:tasks.workunit.client.0.smithi107.stdout:9/695: dwrite d0/dd/d21/d29/f5a [0,4194304] 0 2024-05-17T21:53:00.857 INFO:tasks.workunit.client.0.smithi107.stdout:3/708: creat d6/d8/d3e/d3f/d90/da7/fe5 x:0 0 0 2024-05-17T21:53:00.857 INFO:tasks.workunit.client.1.smithi138.stdout:0/528: write d3/d10/f11 [3158113,37995] 0 2024-05-17T21:53:00.860 INFO:tasks.workunit.client.0.smithi107.stdout:0/758: dread d3/d24/f2a [0,4194304] 0 2024-05-17T21:53:00.862 INFO:tasks.workunit.client.1.smithi138.stdout:4/511: rmdir d0/d3/d1d/d4c 0 2024-05-17T21:53:00.863 INFO:tasks.workunit.client.0.smithi107.stdout:4/693: creat d3/ddf/fe5 x:0 0 0 2024-05-17T21:53:00.864 INFO:tasks.workunit.client.1.smithi138.stdout:1/492: fsync d4/df/d3a/d3c/d67/f85 0 2024-05-17T21:53:00.866 INFO:tasks.workunit.client.1.smithi138.stdout:3/610: unlink d1/d4f/d50/f9e 0 2024-05-17T21:53:00.869 INFO:tasks.workunit.client.0.smithi107.stdout:8/666: dwrite dc/d21/d97/f15 [0,4194304] 0 2024-05-17T21:53:00.870 INFO:tasks.workunit.client.0.smithi107.stdout:7/637: getdents d5/df/d11/d1c/d2f/d65 0 2024-05-17T21:53:00.871 INFO:tasks.workunit.client.1.smithi138.stdout:4/512: read d0/d2f/f82 [265794,18560] 0 2024-05-17T21:53:00.872 INFO:tasks.workunit.client.1.smithi138.stdout:4/513: chown d0/d3/d1d/d38/d9c 466678 1 2024-05-17T21:53:00.880 INFO:tasks.workunit.client.1.smithi138.stdout:4/514: dwrite d0/d2f/d49/d60/d1f/f70 [4194304,4194304] 0 2024-05-17T21:53:00.881 INFO:tasks.workunit.client.1.smithi138.stdout:7/424: write d0/f19 [1618692,98894] 0 2024-05-17T21:53:00.882 INFO:tasks.workunit.client.1.smithi138.stdout:7/425: chown d0/d18/d40/d41/d51/d81 15 1 2024-05-17T21:53:00.887 INFO:tasks.workunit.client.0.smithi107.stdout:3/709: readlink d6/d8/d10/d35/d36/d66/lac 0 2024-05-17T21:53:00.890 INFO:tasks.workunit.client.0.smithi107.stdout:0/759: rmdir d3/dc/d22/d23 39 2024-05-17T21:53:00.892 INFO:tasks.workunit.client.0.smithi107.stdout:4/694: creat d3/d6/d26/d36/d8b/d95/fe6 x:0 0 0 2024-05-17T21:53:00.892 INFO:tasks.workunit.client.1.smithi138.stdout:3/611: chown d1/d3/d8d/d92 3955 1 2024-05-17T21:53:00.895 INFO:tasks.workunit.client.1.smithi138.stdout:4/515: creat d0/d3/d1d/d5b/fb6 x:0 0 0 2024-05-17T21:53:00.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:00.902+0000 7fc8c3598700 1 -- 172.21.15.107:0/2116208213 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fc8bc04bed0 con 0x7fc8bc0fbcc0 2024-05-17T21:53:00.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:00.902+0000 7fc8b1ffb700 1 -- 172.21.15.107:0/2116208213 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+846 (secure 0 0 0) 0x7fc8b8059d10 con 0x7fc8bc0fbcc0 2024-05-17T21:53:00.895 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-05-17T21:53:00.895 INFO:teuthology.orchestra.run.smithi107.stdout: "mon": { 2024-05-17T21:53:00.895 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-952-g9903f776 (9903f7767386a9d9efadb7c1effdcd3064d71804) quincy (stable)": 2 2024-05-17T21:53:00.896 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-05-17T21:53:00.896 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr": { 2024-05-17T21:53:00.896 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-952-g9903f776 (9903f7767386a9d9efadb7c1effdcd3064d71804) quincy (stable)": 1, 2024-05-17T21:53:00.896 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev)": 1 2024-05-17T21:53:00.896 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-05-17T21:53:00.896 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": { 2024-05-17T21:53:00.896 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-952-g9903f776 (9903f7767386a9d9efadb7c1effdcd3064d71804) quincy (stable)": 6 2024-05-17T21:53:00.896 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-05-17T21:53:00.896 INFO:teuthology.orchestra.run.smithi107.stdout: "mds": { 2024-05-17T21:53:00.896 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-952-g9903f776 (9903f7767386a9d9efadb7c1effdcd3064d71804) quincy (stable)": 4 2024-05-17T21:53:00.896 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-05-17T21:53:00.896 INFO:teuthology.orchestra.run.smithi107.stdout: "overall": { 2024-05-17T21:53:00.896 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-952-g9903f776 (9903f7767386a9d9efadb7c1effdcd3064d71804) quincy (stable)": 13, 2024-05-17T21:53:00.896 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev)": 1 2024-05-17T21:53:00.897 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-05-17T21:53:00.897 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-05-17T21:53:00.897 INFO:tasks.workunit.client.0.smithi107.stdout:8/667: mkdir dc/d21/d27/d2f/dba/dee 0 2024-05-17T21:53:00.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:00.906+0000 7fc8c3598700 1 -- 172.21.15.107:0/2116208213 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7fc8a8070bb0 msgr2=0x7fc8a8073060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:53:00.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:00.906+0000 7fc8c3598700 1 --2- 172.21.15.107:0/2116208213 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7fc8a8070bb0 0x7fc8a8073060 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc8ac004990 tx=0x7fc8ac009ee0 comp rx=0 tx=0).stop 2024-05-17T21:53:00.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:00.906+0000 7fc8c3598700 1 -- 172.21.15.107:0/2116208213 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8bc0fbcc0 msgr2=0x7fc8bc10fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:53:00.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:00.906+0000 7fc8c3598700 1 --2- 172.21.15.107:0/2116208213 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8bc0fbcc0 0x7fc8bc10fc20 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fc8b8009150 tx=0x7fc8b8009e30 comp rx=0 tx=0).stop 2024-05-17T21:53:00.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:00.906+0000 7fc8c3598700 1 -- 172.21.15.107:0/2116208213 shutdown_connections 2024-05-17T21:53:00.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:00.906+0000 7fc8c3598700 1 --2- 172.21.15.107:0/2116208213 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7fc8a8070bb0 0x7fc8a8073060 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:53:00.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:00.906+0000 7fc8c3598700 1 --2- 172.21.15.107:0/2116208213 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8bc0fbcc0 0x7fc8bc10fc20 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:53:00.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:00.906+0000 7fc8c3598700 1 --2- 172.21.15.107:0/2116208213 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8bc0fcf70 0x7fc8bc110160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:53:00.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:00.906+0000 7fc8c3598700 1 -- 172.21.15.107:0/2116208213 >> 172.21.15.107:0/2116208213 conn(0x7fc8bc0f7420 msgr2=0x7fc8bc0663d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:53:00.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:00.906+0000 7fc8c3598700 1 -- 172.21.15.107:0/2116208213 shutdown_connections 2024-05-17T21:53:00.899 INFO:tasks.workunit.client.0.smithi107.stdout:4/695: dwrite d3/d4a/d47/f4b [0,4194304] 0 2024-05-17T21:53:00.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:00.906+0000 7fc8c3598700 1 -- 172.21.15.107:0/2116208213 wait complete. 2024-05-17T21:53:00.900 INFO:tasks.workunit.client.0.smithi107.stdout:7/638: write d5/df/d11/d1c/d31/f48 [7333719,7157] 0 2024-05-17T21:53:00.900 INFO:tasks.workunit.client.0.smithi107.stdout:4/696: chown d3/d6/d26/d3b/l76 35472286 1 2024-05-17T21:53:00.902 INFO:tasks.workunit.client.0.smithi107.stdout:9/696: creat d0/dd/d2f/d87/fea x:0 0 0 2024-05-17T21:53:00.902 INFO:tasks.workunit.client.0.smithi107.stdout:7/639: read d5/df/d11/f7a [154508,21920] 0 2024-05-17T21:53:00.903 INFO:tasks.workunit.client.1.smithi138.stdout:1/493: write d4/df/d4e/f55 [1189293,54645] 0 2024-05-17T21:53:00.903 INFO:tasks.workunit.client.1.smithi138.stdout:3/612: rmdir d1/d62/db0 39 2024-05-17T21:53:00.903 INFO:tasks.workunit.client.0.smithi107.stdout:4/697: read d3/db/f33 [5171528,71258] 0 2024-05-17T21:53:00.904 INFO:tasks.workunit.client.1.smithi138.stdout:7/426: dread d0/f1e [0,4194304] 0 2024-05-17T21:53:00.904 INFO:tasks.workunit.client.0.smithi107.stdout:7/640: stat d5/df/d11/d1d/d33/d82/fa0 0 2024-05-17T21:53:00.904 INFO:tasks.workunit.client.0.smithi107.stdout:9/697: chown d0/dd/d21/d29/d3e/fc4 108 1 2024-05-17T21:53:00.906 INFO:tasks.workunit.client.0.smithi107.stdout:1/710: rename d4/df/d69/f6e to d4/df/d1a/d26/fe6 0 2024-05-17T21:53:00.906 INFO:tasks.workunit.client.0.smithi107.stdout:7/641: fsync d5/df/d11/d1c/d31/d39/f58 0 2024-05-17T21:53:00.907 INFO:tasks.workunit.client.1.smithi138.stdout:0/529: getdents d3 0 2024-05-17T21:53:00.910 INFO:tasks.workunit.client.1.smithi138.stdout:1/494: mknod d4/df/d4e/c9c 0 2024-05-17T21:53:00.911 INFO:tasks.workunit.client.0.smithi107.stdout:6/591: link d9/l40 d9/d16/d4d/d67/lb7 0 2024-05-17T21:53:00.911 INFO:tasks.workunit.client.1.smithi138.stdout:3/613: rmdir d1/d3/d8d 39 2024-05-17T21:53:00.914 INFO:tasks.workunit.client.0.smithi107.stdout:7/642: dwrite d5/df/d11/d1c/d2f/fd2 [0,4194304] 0 2024-05-17T21:53:00.916 INFO:tasks.workunit.client.0.smithi107.stdout:5/666: write d5/db/d16/d2a/d73/f9e [1253510,98519] 0 2024-05-17T21:53:00.916 INFO:tasks.workunit.client.0.smithi107.stdout:4/698: rmdir d3/db/d1e/d66 39 2024-05-17T21:53:00.919 INFO:tasks.workunit.client.0.smithi107.stdout:0/760: write d3/d58/d61/d94/f95 [1092309,87855] 0 2024-05-17T21:53:00.919 INFO:tasks.workunit.client.0.smithi107.stdout:0/761: stat d3/d24/d6f/d9f 0 2024-05-17T21:53:00.923 INFO:tasks.workunit.client.0.smithi107.stdout:2/526: rename d0/d9/d11/d15/f4e to d0/d26/d64/fa9 0 2024-05-17T21:53:00.926 INFO:tasks.workunit.client.0.smithi107.stdout:7/643: mknod d5/df/d11/d1c/d31/d39/d92/d9a/ce4 0 2024-05-17T21:53:00.926 INFO:tasks.workunit.client.0.smithi107.stdout:2/527: read d0/f36 [1158832,54207] 0 2024-05-17T21:53:00.927 INFO:tasks.workunit.client.0.smithi107.stdout:4/699: creat d3/d4a/d47/d60/d3c/d9d/fe7 x:0 0 0 2024-05-17T21:53:00.928 INFO:tasks.workunit.client.0.smithi107.stdout:0/762: dwrite d3/d75/d9e/f83 [0,4194304] 0 2024-05-17T21:53:00.928 INFO:tasks.workunit.client.0.smithi107.stdout:9/698: symlink d0/dd/d21/d29/d3e/d45/d68/leb 0 2024-05-17T21:53:00.929 INFO:tasks.workunit.client.0.smithi107.stdout:3/710: rename d6/d8/f9 to d6/d8/d10/d25/fe6 0 2024-05-17T21:53:00.931 INFO:tasks.workunit.client.0.smithi107.stdout:9/699: write d0/dd/d21/d29/dd4/fa0 [685956,5010] 0 2024-05-17T21:53:00.931 INFO:tasks.workunit.client.0.smithi107.stdout:6/592: mkdir d9/d16/d4d/db8 0 2024-05-17T21:53:00.931 INFO:tasks.workunit.client.0.smithi107.stdout:3/711: stat d6/d8/d3e/d3f/d90/f9f 0 2024-05-17T21:53:00.933 INFO:tasks.workunit.client.0.smithi107.stdout:3/712: stat d6/d8/d10/c9c 0 2024-05-17T21:53:00.933 INFO:tasks.workunit.client.0.smithi107.stdout:7/644: fdatasync d5/df/d11/d1c/f2e 0 2024-05-17T21:53:00.935 INFO:tasks.workunit.client.1.smithi138.stdout:9/480: sync 2024-05-17T21:53:00.935 INFO:tasks.workunit.client.1.smithi138.stdout:8/443: sync 2024-05-17T21:53:00.935 INFO:tasks.workunit.client.1.smithi138.stdout:5/496: sync 2024-05-17T21:53:00.936 INFO:tasks.workunit.client.0.smithi107.stdout:0/763: truncate d3/d39/f7f 379895 0 2024-05-17T21:53:00.937 INFO:tasks.workunit.client.0.smithi107.stdout:2/528: rmdir d0/d26/d64 39 2024-05-17T21:53:00.939 INFO:tasks.workunit.client.0.smithi107.stdout:4/700: dwrite d3/d4a/d47/d60/d3c/d58/fe3 [0,4194304] 0 2024-05-17T21:53:00.940 INFO:tasks.workunit.client.0.smithi107.stdout:8/668: rename dc/d44/f84 to dc/d21/d97/db7/fef 0 2024-05-17T21:53:00.941 INFO:tasks.workunit.client.0.smithi107.stdout:4/701: fdatasync d3/d4a/f75 0 2024-05-17T21:53:00.944 INFO:tasks.workunit.client.1.smithi138.stdout:8/444: stat d0/dc/d3b/c67 0 2024-05-17T21:53:00.944 INFO:tasks.workunit.client.1.smithi138.stdout:9/481: stat d6/d7/d16/d29/d48/d3b 0 2024-05-17T21:53:00.944 INFO:tasks.workunit.client.0.smithi107.stdout:9/700: chown d0/dd/d21/d29/d3e/d45/d68/d75 25092025 1 2024-05-17T21:53:00.944 INFO:tasks.workunit.client.1.smithi138.stdout:0/530: creat d3/d10/d6d/d9f/da8/fb3 x:0 0 0 2024-05-17T21:53:00.945 INFO:tasks.workunit.client.1.smithi138.stdout:1/495: rmdir d4/d2e 39 2024-05-17T21:53:00.948 INFO:tasks.workunit.client.1.smithi138.stdout:1/496: read d4/dd/d3b/f53 [136591,102766] 0 2024-05-17T21:53:00.948 INFO:tasks.workunit.client.1.smithi138.stdout:3/614: creat d1/d3/d16/d2d/fd1 x:0 0 0 2024-05-17T21:53:00.950 INFO:tasks.workunit.client.1.smithi138.stdout:1/497: fdatasync d4/dd/d3b/f53 0 2024-05-17T21:53:00.954 INFO:tasks.workunit.client.1.smithi138.stdout:8/445: dwrite d0/f1b [0,4194304] 0 2024-05-17T21:53:00.956 INFO:tasks.workunit.client.0.smithi107.stdout:1/711: dread d4/df/d1a/d26/fe6 [0,4194304] 0 2024-05-17T21:53:00.959 INFO:tasks.workunit.client.1.smithi138.stdout:5/497: dread - d1/d6/d1b/d29/d36/d68/f6c zero size 2024-05-17T21:53:00.961 INFO:tasks.workunit.client.0.smithi107.stdout:6/593: rename d9/d16/d22/d5f/f75 to d9/d16/d22/daf/fb9 0 2024-05-17T21:53:00.961 INFO:tasks.workunit.client.0.smithi107.stdout:6/594: stat d9/f1f 0 2024-05-17T21:53:00.964 INFO:tasks.workunit.client.1.smithi138.stdout:5/498: dread d1/d53/d96/f9b [0,4194304] 0 2024-05-17T21:53:00.965 INFO:tasks.workunit.client.0.smithi107.stdout:5/667: getdents d5/db/d16/d54/da0 0 2024-05-17T21:53:00.965 INFO:tasks.workunit.client.1.smithi138.stdout:5/499: read - d1/d6/d1b/d29/d36/d68/f6c zero size 2024-05-17T21:53:00.967 INFO:tasks.workunit.client.0.smithi107.stdout:4/702: symlink d3/db/d1e/d99/le8 0 2024-05-17T21:53:00.974 INFO:tasks.workunit.client.1.smithi138.stdout:0/531: dwrite d3/f5e [0,4194304] 0 2024-05-17T21:53:00.979 INFO:tasks.workunit.client.1.smithi138.stdout:3/615: dread d1/d3/f25 [0,4194304] 0 2024-05-17T21:53:00.980 INFO:tasks.workunit.client.1.smithi138.stdout:7/427: getdents d0/d18/d40/d5f 0 2024-05-17T21:53:00.980 INFO:tasks.workunit.client.1.smithi138.stdout:3/616: write d1/d3/d16/d3e/f73 [4323254,1203] 0 2024-05-17T21:53:00.981 INFO:tasks.workunit.client.1.smithi138.stdout:1/498: write d4/df/d4e/f84 [4759026,51917] 0 2024-05-17T21:53:00.982 INFO:tasks.workunit.client.1.smithi138.stdout:7/428: dread - d0/d18/d40/d5f/f73 zero size 2024-05-17T21:53:00.983 INFO:tasks.workunit.client.1.smithi138.stdout:5/500: creat d1/d1a/d39/d69/fa2 x:0 0 0 2024-05-17T21:53:00.985 INFO:tasks.workunit.client.0.smithi107.stdout:8/669: dread dc/d21/d35/f2e [0,4194304] 0 2024-05-17T21:53:00.985 INFO:tasks.workunit.client.0.smithi107.stdout:7/645: truncate d5/d16/d3b/d45/d91/fc7 1033497 0 2024-05-17T21:53:00.987 INFO:tasks.workunit.client.0.smithi107.stdout:1/712: write d4/df/d1a/f7b [2605319,71999] 0 2024-05-17T21:53:00.987 INFO:tasks.workunit.client.0.smithi107.stdout:2/529: dread d0/d9/f46 [0,4194304] 0 2024-05-17T21:53:00.989 INFO:tasks.workunit.client.0.smithi107.stdout:1/713: chown d4/df/d14/l39 50 1 2024-05-17T21:53:00.989 INFO:tasks.workunit.client.0.smithi107.stdout:0/764: dread d3/d58/faa [0,4194304] 0 2024-05-17T21:53:00.989 INFO:tasks.workunit.client.0.smithi107.stdout:2/530: stat d0/d9/d11/d15/f57 0 2024-05-17T21:53:00.990 INFO:tasks.workunit.client.0.smithi107.stdout:5/668: creat d5/db/d16/d54/d59/d65/fd6 x:0 0 0 2024-05-17T21:53:00.990 INFO:tasks.workunit.client.1.smithi138.stdout:1/499: dwrite d4/dd/d3b/f73 [0,4194304] 0 2024-05-17T21:53:00.992 INFO:tasks.workunit.client.1.smithi138.stdout:1/500: read - d4/db/f7b zero size 2024-05-17T21:53:00.992 INFO:tasks.workunit.client.1.smithi138.stdout:7/429: dwrite d0/d18/d40/d41/d51/f69 [0,4194304] 0 2024-05-17T21:53:00.993 INFO:tasks.workunit.client.0.smithi107.stdout:4/703: dread - d3/d4a/d47/d60/d3a/d41/d80/fa5 zero size 2024-05-17T21:53:00.993 INFO:tasks.workunit.client.1.smithi138.stdout:0/532: rename d3/d10/l4e to d3/lb4 0 2024-05-17T21:53:00.993 INFO:tasks.workunit.client.0.smithi107.stdout:4/704: dread - d3/ddf/fe5 zero size 2024-05-17T21:53:00.996 INFO:tasks.workunit.client.0.smithi107.stdout:2/531: dwrite d0/d9/d11/df/f47 [12582912,4194304] 0 2024-05-17T21:53:00.998 INFO:tasks.workunit.client.0.smithi107.stdout:8/670: creat dc/d21/daa/dc5/ff0 x:0 0 0 2024-05-17T21:53:01.003 INFO:tasks.workunit.client.0.smithi107.stdout:7/646: rmdir d5/d16/d3b/d45/d91/dc1 39 2024-05-17T21:53:01.007 INFO:tasks.workunit.client.1.smithi138.stdout:1/501: fsync d4/df/d3a/d3c/d67/f85 0 2024-05-17T21:53:01.009 INFO:tasks.workunit.client.1.smithi138.stdout:1/502: dread - d4/df/d3a/d3c/d63/d6d/f87 zero size 2024-05-17T21:53:01.010 INFO:tasks.workunit.client.1.smithi138.stdout:7/430: symlink d0/d18/d40/d41/d51/l86 0 2024-05-17T21:53:01.013 INFO:tasks.workunit.client.0.smithi107.stdout:5/669: rmdir d5/db/d16/d54/d59 39 2024-05-17T21:53:01.013 INFO:tasks.workunit.client.1.smithi138.stdout:8/446: getdents d0/d63 0 2024-05-17T21:53:01.014 INFO:tasks.workunit.client.0.smithi107.stdout:9/701: link d0/dd/d21/d29/d3e/d45/d68/f8c d0/dd/d21/d29/d3e/d45/fec 0 2024-05-17T21:53:01.015 INFO:tasks.workunit.client.0.smithi107.stdout:4/705: creat d3/d6/d26/d3b/d91/fe9 x:0 0 0 2024-05-17T21:53:01.016 INFO:tasks.workunit.client.0.smithi107.stdout:2/532: symlink d0/d9/d1c/d37/laa 0 2024-05-17T21:53:01.016 INFO:tasks.workunit.client.1.smithi138.stdout:0/533: write d3/d10/d38/d70/fa9 [954219,34911] 0 2024-05-17T21:53:01.016 INFO:tasks.workunit.client.0.smithi107.stdout:4/706: write d3/d4a/d47/d60/d3a/d41/d80/d43/f62 [506351,5277] 0 2024-05-17T21:53:01.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:01.054+0000 7f656d94d700 1 -- 172.21.15.107:0/231929601 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f65680fdfd0 msgr2=0x7f65680fe440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:53:01.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:01.054+0000 7f656d94d700 1 --2- 172.21.15.107:0/231929601 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f65680fdfd0 0x7f65680fe440 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f655c004940 tx=0x7f655c009e30 comp rx=0 tx=0).stop 2024-05-17T21:53:01.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:01.054+0000 7f656d94d700 1 -- 172.21.15.107:0/231929601 shutdown_connections 2024-05-17T21:53:01.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:01.054+0000 7f656d94d700 1 --2- 172.21.15.107:0/231929601 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f65680fdfd0 0x7f65680fe440 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:53:01.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:01.054+0000 7f656d94d700 1 --2- 172.21.15.107:0/231929601 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65680fce60 0x7f65680fd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:53:01.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:01.054+0000 7f656d94d700 1 -- 172.21.15.107:0/231929601 >> 172.21.15.107:0/231929601 conn(0x7f65680f8540 msgr2=0x7f65680fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:53:01.047 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:00 smithi107 ceph-mon[28434]: pgmap v38: 65 pgs: 65 active+clean; 1.9 GiB data, 7.0 GiB used, 529 GiB / 536 GiB avail; 65 MiB/s rd, 197 MiB/s wr, 391 op/s 2024-05-17T21:53:01.047 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:00 smithi107 ceph-mon[28434]: from='client.24457 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:53:01.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:01.054+0000 7f656d94d700 1 -- 172.21.15.107:0/231929601 shutdown_connections 2024-05-17T21:53:01.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:01.054+0000 7f656d94d700 1 -- 172.21.15.107:0/231929601 wait complete. 2024-05-17T21:53:01.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:01.055+0000 7f656d94d700 1 Processor -- start 2024-05-17T21:53:01.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:01.055+0000 7f656d94d700 1 -- start start 2024-05-17T21:53:01.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:01.055+0000 7f656d94d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65680fce60 0x7f6568071c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:53:01.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:01.055+0000 7f656d94d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f65680fdfd0 0x7f65680702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:53:01.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:01.055+0000 7f656d94d700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6568072200 con 0x7f65680fdfd0 2024-05-17T21:53:01.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:01.055+0000 7f656d94d700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65680707f0 con 0x7f65680fce60 2024-05-17T21:53:01.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:01.055+0000 7f65667fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f65680fdfd0 0x7f65680702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:53:01.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:01.055+0000 7f6566ffd700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65680fce60 0x7f6568071c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:53:01.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:01.055+0000 7f65667fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f65680fdfd0 0x7f65680702b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:35408/0 (socket says 172.21.15.107:35408) 2024-05-17T21:53:01.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:01.055+0000 7f65667fc700 1 -- 172.21.15.107:0/4022027707 learned_addr learned my addr 172.21.15.107:0/4022027707 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:53:01.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:01.055+0000 7f65667fc700 1 -- 172.21.15.107:0/4022027707 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65680fce60 msgr2=0x7f6568071c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:53:01.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:01.055+0000 7f65667fc700 1 --2- 172.21.15.107:0/4022027707 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65680fce60 0x7f6568071c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:53:01.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:01.055+0000 7f65667fc700 1 -- 172.21.15.107:0/4022027707 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f655c005040 con 0x7f65680fdfd0 2024-05-17T21:53:01.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:01.055+0000 7f65667fc700 1 --2- 172.21.15.107:0/4022027707 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f65680fdfd0 0x7f65680702b0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f655c009150 tx=0x7f655c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:53:01.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:01.056+0000 7f654f7fe700 1 -- 172.21.15.107:0/4022027707 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f655c00e040 con 0x7f65680fdfd0 2024-05-17T21:53:01.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:01.056+0000 7f656d94d700 1 -- 172.21.15.107:0/4022027707 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6568070a40 con 0x7f65680fdfd0 2024-05-17T21:53:01.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:01.056+0000 7f654f7fe700 1 -- 172.21.15.107:0/4022027707 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f655c004290 con 0x7f65680fdfd0 2024-05-17T21:53:01.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:01.056+0000 7f654f7fe700 1 -- 172.21.15.107:0/4022027707 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f655c01c630 con 0x7f65680fdfd0 2024-05-17T21:53:01.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:01.056+0000 7f656d94d700 1 -- 172.21.15.107:0/4022027707 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6568071000 con 0x7f65680fdfd0 2024-05-17T21:53:01.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:01.057+0000 7f656d94d700 1 -- 172.21.15.107:0/4022027707 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f656804c710 con 0x7f65680fdfd0 2024-05-17T21:53:01.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:01.058+0000 7f654f7fe700 1 -- 172.21.15.107:0/4022027707 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) v1 ==== 93316+0+0 (secure 0 0 0) 0x7f655c00bbf0 con 0x7f65680fdfd0 2024-05-17T21:53:01.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:01.058+0000 7f654f7fe700 1 --2- 172.21.15.107:0/4022027707 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7f6550070bc0 0x7f6550073070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:53:01.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:01.058+0000 7f654f7fe700 1 -- 172.21.15.107:0/4022027707 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f655c0915e0 con 0x7f65680fdfd0 2024-05-17T21:53:01.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:01.058+0000 7f6566ffd700 1 --2- 172.21.15.107:0/4022027707 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7f6550070bc0 0x7f6550073070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:53:01.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:01.059+0000 7f6566ffd700 1 --2- 172.21.15.107:0/4022027707 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7f6550070bc0 0x7f6550073070 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6558007fd0 tx=0x7f6558009040 comp rx=0 tx=0).ready entity=mgr.14636 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:53:01.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:01.063+0000 7f654f7fe700 1 -- 172.21.15.107:0/4022027707 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+190121 (secure 0 0 0) 0x7f655c059b30 con 0x7f65680fdfd0 2024-05-17T21:53:01.067 INFO:tasks.workunit.client.1.smithi138.stdout:2/548: sync 2024-05-17T21:53:01.067 INFO:tasks.workunit.client.1.smithi138.stdout:6/531: sync 2024-05-17T21:53:01.067 INFO:tasks.workunit.client.0.smithi107.stdout:8/671: unlink dc/fd2 0 2024-05-17T21:53:01.068 INFO:tasks.workunit.client.0.smithi107.stdout:8/672: readlink dc/d21/d35/dab/lb0 0 2024-05-17T21:53:01.069 INFO:tasks.workunit.client.1.smithi138.stdout:7/431: symlink d0/d18/d40/d41/d51/l87 0 2024-05-17T21:53:01.069 INFO:tasks.workunit.client.0.smithi107.stdout:6/595: link d9/d16/d22/d2d/f31 d9/d16/d4d/db8/fba 0 2024-05-17T21:53:01.070 INFO:tasks.workunit.client.0.smithi107.stdout:6/596: chown d9/d16/d22/d2d/cb4 67 1 2024-05-17T21:53:01.071 INFO:tasks.workunit.client.1.smithi138.stdout:3/617: creat d1/d4f/d50/dbb/fd2 x:0 0 0 2024-05-17T21:53:01.073 INFO:tasks.workunit.client.1.smithi138.stdout:2/549: dwrite d3/d21/f30 [0,4194304] 0 2024-05-17T21:53:01.075 INFO:tasks.workunit.client.0.smithi107.stdout:1/714: mknod d4/df/d14/d2e/ddf/ce7 0 2024-05-17T21:53:01.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:00 smithi138 ceph-mon[35948]: pgmap v38: 65 pgs: 65 active+clean; 1.9 GiB data, 7.0 GiB used, 529 GiB / 536 GiB avail; 65 MiB/s rd, 197 MiB/s wr, 391 op/s 2024-05-17T21:53:01.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:00 smithi138 ceph-mon[35948]: from='client.24457 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:53:01.078 INFO:tasks.workunit.client.0.smithi107.stdout:5/670: mknod d5/db/d16/d2d/d32/cd7 0 2024-05-17T21:53:01.079 INFO:tasks.workunit.client.0.smithi107.stdout:1/715: dread d4/df/d1a/d26/fe6 [0,4194304] 0 2024-05-17T21:53:01.079 INFO:tasks.workunit.client.0.smithi107.stdout:5/671: fdatasync d5/db/d16/d2d/d32/f91 0 2024-05-17T21:53:01.082 INFO:tasks.workunit.client.1.smithi138.stdout:6/532: unlink d7/d8/d78/f81 0 2024-05-17T21:53:01.083 INFO:tasks.workunit.client.1.smithi138.stdout:6/533: write d7/db/d43/d5f/f73 [485337,74849] 0 2024-05-17T21:53:01.085 INFO:tasks.workunit.client.1.smithi138.stdout:7/432: creat d0/d18/d40/d41/d51/d66/f88 x:0 0 0 2024-05-17T21:53:01.086 INFO:tasks.workunit.client.1.smithi138.stdout:7/433: fdatasync d0/d18/f1d 0 2024-05-17T21:53:01.086 INFO:tasks.workunit.client.1.smithi138.stdout:3/618: mknod d1/d62/dad/d5c/cd3 0 2024-05-17T21:53:01.087 INFO:tasks.workunit.client.0.smithi107.stdout:0/765: write d3/dc/d22/d23/d97/d77/d91/fd3 [320768,99819] 0 2024-05-17T21:53:01.088 INFO:tasks.workunit.client.1.smithi138.stdout:2/550: unlink d3/fd 0 2024-05-17T21:53:01.091 INFO:tasks.workunit.client.0.smithi107.stdout:4/707: rmdir d3/d4a/d47/d60/d3c 39 2024-05-17T21:53:01.091 INFO:tasks.workunit.client.0.smithi107.stdout:0/766: dread d3/dc/d22/d3d/dc7/fcb [0,4194304] 0 2024-05-17T21:53:01.093 INFO:tasks.workunit.client.0.smithi107.stdout:6/597: dread d9/d16/d22/d2d/f63 [0,4194304] 0 2024-05-17T21:53:01.100 INFO:tasks.workunit.client.1.smithi138.stdout:6/534: rename d7/db/d43/f59 to d7/da0/fbf 0 2024-05-17T21:53:01.101 INFO:tasks.workunit.client.0.smithi107.stdout:9/702: creat d0/d80/dc9/fed x:0 0 0 2024-05-17T21:53:01.102 INFO:tasks.workunit.client.0.smithi107.stdout:5/672: mknod d5/db/d16/d2d/d51/cd8 0 2024-05-17T21:53:01.103 INFO:tasks.workunit.client.1.smithi138.stdout:7/434: dread d0/d18/d40/f4e [0,4194304] 0 2024-05-17T21:53:01.105 INFO:tasks.workunit.client.1.smithi138.stdout:3/619: mknod d1/d62/dad/de/cd4 0 2024-05-17T21:53:01.105 INFO:tasks.workunit.client.1.smithi138.stdout:3/620: dread - d1/d3/d16/fa2 zero size 2024-05-17T21:53:01.106 INFO:tasks.workunit.client.1.smithi138.stdout:2/551: dread - d3/d21/d2f/d3e/d63/f6e zero size 2024-05-17T21:53:01.109 INFO:tasks.workunit.client.0.smithi107.stdout:0/767: dread d3/dc/d22/d23/d97/d85/fd9 [0,4194304] 0 2024-05-17T21:53:01.109 INFO:tasks.workunit.client.1.smithi138.stdout:0/534: truncate d3/d3d/d48/f5f 1024788 0 2024-05-17T21:53:01.110 INFO:tasks.workunit.client.0.smithi107.stdout:0/768: write d3/f1d [1636028,43113] 0 2024-05-17T21:53:01.110 INFO:tasks.workunit.client.0.smithi107.stdout:8/673: truncate dc/d21/d35/f61 1023756 0 2024-05-17T21:53:01.112 INFO:tasks.workunit.client.0.smithi107.stdout:8/674: write dc/d21/d97/db7/fe8 [96024,64405] 0 2024-05-17T21:53:01.113 INFO:tasks.workunit.client.1.smithi138.stdout:1/503: link d4/d2e/d36/f91 d4/d2e/f9d 0 2024-05-17T21:53:01.115 INFO:tasks.workunit.client.0.smithi107.stdout:6/598: rename d9/d16/d22/d5f/d6e/fa4 to d9/d16/d4d/db8/fbb 0 2024-05-17T21:53:01.120 INFO:tasks.workunit.client.1.smithi138.stdout:2/552: dread d3/fe [0,4194304] 0 2024-05-17T21:53:01.122 INFO:tasks.workunit.client.1.smithi138.stdout:2/553: write d3/d21/d2f/d3e/d55/d99/f8f [217226,93558] 0 2024-05-17T21:53:01.124 INFO:tasks.workunit.client.1.smithi138.stdout:3/621: rmdir d1 39 2024-05-17T21:53:01.125 INFO:tasks.workunit.client.1.smithi138.stdout:7/435: dwrite d0/d39/f80 [0,4194304] 0 2024-05-17T21:53:01.125 INFO:tasks.workunit.client.0.smithi107.stdout:5/673: creat d5/db/d16/d2a/d73/fd9 x:0 0 0 2024-05-17T21:53:01.126 INFO:tasks.workunit.client.1.smithi138.stdout:7/436: fdatasync d0/d18/d40/d41/d51/f69 0 2024-05-17T21:53:01.126 INFO:tasks.workunit.client.0.smithi107.stdout:5/674: stat d5/db/d16/d2d/d40/d5a 0 2024-05-17T21:53:01.127 INFO:tasks.workunit.client.1.smithi138.stdout:7/437: chown d0/f13 46897 1 2024-05-17T21:53:01.129 INFO:tasks.workunit.client.0.smithi107.stdout:2/533: creat d0/d9/d1c/fab x:0 0 0 2024-05-17T21:53:01.130 INFO:tasks.workunit.client.1.smithi138.stdout:0/535: symlink d3/d10/d38/d70/d80/lb5 0 2024-05-17T21:53:01.133 INFO:tasks.workunit.client.1.smithi138.stdout:1/504: unlink d4/df/d3a/d3c/l83 0 2024-05-17T21:53:01.134 INFO:tasks.workunit.client.0.smithi107.stdout:1/716: truncate d4/df/d14/d2e/d43/d47/f94 1972238 0 2024-05-17T21:53:01.138 INFO:tasks.workunit.client.0.smithi107.stdout:0/769: mkdir d3/d75/d9e/d103 0 2024-05-17T21:53:01.139 INFO:tasks.workunit.client.1.smithi138.stdout:2/554: creat d3/d21/d2f/d3e/d69/fb5 x:0 0 0 2024-05-17T21:53:01.150 INFO:tasks.workunit.client.0.smithi107.stdout:9/703: rename d0/dd/d21/d29/d57/dd7 to d0/dd/d69/dee 0 2024-05-17T21:53:01.151 INFO:tasks.workunit.client.0.smithi107.stdout:9/704: chown d0/dd/d21/d29/d3e/d94 3343139 1 2024-05-17T21:53:01.151 INFO:tasks.workunit.client.1.smithi138.stdout:7/438: mknod d0/d18/d40/c89 0 2024-05-17T21:53:01.151 INFO:tasks.workunit.client.1.smithi138.stdout:7/439: chown d0/d18/f45 0 1 2024-05-17T21:53:01.154 INFO:tasks.workunit.client.1.smithi138.stdout:1/505: rmdir d4/dd/d3b 39 2024-05-17T21:53:01.157 INFO:tasks.workunit.client.0.smithi107.stdout:1/717: dread d4/df/d1a/d41/f48 [0,4194304] 0 2024-05-17T21:53:01.158 INFO:tasks.workunit.client.0.smithi107.stdout:9/705: dwrite d0/dd/d21/d29/dd4/fc5 [0,4194304] 0 2024-05-17T21:53:01.159 INFO:tasks.workunit.client.1.smithi138.stdout:0/536: dwrite d3/f8e [0,4194304] 0 2024-05-17T21:53:01.162 INFO:tasks.workunit.client.0.smithi107.stdout:6/599: mkdir d9/d16/d4d/dbc 0 2024-05-17T21:53:01.167 INFO:tasks.workunit.client.1.smithi138.stdout:6/535: dwrite d7/db/d43/d3f/f4a [4194304,4194304] 0 2024-05-17T21:53:01.169 INFO:tasks.workunit.client.1.smithi138.stdout:1/506: rmdir d4/db/d29 39 2024-05-17T21:53:01.178 INFO:tasks.workunit.client.1.smithi138.stdout:0/537: fdatasync d3/f7 0 2024-05-17T21:53:01.179 INFO:tasks.workunit.client.0.smithi107.stdout:2/534: symlink d0/da6/lac 0 2024-05-17T21:53:01.182 INFO:tasks.workunit.client.1.smithi138.stdout:7/440: rename d0/l23 to d0/d18/d40/d41/d50/l8a 0 2024-05-17T21:53:01.184 INFO:tasks.workunit.client.0.smithi107.stdout:2/535: read d0/d9/f60 [1468012,71922] 0 2024-05-17T21:53:01.185 INFO:tasks.workunit.client.1.smithi138.stdout:6/536: symlink d7/d69/lc0 0 2024-05-17T21:53:01.186 INFO:tasks.workunit.client.0.smithi107.stdout:0/770: fdatasync d3/dc/f6d 0 2024-05-17T21:53:01.188 INFO:tasks.workunit.client.1.smithi138.stdout:1/507: write d4/df/f2c [144860,111343] 0 2024-05-17T21:53:01.189 INFO:tasks.workunit.client.0.smithi107.stdout:9/706: creat d0/dd/d69/dee/fef x:0 0 0 2024-05-17T21:53:01.190 INFO:tasks.workunit.client.1.smithi138.stdout:2/555: dread d3/d21/d2f/d3e/d55/d99/d93/f79 [0,4194304] 0 2024-05-17T21:53:01.190 INFO:tasks.workunit.client.1.smithi138.stdout:4/516: sync 2024-05-17T21:53:01.190 INFO:tasks.workunit.client.1.smithi138.stdout:3/622: creat d1/d4f/fd5 x:0 0 0 2024-05-17T21:53:01.191 INFO:tasks.workunit.client.0.smithi107.stdout:2/536: dwrite d0/f7d [0,4194304] 0 2024-05-17T21:53:01.193 INFO:tasks.workunit.client.0.smithi107.stdout:8/675: write f4 [1450334,122039] 0 2024-05-17T21:53:01.195 INFO:tasks.workunit.client.1.smithi138.stdout:0/538: mkdir d3/d10/d6d/db6 0 2024-05-17T21:53:01.198 INFO:tasks.workunit.client.0.smithi107.stdout:5/675: link d5/db/d16/d5b/l7d d5/db/dc/lda 0 2024-05-17T21:53:01.199 INFO:tasks.workunit.client.0.smithi107.stdout:5/676: read - d5/db/d16/d2a/f94 zero size 2024-05-17T21:53:01.202 INFO:tasks.workunit.client.1.smithi138.stdout:1/508: write d4/db/d29/d89/f9a [861764,68744] 0 2024-05-17T21:53:01.204 INFO:tasks.workunit.client.1.smithi138.stdout:2/556: rmdir d3/d21/d2f/d3e/d58 39 2024-05-17T21:53:01.205 INFO:tasks.workunit.client.1.smithi138.stdout:7/441: dread d0/f4a [0,4194304] 0 2024-05-17T21:53:01.205 INFO:tasks.workunit.client.0.smithi107.stdout:0/771: rmdir d3/d58/d61 39 2024-05-17T21:53:01.207 INFO:tasks.workunit.client.1.smithi138.stdout:4/517: creat d0/d1b/d2b/d55/d79/fb7 x:0 0 0 2024-05-17T21:53:01.209 INFO:tasks.workunit.client.1.smithi138.stdout:4/518: chown d0/d3/d1d/d38 531 1 2024-05-17T21:53:01.209 INFO:tasks.workunit.client.1.smithi138.stdout:1/509: dwrite d4/db/f39 [0,4194304] 0 2024-05-17T21:53:01.212 INFO:tasks.workunit.client.1.smithi138.stdout:3/623: creat d1/d3/d16/d3e/d71/fd6 x:0 0 0 2024-05-17T21:53:01.214 INFO:tasks.workunit.client.0.smithi107.stdout:9/707: unlink d0/dd/d69/f6e 0 2024-05-17T21:53:01.216 INFO:tasks.workunit.client.0.smithi107.stdout:2/537: stat d0/d9/d1c/d3e/d4f/f68 0 2024-05-17T21:53:01.218 INFO:tasks.workunit.client.0.smithi107.stdout:1/718: link d4/df/d1a/d26/d72/d85/f92 d4/df/d1a/d26/d72/de1/fe8 0 2024-05-17T21:53:01.220 INFO:tasks.workunit.client.1.smithi138.stdout:3/624: dwrite d1/d3/d16/d3e/d71/fc9 [0,4194304] 0 2024-05-17T21:53:01.221 INFO:tasks.workunit.client.0.smithi107.stdout:8/676: creat dc/d44/d4b/d52/dcb/ff1 x:0 0 0 2024-05-17T21:53:01.221 INFO:tasks.workunit.client.1.smithi138.stdout:6/537: write d7/db/d43/d3f/f41 [763704,99707] 0 2024-05-17T21:53:01.229 INFO:tasks.workunit.client.1.smithi138.stdout:4/519: rename d0/l4 to d0/d3/db/lb8 0 2024-05-17T21:53:01.229 INFO:tasks.workunit.client.1.smithi138.stdout:4/520: stat d0/d3/f40 0 2024-05-17T21:53:01.235 INFO:tasks.workunit.client.1.smithi138.stdout:7/442: creat d0/d18/d40/d41/d51/d81/d84/f8b x:0 0 0 2024-05-17T21:53:01.240 INFO:tasks.workunit.client.0.smithi107.stdout:9/708: creat d0/dd/d21/d29/dca/ff0 x:0 0 0 2024-05-17T21:53:01.241 INFO:tasks.workunit.client.1.smithi138.stdout:4/521: creat d0/d2f/d49/d60/d1f/fb9 x:0 0 0 2024-05-17T21:53:01.242 INFO:tasks.workunit.client.0.smithi107.stdout:2/538: dread d0/d9/f77 [0,4194304] 0 2024-05-17T21:53:01.243 INFO:tasks.workunit.client.0.smithi107.stdout:8/677: mkdir dc/ddb/df2 0 2024-05-17T21:53:01.244 INFO:tasks.workunit.client.1.smithi138.stdout:3/625: mkdir d1/d3/d16/dd7 0 2024-05-17T21:53:01.247 INFO:tasks.workunit.client.1.smithi138.stdout:6/538: mkdir d7/db/d43/d5f/da4/db0/dc1 0 2024-05-17T21:53:01.247 INFO:tasks.workunit.client.1.smithi138.stdout:3/626: read d1/d62/dad/f34 [1022152,9111] 0 2024-05-17T21:53:01.248 INFO:tasks.workunit.client.0.smithi107.stdout:6/600: getdents d9/d16/d22/d2d 0 2024-05-17T21:53:01.248 INFO:tasks.workunit.client.0.smithi107.stdout:1/719: dread d4/df/d1a/d26/d72/de1/fe8 [0,4194304] 0 2024-05-17T21:53:01.248 INFO:tasks.workunit.client.1.smithi138.stdout:4/522: dwrite d0/d3/d17/d21/d78/fa5 [0,4194304] 0 2024-05-17T21:53:01.249 INFO:tasks.workunit.client.1.smithi138.stdout:3/627: fsync d1/d3/f20 0 2024-05-17T21:53:01.250 INFO:tasks.workunit.client.0.smithi107.stdout:6/601: chown d9/d16/d22/f56 126209 1 2024-05-17T21:53:01.251 INFO:tasks.workunit.client.1.smithi138.stdout:4/523: dread - d0/d3/d17/d21/d78/fac zero size 2024-05-17T21:53:01.252 INFO:tasks.workunit.client.0.smithi107.stdout:0/772: rename d3/f35 to d3/dc/f104 0 2024-05-17T21:53:01.253 INFO:tasks.workunit.client.0.smithi107.stdout:9/709: dwrite d0/dd/d2f/d87/fea [0,4194304] 0 2024-05-17T21:53:01.254 INFO:tasks.workunit.client.0.smithi107.stdout:0/773: read - d3/d75/d9e/fea zero size 2024-05-17T21:53:01.255 INFO:tasks.workunit.client.1.smithi138.stdout:7/443: symlink d0/d18/d40/d41/d51/d81/l8c 0 2024-05-17T21:53:01.256 INFO:tasks.workunit.client.1.smithi138.stdout:6/539: dwrite d7/d8/fba [0,4194304] 0 2024-05-17T21:53:01.259 INFO:tasks.workunit.client.1.smithi138.stdout:0/539: link f2 d3/d10/d38/d3e/fb7 0 2024-05-17T21:53:01.261 INFO:tasks.workunit.client.0.smithi107.stdout:5/677: link d5/db/d16/d2d/d32/c35 d5/db/d16/d2d/d40/dc2/cdb 0 2024-05-17T21:53:01.265 INFO:tasks.workunit.client.0.smithi107.stdout:6/602: dwrite d9/d58/d66/f6c [0,4194304] 0 2024-05-17T21:53:01.266 INFO:tasks.workunit.client.1.smithi138.stdout:6/540: dwrite d7/db/d43/d3f/f4a [0,4194304] 0 2024-05-17T21:53:01.269 INFO:tasks.workunit.client.0.smithi107.stdout:2/539: dread d0/d9/d11/df/d5a/d6d/f74 [0,4194304] 0 2024-05-17T21:53:01.271 INFO:tasks.workunit.client.0.smithi107.stdout:1/720: creat d4/d6a/fe9 x:0 0 0 2024-05-17T21:53:01.272 INFO:tasks.workunit.client.1.smithi138.stdout:2/557: truncate d3/d21/d2f/d3e/d58/d77/f86 37463 0 2024-05-17T21:53:01.272 INFO:tasks.workunit.client.0.smithi107.stdout:5/678: read d5/db/d16/d2a/f64 [3053240,110188] 0 2024-05-17T21:53:01.275 INFO:tasks.workunit.client.1.smithi138.stdout:3/628: rename d1/d62/dad/d5c/cd3 to d1/d4d/cd8 0 2024-05-17T21:53:01.279 INFO:tasks.workunit.client.0.smithi107.stdout:6/603: dread d9/d58/d7b/fae [0,4194304] 0 2024-05-17T21:53:01.285 INFO:tasks.workunit.client.1.smithi138.stdout:0/540: symlink d3/d10/d39/lb8 0 2024-05-17T21:53:01.288 INFO:tasks.workunit.client.1.smithi138.stdout:6/541: mkdir d7/d8/dc2 0 2024-05-17T21:53:01.291 INFO:tasks.workunit.client.1.smithi138.stdout:2/558: rename d3/d21/d2f/f50 to d3/d21/d2f/d3e/d63/da3/fb6 0 2024-05-17T21:53:01.293 INFO:tasks.workunit.client.1.smithi138.stdout:4/524: creat d0/d3/d17/d5e/d7f/fba x:0 0 0 2024-05-17T21:53:01.294 INFO:tasks.workunit.client.0.smithi107.stdout:5/679: dread d5/db/d39/f70 [0,4194304] 0 2024-05-17T21:53:01.296 INFO:tasks.workunit.client.1.smithi138.stdout:6/542: dwrite d7/db/f52 [0,4194304] 0 2024-05-17T21:53:01.298 INFO:tasks.workunit.client.0.smithi107.stdout:9/710: link d0/dd/d69/dee/fef d0/dd/d69/ff1 0 2024-05-17T21:53:01.299 INFO:tasks.workunit.client.1.smithi138.stdout:4/525: truncate d0/d1b/d2b/d55/d79/fb7 959268 0 2024-05-17T21:53:01.299 INFO:tasks.workunit.client.1.smithi138.stdout:6/543: stat d7/db/d43/d5f/d6a/c99 0 2024-05-17T21:53:01.299 INFO:tasks.workunit.client.0.smithi107.stdout:6/604: dwrite d9/d16/d22/d2d/fb5 [0,4194304] 0 2024-05-17T21:53:01.300 INFO:tasks.workunit.client.1.smithi138.stdout:3/629: getdents d1/d3/d16/d3e/d71/db2 0 2024-05-17T21:53:01.302 INFO:tasks.workunit.client.0.smithi107.stdout:7/647: sync 2024-05-17T21:53:01.302 INFO:tasks.workunit.client.0.smithi107.stdout:3/713: sync 2024-05-17T21:53:01.302 INFO:tasks.workunit.client.0.smithi107.stdout:3/714: readlink d6/l7d 0 2024-05-17T21:53:01.303 INFO:tasks.workunit.client.1.smithi138.stdout:7/444: dread d0/f24 [0,4194304] 0 2024-05-17T21:53:01.303 INFO:tasks.workunit.client.1.smithi138.stdout:2/559: creat d3/d21/d2f/d8a/fb7 x:0 0 0 2024-05-17T21:53:01.307 INFO:tasks.workunit.client.1.smithi138.stdout:2/560: truncate d3/d21/d2f/d3e/d69/fb5 320343 0 2024-05-17T21:53:01.308 INFO:tasks.workunit.client.1.smithi138.stdout:0/541: mkdir d3/d10/d6d/db6/db9 0 2024-05-17T21:53:01.313 INFO:tasks.workunit.client.1.smithi138.stdout:3/630: write d1/d3/d16/d5d/fb4 [501634,71051] 0 2024-05-17T21:53:01.314 INFO:tasks.workunit.client.0.smithi107.stdout:8/678: getdents dc/d21/d27/d83 0 2024-05-17T21:53:01.316 INFO:tasks.workunit.client.0.smithi107.stdout:2/540: write d0/f36 [3162516,77750] 0 2024-05-17T21:53:01.316 INFO:tasks.workunit.client.0.smithi107.stdout:5/680: dread - d5/db/d16/d54/d59/f7f zero size 2024-05-17T21:53:01.319 INFO:tasks.workunit.client.0.smithi107.stdout:9/711: write d0/dd/f35 [7078284,97679] 0 2024-05-17T21:53:01.320 INFO:tasks.workunit.client.0.smithi107.stdout:6/605: fsync d9/f1a 0 2024-05-17T21:53:01.321 INFO:tasks.workunit.client.0.smithi107.stdout:0/774: truncate d3/dc/d31/f3a 656584 0 2024-05-17T21:53:01.322 INFO:tasks.workunit.client.1.smithi138.stdout:4/526: symlink d0/d3/lbb 0 2024-05-17T21:53:01.326 INFO:tasks.workunit.client.0.smithi107.stdout:3/715: creat d6/d2a/d2d/d48/d4f/fe7 x:0 0 0 2024-05-17T21:53:01.327 INFO:tasks.workunit.client.0.smithi107.stdout:3/716: write d6/f2c [2343139,18260] 0 2024-05-17T21:53:01.327 INFO:tasks.workunit.client.0.smithi107.stdout:1/721: link d4/df/d14/d1d/f2a d4/df/d1a/d41/fea 0 2024-05-17T21:53:01.331 INFO:tasks.workunit.client.0.smithi107.stdout:8/679: symlink dc/dd5/lf3 0 2024-05-17T21:53:01.331 INFO:tasks.workunit.client.1.smithi138.stdout:7/445: truncate d0/d18/d40/d41/d51/f69 3370535 0 2024-05-17T21:53:01.333 INFO:tasks.workunit.client.0.smithi107.stdout:7/648: dread d5/d16/f28 [0,4194304] 0 2024-05-17T21:53:01.333 INFO:tasks.workunit.client.0.smithi107.stdout:5/681: unlink d5/db/d16/d2d/d40/fca 0 2024-05-17T21:53:01.338 INFO:tasks.workunit.client.1.smithi138.stdout:6/544: truncate d7/db/d43/d3f/d79/f9f 2369766 0 2024-05-17T21:53:01.339 INFO:tasks.workunit.client.1.smithi138.stdout:6/545: write d7/d8/fba [3348464,70217] 0 2024-05-17T21:53:01.340 INFO:tasks.workunit.client.0.smithi107.stdout:9/712: dread d0/dd/d21/dcd/fe5 [0,4194304] 0 2024-05-17T21:53:01.343 INFO:tasks.workunit.client.0.smithi107.stdout:0/775: symlink d3/d75/dd0/l105 0 2024-05-17T21:53:01.343 INFO:tasks.workunit.client.0.smithi107.stdout:2/541: dread d0/d9/f60 [0,4194304] 0 2024-05-17T21:53:01.344 INFO:tasks.workunit.client.1.smithi138.stdout:2/561: mknod d3/da0/daa/cb8 0 2024-05-17T21:53:01.348 INFO:tasks.workunit.client.0.smithi107.stdout:1/722: truncate d4/d78/fab 268263 0 2024-05-17T21:53:01.351 INFO:tasks.workunit.client.1.smithi138.stdout:9/482: sync 2024-05-17T21:53:01.351 INFO:tasks.workunit.client.1.smithi138.stdout:5/501: sync 2024-05-17T21:53:01.351 INFO:tasks.workunit.client.1.smithi138.stdout:8/447: sync 2024-05-17T21:53:01.352 INFO:tasks.workunit.client.0.smithi107.stdout:3/717: rename d6/d2a/d2d/d48/d4f/fcf to d6/d8/d10/d35/d36/d4c/dcc/fe8 0 2024-05-17T21:53:01.354 INFO:tasks.workunit.client.0.smithi107.stdout:5/682: mknod d5/db/d16/d54/d59/d92/cdc 0 2024-05-17T21:53:01.358 INFO:tasks.workunit.client.0.smithi107.stdout:3/718: dwrite d6/d8/f14 [0,4194304] 0 2024-05-17T21:53:01.359 INFO:tasks.workunit.client.0.smithi107.stdout:3/719: write d6/d2a/f78 [5176031,75458] 0 2024-05-17T21:53:01.400 INFO:tasks.workunit.client.1.smithi138.stdout:4/527: mknod d0/d2f/d49/d60/cbc 0 2024-05-17T21:53:01.400 INFO:tasks.workunit.client.0.smithi107.stdout:9/713: mknod d0/dd/d69/dee/cf2 0 2024-05-17T21:53:01.401 INFO:tasks.workunit.client.1.smithi138.stdout:6/546: dread d7/db/f82 [0,4194304] 0 2024-05-17T21:53:01.401 INFO:tasks.workunit.client.0.smithi107.stdout:9/714: fsync d0/dd/d21/d29/f8d 0 2024-05-17T21:53:01.402 INFO:tasks.workunit.client.0.smithi107.stdout:0/776: chown d3/dc/d22/d23/d97/d85/fb3 20481760 1 2024-05-17T21:53:01.404 INFO:tasks.workunit.client.1.smithi138.stdout:6/547: readlink d7/db/d43/d5f/d6a/l84 0 2024-05-17T21:53:01.406 INFO:tasks.workunit.client.1.smithi138.stdout:2/562: creat d3/d21/d2f/d3e/d63/da3/fb9 x:0 0 0 2024-05-17T21:53:01.406 INFO:tasks.workunit.client.0.smithi107.stdout:1/723: fdatasync d4/df/d14/d2e/d43/d47/fa3 0 2024-05-17T21:53:01.408 INFO:tasks.workunit.client.1.smithi138.stdout:7/446: dwrite d0/f4a [0,4194304] 0 2024-05-17T21:53:01.410 INFO:tasks.workunit.client.0.smithi107.stdout:0/777: dread d3/d24/d6f/db5/def/ff0 [0,4194304] 0 2024-05-17T21:53:01.411 INFO:tasks.workunit.client.0.smithi107.stdout:0/778: truncate d3/dc/ff4 932552 0 2024-05-17T21:53:01.411 INFO:tasks.workunit.client.1.smithi138.stdout:5/502: creat d1/d6/d19/fa3 x:0 0 0 2024-05-17T21:53:01.414 INFO:tasks.workunit.client.0.smithi107.stdout:8/680: rename dc/d21/d35/dab to dc/d21/d69/df4 0 2024-05-17T21:53:01.416 INFO:tasks.workunit.client.1.smithi138.stdout:4/528: rmdir d0/d2f/d49/d60/d1f 39 2024-05-17T21:53:01.416 INFO:tasks.workunit.client.0.smithi107.stdout:6/606: dwrite d9/d16/d4d/f52 [0,4194304] 0 2024-05-17T21:53:01.416 INFO:tasks.workunit.client.1.smithi138.stdout:4/529: chown d0/d1b/d36 8 1 2024-05-17T21:53:01.426 INFO:tasks.workunit.client.1.smithi138.stdout:8/448: dread d0/dc/f10 [0,4194304] 0 2024-05-17T21:53:01.427 INFO:tasks.workunit.client.1.smithi138.stdout:7/447: creat d0/d18/d40/d5f/f8d x:0 0 0 2024-05-17T21:53:01.428 INFO:tasks.workunit.client.1.smithi138.stdout:7/448: stat d0/d18/f1d 0 2024-05-17T21:53:01.429 INFO:tasks.workunit.client.0.smithi107.stdout:3/720: unlink d6/d8/d10/d35/d91/lbc 0 2024-05-17T21:53:01.431 INFO:tasks.workunit.client.1.smithi138.stdout:5/503: creat d1/d1a/d39/d4e/d58/fa4 x:0 0 0 2024-05-17T21:53:01.433 INFO:tasks.workunit.client.1.smithi138.stdout:4/530: mkdir d0/d3/d1d/d38/d8a/dbd 0 2024-05-17T21:53:01.435 INFO:tasks.workunit.client.1.smithi138.stdout:7/449: dwrite d0/d18/d64/f6e [0,4194304] 0 2024-05-17T21:53:01.436 INFO:tasks.workunit.client.0.smithi107.stdout:5/683: dwrite d5/db/d16/d2d/d32/f38 [0,4194304] 0 2024-05-17T21:53:01.437 INFO:tasks.workunit.client.0.smithi107.stdout:5/684: readlink d5/db/d16/d2d/l37 0 2024-05-17T21:53:01.438 INFO:tasks.workunit.client.0.smithi107.stdout:0/779: rmdir d3/dc/dae/dc0/de1 39 2024-05-17T21:53:01.440 INFO:tasks.workunit.client.0.smithi107.stdout:0/780: dread d3/d24/d6f/db5/fdd [0,4194304] 0 2024-05-17T21:53:01.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:01.454+0000 7f656d94d700 1 -- 172.21.15.107:0/4022027707 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f65681058c0 con 0x7f65680fdfd0 2024-05-17T21:53:01.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:01.454+0000 7f654f7fe700 1 -- 172.21.15.107:0/4022027707 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1840 (secure 0 0 0) 0x7f655c0594d0 con 0x7f65680fdfd0 2024-05-17T21:53:01.447 INFO:teuthology.orchestra.run.smithi107.stdout:e15 2024-05-17T21:53:01.447 INFO:teuthology.orchestra.run.smithi107.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-17T21:53:01.447 INFO:teuthology.orchestra.run.smithi107.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-17T21:53:01.447 INFO:teuthology.orchestra.run.smithi107.stdout:legacy client fscid: 1 2024-05-17T21:53:01.447 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:53:01.447 INFO:teuthology.orchestra.run.smithi107.stdout:Filesystem 'cephfs' (1) 2024-05-17T21:53:01.447 INFO:teuthology.orchestra.run.smithi107.stdout:fs_name cephfs 2024-05-17T21:53:01.447 INFO:teuthology.orchestra.run.smithi107.stdout:epoch 15 2024-05-17T21:53:01.447 INFO:teuthology.orchestra.run.smithi107.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-17T21:53:01.447 INFO:teuthology.orchestra.run.smithi107.stdout:created 2024-05-17T21:49:16.836333+0000 2024-05-17T21:53:01.447 INFO:teuthology.orchestra.run.smithi107.stdout:modified 2024-05-17T21:49:34.873628+0000 2024-05-17T21:53:01.447 INFO:teuthology.orchestra.run.smithi107.stdout:tableserver 0 2024-05-17T21:53:01.448 INFO:teuthology.orchestra.run.smithi107.stdout:root 0 2024-05-17T21:53:01.448 INFO:teuthology.orchestra.run.smithi107.stdout:session_timeout 60 2024-05-17T21:53:01.448 INFO:teuthology.orchestra.run.smithi107.stdout:session_autoclose 300 2024-05-17T21:53:01.448 INFO:teuthology.orchestra.run.smithi107.stdout:max_file_size 1099511627776 2024-05-17T21:53:01.448 INFO:teuthology.orchestra.run.smithi107.stdout:required_client_features {} 2024-05-17T21:53:01.448 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure 0 2024-05-17T21:53:01.448 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure_osd_epoch 46 2024-05-17T21:53:01.448 INFO:teuthology.orchestra.run.smithi107.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-17T21:53:01.448 INFO:teuthology.orchestra.run.smithi107.stdout:max_mds 1 2024-05-17T21:53:01.448 INFO:teuthology.orchestra.run.smithi107.stdout:in 0 2024-05-17T21:53:01.448 INFO:teuthology.orchestra.run.smithi107.stdout:up {0=24259} 2024-05-17T21:53:01.448 INFO:teuthology.orchestra.run.smithi107.stdout:failed 2024-05-17T21:53:01.448 INFO:teuthology.orchestra.run.smithi107.stdout:damaged 2024-05-17T21:53:01.448 INFO:teuthology.orchestra.run.smithi107.stdout:stopped 2024-05-17T21:53:01.449 INFO:teuthology.orchestra.run.smithi107.stdout:data_pools [3] 2024-05-17T21:53:01.449 INFO:teuthology.orchestra.run.smithi107.stdout:metadata_pool 2 2024-05-17T21:53:01.449 INFO:teuthology.orchestra.run.smithi107.stdout:inline_data enabled 2024-05-17T21:53:01.449 INFO:teuthology.orchestra.run.smithi107.stdout:balancer 2024-05-17T21:53:01.449 INFO:teuthology.orchestra.run.smithi107.stdout:standby_count_wanted 1 2024-05-17T21:53:01.449 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi138.gwbtyd{0:24259} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.138:6824/2779634919,v1:172.21.15.138:6825/2779634919] compat {c=[1],r=[1],i=[7ff]}] 2024-05-17T21:53:01.449 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.drqody{0:14486} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.107:6828/2275807100,v1:172.21.15.107:6829/2275807100] compat {c=[1],r=[1],i=[7ff]}] 2024-05-17T21:53:01.449 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:53:01.449 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:53:01.449 INFO:teuthology.orchestra.run.smithi107.stdout:Standby daemons: 2024-05-17T21:53:01.449 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:53:01.449 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.idmbgj{-1:14498} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.107:6826/1460186079,v1:172.21.15.107:6827/1460186079] compat {c=[1],r=[1],i=[7ff]}] 2024-05-17T21:53:01.449 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi138.zckrvs{-1:24267} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.138:6826/170628661,v1:172.21.15.138:6827/170628661] compat {c=[1],r=[1],i=[7ff]}] 2024-05-17T21:53:01.450 INFO:tasks.workunit.client.0.smithi107.stdout:4/708: sync 2024-05-17T21:53:01.450 INFO:tasks.workunit.client.1.smithi138.stdout:6/548: getdents d7/db/d43/d3f 0 2024-05-17T21:53:01.450 INFO:tasks.workunit.client.1.smithi138.stdout:6/549: write d7/d69/f71 [1014577,107896] 0 2024-05-17T21:53:01.450 INFO:tasks.workunit.client.1.smithi138.stdout:6/550: stat d7/d69/f9d 0 2024-05-17T21:53:01.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:01.457+0000 7f656d94d700 1 -- 172.21.15.107:0/4022027707 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7f6550070bc0 msgr2=0x7f6550073070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:53:01.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:01.457+0000 7f656d94d700 1 --2- 172.21.15.107:0/4022027707 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7f6550070bc0 0x7f6550073070 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6558007fd0 tx=0x7f6558009040 comp rx=0 tx=0).stop 2024-05-17T21:53:01.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:01.457+0000 7f656d94d700 1 -- 172.21.15.107:0/4022027707 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f65680fdfd0 msgr2=0x7f65680702b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:53:01.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:01.457+0000 7f656d94d700 1 --2- 172.21.15.107:0/4022027707 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f65680fdfd0 0x7f65680702b0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f655c009150 tx=0x7f655c009e30 comp rx=0 tx=0).stop 2024-05-17T21:53:01.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:01.459+0000 7f656d94d700 1 -- 172.21.15.107:0/4022027707 shutdown_connections 2024-05-17T21:53:01.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:01.459+0000 7f656d94d700 1 --2- 172.21.15.107:0/4022027707 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7f6550070bc0 0x7f6550073070 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:53:01.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:01.459+0000 7f656d94d700 1 --2- 172.21.15.107:0/4022027707 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65680fce60 0x7f6568071c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:53:01.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:01.459+0000 7f656d94d700 1 --2- 172.21.15.107:0/4022027707 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f65680fdfd0 0x7f65680702b0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:53:01.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:01.459+0000 7f656d94d700 1 -- 172.21.15.107:0/4022027707 >> 172.21.15.107:0/4022027707 conn(0x7f65680f8540 msgr2=0x7f65680f9c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:53:01.452 INFO:tasks.workunit.client.0.smithi107.stdout:0/781: chown d3/dc/d22/d23/d97/d77 45 1 2024-05-17T21:53:01.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:01.460+0000 7f656d94d700 1 -- 172.21.15.107:0/4022027707 shutdown_connections 2024-05-17T21:53:01.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:01.460+0000 7f656d94d700 1 -- 172.21.15.107:0/4022027707 wait complete. 2024-05-17T21:53:01.453 INFO:tasks.workunit.client.0.smithi107.stdout:3/721: chown d6/d2a/d2d/d48/d4f/f94 497 1 2024-05-17T21:53:01.453 INFO:tasks.workunit.client.0.smithi107.stdout:9/715: stat d0/dd/d1b/l4b 0 2024-05-17T21:53:01.453 INFO:tasks.workunit.client.0.smithi107.stdout:4/709: write d3/d4a/d47/d60/d3a/d41/fc8 [6813,76901] 0 2024-05-17T21:53:01.454 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 15 2024-05-17T21:53:01.454 INFO:tasks.workunit.client.0.smithi107.stdout:7/649: rename d5/df/l14 to d5/df/d11/d1d/dc4/le5 0 2024-05-17T21:53:01.454 INFO:tasks.workunit.client.0.smithi107.stdout:5/685: creat d5/db/d16/d2d/d40/d5a/d7c/fdd x:0 0 0 2024-05-17T21:53:01.456 INFO:tasks.workunit.client.1.smithi138.stdout:6/551: chown d7/db/d43/f35 13871051 1 2024-05-17T21:53:01.456 INFO:tasks.workunit.client.1.smithi138.stdout:6/552: read - d7/d8/f96 zero size 2024-05-17T21:53:01.457 INFO:tasks.workunit.client.0.smithi107.stdout:9/716: dread d0/dd/d21/daf/fe1 [0,4194304] 0 2024-05-17T21:53:01.457 INFO:tasks.workunit.client.1.smithi138.stdout:8/449: getdents d0/dc/d3b/d4d/d7d 0 2024-05-17T21:53:01.458 INFO:tasks.workunit.client.0.smithi107.stdout:6/607: creat d9/d16/fbd x:0 0 0 2024-05-17T21:53:01.458 INFO:tasks.workunit.client.0.smithi107.stdout:9/717: stat d0/dd/d21/d29/d57/d5f/l86 0 2024-05-17T21:53:01.458 INFO:tasks.workunit.client.1.smithi138.stdout:7/450: creat d0/f8e x:0 0 0 2024-05-17T21:53:01.459 INFO:tasks.workunit.client.0.smithi107.stdout:6/608: write d9/d16/f24 [2812932,25759] 0 2024-05-17T21:53:01.460 INFO:tasks.workunit.client.1.smithi138.stdout:7/451: stat d0/d18/d40/d41/d51/d66/f6f 0 2024-05-17T21:53:01.460 INFO:tasks.workunit.client.1.smithi138.stdout:5/504: getdents d1/d6/d1b 0 2024-05-17T21:53:01.460 INFO:tasks.workunit.client.0.smithi107.stdout:6/609: write d9/f4e [2583459,51396] 0 2024-05-17T21:53:01.462 INFO:tasks.workunit.client.0.smithi107.stdout:0/782: symlink d3/d24/d6f/l106 0 2024-05-17T21:53:01.463 INFO:tasks.workunit.client.1.smithi138.stdout:5/505: symlink d1/d53/la5 0 2024-05-17T21:53:01.464 INFO:tasks.workunit.client.1.smithi138.stdout:5/506: write d1/d1a/f93 [4236057,57487] 0 2024-05-17T21:53:01.466 INFO:tasks.workunit.client.0.smithi107.stdout:9/718: mkdir d0/dd/d21/d29/dca/df3 0 2024-05-17T21:53:01.466 INFO:tasks.workunit.client.1.smithi138.stdout:5/507: write d1/d6/d1b/f1d [4668651,66463] 0 2024-05-17T21:53:01.467 INFO:tasks.workunit.client.0.smithi107.stdout:9/719: chown d0/dd/d2f/d87/d89/de0/ld6 14 1 2024-05-17T21:53:01.468 INFO:tasks.workunit.client.1.smithi138.stdout:7/452: getdents d0/d18/d64/d7c 0 2024-05-17T21:53:01.469 INFO:tasks.workunit.client.0.smithi107.stdout:1/724: dread f3 [0,4194304] 0 2024-05-17T21:53:01.469 INFO:tasks.workunit.client.0.smithi107.stdout:4/710: creat d3/db/d1e/dc6/fea x:0 0 0 2024-05-17T21:53:01.471 INFO:tasks.workunit.client.0.smithi107.stdout:7/650: dread d5/df/d11/d1d/d33/f6a [0,4194304] 0 2024-05-17T21:53:01.471 INFO:tasks.workunit.client.1.smithi138.stdout:5/508: rename d1/d1a/f93 to d1/d1a/d39/fa6 0 2024-05-17T21:53:01.473 INFO:tasks.workunit.client.0.smithi107.stdout:3/722: rename d6/d1a/db1/ce3 to d6/d8/d10/ce9 0 2024-05-17T21:53:01.473 INFO:tasks.workunit.client.0.smithi107.stdout:3/723: write d6/d8/d10/d25/f99 [3141851,24441] 0 2024-05-17T21:53:01.474 INFO:tasks.workunit.client.1.smithi138.stdout:7/453: chown d0/f11 12446672 1 2024-05-17T21:53:01.477 INFO:tasks.workunit.client.0.smithi107.stdout:0/783: mknod d3/d39/c107 0 2024-05-17T21:53:01.477 INFO:tasks.workunit.client.1.smithi138.stdout:4/531: dwrite d0/d2f/d49/d60/d1f/f7a [0,4194304] 0 2024-05-17T21:53:01.478 INFO:tasks.workunit.client.0.smithi107.stdout:0/784: fdatasync d3/d24/d6f/d9f/dab/fb4 0 2024-05-17T21:53:01.479 INFO:tasks.workunit.client.0.smithi107.stdout:3/724: dwrite d6/d8/f12 [0,4194304] 0 2024-05-17T21:53:01.479 INFO:tasks.workunit.client.0.smithi107.stdout:5/686: mknod d5/d75/dc7/dd4/cde 0 2024-05-17T21:53:01.483 INFO:tasks.workunit.client.0.smithi107.stdout:9/720: symlink d0/dd/d69/lf4 0 2024-05-17T21:53:01.485 INFO:tasks.workunit.client.1.smithi138.stdout:4/532: write d0/d3/d17/d21/f75 [1008383,7908] 0 2024-05-17T21:53:01.486 INFO:tasks.workunit.client.1.smithi138.stdout:4/533: chown d0/d2f/d49 54204394 1 2024-05-17T21:53:01.487 INFO:tasks.workunit.client.0.smithi107.stdout:6/610: symlink d9/d16/d9e/lbe 0 2024-05-17T21:53:01.488 INFO:tasks.workunit.client.0.smithi107.stdout:6/611: write d9/d16/d22/d5f/fac [601799,84065] 0 2024-05-17T21:53:01.491 INFO:tasks.workunit.client.1.smithi138.stdout:5/509: creat d1/d1a/fa7 x:0 0 0 2024-05-17T21:53:01.492 INFO:tasks.workunit.client.0.smithi107.stdout:1/725: mkdir d4/df/d1a/d26/d72/daa/deb 0 2024-05-17T21:53:01.494 INFO:tasks.workunit.client.1.smithi138.stdout:4/534: rename d0/d2f/d49/d60/d1f/d32/d71/c80 to d0/d2f/d49/d60/d1f/da1/cbe 0 2024-05-17T21:53:01.494 INFO:tasks.workunit.client.0.smithi107.stdout:7/651: readlink d5/l9 0 2024-05-17T21:53:01.495 INFO:tasks.workunit.client.1.smithi138.stdout:4/535: fdatasync d0/d2f/d49/d60/d1f/d7e/f93 0 2024-05-17T21:53:01.502 INFO:tasks.workunit.client.0.smithi107.stdout:0/785: mknod d3/d24/d6f/db5/def/c108 0 2024-05-17T21:53:01.503 INFO:tasks.workunit.client.0.smithi107.stdout:0/786: write d3/d75/d9e/fea [948590,86224] 0 2024-05-17T21:53:01.508 INFO:tasks.workunit.client.1.smithi138.stdout:7/454: dwrite d0/f2a [0,4194304] 0 2024-05-17T21:53:01.510 INFO:tasks.workunit.client.1.smithi138.stdout:7/455: write d0/d18/d40/d5f/f8d [247852,83326] 0 2024-05-17T21:53:01.511 INFO:tasks.workunit.client.0.smithi107.stdout:4/711: dwrite d3/d4a/f3f [0,4194304] 0 2024-05-17T21:53:01.512 INFO:tasks.workunit.client.0.smithi107.stdout:4/712: chown d3/db/d1e/c38 352476 1 2024-05-17T21:53:01.514 INFO:tasks.workunit.client.1.smithi138.stdout:7/456: creat d0/d18/d40/d5f/f8f x:0 0 0 2024-05-17T21:53:01.518 INFO:tasks.workunit.client.0.smithi107.stdout:6/612: mknod d9/d16/d4d/cbf 0 2024-05-17T21:53:01.519 INFO:tasks.workunit.client.1.smithi138.stdout:4/536: write d0/d2f/f6f [425771,75141] 0 2024-05-17T21:53:01.519 INFO:tasks.workunit.client.0.smithi107.stdout:5/687: dread d5/f8 [0,4194304] 0 2024-05-17T21:53:01.520 INFO:tasks.workunit.client.1.smithi138.stdout:5/510: dwrite d1/d6/f41 [4194304,4194304] 0 2024-05-17T21:53:01.521 INFO:tasks.workunit.client.0.smithi107.stdout:9/721: write d0/dd/d69/f83 [878815,52766] 0 2024-05-17T21:53:01.521 INFO:tasks.workunit.client.0.smithi107.stdout:7/652: dread d5/d16/f23 [0,4194304] 0 2024-05-17T21:53:01.522 INFO:tasks.workunit.client.0.smithi107.stdout:9/722: chown d0/d80/dad 0 1 2024-05-17T21:53:01.523 INFO:tasks.workunit.client.0.smithi107.stdout:6/613: read d9/d16/d22/daf/fb9 [59593,53074] 0 2024-05-17T21:53:01.523 INFO:tasks.workunit.client.1.smithi138.stdout:7/457: symlink d0/d18/d40/d5f/l90 0 2024-05-17T21:53:01.523 INFO:tasks.workunit.client.0.smithi107.stdout:5/688: dread - d5/db/d16/d54/d59/d65/d48/d89/fa8 zero size 2024-05-17T21:53:01.525 INFO:tasks.workunit.client.0.smithi107.stdout:1/726: write d4/df/f4d [60047,68659] 0 2024-05-17T21:53:01.528 INFO:tasks.workunit.client.0.smithi107.stdout:0/787: write d3/d24/d6f/d9f/dab/f89 [2819051,80158] 0 2024-05-17T21:53:01.528 INFO:tasks.workunit.client.0.smithi107.stdout:3/725: rename d6/d2a/d2d/c44 to d6/d8/cea 0 2024-05-17T21:53:01.530 INFO:tasks.workunit.client.1.smithi138.stdout:5/511: dread - d1/d2c/d63/f6d zero size 2024-05-17T21:53:01.531 INFO:tasks.workunit.client.1.smithi138.stdout:5/512: chown d1/d6/d1b/l1f 417589688 1 2024-05-17T21:53:01.533 INFO:tasks.workunit.client.0.smithi107.stdout:6/614: stat d9/d16/d4d/l95 0 2024-05-17T21:53:01.535 INFO:tasks.workunit.client.1.smithi138.stdout:7/458: link d0/d18/d40/f6c d0/f91 0 2024-05-17T21:53:01.537 INFO:tasks.workunit.client.0.smithi107.stdout:0/788: chown d3/dc/c1a 1 1 2024-05-17T21:53:01.538 INFO:tasks.workunit.client.0.smithi107.stdout:7/653: mkdir d5/d16/d3b/d45/d91/dc1/de6 0 2024-05-17T21:53:01.539 INFO:tasks.workunit.client.0.smithi107.stdout:7/654: stat d5/df/d11/d1c/d2f/d65/da7 0 2024-05-17T21:53:01.539 INFO:tasks.workunit.client.1.smithi138.stdout:7/459: creat d0/d18/f92 x:0 0 0 2024-05-17T21:53:01.540 INFO:tasks.workunit.client.0.smithi107.stdout:9/723: truncate d0/dd/d1b/f53 242438 0 2024-05-17T21:53:01.541 INFO:tasks.workunit.client.1.smithi138.stdout:5/513: rename d1/d2c/c2e to d1/d2c/ca8 0 2024-05-17T21:53:01.542 INFO:tasks.workunit.client.1.smithi138.stdout:5/514: chown d1/d53/d96/f9b 0 1 2024-05-17T21:53:01.542 INFO:tasks.workunit.client.1.smithi138.stdout:7/460: mknod d0/d4c/c93 0 2024-05-17T21:53:01.548 INFO:tasks.workunit.client.0.smithi107.stdout:9/724: truncate d0/dd/d21/daf/fe1 1030961 0 2024-05-17T21:53:01.548 INFO:tasks.workunit.client.0.smithi107.stdout:9/725: chown d0/dd/d2f/d87/cdf 31870735 1 2024-05-17T21:53:01.549 INFO:tasks.workunit.client.1.smithi138.stdout:4/537: dread d0/f7 [0,4194304] 0 2024-05-17T21:53:01.549 INFO:tasks.workunit.client.1.smithi138.stdout:4/538: fdatasync d0/d3/db/da0/fa8 0 2024-05-17T21:53:01.551 INFO:tasks.workunit.client.1.smithi138.stdout:5/515: getdents d1/d53 0 2024-05-17T21:53:01.552 INFO:tasks.workunit.client.0.smithi107.stdout:5/689: link d5/db/d16/d2d/d40/d5a/ccc d5/db/d16/d2d/d51/cdf 0 2024-05-17T21:53:01.556 INFO:tasks.workunit.client.1.smithi138.stdout:1/510: sync 2024-05-17T21:53:01.557 INFO:tasks.workunit.client.0.smithi107.stdout:3/726: dread d6/d2a/d2d/f43 [0,4194304] 0 2024-05-17T21:53:01.559 INFO:tasks.workunit.client.0.smithi107.stdout:6/615: symlink d9/d16/d4d/d85/d96/lc0 0 2024-05-17T21:53:01.561 INFO:tasks.workunit.client.1.smithi138.stdout:5/516: mknod d1/d6/ca9 0 2024-05-17T21:53:01.562 INFO:tasks.workunit.client.0.smithi107.stdout:1/727: getdents d4/df/d1a 0 2024-05-17T21:53:01.568 INFO:tasks.workunit.client.0.smithi107.stdout:0/789: dwrite d3/f9 [0,4194304] 0 2024-05-17T21:53:01.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:01.597+0000 7fef480ce700 1 -- 172.21.15.107:0/102699663 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef400fcf60 msgr2=0x7fef400fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:53:01.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:01.597+0000 7fef480ce700 1 --2- 172.21.15.107:0/102699663 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef400fcf60 0x7fef400fd3d0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fef30004940 tx=0x7fef30009e30 comp rx=0 tx=0).stop 2024-05-17T21:53:01.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:01.597+0000 7fef480ce700 1 -- 172.21.15.107:0/102699663 shutdown_connections 2024-05-17T21:53:01.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:01.598+0000 7fef480ce700 1 --2- 172.21.15.107:0/102699663 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef400fcf60 0x7fef400fd3d0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:53:01.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:01.598+0000 7fef480ce700 1 --2- 172.21.15.107:0/102699663 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef400fbcb0 0x7fef400fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:53:01.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:01.598+0000 7fef480ce700 1 -- 172.21.15.107:0/102699663 >> 172.21.15.107:0/102699663 conn(0x7fef400f7430 msgr2=0x7fef400f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:53:01.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:01.598+0000 7fef480ce700 1 -- 172.21.15.107:0/102699663 shutdown_connections 2024-05-17T21:53:01.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:01.598+0000 7fef480ce700 1 -- 172.21.15.107:0/102699663 wait complete. 2024-05-17T21:53:01.591 INFO:tasks.workunit.client.1.smithi138.stdout:4/539: link d0/d3/d17/d5e/f8e d0/d3/d17/d5e/fbf 0 2024-05-17T21:53:01.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:01.598+0000 7fef480ce700 1 Processor -- start 2024-05-17T21:53:01.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:01.598+0000 7fef480ce700 1 -- start start 2024-05-17T21:53:01.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:01.598+0000 7fef480ce700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef400fbcb0 0x7fef4010da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:53:01.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:01.598+0000 7fef480ce700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef400fcf60 0x7fef4010df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:53:01.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:01.598+0000 7fef480ce700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef4010e550 con 0x7fef400fcf60 2024-05-17T21:53:01.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:01.598+0000 7fef480ce700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef4010ed50 con 0x7fef400fbcb0 2024-05-17T21:53:01.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:01.598+0000 7fef45669700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef400fcf60 0x7fef4010df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:53:01.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:01.598+0000 7fef45669700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef400fcf60 0x7fef4010df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:35432/0 (socket says 172.21.15.107:35432) 2024-05-17T21:53:01.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:01.598+0000 7fef45669700 1 -- 172.21.15.107:0/418396411 learned_addr learned my addr 172.21.15.107:0/418396411 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:53:01.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:01.598+0000 7fef45e6a700 1 --2- 172.21.15.107:0/418396411 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef400fbcb0 0x7fef4010da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:53:01.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:01.599+0000 7fef45669700 1 -- 172.21.15.107:0/418396411 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef400fbcb0 msgr2=0x7fef4010da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:53:01.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:01.599+0000 7fef45669700 1 --2- 172.21.15.107:0/418396411 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef400fbcb0 0x7fef4010da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:53:01.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:01.599+0000 7fef45669700 1 -- 172.21.15.107:0/418396411 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef30005040 con 0x7fef400fcf60 2024-05-17T21:53:01.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:01.599+0000 7fef45669700 1 --2- 172.21.15.107:0/418396411 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef400fcf60 0x7fef4010df50 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fef30013870 tx=0x7fef30009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:53:01.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:01.599+0000 7fef367fc700 1 -- 172.21.15.107:0/418396411 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fef30021070 con 0x7fef400fcf60 2024-05-17T21:53:01.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:01.599+0000 7fef480ce700 1 -- 172.21.15.107:0/418396411 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef4010efd0 con 0x7fef400fcf60 2024-05-17T21:53:01.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:01.599+0000 7fef367fc700 1 -- 172.21.15.107:0/418396411 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fef30009430 con 0x7fef400fcf60 2024-05-17T21:53:01.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:01.599+0000 7fef367fc700 1 -- 172.21.15.107:0/418396411 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fef3000e070 con 0x7fef400fcf60 2024-05-17T21:53:01.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:01.599+0000 7fef480ce700 1 -- 172.21.15.107:0/418396411 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef4010f510 con 0x7fef400fcf60 2024-05-17T21:53:01.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:01.600+0000 7fef480ce700 1 -- 172.21.15.107:0/418396411 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef40101af0 con 0x7fef400fcf60 2024-05-17T21:53:01.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:01.601+0000 7fef367fc700 1 -- 172.21.15.107:0/418396411 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) v1 ==== 93316+0+0 (secure 0 0 0) 0x7fef3001c750 con 0x7fef400fcf60 2024-05-17T21:53:01.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:01.602+0000 7fef367fc700 1 --2- 172.21.15.107:0/418396411 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7fef2c070b50 0x7fef2c073000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:53:01.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:01.602+0000 7fef367fc700 1 -- 172.21.15.107:0/418396411 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7fef30091c10 con 0x7fef400fcf60 2024-05-17T21:53:01.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:01.602+0000 7fef45e6a700 1 --2- 172.21.15.107:0/418396411 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7fef2c070b50 0x7fef2c073000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:53:01.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:01.602+0000 7fef45e6a700 1 --2- 172.21.15.107:0/418396411 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7fef2c070b50 0x7fef2c073000 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fef3c009b80 tx=0x7fef3c00b040 comp rx=0 tx=0).ready entity=mgr.14636 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:53:01.595 INFO:tasks.workunit.client.1.smithi138.stdout:1/511: creat d4/dd/d3b/f9e x:0 0 0 2024-05-17T21:53:01.597 INFO:tasks.workunit.client.1.smithi138.stdout:7/461: write d0/d18/d40/f4e [4458055,95029] 0 2024-05-17T21:53:01.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:01.606+0000 7fef367fc700 1 -- 172.21.15.107:0/418396411 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+190121 (secure 0 0 0) 0x7fef3005a160 con 0x7fef400fcf60 2024-05-17T21:53:01.600 INFO:tasks.workunit.client.1.smithi138.stdout:4/540: rename d0/d1b/d2b/f9d to d0/d3/d1d/d38/d8a/dbd/fc0 0 2024-05-17T21:53:01.600 INFO:tasks.workunit.client.1.smithi138.stdout:4/541: dread - d0/d2f/d49/d60/d1f/f53 zero size 2024-05-17T21:53:01.601 INFO:tasks.workunit.client.0.smithi107.stdout:1/728: chown d4/df/d1a/c3f 908 1 2024-05-17T21:53:01.601 INFO:tasks.workunit.client.1.smithi138.stdout:5/517: fsync d1/d2c/d63/f9d 0 2024-05-17T21:53:01.602 INFO:tasks.workunit.client.1.smithi138.stdout:1/512: mkdir d4/dd/d3b/d9f 0 2024-05-17T21:53:01.604 INFO:tasks.workunit.client.0.smithi107.stdout:3/727: dwrite d6/d8/d10/d35/d36/f8b [0,4194304] 0 2024-05-17T21:53:01.604 INFO:tasks.workunit.client.0.smithi107.stdout:3/728: stat d6/l7d 0 2024-05-17T21:53:01.605 INFO:tasks.workunit.client.1.smithi138.stdout:7/462: chown d0/d18/d40/d41/d50/l8a 0 1 2024-05-17T21:53:01.605 INFO:tasks.workunit.client.1.smithi138.stdout:4/542: dread d0/f34 [0,4194304] 0 2024-05-17T21:53:01.606 INFO:tasks.workunit.client.0.smithi107.stdout:0/790: truncate f1 1550601 0 2024-05-17T21:53:01.607 INFO:tasks.workunit.client.0.smithi107.stdout:7/655: getdents d5/df/d11/d1c/d31/d39/d92/d9a 0 2024-05-17T21:53:01.608 INFO:tasks.workunit.client.1.smithi138.stdout:1/513: dread d4/db/d29/d89/f9a [0,4194304] 0 2024-05-17T21:53:01.616 INFO:tasks.workunit.client.1.smithi138.stdout:4/543: chown d0/d3/d17/c66 14 1 2024-05-17T21:53:01.619 INFO:tasks.workunit.client.0.smithi107.stdout:0/791: creat d3/d39/f109 x:0 0 0 2024-05-17T21:53:01.620 INFO:tasks.workunit.client.1.smithi138.stdout:5/518: symlink d1/d2c/laa 0 2024-05-17T21:53:01.621 INFO:tasks.workunit.client.1.smithi138.stdout:5/519: fdatasync d1/d1a/fa7 0 2024-05-17T21:53:01.621 INFO:tasks.workunit.client.0.smithi107.stdout:6/616: rmdir d9/d16/d4d/dbc 0 2024-05-17T21:53:01.622 INFO:tasks.workunit.client.1.smithi138.stdout:5/520: chown d1/d1a/d39/d4e/d58/d5b/d65/c86 779 1 2024-05-17T21:53:01.623 INFO:tasks.workunit.client.0.smithi107.stdout:9/726: getdents d0/dd/d21/d29/d57/d5f 0 2024-05-17T21:53:01.623 INFO:tasks.workunit.client.1.smithi138.stdout:5/521: write d1/d1a/d39/d4e/d58/d5b/d65/f8f [2561609,24973] 0 2024-05-17T21:53:01.624 INFO:tasks.workunit.client.1.smithi138.stdout:4/544: symlink d0/d2f/d49/d60/d1f/lc1 0 2024-05-17T21:53:01.625 INFO:tasks.workunit.client.0.smithi107.stdout:1/729: dwrite d4/df/d1a/d26/d72/d85/fc0 [0,4194304] 0 2024-05-17T21:53:01.625 INFO:tasks.workunit.client.1.smithi138.stdout:1/514: mkdir d4/d2e/da0 0 2024-05-17T21:53:01.627 INFO:tasks.workunit.client.1.smithi138.stdout:1/515: truncate d4/df/f97 767513 0 2024-05-17T21:53:01.627 INFO:tasks.workunit.client.0.smithi107.stdout:7/656: creat d5/d16/d3b/d45/d91/dac/fe7 x:0 0 0 2024-05-17T21:53:01.629 INFO:tasks.workunit.client.1.smithi138.stdout:7/463: link d0/f31 d0/d18/d40/d41/d50/f94 0 2024-05-17T21:53:01.632 INFO:tasks.workunit.client.0.smithi107.stdout:6/617: dread d9/d16/f4a [0,4194304] 0 2024-05-17T21:53:01.632 INFO:tasks.workunit.client.1.smithi138.stdout:5/522: creat d1/d6/d19/fab x:0 0 0 2024-05-17T21:53:01.633 INFO:tasks.workunit.client.0.smithi107.stdout:6/618: write d9/f1a [98709,43042] 0 2024-05-17T21:53:01.635 INFO:tasks.workunit.client.1.smithi138.stdout:1/516: mknod d4/db/d60/ca1 0 2024-05-17T21:53:01.636 INFO:tasks.workunit.client.0.smithi107.stdout:0/792: dread d3/dc/d22/d23/d97/f96 [0,4194304] 0 2024-05-17T21:53:01.637 INFO:tasks.workunit.client.1.smithi138.stdout:7/464: dwrite d0/f4d [0,4194304] 0 2024-05-17T21:53:01.638 INFO:tasks.workunit.client.0.smithi107.stdout:9/727: unlink d0/dd/d3d/ce8 0 2024-05-17T21:53:01.639 INFO:tasks.workunit.client.0.smithi107.stdout:6/619: rmdir d9/d16/d4d/d85/d88 39 2024-05-17T21:53:01.640 INFO:tasks.workunit.client.1.smithi138.stdout:5/523: creat d1/d1a/d39/d62/fac x:0 0 0 2024-05-17T21:53:01.644 INFO:tasks.workunit.client.1.smithi138.stdout:4/545: rmdir d0/d2f/d49/d60/d1f/d32/d71/d95/da6 0 2024-05-17T21:53:01.644 INFO:tasks.workunit.client.1.smithi138.stdout:1/517: dwrite d4/df/f2c [0,4194304] 0 2024-05-17T21:53:01.645 INFO:tasks.workunit.client.1.smithi138.stdout:5/524: truncate d1/d1a/d39/d62/fac 794530 0 2024-05-17T21:53:01.646 INFO:tasks.workunit.client.1.smithi138.stdout:4/546: stat d0/d2f/d49/d60/d1f/d32/fb2 0 2024-05-17T21:53:01.646 INFO:tasks.workunit.client.1.smithi138.stdout:1/518: write d4/dd/d3b/f9e [340437,126653] 0 2024-05-17T21:53:01.647 INFO:tasks.workunit.client.1.smithi138.stdout:5/525: write d1/d6/d1b/d29/d36/f4b [464762,52738] 0 2024-05-17T21:53:01.648 INFO:tasks.workunit.client.0.smithi107.stdout:7/657: creat d5/df/d11/d1c/d31/d39/dc0/fe8 x:0 0 0 2024-05-17T21:53:01.650 INFO:tasks.workunit.client.0.smithi107.stdout:9/728: unlink d0/dd/d2f/d81/f84 0 2024-05-17T21:53:01.651 INFO:tasks.workunit.client.0.smithi107.stdout:6/620: fsync d9/f1f 0 2024-05-17T21:53:01.652 INFO:tasks.workunit.client.1.smithi138.stdout:4/547: dread d0/d2f/d49/d60/d1f/f73 [0,4194304] 0 2024-05-17T21:53:01.652 INFO:tasks.workunit.client.1.smithi138.stdout:1/519: read d4/df/d3a/d3c/f88 [3518064,90761] 0 2024-05-17T21:53:01.653 INFO:tasks.workunit.client.0.smithi107.stdout:1/730: link d4/df/d14/c82 d4/df/d1a/d26/d72/daa/deb/cec 0 2024-05-17T21:53:01.656 INFO:tasks.workunit.client.1.smithi138.stdout:5/526: mkdir d1/d6/d1b/d29/d36/d68/dad 0 2024-05-17T21:53:01.656 INFO:tasks.workunit.client.0.smithi107.stdout:7/658: chown d5/ca 225 1 2024-05-17T21:53:01.657 INFO:tasks.workunit.client.0.smithi107.stdout:9/729: fdatasync d0/fc 0 2024-05-17T21:53:01.659 INFO:tasks.workunit.client.0.smithi107.stdout:1/731: rmdir d4/df/d1a/d26/d50/d5c 39 2024-05-17T21:53:01.660 INFO:tasks.workunit.client.1.smithi138.stdout:1/520: read d4/df/d35/f37 [4079554,14679] 0 2024-05-17T21:53:01.664 INFO:tasks.workunit.client.1.smithi138.stdout:1/521: read d4/db/f4b [4002515,65421] 0 2024-05-17T21:53:01.665 INFO:tasks.workunit.client.0.smithi107.stdout:7/659: creat d5/df/d11/d1c/d31/d39/dc0/fe9 x:0 0 0 2024-05-17T21:53:01.666 INFO:tasks.workunit.client.0.smithi107.stdout:1/732: dwrite d4/df/d1a/d26/d72/fb4 [0,4194304] 0 2024-05-17T21:53:01.666 INFO:tasks.workunit.client.0.smithi107.stdout:6/621: getdents d9/d58/d7b/da3 0 2024-05-17T21:53:01.667 INFO:tasks.workunit.client.0.smithi107.stdout:0/793: dread d3/dc/d22/ffc [0,4194304] 0 2024-05-17T21:53:01.668 INFO:tasks.workunit.client.0.smithi107.stdout:0/794: stat d3/d39/fd6 0 2024-05-17T21:53:01.669 INFO:tasks.workunit.client.0.smithi107.stdout:0/795: fdatasync d3/d24/d6f/d9f/de8/ff1 0 2024-05-17T21:53:01.671 INFO:tasks.workunit.client.0.smithi107.stdout:6/622: dread d9/d16/f5e [0,4194304] 0 2024-05-17T21:53:01.671 INFO:tasks.workunit.client.1.smithi138.stdout:4/548: creat d0/d2f/d49/fc2 x:0 0 0 2024-05-17T21:53:01.673 INFO:tasks.workunit.client.0.smithi107.stdout:7/660: creat d5/d16/d3b/d45/d91/dc1/fea x:0 0 0 2024-05-17T21:53:01.683 INFO:tasks.workunit.client.0.smithi107.stdout:6/623: mknod d9/d16/d9e/cc1 0 2024-05-17T21:53:01.686 INFO:tasks.workunit.client.0.smithi107.stdout:0/796: mkdir d3/d75/d9e/d103/d10a 0 2024-05-17T21:53:01.688 INFO:tasks.workunit.client.0.smithi107.stdout:7/661: creat d5/d16/d3b/d45/d91/dc1/de6/feb x:0 0 0 2024-05-17T21:53:01.689 INFO:tasks.workunit.client.1.smithi138.stdout:7/465: dwrite d0/d18/d40/d41/d50/f61 [0,4194304] 0 2024-05-17T21:53:01.690 INFO:tasks.workunit.client.1.smithi138.stdout:7/466: chown d0/f20 9692338 1 2024-05-17T21:53:01.690 INFO:tasks.workunit.client.0.smithi107.stdout:6/624: symlink d9/d16/d4d/d85/lc2 0 2024-05-17T21:53:01.690 INFO:tasks.workunit.client.0.smithi107.stdout:0/797: rmdir d3/dc/d22/d23/d97/d77 39 2024-05-17T21:53:01.691 INFO:tasks.workunit.client.1.smithi138.stdout:7/467: truncate d0/d18/f92 648856 0 2024-05-17T21:53:01.693 INFO:tasks.workunit.client.1.smithi138.stdout:5/527: write d1/d6/d19/f20 [44819,95657] 0 2024-05-17T21:53:01.693 INFO:tasks.workunit.client.0.smithi107.stdout:0/798: read d3/f1e [36088,78176] 0 2024-05-17T21:53:01.695 INFO:tasks.workunit.client.0.smithi107.stdout:1/733: dread d4/df/d4e/d59/fe3 [4194304,4194304] 0 2024-05-17T21:53:01.697 INFO:tasks.workunit.client.1.smithi138.stdout:5/528: creat d1/d46/d95/fae x:0 0 0 2024-05-17T21:53:01.699 INFO:tasks.workunit.client.0.smithi107.stdout:6/625: dread d9/d16/d22/d2d/f5c [0,4194304] 0 2024-05-17T21:53:01.701 INFO:tasks.workunit.client.0.smithi107.stdout:7/662: creat d5/df/d11/d1c/d2f/fec x:0 0 0 2024-05-17T21:53:01.701 INFO:tasks.workunit.client.1.smithi138.stdout:4/549: write d0/d3/d17/d5e/f8e [726852,68394] 0 2024-05-17T21:53:01.702 INFO:tasks.workunit.client.1.smithi138.stdout:1/522: dwrite d4/df/d3a/d3c/d67/f85 [4194304,4194304] 0 2024-05-17T21:53:01.703 INFO:tasks.workunit.client.1.smithi138.stdout:5/529: creat d1/d60/faf x:0 0 0 2024-05-17T21:53:01.704 INFO:tasks.workunit.client.1.smithi138.stdout:4/550: fdatasync d0/d3/d17/d21/fa7 0 2024-05-17T21:53:01.704 INFO:tasks.workunit.client.1.smithi138.stdout:1/523: dread - d4/db/d29/d5d/f6e zero size 2024-05-17T21:53:01.705 INFO:tasks.workunit.client.0.smithi107.stdout:0/799: rmdir d3/d39 39 2024-05-17T21:53:01.706 INFO:tasks.workunit.client.0.smithi107.stdout:0/800: fdatasync d3/fa1 0 2024-05-17T21:53:01.707 INFO:tasks.workunit.client.0.smithi107.stdout:7/663: dwrite d5/df/d11/d1c/d31/d39/f5b [0,4194304] 0 2024-05-17T21:53:01.707 INFO:tasks.workunit.client.0.smithi107.stdout:6/626: read - d9/d16/d22/f72 zero size 2024-05-17T21:53:01.709 INFO:tasks.workunit.client.1.smithi138.stdout:4/551: symlink d0/d3/d1d/d38/d9c/lc3 0 2024-05-17T21:53:01.710 INFO:tasks.workunit.client.1.smithi138.stdout:1/524: symlink d4/la2 0 2024-05-17T21:53:01.713 INFO:tasks.workunit.client.1.smithi138.stdout:5/530: dwrite d1/d6/d1b/d29/d36/d68/f6f [0,4194304] 0 2024-05-17T21:53:01.714 INFO:tasks.workunit.client.0.smithi107.stdout:7/664: rename d5/df/d11/d1c/d31/d39/dc0/fe8 to d5/d16/d3b/d45/d91/db3/fed 0 2024-05-17T21:53:01.715 INFO:tasks.workunit.client.0.smithi107.stdout:7/665: stat d5/df/d11/d1c 0 2024-05-17T21:53:01.715 INFO:tasks.workunit.client.1.smithi138.stdout:5/531: truncate d1/d60/f87 281019 0 2024-05-17T21:53:01.716 INFO:tasks.workunit.client.1.smithi138.stdout:0/542: sync 2024-05-17T21:53:01.716 INFO:tasks.workunit.client.1.smithi138.stdout:4/552: chown d0/d1b/f3e 0 1 2024-05-17T21:53:01.721 INFO:tasks.workunit.client.0.smithi107.stdout:0/801: unlink d3/dc/d22/d23/d97/d77/d91/ffa 0 2024-05-17T21:53:01.722 INFO:tasks.workunit.client.0.smithi107.stdout:0/802: truncate d3/d75/dc1/fc4 118553 0 2024-05-17T21:53:01.725 INFO:tasks.workunit.client.0.smithi107.stdout:6/627: creat d9/d58/d7b/da3/fc3 x:0 0 0 2024-05-17T21:53:01.727 INFO:tasks.workunit.client.1.smithi138.stdout:1/525: rename d4/df/d3a/d3c/d67/l8d to d4/d2e/d36/la3 0 2024-05-17T21:53:01.728 INFO:tasks.workunit.client.1.smithi138.stdout:7/468: write d0/d18/d40/d41/d51/f69 [964321,95974] 0 2024-05-17T21:53:01.728 INFO:tasks.workunit.client.0.smithi107.stdout:6/628: dread d9/d16/d22/d5f/fac [0,4194304] 0 2024-05-17T21:53:01.729 INFO:tasks.workunit.client.1.smithi138.stdout:0/543: mkdir d3/d3d/d51/dba 0 2024-05-17T21:53:01.729 INFO:tasks.workunit.client.0.smithi107.stdout:8/681: sync 2024-05-17T21:53:01.729 INFO:tasks.workunit.client.0.smithi107.stdout:2/542: sync 2024-05-17T21:53:01.730 INFO:tasks.workunit.client.0.smithi107.stdout:8/682: write f4 [1249967,14380] 0 2024-05-17T21:53:01.732 INFO:tasks.workunit.client.1.smithi138.stdout:0/544: read d3/d3d/f53 [3709708,56495] 0 2024-05-17T21:53:01.732 INFO:tasks.workunit.client.1.smithi138.stdout:0/545: fdatasync d3/d3d/d51/f6e 0 2024-05-17T21:53:01.735 INFO:tasks.workunit.client.0.smithi107.stdout:2/543: dwrite d0/d9/d1c/d37/d9f/fa7 [0,4194304] 0 2024-05-17T21:53:01.736 INFO:tasks.workunit.client.0.smithi107.stdout:2/544: fsync d0/d9/d1c/fab 0 2024-05-17T21:53:01.737 INFO:tasks.workunit.client.0.smithi107.stdout:2/545: stat d0/d9/d11/l67 0 2024-05-17T21:53:01.757 INFO:tasks.workunit.client.1.smithi138.stdout:4/553: rename d0/d3/l30 to d0/d3/d17/d5e/d7f/lc4 0 2024-05-17T21:53:01.768 INFO:tasks.workunit.client.1.smithi138.stdout:4/554: creat d0/d3/d17/fc5 x:0 0 0 2024-05-17T21:53:01.769 INFO:tasks.workunit.client.1.smithi138.stdout:5/532: getdents d1/d6/d1b/d29/d36/d68 0 2024-05-17T21:53:01.773 INFO:tasks.workunit.client.1.smithi138.stdout:4/555: write d0/d3/d17/d5e/fb4 [1943648,10962] 0 2024-05-17T21:53:01.776 INFO:tasks.workunit.client.1.smithi138.stdout:5/533: unlink d1/c4a 0 2024-05-17T21:53:01.779 INFO:tasks.workunit.client.1.smithi138.stdout:7/469: rename d0/d18/d40/d5f/l75 to d0/d18/d40/d41/l95 0 2024-05-17T21:53:01.780 INFO:tasks.workunit.client.1.smithi138.stdout:9/483: sync 2024-05-17T21:53:01.780 INFO:tasks.workunit.client.1.smithi138.stdout:3/631: sync 2024-05-17T21:53:01.783 INFO:tasks.workunit.client.1.smithi138.stdout:5/534: rmdir d1/d2c/d63 39 2024-05-17T21:53:01.787 INFO:tasks.workunit.client.0.smithi107.stdout:4/713: sync 2024-05-17T21:53:01.787 INFO:tasks.workunit.client.0.smithi107.stdout:5/690: sync 2024-05-17T21:53:01.787 INFO:tasks.workunit.client.1.smithi138.stdout:5/535: dread d1/d2c/f8b [0,4194304] 0 2024-05-17T21:53:01.788 INFO:tasks.workunit.client.1.smithi138.stdout:5/536: readlink d1/d1a/d39/d4e/d58/d5b/d65/l6b 0 2024-05-17T21:53:01.788 INFO:tasks.workunit.client.1.smithi138.stdout:3/632: unlink d1/d62/fb8 0 2024-05-17T21:53:01.794 INFO:tasks.workunit.client.1.smithi138.stdout:7/470: truncate d0/d18/d40/f82 2709800 0 2024-05-17T21:53:01.801 INFO:tasks.workunit.client.1.smithi138.stdout:9/484: symlink d6/d7/d16/d29/d6c/d83/d7e/l9d 0 2024-05-17T21:53:01.808 INFO:tasks.workunit.client.0.smithi107.stdout:0/803: mkdir d3/dc/dae/dc0/de1/d10b 0 2024-05-17T21:53:01.811 INFO:tasks.workunit.client.1.smithi138.stdout:7/471: dwrite d0/d18/d40/d41/f77 [0,4194304] 0 2024-05-17T21:53:01.812 INFO:tasks.workunit.client.1.smithi138.stdout:9/485: truncate d6/d7/d16/d29/d38/d40/d44/f87 1414106 0 2024-05-17T21:53:01.814 INFO:tasks.workunit.client.1.smithi138.stdout:0/546: dwrite d3/d10/f20 [4194304,4194304] 0 2024-05-17T21:53:01.815 INFO:tasks.workunit.client.1.smithi138.stdout:1/526: truncate d4/dd/d3b/f73 314911 0 2024-05-17T21:53:01.819 INFO:tasks.workunit.client.1.smithi138.stdout:3/633: rename d1/d4f/d50/f6b to d1/d62/d9b/fd9 0 2024-05-17T21:53:01.821 INFO:tasks.workunit.client.0.smithi107.stdout:8/683: mkdir dc/d21/d27/d34/df5 0 2024-05-17T21:53:01.824 INFO:tasks.workunit.client.1.smithi138.stdout:0/547: dwrite d3/f7 [4194304,4194304] 0 2024-05-17T21:53:01.826 INFO:tasks.workunit.client.1.smithi138.stdout:1/527: dread d4/df/f1b [0,4194304] 0 2024-05-17T21:53:01.827 INFO:tasks.workunit.client.1.smithi138.stdout:1/528: chown d4/df/d3a/d3c/l94 7844 1 2024-05-17T21:53:01.829 INFO:tasks.workunit.client.0.smithi107.stdout:4/714: rename d3/d4a/d47/d60/d3c/fcc to d3/d6/dd5/feb 0 2024-05-17T21:53:01.831 INFO:tasks.workunit.client.1.smithi138.stdout:3/634: chown d1/d3/d16/d2d/c5f 3127395 1 2024-05-17T21:53:01.832 INFO:tasks.workunit.client.1.smithi138.stdout:3/635: write d1/f65 [3681725,28472] 0 2024-05-17T21:53:01.833 INFO:tasks.workunit.client.0.smithi107.stdout:0/804: fdatasync d3/dc/fc3 0 2024-05-17T21:53:01.835 INFO:tasks.workunit.client.1.smithi138.stdout:7/472: unlink d0/d18/d40/d41/d51/d66/l5e 0 2024-05-17T21:53:01.835 INFO:tasks.workunit.client.1.smithi138.stdout:6/553: sync 2024-05-17T21:53:01.836 INFO:tasks.workunit.client.1.smithi138.stdout:2/563: sync 2024-05-17T21:53:01.836 INFO:tasks.workunit.client.1.smithi138.stdout:8/450: sync 2024-05-17T21:53:01.836 INFO:tasks.workunit.client.0.smithi107.stdout:6/629: link d9/d16/d9e/lbe d9/d16/d4d/d85/d96/lc4 0 2024-05-17T21:53:01.838 INFO:tasks.workunit.client.0.smithi107.stdout:8/684: creat dc/d21/d97/d63/d71/d81/de5/ff6 x:0 0 0 2024-05-17T21:53:01.845 INFO:tasks.workunit.client.0.smithi107.stdout:7/666: write d5/df/d11/d1d/d33/d82/fb1 [3424405,125915] 0 2024-05-17T21:53:01.848 INFO:tasks.workunit.client.1.smithi138.stdout:4/556: dwrite d0/d1b/d88/f8b [0,4194304] 0 2024-05-17T21:53:01.851 INFO:tasks.workunit.client.1.smithi138.stdout:5/537: write d1/d6/fe [4262804,52866] 0 2024-05-17T21:53:01.857 INFO:tasks.workunit.client.0.smithi107.stdout:5/691: write d5/db/d39/f43 [7284648,119237] 0 2024-05-17T21:53:01.857 INFO:tasks.workunit.client.1.smithi138.stdout:6/554: read - d7/db/d43/d63/f7d zero size 2024-05-17T21:53:01.860 INFO:tasks.workunit.client.0.smithi107.stdout:7/667: creat d5/df/d11/d1c/d31/d39/fee x:0 0 0 2024-05-17T21:53:01.861 INFO:tasks.workunit.client.0.smithi107.stdout:6/630: read d9/d16/f3e [1329083,17259] 0 2024-05-17T21:53:01.862 INFO:tasks.workunit.client.1.smithi138.stdout:2/564: dread d3/d21/d2f/d3e/f6d [0,4194304] 0 2024-05-17T21:53:01.863 INFO:tasks.workunit.client.0.smithi107.stdout:9/730: sync 2024-05-17T21:53:01.863 INFO:tasks.workunit.client.0.smithi107.stdout:3/729: sync 2024-05-17T21:53:01.863 INFO:tasks.workunit.client.0.smithi107.stdout:1/734: sync 2024-05-17T21:53:01.863 INFO:tasks.workunit.client.0.smithi107.stdout:2/546: dwrite d0/d9/d11/d27/f43 [0,4194304] 0 2024-05-17T21:53:01.864 INFO:tasks.workunit.client.0.smithi107.stdout:6/631: write d9/f15 [784751,25232] 0 2024-05-17T21:53:01.865 INFO:tasks.workunit.client.0.smithi107.stdout:0/805: link d3/d24/d6f/d9f/dab/fb4 d3/d24/d6f/db5/def/f10c 0 2024-05-17T21:53:01.866 INFO:tasks.workunit.client.1.smithi138.stdout:4/557: creat d0/d1b/d88/fc6 x:0 0 0 2024-05-17T21:53:01.867 INFO:tasks.workunit.client.0.smithi107.stdout:9/731: dread - d0/dd/d21/d29/d3e/d45/fec zero size 2024-05-17T21:53:01.868 INFO:tasks.workunit.client.0.smithi107.stdout:6/632: chown d9/f8d 471908735 1 2024-05-17T21:53:01.869 INFO:tasks.workunit.client.1.smithi138.stdout:3/636: dread d1/d62/dad/de/f3f [0,4194304] 0 2024-05-17T21:53:01.869 INFO:tasks.workunit.client.0.smithi107.stdout:1/735: write d4/df/d1a/d26/d72/fb4 [3209403,22691] 0 2024-05-17T21:53:01.869 INFO:tasks.workunit.client.0.smithi107.stdout:8/685: symlink dc/d21/d27/d34/lf7 0 2024-05-17T21:53:01.869 INFO:tasks.workunit.client.1.smithi138.stdout:3/637: read d1/d3/f25 [3539727,106917] 0 2024-05-17T21:53:01.870 INFO:tasks.workunit.client.0.smithi107.stdout:8/686: stat dc/d21 0 2024-05-17T21:53:01.870 INFO:tasks.workunit.client.1.smithi138.stdout:3/638: write d1/d4f/fd5 [694024,37714] 0 2024-05-17T21:53:01.872 INFO:tasks.workunit.client.0.smithi107.stdout:1/736: write d4/df/d1a/d26/d7a/fb9 [205936,20029] 0 2024-05-17T21:53:01.877 INFO:tasks.workunit.client.1.smithi138.stdout:1/529: rename d4/df/d3a/f4c to d4/db/d29/fa4 0 2024-05-17T21:53:01.877 INFO:tasks.workunit.client.0.smithi107.stdout:4/715: write d3/f2c [3470958,84854] 0 2024-05-17T21:53:01.878 INFO:tasks.workunit.client.0.smithi107.stdout:7/668: unlink d5/df/d11/d1c/d2f/d65/f69 0 2024-05-17T21:53:01.879 INFO:tasks.workunit.client.0.smithi107.stdout:9/732: dread d0/d80/fd0 [0,4194304] 0 2024-05-17T21:53:01.880 INFO:tasks.workunit.client.1.smithi138.stdout:6/555: dread - d7/db/d34/da6/f21 zero size 2024-05-17T21:53:01.881 INFO:tasks.workunit.client.1.smithi138.stdout:6/556: readlink d7/db/d43/d63/l67 0 2024-05-17T21:53:01.881 INFO:tasks.workunit.client.1.smithi138.stdout:1/530: dread d4/df/f2c [0,4194304] 0 2024-05-17T21:53:01.882 INFO:tasks.workunit.client.0.smithi107.stdout:3/730: mknod d6/d8/d10/d25/d80/ceb 0 2024-05-17T21:53:01.882 INFO:tasks.workunit.client.1.smithi138.stdout:6/557: stat d7/d69/c8d 0 2024-05-17T21:53:01.882 INFO:tasks.workunit.client.1.smithi138.stdout:6/558: stat d7/d69/c8d 0 2024-05-17T21:53:01.883 INFO:tasks.workunit.client.0.smithi107.stdout:0/806: mkdir d3/d75/d9e/d103/d10d 0 2024-05-17T21:53:01.885 INFO:tasks.workunit.client.0.smithi107.stdout:2/547: mkdir d0/d9/d11/d18/dad 0 2024-05-17T21:53:01.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:01.893+0000 7fef480ce700 1 -- 172.21.15.107:0/418396411 --> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fef401048d0 con 0x7fef2c070b50 2024-05-17T21:53:01.886 INFO:tasks.workunit.client.1.smithi138.stdout:2/565: mknod d3/d81/d96/cba 0 2024-05-17T21:53:01.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:01.894+0000 7fef367fc700 1 -- 172.21.15.107:0/418396411 <== mgr.14636 v2:172.21.15.138:6828/1369055740 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+349 (secure 0 0 0) 0x7fef401048d0 con 0x7fef2c070b50 2024-05-17T21:53:01.887 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-05-17T21:53:01.887 INFO:teuthology.orchestra.run.smithi107.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b", 2024-05-17T21:53:01.887 INFO:teuthology.orchestra.run.smithi107.stdout: "in_progress": true, 2024-05-17T21:53:01.887 INFO:teuthology.orchestra.run.smithi107.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-17T21:53:01.887 INFO:teuthology.orchestra.run.smithi107.stdout: "services_complete": [], 2024-05-17T21:53:01.887 INFO:teuthology.orchestra.run.smithi107.stdout: "progress": "1/21 daemons upgraded", 2024-05-17T21:53:01.887 INFO:teuthology.orchestra.run.smithi107.stdout: "message": "Currently upgrading mgr daemons", 2024-05-17T21:53:01.887 INFO:teuthology.orchestra.run.smithi107.stdout: "is_paused": false 2024-05-17T21:53:01.887 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-05-17T21:53:01.889 INFO:tasks.workunit.client.1.smithi138.stdout:4/558: write d0/d2f/d49/d60/d1f/f7a [3190114,66687] 0 2024-05-17T21:53:01.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:01.897+0000 7fef480ce700 1 -- 172.21.15.107:0/418396411 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7fef2c070b50 msgr2=0x7fef2c073000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:53:01.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:01.897+0000 7fef480ce700 1 --2- 172.21.15.107:0/418396411 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7fef2c070b50 0x7fef2c073000 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fef3c009b80 tx=0x7fef3c00b040 comp rx=0 tx=0).stop 2024-05-17T21:53:01.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:01.897+0000 7fef480ce700 1 -- 172.21.15.107:0/418396411 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef400fcf60 msgr2=0x7fef4010df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:53:01.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:01.897+0000 7fef480ce700 1 --2- 172.21.15.107:0/418396411 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef400fcf60 0x7fef4010df50 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fef30013870 tx=0x7fef30009e30 comp rx=0 tx=0).stop 2024-05-17T21:53:01.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:01.897+0000 7fef480ce700 1 -- 172.21.15.107:0/418396411 shutdown_connections 2024-05-17T21:53:01.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:01.897+0000 7fef480ce700 1 --2- 172.21.15.107:0/418396411 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7fef2c070b50 0x7fef2c073000 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:53:01.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:01.897+0000 7fef480ce700 1 --2- 172.21.15.107:0/418396411 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef400fbcb0 0x7fef4010da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:53:01.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:01.897+0000 7fef480ce700 1 --2- 172.21.15.107:0/418396411 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef400fcf60 0x7fef4010df50 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:53:01.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:01.897+0000 7fef480ce700 1 -- 172.21.15.107:0/418396411 >> 172.21.15.107:0/418396411 conn(0x7fef400f7430 msgr2=0x7fef40065ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:53:01.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:01.897+0000 7fef480ce700 1 -- 172.21.15.107:0/418396411 shutdown_connections 2024-05-17T21:53:01.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:01.897+0000 7fef480ce700 1 -- 172.21.15.107:0/418396411 wait complete. 2024-05-17T21:53:01.891 INFO:tasks.workunit.client.0.smithi107.stdout:6/633: symlink d9/d81/lc5 0 2024-05-17T21:53:01.891 INFO:tasks.workunit.client.0.smithi107.stdout:6/634: fsync d9/d16/d4d/d67/f79 0 2024-05-17T21:53:01.893 INFO:tasks.workunit.client.1.smithi138.stdout:3/639: mkdir d1/d62/dad/d72/dda 0 2024-05-17T21:53:01.893 INFO:tasks.workunit.client.0.smithi107.stdout:5/692: write d5/db/d16/d2d/d40/d5a/d7c/d8e/f90 [4551228,122040] 0 2024-05-17T21:53:01.894 INFO:tasks.workunit.client.1.smithi138.stdout:3/640: fsync d1/d3/d16/d3e/d71/fd6 0 2024-05-17T21:53:01.894 INFO:tasks.workunit.client.0.smithi107.stdout:1/737: creat d4/de/fed x:0 0 0 2024-05-17T21:53:01.895 INFO:tasks.workunit.client.0.smithi107.stdout:1/738: dread - d4/df/d1a/d41/db2/fd9 zero size 2024-05-17T21:53:01.898 INFO:tasks.workunit.client.1.smithi138.stdout:5/538: rename d1/d1a/d39/f6a to d1/d6/d73/fb0 0 2024-05-17T21:53:01.904 INFO:tasks.workunit.client.0.smithi107.stdout:9/733: rmdir d0/dd/d2f/d4a 39 2024-05-17T21:53:01.904 INFO:tasks.workunit.client.1.smithi138.stdout:5/539: dwrite d1/d6/d19/fa3 [0,4194304] 0 2024-05-17T21:53:01.908 INFO:tasks.workunit.client.0.smithi107.stdout:1/739: dread d4/df/d1a/d26/d72/dbc/fad [0,4194304] 0 2024-05-17T21:53:01.909 INFO:tasks.workunit.client.1.smithi138.stdout:6/559: creat d7/db/d34/da6/fc3 x:0 0 0 2024-05-17T21:53:01.911 INFO:tasks.workunit.client.0.smithi107.stdout:0/807: unlink d3/f21 0 2024-05-17T21:53:01.912 INFO:tasks.workunit.client.1.smithi138.stdout:9/486: link d6/d7/d16/d29/d38/d40/d44/f87 d6/d7/d16/d29/d48/d6e/d5a/f9e 0 2024-05-17T21:53:01.913 INFO:tasks.workunit.client.0.smithi107.stdout:2/548: dread - d0/f6f zero size 2024-05-17T21:53:01.914 INFO:tasks.workunit.client.1.smithi138.stdout:2/566: creat d3/da0/fbb x:0 0 0 2024-05-17T21:53:01.917 INFO:tasks.workunit.client.0.smithi107.stdout:3/731: write d6/d8/d10/d25/f47 [3275449,103745] 0 2024-05-17T21:53:01.917 INFO:tasks.workunit.client.1.smithi138.stdout:8/451: getdents d0/dc 0 2024-05-17T21:53:01.922 INFO:tasks.workunit.client.0.smithi107.stdout:5/693: fsync d5/db/d16/d54/d59/f7f 0 2024-05-17T21:53:01.922 INFO:tasks.workunit.client.0.smithi107.stdout:7/669: dwrite d5/df/d11/d1c/d31/d39/f5e [4194304,4194304] 0 2024-05-17T21:53:01.922 INFO:tasks.workunit.client.1.smithi138.stdout:3/641: rename d1/d3/d49/d4b to d1/d3/d49/ddb 0 2024-05-17T21:53:01.924 INFO:tasks.workunit.client.0.smithi107.stdout:4/716: creat d3/d6/d26/d36/d37/d85/fec x:0 0 0 2024-05-17T21:53:01.925 INFO:tasks.workunit.client.1.smithi138.stdout:7/473: dread d0/d18/d40/f82 [0,4194304] 0 2024-05-17T21:53:01.926 INFO:tasks.workunit.client.1.smithi138.stdout:1/531: rename d4/dd/c10 to d4/db/d60/ca5 0 2024-05-17T21:53:01.931 INFO:tasks.workunit.client.0.smithi107.stdout:9/734: creat d0/dd/d21/d29/d57/d5f/ff5 x:0 0 0 2024-05-17T21:53:01.933 INFO:tasks.workunit.client.0.smithi107.stdout:8/687: write dc/d21/d35/f61 [52845,4480] 0 2024-05-17T21:53:01.935 INFO:tasks.workunit.client.1.smithi138.stdout:1/532: dwrite d4/df/d3a/d3c/d67/f85 [8388608,4194304] 0 2024-05-17T21:53:01.935 INFO:tasks.workunit.client.0.smithi107.stdout:1/740: dread d4/df/d1a/d41/fe5 [0,4194304] 0 2024-05-17T21:53:01.938 INFO:tasks.workunit.client.0.smithi107.stdout:0/808: read d3/dc/d22/d23/f29 [5678437,36168] 0 2024-05-17T21:53:01.938 INFO:tasks.workunit.client.1.smithi138.stdout:8/452: chown d0/c8 116 1 2024-05-17T21:53:01.941 INFO:tasks.workunit.client.0.smithi107.stdout:3/732: creat d6/d8/d10/d35/d36/d4c/dcc/fec x:0 0 0 2024-05-17T21:53:01.942 INFO:tasks.workunit.client.1.smithi138.stdout:3/642: rmdir d1/d3/d16/d3a 39 2024-05-17T21:53:01.944 INFO:tasks.workunit.client.1.smithi138.stdout:7/474: creat d0/d18/d40/d5f/f96 x:0 0 0 2024-05-17T21:53:01.945 INFO:tasks.workunit.client.1.smithi138.stdout:7/475: fdatasync d0/d18/d40/d41/d50/f74 0 2024-05-17T21:53:01.947 INFO:tasks.workunit.client.1.smithi138.stdout:9/487: rename d6/d7/d16/d29/d38/c39 to d6/d7/d16/d29/d48/d6e/d74/c9f 0 2024-05-17T21:53:01.949 INFO:tasks.workunit.client.1.smithi138.stdout:9/488: chown d6/d7/d16/d29/d38/d40/d44/d5c/f8f 456 1 2024-05-17T21:53:01.950 INFO:tasks.workunit.client.0.smithi107.stdout:2/549: write d0/d9/d1c/d3e/f3f [1517041,72082] 0 2024-05-17T21:53:01.951 INFO:tasks.workunit.client.0.smithi107.stdout:5/694: creat d5/db/d16/d2d/d4d/d6d/fe0 x:0 0 0 2024-05-17T21:53:01.952 INFO:tasks.workunit.client.0.smithi107.stdout:6/635: write d9/d16/d4d/f5b [1031874,124556] 0 2024-05-17T21:53:01.952 INFO:tasks.workunit.client.1.smithi138.stdout:4/559: truncate d0/d3/d17/d21/d78/f9a 1955840 0 2024-05-17T21:53:01.955 INFO:tasks.workunit.client.0.smithi107.stdout:7/670: creat d5/d16/d3b/d45/d91/dac/fef x:0 0 0 2024-05-17T21:53:01.956 INFO:tasks.workunit.client.1.smithi138.stdout:1/533: chown d4/df/d35/l45 169749 1 2024-05-17T21:53:01.957 INFO:tasks.workunit.client.1.smithi138.stdout:8/453: mknod d0/dc/d3b/c86 0 2024-05-17T21:53:01.958 INFO:tasks.workunit.client.0.smithi107.stdout:9/735: rmdir d0/d80 39 2024-05-17T21:53:01.959 INFO:tasks.workunit.client.0.smithi107.stdout:9/736: readlink d0/dd/d69/la4 0 2024-05-17T21:53:01.959 INFO:tasks.workunit.client.0.smithi107.stdout:9/737: stat d0/dd/d2f/l41 0 2024-05-17T21:53:01.961 INFO:tasks.workunit.client.1.smithi138.stdout:7/476: chown d0/d18/f45 430 1 2024-05-17T21:53:01.965 INFO:tasks.workunit.client.0.smithi107.stdout:1/741: dread d4/df/d1a/d26/d72/dbc/fad [0,4194304] 0 2024-05-17T21:53:01.965 INFO:tasks.workunit.client.0.smithi107.stdout:0/809: rename d3/f33 to d3/d24/d6f/db5/da5/f10e 0 2024-05-17T21:53:01.966 INFO:tasks.workunit.client.0.smithi107.stdout:0/810: stat d3/dc/d22/f68 0 2024-05-17T21:53:01.968 INFO:tasks.workunit.client.1.smithi138.stdout:3/643: write d1/d3/d16/f38 [4879353,58237] 0 2024-05-17T21:53:01.969 INFO:tasks.workunit.client.0.smithi107.stdout:1/742: dread d4/df/d1a/d41/fe5 [0,4194304] 0 2024-05-17T21:53:01.970 INFO:tasks.workunit.client.0.smithi107.stdout:3/733: unlink d6/d2a/d2d/d69/c82 0 2024-05-17T21:53:01.971 INFO:tasks.workunit.client.0.smithi107.stdout:4/717: dread d3/d4a/d47/d60/f12 [0,4194304] 0 2024-05-17T21:53:01.973 INFO:tasks.workunit.client.1.smithi138.stdout:4/560: write d0/d1b/f3e [1590181,94149] 0 2024-05-17T21:53:01.973 INFO:tasks.workunit.client.0.smithi107.stdout:2/550: symlink d0/d9/d11/df/lae 0 2024-05-17T21:53:01.974 INFO:tasks.workunit.client.0.smithi107.stdout:2/551: write d0/d9/d1c/d3e/d4f/f84 [978228,88419] 0 2024-05-17T21:53:01.975 INFO:tasks.workunit.client.0.smithi107.stdout:2/552: chown d0/d9/d1c/d37/c41 996692953 1 2024-05-17T21:53:01.976 INFO:tasks.workunit.client.1.smithi138.stdout:3/644: dwrite d1/d62/dad/de/fb5 [0,4194304] 0 2024-05-17T21:53:01.978 INFO:tasks.workunit.client.0.smithi107.stdout:6/636: dread d9/d16/d22/d2d/f31 [0,4194304] 0 2024-05-17T21:53:01.978 INFO:tasks.workunit.client.0.smithi107.stdout:7/671: unlink d5/df/d11/d1c/f2e 0 2024-05-17T21:53:01.980 INFO:tasks.workunit.client.0.smithi107.stdout:6/637: truncate d9/d16/d22/d5f/d6f/f91 935492 0 2024-05-17T21:53:01.980 INFO:tasks.workunit.client.1.smithi138.stdout:7/477: fsync d0/f2a 0 2024-05-17T21:53:01.981 INFO:tasks.workunit.client.1.smithi138.stdout:7/478: read d0/f4d [2453552,14885] 0 2024-05-17T21:53:01.983 INFO:tasks.workunit.client.0.smithi107.stdout:8/688: symlink dc/lf8 0 2024-05-17T21:53:01.983 INFO:tasks.workunit.client.1.smithi138.stdout:7/479: truncate d0/d18/d40/d5f/f8f 362159 0 2024-05-17T21:53:01.984 INFO:tasks.workunit.client.1.smithi138.stdout:6/560: getdents d7/db/d34/da6 0 2024-05-17T21:53:01.985 INFO:tasks.workunit.client.1.smithi138.stdout:7/480: dread d0/f91 [0,4194304] 0 2024-05-17T21:53:01.987 INFO:tasks.workunit.client.1.smithi138.stdout:9/489: write d6/d7/d16/f1a [561807,59604] 0 2024-05-17T21:53:01.988 INFO:tasks.workunit.client.1.smithi138.stdout:7/481: dread d0/d18/d40/d41/d51/d66/f6f [0,4194304] 0 2024-05-17T21:53:01.990 INFO:tasks.workunit.client.1.smithi138.stdout:9/490: write d6/d7/d16/d29/d48/d3b/f63 [1392053,6730] 0 2024-05-17T21:53:01.990 INFO:tasks.workunit.client.1.smithi138.stdout:2/567: getdents d3/d21/d2f 0 2024-05-17T21:53:01.993 INFO:tasks.workunit.client.0.smithi107.stdout:9/738: dread d0/dd/d3d/f8e [0,4194304] 0 2024-05-17T21:53:01.994 INFO:tasks.workunit.client.0.smithi107.stdout:0/811: symlink d3/d24/d6f/db5/da5/l10f 0 2024-05-17T21:53:01.994 INFO:tasks.workunit.client.1.smithi138.stdout:8/454: dread d0/f40 [0,4194304] 0 2024-05-17T21:53:01.996 INFO:tasks.workunit.client.0.smithi107.stdout:1/743: mkdir d4/df/d14/d2e/ddf/dee 0 2024-05-17T21:53:01.997 INFO:tasks.workunit.client.1.smithi138.stdout:1/534: write f0 [1146290,123333] 0 2024-05-17T21:53:01.998 INFO:tasks.workunit.client.1.smithi138.stdout:7/482: dwrite d0/d18/d40/d41/d50/f59 [4194304,4194304] 0 2024-05-17T21:53:01.999 INFO:tasks.workunit.client.0.smithi107.stdout:4/718: symlink d3/d4a/d47/d60/d3c/d9d/led 0 2024-05-17T21:53:02.000 INFO:tasks.workunit.client.0.smithi107.stdout:4/719: write d3/d6/fe0 [298268,43635] 0 2024-05-17T21:53:02.002 INFO:tasks.workunit.client.1.smithi138.stdout:3/645: creat d1/d62/dad/d72/fdc x:0 0 0 2024-05-17T21:53:02.007 INFO:tasks.workunit.client.0.smithi107.stdout:8/689: creat dc/d21/d97/db7/ff9 x:0 0 0 2024-05-17T21:53:02.007 INFO:tasks.workunit.client.1.smithi138.stdout:6/561: rmdir d7/db/d43/d3f 39 2024-05-17T21:53:02.009 INFO:tasks.workunit.client.0.smithi107.stdout:5/695: rename d5/db/dc/lda to d5/db/dc/d1e/d6e/le1 0 2024-05-17T21:53:02.009 INFO:tasks.workunit.client.0.smithi107.stdout:5/696: dread - d5/db/d16/d2d/d40/d9f/fc3 zero size 2024-05-17T21:53:02.012 INFO:tasks.workunit.client.0.smithi107.stdout:7/672: write d5/df/d11/d1d/d33/f64 [783233,110426] 0 2024-05-17T21:53:02.013 INFO:tasks.workunit.client.0.smithi107.stdout:7/673: read d5/f22 [3567169,19696] 0 2024-05-17T21:53:02.014 INFO:tasks.workunit.client.0.smithi107.stdout:0/812: mkdir d3/dc/d22/d23/d110 0 2024-05-17T21:53:02.014 INFO:tasks.workunit.client.0.smithi107.stdout:0/813: readlink d3/d24/d6f/d9f/l101 0 2024-05-17T21:53:02.015 INFO:tasks.workunit.client.1.smithi138.stdout:2/568: creat d3/d21/d2f/d8a/fbc x:0 0 0 2024-05-17T21:53:02.016 INFO:tasks.workunit.client.1.smithi138.stdout:2/569: chown d3/d21/d2f/f3c 512 1 2024-05-17T21:53:02.019 INFO:tasks.workunit.client.0.smithi107.stdout:3/734: symlink d6/d8/d3e/d3f/d90/da7/ddc/led 0 2024-05-17T21:53:02.021 INFO:tasks.workunit.client.1.smithi138.stdout:4/561: mknod d0/d2f/d49/d60/cc7 0 2024-05-17T21:53:02.021 INFO:tasks.workunit.client.1.smithi138.stdout:9/491: write d6/f49 [796541,90995] 0 2024-05-17T21:53:02.022 INFO:tasks.workunit.client.0.smithi107.stdout:2/553: symlink d0/d9/d11/laf 0 2024-05-17T21:53:02.023 INFO:tasks.workunit.client.0.smithi107.stdout:6/638: dread d9/f1d [4194304,4194304] 0 2024-05-17T21:53:02.026 INFO:tasks.workunit.client.0.smithi107.stdout:4/720: mkdir d3/d6/d26/d36/d8b/d95/dee 0 2024-05-17T21:53:02.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:02.036+0000 7f01d5bd1700 1 -- 172.21.15.107:0/1763250352 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01d00fce70 msgr2=0x7f01d00fd280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:53:02.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:02.036+0000 7f01d5bd1700 1 --2- 172.21.15.107:0/1763250352 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01d00fce70 0x7f01d00fd280 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f01c0004940 tx=0x7f01c0009e30 comp rx=0 tx=0).stop 2024-05-17T21:53:02.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:02.036+0000 7f01d5bd1700 1 -- 172.21.15.107:0/1763250352 shutdown_connections 2024-05-17T21:53:02.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:02.036+0000 7f01d5bd1700 1 --2- 172.21.15.107:0/1763250352 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f01d00fdfe0 0x7f01d00fe450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:53:02.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:02.036+0000 7f01d5bd1700 1 --2- 172.21.15.107:0/1763250352 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01d00fce70 0x7f01d00fd280 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:53:02.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:02.036+0000 7f01d5bd1700 1 -- 172.21.15.107:0/1763250352 >> 172.21.15.107:0/1763250352 conn(0x7f01d00f8550 msgr2=0x7f01d00fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:53:02.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:02.036+0000 7f01d5bd1700 1 -- 172.21.15.107:0/1763250352 shutdown_connections 2024-05-17T21:53:02.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:02.036+0000 7f01d5bd1700 1 -- 172.21.15.107:0/1763250352 wait complete. 2024-05-17T21:53:02.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:02.036+0000 7f01d5bd1700 1 Processor -- start 2024-05-17T21:53:02.029 INFO:tasks.workunit.client.1.smithi138.stdout:8/455: dwrite d0/f40 [0,4194304] 0 2024-05-17T21:53:02.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:02.036+0000 7f01d5bd1700 1 -- start start 2024-05-17T21:53:02.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:02.037+0000 7f01d5bd1700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01d00fce70 0x7f01d010d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:53:02.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:02.037+0000 7f01d5bd1700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f01d00fdfe0 0x7f01d010ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:53:02.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:02.037+0000 7f01d5bd1700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01d010e3f0 con 0x7f01d00fce70 2024-05-17T21:53:02.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:02.037+0000 7f01d5bd1700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01d010e560 con 0x7f01d00fdfe0 2024-05-17T21:53:02.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:02.037+0000 7f01cf7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01d00fce70 0x7f01d010d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:53:02.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:02.037+0000 7f01cf7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01d00fce70 0x7f01d010d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:35444/0 (socket says 172.21.15.107:35444) 2024-05-17T21:53:02.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:02.037+0000 7f01cf7fe700 1 -- 172.21.15.107:0/4293648570 learned_addr learned my addr 172.21.15.107:0/4293648570 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:53:02.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:02.037+0000 7f01ceffd700 1 --2- 172.21.15.107:0/4293648570 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f01d00fdfe0 0x7f01d010ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:53:02.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:02.037+0000 7f01cf7fe700 1 -- 172.21.15.107:0/4293648570 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f01d00fdfe0 msgr2=0x7f01d010ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:53:02.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:02.037+0000 7f01cf7fe700 1 --2- 172.21.15.107:0/4293648570 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f01d00fdfe0 0x7f01d010ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:53:02.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:02.037+0000 7f01cf7fe700 1 -- 172.21.15.107:0/4293648570 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01c0005040 con 0x7f01d00fce70 2024-05-17T21:53:02.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:02.037+0000 7f01cf7fe700 1 --2- 172.21.15.107:0/4293648570 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01d00fce70 0x7f01d010d8b0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f01c0013870 tx=0x7f01c0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:53:02.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:02.037+0000 7f01b7fff700 1 -- 172.21.15.107:0/4293648570 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f01c000e070 con 0x7f01d00fce70 2024-05-17T21:53:02.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:02.038+0000 7f01b7fff700 1 -- 172.21.15.107:0/4293648570 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f01c001c400 con 0x7f01d00fce70 2024-05-17T21:53:02.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:02.038+0000 7f01b7fff700 1 -- 172.21.15.107:0/4293648570 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f01c000b630 con 0x7f01d00fce70 2024-05-17T21:53:02.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:02.038+0000 7f01d5bd1700 1 -- 172.21.15.107:0/4293648570 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01d0112f60 con 0x7f01d00fce70 2024-05-17T21:53:02.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:02.038+0000 7f01d5bd1700 1 -- 172.21.15.107:0/4293648570 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01d01133c0 con 0x7f01d00fce70 2024-05-17T21:53:02.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:02.039+0000 7f01d5bd1700 1 -- 172.21.15.107:0/4293648570 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01d004c710 con 0x7f01d00fce70 2024-05-17T21:53:02.033 INFO:tasks.workunit.client.0.smithi107.stdout:8/690: symlink dc/d21/d97/db7/lfa 0 2024-05-17T21:53:02.033 INFO:tasks.workunit.client.1.smithi138.stdout:7/483: readlink d0/l16 0 2024-05-17T21:53:02.033 INFO:tasks.workunit.client.1.smithi138.stdout:8/456: write d0/f40 [199789,117542] 0 2024-05-17T21:53:02.033 INFO:tasks.workunit.client.1.smithi138.stdout:7/484: chown d0/d18/d40/d41/d51/d81 0 1 2024-05-17T21:53:02.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:02.039+0000 7f01b7fff700 1 -- 172.21.15.107:0/4293648570 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) v1 ==== 93316+0+0 (secure 0 0 0) 0x7f01c0013af0 con 0x7f01d00fce70 2024-05-17T21:53:02.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:02.040+0000 7f01b7fff700 1 --2- 172.21.15.107:0/4293648570 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7f01b8070bc0 0x7f01b8073070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:53:02.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:02.040+0000 7f01b7fff700 1 -- 172.21.15.107:0/4293648570 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f01c00920e0 con 0x7f01d00fce70 2024-05-17T21:53:02.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:02.040+0000 7f01ceffd700 1 --2- 172.21.15.107:0/4293648570 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7f01b8070bc0 0x7f01b8073070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:53:02.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:02.041+0000 7f01ceffd700 1 --2- 172.21.15.107:0/4293648570 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7f01b8070bc0 0x7f01b8073070 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f01c4004600 tx=0x7f01c400b040 comp rx=0 tx=0).ready entity=mgr.14636 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:53:02.034 INFO:tasks.workunit.client.0.smithi107.stdout:5/697: mknod d5/db/d16/d2a/ce2 0 2024-05-17T21:53:02.036 INFO:tasks.workunit.client.0.smithi107.stdout:5/698: truncate d5/db/d16/d2d/d40/d5a/d7c/fdd 21110 0 2024-05-17T21:53:02.037 INFO:tasks.workunit.client.1.smithi138.stdout:1/535: dwrite d4/d2e/d36/f91 [0,4194304] 0 2024-05-17T21:53:02.038 INFO:tasks.workunit.client.1.smithi138.stdout:1/536: write d4/df/f97 [1259878,62547] 0 2024-05-17T21:53:02.040 INFO:tasks.workunit.client.0.smithi107.stdout:7/674: creat d5/df/d11/d1c/d2f/d65/ff0 x:0 0 0 2024-05-17T21:53:02.041 INFO:tasks.workunit.client.1.smithi138.stdout:2/570: mknod d3/d21/d2f/d3e/d55/d99/cbd 0 2024-05-17T21:53:02.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:02.049+0000 7f01b7fff700 1 -- 172.21.15.107:0/4293648570 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+190121 (secure 0 0 0) 0x7f01c005a5d0 con 0x7f01d00fce70 2024-05-17T21:53:02.043 INFO:tasks.workunit.client.1.smithi138.stdout:4/562: rmdir d0/d2f/d49/d60/d1f/d32 39 2024-05-17T21:53:02.045 INFO:tasks.workunit.client.0.smithi107.stdout:1/744: symlink d4/df/d14/d2e/ddf/dee/lef 0 2024-05-17T21:53:02.045 INFO:tasks.workunit.client.0.smithi107.stdout:8/691: dread dc/d21/d97/f8d [0,4194304] 0 2024-05-17T21:53:02.046 INFO:tasks.workunit.client.0.smithi107.stdout:8/692: write dc/d21/db8/fc4 [489922,88730] 0 2024-05-17T21:53:02.047 INFO:tasks.workunit.client.1.smithi138.stdout:9/492: rmdir d6/d7/d16/d29/d48 39 2024-05-17T21:53:02.048 INFO:tasks.workunit.client.0.smithi107.stdout:2/554: dread - d0/d26/d64/f8a zero size 2024-05-17T21:53:02.048 INFO:tasks.workunit.client.1.smithi138.stdout:9/493: fdatasync d6/d7/d16/d29/d38/d40/d44/d5c/f7a 0 2024-05-17T21:53:02.050 INFO:tasks.workunit.client.0.smithi107.stdout:4/721: symlink d3/d4a/d47/d60/d3a/d41/d80/d43/lef 0 2024-05-17T21:53:02.051 INFO:tasks.workunit.client.1.smithi138.stdout:9/494: write d6/d7/d16/d29/d38/d40/d44/d5c/f8f [891163,39599] 0 2024-05-17T21:53:02.052 INFO:tasks.workunit.client.0.smithi107.stdout:4/722: dread - d3/d6/d26/d36/d8b/d95/fe6 zero size 2024-05-17T21:53:02.053 INFO:tasks.workunit.client.0.smithi107.stdout:6/639: dread d9/d16/f7f [0,4194304] 0 2024-05-17T21:53:02.053 INFO:tasks.workunit.client.1.smithi138.stdout:7/485: symlink d0/d39/d78/l97 0 2024-05-17T21:53:02.054 INFO:tasks.workunit.client.0.smithi107.stdout:4/723: fsync d3/d4a/d47/d60/d3a/d41/d80/d43/f62 0 2024-05-17T21:53:02.054 INFO:tasks.workunit.client.0.smithi107.stdout:5/699: creat d5/db/d16/d2d/d40/fe3 x:0 0 0 2024-05-17T21:53:02.054 INFO:tasks.workunit.client.1.smithi138.stdout:4/563: dread d0/d3/d1d/d38/f50 [0,4194304] 0 2024-05-17T21:53:02.055 INFO:tasks.workunit.client.1.smithi138.stdout:3/646: mkdir d1/dc0/ddd 0 2024-05-17T21:53:02.059 INFO:tasks.workunit.client.1.smithi138.stdout:1/537: truncate d4/db/d29/d5d/f6f 3403854 0 2024-05-17T21:53:02.060 INFO:tasks.workunit.client.0.smithi107.stdout:1/745: unlink d4/df/d14/la6 0 2024-05-17T21:53:02.062 INFO:tasks.workunit.client.1.smithi138.stdout:2/571: mkdir d3/d21/d2f/d3e/d55/d99/d84/dbe 0 2024-05-17T21:53:02.065 INFO:tasks.workunit.client.0.smithi107.stdout:0/814: dwrite d3/d24/d6f/f9d [0,4194304] 0 2024-05-17T21:53:02.068 INFO:tasks.workunit.client.0.smithi107.stdout:2/555: rmdir d0/d9/d1c 39 2024-05-17T21:53:02.069 INFO:tasks.workunit.client.0.smithi107.stdout:9/739: link d0/dd/d2f/d87/fb6 d0/dd/ff6 0 2024-05-17T21:53:02.071 INFO:tasks.workunit.client.1.smithi138.stdout:7/486: symlink d0/d18/d40/d41/d51/d81/d84/l98 0 2024-05-17T21:53:02.072 INFO:tasks.workunit.client.1.smithi138.stdout:3/647: mkdir d1/d4f/d50/dbb/dde 0 2024-05-17T21:53:02.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:01 smithi138 ceph-mon[35948]: from='client.14726 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:53:02.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:01 smithi138 ceph-mon[35948]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:53:02.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:01 smithi138 ceph-mon[35948]: from='client.? 172.21.15.107:0/2116208213' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:53:02.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:01 smithi138 ceph-mon[35948]: from='client.? 172.21.15.107:0/4022027707' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-17T21:53:02.077 INFO:tasks.workunit.client.1.smithi138.stdout:4/564: rename d0/d3/d17/d5e/d7f/fba to d0/d1b/fc8 0 2024-05-17T21:53:02.077 INFO:tasks.workunit.client.1.smithi138.stdout:7/487: dwrite d0/d4c/f6d [0,4194304] 0 2024-05-17T21:53:02.080 INFO:tasks.workunit.client.1.smithi138.stdout:2/572: rmdir d3/d21/d2f/d8a 39 2024-05-17T21:53:02.081 INFO:tasks.workunit.client.1.smithi138.stdout:2/573: fdatasync d3/d21/f2b 0 2024-05-17T21:53:02.084 INFO:tasks.workunit.client.0.smithi107.stdout:4/724: read d3/d4a/d47/d60/f79 [228253,14393] 0 2024-05-17T21:53:02.088 INFO:tasks.workunit.client.1.smithi138.stdout:8/457: write d0/dc/d1a/d33/d4a/d68/f6c [4282653,46855] 0 2024-05-17T21:53:02.090 INFO:tasks.workunit.client.1.smithi138.stdout:9/495: symlink d6/d7/d16/d29/d48/d67/la0 0 2024-05-17T21:53:02.096 INFO:tasks.workunit.client.1.smithi138.stdout:1/538: rename d4/df/d35/c4a to d4/df/d5f/ca6 0 2024-05-17T21:53:02.098 INFO:tasks.workunit.client.1.smithi138.stdout:4/565: symlink d0/d3/d1d/d38/lc9 0 2024-05-17T21:53:02.098 INFO:tasks.workunit.client.1.smithi138.stdout:4/566: readlink d0/d3/lbb 0 2024-05-17T21:53:02.099 INFO:tasks.workunit.client.1.smithi138.stdout:7/488: truncate d0/f91 1429733 0 2024-05-17T21:53:02.101 INFO:tasks.workunit.client.1.smithi138.stdout:8/458: rmdir d0/dc/d1a/d51 39 2024-05-17T21:53:02.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:01 smithi107 ceph-mon[28434]: from='client.14726 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:53:02.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:01 smithi107 ceph-mon[28434]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:53:02.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:01 smithi107 ceph-mon[28434]: from='client.? 172.21.15.107:0/2116208213' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:53:02.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:01 smithi107 ceph-mon[28434]: from='client.? 172.21.15.107:0/4022027707' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-17T21:53:02.102 INFO:tasks.workunit.client.1.smithi138.stdout:8/459: chown d0/d11 7 1 2024-05-17T21:53:02.104 INFO:tasks.workunit.client.1.smithi138.stdout:9/496: mknod d6/d7/d16/d29/d38/d40/d44/ca1 0 2024-05-17T21:53:02.107 INFO:tasks.workunit.client.1.smithi138.stdout:4/567: rmdir d0/d2f/d49/d60/d1f/da1 39 2024-05-17T21:53:02.109 INFO:tasks.workunit.client.1.smithi138.stdout:8/460: mkdir d0/dc/d3b/d87 0 2024-05-17T21:53:02.110 INFO:tasks.workunit.client.1.smithi138.stdout:9/497: rmdir d6/d7/d16/d29/d48/d6e/d74 39 2024-05-17T21:53:02.111 INFO:tasks.workunit.client.1.smithi138.stdout:1/539: write d4/df/d3a/d3c/f7c [10378122,118594] 0 2024-05-17T21:53:02.111 INFO:tasks.workunit.client.1.smithi138.stdout:3/648: link d1/d4f/d80/cb9 d1/d62/dad/d72/cdf 0 2024-05-17T21:53:02.113 INFO:tasks.workunit.client.1.smithi138.stdout:4/568: mkdir d0/d1b/d2b/dca 0 2024-05-17T21:53:02.114 INFO:tasks.workunit.client.1.smithi138.stdout:2/574: truncate d3/d21/d2f/d3e/d55/d99/d84/f91 446508 0 2024-05-17T21:53:02.116 INFO:tasks.workunit.client.1.smithi138.stdout:1/540: mkdir d4/dd/d3b/da7 0 2024-05-17T21:53:02.118 INFO:tasks.workunit.client.1.smithi138.stdout:9/498: dwrite d6/d7/d16/d29/d48/d6f/f71 [4194304,4194304] 0 2024-05-17T21:53:02.119 INFO:tasks.workunit.client.1.smithi138.stdout:4/569: unlink d0/d3/db/faf 0 2024-05-17T21:53:02.121 INFO:tasks.workunit.client.1.smithi138.stdout:4/570: dread - d0/d1b/d88/fc6 zero size 2024-05-17T21:53:02.121 INFO:tasks.workunit.client.1.smithi138.stdout:8/461: truncate d0/dc/d1a/f4b 4911703 0 2024-05-17T21:53:02.125 INFO:tasks.workunit.client.1.smithi138.stdout:8/462: write d0/dc/d1a/f3f [4522833,59739] 0 2024-05-17T21:53:02.127 INFO:tasks.workunit.client.1.smithi138.stdout:7/489: getdents d0/d39 0 2024-05-17T21:53:02.130 INFO:tasks.workunit.client.1.smithi138.stdout:8/463: fdatasync d0/dc/d3b/d4d/f77 0 2024-05-17T21:53:02.130 INFO:tasks.workunit.client.1.smithi138.stdout:1/541: dwrite d4/dd/d3b/f9e [0,4194304] 0 2024-05-17T21:53:02.132 INFO:tasks.workunit.client.1.smithi138.stdout:9/499: dwrite d6/d7/d16/d29/d6c/d83/f92 [0,4194304] 0 2024-05-17T21:53:02.137 INFO:tasks.workunit.client.1.smithi138.stdout:7/490: mknod d0/d18/d40/d5f/c99 0 2024-05-17T21:53:02.140 INFO:tasks.workunit.client.1.smithi138.stdout:1/542: creat d4/df/fa8 x:0 0 0 2024-05-17T21:53:02.141 INFO:tasks.workunit.client.1.smithi138.stdout:9/500: mknod d6/d7/d16/d29/d38/d40/d44/d5c/ca2 0 2024-05-17T21:53:02.142 INFO:tasks.workunit.client.1.smithi138.stdout:2/575: dwrite d3/f6f [0,4194304] 0 2024-05-17T21:53:02.142 INFO:tasks.workunit.client.1.smithi138.stdout:7/491: chown d0/d18/d64 1 1 2024-05-17T21:53:02.143 INFO:tasks.workunit.client.1.smithi138.stdout:1/543: fsync d4/dd/d3b/d6c/f78 0 2024-05-17T21:53:02.144 INFO:tasks.workunit.client.1.smithi138.stdout:2/576: dread - d3/d81/d96/fb4 zero size 2024-05-17T21:53:02.145 INFO:tasks.workunit.client.1.smithi138.stdout:2/577: fsync d3/d21/f38 0 2024-05-17T21:53:02.145 INFO:tasks.workunit.client.1.smithi138.stdout:4/571: creat d0/fcb x:0 0 0 2024-05-17T21:53:02.147 INFO:tasks.workunit.client.1.smithi138.stdout:8/464: mknod d0/dc/d1a/d33/d4a/c88 0 2024-05-17T21:53:02.148 INFO:tasks.workunit.client.1.smithi138.stdout:3/649: getdents d1/d3/d8d 0 2024-05-17T21:53:02.148 INFO:tasks.workunit.client.1.smithi138.stdout:8/465: fsync d0/d2f/f45 0 2024-05-17T21:53:02.150 INFO:tasks.workunit.client.1.smithi138.stdout:2/578: chown d3/d21/d2f/d3e/d55/d73/l88 3 1 2024-05-17T21:53:02.154 INFO:tasks.workunit.client.1.smithi138.stdout:9/501: creat d6/d7/d16/d29/d38/fa3 x:0 0 0 2024-05-17T21:53:02.157 INFO:tasks.workunit.client.1.smithi138.stdout:9/502: fdatasync d6/d7/d16/d29/d48/d6e/f56 0 2024-05-17T21:53:02.159 INFO:tasks.workunit.client.1.smithi138.stdout:8/466: dread d0/f1b [0,4194304] 0 2024-05-17T21:53:02.159 INFO:tasks.workunit.client.1.smithi138.stdout:2/579: dread d3/f6f [0,4194304] 0 2024-05-17T21:53:02.159 INFO:tasks.workunit.client.1.smithi138.stdout:3/650: dwrite d1/d3/d16/d3e/d71/fa9 [0,4194304] 0 2024-05-17T21:53:02.162 INFO:tasks.workunit.client.1.smithi138.stdout:0/548: sync 2024-05-17T21:53:02.164 INFO:tasks.workunit.client.1.smithi138.stdout:2/580: write d3/d21/d2f/d3e/d63/da3/fb9 [70350,104023] 0 2024-05-17T21:53:02.165 INFO:tasks.workunit.client.1.smithi138.stdout:3/651: rmdir d1/d3/d8d 39 2024-05-17T21:53:02.166 INFO:tasks.workunit.client.1.smithi138.stdout:2/581: chown d3/d21/d2f/d3e/fad 270 1 2024-05-17T21:53:02.167 INFO:tasks.workunit.client.1.smithi138.stdout:2/582: chown d3/d81/fa7 49 1 2024-05-17T21:53:02.170 INFO:tasks.workunit.client.1.smithi138.stdout:0/549: creat d3/d10/d38/d3e/d59/dad/fbb x:0 0 0 2024-05-17T21:53:02.171 INFO:tasks.workunit.client.1.smithi138.stdout:2/583: mknod d3/d81/cbf 0 2024-05-17T21:53:02.173 INFO:tasks.workunit.client.1.smithi138.stdout:0/550: fsync d3/f29 0 2024-05-17T21:53:02.175 INFO:tasks.workunit.client.0.smithi107.stdout:0/815: fdatasync d3/dc/d22/d23/fb7 0 2024-05-17T21:53:02.176 INFO:tasks.workunit.client.1.smithi138.stdout:0/551: fdatasync d3/f45 0 2024-05-17T21:53:02.177 INFO:tasks.workunit.client.0.smithi107.stdout:0/816: read - d3/dc/d22/ff2 zero size 2024-05-17T21:53:02.177 INFO:tasks.workunit.client.1.smithi138.stdout:0/552: truncate d3/d10/f11 4413125 0 2024-05-17T21:53:02.178 INFO:tasks.workunit.client.0.smithi107.stdout:4/725: creat d3/d6/d26/d36/d37/ff0 x:0 0 0 2024-05-17T21:53:02.178 INFO:tasks.workunit.client.0.smithi107.stdout:3/735: getdents d6/d2a/d2d/da8 0 2024-05-17T21:53:02.179 INFO:tasks.workunit.client.1.smithi138.stdout:4/572: write d0/d2f/d49/d60/f51 [1163100,17866] 0 2024-05-17T21:53:02.179 INFO:tasks.workunit.client.1.smithi138.stdout:3/652: fdatasync d1/fa5 0 2024-05-17T21:53:02.181 INFO:tasks.workunit.client.1.smithi138.stdout:0/553: fsync d3/d10/f35 0 2024-05-17T21:53:02.182 INFO:tasks.workunit.client.0.smithi107.stdout:3/736: chown d6/d2a/d2d/d48/d4f/f67 1751844 1 2024-05-17T21:53:02.183 INFO:tasks.workunit.client.1.smithi138.stdout:7/492: write d0/f10 [7010624,54942] 0 2024-05-17T21:53:02.184 INFO:tasks.workunit.client.0.smithi107.stdout:7/675: dwrite d5/df/f20 [0,4194304] 0 2024-05-17T21:53:02.185 INFO:tasks.workunit.client.1.smithi138.stdout:4/573: dread - d0/d2f/d49/d60/f41 zero size 2024-05-17T21:53:02.186 INFO:tasks.workunit.client.0.smithi107.stdout:8/693: unlink dc/d21/d27/d34/d4c/d7e/c1c 0 2024-05-17T21:53:02.186 INFO:tasks.workunit.client.0.smithi107.stdout:9/740: dwrite d0/dd/d2f/d87/f98 [0,4194304] 0 2024-05-17T21:53:02.186 INFO:tasks.workunit.client.1.smithi138.stdout:4/574: read d0/d2f/d49/d60/d1f/f7a [2995972,11915] 0 2024-05-17T21:53:02.187 INFO:tasks.workunit.client.1.smithi138.stdout:4/575: dread - d0/d2f/d49/d60/d1f/d7e/f93 zero size 2024-05-17T21:53:02.188 INFO:tasks.workunit.client.0.smithi107.stdout:9/741: stat d0/fc 0 2024-05-17T21:53:02.189 INFO:tasks.workunit.client.0.smithi107.stdout:9/742: chown d0/dd/d2f/d81/dc3 1657995 1 2024-05-17T21:53:02.189 INFO:tasks.workunit.client.1.smithi138.stdout:0/554: fsync f0 0 2024-05-17T21:53:02.191 INFO:tasks.workunit.client.1.smithi138.stdout:8/467: dread d0/d2f/f41 [0,4194304] 0 2024-05-17T21:53:02.195 INFO:tasks.workunit.client.0.smithi107.stdout:7/676: dwrite d5/df/d11/d1c/d31/d39/d92/d9a/fdc [0,4194304] 0 2024-05-17T21:53:02.195 INFO:tasks.workunit.client.0.smithi107.stdout:4/726: unlink d3/db/d1e/d99/le8 0 2024-05-17T21:53:02.196 INFO:tasks.workunit.client.1.smithi138.stdout:7/493: mkdir d0/d18/d40/d9a 0 2024-05-17T21:53:02.199 INFO:tasks.workunit.client.1.smithi138.stdout:3/653: creat d1/d3/d16/d3e/d71/db2/fe0 x:0 0 0 2024-05-17T21:53:02.201 INFO:tasks.workunit.client.1.smithi138.stdout:7/494: read d0/d18/f92 [35607,109802] 0 2024-05-17T21:53:02.202 INFO:tasks.workunit.client.1.smithi138.stdout:7/495: truncate d0/f27 4613461 0 2024-05-17T21:53:02.204 INFO:tasks.workunit.client.1.smithi138.stdout:4/576: rename d0/d1b/d36/c5c to d0/d3/d1d/ccc 0 2024-05-17T21:53:02.208 INFO:tasks.workunit.client.0.smithi107.stdout:2/556: mkdir d0/d9/d1c/d22/db0 0 2024-05-17T21:53:02.209 INFO:tasks.workunit.client.1.smithi138.stdout:0/555: dread d3/d10/f7d [0,4194304] 0 2024-05-17T21:53:02.210 INFO:tasks.workunit.client.1.smithi138.stdout:2/584: link d3/d21/d2f/d3e/d63/f80 d3/d21/d2f/fc0 0 2024-05-17T21:53:02.210 INFO:tasks.workunit.client.1.smithi138.stdout:2/585: fsync d3/da0/fbb 0 2024-05-17T21:53:02.210 INFO:tasks.workunit.client.1.smithi138.stdout:0/556: read - d3/d10/d38/d70/d80/d94/fa4 zero size 2024-05-17T21:53:02.211 INFO:tasks.workunit.client.0.smithi107.stdout:6/640: getdents d9/d16/d22 0 2024-05-17T21:53:02.214 INFO:tasks.workunit.client.0.smithi107.stdout:2/557: dwrite d0/d9/d1c/d37/d9f/fa7 [0,4194304] 0 2024-05-17T21:53:02.216 INFO:tasks.workunit.client.1.smithi138.stdout:7/496: creat d0/d39/f9b x:0 0 0 2024-05-17T21:53:02.218 INFO:tasks.workunit.client.1.smithi138.stdout:8/468: rename d0/dc/d1a/f58 to d0/dc/d3b/f89 0 2024-05-17T21:53:02.219 INFO:tasks.workunit.client.0.smithi107.stdout:7/677: mknod d5/df/d11/d2a/cf1 0 2024-05-17T21:53:02.221 INFO:tasks.workunit.client.1.smithi138.stdout:4/577: creat d0/d3/d17/d21/d78/fcd x:0 0 0 2024-05-17T21:53:02.222 INFO:tasks.workunit.client.1.smithi138.stdout:7/497: dread d0/d18/d40/f82 [0,4194304] 0 2024-05-17T21:53:02.224 INFO:tasks.workunit.client.0.smithi107.stdout:1/746: rename d4/c64 to d4/df/d1a/d26/d72/dbc/cf0 0 2024-05-17T21:53:02.227 INFO:tasks.workunit.client.1.smithi138.stdout:2/586: truncate d3/d21/d2f/d8a/fb7 28202 0 2024-05-17T21:53:02.227 INFO:tasks.workunit.client.0.smithi107.stdout:3/737: creat d6/d8/d3e/d3f/dbe/fee x:0 0 0 2024-05-17T21:53:02.228 INFO:tasks.workunit.client.1.smithi138.stdout:7/498: dread d0/f1e [0,4194304] 0 2024-05-17T21:53:02.228 INFO:tasks.workunit.client.0.smithi107.stdout:3/738: stat d6/d8/d10/d25/f95 0 2024-05-17T21:53:02.229 INFO:tasks.workunit.client.1.smithi138.stdout:0/557: mkdir d3/d3d/d48/dbc 0 2024-05-17T21:53:02.229 INFO:tasks.workunit.client.1.smithi138.stdout:0/558: chown d3/d3d/f53 37924 1 2024-05-17T21:53:02.233 INFO:tasks.workunit.client.0.smithi107.stdout:6/641: rmdir d9/d16 39 2024-05-17T21:53:02.236 INFO:tasks.workunit.client.0.smithi107.stdout:9/743: mknod d0/cf7 0 2024-05-17T21:53:02.239 INFO:tasks.workunit.client.0.smithi107.stdout:0/817: write d3/dc/f37 [823906,29542] 0 2024-05-17T21:53:02.239 INFO:tasks.workunit.client.0.smithi107.stdout:0/818: fdatasync d3/f1d 0 2024-05-17T21:53:02.241 INFO:tasks.workunit.client.1.smithi138.stdout:2/587: symlink d3/d21/d2f/d3e/d55/d99/d84/d9f/lc1 0 2024-05-17T21:53:02.242 INFO:tasks.workunit.client.1.smithi138.stdout:3/654: dwrite d1/d2e/fa1 [0,4194304] 0 2024-05-17T21:53:02.243 INFO:tasks.workunit.client.1.smithi138.stdout:2/588: write d3/d21/d2f/d3e/d58/f5d [4237967,54940] 0 2024-05-17T21:53:02.244 INFO:tasks.workunit.client.1.smithi138.stdout:7/499: rmdir d0/d18/d40/d5f 39 2024-05-17T21:53:02.248 INFO:tasks.workunit.client.1.smithi138.stdout:0/559: unlink d3/d10/d38/d3e/f3f 0 2024-05-17T21:53:02.256 INFO:tasks.workunit.client.0.smithi107.stdout:9/744: write d0/dd/d21/d29/d57/fe2 [952875,85695] 0 2024-05-17T21:53:02.258 INFO:tasks.workunit.client.0.smithi107.stdout:9/745: read d0/dd/d2f/f64 [39934,16763] 0 2024-05-17T21:53:02.258 INFO:tasks.workunit.client.1.smithi138.stdout:2/589: truncate d3/d21/f2e 619214 0 2024-05-17T21:53:02.259 INFO:tasks.workunit.client.0.smithi107.stdout:0/819: dread - d3/dc/d22/d23/fb7 zero size 2024-05-17T21:53:02.260 INFO:tasks.workunit.client.1.smithi138.stdout:4/578: dwrite d0/d3/db/f20 [0,4194304] 0 2024-05-17T21:53:02.262 INFO:tasks.workunit.client.0.smithi107.stdout:4/727: creat d3/db/d1e/ff1 x:0 0 0 2024-05-17T21:53:02.266 INFO:tasks.workunit.client.1.smithi138.stdout:7/500: mkdir d0/d18/d9c 0 2024-05-17T21:53:02.269 INFO:tasks.workunit.client.0.smithi107.stdout:8/694: getdents dc/d44/d4b/d52 0 2024-05-17T21:53:02.273 INFO:tasks.workunit.client.1.smithi138.stdout:4/579: unlink d0/d1b/d2b/lb1 0 2024-05-17T21:53:02.274 INFO:tasks.workunit.client.0.smithi107.stdout:1/747: truncate d4/f21 3225711 0 2024-05-17T21:53:02.275 INFO:tasks.workunit.client.1.smithi138.stdout:8/469: link d0/d11/c85 d0/c8a 0 2024-05-17T21:53:02.277 INFO:tasks.workunit.client.1.smithi138.stdout:3/655: creat d1/d62/dad/fe1 x:0 0 0 2024-05-17T21:53:02.277 INFO:tasks.workunit.client.1.smithi138.stdout:0/560: truncate d3/d10/f20 1327728 0 2024-05-17T21:53:02.278 INFO:tasks.workunit.client.0.smithi107.stdout:2/558: truncate d0/d9/f33 1837217 0 2024-05-17T21:53:02.279 INFO:tasks.workunit.client.0.smithi107.stdout:2/559: write d0/d9/f77 [2162217,16973] 0 2024-05-17T21:53:02.282 INFO:tasks.workunit.client.1.smithi138.stdout:8/470: dwrite d0/dc/f22 [8388608,4194304] 0 2024-05-17T21:53:02.282 INFO:tasks.workunit.client.0.smithi107.stdout:4/728: creat d3/db/d35/d55/d8a/ff2 x:0 0 0 2024-05-17T21:53:02.283 INFO:tasks.workunit.client.1.smithi138.stdout:4/580: write d0/f7 [6258331,12284] 0 2024-05-17T21:53:02.285 INFO:tasks.workunit.client.1.smithi138.stdout:4/581: truncate d0/d1b/d2b/f65 1181247 0 2024-05-17T21:53:02.286 INFO:tasks.workunit.client.1.smithi138.stdout:4/582: chown d0/d3/d1d/d38/d8a/dbd 0 1 2024-05-17T21:53:02.290 INFO:tasks.workunit.client.0.smithi107.stdout:3/739: creat d6/fef x:0 0 0 2024-05-17T21:53:02.295 INFO:tasks.workunit.client.1.smithi138.stdout:2/590: write d3/d21/d2f/d3e/d55/d99/d84/f91 [1309417,9881] 0 2024-05-17T21:53:02.295 INFO:tasks.workunit.client.1.smithi138.stdout:0/561: rmdir d3/d10/d6d/d81 39 2024-05-17T21:53:02.297 INFO:tasks.workunit.client.0.smithi107.stdout:9/746: dread d0/dd/f66 [0,4194304] 0 2024-05-17T21:53:02.297 INFO:tasks.workunit.client.0.smithi107.stdout:6/642: mknod d9/d16/d4d/d85/d88/cc6 0 2024-05-17T21:53:02.299 INFO:tasks.workunit.client.1.smithi138.stdout:7/501: mkdir d0/d9d 0 2024-05-17T21:53:02.301 INFO:tasks.workunit.client.1.smithi138.stdout:7/502: dread - d0/d39/f9b zero size 2024-05-17T21:53:02.302 INFO:tasks.workunit.client.1.smithi138.stdout:2/591: dwrite d3/d21/f38 [0,4194304] 0 2024-05-17T21:53:02.302 INFO:tasks.workunit.client.0.smithi107.stdout:5/700: sync 2024-05-17T21:53:02.302 INFO:tasks.workunit.client.0.smithi107.stdout:2/560: creat d0/d9/d1c/d22/fb1 x:0 0 0 2024-05-17T21:53:02.303 INFO:tasks.workunit.client.1.smithi138.stdout:2/592: dread - d3/d21/d2f/d3e/d69/fb0 zero size 2024-05-17T21:53:02.303 INFO:tasks.workunit.client.0.smithi107.stdout:5/701: write d5/db/d16/d2d/d32/f91 [1499238,89355] 0 2024-05-17T21:53:02.307 INFO:tasks.workunit.client.1.smithi138.stdout:2/593: truncate d3/d21/d2f/d3e/d55/d99/d84/f91 1908416 0 2024-05-17T21:53:02.309 INFO:tasks.workunit.client.0.smithi107.stdout:4/729: symlink d3/db/d35/lf3 0 2024-05-17T21:53:02.311 INFO:tasks.workunit.client.0.smithi107.stdout:7/678: rename d5/df/c1f to d5/d16/dd3/cf2 0 2024-05-17T21:53:02.311 INFO:tasks.workunit.client.0.smithi107.stdout:4/730: chown d3/db/d1e/f6c 714051403 1 2024-05-17T21:53:02.314 INFO:tasks.workunit.client.1.smithi138.stdout:3/656: dwrite d1/d2e/f98 [0,4194304] 0 2024-05-17T21:53:02.315 INFO:tasks.workunit.client.0.smithi107.stdout:8/695: dwrite dc/d21/d27/f37 [0,4194304] 0 2024-05-17T21:53:02.315 INFO:tasks.workunit.client.0.smithi107.stdout:9/747: chown d0/dd/d1b/c1d 309056146 1 2024-05-17T21:53:02.319 INFO:tasks.workunit.client.1.smithi138.stdout:8/471: write d0/dc/d1a/d33/d4a/d4e/f7c [992312,21509] 0 2024-05-17T21:53:02.321 INFO:tasks.workunit.client.1.smithi138.stdout:4/583: rmdir d0/d2f/d49/d60/d1f/d32 39 2024-05-17T21:53:02.324 INFO:tasks.workunit.client.0.smithi107.stdout:0/820: dread d3/d39/f7f [0,4194304] 0 2024-05-17T21:53:02.325 INFO:tasks.workunit.client.1.smithi138.stdout:7/503: dwrite d0/d18/d40/d41/d50/f53 [0,4194304] 0 2024-05-17T21:53:02.326 INFO:tasks.workunit.client.0.smithi107.stdout:8/696: dwrite dc/d21/d97/d63/d71/f94 [0,4194304] 0 2024-05-17T21:53:02.328 INFO:tasks.workunit.client.0.smithi107.stdout:5/702: rename d5/db/d16/d2d/d51/l63 to d5/d75/dc7/le4 0 2024-05-17T21:53:02.329 INFO:tasks.workunit.client.0.smithi107.stdout:3/740: dread d6/d8/d10/f23 [4194304,4194304] 0 2024-05-17T21:53:02.330 INFO:tasks.workunit.client.0.smithi107.stdout:5/703: write d5/db/d16/d5b/f2f [1187858,51007] 0 2024-05-17T21:53:02.333 INFO:tasks.workunit.client.1.smithi138.stdout:2/594: dread d3/f4 [0,4194304] 0 2024-05-17T21:53:02.338 INFO:tasks.workunit.client.0.smithi107.stdout:7/679: symlink d5/df/d11/d1c/d2f/d65/da7/lf3 0 2024-05-17T21:53:02.338 INFO:tasks.workunit.client.0.smithi107.stdout:7/680: readlink d5/lbc 0 2024-05-17T21:53:02.344 INFO:tasks.workunit.client.0.smithi107.stdout:6/643: write d9/d16/f3b [410242,17714] 0 2024-05-17T21:53:02.345 INFO:tasks.workunit.client.0.smithi107.stdout:1/748: rename d4/df/d1a/d26/fe6 to d4/df/d1a/d26/ff1 0 2024-05-17T21:53:02.347 INFO:tasks.workunit.client.1.smithi138.stdout:7/504: fdatasync d0/d18/d40/d41/d50/f94 0 2024-05-17T21:53:02.350 INFO:tasks.workunit.client.1.smithi138.stdout:3/657: mknod d1/d3/d8d/d92/ce2 0 2024-05-17T21:53:02.351 INFO:tasks.workunit.client.0.smithi107.stdout:1/749: dwrite d4/df/d1a/f7b [0,4194304] 0 2024-05-17T21:53:02.353 INFO:tasks.workunit.client.0.smithi107.stdout:8/697: mknod dc/d21/d27/d83/cfb 0 2024-05-17T21:53:02.355 INFO:tasks.workunit.client.0.smithi107.stdout:2/561: rename d0/d9/d11/d27/l44 to d0/d9/d1c/d37/lb2 0 2024-05-17T21:53:02.355 INFO:tasks.workunit.client.1.smithi138.stdout:0/562: link d3/d10/d38/f46 d3/d10/d6d/d81/fbd 0 2024-05-17T21:53:02.361 INFO:tasks.workunit.client.0.smithi107.stdout:4/731: dread d3/db/d1e/f77 [0,4194304] 0 2024-05-17T21:53:02.362 INFO:tasks.workunit.client.0.smithi107.stdout:9/748: write d0/dd/d21/d29/d3e/d45/d68/fdc [928239,102317] 0 2024-05-17T21:53:02.362 INFO:tasks.workunit.client.1.smithi138.stdout:7/505: mkdir d0/d18/d40/d41/d51/d66/d9e 0 2024-05-17T21:53:02.363 INFO:tasks.workunit.client.1.smithi138.stdout:4/584: write d0/d2f/d49/d60/d1f/f7a [2505832,79948] 0 2024-05-17T21:53:02.363 INFO:tasks.workunit.client.0.smithi107.stdout:0/821: dread d3/dc/d22/f68 [0,4194304] 0 2024-05-17T21:53:02.364 INFO:tasks.workunit.client.1.smithi138.stdout:0/563: dwrite d3/d10/d38/d70/d80/d94/fa4 [0,4194304] 0 2024-05-17T21:53:02.364 INFO:tasks.workunit.client.0.smithi107.stdout:5/704: creat d5/db/d16/d54/d59/d92/fe5 x:0 0 0 2024-05-17T21:53:02.364 INFO:tasks.workunit.client.1.smithi138.stdout:7/506: truncate d0/f8e 331961 0 2024-05-17T21:53:02.366 INFO:tasks.workunit.client.0.smithi107.stdout:7/681: chown d5/d16/f3a 2968017 1 2024-05-17T21:53:02.372 INFO:tasks.workunit.client.0.smithi107.stdout:6/644: truncate d9/d16/d4d/f59 762490 0 2024-05-17T21:53:02.374 INFO:tasks.workunit.client.1.smithi138.stdout:0/564: symlink d3/d10/d39/lbe 0 2024-05-17T21:53:02.375 INFO:tasks.workunit.client.1.smithi138.stdout:0/565: chown d3/d3d/d48 0 1 2024-05-17T21:53:02.376 INFO:tasks.workunit.client.1.smithi138.stdout:3/658: mknod d1/d62/dad/d72/dda/ce3 0 2024-05-17T21:53:02.377 INFO:tasks.workunit.client.1.smithi138.stdout:3/659: write d1/d62/dad/f24 [66434,93222] 0 2024-05-17T21:53:02.377 INFO:tasks.workunit.client.1.smithi138.stdout:8/472: link d0/d11/c85 d0/dc/d1a/d33/d4a/c8b 0 2024-05-17T21:53:02.378 INFO:tasks.workunit.client.1.smithi138.stdout:8/473: readlink d0/dc/d1a/d33/d4a/d4e/l5b 0 2024-05-17T21:53:02.378 INFO:tasks.workunit.client.0.smithi107.stdout:2/562: creat d0/d93/fb3 x:0 0 0 2024-05-17T21:53:02.380 INFO:tasks.workunit.client.1.smithi138.stdout:3/660: mknod d1/d4f/d87/ce4 0 2024-05-17T21:53:02.383 INFO:tasks.workunit.client.0.smithi107.stdout:9/749: rename d0/dd/d21/d29/d3e/d94/fb0 to d0/dd/d21/d29/dca/ff8 0 2024-05-17T21:53:02.387 INFO:tasks.workunit.client.1.smithi138.stdout:8/474: rename d0/dc/d1a/d33/d4a/d68/f6c to d0/dc/d3b/d4d/f8c 0 2024-05-17T21:53:02.388 INFO:tasks.workunit.client.0.smithi107.stdout:6/645: mkdir d9/d16/d22/daf/dc7 0 2024-05-17T21:53:02.389 INFO:tasks.workunit.client.1.smithi138.stdout:4/585: dread d0/d1b/f18 [0,4194304] 0 2024-05-17T21:53:02.390 INFO:tasks.workunit.client.1.smithi138.stdout:3/661: mkdir d1/d4f/d80/de5 0 2024-05-17T21:53:02.391 INFO:tasks.workunit.client.0.smithi107.stdout:1/750: creat d4/df/d1a/d41/db2/dc8/ff2 x:0 0 0 2024-05-17T21:53:02.392 INFO:tasks.workunit.client.1.smithi138.stdout:7/507: getdents d0/d4c 0 2024-05-17T21:53:02.396 INFO:tasks.workunit.client.1.smithi138.stdout:8/475: mkdir d0/dc/d8d 0 2024-05-17T21:53:02.399 INFO:tasks.workunit.client.1.smithi138.stdout:7/508: dwrite d0/d18/d40/d41/f5b [0,4194304] 0 2024-05-17T21:53:02.399 INFO:tasks.workunit.client.0.smithi107.stdout:8/698: dwrite dc/d21/d97/d63/f99 [0,4194304] 0 2024-05-17T21:53:02.399 INFO:tasks.workunit.client.1.smithi138.stdout:4/586: rename d0/d3/d17/fc5 to d0/d2f/d49/d60/d1f/d7e/fce 0 2024-05-17T21:53:02.405 INFO:tasks.workunit.client.1.smithi138.stdout:3/662: creat d1/d4f/d80/fe6 x:0 0 0 2024-05-17T21:53:02.408 INFO:tasks.workunit.client.0.smithi107.stdout:4/732: rename d3/d4a/d47/d60/d3a/d41/d80/d43/lef to d3/d6/d26/d36/d37/d85/lf4 0 2024-05-17T21:53:02.409 INFO:tasks.workunit.client.0.smithi107.stdout:4/733: dread - d3/d6/fde zero size 2024-05-17T21:53:02.409 INFO:tasks.workunit.client.1.smithi138.stdout:4/587: dwrite d0/d3/d17/d21/fa7 [0,4194304] 0 2024-05-17T21:53:02.413 INFO:tasks.workunit.client.0.smithi107.stdout:5/705: symlink d5/db/d16/d2a/d73/dd0/le6 0 2024-05-17T21:53:02.413 INFO:tasks.workunit.client.0.smithi107.stdout:4/734: dread d3/d4a/f3f [0,4194304] 0 2024-05-17T21:53:02.414 INFO:tasks.workunit.client.0.smithi107.stdout:4/735: chown d3/ddf 89817526 1 2024-05-17T21:53:02.416 INFO:tasks.workunit.client.0.smithi107.stdout:6/646: creat d9/d58/d7b/da3/fc8 x:0 0 0 2024-05-17T21:53:02.416 INFO:tasks.workunit.client.0.smithi107.stdout:2/563: write d0/d9/d1c/d37/f50 [881585,14319] 0 2024-05-17T21:53:02.418 INFO:tasks.workunit.client.1.smithi138.stdout:3/663: mkdir d1/d4f/d80/de7 0 2024-05-17T21:53:02.419 INFO:tasks.workunit.client.0.smithi107.stdout:6/647: dread d9/d16/d22/d2d/f5c [0,4194304] 0 2024-05-17T21:53:02.419 INFO:tasks.workunit.client.0.smithi107.stdout:3/741: getdents d6/d8/d3e/dc9 0 2024-05-17T21:53:02.422 INFO:tasks.workunit.client.1.smithi138.stdout:8/476: fdatasync d0/ff 0 2024-05-17T21:53:02.423 INFO:tasks.workunit.client.0.smithi107.stdout:8/699: creat dc/d21/d69/ffc x:0 0 0 2024-05-17T21:53:02.426 INFO:tasks.workunit.client.0.smithi107.stdout:0/822: rename d3/d75/dc1/fc4 to d3/d75/d9e/d103/d10d/f111 0 2024-05-17T21:53:02.427 INFO:tasks.workunit.client.1.smithi138.stdout:3/664: rmdir d1/d62 39 2024-05-17T21:53:02.428 INFO:tasks.workunit.client.1.smithi138.stdout:3/665: dread - d1/fc3 zero size 2024-05-17T21:53:02.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:02.436+0000 7f01d5bd1700 1 -- 172.21.15.107:0/4293648570 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f01d01138d0 con 0x7f01d00fce70 2024-05-17T21:53:02.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:02.436+0000 7f01b7fff700 1 -- 172.21.15.107:0/4293648570 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f01c0059f70 con 0x7f01d00fce70 2024-05-17T21:53:02.429 INFO:teuthology.orchestra.run.smithi107.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-17T21:53:02.429 INFO:teuthology.orchestra.run.smithi107.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-17T21:53:02.429 INFO:teuthology.orchestra.run.smithi107.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-17T21:53:02.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:02.439+0000 7f01d5bd1700 1 -- 172.21.15.107:0/4293648570 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7f01b8070bc0 msgr2=0x7f01b8073070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:53:02.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:02.439+0000 7f01d5bd1700 1 --2- 172.21.15.107:0/4293648570 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7f01b8070bc0 0x7f01b8073070 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f01c4004600 tx=0x7f01c400b040 comp rx=0 tx=0).stop 2024-05-17T21:53:02.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:02.439+0000 7f01d5bd1700 1 -- 172.21.15.107:0/4293648570 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01d00fce70 msgr2=0x7f01d010d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:53:02.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:02.439+0000 7f01d5bd1700 1 --2- 172.21.15.107:0/4293648570 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01d00fce70 0x7f01d010d8b0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f01c0013870 tx=0x7f01c0009e30 comp rx=0 tx=0).stop 2024-05-17T21:53:02.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:02.439+0000 7f01d5bd1700 1 -- 172.21.15.107:0/4293648570 shutdown_connections 2024-05-17T21:53:02.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:02.439+0000 7f01d5bd1700 1 --2- 172.21.15.107:0/4293648570 >> [v2:172.21.15.138:6828/1369055740,v1:172.21.15.138:6829/1369055740] conn(0x7f01b8070bc0 0x7f01b8073070 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:53:02.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:02.439+0000 7f01d5bd1700 1 --2- 172.21.15.107:0/4293648570 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01d00fce70 0x7f01d010d8b0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:53:02.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:02.439+0000 7f01d5bd1700 1 --2- 172.21.15.107:0/4293648570 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f01d00fdfe0 0x7f01d010ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:53:02.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:02.439+0000 7f01d5bd1700 1 -- 172.21.15.107:0/4293648570 >> 172.21.15.107:0/4293648570 conn(0x7f01d00f8550 msgr2=0x7f01d00f9b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:53:02.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:02.439+0000 7f01d5bd1700 1 -- 172.21.15.107:0/4293648570 shutdown_connections 2024-05-17T21:53:02.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:02.439+0000 7f01d5bd1700 1 -- 172.21.15.107:0/4293648570 wait complete. 2024-05-17T21:53:02.433 INFO:tasks.workunit.client.1.smithi138.stdout:6/562: sync 2024-05-17T21:53:02.433 INFO:tasks.workunit.client.1.smithi138.stdout:5/540: sync 2024-05-17T21:53:02.434 INFO:tasks.workunit.client.0.smithi107.stdout:4/736: rmdir d3/d6/d26 39 2024-05-17T21:53:02.436 INFO:tasks.workunit.client.1.smithi138.stdout:3/666: symlink d1/d3/d49/ddb/le8 0 2024-05-17T21:53:02.437 INFO:tasks.workunit.client.0.smithi107.stdout:2/564: creat d0/d9/d11/df/fb4 x:0 0 0 2024-05-17T21:53:02.438 INFO:tasks.workunit.client.1.smithi138.stdout:4/588: link d0/d3/d17/d5e/d7f/l85 d0/d1b/d42/lcf 0 2024-05-17T21:53:02.438 INFO:tasks.workunit.client.0.smithi107.stdout:2/565: chown d0/d9/d1c/d3e/l7c 21 1 2024-05-17T21:53:02.441 INFO:tasks.workunit.client.1.smithi138.stdout:5/541: mkdir d1/d53/db1 0 2024-05-17T21:53:02.442 INFO:tasks.workunit.client.0.smithi107.stdout:6/648: mkdir d9/d16/d4d/db8/dc9 0 2024-05-17T21:53:02.444 INFO:tasks.workunit.client.0.smithi107.stdout:3/742: symlink d6/d8/d10/d35/d36/d4c/dcc/lf0 0 2024-05-17T21:53:02.444 INFO:tasks.workunit.client.1.smithi138.stdout:4/589: dwrite d0/fcb [0,4194304] 0 2024-05-17T21:53:02.445 INFO:tasks.workunit.client.1.smithi138.stdout:7/509: getdents d0/d18/d40/d41/d50 0 2024-05-17T21:53:02.445 INFO:tasks.workunit.client.0.smithi107.stdout:5/706: dwrite d5/db/d16/d2d/d40/d5a/fa7 [0,4194304] 0 2024-05-17T21:53:02.445 INFO:tasks.workunit.client.0.smithi107.stdout:8/700: dread - dc/d21/d68/fae zero size 2024-05-17T21:53:02.447 INFO:tasks.workunit.client.0.smithi107.stdout:5/707: write d5/db/d16/d54/d59/d65/d48/f8a [3273667,33059] 0 2024-05-17T21:53:02.448 INFO:tasks.workunit.client.1.smithi138.stdout:8/477: dwrite d0/dc/f10 [4194304,4194304] 0 2024-05-17T21:53:02.448 INFO:tasks.workunit.client.1.smithi138.stdout:3/667: rename d1/d3/d49/ddb/c52 to d1/d4f/d50/dbb/ce9 0 2024-05-17T21:53:02.451 INFO:tasks.workunit.client.1.smithi138.stdout:3/668: fsync d1/d3/d16/d3e/f4a 0 2024-05-17T21:53:02.452 INFO:tasks.workunit.client.1.smithi138.stdout:3/669: fsync d1/d3/d16/f95 0 2024-05-17T21:53:02.457 INFO:tasks.workunit.client.1.smithi138.stdout:5/542: dread - d1/d2c/f35 zero size 2024-05-17T21:53:02.460 INFO:tasks.workunit.client.1.smithi138.stdout:3/670: read - d1/d3/d16/d3e/d71/f9a zero size 2024-05-17T21:53:02.461 INFO:tasks.workunit.client.0.smithi107.stdout:2/566: read d0/d26/f6c [716556,19208] 0 2024-05-17T21:53:02.462 INFO:tasks.workunit.client.0.smithi107.stdout:2/567: chown d0/d9/d1c 0 1 2024-05-17T21:53:02.462 INFO:tasks.workunit.client.1.smithi138.stdout:3/671: write d1/d3/d8d/fb7 [933768,111167] 0 2024-05-17T21:53:02.463 INFO:tasks.workunit.client.1.smithi138.stdout:6/563: creat d7/db/d43/fc4 x:0 0 0 2024-05-17T21:53:02.465 INFO:tasks.workunit.client.1.smithi138.stdout:5/543: dwrite d1/d6/fe [0,4194304] 0 2024-05-17T21:53:02.465 INFO:tasks.workunit.client.0.smithi107.stdout:6/649: creat d9/d16/d4d/d67/fca x:0 0 0 2024-05-17T21:53:02.466 INFO:tasks.workunit.client.1.smithi138.stdout:6/564: readlink d7/db/d43/d5f/d6a/d9b/lb3 0 2024-05-17T21:53:02.466 INFO:tasks.workunit.client.0.smithi107.stdout:3/743: fsync d6/d2a/d2d/f43 0 2024-05-17T21:53:02.469 INFO:tasks.workunit.client.1.smithi138.stdout:3/672: creat d1/d4d/fea x:0 0 0 2024-05-17T21:53:02.471 INFO:tasks.workunit.client.1.smithi138.stdout:7/510: creat d0/d18/d64/d7c/f9f x:0 0 0 2024-05-17T21:53:02.472 INFO:tasks.workunit.client.0.smithi107.stdout:6/650: read d9/d16/f41 [105985,117372] 0 2024-05-17T21:53:02.473 INFO:tasks.workunit.client.0.smithi107.stdout:5/708: mknod d5/db/d16/d2d/d40/d9f/dc9/ce7 0 2024-05-17T21:53:02.477 INFO:tasks.workunit.client.0.smithi107.stdout:7/682: rename d5/l3f to d5/df/d11/d1d/d44/lf4 0 2024-05-17T21:53:02.479 INFO:tasks.workunit.client.1.smithi138.stdout:7/511: dwrite d0/d18/d40/f44 [4194304,4194304] 0 2024-05-17T21:53:02.479 INFO:tasks.workunit.client.0.smithi107.stdout:7/683: write d5/df/d11/d1c/d31/d39/d92/d9a/fc2 [772866,19719] 0 2024-05-17T21:53:02.486 INFO:tasks.workunit.client.1.smithi138.stdout:6/565: creat d7/db/d43/d3f/db6/fc5 x:0 0 0 2024-05-17T21:53:02.488 INFO:tasks.workunit.client.1.smithi138.stdout:8/478: creat d0/dc/d1a/d51/d82/f8e x:0 0 0 2024-05-17T21:53:02.489 INFO:tasks.workunit.client.0.smithi107.stdout:8/701: creat dc/d21/d27/d34/df5/ffd x:0 0 0 2024-05-17T21:53:02.490 INFO:tasks.workunit.client.0.smithi107.stdout:3/744: mknod d6/d8/d10/d35/d36/d4c/cf1 0 2024-05-17T21:53:02.491 INFO:tasks.workunit.client.0.smithi107.stdout:8/702: stat dc/d21/d97/l48 0 2024-05-17T21:53:02.491 INFO:tasks.workunit.client.0.smithi107.stdout:4/737: dwrite d3/d4a/d47/d60/f10 [0,4194304] 0 2024-05-17T21:53:02.493 INFO:tasks.workunit.client.1.smithi138.stdout:6/566: fdatasync d7/db/f25 0 2024-05-17T21:53:02.494 INFO:tasks.workunit.client.0.smithi107.stdout:3/745: truncate d6/d8/d10/d25/fda 989813 0 2024-05-17T21:53:02.495 INFO:tasks.workunit.client.0.smithi107.stdout:3/746: write d6/d8/d10/d25/f47 [2664814,48382] 0 2024-05-17T21:53:02.499 INFO:tasks.workunit.client.0.smithi107.stdout:9/750: rename d0/dd/d1f/l3a to d0/dd/d21/dcd/lf9 0 2024-05-17T21:53:02.505 INFO:tasks.workunit.client.0.smithi107.stdout:1/751: link d4/df/d1a/d26/d7a/f93 d4/df/ff3 0 2024-05-17T21:53:02.506 INFO:tasks.workunit.client.1.smithi138.stdout:8/479: dread d0/dc/f2b [0,4194304] 0 2024-05-17T21:53:02.506 INFO:tasks.workunit.client.0.smithi107.stdout:9/751: dwrite d0/dd/d21/d29/d3e/fc4 [0,4194304] 0 2024-05-17T21:53:02.508 INFO:tasks.workunit.client.0.smithi107.stdout:9/752: write d0/dd/d21/d29/d3e/fc7 [944499,39201] 0 2024-05-17T21:53:02.509 INFO:tasks.workunit.client.1.smithi138.stdout:5/544: truncate d1/d6/d1b/d29/d36/d68/f6f 3928281 0 2024-05-17T21:53:02.509 INFO:tasks.workunit.client.1.smithi138.stdout:5/545: readlink d1/d60/l44 0 2024-05-17T21:53:02.510 INFO:tasks.workunit.client.0.smithi107.stdout:6/651: write d9/d16/d22/d2d/f54 [149655,38720] 0 2024-05-17T21:53:02.512 INFO:tasks.workunit.client.1.smithi138.stdout:7/512: getdents d0/d18/d40 0 2024-05-17T21:53:02.514 INFO:tasks.workunit.client.0.smithi107.stdout:8/703: creat dc/d44/d4b/d52/ffe x:0 0 0 2024-05-17T21:53:02.515 INFO:tasks.workunit.client.1.smithi138.stdout:8/480: mknod d0/d11/c8f 0 2024-05-17T21:53:02.515 INFO:tasks.workunit.client.0.smithi107.stdout:8/704: dread - dc/d21/d97/db7/fcc zero size 2024-05-17T21:53:02.516 INFO:tasks.workunit.client.0.smithi107.stdout:1/752: dread d4/df/d1a/fc4 [0,4194304] 0 2024-05-17T21:53:02.518 INFO:tasks.workunit.client.1.smithi138.stdout:5/546: rmdir d1/d60/d2f 39 2024-05-17T21:53:02.520 INFO:tasks.workunit.client.1.smithi138.stdout:7/513: symlink d0/d18/d40/d41/d51/la0 0 2024-05-17T21:53:02.520 INFO:tasks.workunit.client.1.smithi138.stdout:8/481: dread d0/f1b [0,4194304] 0 2024-05-17T21:53:02.521 INFO:tasks.workunit.client.0.smithi107.stdout:3/747: mkdir d6/d8/d10/d35/d36/d4c/df2 0 2024-05-17T21:53:02.522 INFO:tasks.workunit.client.1.smithi138.stdout:6/567: dwrite d7/db/f52 [4194304,4194304] 0 2024-05-17T21:53:02.522 INFO:tasks.workunit.client.1.smithi138.stdout:8/482: stat d0/dc/d1a/l6b 0 2024-05-17T21:53:02.524 INFO:tasks.workunit.client.0.smithi107.stdout:0/823: rename d3/d24/d6f/f9d to d3/d75/dd0/f112 0 2024-05-17T21:53:02.524 INFO:tasks.workunit.client.1.smithi138.stdout:8/483: write d0/dc/d1a/f1d [4333680,98912] 0 2024-05-17T21:53:02.525 INFO:tasks.workunit.client.0.smithi107.stdout:0/824: dread d3/d39/f7f [0,4194304] 0 2024-05-17T21:53:02.526 INFO:tasks.workunit.client.1.smithi138.stdout:8/484: read - d0/dc/d1a/d51/f80 zero size 2024-05-17T21:53:02.529 INFO:tasks.workunit.client.0.smithi107.stdout:4/738: write d3/d4a/d47/d60/f12 [3131727,111547] 0 2024-05-17T21:53:02.534 INFO:tasks.workunit.client.0.smithi107.stdout:2/568: link d0/d26/d64/f8d d0/d9/d1c/fb5 0 2024-05-17T21:53:02.535 INFO:tasks.workunit.client.1.smithi138.stdout:6/568: dread d7/db/f82 [0,4194304] 0 2024-05-17T21:53:02.536 INFO:tasks.workunit.client.0.smithi107.stdout:9/753: creat d0/dd/d21/daf/ffa x:0 0 0 2024-05-17T21:53:02.538 INFO:tasks.workunit.client.1.smithi138.stdout:7/514: creat d0/d18/d40/d41/d51/d66/d9e/fa1 x:0 0 0 2024-05-17T21:53:02.542 INFO:tasks.workunit.client.0.smithi107.stdout:6/652: mknod d9/d58/d7b/da3/ccb 0 2024-05-17T21:53:02.542 INFO:tasks.workunit.client.0.smithi107.stdout:1/753: mknod d4/df/d14/d2e/ddf/cf4 0 2024-05-17T21:53:02.542 INFO:tasks.workunit.client.1.smithi138.stdout:6/569: symlink d7/db/d43/d5f/da4/lc6 0 2024-05-17T21:53:02.543 INFO:tasks.workunit.client.0.smithi107.stdout:8/705: truncate dc/d21/d69/f8e 5233821 0 2024-05-17T21:53:02.546 INFO:tasks.workunit.client.1.smithi138.stdout:6/570: dread d7/db/f52 [4194304,4194304] 0 2024-05-17T21:53:02.552 INFO:tasks.workunit.client.1.smithi138.stdout:8/485: dread d0/dc/d3b/d4d/d7d/f7f [0,4194304] 0 2024-05-17T21:53:02.553 INFO:tasks.workunit.client.1.smithi138.stdout:8/486: write d0/fb [8885384,126646] 0 2024-05-17T21:53:02.554 INFO:tasks.workunit.client.1.smithi138.stdout:6/571: symlink d7/db/d43/d5f/da4/lc7 0 2024-05-17T21:53:02.554 INFO:tasks.workunit.client.0.smithi107.stdout:0/825: dread d3/d24/d6f/f8b [0,4194304] 0 2024-05-17T21:53:02.558 INFO:tasks.workunit.client.0.smithi107.stdout:2/569: rename d0/d9/d1c/d3e/f6b to d0/d9/d11/df/d5a/d6d/fb6 0 2024-05-17T21:53:02.560 INFO:tasks.workunit.client.1.smithi138.stdout:8/487: dread d0/dc/d1a/f2a [0,4194304] 0 2024-05-17T21:53:02.562 INFO:tasks.workunit.client.1.smithi138.stdout:7/515: truncate d0/d18/d40/d41/f77 1850561 0 2024-05-17T21:53:02.562 INFO:tasks.workunit.client.1.smithi138.stdout:8/488: dread d0/d2f/f6e [0,4194304] 0 2024-05-17T21:53:02.563 INFO:tasks.workunit.client.1.smithi138.stdout:8/489: stat d0/dc/d1a 0 2024-05-17T21:53:02.564 INFO:tasks.workunit.client.1.smithi138.stdout:8/490: dread - d0/dc/d3b/d4d/f77 zero size 2024-05-17T21:53:02.566 INFO:tasks.workunit.client.0.smithi107.stdout:1/754: creat d4/df/d14/d1d/ff5 x:0 0 0 2024-05-17T21:53:02.568 INFO:tasks.workunit.client.0.smithi107.stdout:5/709: getdents d5/db/d16/d2d/d4d 0 2024-05-17T21:53:02.569 INFO:tasks.workunit.client.0.smithi107.stdout:5/710: write d5/d75/fae [370677,34524] 0 2024-05-17T21:53:02.569 INFO:tasks.workunit.client.0.smithi107.stdout:9/754: write d0/dd/d21/d29/d3e/d45/f67 [949998,108007] 0 2024-05-17T21:53:02.573 INFO:tasks.workunit.client.1.smithi138.stdout:7/516: rename d0/d18/d40/d41/d51/d81/d84 to d0/d18/d64/d7c/da2 0 2024-05-17T21:53:02.574 INFO:tasks.workunit.client.0.smithi107.stdout:7/684: getdents d5/df/d11/d1c/d31/d39/d92 0 2024-05-17T21:53:02.577 INFO:tasks.workunit.client.0.smithi107.stdout:4/739: rmdir d3/d6/d26/d3b 39 2024-05-17T21:53:02.578 INFO:tasks.workunit.client.0.smithi107.stdout:4/740: read - d3/db/d1e/ff1 zero size 2024-05-17T21:53:02.578 INFO:tasks.workunit.client.0.smithi107.stdout:6/653: dwrite d9/f1d [0,4194304] 0 2024-05-17T21:53:02.579 INFO:tasks.workunit.client.1.smithi138.stdout:8/491: symlink d0/dc/d3b/d4d/d7d/l90 0 2024-05-17T21:53:02.582 INFO:tasks.workunit.client.1.smithi138.stdout:7/517: creat d0/d18/d40/d41/d51/d81/fa3 x:0 0 0 2024-05-17T21:53:02.582 INFO:tasks.workunit.client.1.smithi138.stdout:7/518: stat d0/f2a 0 2024-05-17T21:53:02.583 INFO:tasks.workunit.client.1.smithi138.stdout:7/519: fsync d0/f10 0 2024-05-17T21:53:02.584 INFO:tasks.workunit.client.1.smithi138.stdout:8/492: fsync d0/dc/d3b/f89 0 2024-05-17T21:53:02.586 INFO:tasks.workunit.client.0.smithi107.stdout:8/706: rename dc/d44/d4b/d52/dcb/ff1 to dc/d21/d27/d2f/dba/dee/fff 0 2024-05-17T21:53:02.588 INFO:tasks.workunit.client.1.smithi138.stdout:6/572: dwrite d7/db/d43/d5f/d6a/d77/f85 [0,4194304] 0 2024-05-17T21:53:02.590 INFO:tasks.workunit.client.1.smithi138.stdout:8/493: write d0/dc/f22 [9367220,110676] 0 2024-05-17T21:53:02.591 INFO:tasks.workunit.client.0.smithi107.stdout:1/755: fdatasync d4/df/ff3 0 2024-05-17T21:53:02.592 INFO:tasks.workunit.client.0.smithi107.stdout:3/748: link d6/d8/d10/d35/fb0 d6/d8/d10/ff3 0 2024-05-17T21:53:02.594 INFO:tasks.workunit.client.0.smithi107.stdout:2/570: dwrite d0/d9/d11/df/d5a/d6d/fb6 [0,4194304] 0 2024-05-17T21:53:02.594 INFO:tasks.workunit.client.0.smithi107.stdout:9/755: chown d0/dd/d3d/f8e 30532 1 2024-05-17T21:53:02.595 INFO:tasks.workunit.client.0.smithi107.stdout:2/571: readlink d0/d9/d1c/l66 0 2024-05-17T21:53:02.597 INFO:tasks.workunit.client.0.smithi107.stdout:5/711: mkdir d5/db/dc/d1e/de8 0 2024-05-17T21:53:02.598 INFO:tasks.workunit.client.0.smithi107.stdout:7/685: mknod d5/df/d11/d1c/d2f/d53/cf5 0 2024-05-17T21:53:02.598 INFO:tasks.workunit.client.0.smithi107.stdout:0/826: truncate d3/d39/fb0 1016043 0 2024-05-17T21:53:02.599 INFO:tasks.workunit.client.0.smithi107.stdout:8/707: write dc/d21/d69/f77 [4056510,30514] 0 2024-05-17T21:53:02.602 INFO:tasks.workunit.client.0.smithi107.stdout:4/741: symlink d3/d4a/d47/d60/lf5 0 2024-05-17T21:53:02.602 INFO:tasks.workunit.client.0.smithi107.stdout:7/686: dread - d5/df/d11/d1c/d2f/d53/fe0 zero size 2024-05-17T21:53:02.604 INFO:tasks.workunit.client.0.smithi107.stdout:4/742: dread - d3/db/d1e/dc6/fea zero size 2024-05-17T21:53:02.607 INFO:tasks.workunit.client.0.smithi107.stdout:8/708: symlink dc/d21/daa/dc5/l100 0 2024-05-17T21:53:02.608 INFO:tasks.workunit.client.0.smithi107.stdout:9/756: symlink d0/dd/d2f/d4a/de6/lfb 0 2024-05-17T21:53:02.610 INFO:tasks.workunit.client.0.smithi107.stdout:1/756: getdents d4/d6a 0 2024-05-17T21:53:02.610 INFO:tasks.workunit.client.0.smithi107.stdout:2/572: dwrite d0/d9/d11/f52 [4194304,4194304] 0 2024-05-17T21:53:02.610 INFO:tasks.workunit.client.0.smithi107.stdout:8/709: fdatasync dc/f62 0 2024-05-17T21:53:02.613 INFO:tasks.workunit.client.0.smithi107.stdout:4/743: mkdir d3/d6/d26/d3b/d91/df6 0 2024-05-17T21:53:02.613 INFO:tasks.workunit.client.0.smithi107.stdout:5/712: link d5/db/dc/d1e/d6e/le1 d5/db/d16/d2d/d40/d9f/dc9/le9 0 2024-05-17T21:53:02.614 INFO:tasks.workunit.client.0.smithi107.stdout:9/757: chown d0/dd/d2f/d87/f98 3290 1 2024-05-17T21:53:02.614 INFO:tasks.workunit.client.0.smithi107.stdout:2/573: chown d0/d26/d64 42768825 1 2024-05-17T21:53:02.617 INFO:tasks.workunit.client.0.smithi107.stdout:9/758: dread d0/dd/d21/dcd/fe5 [0,4194304] 0 2024-05-17T21:53:02.617 INFO:tasks.workunit.client.0.smithi107.stdout:2/574: chown d0/d9/d11/d15/c4b 1 1 2024-05-17T21:53:02.619 INFO:tasks.workunit.client.0.smithi107.stdout:5/713: mknod d5/db/d16/d2d/d40/d9f/dc9/cea 0 2024-05-17T21:53:02.622 INFO:tasks.workunit.client.0.smithi107.stdout:9/759: creat d0/dd/d21/d29/d3e/d45/d68/d75/ffc x:0 0 0 2024-05-17T21:53:02.622 INFO:tasks.workunit.client.0.smithi107.stdout:2/575: creat d0/d9/d11/d18/dad/fb7 x:0 0 0 2024-05-17T21:53:02.622 INFO:tasks.workunit.client.0.smithi107.stdout:1/757: dwrite d4/df/d1a/d41/fe5 [0,4194304] 0 2024-05-17T21:53:02.623 INFO:tasks.workunit.client.0.smithi107.stdout:5/714: symlink d5/db/d16/d2d/d40/d5a/db4/leb 0 2024-05-17T21:53:02.624 INFO:tasks.workunit.client.1.smithi138.stdout:1/544: sync 2024-05-17T21:53:02.625 INFO:tasks.workunit.client.1.smithi138.stdout:7/520: rename d0/le to d0/d18/d64/d7c/la4 0 2024-05-17T21:53:02.633 INFO:tasks.workunit.client.0.smithi107.stdout:5/715: dwrite d5/db/d16/d2d/d40/d5a/fa7 [0,4194304] 0 2024-05-17T21:53:02.634 INFO:tasks.workunit.client.0.smithi107.stdout:5/716: fsync d5/d5f/fb3 0 2024-05-17T21:53:02.634 INFO:tasks.workunit.client.1.smithi138.stdout:7/521: unlink d0/cb 0 2024-05-17T21:53:02.634 INFO:tasks.workunit.client.1.smithi138.stdout:7/522: fdatasync d0/f19 0 2024-05-17T21:53:02.637 INFO:tasks.workunit.client.0.smithi107.stdout:1/758: getdents d4/df/d69 0 2024-05-17T21:53:02.638 INFO:tasks.workunit.client.0.smithi107.stdout:1/759: fsync d4/df/d1a/d26/f7c 0 2024-05-17T21:53:02.639 INFO:tasks.workunit.client.0.smithi107.stdout:5/717: mkdir d5/dec 0 2024-05-17T21:53:02.641 INFO:tasks.workunit.client.1.smithi138.stdout:1/545: getdents d4/df/d3a/d3c/d67 0 2024-05-17T21:53:02.644 INFO:tasks.workunit.client.1.smithi138.stdout:7/523: getdents d0/d18/d40/d9a 0 2024-05-17T21:53:02.645 INFO:tasks.workunit.client.0.smithi107.stdout:5/718: dread d5/d75/fae [0,4194304] 0 2024-05-17T21:53:02.648 INFO:tasks.workunit.client.0.smithi107.stdout:5/719: readlink d5/db/d16/d2d/l42 0 2024-05-17T21:53:02.650 INFO:tasks.workunit.client.1.smithi138.stdout:7/524: link d0/d18/d40/d41/d50/f53 d0/d18/d40/d9a/fa5 0 2024-05-17T21:53:02.651 INFO:tasks.workunit.client.0.smithi107.stdout:5/720: read d5/db/d16/d2d/d4d/d6d/f77 [4117161,59729] 0 2024-05-17T21:53:02.652 INFO:tasks.workunit.client.0.smithi107.stdout:2/576: dread d0/d9/d11/f10 [0,4194304] 0 2024-05-17T21:53:02.656 INFO:tasks.workunit.client.1.smithi138.stdout:7/525: rename d0/l28 to d0/la6 0 2024-05-17T21:53:02.657 INFO:tasks.workunit.client.0.smithi107.stdout:6/654: dread d9/d16/d22/d2d/f63 [0,4194304] 0 2024-05-17T21:53:02.658 INFO:tasks.workunit.client.1.smithi138.stdout:7/526: creat d0/d18/d40/d41/d50/fa7 x:0 0 0 2024-05-17T21:53:02.660 INFO:tasks.workunit.client.0.smithi107.stdout:6/655: symlink d9/d16/d22/lcc 0 2024-05-17T21:53:02.661 INFO:tasks.workunit.client.1.smithi138.stdout:7/527: mkdir d0/d18/d40/d41/d51/d66/da8 0 2024-05-17T21:53:02.665 INFO:tasks.workunit.client.1.smithi138.stdout:6/573: write d7/d8/f50 [694682,106975] 0 2024-05-17T21:53:02.667 INFO:tasks.workunit.client.0.smithi107.stdout:6/656: link d9/d16/d4d/db8/fba d9/d81/fcd 0 2024-05-17T21:53:02.668 INFO:tasks.workunit.client.1.smithi138.stdout:7/528: dwrite d0/d18/f57 [0,4194304] 0 2024-05-17T21:53:02.673 INFO:tasks.workunit.client.1.smithi138.stdout:6/574: rename d7/db to d7/d69/db5/dc8 0 2024-05-17T21:53:02.677 INFO:tasks.workunit.client.0.smithi107.stdout:1/760: dread d4/f66 [0,4194304] 0 2024-05-17T21:53:02.679 INFO:tasks.workunit.client.1.smithi138.stdout:6/575: mkdir d7/d69/db5/dc8/d43/d5f/d94/dc9 0 2024-05-17T21:53:02.682 INFO:tasks.workunit.client.1.smithi138.stdout:7/529: truncate d0/f91 586620 0 2024-05-17T21:53:02.682 INFO:tasks.workunit.client.0.smithi107.stdout:3/749: write d6/d2a/d2d/da8/daa/fe0 [4959490,105203] 0 2024-05-17T21:53:02.683 INFO:tasks.workunit.client.1.smithi138.stdout:6/576: creat d7/d69/db5/dc8/d43/d5f/da4/db0/fca x:0 0 0 2024-05-17T21:53:02.684 INFO:tasks.workunit.client.0.smithi107.stdout:0/827: dread d3/dc/f14 [0,4194304] 0 2024-05-17T21:53:02.684 INFO:tasks.workunit.client.1.smithi138.stdout:6/577: dread - d7/d8/d78/fbd zero size 2024-05-17T21:53:02.686 INFO:tasks.workunit.client.1.smithi138.stdout:6/578: write d7/d69/db5/dc8/d43/d3f/d79/f93 [545077,45921] 0 2024-05-17T21:53:02.687 INFO:tasks.workunit.client.1.smithi138.stdout:6/579: write d7/d8/fba [4763152,87469] 0 2024-05-17T21:53:02.687 INFO:tasks.workunit.client.0.smithi107.stdout:3/750: fdatasync d6/d8/d3e/d3f/d90/f9f 0 2024-05-17T21:53:02.688 INFO:tasks.workunit.client.0.smithi107.stdout:3/751: write d6/d2a/d2d/da8/daa/fe0 [5250158,29366] 0 2024-05-17T21:53:02.689 INFO:tasks.workunit.client.1.smithi138.stdout:6/580: write d7/d69/db5/dc8/d43/d3f/d79/fbb [61962,116516] 0 2024-05-17T21:53:02.690 INFO:tasks.workunit.client.1.smithi138.stdout:6/581: stat d7/d69/db5/dc8/c31 0 2024-05-17T21:53:02.692 INFO:tasks.workunit.client.0.smithi107.stdout:0/828: write d3/d58/faa [5690041,85754] 0 2024-05-17T21:53:02.692 INFO:tasks.workunit.client.0.smithi107.stdout:7/687: write d5/d16/d3b/d45/d91/fc7 [1023595,66306] 0 2024-05-17T21:53:02.694 INFO:tasks.workunit.client.0.smithi107.stdout:7/688: dread - d5/d16/d3b/d45/d91/dc1/de6/feb zero size 2024-05-17T21:53:02.694 INFO:tasks.workunit.client.0.smithi107.stdout:4/744: truncate d3/d4a/d47/d60/d3a/f40 2340093 0 2024-05-17T21:53:02.696 INFO:tasks.workunit.client.1.smithi138.stdout:6/582: dwrite d7/d8/d78/fbd [0,4194304] 0 2024-05-17T21:53:02.697 INFO:tasks.workunit.client.0.smithi107.stdout:3/752: rename d6/d8/f14 to d6/d8/d3e/d3f/d90/ff4 0 2024-05-17T21:53:02.699 INFO:tasks.workunit.client.1.smithi138.stdout:6/583: write d7/d69/db5/dc8/d43/d5f/da4/db0/fca [692087,1682] 0 2024-05-17T21:53:02.700 INFO:tasks.workunit.client.0.smithi107.stdout:9/760: write d0/dd/d1b/f53 [927382,2144] 0 2024-05-17T21:53:02.701 INFO:tasks.workunit.client.0.smithi107.stdout:9/761: chown d0/cf7 1641 1 2024-05-17T21:53:02.703 INFO:tasks.workunit.client.0.smithi107.stdout:4/745: mkdir d3/db/d1e/d99/df7 0 2024-05-17T21:53:02.704 INFO:tasks.workunit.client.0.smithi107.stdout:7/689: dread - d5/df/d11/d1c/d2f/d65/da7/fad zero size 2024-05-17T21:53:02.704 INFO:tasks.workunit.client.0.smithi107.stdout:7/690: chown d5/df/fc9 46865809 1 2024-05-17T21:53:02.705 INFO:tasks.workunit.client.1.smithi138.stdout:8/494: write d0/dc/d1a/f4b [1451359,42501] 0 2024-05-17T21:53:02.711 INFO:tasks.workunit.client.0.smithi107.stdout:3/753: dread d6/d8/d10/f38 [0,4194304] 0 2024-05-17T21:53:02.714 INFO:tasks.workunit.client.1.smithi138.stdout:1/546: dwrite d4/df/f1b [0,4194304] 0 2024-05-17T21:53:02.716 INFO:tasks.workunit.client.1.smithi138.stdout:1/547: write d4/df/fa8 [832205,27837] 0 2024-05-17T21:53:02.717 INFO:tasks.workunit.client.0.smithi107.stdout:9/762: dread d0/dd/d21/d29/d3e/d45/f61 [0,4194304] 0 2024-05-17T21:53:02.719 INFO:tasks.workunit.client.0.smithi107.stdout:9/763: write d0/dd/d1f/f40 [4263359,281] 0 2024-05-17T21:53:02.720 INFO:tasks.workunit.client.0.smithi107.stdout:4/746: creat d3/db/d1e/d99/df7/ff8 x:0 0 0 2024-05-17T21:53:02.721 INFO:tasks.workunit.client.0.smithi107.stdout:5/721: dwrite d5/db/d16/d54/d59/f7f [0,4194304] 0 2024-05-17T21:53:02.725 INFO:tasks.workunit.client.1.smithi138.stdout:1/548: dread d4/df/d4e/f55 [0,4194304] 0 2024-05-17T21:53:02.739 INFO:tasks.workunit.client.0.smithi107.stdout:1/761: dwrite d4/df/d4e/d59/fe3 [4194304,4194304] 0 2024-05-17T21:53:02.740 INFO:tasks.workunit.client.0.smithi107.stdout:1/762: write d4/df/d4e/d59/f5a [976082,96921] 0 2024-05-17T21:53:02.741 INFO:tasks.workunit.client.0.smithi107.stdout:1/763: stat d4/fa 0 2024-05-17T21:53:02.748 INFO:tasks.workunit.client.0.smithi107.stdout:0/829: write d3/d39/f7f [4452,17204] 0 2024-05-17T21:53:02.750 INFO:tasks.workunit.client.0.smithi107.stdout:3/754: dread d6/d8/d10/d25/f95 [0,4194304] 0 2024-05-17T21:53:02.752 INFO:tasks.workunit.client.1.smithi138.stdout:6/584: write d7/d69/f90 [788854,61814] 0 2024-05-17T21:53:02.758 INFO:tasks.workunit.client.1.smithi138.stdout:8/495: dwrite d0/dc/f2b [0,4194304] 0 2024-05-17T21:53:02.759 INFO:tasks.workunit.client.0.smithi107.stdout:7/691: dwrite d5/df/d11/d1d/f8c [0,4194304] 0 2024-05-17T21:53:02.763 INFO:tasks.workunit.client.1.smithi138.stdout:7/530: write d0/f91 [1250948,23574] 0 2024-05-17T21:53:02.778 INFO:tasks.workunit.client.1.smithi138.stdout:1/549: mkdir d4/dd/d3b/d9f/da9 0 2024-05-17T21:53:02.781 INFO:tasks.workunit.client.1.smithi138.stdout:6/585: rmdir d7/d69/db5/dc8/d43/d5f/d6a 39 2024-05-17T21:53:02.783 INFO:tasks.workunit.client.1.smithi138.stdout:1/550: dwrite f2 [0,4194304] 0 2024-05-17T21:53:02.787 INFO:tasks.workunit.client.1.smithi138.stdout:7/531: unlink d0/d18/d40/l70 0 2024-05-17T21:53:02.788 INFO:tasks.workunit.client.1.smithi138.stdout:7/532: chown d0/d18/d40/d41/l5d 31536127 1 2024-05-17T21:53:02.789 INFO:tasks.workunit.client.1.smithi138.stdout:9/503: sync 2024-05-17T21:53:02.792 INFO:tasks.workunit.client.1.smithi138.stdout:6/586: write d7/d69/db5/dc8/d43/d5f/d6a/f89 [823968,5321] 0 2024-05-17T21:53:02.793 INFO:tasks.workunit.client.1.smithi138.stdout:8/496: creat d0/d11/d6a/f91 x:0 0 0 2024-05-17T21:53:02.794 INFO:tasks.workunit.client.1.smithi138.stdout:6/587: stat d7/d69/db5/dc8/d34/da6/c1f 0 2024-05-17T21:53:02.797 INFO:tasks.workunit.client.1.smithi138.stdout:7/533: dwrite d0/d39/f85 [0,4194304] 0 2024-05-17T21:53:02.803 INFO:tasks.workunit.client.1.smithi138.stdout:1/551: mkdir d4/d2e/da0/daa 0 2024-05-17T21:53:02.805 INFO:tasks.workunit.client.1.smithi138.stdout:6/588: dread d7/d8/f70 [0,4194304] 0 2024-05-17T21:53:02.807 INFO:tasks.workunit.client.1.smithi138.stdout:8/497: truncate d0/d2f/f41 2950617 0 2024-05-17T21:53:02.809 INFO:tasks.workunit.client.1.smithi138.stdout:8/498: truncate d0/d11/f69 8444094 0 2024-05-17T21:53:02.811 INFO:tasks.workunit.client.1.smithi138.stdout:8/499: dread - d0/dc/d3b/d4d/f77 zero size 2024-05-17T21:53:02.811 INFO:tasks.workunit.client.1.smithi138.stdout:6/589: dwrite d7/d8/f9a [0,4194304] 0 2024-05-17T21:53:02.811 INFO:tasks.workunit.client.1.smithi138.stdout:6/590: chown d7/d69/db5 304 1 2024-05-17T21:53:02.820 INFO:tasks.workunit.client.0.smithi107.stdout:4/747: unlink d3/d4a/d47/d60/d3a/d41/d80/d43/f53 0 2024-05-17T21:53:02.822 INFO:tasks.workunit.client.0.smithi107.stdout:1/764: rmdir d4/df/d1a/d41/db2 39 2024-05-17T21:53:02.823 INFO:tasks.workunit.client.0.smithi107.stdout:0/830: dread - d3/dc/d22/d23/d97/d77/fca zero size 2024-05-17T21:53:02.824 INFO:tasks.workunit.client.0.smithi107.stdout:3/755: truncate d6/d2a/f42 1497325 0 2024-05-17T21:53:02.825 INFO:tasks.workunit.client.0.smithi107.stdout:0/831: truncate d3/d24/d6f/db5/def/ff0 1605029 0 2024-05-17T21:53:02.826 INFO:tasks.workunit.client.0.smithi107.stdout:0/832: chown d3/d75/lb2 7237772 1 2024-05-17T21:53:02.828 INFO:tasks.workunit.client.0.smithi107.stdout:4/748: creat d3/d6/d26/d36/d8b/d95/ff9 x:0 0 0 2024-05-17T21:53:02.829 INFO:tasks.workunit.client.0.smithi107.stdout:1/765: truncate d4/f66 243586 0 2024-05-17T21:53:02.831 INFO:tasks.workunit.client.0.smithi107.stdout:7/692: mkdir d5/df/d11/df6 0 2024-05-17T21:53:02.832 INFO:tasks.workunit.client.0.smithi107.stdout:7/693: chown d5/df/d11/d1c/d31/d56 584898227 1 2024-05-17T21:53:02.832 INFO:tasks.workunit.client.0.smithi107.stdout:7/694: truncate d5/fe 4345622 0 2024-05-17T21:53:02.834 INFO:tasks.workunit.client.0.smithi107.stdout:1/766: unlink d4/df/d1a/d26/c8c 0 2024-05-17T21:53:02.836 INFO:tasks.workunit.client.0.smithi107.stdout:1/767: read d4/df/f13 [2500833,32864] 0 2024-05-17T21:53:02.837 INFO:tasks.workunit.client.0.smithi107.stdout:6/657: rename d9/d16/d22/d5f/d6e/ca0 to d9/d16/d4d/d85/cce 0 2024-05-17T21:53:02.838 INFO:tasks.workunit.client.0.smithi107.stdout:6/658: chown d9/d16/c80 19604 1 2024-05-17T21:53:02.838 INFO:tasks.workunit.client.0.smithi107.stdout:1/768: dread d4/df/d1a/d26/d72/d85/fda [0,4194304] 0 2024-05-17T21:53:02.839 INFO:tasks.workunit.client.0.smithi107.stdout:4/749: mkdir d3/d4a/d47/d60/dfa 0 2024-05-17T21:53:02.839 INFO:tasks.workunit.client.0.smithi107.stdout:6/659: chown d9/d16/f3b 1940654 1 2024-05-17T21:53:02.839 INFO:tasks.workunit.client.0.smithi107.stdout:4/750: readlink d3/d6/ld3 0 2024-05-17T21:53:02.840 INFO:tasks.workunit.client.0.smithi107.stdout:1/769: read d4/df/d1a/d26/d7a/fa5 [487088,59609] 0 2024-05-17T21:53:02.842 INFO:tasks.workunit.client.0.smithi107.stdout:1/770: chown d4/df/l84 122090744 1 2024-05-17T21:53:02.844 INFO:tasks.workunit.client.0.smithi107.stdout:5/722: write d5/db/d16/d2a/f64 [3908297,68077] 0 2024-05-17T21:53:02.845 INFO:tasks.workunit.client.0.smithi107.stdout:6/660: dwrite f5 [0,4194304] 0 2024-05-17T21:53:02.849 INFO:tasks.workunit.client.0.smithi107.stdout:1/771: mkdir d4/df/d14/d2e/dd3/df6 0 2024-05-17T21:53:02.855 INFO:tasks.workunit.client.0.smithi107.stdout:3/756: rename d6/d8/d10/d35/d36/d66 to d6/d2a/d2d/da8/daa/df5 0 2024-05-17T21:53:02.856 INFO:tasks.workunit.client.0.smithi107.stdout:1/772: rmdir d4/df/d4e/d59 39 2024-05-17T21:53:02.861 INFO:tasks.workunit.client.0.smithi107.stdout:1/773: symlink d4/d78/lf7 0 2024-05-17T21:53:02.861 INFO:tasks.workunit.client.0.smithi107.stdout:1/774: readlink d4/df/d69/l75 0 2024-05-17T21:53:02.862 INFO:tasks.workunit.client.0.smithi107.stdout:1/775: chown d4/df/d1a/d26/d72/d85/fd7 47 1 2024-05-17T21:53:02.863 INFO:tasks.workunit.client.0.smithi107.stdout:3/757: creat d6/d2a/d2d/d69/dad/dcd/ff6 x:0 0 0 2024-05-17T21:53:02.865 INFO:tasks.workunit.client.0.smithi107.stdout:0/833: rename d3/f11 to d3/dc/f113 0 2024-05-17T21:53:02.868 INFO:tasks.workunit.client.0.smithi107.stdout:7/695: dwrite d5/d16/d3b/d45/f6c [0,4194304] 0 2024-05-17T21:53:02.870 INFO:tasks.workunit.client.0.smithi107.stdout:5/723: rename d5/db/d16/d5b/f3d to d5/db/d16/d2d/d40/fed 0 2024-05-17T21:53:02.874 INFO:tasks.workunit.client.0.smithi107.stdout:4/751: dwrite d3/d4a/d47/d60/f79 [0,4194304] 0 2024-05-17T21:53:02.877 INFO:tasks.workunit.client.0.smithi107.stdout:0/834: mkdir d3/d24/d6f/d114 0 2024-05-17T21:53:02.877 INFO:tasks.workunit.client.0.smithi107.stdout:3/758: link d6/d2a/d2d/d48/d4f/l7f d6/d2a/d2d/d37/lf7 0 2024-05-17T21:53:02.880 INFO:tasks.workunit.client.0.smithi107.stdout:6/661: dwrite d9/d16/d22/d2d/f2f [0,4194304] 0 2024-05-17T21:53:02.882 INFO:tasks.workunit.client.0.smithi107.stdout:0/835: chown d3/d58/d61/l87 102692 1 2024-05-17T21:53:02.882 INFO:tasks.workunit.client.0.smithi107.stdout:3/759: chown d6/d8/d10/d25/f99 3450 1 2024-05-17T21:53:02.884 INFO:tasks.workunit.client.1.smithi138.stdout:9/504: link d6/d7/d16/d29/d48/d3b/f63 d6/d7/d16/d29/d38/d40/d44/fa4 0 2024-05-17T21:53:02.884 INFO:tasks.workunit.client.0.smithi107.stdout:5/724: dwrite d5/db/d16/d5b/f97 [0,4194304] 0 2024-05-17T21:53:02.885 INFO:tasks.workunit.client.0.smithi107.stdout:3/760: stat d6/d8/d3e/d3f/d90/da7/ddc/led 0 2024-05-17T21:53:02.886 INFO:tasks.workunit.client.1.smithi138.stdout:9/505: read d6/f2a [533774,7160] 0 2024-05-17T21:53:02.890 INFO:tasks.workunit.client.1.smithi138.stdout:6/591: dread d7/d69/db5/dc8/d43/d5f/d6a/f89 [0,4194304] 0 2024-05-17T21:53:02.891 INFO:tasks.workunit.client.0.smithi107.stdout:7/696: write d5/df/d11/d1c/d2f/d53/fa9 [1036630,88692] 0 2024-05-17T21:53:02.891 INFO:tasks.workunit.client.0.smithi107.stdout:5/725: mkdir d5/db/d16/d2d/d40/d5a/d7c/dee 0 2024-05-17T21:53:02.892 INFO:tasks.workunit.client.1.smithi138.stdout:6/592: chown d7/d8/l6f 9997 1 2024-05-17T21:53:02.892 INFO:tasks.workunit.client.0.smithi107.stdout:6/662: truncate d9/d16/d22/d5f/d6f/fa7 315095 0 2024-05-17T21:53:02.894 INFO:tasks.workunit.client.1.smithi138.stdout:9/506: dwrite d6/d7/d16/d29/f3a [0,4194304] 0 2024-05-17T21:53:02.894 INFO:tasks.workunit.client.0.smithi107.stdout:3/761: mknod d6/d8/d3e/dc9/cf8 0 2024-05-17T21:53:02.895 INFO:tasks.workunit.client.1.smithi138.stdout:7/534: getdents d0/d18/d64 0 2024-05-17T21:53:02.896 INFO:tasks.workunit.client.0.smithi107.stdout:0/836: creat d3/d24/d6f/f115 x:0 0 0 2024-05-17T21:53:02.897 INFO:tasks.workunit.client.0.smithi107.stdout:0/837: chown d3/dc/f14 16054338 1 2024-05-17T21:53:02.898 INFO:tasks.workunit.client.1.smithi138.stdout:8/500: write d0/d2f/f41 [710844,73819] 0 2024-05-17T21:53:02.901 INFO:tasks.workunit.client.0.smithi107.stdout:5/726: rename d5/db/f5c to d5/db/d16/d2d/d40/d9f/fef 0 2024-05-17T21:53:02.902 INFO:tasks.workunit.client.1.smithi138.stdout:7/535: chown d0/d18/f45 15201039 1 2024-05-17T21:53:02.903 INFO:tasks.workunit.client.1.smithi138.stdout:8/501: dread d0/f40 [0,4194304] 0 2024-05-17T21:53:02.907 INFO:tasks.workunit.client.0.smithi107.stdout:3/762: rename d6/d2a/d2d/d69/dad/dcd/ff6 to d6/d8/d10/d35/d91/da6/ff9 0 2024-05-17T21:53:02.909 INFO:tasks.workunit.client.1.smithi138.stdout:8/502: unlink d0/dc/d1a/d44/l5d 0 2024-05-17T21:53:02.911 INFO:tasks.workunit.client.1.smithi138.stdout:6/593: creat d7/d69/db5/dc8/d43/da1/fcb x:0 0 0 2024-05-17T21:53:02.913 INFO:tasks.workunit.client.0.smithi107.stdout:7/697: getdents d5/df/d11/d1c 0 2024-05-17T21:53:02.913 INFO:tasks.workunit.client.0.smithi107.stdout:7/698: chown d5/d16/d3b/d45/f6c 34630672 1 2024-05-17T21:53:02.914 INFO:tasks.workunit.client.0.smithi107.stdout:3/763: creat d6/d1a/d57/ffa x:0 0 0 2024-05-17T21:53:02.914 INFO:tasks.workunit.client.0.smithi107.stdout:7/699: chown d5/df/d11/d1d/d44 8 1 2024-05-17T21:53:02.915 INFO:tasks.workunit.client.0.smithi107.stdout:5/727: mknod d5/db/d16/d2d/d40/d5a/d7c/dee/cf0 0 2024-05-17T21:53:02.917 INFO:tasks.workunit.client.0.smithi107.stdout:3/764: unlink d6/d8/d10/d35/d36/d4c/c81 0 2024-05-17T21:53:02.917 INFO:tasks.workunit.client.0.smithi107.stdout:7/700: fdatasync d5/df/d11/d1d/d33/f46 0 2024-05-17T21:53:02.919 INFO:tasks.workunit.client.0.smithi107.stdout:5/728: creat d5/db/d16/d54/d59/d65/ff1 x:0 0 0 2024-05-17T21:53:02.920 INFO:tasks.workunit.client.0.smithi107.stdout:5/729: fsync d5/db/d16/d54/d59/f87 0 2024-05-17T21:53:02.920 INFO:tasks.workunit.client.0.smithi107.stdout:7/701: truncate d5/d16/d3b/d45/d91/dac/fef 935251 0 2024-05-17T21:53:02.926 INFO:tasks.workunit.client.0.smithi107.stdout:6/663: write d9/d16/d22/d2d/f9a [43174,86519] 0 2024-05-17T21:53:02.926 INFO:tasks.workunit.client.0.smithi107.stdout:6/664: readlink d9/d16/d22/lcc 0 2024-05-17T21:53:02.931 INFO:tasks.workunit.client.0.smithi107.stdout:7/702: link d5/df/d11/d1d/l74 d5/df/d78/lf7 0 2024-05-17T21:53:02.931 INFO:tasks.workunit.client.0.smithi107.stdout:0/838: write d3/dc/d22/d3d/fa8 [1523188,31673] 0 2024-05-17T21:53:02.932 INFO:tasks.workunit.client.0.smithi107.stdout:0/839: stat d3/d39/c107 0 2024-05-17T21:53:02.933 INFO:tasks.workunit.client.0.smithi107.stdout:6/665: rename d9/l35 to d9/d16/d22/daf/lcf 0 2024-05-17T21:53:02.934 INFO:tasks.workunit.client.0.smithi107.stdout:0/840: write d3/d58/d61/d94/f95 [4042725,67110] 0 2024-05-17T21:53:02.934 INFO:tasks.workunit.client.1.smithi138.stdout:7/536: write d0/d18/d40/d41/f43 [988851,24233] 0 2024-05-17T21:53:02.937 INFO:tasks.workunit.client.1.smithi138.stdout:7/537: symlink d0/d39/la9 0 2024-05-17T21:53:02.939 INFO:tasks.workunit.client.0.smithi107.stdout:0/841: dwrite d3/d24/d6f/d9f/de8/ff1 [0,4194304] 0 2024-05-17T21:53:02.941 INFO:tasks.workunit.client.0.smithi107.stdout:6/666: rename d9/d16/d22/d2d to d9/d16/d4d/db8/dc9/dd0 0 2024-05-17T21:53:02.942 INFO:tasks.workunit.client.1.smithi138.stdout:8/503: dwrite d0/dc/f79 [0,4194304] 0 2024-05-17T21:53:02.944 INFO:tasks.workunit.client.1.smithi138.stdout:7/538: creat d0/d18/d40/d41/d50/faa x:0 0 0 2024-05-17T21:53:02.948 INFO:tasks.workunit.client.0.smithi107.stdout:3/765: write d6/d2a/d2d/da8/daa/df5/fbf [543288,32712] 0 2024-05-17T21:53:02.950 INFO:tasks.workunit.client.0.smithi107.stdout:8/710: sync 2024-05-17T21:53:02.950 INFO:tasks.workunit.client.0.smithi107.stdout:2/577: sync 2024-05-17T21:53:02.950 INFO:tasks.workunit.client.0.smithi107.stdout:9/764: sync 2024-05-17T21:53:02.950 INFO:tasks.workunit.client.0.smithi107.stdout:5/730: write d5/db/d16/f21 [1182198,16665] 0 2024-05-17T21:53:02.951 INFO:tasks.workunit.client.0.smithi107.stdout:7/703: getdents d5/d16/d3b/d45 0 2024-05-17T21:53:02.952 INFO:tasks.workunit.client.0.smithi107.stdout:6/667: rename d9/f4e to d9/d16/d9e/fd1 0 2024-05-17T21:53:02.955 INFO:tasks.workunit.client.0.smithi107.stdout:5/731: readlink d5/db/d16/d2d/d4d/ld5 0 2024-05-17T21:53:02.955 INFO:tasks.workunit.client.0.smithi107.stdout:0/842: dread d3/f18 [0,4194304] 0 2024-05-17T21:53:02.958 INFO:tasks.workunit.client.0.smithi107.stdout:7/704: creat d5/d16/d3b/d45/ff8 x:0 0 0 2024-05-17T21:53:02.958 INFO:tasks.workunit.client.0.smithi107.stdout:6/668: rmdir d9/d16/d4d/d85/d88 39 2024-05-17T21:53:02.959 INFO:tasks.workunit.client.0.smithi107.stdout:9/765: mknod d0/dd/d69/cfd 0 2024-05-17T21:53:02.959 INFO:tasks.workunit.client.0.smithi107.stdout:0/843: mkdir d3/dc/d22/d23/d116 0 2024-05-17T21:53:02.961 INFO:tasks.workunit.client.0.smithi107.stdout:2/578: dwrite d0/f7d [4194304,4194304] 0 2024-05-17T21:53:02.964 INFO:tasks.workunit.client.0.smithi107.stdout:9/766: rmdir d0/dd/d21/d29/d57 39 2024-05-17T21:53:02.965 INFO:tasks.workunit.client.0.smithi107.stdout:5/732: dwrite d5/db/d16/d54/d59/d65/fd6 [0,4194304] 0 2024-05-17T21:53:02.965 INFO:tasks.workunit.client.0.smithi107.stdout:0/844: unlink d3/dc/d31/l45 0 2024-05-17T21:53:02.966 INFO:tasks.workunit.client.0.smithi107.stdout:6/669: mknod d9/d16/d22/daf/cd2 0 2024-05-17T21:53:02.973 INFO:tasks.workunit.client.1.smithi138.stdout:0/566: sync 2024-05-17T21:53:02.973 INFO:tasks.workunit.client.1.smithi138.stdout:2/595: sync 2024-05-17T21:53:02.976 INFO:tasks.workunit.client.1.smithi138.stdout:2/596: stat d3/d81/c8e 0 2024-05-17T21:53:02.980 INFO:tasks.workunit.client.1.smithi138.stdout:8/504: mkdir d0/d71/d92 0 2024-05-17T21:53:02.981 INFO:tasks.workunit.client.1.smithi138.stdout:7/539: creat d0/d9d/fab x:0 0 0 2024-05-17T21:53:02.984 INFO:tasks.workunit.client.1.smithi138.stdout:0/567: mknod d3/d10/d6d/db6/db9/cbf 0 2024-05-17T21:53:02.987 INFO:tasks.workunit.client.1.smithi138.stdout:2/597: dwrite d3/f8c [0,4194304] 0 2024-05-17T21:53:02.988 INFO:tasks.workunit.client.1.smithi138.stdout:2/598: chown d3/f20 0 1 2024-05-17T21:53:02.992 INFO:tasks.workunit.client.1.smithi138.stdout:8/505: symlink d0/l93 0 2024-05-17T21:53:02.993 INFO:tasks.workunit.client.1.smithi138.stdout:2/599: dread - d3/f66 zero size 2024-05-17T21:53:02.993 INFO:tasks.workunit.client.1.smithi138.stdout:7/540: mknod d0/d18/d9c/cac 0 2024-05-17T21:53:02.994 INFO:tasks.workunit.client.1.smithi138.stdout:0/568: getdents d3/d3d/da2 0 2024-05-17T21:53:02.997 INFO:tasks.workunit.client.1.smithi138.stdout:2/600: mknod d3/d21/d2f/d3e/d55/d99/d93/cc2 0 2024-05-17T21:53:02.999 INFO:tasks.workunit.client.1.smithi138.stdout:8/506: creat d0/d11/f94 x:0 0 0 2024-05-17T21:53:03.002 INFO:tasks.workunit.client.1.smithi138.stdout:7/541: dwrite d0/d18/d40/d41/d51/f69 [0,4194304] 0 2024-05-17T21:53:03.007 INFO:tasks.workunit.client.1.smithi138.stdout:8/507: rename d0/dc/d1a/d33/d4a/d4e/l5b to d0/dc/d3b/l95 0 2024-05-17T21:53:03.010 INFO:tasks.workunit.client.1.smithi138.stdout:7/542: link d0/d18/d40/f4e d0/d18/d64/d7c/fad 0 2024-05-17T21:53:03.011 INFO:tasks.workunit.client.1.smithi138.stdout:8/508: rmdir d0/dc/d1a/d33/d4a 39 2024-05-17T21:53:03.012 INFO:tasks.workunit.client.1.smithi138.stdout:7/543: dread d0/d18/f92 [0,4194304] 0 2024-05-17T21:53:03.017 INFO:tasks.workunit.client.1.smithi138.stdout:7/544: symlink d0/d18/d40/d41/d51/d66/lae 0 2024-05-17T21:53:03.021 INFO:tasks.workunit.client.1.smithi138.stdout:0/569: getdents d3/d10/d6d/db6/db9 0 2024-05-17T21:53:03.024 INFO:tasks.workunit.client.1.smithi138.stdout:0/570: rename d3/d10/d38/d3e/f67 to d3/d10/fc0 0 2024-05-17T21:53:03.025 INFO:tasks.workunit.client.1.smithi138.stdout:2/601: write d3/d21/d2f/f33 [2945174,14925] 0 2024-05-17T21:53:03.028 INFO:tasks.workunit.client.1.smithi138.stdout:2/602: mkdir d3/d21/d2f/d3e/d58/dc3 0 2024-05-17T21:53:03.029 INFO:tasks.workunit.client.1.smithi138.stdout:8/509: write d0/d2f/f6e [686188,100502] 0 2024-05-17T21:53:03.030 INFO:tasks.workunit.client.1.smithi138.stdout:8/510: write d0/fb [11173831,62044] 0 2024-05-17T21:53:03.033 INFO:tasks.workunit.client.1.smithi138.stdout:0/571: write f0 [2275052,27918] 0 2024-05-17T21:53:03.035 INFO:tasks.workunit.client.1.smithi138.stdout:8/511: mknod d0/dc/d1a/d78/c96 0 2024-05-17T21:53:03.036 INFO:tasks.workunit.client.1.smithi138.stdout:8/512: fsync d0/dc/d3b/d4d/f8c 0 2024-05-17T21:53:03.038 INFO:tasks.workunit.client.1.smithi138.stdout:7/545: link d0/d18/d40/d41/c56 d0/d18/d40/d41/d51/d66/caf 0 2024-05-17T21:53:03.040 INFO:tasks.workunit.client.0.smithi107.stdout:9/767: rmdir d0/dd/d69/db2 39 2024-05-17T21:53:03.040 INFO:tasks.workunit.client.0.smithi107.stdout:5/733: mknod d5/d75/dc7/dd4/cf2 0 2024-05-17T21:53:03.041 INFO:tasks.workunit.client.1.smithi138.stdout:7/546: creat d0/d18/d64/d7c/da2/fb0 x:0 0 0 2024-05-17T21:53:03.041 INFO:tasks.workunit.client.0.smithi107.stdout:2/579: dread d0/d9/d1c/d37/d9f/fa7 [0,4194304] 0 2024-05-17T21:53:03.042 INFO:tasks.workunit.client.1.smithi138.stdout:8/513: dwrite d0/d11/d6a/f91 [0,4194304] 0 2024-05-17T21:53:03.042 INFO:tasks.workunit.client.0.smithi107.stdout:9/768: stat d0/dd/d21/dcd 0 2024-05-17T21:53:03.043 INFO:tasks.workunit.client.0.smithi107.stdout:7/705: truncate d5/f6d 2867577 0 2024-05-17T21:53:03.043 INFO:tasks.workunit.client.1.smithi138.stdout:8/514: fdatasync d0/dc/d1a/d51/f73 0 2024-05-17T21:53:03.047 INFO:tasks.workunit.client.1.smithi138.stdout:8/515: mkdir d0/dc/d3b/d4d/d7d/d97 0 2024-05-17T21:53:03.050 INFO:tasks.workunit.client.0.smithi107.stdout:2/580: unlink d0/d9/d1c/d37/l49 0 2024-05-17T21:53:03.051 INFO:tasks.workunit.client.0.smithi107.stdout:3/766: dwrite d6/d8/f17 [0,4194304] 0 2024-05-17T21:53:03.051 INFO:tasks.workunit.client.1.smithi138.stdout:7/547: write d0/d18/d64/d7c/fad [3779496,12402] 0 2024-05-17T21:53:03.052 INFO:tasks.workunit.client.0.smithi107.stdout:8/711: dwrite dc/d21/d97/d63/d71/fbb [0,4194304] 0 2024-05-17T21:53:03.054 INFO:tasks.workunit.client.0.smithi107.stdout:8/712: write dc/d21/d97/d63/f99 [3542031,48259] 0 2024-05-17T21:53:03.055 INFO:tasks.workunit.client.0.smithi107.stdout:9/769: mkdir d0/dd/d21/d29/d3e/dfe 0 2024-05-17T21:53:03.055 INFO:tasks.workunit.client.1.smithi138.stdout:8/516: dwrite d0/dc/f79 [0,4194304] 0 2024-05-17T21:53:03.056 INFO:tasks.workunit.client.0.smithi107.stdout:5/734: fsync d5/db/f1a 0 2024-05-17T21:53:03.057 INFO:tasks.workunit.client.0.smithi107.stdout:2/581: read d0/d9/d11/df/f35 [2709787,82144] 0 2024-05-17T21:53:03.058 INFO:tasks.workunit.client.0.smithi107.stdout:5/735: write d5/db/d16/d54/d59/d92/fe5 [239220,101475] 0 2024-05-17T21:53:03.059 INFO:tasks.workunit.client.0.smithi107.stdout:5/736: write d5/d5f/fb3 [427293,92492] 0 2024-05-17T21:53:03.060 INFO:tasks.workunit.client.0.smithi107.stdout:7/706: symlink d5/df/d11/d1c/d31/d56/lf9 0 2024-05-17T21:53:03.062 INFO:tasks.workunit.client.1.smithi138.stdout:8/517: dread d0/d2f/f45 [0,4194304] 0 2024-05-17T21:53:03.063 INFO:tasks.workunit.client.1.smithi138.stdout:7/548: truncate d0/d18/d40/d41/d50/f53 2518525 0 2024-05-17T21:53:03.065 INFO:tasks.workunit.client.1.smithi138.stdout:7/549: chown d0/d18/d40/d41/f5b 1050 1 2024-05-17T21:53:03.067 INFO:tasks.workunit.client.1.smithi138.stdout:7/550: dread - d0/d18/d40/d5f/f96 zero size 2024-05-17T21:53:03.068 INFO:tasks.workunit.client.0.smithi107.stdout:0/845: link d3/d24/c5c d3/d24/d6f/d9f/dab/c117 0 2024-05-17T21:53:03.068 INFO:tasks.workunit.client.1.smithi138.stdout:7/551: dread d0/f8e [0,4194304] 0 2024-05-17T21:53:03.070 INFO:tasks.workunit.client.0.smithi107.stdout:8/713: unlink dc/d21/d97/f8d 0 2024-05-17T21:53:03.073 INFO:tasks.workunit.client.0.smithi107.stdout:8/714: chown dc/d21/d27/f37 4704292 1 2024-05-17T21:53:03.073 INFO:tasks.workunit.client.0.smithi107.stdout:9/770: creat d0/dd/d21/d29/d3e/d45/fff x:0 0 0 2024-05-17T21:53:03.074 INFO:tasks.workunit.client.1.smithi138.stdout:8/518: rename d0/d11/c1e to d0/d63/c98 0 2024-05-17T21:53:03.075 INFO:tasks.workunit.client.0.smithi107.stdout:3/767: dwrite d6/d8/d10/f23 [4194304,4194304] 0 2024-05-17T21:53:03.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:02 smithi138 ceph-mon[35948]: pgmap v39: 65 pgs: 65 active+clean; 2.5 GiB data, 8.7 GiB used, 528 GiB / 536 GiB avail; 115 MiB/s rd, 306 MiB/s wr, 631 op/s 2024-05-17T21:53:03.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:02 smithi138 ceph-mon[35948]: from='client.? 172.21.15.107:0/4293648570' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-17T21:53:03.077 INFO:tasks.workunit.client.0.smithi107.stdout:6/670: write d9/d16/d4d/db8/dc9/dd0/f31 [1362004,65015] 0 2024-05-17T21:53:03.079 INFO:tasks.workunit.client.0.smithi107.stdout:2/582: mkdir d0/d9/d11/d18/dad/db8 0 2024-05-17T21:53:03.082 INFO:tasks.workunit.client.0.smithi107.stdout:5/737: symlink d5/db/dc/dc5/lf3 0 2024-05-17T21:53:03.085 INFO:tasks.workunit.client.0.smithi107.stdout:7/707: symlink d5/df/d78/lfa 0 2024-05-17T21:53:03.088 INFO:tasks.workunit.client.1.smithi138.stdout:7/552: rename d0/c1 to d0/d18/d9c/cb1 0 2024-05-17T21:53:03.090 INFO:tasks.workunit.client.1.smithi138.stdout:7/553: fsync d0/f30 0 2024-05-17T21:53:03.090 INFO:tasks.workunit.client.0.smithi107.stdout:5/738: dread d5/db/d16/d54/d59/d65/fac [0,4194304] 0 2024-05-17T21:53:03.091 INFO:tasks.workunit.client.0.smithi107.stdout:8/715: creat dc/d21/d27/d34/d4c/d7e/f101 x:0 0 0 2024-05-17T21:53:03.099 INFO:tasks.workunit.client.1.smithi138.stdout:7/554: symlink d0/d18/d40/d41/lb2 0 2024-05-17T21:53:03.101 INFO:tasks.workunit.client.0.smithi107.stdout:0/846: mkdir d3/dc/d22/d3d/d118 0 2024-05-17T21:53:03.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:02 smithi107 ceph-mon[28434]: pgmap v39: 65 pgs: 65 active+clean; 2.5 GiB data, 8.7 GiB used, 528 GiB / 536 GiB avail; 115 MiB/s rd, 306 MiB/s wr, 631 op/s 2024-05-17T21:53:03.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:02 smithi107 ceph-mon[28434]: from='client.? 172.21.15.107:0/4293648570' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-17T21:53:03.102 INFO:tasks.workunit.client.0.smithi107.stdout:0/847: write d3/dc/f16 [354091,98117] 0 2024-05-17T21:53:03.104 INFO:tasks.workunit.client.0.smithi107.stdout:3/768: dread d6/d1a/d57/f73 [0,4194304] 0 2024-05-17T21:53:03.105 INFO:tasks.workunit.client.0.smithi107.stdout:5/739: mkdir d5/db/d16/d2d/d40/d5a/db4/df4 0 2024-05-17T21:53:03.108 INFO:tasks.workunit.client.0.smithi107.stdout:6/671: mknod d9/d16/d22/d5f/cd3 0 2024-05-17T21:53:03.108 INFO:tasks.workunit.client.0.smithi107.stdout:6/672: readlink d9/d58/d66/la5 0 2024-05-17T21:53:03.109 INFO:tasks.workunit.client.0.smithi107.stdout:6/673: write d9/f1c [641727,35350] 0 2024-05-17T21:53:03.110 INFO:tasks.workunit.client.0.smithi107.stdout:2/583: rename d0/d9/f33 to d0/d9/d11/df/d5a/d6d/d82/fb9 0 2024-05-17T21:53:03.111 INFO:tasks.workunit.client.1.smithi138.stdout:8/519: dwrite d0/d31/f5a [0,4194304] 0 2024-05-17T21:53:03.111 INFO:tasks.workunit.client.0.smithi107.stdout:2/584: truncate d0/d26/d64/f8d 594434 0 2024-05-17T21:53:03.112 INFO:tasks.workunit.client.1.smithi138.stdout:7/555: rename d0/d39/la9 to d0/d18/d40/d41/d51/d66/lb3 0 2024-05-17T21:53:03.112 INFO:tasks.workunit.client.1.smithi138.stdout:8/520: readlink d0/dc/d1a/d33/l6f 0 2024-05-17T21:53:03.113 INFO:tasks.workunit.client.1.smithi138.stdout:8/521: stat d0/dc/d3b/c67 0 2024-05-17T21:53:03.113 INFO:tasks.workunit.client.1.smithi138.stdout:8/522: dread - d0/d11/f94 zero size 2024-05-17T21:53:03.116 INFO:tasks.workunit.client.0.smithi107.stdout:0/848: truncate d3/dc/f14 190320 0 2024-05-17T21:53:03.118 INFO:tasks.workunit.client.1.smithi138.stdout:8/523: fdatasync d0/d11/f42 0 2024-05-17T21:53:03.119 INFO:tasks.workunit.client.1.smithi138.stdout:8/524: fdatasync d0/dc/f22 0 2024-05-17T21:53:03.120 INFO:tasks.workunit.client.0.smithi107.stdout:3/769: symlink d6/d8/d3e/d3f/dbe/lfb 0 2024-05-17T21:53:03.121 INFO:tasks.workunit.client.1.smithi138.stdout:7/556: mknod d0/d18/cb4 0 2024-05-17T21:53:03.123 INFO:tasks.workunit.client.0.smithi107.stdout:5/740: symlink d5/db/d16/d2d/d40/d5a/db4/lf5 0 2024-05-17T21:53:03.127 INFO:tasks.workunit.client.0.smithi107.stdout:8/716: write dc/f67 [1882721,65515] 0 2024-05-17T21:53:03.128 INFO:tasks.workunit.client.1.smithi138.stdout:7/557: dwrite d0/d18/d40/f44 [4194304,4194304] 0 2024-05-17T21:53:03.143 INFO:tasks.workunit.client.1.smithi138.stdout:7/558: link d0/f20 d0/d9d/fb5 0 2024-05-17T21:53:03.152 INFO:tasks.workunit.client.1.smithi138.stdout:8/525: dwrite d0/dc/d1a/f2a [0,4194304] 0 2024-05-17T21:53:03.152 INFO:tasks.workunit.client.1.smithi138.stdout:7/559: dwrite d0/d9d/fab [0,4194304] 0 2024-05-17T21:53:03.155 INFO:tasks.workunit.client.1.smithi138.stdout:8/526: creat d0/dc/d3b/f99 x:0 0 0 2024-05-17T21:53:03.157 INFO:tasks.workunit.client.1.smithi138.stdout:7/560: rmdir d0/d18/d40/d41/d51 39 2024-05-17T21:53:03.158 INFO:tasks.workunit.client.1.smithi138.stdout:8/527: rmdir d0/dc/d1a/d51 39 2024-05-17T21:53:03.161 INFO:tasks.workunit.client.1.smithi138.stdout:8/528: write d0/dc/d1a/d51/d82/f8e [251855,54963] 0 2024-05-17T21:53:03.163 INFO:tasks.workunit.client.1.smithi138.stdout:8/529: write d0/dc/d3b/d4d/f8c [1673246,13029] 0 2024-05-17T21:53:03.186 INFO:tasks.workunit.client.0.smithi107.stdout:1/776: sync 2024-05-17T21:53:03.190 INFO:tasks.workunit.client.0.smithi107.stdout:6/674: creat d9/d16/d4d/d85/d96/fd4 x:0 0 0 2024-05-17T21:53:03.190 INFO:tasks.workunit.client.0.smithi107.stdout:5/741: dwrite d5/db/d16/f9d [0,4194304] 0 2024-05-17T21:53:03.190 INFO:tasks.workunit.client.0.smithi107.stdout:9/771: dwrite d0/dd/d21/d29/d57/f8a [0,4194304] 0 2024-05-17T21:53:03.190 INFO:tasks.workunit.client.0.smithi107.stdout:0/849: symlink d3/d75/d9e/d103/d10d/l119 0 2024-05-17T21:53:03.191 INFO:tasks.workunit.client.0.smithi107.stdout:5/742: readlink d5/d75/l96 0 2024-05-17T21:53:03.192 INFO:tasks.workunit.client.0.smithi107.stdout:8/717: rename f4 to dc/d21/db8/f102 0 2024-05-17T21:53:03.193 INFO:tasks.workunit.client.0.smithi107.stdout:9/772: chown d0/dd/d21/d29/d3e/f91 908 1 2024-05-17T21:53:03.193 INFO:tasks.workunit.client.1.smithi138.stdout:5/547: sync 2024-05-17T21:53:03.193 INFO:tasks.workunit.client.1.smithi138.stdout:3/673: sync 2024-05-17T21:53:03.193 INFO:tasks.workunit.client.1.smithi138.stdout:4/590: sync 2024-05-17T21:53:03.194 INFO:tasks.workunit.client.0.smithi107.stdout:6/675: mkdir d9/d58/d66/dd5 0 2024-05-17T21:53:03.197 INFO:tasks.workunit.client.1.smithi138.stdout:3/674: dread - d1/d3/d16/d2d/faa zero size 2024-05-17T21:53:03.198 INFO:tasks.workunit.client.1.smithi138.stdout:5/548: chown d1/d6/d1b/d29/d36/d68/f6f 56049570 1 2024-05-17T21:53:03.201 INFO:tasks.workunit.client.0.smithi107.stdout:2/585: rmdir d0/d26/d64/d7a 0 2024-05-17T21:53:03.203 INFO:tasks.workunit.client.0.smithi107.stdout:3/770: dwrite d6/d2a/d2d/d48/d4f/f67 [0,4194304] 0 2024-05-17T21:53:03.203 INFO:tasks.workunit.client.1.smithi138.stdout:3/675: readlink d1/d62/dad/d72/l8a 0 2024-05-17T21:53:03.207 INFO:tasks.workunit.client.0.smithi107.stdout:9/773: rmdir d0/dd/d21/d29/d3e/d94 39 2024-05-17T21:53:03.209 INFO:tasks.workunit.client.1.smithi138.stdout:3/676: creat d1/d3/d16/d5d/feb x:0 0 0 2024-05-17T21:53:03.210 INFO:tasks.workunit.client.0.smithi107.stdout:0/850: symlink d3/d24/d6f/db5/dd7/l11a 0 2024-05-17T21:53:03.215 INFO:tasks.workunit.client.0.smithi107.stdout:6/676: mkdir d9/d16/d22/d5f/d6e/dd6 0 2024-05-17T21:53:03.216 INFO:tasks.workunit.client.1.smithi138.stdout:3/677: mkdir d1/d62/dad/d5c/d9c/dec 0 2024-05-17T21:53:03.217 INFO:tasks.workunit.client.1.smithi138.stdout:4/591: link d0/d1b/d36/c58 d0/d1b/d42/d45/cd0 0 2024-05-17T21:53:03.218 INFO:tasks.workunit.client.0.smithi107.stdout:2/586: mkdir d0/d93/dba 0 2024-05-17T21:53:03.219 INFO:tasks.workunit.client.1.smithi138.stdout:5/549: link d1/d2c/laa d1/d1a/d39/d62/lb2 0 2024-05-17T21:53:03.219 INFO:tasks.workunit.client.1.smithi138.stdout:5/550: readlink d1/d60/l44 0 2024-05-17T21:53:03.223 INFO:tasks.workunit.client.1.smithi138.stdout:5/551: read d1/d6/d1b/f54 [505747,94364] 0 2024-05-17T21:53:03.223 INFO:tasks.workunit.client.0.smithi107.stdout:9/774: creat d0/dd/d21/d29/d57/d5f/f100 x:0 0 0 2024-05-17T21:53:03.225 INFO:tasks.workunit.client.1.smithi138.stdout:8/530: truncate d0/dc/f79 1735463 0 2024-05-17T21:53:03.229 INFO:tasks.workunit.client.1.smithi138.stdout:5/552: write d1/d2c/d63/f9d [320213,71615] 0 2024-05-17T21:53:03.231 INFO:tasks.workunit.client.0.smithi107.stdout:1/777: dwrite f3 [0,4194304] 0 2024-05-17T21:53:03.232 INFO:tasks.workunit.client.0.smithi107.stdout:1/778: chown d4/df/d4e/l62 1601902 1 2024-05-17T21:53:03.235 INFO:tasks.workunit.client.0.smithi107.stdout:0/851: mkdir d3/dc/dae/dc0/de1/d10b/d11b 0 2024-05-17T21:53:03.237 INFO:tasks.workunit.client.0.smithi107.stdout:4/752: sync 2024-05-17T21:53:03.238 INFO:tasks.workunit.client.0.smithi107.stdout:5/743: dwrite d5/db/d16/d54/d59/d65/f47 [0,4194304] 0 2024-05-17T21:53:03.239 INFO:tasks.workunit.client.0.smithi107.stdout:5/744: chown d5/db/d16/d2d/d4d 124 1 2024-05-17T21:53:03.242 INFO:tasks.workunit.client.1.smithi138.stdout:3/678: getdents d1/d62/dad/d5c/d9c 0 2024-05-17T21:53:03.245 INFO:tasks.workunit.client.0.smithi107.stdout:3/771: rename d6/d8/d10/d25/fda to d6/d8/d3e/d3f/ffc 0 2024-05-17T21:53:03.246 INFO:tasks.workunit.client.1.smithi138.stdout:5/553: creat d1/d6/d1b/d29/d36/d68/fb3 x:0 0 0 2024-05-17T21:53:03.249 INFO:tasks.workunit.client.1.smithi138.stdout:8/531: rename d0/dc/f18 to d0/dc/d3b/d4d/d7d/d97/f9a 0 2024-05-17T21:53:03.250 INFO:tasks.workunit.client.1.smithi138.stdout:8/532: write d0/d2f/f45 [412945,106744] 0 2024-05-17T21:53:03.251 INFO:tasks.workunit.client.1.smithi138.stdout:8/533: fsync d0/f2c 0 2024-05-17T21:53:03.253 INFO:tasks.workunit.client.0.smithi107.stdout:0/852: fdatasync d3/d58/f84 0 2024-05-17T21:53:03.253 INFO:tasks.workunit.client.1.smithi138.stdout:8/534: truncate d0/dc/d3b/d4d/f77 725884 0 2024-05-17T21:53:03.254 INFO:tasks.workunit.client.1.smithi138.stdout:8/535: write d0/dc/f10 [8119614,115863] 0 2024-05-17T21:53:03.255 INFO:tasks.workunit.client.0.smithi107.stdout:8/718: getdents dc/d21/d97/d63/d71/d81 0 2024-05-17T21:53:03.257 INFO:tasks.workunit.client.0.smithi107.stdout:9/775: dwrite d0/dd/d3d/f8e [0,4194304] 0 2024-05-17T21:53:03.258 INFO:tasks.workunit.client.1.smithi138.stdout:4/592: getdents d0/d1b/d42/d45/d84 0 2024-05-17T21:53:03.259 INFO:tasks.workunit.client.0.smithi107.stdout:5/745: symlink d5/db/d16/d2a/d73/dd0/lf6 0 2024-05-17T21:53:03.259 INFO:tasks.workunit.client.1.smithi138.stdout:4/593: readlink d0/l9 0 2024-05-17T21:53:03.262 INFO:tasks.workunit.client.0.smithi107.stdout:9/776: read - d0/dd/d69/dee/fef zero size 2024-05-17T21:53:03.263 INFO:tasks.workunit.client.1.smithi138.stdout:3/679: unlink d1/d3/d16/d5d/c8c 0 2024-05-17T21:53:03.264 INFO:tasks.workunit.client.0.smithi107.stdout:6/677: dwrite d9/d16/d22/f56 [0,4194304] 0 2024-05-17T21:53:03.266 INFO:tasks.workunit.client.0.smithi107.stdout:2/587: rename d0/d9/d11/d15/f89 to d0/d9/d11/d15/d99/fbb 0 2024-05-17T21:53:03.270 INFO:tasks.workunit.client.0.smithi107.stdout:3/772: readlink d6/d2a/d2d/da8/daa/df5/la0 0 2024-05-17T21:53:03.270 INFO:tasks.workunit.client.0.smithi107.stdout:3/773: readlink d6/d2a/d2d/da8/daa/df5/la0 0 2024-05-17T21:53:03.277 INFO:tasks.workunit.client.0.smithi107.stdout:4/753: dwrite d3/db/d1e/f9a [0,4194304] 0 2024-05-17T21:53:03.278 INFO:tasks.workunit.client.0.smithi107.stdout:0/853: dread d3/dc/d22/d3d/dc7/fcb [0,4194304] 0 2024-05-17T21:53:03.279 INFO:tasks.workunit.client.0.smithi107.stdout:1/779: write d4/f21 [3832028,13660] 0 2024-05-17T21:53:03.279 INFO:tasks.workunit.client.1.smithi138.stdout:5/554: getdents d1/d6/d1b 0 2024-05-17T21:53:03.287 INFO:tasks.workunit.client.1.smithi138.stdout:8/536: link d0/dc/d1a/c53 d0/d71/d92/c9b 0 2024-05-17T21:53:03.287 INFO:tasks.workunit.client.1.smithi138.stdout:5/555: dwrite d1/d6/f41 [0,4194304] 0 2024-05-17T21:53:03.292 INFO:tasks.workunit.client.1.smithi138.stdout:4/594: link d0/lf d0/d3/d17/d5e/d7f/ld1 0 2024-05-17T21:53:03.293 INFO:tasks.workunit.client.0.smithi107.stdout:6/678: creat d9/d58/d7b/da3/fd7 x:0 0 0 2024-05-17T21:53:03.294 INFO:tasks.workunit.client.1.smithi138.stdout:3/680: link d1/d62/dad/cba d1/dc0/ddd/ced 0 2024-05-17T21:53:03.296 INFO:tasks.workunit.client.0.smithi107.stdout:2/588: symlink d0/d9/d1c/d3e/d4f/lbc 0 2024-05-17T21:53:03.298 INFO:tasks.workunit.client.1.smithi138.stdout:8/537: creat d0/d63/f9c x:0 0 0 2024-05-17T21:53:03.299 INFO:tasks.workunit.client.1.smithi138.stdout:8/538: readlink d0/dc/d3b/d4d/d7d/l90 0 2024-05-17T21:53:03.299 INFO:tasks.workunit.client.1.smithi138.stdout:5/556: rename d1/f45 to d1/d1a/d39/d4e/fb4 0 2024-05-17T21:53:03.300 INFO:tasks.workunit.client.1.smithi138.stdout:4/595: rmdir d0/d1b/d42 39 2024-05-17T21:53:03.301 INFO:tasks.workunit.client.1.smithi138.stdout:3/681: fsync d1/d62/dad/d72/fa4 0 2024-05-17T21:53:03.302 INFO:tasks.workunit.client.1.smithi138.stdout:3/682: truncate d1/d4d/fea 261347 0 2024-05-17T21:53:03.307 INFO:tasks.workunit.client.1.smithi138.stdout:5/557: rename d1/d6 to d1/d2c/d63/db5 0 2024-05-17T21:53:03.308 INFO:tasks.workunit.client.1.smithi138.stdout:4/596: dwrite d0/d2f/d49/d60/d1f/d7e/f93 [0,4194304] 0 2024-05-17T21:53:03.309 INFO:tasks.workunit.client.0.smithi107.stdout:7/708: sync 2024-05-17T21:53:03.309 INFO:tasks.workunit.client.1.smithi138.stdout:4/597: stat d0/d2f/d49/d60/d1f/f70 0 2024-05-17T21:53:03.310 INFO:tasks.workunit.client.0.smithi107.stdout:7/709: chown d5/df/d11/d1c/d31/d39/d92/d9a/cc6 2895 1 2024-05-17T21:53:03.312 INFO:tasks.workunit.client.0.smithi107.stdout:7/710: dread - d5/df/d11/d1c/d31/d56/f9c zero size 2024-05-17T21:53:03.312 INFO:tasks.workunit.client.0.smithi107.stdout:4/754: dread d3/db/f46 [0,4194304] 0 2024-05-17T21:53:03.313 INFO:tasks.workunit.client.0.smithi107.stdout:5/746: mknod d5/db/dc/d1e/de8/cf7 0 2024-05-17T21:53:03.314 INFO:tasks.workunit.client.1.smithi138.stdout:3/683: fsync d1/d62/dad/f34 0 2024-05-17T21:53:03.316 INFO:tasks.workunit.client.0.smithi107.stdout:9/777: fsync d0/dd/d2f/d87/d89/f97 0 2024-05-17T21:53:03.321 INFO:tasks.workunit.client.0.smithi107.stdout:2/589: rmdir d0/d26 39 2024-05-17T21:53:03.322 INFO:tasks.workunit.client.1.smithi138.stdout:4/598: symlink d0/d3/d17/d21/ld2 0 2024-05-17T21:53:03.323 INFO:tasks.workunit.client.0.smithi107.stdout:6/679: dread d9/d58/d7b/fae [0,4194304] 0 2024-05-17T21:53:03.323 INFO:tasks.workunit.client.1.smithi138.stdout:8/539: truncate d0/dc/d1a/f1d 3909299 0 2024-05-17T21:53:03.323 INFO:tasks.workunit.client.1.smithi138.stdout:4/599: readlink d0/d2f/d49/d60/l28 0 2024-05-17T21:53:03.324 INFO:tasks.workunit.client.0.smithi107.stdout:3/774: mkdir d6/dfd 0 2024-05-17T21:53:03.325 INFO:tasks.workunit.client.0.smithi107.stdout:0/854: dwrite d3/dc/d22/d23/f44 [0,4194304] 0 2024-05-17T21:53:03.328 INFO:tasks.workunit.client.1.smithi138.stdout:5/558: creat d1/d53/d88/d99/fb6 x:0 0 0 2024-05-17T21:53:03.329 INFO:tasks.workunit.client.1.smithi138.stdout:5/559: dread - d1/d2c/d63/db5/d1b/d29/d36/d68/f6c zero size 2024-05-17T21:53:03.330 INFO:tasks.workunit.client.1.smithi138.stdout:4/600: read - d0/d1b/d2b/d55/d79/f7b zero size 2024-05-17T21:53:03.331 INFO:tasks.workunit.client.1.smithi138.stdout:5/560: chown d1/d2c/d63/db5/d1b/d29/d36/f7a 786546271 1 2024-05-17T21:53:03.332 INFO:tasks.workunit.client.0.smithi107.stdout:0/855: dwrite d3/dc/d22/d23/f44 [0,4194304] 0 2024-05-17T21:53:03.332 INFO:tasks.workunit.client.1.smithi138.stdout:3/684: mkdir d1/dee 0 2024-05-17T21:53:03.333 INFO:tasks.workunit.client.1.smithi138.stdout:3/685: chown d1/d62/dad/f64 1 1 2024-05-17T21:53:03.334 INFO:tasks.workunit.client.0.smithi107.stdout:4/755: dread d3/d4a/d47/d60/f12 [0,4194304] 0 2024-05-17T21:53:03.334 INFO:tasks.workunit.client.1.smithi138.stdout:8/540: creat d0/d71/d92/f9d x:0 0 0 2024-05-17T21:53:03.335 INFO:tasks.workunit.client.1.smithi138.stdout:4/601: creat d0/d3/db/fd3 x:0 0 0 2024-05-17T21:53:03.337 INFO:tasks.workunit.client.0.smithi107.stdout:9/778: unlink d0/dd/d21/d29/dd4/lb1 0 2024-05-17T21:53:03.337 INFO:tasks.workunit.client.1.smithi138.stdout:3/686: mknod d1/d62/d9b/cef 0 2024-05-17T21:53:03.338 INFO:tasks.workunit.client.0.smithi107.stdout:4/756: dread d3/db/d1e/f9a [0,4194304] 0 2024-05-17T21:53:03.338 INFO:tasks.workunit.client.1.smithi138.stdout:8/541: creat d0/dc/d3b/d4d/d7d/f9e x:0 0 0 2024-05-17T21:53:03.339 INFO:tasks.workunit.client.0.smithi107.stdout:4/757: chown d3/db/d1e/d99/dd6 1010507 1 2024-05-17T21:53:03.342 INFO:tasks.workunit.client.0.smithi107.stdout:8/719: getdents dc/ddb 0 2024-05-17T21:53:03.342 INFO:tasks.workunit.client.1.smithi138.stdout:3/687: chown d1/d4f/d50/c99 0 1 2024-05-17T21:53:03.343 INFO:tasks.workunit.client.1.smithi138.stdout:4/602: mkdir d0/d1b/dd4 0 2024-05-17T21:53:03.344 INFO:tasks.workunit.client.0.smithi107.stdout:3/775: mknod d6/d8/d10/d35/d36/d4c/cfe 0 2024-05-17T21:53:03.344 INFO:tasks.workunit.client.1.smithi138.stdout:3/688: write d1/d3/d16/d5d/feb [916301,72471] 0 2024-05-17T21:53:03.345 INFO:tasks.workunit.client.1.smithi138.stdout:8/542: chown d0/d11/c85 1727 1 2024-05-17T21:53:03.346 INFO:tasks.workunit.client.1.smithi138.stdout:4/603: truncate d0/d2f/d49/d60/d1f/d7e/fce 611560 0 2024-05-17T21:53:03.347 INFO:tasks.workunit.client.1.smithi138.stdout:3/689: write d1/d62/dad/f34 [4522004,70565] 0 2024-05-17T21:53:03.348 INFO:tasks.workunit.client.0.smithi107.stdout:1/780: creat d4/df/d14/d2e/d43/ff8 x:0 0 0 2024-05-17T21:53:03.349 INFO:tasks.workunit.client.0.smithi107.stdout:5/747: mkdir d5/dec/df8 0 2024-05-17T21:53:03.349 INFO:tasks.workunit.client.0.smithi107.stdout:2/590: dread d0/d26/f6c [0,4194304] 0 2024-05-17T21:53:03.350 INFO:tasks.workunit.client.0.smithi107.stdout:5/748: chown d5/db/d16/d2a/d73 330902630 1 2024-05-17T21:53:03.351 INFO:tasks.workunit.client.1.smithi138.stdout:4/604: creat d0/d1b/d2b/d55/d79/fd5 x:0 0 0 2024-05-17T21:53:03.354 INFO:tasks.workunit.client.1.smithi138.stdout:3/690: chown d1/dc0/ddd/ced 1 1 2024-05-17T21:53:03.354 INFO:tasks.workunit.client.1.smithi138.stdout:8/543: dwrite d0/dc/f22 [8388608,4194304] 0 2024-05-17T21:53:03.354 INFO:tasks.workunit.client.1.smithi138.stdout:8/544: read d0/d2f/f6e [638690,37048] 0 2024-05-17T21:53:03.355 INFO:tasks.workunit.client.1.smithi138.stdout:4/605: mknod d0/d1b/d88/cd6 0 2024-05-17T21:53:03.357 INFO:tasks.workunit.client.0.smithi107.stdout:5/749: dwrite d5/db/d16/d54/da0/fc0 [0,4194304] 0 2024-05-17T21:53:03.358 INFO:tasks.workunit.client.1.smithi138.stdout:5/561: dread d1/d1a/d39/d4e/fb4 [0,4194304] 0 2024-05-17T21:53:03.359 INFO:tasks.workunit.client.1.smithi138.stdout:5/562: fdatasync d1/d1a/d39/d4e/f51 0 2024-05-17T21:53:03.361 INFO:tasks.workunit.client.1.smithi138.stdout:3/691: creat d1/d62/dad/d5c/d9c/ff0 x:0 0 0 2024-05-17T21:53:03.361 INFO:tasks.workunit.client.1.smithi138.stdout:3/692: stat d1/d4f/d80 0 2024-05-17T21:53:03.367 INFO:tasks.workunit.client.0.smithi107.stdout:1/781: mkdir d4/df/d1a/d26/d72/d85/df9 0 2024-05-17T21:53:03.367 INFO:tasks.workunit.client.0.smithi107.stdout:1/782: stat d4/df/d1a/cd0 0 2024-05-17T21:53:03.368 INFO:tasks.workunit.client.1.smithi138.stdout:8/545: creat d0/d71/f9f x:0 0 0 2024-05-17T21:53:03.371 INFO:tasks.workunit.client.0.smithi107.stdout:7/711: link d5/df/d11/d1c/d2f/d53/fb2 d5/df/d11/d1d/d33/d82/ffb 0 2024-05-17T21:53:03.372 INFO:tasks.workunit.client.1.smithi138.stdout:5/563: mkdir d1/d2c/d63/db5/d1b/db7 0 2024-05-17T21:53:03.373 INFO:tasks.workunit.client.0.smithi107.stdout:6/680: write d9/d16/d4d/db8/dc9/dd0/f70 [160772,47259] 0 2024-05-17T21:53:03.373 INFO:tasks.workunit.client.0.smithi107.stdout:6/681: chown d9/d16/ca6 71177253 1 2024-05-17T21:53:03.374 INFO:tasks.workunit.client.0.smithi107.stdout:7/712: read d5/d16/d3b/d45/d91/fc7 [872553,12840] 0 2024-05-17T21:53:03.374 INFO:tasks.workunit.client.1.smithi138.stdout:5/564: truncate d1/d1a/d39/fa6 4384150 0 2024-05-17T21:53:03.374 INFO:tasks.workunit.client.0.smithi107.stdout:7/713: fsync d5/d16/f3a 0 2024-05-17T21:53:03.376 INFO:tasks.workunit.client.0.smithi107.stdout:7/714: chown d5/df/d11/d1c/d2f/d65/da7/fad 13 1 2024-05-17T21:53:03.377 INFO:tasks.workunit.client.1.smithi138.stdout:4/606: creat d0/d1b/d42/fd7 x:0 0 0 2024-05-17T21:53:03.379 INFO:tasks.workunit.client.0.smithi107.stdout:9/779: rename d0/dd/f35 to d0/dd/d2f/d4a/f101 0 2024-05-17T21:53:03.380 INFO:tasks.workunit.client.1.smithi138.stdout:8/546: mkdir d0/dc/d3b/da0 0 2024-05-17T21:53:03.380 INFO:tasks.workunit.client.0.smithi107.stdout:6/682: dwrite d9/d16/d4d/db8/dc9/dd0/fb5 [0,4194304] 0 2024-05-17T21:53:03.382 INFO:tasks.workunit.client.1.smithi138.stdout:5/565: chown d1/d60/d2f/c56 13 1 2024-05-17T21:53:03.383 INFO:tasks.workunit.client.0.smithi107.stdout:9/780: write d0/dd/d21/d29/d57/d5f/ff5 [681558,74076] 0 2024-05-17T21:53:03.383 INFO:tasks.workunit.client.0.smithi107.stdout:4/758: mkdir d3/d6/d26/d36/d8b/d95/dee/dfb 0 2024-05-17T21:53:03.384 INFO:tasks.workunit.client.0.smithi107.stdout:6/683: chown d9/d16/d4d/db8/dc9/dd0/c51 31152530 1 2024-05-17T21:53:03.384 INFO:tasks.workunit.client.0.smithi107.stdout:7/715: dwrite d5/d16/dd3/fdd [0,4194304] 0 2024-05-17T21:53:03.385 INFO:tasks.workunit.client.1.smithi138.stdout:3/693: symlink d1/d62/db0/lf1 0 2024-05-17T21:53:03.386 INFO:tasks.workunit.client.0.smithi107.stdout:5/750: fsync d5/db/d16/d54/d59/d65/fac 0 2024-05-17T21:53:03.386 INFO:tasks.workunit.client.0.smithi107.stdout:6/684: write d9/d58/d7b/da3/fc8 [43006,59722] 0 2024-05-17T21:53:03.388 INFO:tasks.workunit.client.1.smithi138.stdout:4/607: mkdir d0/d3/d17/dd8 0 2024-05-17T21:53:03.389 INFO:tasks.workunit.client.0.smithi107.stdout:5/751: read d5/db/dc/d1e/fb5 [396616,26868] 0 2024-05-17T21:53:03.390 INFO:tasks.workunit.client.0.smithi107.stdout:9/781: dread d0/dd/d1b/f53 [0,4194304] 0 2024-05-17T21:53:03.394 INFO:tasks.workunit.client.0.smithi107.stdout:5/752: dwrite d5/db/d16/f9d [0,4194304] 0 2024-05-17T21:53:03.402 INFO:tasks.workunit.client.0.smithi107.stdout:5/753: dwrite d5/db/d16/d54/d59/d65/f47 [4194304,4194304] 0 2024-05-17T21:53:03.454 INFO:tasks.workunit.client.1.smithi138.stdout:1/552: sync 2024-05-17T21:53:03.455 INFO:tasks.workunit.client.1.smithi138.stdout:1/553: chown d4/df/d4e/c9c 1466018 1 2024-05-17T21:53:03.458 INFO:tasks.workunit.client.1.smithi138.stdout:3/694: mkdir d1/d4d/df2 0 2024-05-17T21:53:03.459 INFO:tasks.workunit.client.1.smithi138.stdout:3/695: chown d1/d3/d8d 0 1 2024-05-17T21:53:03.463 INFO:tasks.workunit.client.0.smithi107.stdout:8/720: dwrite dc/f18 [4194304,4194304] 0 2024-05-17T21:53:03.464 INFO:tasks.workunit.client.0.smithi107.stdout:8/721: fsync dc/d21/d27/d34/fe1 0 2024-05-17T21:53:03.464 INFO:tasks.workunit.client.0.smithi107.stdout:0/856: link d3/dc/d22/d23/f29 d3/dc/dae/dc0/de1/f11c 0 2024-05-17T21:53:03.465 INFO:tasks.workunit.client.0.smithi107.stdout:8/722: write dc/d21/d27/d34/f6e [1566058,14186] 0 2024-05-17T21:53:03.467 INFO:tasks.workunit.client.0.smithi107.stdout:8/723: write dc/d21/d27/d2f/f30 [3047086,83982] 0 2024-05-17T21:53:03.469 INFO:tasks.workunit.client.1.smithi138.stdout:3/696: creat d1/d4f/d87/ff3 x:0 0 0 2024-05-17T21:53:03.471 INFO:tasks.workunit.client.1.smithi138.stdout:3/697: fdatasync d1/d3/d8d/fc7 0 2024-05-17T21:53:03.472 INFO:tasks.workunit.client.1.smithi138.stdout:4/608: symlink d0/d3/ld9 0 2024-05-17T21:53:03.472 INFO:tasks.workunit.client.1.smithi138.stdout:4/609: stat d0/d3/d17/c66 0 2024-05-17T21:53:03.479 INFO:tasks.workunit.client.1.smithi138.stdout:3/698: creat d1/d4f/d87/ff4 x:0 0 0 2024-05-17T21:53:03.479 INFO:tasks.workunit.client.1.smithi138.stdout:4/610: dwrite d0/fcb [0,4194304] 0 2024-05-17T21:53:03.483 INFO:tasks.workunit.client.1.smithi138.stdout:5/566: getdents d1/d8a 0 2024-05-17T21:53:03.484 INFO:tasks.workunit.client.1.smithi138.stdout:3/699: symlink d1/d4d/lf5 0 2024-05-17T21:53:03.487 INFO:tasks.workunit.client.0.smithi107.stdout:6/685: chown d9/d16/d9e/lbe 2 1 2024-05-17T21:53:03.491 INFO:tasks.workunit.client.0.smithi107.stdout:3/776: link d6/d1a/d57/lc2 d6/d8/d10/d25/lff 0 2024-05-17T21:53:03.493 INFO:tasks.workunit.client.1.smithi138.stdout:5/567: mkdir d1/d53/db1/db8 0 2024-05-17T21:53:03.500 INFO:tasks.workunit.client.1.smithi138.stdout:5/568: rename d1/d1a to d1/d53/d88/db9 0 2024-05-17T21:53:03.501 INFO:tasks.workunit.client.1.smithi138.stdout:5/569: chown d1/d2c/d63/db5/d1b/f1d 215 1 2024-05-17T21:53:03.505 INFO:tasks.workunit.client.1.smithi138.stdout:4/611: getdents d0 0 2024-05-17T21:53:03.506 INFO:tasks.workunit.client.0.smithi107.stdout:7/716: write d5/df/d11/d1c/d2f/d53/f90 [1471880,73300] 0 2024-05-17T21:53:03.506 INFO:tasks.workunit.client.1.smithi138.stdout:3/700: getdents d1/d62/dad/d72 0 2024-05-17T21:53:03.508 INFO:tasks.workunit.client.0.smithi107.stdout:4/759: dwrite d3/db/f4d [0,4194304] 0 2024-05-17T21:53:03.508 INFO:tasks.workunit.client.0.smithi107.stdout:7/717: chown d5/df/d11/d1d/d33/d82/fa0 342396880 1 2024-05-17T21:53:03.510 INFO:tasks.workunit.client.0.smithi107.stdout:0/857: rmdir d3/d24/d6f/d9f/dab 39 2024-05-17T21:53:03.513 INFO:tasks.workunit.client.1.smithi138.stdout:4/612: mknod d0/d3/d1d/d38/d9c/cda 0 2024-05-17T21:53:03.514 INFO:tasks.workunit.client.0.smithi107.stdout:5/754: dread d5/db/d16/d54/d59/d65/f6f [0,4194304] 0 2024-05-17T21:53:03.515 INFO:tasks.workunit.client.1.smithi138.stdout:5/570: dwrite d1/d53/d88/db9/d39/d4e/fb4 [0,4194304] 0 2024-05-17T21:53:03.519 INFO:tasks.workunit.client.1.smithi138.stdout:3/701: mkdir d1/d3/d16/d2d/df6 0 2024-05-17T21:53:03.521 INFO:tasks.workunit.client.1.smithi138.stdout:3/702: readlink d1/d4f/d87/lce 0 2024-05-17T21:53:03.521 INFO:tasks.workunit.client.1.smithi138.stdout:4/613: mknod d0/d3/d17/d21/d78/cdb 0 2024-05-17T21:53:03.523 INFO:tasks.workunit.client.0.smithi107.stdout:1/783: rename d4/df/d1a/d26/ff1 to d4/df/d14/d2e/ffa 0 2024-05-17T21:53:03.523 INFO:tasks.workunit.client.1.smithi138.stdout:5/571: creat d1/d2c/d63/db5/d1b/fba x:0 0 0 2024-05-17T21:53:03.525 INFO:tasks.workunit.client.1.smithi138.stdout:3/703: creat d1/d3/d49/ff7 x:0 0 0 2024-05-17T21:53:03.525 INFO:tasks.workunit.client.0.smithi107.stdout:8/724: dread dc/d44/d4b/d52/d70/d76/f78 [0,4194304] 0 2024-05-17T21:53:03.526 INFO:tasks.workunit.client.1.smithi138.stdout:4/614: mkdir d0/d3/d1d/d38/d8a/dbd/ddc 0 2024-05-17T21:53:03.526 INFO:tasks.workunit.client.0.smithi107.stdout:1/784: write d4/df/d14/d2e/d43/ff8 [10421,58448] 0 2024-05-17T21:53:03.528 INFO:tasks.workunit.client.0.smithi107.stdout:1/785: stat d4/df/d1a/d26/d72/daa/le2 0 2024-05-17T21:53:03.529 INFO:tasks.workunit.client.0.smithi107.stdout:6/686: mkdir d9/d16/d22/d5f/dd8 0 2024-05-17T21:53:03.530 INFO:tasks.workunit.client.0.smithi107.stdout:6/687: chown d9/d58 3 1 2024-05-17T21:53:03.530 INFO:tasks.workunit.client.0.smithi107.stdout:6/688: truncate f5 4773416 0 2024-05-17T21:53:03.536 INFO:tasks.workunit.client.0.smithi107.stdout:2/591: creat d0/d9/d11/d18/dad/db8/fbd x:0 0 0 2024-05-17T21:53:03.537 INFO:tasks.workunit.client.0.smithi107.stdout:4/760: fsync d3/d6/d26/d36/d7e/f89 0 2024-05-17T21:53:03.538 INFO:tasks.workunit.client.0.smithi107.stdout:2/592: truncate d0/d9/d1c/d37/d9f/fa7 4442944 0 2024-05-17T21:53:03.539 INFO:tasks.workunit.client.0.smithi107.stdout:7/718: creat d5/df/d11/d1d/d33/d82/ffc x:0 0 0 2024-05-17T21:53:03.540 INFO:tasks.workunit.client.0.smithi107.stdout:5/755: symlink d5/db/d16/d54/d59/d65/d48/d89/lf9 0 2024-05-17T21:53:03.541 INFO:tasks.workunit.client.0.smithi107.stdout:5/756: write d5/db/dc/d1e/d6e/fc8 [521767,106791] 0 2024-05-17T21:53:03.542 INFO:tasks.workunit.client.0.smithi107.stdout:8/725: creat dc/d21/d35/db6/f103 x:0 0 0 2024-05-17T21:53:03.543 INFO:tasks.workunit.client.0.smithi107.stdout:8/726: dread - dc/d21/d97/db7/ff9 zero size 2024-05-17T21:53:03.543 INFO:tasks.workunit.client.1.smithi138.stdout:5/572: dwrite d1/d2c/d63/db5/d1b/d29/d36/d68/f6f [0,4194304] 0 2024-05-17T21:53:03.544 INFO:tasks.workunit.client.0.smithi107.stdout:6/689: mknod d9/d16/d4d/db8/cd9 0 2024-05-17T21:53:03.544 INFO:tasks.workunit.client.0.smithi107.stdout:8/727: chown dc/d21/d27/d2f/c9e 97838 1 2024-05-17T21:53:03.546 INFO:tasks.workunit.client.0.smithi107.stdout:9/782: link d0/dd/d1f/l2b d0/dd/d3d/d9d/l102 0 2024-05-17T21:53:03.547 INFO:tasks.workunit.client.0.smithi107.stdout:8/728: read dc/d21/d27/d2f/f4a [340004,114007] 0 2024-05-17T21:53:03.547 INFO:tasks.workunit.client.0.smithi107.stdout:3/777: dwrite d6/f1c [0,4194304] 0 2024-05-17T21:53:03.548 INFO:tasks.workunit.client.1.smithi138.stdout:4/615: dwrite d0/d3/d17/d21/d78/f9a [0,4194304] 0 2024-05-17T21:53:03.548 INFO:tasks.workunit.client.1.smithi138.stdout:5/573: fdatasync d1/d60/f5c 0 2024-05-17T21:53:03.551 INFO:tasks.workunit.client.0.smithi107.stdout:0/858: mknod d3/dc/dae/c11d 0 2024-05-17T21:53:03.552 INFO:tasks.workunit.client.0.smithi107.stdout:7/719: creat d5/df/d11/d1c/d31/d39/d92/d9a/ffd x:0 0 0 2024-05-17T21:53:03.555 INFO:tasks.workunit.client.1.smithi138.stdout:4/616: rename d0/d3/d17/dd8 to d0/d3/d17/d21/ddd 0 2024-05-17T21:53:03.557 INFO:tasks.workunit.client.0.smithi107.stdout:1/786: creat d4/df/d1a/d26/d72/d85/df9/ffb x:0 0 0 2024-05-17T21:53:03.559 INFO:tasks.workunit.client.0.smithi107.stdout:5/757: chown d5/db/d16/d2d/d40/d9f/fef 1934807 1 2024-05-17T21:53:03.561 INFO:tasks.workunit.client.0.smithi107.stdout:7/720: dwrite d5/df/d11/d1d/f8c [0,4194304] 0 2024-05-17T21:53:03.562 INFO:tasks.workunit.client.1.smithi138.stdout:4/617: rmdir d0/d3/db 39 2024-05-17T21:53:03.564 INFO:tasks.workunit.client.1.smithi138.stdout:4/618: rmdir d0 39 2024-05-17T21:53:03.566 INFO:tasks.workunit.client.0.smithi107.stdout:6/690: dwrite d9/d16/d9e/fd1 [0,4194304] 0 2024-05-17T21:53:03.576 INFO:tasks.workunit.client.0.smithi107.stdout:2/593: rename d0/d9/d11/d15/d99/c9a to d0/d9/d1c/d3e/d4f/cbe 0 2024-05-17T21:53:03.579 INFO:tasks.workunit.client.1.smithi138.stdout:4/619: unlink d0/d2f/d49/d60/d1f/d32/c4b 0 2024-05-17T21:53:03.580 INFO:tasks.workunit.client.0.smithi107.stdout:3/778: truncate f5 1353709 0 2024-05-17T21:53:03.581 INFO:tasks.workunit.client.0.smithi107.stdout:9/783: creat d0/dd/d21/d29/d3e/dfe/f103 x:0 0 0 2024-05-17T21:53:03.583 INFO:tasks.workunit.client.0.smithi107.stdout:4/761: creat d3/d6/ffc x:0 0 0 2024-05-17T21:53:03.584 INFO:tasks.workunit.client.0.smithi107.stdout:4/762: fdatasync d3/db/d1e/d99/df7/ff8 0 2024-05-17T21:53:03.585 INFO:tasks.workunit.client.1.smithi138.stdout:5/574: dread d1/d2c/d63/db5/fa [0,4194304] 0 2024-05-17T21:53:03.590 INFO:tasks.workunit.client.1.smithi138.stdout:4/620: getdents d0/d1b/d42/d45 0 2024-05-17T21:53:03.592 INFO:tasks.workunit.client.1.smithi138.stdout:5/575: fsync d1/d53/d88/db9/d39/d62/f7e 0 2024-05-17T21:53:03.595 INFO:tasks.workunit.client.0.smithi107.stdout:5/758: write d5/db/d16/d54/d59/d65/fac [1329298,117498] 0 2024-05-17T21:53:03.595 INFO:tasks.workunit.client.0.smithi107.stdout:0/859: dwrite d3/dc/d22/d3d/dc7/fcc [0,4194304] 0 2024-05-17T21:53:03.601 INFO:tasks.workunit.client.0.smithi107.stdout:2/594: unlink d0/d9/d11/d18/l5e 0 2024-05-17T21:53:03.605 INFO:tasks.workunit.client.1.smithi138.stdout:4/621: rename d0/d2f/la2 to d0/d2f/d49/d60/d1f/d32/d71/d95/lde 0 2024-05-17T21:53:03.606 INFO:tasks.workunit.client.0.smithi107.stdout:3/779: write d6/d2a/d2d/d69/fc5 [865274,60471] 0 2024-05-17T21:53:03.606 INFO:tasks.workunit.client.1.smithi138.stdout:5/576: read d1/d2c/d63/db5/f31 [865740,6827] 0 2024-05-17T21:53:03.607 INFO:tasks.workunit.client.0.smithi107.stdout:6/691: dwrite d9/d16/d4d/d67/f86 [0,4194304] 0 2024-05-17T21:53:03.608 INFO:tasks.workunit.client.1.smithi138.stdout:4/622: truncate d0/d1b/d42/f7c 546668 0 2024-05-17T21:53:03.611 INFO:tasks.workunit.client.0.smithi107.stdout:5/759: creat d5/db/dc/d1e/d6e/ffa x:0 0 0 2024-05-17T21:53:03.612 INFO:tasks.workunit.client.0.smithi107.stdout:5/760: chown d5/db/d16/d2d/d4d/f6a 14 1 2024-05-17T21:53:03.613 INFO:tasks.workunit.client.0.smithi107.stdout:1/787: dread d4/df/d4e/d59/fcc [0,4194304] 0 2024-05-17T21:53:03.615 INFO:tasks.workunit.client.0.smithi107.stdout:5/761: write d5/db/d16/d5b/f97 [2101939,14530] 0 2024-05-17T21:53:03.616 INFO:tasks.workunit.client.0.smithi107.stdout:4/763: truncate d3/db/f33 3250537 0 2024-05-17T21:53:03.617 INFO:tasks.workunit.client.1.smithi138.stdout:4/623: mknod d0/d1b/d2b/d55/cdf 0 2024-05-17T21:53:03.618 INFO:tasks.workunit.client.0.smithi107.stdout:4/764: chown d3/db/d1e/l2a 6 1 2024-05-17T21:53:03.618 INFO:tasks.workunit.client.0.smithi107.stdout:6/692: dwrite d9/d16/d4d/d67/f79 [4194304,4194304] 0 2024-05-17T21:53:03.619 INFO:tasks.workunit.client.1.smithi138.stdout:5/577: creat d1/d2c/d63/db5/d1b/d29/fbb x:0 0 0 2024-05-17T21:53:03.621 INFO:tasks.workunit.client.0.smithi107.stdout:7/721: getdents d5/df/d11/d1c/d2f 0 2024-05-17T21:53:03.621 INFO:tasks.workunit.client.1.smithi138.stdout:4/624: chown d0/d3/d17/d5e/d7f/l85 2391 1 2024-05-17T21:53:03.624 INFO:tasks.workunit.client.0.smithi107.stdout:2/595: write d0/fa [6994304,57735] 0 2024-05-17T21:53:03.628 INFO:tasks.workunit.client.1.smithi138.stdout:5/578: dwrite d1/d53/d88/db9/d39/d62/fac [0,4194304] 0 2024-05-17T21:53:03.634 INFO:tasks.workunit.client.0.smithi107.stdout:6/693: truncate d9/f48 22156 0 2024-05-17T21:53:03.635 INFO:tasks.workunit.client.0.smithi107.stdout:7/722: write d5/d16/d3b/d45/d91/db3/fed [801495,232] 0 2024-05-17T21:53:03.637 INFO:tasks.workunit.client.0.smithi107.stdout:7/723: dread - d5/d16/d3b/d45/d91/db3/fd6 zero size 2024-05-17T21:53:03.638 INFO:tasks.workunit.client.0.smithi107.stdout:2/596: mkdir d0/d9/d11/d27/dbf 0 2024-05-17T21:53:03.638 INFO:tasks.workunit.client.1.smithi138.stdout:5/579: dread d1/d2c/f8b [0,4194304] 0 2024-05-17T21:53:03.640 INFO:tasks.workunit.client.0.smithi107.stdout:7/724: truncate d5/d16/d3b/d45/d91/dc1/fea 12066 0 2024-05-17T21:53:03.641 INFO:tasks.workunit.client.0.smithi107.stdout:7/725: chown d5/df/d11/d1d/f8c 96520 1 2024-05-17T21:53:03.641 INFO:tasks.workunit.client.0.smithi107.stdout:1/788: rename d4/d78/fab to d4/df/d1a/d26/d72/daa/ffc 0 2024-05-17T21:53:03.642 INFO:tasks.workunit.client.0.smithi107.stdout:7/726: read - d5/df/d11/d1d/d33/d82/ffc zero size 2024-05-17T21:53:03.643 INFO:tasks.workunit.client.0.smithi107.stdout:4/765: truncate d3/d4a/f73 1877275 0 2024-05-17T21:53:03.645 INFO:tasks.workunit.client.1.smithi138.stdout:5/580: creat d1/d53/d88/db9/d39/d62/fbc x:0 0 0 2024-05-17T21:53:03.647 INFO:tasks.workunit.client.0.smithi107.stdout:0/860: getdents d3/d24/d6f/db5/da5 0 2024-05-17T21:53:03.648 INFO:tasks.workunit.client.1.smithi138.stdout:5/581: unlink d1/d2c/d63/db5/d1b/d29/d36/f37 0 2024-05-17T21:53:03.650 INFO:tasks.workunit.client.0.smithi107.stdout:3/780: truncate f3 4450410 0 2024-05-17T21:53:03.650 INFO:tasks.workunit.client.0.smithi107.stdout:2/597: read d0/d9/d1c/d3e/d4f/f68 [262458,19532] 0 2024-05-17T21:53:03.651 INFO:tasks.workunit.client.0.smithi107.stdout:5/762: rename d5/db/dc/c10 to d5/db/d16/d54/d59/d92/cfb 0 2024-05-17T21:53:03.652 INFO:tasks.workunit.client.0.smithi107.stdout:3/781: dread d6/d8/d3e/d3f/ffc [0,4194304] 0 2024-05-17T21:53:03.652 INFO:tasks.workunit.client.1.smithi138.stdout:5/582: symlink d1/d53/d88/d99/lbd 0 2024-05-17T21:53:03.652 INFO:tasks.workunit.client.0.smithi107.stdout:3/782: readlink d6/d8/d3e/d3f/d90/l9b 0 2024-05-17T21:53:03.653 INFO:tasks.workunit.client.0.smithi107.stdout:3/783: write d6/d2a/d2d/d69/fc5 [256450,19341] 0 2024-05-17T21:53:03.655 INFO:tasks.workunit.client.0.smithi107.stdout:3/784: chown d6/d8/d10/d25/f49 4 1 2024-05-17T21:53:03.657 INFO:tasks.workunit.client.0.smithi107.stdout:7/727: mkdir d5/df/d11/d1c/d2f/dfe 0 2024-05-17T21:53:03.657 INFO:tasks.workunit.client.0.smithi107.stdout:7/728: chown d5/df 7 1 2024-05-17T21:53:03.665 INFO:tasks.workunit.client.0.smithi107.stdout:4/766: rename d3/d6/d26/d36/d37/c84 to d3/d4a/d47/d60/d3c/d9d/cfd 0 2024-05-17T21:53:03.666 INFO:tasks.workunit.client.1.smithi138.stdout:5/583: write d1/d46/f78 [461732,71664] 0 2024-05-17T21:53:03.668 INFO:tasks.workunit.client.0.smithi107.stdout:2/598: dwrite d0/d9/d1c/d3e/f62 [0,4194304] 0 2024-05-17T21:53:03.670 INFO:tasks.workunit.client.0.smithi107.stdout:2/599: fdatasync d0/d9/d1c/d22/f65 0 2024-05-17T21:53:03.670 INFO:tasks.workunit.client.0.smithi107.stdout:7/729: stat d5/d16/f28 0 2024-05-17T21:53:03.671 INFO:tasks.workunit.client.0.smithi107.stdout:7/730: dread - d5/d16/d3b/d45/ff8 zero size 2024-05-17T21:53:03.675 INFO:tasks.workunit.client.0.smithi107.stdout:1/789: rename d4/df/d69/lba to d4/df/d1a/d41/db2/dc8/lfd 0 2024-05-17T21:53:03.676 INFO:tasks.workunit.client.0.smithi107.stdout:4/767: mknod d3/d4a/d47/d60/d3a/d41/cfe 0 2024-05-17T21:53:03.677 INFO:tasks.workunit.client.0.smithi107.stdout:4/768: dread - d3/db/d1e/dc6/fea zero size 2024-05-17T21:53:03.678 INFO:tasks.workunit.client.0.smithi107.stdout:6/694: link d9/d16/d4d/l99 d9/d16/d22/d5f/dd8/lda 0 2024-05-17T21:53:03.679 INFO:tasks.workunit.client.0.smithi107.stdout:2/600: chown d0/d9/d11/d18/f20 102828 1 2024-05-17T21:53:03.681 INFO:tasks.workunit.client.0.smithi107.stdout:7/731: creat d5/d16/d3b/d45/d91/db3/fff x:0 0 0 2024-05-17T21:53:03.683 INFO:tasks.workunit.client.0.smithi107.stdout:1/790: fsync d4/df/d14/d1d/f2f 0 2024-05-17T21:53:03.684 INFO:tasks.workunit.client.0.smithi107.stdout:1/791: chown d4/d6a/fe9 42 1 2024-05-17T21:53:03.684 INFO:tasks.workunit.client.0.smithi107.stdout:6/695: symlink d9/d58/d7b/ldb 0 2024-05-17T21:53:03.685 INFO:tasks.workunit.client.0.smithi107.stdout:2/601: truncate d0/d9/d1c/f3d 483335 0 2024-05-17T21:53:03.685 INFO:tasks.workunit.client.0.smithi107.stdout:0/861: link d3/f19 d3/d24/f11e 0 2024-05-17T21:53:03.685 INFO:tasks.workunit.client.0.smithi107.stdout:6/696: readlink d9/d58/l69 0 2024-05-17T21:53:03.686 INFO:tasks.workunit.client.0.smithi107.stdout:5/763: getdents d5/d75/dc7 0 2024-05-17T21:53:03.691 INFO:tasks.workunit.client.1.smithi138.stdout:0/572: sync 2024-05-17T21:53:03.691 INFO:tasks.workunit.client.1.smithi138.stdout:6/594: sync 2024-05-17T21:53:03.691 INFO:tasks.workunit.client.1.smithi138.stdout:9/507: sync 2024-05-17T21:53:03.692 INFO:tasks.workunit.client.1.smithi138.stdout:2/603: sync 2024-05-17T21:53:03.692 INFO:tasks.workunit.client.0.smithi107.stdout:1/792: unlink f3 0 2024-05-17T21:53:03.698 INFO:tasks.workunit.client.0.smithi107.stdout:7/732: mkdir d5/df/d11/d100 0 2024-05-17T21:53:03.699 INFO:tasks.workunit.client.0.smithi107.stdout:1/793: dwrite d4/df/d1a/d26/d72/d85/fd7 [0,4194304] 0 2024-05-17T21:53:03.700 INFO:tasks.workunit.client.0.smithi107.stdout:1/794: chown d4/df/d1a/d26/d72/d85 26 1 2024-05-17T21:53:03.702 INFO:tasks.workunit.client.1.smithi138.stdout:5/584: dwrite d1/d60/f5c [0,4194304] 0 2024-05-17T21:53:03.710 INFO:tasks.workunit.client.1.smithi138.stdout:2/604: mknod d3/d21/d2f/d3e/d55/d99/d84/cc4 0 2024-05-17T21:53:03.712 INFO:tasks.workunit.client.1.smithi138.stdout:6/595: dread d7/f68 [0,4194304] 0 2024-05-17T21:53:03.712 INFO:tasks.workunit.client.1.smithi138.stdout:0/573: dread d3/d10/f22 [4194304,4194304] 0 2024-05-17T21:53:03.714 INFO:tasks.workunit.client.0.smithi107.stdout:0/862: mkdir d3/dc/dae/dc0/de1/d10b/d11f 0 2024-05-17T21:53:03.718 INFO:tasks.workunit.client.1.smithi138.stdout:9/508: mknod d6/d7/d16/d29/d48/d6e/d74/d75/d76/ca5 0 2024-05-17T21:53:03.720 INFO:tasks.workunit.client.0.smithi107.stdout:2/602: dread d0/d9/d11/f52 [0,4194304] 0 2024-05-17T21:53:03.721 INFO:tasks.workunit.client.0.smithi107.stdout:4/769: rename d3/d4a/d47/d60/d3a/d41 to d3/db/d35/d5e/dff 0 2024-05-17T21:53:03.722 INFO:tasks.workunit.client.0.smithi107.stdout:2/603: truncate d0/d9/d11/df/d5a/d6d/d82/f92 129851 0 2024-05-17T21:53:03.725 INFO:tasks.workunit.client.1.smithi138.stdout:9/509: dwrite d6/d7/d16/d29/d48/d3b/f88 [0,4194304] 0 2024-05-17T21:53:03.726 INFO:tasks.workunit.client.1.smithi138.stdout:5/585: rename d1/d2c/d63/db5/c8c to d1/d53/db1/cbe 0 2024-05-17T21:53:03.730 INFO:tasks.workunit.client.0.smithi107.stdout:7/733: symlink d5/df/d11/d2a/l101 0 2024-05-17T21:53:03.731 INFO:tasks.workunit.client.1.smithi138.stdout:6/596: readlink d7/d69/db5/dc8/d43/d5f/d94/lac 0 2024-05-17T21:53:03.732 INFO:tasks.workunit.client.1.smithi138.stdout:6/597: chown d7/d69/db5/dc8/d34/c47 0 1 2024-05-17T21:53:03.733 INFO:tasks.workunit.client.0.smithi107.stdout:6/697: write d9/d16/f7f [1423862,29531] 0 2024-05-17T21:53:03.738 INFO:tasks.workunit.client.0.smithi107.stdout:1/795: dread d4/df/fa8 [0,4194304] 0 2024-05-17T21:53:03.739 INFO:tasks.workunit.client.1.smithi138.stdout:2/605: rename d3/d21/d2f/d8a to d3/d21/d2f/d3e/d58/d77/da4/dc5 0 2024-05-17T21:53:03.739 INFO:tasks.workunit.client.0.smithi107.stdout:0/863: truncate d3/dc/d22/d23/d97/f96 1136058 0 2024-05-17T21:53:03.745 INFO:tasks.workunit.client.0.smithi107.stdout:0/864: dwrite d3/dc/d22/d23/d97/d77/d91/f100 [0,4194304] 0 2024-05-17T21:53:03.753 INFO:tasks.workunit.client.1.smithi138.stdout:0/574: truncate d3/d3d/f6a 192727 0 2024-05-17T21:53:03.759 INFO:tasks.workunit.client.1.smithi138.stdout:0/575: dwrite d3/d10/d6d/fa5 [0,4194304] 0 2024-05-17T21:53:03.791 INFO:tasks.workunit.client.1.smithi138.stdout:9/510: mkdir d6/d7/d16/d29/da6 0 2024-05-17T21:53:03.793 INFO:tasks.workunit.client.1.smithi138.stdout:6/598: mkdir d7/d69/db5/dc8/d43/d5f/da4/dcc 0 2024-05-17T21:53:03.797 INFO:tasks.workunit.client.1.smithi138.stdout:0/576: chown d3/d3d/f58 163930865 1 2024-05-17T21:53:03.798 INFO:tasks.workunit.client.1.smithi138.stdout:9/511: unlink d6/d7/d16/d29/d48/d67/d86/l96 0 2024-05-17T21:53:03.799 INFO:tasks.workunit.client.1.smithi138.stdout:9/512: fsync d6/d7/f99 0 2024-05-17T21:53:03.800 INFO:tasks.workunit.client.1.smithi138.stdout:6/599: creat d7/d69/db5/dc8/d34/da6/fcd x:0 0 0 2024-05-17T21:53:03.800 INFO:tasks.workunit.client.1.smithi138.stdout:6/600: fsync d7/d69/fb8 0 2024-05-17T21:53:03.802 INFO:tasks.workunit.client.0.smithi107.stdout:7/734: chown d5/df/d11/f5c 2 1 2024-05-17T21:53:03.804 INFO:tasks.workunit.client.0.smithi107.stdout:6/698: rename d9/d16/d9e/fd1 to d9/d16/d22/d5f/fdc 0 2024-05-17T21:53:03.806 INFO:tasks.workunit.client.0.smithi107.stdout:1/796: chown d4/df/c2d 25 1 2024-05-17T21:53:03.807 INFO:tasks.workunit.client.1.smithi138.stdout:5/586: rename d1/d8a/d9a to d1/d2c/dbf 0 2024-05-17T21:53:03.808 INFO:tasks.workunit.client.0.smithi107.stdout:4/770: symlink d3/d6/l100 0 2024-05-17T21:53:03.810 INFO:tasks.workunit.client.1.smithi138.stdout:0/577: dwrite d3/fa1 [0,4194304] 0 2024-05-17T21:53:03.810 INFO:tasks.workunit.client.1.smithi138.stdout:2/606: link d3/l2c d3/d21/d2f/d3e/d55/d99/d93/lc6 0 2024-05-17T21:53:03.814 INFO:tasks.workunit.client.1.smithi138.stdout:9/513: creat d6/d7/d16/d29/d48/d6e/d74/d75/d76/d8e/fa7 x:0 0 0 2024-05-17T21:53:03.814 INFO:tasks.workunit.client.0.smithi107.stdout:2/604: truncate d0/d9/d1c/d22/f65 1422404 0 2024-05-17T21:53:03.815 INFO:tasks.workunit.client.0.smithi107.stdout:4/771: dread d3/db/d1e/f6c [0,4194304] 0 2024-05-17T21:53:03.817 INFO:tasks.workunit.client.0.smithi107.stdout:0/865: creat d3/d75/d9e/f120 x:0 0 0 2024-05-17T21:53:03.818 INFO:tasks.workunit.client.1.smithi138.stdout:9/514: dread d6/f49 [0,4194304] 0 2024-05-17T21:53:03.823 INFO:tasks.workunit.client.1.smithi138.stdout:2/607: mknod d3/d81/cc7 0 2024-05-17T21:53:03.824 INFO:tasks.workunit.client.0.smithi107.stdout:5/764: getdents d5/db/d16/d2d/d40/dc2 0 2024-05-17T21:53:03.830 INFO:tasks.workunit.client.0.smithi107.stdout:1/797: creat d4/df/d1a/ffe x:0 0 0 2024-05-17T21:53:03.833 INFO:tasks.workunit.client.1.smithi138.stdout:9/515: write d6/d7/d16/d29/d38/d3c/f4e [1891401,14375] 0 2024-05-17T21:53:03.835 INFO:tasks.workunit.client.1.smithi138.stdout:7/561: sync 2024-05-17T21:53:03.835 INFO:tasks.workunit.client.1.smithi138.stdout:2/608: mknod d3/d21/d2f/d3e/d63/da3/cc8 0 2024-05-17T21:53:03.836 INFO:tasks.workunit.client.0.smithi107.stdout:2/605: dread d0/d9/d11/df/d5a/d6d/fb6 [0,4194304] 0 2024-05-17T21:53:03.837 INFO:tasks.workunit.client.0.smithi107.stdout:4/772: creat d3/db/d35/f101 x:0 0 0 2024-05-17T21:53:03.838 INFO:tasks.workunit.client.0.smithi107.stdout:0/866: mknod d3/dc/d22/d23/d97/d85/c121 0 2024-05-17T21:53:03.840 INFO:tasks.workunit.client.1.smithi138.stdout:2/609: dwrite d3/fe [0,4194304] 0 2024-05-17T21:53:03.840 INFO:tasks.workunit.client.0.smithi107.stdout:7/735: dread - d5/df/d11/d1c/d2f/d65/ff0 zero size 2024-05-17T21:53:03.841 INFO:tasks.workunit.client.0.smithi107.stdout:5/765: rmdir d5/db/d16 39 2024-05-17T21:53:03.844 INFO:tasks.workunit.client.1.smithi138.stdout:9/516: fsync d6/d7/fc 0 2024-05-17T21:53:03.846 INFO:tasks.workunit.client.1.smithi138.stdout:9/517: write d6/d7/d16/d29/d48/d3b/f88 [4134239,127314] 0 2024-05-17T21:53:03.850 INFO:tasks.workunit.client.0.smithi107.stdout:4/773: dread d3/db/d35/d5e/f63 [0,4194304] 0 2024-05-17T21:53:03.850 INFO:tasks.workunit.client.0.smithi107.stdout:4/774: dread - d3/db/f72 zero size 2024-05-17T21:53:03.851 INFO:tasks.workunit.client.1.smithi138.stdout:5/587: rename d1/d2c/l30 to d1/d2c/d63/db5/lc0 0 2024-05-17T21:53:03.854 INFO:tasks.workunit.client.1.smithi138.stdout:9/518: dwrite d6/d7/d16/d29/d48/f98 [0,4194304] 0 2024-05-17T21:53:03.854 INFO:tasks.workunit.client.0.smithi107.stdout:2/606: rename d0/f36 to d0/d9/d11/df/d5a/fc0 0 2024-05-17T21:53:03.854 INFO:tasks.workunit.client.0.smithi107.stdout:2/607: chown d0/d9/d1c/d37/c9d 197 1 2024-05-17T21:53:03.855 INFO:tasks.workunit.client.1.smithi138.stdout:2/610: creat d3/d21/d2f/d3e/d55/d99/d84/fc9 x:0 0 0 2024-05-17T21:53:03.860 INFO:tasks.workunit.client.0.smithi107.stdout:6/699: mkdir d9/d16/d4d/d85/d88/db1/ddd 0 2024-05-17T21:53:03.861 INFO:tasks.workunit.client.1.smithi138.stdout:9/519: dread d6/d7/d16/d29/d38/d40/d44/fa4 [0,4194304] 0 2024-05-17T21:53:03.862 INFO:tasks.workunit.client.1.smithi138.stdout:9/520: write d6/d7/d16/d29/d48/f98 [827737,76773] 0 2024-05-17T21:53:03.863 INFO:tasks.workunit.client.1.smithi138.stdout:5/588: mknod d1/d53/d88/db9/d39/d69/cc1 0 2024-05-17T21:53:03.864 INFO:tasks.workunit.client.0.smithi107.stdout:7/736: dwrite d5/df/d11/d1c/d2f/fec [0,4194304] 0 2024-05-17T21:53:03.875 INFO:tasks.workunit.client.1.smithi138.stdout:2/611: dread d3/d21/d2f/d3e/d58/f71 [0,4194304] 0 2024-05-17T21:53:03.876 INFO:tasks.workunit.client.1.smithi138.stdout:5/589: rename d1/d53/d88/d99 to d1/d53/d88/db9/d39/d69/dc2 0 2024-05-17T21:53:03.878 INFO:tasks.workunit.client.0.smithi107.stdout:1/798: creat d4/df/fff x:0 0 0 2024-05-17T21:53:03.880 INFO:tasks.workunit.client.0.smithi107.stdout:7/737: chown d5/df/d11/d1c/d2f/l57 152865716 1 2024-05-17T21:53:03.881 INFO:tasks.workunit.client.1.smithi138.stdout:2/612: rename d3/d81/f92 to d3/d21/d2f/d3e/d55/d99/d93/fca 0 2024-05-17T21:53:03.883 INFO:tasks.workunit.client.0.smithi107.stdout:0/867: rename d3/d24/c55 to d3/d24/dcf/c122 0 2024-05-17T21:53:03.885 INFO:tasks.workunit.client.0.smithi107.stdout:5/766: mknod d5/db/d16/d54/cfc 0 2024-05-17T21:53:03.886 INFO:tasks.workunit.client.1.smithi138.stdout:2/613: creat d3/d21/d2f/d3e/d55/d99/d84/d9f/fcb x:0 0 0 2024-05-17T21:53:03.887 INFO:tasks.workunit.client.1.smithi138.stdout:9/521: link d6/d7/d16/d29/d38/d40/l4f d6/d7/d16/d29/d48/d6e/d74/d75/la8 0 2024-05-17T21:53:03.890 INFO:tasks.workunit.client.0.smithi107.stdout:1/799: dwrite d4/df/d1a/d26/d72/daa/f40 [0,4194304] 0 2024-05-17T21:53:03.891 INFO:tasks.workunit.client.0.smithi107.stdout:1/800: chown d4/df/d1a/d26/d72/daa/f40 20393 1 2024-05-17T21:53:03.892 INFO:tasks.workunit.client.1.smithi138.stdout:8/547: sync 2024-05-17T21:53:03.892 INFO:tasks.workunit.client.1.smithi138.stdout:1/554: sync 2024-05-17T21:53:03.924 INFO:tasks.workunit.client.0.smithi107.stdout:7/738: mknod d5/df/d78/c102 0 2024-05-17T21:53:03.925 INFO:tasks.workunit.client.1.smithi138.stdout:2/614: write d3/f9b [424639,105908] 0 2024-05-17T21:53:03.926 INFO:tasks.workunit.client.1.smithi138.stdout:9/522: fsync d6/d7/f17 0 2024-05-17T21:53:03.928 INFO:tasks.workunit.client.0.smithi107.stdout:2/608: rename d0/d9/d1c/d3e/f62 to d0/da6/fc1 0 2024-05-17T21:53:03.932 INFO:tasks.workunit.client.1.smithi138.stdout:1/555: symlink d4/d2e/da0/lab 0 2024-05-17T21:53:03.933 INFO:tasks.workunit.client.1.smithi138.stdout:9/523: dwrite d6/d7/d16/d29/d6c/d83/f95 [0,4194304] 0 2024-05-17T21:53:03.934 INFO:tasks.workunit.client.0.smithi107.stdout:1/801: read d4/df/d14/d1d/f2a [120804,104096] 0 2024-05-17T21:53:03.934 INFO:tasks.workunit.client.1.smithi138.stdout:2/615: rmdir d3/da0/daa 39 2024-05-17T21:53:04.013 INFO:tasks.workunit.client.0.smithi107.stdout:9/784: sync 2024-05-17T21:53:04.014 INFO:tasks.workunit.client.0.smithi107.stdout:6/700: sync 2024-05-17T21:53:04.014 INFO:tasks.workunit.client.0.smithi107.stdout:3/785: sync 2024-05-17T21:53:04.014 INFO:tasks.workunit.client.0.smithi107.stdout:8/729: sync 2024-05-17T21:53:04.014 INFO:tasks.workunit.client.0.smithi107.stdout:2/609: fsync d0/d9/d11/f5f 0 2024-05-17T21:53:04.015 INFO:tasks.workunit.client.1.smithi138.stdout:1/556: creat d4/d2e/d36/fac x:0 0 0 2024-05-17T21:53:04.016 INFO:tasks.workunit.client.1.smithi138.stdout:1/557: fsync d4/df/d3a/d3c/d63/d6d/f87 0 2024-05-17T21:53:04.019 INFO:tasks.workunit.client.1.smithi138.stdout:9/524: mknod d6/d7/d16/d29/d48/d3b/ca9 0 2024-05-17T21:53:04.020 INFO:tasks.workunit.client.1.smithi138.stdout:9/525: readlink d6/d7/d16/l1d 0 2024-05-17T21:53:04.021 INFO:tasks.workunit.client.1.smithi138.stdout:9/526: chown d6/d7/d16/d29/d48/d6e/d74/d75/d76/d8e/fa7 11863452 1 2024-05-17T21:53:04.022 INFO:tasks.workunit.client.1.smithi138.stdout:2/616: truncate d3/f27 1492086 0 2024-05-17T21:53:04.028 INFO:tasks.workunit.client.1.smithi138.stdout:2/617: dwrite d3/f74 [4194304,4194304] 0 2024-05-17T21:53:04.031 INFO:tasks.workunit.client.1.smithi138.stdout:2/618: write d3/d81/d96/fb4 [689245,87448] 0 2024-05-17T21:53:04.032 INFO:tasks.workunit.client.1.smithi138.stdout:6/601: truncate d7/d69/db5/dc8/d43/f35 6736932 0 2024-05-17T21:53:04.034 INFO:tasks.workunit.client.1.smithi138.stdout:1/558: creat d4/d2e/da0/daa/fad x:0 0 0 2024-05-17T21:53:04.038 INFO:tasks.workunit.client.1.smithi138.stdout:9/527: symlink d6/d7/d16/d29/d6c/d9c/laa 0 2024-05-17T21:53:04.039 INFO:tasks.workunit.client.1.smithi138.stdout:0/578: write d3/d10/f25 [3761936,95674] 0 2024-05-17T21:53:04.045 INFO:tasks.workunit.client.1.smithi138.stdout:1/559: unlink d4/db/d29/d89/l58 0 2024-05-17T21:53:04.052 INFO:tasks.workunit.client.1.smithi138.stdout:6/602: dread d7/d8/ff [0,4194304] 0 2024-05-17T21:53:04.059 INFO:tasks.workunit.client.1.smithi138.stdout:1/560: dread d4/d2e/d36/f91 [0,4194304] 0 2024-05-17T21:53:04.064 INFO:tasks.workunit.client.1.smithi138.stdout:1/561: dread d4/df/f97 [0,4194304] 0 2024-05-17T21:53:04.064 INFO:tasks.workunit.client.1.smithi138.stdout:1/562: chown d4/dd/c56 1891 1 2024-05-17T21:53:04.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:03 smithi138 ceph-mon[35948]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:53:04.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:03 smithi138 ceph-mon[35948]: pgmap v40: 65 pgs: 65 active+clean; 2.5 GiB data, 8.7 GiB used, 528 GiB / 536 GiB avail; 93 MiB/s rd, 213 MiB/s wr, 455 op/s 2024-05-17T21:53:04.079 INFO:tasks.workunit.client.1.smithi138.stdout:4/625: sync 2024-05-17T21:53:04.079 INFO:tasks.workunit.client.1.smithi138.stdout:3/704: sync 2024-05-17T21:53:04.081 INFO:tasks.workunit.client.1.smithi138.stdout:3/705: write d1/d3/d16/d3e/d71/fa9 [5000752,111378] 0 2024-05-17T21:53:04.091 INFO:tasks.workunit.client.1.smithi138.stdout:7/562: dwrite d0/f8e [0,4194304] 0 2024-05-17T21:53:04.092 INFO:tasks.workunit.client.1.smithi138.stdout:6/603: creat d7/d69/db5/dc8/d43/d63/fce x:0 0 0 2024-05-17T21:53:04.094 INFO:tasks.workunit.client.0.smithi107.stdout:9/785: creat d0/dd/d2f/d81/dc3/f104 x:0 0 0 2024-05-17T21:53:04.101 INFO:tasks.workunit.client.0.smithi107.stdout:4/775: write d3/d6/d26/d36/f6e [966631,48831] 0 2024-05-17T21:53:04.101 INFO:tasks.workunit.client.1.smithi138.stdout:9/528: write d6/d7/d16/d29/d38/d40/d44/fa4 [2019710,49066] 0 2024-05-17T21:53:04.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:03 smithi107 ceph-mon[28434]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:53:04.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:03 smithi107 ceph-mon[28434]: pgmap v40: 65 pgs: 65 active+clean; 2.5 GiB data, 8.7 GiB used, 528 GiB / 536 GiB avail; 93 MiB/s rd, 213 MiB/s wr, 455 op/s 2024-05-17T21:53:04.117 INFO:tasks.workunit.client.1.smithi138.stdout:7/563: dread d0/d18/d40/d41/f77 [0,4194304] 0 2024-05-17T21:53:04.119 INFO:tasks.workunit.client.0.smithi107.stdout:4/776: mknod d3/db/d1e/d99/dd6/c102 0 2024-05-17T21:53:04.123 INFO:tasks.workunit.client.1.smithi138.stdout:1/563: mkdir d4/dd/d3b/d8c/dae 0 2024-05-17T21:53:04.125 INFO:tasks.workunit.client.1.smithi138.stdout:7/564: dwrite d0/d18/d40/f44 [8388608,4194304] 0 2024-05-17T21:53:04.126 INFO:tasks.workunit.client.0.smithi107.stdout:4/777: creat d3/ddf/f103 x:0 0 0 2024-05-17T21:53:04.128 INFO:tasks.workunit.client.1.smithi138.stdout:4/626: mknod d0/d2f/d49/d60/d1f/da1/ce0 0 2024-05-17T21:53:04.129 INFO:tasks.workunit.client.1.smithi138.stdout:4/627: dread - d0/d1b/fc8 zero size 2024-05-17T21:53:04.131 INFO:tasks.workunit.client.0.smithi107.stdout:5/767: write d5/db/d39/f3e [4244714,70447] 0 2024-05-17T21:53:04.132 INFO:tasks.workunit.client.0.smithi107.stdout:5/768: chown d5/db/d16/d2a/f94 14906657 1 2024-05-17T21:53:04.133 INFO:tasks.workunit.client.1.smithi138.stdout:3/706: link d1/d2e/fa1 d1/d4f/d80/ff8 0 2024-05-17T21:53:04.141 INFO:tasks.workunit.client.0.smithi107.stdout:4/778: symlink d3/d4a/d47/d60/d3a/ddc/l104 0 2024-05-17T21:53:04.145 INFO:tasks.workunit.client.1.smithi138.stdout:1/564: creat d4/d93/faf x:0 0 0 2024-05-17T21:53:04.147 INFO:tasks.workunit.client.0.smithi107.stdout:4/779: dwrite d3/db/f69 [0,4194304] 0 2024-05-17T21:53:04.150 INFO:tasks.workunit.client.1.smithi138.stdout:7/565: symlink d0/d18/d9c/lb6 0 2024-05-17T21:53:04.151 INFO:tasks.workunit.client.0.smithi107.stdout:5/769: link d5/db/d16/d2d/l37 d5/db/d16/d2d/d40/lfd 0 2024-05-17T21:53:04.157 INFO:tasks.workunit.client.0.smithi107.stdout:4/780: rmdir d3/d4a/d47/d60/d3c/d9d 39 2024-05-17T21:53:04.161 INFO:tasks.workunit.client.0.smithi107.stdout:5/770: symlink d5/db/d16/d2d/d40/d9f/dc9/lfe 0 2024-05-17T21:53:04.163 INFO:tasks.workunit.client.1.smithi138.stdout:3/707: dread d1/d3/d16/d3e/f74 [0,4194304] 0 2024-05-17T21:53:04.164 INFO:tasks.workunit.client.1.smithi138.stdout:3/708: write d1/d62/dad/fe1 [101499,119233] 0 2024-05-17T21:53:04.165 INFO:tasks.workunit.client.1.smithi138.stdout:3/709: write d1/d3/d16/f38 [4963809,21911] 0 2024-05-17T21:53:04.170 INFO:tasks.workunit.client.0.smithi107.stdout:5/771: rmdir d5/db/d39 39 2024-05-17T21:53:04.172 INFO:tasks.workunit.client.1.smithi138.stdout:7/566: symlink d0/d18/d40/d41/d50/lb7 0 2024-05-17T21:53:04.178 INFO:tasks.workunit.client.1.smithi138.stdout:5/590: sync 2024-05-17T21:53:04.179 INFO:tasks.workunit.client.1.smithi138.stdout:5/591: stat d1/d2c/d63/db5/l9f 0 2024-05-17T21:53:04.180 INFO:tasks.workunit.client.1.smithi138.stdout:1/565: rename d4/db/d29/d5d/c7d to d4/db/d29/d5d/cb0 0 2024-05-17T21:53:04.185 INFO:tasks.workunit.client.1.smithi138.stdout:7/567: creat d0/d18/d40/d9a/fb8 x:0 0 0 2024-05-17T21:53:04.185 INFO:tasks.workunit.client.1.smithi138.stdout:3/710: dread d1/d3/d16/d3e/f4e [0,4194304] 0 2024-05-17T21:53:04.187 INFO:tasks.workunit.client.1.smithi138.stdout:3/711: write d1/d4f/d80/f86 [393033,8568] 0 2024-05-17T21:53:04.189 INFO:tasks.workunit.client.1.smithi138.stdout:1/566: creat d4/db/d60/fb1 x:0 0 0 2024-05-17T21:53:04.195 INFO:tasks.workunit.client.1.smithi138.stdout:1/567: dwrite d4/db/d29/d5d/f6e [0,4194304] 0 2024-05-17T21:53:04.197 INFO:tasks.workunit.client.1.smithi138.stdout:1/568: fsync d4/db/d29/fa4 0 2024-05-17T21:53:04.198 INFO:tasks.workunit.client.1.smithi138.stdout:1/569: chown d4/d2e/d36/f4f 27711 1 2024-05-17T21:53:04.200 INFO:tasks.workunit.client.1.smithi138.stdout:1/570: symlink d4/df/lb2 0 2024-05-17T21:53:04.208 INFO:tasks.workunit.client.1.smithi138.stdout:2/619: symlink d3/d21/d2f/d3e/d58/lcc 0 2024-05-17T21:53:04.209 INFO:tasks.workunit.client.1.smithi138.stdout:2/620: stat d3/d21/f30 0 2024-05-17T21:53:04.212 INFO:tasks.workunit.client.0.smithi107.stdout:9/786: rename d0/dd/d21/d29/d57/d5f/ff5 to d0/d80/f105 0 2024-05-17T21:53:04.217 INFO:tasks.workunit.client.0.smithi107.stdout:5/772: rename d5/ca2 to d5/db/dc/d1e/cff 0 2024-05-17T21:53:04.217 INFO:tasks.workunit.client.0.smithi107.stdout:7/739: dwrite d5/df/d11/d1c/d31/d39/f5b [4194304,4194304] 0 2024-05-17T21:53:04.219 INFO:tasks.workunit.client.0.smithi107.stdout:5/773: chown d5/db/d16/d5b/l7a 25770885 1 2024-05-17T21:53:04.220 INFO:tasks.workunit.client.0.smithi107.stdout:5/774: chown d5/db/d16/f81 2 1 2024-05-17T21:53:04.222 INFO:tasks.workunit.client.0.smithi107.stdout:7/740: truncate d5/df/d78/f95 3728004 0 2024-05-17T21:53:04.222 INFO:tasks.workunit.client.0.smithi107.stdout:5/775: write d5/db/d16/f9d [78428,16418] 0 2024-05-17T21:53:04.223 INFO:tasks.workunit.client.0.smithi107.stdout:7/741: write d5/df/d11/d1c/d31/d56/f9c [865214,58218] 0 2024-05-17T21:53:04.226 INFO:tasks.workunit.client.0.smithi107.stdout:5/776: symlink d5/db/d16/d2d/d40/dc2/l100 0 2024-05-17T21:53:04.228 INFO:tasks.workunit.client.0.smithi107.stdout:7/742: creat d5/d16/d3b/d45/d91/db3/f103 x:0 0 0 2024-05-17T21:53:04.229 INFO:tasks.workunit.client.0.smithi107.stdout:3/786: creat d6/d8/d10/d35/f100 x:0 0 0 2024-05-17T21:53:04.230 INFO:tasks.workunit.client.0.smithi107.stdout:7/743: write d5/df/d11/d1d/f8c [3652529,31248] 0 2024-05-17T21:53:04.231 INFO:tasks.workunit.client.0.smithi107.stdout:3/787: write d6/d8/d3e/d3f/fba [1011409,104013] 0 2024-05-17T21:53:04.232 INFO:tasks.workunit.client.0.smithi107.stdout:3/788: stat d6/d2a/d2d/d48/d4f/f67 0 2024-05-17T21:53:04.234 INFO:tasks.workunit.client.1.smithi138.stdout:2/621: dread d3/d81/fa7 [0,4194304] 0 2024-05-17T21:53:04.234 INFO:tasks.workunit.client.0.smithi107.stdout:3/789: chown d6/d8/d10/d25/f95 285 1 2024-05-17T21:53:04.237 INFO:tasks.workunit.client.0.smithi107.stdout:7/744: dwrite d5/df/d11/f4c [0,4194304] 0 2024-05-17T21:53:04.239 INFO:tasks.workunit.client.0.smithi107.stdout:3/790: symlink d6/d2a/d2d/d37/dc6/l101 0 2024-05-17T21:53:04.243 INFO:tasks.workunit.client.0.smithi107.stdout:3/791: readlink d6/d2a/d2d/d48/d4f/l7f 0 2024-05-17T21:53:04.244 INFO:tasks.workunit.client.0.smithi107.stdout:9/787: dread d0/dd/d3d/fb4 [0,4194304] 0 2024-05-17T21:53:04.244 INFO:tasks.workunit.client.0.smithi107.stdout:3/792: write d6/d8/d10/d35/d36/f8e [24471,65796] 0 2024-05-17T21:53:04.246 INFO:tasks.workunit.client.0.smithi107.stdout:9/788: write d0/f7 [4463142,68591] 0 2024-05-17T21:53:04.251 INFO:tasks.workunit.client.0.smithi107.stdout:0/868: write d3/d39/fb0 [148218,67796] 0 2024-05-17T21:53:04.253 INFO:tasks.workunit.client.0.smithi107.stdout:9/789: read d0/dd/d2f/d87/d89/f97 [1975326,101891] 0 2024-05-17T21:53:04.258 INFO:tasks.workunit.client.0.smithi107.stdout:1/802: write d4/df/d4e/f63 [1064968,45276] 0 2024-05-17T21:53:04.261 INFO:tasks.workunit.client.0.smithi107.stdout:1/803: fsync d4/df/d1a/d41/f48 0 2024-05-17T21:53:04.269 INFO:tasks.workunit.client.0.smithi107.stdout:0/869: dread d3/d24/d6f/d9f/fbd [0,4194304] 0 2024-05-17T21:53:04.269 INFO:tasks.workunit.client.1.smithi138.stdout:2/622: link d3/d21/d2f/d3e/d55/d99/f8f d3/d21/d2f/d3e/d63/fcd 0 2024-05-17T21:53:04.275 INFO:tasks.workunit.client.1.smithi138.stdout:2/623: mkdir d3/da0/daa/dce 0 2024-05-17T21:53:04.276 INFO:tasks.workunit.client.0.smithi107.stdout:0/870: creat d3/dc/d22/d3d/f123 x:0 0 0 2024-05-17T21:53:04.277 INFO:tasks.workunit.client.0.smithi107.stdout:0/871: chown d3/dc/ff4 0 1 2024-05-17T21:53:04.278 INFO:tasks.workunit.client.1.smithi138.stdout:2/624: read d3/fa [554220,9162] 0 2024-05-17T21:53:04.280 INFO:tasks.workunit.client.0.smithi107.stdout:0/872: unlink d3/d39/l90 0 2024-05-17T21:53:04.281 INFO:tasks.workunit.client.0.smithi107.stdout:6/701: dwrite d9/d16/d22/f27 [0,4194304] 0 2024-05-17T21:53:04.283 INFO:tasks.workunit.client.1.smithi138.stdout:2/625: rename d3/d21/d2f/d3e/d58/l7d to d3/d81/d96/da8/lcf 0 2024-05-17T21:53:04.283 INFO:tasks.workunit.client.0.smithi107.stdout:6/702: chown d9/d58/d7b/da3/fc8 16113 1 2024-05-17T21:53:04.287 INFO:tasks.workunit.client.1.smithi138.stdout:2/626: rename d3/d21/d2f/d3e/d69 to d3/da0/daa/dce/dd0 0 2024-05-17T21:53:04.289 INFO:tasks.workunit.client.0.smithi107.stdout:0/873: mknod d3/d58/c124 0 2024-05-17T21:53:04.290 INFO:tasks.workunit.client.0.smithi107.stdout:2/610: write d0/d9/d11/df/d5a/d6d/d82/fb9 [4728,48350] 0 2024-05-17T21:53:04.290 INFO:tasks.workunit.client.0.smithi107.stdout:2/611: chown d0/f40 182936723 1 2024-05-17T21:53:04.292 INFO:tasks.workunit.client.0.smithi107.stdout:8/730: truncate dc/d21/d27/d2f/f30 305680 0 2024-05-17T21:53:04.293 INFO:tasks.workunit.client.0.smithi107.stdout:0/874: symlink d3/d24/d6f/d9f/de8/l125 0 2024-05-17T21:53:04.293 INFO:tasks.workunit.client.0.smithi107.stdout:0/875: stat d3/d58/f84 0 2024-05-17T21:53:04.293 INFO:tasks.workunit.client.0.smithi107.stdout:0/876: stat d3/f6e 0 2024-05-17T21:53:04.294 INFO:tasks.workunit.client.1.smithi138.stdout:2/627: dwrite d3/d81/d96/fb4 [0,4194304] 0 2024-05-17T21:53:04.295 INFO:tasks.workunit.client.0.smithi107.stdout:2/612: symlink d0/d9/d11/d18/dad/db8/lc2 0 2024-05-17T21:53:04.300 INFO:tasks.workunit.client.0.smithi107.stdout:8/731: dwrite dc/d21/d35/fda [0,4194304] 0 2024-05-17T21:53:04.300 INFO:tasks.workunit.client.1.smithi138.stdout:2/628: dread d3/d21/d2f/d3e/d63/fcd [0,4194304] 0 2024-05-17T21:53:04.300 INFO:tasks.workunit.client.0.smithi107.stdout:2/613: truncate d0/d9/d11/df/d5a/d6d/f74 4789146 0 2024-05-17T21:53:04.304 INFO:tasks.workunit.client.1.smithi138.stdout:2/629: creat d3/d21/d2f/d3e/d58/d77/fd1 x:0 0 0 2024-05-17T21:53:04.307 INFO:tasks.workunit.client.0.smithi107.stdout:8/732: mkdir dc/d21/d27/d2f/dba/d104 0 2024-05-17T21:53:04.309 INFO:tasks.workunit.client.1.smithi138.stdout:6/604: write d7/d69/db5/dc8/d43/d3f/d79/f6c [945768,90490] 0 2024-05-17T21:53:04.310 INFO:tasks.workunit.client.1.smithi138.stdout:6/605: dread - d7/d69/db5/dc8/d34/da6/fcd zero size 2024-05-17T21:53:04.312 INFO:tasks.workunit.client.1.smithi138.stdout:6/606: dread - d7/d69/fb8 zero size 2024-05-17T21:53:04.322 INFO:tasks.workunit.client.0.smithi107.stdout:2/614: dread d0/da6/fc1 [4194304,4194304] 0 2024-05-17T21:53:04.322 INFO:tasks.workunit.client.0.smithi107.stdout:0/877: dread d3/dc/dae/dc0/de1/f11c [4194304,4194304] 0 2024-05-17T21:53:04.326 INFO:tasks.workunit.client.1.smithi138.stdout:8/548: creat d0/dc/d1a/fa1 x:0 0 0 2024-05-17T21:53:04.327 INFO:tasks.workunit.client.1.smithi138.stdout:0/579: sync 2024-05-17T21:53:04.328 INFO:tasks.workunit.client.1.smithi138.stdout:8/549: chown d0/dc/l25 1490347 1 2024-05-17T21:53:04.331 INFO:tasks.workunit.client.1.smithi138.stdout:0/580: fsync d3/d3d/f7f 0 2024-05-17T21:53:04.333 INFO:tasks.workunit.client.1.smithi138.stdout:0/581: dread d3/d10/d39/f72 [0,4194304] 0 2024-05-17T21:53:04.342 INFO:tasks.workunit.client.1.smithi138.stdout:4/628: truncate d0/f7 4028364 0 2024-05-17T21:53:04.348 INFO:tasks.workunit.client.1.smithi138.stdout:4/629: dwrite d0/d2f/d49/d60/d1f/d32/f43 [4194304,4194304] 0 2024-05-17T21:53:04.353 INFO:tasks.workunit.client.1.smithi138.stdout:4/630: rmdir d0 39 2024-05-17T21:53:04.353 INFO:tasks.workunit.client.1.smithi138.stdout:9/529: creat d6/d7/d16/d29/d48/d6e/fab x:0 0 0 2024-05-17T21:53:04.354 INFO:tasks.workunit.client.1.smithi138.stdout:5/592: dwrite d1/d2c/d63/db5/d73/fb0 [0,4194304] 0 2024-05-17T21:53:04.365 INFO:tasks.workunit.client.1.smithi138.stdout:4/631: read d0/d3/d17/fa9 [3516805,22021] 0 2024-05-17T21:53:04.365 INFO:tasks.workunit.client.0.smithi107.stdout:4/781: creat d3/d4a/d47/d60/d3a/f105 x:0 0 0 2024-05-17T21:53:04.366 INFO:tasks.workunit.client.1.smithi138.stdout:4/632: chown d0/d2f/d49/f90 368 1 2024-05-17T21:53:04.368 INFO:tasks.workunit.client.0.smithi107.stdout:4/782: mknod d3/db/d35/d4f/d93/c106 0 2024-05-17T21:53:04.368 INFO:tasks.workunit.client.0.smithi107.stdout:4/783: fdatasync d3/d4a/d47/d60/f10 0 2024-05-17T21:53:04.369 INFO:tasks.workunit.client.1.smithi138.stdout:1/571: symlink d4/dd/d3b/lb3 0 2024-05-17T21:53:04.370 INFO:tasks.workunit.client.0.smithi107.stdout:4/784: truncate d3/db/d35/d5e/fcf 923740 0 2024-05-17T21:53:04.371 INFO:tasks.workunit.client.0.smithi107.stdout:9/790: symlink d0/dd/d21/l106 0 2024-05-17T21:53:04.371 INFO:tasks.workunit.client.0.smithi107.stdout:9/791: chown d0/dd/d69/fc1 15753482 1 2024-05-17T21:53:04.372 INFO:tasks.workunit.client.1.smithi138.stdout:4/633: dwrite d0/d2f/d49/d60/d1f/d32/f43 [0,4194304] 0 2024-05-17T21:53:04.380 INFO:tasks.workunit.client.1.smithi138.stdout:4/634: fsync d0/d2f/d49/d60/f89 0 2024-05-17T21:53:04.382 INFO:tasks.workunit.client.1.smithi138.stdout:1/572: mkdir d4/dd/db4 0 2024-05-17T21:53:04.386 INFO:tasks.workunit.client.1.smithi138.stdout:4/635: fsync d0/d3/db/da0/fa8 0 2024-05-17T21:53:04.387 INFO:tasks.workunit.client.0.smithi107.stdout:9/792: getdents d0/dd/d2f/d4a 0 2024-05-17T21:53:04.392 INFO:tasks.workunit.client.0.smithi107.stdout:9/793: unlink d0/dd/d21/c3c 0 2024-05-17T21:53:04.392 INFO:tasks.workunit.client.0.smithi107.stdout:9/794: chown d0/dd/d2f/d4a/de6/lfb 240924101 1 2024-05-17T21:53:04.393 INFO:tasks.workunit.client.1.smithi138.stdout:4/636: creat d0/d2f/d49/d60/d1f/d32/fe1 x:0 0 0 2024-05-17T21:53:04.394 INFO:tasks.workunit.client.0.smithi107.stdout:9/795: symlink d0/dd/d1b/l107 0 2024-05-17T21:53:04.396 INFO:tasks.workunit.client.1.smithi138.stdout:4/637: creat d0/fe2 x:0 0 0 2024-05-17T21:53:04.398 INFO:tasks.workunit.client.0.smithi107.stdout:9/796: read d0/dd/f78 [2524260,54875] 0 2024-05-17T21:53:04.399 INFO:tasks.workunit.client.0.smithi107.stdout:7/745: unlink d5/d16/d3b/d45/f6c 0 2024-05-17T21:53:04.399 INFO:tasks.workunit.client.1.smithi138.stdout:1/573: dread d4/df/d35/f37 [0,4194304] 0 2024-05-17T21:53:04.404 INFO:tasks.workunit.client.0.smithi107.stdout:5/777: sync 2024-05-17T21:53:04.404 INFO:tasks.workunit.client.0.smithi107.stdout:9/797: dread d0/fa5 [0,4194304] 0 2024-05-17T21:53:04.410 INFO:tasks.workunit.client.0.smithi107.stdout:5/778: rename d5/db/d16/d54/d59/d65/f47 to d5/db/d16/d2d/d51/f101 0 2024-05-17T21:53:04.412 INFO:tasks.workunit.client.0.smithi107.stdout:3/793: dwrite d6/d8/d3e/d3f/d90/f9f [0,4194304] 0 2024-05-17T21:53:04.416 INFO:tasks.workunit.client.1.smithi138.stdout:1/574: dread d4/db/f39 [0,4194304] 0 2024-05-17T21:53:04.417 INFO:tasks.workunit.client.1.smithi138.stdout:1/575: fsync d4/dd/d3b/d6c/f92 0 2024-05-17T21:53:04.418 INFO:tasks.workunit.client.0.smithi107.stdout:3/794: write d6/d8/d10/d35/d91/da6/ff9 [330926,27084] 0 2024-05-17T21:53:04.419 INFO:tasks.workunit.client.0.smithi107.stdout:3/795: dread - d6/fef zero size 2024-05-17T21:53:04.419 INFO:tasks.workunit.client.0.smithi107.stdout:7/746: dread d5/df/d11/d1c/d31/d39/d92/d9a/fdc [0,4194304] 0 2024-05-17T21:53:04.420 INFO:tasks.workunit.client.0.smithi107.stdout:9/798: link d0/dd/d21/d29/d57/l73 d0/dd/d21/d29/dca/df3/l108 0 2024-05-17T21:53:04.426 INFO:tasks.workunit.client.1.smithi138.stdout:1/576: dread d4/dd/d3b/d6c/f78 [0,4194304] 0 2024-05-17T21:53:04.429 INFO:tasks.workunit.client.1.smithi138.stdout:7/568: rmdir d0 39 2024-05-17T21:53:04.431 INFO:tasks.workunit.client.1.smithi138.stdout:1/577: mknod d4/dd/d3b/cb5 0 2024-05-17T21:53:04.431 INFO:tasks.workunit.client.1.smithi138.stdout:1/578: read - d4/dd/f90 zero size 2024-05-17T21:53:04.433 INFO:tasks.workunit.client.0.smithi107.stdout:7/747: rmdir d5/df/d11/d1c/d2f/d53 39 2024-05-17T21:53:04.434 INFO:tasks.workunit.client.0.smithi107.stdout:7/748: write d5/d16/f3a [3502569,82170] 0 2024-05-17T21:53:04.438 INFO:tasks.workunit.client.0.smithi107.stdout:5/779: getdents d5/db/dc/d1e/de8 0 2024-05-17T21:53:04.441 INFO:tasks.workunit.client.1.smithi138.stdout:1/579: symlink d4/db/lb6 0 2024-05-17T21:53:04.441 INFO:tasks.workunit.client.1.smithi138.stdout:1/580: stat d4/df/l1a 0 2024-05-17T21:53:04.441 INFO:tasks.workunit.client.1.smithi138.stdout:1/581: fdatasync f0 0 2024-05-17T21:53:04.444 INFO:tasks.workunit.client.0.smithi107.stdout:3/796: rmdir d6/d2a/d2d/d69/dad/dcd 0 2024-05-17T21:53:04.445 INFO:tasks.workunit.client.0.smithi107.stdout:3/797: chown d6/d8/d10/d35/d36/d4c/laf 1809 1 2024-05-17T21:53:04.447 INFO:tasks.workunit.client.0.smithi107.stdout:3/798: write d6/d8/d10/d25/f55 [867067,71029] 0 2024-05-17T21:53:04.449 INFO:tasks.workunit.client.0.smithi107.stdout:3/799: read d6/d2a/d2d/da8/daa/df5/fbf [37348,55437] 0 2024-05-17T21:53:04.450 INFO:tasks.workunit.client.0.smithi107.stdout:5/780: mknod d5/db/d16/d2d/d40/d5a/db4/c102 0 2024-05-17T21:53:04.455 INFO:tasks.workunit.client.1.smithi138.stdout:9/530: symlink d6/d7/d16/d29/d48/lac 0 2024-05-17T21:53:04.455 INFO:tasks.workunit.client.0.smithi107.stdout:3/800: chown f3 16 1 2024-05-17T21:53:04.456 INFO:tasks.workunit.client.1.smithi138.stdout:9/531: dread - d6/d7/d16/d29/d38/f97 zero size 2024-05-17T21:53:04.456 INFO:tasks.workunit.client.1.smithi138.stdout:1/582: dread d4/db/d29/d5d/f86 [0,4194304] 0 2024-05-17T21:53:04.457 INFO:tasks.workunit.client.0.smithi107.stdout:1/804: unlink d4/df/c2d 0 2024-05-17T21:53:04.459 INFO:tasks.workunit.client.1.smithi138.stdout:7/569: getdents d0/d18/d40/d41/d51/d66/d9e 0 2024-05-17T21:53:04.459 INFO:tasks.workunit.client.1.smithi138.stdout:3/712: sync 2024-05-17T21:53:04.461 INFO:tasks.workunit.client.0.smithi107.stdout:7/749: creat d5/df/d11/d1c/d2f/f104 x:0 0 0 2024-05-17T21:53:04.462 INFO:tasks.workunit.client.0.smithi107.stdout:7/750: chown d5/df/d11/d1d/d33/d82/fd5 3 1 2024-05-17T21:53:04.462 INFO:tasks.workunit.client.1.smithi138.stdout:9/532: dwrite d6/d7/d16/d29/d48/d6f/f71 [4194304,4194304] 0 2024-05-17T21:53:04.464 INFO:tasks.workunit.client.1.smithi138.stdout:2/630: rename d3/d81/d96/db2 to d3/d21/d2f/d3e/d55/d99/dd2 0 2024-05-17T21:53:04.466 INFO:tasks.workunit.client.0.smithi107.stdout:6/703: sync 2024-05-17T21:53:04.467 INFO:tasks.workunit.client.1.smithi138.stdout:2/631: truncate d3/d21/d2f/d3e/d55/d99/d84/d9f/fcb 377808 0 2024-05-17T21:53:04.468 INFO:tasks.workunit.client.0.smithi107.stdout:6/704: fsync d9/d16/d4d/db8/dc9/dd0/f39 0 2024-05-17T21:53:04.469 INFO:tasks.workunit.client.0.smithi107.stdout:1/805: mknod d4/df/d1a/d41/db2/c100 0 2024-05-17T21:53:04.470 INFO:tasks.workunit.client.1.smithi138.stdout:1/583: symlink d4/d2e/lb7 0 2024-05-17T21:53:04.470 INFO:tasks.workunit.client.1.smithi138.stdout:1/584: stat d4/db/f21 0 2024-05-17T21:53:04.474 INFO:tasks.workunit.client.0.smithi107.stdout:8/733: dwrite dc/d21/daa/db4/fe0 [0,4194304] 0 2024-05-17T21:53:04.474 INFO:tasks.workunit.client.1.smithi138.stdout:7/570: rename d0/d39/d78/l7e to d0/d18/d40/d41/d50/lb9 0 2024-05-17T21:53:04.477 INFO:tasks.workunit.client.1.smithi138.stdout:1/585: dwrite d4/dd/d3b/f9e [0,4194304] 0 2024-05-17T21:53:04.479 INFO:tasks.workunit.client.1.smithi138.stdout:0/582: sync 2024-05-17T21:53:04.479 INFO:tasks.workunit.client.1.smithi138.stdout:8/550: unlink d0/l16 0 2024-05-17T21:53:04.481 INFO:tasks.workunit.client.0.smithi107.stdout:3/801: link d6/d8/d10/d35/d36/f8e d6/d8/d10/d35/d36/d4c/dcc/f102 0 2024-05-17T21:53:04.482 INFO:tasks.workunit.client.0.smithi107.stdout:2/615: write d0/d9/d11/f8b [573240,97978] 0 2024-05-17T21:53:04.483 INFO:tasks.workunit.client.1.smithi138.stdout:2/632: write d3/d21/d2f/f5f [3386226,125550] 0 2024-05-17T21:53:04.483 INFO:tasks.workunit.client.0.smithi107.stdout:3/802: chown d6/d2a/l2b 17016 1 2024-05-17T21:53:04.487 INFO:tasks.workunit.client.0.smithi107.stdout:6/705: symlink d9/d58/d7b/da3/lde 0 2024-05-17T21:53:04.489 INFO:tasks.workunit.client.1.smithi138.stdout:9/533: rename d6/f25 to d6/d7/d16/d29/d38/d3c/fad 0 2024-05-17T21:53:04.490 INFO:tasks.workunit.client.0.smithi107.stdout:0/878: dwrite d3/dc/d22/ffc [0,4194304] 0 2024-05-17T21:53:04.498 INFO:tasks.workunit.client.0.smithi107.stdout:8/734: rmdir dc/d21/d27/d83 39 2024-05-17T21:53:04.499 INFO:tasks.workunit.client.0.smithi107.stdout:8/735: dread - dc/d21/d97/db7/fcc zero size 2024-05-17T21:53:04.499 INFO:tasks.workunit.client.1.smithi138.stdout:7/571: dwrite d0/d18/d40/f82 [0,4194304] 0 2024-05-17T21:53:04.500 INFO:tasks.workunit.client.0.smithi107.stdout:8/736: write dc/d21/d27/d34/df5/ffd [623152,125798] 0 2024-05-17T21:53:04.501 INFO:tasks.workunit.client.1.smithi138.stdout:7/572: readlink d0/d18/d40/d41/lb2 0 2024-05-17T21:53:04.501 INFO:tasks.workunit.client.0.smithi107.stdout:8/737: stat dc/d21/d35/f36 0 2024-05-17T21:53:04.502 INFO:tasks.workunit.client.1.smithi138.stdout:8/551: dread d0/dc/d1a/d51/f73 [0,4194304] 0 2024-05-17T21:53:04.505 INFO:tasks.workunit.client.1.smithi138.stdout:0/583: creat d3/d10/d6d/db6/fc1 x:0 0 0 2024-05-17T21:53:04.505 INFO:tasks.workunit.client.1.smithi138.stdout:8/552: fsync d0/dc/d3b/d4d/d7d/f9e 0 2024-05-17T21:53:04.506 INFO:tasks.workunit.client.0.smithi107.stdout:3/803: rename d6/c5a to d6/d8/d3e/c103 0 2024-05-17T21:53:04.509 INFO:tasks.workunit.client.1.smithi138.stdout:5/593: dwrite d1/d2c/d63/db5/d1b/d29/f67 [4194304,4194304] 0 2024-05-17T21:53:04.510 INFO:tasks.workunit.client.1.smithi138.stdout:2/633: creat d3/d21/d2f/d3e/d63/da3/fd3 x:0 0 0 2024-05-17T21:53:04.515 INFO:tasks.workunit.client.1.smithi138.stdout:1/586: symlink d4/d2e/lb8 0 2024-05-17T21:53:04.517 INFO:tasks.workunit.client.1.smithi138.stdout:8/553: dwrite d0/d63/f9c [0,4194304] 0 2024-05-17T21:53:04.520 INFO:tasks.workunit.client.1.smithi138.stdout:9/534: dread d6/d7/d16/f1a [0,4194304] 0 2024-05-17T21:53:04.522 INFO:tasks.workunit.client.1.smithi138.stdout:9/535: stat d6/d7/d16/d29/d48/d6e/d74 0 2024-05-17T21:53:04.523 INFO:tasks.workunit.client.0.smithi107.stdout:4/785: dwrite d3/d6/d26/d3b/d91/fb9 [0,4194304] 0 2024-05-17T21:53:04.523 INFO:tasks.workunit.client.1.smithi138.stdout:0/584: dread - d3/d10/d6d/f74 zero size 2024-05-17T21:53:04.525 INFO:tasks.workunit.client.0.smithi107.stdout:8/738: truncate dc/d21/f92 3314375 0 2024-05-17T21:53:04.527 INFO:tasks.workunit.client.0.smithi107.stdout:8/739: dread - dc/d21/d27/d2f/f2c zero size 2024-05-17T21:53:04.529 INFO:tasks.workunit.client.0.smithi107.stdout:8/740: write dc/d21/d27/d34/d4c/d7e/f101 [371469,34089] 0 2024-05-17T21:53:04.533 INFO:tasks.workunit.client.1.smithi138.stdout:5/594: rename d1/d53/d88/db9/d39/d69/fa2 to d1/d2c/dbf/fc3 0 2024-05-17T21:53:04.534 INFO:tasks.workunit.client.1.smithi138.stdout:5/595: chown d1/d2c/d63/db5/l75 58842 1 2024-05-17T21:53:04.535 INFO:tasks.workunit.client.1.smithi138.stdout:5/596: fsync d1/d46/f78 0 2024-05-17T21:53:04.535 INFO:tasks.workunit.client.0.smithi107.stdout:6/706: mknod d9/d58/d66/dd5/cdf 0 2024-05-17T21:53:04.536 INFO:tasks.workunit.client.1.smithi138.stdout:5/597: chown d1/d2c/d63/db5/d1b/d29/l40 7 1 2024-05-17T21:53:04.538 INFO:tasks.workunit.client.1.smithi138.stdout:7/573: mknod d0/d18/d40/d41/cba 0 2024-05-17T21:53:04.540 INFO:tasks.workunit.client.0.smithi107.stdout:0/879: chown d3/d24/d6f/d9f/dab/c117 245 1 2024-05-17T21:53:04.541 INFO:tasks.workunit.client.0.smithi107.stdout:0/880: chown d3/dc/f37 1611539268 1 2024-05-17T21:53:04.541 INFO:tasks.workunit.client.1.smithi138.stdout:8/554: dread d0/dc/d3b/d4d/d7d/d97/f9a [0,4194304] 0 2024-05-17T21:53:04.541 INFO:tasks.workunit.client.1.smithi138.stdout:0/585: creat d3/d10/d38/d70/d80/fc2 x:0 0 0 2024-05-17T21:53:04.544 INFO:tasks.workunit.client.0.smithi107.stdout:4/786: creat d3/d4a/d47/d60/d3c/d58/f107 x:0 0 0 2024-05-17T21:53:04.545 INFO:tasks.workunit.client.1.smithi138.stdout:2/634: mkdir d3/d21/d2f/dd4 0 2024-05-17T21:53:04.548 INFO:tasks.workunit.client.0.smithi107.stdout:2/616: link d0/d9/d11/f5f d0/d9/d11/d18/dad/fc3 0 2024-05-17T21:53:04.548 INFO:tasks.workunit.client.1.smithi138.stdout:6/607: chown d7/d69/db5/dc8/d43/f35 111 1 2024-05-17T21:53:04.550 INFO:tasks.workunit.client.1.smithi138.stdout:1/587: rename d4/dd/d3b/d8c to d4/dd/d3b/d9f/db9 0 2024-05-17T21:53:04.552 INFO:tasks.workunit.client.0.smithi107.stdout:9/799: write d0/dd/f46 [1197585,86209] 0 2024-05-17T21:53:04.556 INFO:tasks.workunit.client.0.smithi107.stdout:6/707: write d9/d16/d22/daf/fb9 [1182089,75781] 0 2024-05-17T21:53:04.558 INFO:tasks.workunit.client.0.smithi107.stdout:6/708: chown d9/d16/d4d/db8/dc9/dd0/f37 378162366 1 2024-05-17T21:53:04.558 INFO:tasks.workunit.client.0.smithi107.stdout:0/881: rmdir d3/dc/dae/dc0 39 2024-05-17T21:53:04.560 INFO:tasks.workunit.client.1.smithi138.stdout:7/574: creat d0/d18/d40/d9a/fbb x:0 0 0 2024-05-17T21:53:04.560 INFO:tasks.workunit.client.1.smithi138.stdout:7/575: write d0/d39/f85 [3806665,6597] 0 2024-05-17T21:53:04.563 INFO:tasks.workunit.client.0.smithi107.stdout:2/617: unlink d0/d9/d1c/d22/l6e 0 2024-05-17T21:53:04.566 INFO:tasks.workunit.client.0.smithi107.stdout:8/741: mknod dc/d21/d27/c105 0 2024-05-17T21:53:04.566 INFO:tasks.workunit.client.1.smithi138.stdout:7/576: dwrite d0/f4b [0,4194304] 0 2024-05-17T21:53:04.567 INFO:tasks.workunit.client.1.smithi138.stdout:7/577: truncate d0/f19 1805423 0 2024-05-17T21:53:04.568 INFO:tasks.workunit.client.0.smithi107.stdout:9/800: symlink d0/dd/d1b/l109 0 2024-05-17T21:53:04.569 INFO:tasks.workunit.client.0.smithi107.stdout:9/801: stat d0/dd/d3d/fb4 0 2024-05-17T21:53:04.571 INFO:tasks.workunit.client.1.smithi138.stdout:3/713: write d1/d4f/d80/ff8 [321927,111842] 0 2024-05-17T21:53:04.571 INFO:tasks.workunit.client.0.smithi107.stdout:3/804: link d6/d8/d10/d35/d36/d4c/c9d d6/d8/d10/d25/c104 0 2024-05-17T21:53:04.576 INFO:tasks.workunit.client.1.smithi138.stdout:4/638: mkdir d0/d3/d1d/d38/de3 0 2024-05-17T21:53:04.578 INFO:tasks.workunit.client.0.smithi107.stdout:5/781: dwrite d5/db/d39/f3c [0,4194304] 0 2024-05-17T21:53:04.581 INFO:tasks.workunit.client.1.smithi138.stdout:2/635: dread d3/d21/d2f/f45 [0,4194304] 0 2024-05-17T21:53:04.583 INFO:tasks.workunit.client.0.smithi107.stdout:6/709: truncate d9/d58/d7b/fae 1174871 0 2024-05-17T21:53:04.585 INFO:tasks.workunit.client.0.smithi107.stdout:6/710: write d9/d16/d4d/db8/dc9/dd0/fb5 [4690484,100836] 0 2024-05-17T21:53:04.586 INFO:tasks.workunit.client.0.smithi107.stdout:7/751: dwrite f4 [0,4194304] 0 2024-05-17T21:53:04.587 INFO:tasks.workunit.client.0.smithi107.stdout:7/752: chown d5/d16/d3b/d4f/c5f 1 1 2024-05-17T21:53:04.588 INFO:tasks.workunit.client.0.smithi107.stdout:4/787: link d3/d4a/f75 d3/d6/d26/d36/f108 0 2024-05-17T21:53:04.588 INFO:tasks.workunit.client.0.smithi107.stdout:7/753: chown d5/df/d11/f4c 1556 1 2024-05-17T21:53:04.593 INFO:tasks.workunit.client.1.smithi138.stdout:7/578: dread d0/f11 [0,4194304] 0 2024-05-17T21:53:04.594 INFO:tasks.workunit.client.1.smithi138.stdout:7/579: readlink d0/d18/d40/d41/lb2 0 2024-05-17T21:53:04.597 INFO:tasks.workunit.client.1.smithi138.stdout:0/586: dread d3/f5e [0,4194304] 0 2024-05-17T21:53:04.602 INFO:tasks.workunit.client.0.smithi107.stdout:9/802: creat d0/d80/dc9/f10a x:0 0 0 2024-05-17T21:53:04.604 INFO:tasks.workunit.client.0.smithi107.stdout:9/803: write d0/dd/d21/d29/d3e/dfe/f103 [434456,26534] 0 2024-05-17T21:53:04.605 INFO:tasks.workunit.client.1.smithi138.stdout:3/714: chown d1/d4d/cd8 120340901 1 2024-05-17T21:53:04.608 INFO:tasks.workunit.client.1.smithi138.stdout:6/608: truncate d7/d69/db5/dc8/d43/f64 1871844 0 2024-05-17T21:53:04.609 INFO:tasks.workunit.client.0.smithi107.stdout:5/782: mkdir d5/db/d16/d2d/d40/d5a/d7c/d103 0 2024-05-17T21:53:04.614 INFO:tasks.workunit.client.0.smithi107.stdout:7/754: creat d5/df/d11/d1c/d31/d39/d92/d9a/f105 x:0 0 0 2024-05-17T21:53:04.615 INFO:tasks.workunit.client.1.smithi138.stdout:2/636: mknod d3/d21/d2f/d3e/d55/d99/dd2/cd5 0 2024-05-17T21:53:04.616 INFO:tasks.workunit.client.0.smithi107.stdout:1/806: dwrite d4/df/d1a/d26/d72/daa/ffc [0,4194304] 0 2024-05-17T21:53:04.617 INFO:tasks.workunit.client.0.smithi107.stdout:2/618: unlink d0/d9/d11/l1d 0 2024-05-17T21:53:04.618 INFO:tasks.workunit.client.1.smithi138.stdout:9/536: getdents d6/d7/d16/d29/d6c/d83 0 2024-05-17T21:53:04.620 INFO:tasks.workunit.client.1.smithi138.stdout:9/537: chown d6/d7/d16/d29/d48/d6e/d74/d75 97718803 1 2024-05-17T21:53:04.620 INFO:tasks.workunit.client.0.smithi107.stdout:8/742: mkdir dc/d21/d97/db7/dcd/d106 0 2024-05-17T21:53:04.621 INFO:tasks.workunit.client.0.smithi107.stdout:6/711: dread d9/d16/f3e [0,4194304] 0 2024-05-17T21:53:04.622 INFO:tasks.workunit.client.1.smithi138.stdout:7/580: symlink d0/d4c/lbc 0 2024-05-17T21:53:04.622 INFO:tasks.workunit.client.1.smithi138.stdout:7/581: chown d0/d18/d40/d41/d51/d66 8019211 1 2024-05-17T21:53:04.624 INFO:tasks.workunit.client.0.smithi107.stdout:5/783: dwrite d5/db/d16/d54/d59/d65/d48/d89/fa8 [0,4194304] 0 2024-05-17T21:53:04.624 INFO:tasks.workunit.client.0.smithi107.stdout:3/805: link d6/d2a/d2d/d37/fc8 d6/d2a/d2d/d37/f105 0 2024-05-17T21:53:04.627 INFO:tasks.workunit.client.0.smithi107.stdout:7/755: dread f4 [0,4194304] 0 2024-05-17T21:53:04.630 INFO:tasks.workunit.client.0.smithi107.stdout:3/806: dread - d6/d2a/d2d/da8/fb8 zero size 2024-05-17T21:53:04.632 INFO:tasks.workunit.client.0.smithi107.stdout:3/807: chown d6/d2a/d2d/l53 423365 1 2024-05-17T21:53:04.634 INFO:tasks.workunit.client.0.smithi107.stdout:8/743: dwrite dc/d21/db8/fc4 [0,4194304] 0 2024-05-17T21:53:04.639 INFO:tasks.workunit.client.1.smithi138.stdout:3/715: truncate d1/d4f/d80/fac 1088369 0 2024-05-17T21:53:04.640 INFO:tasks.workunit.client.1.smithi138.stdout:3/716: write d1/d3/d49/ddb/f6e [3068413,116647] 0 2024-05-17T21:53:04.644 INFO:tasks.workunit.client.0.smithi107.stdout:2/619: mkdir d0/d26/d8e/dc4 0 2024-05-17T21:53:04.650 INFO:tasks.workunit.client.0.smithi107.stdout:2/620: dwrite d0/f7d [0,4194304] 0 2024-05-17T21:53:04.650 INFO:tasks.workunit.client.0.smithi107.stdout:0/882: dwrite d3/dc/d22/fb9 [0,4194304] 0 2024-05-17T21:53:04.652 INFO:tasks.workunit.client.0.smithi107.stdout:2/621: chown d0/d9/d11/df/d5a/d6d 404 1 2024-05-17T21:53:04.652 INFO:tasks.workunit.client.1.smithi138.stdout:5/598: getdents d1/d53/d88/db9/d39/d69/dc2 0 2024-05-17T21:53:04.653 INFO:tasks.workunit.client.0.smithi107.stdout:2/622: chown d0/d9/d11/df/d5a/c5d 167264068 1 2024-05-17T21:53:04.655 INFO:tasks.workunit.client.1.smithi138.stdout:2/637: stat d3/d21/d2f/f3f 0 2024-05-17T21:53:04.656 INFO:tasks.workunit.client.1.smithi138.stdout:2/638: write d3/da0/daa/dce/dd0/fb0 [885660,84198] 0 2024-05-17T21:53:04.659 INFO:tasks.workunit.client.0.smithi107.stdout:6/712: dwrite d9/d16/d4d/db8/dc9/dd0/f63 [0,4194304] 0 2024-05-17T21:53:04.664 INFO:tasks.workunit.client.0.smithi107.stdout:5/784: dwrite d5/db/d16/d2d/d51/f101 [0,4194304] 0 2024-05-17T21:53:04.678 INFO:tasks.workunit.client.1.smithi138.stdout:6/609: mkdir d7/d69/db5/dc8/d43/d5f/d94/dc9/dcf 0 2024-05-17T21:53:04.688 INFO:tasks.workunit.client.0.smithi107.stdout:2/623: mkdir d0/d93/dc5 0 2024-05-17T21:53:04.689 INFO:tasks.workunit.client.1.smithi138.stdout:8/555: getdents d0/dc/d1a/d33/d4a 0 2024-05-17T21:53:04.690 INFO:tasks.workunit.client.0.smithi107.stdout:1/807: dwrite d4/df/ff3 [0,4194304] 0 2024-05-17T21:53:04.690 INFO:tasks.workunit.client.1.smithi138.stdout:4/639: truncate d0/d3/d17/d21/d78/f9a 4050665 0 2024-05-17T21:53:04.691 INFO:tasks.workunit.client.1.smithi138.stdout:8/556: chown d0/dc/d1a/l70 6136130 1 2024-05-17T21:53:04.692 INFO:tasks.workunit.client.1.smithi138.stdout:8/557: truncate d0/d71/f9f 674758 0 2024-05-17T21:53:04.694 INFO:tasks.workunit.client.1.smithi138.stdout:3/717: getdents d1/d4f/d50/dbb/dde 0 2024-05-17T21:53:04.695 INFO:tasks.workunit.client.0.smithi107.stdout:9/804: write d0/dd/d2f/fd9 [2634233,99974] 0 2024-05-17T21:53:04.697 INFO:tasks.workunit.client.1.smithi138.stdout:0/587: write d3/fd [63744,96143] 0 2024-05-17T21:53:04.697 INFO:tasks.workunit.client.0.smithi107.stdout:5/785: symlink d5/db/d16/d54/d59/l104 0 2024-05-17T21:53:04.699 INFO:tasks.workunit.client.1.smithi138.stdout:5/599: rename d1/l4 to d1/d53/db1/db8/lc4 0 2024-05-17T21:53:04.704 INFO:tasks.workunit.client.1.smithi138.stdout:3/718: dwrite d1/d4d/fc6 [4194304,4194304] 0 2024-05-17T21:53:04.705 INFO:tasks.workunit.client.0.smithi107.stdout:3/808: dwrite d6/d8/d10/ff3 [0,4194304] 0 2024-05-17T21:53:04.710 INFO:tasks.workunit.client.1.smithi138.stdout:5/600: dwrite d1/d53/d88/db9/d39/d62/fac [0,4194304] 0 2024-05-17T21:53:04.710 INFO:tasks.workunit.client.1.smithi138.stdout:6/610: truncate d7/da0/fbf 1088091 0 2024-05-17T21:53:04.712 INFO:tasks.workunit.client.0.smithi107.stdout:0/883: dwrite d3/d24/d6f/d9f/fa0 [0,4194304] 0 2024-05-17T21:53:04.712 INFO:tasks.workunit.client.0.smithi107.stdout:1/808: rmdir d4/df/d14/d2e/d43 39 2024-05-17T21:53:04.714 INFO:tasks.workunit.client.1.smithi138.stdout:6/611: write d7/d69/db5/dc8/d43/d3f/d79/fbb [277155,16666] 0 2024-05-17T21:53:04.715 INFO:tasks.workunit.client.0.smithi107.stdout:7/756: link d5/df/fc9 d5/d16/d3b/d45/d91/dc1/f106 0 2024-05-17T21:53:04.718 INFO:tasks.workunit.client.0.smithi107.stdout:7/757: readlink d5/df/d11/d1c/d31/d56/lf9 0 2024-05-17T21:53:04.724 INFO:tasks.workunit.client.0.smithi107.stdout:4/788: sync 2024-05-17T21:53:04.724 INFO:tasks.workunit.client.1.smithi138.stdout:6/612: dwrite d7/d69/db5/dc8/d43/d5f/d6a/d9b/faf [0,4194304] 0 2024-05-17T21:53:04.729 INFO:tasks.workunit.client.0.smithi107.stdout:5/786: rmdir d5/d75/dc7/dd4 39 2024-05-17T21:53:04.733 INFO:tasks.workunit.client.1.smithi138.stdout:9/538: getdents d6/d7/d16/d29/d48/d6e/d74 0 2024-05-17T21:53:04.739 INFO:tasks.workunit.client.0.smithi107.stdout:3/809: rename d6/d8/l32 to d6/d8/d10/d35/d36/d4c/dcc/l106 0 2024-05-17T21:53:04.741 INFO:tasks.workunit.client.0.smithi107.stdout:7/758: dread d5/df/d11/d1c/d31/f48 [0,4194304] 0 2024-05-17T21:53:04.743 INFO:tasks.workunit.client.0.smithi107.stdout:1/809: mknod d4/df/d14/d1d/c101 0 2024-05-17T21:53:04.744 INFO:tasks.workunit.client.1.smithi138.stdout:8/558: symlink d0/dc/d8d/la2 0 2024-05-17T21:53:04.745 INFO:tasks.workunit.client.0.smithi107.stdout:0/884: dread d3/dc/f26 [0,4194304] 0 2024-05-17T21:53:04.747 INFO:tasks.workunit.client.0.smithi107.stdout:4/789: mkdir d3/db/d35/d109 0 2024-05-17T21:53:04.751 INFO:tasks.workunit.client.0.smithi107.stdout:9/805: dwrite d0/dd/d21/d29/d3e/d45/d68/f6c [0,4194304] 0 2024-05-17T21:53:04.755 INFO:tasks.workunit.client.0.smithi107.stdout:9/806: write d0/dd/d3d/f8e [870626,121627] 0 2024-05-17T21:53:04.757 INFO:tasks.workunit.client.1.smithi138.stdout:3/719: write d1/d3/d16/d3e/f43 [1578599,98258] 0 2024-05-17T21:53:04.763 INFO:tasks.workunit.client.1.smithi138.stdout:1/588: sync 2024-05-17T21:53:04.764 INFO:tasks.workunit.client.0.smithi107.stdout:5/787: dread d5/db/d16/d2d/d4d/f6a [0,4194304] 0 2024-05-17T21:53:04.767 INFO:tasks.workunit.client.1.smithi138.stdout:1/589: dread d4/dd/d3b/f9e [0,4194304] 0 2024-05-17T21:53:04.768 INFO:tasks.workunit.client.1.smithi138.stdout:0/588: write d3/d10/f20 [1044683,32484] 0 2024-05-17T21:53:04.771 INFO:tasks.workunit.client.1.smithi138.stdout:9/539: write d6/d7/f9 [4069551,111496] 0 2024-05-17T21:53:04.775 INFO:tasks.workunit.client.1.smithi138.stdout:0/589: dwrite d3/d10/f11 [0,4194304] 0 2024-05-17T21:53:04.786 INFO:tasks.workunit.client.0.smithi107.stdout:1/810: dread d4/df/d1a/d41/fe5 [0,4194304] 0 2024-05-17T21:53:04.793 INFO:tasks.workunit.client.1.smithi138.stdout:2/639: getdents d3/d21 0 2024-05-17T21:53:04.796 INFO:tasks.workunit.client.0.smithi107.stdout:3/810: stat d6/d2a/f42 0 2024-05-17T21:53:04.798 INFO:tasks.workunit.client.0.smithi107.stdout:6/713: getdents d9/d16/d22/d5f/dd8 0 2024-05-17T21:53:04.799 INFO:tasks.workunit.client.1.smithi138.stdout:6/613: symlink d7/d69/db5/dc8/d43/ld0 0 2024-05-17T21:53:04.800 INFO:tasks.workunit.client.0.smithi107.stdout:7/759: dread d5/f70 [0,4194304] 0 2024-05-17T21:53:04.800 INFO:tasks.workunit.client.0.smithi107.stdout:6/714: read d9/d16/d22/daf/fb9 [1042102,58408] 0 2024-05-17T21:53:04.801 INFO:tasks.workunit.client.1.smithi138.stdout:6/614: write d7/d69/db5/dc8/d43/d3f/db6/fc5 [883136,63058] 0 2024-05-17T21:53:04.801 INFO:tasks.workunit.client.0.smithi107.stdout:8/744: sync 2024-05-17T21:53:04.801 INFO:tasks.workunit.client.0.smithi107.stdout:4/790: mknod d3/d6/d26/d3b/d91/c10a 0 2024-05-17T21:53:04.803 INFO:tasks.workunit.client.0.smithi107.stdout:2/624: mkdir d0/d9/d11/d15/d99/dc6 0 2024-05-17T21:53:04.804 INFO:tasks.workunit.client.0.smithi107.stdout:8/745: dread dc/d21/db8/fc4 [0,4194304] 0 2024-05-17T21:53:04.806 INFO:tasks.workunit.client.0.smithi107.stdout:8/746: read dc/d21/d27/d2f/f4a [135879,97076] 0 2024-05-17T21:53:04.807 INFO:tasks.workunit.client.0.smithi107.stdout:9/807: unlink d0/dd/d21/d29/d57/d5f/c99 0 2024-05-17T21:53:04.809 INFO:tasks.workunit.client.0.smithi107.stdout:5/788: mknod d5/db/dc/d1e/d6e/c105 0 2024-05-17T21:53:04.810 INFO:tasks.workunit.client.0.smithi107.stdout:5/789: chown d5/db/d16/d54/d59/d92/cdc 216 1 2024-05-17T21:53:04.811 INFO:tasks.workunit.client.0.smithi107.stdout:1/811: mknod d4/d78/c102 0 2024-05-17T21:53:04.815 INFO:tasks.workunit.client.1.smithi138.stdout:3/720: dread d1/d62/d9b/fd9 [0,4194304] 0 2024-05-17T21:53:04.815 INFO:tasks.workunit.client.0.smithi107.stdout:1/812: dread d4/f1b [0,4194304] 0 2024-05-17T21:53:04.815 INFO:tasks.workunit.client.0.smithi107.stdout:3/811: rename d6/d8/d10/d25/c51 to d6/d8/d3e/d3f/d90/da7/ddc/c107 0 2024-05-17T21:53:04.816 INFO:tasks.workunit.client.0.smithi107.stdout:3/812: dread - d6/d1a/d57/ffa zero size 2024-05-17T21:53:04.817 INFO:tasks.workunit.client.1.smithi138.stdout:2/640: chown d3/l2c 1260064924 1 2024-05-17T21:53:04.823 INFO:tasks.workunit.client.0.smithi107.stdout:7/760: mkdir d5/d16/d3b/d45/d91/dac/d107 0 2024-05-17T21:53:04.825 INFO:tasks.workunit.client.0.smithi107.stdout:6/715: creat d9/d16/d4d/db8/dc9/fe0 x:0 0 0 2024-05-17T21:53:04.826 INFO:tasks.workunit.client.1.smithi138.stdout:1/590: mkdir d4/dd/db4/dba 0 2024-05-17T21:53:04.828 INFO:tasks.workunit.client.1.smithi138.stdout:9/540: creat d6/d7/d16/d29/da6/fae x:0 0 0 2024-05-17T21:53:04.831 INFO:tasks.workunit.client.1.smithi138.stdout:0/590: mkdir d3/d10/d38/d3e/d59/d9b/dc3 0 2024-05-17T21:53:04.833 INFO:tasks.workunit.client.1.smithi138.stdout:3/721: mknod d1/d62/db0/cf9 0 2024-05-17T21:53:04.834 INFO:tasks.workunit.client.1.smithi138.stdout:2/641: readlink d3/d21/d2f/lb3 0 2024-05-17T21:53:04.836 INFO:tasks.workunit.client.1.smithi138.stdout:2/642: read d3/d21/d2f/d3e/d55/d99/d84/d9f/fcb [149363,117308] 0 2024-05-17T21:53:04.836 INFO:tasks.workunit.client.1.smithi138.stdout:5/601: getdents d1/d60 0 2024-05-17T21:53:04.838 INFO:tasks.workunit.client.1.smithi138.stdout:5/602: fdatasync d1/d60/d2f/d5e/f79 0 2024-05-17T21:53:04.838 INFO:tasks.workunit.client.1.smithi138.stdout:8/559: link d0/d11/l2e d0/d71/la3 0 2024-05-17T21:53:04.839 INFO:tasks.workunit.client.0.smithi107.stdout:9/808: dwrite d0/dd/f60 [0,4194304] 0 2024-05-17T21:53:04.839 INFO:tasks.workunit.client.1.smithi138.stdout:8/560: chown d0/dc/d3b/d4d/d7d/l90 222269570 1 2024-05-17T21:53:04.841 INFO:tasks.workunit.client.0.smithi107.stdout:1/813: mkdir d4/df/d1a/d26/d72/d85/df9/d103 0 2024-05-17T21:53:04.842 INFO:tasks.workunit.client.1.smithi138.stdout:8/561: truncate d0/dc/d3b/d4d/d7d/f9e 223653 0 2024-05-17T21:53:04.843 INFO:tasks.workunit.client.1.smithi138.stdout:1/591: dread - d4/db/d29/f5b zero size 2024-05-17T21:53:04.844 INFO:tasks.workunit.client.1.smithi138.stdout:8/562: write d0/dc/d1a/f4b [414956,115678] 0 2024-05-17T21:53:04.844 INFO:tasks.workunit.client.1.smithi138.stdout:1/592: truncate d4/f3f 1921405 0 2024-05-17T21:53:04.846 INFO:tasks.workunit.client.1.smithi138.stdout:0/591: fsync d3/d3d/d51/f54 0 2024-05-17T21:53:04.848 INFO:tasks.workunit.client.1.smithi138.stdout:0/592: chown d3/d10/f2d 1847957585 1 2024-05-17T21:53:04.850 INFO:tasks.workunit.client.1.smithi138.stdout:3/722: dread d1/d3/d8d/d92/f96 [0,4194304] 0 2024-05-17T21:53:04.852 INFO:tasks.workunit.client.1.smithi138.stdout:1/593: dwrite d4/df/f2c [0,4194304] 0 2024-05-17T21:53:04.853 INFO:tasks.workunit.client.0.smithi107.stdout:7/761: creat d5/d16/d3b/d45/f108 x:0 0 0 2024-05-17T21:53:04.854 INFO:tasks.workunit.client.1.smithi138.stdout:4/640: truncate d0/d2f/d49/d60/d1f/d7e/f93 4065447 0 2024-05-17T21:53:04.854 INFO:tasks.workunit.client.1.smithi138.stdout:4/641: stat d0/d2f/f6f 0 2024-05-17T21:53:04.855 INFO:tasks.workunit.client.0.smithi107.stdout:6/716: creat d9/d58/d7b/da3/fe1 x:0 0 0 2024-05-17T21:53:04.856 INFO:tasks.workunit.client.1.smithi138.stdout:4/642: write d0/d3/d1d/f2e [797173,8891] 0 2024-05-17T21:53:04.860 INFO:tasks.workunit.client.1.smithi138.stdout:5/603: rename d1/d2c/d63/db5/d1b/d29 to d1/d2c/d63/db5/d1b/dc5 0 2024-05-17T21:53:04.862 INFO:tasks.workunit.client.1.smithi138.stdout:2/643: dwrite d3/d21/d2f/d3e/d63/da3/fb6 [8388608,4194304] 0 2024-05-17T21:53:04.862 INFO:tasks.workunit.client.1.smithi138.stdout:5/604: read - d1/d2c/d63/db5/d1b/fba zero size 2024-05-17T21:53:04.872 INFO:tasks.workunit.client.1.smithi138.stdout:0/593: mkdir d3/d10/d6d/d81/dc4 0 2024-05-17T21:53:04.874 INFO:tasks.workunit.client.0.smithi107.stdout:5/790: mkdir d5/d106 0 2024-05-17T21:53:04.879 INFO:tasks.workunit.client.1.smithi138.stdout:5/605: creat d1/d53/d96/fc6 x:0 0 0 2024-05-17T21:53:04.880 INFO:tasks.workunit.client.0.smithi107.stdout:1/814: truncate d4/df/d14/d2e/d43/ff8 1081813 0 2024-05-17T21:53:04.881 INFO:tasks.workunit.client.1.smithi138.stdout:2/644: dread - d3/d21/d2f/d3e/d63/f7b zero size 2024-05-17T21:53:04.882 INFO:tasks.workunit.client.1.smithi138.stdout:6/615: dwrite d7/d8/d78/f7f [0,4194304] 0 2024-05-17T21:53:04.882 INFO:tasks.workunit.client.0.smithi107.stdout:1/815: truncate d4/df/d1a/d26/d7a/f93 4819834 0 2024-05-17T21:53:04.882 INFO:tasks.workunit.client.0.smithi107.stdout:1/816: chown d4/de/f3c 12242074 1 2024-05-17T21:53:04.883 INFO:tasks.workunit.client.0.smithi107.stdout:0/885: getdents d3/dc/d31/ddc 0 2024-05-17T21:53:04.884 INFO:tasks.workunit.client.0.smithi107.stdout:0/886: write d3/dc/f16 [2641098,15797] 0 2024-05-17T21:53:04.885 INFO:tasks.workunit.client.1.smithi138.stdout:4/643: mkdir d0/d3/d17/d21/d78/d9f/de4 0 2024-05-17T21:53:04.887 INFO:tasks.workunit.client.0.smithi107.stdout:7/762: mknod d5/df/d78/c109 0 2024-05-17T21:53:04.888 INFO:tasks.workunit.client.0.smithi107.stdout:4/791: dwrite d3/d6/d26/d36/d7e/f87 [0,4194304] 0 2024-05-17T21:53:04.895 INFO:tasks.workunit.client.0.smithi107.stdout:2/625: mkdir d0/d9/d11/df/d5a/dc7 0 2024-05-17T21:53:04.897 INFO:tasks.workunit.client.0.smithi107.stdout:3/813: write d6/d2a/d2d/d37/f3a [1950940,107683] 0 2024-05-17T21:53:04.898 INFO:tasks.workunit.client.1.smithi138.stdout:0/594: symlink d3/lc5 0 2024-05-17T21:53:04.899 INFO:tasks.workunit.client.0.smithi107.stdout:8/747: link dc/d44/d4b/d52/d70/d76/la6 dc/d21/d97/db7/dcd/l107 0 2024-05-17T21:53:04.902 INFO:tasks.workunit.client.0.smithi107.stdout:5/791: mkdir d5/db/d16/d107 0 2024-05-17T21:53:04.903 INFO:tasks.workunit.client.1.smithi138.stdout:4/644: fsync d0/d1b/d2b/f65 0 2024-05-17T21:53:04.913 INFO:tasks.workunit.client.0.smithi107.stdout:0/887: mkdir d3/d24/dcd/d126 0 2024-05-17T21:53:04.917 INFO:tasks.workunit.client.0.smithi107.stdout:4/792: rename d3/db/f1f to d3/d6/d26/d36/d8b/d95/dee/f10b 0 2024-05-17T21:53:04.918 INFO:tasks.workunit.client.0.smithi107.stdout:4/793: truncate d3/db/d35/d5e/dff/fc8 508630 0 2024-05-17T21:53:04.921 INFO:tasks.workunit.client.1.smithi138.stdout:4/645: mknod d0/d3/db/da0/ce5 0 2024-05-17T21:53:04.928 INFO:tasks.workunit.client.0.smithi107.stdout:8/748: fsync dc/d21/d27/d2f/f7f 0 2024-05-17T21:53:04.929 INFO:tasks.workunit.client.0.smithi107.stdout:5/792: mknod d5/db/d16/d2d/d40/d9f/c108 0 2024-05-17T21:53:04.930 INFO:tasks.workunit.client.0.smithi107.stdout:3/814: dwrite d6/d2a/f6a [0,4194304] 0 2024-05-17T21:53:04.931 INFO:tasks.workunit.client.1.smithi138.stdout:4/646: mknod d0/d1b/ce6 0 2024-05-17T21:53:04.931 INFO:tasks.workunit.client.1.smithi138.stdout:2/645: write f2 [1743996,21851] 0 2024-05-17T21:53:04.931 INFO:tasks.workunit.client.0.smithi107.stdout:2/626: rmdir d0/d9/d11/df 39 2024-05-17T21:53:04.934 INFO:tasks.workunit.client.0.smithi107.stdout:1/817: mkdir d4/df/d1a/d26/d72/daa/deb/d104 0 2024-05-17T21:53:04.934 INFO:tasks.workunit.client.0.smithi107.stdout:0/888: rmdir d3/d24/d6f/db5 39 2024-05-17T21:53:04.936 INFO:tasks.workunit.client.1.smithi138.stdout:4/647: creat d0/d3/d17/d5e/d7f/fe7 x:0 0 0 2024-05-17T21:53:04.938 INFO:tasks.workunit.client.1.smithi138.stdout:2/646: creat d3/da0/daa/dce/fd6 x:0 0 0 2024-05-17T21:53:04.939 INFO:tasks.workunit.client.1.smithi138.stdout:2/647: fdatasync f2 0 2024-05-17T21:53:04.940 INFO:tasks.workunit.client.1.smithi138.stdout:6/616: truncate d7/d69/f71 336900 0 2024-05-17T21:53:04.940 INFO:tasks.workunit.client.0.smithi107.stdout:9/809: dwrite d0/dd/d21/daf/fe1 [0,4194304] 0 2024-05-17T21:53:04.941 INFO:tasks.workunit.client.1.smithi138.stdout:2/648: write d3/d21/d2f/d3e/d63/da3/fb9 [411494,94906] 0 2024-05-17T21:53:04.945 INFO:tasks.workunit.client.1.smithi138.stdout:4/648: read d0/fcb [941103,109142] 0 2024-05-17T21:53:04.950 INFO:tasks.workunit.client.1.smithi138.stdout:6/617: dwrite d7/d69/db5/dc8/d43/d63/fce [0,4194304] 0 2024-05-17T21:53:04.951 INFO:tasks.workunit.client.0.smithi107.stdout:8/749: symlink dc/d21/d97/d63/d71/d81/de5/l108 0 2024-05-17T21:53:04.952 INFO:tasks.workunit.client.0.smithi107.stdout:8/750: stat dc/d21/d27/d2f/fa1 0 2024-05-17T21:53:04.953 INFO:tasks.workunit.client.1.smithi138.stdout:0/595: write d3/d10/d38/d3e/f5b [420131,119225] 0 2024-05-17T21:53:04.953 INFO:tasks.workunit.client.0.smithi107.stdout:3/815: rmdir d6/d8/d3e/d3f/d90/da7/ddc 39 2024-05-17T21:53:04.954 INFO:tasks.workunit.client.0.smithi107.stdout:6/717: dwrite d9/d58/f9b [0,4194304] 0 2024-05-17T21:53:04.955 INFO:tasks.workunit.client.0.smithi107.stdout:2/627: write d0/d9/f77 [3366761,43693] 0 2024-05-17T21:53:04.956 INFO:tasks.workunit.client.1.smithi138.stdout:2/649: fsync d3/f35 0 2024-05-17T21:53:04.958 INFO:tasks.workunit.client.0.smithi107.stdout:0/889: rmdir d3/dc/d22/d3d/d56 39 2024-05-17T21:53:04.960 INFO:tasks.workunit.client.1.smithi138.stdout:4/649: mkdir d0/d2f/d49/d60/d1f/d7e/de8 0 2024-05-17T21:53:04.963 INFO:tasks.workunit.client.1.smithi138.stdout:6/618: unlink l5 0 2024-05-17T21:53:04.965 INFO:tasks.workunit.client.0.smithi107.stdout:4/794: creat d3/db/d1e/d66/f10c x:0 0 0 2024-05-17T21:53:04.966 INFO:tasks.workunit.client.0.smithi107.stdout:4/795: chown d3/db/f46 29167 1 2024-05-17T21:53:04.967 INFO:tasks.workunit.client.0.smithi107.stdout:4/796: fdatasync d3/d4a/d47/f4b 0 2024-05-17T21:53:04.968 INFO:tasks.workunit.client.0.smithi107.stdout:9/810: mkdir d0/dd/d2f/d87/d89/d10b 0 2024-05-17T21:53:04.973 INFO:tasks.workunit.client.0.smithi107.stdout:8/751: symlink dc/d21/d97/l109 0 2024-05-17T21:53:04.977 INFO:tasks.workunit.client.1.smithi138.stdout:7/582: sync 2024-05-17T21:53:04.977 INFO:tasks.workunit.client.0.smithi107.stdout:6/718: mkdir d9/d58/d66/de2 0 2024-05-17T21:53:04.978 INFO:tasks.workunit.client.1.smithi138.stdout:7/583: read - d0/d18/d40/d9a/fbb zero size 2024-05-17T21:53:04.979 INFO:tasks.workunit.client.1.smithi138.stdout:7/584: write d0/d18/d40/d41/d50/f61 [3086673,95721] 0 2024-05-17T21:53:04.980 INFO:tasks.workunit.client.1.smithi138.stdout:7/585: chown d0/f12 0 1 2024-05-17T21:53:04.981 INFO:tasks.workunit.client.1.smithi138.stdout:7/586: write d0/d18/d40/d5f/f8f [729817,105687] 0 2024-05-17T21:53:04.985 INFO:tasks.workunit.client.1.smithi138.stdout:4/650: symlink d0/d1b/dd4/le9 0 2024-05-17T21:53:04.990 INFO:tasks.workunit.client.1.smithi138.stdout:6/619: creat d7/d8/dc2/fd1 x:0 0 0 2024-05-17T21:53:04.992 INFO:tasks.workunit.client.1.smithi138.stdout:0/596: creat d3/d10/d38/d3e/fc6 x:0 0 0 2024-05-17T21:53:04.993 INFO:tasks.workunit.client.0.smithi107.stdout:8/752: chown dc/d44/d4b/l58 93457 1 2024-05-17T21:53:04.994 INFO:tasks.workunit.client.0.smithi107.stdout:3/816: write d6/d8/f76 [3815902,59321] 0 2024-05-17T21:53:04.996 INFO:tasks.workunit.client.1.smithi138.stdout:7/587: truncate d0/d39/f3d 61128 0 2024-05-17T21:53:04.997 INFO:tasks.workunit.client.1.smithi138.stdout:7/588: chown d0/d39/d78 1637 1 2024-05-17T21:53:04.998 INFO:tasks.workunit.client.1.smithi138.stdout:2/650: dwrite d3/f15 [0,4194304] 0 2024-05-17T21:53:04.998 INFO:tasks.workunit.client.0.smithi107.stdout:0/890: write d3/d39/f41 [205331,122272] 0 2024-05-17T21:53:04.999 INFO:tasks.workunit.client.0.smithi107.stdout:0/891: stat d3/dc/d22/d23/d97/l82 0 2024-05-17T21:53:04.999 INFO:tasks.workunit.client.0.smithi107.stdout:2/628: mkdir d0/d9/d11/df/d5a/d6d/dc8 0 2024-05-17T21:53:05.002 INFO:tasks.workunit.client.0.smithi107.stdout:6/719: dwrite d9/d16/f4a [0,4194304] 0 2024-05-17T21:53:05.004 INFO:tasks.workunit.client.0.smithi107.stdout:1/818: creat d4/df/f105 x:0 0 0 2024-05-17T21:53:05.005 INFO:tasks.workunit.client.0.smithi107.stdout:1/819: readlink d4/df/d14/d2e/d43/d47/l89 0 2024-05-17T21:53:05.007 INFO:tasks.workunit.client.1.smithi138.stdout:4/651: dwrite d0/f34 [0,4194304] 0 2024-05-17T21:53:05.011 INFO:tasks.workunit.client.1.smithi138.stdout:6/620: dwrite d7/d8/f50 [0,4194304] 0 2024-05-17T21:53:05.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:04 smithi138 ceph-mon[35948]: from='mgr.14636 172.21.15.138:0/3120106723' entity='mgr.smithi138.napnll' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:53:05.082 INFO:tasks.workunit.client.0.smithi107.stdout:9/811: write d0/dd/d3d/fb4 [1996131,65508] 0 2024-05-17T21:53:05.083 INFO:tasks.workunit.client.0.smithi107.stdout:4/797: symlink d3/d6/d26/l10d 0 2024-05-17T21:53:05.084 INFO:tasks.workunit.client.0.smithi107.stdout:4/798: dread - d3/d6/d26/d36/d8b/d95/ff9 zero size 2024-05-17T21:53:05.088 INFO:tasks.workunit.client.1.smithi138.stdout:7/589: symlink d0/d18/d40/d5f/lbd 0 2024-05-17T21:53:05.089 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:04 smithi107 ceph-mon[28434]: from='mgr.14636 172.21.15.138:0/3120106723' entity='mgr.smithi138.napnll' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:53:05.090 INFO:tasks.workunit.client.1.smithi138.stdout:2/651: creat d3/d21/d2f/d3e/d55/d99/dd2/fd7 x:0 0 0 2024-05-17T21:53:05.093 INFO:tasks.workunit.client.0.smithi107.stdout:2/629: chown d0/d9/d11/c5 707833914 1 2024-05-17T21:53:05.097 INFO:tasks.workunit.client.0.smithi107.stdout:2/630: dread d0/d9/d11/df/d5a/d6d/fb6 [0,4194304] 0 2024-05-17T21:53:05.097 INFO:tasks.workunit.client.0.smithi107.stdout:6/720: mknod d9/d58/d66/dd5/ce3 0 2024-05-17T21:53:05.099 INFO:tasks.workunit.client.0.smithi107.stdout:1/820: unlink d4/df/d14/d2e/d43/c4f 0 2024-05-17T21:53:05.100 INFO:tasks.workunit.client.0.smithi107.stdout:1/821: write d4/df/d14/d1d/ff5 [38504,55563] 0 2024-05-17T21:53:05.100 INFO:tasks.workunit.client.1.smithi138.stdout:7/590: unlink d0/ld 0 2024-05-17T21:53:05.101 INFO:tasks.workunit.client.1.smithi138.stdout:6/621: write d7/d8/f5d [3930844,52628] 0 2024-05-17T21:53:05.102 INFO:tasks.workunit.client.0.smithi107.stdout:9/812: mknod d0/dd/d21/d29/d3e/dfe/c10c 0 2024-05-17T21:53:05.103 INFO:tasks.workunit.client.0.smithi107.stdout:9/813: chown d0/dd/d2f/f64 7 1 2024-05-17T21:53:05.104 INFO:tasks.workunit.client.1.smithi138.stdout:4/652: mknod d0/d2f/cea 0 2024-05-17T21:53:05.108 INFO:tasks.workunit.client.1.smithi138.stdout:0/597: creat d3/d10/d6d/fc7 x:0 0 0 2024-05-17T21:53:05.109 INFO:tasks.workunit.client.0.smithi107.stdout:7/763: sync 2024-05-17T21:53:05.109 INFO:tasks.workunit.client.0.smithi107.stdout:9/814: dwrite d0/dd/d2f/d87/fea [0,4194304] 0 2024-05-17T21:53:05.109 INFO:tasks.workunit.client.0.smithi107.stdout:2/631: symlink d0/d9/d11/df/d5a/d6d/lc9 0 2024-05-17T21:53:05.111 INFO:tasks.workunit.client.0.smithi107.stdout:2/632: truncate d0/d26/d64/f8d 1490080 0 2024-05-17T21:53:05.112 INFO:tasks.workunit.client.0.smithi107.stdout:4/799: write d3/db/d1e/f9a [2693897,103167] 0 2024-05-17T21:53:05.112 INFO:tasks.workunit.client.1.smithi138.stdout:2/652: mkdir d3/d21/d2f/d3e/d55/d99/d84/dbe/dd8 0 2024-05-17T21:53:05.113 INFO:tasks.workunit.client.0.smithi107.stdout:8/753: write dc/d21/d69/f8e [3364760,13237] 0 2024-05-17T21:53:05.119 INFO:tasks.workunit.client.0.smithi107.stdout:0/892: dwrite d3/d24/d6f/db5/da5/f10e [4194304,4194304] 0 2024-05-17T21:53:05.121 INFO:tasks.workunit.client.1.smithi138.stdout:2/653: rmdir d3/d81/d96 39 2024-05-17T21:53:05.129 INFO:tasks.workunit.client.0.smithi107.stdout:2/633: creat d0/d9/d11/df/d5a/d6d/fca x:0 0 0 2024-05-17T21:53:05.131 INFO:tasks.workunit.client.0.smithi107.stdout:4/800: creat d3/d4a/d47/da3/f10e x:0 0 0 2024-05-17T21:53:05.134 INFO:tasks.workunit.client.1.smithi138.stdout:4/653: dwrite d0/d2f/d49/d60/d1f/d7e/f83 [0,4194304] 0 2024-05-17T21:53:05.139 INFO:tasks.workunit.client.1.smithi138.stdout:9/541: sync 2024-05-17T21:53:05.139 INFO:tasks.workunit.client.1.smithi138.stdout:7/591: getdents d0/d18/d40/d41/d51/d66 0 2024-05-17T21:53:05.139 INFO:tasks.workunit.client.1.smithi138.stdout:2/654: dread d3/d81/fa7 [0,4194304] 0 2024-05-17T21:53:05.139 INFO:tasks.workunit.client.0.smithi107.stdout:1/822: write d4/f66 [1161224,88790] 0 2024-05-17T21:53:05.140 INFO:tasks.workunit.client.1.smithi138.stdout:9/542: chown d6/f2a 277 1 2024-05-17T21:53:05.140 INFO:tasks.workunit.client.0.smithi107.stdout:1/823: fdatasync d4/df/d1a/f7b 0 2024-05-17T21:53:05.141 INFO:tasks.workunit.client.0.smithi107.stdout:2/634: chown d0/d9/d11/laf 2 1 2024-05-17T21:53:05.142 INFO:tasks.workunit.client.1.smithi138.stdout:6/622: truncate d7/d69/db5/dc8/d43/d3f/d79/f93 264225 0 2024-05-17T21:53:05.145 INFO:tasks.workunit.client.1.smithi138.stdout:6/623: stat d7/d69/db5/dc8/d43/d63/ca8 0 2024-05-17T21:53:05.146 INFO:tasks.workunit.client.1.smithi138.stdout:0/598: dwrite d3/d10/f7d [0,4194304] 0 2024-05-17T21:53:05.147 INFO:tasks.workunit.client.1.smithi138.stdout:7/592: truncate d0/f1a 2085027 0 2024-05-17T21:53:05.148 INFO:tasks.workunit.client.0.smithi107.stdout:6/721: getdents d9/d16/d4d/db8 0 2024-05-17T21:53:05.149 INFO:tasks.workunit.client.1.smithi138.stdout:2/655: unlink d3/d21/l46 0 2024-05-17T21:53:05.152 INFO:tasks.workunit.client.0.smithi107.stdout:7/764: creat d5/df/d11/d1d/f10a x:0 0 0 2024-05-17T21:53:05.153 INFO:tasks.workunit.client.1.smithi138.stdout:2/656: dread d3/d21/d2f/f45 [0,4194304] 0 2024-05-17T21:53:05.154 INFO:tasks.workunit.client.1.smithi138.stdout:9/543: rename d6/d7/f9 to d6/d7/d16/d29/d48/d3b/faf 0 2024-05-17T21:53:05.158 INFO:tasks.workunit.client.0.smithi107.stdout:9/815: rmdir d0/dd/d1f/de4 0 2024-05-17T21:53:05.159 INFO:tasks.workunit.client.1.smithi138.stdout:9/544: read d6/d7/d16/d29/d48/d6e/f56 [3755713,99771] 0 2024-05-17T21:53:05.161 INFO:tasks.workunit.client.1.smithi138.stdout:4/654: link d0/d2f/d49/d60/d1f/fb9 d0/d2f/d49/d60/d1f/d32/feb 0 2024-05-17T21:53:05.162 INFO:tasks.workunit.client.1.smithi138.stdout:4/655: chown d0/d1b/d88/cd6 0 1 2024-05-17T21:53:05.165 INFO:tasks.workunit.client.0.smithi107.stdout:4/801: mknod d3/db/d1e/c10f 0 2024-05-17T21:53:05.168 INFO:tasks.workunit.client.0.smithi107.stdout:2/635: rename d0/d9/d1c/d3e/d4f/c58 to d0/d9/d11/d15/d99/ccb 0 2024-05-17T21:53:05.170 INFO:tasks.workunit.client.0.smithi107.stdout:8/754: link dc/d21/d27/d2f/lc8 dc/d21/d69/l10a 0 2024-05-17T21:53:05.178 INFO:tasks.workunit.client.0.smithi107.stdout:7/765: mknod d5/d16/d3b/d45/d91/dc1/de6/c10b 0 2024-05-17T21:53:05.182 INFO:tasks.workunit.client.0.smithi107.stdout:4/802: creat d3/db/d1e/db5/f110 x:0 0 0 2024-05-17T21:53:05.182 INFO:tasks.workunit.client.1.smithi138.stdout:0/599: mknod d3/cc8 0 2024-05-17T21:53:05.185 INFO:tasks.workunit.client.0.smithi107.stdout:8/755: mknod dc/d21/d97/d63/d71/d81/de5/c10b 0 2024-05-17T21:53:05.187 INFO:tasks.workunit.client.0.smithi107.stdout:4/803: dwrite d3/db/d35/f101 [0,4194304] 0 2024-05-17T21:53:05.188 INFO:tasks.workunit.client.1.smithi138.stdout:6/624: write d7/d69/db5/dc8/d43/d3f/f5b [4525754,47566] 0 2024-05-17T21:53:05.189 INFO:tasks.workunit.client.1.smithi138.stdout:6/625: chown d7/d8/l6f 333444488 1 2024-05-17T21:53:05.189 INFO:tasks.workunit.client.0.smithi107.stdout:4/804: write d3/d4a/d47/d60/d3a/fd0 [408427,124062] 0 2024-05-17T21:53:05.191 INFO:tasks.workunit.client.0.smithi107.stdout:0/893: getdents d3/d24 0 2024-05-17T21:53:05.191 INFO:tasks.workunit.client.0.smithi107.stdout:7/766: dread - d5/df/d11/d1c/d31/d39/f94 zero size 2024-05-17T21:53:05.192 INFO:tasks.workunit.client.1.smithi138.stdout:2/657: write d3/f9 [2260157,18055] 0 2024-05-17T21:53:05.192 INFO:tasks.workunit.client.1.smithi138.stdout:2/658: chown d3/f35 51068759 1 2024-05-17T21:53:05.194 INFO:tasks.workunit.client.0.smithi107.stdout:2/636: mknod d0/d9/d11/d27/ccc 0 2024-05-17T21:53:05.198 INFO:tasks.workunit.client.0.smithi107.stdout:4/805: dwrite d3/d6/d26/d36/f6e [0,4194304] 0 2024-05-17T21:53:05.200 INFO:tasks.workunit.client.1.smithi138.stdout:7/593: write d0/d18/d40/d9a/fa5 [3481163,123512] 0 2024-05-17T21:53:05.201 INFO:tasks.workunit.client.0.smithi107.stdout:6/722: link d9/d16/f5e d9/d16/d22/d5f/d6f/fe4 0 2024-05-17T21:53:05.202 INFO:tasks.workunit.client.1.smithi138.stdout:7/594: write d0/d18/d40/d41/d50/f74 [476307,116779] 0 2024-05-17T21:53:05.204 INFO:tasks.workunit.client.1.smithi138.stdout:8/563: sync 2024-05-17T21:53:05.205 INFO:tasks.workunit.client.1.smithi138.stdout:8/564: fdatasync d0/d63/f9c 0 2024-05-17T21:53:05.205 INFO:tasks.workunit.client.1.smithi138.stdout:6/626: creat d7/da0/fd2 x:0 0 0 2024-05-17T21:53:05.206 INFO:tasks.workunit.client.1.smithi138.stdout:6/627: readlink d7/d69/db5/dc8/d43/ld0 0 2024-05-17T21:53:05.206 INFO:tasks.workunit.client.1.smithi138.stdout:8/565: chown d0/dc/d3b/d4d/d7d 86876749 1 2024-05-17T21:53:05.207 INFO:tasks.workunit.client.1.smithi138.stdout:8/566: stat d0/d11/c8f 0 2024-05-17T21:53:05.211 INFO:tasks.workunit.client.1.smithi138.stdout:6/628: dread d7/d69/db5/dc8/d43/d63/fce [0,4194304] 0 2024-05-17T21:53:05.212 INFO:tasks.workunit.client.0.smithi107.stdout:7/767: symlink d5/df/d11/d1c/d31/d39/dc0/l10c 0 2024-05-17T21:53:05.212 INFO:tasks.workunit.client.1.smithi138.stdout:9/545: truncate d6/d7/d16/d29/f3a 2517626 0 2024-05-17T21:53:05.213 INFO:tasks.workunit.client.0.smithi107.stdout:6/723: dread d9/d16/d4d/f5b [0,4194304] 0 2024-05-17T21:53:05.213 INFO:tasks.workunit.client.0.smithi107.stdout:2/637: truncate d0/d9/d1c/d22/f5b 542507 0 2024-05-17T21:53:05.214 INFO:tasks.workunit.client.1.smithi138.stdout:9/546: fsync d6/d7/d16/d29/d38/d40/d44/fa4 0 2024-05-17T21:53:05.214 INFO:tasks.workunit.client.0.smithi107.stdout:5/793: sync 2024-05-17T21:53:05.214 INFO:tasks.workunit.client.0.smithi107.stdout:3/817: sync 2024-05-17T21:53:05.215 INFO:tasks.workunit.client.0.smithi107.stdout:6/724: stat d9/d16/d4d/d85/d96/lc4 0 2024-05-17T21:53:05.215 INFO:tasks.workunit.client.1.smithi138.stdout:7/595: chown d0/d39/f3d 42046760 1 2024-05-17T21:53:05.216 INFO:tasks.workunit.client.0.smithi107.stdout:4/806: chown d3/db/d1e/lb1 7 1 2024-05-17T21:53:05.217 INFO:tasks.workunit.client.1.smithi138.stdout:6/629: dread d7/d69/db5/dc8/d43/d3f/db6/fc5 [0,4194304] 0 2024-05-17T21:53:05.218 INFO:tasks.workunit.client.1.smithi138.stdout:2/659: mknod d3/cd9 0 2024-05-17T21:53:05.221 INFO:tasks.workunit.client.1.smithi138.stdout:4/656: dwrite d0/d2f/d49/d60/f3c [0,4194304] 0 2024-05-17T21:53:05.223 INFO:tasks.workunit.client.0.smithi107.stdout:9/816: getdents d0/dd 0 2024-05-17T21:53:05.226 INFO:tasks.workunit.client.1.smithi138.stdout:8/567: unlink d0/dc/d1a/d33/d4a/d4e/c5c 0 2024-05-17T21:53:05.229 INFO:tasks.workunit.client.0.smithi107.stdout:0/894: rename d3/dc/f113 to d3/d75/dd0/f127 0 2024-05-17T21:53:05.231 INFO:tasks.workunit.client.1.smithi138.stdout:0/600: rename d3/d10/f35 to d3/fc9 0 2024-05-17T21:53:05.235 INFO:tasks.workunit.client.0.smithi107.stdout:2/638: fdatasync d0/d9/d11/d18/dad/db8/fbd 0 2024-05-17T21:53:05.237 INFO:tasks.workunit.client.0.smithi107.stdout:5/794: rmdir d5/db/d16/d54/d59/d65 39 2024-05-17T21:53:05.240 INFO:tasks.workunit.client.0.smithi107.stdout:3/818: mkdir d6/d2a/d2d/d37/d108 0 2024-05-17T21:53:05.240 INFO:tasks.workunit.client.0.smithi107.stdout:3/819: fdatasync d6/d8/f17 0 2024-05-17T21:53:05.242 INFO:tasks.workunit.client.1.smithi138.stdout:7/596: mkdir d0/d18/d40/d5f/dbe 0 2024-05-17T21:53:05.246 INFO:tasks.workunit.client.1.smithi138.stdout:6/630: creat d7/d69/db5/dc8/d43/d5f/d6a/fd3 x:0 0 0 2024-05-17T21:53:05.247 INFO:tasks.workunit.client.0.smithi107.stdout:8/756: link dc/d21/d97/l57 dc/d21/d27/d34/d4c/d7e/l10c 0 2024-05-17T21:53:05.248 INFO:tasks.workunit.client.1.smithi138.stdout:6/631: dread - d7/d69/db5/dc8/d43/d5f/d6a/d9b/db2/fb9 zero size 2024-05-17T21:53:05.248 INFO:tasks.workunit.client.1.smithi138.stdout:9/547: write d6/d7/f15 [3246325,1901] 0 2024-05-17T21:53:05.250 INFO:tasks.workunit.client.1.smithi138.stdout:2/660: symlink d3/d21/d2f/d3e/d55/d99/dd2/lda 0 2024-05-17T21:53:05.252 INFO:tasks.workunit.client.1.smithi138.stdout:4/657: mknod d0/d3/d17/d21/ddd/cec 0 2024-05-17T21:53:05.253 INFO:tasks.workunit.client.1.smithi138.stdout:3/723: sync 2024-05-17T21:53:05.253 INFO:tasks.workunit.client.1.smithi138.stdout:1/594: sync 2024-05-17T21:53:05.254 INFO:tasks.workunit.client.1.smithi138.stdout:5/606: sync 2024-05-17T21:53:05.254 INFO:tasks.workunit.client.0.smithi107.stdout:0/895: fsync d3/f1e 0 2024-05-17T21:53:05.255 INFO:tasks.workunit.client.1.smithi138.stdout:8/568: mkdir d0/dc/d3b/da4 0 2024-05-17T21:53:05.256 INFO:tasks.workunit.client.1.smithi138.stdout:8/569: stat d0/d31/c38 0 2024-05-17T21:53:05.256 INFO:tasks.workunit.client.1.smithi138.stdout:5/607: read - d1/d2c/d63/db5/d1b/dc5/d36/f98 zero size 2024-05-17T21:53:05.256 INFO:tasks.workunit.client.1.smithi138.stdout:1/595: write d4/df/d3a/d3c/d63/d6d/f87 [683268,15781] 0 2024-05-17T21:53:05.257 INFO:tasks.workunit.client.1.smithi138.stdout:8/570: readlink d0/dc/d1a/l70 0 2024-05-17T21:53:05.257 INFO:tasks.workunit.client.1.smithi138.stdout:8/571: chown d0/d71/d92 181190969 1 2024-05-17T21:53:05.257 INFO:tasks.workunit.client.0.smithi107.stdout:6/725: dwrite d9/d81/f82 [0,4194304] 0 2024-05-17T21:53:05.258 INFO:tasks.workunit.client.1.smithi138.stdout:0/601: mknod d3/d10/d6d/db6/cca 0 2024-05-17T21:53:05.258 INFO:tasks.workunit.client.1.smithi138.stdout:1/596: read d4/df/f1f [3069474,107703] 0 2024-05-17T21:53:05.259 INFO:tasks.workunit.client.0.smithi107.stdout:6/726: stat d9/d16/d4d/d85/l94 0 2024-05-17T21:53:05.259 INFO:tasks.workunit.client.0.smithi107.stdout:0/896: dread d3/dc/f26 [0,4194304] 0 2024-05-17T21:53:05.260 INFO:tasks.workunit.client.0.smithi107.stdout:6/727: chown d9 10547 1 2024-05-17T21:53:05.260 INFO:tasks.workunit.client.0.smithi107.stdout:9/817: write d0/dd/d21/d29/d3e/d45/d68/d75/f7c [1338790,109305] 0 2024-05-17T21:53:05.262 INFO:tasks.workunit.client.0.smithi107.stdout:0/897: write d3/dc/d22/ffc [1433030,8626] 0 2024-05-17T21:53:05.263 INFO:tasks.workunit.client.0.smithi107.stdout:5/795: symlink d5/db/dc/d1e/d6e/l109 0 2024-05-17T21:53:05.265 INFO:tasks.workunit.client.0.smithi107.stdout:3/820: symlink d6/d2a/d2d/da8/l109 0 2024-05-17T21:53:05.269 INFO:tasks.workunit.client.0.smithi107.stdout:4/807: mkdir d3/d6/d26/d3b/d91/df6/d111 0 2024-05-17T21:53:05.269 INFO:tasks.workunit.client.0.smithi107.stdout:0/898: dwrite d3/dc/d22/d3d/fa8 [0,4194304] 0 2024-05-17T21:53:05.270 INFO:tasks.workunit.client.1.smithi138.stdout:2/661: unlink d3/f74 0 2024-05-17T21:53:05.272 INFO:tasks.workunit.client.0.smithi107.stdout:8/757: creat dc/d21/daa/f10d x:0 0 0 2024-05-17T21:53:05.272 INFO:tasks.workunit.client.1.smithi138.stdout:4/658: creat d0/d1b/dd4/fed x:0 0 0 2024-05-17T21:53:05.276 INFO:tasks.workunit.client.1.smithi138.stdout:5/608: symlink d1/d53/d88/lc7 0 2024-05-17T21:53:05.278 INFO:tasks.workunit.client.0.smithi107.stdout:7/768: link d5/lbc d5/df/d11/d1c/d31/d39/l10d 0 2024-05-17T21:53:05.278 INFO:tasks.workunit.client.1.smithi138.stdout:5/609: fsync d1/d2c/d63/db5/d1b/fba 0 2024-05-17T21:53:05.279 INFO:tasks.workunit.client.0.smithi107.stdout:8/758: dwrite dc/d21/d27/d34/fe1 [0,4194304] 0 2024-05-17T21:53:05.283 INFO:tasks.workunit.client.1.smithi138.stdout:7/597: mkdir d0/d18/dbf 0 2024-05-17T21:53:05.284 INFO:tasks.workunit.client.1.smithi138.stdout:0/602: read d3/fa1 [3737665,97654] 0 2024-05-17T21:53:05.284 INFO:tasks.workunit.client.0.smithi107.stdout:9/818: unlink d0/dd/d2f/d4a/fd5 0 2024-05-17T21:53:05.285 INFO:tasks.workunit.client.1.smithi138.stdout:4/659: dwrite d0/d3/d17/d5e/f8e [0,4194304] 0 2024-05-17T21:53:05.286 INFO:tasks.workunit.client.1.smithi138.stdout:8/572: dread d0/dc/f2b [0,4194304] 0 2024-05-17T21:53:05.287 INFO:tasks.workunit.client.0.smithi107.stdout:6/728: rename d9/d58/d7b/f8f to d9/d58/d66/de2/fe5 0 2024-05-17T21:53:05.289 INFO:tasks.workunit.client.1.smithi138.stdout:6/632: fsync d7/d69/db5/dc8/d34/da6/f45 0 2024-05-17T21:53:05.293 INFO:tasks.workunit.client.0.smithi107.stdout:5/796: fdatasync d5/db/d16/d2a/f94 0 2024-05-17T21:53:05.293 INFO:tasks.workunit.client.0.smithi107.stdout:1/824: sync 2024-05-17T21:53:05.294 INFO:tasks.workunit.client.1.smithi138.stdout:5/610: dwrite d1/d53/d88/db9/d39/d4e/fb4 [0,4194304] 0 2024-05-17T21:53:05.294 INFO:tasks.workunit.client.0.smithi107.stdout:6/729: dwrite d9/d16/d4d/f52 [0,4194304] 0 2024-05-17T21:53:05.295 INFO:tasks.workunit.client.1.smithi138.stdout:4/660: truncate d0/d2f/d49/d60/d1f/fb9 1031411 0 2024-05-17T21:53:05.295 INFO:tasks.workunit.client.1.smithi138.stdout:6/633: read d7/d69/f9d [3407238,66506] 0 2024-05-17T21:53:05.300 INFO:tasks.workunit.client.1.smithi138.stdout:5/611: dread - d1/d53/d88/db9/fa7 zero size 2024-05-17T21:53:05.301 INFO:tasks.workunit.client.1.smithi138.stdout:7/598: dwrite d0/d18/d40/d41/d50/f53 [0,4194304] 0 2024-05-17T21:53:05.307 INFO:tasks.workunit.client.1.smithi138.stdout:4/661: dwrite d0/d3/db/fd3 [0,4194304] 0 2024-05-17T21:53:05.312 INFO:tasks.workunit.client.0.smithi107.stdout:7/769: mkdir d5/df/d11/d2a/d10e 0 2024-05-17T21:53:05.317 INFO:tasks.workunit.client.0.smithi107.stdout:2/639: write d0/d9/d11/d15/d99/fbb [127977,19199] 0 2024-05-17T21:53:05.317 INFO:tasks.workunit.client.0.smithi107.stdout:9/819: fdatasync d0/f43 0 2024-05-17T21:53:05.317 INFO:tasks.workunit.client.1.smithi138.stdout:8/573: creat d0/d71/fa5 x:0 0 0 2024-05-17T21:53:05.319 INFO:tasks.workunit.client.1.smithi138.stdout:5/612: dwrite d1/d60/d2f/d5e/f79 [0,4194304] 0 2024-05-17T21:53:05.322 INFO:tasks.workunit.client.1.smithi138.stdout:3/724: dwrite d1/d4f/d87/f88 [0,4194304] 0 2024-05-17T21:53:05.323 INFO:tasks.workunit.client.0.smithi107.stdout:3/821: getdents d6/d2a/d2d/d37/d108 0 2024-05-17T21:53:05.324 INFO:tasks.workunit.client.1.smithi138.stdout:6/634: mknod d7/d69/db5/dc8/d34/da6/cd4 0 2024-05-17T21:53:05.325 INFO:tasks.workunit.client.1.smithi138.stdout:0/603: read d3/d10/d6d/f75 [3030850,78981] 0 2024-05-17T21:53:05.325 INFO:tasks.workunit.client.0.smithi107.stdout:0/899: mkdir d3/d24/d6f/d114/d128 0 2024-05-17T21:53:05.326 INFO:tasks.workunit.client.1.smithi138.stdout:6/635: chown d7/d69/db5/dc8/d43/d3f/db6/fc5 44252 1 2024-05-17T21:53:05.327 INFO:tasks.workunit.client.0.smithi107.stdout:4/808: creat d3/d6/d26/d3b/d91/df6/d111/f112 x:0 0 0 2024-05-17T21:53:05.328 INFO:tasks.workunit.client.1.smithi138.stdout:0/604: fdatasync d3/d10/d6d/d9f/faf 0 2024-05-17T21:53:05.329 INFO:tasks.workunit.client.1.smithi138.stdout:7/599: creat d0/d4c/fc0 x:0 0 0 2024-05-17T21:53:05.330 INFO:tasks.workunit.client.0.smithi107.stdout:7/770: symlink d5/df/d11/d1c/d31/d39/d92/d9a/l10f 0 2024-05-17T21:53:05.330 INFO:tasks.workunit.client.1.smithi138.stdout:1/597: link d4/df/l23 d4/df/d3a/lbb 0 2024-05-17T21:53:05.332 INFO:tasks.workunit.client.1.smithi138.stdout:7/600: chown d0/d39/f85 5493 1 2024-05-17T21:53:05.332 INFO:tasks.workunit.client.0.smithi107.stdout:9/820: symlink d0/dd/d3d/d9d/l10d 0 2024-05-17T21:53:05.334 INFO:tasks.workunit.client.1.smithi138.stdout:9/548: getdents d6/d7/d16 0 2024-05-17T21:53:05.336 INFO:tasks.workunit.client.0.smithi107.stdout:1/825: creat d4/df/d14/d2e/dd3/df6/f106 x:0 0 0 2024-05-17T21:53:05.337 INFO:tasks.workunit.client.0.smithi107.stdout:6/730: mkdir d9/d16/d22/de6 0 2024-05-17T21:53:05.340 INFO:tasks.workunit.client.0.smithi107.stdout:3/822: rmdir d6/d1a/d57 39 2024-05-17T21:53:05.340 INFO:tasks.workunit.client.0.smithi107.stdout:3/823: fsync d6/d1a/fde 0 2024-05-17T21:53:05.342 INFO:tasks.workunit.client.0.smithi107.stdout:0/900: creat d3/d75/d9e/d103/f129 x:0 0 0 2024-05-17T21:53:05.343 INFO:tasks.workunit.client.0.smithi107.stdout:3/824: dread d6/d8/fc0 [0,4194304] 0 2024-05-17T21:53:05.343 INFO:tasks.workunit.client.0.smithi107.stdout:3/825: read - d6/d1a/fde zero size 2024-05-17T21:53:05.345 INFO:tasks.workunit.client.0.smithi107.stdout:3/826: write d6/d8/d10/d25/f99 [4914058,90957] 0 2024-05-17T21:53:05.346 INFO:tasks.workunit.client.0.smithi107.stdout:4/809: unlink d3/db/f4d 0 2024-05-17T21:53:05.350 INFO:tasks.workunit.client.1.smithi138.stdout:3/725: mknod d1/d3/d49/ddb/cfa 0 2024-05-17T21:53:05.351 INFO:tasks.workunit.client.1.smithi138.stdout:3/726: chown d1/d2e/fa1 1838777 1 2024-05-17T21:53:05.355 INFO:tasks.workunit.client.0.smithi107.stdout:1/826: dread d4/df/d1a/d26/d72/fb4 [0,4194304] 0 2024-05-17T21:53:05.356 INFO:tasks.workunit.client.0.smithi107.stdout:1/827: write d4/df/d1a/d26/d72/d85/fd7 [3205867,113548] 0 2024-05-17T21:53:05.358 INFO:tasks.workunit.client.1.smithi138.stdout:5/613: dread d1/d53/f6e [0,4194304] 0 2024-05-17T21:53:05.361 INFO:tasks.workunit.client.1.smithi138.stdout:5/614: chown d1/d2c/d63/db5/fe 17842 1 2024-05-17T21:53:05.361 INFO:tasks.workunit.client.1.smithi138.stdout:3/727: dwrite d1/d3/d16/d3e/d71/fd6 [0,4194304] 0 2024-05-17T21:53:05.362 INFO:tasks.workunit.client.0.smithi107.stdout:6/731: unlink d9/d16/d4d/db8/dc9/dd0/f2f 0 2024-05-17T21:53:05.363 INFO:tasks.workunit.client.1.smithi138.stdout:5/615: dread - d1/d2c/d63/db5/d1b/dc5/fbb zero size 2024-05-17T21:53:05.366 INFO:tasks.workunit.client.1.smithi138.stdout:7/601: dread d0/d18/d40/d41/d50/f59 [0,4194304] 0 2024-05-17T21:53:05.367 INFO:tasks.workunit.client.0.smithi107.stdout:0/901: rmdir d3/d58/d88 39 2024-05-17T21:53:05.367 INFO:tasks.workunit.client.1.smithi138.stdout:9/549: fdatasync d6/d7/d16/d29/d48/d3b/faf 0 2024-05-17T21:53:05.367 INFO:tasks.workunit.client.1.smithi138.stdout:7/602: fsync d0/d18/d40/f82 0 2024-05-17T21:53:05.369 INFO:tasks.workunit.client.1.smithi138.stdout:7/603: truncate d0/d9d/fab 4483827 0 2024-05-17T21:53:05.370 INFO:tasks.workunit.client.1.smithi138.stdout:2/662: link d3/d21/d2f/d3e/d63/f80 d3/d21/d2f/dd4/fdb 0 2024-05-17T21:53:05.372 INFO:tasks.workunit.client.1.smithi138.stdout:9/550: truncate d6/d7/d16/d29/d48/d6e/d74/d75/d76/d8e/fa7 525179 0 2024-05-17T21:53:05.374 INFO:tasks.workunit.client.0.smithi107.stdout:0/902: dwrite d3/d39/fb0 [0,4194304] 0 2024-05-17T21:53:05.375 INFO:tasks.workunit.client.0.smithi107.stdout:3/827: rmdir d6/d8/d10/d35/d91/da6 39 2024-05-17T21:53:05.380 INFO:tasks.workunit.client.0.smithi107.stdout:9/821: mkdir d0/dd/d21/d10e 0 2024-05-17T21:53:05.382 INFO:tasks.workunit.client.0.smithi107.stdout:9/822: write d0/dd/d21/d29/dd4/fc5 [3351031,39400] 0 2024-05-17T21:53:05.385 INFO:tasks.workunit.client.1.smithi138.stdout:0/605: dread d3/f21 [0,4194304] 0 2024-05-17T21:53:05.385 INFO:tasks.workunit.client.0.smithi107.stdout:5/797: getdents d5/db/d16/d2a 0 2024-05-17T21:53:05.386 INFO:tasks.workunit.client.0.smithi107.stdout:5/798: truncate d5/db/fcf 325339 0 2024-05-17T21:53:05.386 INFO:tasks.workunit.client.1.smithi138.stdout:5/616: write d1/d53/d88/db9/d39/fa6 [1579099,19669] 0 2024-05-17T21:53:05.387 INFO:tasks.workunit.client.0.smithi107.stdout:2/640: truncate d0/d9/d11/d15/f57 910478 0 2024-05-17T21:53:05.387 INFO:tasks.workunit.client.1.smithi138.stdout:5/617: chown d1/d2c/d63/db5 46 1 2024-05-17T21:53:05.388 INFO:tasks.workunit.client.1.smithi138.stdout:5/618: fdatasync d1/d2c/d63/db5/d1b/f94 0 2024-05-17T21:53:05.390 INFO:tasks.workunit.client.1.smithi138.stdout:8/574: creat d0/d11/fa6 x:0 0 0 2024-05-17T21:53:05.390 INFO:tasks.workunit.client.1.smithi138.stdout:3/728: dread d1/d3/f11 [0,4194304] 0 2024-05-17T21:53:05.391 INFO:tasks.workunit.client.1.smithi138.stdout:3/729: readlink d1/d2e/l39 0 2024-05-17T21:53:05.395 INFO:tasks.workunit.client.1.smithi138.stdout:7/604: creat d0/d18/d64/d7c/da2/fc1 x:0 0 0 2024-05-17T21:53:05.398 INFO:tasks.workunit.client.0.smithi107.stdout:9/823: dread d0/dd/d1f/f40 [0,4194304] 0 2024-05-17T21:53:05.399 INFO:tasks.workunit.client.0.smithi107.stdout:9/824: stat d0/dd/d21/d29/d3e/f91 0 2024-05-17T21:53:05.402 INFO:tasks.workunit.client.1.smithi138.stdout:9/551: mknod d6/d7/d16/d29/d6c/d83/cb0 0 2024-05-17T21:53:05.402 INFO:tasks.workunit.client.0.smithi107.stdout:9/825: fdatasync d0/dd/d21/d29/dd4/fa0 0 2024-05-17T21:53:05.403 INFO:tasks.workunit.client.0.smithi107.stdout:4/810: dwrite d3/db/d35/d5e/dff/f71 [0,4194304] 0 2024-05-17T21:53:05.405 INFO:tasks.workunit.client.1.smithi138.stdout:4/662: getdents d0/d3/d17/d5e 0 2024-05-17T21:53:05.407 INFO:tasks.workunit.client.0.smithi107.stdout:7/771: rename d5/df/d11/c88 to d5/df/d11/d1d/c110 0 2024-05-17T21:53:05.408 INFO:tasks.workunit.client.1.smithi138.stdout:0/606: creat d3/d10/d6d/d9f/da8/daa/fcb x:0 0 0 2024-05-17T21:53:05.409 INFO:tasks.workunit.client.0.smithi107.stdout:9/826: dwrite d0/dd/d69/f83 [0,4194304] 0 2024-05-17T21:53:05.414 INFO:tasks.workunit.client.1.smithi138.stdout:3/730: rmdir d1/dc0/ddd 39 2024-05-17T21:53:05.414 INFO:tasks.workunit.client.0.smithi107.stdout:6/732: dwrite d9/d16/d4d/f5b [0,4194304] 0 2024-05-17T21:53:05.416 INFO:tasks.workunit.client.0.smithi107.stdout:6/733: chown d9/d16/d4d/d85/d88/f89 28489852 1 2024-05-17T21:53:05.416 INFO:tasks.workunit.client.0.smithi107.stdout:5/799: creat d5/db/d16/d2a/d73/f10a x:0 0 0 2024-05-17T21:53:05.417 INFO:tasks.workunit.client.0.smithi107.stdout:3/828: write d6/fc1 [45276,105423] 0 2024-05-17T21:53:05.418 INFO:tasks.workunit.client.0.smithi107.stdout:8/759: sync 2024-05-17T21:53:05.423 INFO:tasks.workunit.client.1.smithi138.stdout:2/663: fdatasync d3/d21/d2f/f39 0 2024-05-17T21:53:05.426 INFO:tasks.workunit.client.0.smithi107.stdout:2/641: mknod d0/d9/d11/d18/dad/ccd 0 2024-05-17T21:53:05.426 INFO:tasks.workunit.client.1.smithi138.stdout:6/636: link d7/fc d7/d69/db5/dc8/d43/d5f/fd5 0 2024-05-17T21:53:05.428 INFO:tasks.workunit.client.1.smithi138.stdout:4/663: mknod d0/d2f/d49/d60/d1f/d7e/cee 0 2024-05-17T21:53:05.430 INFO:tasks.workunit.client.1.smithi138.stdout:5/619: symlink d1/d2c/lc8 0 2024-05-17T21:53:05.434 INFO:tasks.workunit.client.0.smithi107.stdout:4/811: creat d3/db/d1e/db5/f113 x:0 0 0 2024-05-17T21:53:05.435 INFO:tasks.workunit.client.1.smithi138.stdout:4/664: dwrite d0/d2f/d49/d60/d1f/d7e/f83 [4194304,4194304] 0 2024-05-17T21:53:05.438 INFO:tasks.workunit.client.0.smithi107.stdout:1/828: write d4/de/f19 [2430641,24299] 0 2024-05-17T21:53:05.439 INFO:tasks.workunit.client.0.smithi107.stdout:9/827: mknod d0/dd/d21/d29/d3e/dfe/c10f 0 2024-05-17T21:53:05.441 INFO:tasks.workunit.client.1.smithi138.stdout:2/664: creat d3/da0/daa/dce/fdc x:0 0 0 2024-05-17T21:53:05.449 INFO:tasks.workunit.client.1.smithi138.stdout:3/731: write d1/d3/d49/ddb/f85 [2180376,107225] 0 2024-05-17T21:53:05.451 INFO:tasks.workunit.client.0.smithi107.stdout:6/734: creat d9/d16/d22/d5f/fe7 x:0 0 0 2024-05-17T21:53:05.452 INFO:tasks.workunit.client.1.smithi138.stdout:8/575: creat d0/dc/fa7 x:0 0 0 2024-05-17T21:53:05.453 INFO:tasks.workunit.client.0.smithi107.stdout:7/772: write d5/df/d11/d1c/f75 [442976,128329] 0 2024-05-17T21:53:05.455 INFO:tasks.workunit.client.1.smithi138.stdout:5/620: mkdir d1/d53/d88/dc9 0 2024-05-17T21:53:05.456 INFO:tasks.workunit.client.0.smithi107.stdout:3/829: creat d6/d8/d10/d25/d80/f10a x:0 0 0 2024-05-17T21:53:05.457 INFO:tasks.workunit.client.0.smithi107.stdout:8/760: creat dc/d21/d97/db7/f10e x:0 0 0 2024-05-17T21:53:05.459 INFO:tasks.workunit.client.1.smithi138.stdout:0/607: write d3/f83 [3871103,86397] 0 2024-05-17T21:53:05.460 INFO:tasks.workunit.client.1.smithi138.stdout:7/605: truncate d0/d18/f1d 2952490 0 2024-05-17T21:53:05.463 INFO:tasks.workunit.client.0.smithi107.stdout:0/903: creat d3/dc/d22/d23/d97/f12a x:0 0 0 2024-05-17T21:53:05.463 INFO:tasks.workunit.client.0.smithi107.stdout:3/830: dwrite d6/d2a/d2d/d69/dad/fdf [0,4194304] 0 2024-05-17T21:53:05.463 INFO:tasks.workunit.client.1.smithi138.stdout:7/606: dread d0/d18/d40/d41/d50/f74 [0,4194304] 0 2024-05-17T21:53:05.468 INFO:tasks.workunit.client.0.smithi107.stdout:4/812: rename d3/db/d35/d55/d8a/fbc to d3/d4a/d47/da3/f114 0 2024-05-17T21:53:05.469 INFO:tasks.workunit.client.0.smithi107.stdout:4/813: write d3/fe4 [969104,94879] 0 2024-05-17T21:53:05.469 INFO:tasks.workunit.client.0.smithi107.stdout:4/814: chown d3/d6/d26/d36/d37/f5f 106488 1 2024-05-17T21:53:05.470 INFO:tasks.workunit.client.1.smithi138.stdout:6/637: creat d7/da0/dbe/fd6 x:0 0 0 2024-05-17T21:53:05.470 INFO:tasks.workunit.client.1.smithi138.stdout:7/607: dwrite d0/d18/d64/d7c/f9f [0,4194304] 0 2024-05-17T21:53:05.471 INFO:tasks.workunit.client.1.smithi138.stdout:6/638: chown d7/d69/db5/dc8/d43/d5f/d6a/d77 14554130 1 2024-05-17T21:53:05.476 INFO:tasks.workunit.client.0.smithi107.stdout:1/829: unlink d4/df/d1a/f28 0 2024-05-17T21:53:05.476 INFO:tasks.workunit.client.0.smithi107.stdout:1/830: chown d4/df/d14/d2e/c7d 1157 1 2024-05-17T21:53:05.477 INFO:tasks.workunit.client.0.smithi107.stdout:1/831: truncate d4/df/fff 890703 0 2024-05-17T21:53:05.478 INFO:tasks.workunit.client.1.smithi138.stdout:3/732: mknod d1/d3/d16/d5d/cfb 0 2024-05-17T21:53:05.479 INFO:tasks.workunit.client.0.smithi107.stdout:9/828: creat d0/dd/d2f/d87/d89/de0/f110 x:0 0 0 2024-05-17T21:53:05.481 INFO:tasks.workunit.client.1.smithi138.stdout:8/576: creat d0/dc/d1a/d33/d4a/fa8 x:0 0 0 2024-05-17T21:53:05.482 INFO:tasks.workunit.client.1.smithi138.stdout:5/621: mkdir d1/d53/d88/db9/d39/d4e/d58/d5b/d65/dca 0 2024-05-17T21:53:05.483 INFO:tasks.workunit.client.1.smithi138.stdout:2/665: dwrite d3/d21/d2f/d3e/d63/fcd [0,4194304] 0 2024-05-17T21:53:05.486 INFO:tasks.workunit.client.1.smithi138.stdout:5/622: truncate d1/d2c/d63/db5/d1b/f94 585528 0 2024-05-17T21:53:05.489 INFO:tasks.workunit.client.1.smithi138.stdout:4/665: symlink d0/d2f/lef 0 2024-05-17T21:53:05.491 INFO:tasks.workunit.client.0.smithi107.stdout:0/904: mkdir d3/dc/d22/d23/d97/d77/d91/d12b 0 2024-05-17T21:53:05.492 INFO:tasks.workunit.client.1.smithi138.stdout:4/666: read d0/d3/d17/d5e/fb4 [2780677,31105] 0 2024-05-17T21:53:05.492 INFO:tasks.workunit.client.1.smithi138.stdout:4/667: chown d0/d1b/d2b/d55/c8c 1816666059 1 2024-05-17T21:53:05.494 INFO:tasks.workunit.client.1.smithi138.stdout:7/608: symlink d0/d4c/lc2 0 2024-05-17T21:53:05.495 INFO:tasks.workunit.client.0.smithi107.stdout:4/815: symlink d3/d6/d26/d36/l115 0 2024-05-17T21:53:05.499 INFO:tasks.workunit.client.0.smithi107.stdout:1/832: read d4/de/f1e [1833214,128969] 0 2024-05-17T21:53:05.501 INFO:tasks.workunit.client.1.smithi138.stdout:3/733: dread d1/d3/d16/d3e/f4a [0,4194304] 0 2024-05-17T21:53:05.502 INFO:tasks.workunit.client.1.smithi138.stdout:7/609: dwrite d0/d18/d40/d5f/f8d [0,4194304] 0 2024-05-17T21:53:05.504 INFO:tasks.workunit.client.0.smithi107.stdout:6/735: mkdir d9/d16/d4d/d85/de8 0 2024-05-17T21:53:05.507 INFO:tasks.workunit.client.1.smithi138.stdout:8/577: mknod d0/d71/ca9 0 2024-05-17T21:53:05.508 INFO:tasks.workunit.client.0.smithi107.stdout:8/761: mknod dc/d21/d97/d63/c10f 0 2024-05-17T21:53:05.510 INFO:tasks.workunit.client.0.smithi107.stdout:2/642: creat d0/d9/d11/d18/fce x:0 0 0 2024-05-17T21:53:05.511 INFO:tasks.workunit.client.0.smithi107.stdout:3/831: write d6/d2a/d2d/d69/f6d [6040344,98489] 0 2024-05-17T21:53:05.512 INFO:tasks.workunit.client.0.smithi107.stdout:5/800: dread d5/db/d16/d54/d59/d65/f27 [0,4194304] 0 2024-05-17T21:53:05.513 INFO:tasks.workunit.client.0.smithi107.stdout:5/801: read - d5/db/dc/d1e/d6e/ffa zero size 2024-05-17T21:53:05.514 INFO:tasks.workunit.client.0.smithi107.stdout:7/773: rename d5/df/d11/d1c/d31/d39/f85 to d5/df/d78/f111 0 2024-05-17T21:53:05.518 INFO:tasks.workunit.client.0.smithi107.stdout:3/832: dwrite d6/d8/d10/d25/d80/fe2 [0,4194304] 0 2024-05-17T21:53:05.518 INFO:tasks.workunit.client.0.smithi107.stdout:9/829: mkdir d0/dd/d21/d29/d3e/d94/d111 0 2024-05-17T21:53:05.520 INFO:tasks.workunit.client.1.smithi138.stdout:6/639: dwrite d7/d69/db5/dc8/d34/da6/f27 [0,4194304] 0 2024-05-17T21:53:05.523 INFO:tasks.workunit.client.0.smithi107.stdout:8/762: creat dc/ddb/f110 x:0 0 0 2024-05-17T21:53:05.525 INFO:tasks.workunit.client.0.smithi107.stdout:3/833: dwrite d6/d1a/f4e [0,4194304] 0 2024-05-17T21:53:05.527 INFO:tasks.workunit.client.0.smithi107.stdout:2/643: creat d0/d93/fcf x:0 0 0 2024-05-17T21:53:05.527 INFO:tasks.workunit.client.0.smithi107.stdout:2/644: readlink d0/d9/d11/d27/l56 0 2024-05-17T21:53:05.528 INFO:tasks.workunit.client.0.smithi107.stdout:5/802: fdatasync d5/db/f25 0 2024-05-17T21:53:05.528 INFO:tasks.workunit.client.0.smithi107.stdout:5/803: chown d5/db/dc/d1e/d6e 581867 1 2024-05-17T21:53:05.532 INFO:tasks.workunit.client.1.smithi138.stdout:3/734: creat d1/d62/d9b/ffc x:0 0 0 2024-05-17T21:53:05.536 INFO:tasks.workunit.client.1.smithi138.stdout:4/668: write d0/d3/db/f69 [134177,35387] 0 2024-05-17T21:53:05.537 INFO:tasks.workunit.client.1.smithi138.stdout:8/578: symlink d0/dc/d3b/laa 0 2024-05-17T21:53:05.541 INFO:tasks.workunit.client.1.smithi138.stdout:2/666: symlink d3/d21/d2f/dd4/ldd 0 2024-05-17T21:53:05.544 INFO:tasks.workunit.client.1.smithi138.stdout:0/608: mknod d3/d3d/da2/ccc 0 2024-05-17T21:53:05.545 INFO:tasks.workunit.client.1.smithi138.stdout:5/623: truncate d1/d53/d88/db9/f26 583912 0 2024-05-17T21:53:05.548 INFO:tasks.workunit.client.1.smithi138.stdout:2/667: dwrite f2 [0,4194304] 0 2024-05-17T21:53:05.550 INFO:tasks.workunit.client.1.smithi138.stdout:6/640: mkdir d7/d69/db5/dc8/d43/d3f/dd7 0 2024-05-17T21:53:05.554 INFO:tasks.workunit.client.1.smithi138.stdout:3/735: rename d1/d4f/d80/fa7 to d1/d2e/ffd 0 2024-05-17T21:53:05.555 INFO:tasks.workunit.client.1.smithi138.stdout:3/736: write d1/fc3 [456008,128223] 0 2024-05-17T21:53:05.562 INFO:tasks.workunit.client.1.smithi138.stdout:4/669: unlink d0/d2f/d49/d60/d1f/fb9 0 2024-05-17T21:53:05.564 INFO:tasks.workunit.client.1.smithi138.stdout:5/624: creat d1/d60/fcb x:0 0 0 2024-05-17T21:53:05.564 INFO:tasks.workunit.client.1.smithi138.stdout:0/609: truncate d3/d10/f14 1399762 0 2024-05-17T21:53:05.565 INFO:tasks.workunit.client.1.smithi138.stdout:0/610: chown d3/d3d/d51/dba 40 1 2024-05-17T21:53:05.565 INFO:tasks.workunit.client.1.smithi138.stdout:2/668: fsync d3/f66 0 2024-05-17T21:53:05.567 INFO:tasks.workunit.client.1.smithi138.stdout:0/611: write d3/d10/f25 [3284534,49777] 0 2024-05-17T21:53:05.568 INFO:tasks.workunit.client.1.smithi138.stdout:6/641: creat d7/d69/db5/dc8/d43/d3f/db6/fd8 x:0 0 0 2024-05-17T21:53:05.569 INFO:tasks.workunit.client.0.smithi107.stdout:7/774: mkdir d5/df/d11/d1c/d2f/d65/da7/d112 0 2024-05-17T21:53:05.569 INFO:tasks.workunit.client.0.smithi107.stdout:8/763: creat dc/d21/daa/dc5/f111 x:0 0 0 2024-05-17T21:53:05.570 INFO:tasks.workunit.client.1.smithi138.stdout:3/737: symlink d1/d3/d16/d3e/lfe 0 2024-05-17T21:53:05.570 INFO:tasks.workunit.client.0.smithi107.stdout:6/736: truncate d9/d16/d4d/db8/dc9/dd0/f63 1397664 0 2024-05-17T21:53:05.570 INFO:tasks.workunit.client.0.smithi107.stdout:5/804: write d5/d75/fae [381041,35990] 0 2024-05-17T21:53:05.571 INFO:tasks.workunit.client.1.smithi138.stdout:7/610: truncate d0/d18/d40/f44 6125431 0 2024-05-17T21:53:05.571 INFO:tasks.workunit.client.0.smithi107.stdout:6/737: read d9/d16/d22/d5f/d6f/fa7 [3455809,9113] 0 2024-05-17T21:53:05.572 INFO:tasks.workunit.client.1.smithi138.stdout:8/579: mknod d0/dc/d1a/d33/cab 0 2024-05-17T21:53:05.573 INFO:tasks.workunit.client.1.smithi138.stdout:7/611: write d0/d18/d40/d41/d51/d66/d9e/fa1 [181104,106178] 0 2024-05-17T21:53:05.575 INFO:tasks.workunit.client.1.smithi138.stdout:4/670: creat d0/d3/d17/d5e/d7f/ff0 x:0 0 0 2024-05-17T21:53:05.577 INFO:tasks.workunit.client.0.smithi107.stdout:6/738: dwrite d9/d16/f64 [0,4194304] 0 2024-05-17T21:53:05.578 INFO:tasks.workunit.client.0.smithi107.stdout:6/739: truncate f5 5639713 0 2024-05-17T21:53:05.578 INFO:tasks.workunit.client.1.smithi138.stdout:4/671: write d0/d1b/d88/f8b [5179343,119028] 0 2024-05-17T21:53:05.579 INFO:tasks.workunit.client.0.smithi107.stdout:6/740: write d9/d16/f7f [1812454,91730] 0 2024-05-17T21:53:05.582 INFO:tasks.workunit.client.1.smithi138.stdout:4/672: write d0/d3/d17/d21/d78/fcd [5537,79661] 0 2024-05-17T21:53:05.583 INFO:tasks.workunit.client.1.smithi138.stdout:7/612: dwrite d0/d18/d40/d9a/fb8 [0,4194304] 0 2024-05-17T21:53:05.586 INFO:tasks.workunit.client.0.smithi107.stdout:2/645: unlink d0/d9/l59 0 2024-05-17T21:53:05.588 INFO:tasks.workunit.client.1.smithi138.stdout:2/669: dread d3/d21/d2f/d3e/d63/da3/fb6 [4194304,4194304] 0 2024-05-17T21:53:05.592 INFO:tasks.workunit.client.0.smithi107.stdout:8/764: symlink dc/dd5/l112 0 2024-05-17T21:53:05.594 INFO:tasks.workunit.client.0.smithi107.stdout:2/646: dwrite d0/d9/d1c/d3e/d4f/f84 [0,4194304] 0 2024-05-17T21:53:05.597 INFO:tasks.workunit.client.1.smithi138.stdout:3/738: creat d1/d3/d49/fff x:0 0 0 2024-05-17T21:53:05.598 INFO:tasks.workunit.client.1.smithi138.stdout:1/598: sync 2024-05-17T21:53:05.598 INFO:tasks.workunit.client.1.smithi138.stdout:9/552: sync 2024-05-17T21:53:05.600 INFO:tasks.workunit.client.0.smithi107.stdout:1/833: rename d4/df/d69 to d4/df/d14/d2e/d107 0 2024-05-17T21:53:05.603 INFO:tasks.workunit.client.0.smithi107.stdout:4/816: getdents d3/d6 0 2024-05-17T21:53:05.606 INFO:tasks.workunit.client.1.smithi138.stdout:5/625: mkdir d1/d53/d88/db9/dcc 0 2024-05-17T21:53:05.608 INFO:tasks.workunit.client.1.smithi138.stdout:5/626: chown d1/d53/d88/db9/d39/d4e/d58/d5b/d65/c89 8337833 1 2024-05-17T21:53:05.608 INFO:tasks.workunit.client.0.smithi107.stdout:6/741: mkdir d9/d58/d66/dd5/de9 0 2024-05-17T21:53:05.611 INFO:tasks.workunit.client.0.smithi107.stdout:3/834: creat d6/d2a/d2d/f10b x:0 0 0 2024-05-17T21:53:05.615 INFO:tasks.workunit.client.0.smithi107.stdout:6/742: dwrite d9/d16/d4d/db8/dc9/dd0/f9a [0,4194304] 0 2024-05-17T21:53:05.617 INFO:tasks.workunit.client.1.smithi138.stdout:7/613: mknod d0/d18/d9c/cc3 0 2024-05-17T21:53:05.620 INFO:tasks.workunit.client.1.smithi138.stdout:2/670: creat d3/d21/d2f/d3e/d55/d99/dd2/fde x:0 0 0 2024-05-17T21:53:05.621 INFO:tasks.workunit.client.0.smithi107.stdout:5/805: truncate d5/db/d16/d2d/d40/fed 3343389 0 2024-05-17T21:53:05.623 INFO:tasks.workunit.client.0.smithi107.stdout:1/834: rename d4/df/d1a/cd0 to d4/df/d14/d2e/dd3/c108 0 2024-05-17T21:53:05.630 INFO:tasks.workunit.client.0.smithi107.stdout:3/835: unlink d6/d8/d10/ff3 0 2024-05-17T21:53:05.638 INFO:tasks.workunit.client.0.smithi107.stdout:5/806: creat d5/db/d16/d2d/d40/dc2/f10b x:0 0 0 2024-05-17T21:53:05.641 INFO:tasks.workunit.client.1.smithi138.stdout:5/627: mknod d1/d53/d88/db9/d39/d62/ccd 0 2024-05-17T21:53:05.642 INFO:tasks.workunit.client.1.smithi138.stdout:5/628: write d1/d2c/d63/db5/d1b/dc5/fbb [664261,121755] 0 2024-05-17T21:53:05.643 INFO:tasks.workunit.client.0.smithi107.stdout:1/835: mknod d4/df/d1a/d26/d7a/c109 0 2024-05-17T21:53:05.645 INFO:tasks.workunit.client.0.smithi107.stdout:4/817: write d3/d4a/d47/d60/d3c/fb0 [325039,90841] 0 2024-05-17T21:53:05.646 INFO:tasks.workunit.client.1.smithi138.stdout:6/642: truncate d7/d8/d78/f7f 4015903 0 2024-05-17T21:53:05.649 INFO:tasks.workunit.client.1.smithi138.stdout:0/612: link d3/d10/d38/d3e/fb7 d3/d10/d38/d70/d80/d94/fcd 0 2024-05-17T21:53:05.651 INFO:tasks.workunit.client.0.smithi107.stdout:3/836: creat d6/d8/d10/d35/f10c x:0 0 0 2024-05-17T21:53:05.652 INFO:tasks.workunit.client.1.smithi138.stdout:9/553: dread d6/d7/d16/d29/d48/d6e/d74/d75/d76/f7f [0,4194304] 0 2024-05-17T21:53:05.653 INFO:tasks.workunit.client.0.smithi107.stdout:6/743: mknod d9/d16/d4d/d85/de8/cea 0 2024-05-17T21:53:05.654 INFO:tasks.workunit.client.0.smithi107.stdout:2/647: symlink d0/d9/d11/d18/d9e/ld0 0 2024-05-17T21:53:05.658 INFO:tasks.workunit.client.0.smithi107.stdout:1/836: rename d4/df/d4e/l62 to d4/df/d4e/d59/l10a 0 2024-05-17T21:53:05.659 INFO:tasks.workunit.client.1.smithi138.stdout:8/580: creat d0/dc/d1a/d33/d4a/d68/fac x:0 0 0 2024-05-17T21:53:05.660 INFO:tasks.workunit.client.0.smithi107.stdout:4/818: creat d3/d6/d26/d36/f116 x:0 0 0 2024-05-17T21:53:05.661 INFO:tasks.workunit.client.1.smithi138.stdout:5/629: symlink d1/d53/d88/lce 0 2024-05-17T21:53:05.662 INFO:tasks.workunit.client.0.smithi107.stdout:4/819: dread - d3/d6/d26/d3b/d91/df6/d111/f112 zero size 2024-05-17T21:53:05.665 INFO:tasks.workunit.client.0.smithi107.stdout:8/765: getdents dc/d21/d27 0 2024-05-17T21:53:05.666 INFO:tasks.workunit.client.0.smithi107.stdout:8/766: chown dc/d21/f80 1324368852 1 2024-05-17T21:53:05.667 INFO:tasks.workunit.client.1.smithi138.stdout:3/739: truncate d1/f65 3235544 0 2024-05-17T21:53:05.668 INFO:tasks.workunit.client.1.smithi138.stdout:3/740: write d1/d4f/fd5 [1517591,54384] 0 2024-05-17T21:53:05.668 INFO:tasks.workunit.client.1.smithi138.stdout:0/613: mknod d3/d10/d38/d3e/d59/cce 0 2024-05-17T21:53:05.669 INFO:tasks.workunit.client.0.smithi107.stdout:0/905: sync 2024-05-17T21:53:05.669 INFO:tasks.workunit.client.0.smithi107.stdout:0/906: readlink d3/d75/lf7 0 2024-05-17T21:53:05.670 INFO:tasks.workunit.client.1.smithi138.stdout:0/614: stat d3/d10/d6d/d81/l87 0 2024-05-17T21:53:05.673 INFO:tasks.workunit.client.1.smithi138.stdout:1/599: link d4/dd/d3b/f9e d4/db/d29/d5d/fbc 0 2024-05-17T21:53:05.674 INFO:tasks.workunit.client.1.smithi138.stdout:1/600: fdatasync d4/db/d29/fa4 0 2024-05-17T21:53:05.675 INFO:tasks.workunit.client.0.smithi107.stdout:6/744: dwrite d9/d16/d22/d5f/fdc [0,4194304] 0 2024-05-17T21:53:05.676 INFO:tasks.workunit.client.0.smithi107.stdout:1/837: rename d4/df/d14/d2e/d43/ff8 to d4/df/d14/d2e/ddf/dee/f10b 0 2024-05-17T21:53:05.677 INFO:tasks.workunit.client.0.smithi107.stdout:5/807: dwrite d5/f1b [0,4194304] 0 2024-05-17T21:53:05.677 INFO:tasks.workunit.client.1.smithi138.stdout:8/581: creat d0/dc/d8d/fad x:0 0 0 2024-05-17T21:53:05.677 INFO:tasks.workunit.client.0.smithi107.stdout:6/745: stat d9/d16/d22/f72 0 2024-05-17T21:53:05.679 INFO:tasks.workunit.client.0.smithi107.stdout:4/820: creat d3/db/d35/d55/f117 x:0 0 0 2024-05-17T21:53:05.679 INFO:tasks.workunit.client.0.smithi107.stdout:1/838: dread - d4/df/d1a/d41/db2/dc8/ff2 zero size 2024-05-17T21:53:05.679 INFO:tasks.workunit.client.1.smithi138.stdout:5/630: chown d1/d60 72862 1 2024-05-17T21:53:05.681 INFO:tasks.workunit.client.0.smithi107.stdout:8/767: dread dc/d21/daa/db4/fe0 [0,4194304] 0 2024-05-17T21:53:05.682 INFO:tasks.workunit.client.0.smithi107.stdout:8/768: fdatasync dc/d21/d69/ffc 0 2024-05-17T21:53:05.682 INFO:tasks.workunit.client.0.smithi107.stdout:1/839: chown d4/df/d1a/d41/c65 15705 1 2024-05-17T21:53:05.683 INFO:tasks.workunit.client.0.smithi107.stdout:8/769: stat dc/d21/fd7 0 2024-05-17T21:53:05.684 INFO:tasks.workunit.client.0.smithi107.stdout:1/840: dread - d4/df/d14/d2e/dd3/fe4 zero size 2024-05-17T21:53:05.687 INFO:tasks.workunit.client.0.smithi107.stdout:1/841: write d4/d6a/fe9 [261306,37208] 0 2024-05-17T21:53:05.687 INFO:tasks.workunit.client.0.smithi107.stdout:1/842: chown d4/df/d4e/d59/lae 66233788 1 2024-05-17T21:53:05.689 INFO:tasks.workunit.client.0.smithi107.stdout:6/746: creat d9/d81/feb x:0 0 0 2024-05-17T21:53:05.690 INFO:tasks.workunit.client.0.smithi107.stdout:5/808: creat d5/db/d16/d2d/d40/d9f/f10c x:0 0 0 2024-05-17T21:53:05.690 INFO:tasks.workunit.client.0.smithi107.stdout:6/747: write d9/d81/fcd [1939628,21795] 0 2024-05-17T21:53:05.692 INFO:tasks.workunit.client.0.smithi107.stdout:3/837: creat d6/d2a/f10d x:0 0 0 2024-05-17T21:53:05.692 INFO:tasks.workunit.client.0.smithi107.stdout:3/838: readlink d6/d1a/l6e 0 2024-05-17T21:53:05.693 INFO:tasks.workunit.client.0.smithi107.stdout:4/821: symlink d3/d6/d26/d36/d8b/l118 0 2024-05-17T21:53:05.703 INFO:tasks.workunit.client.0.smithi107.stdout:0/907: dread d3/dc/d22/d23/f48 [0,4194304] 0 2024-05-17T21:53:05.706 INFO:tasks.workunit.client.1.smithi138.stdout:2/671: truncate d3/d21/d2f/f36 1968009 0 2024-05-17T21:53:05.708 INFO:tasks.workunit.client.1.smithi138.stdout:6/643: dwrite d7/d69/f71 [0,4194304] 0 2024-05-17T21:53:05.713 INFO:tasks.workunit.client.0.smithi107.stdout:2/648: dwrite d0/d9/d1c/d22/f5b [0,4194304] 0 2024-05-17T21:53:05.713 INFO:tasks.workunit.client.1.smithi138.stdout:1/601: mkdir d4/dd/d3b/da7/dbd 0 2024-05-17T21:53:05.714 INFO:tasks.workunit.client.1.smithi138.stdout:1/602: stat d4/df/d35/l46 0 2024-05-17T21:53:05.715 INFO:tasks.workunit.client.0.smithi107.stdout:4/822: creat d3/d6/d26/d36/d37/d85/f119 x:0 0 0 2024-05-17T21:53:05.715 INFO:tasks.workunit.client.1.smithi138.stdout:1/603: chown d4/f3d 2931 1 2024-05-17T21:53:05.717 INFO:tasks.workunit.client.0.smithi107.stdout:8/770: symlink dc/ddb/df2/l113 0 2024-05-17T21:53:05.718 INFO:tasks.workunit.client.1.smithi138.stdout:9/554: truncate d6/d7/d16/d29/d48/f98 2455405 0 2024-05-17T21:53:05.720 INFO:tasks.workunit.client.0.smithi107.stdout:1/843: dread d4/df/d4e/f63 [0,4194304] 0 2024-05-17T21:53:05.720 INFO:tasks.workunit.client.0.smithi107.stdout:7/775: sync 2024-05-17T21:53:05.720 INFO:tasks.workunit.client.0.smithi107.stdout:9/830: sync 2024-05-17T21:53:05.721 INFO:tasks.workunit.client.1.smithi138.stdout:3/741: creat d1/d4f/d50/f100 x:0 0 0 2024-05-17T21:53:05.724 INFO:tasks.workunit.client.0.smithi107.stdout:0/908: unlink d3/d24/l2b 0 2024-05-17T21:53:05.726 INFO:tasks.workunit.client.0.smithi107.stdout:7/776: dread d5/df/d11/d1c/d31/d56/f9c [0,4194304] 0 2024-05-17T21:53:05.727 INFO:tasks.workunit.client.0.smithi107.stdout:1/844: dwrite d4/df/d4e/d59/fe3 [0,4194304] 0 2024-05-17T21:53:05.727 INFO:tasks.workunit.client.0.smithi107.stdout:5/809: creat d5/db/d16/d2d/d32/dbe/f10d x:0 0 0 2024-05-17T21:53:05.728 INFO:tasks.workunit.client.0.smithi107.stdout:1/845: fsync d4/df/d1a/d41/f48 0 2024-05-17T21:53:05.729 INFO:tasks.workunit.client.0.smithi107.stdout:5/810: chown d5/db/d16/d2d/d40/d9f/dc9/lfe 0 1 2024-05-17T21:53:05.729 INFO:tasks.workunit.client.1.smithi138.stdout:2/672: unlink d3/d81/cc7 0 2024-05-17T21:53:05.730 INFO:tasks.workunit.client.1.smithi138.stdout:5/631: write d1/d53/f6e [463683,64472] 0 2024-05-17T21:53:05.730 INFO:tasks.workunit.client.1.smithi138.stdout:5/632: stat d1/d2c 0 2024-05-17T21:53:05.731 INFO:tasks.workunit.client.0.smithi107.stdout:5/811: dread d5/db/d16/d2d/d40/d9f/fef [0,4194304] 0 2024-05-17T21:53:05.735 INFO:tasks.workunit.client.1.smithi138.stdout:4/673: getdents d0/d2f/d49/d60/d1f/d32/d71 0 2024-05-17T21:53:05.736 INFO:tasks.workunit.client.1.smithi138.stdout:0/615: write d3/f2b [1386348,53730] 0 2024-05-17T21:53:05.738 INFO:tasks.workunit.client.1.smithi138.stdout:6/644: dread - d7/d69/db5/dc8/d43/f98 zero size 2024-05-17T21:53:05.739 INFO:tasks.workunit.client.0.smithi107.stdout:3/839: write d6/d8/d10/f38 [1747484,82787] 0 2024-05-17T21:53:05.741 INFO:tasks.workunit.client.1.smithi138.stdout:8/582: creat d0/dc/d1a/d33/d4a/fae x:0 0 0 2024-05-17T21:53:05.742 INFO:tasks.workunit.client.0.smithi107.stdout:4/823: creat d3/db/d1e/d66/f11a x:0 0 0 2024-05-17T21:53:05.745 INFO:tasks.workunit.client.0.smithi107.stdout:2/649: dread d0/d9/d11/f8b [0,4194304] 0 2024-05-17T21:53:05.745 INFO:tasks.workunit.client.0.smithi107.stdout:8/771: mknod dc/d21/daa/db4/c114 0 2024-05-17T21:53:05.746 INFO:tasks.workunit.client.0.smithi107.stdout:4/824: dread d3/d6/d26/d36/d7e/f87 [0,4194304] 0 2024-05-17T21:53:05.746 INFO:tasks.workunit.client.1.smithi138.stdout:1/604: unlink d4/db/f4d 0 2024-05-17T21:53:05.746 INFO:tasks.workunit.client.1.smithi138.stdout:9/555: rmdir d6/d7/d16/d29/d6c/d83/d7e 39 2024-05-17T21:53:05.747 INFO:tasks.workunit.client.0.smithi107.stdout:4/825: chown d3/d6/d26/d3b/d91/df6/d111 401503281 1 2024-05-17T21:53:05.747 INFO:tasks.workunit.client.1.smithi138.stdout:9/556: fsync d6/d7/d16/d29/d48/d6e/fab 0 2024-05-17T21:53:05.749 INFO:tasks.workunit.client.1.smithi138.stdout:1/605: read d4/df/d3a/d3c/d63/d6d/f87 [656172,56884] 0 2024-05-17T21:53:05.750 INFO:tasks.workunit.client.0.smithi107.stdout:9/831: unlink d0/dd/f60 0 2024-05-17T21:53:05.757 INFO:tasks.workunit.client.0.smithi107.stdout:6/748: write d9/d16/d4d/db8/dc9/dd0/f63 [559435,99165] 0 2024-05-17T21:53:05.757 INFO:tasks.workunit.client.1.smithi138.stdout:3/742: dread d1/d3/d16/f29 [0,4194304] 0 2024-05-17T21:53:05.760 INFO:tasks.workunit.client.1.smithi138.stdout:5/633: unlink d1/d2c/d63/db5/d1b/l70 0 2024-05-17T21:53:05.763 INFO:tasks.workunit.client.0.smithi107.stdout:1/846: mkdir d4/df/d14/d2e/dd3/d10c 0 2024-05-17T21:53:05.763 INFO:tasks.workunit.client.1.smithi138.stdout:2/673: dwrite d3/f4 [8388608,4194304] 0 2024-05-17T21:53:05.764 INFO:tasks.workunit.client.0.smithi107.stdout:1/847: readlink d4/df/d14/d2e/d43/d47/l5b 0 2024-05-17T21:53:05.764 INFO:tasks.workunit.client.0.smithi107.stdout:1/848: readlink d4/df/d1a/d26/la9 0 2024-05-17T21:53:05.765 INFO:tasks.workunit.client.1.smithi138.stdout:0/616: symlink d3/d10/d6d/d9f/lcf 0 2024-05-17T21:53:05.768 INFO:tasks.workunit.client.1.smithi138.stdout:6/645: readlink d7/d69/db5/dc8/d43/l3d 0 2024-05-17T21:53:05.769 INFO:tasks.workunit.client.1.smithi138.stdout:6/646: chown d7/d69/db5/dc8/d43/d3f/f4c 1 1 2024-05-17T21:53:05.771 INFO:tasks.workunit.client.1.smithi138.stdout:8/583: creat d0/dc/d8d/faf x:0 0 0 2024-05-17T21:53:05.775 INFO:tasks.workunit.client.0.smithi107.stdout:7/777: write f4 [3945036,61997] 0 2024-05-17T21:53:05.776 INFO:tasks.workunit.client.0.smithi107.stdout:7/778: stat d5/df/d11/f5c 0 2024-05-17T21:53:05.780 INFO:tasks.workunit.client.1.smithi138.stdout:7/614: sync 2024-05-17T21:53:05.784 INFO:tasks.workunit.client.0.smithi107.stdout:5/812: dwrite d5/db/d16/d2d/f4f [0,4194304] 0 2024-05-17T21:53:05.801 INFO:tasks.workunit.client.0.smithi107.stdout:7/779: dread d5/df/d11/f5c [0,4194304] 0 2024-05-17T21:53:05.811 INFO:tasks.workunit.client.1.smithi138.stdout:1/606: unlink d4/dd/d3b/d6c/f78 0 2024-05-17T21:53:05.812 INFO:tasks.workunit.client.1.smithi138.stdout:3/743: mkdir d1/d3/d16/d3e/d71/d101 0 2024-05-17T21:53:05.822 INFO:tasks.workunit.client.1.smithi138.stdout:0/617: dread d3/d10/f1a [0,4194304] 0 2024-05-17T21:53:05.823 INFO:tasks.workunit.client.1.smithi138.stdout:8/584: mknod d0/dc/d8d/cb0 0 2024-05-17T21:53:05.825 INFO:tasks.workunit.client.0.smithi107.stdout:0/909: mkdir d3/dc/d22/d12c 0 2024-05-17T21:53:05.826 INFO:tasks.workunit.client.0.smithi107.stdout:6/749: truncate d9/d16/d22/f72 353302 0 2024-05-17T21:53:05.827 INFO:tasks.workunit.client.1.smithi138.stdout:9/557: mknod d6/d7/d16/d29/cb1 0 2024-05-17T21:53:05.828 INFO:tasks.workunit.client.0.smithi107.stdout:6/750: write d9/d16/f4a [3237496,3949] 0 2024-05-17T21:53:05.830 INFO:tasks.workunit.client.1.smithi138.stdout:1/607: read - d4/df/f98 zero size 2024-05-17T21:53:05.833 INFO:tasks.workunit.client.0.smithi107.stdout:5/813: mknod d5/db/d16/c10e 0 2024-05-17T21:53:05.836 INFO:tasks.workunit.client.1.smithi138.stdout:4/674: creat d0/d1b/d2b/d55/ff1 x:0 0 0 2024-05-17T21:53:05.836 INFO:tasks.workunit.client.1.smithi138.stdout:4/675: dread - d0/d2f/d49/fc2 zero size 2024-05-17T21:53:05.837 INFO:tasks.workunit.client.1.smithi138.stdout:2/674: mkdir d3/d21/d2f/d3e/d58/dc3/ddf 0 2024-05-17T21:53:05.839 INFO:tasks.workunit.client.1.smithi138.stdout:2/675: dread - d3/d21/d2f/d3e/d55/d99/dd2/fd7 zero size 2024-05-17T21:53:05.840 INFO:tasks.workunit.client.1.smithi138.stdout:6/647: mknod d7/d69/db5/dc8/d60/cd9 0 2024-05-17T21:53:05.843 INFO:tasks.workunit.client.0.smithi107.stdout:2/650: rename d0/d9/d11/ca5 to d0/d26/d64/cd1 0 2024-05-17T21:53:05.845 INFO:tasks.workunit.client.1.smithi138.stdout:0/618: dread d3/d10/f14 [0,4194304] 0 2024-05-17T21:53:05.846 INFO:tasks.workunit.client.0.smithi107.stdout:4/826: link d3/d6/fe0 d3/db/d35/d5e/dff/d80/d43/f11b 0 2024-05-17T21:53:05.849 INFO:tasks.workunit.client.0.smithi107.stdout:2/651: dread d0/d9/d1c/d3e/f3f [0,4194304] 0 2024-05-17T21:53:05.852 INFO:tasks.workunit.client.1.smithi138.stdout:1/608: rename d4/d2e/d36/f4f to d4/df/d3a/fbe 0 2024-05-17T21:53:05.853 INFO:tasks.workunit.client.0.smithi107.stdout:6/751: readlink d9/d16/d4d/l99 0 2024-05-17T21:53:05.853 INFO:tasks.workunit.client.0.smithi107.stdout:0/910: dwrite d3/d75/d9e/f83 [0,4194304] 0 2024-05-17T21:53:05.854 INFO:tasks.workunit.client.1.smithi138.stdout:3/744: symlink d1/d3/d16/d3a/l102 0 2024-05-17T21:53:05.863 INFO:tasks.workunit.client.1.smithi138.stdout:6/648: creat d7/d69/db5/dc8/d60/fda x:0 0 0 2024-05-17T21:53:05.870 INFO:tasks.workunit.client.0.smithi107.stdout:9/832: mkdir d0/dd/d21/d29/d57/d5f/d112 0 2024-05-17T21:53:05.871 INFO:tasks.workunit.client.0.smithi107.stdout:1/849: write d4/df/d1a/d26/d72/d85/fda [562976,102896] 0 2024-05-17T21:53:05.872 INFO:tasks.workunit.client.0.smithi107.stdout:2/652: symlink d0/d9/d1c/d3e/d4f/ld2 0 2024-05-17T21:53:05.875 INFO:tasks.workunit.client.1.smithi138.stdout:0/619: dwrite d3/d3d/d51/f64 [0,4194304] 0 2024-05-17T21:53:05.878 INFO:tasks.workunit.client.0.smithi107.stdout:2/653: dwrite d0/d9/d11/d18/fce [0,4194304] 0 2024-05-17T21:53:05.878 INFO:tasks.workunit.client.1.smithi138.stdout:1/609: unlink d4/df/f98 0 2024-05-17T21:53:05.879 INFO:tasks.workunit.client.1.smithi138.stdout:1/610: dread - d4/d93/faf zero size 2024-05-17T21:53:05.880 INFO:tasks.workunit.client.1.smithi138.stdout:3/745: symlink d1/d3/d49/ddb/l103 0 2024-05-17T21:53:05.881 INFO:tasks.workunit.client.1.smithi138.stdout:8/585: write d0/dc/f79 [985124,115720] 0 2024-05-17T21:53:05.883 INFO:tasks.workunit.client.0.smithi107.stdout:3/840: getdents d6/d1a 0 2024-05-17T21:53:05.884 INFO:tasks.workunit.client.1.smithi138.stdout:5/634: getdents d1/d53/d88/db9/d39/d62 0 2024-05-17T21:53:05.886 INFO:tasks.workunit.client.1.smithi138.stdout:9/558: write d6/d7/d16/d29/d38/f81 [1541069,84927] 0 2024-05-17T21:53:05.887 INFO:tasks.workunit.client.0.smithi107.stdout:8/772: link dc/d21/d97/db7/fef dc/d21/d27/d2f/dba/dee/f115 0 2024-05-17T21:53:05.887 INFO:tasks.workunit.client.1.smithi138.stdout:4/676: getdents d0/d1b/d42/d45/d84 0 2024-05-17T21:53:05.888 INFO:tasks.workunit.client.1.smithi138.stdout:9/559: chown d6/d7/d16/f6d 15 1 2024-05-17T21:53:05.891 INFO:tasks.workunit.client.1.smithi138.stdout:2/676: chown d3/d81/d96/da8/lcf 42448139 1 2024-05-17T21:53:05.892 INFO:tasks.workunit.client.0.smithi107.stdout:7/780: rename d5/df/fc9 to d5/df/d11/d1c/d31/d39/d92/f113 0 2024-05-17T21:53:05.894 INFO:tasks.workunit.client.1.smithi138.stdout:6/649: rmdir d7/d69/db5/dc8 39 2024-05-17T21:53:05.898 INFO:tasks.workunit.client.1.smithi138.stdout:7/615: link d0/d18/c36 d0/d9d/cc4 0 2024-05-17T21:53:05.900 INFO:tasks.workunit.client.1.smithi138.stdout:7/616: readlink d0/d18/d40/d41/d51/d81/l83 0 2024-05-17T21:53:05.900 INFO:tasks.workunit.client.1.smithi138.stdout:0/620: creat d3/d10/d38/d3e/d59/fd0 x:0 0 0 2024-05-17T21:53:05.902 INFO:tasks.workunit.client.1.smithi138.stdout:1/611: creat d4/d2e/da0/fbf x:0 0 0 2024-05-17T21:53:05.903 INFO:tasks.workunit.client.0.smithi107.stdout:1/850: mknod d4/df/d14/d2e/dd3/c10d 0 2024-05-17T21:53:05.906 INFO:tasks.workunit.client.0.smithi107.stdout:4/827: dwrite d3/db/f46 [0,4194304] 0 2024-05-17T21:53:05.907 INFO:tasks.workunit.client.1.smithi138.stdout:8/586: creat d0/dc/d3b/d4d/d7d/fb1 x:0 0 0 2024-05-17T21:53:05.909 INFO:tasks.workunit.client.1.smithi138.stdout:3/746: dread d1/d3/d49/ddb/f85 [0,4194304] 0 2024-05-17T21:53:05.910 INFO:tasks.workunit.client.1.smithi138.stdout:5/635: dread d1/d2c/d63/db5/f31 [0,4194304] 0 2024-05-17T21:53:05.910 INFO:tasks.workunit.client.1.smithi138.stdout:9/560: mknod d6/d7/d16/d29/d48/d67/d86/cb2 0 2024-05-17T21:53:05.911 INFO:tasks.workunit.client.1.smithi138.stdout:9/561: readlink d6/d7/d16/d29/d48/lac 0 2024-05-17T21:53:05.912 INFO:tasks.workunit.client.1.smithi138.stdout:2/677: truncate d3/f35 2152123 0 2024-05-17T21:53:05.912 INFO:tasks.workunit.client.1.smithi138.stdout:5/636: write d1/d2c/d63/db5/d19/f20 [9279,80697] 0 2024-05-17T21:53:05.915 INFO:tasks.workunit.client.1.smithi138.stdout:6/650: fdatasync d7/d69/db5/dc8/d43/d3f/f5b 0 2024-05-17T21:53:05.915 INFO:tasks.workunit.client.0.smithi107.stdout:2/654: dread d0/d9/d1c/f3d [0,4194304] 0 2024-05-17T21:53:05.915 INFO:tasks.workunit.client.0.smithi107.stdout:2/655: readlink d0/da6/lac 0 2024-05-17T21:53:05.916 INFO:tasks.workunit.client.1.smithi138.stdout:7/617: mkdir d0/d18/d64/d7c/da2/dc5 0 2024-05-17T21:53:05.916 INFO:tasks.workunit.client.0.smithi107.stdout:2/656: chown d0/d9/d11/d27 592867751 1 2024-05-17T21:53:05.919 INFO:tasks.workunit.client.0.smithi107.stdout:9/833: rename d0/dd/d2f/d87/d89/fd3 to d0/dd/d21/d29/dca/f113 0 2024-05-17T21:53:05.921 INFO:tasks.workunit.client.0.smithi107.stdout:1/851: creat d4/de/f10e x:0 0 0 2024-05-17T21:53:05.922 INFO:tasks.workunit.client.1.smithi138.stdout:4/677: mknod d0/d3/d1d/d38/d8a/dbd/ddc/cf2 0 2024-05-17T21:53:05.925 INFO:tasks.workunit.client.1.smithi138.stdout:8/587: rename d0/dc/d1a/l6b to d0/dc/d3b/d87/lb2 0 2024-05-17T21:53:05.928 INFO:tasks.workunit.client.1.smithi138.stdout:3/747: fdatasync d1/d62/dad/d5c/d9c/fa3 0 2024-05-17T21:53:05.930 INFO:tasks.workunit.client.0.smithi107.stdout:6/752: write d9/d16/d4d/faa [892017,97933] 0 2024-05-17T21:53:05.932 INFO:tasks.workunit.client.1.smithi138.stdout:2/678: creat d3/d81/d96/da8/fe0 x:0 0 0 2024-05-17T21:53:05.933 INFO:tasks.workunit.client.0.smithi107.stdout:5/814: getdents d5/db/dc 0 2024-05-17T21:53:05.933 INFO:tasks.workunit.client.0.smithi107.stdout:0/911: write d3/dc/d22/f32 [1936877,111486] 0 2024-05-17T21:53:05.935 INFO:tasks.workunit.client.0.smithi107.stdout:3/841: mknod d6/d2a/d2d/c10e 0 2024-05-17T21:53:05.939 INFO:tasks.workunit.client.0.smithi107.stdout:8/773: symlink dc/d21/d27/d34/l116 0 2024-05-17T21:53:05.942 INFO:tasks.workunit.client.0.smithi107.stdout:7/781: dread d5/d16/d3b/d45/fd0 [0,4194304] 0 2024-05-17T21:53:05.943 INFO:tasks.workunit.client.0.smithi107.stdout:2/657: rename d0/da6 to d0/d9/d11/d27/dd3 0 2024-05-17T21:53:05.945 INFO:tasks.workunit.client.1.smithi138.stdout:6/651: mknod d7/da0/dbe/cdb 0 2024-05-17T21:53:05.946 INFO:tasks.workunit.client.0.smithi107.stdout:1/852: truncate d4/df/d14/d2e/d43/fc6 460735 0 2024-05-17T21:53:05.946 INFO:tasks.workunit.client.1.smithi138.stdout:1/612: mknod d4/dd/cc0 0 2024-05-17T21:53:05.949 INFO:tasks.workunit.client.0.smithi107.stdout:6/753: rmdir d9/d81 39 2024-05-17T21:53:05.951 INFO:tasks.workunit.client.1.smithi138.stdout:0/621: write d3/f18 [92945,124362] 0 2024-05-17T21:53:05.954 INFO:tasks.workunit.client.1.smithi138.stdout:7/618: rename d0/f8e to d0/d39/fc6 0 2024-05-17T21:53:05.954 INFO:tasks.workunit.client.1.smithi138.stdout:7/619: readlink d0/d18/d40/d41/d50/l8a 0 2024-05-17T21:53:05.955 INFO:tasks.workunit.client.0.smithi107.stdout:5/815: creat d5/db/d16/d2d/d40/f10f x:0 0 0 2024-05-17T21:53:05.955 INFO:tasks.workunit.client.1.smithi138.stdout:7/620: stat d0/d18/d40/d5f/c7b 0 2024-05-17T21:53:05.956 INFO:tasks.workunit.client.1.smithi138.stdout:8/588: symlink d0/dc/d3b/d4d/d7d/d97/lb3 0 2024-05-17T21:53:05.966 INFO:tasks.workunit.client.1.smithi138.stdout:3/748: write d1/d3/d16/f29 [1232142,61365] 0 2024-05-17T21:53:05.968 INFO:tasks.workunit.client.1.smithi138.stdout:6/652: mknod d7/d69/db5/dc8/d43/d5f/d94/dc9/cdc 0 2024-05-17T21:53:05.969 INFO:tasks.workunit.client.0.smithi107.stdout:2/658: unlink d0/d9/d1c/fab 0 2024-05-17T21:53:05.970 INFO:tasks.workunit.client.0.smithi107.stdout:9/834: write d0/dd/d3d/fd8 [15984,41448] 0 2024-05-17T21:53:05.970 INFO:tasks.workunit.client.0.smithi107.stdout:4/828: link d3/d6/ld3 d3/d6/d26/d36/d8b/d95/dee/l11c 0 2024-05-17T21:53:05.970 INFO:tasks.workunit.client.1.smithi138.stdout:1/613: mkdir d4/dd/d3b/d6c/dc1 0 2024-05-17T21:53:05.970 INFO:tasks.workunit.client.0.smithi107.stdout:5/816: mknod d5/d75/c110 0 2024-05-17T21:53:05.971 INFO:tasks.workunit.client.0.smithi107.stdout:0/912: chown d3/d24/d6f/l106 11533 1 2024-05-17T21:53:05.973 INFO:tasks.workunit.client.0.smithi107.stdout:8/774: rename dc/d21/d27/d2f/f30 to dc/d21/d27/d2f/dba/d104/f117 0 2024-05-17T21:53:05.973 INFO:tasks.workunit.client.1.smithi138.stdout:0/622: creat d3/d10/d38/d3e/d59/dad/fd1 x:0 0 0 2024-05-17T21:53:05.975 INFO:tasks.workunit.client.0.smithi107.stdout:1/853: mknod d4/df/d1a/d26/d72/daa/deb/d104/c10f 0 2024-05-17T21:53:05.976 INFO:tasks.workunit.client.1.smithi138.stdout:7/621: creat d0/d18/d40/fc7 x:0 0 0 2024-05-17T21:53:05.985 INFO:tasks.workunit.client.1.smithi138.stdout:2/679: mkdir d3/d21/d2f/de1 0 2024-05-17T21:53:05.991 INFO:tasks.workunit.client.1.smithi138.stdout:6/653: read d7/d69/fa7 [2382481,25335] 0 2024-05-17T21:53:05.991 INFO:tasks.workunit.client.1.smithi138.stdout:4/678: creat d0/d2f/ff3 x:0 0 0 2024-05-17T21:53:05.991 INFO:tasks.workunit.client.0.smithi107.stdout:4/829: rmdir d3/d6/d26/d36 39 2024-05-17T21:53:05.991 INFO:tasks.workunit.client.0.smithi107.stdout:5/817: rmdir d5/db/d16/d2d/d32 39 2024-05-17T21:53:05.992 INFO:tasks.workunit.client.0.smithi107.stdout:1/854: write d4/df/d1a/f7b [4523474,62045] 0 2024-05-17T21:53:05.992 INFO:tasks.workunit.client.0.smithi107.stdout:3/842: dread d6/d2a/f52 [0,4194304] 0 2024-05-17T21:53:05.992 INFO:tasks.workunit.client.0.smithi107.stdout:4/830: stat d3/db/d35/c9b 0 2024-05-17T21:53:05.993 INFO:tasks.workunit.client.1.smithi138.stdout:0/623: write d3/f45 [1197715,11732] 0 2024-05-17T21:53:05.993 INFO:tasks.workunit.client.0.smithi107.stdout:2/659: mkdir d0/d9/d1c/dd4 0 2024-05-17T21:53:05.994 INFO:tasks.workunit.client.1.smithi138.stdout:7/622: truncate d0/d18/d40/f58 932132 0 2024-05-17T21:53:05.995 INFO:tasks.workunit.client.0.smithi107.stdout:9/835: dread d0/dd/d21/d29/d3e/d45/f67 [0,4194304] 0 2024-05-17T21:53:05.995 INFO:tasks.workunit.client.0.smithi107.stdout:2/660: fdatasync d0/d93/fb3 0 2024-05-17T21:53:05.995 INFO:tasks.workunit.client.1.smithi138.stdout:1/614: write d4/df/f97 [1683364,103208] 0 2024-05-17T21:53:05.996 INFO:tasks.workunit.client.1.smithi138.stdout:0/624: dread d3/d10/f14 [0,4194304] 0 2024-05-17T21:53:05.997 INFO:tasks.workunit.client.0.smithi107.stdout:3/843: truncate d6/d8/d10/d35/f100 945652 0 2024-05-17T21:53:05.998 INFO:tasks.workunit.client.0.smithi107.stdout:3/844: fdatasync d6/d8/d10/d35/f100 0 2024-05-17T21:53:05.999 INFO:tasks.workunit.client.0.smithi107.stdout:6/754: rename d9/c3f to d9/d16/d4d/db8/dc9/dd0/cec 0 2024-05-17T21:53:06.000 INFO:tasks.workunit.client.1.smithi138.stdout:9/562: getdents d6/d7/d16/d29 0 2024-05-17T21:53:06.001 INFO:tasks.workunit.client.1.smithi138.stdout:9/563: write d6/d7/f99 [1010243,47110] 0 2024-05-17T21:53:06.001 INFO:tasks.workunit.client.0.smithi107.stdout:2/661: dread d0/d9/d1c/f73 [0,4194304] 0 2024-05-17T21:53:06.002 INFO:tasks.workunit.client.1.smithi138.stdout:9/564: chown d6/d7/d16/d29/f37 106333 1 2024-05-17T21:53:06.003 INFO:tasks.workunit.client.0.smithi107.stdout:2/662: readlink d0/d9/d1c/l3c 0 2024-05-17T21:53:06.005 INFO:tasks.workunit.client.0.smithi107.stdout:0/913: fsync d3/dc/d22/d3d/d56/f5b 0 2024-05-17T21:53:06.005 INFO:tasks.workunit.client.0.smithi107.stdout:6/755: dwrite d9/f1a [0,4194304] 0 2024-05-17T21:53:06.007 INFO:tasks.workunit.client.1.smithi138.stdout:8/589: dwrite d0/f17 [0,4194304] 0 2024-05-17T21:53:06.007 INFO:tasks.workunit.client.1.smithi138.stdout:5/637: getdents d1/d2c/d63/db5/d19 0 2024-05-17T21:53:06.008 INFO:tasks.workunit.client.1.smithi138.stdout:8/590: fsync d0/d63/f9c 0 2024-05-17T21:53:06.010 INFO:tasks.workunit.client.1.smithi138.stdout:2/680: write d3/d21/d2f/d3e/d58/d77/da4/dc5/fa2 [121265,111944] 0 2024-05-17T21:53:06.010 INFO:tasks.workunit.client.0.smithi107.stdout:4/831: truncate d3/db/d35/d5e/dff/d80/d43/faa 406366 0 2024-05-17T21:53:06.018 INFO:tasks.workunit.client.1.smithi138.stdout:1/615: symlink d4/df/d3a/d3c/lc2 0 2024-05-17T21:53:06.018 INFO:tasks.workunit.client.0.smithi107.stdout:8/775: link dc/dd5/le4 dc/d21/d27/l118 0 2024-05-17T21:53:06.019 INFO:tasks.workunit.client.0.smithi107.stdout:9/836: write d0/dd/d3d/fb8 [1416371,11447] 0 2024-05-17T21:53:06.021 INFO:tasks.workunit.client.0.smithi107.stdout:5/818: dread d5/db/d16/d54/d59/d65/f6f [0,4194304] 0 2024-05-17T21:53:06.022 INFO:tasks.workunit.client.0.smithi107.stdout:5/819: write d5/d75/fae [659710,71855] 0 2024-05-17T21:53:06.023 INFO:tasks.workunit.client.1.smithi138.stdout:3/749: dwrite d1/f65 [0,4194304] 0 2024-05-17T21:53:06.023 INFO:tasks.workunit.client.0.smithi107.stdout:1/855: symlink d4/df/dc1/l110 0 2024-05-17T21:53:06.025 INFO:tasks.workunit.client.0.smithi107.stdout:6/756: chown d9/d16/c4f 0 1 2024-05-17T21:53:06.028 INFO:tasks.workunit.client.0.smithi107.stdout:3/845: dwrite d6/d2a/d2d/d37/f7c [0,4194304] 0 2024-05-17T21:53:06.028 INFO:tasks.workunit.client.0.smithi107.stdout:4/832: mknod d3/db/d35/c11d 0 2024-05-17T21:53:06.029 INFO:tasks.workunit.client.1.smithi138.stdout:5/638: mknod d1/d53/d96/ccf 0 2024-05-17T21:53:06.030 INFO:tasks.workunit.client.1.smithi138.stdout:5/639: fdatasync d1/d46/f78 0 2024-05-17T21:53:06.037 INFO:tasks.workunit.client.1.smithi138.stdout:1/616: mknod d4/d2e/da0/daa/cc3 0 2024-05-17T21:53:06.041 INFO:tasks.workunit.client.1.smithi138.stdout:0/625: mknod d3/cd2 0 2024-05-17T21:53:06.042 INFO:tasks.workunit.client.1.smithi138.stdout:3/750: dread - d1/d62/dad/de/f8e zero size 2024-05-17T21:53:06.044 INFO:tasks.workunit.client.0.smithi107.stdout:1/856: unlink d4/df/d14/d2e/ddf/cf4 0 2024-05-17T21:53:06.046 INFO:tasks.workunit.client.0.smithi107.stdout:8/776: readlink dc/d21/d27/d2f/l95 0 2024-05-17T21:53:06.049 INFO:tasks.workunit.client.0.smithi107.stdout:0/914: dwrite d3/dc/d22/d23/fb7 [0,4194304] 0 2024-05-17T21:53:06.050 INFO:tasks.workunit.client.0.smithi107.stdout:2/663: creat d0/d9/d11/d18/fd5 x:0 0 0 2024-05-17T21:53:06.053 INFO:tasks.workunit.client.1.smithi138.stdout:9/565: dwrite d6/d7/d16/d29/d48/f7d [0,4194304] 0 2024-05-17T21:53:06.056 INFO:tasks.workunit.client.1.smithi138.stdout:1/617: mkdir d4/df/d3a/dc4 0 2024-05-17T21:53:06.059 INFO:tasks.workunit.client.1.smithi138.stdout:8/591: dwrite d0/d2f/f6e [0,4194304] 0 2024-05-17T21:53:06.061 INFO:tasks.workunit.client.1.smithi138.stdout:8/592: dread - d0/dc/d8d/faf zero size 2024-05-17T21:53:06.062 INFO:tasks.workunit.client.1.smithi138.stdout:8/593: write d0/d63/f9c [2238648,25211] 0 2024-05-17T21:53:06.063 INFO:tasks.workunit.client.1.smithi138.stdout:7/623: truncate d0/d18/d40/d5f/f8d 744684 0 2024-05-17T21:53:06.063 INFO:tasks.workunit.client.1.smithi138.stdout:7/624: readlink d0/d18/d40/d5f/l90 0 2024-05-17T21:53:06.064 INFO:tasks.workunit.client.0.smithi107.stdout:6/757: mkdir d9/ded 0 2024-05-17T21:53:06.064 INFO:tasks.workunit.client.1.smithi138.stdout:8/594: read d0/d2f/f45 [107976,97219] 0 2024-05-17T21:53:06.067 INFO:tasks.workunit.client.1.smithi138.stdout:3/751: unlink d1/d2e/l39 0 2024-05-17T21:53:06.069 INFO:tasks.workunit.client.1.smithi138.stdout:5/640: write d1/d2c/d63/db5/f31 [4568534,123234] 0 2024-05-17T21:53:06.071 INFO:tasks.workunit.client.1.smithi138.stdout:2/681: truncate d3/d21/d2f/d3e/f6d 1459761 0 2024-05-17T21:53:06.071 INFO:tasks.workunit.client.1.smithi138.stdout:4/679: getdents d0/d3/d17/d5e/d7f 0 2024-05-17T21:53:06.073 INFO:tasks.workunit.client.1.smithi138.stdout:4/680: write d0/d3/db/da0/fa8 [718061,5787] 0 2024-05-17T21:53:06.074 INFO:tasks.workunit.client.0.smithi107.stdout:3/846: rename d6/d8/d10/d35/d36/d4c/dcc/l106 to d6/d8/d10/d35/d36/d4c/dcc/l10f 0 2024-05-17T21:53:06.075 INFO:tasks.workunit.client.1.smithi138.stdout:6/654: link d7/d69/db5/dc8/d43/d5f/d94/lac d7/d69/db5/dc8/d43/ldd 0 2024-05-17T21:53:06.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:05 smithi138 ceph-mon[35948]: pgmap v41: 65 pgs: 65 active+clean; 2.7 GiB data, 9.2 GiB used, 527 GiB / 536 GiB avail; 109 MiB/s rd, 251 MiB/s wr, 584 op/s 2024-05-17T21:53:06.079 INFO:tasks.workunit.client.1.smithi138.stdout:1/618: creat d4/df/d3a/fc5 x:0 0 0 2024-05-17T21:53:06.079 INFO:tasks.workunit.client.0.smithi107.stdout:9/837: link d0/dd/d21/d29/d3e/d45/d68/f8c d0/dd/d21/d29/d3e/d94/d111/f114 0 2024-05-17T21:53:06.080 INFO:tasks.workunit.client.0.smithi107.stdout:9/838: readlink d0/dd/d21/d29/d3e/d45/d68/leb 0 2024-05-17T21:53:06.082 INFO:tasks.workunit.client.0.smithi107.stdout:9/839: chown d0/dd/d2f/d81/dc3/f104 1 1 2024-05-17T21:53:06.082 INFO:tasks.workunit.client.0.smithi107.stdout:0/915: dread d3/dc/d22/f40 [0,4194304] 0 2024-05-17T21:53:06.084 INFO:tasks.workunit.client.1.smithi138.stdout:8/595: unlink d0/f1b 0 2024-05-17T21:53:06.084 INFO:tasks.workunit.client.1.smithi138.stdout:8/596: stat d0/dc/f79 0 2024-05-17T21:53:06.085 INFO:tasks.workunit.client.1.smithi138.stdout:8/597: chown d0/d2f/c55 1157 1 2024-05-17T21:53:06.092 INFO:tasks.workunit.client.1.smithi138.stdout:2/682: rmdir d3/d21/d2f/d3e/d55/d99/d84/d9f 39 2024-05-17T21:53:06.095 INFO:tasks.workunit.client.0.smithi107.stdout:3/847: mkdir d6/d8/d10/d35/d36/d4c/dcc/d110 0 2024-05-17T21:53:06.097 INFO:tasks.workunit.client.1.smithi138.stdout:9/566: symlink d6/d7/d16/d29/d38/d8b/lb3 0 2024-05-17T21:53:06.100 INFO:tasks.workunit.client.1.smithi138.stdout:6/655: rename d7/d69/db5/dc8/d43/d3f/f5b to d7/d69/db5/dc8/d43/d5f/d6a/fde 0 2024-05-17T21:53:06.101 INFO:tasks.workunit.client.0.smithi107.stdout:5/820: getdents d5/db/d16/d54 0 2024-05-17T21:53:06.102 INFO:tasks.workunit.client.1.smithi138.stdout:9/567: dread d6/f2a [0,4194304] 0 2024-05-17T21:53:06.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:05 smithi107 ceph-mon[28434]: pgmap v41: 65 pgs: 65 active+clean; 2.7 GiB data, 9.2 GiB used, 527 GiB / 536 GiB avail; 109 MiB/s rd, 251 MiB/s wr, 584 op/s 2024-05-17T21:53:06.102 INFO:tasks.workunit.client.0.smithi107.stdout:2/664: dwrite d0/d9/d1c/f96 [0,4194304] 0 2024-05-17T21:53:06.103 INFO:tasks.workunit.client.0.smithi107.stdout:6/758: write d9/d16/f3e [4456802,120588] 0 2024-05-17T21:53:06.103 INFO:tasks.workunit.client.1.smithi138.stdout:9/568: fsync d6/d7/d16/d29/d38/fa3 0 2024-05-17T21:53:06.105 INFO:tasks.workunit.client.1.smithi138.stdout:0/626: dwrite d3/f84 [0,4194304] 0 2024-05-17T21:53:06.106 INFO:tasks.workunit.client.0.smithi107.stdout:0/916: mkdir d3/d75/dd0/d12d 0 2024-05-17T21:53:06.109 INFO:tasks.workunit.client.0.smithi107.stdout:4/833: truncate d3/d6/d26/d36/f6e 1785752 0 2024-05-17T21:53:06.111 INFO:tasks.workunit.client.0.smithi107.stdout:9/840: symlink d0/dd/d21/daf/l115 0 2024-05-17T21:53:06.112 INFO:tasks.workunit.client.1.smithi138.stdout:7/625: dwrite d0/d18/d40/d41/d51/d66/f6f [0,4194304] 0 2024-05-17T21:53:06.115 INFO:tasks.workunit.client.0.smithi107.stdout:1/857: rename d4/df/d4e/d59/fcc to d4/df/d14/d2e/d43/f111 0 2024-05-17T21:53:06.116 INFO:tasks.workunit.client.0.smithi107.stdout:8/777: mkdir dc/d21/d27/d34/d119 0 2024-05-17T21:53:06.117 INFO:tasks.workunit.client.1.smithi138.stdout:5/641: mkdir d1/d53/d88/db9/d39/d4e/d58/d5b/d65/dca/dd0 0 2024-05-17T21:53:06.117 INFO:tasks.workunit.client.1.smithi138.stdout:5/642: dread - d1/d46/d95/fae zero size 2024-05-17T21:53:06.118 INFO:tasks.workunit.client.1.smithi138.stdout:2/683: creat d3/d21/fe2 x:0 0 0 2024-05-17T21:53:06.121 INFO:tasks.workunit.client.1.smithi138.stdout:4/681: symlink d0/lf4 0 2024-05-17T21:53:06.122 INFO:tasks.workunit.client.0.smithi107.stdout:2/665: symlink d0/d9/d1c/d22/ld6 0 2024-05-17T21:53:06.123 INFO:tasks.workunit.client.1.smithi138.stdout:2/684: dread f2 [0,4194304] 0 2024-05-17T21:53:06.124 INFO:tasks.workunit.client.1.smithi138.stdout:6/656: readlink d7/d69/db5/dc8/l6b 0 2024-05-17T21:53:06.125 INFO:tasks.workunit.client.0.smithi107.stdout:4/834: symlink d3/db/d35/d5e/dff/d80/l11e 0 2024-05-17T21:53:06.126 INFO:tasks.workunit.client.1.smithi138.stdout:6/657: dread - d7/d69/db5/dc8/d43/d5f/d6a/d9b/db2/fb9 zero size 2024-05-17T21:53:06.127 INFO:tasks.workunit.client.1.smithi138.stdout:6/658: write d7/d69/f90 [434845,78998] 0 2024-05-17T21:53:06.128 INFO:tasks.workunit.client.1.smithi138.stdout:1/619: unlink d4/db/d29/d5d/cb0 0 2024-05-17T21:53:06.129 INFO:tasks.workunit.client.1.smithi138.stdout:1/620: chown d4/df/d35/l38 258 1 2024-05-17T21:53:06.130 INFO:tasks.workunit.client.0.smithi107.stdout:3/848: rename d6/d8/d10/dce to d6/d8/d10/d35/d36/d4c/dcc/d111 0 2024-05-17T21:53:06.130 INFO:tasks.workunit.client.0.smithi107.stdout:3/849: chown d6/d8/d10/d25/f55 6445 1 2024-05-17T21:53:06.131 INFO:tasks.workunit.client.1.smithi138.stdout:9/569: mkdir d6/d7/d16/d29/d38/d40/d44/d5c/db4 0 2024-05-17T21:53:06.137 INFO:tasks.workunit.client.0.smithi107.stdout:8/778: dread dc/d21/daa/db4/fe0 [0,4194304] 0 2024-05-17T21:53:06.137 INFO:tasks.workunit.client.1.smithi138.stdout:0/627: creat d3/d10/d6d/d81/fd3 x:0 0 0 2024-05-17T21:53:06.138 INFO:tasks.workunit.client.0.smithi107.stdout:8/779: chown dc/d44/d4b/d52/d70/d76 172674663 1 2024-05-17T21:53:06.140 INFO:tasks.workunit.client.0.smithi107.stdout:6/759: getdents d9/d16/d4d/d85/d88/db1/ddd 0 2024-05-17T21:53:06.140 INFO:tasks.workunit.client.1.smithi138.stdout:8/598: dread d0/f2 [4194304,4194304] 0 2024-05-17T21:53:06.141 INFO:tasks.workunit.client.1.smithi138.stdout:7/626: rmdir d0/d18/d9c 39 2024-05-17T21:53:06.143 INFO:tasks.workunit.client.1.smithi138.stdout:2/685: dread d3/d21/f2b [0,4194304] 0 2024-05-17T21:53:06.148 INFO:tasks.workunit.client.0.smithi107.stdout:5/821: dwrite d5/db/d16/d2d/d40/d5a/db4/fb9 [0,4194304] 0 2024-05-17T21:53:06.148 INFO:tasks.workunit.client.1.smithi138.stdout:2/686: dread d3/d21/f67 [0,4194304] 0 2024-05-17T21:53:06.148 INFO:tasks.workunit.client.1.smithi138.stdout:3/752: creat d1/d62/dad/f104 x:0 0 0 2024-05-17T21:53:06.150 INFO:tasks.workunit.client.0.smithi107.stdout:9/841: mkdir d0/dd/d69/dee/d116 0 2024-05-17T21:53:06.151 INFO:tasks.workunit.client.0.smithi107.stdout:9/842: truncate d0/dd/d2f/d87/d89/de0/f110 495742 0 2024-05-17T21:53:06.154 INFO:tasks.workunit.client.0.smithi107.stdout:3/850: creat d6/d8/d10/d25/d80/f112 x:0 0 0 2024-05-17T21:53:06.160 INFO:tasks.workunit.client.1.smithi138.stdout:6/659: write d7/d69/db5/dc8/d43/d5f/d6a/fde [453167,95197] 0 2024-05-17T21:53:06.162 INFO:tasks.workunit.client.0.smithi107.stdout:4/835: write d3/d6/d26/f2f [797468,51078] 0 2024-05-17T21:53:06.163 INFO:tasks.workunit.client.0.smithi107.stdout:4/836: chown d3/db/d35/d109 917 1 2024-05-17T21:53:06.164 INFO:tasks.workunit.client.0.smithi107.stdout:2/666: mknod d0/d9/d1c/d22/db0/cd7 0 2024-05-17T21:53:06.167 INFO:tasks.workunit.client.0.smithi107.stdout:0/917: symlink d3/dc/dae/dc0/de1/d10b/d11b/l12e 0 2024-05-17T21:53:06.169 INFO:tasks.workunit.client.1.smithi138.stdout:8/599: chown d0/dc/d1a/c53 1216570 1 2024-05-17T21:53:06.170 INFO:tasks.workunit.client.1.smithi138.stdout:7/627: creat d0/d18/d40/d41/d51/d66/d9e/fc8 x:0 0 0 2024-05-17T21:53:06.171 INFO:tasks.workunit.client.1.smithi138.stdout:8/600: readlink d0/dc/d1a/d33/d4a/l84 0 2024-05-17T21:53:06.172 INFO:tasks.workunit.client.0.smithi107.stdout:5/822: unlink d5/db/dc/d1e/d6e/c7e 0 2024-05-17T21:53:06.172 INFO:tasks.workunit.client.1.smithi138.stdout:8/601: fsync d0/d11/f69 0 2024-05-17T21:53:06.174 INFO:tasks.workunit.client.1.smithi138.stdout:5/643: dread d1/d2c/d63/db5/d1b/dc5/f3b [0,4194304] 0 2024-05-17T21:53:06.179 INFO:tasks.workunit.client.1.smithi138.stdout:3/753: unlink d1/d3/d16/d3e/d71/fa9 0 2024-05-17T21:53:06.179 INFO:tasks.workunit.client.0.smithi107.stdout:8/780: write dc/d21/fd3 [411586,34676] 0 2024-05-17T21:53:06.180 INFO:tasks.workunit.client.0.smithi107.stdout:1/858: rmdir d4/df/d14/d2e/dd3/d10c 0 2024-05-17T21:53:06.181 INFO:tasks.workunit.client.1.smithi138.stdout:2/687: truncate d3/d21/d2f/d3e/d63/f7b 1000943 0 2024-05-17T21:53:06.183 INFO:tasks.workunit.client.0.smithi107.stdout:6/760: mknod d9/d16/d22/de6/cee 0 2024-05-17T21:53:06.185 INFO:tasks.workunit.client.1.smithi138.stdout:4/682: creat d0/d3/d1d/d38/d8a/dbd/ff5 x:0 0 0 2024-05-17T21:53:06.187 INFO:tasks.workunit.client.0.smithi107.stdout:5/823: mknod d5/db/d16/d2a/d73/dd0/c111 0 2024-05-17T21:53:06.189 INFO:tasks.workunit.client.1.smithi138.stdout:1/621: mknod d4/dd/d3b/d9f/da9/cc6 0 2024-05-17T21:53:06.190 INFO:tasks.workunit.client.1.smithi138.stdout:0/628: dwrite d3/d10/d38/d3e/d59/f9d [0,4194304] 0 2024-05-17T21:53:06.190 INFO:tasks.workunit.client.1.smithi138.stdout:9/570: symlink d6/d7/d16/d29/d6c/d83/d7e/lb5 0 2024-05-17T21:53:06.192 INFO:tasks.workunit.client.1.smithi138.stdout:9/571: dread - d6/d7/d16/f6d zero size 2024-05-17T21:53:06.193 INFO:tasks.workunit.client.0.smithi107.stdout:3/851: symlink d6/d2a/d2d/d69/dad/de1/l113 0 2024-05-17T21:53:06.195 INFO:tasks.workunit.client.0.smithi107.stdout:5/824: dread d5/db/d16/f21 [0,4194304] 0 2024-05-17T21:53:06.195 INFO:tasks.workunit.client.0.smithi107.stdout:8/781: readlink dc/d21/d27/d34/d4c/l5a 0 2024-05-17T21:53:06.198 INFO:tasks.workunit.client.0.smithi107.stdout:6/761: symlink d9/d16/d4d/db8/dc9/dd0/lef 0 2024-05-17T21:53:06.199 INFO:tasks.workunit.client.1.smithi138.stdout:8/602: symlink d0/d63/lb4 0 2024-05-17T21:53:06.200 INFO:tasks.workunit.client.1.smithi138.stdout:8/603: write d0/d63/f9c [4247009,129611] 0 2024-05-17T21:53:06.201 INFO:tasks.workunit.client.0.smithi107.stdout:4/837: mkdir d3/d4a/d47/d60/d11f 0 2024-05-17T21:53:06.201 INFO:tasks.workunit.client.0.smithi107.stdout:6/762: readlink d9/d16/d4d/d85/l94 0 2024-05-17T21:53:06.202 INFO:tasks.workunit.client.0.smithi107.stdout:9/843: dwrite d0/fa5 [0,4194304] 0 2024-05-17T21:53:06.203 INFO:tasks.workunit.client.1.smithi138.stdout:1/622: dread d4/df/d35/f37 [4194304,4194304] 0 2024-05-17T21:53:06.204 INFO:tasks.workunit.client.1.smithi138.stdout:3/754: rmdir d1/d62/dad/d5c/d9c 39 2024-05-17T21:53:06.205 INFO:tasks.workunit.client.1.smithi138.stdout:2/688: mknod d3/d81/d96/da8/ce3 0 2024-05-17T21:53:06.206 INFO:tasks.workunit.client.0.smithi107.stdout:9/844: chown d0/dd/d21/faa 389430643 1 2024-05-17T21:53:06.206 INFO:tasks.workunit.client.0.smithi107.stdout:1/859: mkdir d4/df/d1a/d26/d72/de1/d112 0 2024-05-17T21:53:06.207 INFO:tasks.workunit.client.1.smithi138.stdout:4/683: mknod d0/d3/d17/d21/cf6 0 2024-05-17T21:53:06.208 INFO:tasks.workunit.client.0.smithi107.stdout:3/852: dwrite d6/d8/d10/d25/f47 [0,4194304] 0 2024-05-17T21:53:06.210 INFO:tasks.workunit.client.0.smithi107.stdout:1/860: dread - d4/df/d1a/ffe zero size 2024-05-17T21:53:06.212 INFO:tasks.workunit.client.0.smithi107.stdout:3/853: chown d6/d2a/d2d/f10b 3719 1 2024-05-17T21:53:06.212 INFO:tasks.workunit.client.0.smithi107.stdout:5/825: dwrite d5/db/d16/d54/d59/d92/fe5 [0,4194304] 0 2024-05-17T21:53:06.213 INFO:tasks.workunit.client.1.smithi138.stdout:9/572: mkdir d6/d7/d16/d29/db6 0 2024-05-17T21:53:06.215 INFO:tasks.workunit.client.0.smithi107.stdout:5/826: write d5/db/d16/d2a/d73/fd9 [403838,40642] 0 2024-05-17T21:53:06.215 INFO:tasks.workunit.client.1.smithi138.stdout:7/628: rename d0/l2f to d0/d39/lc9 0 2024-05-17T21:53:06.217 INFO:tasks.workunit.client.1.smithi138.stdout:7/629: chown d0/d18/d40/f4e 202796220 1 2024-05-17T21:53:06.219 INFO:tasks.workunit.client.0.smithi107.stdout:9/845: unlink d0/dd/d3d/fd8 0 2024-05-17T21:53:06.219 INFO:tasks.workunit.client.1.smithi138.stdout:7/630: read d0/f1e [671350,15724] 0 2024-05-17T21:53:06.221 INFO:tasks.workunit.client.1.smithi138.stdout:8/604: mknod d0/dc/d8d/cb5 0 2024-05-17T21:53:06.227 INFO:tasks.workunit.client.1.smithi138.stdout:3/755: mknod d1/d3/d16/d3e/c105 0 2024-05-17T21:53:06.229 INFO:tasks.workunit.client.0.smithi107.stdout:4/838: symlink d3/d4a/d47/d60/l120 0 2024-05-17T21:53:06.230 INFO:tasks.workunit.client.1.smithi138.stdout:2/689: stat d3/d21/d2f/c4f 0 2024-05-17T21:53:06.232 INFO:tasks.workunit.client.0.smithi107.stdout:3/854: mkdir d6/d8/d10/d35/d36/d4c/d114 0 2024-05-17T21:53:06.234 INFO:tasks.workunit.client.1.smithi138.stdout:3/756: dwrite d1/d3/d16/d3e/d71/fd6 [0,4194304] 0 2024-05-17T21:53:06.237 INFO:tasks.workunit.client.0.smithi107.stdout:6/763: creat d9/d16/d22/daf/dc7/ff0 x:0 0 0 2024-05-17T21:53:06.245 INFO:tasks.workunit.client.0.smithi107.stdout:8/782: link dc/d21/d27/d34/d4c/d7e/f45 dc/d21/d69/df4/f11a 0 2024-05-17T21:53:06.246 INFO:tasks.workunit.client.1.smithi138.stdout:7/631: rmdir d0/d18/d40/d41/d50 39 2024-05-17T21:53:06.247 INFO:tasks.workunit.client.1.smithi138.stdout:7/632: chown d0/d18/d40/f6c 0 1 2024-05-17T21:53:06.249 INFO:tasks.workunit.client.1.smithi138.stdout:6/660: write d7/d69/db5/dc8/f82 [128900,71411] 0 2024-05-17T21:53:06.250 INFO:tasks.workunit.client.0.smithi107.stdout:4/839: dread d3/d6/d26/d36/f6e [0,4194304] 0 2024-05-17T21:53:06.251 INFO:tasks.workunit.client.0.smithi107.stdout:1/861: symlink d4/df/d1a/d26/d72/d85/df9/d103/l113 0 2024-05-17T21:53:06.252 INFO:tasks.workunit.client.0.smithi107.stdout:1/862: dread - d4/df/d4e/d59/f73 zero size 2024-05-17T21:53:06.254 INFO:tasks.workunit.client.0.smithi107.stdout:6/764: truncate d9/d16/d22/d5f/d6f/fa7 539092 0 2024-05-17T21:53:06.261 INFO:tasks.workunit.client.1.smithi138.stdout:0/629: truncate d3/d10/f7d 1117743 0 2024-05-17T21:53:06.262 INFO:tasks.workunit.client.1.smithi138.stdout:0/630: chown d3/d10/d38/d3e 24 1 2024-05-17T21:53:06.262 INFO:tasks.workunit.client.0.smithi107.stdout:8/783: dread dc/d21/f5d [0,4194304] 0 2024-05-17T21:53:06.263 INFO:tasks.workunit.client.0.smithi107.stdout:3/855: unlink d6/d8/c15 0 2024-05-17T21:53:06.269 INFO:tasks.workunit.client.0.smithi107.stdout:4/840: mkdir d3/db/d1e/d99/dd6/d121 0 2024-05-17T21:53:06.276 INFO:tasks.workunit.client.1.smithi138.stdout:5/644: rename d1/d53/d88/db9/d39/d4e/d58/d5b/d65 to d1/dd1 0 2024-05-17T21:53:06.278 INFO:tasks.workunit.client.1.smithi138.stdout:7/633: rmdir d0/d18/d9c 39 2024-05-17T21:53:06.279 INFO:tasks.workunit.client.1.smithi138.stdout:7/634: dread - d0/d18/d40/fc7 zero size 2024-05-17T21:53:06.280 INFO:tasks.workunit.client.0.smithi107.stdout:8/784: creat dc/ddb/f11b x:0 0 0 2024-05-17T21:53:06.281 INFO:tasks.workunit.client.1.smithi138.stdout:0/631: rmdir d3/d10/d6d/d81 39 2024-05-17T21:53:06.281 INFO:tasks.workunit.client.1.smithi138.stdout:6/661: dwrite d7/f22 [0,4194304] 0 2024-05-17T21:53:06.281 INFO:tasks.workunit.client.1.smithi138.stdout:4/684: write d0/d2f/d49/d60/d1f/d7e/f93 [291797,11352] 0 2024-05-17T21:53:06.283 INFO:tasks.workunit.client.0.smithi107.stdout:3/856: truncate d6/d8/d10/d25/f95 3158194 0 2024-05-17T21:53:06.284 INFO:tasks.workunit.client.0.smithi107.stdout:3/857: stat d6/d2a/f52 0 2024-05-17T21:53:06.285 INFO:tasks.workunit.client.1.smithi138.stdout:8/605: creat d0/fb6 x:0 0 0 2024-05-17T21:53:06.286 INFO:tasks.workunit.client.0.smithi107.stdout:4/841: fdatasync d3/d6/d26/d36/d8b/fbb 0 2024-05-17T21:53:06.289 INFO:tasks.workunit.client.0.smithi107.stdout:5/827: getdents d5/db/dc/d1e 0 2024-05-17T21:53:06.290 INFO:tasks.workunit.client.0.smithi107.stdout:9/846: link d0/dd/d2f/d87/cb9 d0/dd/d3d/c117 0 2024-05-17T21:53:06.290 INFO:tasks.workunit.client.1.smithi138.stdout:9/573: rmdir d6/d7/d16/d29/d38/d40/d44 39 2024-05-17T21:53:06.293 INFO:tasks.workunit.client.0.smithi107.stdout:8/785: mkdir dc/d44/d4b/d52/dcb/d11c 0 2024-05-17T21:53:06.294 INFO:tasks.workunit.client.0.smithi107.stdout:8/786: dread - dc/d21/d68/fae zero size 2024-05-17T21:53:06.296 INFO:tasks.workunit.client.1.smithi138.stdout:1/623: rename d4/db/d60/fb1 to d4/df/d4e/fc7 0 2024-05-17T21:53:06.296 INFO:tasks.workunit.client.1.smithi138.stdout:1/624: fsync d4/f3d 0 2024-05-17T21:53:06.297 INFO:tasks.workunit.client.1.smithi138.stdout:5/645: fsync d1/d2c/d63/db5/d1b/f33 0 2024-05-17T21:53:06.297 INFO:tasks.workunit.client.1.smithi138.stdout:1/625: dread - d4/d2e/d36/fac zero size 2024-05-17T21:53:06.300 INFO:tasks.workunit.client.0.smithi107.stdout:5/828: mknod d5/db/d16/d54/c112 0 2024-05-17T21:53:06.303 INFO:tasks.workunit.client.0.smithi107.stdout:4/842: dwrite d3/d6/dd5/feb [0,4194304] 0 2024-05-17T21:53:06.305 INFO:tasks.workunit.client.0.smithi107.stdout:3/858: truncate d6/d2a/d2d/f43 1492738 0 2024-05-17T21:53:06.305 INFO:tasks.workunit.client.1.smithi138.stdout:6/662: mkdir d7/d69/db5/dc8/d43/d3f/d79/ddf 0 2024-05-17T21:53:06.306 INFO:tasks.workunit.client.0.smithi107.stdout:8/787: dread - dc/d21/daa/db4/fc9 zero size 2024-05-17T21:53:06.307 INFO:tasks.workunit.client.1.smithi138.stdout:8/606: fdatasync d0/dc/f2b 0 2024-05-17T21:53:06.308 INFO:tasks.workunit.client.0.smithi107.stdout:8/788: fdatasync dc/d21/daa/dc5/ff0 0 2024-05-17T21:53:06.308 INFO:tasks.workunit.client.1.smithi138.stdout:8/607: read d0/d2f/f6e [1442768,68636] 0 2024-05-17T21:53:06.309 INFO:tasks.workunit.client.0.smithi107.stdout:8/789: truncate dc/ddb/f11b 474382 0 2024-05-17T21:53:06.310 INFO:tasks.workunit.client.0.smithi107.stdout:4/843: dwrite d3/db/f46 [0,4194304] 0 2024-05-17T21:53:06.314 INFO:tasks.workunit.client.0.smithi107.stdout:7/782: sync 2024-05-17T21:53:06.315 INFO:tasks.workunit.client.1.smithi138.stdout:0/632: dwrite d3/d10/d38/d70/d80/f8b [0,4194304] 0 2024-05-17T21:53:06.318 INFO:tasks.workunit.client.1.smithi138.stdout:1/626: symlink d4/dd/d3b/lc8 0 2024-05-17T21:53:06.325 INFO:tasks.workunit.client.0.smithi107.stdout:8/790: mkdir dc/d21/d97/d11d 0 2024-05-17T21:53:06.327 INFO:tasks.workunit.client.0.smithi107.stdout:4/844: rename d3/d4a/d47/da3/ld4 to d3/db/d35/d5e/dff/d80/l122 0 2024-05-17T21:53:06.328 INFO:tasks.workunit.client.0.smithi107.stdout:4/845: fsync d3/db/d35/f101 0 2024-05-17T21:53:06.328 INFO:tasks.workunit.client.0.smithi107.stdout:9/847: truncate d0/dd/d21/d29/dd4/fc5 1646904 0 2024-05-17T21:53:06.329 INFO:tasks.workunit.client.0.smithi107.stdout:4/846: chown d3/db/d35/ca7 146 1 2024-05-17T21:53:06.330 INFO:tasks.workunit.client.1.smithi138.stdout:8/608: symlink d0/dc/d3b/d4d/d7d/lb7 0 2024-05-17T21:53:06.331 INFO:tasks.workunit.client.1.smithi138.stdout:8/609: fdatasync d0/dc/d1a/d51/d82/f8e 0 2024-05-17T21:53:06.331 INFO:tasks.workunit.client.0.smithi107.stdout:4/847: dread - d3/db/d35/d55/d8a/ff2 zero size 2024-05-17T21:53:06.332 INFO:tasks.workunit.client.0.smithi107.stdout:4/848: chown d3/db/d1e/f9a 1372 1 2024-05-17T21:53:06.332 INFO:tasks.workunit.client.1.smithi138.stdout:3/757: mknod d1/d62/dad/c106 0 2024-05-17T21:53:06.332 INFO:tasks.workunit.client.0.smithi107.stdout:4/849: fsync d3/ddf/f103 0 2024-05-17T21:53:06.334 INFO:tasks.workunit.client.0.smithi107.stdout:9/848: read d0/dd/d21/d29/dca/ff8 [112759,128367] 0 2024-05-17T21:53:06.334 INFO:tasks.workunit.client.0.smithi107.stdout:9/849: chown d0/dd/d69 3010 1 2024-05-17T21:53:06.336 INFO:tasks.workunit.client.1.smithi138.stdout:9/574: mkdir d6/d7/d16/d29/d6c/d83/d7e/db7 0 2024-05-17T21:53:06.337 INFO:tasks.workunit.client.1.smithi138.stdout:9/575: readlink d6/d7/l32 0 2024-05-17T21:53:06.337 INFO:tasks.workunit.client.0.smithi107.stdout:8/791: mkdir dc/d21/d69/d11e 0 2024-05-17T21:53:06.338 INFO:tasks.workunit.client.1.smithi138.stdout:0/633: chown d3/d3d/l41 28130 1 2024-05-17T21:53:06.338 INFO:tasks.workunit.client.0.smithi107.stdout:8/792: stat dc/d21/daa/dc5/l100 0 2024-05-17T21:53:06.339 INFO:tasks.workunit.client.0.smithi107.stdout:8/793: dread - dc/d44/d4b/d52/d70/fa4 zero size 2024-05-17T21:53:06.341 INFO:tasks.workunit.client.1.smithi138.stdout:5/646: dread d1/d2c/d63/db5/d1b/dc5/f7c [0,4194304] 0 2024-05-17T21:53:06.343 INFO:tasks.workunit.client.0.smithi107.stdout:7/783: unlink d5/df/l1b 0 2024-05-17T21:53:06.344 INFO:tasks.workunit.client.1.smithi138.stdout:1/627: unlink d4/dd/l82 0 2024-05-17T21:53:06.346 INFO:tasks.workunit.client.0.smithi107.stdout:3/859: truncate d6/d2a/d2d/da8/daa/fe0 3592972 0 2024-05-17T21:53:06.350 INFO:tasks.workunit.client.1.smithi138.stdout:8/610: creat d0/dc/d3b/d87/fb8 x:0 0 0 2024-05-17T21:53:06.352 INFO:tasks.workunit.client.0.smithi107.stdout:7/784: truncate d5/df/d78/fa2 719766 0 2024-05-17T21:53:06.354 INFO:tasks.workunit.client.1.smithi138.stdout:9/576: read d6/d7/d16/d29/d38/d40/d44/f87 [782766,12704] 0 2024-05-17T21:53:06.354 INFO:tasks.workunit.client.0.smithi107.stdout:5/829: symlink d5/db/d16/d54/d59/l113 0 2024-05-17T21:53:06.354 INFO:tasks.workunit.client.1.smithi138.stdout:0/634: mknod d3/d10/d39/cd4 0 2024-05-17T21:53:06.356 INFO:tasks.workunit.client.0.smithi107.stdout:4/850: truncate d3/db/fc 4124053 0 2024-05-17T21:53:06.357 INFO:tasks.workunit.client.0.smithi107.stdout:7/785: dwrite d5/df/d11/d1c/d31/d39/f5e [0,4194304] 0 2024-05-17T21:53:06.358 INFO:tasks.workunit.client.0.smithi107.stdout:3/860: chown d6/d8/d10/d25/l70 26 1 2024-05-17T21:53:06.359 INFO:tasks.workunit.client.0.smithi107.stdout:7/786: read d5/d16/d3b/d45/d91/dc1/fea [11056,10608] 0 2024-05-17T21:53:06.359 INFO:tasks.workunit.client.1.smithi138.stdout:1/628: dread f2 [0,4194304] 0 2024-05-17T21:53:06.364 INFO:tasks.workunit.client.1.smithi138.stdout:8/611: write d0/dc/d3b/d4d/d7d/f7f [4268906,129949] 0 2024-05-17T21:53:06.367 INFO:tasks.workunit.client.0.smithi107.stdout:8/794: fdatasync dc/d21/d97/db7/fef 0 2024-05-17T21:53:06.370 INFO:tasks.workunit.client.1.smithi138.stdout:3/758: dread - d1/d3/d16/fa2 zero size 2024-05-17T21:53:06.370 INFO:tasks.workunit.client.1.smithi138.stdout:8/612: read d0/d11/f69 [5502440,65400] 0 2024-05-17T21:53:06.375 INFO:tasks.workunit.client.1.smithi138.stdout:9/577: unlink d6/d7/d16/d29/d6c/d83/d7e/l9d 0 2024-05-17T21:53:06.379 INFO:tasks.workunit.client.1.smithi138.stdout:5/647: mknod d1/d2c/d63/db5/d1b/dc5/d36/d68/dad/cd2 0 2024-05-17T21:53:06.382 INFO:tasks.workunit.client.1.smithi138.stdout:6/663: link d7/d69/db5/dc8/d43/d5f/d6a/d9b/caa d7/d69/db5/dc8/d60/ce0 0 2024-05-17T21:53:06.387 INFO:tasks.workunit.client.0.smithi107.stdout:4/851: write d3/db/d35/d5e/dff/f5a [2082406,19243] 0 2024-05-17T21:53:06.389 INFO:tasks.workunit.client.0.smithi107.stdout:3/861: write d6/d2a/f42 [1312857,75868] 0 2024-05-17T21:53:06.391 INFO:tasks.workunit.client.1.smithi138.stdout:0/635: rename d3/d10/d6d/d81/dc4 to d3/d10/d38/d3e/d59/d9b/dc3/dd5 0 2024-05-17T21:53:06.392 INFO:tasks.workunit.client.0.smithi107.stdout:8/795: mknod dc/d21/d97/db7/c11f 0 2024-05-17T21:53:06.398 INFO:tasks.workunit.client.1.smithi138.stdout:6/664: mknod d7/da0/ce1 0 2024-05-17T21:53:06.399 INFO:tasks.workunit.client.1.smithi138.stdout:6/665: chown d7/d69/db5/dc8/d43/d5f/d6a/d9b/db2 397730 1 2024-05-17T21:53:06.401 INFO:tasks.workunit.client.1.smithi138.stdout:8/613: unlink d0/dc/d3b/d87/lb2 0 2024-05-17T21:53:06.402 INFO:tasks.workunit.client.0.smithi107.stdout:7/787: getdents d5/d16/d3b/d45/d91/dac/d107 0 2024-05-17T21:53:06.402 INFO:tasks.workunit.client.0.smithi107.stdout:9/850: link d0/l1a d0/dd/d21/daf/l118 0 2024-05-17T21:53:06.403 INFO:tasks.workunit.client.0.smithi107.stdout:4/852: dread d3/db/d35/f81 [0,4194304] 0 2024-05-17T21:53:06.404 INFO:tasks.workunit.client.0.smithi107.stdout:4/853: stat d3/db/d35/d4f/lc0 0 2024-05-17T21:53:06.407 INFO:tasks.workunit.client.0.smithi107.stdout:8/796: rmdir dc/d21/d97 39 2024-05-17T21:53:06.408 INFO:tasks.workunit.client.0.smithi107.stdout:4/854: dread d3/d4a/d47/d60/d3c/fb0 [0,4194304] 0 2024-05-17T21:53:06.408 INFO:tasks.workunit.client.0.smithi107.stdout:1/863: sync 2024-05-17T21:53:06.408 INFO:tasks.workunit.client.0.smithi107.stdout:6/765: sync 2024-05-17T21:53:06.408 INFO:tasks.workunit.client.0.smithi107.stdout:0/918: sync 2024-05-17T21:53:06.408 INFO:tasks.workunit.client.0.smithi107.stdout:2/667: sync 2024-05-17T21:53:06.408 INFO:tasks.workunit.client.1.smithi138.stdout:6/666: creat d7/da0/dbe/fe2 x:0 0 0 2024-05-17T21:53:06.409 INFO:tasks.workunit.client.0.smithi107.stdout:4/855: chown d3/db/d35/d4f 22569800 1 2024-05-17T21:53:06.411 INFO:tasks.workunit.client.1.smithi138.stdout:8/614: unlink d0/d31/f5a 0 2024-05-17T21:53:06.413 INFO:tasks.workunit.client.1.smithi138.stdout:0/636: symlink d3/d10/d38/d3e/ld6 0 2024-05-17T21:53:06.415 INFO:tasks.workunit.client.0.smithi107.stdout:4/856: dwrite d3/d6/d26/d36/f116 [0,4194304] 0 2024-05-17T21:53:06.415 INFO:tasks.workunit.client.1.smithi138.stdout:9/578: rmdir d6/d7/d16/d29/db6 0 2024-05-17T21:53:06.416 INFO:tasks.workunit.client.0.smithi107.stdout:3/862: truncate d6/d2a/d2d/d37/fc8 2210895 0 2024-05-17T21:53:06.417 INFO:tasks.workunit.client.1.smithi138.stdout:8/615: dwrite d0/dc/d3b/f89 [4194304,4194304] 0 2024-05-17T21:53:06.417 INFO:tasks.workunit.client.1.smithi138.stdout:5/648: link d1/d2c/d63/db5/d1b/dc5/f7c d1/d2c/dbf/fd3 0 2024-05-17T21:53:06.418 INFO:tasks.workunit.client.1.smithi138.stdout:5/649: read d1/d2c/dbf/fd3 [3028364,74243] 0 2024-05-17T21:53:06.420 INFO:tasks.workunit.client.1.smithi138.stdout:3/759: getdents d1/d3 0 2024-05-17T21:53:06.421 INFO:tasks.workunit.client.1.smithi138.stdout:5/650: dread - d1/d53/d88/db9/d39/d4e/d58/fa4 zero size 2024-05-17T21:53:06.421 INFO:tasks.workunit.client.0.smithi107.stdout:8/797: creat dc/d21/daa/db4/f120 x:0 0 0 2024-05-17T21:53:06.422 INFO:tasks.workunit.client.1.smithi138.stdout:3/760: chown d1/d3/d16/d3e/c105 576725 1 2024-05-17T21:53:06.423 INFO:tasks.workunit.client.0.smithi107.stdout:2/668: dread d0/d9/d11/df/f47 [4194304,4194304] 0 2024-05-17T21:53:06.424 INFO:tasks.workunit.client.0.smithi107.stdout:0/919: mkdir d3/d39/d12f 0 2024-05-17T21:53:06.429 INFO:tasks.workunit.client.1.smithi138.stdout:0/637: creat d3/d3d/da2/fd7 x:0 0 0 2024-05-17T21:53:06.429 INFO:tasks.workunit.client.0.smithi107.stdout:6/766: mkdir d9/d58/d66/dd5/df1 0 2024-05-17T21:53:06.430 INFO:tasks.workunit.client.0.smithi107.stdout:1/864: creat d4/df/d1a/d26/d72/d85/f114 x:0 0 0 2024-05-17T21:53:06.431 INFO:tasks.workunit.client.1.smithi138.stdout:8/616: readlink d0/l19 0 2024-05-17T21:53:06.433 INFO:tasks.workunit.client.1.smithi138.stdout:5/651: write d1/d2c/d63/db5/fa [6228763,63430] 0 2024-05-17T21:53:06.435 INFO:tasks.workunit.client.0.smithi107.stdout:9/851: rename d0/dd/d1b/fcb to d0/dd/d69/dee/f119 0 2024-05-17T21:53:06.436 INFO:tasks.workunit.client.0.smithi107.stdout:9/852: write d0/dd/d21/d29/d3e/d45/fff [820081,44751] 0 2024-05-17T21:53:06.437 INFO:tasks.workunit.client.0.smithi107.stdout:4/857: symlink d3/db/d35/d55/l123 0 2024-05-17T21:53:06.438 INFO:tasks.workunit.client.0.smithi107.stdout:4/858: stat d3/d6/d26/d36/d8b 0 2024-05-17T21:53:06.438 INFO:tasks.workunit.client.0.smithi107.stdout:3/863: creat d6/d2a/d2d/d37/f115 x:0 0 0 2024-05-17T21:53:06.439 INFO:tasks.workunit.client.0.smithi107.stdout:4/859: fdatasync d3/d6/dd5/feb 0 2024-05-17T21:53:06.444 INFO:tasks.workunit.client.0.smithi107.stdout:2/669: symlink d0/d9/d1c/d37/d9f/ld8 0 2024-05-17T21:53:06.445 INFO:tasks.workunit.client.1.smithi138.stdout:5/652: rmdir d1/d53/d88/db9/d39/d4e/d58/d5b 39 2024-05-17T21:53:06.446 INFO:tasks.workunit.client.0.smithi107.stdout:8/798: dwrite dc/d21/d97/d63/f99 [0,4194304] 0 2024-05-17T21:53:06.447 INFO:tasks.workunit.client.0.smithi107.stdout:0/920: chown d3/d24/dcd/d126 227533264 1 2024-05-17T21:53:06.448 INFO:tasks.workunit.client.0.smithi107.stdout:8/799: dread - dc/d21/d97/d63/d71/d81/de5/ff6 zero size 2024-05-17T21:53:06.449 INFO:tasks.workunit.client.0.smithi107.stdout:0/921: chown d3/dc/d22/d23/d97/ld1 391949617 1 2024-05-17T21:53:06.450 INFO:tasks.workunit.client.1.smithi138.stdout:8/617: dread d0/dc/f79 [0,4194304] 0 2024-05-17T21:53:06.451 INFO:tasks.workunit.client.1.smithi138.stdout:9/579: dread d6/d7/f15 [0,4194304] 0 2024-05-17T21:53:06.451 INFO:tasks.workunit.client.0.smithi107.stdout:9/853: dread d0/dd/d21/d29/d57/f8a [0,4194304] 0 2024-05-17T21:53:06.453 INFO:tasks.workunit.client.1.smithi138.stdout:6/667: getdents d7/d8/d78 0 2024-05-17T21:53:06.457 INFO:tasks.workunit.client.1.smithi138.stdout:5/653: unlink d1/d2c/d63/db5/d1b/dc5/d36/l4d 0 2024-05-17T21:53:06.458 INFO:tasks.workunit.client.1.smithi138.stdout:8/618: dwrite d0/dc/d3b/f99 [0,4194304] 0 2024-05-17T21:53:06.460 INFO:tasks.workunit.client.1.smithi138.stdout:0/638: creat d3/d10/fd8 x:0 0 0 2024-05-17T21:53:06.468 INFO:tasks.workunit.client.1.smithi138.stdout:3/761: dwrite d1/d62/dad/f28 [0,4194304] 0 2024-05-17T21:53:06.469 INFO:tasks.workunit.client.0.smithi107.stdout:4/860: rename d3/db/d1e/d99/dd6/ce2 to d3/db/d35/d5e/dff/d80/c124 0 2024-05-17T21:53:06.469 INFO:tasks.workunit.client.1.smithi138.stdout:3/762: readlink d1/d62/dad/l8f 0 2024-05-17T21:53:06.470 INFO:tasks.workunit.client.1.smithi138.stdout:6/668: mknod d7/d69/db5/dc8/d43/d5f/d94/ce3 0 2024-05-17T21:53:06.472 INFO:tasks.workunit.client.0.smithi107.stdout:6/767: write d9/d58/fa1 [784136,3161] 0 2024-05-17T21:53:06.474 INFO:tasks.workunit.client.1.smithi138.stdout:5/654: symlink d1/d53/ld4 0 2024-05-17T21:53:06.476 INFO:tasks.workunit.client.0.smithi107.stdout:2/670: unlink d0/d26/c2d 0 2024-05-17T21:53:06.478 INFO:tasks.workunit.client.1.smithi138.stdout:0/639: unlink d3/d3d/f44 0 2024-05-17T21:53:06.479 INFO:tasks.workunit.client.1.smithi138.stdout:5/655: dread d1/d2c/d63/db5/d1b/dc5/f3b [0,4194304] 0 2024-05-17T21:53:06.479 INFO:tasks.workunit.client.0.smithi107.stdout:1/865: dread d4/df/d1a/d26/f60 [0,4194304] 0 2024-05-17T21:53:06.481 INFO:tasks.workunit.client.0.smithi107.stdout:1/866: write d4/df/d1a/d26/d72/d85/fd7 [1974210,56282] 0 2024-05-17T21:53:06.483 INFO:tasks.workunit.client.0.smithi107.stdout:2/671: dwrite d0/d9/d11/d27/f43 [4194304,4194304] 0 2024-05-17T21:53:06.485 INFO:tasks.workunit.client.1.smithi138.stdout:8/619: dread d0/f40 [0,4194304] 0 2024-05-17T21:53:06.486 INFO:tasks.workunit.client.1.smithi138.stdout:0/640: dwrite d3/f83 [0,4194304] 0 2024-05-17T21:53:06.488 INFO:tasks.workunit.client.1.smithi138.stdout:0/641: chown d3/d10/d38/d3e/d59/d9b 88022221 1 2024-05-17T21:53:06.488 INFO:tasks.workunit.client.1.smithi138.stdout:0/642: readlink d3/d10/d38/d3e/d59/l85 0 2024-05-17T21:53:06.490 INFO:tasks.workunit.client.1.smithi138.stdout:0/643: fsync d3/d10/d39/f72 0 2024-05-17T21:53:06.496 INFO:tasks.workunit.client.0.smithi107.stdout:9/854: rename d0/dd/d2f/fd9 to d0/dd/d69/dee/f11a 0 2024-05-17T21:53:06.496 INFO:tasks.workunit.client.1.smithi138.stdout:9/580: write d6/d7/d16/d29/d48/d6e/d74/d75/d76/f7f [532901,60940] 0 2024-05-17T21:53:06.497 INFO:tasks.workunit.client.1.smithi138.stdout:9/581: chown d6/d7/d16/d29/d48 183322 1 2024-05-17T21:53:06.499 INFO:tasks.workunit.client.1.smithi138.stdout:6/669: rename d7/d69/db5/dc8/d43/d5f/d6a/d77/cae to d7/d69/db5/dc8/d43/d3f/d79/ddf/ce4 0 2024-05-17T21:53:06.501 INFO:tasks.workunit.client.1.smithi138.stdout:5/656: symlink d1/dd1/dca/dd0/ld5 0 2024-05-17T21:53:06.502 INFO:tasks.workunit.client.1.smithi138.stdout:0/644: dread - d3/d10/d6d/d81/f9e zero size 2024-05-17T21:53:06.504 INFO:tasks.workunit.client.0.smithi107.stdout:0/922: creat d3/d39/d12f/f130 x:0 0 0 2024-05-17T21:53:06.506 INFO:tasks.workunit.client.0.smithi107.stdout:4/861: truncate d3/f9 370512 0 2024-05-17T21:53:06.507 INFO:tasks.workunit.client.1.smithi138.stdout:6/670: mkdir d7/d69/db5/dc8/d43/d5f/d6a/d77/de5 0 2024-05-17T21:53:06.509 INFO:tasks.workunit.client.1.smithi138.stdout:9/582: read d6/d7/f26 [302788,98078] 0 2024-05-17T21:53:06.509 INFO:tasks.workunit.client.1.smithi138.stdout:5/657: mknod d1/d46/d95/cd6 0 2024-05-17T21:53:06.510 INFO:tasks.workunit.client.0.smithi107.stdout:4/862: read d3/db/f69 [1279666,81344] 0 2024-05-17T21:53:06.511 INFO:tasks.workunit.client.0.smithi107.stdout:4/863: chown d3/db/d35/d5e/dff/cda 184749 1 2024-05-17T21:53:06.511 INFO:tasks.workunit.client.0.smithi107.stdout:4/864: chown d3/d4a/d47/d60/f79 2125 1 2024-05-17T21:53:06.511 INFO:tasks.workunit.client.0.smithi107.stdout:7/788: rename d5/df/d11/d1c/d31/d39/l89 to d5/d16/dd3/l114 0 2024-05-17T21:53:06.512 INFO:tasks.workunit.client.0.smithi107.stdout:6/768: rename d9 to d9/d16/d4d/d67/df2 22 2024-05-17T21:53:06.512 INFO:tasks.workunit.client.0.smithi107.stdout:4/865: write d3/db/d1e/db5/f113 [503908,120772] 0 2024-05-17T21:53:06.513 INFO:tasks.workunit.client.0.smithi107.stdout:7/789: chown d5/d16/d3b/d45/d91/dc1 20 1 2024-05-17T21:53:06.514 INFO:tasks.workunit.client.0.smithi107.stdout:5/830: sync 2024-05-17T21:53:06.514 INFO:tasks.workunit.client.0.smithi107.stdout:7/790: chown d5/df/d11/d1c/d2f/d65/c7e 819668 1 2024-05-17T21:53:06.514 INFO:tasks.workunit.client.1.smithi138.stdout:0/645: unlink f2 0 2024-05-17T21:53:06.515 INFO:tasks.workunit.client.0.smithi107.stdout:7/791: write d5/df/d11/d1c/d2f/d65/ff0 [949866,5153] 0 2024-05-17T21:53:06.516 INFO:tasks.workunit.client.0.smithi107.stdout:7/792: readlink d5/df/d11/l8e 0 2024-05-17T21:53:06.519 INFO:tasks.workunit.client.0.smithi107.stdout:8/800: rmdir dc/d21/d97/da0 0 2024-05-17T21:53:06.520 INFO:tasks.workunit.client.0.smithi107.stdout:5/831: dwrite d5/db/d16/d54/d59/d65/fac [0,4194304] 0 2024-05-17T21:53:06.520 INFO:tasks.workunit.client.1.smithi138.stdout:8/620: creat d0/dc/fb9 x:0 0 0 2024-05-17T21:53:06.524 INFO:tasks.workunit.client.0.smithi107.stdout:2/672: creat d0/d9/d1c/dd4/fd9 x:0 0 0 2024-05-17T21:53:06.525 INFO:tasks.workunit.client.0.smithi107.stdout:1/867: dwrite d4/df/f4d [0,4194304] 0 2024-05-17T21:53:06.530 INFO:tasks.workunit.client.1.smithi138.stdout:5/658: mknod d1/d46/cd7 0 2024-05-17T21:53:06.531 INFO:tasks.workunit.client.1.smithi138.stdout:5/659: write d1/d2c/d63/f9d [1248012,101352] 0 2024-05-17T21:53:06.536 INFO:tasks.workunit.client.0.smithi107.stdout:9/855: write d0/dd/d21/f37 [608354,118140] 0 2024-05-17T21:53:06.536 INFO:tasks.workunit.client.1.smithi138.stdout:3/763: getdents d1/d2e 0 2024-05-17T21:53:06.536 INFO:tasks.workunit.client.0.smithi107.stdout:6/769: write d9/d81/f82 [8154,108578] 0 2024-05-17T21:53:06.537 INFO:tasks.workunit.client.0.smithi107.stdout:9/856: chown d0/dd/d2f/cda 746 1 2024-05-17T21:53:06.538 INFO:tasks.workunit.client.0.smithi107.stdout:6/770: fsync d9/d16/d4d/db8/dc9/dd0/f9a 0 2024-05-17T21:53:06.538 INFO:tasks.workunit.client.0.smithi107.stdout:4/866: mknod d3/db/d35/d5e/dff/d80/dad/c125 0 2024-05-17T21:53:06.541 INFO:tasks.workunit.client.1.smithi138.stdout:9/583: write d6/d7/d16/f21 [3927951,64098] 0 2024-05-17T21:53:06.543 INFO:tasks.workunit.client.1.smithi138.stdout:6/671: creat d7/d69/db5/dc8/d43/d5f/da4/dcc/fe6 x:0 0 0 2024-05-17T21:53:06.543 INFO:tasks.workunit.client.1.smithi138.stdout:9/584: write d6/d7/d16/d29/d48/d3b/f88 [1071891,115017] 0 2024-05-17T21:53:06.544 INFO:tasks.workunit.client.0.smithi107.stdout:4/867: dread d3/d4a/d47/d60/d3a/fd0 [0,4194304] 0 2024-05-17T21:53:06.546 INFO:tasks.workunit.client.0.smithi107.stdout:8/801: write dc/d21/d27/d2f/dba/fd6 [905701,101471] 0 2024-05-17T21:53:06.547 INFO:tasks.workunit.client.1.smithi138.stdout:5/660: read d1/d2c/f7b [621017,112089] 0 2024-05-17T21:53:06.548 INFO:tasks.workunit.client.1.smithi138.stdout:0/646: unlink d3/d10/d38/f46 0 2024-05-17T21:53:06.550 INFO:tasks.workunit.client.1.smithi138.stdout:3/764: truncate d1/d62/dad/de/f8e 1013799 0 2024-05-17T21:53:06.550 INFO:tasks.workunit.client.1.smithi138.stdout:0/647: dread d3/d10/f17 [0,4194304] 0 2024-05-17T21:53:06.551 INFO:tasks.workunit.client.0.smithi107.stdout:4/868: dread d3/db/d35/d5e/dff/d80/d43/f62 [0,4194304] 0 2024-05-17T21:53:06.551 INFO:tasks.workunit.client.0.smithi107.stdout:0/923: mkdir d3/d24/dcd/d126/d131 0 2024-05-17T21:53:06.555 INFO:tasks.workunit.client.1.smithi138.stdout:8/621: dwrite d0/f2 [4194304,4194304] 0 2024-05-17T21:53:06.556 INFO:tasks.workunit.client.1.smithi138.stdout:8/622: write d0/f2 [7144643,31687] 0 2024-05-17T21:53:06.566 INFO:tasks.workunit.client.1.smithi138.stdout:6/672: creat d7/d69/db5/dc8/d43/d5f/d6a/d9b/db2/fe7 x:0 0 0 2024-05-17T21:53:06.567 INFO:tasks.workunit.client.1.smithi138.stdout:6/673: write d7/d69/fb8 [189486,15712] 0 2024-05-17T21:53:06.568 INFO:tasks.workunit.client.0.smithi107.stdout:9/857: mkdir d0/dd/d21/d29/dd4/d11b 0 2024-05-17T21:53:06.569 INFO:tasks.workunit.client.0.smithi107.stdout:6/771: mkdir d9/d58/d66/dd5/df3 0 2024-05-17T21:53:06.571 INFO:tasks.workunit.client.1.smithi138.stdout:0/648: mkdir d3/d10/d39/dd9 0 2024-05-17T21:53:06.572 INFO:tasks.workunit.client.1.smithi138.stdout:6/674: dread d7/f22 [0,4194304] 0 2024-05-17T21:53:06.573 INFO:tasks.workunit.client.0.smithi107.stdout:3/864: sync 2024-05-17T21:53:06.574 INFO:tasks.workunit.client.0.smithi107.stdout:3/865: read d6/d8/d10/d35/f100 [919915,62895] 0 2024-05-17T21:53:06.574 INFO:tasks.workunit.client.0.smithi107.stdout:3/866: readlink d6/d2a/d2d/da8/daa/df5/la0 0 2024-05-17T21:53:06.576 INFO:tasks.workunit.client.0.smithi107.stdout:5/832: mkdir d5/db/d16/d54/d59/d114 0 2024-05-17T21:53:06.578 INFO:tasks.workunit.client.1.smithi138.stdout:9/585: creat d6/d7/d16/d29/d38/d40/d44/fb8 x:0 0 0 2024-05-17T21:53:06.578 INFO:tasks.workunit.client.0.smithi107.stdout:5/833: write d5/db/d16/d2d/d51/f101 [2999476,127859] 0 2024-05-17T21:53:06.579 INFO:tasks.workunit.client.0.smithi107.stdout:8/802: unlink dc/d44/d4b/d52/d70/fc1 0 2024-05-17T21:53:06.580 INFO:tasks.workunit.client.1.smithi138.stdout:5/661: dwrite d1/d53/d96/da0/fa1 [0,4194304] 0 2024-05-17T21:53:06.580 INFO:tasks.workunit.client.0.smithi107.stdout:3/867: dwrite d6/d8/d10/d25/f99 [0,4194304] 0 2024-05-17T21:53:06.580 INFO:tasks.workunit.client.0.smithi107.stdout:5/834: readlink d5/db/d16/d2d/d40/dc2/l100 0 2024-05-17T21:53:06.586 INFO:tasks.workunit.client.1.smithi138.stdout:5/662: dread d1/d2c/d63/db5/d1b/dc5/d36/f4b [0,4194304] 0 2024-05-17T21:53:06.589 INFO:tasks.workunit.client.1.smithi138.stdout:5/663: truncate d1/d53/d96/fc6 790793 0 2024-05-17T21:53:06.589 INFO:tasks.workunit.client.0.smithi107.stdout:9/858: dread d0/dd/d69/dee/f11a [0,4194304] 0 2024-05-17T21:53:06.589 INFO:tasks.workunit.client.1.smithi138.stdout:5/664: chown d1/d46 6314066 1 2024-05-17T21:53:06.591 INFO:tasks.workunit.client.0.smithi107.stdout:7/793: write d5/df/d11/d1c/d2f/f9f [1738003,17888] 0 2024-05-17T21:53:06.592 INFO:tasks.workunit.client.0.smithi107.stdout:4/869: symlink d3/db/d35/l126 0 2024-05-17T21:53:06.596 INFO:tasks.workunit.client.1.smithi138.stdout:6/675: creat d7/d69/db5/dc8/d43/d3f/db6/fe8 x:0 0 0 2024-05-17T21:53:06.597 INFO:tasks.workunit.client.1.smithi138.stdout:6/676: write d7/da0/dbe/fd6 [804059,256] 0 2024-05-17T21:53:06.598 INFO:tasks.workunit.client.1.smithi138.stdout:9/586: mknod d6/d7/d16/d29/d48/d67/d86/cb9 0 2024-05-17T21:53:06.605 INFO:tasks.workunit.client.1.smithi138.stdout:5/665: unlink d1/d53/d88/db9/d39/d62/f7e 0 2024-05-17T21:53:06.611 INFO:tasks.workunit.client.1.smithi138.stdout:5/666: dwrite d1/d2c/d63/db5/d1b/fba [0,4194304] 0 2024-05-17T21:53:06.612 INFO:tasks.workunit.client.0.smithi107.stdout:0/924: write d3/d24/d6f/db5/def/f10c [823379,97923] 0 2024-05-17T21:53:06.619 INFO:tasks.workunit.client.0.smithi107.stdout:6/772: dwrite d9/d16/d22/daf/fb9 [0,4194304] 0 2024-05-17T21:53:06.621 INFO:tasks.workunit.client.0.smithi107.stdout:9/859: read d0/dd/d21/d29/dca/f113 [1571784,44879] 0 2024-05-17T21:53:06.622 INFO:tasks.workunit.client.1.smithi138.stdout:3/765: getdents d1/d62/dad/de 0 2024-05-17T21:53:06.623 INFO:tasks.workunit.client.0.smithi107.stdout:7/794: truncate d5/df/d11/d1d/d33/f6a 2224345 0 2024-05-17T21:53:06.623 INFO:tasks.workunit.client.0.smithi107.stdout:3/868: write d6/d2a/f5b [2954497,109688] 0 2024-05-17T21:53:06.626 INFO:tasks.workunit.client.0.smithi107.stdout:8/803: write dc/d21/d27/d2f/fe9 [101635,31529] 0 2024-05-17T21:53:06.628 INFO:tasks.workunit.client.1.smithi138.stdout:0/649: rename d3/f2b to d3/d10/d38/d3e/d59/fda 0 2024-05-17T21:53:06.628 INFO:tasks.workunit.client.0.smithi107.stdout:6/773: dwrite d9/d16/f4a [0,4194304] 0 2024-05-17T21:53:06.629 INFO:tasks.workunit.client.1.smithi138.stdout:9/587: write d6/d7/d16/d29/d48/f34 [4385887,17700] 0 2024-05-17T21:53:06.630 INFO:tasks.workunit.client.1.smithi138.stdout:5/667: mknod d1/d53/d88/db9/d39/cd8 0 2024-05-17T21:53:06.630 INFO:tasks.workunit.client.0.smithi107.stdout:8/804: dread dc/d21/f5d [0,4194304] 0 2024-05-17T21:53:06.633 INFO:tasks.workunit.client.0.smithi107.stdout:8/805: dread - dc/d21/daa/fe6 zero size 2024-05-17T21:53:06.634 INFO:tasks.workunit.client.0.smithi107.stdout:3/869: dwrite d6/d8/d3e/d3f/fba [0,4194304] 0 2024-05-17T21:53:06.636 INFO:tasks.workunit.client.0.smithi107.stdout:2/673: dwrite d0/d9/d1c/fb5 [0,4194304] 0 2024-05-17T21:53:06.638 INFO:tasks.workunit.client.0.smithi107.stdout:3/870: stat d6/d8/d10/d35 0 2024-05-17T21:53:06.638 INFO:tasks.workunit.client.0.smithi107.stdout:8/806: write dc/d21/daa/dc5/ff0 [511803,53636] 0 2024-05-17T21:53:06.638 INFO:tasks.workunit.client.0.smithi107.stdout:0/925: rename d3/d24/d6f/d9f to d3/d75/d9e/d103/d10d/d132 0 2024-05-17T21:53:06.640 INFO:tasks.workunit.client.0.smithi107.stdout:8/807: chown dc/ddb/f110 0 1 2024-05-17T21:53:06.640 INFO:tasks.workunit.client.0.smithi107.stdout:0/926: chown d3/dc/d31/c5e 93923 1 2024-05-17T21:53:06.645 INFO:tasks.workunit.client.0.smithi107.stdout:1/868: getdents d4/df/d1a/d26/d72 0 2024-05-17T21:53:06.647 INFO:tasks.workunit.client.0.smithi107.stdout:4/870: unlink d3/db/fc 0 2024-05-17T21:53:06.647 INFO:tasks.workunit.client.1.smithi138.stdout:0/650: mknod d3/d10/d39/dd9/cdb 0 2024-05-17T21:53:06.649 INFO:tasks.workunit.client.0.smithi107.stdout:1/869: write d4/df/fff [115497,16854] 0 2024-05-17T21:53:06.649 INFO:tasks.workunit.client.0.smithi107.stdout:3/871: dwrite d6/d1a/d57/ffa [0,4194304] 0 2024-05-17T21:53:06.650 INFO:tasks.workunit.client.1.smithi138.stdout:5/668: link d1/d53/f6e d1/d53/d88/fd9 0 2024-05-17T21:53:06.652 INFO:tasks.workunit.client.0.smithi107.stdout:7/795: creat d5/d16/d3b/f115 x:0 0 0 2024-05-17T21:53:06.653 INFO:tasks.workunit.client.1.smithi138.stdout:6/677: getdents d7/d8 0 2024-05-17T21:53:06.658 INFO:tasks.workunit.client.1.smithi138.stdout:3/766: getdents d1/d3/d16/d3e/d71/db2 0 2024-05-17T21:53:06.660 INFO:tasks.workunit.client.1.smithi138.stdout:0/651: mkdir d3/d10/d38/d3e/ddc 0 2024-05-17T21:53:06.661 INFO:tasks.workunit.client.0.smithi107.stdout:5/835: creat d5/db/d16/d54/f115 x:0 0 0 2024-05-17T21:53:06.662 INFO:tasks.workunit.client.1.smithi138.stdout:5/669: creat d1/d2c/d63/db5/d1b/db7/fda x:0 0 0 2024-05-17T21:53:06.662 INFO:tasks.workunit.client.0.smithi107.stdout:5/836: stat d5/db/d16/d2d/d40/d5a/d7c/d8e/f90 0 2024-05-17T21:53:06.663 INFO:tasks.workunit.client.0.smithi107.stdout:5/837: chown d5/db/d16/d2d/d40/d5a/d7c/d103 252 1 2024-05-17T21:53:06.663 INFO:tasks.workunit.client.1.smithi138.stdout:5/670: read - d1/d53/d88/db9/d39/d4e/d58/fa4 zero size 2024-05-17T21:53:06.664 INFO:tasks.workunit.client.1.smithi138.stdout:3/767: fsync d1/d3/f11 0 2024-05-17T21:53:06.664 INFO:tasks.workunit.client.0.smithi107.stdout:5/838: readlink d5/db/d16/d54/d59/l104 0 2024-05-17T21:53:06.665 INFO:tasks.workunit.client.1.smithi138.stdout:5/671: dread - d1/d2c/d63/db5/d1b/dc5/d36/d68/f6c zero size 2024-05-17T21:53:06.670 INFO:tasks.workunit.client.0.smithi107.stdout:9/860: unlink d0/dd/d1f/c7d 0 2024-05-17T21:53:06.671 INFO:tasks.workunit.client.1.smithi138.stdout:5/672: dwrite d1/d53/d96/fc6 [0,4194304] 0 2024-05-17T21:53:06.671 INFO:tasks.workunit.client.0.smithi107.stdout:9/861: dread - d0/d80/dc9/fed zero size 2024-05-17T21:53:06.671 INFO:tasks.workunit.client.0.smithi107.stdout:9/862: write d0/d80/dc9/fed [649963,120010] 0 2024-05-17T21:53:06.672 INFO:tasks.workunit.client.0.smithi107.stdout:3/872: rmdir d6/d2a/d2d/d48/d4f 39 2024-05-17T21:53:06.676 INFO:tasks.workunit.client.1.smithi138.stdout:3/768: unlink d1/d4f/d80/f86 0 2024-05-17T21:53:06.677 INFO:tasks.workunit.client.0.smithi107.stdout:4/871: symlink d3/d6/d26/d36/d37/l127 0 2024-05-17T21:53:06.678 INFO:tasks.workunit.client.0.smithi107.stdout:1/870: symlink d4/df/d14/d1d/l115 0 2024-05-17T21:53:06.678 INFO:tasks.workunit.client.0.smithi107.stdout:1/871: chown d4/df/d14/l6c 54 1 2024-05-17T21:53:06.681 INFO:tasks.workunit.client.1.smithi138.stdout:3/769: truncate d1/d62/dad/de/f27 1979649 0 2024-05-17T21:53:06.682 INFO:tasks.workunit.client.1.smithi138.stdout:3/770: truncate d1/d3/d16/d5d/feb 1745688 0 2024-05-17T21:53:06.683 INFO:tasks.workunit.client.1.smithi138.stdout:3/771: chown d1/d4f/d50/c99 1740073 1 2024-05-17T21:53:06.684 INFO:tasks.workunit.client.0.smithi107.stdout:7/796: dread d5/f83 [0,4194304] 0 2024-05-17T21:53:06.684 INFO:tasks.workunit.client.0.smithi107.stdout:1/872: chown d4/df/d14/c82 470036260 1 2024-05-17T21:53:06.684 INFO:tasks.workunit.client.0.smithi107.stdout:7/797: readlink d5/df/d11/l8e 0 2024-05-17T21:53:06.685 INFO:tasks.workunit.client.0.smithi107.stdout:6/774: creat d9/d16/d4d/d85/ff4 x:0 0 0 2024-05-17T21:53:06.686 INFO:tasks.workunit.client.0.smithi107.stdout:0/927: creat d3/f133 x:0 0 0 2024-05-17T21:53:06.687 INFO:tasks.workunit.client.0.smithi107.stdout:5/839: creat d5/d75/dc7/dd4/f116 x:0 0 0 2024-05-17T21:53:06.689 INFO:tasks.workunit.client.0.smithi107.stdout:5/840: truncate d5/db/d16/d54/d59/d65/fac 4976810 0 2024-05-17T21:53:06.689 INFO:tasks.workunit.client.0.smithi107.stdout:8/808: rename dc/d21/d97/d63/d71/d81 to dc/d44/d4b/d121 0 2024-05-17T21:53:06.689 INFO:tasks.workunit.client.0.smithi107.stdout:9/863: symlink d0/dd/d21/d29/d57/d5f/dc6/l11c 0 2024-05-17T21:53:06.690 INFO:tasks.workunit.client.0.smithi107.stdout:3/873: mkdir d6/d2a/d2d/d48/d4f/d116 0 2024-05-17T21:53:06.691 INFO:tasks.workunit.client.0.smithi107.stdout:4/872: mkdir d3/d6/d26/d36/d8b/d95/dee/dfb/d128 0 2024-05-17T21:53:06.692 INFO:tasks.workunit.client.0.smithi107.stdout:1/873: creat d4/df/d1a/d26/d72/daa/f116 x:0 0 0 2024-05-17T21:53:06.692 INFO:tasks.workunit.client.0.smithi107.stdout:7/798: write d5/df/d11/d1c/d2f/f7c [3174837,118151] 0 2024-05-17T21:53:06.693 INFO:tasks.workunit.client.0.smithi107.stdout:6/775: dread - d9/d16/d22/f92 zero size 2024-05-17T21:53:06.694 INFO:tasks.workunit.client.0.smithi107.stdout:1/874: readlink d4/df/d4e/l90 0 2024-05-17T21:53:06.694 INFO:tasks.workunit.client.0.smithi107.stdout:9/864: dread d0/dd/d21/d29/d3e/fc7 [0,4194304] 0 2024-05-17T21:53:06.695 INFO:tasks.workunit.client.0.smithi107.stdout:5/841: unlink d5/db/d39/f43 0 2024-05-17T21:53:06.695 INFO:tasks.workunit.client.0.smithi107.stdout:1/875: write d4/df/ff3 [1762171,91809] 0 2024-05-17T21:53:06.696 INFO:tasks.workunit.client.0.smithi107.stdout:8/809: readlink dc/d44/d4b/d52/d70/d76/la6 0 2024-05-17T21:53:06.698 INFO:tasks.workunit.client.0.smithi107.stdout:8/810: read dc/d21/d27/d34/df5/ffd [262255,111550] 0 2024-05-17T21:53:06.698 INFO:tasks.workunit.client.0.smithi107.stdout:8/811: read - dc/d44/d4b/d121/de5/ff6 zero size 2024-05-17T21:53:06.700 INFO:tasks.workunit.client.0.smithi107.stdout:0/928: rmdir d3/d24/dcf 39 2024-05-17T21:53:06.701 INFO:tasks.workunit.client.0.smithi107.stdout:7/799: readlink d5/lc 0 2024-05-17T21:53:06.702 INFO:tasks.workunit.client.0.smithi107.stdout:7/800: write d5/d16/d3b/d45/d91/dc1/de6/feb [704978,12550] 0 2024-05-17T21:53:06.702 INFO:tasks.workunit.client.0.smithi107.stdout:6/776: mknod d9/d16/d9e/cf5 0 2024-05-17T21:53:06.707 INFO:tasks.workunit.client.0.smithi107.stdout:1/876: creat d4/df/d1a/d26/d72/f117 x:0 0 0 2024-05-17T21:53:06.709 INFO:tasks.workunit.client.0.smithi107.stdout:3/874: link d6/d8/d3e/d3f/d90/ff4 d6/d8/d10/d25/f117 0 2024-05-17T21:53:06.710 INFO:tasks.workunit.client.0.smithi107.stdout:8/812: truncate dc/d21/f5d 1169158 0 2024-05-17T21:53:06.710 INFO:tasks.workunit.client.0.smithi107.stdout:8/813: readlink dc/d21/daa/dc5/lde 0 2024-05-17T21:53:06.712 INFO:tasks.workunit.client.0.smithi107.stdout:9/865: creat d0/dd/d69/dee/d116/f11d x:0 0 0 2024-05-17T21:53:06.713 INFO:tasks.workunit.client.0.smithi107.stdout:7/801: mkdir d5/d16/d3b/d45/d91/dc1/de6/d116 0 2024-05-17T21:53:06.715 INFO:tasks.workunit.client.0.smithi107.stdout:1/877: truncate d4/df/d14/d1d/f2f 1356893 0 2024-05-17T21:53:06.716 INFO:tasks.workunit.client.0.smithi107.stdout:8/814: dread - dc/d21/daa/fc3 zero size 2024-05-17T21:53:06.717 INFO:tasks.workunit.client.0.smithi107.stdout:9/866: fdatasync d0/d80/fcf 0 2024-05-17T21:53:06.718 INFO:tasks.workunit.client.0.smithi107.stdout:9/867: chown d0/dd/d1f/f40 1 1 2024-05-17T21:53:06.721 INFO:tasks.workunit.client.0.smithi107.stdout:0/929: rmdir d3/d75/dd0/d12d 0 2024-05-17T21:53:06.724 INFO:tasks.workunit.client.0.smithi107.stdout:9/868: fsync d0/dd/d21/dcd/fe5 0 2024-05-17T21:53:06.726 INFO:tasks.workunit.client.0.smithi107.stdout:7/802: rename d5/df/d78/fa2 to d5/df/d11/d1c/d2f/d65/da7/f117 0 2024-05-17T21:53:06.727 INFO:tasks.workunit.client.0.smithi107.stdout:0/930: rmdir d3/dc/dae 39 2024-05-17T21:53:06.727 INFO:tasks.workunit.client.0.smithi107.stdout:7/803: chown d5/df/d11/d1c/d31/d39/d92/f98 47291 1 2024-05-17T21:53:06.729 INFO:tasks.workunit.client.0.smithi107.stdout:9/869: fdatasync d0/f43 0 2024-05-17T21:53:06.730 INFO:tasks.workunit.client.0.smithi107.stdout:8/815: rename dc/d21/d27/d34/fe1 to dc/d21/d69/df4/f122 0 2024-05-17T21:53:06.731 INFO:tasks.workunit.client.0.smithi107.stdout:0/931: creat d3/dc/d22/d23/d97/d77/f134 x:0 0 0 2024-05-17T21:53:06.734 INFO:tasks.workunit.client.0.smithi107.stdout:7/804: dwrite d5/d16/dd3/fdd [4194304,4194304] 0 2024-05-17T21:53:06.745 INFO:tasks.workunit.client.0.smithi107.stdout:0/932: dread d3/d24/d6f/db5/fdd [0,4194304] 0 2024-05-17T21:53:06.755 INFO:tasks.workunit.client.0.smithi107.stdout:5/842: read d5/db/d16/d54/d59/f7f [2975827,70350] 0 2024-05-17T21:53:06.764 INFO:tasks.workunit.client.1.smithi138.stdout:6/678: write d7/f22 [2195778,55442] 0 2024-05-17T21:53:06.764 INFO:tasks.workunit.client.0.smithi107.stdout:9/870: dread d0/dd/d3d/f92 [0,4194304] 0 2024-05-17T21:53:06.768 INFO:tasks.workunit.client.1.smithi138.stdout:5/673: dread d1/d2c/d63/db5/d1b/dc5/f67 [0,4194304] 0 2024-05-17T21:53:06.772 INFO:tasks.workunit.client.1.smithi138.stdout:0/652: write d3/d10/f7d [503880,66949] 0 2024-05-17T21:53:06.784 INFO:tasks.workunit.client.1.smithi138.stdout:3/772: truncate d1/d2e/f98 472423 0 2024-05-17T21:53:06.798 INFO:tasks.workunit.client.0.smithi107.stdout:0/933: dread d3/f6e [0,4194304] 0 2024-05-17T21:53:06.799 INFO:tasks.workunit.client.1.smithi138.stdout:3/773: dread d1/d4f/f77 [0,4194304] 0 2024-05-17T21:53:06.800 INFO:tasks.workunit.client.0.smithi107.stdout:0/934: chown d3/dc/d22/d23/d97/d85/fd9 2027 1 2024-05-17T21:53:06.800 INFO:tasks.workunit.client.0.smithi107.stdout:0/935: chown d3/d58/d61 413591 1 2024-05-17T21:53:06.800 INFO:tasks.workunit.client.1.smithi138.stdout:3/774: dread - d1/d3/d49/fff zero size 2024-05-17T21:53:06.801 INFO:tasks.workunit.client.0.smithi107.stdout:2/674: sync 2024-05-17T21:53:06.801 INFO:tasks.workunit.client.0.smithi107.stdout:0/936: chown d3/d75/dc1 4127654 1 2024-05-17T21:53:06.801 INFO:tasks.workunit.client.0.smithi107.stdout:0/937: write d3/dc/d22/ff2 [268103,77155] 0 2024-05-17T21:53:06.803 INFO:tasks.workunit.client.0.smithi107.stdout:5/843: symlink d5/db/d16/d2d/l117 0 2024-05-17T21:53:06.804 INFO:tasks.workunit.client.0.smithi107.stdout:5/844: chown d5/db/d16/d2d/d4d/c5e 39004 1 2024-05-17T21:53:06.804 INFO:tasks.workunit.client.0.smithi107.stdout:6/777: truncate d9/d16/d4d/d67/f86 1307846 0 2024-05-17T21:53:06.806 INFO:tasks.workunit.client.0.smithi107.stdout:1/878: write d4/df/d1a/d26/d72/fbe [673440,44283] 0 2024-05-17T21:53:06.807 INFO:tasks.workunit.client.0.smithi107.stdout:2/675: rmdir d0/d9/d1c/d3e 39 2024-05-17T21:53:06.807 INFO:tasks.workunit.client.0.smithi107.stdout:0/938: dread d3/d39/f49 [0,4194304] 0 2024-05-17T21:53:06.808 INFO:tasks.workunit.client.1.smithi138.stdout:6/679: mkdir d7/d8/de9 0 2024-05-17T21:53:06.810 INFO:tasks.workunit.client.0.smithi107.stdout:9/871: link d0/dd/d21/dcd/fe5 d0/dd/d21/d29/d57/d5f/dc6/f11e 0 2024-05-17T21:53:06.811 INFO:tasks.workunit.client.1.smithi138.stdout:5/674: write d1/dd1/f8f [182937,16226] 0 2024-05-17T21:53:06.812 INFO:tasks.workunit.client.1.smithi138.stdout:5/675: truncate d1/d46/f78 1001640 0 2024-05-17T21:53:06.813 INFO:tasks.workunit.client.1.smithi138.stdout:5/676: fsync d1/d2c/d63/db5/fa 0 2024-05-17T21:53:06.814 INFO:tasks.workunit.client.1.smithi138.stdout:0/653: rename d3/d10/d6d/d81/l87 to d3/d10/d6d/d9f/da8/daa/ldd 0 2024-05-17T21:53:06.818 INFO:tasks.workunit.client.1.smithi138.stdout:6/680: symlink d7/d69/db5/dc8/d43/d5f/d6a/d77/lea 0 2024-05-17T21:53:06.818 INFO:tasks.workunit.client.0.smithi107.stdout:8/816: write dc/d44/fbe [363062,41887] 0 2024-05-17T21:53:06.882 INFO:tasks.workunit.client.0.smithi107.stdout:4/873: sync 2024-05-17T21:53:06.882 INFO:tasks.workunit.client.0.smithi107.stdout:3/875: sync 2024-05-17T21:53:06.905 INFO:tasks.workunit.client.0.smithi107.stdout:6/778: rename d9/cb0 to d9/d16/d4d/db8/dc9/cf6 0 2024-05-17T21:53:06.907 INFO:tasks.workunit.client.1.smithi138.stdout:0/654: creat d3/d10/d39/fde x:0 0 0 2024-05-17T21:53:06.909 INFO:tasks.workunit.client.1.smithi138.stdout:6/681: mknod d7/d69/db5/dc8/d43/d63/ceb 0 2024-05-17T21:53:06.910 INFO:tasks.workunit.client.0.smithi107.stdout:7/805: dwrite d5/d16/d3b/d45/d91/dc1/f106 [0,4194304] 0 2024-05-17T21:53:06.911 INFO:tasks.workunit.client.1.smithi138.stdout:0/655: mknod d3/d10/d38/d3e/d59/d9b/dc3/dd5/cdf 0 2024-05-17T21:53:06.912 INFO:tasks.workunit.client.0.smithi107.stdout:0/939: truncate d3/dc/d22/f3e 13279 0 2024-05-17T21:53:06.914 INFO:tasks.workunit.client.0.smithi107.stdout:9/872: unlink d0/dd/d21/d29/l2d 0 2024-05-17T21:53:06.915 INFO:tasks.workunit.client.0.smithi107.stdout:9/873: write d0/fa5 [3122727,30212] 0 2024-05-17T21:53:06.916 INFO:tasks.workunit.client.0.smithi107.stdout:0/940: read d3/dc/d22/ffc [3268348,58318] 0 2024-05-17T21:53:06.916 INFO:tasks.workunit.client.0.smithi107.stdout:5/845: mkdir d5/db/d16/d54/da0/d118 0 2024-05-17T21:53:06.917 INFO:tasks.workunit.client.1.smithi138.stdout:6/682: dwrite d7/d69/db5/dc8/d43/d5f/d6a/fd3 [0,4194304] 0 2024-05-17T21:53:06.918 INFO:tasks.workunit.client.0.smithi107.stdout:9/874: write d0/dd/d21/d29/dd4/fa0 [298634,99490] 0 2024-05-17T21:53:06.918 INFO:tasks.workunit.client.1.smithi138.stdout:6/683: write d7/da0/dbe/fe2 [363623,99587] 0 2024-05-17T21:53:06.920 INFO:tasks.workunit.client.1.smithi138.stdout:6/684: chown d7/da0/fd2 14309033 1 2024-05-17T21:53:06.921 INFO:tasks.workunit.client.0.smithi107.stdout:1/879: dread d4/df/d14/d1d/f27 [0,4194304] 0 2024-05-17T21:53:06.922 INFO:tasks.workunit.client.1.smithi138.stdout:6/685: dread - d7/d69/db5/dc8/d43/d5f/da4/dcc/fe6 zero size 2024-05-17T21:53:06.924 INFO:tasks.workunit.client.0.smithi107.stdout:8/817: rename dc/d21/d97/l57 to dc/d21/d69/l123 0 2024-05-17T21:53:06.929 INFO:tasks.workunit.client.1.smithi138.stdout:6/686: rename d7/d69/db5/dc8/d43/d5f/da4/db0/lbc to d7/da0/dbe/lec 0 2024-05-17T21:53:06.931 INFO:tasks.workunit.client.0.smithi107.stdout:8/818: dwrite dc/d21/daa/dc5/ff0 [0,4194304] 0 2024-05-17T21:53:06.932 INFO:tasks.workunit.client.0.smithi107.stdout:7/806: mkdir d5/df/dd7/d118 0 2024-05-17T21:53:06.935 INFO:tasks.workunit.client.1.smithi138.stdout:6/687: dwrite d7/d69/db5/dc8/d43/d5f/d6a/d77/f85 [0,4194304] 0 2024-05-17T21:53:06.937 INFO:tasks.workunit.client.0.smithi107.stdout:5/846: symlink d5/db/d16/d2a/d73/l119 0 2024-05-17T21:53:06.937 INFO:tasks.workunit.client.1.smithi138.stdout:6/688: write d7/d8/f50 [4035963,24731] 0 2024-05-17T21:53:06.943 INFO:tasks.workunit.client.1.smithi138.stdout:6/689: dread d7/da0/dbe/fe2 [0,4194304] 0 2024-05-17T21:53:06.943 INFO:tasks.workunit.client.0.smithi107.stdout:0/941: dread d3/d75/dd0/f127 [4194304,4194304] 0 2024-05-17T21:53:06.945 INFO:tasks.workunit.client.0.smithi107.stdout:0/942: chown d3/d58/f84 16269583 1 2024-05-17T21:53:06.947 INFO:tasks.workunit.client.0.smithi107.stdout:0/943: readlink d3/d24/d6f/db5/dd7/l11a 0 2024-05-17T21:53:06.949 INFO:tasks.workunit.client.0.smithi107.stdout:2/676: creat d0/d9/d11/fda x:0 0 0 2024-05-17T21:53:06.950 INFO:tasks.workunit.client.0.smithi107.stdout:2/677: write d0/d9/d11/fda [176115,14006] 0 2024-05-17T21:53:06.951 INFO:tasks.workunit.client.0.smithi107.stdout:8/819: read dc/d44/d4b/d121/f9b [1163608,38164] 0 2024-05-17T21:53:06.955 INFO:tasks.workunit.client.1.smithi138.stdout:6/690: getdents d7/d69/db5/dc8/d43/d3f/d79/ddf 0 2024-05-17T21:53:06.956 INFO:tasks.workunit.client.0.smithi107.stdout:1/880: dread d4/df/ff3 [0,4194304] 0 2024-05-17T21:53:06.957 INFO:tasks.workunit.client.0.smithi107.stdout:9/875: rename d0/dd/f46 to d0/dd/d21/d29/dd4/f11f 0 2024-05-17T21:53:06.959 INFO:tasks.workunit.client.0.smithi107.stdout:0/944: fsync d3/d75/d9e/d103/d10d/d132/fbd 0 2024-05-17T21:53:06.963 INFO:tasks.workunit.client.0.smithi107.stdout:7/807: truncate d5/d16/f23 145278 0 2024-05-17T21:53:06.966 INFO:tasks.workunit.client.0.smithi107.stdout:2/678: dwrite d0/d26/d64/f8d [0,4194304] 0 2024-05-17T21:53:06.967 INFO:tasks.workunit.client.0.smithi107.stdout:2/679: chown d0/d9/d11/d15/d99/fbb 120091525 1 2024-05-17T21:53:06.970 INFO:tasks.workunit.client.0.smithi107.stdout:2/680: dread d0/d9/d11/fda [0,4194304] 0 2024-05-17T21:53:06.975 INFO:tasks.workunit.client.0.smithi107.stdout:2/681: dwrite d0/d9/d1c/f96 [4194304,4194304] 0 2024-05-17T21:53:06.988 INFO:tasks.workunit.client.0.smithi107.stdout:1/881: fsync d4/df/d14/f4c 0 2024-05-17T21:53:06.995 INFO:tasks.workunit.client.0.smithi107.stdout:8/820: mkdir dc/d21/d27/d83/d124 0 2024-05-17T21:53:06.995 INFO:tasks.workunit.client.0.smithi107.stdout:8/821: readlink dc/d44/d4b/d121/la5 0 2024-05-17T21:53:06.996 INFO:tasks.workunit.client.0.smithi107.stdout:7/808: creat d5/d16/d3b/ddf/f119 x:0 0 0 2024-05-17T21:53:06.996 INFO:tasks.workunit.client.0.smithi107.stdout:8/822: dread - dc/d44/d4b/d52/d70/fa4 zero size 2024-05-17T21:53:06.997 INFO:tasks.workunit.client.0.smithi107.stdout:8/823: chown dc/d21/d27/d34/d4c/d7e 123752818 1 2024-05-17T21:53:07.002 INFO:tasks.workunit.client.0.smithi107.stdout:8/824: chown dc/d44/d4b/d121/de5/cb5 0 1 2024-05-17T21:53:07.004 INFO:tasks.workunit.client.0.smithi107.stdout:2/682: truncate d0/d9/d11/d18/dad/fc3 3533767 0 2024-05-17T21:53:07.008 INFO:tasks.workunit.client.0.smithi107.stdout:2/683: dread d0/d9/d11/f5c [0,4194304] 0 2024-05-17T21:53:07.012 INFO:tasks.workunit.client.0.smithi107.stdout:4/874: mknod d3/d4a/d47/d60/d3c/d58/c129 0 2024-05-17T21:53:07.012 INFO:tasks.workunit.client.0.smithi107.stdout:1/882: getdents d4/df/d1a/d41 0 2024-05-17T21:53:07.013 INFO:tasks.workunit.client.0.smithi107.stdout:2/684: mkdir d0/d26/d64/ddb 0 2024-05-17T21:53:07.013 INFO:tasks.workunit.client.0.smithi107.stdout:1/883: dread - d4/df/d14/d2e/dd3/df6/f106 zero size 2024-05-17T21:53:07.016 INFO:tasks.workunit.client.0.smithi107.stdout:2/685: creat d0/d9/d11/d18/dad/db8/fdc x:0 0 0 2024-05-17T21:53:07.016 INFO:tasks.workunit.client.1.smithi138.stdout:3/775: write d1/d3/d16/d3e/d71/f7b [2505030,107542] 0 2024-05-17T21:53:07.017 INFO:tasks.workunit.client.0.smithi107.stdout:4/875: creat d3/d4a/d47/d60/d3c/d58/f12a x:0 0 0 2024-05-17T21:53:07.018 INFO:tasks.workunit.client.0.smithi107.stdout:4/876: write d3/d6/d26/d36/d37/d85/fec [203394,42851] 0 2024-05-17T21:53:07.019 INFO:tasks.workunit.client.0.smithi107.stdout:1/884: mknod d4/df/d1a/d41/db2/dc8/c118 0 2024-05-17T21:53:07.019 INFO:tasks.workunit.client.0.smithi107.stdout:1/885: readlink d4/d78/lf7 0 2024-05-17T21:53:07.021 INFO:tasks.workunit.client.0.smithi107.stdout:2/686: read d0/d9/d11/df/f35 [2199772,108796] 0 2024-05-17T21:53:07.022 INFO:tasks.workunit.client.0.smithi107.stdout:4/877: unlink d3/d6/d26/d3b/d91/fe9 0 2024-05-17T21:53:07.025 INFO:tasks.workunit.client.1.smithi138.stdout:3/776: dwrite d1/fc3 [0,4194304] 0 2024-05-17T21:53:07.027 INFO:tasks.workunit.client.0.smithi107.stdout:2/687: rmdir d0/d93 39 2024-05-17T21:53:07.032 INFO:tasks.workunit.client.0.smithi107.stdout:3/876: write d6/d8/d10/d35/d36/d4c/f65 [4825081,115809] 0 2024-05-17T21:53:07.033 INFO:tasks.workunit.client.0.smithi107.stdout:3/877: stat d6/c2e 0 2024-05-17T21:53:07.036 INFO:tasks.workunit.client.1.smithi138.stdout:3/777: rename d1/d4d to d1/d62/d9b/d107 0 2024-05-17T21:53:07.042 INFO:tasks.workunit.client.0.smithi107.stdout:4/878: rmdir d3/db/d35/d109 0 2024-05-17T21:53:07.045 INFO:tasks.workunit.client.1.smithi138.stdout:3/778: link d1/d3/f10 d1/d62/dad/de/f108 0 2024-05-17T21:53:07.047 INFO:tasks.workunit.client.0.smithi107.stdout:6/779: write d9/d58/d7b/fae [392509,81948] 0 2024-05-17T21:53:07.053 INFO:tasks.workunit.client.1.smithi138.stdout:6/691: write d7/d69/db5/dc8/d43/d3f/d79/f93 [914529,108200] 0 2024-05-17T21:53:07.054 INFO:tasks.workunit.client.1.smithi138.stdout:6/692: write d7/d69/db5/dc8/d43/d3f/d79/fbb [1150017,84578] 0 2024-05-17T21:53:07.057 INFO:tasks.workunit.client.0.smithi107.stdout:5/847: truncate d5/db/d16/d54/d59/d65/fd6 2712706 0 2024-05-17T21:53:07.059 INFO:tasks.workunit.client.0.smithi107.stdout:4/879: link d3/d4a/f73 d3/db/d1e/db5/f12b 0 2024-05-17T21:53:07.060 INFO:tasks.workunit.client.1.smithi138.stdout:3/779: link d1/d62/dad/d72/l9d d1/dc0/l109 0 2024-05-17T21:53:07.060 INFO:tasks.workunit.client.1.smithi138.stdout:3/780: chown d1/d4f/d50/dbb 38 1 2024-05-17T21:53:07.061 INFO:tasks.workunit.client.0.smithi107.stdout:0/945: write d3/f7b [3586174,82136] 0 2024-05-17T21:53:07.062 INFO:tasks.workunit.client.0.smithi107.stdout:5/848: stat d5/c8b 0 2024-05-17T21:53:07.063 INFO:tasks.workunit.client.0.smithi107.stdout:6/780: link d9/d16/ca6 d9/d16/d22/d5f/d6e/dd6/cf7 0 2024-05-17T21:53:07.063 INFO:tasks.workunit.client.0.smithi107.stdout:0/946: read d3/dc/d22/d23/f44 [6527711,16830] 0 2024-05-17T21:53:07.064 INFO:tasks.workunit.client.1.smithi138.stdout:3/781: readlink d1/d62/dad/d5c/d9c/lc2 0 2024-05-17T21:53:07.065 INFO:tasks.workunit.client.0.smithi107.stdout:0/947: read d3/d39/fb0 [1056552,34314] 0 2024-05-17T21:53:07.065 INFO:tasks.workunit.client.0.smithi107.stdout:0/948: dread - d3/d24/d6f/f115 zero size 2024-05-17T21:53:07.080 INFO:tasks.workunit.client.0.smithi107.stdout:5/849: dread d5/db/d16/d2d/d40/fed [0,4194304] 0 2024-05-17T21:53:07.080 INFO:tasks.workunit.client.1.smithi138.stdout:6/693: getdents d7/da0/dbe 0 2024-05-17T21:53:07.081 INFO:tasks.workunit.client.0.smithi107.stdout:5/850: write d5/db/d16/d2d/d4d/d6d/fe0 [627598,38080] 0 2024-05-17T21:53:07.084 INFO:tasks.workunit.client.0.smithi107.stdout:1/886: write d4/df/d1a/d26/d72/daa/fa1 [1364777,19170] 0 2024-05-17T21:53:07.084 INFO:tasks.workunit.client.1.smithi138.stdout:3/782: link d1/d3/d16/lbd d1/d4f/d80/de5/l10a 0 2024-05-17T21:53:07.085 INFO:tasks.workunit.client.1.smithi138.stdout:3/783: chown d1/d62/dad/d72/dda/ce3 189 1 2024-05-17T21:53:07.087 INFO:tasks.workunit.client.1.smithi138.stdout:6/694: rename d7/d69/db5/dc8/d34/da6/c2f to d7/da0/dbe/ced 0 2024-05-17T21:53:07.088 INFO:tasks.workunit.client.0.smithi107.stdout:5/851: symlink d5/db/dc/dc5/l11a 0 2024-05-17T21:53:07.090 INFO:tasks.workunit.client.0.smithi107.stdout:4/880: link d3/d6/d26/d36/d8b/d95/dee/l11c d3/d6/d26/l12c 0 2024-05-17T21:53:07.093 INFO:tasks.workunit.client.0.smithi107.stdout:2/688: dwrite d0/d9/d11/d18/f20 [0,4194304] 0 2024-05-17T21:53:07.093 INFO:tasks.workunit.client.1.smithi138.stdout:3/784: unlink d1/d62/dad/de/f27 0 2024-05-17T21:53:07.094 INFO:tasks.workunit.client.0.smithi107.stdout:3/878: dwrite d6/d2a/d2d/f6c [0,4194304] 0 2024-05-17T21:53:07.095 INFO:tasks.workunit.client.0.smithi107.stdout:2/689: write d0/d9/d11/d18/fd5 [538738,36505] 0 2024-05-17T21:53:07.095 INFO:tasks.workunit.client.0.smithi107.stdout:1/887: symlink d4/da7/daf/l119 0 2024-05-17T21:53:07.097 INFO:tasks.workunit.client.0.smithi107.stdout:0/949: creat d3/d75/d9e/d103/d10d/d132/f135 x:0 0 0 2024-05-17T21:53:07.105 INFO:tasks.workunit.client.0.smithi107.stdout:2/690: stat d0/d9/d11/l3b 0 2024-05-17T21:53:07.105 INFO:tasks.workunit.client.1.smithi138.stdout:3/785: getdents d1/d62/dad/d72 0 2024-05-17T21:53:07.106 INFO:tasks.workunit.client.1.smithi138.stdout:3/786: chown d1/d3/d16 0 1 2024-05-17T21:53:07.107 INFO:tasks.workunit.client.0.smithi107.stdout:3/879: mknod d6/d2a/d2d/d37/dc6/c118 0 2024-05-17T21:53:07.108 INFO:tasks.workunit.client.0.smithi107.stdout:2/691: truncate d0/d9/d11/d18/dad/db8/fdc 208441 0 2024-05-17T21:53:07.112 INFO:tasks.workunit.client.1.smithi138.stdout:3/787: rename d1/d62/d9b/d107/cd8 to d1/d4f/d50/dbb/c10b 0 2024-05-17T21:53:07.115 INFO:tasks.workunit.client.1.smithi138.stdout:7/635: sync 2024-05-17T21:53:07.115 INFO:tasks.workunit.client.1.smithi138.stdout:2/690: sync 2024-05-17T21:53:07.116 INFO:tasks.workunit.client.1.smithi138.stdout:7/636: write d0/f91 [679584,107371] 0 2024-05-17T21:53:07.118 INFO:tasks.workunit.client.1.smithi138.stdout:3/788: fdatasync d1/fa5 0 2024-05-17T21:53:07.119 INFO:tasks.workunit.client.1.smithi138.stdout:3/789: chown d1/d4f/d50/c99 4792 1 2024-05-17T21:53:07.120 INFO:tasks.workunit.client.1.smithi138.stdout:7/637: creat d0/d18/d64/fca x:0 0 0 2024-05-17T21:53:07.121 INFO:tasks.workunit.client.1.smithi138.stdout:2/691: fdatasync d3/d21/d2f/d3e/d55/d99/d84/d9f/fcb 0 2024-05-17T21:53:07.122 INFO:tasks.workunit.client.1.smithi138.stdout:2/692: readlink d3/l51 0 2024-05-17T21:53:07.123 INFO:tasks.workunit.client.1.smithi138.stdout:3/790: mkdir d1/d3/d16/d3e/d71/d10c 0 2024-05-17T21:53:07.123 INFO:tasks.workunit.client.1.smithi138.stdout:7/638: mknod d0/d18/d40/d5f/ccb 0 2024-05-17T21:53:07.127 INFO:tasks.workunit.client.1.smithi138.stdout:7/639: write d0/d18/d40/d41/d50/f61 [2970203,120412] 0 2024-05-17T21:53:07.127 INFO:tasks.workunit.client.0.smithi107.stdout:9/876: dwrite d0/dd/d69/db2/fbd [0,4194304] 0 2024-05-17T21:53:07.134 INFO:tasks.workunit.client.1.smithi138.stdout:7/640: dread d0/d18/d40/f58 [0,4194304] 0 2024-05-17T21:53:07.139 INFO:tasks.workunit.client.1.smithi138.stdout:3/791: dread d1/d62/dad/de/f108 [4194304,4194304] 0 2024-05-17T21:53:07.140 INFO:tasks.workunit.client.1.smithi138.stdout:3/792: read d1/fc3 [3247031,30847] 0 2024-05-17T21:53:07.142 INFO:tasks.workunit.client.1.smithi138.stdout:7/641: symlink d0/d18/d40/d41/d51/d66/da8/lcc 0 2024-05-17T21:53:07.144 INFO:tasks.workunit.client.1.smithi138.stdout:3/793: dread d1/fc3 [0,4194304] 0 2024-05-17T21:53:07.145 INFO:tasks.workunit.client.1.smithi138.stdout:3/794: readlink d1/d3/d16/d3e/d71/l79 0 2024-05-17T21:53:07.147 INFO:tasks.workunit.client.1.smithi138.stdout:7/642: creat d0/d18/d64/fcd x:0 0 0 2024-05-17T21:53:07.147 INFO:tasks.workunit.client.0.smithi107.stdout:5/852: dread d5/db/d16/d54/d59/d65/d48/f8a [0,4194304] 0 2024-05-17T21:53:07.148 INFO:tasks.workunit.client.1.smithi138.stdout:7/643: dread - d0/d18/d40/d41/d51/d81/fa3 zero size 2024-05-17T21:53:07.148 INFO:tasks.workunit.client.0.smithi107.stdout:4/881: write d3/db/d35/d5e/f63 [229290,89762] 0 2024-05-17T21:53:07.150 INFO:tasks.workunit.client.0.smithi107.stdout:1/888: write d4/df/d1a/d26/fcf [685019,92734] 0 2024-05-17T21:53:07.152 INFO:tasks.workunit.client.1.smithi138.stdout:7/644: creat d0/d18/d40/d41/d51/fce x:0 0 0 2024-05-17T21:53:07.153 INFO:tasks.workunit.client.1.smithi138.stdout:7/645: readlink d0/d18/d40/l52 0 2024-05-17T21:53:07.154 INFO:tasks.workunit.client.1.smithi138.stdout:7/646: dread d0/d39/f3d [0,4194304] 0 2024-05-17T21:53:07.167 INFO:tasks.workunit.client.1.smithi138.stdout:2/693: dwrite d3/d21/d2f/d3e/d58/d77/f86 [0,4194304] 0 2024-05-17T21:53:07.172 INFO:tasks.workunit.client.1.smithi138.stdout:3/795: write d1/d62/dad/f64 [1163366,113574] 0 2024-05-17T21:53:07.172 INFO:tasks.workunit.client.1.smithi138.stdout:3/796: readlink d1/d3/l17 0 2024-05-17T21:53:07.172 INFO:tasks.workunit.client.1.smithi138.stdout:2/694: creat d3/d21/d2f/d3e/d55/d99/dd2/fe4 x:0 0 0 2024-05-17T21:53:07.172 INFO:tasks.workunit.client.1.smithi138.stdout:7/647: dread d0/d18/d40/d41/f5b [0,4194304] 0 2024-05-17T21:53:07.173 INFO:tasks.workunit.client.1.smithi138.stdout:7/648: chown d0/d18/d40/d41/f5b 233310972 1 2024-05-17T21:53:07.174 INFO:tasks.workunit.client.1.smithi138.stdout:2/695: mknod d3/da0/daa/dce/ce5 0 2024-05-17T21:53:07.175 INFO:tasks.workunit.client.1.smithi138.stdout:7/649: chown d0/d18/d40/d41/d51/d66/lb3 2071787 1 2024-05-17T21:53:07.175 INFO:tasks.workunit.client.1.smithi138.stdout:2/696: chown d3/d21/d2f/d3e/l4d 0 1 2024-05-17T21:53:07.176 INFO:tasks.workunit.client.1.smithi138.stdout:2/697: dread - d3/d21/fe2 zero size 2024-05-17T21:53:07.177 INFO:tasks.workunit.client.1.smithi138.stdout:3/797: rename d1/d3/d16/d3a to d1/d3/d16/d3e/d71/d10d 0 2024-05-17T21:53:07.180 INFO:tasks.workunit.client.1.smithi138.stdout:7/650: mkdir d0/d18/d64/d7c/da2/dc5/dcf 0 2024-05-17T21:53:07.180 INFO:tasks.workunit.client.1.smithi138.stdout:7/651: dread - d0/d18/d64/fcd zero size 2024-05-17T21:53:07.182 INFO:tasks.workunit.client.1.smithi138.stdout:2/698: truncate d3/f27 166841 0 2024-05-17T21:53:07.183 INFO:tasks.workunit.client.1.smithi138.stdout:7/652: mknod d0/d18/d40/d41/d51/d66/da8/cd0 0 2024-05-17T21:53:07.184 INFO:tasks.workunit.client.1.smithi138.stdout:2/699: truncate d3/d21/d2f/d3e/d55/d73/f97 219119 0 2024-05-17T21:53:07.189 INFO:tasks.workunit.client.1.smithi138.stdout:7/653: dwrite d0/f91 [0,4194304] 0 2024-05-17T21:53:07.190 INFO:tasks.workunit.client.1.smithi138.stdout:7/654: readlink d0/d18/d40/d41/d50/lb7 0 2024-05-17T21:53:07.195 INFO:tasks.workunit.client.1.smithi138.stdout:7/655: mkdir d0/d18/dbf/dd1 0 2024-05-17T21:53:07.196 INFO:tasks.workunit.client.1.smithi138.stdout:2/700: write d3/f61 [5005448,72917] 0 2024-05-17T21:53:07.198 INFO:tasks.workunit.client.1.smithi138.stdout:3/798: dwrite d1/d3/f1c [4194304,4194304] 0 2024-05-17T21:53:07.203 INFO:tasks.workunit.client.1.smithi138.stdout:2/701: creat d3/d21/d2f/d3e/d55/d99/d93/fe6 x:0 0 0 2024-05-17T21:53:07.209 INFO:tasks.workunit.client.1.smithi138.stdout:3/799: write d1/d3/d16/d3e/d71/f9a [105953,105080] 0 2024-05-17T21:53:07.210 INFO:tasks.workunit.client.1.smithi138.stdout:2/702: dwrite d3/d21/d2f/d3e/d55/d99/f8f [0,4194304] 0 2024-05-17T21:53:07.212 INFO:tasks.workunit.client.1.smithi138.stdout:3/800: mkdir d1/d62/dad/de/d10e 0 2024-05-17T21:53:07.213 INFO:tasks.workunit.client.1.smithi138.stdout:2/703: chown d3/l28 4296 1 2024-05-17T21:53:07.217 INFO:tasks.workunit.client.1.smithi138.stdout:3/801: symlink d1/dee/l10f 0 2024-05-17T21:53:07.220 INFO:tasks.workunit.client.0.smithi107.stdout:3/880: getdents d6/d8/d10/d25/d80 0 2024-05-17T21:53:07.220 INFO:tasks.workunit.client.1.smithi138.stdout:3/802: creat d1/d3/d16/d3e/d71/f110 x:0 0 0 2024-05-17T21:53:07.220 INFO:tasks.workunit.client.1.smithi138.stdout:2/704: dwrite d3/f61 [0,4194304] 0 2024-05-17T21:53:07.220 INFO:tasks.workunit.client.0.smithi107.stdout:5/853: symlink d5/db/d16/d2d/d40/d5a/d7c/d8e/l11b 0 2024-05-17T21:53:07.220 INFO:tasks.workunit.client.0.smithi107.stdout:0/950: write d3/dc/d22/f7d [9267824,80406] 0 2024-05-17T21:53:07.221 INFO:tasks.workunit.client.0.smithi107.stdout:9/877: creat d0/dd/f120 x:0 0 0 2024-05-17T21:53:07.222 INFO:tasks.workunit.client.0.smithi107.stdout:2/692: write d0/d9/d11/fda [289798,46914] 0 2024-05-17T21:53:07.224 INFO:tasks.workunit.client.1.smithi138.stdout:7/656: dread d0/d18/d40/d9a/fa5 [0,4194304] 0 2024-05-17T21:53:07.225 INFO:tasks.workunit.client.1.smithi138.stdout:7/657: stat d0/d18/d40/d9a 0 2024-05-17T21:53:07.228 INFO:tasks.workunit.client.0.smithi107.stdout:4/882: truncate d3/d4a/d47/d60/d3a/f105 975313 0 2024-05-17T21:53:07.228 INFO:tasks.workunit.client.0.smithi107.stdout:1/889: read d4/df/d14/d1d/f2c [1695433,103466] 0 2024-05-17T21:53:07.231 INFO:tasks.workunit.client.1.smithi138.stdout:2/705: rename d3/f1c to d3/d21/d2f/d3e/fe7 0 2024-05-17T21:53:07.232 INFO:tasks.workunit.client.0.smithi107.stdout:0/951: dread d3/dc/d22/d3d/dc7/fcb [0,4194304] 0 2024-05-17T21:53:07.232 INFO:tasks.workunit.client.1.smithi138.stdout:3/803: link d1/d3/d49/ddb/le8 d1/l111 0 2024-05-17T21:53:07.233 INFO:tasks.workunit.client.0.smithi107.stdout:5/854: dwrite d5/db/d16/d54/d59/d65/d48/d89/fa8 [4194304,4194304] 0 2024-05-17T21:53:07.233 INFO:tasks.workunit.client.1.smithi138.stdout:3/804: write d1/d4f/d80/fe6 [580079,14268] 0 2024-05-17T21:53:07.235 INFO:tasks.workunit.client.1.smithi138.stdout:3/805: read d1/d3/f1b [1103261,107779] 0 2024-05-17T21:53:07.236 INFO:tasks.workunit.client.1.smithi138.stdout:3/806: dread - d1/d62/dad/d5c/d9c/ff0 zero size 2024-05-17T21:53:07.238 INFO:tasks.workunit.client.0.smithi107.stdout:4/883: write d3/fa [1059689,124195] 0 2024-05-17T21:53:07.240 INFO:tasks.workunit.client.0.smithi107.stdout:9/878: dwrite d0/dd/d21/d29/d3e/d45/f67 [0,4194304] 0 2024-05-17T21:53:07.240 INFO:tasks.workunit.client.1.smithi138.stdout:3/807: dread d1/d62/dad/fe1 [0,4194304] 0 2024-05-17T21:53:07.240 INFO:tasks.workunit.client.1.smithi138.stdout:7/658: write d0/d18/d40/d41/f77 [619901,23589] 0 2024-05-17T21:53:07.241 INFO:tasks.workunit.client.0.smithi107.stdout:9/879: chown d0/dd/d21/d29/d3e/d45/f61 2110211 1 2024-05-17T21:53:07.242 INFO:tasks.workunit.client.1.smithi138.stdout:3/808: readlink d1/d62/d9b/d107/l6c 0 2024-05-17T21:53:07.242 INFO:tasks.workunit.client.1.smithi138.stdout:7/659: write d0/d18/d40/d41/d50/f61 [3987167,2786] 0 2024-05-17T21:53:07.243 INFO:tasks.workunit.client.0.smithi107.stdout:1/890: read d4/f38 [660083,19037] 0 2024-05-17T21:53:07.243 INFO:tasks.workunit.client.0.smithi107.stdout:3/881: link d6/d1a/l24 d6/d1a/d57/l119 0 2024-05-17T21:53:07.249 INFO:tasks.workunit.client.1.smithi138.stdout:3/809: creat d1/dee/f112 x:0 0 0 2024-05-17T21:53:07.250 INFO:tasks.workunit.client.0.smithi107.stdout:4/884: dwrite d3/d6/d26/d36/d37/ff0 [0,4194304] 0 2024-05-17T21:53:07.251 INFO:tasks.workunit.client.0.smithi107.stdout:0/952: mknod d3/d58/d61/c136 0 2024-05-17T21:53:07.254 INFO:tasks.workunit.client.0.smithi107.stdout:5/855: mknod d5/c11c 0 2024-05-17T21:53:07.254 INFO:tasks.workunit.client.0.smithi107.stdout:3/882: dwrite d6/d8/d10/d25/d80/f10a [0,4194304] 0 2024-05-17T21:53:07.256 INFO:tasks.workunit.client.0.smithi107.stdout:4/885: dread d3/d6/d26/f31 [0,4194304] 0 2024-05-17T21:53:07.257 INFO:tasks.workunit.client.0.smithi107.stdout:2/693: rename d0/d9/d1c/d3e/d4f to d0/d26/ddd 0 2024-05-17T21:53:07.259 INFO:tasks.workunit.client.1.smithi138.stdout:3/810: truncate d1/d4f/f77 4981650 0 2024-05-17T21:53:07.260 INFO:tasks.workunit.client.1.smithi138.stdout:3/811: chown d1/d3/d8d/d92/ce2 241053349 1 2024-05-17T21:53:07.261 INFO:tasks.workunit.client.1.smithi138.stdout:3/812: dread - d1/d3/d16/d3e/d71/f110 zero size 2024-05-17T21:53:07.263 INFO:tasks.workunit.client.0.smithi107.stdout:1/891: write d4/df/d14/d2e/d43/f111 [4636161,30223] 0 2024-05-17T21:53:07.265 INFO:tasks.workunit.client.1.smithi138.stdout:3/813: creat d1/d3/d16/d3e/f113 x:0 0 0 2024-05-17T21:53:07.265 INFO:tasks.workunit.client.1.smithi138.stdout:7/660: getdents d0/d18/d40/d41/d51/d81 0 2024-05-17T21:53:07.266 INFO:tasks.workunit.client.1.smithi138.stdout:3/814: chown d1/d62/dad/f64 50427 1 2024-05-17T21:53:07.275 INFO:tasks.workunit.client.0.smithi107.stdout:9/880: rename d0/lf to d0/dd/d1b/de7/l121 0 2024-05-17T21:53:07.276 INFO:tasks.workunit.client.0.smithi107.stdout:2/694: write d0/d9/d11/f9b [1554910,84112] 0 2024-05-17T21:53:07.277 INFO:tasks.workunit.client.1.smithi138.stdout:3/815: read d1/d4f/d50/f54 [3361162,106077] 0 2024-05-17T21:53:07.277 INFO:tasks.workunit.client.1.smithi138.stdout:3/816: readlink d1/d3/d16/d2d/l7c 0 2024-05-17T21:53:07.278 INFO:tasks.workunit.client.0.smithi107.stdout:1/892: dread d4/f66 [0,4194304] 0 2024-05-17T21:53:07.278 INFO:tasks.workunit.client.0.smithi107.stdout:2/695: readlink d0/d9/d11/l3b 0 2024-05-17T21:53:07.278 INFO:tasks.workunit.client.1.smithi138.stdout:3/817: write d1/d62/dad/f28 [127788,123649] 0 2024-05-17T21:53:07.279 INFO:tasks.workunit.client.0.smithi107.stdout:0/953: rename d3/dc/d22/ce0 to d3/d24/d6f/db5/def/c137 0 2024-05-17T21:53:07.281 INFO:tasks.workunit.client.1.smithi138.stdout:3/818: mknod d1/d4f/d87/c114 0 2024-05-17T21:53:07.282 INFO:tasks.workunit.client.0.smithi107.stdout:2/696: creat d0/d9/d11/d18/dad/db8/fde x:0 0 0 2024-05-17T21:53:07.284 INFO:tasks.workunit.client.0.smithi107.stdout:9/881: dwrite d0/dd/d2f/d4a/f101 [4194304,4194304] 0 2024-05-17T21:53:07.284 INFO:tasks.workunit.client.0.smithi107.stdout:0/954: creat d3/d75/d9e/d103/d10d/f138 x:0 0 0 2024-05-17T21:53:07.288 INFO:tasks.workunit.client.1.smithi138.stdout:3/819: creat d1/d3/d16/d3e/d71/d101/f115 x:0 0 0 2024-05-17T21:53:07.289 INFO:tasks.workunit.client.0.smithi107.stdout:2/697: dread d0/d9/d11/d18/fd5 [0,4194304] 0 2024-05-17T21:53:07.289 INFO:tasks.workunit.client.1.smithi138.stdout:3/820: chown d1/d62/dad/d72 615 1 2024-05-17T21:53:07.290 INFO:tasks.workunit.client.0.smithi107.stdout:4/886: dread d3/d4a/d47/d60/f79 [0,4194304] 0 2024-05-17T21:53:07.294 INFO:tasks.workunit.client.0.smithi107.stdout:9/882: write d0/dd/d69/dee/d116/f11d [609674,28488] 0 2024-05-17T21:53:07.295 INFO:tasks.workunit.client.0.smithi107.stdout:8/825: sync 2024-05-17T21:53:07.295 INFO:tasks.workunit.client.0.smithi107.stdout:7/809: sync 2024-05-17T21:53:07.295 INFO:tasks.workunit.client.1.smithi138.stdout:3/821: creat d1/d3/f116 x:0 0 0 2024-05-17T21:53:07.296 INFO:tasks.workunit.client.0.smithi107.stdout:1/893: link d4/de/l91 d4/df/d1a/d26/d72/daa/deb/d104/l11a 0 2024-05-17T21:53:07.299 INFO:tasks.workunit.client.0.smithi107.stdout:8/826: read dc/d44/d4b/d121/f9b [2870216,42001] 0 2024-05-17T21:53:07.301 INFO:tasks.workunit.client.0.smithi107.stdout:0/955: mkdir d3/dc/dae/dc0/de1/d10b/d139 0 2024-05-17T21:53:07.302 INFO:tasks.workunit.client.1.smithi138.stdout:7/661: dwrite d0/d18/d40/d41/d50/f74 [0,4194304] 0 2024-05-17T21:53:07.303 INFO:tasks.workunit.client.0.smithi107.stdout:3/883: dwrite d6/d8/d10/f23 [0,4194304] 0 2024-05-17T21:53:07.303 INFO:tasks.workunit.client.0.smithi107.stdout:9/883: symlink d0/dd/d2f/d81/l122 0 2024-05-17T21:53:07.304 INFO:tasks.workunit.client.0.smithi107.stdout:5/856: dwrite d5/db/d16/d2a/d73/f9e [0,4194304] 0 2024-05-17T21:53:07.306 INFO:tasks.workunit.client.0.smithi107.stdout:9/884: chown d0/dd/d21/d29/d3e/d45/f67 52318 1 2024-05-17T21:53:07.311 INFO:tasks.workunit.client.0.smithi107.stdout:4/887: mkdir d3/d4a/d47/d60/d3c/d12d 0 2024-05-17T21:53:07.311 INFO:tasks.workunit.client.0.smithi107.stdout:2/698: creat d0/d9/d11/d27/dbf/fdf x:0 0 0 2024-05-17T21:53:07.311 INFO:tasks.workunit.client.0.smithi107.stdout:1/894: dread d4/df/d1a/d26/d7a/fb5 [0,4194304] 0 2024-05-17T21:53:07.312 INFO:tasks.workunit.client.1.smithi138.stdout:7/662: creat d0/d18/fd2 x:0 0 0 2024-05-17T21:53:07.313 INFO:tasks.workunit.client.1.smithi138.stdout:7/663: stat d0/f1e 0 2024-05-17T21:53:07.318 INFO:tasks.workunit.client.0.smithi107.stdout:5/857: rename d5/d75/dc7/dd4 to d5/db/d16/d11d 0 2024-05-17T21:53:07.319 INFO:tasks.workunit.client.0.smithi107.stdout:7/810: symlink d5/df/d11/d100/l11a 0 2024-05-17T21:53:07.321 INFO:tasks.workunit.client.1.smithi138.stdout:7/664: truncate d0/d18/d40/d5f/f73 794845 0 2024-05-17T21:53:07.326 INFO:tasks.workunit.client.0.smithi107.stdout:1/895: symlink d4/da7/daf/l11b 0 2024-05-17T21:53:07.326 INFO:tasks.workunit.client.0.smithi107.stdout:0/956: creat d3/d24/d6f/d114/d128/f13a x:0 0 0 2024-05-17T21:53:07.327 INFO:tasks.workunit.client.0.smithi107.stdout:5/858: readlink d5/db/d16/d2d/d40/lfd 0 2024-05-17T21:53:07.328 INFO:tasks.workunit.client.0.smithi107.stdout:7/811: symlink d5/d16/d3b/l11b 0 2024-05-17T21:53:07.331 INFO:tasks.workunit.client.0.smithi107.stdout:0/957: fdatasync d3/ff 0 2024-05-17T21:53:07.332 INFO:tasks.workunit.client.0.smithi107.stdout:3/884: link d6/d2a/d2d/d48/f93 d6/d2a/d2d/d69/dad/de1/f11a 0 2024-05-17T21:53:07.332 INFO:tasks.workunit.client.0.smithi107.stdout:5/859: mkdir d5/d11e 0 2024-05-17T21:53:07.333 INFO:tasks.workunit.client.0.smithi107.stdout:0/958: fdatasync d3/f133 0 2024-05-17T21:53:07.336 INFO:tasks.workunit.client.0.smithi107.stdout:3/885: chown d6/d2a/f5b 5 1 2024-05-17T21:53:07.339 INFO:tasks.workunit.client.0.smithi107.stdout:5/860: write d5/db/d16/d11d/f116 [772046,5621] 0 2024-05-17T21:53:07.340 INFO:tasks.workunit.client.0.smithi107.stdout:4/888: dwrite d3/d4a/d47/d60/f12 [0,4194304] 0 2024-05-17T21:53:07.340 INFO:tasks.workunit.client.0.smithi107.stdout:1/896: dwrite d4/f66 [0,4194304] 0 2024-05-17T21:53:07.343 INFO:tasks.workunit.client.0.smithi107.stdout:3/886: creat d6/d2a/d2d/d37/d108/f11b x:0 0 0 2024-05-17T21:53:07.343 INFO:tasks.workunit.client.0.smithi107.stdout:2/699: dread d0/d9/d11/d27/f43 [0,4194304] 0 2024-05-17T21:53:07.345 INFO:tasks.workunit.client.0.smithi107.stdout:5/861: write d5/db/dc/d1e/d6e/ffa [27865,63768] 0 2024-05-17T21:53:07.345 INFO:tasks.workunit.client.0.smithi107.stdout:1/897: dread - d4/df/d1a/d26/d72/f117 zero size 2024-05-17T21:53:07.347 INFO:tasks.workunit.client.0.smithi107.stdout:0/959: rename d3/d24/d6f/db5/def/c137 to d3/dc/d22/d23/d97/d77/d91/d12b/c13b 0 2024-05-17T21:53:07.348 INFO:tasks.workunit.client.0.smithi107.stdout:5/862: fdatasync d5/db/d16/d5b/f97 0 2024-05-17T21:53:07.350 INFO:tasks.workunit.client.0.smithi107.stdout:4/889: creat d3/d4a/d47/d60/f12e x:0 0 0 2024-05-17T21:53:07.351 INFO:tasks.workunit.client.0.smithi107.stdout:1/898: mkdir d4/df/d14/d2e/dd3/df6/d11c 0 2024-05-17T21:53:07.352 INFO:tasks.workunit.client.0.smithi107.stdout:0/960: fdatasync d3/d24/ff8 0 2024-05-17T21:53:07.353 INFO:tasks.workunit.client.0.smithi107.stdout:1/899: dread - d4/df/d1a/d26/d72/daa/fd2 zero size 2024-05-17T21:53:07.358 INFO:tasks.workunit.client.0.smithi107.stdout:1/900: getdents d4/df/d4e/d59 0 2024-05-17T21:53:07.359 INFO:tasks.workunit.client.0.smithi107.stdout:7/812: dread d5/d16/f1e [0,4194304] 0 2024-05-17T21:53:07.359 INFO:tasks.workunit.client.0.smithi107.stdout:4/890: dwrite d3/db/d1e/d66/f10c [0,4194304] 0 2024-05-17T21:53:07.360 INFO:tasks.workunit.client.0.smithi107.stdout:1/901: chown d4/da7/daf/lca 61300514 1 2024-05-17T21:53:07.363 INFO:tasks.workunit.client.0.smithi107.stdout:4/891: truncate d3/db/f69 3970721 0 2024-05-17T21:53:07.363 INFO:tasks.workunit.client.0.smithi107.stdout:1/902: symlink d4/df/d1a/d26/d72/l11d 0 2024-05-17T21:53:07.364 INFO:tasks.workunit.client.0.smithi107.stdout:7/813: creat d5/df/d11/d1c/d31/f11c x:0 0 0 2024-05-17T21:53:07.365 INFO:tasks.workunit.client.0.smithi107.stdout:4/892: write d3/d4a/d47/d60/f12 [4485196,24099] 0 2024-05-17T21:53:07.366 INFO:tasks.workunit.client.0.smithi107.stdout:4/893: read - d3/db/d1e/ff1 zero size 2024-05-17T21:53:07.367 INFO:tasks.workunit.client.0.smithi107.stdout:7/814: creat d5/df/d11/d1d/d33/d82/f11d x:0 0 0 2024-05-17T21:53:07.368 INFO:tasks.workunit.client.0.smithi107.stdout:7/815: write d5/d16/d3b/ddf/f119 [681985,22180] 0 2024-05-17T21:53:07.369 INFO:tasks.workunit.client.0.smithi107.stdout:7/816: read d5/d16/d3b/d45/d91/dac/fef [227878,78672] 0 2024-05-17T21:53:07.370 INFO:tasks.workunit.client.0.smithi107.stdout:7/817: chown d5/df/d11/d1c/d31/d39/l8f 33 1 2024-05-17T21:53:07.372 INFO:tasks.workunit.client.0.smithi107.stdout:7/818: getdents d5/df/d11/df6 0 2024-05-17T21:53:07.373 INFO:tasks.workunit.client.1.smithi138.stdout:4/685: sync 2024-05-17T21:53:07.374 INFO:tasks.workunit.client.0.smithi107.stdout:7/819: chown d5/df/d11/d1c/d31/d56/f9c 629345 1 2024-05-17T21:53:07.383 INFO:tasks.workunit.client.1.smithi138.stdout:3/822: rename d1/d62/db0/cb6 to d1/d3/d16/d2d/c117 0 2024-05-17T21:53:07.384 INFO:tasks.workunit.client.0.smithi107.stdout:4/894: dread d3/fa [0,4194304] 0 2024-05-17T21:53:07.385 INFO:tasks.workunit.client.0.smithi107.stdout:4/895: stat d3/d4a/d47/f4b 0 2024-05-17T21:53:07.385 INFO:tasks.workunit.client.1.smithi138.stdout:3/823: truncate d1/d3/d16/d3e/d71/f110 971907 0 2024-05-17T21:53:07.386 INFO:tasks.workunit.client.1.smithi138.stdout:4/686: symlink d0/d1b/dd4/lf7 0 2024-05-17T21:53:07.387 INFO:tasks.workunit.client.1.smithi138.stdout:4/687: chown d0/d2f/d49/d60/d1f/d7e/f93 668146987 1 2024-05-17T21:53:07.388 INFO:tasks.workunit.client.0.smithi107.stdout:4/896: fsync d3/db/d35/d5e/dff/d80/f45 0 2024-05-17T21:53:07.389 INFO:tasks.workunit.client.0.smithi107.stdout:7/820: getdents d5/df/d11/d1c 0 2024-05-17T21:53:07.390 INFO:tasks.workunit.client.1.smithi138.stdout:3/824: rename d1/d3/d8d/d92/fa6 to d1/d4f/d50/dbb/f118 0 2024-05-17T21:53:07.392 INFO:tasks.workunit.client.1.smithi138.stdout:4/688: mknod d0/d2f/d49/d60/d1f/d7e/de8/cf8 0 2024-05-17T21:53:07.394 INFO:tasks.workunit.client.1.smithi138.stdout:3/825: creat d1/d62/dad/de/f119 x:0 0 0 2024-05-17T21:53:07.395 INFO:tasks.workunit.client.0.smithi107.stdout:7/821: chown d5/df/d11/d1c/d31/d39/d92/d9a/lce 78029514 1 2024-05-17T21:53:07.395 INFO:tasks.workunit.client.0.smithi107.stdout:7/822: fsync d5/d16/d3b/d45/d91/dc1/f106 0 2024-05-17T21:53:07.397 INFO:tasks.workunit.client.1.smithi138.stdout:4/689: rename d0/d3/d17/d5e/d7f/l85 to d0/d2f/d49/d60/d1f/d7e/lf9 0 2024-05-17T21:53:07.399 INFO:tasks.workunit.client.1.smithi138.stdout:3/826: rmdir d1/d3/d16/d3e/d71 39 2024-05-17T21:53:07.401 INFO:tasks.workunit.client.1.smithi138.stdout:4/690: readlink d0/d1b/d42/lcf 0 2024-05-17T21:53:07.403 INFO:tasks.workunit.client.0.smithi107.stdout:7/823: symlink d5/d16/d3b/d45/d91/dc1/dde/l11e 0 2024-05-17T21:53:07.404 INFO:tasks.workunit.client.0.smithi107.stdout:9/885: dread d0/dd/d2f/d87/d89/f97 [0,4194304] 0 2024-05-17T21:53:07.404 INFO:tasks.workunit.client.0.smithi107.stdout:7/824: fdatasync d5/df/d11/d1c/f4e 0 2024-05-17T21:53:07.405 INFO:tasks.workunit.client.0.smithi107.stdout:7/825: readlink d5/d16/d3b/d45/l61 0 2024-05-17T21:53:07.408 INFO:tasks.workunit.client.0.smithi107.stdout:4/897: dread d3/d6/d26/d36/f108 [0,4194304] 0 2024-05-17T21:53:07.408 INFO:tasks.workunit.client.0.smithi107.stdout:7/826: symlink d5/df/d11/d1c/d2f/dfe/l11f 0 2024-05-17T21:53:07.409 INFO:tasks.workunit.client.0.smithi107.stdout:4/898: chown d3/d6/d26/d36/d37/c39 83217 1 2024-05-17T21:53:07.409 INFO:tasks.workunit.client.0.smithi107.stdout:7/827: chown d5/df/d11/d1c/d2f/l57 177961441 1 2024-05-17T21:53:07.411 INFO:tasks.workunit.client.0.smithi107.stdout:4/899: creat d3/db/d35/f12f x:0 0 0 2024-05-17T21:53:07.412 INFO:tasks.workunit.client.0.smithi107.stdout:1/903: dread d4/df/d1a/d26/d72/dbc/f9f [0,4194304] 0 2024-05-17T21:53:07.413 INFO:tasks.workunit.client.0.smithi107.stdout:9/886: getdents d0/dd/d2f/d81 0 2024-05-17T21:53:07.416 INFO:tasks.workunit.client.0.smithi107.stdout:4/900: mknod d3/db/d35/d5e/dff/d80/c130 0 2024-05-17T21:53:07.417 INFO:tasks.workunit.client.0.smithi107.stdout:7/828: dwrite d5/df/d11/d1d/f10a [0,4194304] 0 2024-05-17T21:53:07.418 INFO:tasks.workunit.client.0.smithi107.stdout:9/887: chown d0/dd/d3d/d9d/l10d 111986 1 2024-05-17T21:53:07.420 INFO:tasks.workunit.client.0.smithi107.stdout:7/829: creat d5/d16/d3b/d45/d91/dc1/de6/f120 x:0 0 0 2024-05-17T21:53:07.422 INFO:tasks.workunit.client.0.smithi107.stdout:1/904: mkdir d4/df/d11e 0 2024-05-17T21:53:07.426 INFO:tasks.workunit.client.0.smithi107.stdout:1/905: mknod d4/d78/c11f 0 2024-05-17T21:53:07.426 INFO:tasks.workunit.client.0.smithi107.stdout:7/830: chown d5/df/d11/d1c/d31/d39/dc0/fe9 223 1 2024-05-17T21:53:07.428 INFO:tasks.workunit.client.0.smithi107.stdout:4/901: dwrite d3/d6/d26/d3b/d91/fb9 [4194304,4194304] 0 2024-05-17T21:53:07.428 INFO:tasks.workunit.client.1.smithi138.stdout:7/665: dwrite d0/f13 [0,4194304] 0 2024-05-17T21:53:07.429 INFO:tasks.workunit.client.0.smithi107.stdout:1/906: write d4/d6a/fe9 [606488,53484] 0 2024-05-17T21:53:07.430 INFO:tasks.workunit.client.1.smithi138.stdout:7/666: fsync d0/f91 0 2024-05-17T21:53:07.431 INFO:tasks.workunit.client.0.smithi107.stdout:8/827: dwrite dc/d21/f92 [0,4194304] 0 2024-05-17T21:53:07.432 INFO:tasks.workunit.client.0.smithi107.stdout:6/781: sync 2024-05-17T21:53:07.433 INFO:tasks.workunit.client.0.smithi107.stdout:8/828: chown dc/d44/d4b/d52/dcb 6 1 2024-05-17T21:53:07.434 INFO:tasks.workunit.client.0.smithi107.stdout:7/831: stat d5/df/d11/d1d/l74 0 2024-05-17T21:53:07.436 INFO:tasks.workunit.client.1.smithi138.stdout:7/667: mknod d0/cd3 0 2024-05-17T21:53:07.439 INFO:tasks.workunit.client.1.smithi138.stdout:7/668: write d0/d18/d64/d7c/da2/fb0 [117373,4245] 0 2024-05-17T21:53:07.440 INFO:tasks.workunit.client.0.smithi107.stdout:4/902: mknod d3/d4a/d47/d60/d3c/d12d/c131 0 2024-05-17T21:53:07.440 INFO:tasks.workunit.client.0.smithi107.stdout:6/782: chown d9/d58/d66/de2/fe5 126035 1 2024-05-17T21:53:07.441 INFO:tasks.workunit.client.0.smithi107.stdout:1/907: dread d4/df/d1a/d26/d7a/fb5 [0,4194304] 0 2024-05-17T21:53:07.444 INFO:tasks.workunit.client.1.smithi138.stdout:7/669: mkdir d0/d18/d9c/dd4 0 2024-05-17T21:53:07.447 INFO:tasks.workunit.client.0.smithi107.stdout:3/887: dwrite d6/d8/d3e/d3f/d90/ff4 [0,4194304] 0 2024-05-17T21:53:07.450 INFO:tasks.workunit.client.0.smithi107.stdout:5/863: dwrite d5/f9 [0,4194304] 0 2024-05-17T21:53:07.450 INFO:tasks.workunit.client.0.smithi107.stdout:8/829: symlink dc/d44/d4b/d52/d70/l125 0 2024-05-17T21:53:07.451 INFO:tasks.workunit.client.0.smithi107.stdout:6/783: symlink d9/d58/d7b/da3/lf8 0 2024-05-17T21:53:07.453 INFO:tasks.workunit.client.0.smithi107.stdout:7/832: mkdir d5/df/d11/d1c/d2f/d53/d121 0 2024-05-17T21:53:07.455 INFO:tasks.workunit.client.0.smithi107.stdout:1/908: creat d4/df/d1a/d26/d72/daa/deb/f120 x:0 0 0 2024-05-17T21:53:07.457 INFO:tasks.workunit.client.0.smithi107.stdout:6/784: mkdir d9/d81/df9 0 2024-05-17T21:53:07.458 INFO:tasks.workunit.client.0.smithi107.stdout:0/961: dwrite d3/dc/dae/fbb [4194304,4194304] 0 2024-05-17T21:53:07.460 INFO:tasks.workunit.client.0.smithi107.stdout:6/785: stat d9/d58/d66/la5 0 2024-05-17T21:53:07.460 INFO:tasks.workunit.client.0.smithi107.stdout:8/830: fdatasync dc/d44/d4b/d52/f89 0 2024-05-17T21:53:07.462 INFO:tasks.workunit.client.0.smithi107.stdout:3/888: rename d6/d2a/d2d/l53 to d6/d2a/d2d/d69/l11c 0 2024-05-17T21:53:07.464 INFO:tasks.workunit.client.0.smithi107.stdout:4/903: truncate d3/d4a/d47/d60/d3c/d58/fe3 3685352 0 2024-05-17T21:53:07.465 INFO:tasks.workunit.client.0.smithi107.stdout:5/864: mkdir d5/db/d16/d54/d59/d65/d48/d11f 0 2024-05-17T21:53:07.469 INFO:tasks.workunit.client.1.smithi138.stdout:4/691: write d0/d3/d17/d21/d78/f9a [4569068,117493] 0 2024-05-17T21:53:07.473 INFO:tasks.workunit.client.0.smithi107.stdout:1/909: creat d4/df/d1a/d26/d7a/f121 x:0 0 0 2024-05-17T21:53:07.474 INFO:tasks.workunit.client.1.smithi138.stdout:4/692: write d0/d3/d1d/d38/d8a/dbd/fc0 [987449,124004] 0 2024-05-17T21:53:07.477 INFO:tasks.workunit.client.0.smithi107.stdout:3/889: write d6/d8/d10/d35/d91/da6/ff9 [1201684,34356] 0 2024-05-17T21:53:07.478 INFO:tasks.workunit.client.0.smithi107.stdout:2/700: sync 2024-05-17T21:53:07.479 INFO:tasks.workunit.client.0.smithi107.stdout:7/833: symlink d5/df/d11/d1c/d31/l122 0 2024-05-17T21:53:07.480 INFO:tasks.workunit.client.0.smithi107.stdout:7/834: chown d5/d16/d3b/d4f/c5f 54838 1 2024-05-17T21:53:07.481 INFO:tasks.workunit.client.0.smithi107.stdout:6/786: fdatasync d9/d16/d22/f72 0 2024-05-17T21:53:07.482 INFO:tasks.workunit.client.0.smithi107.stdout:6/787: dread - d9/d81/feb zero size 2024-05-17T21:53:07.484 INFO:tasks.workunit.client.0.smithi107.stdout:1/910: unlink d4/l68 0 2024-05-17T21:53:07.485 INFO:tasks.workunit.client.0.smithi107.stdout:9/888: write d0/dd/d69/ff1 [750,6471] 0 2024-05-17T21:53:07.486 INFO:tasks.workunit.client.1.smithi138.stdout:3/827: dread d1/d4f/d50/dbb/f118 [0,4194304] 0 2024-05-17T21:53:07.487 INFO:tasks.workunit.client.1.smithi138.stdout:3/828: fdatasync d1/d4f/d87/ff3 0 2024-05-17T21:53:07.489 INFO:tasks.workunit.client.0.smithi107.stdout:9/889: dread d0/dd/d21/d29/d3e/dfe/f103 [0,4194304] 0 2024-05-17T21:53:07.495 INFO:tasks.workunit.client.0.smithi107.stdout:8/831: rename dc/d21/daa/fc3 to dc/d21/f126 0 2024-05-17T21:53:07.496 INFO:tasks.workunit.client.0.smithi107.stdout:8/832: chown dc/d21/daa/db4/f120 13078564 1 2024-05-17T21:53:07.497 INFO:tasks.workunit.client.0.smithi107.stdout:8/833: readlink dc/d21/l2b 0 2024-05-17T21:53:07.498 INFO:tasks.workunit.client.0.smithi107.stdout:6/788: dread d9/d16/d22/d5f/fdc [0,4194304] 0 2024-05-17T21:53:07.498 INFO:tasks.workunit.client.0.smithi107.stdout:4/904: symlink d3/d4a/d47/d60/dfa/l132 0 2024-05-17T21:53:07.501 INFO:tasks.workunit.client.0.smithi107.stdout:3/890: creat d6/d8/d3e/d3f/d90/da7/ddc/f11d x:0 0 0 2024-05-17T21:53:07.501 INFO:tasks.workunit.client.0.smithi107.stdout:3/891: write f3 [3615863,99797] 0 2024-05-17T21:53:07.502 INFO:tasks.workunit.client.0.smithi107.stdout:2/701: mknod d0/d26/d8e/dc4/ce0 0 2024-05-17T21:53:07.503 INFO:tasks.workunit.client.0.smithi107.stdout:0/962: write d3/ff [5745028,37116] 0 2024-05-17T21:53:07.504 INFO:tasks.workunit.client.0.smithi107.stdout:6/789: dread d9/d16/f3b [0,4194304] 0 2024-05-17T21:53:07.506 INFO:tasks.workunit.client.0.smithi107.stdout:8/834: stat dc/d21/d27/l40 0 2024-05-17T21:53:07.508 INFO:tasks.workunit.client.0.smithi107.stdout:4/905: mkdir d3/db/d35/d5e/dff/d80/d133 0 2024-05-17T21:53:07.510 INFO:tasks.workunit.client.0.smithi107.stdout:3/892: dread - d6/d2a/d2d/da8/f92 zero size 2024-05-17T21:53:07.510 INFO:tasks.workunit.client.0.smithi107.stdout:8/835: chown dc/d21/l8c 384 1 2024-05-17T21:53:07.511 INFO:tasks.workunit.client.0.smithi107.stdout:2/702: creat d0/d9/d11/d18/d9e/fe1 x:0 0 0 2024-05-17T21:53:07.514 INFO:tasks.workunit.client.0.smithi107.stdout:1/911: link d4/df/lbb d4/df/d1a/d41/l122 0 2024-05-17T21:53:07.515 INFO:tasks.workunit.client.0.smithi107.stdout:9/890: dwrite d0/dd/d69/dee/f11a [0,4194304] 0 2024-05-17T21:53:07.516 INFO:tasks.workunit.client.0.smithi107.stdout:5/865: dwrite d5/db/d16/d2d/d4d/f6a [0,4194304] 0 2024-05-17T21:53:07.517 INFO:tasks.workunit.client.1.smithi138.stdout:3/829: dwrite d1/d62/dad/de/f108 [0,4194304] 0 2024-05-17T21:53:07.519 INFO:tasks.workunit.client.0.smithi107.stdout:5/866: stat d5/db/fc6 0 2024-05-17T21:53:07.520 INFO:tasks.workunit.client.0.smithi107.stdout:6/790: dwrite d9/d16/d22/f27 [0,4194304] 0 2024-05-17T21:53:07.523 INFO:tasks.workunit.client.1.smithi138.stdout:3/830: creat d1/d3/d16/d5d/f11a x:0 0 0 2024-05-17T21:53:07.524 INFO:tasks.workunit.client.1.smithi138.stdout:1/629: sync 2024-05-17T21:53:07.525 INFO:tasks.workunit.client.0.smithi107.stdout:9/891: write d0/dd/d21/d29/d3e/d45/d68/d75/ffc [61115,128153] 0 2024-05-17T21:53:07.527 INFO:tasks.workunit.client.0.smithi107.stdout:4/906: dwrite d3/d4a/d47/d60/f12 [0,4194304] 0 2024-05-17T21:53:07.527 INFO:tasks.workunit.client.0.smithi107.stdout:1/912: dread d4/df/d1a/fc4 [0,4194304] 0 2024-05-17T21:53:07.529 INFO:tasks.workunit.client.1.smithi138.stdout:3/831: creat d1/d4f/d87/f11b x:0 0 0 2024-05-17T21:53:07.530 INFO:tasks.workunit.client.0.smithi107.stdout:1/913: write d4/df/d1a/d26/d72/d85/f114 [665780,101348] 0 2024-05-17T21:53:07.532 INFO:tasks.workunit.client.1.smithi138.stdout:1/630: creat d4/dd/d3b/d6c/fc9 x:0 0 0 2024-05-17T21:53:07.538 INFO:tasks.workunit.client.0.smithi107.stdout:0/963: write d3/f1d [4398355,71010] 0 2024-05-17T21:53:07.538 INFO:tasks.workunit.client.0.smithi107.stdout:4/907: dwrite d3/d6/d26/f2f [0,4194304] 0 2024-05-17T21:53:07.541 INFO:tasks.workunit.client.0.smithi107.stdout:5/867: symlink d5/dec/l120 0 2024-05-17T21:53:07.541 INFO:tasks.workunit.client.1.smithi138.stdout:1/631: chown d4/df/c18 928 1 2024-05-17T21:53:07.542 INFO:tasks.workunit.client.1.smithi138.stdout:1/632: readlink d4/dd/l7a 0 2024-05-17T21:53:07.545 INFO:tasks.workunit.client.1.smithi138.stdout:3/832: symlink d1/d62/d9b/d107/df2/l11c 0 2024-05-17T21:53:07.546 INFO:tasks.workunit.client.0.smithi107.stdout:5/868: dread d5/db/d16/d2a/d73/f9e [0,4194304] 0 2024-05-17T21:53:07.546 INFO:tasks.workunit.client.0.smithi107.stdout:6/791: symlink d9/d16/d4d/d85/d96/lfa 0 2024-05-17T21:53:07.548 INFO:tasks.workunit.client.0.smithi107.stdout:3/893: truncate d6/d2a/d2d/f43 742240 0 2024-05-17T21:53:07.549 INFO:tasks.workunit.client.1.smithi138.stdout:3/833: mknod d1/dc0/c11d 0 2024-05-17T21:53:07.549 INFO:tasks.workunit.client.0.smithi107.stdout:8/836: getdents dc/d21/d27/d34/d119 0 2024-05-17T21:53:07.550 INFO:tasks.workunit.client.1.smithi138.stdout:3/834: write d1/d3/d16/d3e/f43 [87606,127079] 0 2024-05-17T21:53:07.550 INFO:tasks.workunit.client.1.smithi138.stdout:1/633: creat d4/db/d29/fca x:0 0 0 2024-05-17T21:53:07.551 INFO:tasks.workunit.client.0.smithi107.stdout:1/914: symlink d4/df/d1a/d26/d72/daa/l123 0 2024-05-17T21:53:07.553 INFO:tasks.workunit.client.1.smithi138.stdout:3/835: chown d1/d4f/d80/fac 910119499 1 2024-05-17T21:53:07.553 INFO:tasks.workunit.client.0.smithi107.stdout:2/703: symlink d0/d9/d11/df/d5a/dc7/le2 0 2024-05-17T21:53:07.555 INFO:tasks.workunit.client.0.smithi107.stdout:0/964: readlink d3/dc/d22/d3d/la7 0 2024-05-17T21:53:07.556 INFO:tasks.workunit.client.1.smithi138.stdout:3/836: stat d1/d4f/d50/c83 0 2024-05-17T21:53:07.560 INFO:tasks.workunit.client.0.smithi107.stdout:5/869: mkdir d5/dec/d121 0 2024-05-17T21:53:07.561 INFO:tasks.workunit.client.1.smithi138.stdout:3/837: creat d1/d62/dad/d5c/d9c/dec/f11e x:0 0 0 2024-05-17T21:53:07.563 INFO:tasks.workunit.client.0.smithi107.stdout:6/792: unlink d9/d16/d22/de6/cee 0 2024-05-17T21:53:07.564 INFO:tasks.workunit.client.0.smithi107.stdout:8/837: creat dc/d21/d35/db6/f127 x:0 0 0 2024-05-17T21:53:07.564 INFO:tasks.workunit.client.0.smithi107.stdout:1/915: rmdir d4/df/d4e/d59 39 2024-05-17T21:53:07.565 INFO:tasks.workunit.client.0.smithi107.stdout:2/704: fsync d0/d9/d1c/f73 0 2024-05-17T21:53:07.567 INFO:tasks.workunit.client.1.smithi138.stdout:3/838: rename d1/d3/d16/d3e/c105 to d1/d3/d49/c11f 0 2024-05-17T21:53:07.568 INFO:tasks.workunit.client.0.smithi107.stdout:9/892: link d0/dd/d2f/f5b d0/dd/d2f/d87/d89/f123 0 2024-05-17T21:53:07.570 INFO:tasks.workunit.client.0.smithi107.stdout:3/894: symlink d6/dfd/l11e 0 2024-05-17T21:53:07.572 INFO:tasks.workunit.client.0.smithi107.stdout:6/793: rename d9/d16/d4d/d85/d96/fd4 to d9/d16/d4d/d85/d96/ffb 0 2024-05-17T21:53:07.572 INFO:tasks.workunit.client.0.smithi107.stdout:8/838: creat dc/d44/d4b/d121/f128 x:0 0 0 2024-05-17T21:53:07.574 INFO:tasks.workunit.client.1.smithi138.stdout:3/839: dwrite d1/d3/d16/d3e/f113 [0,4194304] 0 2024-05-17T21:53:07.576 INFO:tasks.workunit.client.0.smithi107.stdout:0/965: symlink d3/d58/d61/l13c 0 2024-05-17T21:53:07.578 INFO:tasks.workunit.client.0.smithi107.stdout:9/893: fdatasync d0/dd/d21/d29/d3e/d45/f6b 0 2024-05-17T21:53:07.580 INFO:tasks.workunit.client.1.smithi138.stdout:3/840: mkdir d1/d62/dad/de/d120 0 2024-05-17T21:53:07.581 INFO:tasks.workunit.client.1.smithi138.stdout:3/841: write d1/d4f/d80/fe6 [1075237,77621] 0 2024-05-17T21:53:07.592 INFO:tasks.workunit.client.1.smithi138.stdout:3/842: rename d1/d3/d16/d3e/d71/fc9 to d1/d3/d16/f121 0 2024-05-17T21:53:07.595 INFO:tasks.workunit.client.1.smithi138.stdout:1/634: dwrite d4/db/f4b [0,4194304] 0 2024-05-17T21:53:07.600 INFO:tasks.workunit.client.1.smithi138.stdout:3/843: dwrite d1/d3/d8d/fb7 [0,4194304] 0 2024-05-17T21:53:07.601 INFO:tasks.workunit.client.1.smithi138.stdout:3/844: stat d1/d62/dad/l44 0 2024-05-17T21:53:07.602 INFO:tasks.workunit.client.1.smithi138.stdout:1/635: link d4/df/fa8 d4/dd/d3b/da7/dbd/fcb 0 2024-05-17T21:53:07.603 INFO:tasks.workunit.client.1.smithi138.stdout:1/636: fsync f0 0 2024-05-17T21:53:07.605 INFO:tasks.workunit.client.0.smithi107.stdout:3/895: unlink d6/d8/fc0 0 2024-05-17T21:53:07.605 INFO:tasks.workunit.client.1.smithi138.stdout:3/845: creat d1/dee/f122 x:0 0 0 2024-05-17T21:53:07.605 INFO:tasks.workunit.client.0.smithi107.stdout:6/794: creat d9/d16/d4d/db8/dc9/dd0/ffc x:0 0 0 2024-05-17T21:53:07.607 INFO:tasks.workunit.client.0.smithi107.stdout:4/908: write d3/d6/d26/d36/d7e/f87 [937305,55100] 0 2024-05-17T21:53:07.610 INFO:tasks.workunit.client.0.smithi107.stdout:4/909: chown d3/db/d35/d5e/dff/d80/f94 22007426 1 2024-05-17T21:53:07.614 INFO:tasks.workunit.client.1.smithi138.stdout:3/846: mknod d1/d3/d16/d2d/df6/c123 0 2024-05-17T21:53:07.615 INFO:tasks.workunit.client.1.smithi138.stdout:3/847: chown d1/d3/d8d/fc7 25674309 1 2024-05-17T21:53:07.615 INFO:tasks.workunit.client.0.smithi107.stdout:5/870: dwrite d5/db/d16/f21 [0,4194304] 0 2024-05-17T21:53:07.616 INFO:tasks.workunit.client.0.smithi107.stdout:9/894: creat d0/dd/d21/d29/dca/f124 x:0 0 0 2024-05-17T21:53:07.617 INFO:tasks.workunit.client.1.smithi138.stdout:3/848: truncate d1/d3/d16/d3e/f43 4413613 0 2024-05-17T21:53:07.617 INFO:tasks.workunit.client.0.smithi107.stdout:8/839: link dc/d44/d4b/d121/la5 dc/d44/d4b/d52/l129 0 2024-05-17T21:53:07.617 INFO:tasks.workunit.client.1.smithi138.stdout:3/849: dread - d1/d62/dad/f104 zero size 2024-05-17T21:53:07.618 INFO:tasks.workunit.client.0.smithi107.stdout:0/966: write d3/d75/d9e/d103/d10d/d132/dab/f8e [960859,87002] 0 2024-05-17T21:53:07.618 INFO:tasks.workunit.client.1.smithi138.stdout:3/850: dread - d1/d3/d16/d3e/d71/db2/fe0 zero size 2024-05-17T21:53:07.623 INFO:tasks.workunit.client.0.smithi107.stdout:5/871: rmdir d5/d5f 39 2024-05-17T21:53:07.624 INFO:tasks.workunit.client.0.smithi107.stdout:5/872: fdatasync d5/db/d39/f3e 0 2024-05-17T21:53:07.624 INFO:tasks.workunit.client.1.smithi138.stdout:3/851: dread d1/d3/d8d/fb7 [0,4194304] 0 2024-05-17T21:53:07.626 INFO:tasks.workunit.client.0.smithi107.stdout:1/916: dread d4/df/d4e/f63 [0,4194304] 0 2024-05-17T21:53:07.627 INFO:tasks.workunit.client.0.smithi107.stdout:9/895: chown d0/dd/d3d/c44 817 1 2024-05-17T21:53:07.629 INFO:tasks.workunit.client.0.smithi107.stdout:9/896: chown d0/dd/d21/d10e 293359 1 2024-05-17T21:53:07.630 INFO:tasks.workunit.client.1.smithi138.stdout:3/852: dwrite d1/d62/dad/d72/fa4 [0,4194304] 0 2024-05-17T21:53:07.632 INFO:tasks.workunit.client.1.smithi138.stdout:3/853: write d1/d62/dad/d5c/d9c/dec/f11e [968067,82843] 0 2024-05-17T21:53:07.633 INFO:tasks.workunit.client.0.smithi107.stdout:6/795: dread d9/d16/f24 [0,4194304] 0 2024-05-17T21:53:07.635 INFO:tasks.workunit.client.0.smithi107.stdout:0/967: dread - d3/d39/fd6 zero size 2024-05-17T21:53:07.635 INFO:tasks.workunit.client.1.smithi138.stdout:3/854: dread d1/d62/dad/de/f8e [0,4194304] 0 2024-05-17T21:53:07.638 INFO:tasks.workunit.client.0.smithi107.stdout:3/896: write d6/d2a/f52 [3350500,128611] 0 2024-05-17T21:53:07.640 INFO:tasks.workunit.client.1.smithi138.stdout:1/637: dwrite d4/df/d3a/d3c/d63/d6d/f87 [0,4194304] 0 2024-05-17T21:53:07.642 INFO:tasks.workunit.client.0.smithi107.stdout:4/910: creat d3/d4a/d47/d60/f134 x:0 0 0 2024-05-17T21:53:07.643 INFO:tasks.workunit.client.1.smithi138.stdout:1/638: write d4/df/d3a/d3c/d63/d6d/f87 [1579673,11090] 0 2024-05-17T21:53:07.646 INFO:tasks.workunit.client.0.smithi107.stdout:6/796: fdatasync d9/f10 0 2024-05-17T21:53:07.647 INFO:tasks.workunit.client.1.smithi138.stdout:3/855: dwrite d1/d4f/d87/f11b [0,4194304] 0 2024-05-17T21:53:07.650 INFO:tasks.workunit.client.1.smithi138.stdout:3/856: write d1/d3/d16/fa2 [990281,96797] 0 2024-05-17T21:53:07.651 INFO:tasks.workunit.client.0.smithi107.stdout:7/835: sync 2024-05-17T21:53:07.652 INFO:tasks.workunit.client.1.smithi138.stdout:1/639: mkdir d4/db/d60/dcc 0 2024-05-17T21:53:07.653 INFO:tasks.workunit.client.0.smithi107.stdout:5/873: getdents d5/d106 0 2024-05-17T21:53:07.655 INFO:tasks.workunit.client.0.smithi107.stdout:1/917: chown d4/df/d14/d2e/d43/d47/l89 0 1 2024-05-17T21:53:07.655 INFO:tasks.workunit.client.0.smithi107.stdout:3/897: mkdir d6/d8/d3e/dc9/d11f 0 2024-05-17T21:53:07.657 INFO:tasks.workunit.client.1.smithi138.stdout:3/857: dwrite d1/d3/d16/fa2 [0,4194304] 0 2024-05-17T21:53:07.658 INFO:tasks.workunit.client.0.smithi107.stdout:4/911: mknod d3/d6/d26/d36/d37/d85/c135 0 2024-05-17T21:53:07.659 INFO:tasks.workunit.client.1.smithi138.stdout:3/858: chown d1/d62/d9b/ffc 53723 1 2024-05-17T21:53:07.660 INFO:tasks.workunit.client.0.smithi107.stdout:1/918: write d4/df/d1a/d26/d7a/f121 [926783,81886] 0 2024-05-17T21:53:07.661 INFO:tasks.workunit.client.1.smithi138.stdout:3/859: write d1/d3/d49/ddb/f6e [4771172,93532] 0 2024-05-17T21:53:07.661 INFO:tasks.workunit.client.0.smithi107.stdout:8/840: dwrite dc/d21/d27/d34/f38 [0,4194304] 0 2024-05-17T21:53:07.663 INFO:tasks.workunit.client.0.smithi107.stdout:3/898: write d6/d8/d10/d25/d80/f112 [725038,108637] 0 2024-05-17T21:53:07.665 INFO:tasks.workunit.client.1.smithi138.stdout:1/640: truncate d4/db/d29/fa4 2303784 0 2024-05-17T21:53:07.667 INFO:tasks.workunit.client.0.smithi107.stdout:4/912: write d3/d6/d26/d36/f116 [1199825,58089] 0 2024-05-17T21:53:07.667 INFO:tasks.workunit.client.0.smithi107.stdout:8/841: read - dc/d44/d4b/d121/de5/ff6 zero size 2024-05-17T21:53:07.668 INFO:tasks.workunit.client.0.smithi107.stdout:0/968: rename d3/d24/l8c to d3/d75/dd0/df6/l13d 0 2024-05-17T21:53:07.668 INFO:tasks.workunit.client.1.smithi138.stdout:3/860: mkdir d1/d3/d8d/d92/d124 0 2024-05-17T21:53:07.669 INFO:tasks.workunit.client.0.smithi107.stdout:6/797: rename d9/d16/d4d to d9/d16/d4d/d85/d96/dfd 22 2024-05-17T21:53:07.670 INFO:tasks.workunit.client.0.smithi107.stdout:8/842: dread - dc/d21/d27/d2f/f2c zero size 2024-05-17T21:53:07.674 INFO:tasks.workunit.client.0.smithi107.stdout:1/919: read d4/de/f19 [994131,109858] 0 2024-05-17T21:53:07.675 INFO:tasks.workunit.client.1.smithi138.stdout:1/641: symlink d4/dd/d3b/da7/dbd/lcd 0 2024-05-17T21:53:07.676 INFO:tasks.workunit.client.1.smithi138.stdout:1/642: write d4/f3f [2531005,127303] 0 2024-05-17T21:53:07.677 INFO:tasks.workunit.client.0.smithi107.stdout:3/899: mknod d6/d8/d3e/d3f/d90/c120 0 2024-05-17T21:53:07.679 INFO:tasks.workunit.client.0.smithi107.stdout:2/705: sync 2024-05-17T21:53:07.689 INFO:tasks.workunit.client.0.smithi107.stdout:1/920: dwrite d4/df/d1a/d26/d72/daa/f116 [0,4194304] 0 2024-05-17T21:53:07.694 INFO:tasks.workunit.client.1.smithi138.stdout:3/861: write d1/d2e/f98 [556279,35499] 0 2024-05-17T21:53:07.694 INFO:tasks.workunit.client.0.smithi107.stdout:1/921: write d4/df/d1a/d26/d7a/fb9 [1112130,59921] 0 2024-05-17T21:53:07.699 INFO:tasks.workunit.client.0.smithi107.stdout:9/897: dwrite d0/dd/f66 [0,4194304] 0 2024-05-17T21:53:07.700 INFO:tasks.workunit.client.0.smithi107.stdout:2/706: dwrite d0/d9/d11/df/d5a/d6d/fca [0,4194304] 0 2024-05-17T21:53:07.700 INFO:tasks.workunit.client.1.smithi138.stdout:1/643: dwrite d4/df/d3a/fbe [0,4194304] 0 2024-05-17T21:53:07.703 INFO:tasks.workunit.client.0.smithi107.stdout:7/836: rename d5/df/d11/d2a/l101 to d5/df/d11/d1c/l123 0 2024-05-17T21:53:07.703 INFO:tasks.workunit.client.1.smithi138.stdout:9/588: sync 2024-05-17T21:53:07.703 INFO:tasks.workunit.client.1.smithi138.stdout:5/677: sync 2024-05-17T21:53:07.704 INFO:tasks.workunit.client.1.smithi138.stdout:8/623: sync 2024-05-17T21:53:07.704 INFO:tasks.workunit.client.0.smithi107.stdout:5/874: dread d5/db/d16/d5b/f97 [0,4194304] 0 2024-05-17T21:53:07.705 INFO:tasks.workunit.client.0.smithi107.stdout:3/900: mknod d6/d2a/d2d/da8/c121 0 2024-05-17T21:53:07.706 INFO:tasks.workunit.client.0.smithi107.stdout:4/913: link d3/d4a/f75 d3/db/d1e/d66/f136 0 2024-05-17T21:53:07.707 INFO:tasks.workunit.client.1.smithi138.stdout:1/644: creat d4/dd/d3b/d6c/dc1/fce x:0 0 0 2024-05-17T21:53:07.710 INFO:tasks.workunit.client.0.smithi107.stdout:3/901: write d6/d2a/f6a [706595,66444] 0 2024-05-17T21:53:07.711 INFO:tasks.workunit.client.0.smithi107.stdout:1/922: dwrite d4/df/d1a/f7b [0,4194304] 0 2024-05-17T21:53:07.715 INFO:tasks.workunit.client.1.smithi138.stdout:5/678: mkdir d1/d2c/d63/db5/d1b/ddb 0 2024-05-17T21:53:07.715 INFO:tasks.workunit.client.1.smithi138.stdout:1/645: fdatasync d4/df/d4e/f55 0 2024-05-17T21:53:07.719 INFO:tasks.workunit.client.1.smithi138.stdout:9/589: mkdir d6/d7/d16/d29/d48/d6e/d74/d75/d76/d8e/dba 0 2024-05-17T21:53:07.721 INFO:tasks.workunit.client.0.smithi107.stdout:0/969: dwrite d3/f43 [0,4194304] 0 2024-05-17T21:53:07.722 INFO:tasks.workunit.client.0.smithi107.stdout:6/798: dwrite d9/d58/d7b/fa2 [0,4194304] 0 2024-05-17T21:53:07.723 INFO:tasks.workunit.client.0.smithi107.stdout:5/875: mknod d5/db/d16/d2d/d4d/d6d/c122 0 2024-05-17T21:53:07.724 INFO:tasks.workunit.client.0.smithi107.stdout:3/902: read d6/d8/f17 [2625920,130073] 0 2024-05-17T21:53:07.724 INFO:tasks.workunit.client.1.smithi138.stdout:5/679: rmdir d1/d2c/d63/db5/d1b/dc5/d36/d68/dad 39 2024-05-17T21:53:07.724 INFO:tasks.workunit.client.0.smithi107.stdout:8/843: dwrite dc/d21/d27/d34/d4c/d7e/f3b [0,4194304] 0 2024-05-17T21:53:07.730 INFO:tasks.workunit.client.0.smithi107.stdout:1/923: dread d4/df/d1a/f7b [4194304,4194304] 0 2024-05-17T21:53:07.734 INFO:tasks.workunit.client.0.smithi107.stdout:5/876: dwrite d5/db/d16/d2d/f4a [0,4194304] 0 2024-05-17T21:53:07.734 INFO:tasks.workunit.client.1.smithi138.stdout:5/680: rmdir d1/d60/d2f/d5e 39 2024-05-17T21:53:07.735 INFO:tasks.workunit.client.1.smithi138.stdout:5/681: chown d1/d53/d88/lc7 37 1 2024-05-17T21:53:07.736 INFO:tasks.workunit.client.0.smithi107.stdout:0/970: chown d3/d24/d6f/db5/ca3 510588913 1 2024-05-17T21:53:07.736 INFO:tasks.workunit.client.1.smithi138.stdout:1/646: getdents d4/df/d5f 0 2024-05-17T21:53:07.737 INFO:tasks.workunit.client.0.smithi107.stdout:0/971: chown d3/dc/d22/d23/d97/d77/d91/f100 4961215 1 2024-05-17T21:53:07.745 INFO:tasks.workunit.client.1.smithi138.stdout:1/647: dwrite d4/db/f4b [4194304,4194304] 0 2024-05-17T21:53:07.750 INFO:tasks.workunit.client.1.smithi138.stdout:8/624: dread d0/f2 [0,4194304] 0 2024-05-17T21:53:07.750 INFO:tasks.workunit.client.1.smithi138.stdout:5/682: mknod d1/d2c/d63/db5/d1b/dc5/d36/d68/dad/cdc 0 2024-05-17T21:53:07.750 INFO:tasks.workunit.client.0.smithi107.stdout:8/844: rename dc/d21/d27/d34/lf7 to dc/d21/d27/d2f/dba/l12a 0 2024-05-17T21:53:07.753 INFO:tasks.workunit.client.0.smithi107.stdout:7/837: getdents d5 0 2024-05-17T21:53:07.753 INFO:tasks.workunit.client.0.smithi107.stdout:2/707: getdents d0/d9/d11/df/d5a/d6d/d82 0 2024-05-17T21:53:07.753 INFO:tasks.workunit.client.0.smithi107.stdout:0/972: mkdir d3/d39/d13e 0 2024-05-17T21:53:07.755 INFO:tasks.workunit.client.0.smithi107.stdout:0/973: write d3/d58/d61/d94/f95 [3877756,62796] 0 2024-05-17T21:53:07.757 INFO:tasks.workunit.client.1.smithi138.stdout:1/648: rename d4/df/d3a/d3c/d67 to d4/dd/d3b/d6c/dcf 0 2024-05-17T21:53:07.769 INFO:tasks.workunit.client.0.smithi107.stdout:9/898: truncate d0/dd/d21/f37 556337 0 2024-05-17T21:53:07.772 INFO:tasks.workunit.client.1.smithi138.stdout:9/590: dwrite d6/d7/d16/d29/d48/d6e/f56 [0,4194304] 0 2024-05-17T21:53:07.774 INFO:tasks.workunit.client.0.smithi107.stdout:3/903: dread d6/f2c [0,4194304] 0 2024-05-17T21:53:07.774 INFO:tasks.workunit.client.1.smithi138.stdout:9/591: write d6/d7/d16/d29/d38/f97 [495847,83052] 0 2024-05-17T21:53:07.799 INFO:tasks.workunit.client.0.smithi107.stdout:3/904: dread d6/d8/d10/d25/f47 [0,4194304] 0 2024-05-17T21:53:07.807 INFO:tasks.workunit.client.0.smithi107.stdout:6/799: link d9/d16/d4d/d85/f98 d9/d16/d4d/d85/d88/db1/ffe 0 2024-05-17T21:53:07.807 INFO:tasks.workunit.client.0.smithi107.stdout:6/800: write d9/d16/d22/f27 [1796297,8159] 0 2024-05-17T21:53:07.808 INFO:tasks.workunit.client.0.smithi107.stdout:7/838: fdatasync d5/d16/f28 0 2024-05-17T21:53:07.811 INFO:tasks.workunit.client.1.smithi138.stdout:1/649: mknod d4/df/d3a/d3c/cd0 0 2024-05-17T21:53:07.813 INFO:tasks.workunit.client.0.smithi107.stdout:7/839: dread d5/d16/d3b/d45/d91/db3/fed [0,4194304] 0 2024-05-17T21:53:07.813 INFO:tasks.workunit.client.0.smithi107.stdout:2/708: rename d0/d9/d1c/f3d to d0/d26/d64/ddb/fe3 0 2024-05-17T21:53:07.813 INFO:tasks.workunit.client.0.smithi107.stdout:6/801: dread d9/d16/f41 [0,4194304] 0 2024-05-17T21:53:07.814 INFO:tasks.workunit.client.0.smithi107.stdout:6/802: stat d9/d16/c5a 0 2024-05-17T21:53:07.814 INFO:tasks.workunit.client.0.smithi107.stdout:7/840: dread - d5/d16/d3b/d45/faf zero size 2024-05-17T21:53:07.816 INFO:tasks.workunit.client.0.smithi107.stdout:0/974: creat d3/dc/d22/d23/d97/d77/d91/f13f x:0 0 0 2024-05-17T21:53:07.822 INFO:tasks.workunit.client.1.smithi138.stdout:1/650: mknod d4/dd/d3b/d9f/db9/cd1 0 2024-05-17T21:53:07.826 INFO:tasks.workunit.client.1.smithi138.stdout:4/693: sync 2024-05-17T21:53:07.826 INFO:tasks.workunit.client.1.smithi138.stdout:6/695: sync 2024-05-17T21:53:07.826 INFO:tasks.workunit.client.1.smithi138.stdout:2/706: sync 2024-05-17T21:53:07.826 INFO:tasks.workunit.client.1.smithi138.stdout:7/670: sync 2024-05-17T21:53:07.826 INFO:tasks.workunit.client.1.smithi138.stdout:0/656: sync 2024-05-17T21:53:07.827 INFO:tasks.workunit.client.1.smithi138.stdout:8/625: getdents d0/dc/d1a/d44 0 2024-05-17T21:53:07.827 INFO:tasks.workunit.client.0.smithi107.stdout:2/709: dread d0/d9/d1c/fb5 [0,4194304] 0 2024-05-17T21:53:07.827 INFO:tasks.workunit.client.1.smithi138.stdout:2/707: read d3/d21/f2e [219469,56825] 0 2024-05-17T21:53:07.828 INFO:tasks.workunit.client.1.smithi138.stdout:0/657: truncate d3/d10/f7d 1903270 0 2024-05-17T21:53:07.829 INFO:tasks.workunit.client.1.smithi138.stdout:1/651: truncate d4/d2e/f9d 2939677 0 2024-05-17T21:53:07.829 INFO:tasks.workunit.client.1.smithi138.stdout:0/658: chown d3/d10/d38/d70/fa9 79846 1 2024-05-17T21:53:07.830 INFO:tasks.workunit.client.0.smithi107.stdout:3/905: dwrite d6/d8/d3e/d3f/dbe/fee [0,4194304] 0 2024-05-17T21:53:07.832 INFO:tasks.workunit.client.1.smithi138.stdout:2/708: fsync d3/d21/d2f/d3e/d63/f6e 0 2024-05-17T21:53:07.833 INFO:tasks.workunit.client.1.smithi138.stdout:1/652: rmdir d4/dd/d3b/da7 39 2024-05-17T21:53:07.834 INFO:tasks.workunit.client.0.smithi107.stdout:2/710: dwrite d0/d9/d11/f52 [0,4194304] 0 2024-05-17T21:53:07.838 INFO:tasks.workunit.client.0.smithi107.stdout:1/924: rename d4/df/d1a/d41/db2/dc8/c118 to d4/df/d11e/c124 0 2024-05-17T21:53:07.851 INFO:tasks.workunit.client.0.smithi107.stdout:5/877: getdents d5/db/d16/d2d/d4d/d6d 0 2024-05-17T21:53:07.853 INFO:tasks.workunit.client.1.smithi138.stdout:6/696: fdatasync d7/d69/db5/dc8/d34/f7c 0 2024-05-17T21:53:07.854 INFO:tasks.workunit.client.0.smithi107.stdout:8/845: sync 2024-05-17T21:53:07.854 INFO:tasks.workunit.client.0.smithi107.stdout:4/914: sync 2024-05-17T21:53:07.854 INFO:tasks.workunit.client.1.smithi138.stdout:1/653: dwrite d4/db/f39 [0,4194304] 0 2024-05-17T21:53:07.856 INFO:tasks.workunit.client.1.smithi138.stdout:4/694: creat d0/d2f/d49/d60/d1f/d7e/de8/ffa x:0 0 0 2024-05-17T21:53:07.861 INFO:tasks.workunit.client.0.smithi107.stdout:3/906: rmdir d6/d8/d10/d35 39 2024-05-17T21:53:07.862 INFO:tasks.workunit.client.1.smithi138.stdout:9/592: getdents d6/d7/d16/d29/da6 0 2024-05-17T21:53:07.865 INFO:tasks.workunit.client.1.smithi138.stdout:9/593: dread - d6/d7/d16/d29/d48/f4c zero size 2024-05-17T21:53:07.868 INFO:tasks.workunit.client.0.smithi107.stdout:0/975: rename d3/d58/d61/d94/lb8 to d3/dc/d22/d23/d97/d77/l140 0 2024-05-17T21:53:07.870 INFO:tasks.workunit.client.1.smithi138.stdout:6/697: dread - d7/d69/db5/dc8/d43/d3f/d79/f6d zero size 2024-05-17T21:53:07.873 INFO:tasks.workunit.client.0.smithi107.stdout:6/803: symlink d9/d16/d4d/lff 0 2024-05-17T21:53:07.873 INFO:tasks.workunit.client.1.smithi138.stdout:4/695: dread - d0/d3/d17/d21/d78/fac zero size 2024-05-17T21:53:07.874 INFO:tasks.workunit.client.1.smithi138.stdout:3/862: sync 2024-05-17T21:53:07.876 INFO:tasks.workunit.client.1.smithi138.stdout:3/863: dread d1/d62/dad/de/f8e [0,4194304] 0 2024-05-17T21:53:07.876 INFO:tasks.workunit.client.1.smithi138.stdout:7/671: link d0/d18/d64/d7c/da2/fb0 d0/d18/d40/d41/d51/d66/d9e/fd5 0 2024-05-17T21:53:07.879 INFO:tasks.workunit.client.1.smithi138.stdout:0/659: getdents d3/d10/d38/d3e 0 2024-05-17T21:53:07.879 INFO:tasks.workunit.client.1.smithi138.stdout:0/660: chown d3/d10/d38/d3e 1266 1 2024-05-17T21:53:07.879 INFO:tasks.workunit.client.0.smithi107.stdout:4/915: creat d3/db/d35/d55/d8a/f137 x:0 0 0 2024-05-17T21:53:07.881 INFO:tasks.workunit.client.1.smithi138.stdout:1/654: symlink d4/dd/d3b/ld2 0 2024-05-17T21:53:07.881 INFO:tasks.workunit.client.1.smithi138.stdout:9/594: dread d6/d7/f17 [0,4194304] 0 2024-05-17T21:53:07.881 INFO:tasks.workunit.client.1.smithi138.stdout:9/595: readlink d6/l91 0 2024-05-17T21:53:07.882 INFO:tasks.workunit.client.1.smithi138.stdout:9/596: chown d6/f1b 4220385 1 2024-05-17T21:53:07.882 INFO:tasks.workunit.client.0.smithi107.stdout:8/846: mkdir dc/d44/d4b/d52/dcb/d12b 0 2024-05-17T21:53:07.884 INFO:tasks.workunit.client.1.smithi138.stdout:6/698: rename d7/d8/ff to d7/d69/db5/dc8/d43/d5f/da4/fee 0 2024-05-17T21:53:07.885 INFO:tasks.workunit.client.0.smithi107.stdout:3/907: readlink d6/d8/d10/l6f 0 2024-05-17T21:53:07.887 INFO:tasks.workunit.client.0.smithi107.stdout:2/711: symlink d0/d9/d11/le4 0 2024-05-17T21:53:07.889 INFO:tasks.workunit.client.0.smithi107.stdout:1/925: symlink d4/df/d1a/d26/d72/de1/d112/l125 0 2024-05-17T21:53:07.890 INFO:tasks.workunit.client.1.smithi138.stdout:3/864: creat d1/d62/d9b/d107/f125 x:0 0 0 2024-05-17T21:53:07.892 INFO:tasks.workunit.client.0.smithi107.stdout:6/804: dread - d9/d58/f7e zero size 2024-05-17T21:53:07.893 INFO:tasks.workunit.client.0.smithi107.stdout:6/805: dread - d9/d58/d7b/da3/fd7 zero size 2024-05-17T21:53:07.894 INFO:tasks.workunit.client.1.smithi138.stdout:9/597: creat d6/d7/d16/d29/d48/d6e/d74/d75/fbb x:0 0 0 2024-05-17T21:53:07.896 INFO:tasks.workunit.client.0.smithi107.stdout:4/916: creat d3/db/d1e/d66/f138 x:0 0 0 2024-05-17T21:53:07.897 INFO:tasks.workunit.client.1.smithi138.stdout:6/699: rename d7/d8/dc2 to d7/d8/def 0 2024-05-17T21:53:07.903 INFO:tasks.workunit.client.1.smithi138.stdout:0/661: symlink d3/d10/d6d/le0 0 2024-05-17T21:53:07.904 INFO:tasks.workunit.client.0.smithi107.stdout:0/976: mkdir d3/d75/d9e/d103/d10d/d132/d141 0 2024-05-17T21:53:07.905 INFO:tasks.workunit.client.1.smithi138.stdout:1/655: mknod d4/df/d3a/dc4/cd3 0 2024-05-17T21:53:07.906 INFO:tasks.workunit.client.1.smithi138.stdout:1/656: write d4/dd/f15 [7043373,68292] 0 2024-05-17T21:53:07.907 INFO:tasks.workunit.client.0.smithi107.stdout:4/917: creat d3/db/d1e/d99/dd6/f139 x:0 0 0 2024-05-17T21:53:07.910 INFO:tasks.workunit.client.1.smithi138.stdout:6/700: rmdir d7/d69/db5/dc8/d43/d5f/da4 39 2024-05-17T21:53:07.910 INFO:tasks.workunit.client.0.smithi107.stdout:6/806: dread d9/d16/d4d/db8/fbb [0,4194304] 0 2024-05-17T21:53:07.912 INFO:tasks.workunit.client.1.smithi138.stdout:6/701: read d7/da0/dbe/fd6 [151605,22469] 0 2024-05-17T21:53:07.913 INFO:tasks.workunit.client.1.smithi138.stdout:7/672: creat d0/fd6 x:0 0 0 2024-05-17T21:53:07.915 INFO:tasks.workunit.client.1.smithi138.stdout:9/598: dread d6/d7/d16/d29/d48/d6f/f71 [4194304,4194304] 0 2024-05-17T21:53:07.916 INFO:tasks.workunit.client.1.smithi138.stdout:5/683: dwrite d1/f13 [0,4194304] 0 2024-05-17T21:53:07.921 INFO:tasks.workunit.client.1.smithi138.stdout:1/657: symlink d4/dd/d64/ld4 0 2024-05-17T21:53:07.926 INFO:tasks.workunit.client.0.smithi107.stdout:9/899: dwrite d0/dd/d21/d29/d57/d5f/dc6/f11e [0,4194304] 0 2024-05-17T21:53:07.928 INFO:tasks.workunit.client.1.smithi138.stdout:7/673: mkdir d0/d9d/dd7 0 2024-05-17T21:53:07.928 INFO:tasks.workunit.client.0.smithi107.stdout:3/908: mkdir d6/d8/d10/d35/d91/de4/d122 0 2024-05-17T21:53:07.929 INFO:tasks.workunit.client.1.smithi138.stdout:3/865: creat d1/d4f/f126 x:0 0 0 2024-05-17T21:53:07.930 INFO:tasks.workunit.client.1.smithi138.stdout:2/709: write d3/d81/d96/fb4 [4773135,64576] 0 2024-05-17T21:53:07.933 INFO:tasks.workunit.client.0.smithi107.stdout:1/926: creat d4/df/d1a/d26/f126 x:0 0 0 2024-05-17T21:53:07.933 INFO:tasks.workunit.client.0.smithi107.stdout:7/841: write d5/d16/d3b/d45/d91/db3/fd9 [764467,126659] 0 2024-05-17T21:53:07.933 INFO:tasks.workunit.client.1.smithi138.stdout:3/866: write d1/d3/d16/d3e/d71/d101/f115 [285582,100531] 0 2024-05-17T21:53:07.934 INFO:tasks.workunit.client.0.smithi107.stdout:1/927: stat d4/df/d14/f4c 0 2024-05-17T21:53:07.936 INFO:tasks.workunit.client.1.smithi138.stdout:5/684: creat d1/d53/db1/fdd x:0 0 0 2024-05-17T21:53:07.938 INFO:tasks.workunit.client.0.smithi107.stdout:4/918: rename d3/d4a/d47/da3/f114 to d3/d6/dd5/f13a 0 2024-05-17T21:53:07.942 INFO:tasks.workunit.client.1.smithi138.stdout:1/658: creat d4/dd/d3b/d6c/dcf/fd5 x:0 0 0 2024-05-17T21:53:07.943 INFO:tasks.workunit.client.1.smithi138.stdout:1/659: stat d4/df/c79 0 2024-05-17T21:53:07.943 INFO:tasks.workunit.client.0.smithi107.stdout:6/807: truncate d9/d16/d4d/f8b 316290 0 2024-05-17T21:53:07.946 INFO:tasks.workunit.client.0.smithi107.stdout:9/900: creat d0/dd/d1b/de7/f125 x:0 0 0 2024-05-17T21:53:07.946 INFO:tasks.workunit.client.1.smithi138.stdout:8/626: truncate d0/f2 6525586 0 2024-05-17T21:53:07.951 INFO:tasks.workunit.client.0.smithi107.stdout:0/977: creat d3/dc/d22/f142 x:0 0 0 2024-05-17T21:53:07.951 INFO:tasks.workunit.client.0.smithi107.stdout:5/878: write d5/db/d16/d2a/fab [572043,111786] 0 2024-05-17T21:53:07.951 INFO:tasks.workunit.client.1.smithi138.stdout:6/702: creat d7/d69/db5/dc8/d43/d5f/da4/dcc/ff0 x:0 0 0 2024-05-17T21:53:07.952 INFO:tasks.workunit.client.0.smithi107.stdout:5/879: fsync d5/db/d16/f9d 0 2024-05-17T21:53:07.953 INFO:tasks.workunit.client.0.smithi107.stdout:7/842: mknod d5/df/d11/d1c/d31/d39/c124 0 2024-05-17T21:53:07.954 INFO:tasks.workunit.client.1.smithi138.stdout:5/685: mknod d1/d53/d96/da0/cde 0 2024-05-17T21:53:07.955 INFO:tasks.workunit.client.0.smithi107.stdout:4/919: dread d3/d6/d26/d36/f6e [0,4194304] 0 2024-05-17T21:53:07.955 INFO:tasks.workunit.client.1.smithi138.stdout:5/686: truncate d1/d2c/dbf/fc3 439996 0 2024-05-17T21:53:07.958 INFO:tasks.workunit.client.1.smithi138.stdout:1/660: symlink d4/d93/ld6 0 2024-05-17T21:53:07.960 INFO:tasks.workunit.client.1.smithi138.stdout:1/661: read d4/df/d3a/fbe [2304738,23560] 0 2024-05-17T21:53:07.961 INFO:tasks.workunit.client.1.smithi138.stdout:7/674: creat d0/d18/d64/d7c/da2/dc5/dcf/fd8 x:0 0 0 2024-05-17T21:53:07.961 INFO:tasks.workunit.client.0.smithi107.stdout:9/901: fsync d0/dd/d1b/f74 0 2024-05-17T21:53:07.962 INFO:tasks.workunit.client.1.smithi138.stdout:4/696: dwrite d0/d2f/d49/d60/f41 [0,4194304] 0 2024-05-17T21:53:07.962 INFO:tasks.workunit.client.0.smithi107.stdout:9/902: chown d0/dd/d1b/cbe 114899 1 2024-05-17T21:53:07.964 INFO:tasks.workunit.client.1.smithi138.stdout:7/675: read d0/d18/d40/d41/d51/f69 [1601034,27729] 0 2024-05-17T21:53:07.968 INFO:tasks.workunit.client.1.smithi138.stdout:5/687: creat d1/d2c/d63/db5/d1b/db7/fdf x:0 0 0 2024-05-17T21:53:07.968 INFO:tasks.workunit.client.1.smithi138.stdout:5/688: fsync d1/f13 0 2024-05-17T21:53:07.970 INFO:tasks.workunit.client.0.smithi107.stdout:8/847: truncate dc/d21/daa/dc5/ff0 3951999 0 2024-05-17T21:53:07.972 INFO:tasks.workunit.client.0.smithi107.stdout:8/848: write dc/d21/d27/d34/f6e [2372240,64104] 0 2024-05-17T21:53:07.973 INFO:tasks.workunit.client.0.smithi107.stdout:3/909: rename d6/d8/d10/d35/d36/d4c/c9d to d6/d2a/d2d/d48/d4f/d116/c123 0 2024-05-17T21:53:07.977 INFO:tasks.workunit.client.1.smithi138.stdout:1/662: creat d4/df/d4e/fd7 x:0 0 0 2024-05-17T21:53:07.978 INFO:tasks.workunit.client.0.smithi107.stdout:9/903: symlink d0/dd/d21/d29/d57/d5f/l126 0 2024-05-17T21:53:07.979 INFO:tasks.workunit.client.0.smithi107.stdout:2/712: dwrite d0/d9/f46 [0,4194304] 0 2024-05-17T21:53:07.981 INFO:tasks.workunit.client.0.smithi107.stdout:5/880: mkdir d5/db/d16/d2d/d123 0 2024-05-17T21:53:07.994 INFO:tasks.workunit.client.0.smithi107.stdout:6/808: link d9/d16/d4d/lff d9/d58/d66/dd5/l100 0 2024-05-17T21:53:07.997 INFO:tasks.workunit.client.0.smithi107.stdout:9/904: mknod d0/dd/d69/dee/c127 0 2024-05-17T21:53:07.999 INFO:tasks.workunit.client.0.smithi107.stdout:0/978: creat d3/d75/d9e/d103/d10d/d132/f143 x:0 0 0 2024-05-17T21:53:07.999 INFO:tasks.workunit.client.0.smithi107.stdout:0/979: dread - d3/dc/d22/f142 zero size 2024-05-17T21:53:08.000 INFO:tasks.workunit.client.0.smithi107.stdout:0/980: dread - d3/d24/d6f/db5/da5/fd8 zero size 2024-05-17T21:53:08.002 INFO:tasks.workunit.client.1.smithi138.stdout:2/710: rename d3/d21/d2f/l44 to d3/d21/d2f/d3e/d55/d99/d84/le8 0 2024-05-17T21:53:08.003 INFO:tasks.workunit.client.0.smithi107.stdout:5/881: mknod d5/db/d16/d54/d59/d65/d48/c124 0 2024-05-17T21:53:08.005 INFO:tasks.workunit.client.1.smithi138.stdout:0/662: link d3/d3d/d48/l50 d3/d3d/d51/le1 0 2024-05-17T21:53:08.006 INFO:tasks.workunit.client.1.smithi138.stdout:2/711: dread d3/d21/f67 [0,4194304] 0 2024-05-17T21:53:08.009 INFO:tasks.workunit.client.0.smithi107.stdout:6/809: creat d9/d16/d22/daf/dc7/f101 x:0 0 0 2024-05-17T21:53:08.011 INFO:tasks.workunit.client.0.smithi107.stdout:9/905: write d0/dd/d21/d29/d3e/d45/d68/fdc [448291,106483] 0 2024-05-17T21:53:08.013 INFO:tasks.workunit.client.0.smithi107.stdout:2/713: mknod d0/d9/ce5 0 2024-05-17T21:53:08.015 INFO:tasks.workunit.client.0.smithi107.stdout:0/981: write d3/d75/dd0/f112 [1963564,86608] 0 2024-05-17T21:53:08.018 INFO:tasks.workunit.client.0.smithi107.stdout:8/849: write dc/d21/d35/f61 [938272,24153] 0 2024-05-17T21:53:08.019 INFO:tasks.workunit.client.0.smithi107.stdout:8/850: write dc/d21/fd3 [1005689,2251] 0 2024-05-17T21:53:08.020 INFO:tasks.workunit.client.0.smithi107.stdout:6/810: symlink d9/d16/d4d/db8/dc9/dd0/l102 0 2024-05-17T21:53:08.022 INFO:tasks.workunit.client.1.smithi138.stdout:5/689: dwrite d1/d2c/f8b [0,4194304] 0 2024-05-17T21:53:08.024 INFO:tasks.workunit.client.1.smithi138.stdout:8/627: write d0/f2 [6849593,6329] 0 2024-05-17T21:53:08.025 INFO:tasks.workunit.client.1.smithi138.stdout:8/628: chown d0/dc/d1a/d51/d82/f8e 8298 1 2024-05-17T21:53:08.025 INFO:tasks.workunit.client.0.smithi107.stdout:8/851: dwrite dc/d21/d35/fda [0,4194304] 0 2024-05-17T21:53:08.028 INFO:tasks.workunit.client.0.smithi107.stdout:8/852: write dc/d21/d27/d34/d4c/d7e/f3b [2597325,2812] 0 2024-05-17T21:53:08.032 INFO:tasks.workunit.client.0.smithi107.stdout:1/928: rename d4/df/d14/d2e/d43/fc6 to d4/df/d1a/d26/f127 0 2024-05-17T21:53:08.036 INFO:tasks.workunit.client.1.smithi138.stdout:6/703: rename d7/d69/c8d to d7/d69/db5/dc8/d43/cf1 0 2024-05-17T21:53:08.036 INFO:tasks.workunit.client.1.smithi138.stdout:1/663: read d4/d2e/f9d [2213456,38594] 0 2024-05-17T21:53:08.038 INFO:tasks.workunit.client.1.smithi138.stdout:5/690: symlink d1/d53/le0 0 2024-05-17T21:53:08.039 INFO:tasks.workunit.client.0.smithi107.stdout:6/811: rmdir d9/d81 39 2024-05-17T21:53:08.039 INFO:tasks.workunit.client.0.smithi107.stdout:6/812: fsync d9/d16/f64 0 2024-05-17T21:53:08.041 INFO:tasks.workunit.client.0.smithi107.stdout:0/982: write d3/dc/d22/f98 [1021270,116651] 0 2024-05-17T21:53:08.042 INFO:tasks.workunit.client.0.smithi107.stdout:9/906: mkdir d0/dd/d21/d29/d57/d5f/d112/d128 0 2024-05-17T21:53:08.044 INFO:tasks.workunit.client.1.smithi138.stdout:2/712: dwrite d3/da0/daa/dce/dd0/fb5 [0,4194304] 0 2024-05-17T21:53:08.045 INFO:tasks.workunit.client.1.smithi138.stdout:6/704: symlink d7/d69/db5/dc8/d43/d63/lf2 0 2024-05-17T21:53:08.055 INFO:tasks.workunit.client.1.smithi138.stdout:6/705: dread d7/d8/fb4 [0,4194304] 0 2024-05-17T21:53:08.070 INFO:tasks.workunit.client.1.smithi138.stdout:1/664: read d4/df/f12 [2100472,11002] 0 2024-05-17T21:53:08.070 INFO:tasks.workunit.client.1.smithi138.stdout:0/663: getdents d3/d10/d38/d70/d80/d94 0 2024-05-17T21:53:08.072 INFO:tasks.workunit.client.1.smithi138.stdout:8/629: link d0/dc/fb9 d0/dc/d3b/fba 0 2024-05-17T21:53:08.073 INFO:tasks.workunit.client.0.smithi107.stdout:2/714: write d0/d9/d11/f5f [2277411,64767] 0 2024-05-17T21:53:08.077 INFO:tasks.workunit.client.0.smithi107.stdout:4/920: rename d3/db/d35/d5e/dff/f88 to d3/d4a/d47/f13b 0 2024-05-17T21:53:08.084 INFO:tasks.workunit.client.1.smithi138.stdout:2/713: write d3/da0/daa/dce/dd0/f9c [2661454,46458] 0 2024-05-17T21:53:08.086 INFO:tasks.workunit.client.1.smithi138.stdout:8/630: creat d0/d63/fbb x:0 0 0 2024-05-17T21:53:08.087 INFO:tasks.workunit.client.0.smithi107.stdout:8/853: mknod dc/d21/d27/d34/c12c 0 2024-05-17T21:53:08.088 INFO:tasks.workunit.client.1.smithi138.stdout:4/697: rename d0/d1b/f3e to d0/d2f/d49/d60/d1f/ffb 0 2024-05-17T21:53:08.089 INFO:tasks.workunit.client.0.smithi107.stdout:2/715: dread d0/d9/d11/d18/dad/fc3 [0,4194304] 0 2024-05-17T21:53:08.090 INFO:tasks.workunit.client.1.smithi138.stdout:1/665: getdents d4/dd/db4/dba 0 2024-05-17T21:53:08.090 INFO:tasks.workunit.client.0.smithi107.stdout:0/983: dwrite d3/dc/d22/d23/d97/d77/d91/fb6 [0,4194304] 0 2024-05-17T21:53:08.091 INFO:tasks.workunit.client.0.smithi107.stdout:4/921: symlink d3/db/l13c 0 2024-05-17T21:53:08.092 INFO:tasks.workunit.client.1.smithi138.stdout:6/706: truncate d7/d69/db5/dc8/f25 1005032 0 2024-05-17T21:53:08.092 INFO:tasks.workunit.client.0.smithi107.stdout:5/882: getdents d5/db/dc 0 2024-05-17T21:53:08.094 INFO:tasks.workunit.client.1.smithi138.stdout:7/676: rename d0/l16 to d0/d9d/ld9 0 2024-05-17T21:53:08.094 INFO:tasks.workunit.client.1.smithi138.stdout:7/677: fsync d0/d39/f48 0 2024-05-17T21:53:08.096 INFO:tasks.workunit.client.1.smithi138.stdout:7/678: rename d0/d18/d40/d5f to d0/d18/d40/d5f/dbe/dda 22 2024-05-17T21:53:08.097 INFO:tasks.workunit.client.1.smithi138.stdout:1/666: creat d4/db/d60/fd8 x:0 0 0 2024-05-17T21:53:08.098 INFO:tasks.workunit.client.1.smithi138.stdout:1/667: stat d4/db/d29 0 2024-05-17T21:53:08.098 INFO:tasks.workunit.client.1.smithi138.stdout:2/714: mknod d3/da0/ce9 0 2024-05-17T21:53:08.100 INFO:tasks.workunit.client.1.smithi138.stdout:8/631: mkdir d0/dc/d3b/da4/dbc 0 2024-05-17T21:53:08.100 INFO:tasks.workunit.client.0.smithi107.stdout:0/984: chown d3/l53 7 1 2024-05-17T21:53:08.102 INFO:tasks.workunit.client.0.smithi107.stdout:1/929: creat d4/df/d1a/d26/d72/dbc/f128 x:0 0 0 2024-05-17T21:53:08.103 INFO:tasks.workunit.client.0.smithi107.stdout:2/716: creat d0/d9/d11/d18/dad/fe6 x:0 0 0 2024-05-17T21:53:08.104 INFO:tasks.workunit.client.1.smithi138.stdout:7/679: readlink d0/d18/d64/d7c/la4 0 2024-05-17T21:53:08.105 INFO:tasks.workunit.client.1.smithi138.stdout:7/680: read d0/f19 [958875,15884] 0 2024-05-17T21:53:08.106 INFO:tasks.workunit.client.0.smithi107.stdout:4/922: rmdir d3/d6/d26/d3b/d91 39 2024-05-17T21:53:08.108 INFO:tasks.workunit.client.0.smithi107.stdout:5/883: rmdir d5/db/d16/d2d/d40/dc2 39 2024-05-17T21:53:08.109 INFO:tasks.workunit.client.0.smithi107.stdout:8/854: dread dc/f18 [4194304,4194304] 0 2024-05-17T21:53:08.111 INFO:tasks.workunit.client.1.smithi138.stdout:4/698: write d0/d2f/d49/d60/d1f/f4d [422886,54360] 0 2024-05-17T21:53:08.112 INFO:tasks.workunit.client.0.smithi107.stdout:4/923: dwrite d3/db/d1e/d66/f10c [0,4194304] 0 2024-05-17T21:53:08.113 INFO:tasks.workunit.client.0.smithi107.stdout:3/910: rename d6/d2a/d2d/d48/c5c to d6/d8/d3e/d3f/c124 0 2024-05-17T21:53:08.113 INFO:tasks.workunit.client.1.smithi138.stdout:4/699: write d0/d3/d17/d5e/f8e [2944943,5278] 0 2024-05-17T21:53:08.120 INFO:tasks.workunit.client.0.smithi107.stdout:2/717: write d0/fa [863592,29141] 0 2024-05-17T21:53:08.120 INFO:tasks.workunit.client.1.smithi138.stdout:2/715: mknod d3/d21/d2f/d3e/d55/cea 0 2024-05-17T21:53:08.122 INFO:tasks.workunit.client.1.smithi138.stdout:2/716: write d3/da0/daa/dce/fdc [210289,20412] 0 2024-05-17T21:53:08.124 INFO:tasks.workunit.client.1.smithi138.stdout:8/632: unlink d0/d63/c98 0 2024-05-17T21:53:08.124 INFO:tasks.workunit.client.1.smithi138.stdout:8/633: stat d0/fb6 0 2024-05-17T21:53:08.125 INFO:tasks.workunit.client.1.smithi138.stdout:8/634: truncate d0/dc/fa7 1004177 0 2024-05-17T21:53:08.126 INFO:tasks.workunit.client.0.smithi107.stdout:5/884: mkdir d5/db/dc/d1e/d125 0 2024-05-17T21:53:08.128 INFO:tasks.workunit.client.0.smithi107.stdout:8/855: stat dc/d21/d27/d2f/la9 0 2024-05-17T21:53:08.129 INFO:tasks.workunit.client.0.smithi107.stdout:7/843: sync 2024-05-17T21:53:08.130 INFO:tasks.workunit.client.1.smithi138.stdout:6/707: dwrite d7/d69/db5/dc8/d43/d5f/da4/fee [0,4194304] 0 2024-05-17T21:53:08.130 INFO:tasks.workunit.client.0.smithi107.stdout:4/924: symlink d3/d6/d26/d36/d7e/l13d 0 2024-05-17T21:53:08.133 INFO:tasks.workunit.client.1.smithi138.stdout:4/700: read d0/d2f/d49/d60/d1f/d32/f43 [6401496,6036] 0 2024-05-17T21:53:08.134 INFO:tasks.workunit.client.1.smithi138.stdout:1/668: truncate d4/db/d29/d5d/f6e 1488925 0 2024-05-17T21:53:08.135 INFO:tasks.workunit.client.1.smithi138.stdout:4/701: write d0/d2f/d49/d60/f41 [2322476,23752] 0 2024-05-17T21:53:08.135 INFO:tasks.workunit.client.0.smithi107.stdout:0/985: dwrite d3/dc/d22/d23/f29 [8388608,4194304] 0 2024-05-17T21:53:08.136 INFO:tasks.workunit.client.1.smithi138.stdout:1/669: chown d4/dd/c56 5 1 2024-05-17T21:53:08.136 INFO:tasks.workunit.client.1.smithi138.stdout:4/702: write d0/d3/d17/d21/d78/fcd [495718,83046] 0 2024-05-17T21:53:08.137 INFO:tasks.workunit.client.1.smithi138.stdout:4/703: stat d0/d2f 0 2024-05-17T21:53:08.139 INFO:tasks.workunit.client.0.smithi107.stdout:5/885: dwrite d5/db/d16/d2d/d40/d5a/db4/fb9 [0,4194304] 0 2024-05-17T21:53:08.143 INFO:tasks.workunit.client.0.smithi107.stdout:8/856: fdatasync dc/f86 0 2024-05-17T21:53:08.143 INFO:tasks.workunit.client.1.smithi138.stdout:6/708: mkdir d7/d69/db5/dc8/d43/d3f/df3 0 2024-05-17T21:53:08.144 INFO:tasks.workunit.client.0.smithi107.stdout:7/844: chown d5/d16/c8b 397 1 2024-05-17T21:53:08.145 INFO:tasks.workunit.client.0.smithi107.stdout:3/911: mkdir d6/d8/d3e/dc9/d11f/d125 0 2024-05-17T21:53:08.146 INFO:tasks.workunit.client.0.smithi107.stdout:0/986: truncate d3/d75/d9e/d103/d10d/f138 243181 0 2024-05-17T21:53:08.149 INFO:tasks.workunit.client.0.smithi107.stdout:1/930: dread d4/df/d1a/d26/d72/dbc/fad [0,4194304] 0 2024-05-17T21:53:08.150 INFO:tasks.workunit.client.0.smithi107.stdout:6/813: rename d9/f1a to d9/d16/d22/f103 0 2024-05-17T21:53:08.150 INFO:tasks.workunit.client.0.smithi107.stdout:4/925: dwrite d3/db/d1e/ff1 [0,4194304] 0 2024-05-17T21:53:08.157 INFO:tasks.workunit.client.1.smithi138.stdout:4/704: dread - d0/d1b/d88/fc6 zero size 2024-05-17T21:53:08.167 INFO:tasks.workunit.client.0.smithi107.stdout:7/845: mknod d5/d16/d3b/c125 0 2024-05-17T21:53:08.169 INFO:tasks.workunit.client.0.smithi107.stdout:0/987: mkdir d3/dc/dae/dc0/de1/d10b/d144 0 2024-05-17T21:53:08.171 INFO:tasks.workunit.client.1.smithi138.stdout:7/681: getdents d0/d18/d40/d41/d51/d66/d9e 0 2024-05-17T21:53:08.171 INFO:tasks.workunit.client.0.smithi107.stdout:0/988: write d3/d75/d9e/d103/d10d/d132/dab/fb4 [754218,78095] 0 2024-05-17T21:53:08.172 INFO:tasks.workunit.client.0.smithi107.stdout:2/718: write d0/d9/d11/df/d5a/d6d/f74 [789382,60429] 0 2024-05-17T21:53:08.174 INFO:tasks.workunit.client.0.smithi107.stdout:2/719: write d0/d9/d11/d15/f57 [261781,38223] 0 2024-05-17T21:53:08.175 INFO:tasks.workunit.client.0.smithi107.stdout:2/720: stat d0/d9/d1c/d37/f98 0 2024-05-17T21:53:08.178 INFO:tasks.workunit.client.0.smithi107.stdout:5/886: link d5/db/d16/f21 d5/db/d16/d2d/d4d/f126 0 2024-05-17T21:53:08.179 INFO:tasks.workunit.client.1.smithi138.stdout:6/709: creat d7/d8/de9/ff4 x:0 0 0 2024-05-17T21:53:08.179 INFO:tasks.workunit.client.1.smithi138.stdout:1/670: write d4/db/f22 [4740572,33652] 0 2024-05-17T21:53:08.181 INFO:tasks.workunit.client.0.smithi107.stdout:8/857: write dc/d44/d4b/d52/d70/d76/f78 [33172,25187] 0 2024-05-17T21:53:08.181 INFO:tasks.workunit.client.1.smithi138.stdout:8/635: truncate d0/fb 9607214 0 2024-05-17T21:53:08.183 INFO:tasks.workunit.client.1.smithi138.stdout:2/717: getdents d3/d21/d2f/d3e/d58 0 2024-05-17T21:53:08.184 INFO:tasks.workunit.client.0.smithi107.stdout:7/846: fsync d5/d16/d3b/d4f/fe2 0 2024-05-17T21:53:08.185 INFO:tasks.workunit.client.0.smithi107.stdout:0/989: dread d3/dc/f14 [0,4194304] 0 2024-05-17T21:53:08.186 INFO:tasks.workunit.client.1.smithi138.stdout:4/705: symlink d0/d1b/lfc 0 2024-05-17T21:53:08.188 INFO:tasks.workunit.client.1.smithi138.stdout:7/682: creat d0/d9d/dd7/fdb x:0 0 0 2024-05-17T21:53:08.188 INFO:tasks.workunit.client.1.smithi138.stdout:8/636: dwrite d0/d11/fa6 [0,4194304] 0 2024-05-17T21:53:08.192 INFO:tasks.workunit.client.0.smithi107.stdout:6/814: fdatasync d9/d16/d22/d5f/d6f/fa7 0 2024-05-17T21:53:08.193 INFO:tasks.workunit.client.1.smithi138.stdout:2/718: creat d3/da0/daa/feb x:0 0 0 2024-05-17T21:53:08.196 INFO:tasks.workunit.client.0.smithi107.stdout:1/931: write d4/df/d14/d2e/d43/d47/fa3 [507217,125172] 0 2024-05-17T21:53:08.197 INFO:tasks.workunit.client.0.smithi107.stdout:4/926: dwrite d3/d6/fe0 [0,4194304] 0 2024-05-17T21:53:08.201 INFO:tasks.workunit.client.1.smithi138.stdout:7/683: fsync d0/f27 0 2024-05-17T21:53:08.202 INFO:tasks.workunit.client.1.smithi138.stdout:7/684: chown d0/d18/d40/d41/d51/d66/f88 91 1 2024-05-17T21:53:08.208 INFO:tasks.workunit.client.1.smithi138.stdout:8/637: creat d0/dc/d3b/d4d/fbd x:0 0 0 2024-05-17T21:53:08.209 INFO:tasks.workunit.client.0.smithi107.stdout:3/912: creat d6/d2a/d2d/d48/f126 x:0 0 0 2024-05-17T21:53:08.211 INFO:tasks.workunit.client.1.smithi138.stdout:2/719: fdatasync d3/f6f 0 2024-05-17T21:53:08.212 INFO:tasks.workunit.client.1.smithi138.stdout:9/599: sync 2024-05-17T21:53:08.215 INFO:tasks.workunit.client.0.smithi107.stdout:3/913: dwrite f3 [0,4194304] 0 2024-05-17T21:53:08.216 INFO:tasks.workunit.client.0.smithi107.stdout:3/914: write d6/d8/f12 [1206916,69238] 0 2024-05-17T21:53:08.217 INFO:tasks.workunit.client.0.smithi107.stdout:3/915: write d6/fef [198880,74336] 0 2024-05-17T21:53:08.219 INFO:tasks.workunit.client.0.smithi107.stdout:3/916: fdatasync d6/d2a/d2d/d37/f115 0 2024-05-17T21:53:08.219 INFO:tasks.workunit.client.1.smithi138.stdout:7/685: rename d0/l34 to d0/d9d/dd7/ldc 0 2024-05-17T21:53:08.220 INFO:tasks.workunit.client.1.smithi138.stdout:7/686: chown d0/d18/d40/d41/d51/d66/d9e 21 1 2024-05-17T21:53:08.221 INFO:tasks.workunit.client.1.smithi138.stdout:7/687: stat d0/d18/d64/d7c/da2/fb0 0 2024-05-17T21:53:08.222 INFO:tasks.workunit.client.1.smithi138.stdout:8/638: rmdir d0/dc/d8d 39 2024-05-17T21:53:08.222 INFO:tasks.workunit.client.1.smithi138.stdout:7/688: truncate d0/f91 4444120 0 2024-05-17T21:53:08.224 INFO:tasks.workunit.client.0.smithi107.stdout:9/907: sync 2024-05-17T21:53:08.224 INFO:tasks.workunit.client.1.smithi138.stdout:8/639: dread d0/dc/d3b/d4d/f77 [0,4194304] 0 2024-05-17T21:53:08.224 INFO:tasks.workunit.client.1.smithi138.stdout:7/689: chown d0/d18/d40/d41/d51/d81/fa3 1222 1 2024-05-17T21:53:08.225 INFO:tasks.workunit.client.1.smithi138.stdout:2/720: symlink d3/d21/d2f/d3e/d58/d77/da4/dc5/lec 0 2024-05-17T21:53:08.227 INFO:tasks.workunit.client.0.smithi107.stdout:6/815: mkdir d9/d16/d4d/d85/d96/d104 0 2024-05-17T21:53:08.229 INFO:tasks.workunit.client.1.smithi138.stdout:6/710: write d7/d69/db5/dc8/d43/d3f/f56 [2131296,12428] 0 2024-05-17T21:53:08.231 INFO:tasks.workunit.client.0.smithi107.stdout:4/927: mkdir d3/db/d1e/d66/d13e 0 2024-05-17T21:53:08.236 INFO:tasks.workunit.client.1.smithi138.stdout:9/600: truncate d6/d7/fc 217924 0 2024-05-17T21:53:08.244 INFO:tasks.workunit.client.1.smithi138.stdout:9/601: dwrite d6/d7/f99 [0,4194304] 0 2024-05-17T21:53:08.245 INFO:tasks.workunit.client.0.smithi107.stdout:3/917: mknod d6/d2a/d2d/d69/dad/de1/c127 0 2024-05-17T21:53:08.246 INFO:tasks.workunit.client.1.smithi138.stdout:4/706: dwrite d0/d3/d17/f63 [0,4194304] 0 2024-05-17T21:53:08.252 INFO:tasks.workunit.client.0.smithi107.stdout:6/816: unlink d9/d16/d22/f103 0 2024-05-17T21:53:08.253 INFO:tasks.workunit.client.0.smithi107.stdout:6/817: fdatasync d9/d16/d4d/faa 0 2024-05-17T21:53:08.256 INFO:tasks.workunit.client.1.smithi138.stdout:7/690: link d0/d18/d40/d41/d51/d81/l8c d0/d39/d78/ldd 0 2024-05-17T21:53:08.257 INFO:tasks.workunit.client.0.smithi107.stdout:1/932: mkdir d4/df/d14/d129 0 2024-05-17T21:53:08.259 INFO:tasks.workunit.client.0.smithi107.stdout:2/721: link d0/d26/c48 d0/d9/d1c/dd4/ce7 0 2024-05-17T21:53:08.260 INFO:tasks.workunit.client.0.smithi107.stdout:5/887: link d5/db/c68 d5/db/d16/d54/da0/c127 0 2024-05-17T21:53:08.263 INFO:tasks.workunit.client.0.smithi107.stdout:2/722: dwrite d0/d26/ddd/f84 [0,4194304] 0 2024-05-17T21:53:08.265 INFO:tasks.workunit.client.0.smithi107.stdout:8/858: rename dc/d21/d27/d2f/l95 to dc/d44/d4b/l12d 0 2024-05-17T21:53:08.267 INFO:tasks.workunit.client.0.smithi107.stdout:3/918: chown d6/d2a/d2d/d69/dad/de1/f11a 67646268 1 2024-05-17T21:53:08.273 INFO:tasks.workunit.client.0.smithi107.stdout:0/990: truncate d3/d58/d61/d94/f95 620880 0 2024-05-17T21:53:08.274 INFO:tasks.workunit.client.0.smithi107.stdout:5/888: dread d5/db/dc/d1e/fb5 [0,4194304] 0 2024-05-17T21:53:08.274 INFO:tasks.workunit.client.0.smithi107.stdout:6/818: creat d9/d58/d66/de2/f105 x:0 0 0 2024-05-17T21:53:08.275 INFO:tasks.workunit.client.1.smithi138.stdout:7/691: creat d0/d18/d40/d41/d51/d66/d9e/fde x:0 0 0 2024-05-17T21:53:08.276 INFO:tasks.workunit.client.0.smithi107.stdout:1/933: creat d4/df/d1a/d26/d72/d85/f12a x:0 0 0 2024-05-17T21:53:08.277 INFO:tasks.workunit.client.1.smithi138.stdout:2/721: getdents d3/d21/d2f/d3e 0 2024-05-17T21:53:08.277 INFO:tasks.workunit.client.1.smithi138.stdout:7/692: dread - d0/d18/d40/d41/d50/faa zero size 2024-05-17T21:53:08.281 INFO:tasks.workunit.client.0.smithi107.stdout:7/847: rename d5/df/d11/d1d/f8c to d5/df/d11/d1c/d2f/d53/d121/f126 0 2024-05-17T21:53:08.284 INFO:tasks.workunit.client.0.smithi107.stdout:3/919: creat d6/d8/d3e/dc9/f128 x:0 0 0 2024-05-17T21:53:08.285 INFO:tasks.workunit.client.1.smithi138.stdout:6/711: write d7/da0/fbf [920233,108046] 0 2024-05-17T21:53:08.285 INFO:tasks.workunit.client.1.smithi138.stdout:6/712: read - d7/d8/de9/ff4 zero size 2024-05-17T21:53:08.286 INFO:tasks.workunit.client.1.smithi138.stdout:6/713: readlink d7/d69/db5/dc8/d43/d5f/d6a/lb7 0 2024-05-17T21:53:08.287 INFO:tasks.workunit.client.1.smithi138.stdout:2/722: mkdir d3/d21/d2f/d3e/d55/d99/d84/ded 0 2024-05-17T21:53:08.288 INFO:tasks.workunit.client.1.smithi138.stdout:2/723: readlink d3/d21/d2f/d3e/d63/l78 0 2024-05-17T21:53:08.288 INFO:tasks.workunit.client.0.smithi107.stdout:0/991: rmdir d3/dc/d22/d3d/d56 39 2024-05-17T21:53:08.291 INFO:tasks.workunit.client.0.smithi107.stdout:6/819: mkdir d9/d58/d7b/d106 0 2024-05-17T21:53:08.294 INFO:tasks.workunit.client.0.smithi107.stdout:1/934: mkdir d4/de/d12b 0 2024-05-17T21:53:08.295 INFO:tasks.workunit.client.1.smithi138.stdout:4/707: write d0/d2f/d49/d60/f89 [403082,73588] 0 2024-05-17T21:53:08.295 INFO:tasks.workunit.client.1.smithi138.stdout:6/714: truncate d7/d69/db5/dc8/d43/fc4 350169 0 2024-05-17T21:53:08.297 INFO:tasks.workunit.client.1.smithi138.stdout:9/602: dwrite d6/d7/f82 [0,4194304] 0 2024-05-17T21:53:08.299 INFO:tasks.workunit.client.1.smithi138.stdout:2/724: link d3/f15 d3/da0/daa/fee 0 2024-05-17T21:53:08.302 INFO:tasks.workunit.client.0.smithi107.stdout:2/723: write d0/d26/d64/f8a [390402,122671] 0 2024-05-17T21:53:08.304 INFO:tasks.workunit.client.1.smithi138.stdout:4/708: chown d0/d2f/d49/d60/d1f/d7e/lf9 3591499 1 2024-05-17T21:53:08.306 INFO:tasks.workunit.client.1.smithi138.stdout:6/715: symlink d7/d69/db5/dc8/d34/lf5 0 2024-05-17T21:53:08.308 INFO:tasks.workunit.client.0.smithi107.stdout:4/928: rename d3/db/d35/f101 to d3/db/d1e/d66/d13e/f13f 0 2024-05-17T21:53:08.309 INFO:tasks.workunit.client.0.smithi107.stdout:9/908: link d0/dd/d3d/d9d/l102 d0/dd/d1f/l129 0 2024-05-17T21:53:08.309 INFO:tasks.workunit.client.1.smithi138.stdout:4/709: fsync d0/d2f/d49/d60/f51 0 2024-05-17T21:53:08.310 INFO:tasks.workunit.client.0.smithi107.stdout:6/820: dread d9/d16/d4d/db8/dc9/dd0/f9a [0,4194304] 0 2024-05-17T21:53:08.310 INFO:tasks.workunit.client.1.smithi138.stdout:4/710: fsync d0/d3/db/fd3 0 2024-05-17T21:53:08.311 INFO:tasks.workunit.client.0.smithi107.stdout:9/909: fsync d0/dd/d2f/d87/d89/de0/f110 0 2024-05-17T21:53:08.311 INFO:tasks.workunit.client.0.smithi107.stdout:6/821: dread - d9/d16/d4d/d85/d96/ffb zero size 2024-05-17T21:53:08.312 INFO:tasks.workunit.client.1.smithi138.stdout:6/716: mknod d7/da0/cf6 0 2024-05-17T21:53:08.313 INFO:tasks.workunit.client.0.smithi107.stdout:5/889: mkdir d5/db/d16/d54/da0/d118/d128 0 2024-05-17T21:53:08.315 INFO:tasks.workunit.client.0.smithi107.stdout:1/935: chown d4/df/d1a/d26/d72/daa/c3d 6818256 1 2024-05-17T21:53:08.315 INFO:tasks.workunit.client.1.smithi138.stdout:2/725: mknod d3/d21/d2f/cef 0 2024-05-17T21:53:08.320 INFO:tasks.workunit.client.0.smithi107.stdout:8/859: link dc/d21/d27/d83/cfb dc/dd5/c12e 0 2024-05-17T21:53:08.321 INFO:tasks.workunit.client.1.smithi138.stdout:4/711: creat d0/d3/d17/d5e/d7f/ffd x:0 0 0 2024-05-17T21:53:08.324 INFO:tasks.workunit.client.1.smithi138.stdout:6/717: mknod d7/d69/db5/dc8/d43/d5f/d6a/d77/cf7 0 2024-05-17T21:53:08.326 INFO:tasks.workunit.client.1.smithi138.stdout:9/603: rename d6/d7/d16/d29/d48/d6e/d5a to d6/dbc 0 2024-05-17T21:53:08.328 INFO:tasks.workunit.client.1.smithi138.stdout:9/604: dread d6/d7/d16/d29/d48/d6e/d74/d75/d76/d8e/fa7 [0,4194304] 0 2024-05-17T21:53:08.329 INFO:tasks.workunit.client.1.smithi138.stdout:2/726: mkdir d3/d21/d2f/d3e/d55/d99/d84/dbe/df0 0 2024-05-17T21:53:08.332 INFO:tasks.workunit.client.1.smithi138.stdout:4/712: readlink d0/d3/db/lb8 0 2024-05-17T21:53:08.333 INFO:tasks.workunit.client.0.smithi107.stdout:9/910: dread - d0/dd/d1f/fa7 zero size 2024-05-17T21:53:08.336 INFO:tasks.workunit.client.0.smithi107.stdout:0/992: dwrite d3/d75/d9e/d103/d10d/d132/dab/fdb [0,4194304] 0 2024-05-17T21:53:08.337 INFO:tasks.workunit.client.1.smithi138.stdout:9/605: truncate d6/d7/d16/d29/d6c/d83/f95 4468698 0 2024-05-17T21:53:08.338 INFO:tasks.workunit.client.0.smithi107.stdout:5/890: write d5/db/d16/d54/d59/d65/d48/f8a [775313,125293] 0 2024-05-17T21:53:08.339 INFO:tasks.workunit.client.1.smithi138.stdout:2/727: rename d3/d81/d96/cba to d3/d21/d2f/d3e/d55/d99/dd2/cf1 0 2024-05-17T21:53:08.339 INFO:tasks.workunit.client.0.smithi107.stdout:5/891: chown d5/db/d16/d2a/d73/dd0/lf6 7 1 2024-05-17T21:53:08.341 INFO:tasks.workunit.client.0.smithi107.stdout:1/936: mknod d4/df/d14/d2e/d107/c12c 0 2024-05-17T21:53:08.342 INFO:tasks.workunit.client.1.smithi138.stdout:6/718: dread d7/d69/db5/dc8/d43/d63/fce [0,4194304] 0 2024-05-17T21:53:08.342 INFO:tasks.workunit.client.0.smithi107.stdout:3/920: dwrite d6/d8/d10/d35/fb0 [0,4194304] 0 2024-05-17T21:53:08.343 INFO:tasks.workunit.client.1.smithi138.stdout:6/719: readlink d7/d8/l6f 0 2024-05-17T21:53:08.344 INFO:tasks.workunit.client.1.smithi138.stdout:4/713: creat d0/d3/d1d/d38/d8a/dbd/ddc/ffe x:0 0 0 2024-05-17T21:53:08.351 INFO:tasks.workunit.client.1.smithi138.stdout:2/728: rename d3/da0/daa/dce to d3/d21/d2f/d3e/d58/d77/da4/df2 0 2024-05-17T21:53:08.352 INFO:tasks.workunit.client.0.smithi107.stdout:4/929: dwrite d3/d6/d26/d36/d8b/fbb [0,4194304] 0 2024-05-17T21:53:08.353 INFO:tasks.workunit.client.1.smithi138.stdout:2/729: chown d3/d21/d2f/d3e/d55/d99/d93 128719 1 2024-05-17T21:53:08.354 INFO:tasks.workunit.client.1.smithi138.stdout:4/714: dwrite d0/d3/d17/d5e/d7f/ffd [0,4194304] 0 2024-05-17T21:53:08.355 INFO:tasks.workunit.client.1.smithi138.stdout:6/720: mknod d7/d69/db5/dc8/d34/cf8 0 2024-05-17T21:53:08.356 INFO:tasks.workunit.client.1.smithi138.stdout:9/606: dread d6/f49 [0,4194304] 0 2024-05-17T21:53:08.356 INFO:tasks.workunit.client.1.smithi138.stdout:4/715: chown d0/d2f/d49/d60/d1f/f4d 29 1 2024-05-17T21:53:08.361 INFO:tasks.workunit.client.0.smithi107.stdout:6/822: mknod d9/d58/d66/dd5/df1/c107 0 2024-05-17T21:53:08.362 INFO:tasks.workunit.client.1.smithi138.stdout:2/730: creat d3/d21/d2f/dd4/ff3 x:0 0 0 2024-05-17T21:53:08.365 INFO:tasks.workunit.client.0.smithi107.stdout:9/911: rename d0/dd/d1f/fa7 to d0/dd/d2f/d87/d89/f12a 0 2024-05-17T21:53:08.365 INFO:tasks.workunit.client.1.smithi138.stdout:6/721: mknod d7/da0/cf9 0 2024-05-17T21:53:08.369 INFO:tasks.workunit.client.0.smithi107.stdout:5/892: unlink d5/db/d16/d54/d59/d65/f27 0 2024-05-17T21:53:08.370 INFO:tasks.workunit.client.0.smithi107.stdout:5/893: chown d5/db/dc/f24 274901 1 2024-05-17T21:53:08.370 INFO:tasks.workunit.client.1.smithi138.stdout:4/716: symlink d0/d1b/d2b/lff 0 2024-05-17T21:53:08.373 INFO:tasks.workunit.client.1.smithi138.stdout:6/722: dwrite d7/d69/db5/dc8/d43/d5f/d6a/fd3 [0,4194304] 0 2024-05-17T21:53:08.379 INFO:tasks.workunit.client.0.smithi107.stdout:4/930: mknod d3/db/d1e/d66/c140 0 2024-05-17T21:53:08.380 INFO:tasks.workunit.client.0.smithi107.stdout:6/823: mkdir d9/d16/d22/d5f/dd8/d108 0 2024-05-17T21:53:08.381 INFO:tasks.workunit.client.0.smithi107.stdout:6/824: dread - d9/d16/d22/f92 zero size 2024-05-17T21:53:08.381 INFO:tasks.workunit.client.1.smithi138.stdout:6/723: write d7/d69/db5/dc8/d43/d63/fce [3155751,70377] 0 2024-05-17T21:53:08.381 INFO:tasks.workunit.client.0.smithi107.stdout:6/825: chown d9/d58 123 1 2024-05-17T21:53:08.382 INFO:tasks.workunit.client.0.smithi107.stdout:9/912: mkdir d0/dd/d21/d29/d3e/dfe/d12b 0 2024-05-17T21:53:08.383 INFO:tasks.workunit.client.1.smithi138.stdout:6/724: dread d7/da0/dbe/fe2 [0,4194304] 0 2024-05-17T21:53:08.384 INFO:tasks.workunit.client.1.smithi138.stdout:6/725: chown d7/d69/db5/dc8/d43/d5f/d6a/d9b/faf 0 1 2024-05-17T21:53:08.385 INFO:tasks.workunit.client.0.smithi107.stdout:1/937: fdatasync d4/df/d1a/d26/f127 0 2024-05-17T21:53:08.389 INFO:tasks.workunit.client.1.smithi138.stdout:0/664: sync 2024-05-17T21:53:08.389 INFO:tasks.workunit.client.1.smithi138.stdout:5/691: sync 2024-05-17T21:53:08.389 INFO:tasks.workunit.client.1.smithi138.stdout:3/867: sync 2024-05-17T21:53:08.390 INFO:tasks.workunit.client.0.smithi107.stdout:1/938: symlink d4/df/d14/d2e/ddf/dee/l12d 0 2024-05-17T21:53:08.391 INFO:tasks.workunit.client.1.smithi138.stdout:6/726: dwrite d7/d69/fb8 [0,4194304] 0 2024-05-17T21:53:08.392 INFO:tasks.workunit.client.0.smithi107.stdout:6/826: link d9/d16/d4d/db8/dc9/dd0/f31 d9/d58/d66/de2/f109 0 2024-05-17T21:53:08.392 INFO:tasks.workunit.client.1.smithi138.stdout:5/692: read d1/d2c/f7b [3143936,80774] 0 2024-05-17T21:53:08.395 INFO:tasks.workunit.client.0.smithi107.stdout:2/724: truncate d0/d9/d11/f5f 1666175 0 2024-05-17T21:53:08.396 INFO:tasks.workunit.client.0.smithi107.stdout:6/827: dread d9/d16/f24 [0,4194304] 0 2024-05-17T21:53:08.399 INFO:tasks.workunit.client.0.smithi107.stdout:1/939: mkdir d4/df/d1a/d26/d72/d85/df9/d12e 0 2024-05-17T21:53:08.400 INFO:tasks.workunit.client.0.smithi107.stdout:1/940: stat d4/de/d12b 0 2024-05-17T21:53:08.400 INFO:tasks.workunit.client.1.smithi138.stdout:6/727: dread - d7/d69/db5/dc8/d43/d63/f87 zero size 2024-05-17T21:53:08.402 INFO:tasks.workunit.client.0.smithi107.stdout:9/913: creat d0/dd/d21/d29/d3e/f12c x:0 0 0 2024-05-17T21:53:08.404 INFO:tasks.workunit.client.1.smithi138.stdout:5/693: creat d1/d53/d88/db9/d39/d69/fe1 x:0 0 0 2024-05-17T21:53:08.407 INFO:tasks.workunit.client.0.smithi107.stdout:2/725: rename d0/d26/d64 to d0/d9/d11/df/d5a/d6d/dc8/de8 0 2024-05-17T21:53:08.407 INFO:tasks.workunit.client.0.smithi107.stdout:6/828: dread d9/d58/f9b [0,4194304] 0 2024-05-17T21:53:08.408 INFO:tasks.workunit.client.1.smithi138.stdout:3/868: mknod d1/d62/c127 0 2024-05-17T21:53:08.409 INFO:tasks.workunit.client.0.smithi107.stdout:1/941: dread - d4/df/d1a/d26/f99 zero size 2024-05-17T21:53:08.410 INFO:tasks.workunit.client.0.smithi107.stdout:0/993: dwrite d3/d58/faa [0,4194304] 0 2024-05-17T21:53:08.411 INFO:tasks.workunit.client.0.smithi107.stdout:9/914: fdatasync d0/dd/d3d/f92 0 2024-05-17T21:53:08.411 INFO:tasks.workunit.client.1.smithi138.stdout:5/694: dwrite d1/d2c/d63/db5/d1b/dc5/fbb [0,4194304] 0 2024-05-17T21:53:08.412 INFO:tasks.workunit.client.1.smithi138.stdout:2/731: write d3/d21/d2f/d3e/d58/f5d [809354,15087] 0 2024-05-17T21:53:08.414 INFO:tasks.workunit.client.0.smithi107.stdout:3/921: write d6/d1a/d57/f73 [2437940,100366] 0 2024-05-17T21:53:08.414 INFO:tasks.workunit.client.1.smithi138.stdout:9/607: dwrite d6/f33 [0,4194304] 0 2024-05-17T21:53:08.416 INFO:tasks.workunit.client.1.smithi138.stdout:9/608: dread - d6/d7/d16/d29/d38/fa3 zero size 2024-05-17T21:53:08.416 INFO:tasks.workunit.client.0.smithi107.stdout:3/922: chown d6/d2a/d2d/d48/l61 23973183 1 2024-05-17T21:53:08.418 INFO:tasks.workunit.client.1.smithi138.stdout:6/728: unlink d7/d69/db5/dc8/d43/d5f/d6a/c99 0 2024-05-17T21:53:08.423 INFO:tasks.workunit.client.0.smithi107.stdout:8/860: dwrite dc/d21/f126 [0,4194304] 0 2024-05-17T21:53:08.424 INFO:tasks.workunit.client.0.smithi107.stdout:8/861: chown dc/f67 1709 1 2024-05-17T21:53:08.428 INFO:tasks.workunit.client.1.smithi138.stdout:3/869: readlink d1/d3/l18 0 2024-05-17T21:53:08.430 INFO:tasks.workunit.client.0.smithi107.stdout:8/862: dread dc/f4e [0,4194304] 0 2024-05-17T21:53:08.431 INFO:tasks.workunit.client.1.smithi138.stdout:4/717: dwrite d0/d1b/d2b/d55/d79/f7b [0,4194304] 0 2024-05-17T21:53:08.431 INFO:tasks.workunit.client.1.smithi138.stdout:0/665: rmdir d3/d10/d38/d3e/ddc 0 2024-05-17T21:53:08.431 INFO:tasks.workunit.client.0.smithi107.stdout:5/894: write d5/db/d16/d2d/d40/d9f/fef [13289,97012] 0 2024-05-17T21:53:08.432 INFO:tasks.workunit.client.0.smithi107.stdout:8/863: chown dc/d21/d27/d34/lca 110 1 2024-05-17T21:53:08.432 INFO:tasks.workunit.client.0.smithi107.stdout:0/994: read - d3/d75/f102 zero size 2024-05-17T21:53:08.434 INFO:tasks.workunit.client.0.smithi107.stdout:5/895: write d5/db/d39/f3e [2799424,99435] 0 2024-05-17T21:53:08.434 INFO:tasks.workunit.client.0.smithi107.stdout:8/864: truncate dc/d21/daa/dc5/f111 690367 0 2024-05-17T21:53:08.435 INFO:tasks.workunit.client.0.smithi107.stdout:5/896: chown d5/db/dc/d1e/d125 24440994 1 2024-05-17T21:53:08.437 INFO:tasks.workunit.client.0.smithi107.stdout:5/897: fdatasync d5/db/d39/f3c 0 2024-05-17T21:53:08.437 INFO:tasks.workunit.client.1.smithi138.stdout:9/609: creat d6/d7/d16/d29/d38/d3c/fbd x:0 0 0 2024-05-17T21:53:08.438 INFO:tasks.workunit.client.0.smithi107.stdout:9/915: creat d0/dd/d69/dee/d116/f12d x:0 0 0 2024-05-17T21:53:08.439 INFO:tasks.workunit.client.1.smithi138.stdout:6/729: fdatasync d7/d8/fba 0 2024-05-17T21:53:08.441 INFO:tasks.workunit.client.0.smithi107.stdout:3/923: chown d6/d8/d3e/d3f/c124 470739 1 2024-05-17T21:53:08.441 INFO:tasks.workunit.client.0.smithi107.stdout:0/995: dread d3/d58/faa [0,4194304] 0 2024-05-17T21:53:08.442 INFO:tasks.workunit.client.0.smithi107.stdout:6/829: fdatasync d9/f48 0 2024-05-17T21:53:08.444 INFO:tasks.workunit.client.1.smithi138.stdout:5/695: dread d1/d53/f6e [0,4194304] 0 2024-05-17T21:53:08.445 INFO:tasks.workunit.client.1.smithi138.stdout:5/696: stat d1/d53/d96/da0/fa1 0 2024-05-17T21:53:08.445 INFO:tasks.workunit.client.0.smithi107.stdout:4/931: dwrite d3/d4a/f3f [0,4194304] 0 2024-05-17T21:53:08.446 INFO:tasks.workunit.client.0.smithi107.stdout:4/932: chown d3/db/lac 5 1 2024-05-17T21:53:08.446 INFO:tasks.workunit.client.1.smithi138.stdout:9/610: mknod d6/d7/d16/d29/d48/d67/d86/cbe 0 2024-05-17T21:53:08.448 INFO:tasks.workunit.client.1.smithi138.stdout:6/730: mkdir d7/d69/db5/dc8/d34/da6/dfa 0 2024-05-17T21:53:08.448 INFO:tasks.workunit.client.0.smithi107.stdout:3/924: dread d6/f2c [0,4194304] 0 2024-05-17T21:53:08.448 INFO:tasks.workunit.client.1.smithi138.stdout:6/731: chown d7/d8 7843448 1 2024-05-17T21:53:08.448 INFO:tasks.workunit.client.1.smithi138.stdout:3/870: getdents d1/d3/d16/d3e/d71/db2 0 2024-05-17T21:53:08.450 INFO:tasks.workunit.client.1.smithi138.stdout:3/871: write d1/d62/dad/de/f119 [195419,66838] 0 2024-05-17T21:53:08.451 INFO:tasks.workunit.client.1.smithi138.stdout:3/872: chown d1/d3/d49/ddb/l103 15 1 2024-05-17T21:53:08.453 INFO:tasks.workunit.client.0.smithi107.stdout:4/933: dwrite d3/db/d35/d55/d8a/ff2 [0,4194304] 0 2024-05-17T21:53:08.456 INFO:tasks.workunit.client.1.smithi138.stdout:6/732: rmdir d7/d69/db5 39 2024-05-17T21:53:08.459 INFO:tasks.workunit.client.1.smithi138.stdout:2/732: dread d3/d21/d2f/d3e/d55/d99/d93/f79 [0,4194304] 0 2024-05-17T21:53:08.461 INFO:tasks.workunit.client.0.smithi107.stdout:6/830: fsync d9/f48 0 2024-05-17T21:53:08.464 INFO:tasks.workunit.client.1.smithi138.stdout:9/611: dwrite d6/d7/d16/d29/d48/f7d [0,4194304] 0 2024-05-17T21:53:08.467 INFO:tasks.workunit.client.0.smithi107.stdout:5/898: truncate d5/db/d16/d54/d59/d65/fbb 358982 0 2024-05-17T21:53:08.467 INFO:tasks.workunit.client.1.smithi138.stdout:9/612: chown d6/d7/d16/d29/d38/d40/d44/fa4 703 1 2024-05-17T21:53:08.470 INFO:tasks.workunit.client.1.smithi138.stdout:3/873: truncate d1/d3/f11 1372800 0 2024-05-17T21:53:08.474 INFO:tasks.workunit.client.1.smithi138.stdout:2/733: symlink d3/d21/d2f/d3e/d55/d99/d84/d9f/lf4 0 2024-05-17T21:53:08.476 INFO:tasks.workunit.client.0.smithi107.stdout:1/942: creat d4/f12f x:0 0 0 2024-05-17T21:53:08.477 INFO:tasks.workunit.client.1.smithi138.stdout:6/733: fdatasync d7/d69/db5/dc8/d43/d3f/f4c 0 2024-05-17T21:53:08.479 INFO:tasks.workunit.client.1.smithi138.stdout:4/718: dread d0/d3/d17/fa9 [0,4194304] 0 2024-05-17T21:53:08.484 INFO:tasks.workunit.client.0.smithi107.stdout:2/726: dwrite d0/d9/d11/f8b [0,4194304] 0 2024-05-17T21:53:08.485 INFO:tasks.workunit.client.1.smithi138.stdout:6/734: fdatasync d7/f2e 0 2024-05-17T21:53:08.487 INFO:tasks.workunit.client.0.smithi107.stdout:4/934: unlink d3/db/d35/d5e/dff/d80/f44 0 2024-05-17T21:53:08.489 INFO:tasks.workunit.client.1.smithi138.stdout:0/666: write d3/d10/d38/d70/d80/d94/fa4 [567178,54210] 0 2024-05-17T21:53:08.490 INFO:tasks.workunit.client.1.smithi138.stdout:4/719: rmdir d0/d3/d1d/d38/d8a/dbd/ddc 39 2024-05-17T21:53:08.498 INFO:tasks.workunit.client.1.smithi138.stdout:6/735: chown d7/d69/db5/dc8/f25 18814 1 2024-05-17T21:53:08.498 INFO:tasks.workunit.client.0.smithi107.stdout:9/916: write d0/dd/d21/faa [3646395,49757] 0 2024-05-17T21:53:08.501 INFO:tasks.workunit.client.0.smithi107.stdout:3/925: link d6/d2a/d2d/d48/f126 d6/d8/d10/d35/d36/d4c/dcc/d110/f129 0 2024-05-17T21:53:08.503 INFO:tasks.workunit.client.1.smithi138.stdout:5/697: dwrite d1/d2c/f7b [0,4194304] 0 2024-05-17T21:53:08.506 INFO:tasks.workunit.client.1.smithi138.stdout:3/874: getdents d1/d62/dad/d72 0 2024-05-17T21:53:08.508 INFO:tasks.workunit.client.0.smithi107.stdout:2/727: creat d0/d9/d11/d27/dbf/fe9 x:0 0 0 2024-05-17T21:53:08.508 INFO:tasks.workunit.client.1.smithi138.stdout:6/736: dwrite d7/d69/db5/dc8/d43/d5f/d6a/d9b/faf [0,4194304] 0 2024-05-17T21:53:08.511 INFO:tasks.workunit.client.1.smithi138.stdout:9/613: getdents d6/d7/d16/d29/d6c/d9c 0 2024-05-17T21:53:08.512 INFO:tasks.workunit.client.1.smithi138.stdout:9/614: read - d6/d7/d16/f6d zero size 2024-05-17T21:53:08.514 INFO:tasks.workunit.client.1.smithi138.stdout:9/615: chown d6/c8d 57908873 1 2024-05-17T21:53:08.517 INFO:tasks.workunit.client.0.smithi107.stdout:6/831: symlink d9/l10a 0 2024-05-17T21:53:08.518 INFO:tasks.workunit.client.1.smithi138.stdout:5/698: symlink d1/d53/d88/db9/d39/d69/le2 0 2024-05-17T21:53:08.518 INFO:tasks.workunit.client.1.smithi138.stdout:5/699: readlink d1/d53/d88/lce 0 2024-05-17T21:53:08.521 INFO:tasks.workunit.client.0.smithi107.stdout:8/865: getdents dc/d44 0 2024-05-17T21:53:08.522 INFO:tasks.workunit.client.1.smithi138.stdout:4/720: write d0/d3/d17/d21/d78/fa5 [3858517,82446] 0 2024-05-17T21:53:08.522 INFO:tasks.workunit.client.0.smithi107.stdout:8/866: stat dc/d21/d27/d34/l116 0 2024-05-17T21:53:08.523 INFO:tasks.workunit.client.1.smithi138.stdout:2/734: getdents d3/d21/d2f 0 2024-05-17T21:53:08.524 INFO:tasks.workunit.client.0.smithi107.stdout:1/943: dwrite d4/df/d14/f4c [0,4194304] 0 2024-05-17T21:53:08.525 INFO:tasks.workunit.client.1.smithi138.stdout:6/737: symlink d7/d69/db5/dc8/d43/d5f/d6a/d77/lfb 0 2024-05-17T21:53:08.527 INFO:tasks.workunit.client.1.smithi138.stdout:9/616: unlink d6/d7/d16/d29/d48/d67/d86/cbe 0 2024-05-17T21:53:08.530 INFO:tasks.workunit.client.1.smithi138.stdout:3/875: mknod d1/d62/dad/d5c/c128 0 2024-05-17T21:53:08.532 INFO:tasks.workunit.client.0.smithi107.stdout:4/935: creat d3/db/d35/d5e/dff/d80/d133/f141 x:0 0 0 2024-05-17T21:53:08.533 INFO:tasks.workunit.client.0.smithi107.stdout:4/936: dread - d3/db/d35/d55/f117 zero size 2024-05-17T21:53:08.533 INFO:tasks.workunit.client.1.smithi138.stdout:2/735: creat d3/d21/d2f/d3e/d58/dc3/ff5 x:0 0 0 2024-05-17T21:53:08.534 INFO:tasks.workunit.client.0.smithi107.stdout:0/996: link d3/d75/dd0/df6/l13d d3/d75/d9e/d103/d10d/d132/de8/l145 0 2024-05-17T21:53:08.534 INFO:tasks.workunit.client.1.smithi138.stdout:2/736: chown d3/f10 519654776 1 2024-05-17T21:53:08.534 INFO:tasks.workunit.client.1.smithi138.stdout:2/737: chown d3/d21/d2f/d3e/d58/d77/da4/df2/dd0 442 1 2024-05-17T21:53:08.535 INFO:tasks.workunit.client.1.smithi138.stdout:6/738: mkdir d7/d69/db5/dc8/d60/dfc 0 2024-05-17T21:53:08.536 INFO:tasks.workunit.client.1.smithi138.stdout:2/738: chown d3/d21/d2f/d3e/d55/d73/ca6 68280 1 2024-05-17T21:53:08.536 INFO:tasks.workunit.client.0.smithi107.stdout:6/832: mknod d9/d16/d22/de6/c10b 0 2024-05-17T21:53:08.538 INFO:tasks.workunit.client.1.smithi138.stdout:5/700: mknod d1/d53/d88/db9/ce3 0 2024-05-17T21:53:08.539 INFO:tasks.workunit.client.0.smithi107.stdout:5/899: link d5/db/dc/f1f d5/db/d16/d54/da0/d118/f129 0 2024-05-17T21:53:08.539 INFO:tasks.workunit.client.0.smithi107.stdout:5/900: chown d5/db/d16/d5b/l7a 26 1 2024-05-17T21:53:08.540 INFO:tasks.workunit.client.0.smithi107.stdout:5/901: write d5/db/d16/d54/d59/d92/fe5 [3723122,86485] 0 2024-05-17T21:53:08.541 INFO:tasks.workunit.client.1.smithi138.stdout:9/617: creat d6/d7/d16/d29/d48/d6e/d74/d75/d76/fbf x:0 0 0 2024-05-17T21:53:08.544 INFO:tasks.workunit.client.1.smithi138.stdout:4/721: fsync d0/d2f/d49/d60/d1f/d32/fb2 0 2024-05-17T21:53:08.545 INFO:tasks.workunit.client.0.smithi107.stdout:8/867: read dc/d21/d97/db7/fef [235743,125940] 0 2024-05-17T21:53:08.547 INFO:tasks.workunit.client.0.smithi107.stdout:1/944: symlink d4/da7/daf/l130 0 2024-05-17T21:53:08.549 INFO:tasks.workunit.client.0.smithi107.stdout:2/728: dwrite d0/d26/f6c [0,4194304] 0 2024-05-17T21:53:08.550 INFO:tasks.workunit.client.1.smithi138.stdout:5/701: dread - d1/d46/f9c zero size 2024-05-17T21:53:08.551 INFO:tasks.workunit.client.0.smithi107.stdout:3/926: truncate d6/d8/f17 1232408 0 2024-05-17T21:53:08.552 INFO:tasks.workunit.client.1.smithi138.stdout:6/739: dread d7/d8/f24 [0,4194304] 0 2024-05-17T21:53:08.552 INFO:tasks.workunit.client.0.smithi107.stdout:3/927: chown d6/d2a/d2d/d37/cca 749013131 1 2024-05-17T21:53:08.553 INFO:tasks.workunit.client.1.smithi138.stdout:9/618: truncate f5 506589 0 2024-05-17T21:53:08.556 INFO:tasks.workunit.client.0.smithi107.stdout:4/937: creat d3/d6/d26/d36/d37/f142 x:0 0 0 2024-05-17T21:53:08.559 INFO:tasks.workunit.client.1.smithi138.stdout:2/739: mkdir d3/da0/df6 0 2024-05-17T21:53:08.560 INFO:tasks.workunit.client.1.smithi138.stdout:2/740: dread - d3/d21/d2f/d3e/d58/d77/da4/df2/fd6 zero size 2024-05-17T21:53:08.561 INFO:tasks.workunit.client.0.smithi107.stdout:5/902: symlink d5/db/d16/d54/da0/d118/l12a 0 2024-05-17T21:53:08.562 INFO:tasks.workunit.client.0.smithi107.stdout:4/938: dwrite d3/d4a/d47/d60/d3a/f105 [0,4194304] 0 2024-05-17T21:53:08.564 INFO:tasks.workunit.client.0.smithi107.stdout:8/868: mknod dc/d44/d4b/d52/dcb/c12f 0 2024-05-17T21:53:08.566 INFO:tasks.workunit.client.1.smithi138.stdout:3/876: link d1/d62/d9b/d107/l6a d1/d2e/l129 0 2024-05-17T21:53:08.568 INFO:tasks.workunit.client.0.smithi107.stdout:2/729: creat d0/d9/d1c/d37/d9f/fea x:0 0 0 2024-05-17T21:53:08.571 INFO:tasks.workunit.client.1.smithi138.stdout:5/702: rename d1/d53/d88/db9/d39/d62 to d1/d60/d2f/d5e/de4 0 2024-05-17T21:53:08.572 INFO:tasks.workunit.client.0.smithi107.stdout:7/848: sync 2024-05-17T21:53:08.579 INFO:tasks.workunit.client.1.smithi138.stdout:9/619: mkdir d6/d7/dc0 0 2024-05-17T21:53:08.580 INFO:tasks.workunit.client.0.smithi107.stdout:6/833: symlink d9/d58/d66/dd5/df3/l10c 0 2024-05-17T21:53:08.581 INFO:tasks.workunit.client.0.smithi107.stdout:6/834: write d9/d16/d22/daf/dc7/f101 [508113,9977] 0 2024-05-17T21:53:08.581 INFO:tasks.workunit.client.1.smithi138.stdout:4/722: creat d0/d1b/f100 x:0 0 0 2024-05-17T21:53:08.582 INFO:tasks.workunit.client.1.smithi138.stdout:6/740: write d7/d69/db5/dc8/d34/f53 [3188928,110619] 0 2024-05-17T21:53:08.585 INFO:tasks.workunit.client.1.smithi138.stdout:3/877: rename d1/d3/d16/d3e/d71/d10d/l102 to d1/d3/d16/dd7/l12a 0 2024-05-17T21:53:08.587 INFO:tasks.workunit.client.0.smithi107.stdout:9/917: getdents d0/d80 0 2024-05-17T21:53:08.588 INFO:tasks.workunit.client.1.smithi138.stdout:2/741: dread d3/d21/d2f/fc0 [0,4194304] 0 2024-05-17T21:53:08.589 INFO:tasks.workunit.client.1.smithi138.stdout:5/703: mkdir d1/d53/d88/de5 0 2024-05-17T21:53:08.590 INFO:tasks.workunit.client.0.smithi107.stdout:4/939: mkdir d3/db/d35/d55/d8a/d143 0 2024-05-17T21:53:08.594 INFO:tasks.workunit.client.0.smithi107.stdout:6/835: dread d9/d16/d22/f56 [0,4194304] 0 2024-05-17T21:53:08.595 INFO:tasks.workunit.client.1.smithi138.stdout:9/620: dread d6/d7/d16/d29/d48/d3b/faf [0,4194304] 0 2024-05-17T21:53:08.595 INFO:tasks.workunit.client.0.smithi107.stdout:8/869: dread dc/f4e [0,4194304] 0 2024-05-17T21:53:08.596 INFO:tasks.workunit.client.0.smithi107.stdout:6/836: write d9/d16/d4d/db8/dc9/dd0/f39 [3871058,75101] 0 2024-05-17T21:53:08.596 INFO:tasks.workunit.client.0.smithi107.stdout:8/870: dread - dc/d44/fdc zero size 2024-05-17T21:53:08.601 INFO:tasks.workunit.client.1.smithi138.stdout:5/704: truncate d1/d53/d88/db9/d39/d69/f92 535444 0 2024-05-17T21:53:08.602 INFO:tasks.workunit.client.1.smithi138.stdout:5/705: write d1/d60/d2f/d5e/de4/fac [2182199,108158] 0 2024-05-17T21:53:08.602 INFO:tasks.workunit.client.0.smithi107.stdout:7/849: truncate d5/d16/d3b/d45/d91/dc1/fea 1058409 0 2024-05-17T21:53:08.603 INFO:tasks.workunit.client.0.smithi107.stdout:1/945: write d4/df/d14/d2e/d43/f51 [1170539,116083] 0 2024-05-17T21:53:08.605 INFO:tasks.workunit.client.0.smithi107.stdout:0/997: rename d3/d75/d9e/d103/f129 to d3/d24/d6f/f146 0 2024-05-17T21:53:08.608 INFO:tasks.workunit.client.1.smithi138.stdout:4/723: mkdir d0/d3/d1d/d38/d8a/dbd/ddc/d101 0 2024-05-17T21:53:08.609 INFO:tasks.workunit.client.0.smithi107.stdout:2/730: dwrite d0/d9/d1c/d3e/f3f [0,4194304] 0 2024-05-17T21:53:08.610 INFO:tasks.workunit.client.0.smithi107.stdout:2/731: chown d0/d9/d11/d18/dad/ccd 61599064 1 2024-05-17T21:53:08.611 INFO:tasks.workunit.client.1.smithi138.stdout:3/878: link d1/d62/dad/de/f119 d1/d62/dad/d5c/d9c/dec/f12b 0 2024-05-17T21:53:08.611 INFO:tasks.workunit.client.1.smithi138.stdout:2/742: dwrite d3/d21/d2f/f5b [0,4194304] 0 2024-05-17T21:53:08.612 INFO:tasks.workunit.client.1.smithi138.stdout:3/879: write d1/d4f/f126 [268102,65531] 0 2024-05-17T21:53:08.617 INFO:tasks.workunit.client.1.smithi138.stdout:9/621: rename d6/d7/d16/d29/d6c/d83 to d6/d7/d16/d29/d6c/d9c/dc1 0 2024-05-17T21:53:08.618 INFO:tasks.workunit.client.1.smithi138.stdout:2/743: rename d3/d21/d2f/d3e to d3/d21/d2f/d3e/d63/df7 22 2024-05-17T21:53:08.620 INFO:tasks.workunit.client.0.smithi107.stdout:6/837: symlink d9/d58/d66/de2/l10d 0 2024-05-17T21:53:08.621 INFO:tasks.workunit.client.1.smithi138.stdout:5/706: chown d1/d60/c50 11692 1 2024-05-17T21:53:08.624 INFO:tasks.workunit.client.1.smithi138.stdout:6/741: creat d7/d69/ffd x:0 0 0 2024-05-17T21:53:08.627 INFO:tasks.workunit.client.0.smithi107.stdout:7/850: truncate d5/df/d11/d1d/d33/f99 243755 0 2024-05-17T21:53:08.628 INFO:tasks.workunit.client.1.smithi138.stdout:3/880: symlink d1/d4f/d87/l12c 0 2024-05-17T21:53:08.630 INFO:tasks.workunit.client.1.smithi138.stdout:9/622: mkdir d6/d7/d16/dc2 0 2024-05-17T21:53:08.630 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:08 smithi107 ceph-mon[28434]: pgmap v42: 65 pgs: 65 active+clean; 3.0 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 112 MiB/s rd, 266 MiB/s wr, 604 op/s 2024-05-17T21:53:08.632 INFO:tasks.workunit.client.0.smithi107.stdout:5/903: rename d5/db/d16/d2d/d40/lfd to d5/db/d16/l12b 0 2024-05-17T21:53:08.632 INFO:tasks.workunit.client.0.smithi107.stdout:9/918: dread d0/dd/d2f/d87/d89/f123 [0,4194304] 0 2024-05-17T21:53:08.633 INFO:tasks.workunit.client.1.smithi138.stdout:1/671: sync 2024-05-17T21:53:08.633 INFO:tasks.workunit.client.1.smithi138.stdout:8/640: sync 2024-05-17T21:53:08.635 INFO:tasks.workunit.client.0.smithi107.stdout:8/871: write dc/d21/d27/d2f/dba/dee/f115 [838530,45636] 0 2024-05-17T21:53:08.640 INFO:tasks.workunit.client.0.smithi107.stdout:2/732: mkdir d0/d9/d11/d18/d9e/deb 0 2024-05-17T21:53:08.641 INFO:tasks.workunit.client.0.smithi107.stdout:1/946: write d4/df/d1a/d41/db2/fd9 [541251,111047] 0 2024-05-17T21:53:08.643 INFO:tasks.workunit.client.0.smithi107.stdout:3/928: link d6/d8/d3e/d3f/d90/da7/ddc/c107 d6/d2a/d2d/d37/dc6/c12a 0 2024-05-17T21:53:08.645 INFO:tasks.workunit.client.0.smithi107.stdout:7/851: creat d5/df/d11/d100/f127 x:0 0 0 2024-05-17T21:53:08.646 INFO:tasks.workunit.client.0.smithi107.stdout:5/904: fsync d5/db/dc/f1f 0 2024-05-17T21:53:08.648 INFO:tasks.workunit.client.0.smithi107.stdout:8/872: readlink dc/d21/d27/d34/d4c/lcf 0 2024-05-17T21:53:08.648 INFO:tasks.workunit.client.0.smithi107.stdout:4/940: creat d3/db/d35/d5e/f144 x:0 0 0 2024-05-17T21:53:08.649 INFO:tasks.workunit.client.1.smithi138.stdout:6/742: rename d7/d69/db5/dc8/d43/d5f/f73 to d7/d69/db5/dc8/d43/d3f/d79/ffe 0 2024-05-17T21:53:08.649 INFO:tasks.workunit.client.0.smithi107.stdout:2/733: mkdir d0/d9/d1c/dd4/dec 0 2024-05-17T21:53:08.651 INFO:tasks.workunit.client.1.smithi138.stdout:3/881: write d1/d3/d16/f30 [1446500,47570] 0 2024-05-17T21:53:08.651 INFO:tasks.workunit.client.1.smithi138.stdout:8/641: mkdir d0/d31/dbe 0 2024-05-17T21:53:08.652 INFO:tasks.workunit.client.0.smithi107.stdout:3/929: dwrite d6/d2a/d2d/d37/f7c [4194304,4194304] 0 2024-05-17T21:53:08.653 INFO:tasks.workunit.client.1.smithi138.stdout:1/672: mkdir d4/dd/d3b/d9f/da9/dd9 0 2024-05-17T21:53:08.655 INFO:tasks.workunit.client.1.smithi138.stdout:3/882: truncate d1/d3/d16/d5d/f11a 846160 0 2024-05-17T21:53:08.656 INFO:tasks.workunit.client.1.smithi138.stdout:6/743: symlink d7/d69/db5/dc8/d43/d5f/da4/dcc/lff 0 2024-05-17T21:53:08.657 INFO:tasks.workunit.client.0.smithi107.stdout:7/852: dwrite d5/df/d11/d1c/f4e [0,4194304] 0 2024-05-17T21:53:08.658 INFO:tasks.workunit.client.1.smithi138.stdout:3/883: write d1/d3/d16/d3e/f113 [925244,77590] 0 2024-05-17T21:53:08.659 INFO:tasks.workunit.client.0.smithi107.stdout:1/947: dwrite d4/df/d1a/d26/d7a/f93 [0,4194304] 0 2024-05-17T21:53:08.659 INFO:tasks.workunit.client.0.smithi107.stdout:0/998: creat d3/d24/f147 x:0 0 0 2024-05-17T21:53:08.659 INFO:tasks.workunit.client.1.smithi138.stdout:4/724: dread d0/d1b/d42/f4f [0,4194304] 0 2024-05-17T21:53:08.660 INFO:tasks.workunit.client.0.smithi107.stdout:7/853: truncate d5/df/d11/d1c/d2f/d53/f90 4444814 0 2024-05-17T21:53:08.661 INFO:tasks.workunit.client.1.smithi138.stdout:1/673: fsync d4/df/d4e/fc7 0 2024-05-17T21:53:08.662 INFO:tasks.workunit.client.1.smithi138.stdout:6/744: chown d7/d69/db5/dc8/d43/d5f/c66 60421249 1 2024-05-17T21:53:08.663 INFO:tasks.workunit.client.1.smithi138.stdout:1/674: chown d4/d2e/d36/l71 1 1 2024-05-17T21:53:08.668 INFO:tasks.workunit.client.1.smithi138.stdout:5/707: dread d1/d2c/d63/db5/fe [0,4194304] 0 2024-05-17T21:53:08.671 INFO:tasks.workunit.client.1.smithi138.stdout:9/623: dread d6/d7/d16/d29/d38/f73 [0,4194304] 0 2024-05-17T21:53:08.672 INFO:tasks.workunit.client.1.smithi138.stdout:9/624: read - d6/d7/d16/d29/d48/d6e/fab zero size 2024-05-17T21:53:08.673 INFO:tasks.workunit.client.0.smithi107.stdout:6/838: rename d9/d58 to d9/d10e 0 2024-05-17T21:53:08.676 INFO:tasks.workunit.client.1.smithi138.stdout:3/884: creat d1/d62/d9b/d107/df2/f12d x:0 0 0 2024-05-17T21:53:08.677 INFO:tasks.workunit.client.0.smithi107.stdout:5/905: mkdir d5/db/d16/d54/d59/d65/d12c 0 2024-05-17T21:53:08.678 INFO:tasks.workunit.client.1.smithi138.stdout:6/745: creat d7/d8/def/f100 x:0 0 0 2024-05-17T21:53:08.679 INFO:tasks.workunit.client.1.smithi138.stdout:1/675: write d4/db/d29/d5d/f86 [2327598,41706] 0 2024-05-17T21:53:08.680 INFO:tasks.workunit.client.1.smithi138.stdout:6/746: write d7/d69/db5/dc8/d43/d3f/db6/fe8 [9500,57036] 0 2024-05-17T21:53:08.680 INFO:tasks.workunit.client.0.smithi107.stdout:8/873: creat dc/d21/d97/db7/dcd/f130 x:0 0 0 2024-05-17T21:53:08.682 INFO:tasks.workunit.client.0.smithi107.stdout:2/734: symlink d0/d9/d11/df/d5a/d6d/dc8/led 0 2024-05-17T21:53:08.683 INFO:tasks.workunit.client.0.smithi107.stdout:2/735: chown d0/d9/d11/d18/dad/db8 42592103 1 2024-05-17T21:53:08.683 INFO:tasks.workunit.client.1.smithi138.stdout:3/885: creat d1/d3/d49/f12e x:0 0 0 2024-05-17T21:53:08.684 INFO:tasks.workunit.client.0.smithi107.stdout:3/930: truncate d6/d2a/d2d/d37/fc8 1773677 0 2024-05-17T21:53:08.685 INFO:tasks.workunit.client.0.smithi107.stdout:1/948: mknod d4/df/d1a/d26/d72/c131 0 2024-05-17T21:53:08.686 INFO:tasks.workunit.client.0.smithi107.stdout:1/949: chown d4/df/d14/d1d 2 1 2024-05-17T21:53:08.690 INFO:tasks.workunit.client.0.smithi107.stdout:3/931: dread d6/fc1 [0,4194304] 0 2024-05-17T21:53:08.690 INFO:tasks.workunit.client.0.smithi107.stdout:9/919: creat d0/dd/d2f/f12e x:0 0 0 2024-05-17T21:53:08.691 INFO:tasks.workunit.client.1.smithi138.stdout:9/625: mknod d6/d7/d16/d29/d6c/d9c/dc1/d7e/db7/cc3 0 2024-05-17T21:53:08.693 INFO:tasks.workunit.client.0.smithi107.stdout:6/839: creat d9/d10e/d66/f10f x:0 0 0 2024-05-17T21:53:08.694 INFO:tasks.workunit.client.0.smithi107.stdout:5/906: dread - d5/db/d16/d2a/d73/f99 zero size 2024-05-17T21:53:08.698 INFO:tasks.workunit.client.1.smithi138.stdout:6/747: mkdir d7/d69/db5/dc8/d34/da6/dfa/d101 0 2024-05-17T21:53:08.700 INFO:tasks.workunit.client.0.smithi107.stdout:8/874: creat dc/d44/f131 x:0 0 0 2024-05-17T21:53:08.700 INFO:tasks.workunit.client.0.smithi107.stdout:8/875: chown dc/d21/d27/d34/c3a 17 1 2024-05-17T21:53:08.701 INFO:tasks.workunit.client.1.smithi138.stdout:9/626: stat d6/d7/fc 0 2024-05-17T21:53:08.702 INFO:tasks.workunit.client.0.smithi107.stdout:2/736: symlink d0/d9/d1c/d37/lee 0 2024-05-17T21:53:08.703 INFO:tasks.workunit.client.1.smithi138.stdout:9/627: chown d6/d7/d16/d29/d6c/d9c/laa 67570 1 2024-05-17T21:53:08.703 INFO:tasks.workunit.client.1.smithi138.stdout:6/748: dwrite d7/da0/fd2 [0,4194304] 0 2024-05-17T21:53:08.705 INFO:tasks.workunit.client.0.smithi107.stdout:1/950: chown d4/df/d1a/d41/db2/dc8/lfd 15 1 2024-05-17T21:53:08.707 INFO:tasks.workunit.client.0.smithi107.stdout:2/737: dwrite d0/d9/d11/f52 [0,4194304] 0 2024-05-17T21:53:08.708 INFO:tasks.workunit.client.1.smithi138.stdout:5/708: truncate d1/d53/d96/fc6 3740052 0 2024-05-17T21:53:08.709 INFO:tasks.workunit.client.0.smithi107.stdout:0/999: write d3/d75/d9e/d103/d10d/d132/dab/f89 [3980361,33860] 0 2024-05-17T21:53:08.711 INFO:tasks.workunit.client.1.smithi138.stdout:6/749: creat d7/d8/f102 x:0 0 0 2024-05-17T21:53:08.712 INFO:tasks.workunit.client.0.smithi107.stdout:7/854: write d5/f83 [1280344,4652] 0 2024-05-17T21:53:08.712 INFO:tasks.workunit.client.1.smithi138.stdout:5/709: read - d1/d53/d88/db9/d39/d69/fe1 zero size 2024-05-17T21:53:08.714 INFO:tasks.workunit.client.1.smithi138.stdout:6/750: write d7/d8/def/f100 [595263,33617] 0 2024-05-17T21:53:08.714 INFO:tasks.workunit.client.1.smithi138.stdout:9/628: dwrite d6/d7/d16/d29/d38/d3c/fbd [0,4194304] 0 2024-05-17T21:53:08.715 INFO:tasks.workunit.client.0.smithi107.stdout:3/932: mkdir d6/d8/db6/d12b 0 2024-05-17T21:53:08.718 INFO:tasks.workunit.client.0.smithi107.stdout:9/920: rename d0/dd/d2f/d81/l122 to d0/d80/dad/l12f 0 2024-05-17T21:53:08.719 INFO:tasks.workunit.client.1.smithi138.stdout:3/886: write d1/d62/dad/de/fb5 [3422998,83469] 0 2024-05-17T21:53:08.721 INFO:tasks.workunit.client.1.smithi138.stdout:1/676: write d4/db/d29/d5d/f6f [1390835,35350] 0 2024-05-17T21:53:08.721 INFO:tasks.workunit.client.0.smithi107.stdout:9/921: chown d0/dd/d2f/d87/d89/de0 46980 1 2024-05-17T21:53:08.722 INFO:tasks.workunit.client.0.smithi107.stdout:3/933: dwrite d6/d8/d3e/dc9/f128 [0,4194304] 0 2024-05-17T21:53:08.722 INFO:tasks.workunit.client.1.smithi138.stdout:9/629: truncate d6/d7/fc 530011 0 2024-05-17T21:53:08.726 INFO:tasks.workunit.client.1.smithi138.stdout:6/751: rename d7/d8/l6f to d7/d69/db5/dc8/d43/d5f/d6a/d77/l103 0 2024-05-17T21:53:08.729 INFO:tasks.workunit.client.1.smithi138.stdout:3/887: rmdir d1/d62/dad/d72 39 2024-05-17T21:53:08.730 INFO:tasks.workunit.client.0.smithi107.stdout:4/941: dwrite d3/d4a/f73 [0,4194304] 0 2024-05-17T21:53:08.730 INFO:tasks.workunit.client.0.smithi107.stdout:8/876: write dc/d21/d69/df4/f122 [1062482,89293] 0 2024-05-17T21:53:08.731 INFO:tasks.workunit.client.1.smithi138.stdout:3/888: fsync d1/d62/dad/d5c/d9c/dec/f11e 0 2024-05-17T21:53:08.734 INFO:tasks.workunit.client.1.smithi138.stdout:1/677: creat d4/df/d3a/d3c/d63/d6d/fda x:0 0 0 2024-05-17T21:53:08.735 INFO:tasks.workunit.client.1.smithi138.stdout:1/678: dread - d4/dd/d3b/d6c/fc9 zero size 2024-05-17T21:53:08.737 INFO:tasks.workunit.client.1.smithi138.stdout:9/630: read d6/d7/d16/d29/d6c/d9c/dc1/f92 [2308490,72667] 0 2024-05-17T21:53:08.738 INFO:tasks.workunit.client.1.smithi138.stdout:6/752: fdatasync d7/d69/db5/dc8/f25 0 2024-05-17T21:53:08.738 INFO:tasks.workunit.client.0.smithi107.stdout:1/951: creat d4/df/d14/d2e/dd3/f132 x:0 0 0 2024-05-17T21:53:08.740 INFO:tasks.workunit.client.0.smithi107.stdout:7/855: mknod d5/df/d11/d1d/d33/c128 0 2024-05-17T21:53:08.743 INFO:tasks.workunit.client.0.smithi107.stdout:2/738: rename d0/d9/d1c/l66 to d0/d9/d1c/dd4/lef 0 2024-05-17T21:53:08.745 INFO:tasks.workunit.client.0.smithi107.stdout:2/739: write d0/d9/d11/df/d5a/d6d/dc8/de8/f8a [24253,124752] 0 2024-05-17T21:53:08.746 INFO:tasks.workunit.client.0.smithi107.stdout:9/922: creat d0/dd/d21/d29/dca/f130 x:0 0 0 2024-05-17T21:53:08.747 INFO:tasks.workunit.client.0.smithi107.stdout:9/923: write d0/dd/d69/f83 [2820817,119249] 0 2024-05-17T21:53:08.748 INFO:tasks.workunit.client.0.smithi107.stdout:9/924: readlink d0/dd/d2f/l6d 0 2024-05-17T21:53:08.748 INFO:tasks.workunit.client.0.smithi107.stdout:5/907: fdatasync d5/db/d16/d2d/d32/f38 0 2024-05-17T21:53:08.749 INFO:tasks.workunit.client.1.smithi138.stdout:9/631: creat d6/d7/d16/d29/d6c/d9c/dc1/d7e/db7/fc4 x:0 0 0 2024-05-17T21:53:08.751 INFO:tasks.workunit.client.1.smithi138.stdout:3/889: mkdir d1/d12f 0 2024-05-17T21:53:08.755 INFO:tasks.workunit.client.0.smithi107.stdout:8/877: creat dc/d21/d27/d2f/dba/d104/f132 x:0 0 0 2024-05-17T21:53:08.756 INFO:tasks.workunit.client.0.smithi107.stdout:1/952: symlink d4/df/d1a/d41/db2/l133 0 2024-05-17T21:53:08.759 INFO:tasks.workunit.client.0.smithi107.stdout:4/942: rename d3/db/d35/d5e/dff/d80/d133/f141 to d3/d4a/d47/da3/f145 0 2024-05-17T21:53:08.761 INFO:tasks.workunit.client.0.smithi107.stdout:2/740: dread - d0/f6f zero size 2024-05-17T21:53:08.762 INFO:tasks.workunit.client.0.smithi107.stdout:3/934: symlink d6/d2a/l12c 0 2024-05-17T21:53:08.769 INFO:tasks.workunit.client.0.smithi107.stdout:6/840: dwrite d9/d16/d4d/d85/f98 [0,4194304] 0 2024-05-17T21:53:08.784 INFO:tasks.workunit.client.0.smithi107.stdout:1/953: mknod d4/df/d14/d2e/dd3/c134 0 2024-05-17T21:53:08.785 INFO:tasks.workunit.client.0.smithi107.stdout:7/856: rename d5/df/d78/c109 to d5/df/d11/d1d/d33/c129 0 2024-05-17T21:53:08.786 INFO:tasks.workunit.client.0.smithi107.stdout:4/943: readlink d3/db/d35/d5e/dff/d80/l122 0 2024-05-17T21:53:08.787 INFO:tasks.workunit.client.0.smithi107.stdout:6/841: mkdir d9/d16/d4d/db8/d110 0 2024-05-17T21:53:08.789 INFO:tasks.workunit.client.0.smithi107.stdout:3/935: mkdir d6/d2a/d2d/da8/daa/d12d 0 2024-05-17T21:53:08.791 INFO:tasks.workunit.client.0.smithi107.stdout:4/944: mknod d3/db/d35/d5e/dff/c146 0 2024-05-17T21:53:08.792 INFO:tasks.workunit.client.0.smithi107.stdout:6/842: dread d9/f1f [0,4194304] 0 2024-05-17T21:53:08.792 INFO:tasks.workunit.client.0.smithi107.stdout:1/954: symlink d4/df/d1a/d41/l135 0 2024-05-17T21:53:08.793 INFO:tasks.workunit.client.0.smithi107.stdout:4/945: chown d3/d6/d26/d36/d7e/f89 400900954 1 2024-05-17T21:53:08.794 INFO:tasks.workunit.client.0.smithi107.stdout:4/946: truncate d3/db/d35/f12f 514764 0 2024-05-17T21:53:08.795 INFO:tasks.workunit.client.0.smithi107.stdout:9/925: getdents d0/dd/d1f 0 2024-05-17T21:53:08.798 INFO:tasks.workunit.client.1.smithi138.stdout:5/710: link d1/l14 d1/d53/d88/db9/le6 0 2024-05-17T21:53:08.798 INFO:tasks.workunit.client.1.smithi138.stdout:0/667: sync 2024-05-17T21:53:08.798 INFO:tasks.workunit.client.1.smithi138.stdout:7/693: sync 2024-05-17T21:53:08.799 INFO:tasks.workunit.client.0.smithi107.stdout:8/878: rename dc/d44/d4b/d52/d70/d76/f78 to dc/d21/d97/d63/f133 0 2024-05-17T21:53:08.800 INFO:tasks.workunit.client.1.smithi138.stdout:3/890: dwrite d1/d4f/f126 [0,4194304] 0 2024-05-17T21:53:08.803 INFO:tasks.workunit.client.0.smithi107.stdout:1/955: creat d4/df/d1a/d41/db2/dc8/f136 x:0 0 0 2024-05-17T21:53:08.805 INFO:tasks.workunit.client.1.smithi138.stdout:6/753: dread f1 [4194304,4194304] 0 2024-05-17T21:53:08.808 INFO:tasks.workunit.client.0.smithi107.stdout:1/956: dread d4/df/d14/d2e/d43/d47/fa3 [0,4194304] 0 2024-05-17T21:53:08.808 INFO:tasks.workunit.client.0.smithi107.stdout:4/947: mknod d3/d6/d26/d36/d8b/d95/dee/c147 0 2024-05-17T21:53:08.809 INFO:tasks.workunit.client.0.smithi107.stdout:9/926: chown d0/d80/dad/l12f 11 1 2024-05-17T21:53:08.810 INFO:tasks.workunit.client.0.smithi107.stdout:5/908: dwrite d5/db/d16/d2d/d40/fe3 [0,4194304] 0 2024-05-17T21:53:08.810 INFO:tasks.workunit.client.0.smithi107.stdout:3/936: truncate d6/d2a/d2d/d37/f105 2257284 0 2024-05-17T21:53:08.812 INFO:tasks.workunit.client.0.smithi107.stdout:9/927: chown d0/dd/d21/d29/d3e/d94/cd2 2 1 2024-05-17T21:53:08.816 INFO:tasks.workunit.client.0.smithi107.stdout:2/741: rename d0/d9/d1c/d22/f31 to d0/d9/d11/d18/d9e/deb/ff0 0 2024-05-17T21:53:08.818 INFO:tasks.workunit.client.0.smithi107.stdout:9/928: dwrite d0/dd/d69/db2/fbd [0,4194304] 0 2024-05-17T21:53:08.818 INFO:tasks.workunit.client.0.smithi107.stdout:8/879: mkdir dc/d21/d35/d134 0 2024-05-17T21:53:08.819 INFO:tasks.workunit.client.0.smithi107.stdout:1/957: dread d4/df/d14/d2e/d43/f51 [0,4194304] 0 2024-05-17T21:53:08.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:08 smithi138 ceph-mon[35948]: pgmap v42: 65 pgs: 65 active+clean; 3.0 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 112 MiB/s rd, 266 MiB/s wr, 604 op/s 2024-05-17T21:53:08.827 INFO:tasks.workunit.client.0.smithi107.stdout:7/857: link d5/d16/l3c d5/df/d11/d1c/d2f/d53/l12a 0 2024-05-17T21:53:08.836 INFO:tasks.workunit.client.0.smithi107.stdout:5/909: mknod d5/db/d16/d54/d59/d65/d48/c12d 0 2024-05-17T21:53:08.840 INFO:tasks.workunit.client.0.smithi107.stdout:9/929: mknod d0/dd/d21/d29/d57/c131 0 2024-05-17T21:53:08.842 INFO:tasks.workunit.client.0.smithi107.stdout:8/880: creat dc/d44/d4b/d121/de5/f135 x:0 0 0 2024-05-17T21:53:08.848 INFO:tasks.workunit.client.0.smithi107.stdout:7/858: dread - d5/d16/d3b/d45/d91/db3/fff zero size 2024-05-17T21:53:08.848 INFO:tasks.workunit.client.0.smithi107.stdout:7/859: fdatasync d5/f83 0 2024-05-17T21:53:08.849 INFO:tasks.workunit.client.0.smithi107.stdout:7/860: stat d5/df/d11/d2a/ca6 0 2024-05-17T21:53:08.853 INFO:tasks.workunit.client.0.smithi107.stdout:5/910: unlink d5/db/dc/d1e/d6e/c95 0 2024-05-17T21:53:08.858 INFO:tasks.workunit.client.1.smithi138.stdout:1/679: rename d4/d2e/d36/la3 to d4/ldb 0 2024-05-17T21:53:08.859 INFO:tasks.workunit.client.0.smithi107.stdout:8/881: fsync dc/d21/db8/fc4 0 2024-05-17T21:53:08.860 INFO:tasks.workunit.client.1.smithi138.stdout:5/711: creat d1/d46/fe7 x:0 0 0 2024-05-17T21:53:08.864 INFO:tasks.workunit.client.1.smithi138.stdout:0/668: symlink d3/d10/d6d/d9f/da8/le2 0 2024-05-17T21:53:08.869 INFO:tasks.workunit.client.0.smithi107.stdout:7/861: read d5/df/d11/d1c/d31/d56/fa8 [2220473,32363] 0 2024-05-17T21:53:08.871 INFO:tasks.workunit.client.0.smithi107.stdout:3/937: rename d6/d2a/d2d/d48/f93 to d6/d2a/d2d/d69/dad/de1/f12e 0 2024-05-17T21:53:08.871 INFO:tasks.workunit.client.0.smithi107.stdout:4/948: link d3/db/d1e/d66/c140 d3/db/d35/d5e/dff/d80/dad/c148 0 2024-05-17T21:53:08.872 INFO:tasks.workunit.client.1.smithi138.stdout:1/680: creat d4/df/d3a/d3c/d63/d6d/fdc x:0 0 0 2024-05-17T21:53:08.873 INFO:tasks.workunit.client.0.smithi107.stdout:5/911: symlink d5/db/d16/d2a/d73/dd0/l12e 0 2024-05-17T21:53:08.873 INFO:tasks.workunit.client.0.smithi107.stdout:5/912: fsync d5/db/d16/d11d/f116 0 2024-05-17T21:53:08.879 INFO:tasks.workunit.client.1.smithi138.stdout:3/891: link d1/d3/d16/d3e/f4a d1/d4f/d80/de5/f130 0 2024-05-17T21:53:08.880 INFO:tasks.workunit.client.1.smithi138.stdout:3/892: readlink d1/d3/d49/ddb/l103 0 2024-05-17T21:53:08.881 INFO:tasks.workunit.client.0.smithi107.stdout:8/882: dread dc/d21/d27/d34/df5/ffd [0,4194304] 0 2024-05-17T21:53:08.882 INFO:tasks.workunit.client.0.smithi107.stdout:1/958: truncate d4/df/d1a/d41/f48 2491898 0 2024-05-17T21:53:08.884 INFO:tasks.workunit.client.1.smithi138.stdout:9/632: rename d6/d7/d16/d29/l50 to d6/d7/d16/dc2/lc5 0 2024-05-17T21:53:08.885 INFO:tasks.workunit.client.1.smithi138.stdout:9/633: chown d6/d7/d16/d29/d38/d40/d44/d5c/db4 0 1 2024-05-17T21:53:08.886 INFO:tasks.workunit.client.0.smithi107.stdout:7/862: creat d5/df/d11/d1c/d31/d39/d92/d9a/f12b x:0 0 0 2024-05-17T21:53:08.886 INFO:tasks.workunit.client.1.smithi138.stdout:9/634: read d6/d7/f10 [3358173,9836] 0 2024-05-17T21:53:08.889 INFO:tasks.workunit.client.0.smithi107.stdout:4/949: symlink d3/d4a/d47/l149 0 2024-05-17T21:53:08.894 INFO:tasks.workunit.client.1.smithi138.stdout:9/635: creat d6/d7/d16/d29/d38/d8b/fc6 x:0 0 0 2024-05-17T21:53:08.898 INFO:tasks.workunit.client.0.smithi107.stdout:1/959: truncate d4/de/f1e 1054220 0 2024-05-17T21:53:08.898 INFO:tasks.workunit.client.0.smithi107.stdout:7/863: creat d5/df/d11/d1d/dc4/f12c x:0 0 0 2024-05-17T21:53:08.900 INFO:tasks.workunit.client.1.smithi138.stdout:9/636: symlink d6/d7/d16/d29/d48/d6e/d74/d75/lc7 0 2024-05-17T21:53:08.900 INFO:tasks.workunit.client.0.smithi107.stdout:6/843: dwrite d9/d81/fb6 [0,4194304] 0 2024-05-17T21:53:08.900 INFO:tasks.workunit.client.0.smithi107.stdout:7/864: truncate d5/df/d11/d1c/d2f/f9f 2543243 0 2024-05-17T21:53:08.901 INFO:tasks.workunit.client.0.smithi107.stdout:7/865: rename d5 to d5/d16/d3b/d45/d12d 22 2024-05-17T21:53:08.901 INFO:tasks.workunit.client.0.smithi107.stdout:6/844: chown d9/d10e/d7b/fae 57 1 2024-05-17T21:53:08.903 INFO:tasks.workunit.client.1.smithi138.stdout:0/669: rename d3/d10/d6d/d9f/faf to d3/fe3 0 2024-05-17T21:53:08.905 INFO:tasks.workunit.client.0.smithi107.stdout:5/913: creat d5/d5f/f12f x:0 0 0 2024-05-17T21:53:08.907 INFO:tasks.workunit.client.1.smithi138.stdout:1/681: link d4/df/d3a/d3c/lc2 d4/dd/d3b/da7/ldd 0 2024-05-17T21:53:08.909 INFO:tasks.workunit.client.0.smithi107.stdout:8/883: fsync dc/d21/d27/d2f/dba/d104/f117 0 2024-05-17T21:53:08.911 INFO:tasks.workunit.client.1.smithi138.stdout:9/637: fsync d6/d7/d16/d29/d38/d40/d44/f90 0 2024-05-17T21:53:08.918 INFO:tasks.workunit.client.1.smithi138.stdout:3/893: rename d1/d3/d16/d3e/d71/f110 to d1/d62/dad/d5c/d9c/dec/f131 0 2024-05-17T21:53:08.918 INFO:tasks.workunit.client.1.smithi138.stdout:9/638: dwrite d6/d7/d16/d29/d48/f7d [0,4194304] 0 2024-05-17T21:53:08.920 INFO:tasks.workunit.client.1.smithi138.stdout:0/670: creat d3/d3d/d48/fe4 x:0 0 0 2024-05-17T21:53:08.922 INFO:tasks.workunit.client.0.smithi107.stdout:6/845: symlink d9/d10e/d66/l111 0 2024-05-17T21:53:08.923 INFO:tasks.workunit.client.0.smithi107.stdout:6/846: readlink d9/d16/d4d/d85/lc2 0 2024-05-17T21:53:08.924 INFO:tasks.workunit.client.1.smithi138.stdout:5/712: write d1/d2c/d63/db5/d1b/dc5/f3b [544641,105482] 0 2024-05-17T21:53:08.927 INFO:tasks.workunit.client.0.smithi107.stdout:7/866: rename d5/d16/d3b/d45/d91 to d5/d16/d3b/ddf/d12e 0 2024-05-17T21:53:08.932 INFO:tasks.workunit.client.1.smithi138.stdout:6/754: truncate d7/f68 97608 0 2024-05-17T21:53:08.934 INFO:tasks.workunit.client.1.smithi138.stdout:3/894: readlink d1/d4f/lb1 0 2024-05-17T21:53:08.936 INFO:tasks.workunit.client.1.smithi138.stdout:3/895: read d1/d62/dad/d5c/d9c/dec/f11e [879365,77294] 0 2024-05-17T21:53:08.936 INFO:tasks.workunit.client.1.smithi138.stdout:3/896: stat d1/d4f/d80/cb9 0 2024-05-17T21:53:08.950 INFO:tasks.workunit.client.1.smithi138.stdout:2/744: sync 2024-05-17T21:53:08.968 INFO:tasks.workunit.client.1.smithi138.stdout:8/642: sync 2024-05-17T21:53:08.968 INFO:tasks.workunit.client.1.smithi138.stdout:7/694: sync 2024-05-17T21:53:08.968 INFO:tasks.workunit.client.1.smithi138.stdout:4/725: sync 2024-05-17T21:53:08.970 INFO:tasks.workunit.client.1.smithi138.stdout:8/643: write d0/dc/d1a/d51/d82/f8e [904227,15219] 0 2024-05-17T21:53:08.971 INFO:tasks.workunit.client.1.smithi138.stdout:7/695: write d0/d18/d40/d41/d50/f74 [3690210,8847] 0 2024-05-17T21:53:08.972 INFO:tasks.workunit.client.1.smithi138.stdout:8/644: write d0/dc/d1a/d33/d4a/fa8 [212397,16598] 0 2024-05-17T21:53:08.979 INFO:tasks.workunit.client.1.smithi138.stdout:9/639: unlink d6/f49 0 2024-05-17T21:53:08.980 INFO:tasks.workunit.client.1.smithi138.stdout:8/645: dwrite d0/d71/fa5 [0,4194304] 0 2024-05-17T21:53:08.982 INFO:tasks.workunit.client.1.smithi138.stdout:8/646: readlink d0/d11/l54 0 2024-05-17T21:53:09.037 INFO:tasks.workunit.client.0.smithi107.stdout:3/938: write d6/f2c [1145602,58758] 0 2024-05-17T21:53:09.039 INFO:tasks.workunit.client.0.smithi107.stdout:3/939: read - d6/d2a/d2d/d37/f115 zero size 2024-05-17T21:53:09.042 INFO:tasks.workunit.client.0.smithi107.stdout:8/884: rename dc/d44/d4b/cb2 to dc/d21/d35/c136 0 2024-05-17T21:53:09.045 INFO:tasks.workunit.client.1.smithi138.stdout:5/713: mknod d1/d46/ce8 0 2024-05-17T21:53:09.049 INFO:tasks.workunit.client.1.smithi138.stdout:3/897: fdatasync d1/d3/d8d/fc7 0 2024-05-17T21:53:09.050 INFO:tasks.workunit.client.1.smithi138.stdout:3/898: chown d1/d62/dad/d5c/d9c/fa3 14 1 2024-05-17T21:53:09.051 INFO:tasks.workunit.client.0.smithi107.stdout:4/950: getdents d3/d4a/d47 0 2024-05-17T21:53:09.057 INFO:tasks.workunit.client.1.smithi138.stdout:4/726: unlink d0/d3/lbb 0 2024-05-17T21:53:09.057 INFO:tasks.workunit.client.0.smithi107.stdout:3/940: creat d6/d8/d3e/dc9/f12f x:0 0 0 2024-05-17T21:53:09.058 INFO:tasks.workunit.client.1.smithi138.stdout:4/727: read d0/d2f/d49/d60/d1f/d7e/f83 [5694149,18921] 0 2024-05-17T21:53:09.060 INFO:tasks.workunit.client.1.smithi138.stdout:4/728: chown d0/d3/d1d/d38/d8a/dbd/ff5 1049929 1 2024-05-17T21:53:09.060 INFO:tasks.workunit.client.0.smithi107.stdout:8/885: mkdir dc/dd5/d137 0 2024-05-17T21:53:09.064 INFO:tasks.workunit.client.0.smithi107.stdout:1/960: dwrite d4/df/d1a/d26/d72/d85/df9/ffb [0,4194304] 0 2024-05-17T21:53:09.067 INFO:tasks.workunit.client.0.smithi107.stdout:4/951: creat d3/d4a/d47/da3/f14a x:0 0 0 2024-05-17T21:53:09.069 INFO:tasks.workunit.client.1.smithi138.stdout:9/640: mkdir d6/d7/d16/d29/d38/d40/d44/d5c/dc8 0 2024-05-17T21:53:09.071 INFO:tasks.workunit.client.0.smithi107.stdout:3/941: symlink d6/d8/d10/d35/d91/de4/l130 0 2024-05-17T21:53:09.073 INFO:tasks.workunit.client.0.smithi107.stdout:5/914: dwrite d5/d75/fce [0,4194304] 0 2024-05-17T21:53:09.077 INFO:tasks.workunit.client.0.smithi107.stdout:8/886: creat dc/d21/db8/f138 x:0 0 0 2024-05-17T21:53:09.079 INFO:tasks.workunit.client.1.smithi138.stdout:1/682: link d4/df/d35/l45 d4/df/d3a/d3c/d63/d6d/lde 0 2024-05-17T21:53:09.082 INFO:tasks.workunit.client.1.smithi138.stdout:0/671: dwrite d3/d10/d38/d3e/d59/f9d [0,4194304] 0 2024-05-17T21:53:09.087 INFO:tasks.workunit.client.1.smithi138.stdout:6/755: link d7/da0/fbf d7/d69/db5/dc8/d43/d63/f104 0 2024-05-17T21:53:09.102 INFO:tasks.workunit.client.0.smithi107.stdout:3/942: dread d6/d2a/d2d/d37/f3a [0,4194304] 0 2024-05-17T21:53:09.114 INFO:tasks.workunit.client.0.smithi107.stdout:3/943: mkdir d6/d8/d10/d35/d36/d4c/d114/d131 0 2024-05-17T21:53:09.115 INFO:tasks.workunit.client.1.smithi138.stdout:1/683: read d4/dd/d3b/f53 [356075,72895] 0 2024-05-17T21:53:09.117 INFO:tasks.workunit.client.0.smithi107.stdout:3/944: symlink d6/d8/d3e/d3f/dbe/l132 0 2024-05-17T21:53:09.118 INFO:tasks.workunit.client.0.smithi107.stdout:3/945: write d6/d2a/f42 [923552,74372] 0 2024-05-17T21:53:09.122 INFO:tasks.workunit.client.1.smithi138.stdout:3/899: mknod d1/d4f/d80/de5/c132 0 2024-05-17T21:53:09.122 INFO:tasks.workunit.client.1.smithi138.stdout:3/900: readlink d1/d4f/d87/l12c 0 2024-05-17T21:53:09.123 INFO:tasks.workunit.client.1.smithi138.stdout:3/901: chown d1/d62/dad/c106 39 1 2024-05-17T21:53:09.124 INFO:tasks.workunit.client.1.smithi138.stdout:3/902: chown d1/d3/d49/ddb/f85 77 1 2024-05-17T21:53:09.129 INFO:tasks.workunit.client.1.smithi138.stdout:7/696: link d0/d18/d40/d41/d50/l8a d0/d18/d40/d9a/ldf 0 2024-05-17T21:53:09.131 INFO:tasks.workunit.client.1.smithi138.stdout:9/641: creat d6/d7/d16/d29/d48/d6e/d74/d75/d76/d8e/dba/fc9 x:0 0 0 2024-05-17T21:53:09.132 INFO:tasks.workunit.client.1.smithi138.stdout:1/684: dread d4/db/f4b [0,4194304] 0 2024-05-17T21:53:09.136 INFO:tasks.workunit.client.1.smithi138.stdout:0/672: dread d3/d3d/f53 [0,4194304] 0 2024-05-17T21:53:09.137 INFO:tasks.workunit.client.1.smithi138.stdout:0/673: chown d3/d10/d6d/d9f/da8 4587 1 2024-05-17T21:53:09.137 INFO:tasks.workunit.client.1.smithi138.stdout:9/642: dwrite d6/d7/d16/f21 [0,4194304] 0 2024-05-17T21:53:09.138 INFO:tasks.workunit.client.1.smithi138.stdout:3/903: rename d1/d62/dad/l44 to d1/d3/d8d/d92/d124/l133 0 2024-05-17T21:53:09.140 INFO:tasks.workunit.client.1.smithi138.stdout:7/697: mkdir d0/d18/d64/d7c/da2/de0 0 2024-05-17T21:53:09.141 INFO:tasks.workunit.client.1.smithi138.stdout:3/904: truncate d1/d3/d16/fa2 4943841 0 2024-05-17T21:53:09.142 INFO:tasks.workunit.client.1.smithi138.stdout:5/714: getdents d1/d53/d88/db9 0 2024-05-17T21:53:09.143 INFO:tasks.workunit.client.1.smithi138.stdout:5/715: stat d1/d53/d88/db9/f26 0 2024-05-17T21:53:09.146 INFO:tasks.workunit.client.1.smithi138.stdout:6/756: rmdir d7/d69/db5/dc8/d43/d3f/df3 0 2024-05-17T21:53:09.149 INFO:tasks.workunit.client.1.smithi138.stdout:3/905: dwrite d1/d62/dad/de/f108 [0,4194304] 0 2024-05-17T21:53:09.151 INFO:tasks.workunit.client.1.smithi138.stdout:7/698: truncate d0/f11 3857818 0 2024-05-17T21:53:09.155 INFO:tasks.workunit.client.1.smithi138.stdout:5/716: creat d1/d2c/d63/db5/d73/fe9 x:0 0 0 2024-05-17T21:53:09.158 INFO:tasks.workunit.client.1.smithi138.stdout:3/906: chown d1/d62/dad/d72/fdc 517957 1 2024-05-17T21:53:09.160 INFO:tasks.workunit.client.1.smithi138.stdout:7/699: creat d0/d18/d9c/fe1 x:0 0 0 2024-05-17T21:53:09.160 INFO:tasks.workunit.client.1.smithi138.stdout:7/700: stat d0/d39/f3d 0 2024-05-17T21:53:09.163 INFO:tasks.workunit.client.1.smithi138.stdout:5/717: unlink d1/dd1/dca/dd0/ld5 0 2024-05-17T21:53:09.167 INFO:tasks.workunit.client.1.smithi138.stdout:3/907: mkdir d1/d3/d8d/d92/d124/d134 0 2024-05-17T21:53:09.170 INFO:tasks.workunit.client.1.smithi138.stdout:1/685: getdents d4/df/d3a/d3c 0 2024-05-17T21:53:09.172 INFO:tasks.workunit.client.1.smithi138.stdout:3/908: dread d1/d3/d16/d5d/feb [0,4194304] 0 2024-05-17T21:53:09.172 INFO:tasks.workunit.client.1.smithi138.stdout:5/718: fsync d1/d2c/d63/db5/fe 0 2024-05-17T21:53:09.177 INFO:tasks.workunit.client.1.smithi138.stdout:1/686: creat d4/dd/d3b/d6c/dc1/fdf x:0 0 0 2024-05-17T21:53:09.180 INFO:tasks.workunit.client.1.smithi138.stdout:3/909: dread d1/d62/dad/d5c/f9f [0,4194304] 0 2024-05-17T21:53:09.183 INFO:tasks.workunit.client.1.smithi138.stdout:3/910: dread - d1/d62/dad/d72/fdc zero size 2024-05-17T21:53:09.185 INFO:tasks.workunit.client.1.smithi138.stdout:3/911: fsync d1/d3/d16/d5d/feb 0 2024-05-17T21:53:09.217 INFO:tasks.workunit.client.0.smithi107.stdout:3/946: symlink d6/d2a/d2d/d69/dad/l133 0 2024-05-17T21:53:09.219 INFO:tasks.workunit.client.0.smithi107.stdout:3/947: symlink d6/d8/d10/d35/d36/d4c/dcc/d111/l134 0 2024-05-17T21:53:09.222 INFO:tasks.workunit.client.0.smithi107.stdout:6/847: write d9/f7c [62747,125544] 0 2024-05-17T21:53:09.225 INFO:tasks.workunit.client.0.smithi107.stdout:6/848: creat d9/d16/d4d/db8/dc9/f112 x:0 0 0 2024-05-17T21:53:09.229 INFO:tasks.workunit.client.0.smithi107.stdout:6/849: creat d9/d10e/d66/dd5/de9/f113 x:0 0 0 2024-05-17T21:53:09.230 INFO:tasks.workunit.client.0.smithi107.stdout:7/867: dwrite d5/d16/d3b/ddf/d12e/db3/fed [0,4194304] 0 2024-05-17T21:53:09.231 INFO:tasks.workunit.client.0.smithi107.stdout:6/850: mkdir d9/d10e/d7b/d114 0 2024-05-17T21:53:09.236 INFO:tasks.workunit.client.0.smithi107.stdout:6/851: unlink d9/d16/d4d/db8/dc9/dd0/f39 0 2024-05-17T21:53:09.238 INFO:tasks.workunit.client.0.smithi107.stdout:7/868: dwrite d5/df/d11/d1c/d2f/d65/ff0 [0,4194304] 0 2024-05-17T21:53:09.242 INFO:tasks.workunit.client.0.smithi107.stdout:7/869: symlink d5/d16/d3b/ddf/d12e/dac/l12f 0 2024-05-17T21:53:09.244 INFO:tasks.workunit.client.0.smithi107.stdout:6/852: link d9/d16/d4d/db8/dc9/dd0/fb5 d9/d10e/d66/dd5/de9/f115 0 2024-05-17T21:53:09.248 INFO:tasks.workunit.client.0.smithi107.stdout:6/853: truncate d9/f1d 1026583 0 2024-05-17T21:53:09.249 INFO:tasks.workunit.client.0.smithi107.stdout:6/854: chown d9/d10e/f9b 573093 1 2024-05-17T21:53:09.249 INFO:tasks.workunit.client.0.smithi107.stdout:7/870: dread d5/df/d11/d1c/d31/d39/f8d [0,4194304] 0 2024-05-17T21:53:09.256 INFO:tasks.workunit.client.0.smithi107.stdout:6/855: link d9/d16/d4d/db8/dc9/dd0/l102 d9/d10e/d66/dd5/l116 0 2024-05-17T21:53:09.258 INFO:tasks.workunit.client.0.smithi107.stdout:2/742: sync 2024-05-17T21:53:09.266 INFO:tasks.workunit.client.0.smithi107.stdout:8/887: truncate dc/d21/d27/d34/fd4 8317309 0 2024-05-17T21:53:09.266 INFO:tasks.workunit.client.1.smithi138.stdout:2/745: dwrite d3/d21/d2f/d3e/d55/d99/d93/fca [0,4194304] 0 2024-05-17T21:53:09.267 INFO:tasks.workunit.client.0.smithi107.stdout:8/888: truncate dc/d44/f131 921009 0 2024-05-17T21:53:09.270 INFO:tasks.workunit.client.1.smithi138.stdout:2/746: creat d3/d21/d2f/d3e/d58/d77/da4/df2/dd0/ff8 x:0 0 0 2024-05-17T21:53:09.272 INFO:tasks.workunit.client.0.smithi107.stdout:8/889: symlink dc/d21/d35/d134/l139 0 2024-05-17T21:53:09.273 INFO:tasks.workunit.client.1.smithi138.stdout:2/747: creat d3/d21/d2f/d3e/d55/d99/d93/ff9 x:0 0 0 2024-05-17T21:53:09.277 INFO:tasks.workunit.client.1.smithi138.stdout:2/748: symlink d3/d81/d96/da8/lfa 0 2024-05-17T21:53:09.279 INFO:tasks.workunit.client.1.smithi138.stdout:2/749: chown d3/d21/d2f/d3e/f6d 1127412 1 2024-05-17T21:53:09.282 INFO:tasks.workunit.client.1.smithi138.stdout:2/750: creat d3/d21/d2f/d3e/d58/dc3/ddf/ffb x:0 0 0 2024-05-17T21:53:09.283 INFO:tasks.workunit.client.0.smithi107.stdout:7/871: mknod d5/df/d11/c130 0 2024-05-17T21:53:09.285 INFO:tasks.workunit.client.1.smithi138.stdout:4/729: write d0/fcb [117173,59689] 0 2024-05-17T21:53:09.286 INFO:tasks.workunit.client.0.smithi107.stdout:7/872: dread - d5/d16/d3b/ddf/d12e/dac/fe7 zero size 2024-05-17T21:53:09.286 INFO:tasks.workunit.client.1.smithi138.stdout:4/730: chown d0/d3/d1d/d38/de3 6630564 1 2024-05-17T21:53:09.287 INFO:tasks.workunit.client.1.smithi138.stdout:4/731: write d0/d1b/d2b/f65 [1244194,124154] 0 2024-05-17T21:53:09.288 INFO:tasks.workunit.client.0.smithi107.stdout:7/873: mknod d5/df/d11/d1d/d44/c131 0 2024-05-17T21:53:09.289 INFO:tasks.workunit.client.0.smithi107.stdout:7/874: readlink d5/df/d11/d1c/d2f/d53/l5a 0 2024-05-17T21:53:09.292 INFO:tasks.workunit.client.1.smithi138.stdout:4/732: dread d0/d2f/d49/f90 [0,4194304] 0 2024-05-17T21:53:09.292 INFO:tasks.workunit.client.1.smithi138.stdout:4/733: dread - d0/d2f/d49/fc2 zero size 2024-05-17T21:53:09.296 INFO:tasks.workunit.client.1.smithi138.stdout:0/674: write d3/d10/fc0 [1755183,84049] 0 2024-05-17T21:53:09.296 INFO:tasks.workunit.client.1.smithi138.stdout:4/734: getdents d0/d1b/d36 0 2024-05-17T21:53:09.298 INFO:tasks.workunit.client.1.smithi138.stdout:0/675: rmdir d3/d10/d38/d3e/d59/dad 39 2024-05-17T21:53:09.299 INFO:tasks.workunit.client.1.smithi138.stdout:0/676: read - d3/d10/d6d/fc7 zero size 2024-05-17T21:53:09.300 INFO:tasks.workunit.client.1.smithi138.stdout:4/735: symlink d0/d1b/d42/l102 0 2024-05-17T21:53:09.303 INFO:tasks.workunit.client.1.smithi138.stdout:4/736: truncate d0/d2f/d49/d60/d1f/d32/f43 2208027 0 2024-05-17T21:53:09.303 INFO:tasks.workunit.client.1.smithi138.stdout:4/737: fdatasync d0/d2f/d49/d60/f89 0 2024-05-17T21:53:09.304 INFO:tasks.workunit.client.1.smithi138.stdout:4/738: write d0/d3/db/fd3 [3852302,85105] 0 2024-05-17T21:53:09.305 INFO:tasks.workunit.client.0.smithi107.stdout:5/915: rename d5/db/d16/d2d/d40/d5a/fa7 to d5/db/d16/d2d/d40/f130 0 2024-05-17T21:53:09.305 INFO:tasks.workunit.client.1.smithi138.stdout:4/739: fsync d0/d3/db/da0/fa8 0 2024-05-17T21:53:09.308 INFO:tasks.workunit.client.0.smithi107.stdout:6/856: rename d9/d81/df9 to d9/d16/d4d/d85/d96/d117 0 2024-05-17T21:53:09.311 INFO:tasks.workunit.client.1.smithi138.stdout:4/740: dwrite d0/d2f/d49/d60/f3c [0,4194304] 0 2024-05-17T21:53:09.313 INFO:tasks.workunit.client.1.smithi138.stdout:4/741: fsync d0/d3/db/da0/fa8 0 2024-05-17T21:53:09.316 INFO:tasks.workunit.client.0.smithi107.stdout:7/875: rename d5/df/d11/d1c/d2f/d53/l12a to d5/df/d11/d1c/l132 0 2024-05-17T21:53:09.320 INFO:tasks.workunit.client.0.smithi107.stdout:6/857: mknod d9/d16/d22/d5f/d6e/dd6/c118 0 2024-05-17T21:53:09.321 INFO:tasks.workunit.client.0.smithi107.stdout:3/948: symlink d6/d8/l135 0 2024-05-17T21:53:09.322 INFO:tasks.workunit.client.1.smithi138.stdout:4/742: chown d0/d1b/d42/d45/cd0 9 1 2024-05-17T21:53:09.324 INFO:tasks.workunit.client.0.smithi107.stdout:6/858: fdatasync d9/d16/d4d/db8/dc9/dd0/f31 0 2024-05-17T21:53:09.326 INFO:tasks.workunit.client.1.smithi138.stdout:4/743: creat d0/d2f/d49/d60/d1f/d7e/f103 x:0 0 0 2024-05-17T21:53:09.328 INFO:tasks.workunit.client.0.smithi107.stdout:3/949: creat d6/d8/d3e/d3f/d90/da7/ddc/f136 x:0 0 0 2024-05-17T21:53:09.328 INFO:tasks.workunit.client.0.smithi107.stdout:7/876: fdatasync d5/d16/f23 0 2024-05-17T21:53:09.331 INFO:tasks.workunit.client.0.smithi107.stdout:6/859: mkdir d9/d16/d4d/d85/d96/d117/d119 0 2024-05-17T21:53:09.332 INFO:tasks.workunit.client.1.smithi138.stdout:4/744: dread d0/d2f/d49/d60/d1f/d32/f87 [0,4194304] 0 2024-05-17T21:53:09.332 INFO:tasks.workunit.client.0.smithi107.stdout:6/860: dread d9/d16/d22/f72 [0,4194304] 0 2024-05-17T21:53:09.333 INFO:tasks.workunit.client.1.smithi138.stdout:4/745: write d0/d2f/d49/d60/d1f/f7a [3061318,108081] 0 2024-05-17T21:53:09.337 INFO:tasks.workunit.client.1.smithi138.stdout:4/746: mkdir d0/d3/d17/d21/d78/d9f/d104 0 2024-05-17T21:53:09.339 INFO:tasks.workunit.client.0.smithi107.stdout:6/861: mkdir d9/d16/d4d/d85/d88/db1/ddd/d11a 0 2024-05-17T21:53:09.341 INFO:tasks.workunit.client.0.smithi107.stdout:6/862: mknod d9/d10e/d66/dd5/df1/c11b 0 2024-05-17T21:53:09.342 INFO:tasks.workunit.client.1.smithi138.stdout:4/747: dwrite d0/d3/d17/d21/d78/fa5 [0,4194304] 0 2024-05-17T21:53:09.343 INFO:tasks.workunit.client.1.smithi138.stdout:4/748: chown d0/d2f/d49/d60/d1f/lc1 6648609 1 2024-05-17T21:53:09.348 INFO:tasks.workunit.client.1.smithi138.stdout:8/647: link d0/f2c d0/dc/fbf 0 2024-05-17T21:53:09.351 INFO:tasks.workunit.client.1.smithi138.stdout:8/648: getdents d0/dc/d3b/d4d/d7d 0 2024-05-17T21:53:09.352 INFO:tasks.workunit.client.1.smithi138.stdout:8/649: chown d0/dc/d1a/d33/l66 3799 1 2024-05-17T21:53:09.357 INFO:tasks.workunit.client.1.smithi138.stdout:9/643: rename d6/f2a to d6/d7/d16/d29/fca 0 2024-05-17T21:53:09.358 INFO:tasks.workunit.client.1.smithi138.stdout:9/644: chown d6/d7/d16/d29/d38/d8b/fc6 5827 1 2024-05-17T21:53:09.360 INFO:tasks.workunit.client.1.smithi138.stdout:7/701: rename d0/l3f to d0/d39/le2 0 2024-05-17T21:53:09.361 INFO:tasks.workunit.client.1.smithi138.stdout:9/645: creat d6/d7/d16/d29/d38/d40/d44/fcb x:0 0 0 2024-05-17T21:53:09.364 INFO:tasks.workunit.client.1.smithi138.stdout:1/687: rename d4/dd/d3b/da7/dbd to d4/df/d3a/d3c/d63/d6d/de0 0 2024-05-17T21:53:09.367 INFO:tasks.workunit.client.1.smithi138.stdout:9/646: mknod d6/d7/d16/d29/d6c/d9c/dc1/ccc 0 2024-05-17T21:53:09.369 INFO:tasks.workunit.client.1.smithi138.stdout:8/650: dread d0/dc/f22 [0,4194304] 0 2024-05-17T21:53:09.369 INFO:tasks.workunit.client.0.smithi107.stdout:4/952: sync 2024-05-17T21:53:09.369 INFO:tasks.workunit.client.0.smithi107.stdout:1/961: sync 2024-05-17T21:53:09.370 INFO:tasks.workunit.client.0.smithi107.stdout:9/930: sync 2024-05-17T21:53:09.370 INFO:tasks.workunit.client.1.smithi138.stdout:7/702: dwrite d0/d18/d40/d41/d51/d81/fa3 [0,4194304] 0 2024-05-17T21:53:09.370 INFO:tasks.workunit.client.1.smithi138.stdout:9/647: write d6/d7/f82 [1306753,50644] 0 2024-05-17T21:53:09.373 INFO:tasks.workunit.client.1.smithi138.stdout:9/648: write d6/d7/d16/f21 [1245310,29004] 0 2024-05-17T21:53:09.374 INFO:tasks.workunit.client.1.smithi138.stdout:4/749: rename d0/d3/db/cb3 to d0/d1b/d2b/dca/c105 0 2024-05-17T21:53:09.383 INFO:tasks.workunit.client.0.smithi107.stdout:4/953: dread d3/fe4 [0,4194304] 0 2024-05-17T21:53:09.384 INFO:tasks.workunit.client.0.smithi107.stdout:4/954: stat d3/db/ca6 0 2024-05-17T21:53:09.387 INFO:tasks.workunit.client.0.smithi107.stdout:4/955: mknod d3/db/d1e/d99/c14b 0 2024-05-17T21:53:09.389 INFO:tasks.workunit.client.0.smithi107.stdout:4/956: write d3/fa [8911074,30416] 0 2024-05-17T21:53:09.389 INFO:tasks.workunit.client.0.smithi107.stdout:2/743: write d0/d26/ddd/f68 [40781,74331] 0 2024-05-17T21:53:09.394 INFO:tasks.workunit.client.0.smithi107.stdout:4/957: dread d3/db/d1e/d66/f70 [0,4194304] 0 2024-05-17T21:53:09.395 INFO:tasks.workunit.client.0.smithi107.stdout:2/744: symlink d0/d9/d11/d18/dad/db8/lf1 0 2024-05-17T21:53:09.397 INFO:tasks.workunit.client.0.smithi107.stdout:4/958: mkdir d3/db/d1e/dc6/d14c 0 2024-05-17T21:53:09.398 INFO:tasks.workunit.client.0.smithi107.stdout:2/745: mkdir d0/d9/d11/d15/df2 0 2024-05-17T21:53:09.399 INFO:tasks.workunit.client.0.smithi107.stdout:4/959: write d3/fe4 [196869,127614] 0 2024-05-17T21:53:09.406 INFO:tasks.workunit.client.0.smithi107.stdout:8/890: dwrite dc/d21/d27/d34/d4c/d7e/f3b [4194304,4194304] 0 2024-05-17T21:53:09.409 INFO:tasks.workunit.client.0.smithi107.stdout:4/960: getdents d3/db 0 2024-05-17T21:53:09.409 INFO:tasks.workunit.client.0.smithi107.stdout:2/746: dwrite d0/d9/d11/d27/dd3/fc1 [0,4194304] 0 2024-05-17T21:53:09.410 INFO:tasks.workunit.client.0.smithi107.stdout:5/916: write d5/db/d16/d2d/d40/d9f/fc3 [190660,53646] 0 2024-05-17T21:53:09.411 INFO:tasks.workunit.client.0.smithi107.stdout:8/891: unlink dc/d44/d4b/d52/d70/ca3 0 2024-05-17T21:53:09.414 INFO:tasks.workunit.client.0.smithi107.stdout:8/892: truncate dc/d44/d4b/d121/f128 732696 0 2024-05-17T21:53:09.414 INFO:tasks.workunit.client.0.smithi107.stdout:4/961: creat d3/d4a/f14d x:0 0 0 2024-05-17T21:53:09.415 INFO:tasks.workunit.client.0.smithi107.stdout:4/962: chown d3/db/d1e/c38 115 1 2024-05-17T21:53:09.416 INFO:tasks.workunit.client.0.smithi107.stdout:4/963: read - d3/d4a/d47/d60/f134 zero size 2024-05-17T21:53:09.417 INFO:tasks.workunit.client.0.smithi107.stdout:5/917: rename d5/db/l36 to d5/db/d16/d2a/d73/dd0/l131 0 2024-05-17T21:53:09.418 INFO:tasks.workunit.client.0.smithi107.stdout:2/747: mkdir d0/d9/df3 0 2024-05-17T21:53:09.420 INFO:tasks.workunit.client.0.smithi107.stdout:4/964: readlink d3/d4a/d47/d60/l2e 0 2024-05-17T21:53:09.422 INFO:tasks.workunit.client.0.smithi107.stdout:5/918: unlink d5/db/d16/d11d/cf2 0 2024-05-17T21:53:09.423 INFO:tasks.workunit.client.0.smithi107.stdout:3/950: write d6/d8/d10/d25/f55 [4672485,41663] 0 2024-05-17T21:53:09.425 INFO:tasks.workunit.client.0.smithi107.stdout:7/877: write d5/df/d11/f7a [649830,12262] 0 2024-05-17T21:53:09.425 INFO:tasks.workunit.client.0.smithi107.stdout:5/919: stat d5/db/d16/d54/d59/d65/ff1 0 2024-05-17T21:53:09.427 INFO:tasks.workunit.client.0.smithi107.stdout:6/863: write d9/d10e/d66/f93 [466091,51561] 0 2024-05-17T21:53:09.428 INFO:tasks.workunit.client.0.smithi107.stdout:5/920: chown d5/db/d16/d2d/d32 20 1 2024-05-17T21:53:09.430 INFO:tasks.workunit.client.0.smithi107.stdout:2/748: dwrite d0/d9/d11/f52 [0,4194304] 0 2024-05-17T21:53:09.430 INFO:tasks.workunit.client.0.smithi107.stdout:3/951: rmdir d6/d2a/d2d/da8/daa 39 2024-05-17T21:53:09.435 INFO:tasks.workunit.client.0.smithi107.stdout:4/965: dwrite d3/d4a/f73 [4194304,4194304] 0 2024-05-17T21:53:09.435 INFO:tasks.workunit.client.0.smithi107.stdout:5/921: creat d5/db/d16/d11d/f132 x:0 0 0 2024-05-17T21:53:09.437 INFO:tasks.workunit.client.0.smithi107.stdout:1/962: write d4/df/d14/d2e/d43/f51 [2181311,83422] 0 2024-05-17T21:53:09.443 INFO:tasks.workunit.client.0.smithi107.stdout:2/749: rename d0/d9/d11/d18/dad/ccd to d0/d9/d11/df/d5a/cf4 0 2024-05-17T21:53:09.449 INFO:tasks.workunit.client.0.smithi107.stdout:3/952: dwrite d6/d2a/d2d/da8/daa/df5/f7b [0,4194304] 0 2024-05-17T21:53:09.451 INFO:tasks.workunit.client.0.smithi107.stdout:3/953: chown d6/d2a/d2d/d69/dad/cb7 7707 1 2024-05-17T21:53:09.452 INFO:tasks.workunit.client.0.smithi107.stdout:4/966: mknod d3/d4a/d47/d60/d3c/d9d/c14e 0 2024-05-17T21:53:09.458 INFO:tasks.workunit.client.0.smithi107.stdout:5/922: dread d5/db/d39/f3e [0,4194304] 0 2024-05-17T21:53:09.460 INFO:tasks.workunit.client.0.smithi107.stdout:3/954: rename d6/d2a/d2d/d37/f105 to d6/d2a/d2d/d48/d4f/f137 0 2024-05-17T21:53:09.463 INFO:tasks.workunit.client.0.smithi107.stdout:7/878: dwrite d5/df/d11/d1d/d33/f87 [0,4194304] 0 2024-05-17T21:53:09.463 INFO:tasks.workunit.client.0.smithi107.stdout:3/955: rmdir d6/d8/d10/d35/d36 39 2024-05-17T21:53:09.464 INFO:tasks.workunit.client.0.smithi107.stdout:2/750: write d0/f6f [793844,92223] 0 2024-05-17T21:53:09.465 INFO:tasks.workunit.client.1.smithi138.stdout:7/703: rmdir d0/d18/d40/d9a 39 2024-05-17T21:53:09.465 INFO:tasks.workunit.client.0.smithi107.stdout:7/879: write f4 [3912548,92824] 0 2024-05-17T21:53:09.466 INFO:tasks.workunit.client.1.smithi138.stdout:8/651: symlink d0/dc/d1a/d51/lc0 0 2024-05-17T21:53:09.466 INFO:tasks.workunit.client.0.smithi107.stdout:3/956: mkdir d6/d2a/d2d/d37/d138 0 2024-05-17T21:53:09.467 INFO:tasks.workunit.client.1.smithi138.stdout:1/688: dwrite d4/db/d29/d5d/f86 [0,4194304] 0 2024-05-17T21:53:09.468 INFO:tasks.workunit.client.1.smithi138.stdout:4/750: write d0/d2f/d49/d60/d1f/d7e/f103 [212268,38455] 0 2024-05-17T21:53:09.469 INFO:tasks.workunit.client.0.smithi107.stdout:1/963: truncate d4/df/d1a/d26/d7a/f93 2150881 0 2024-05-17T21:53:09.469 INFO:tasks.workunit.client.1.smithi138.stdout:0/677: write d3/d3d/d48/fb0 [288487,51661] 0 2024-05-17T21:53:09.470 INFO:tasks.workunit.client.0.smithi107.stdout:3/957: write d6/d8/d3e/d3f/dbe/fee [24612,56021] 0 2024-05-17T21:53:09.472 INFO:tasks.workunit.client.1.smithi138.stdout:1/689: fdatasync d4/df/d3a/d3c/d63/d6d/fda 0 2024-05-17T21:53:09.476 INFO:tasks.workunit.client.1.smithi138.stdout:0/678: chown d3/d10/d38/d70/d80/f8b 14394 1 2024-05-17T21:53:09.477 INFO:tasks.workunit.client.0.smithi107.stdout:5/923: write d5/db/d16/f4b [4945775,972] 0 2024-05-17T21:53:09.477 INFO:tasks.workunit.client.1.smithi138.stdout:1/690: write d4/df/d4e/fd7 [369439,35263] 0 2024-05-17T21:53:09.477 INFO:tasks.workunit.client.0.smithi107.stdout:4/967: dwrite d3/d4a/d47/d60/d3c/fd9 [0,4194304] 0 2024-05-17T21:53:09.479 INFO:tasks.workunit.client.1.smithi138.stdout:2/751: dwrite d3/d21/d2f/f36 [0,4194304] 0 2024-05-17T21:53:09.479 INFO:tasks.workunit.client.0.smithi107.stdout:4/968: chown d3/ddf/fe5 1 1 2024-05-17T21:53:09.479 INFO:tasks.workunit.client.1.smithi138.stdout:1/691: chown d4/dd/d3b/d9f/db9/dae 60088 1 2024-05-17T21:53:09.481 INFO:tasks.workunit.client.0.smithi107.stdout:2/751: mkdir d0/d93/df5 0 2024-05-17T21:53:09.481 INFO:tasks.workunit.client.1.smithi138.stdout:7/704: fdatasync d0/d18/d40/d41/d50/f59 0 2024-05-17T21:53:09.482 INFO:tasks.workunit.client.1.smithi138.stdout:8/652: mkdir d0/dc/d3b/d4d/d7d/d97/dc1 0 2024-05-17T21:53:09.484 INFO:tasks.workunit.client.1.smithi138.stdout:8/653: chown d0/d11/l27 857172376 1 2024-05-17T21:53:09.484 INFO:tasks.workunit.client.1.smithi138.stdout:4/751: truncate d0/d2f/d49/d60/d1f/f73 718390 0 2024-05-17T21:53:09.485 INFO:tasks.workunit.client.0.smithi107.stdout:1/964: truncate d4/df/d1a/fc4 429753 0 2024-05-17T21:53:09.491 INFO:tasks.workunit.client.1.smithi138.stdout:0/679: unlink d3/d10/d6d/db6/cca 0 2024-05-17T21:53:09.491 INFO:tasks.workunit.client.0.smithi107.stdout:4/969: dwrite d3/d6/d26/d36/d8b/d95/dee/f10b [4194304,4194304] 0 2024-05-17T21:53:09.493 INFO:tasks.workunit.client.1.smithi138.stdout:1/692: dread f0 [0,4194304] 0 2024-05-17T21:53:09.493 INFO:tasks.workunit.client.0.smithi107.stdout:3/958: mkdir d6/d8/d10/d35/d91/d139 0 2024-05-17T21:53:09.495 INFO:tasks.workunit.client.1.smithi138.stdout:2/752: rename d3/d21/d2f/d3e/d55/d99/dd2/fe4 to d3/d21/d2f/d3e/d63/ffc 0 2024-05-17T21:53:09.496 INFO:tasks.workunit.client.1.smithi138.stdout:2/753: chown d3/d21/d2f/d3e/d55/d99/f8f 14740 1 2024-05-17T21:53:09.499 INFO:tasks.workunit.client.1.smithi138.stdout:4/752: read d0/d2f/d49/d60/d1f/d7e/f83 [6394657,79210] 0 2024-05-17T21:53:09.499 INFO:tasks.workunit.client.1.smithi138.stdout:9/649: symlink d6/d7/d16/d29/d48/d6e/d74/d75/lcd 0 2024-05-17T21:53:09.500 INFO:tasks.workunit.client.0.smithi107.stdout:1/965: dwrite d4/df/d1a/d26/f126 [0,4194304] 0 2024-05-17T21:53:09.501 INFO:tasks.workunit.client.0.smithi107.stdout:4/970: dread d3/d4a/d47/d60/d3a/f105 [0,4194304] 0 2024-05-17T21:53:09.507 INFO:tasks.workunit.client.1.smithi138.stdout:2/754: chown d3/d21/l23 12209 1 2024-05-17T21:53:09.508 INFO:tasks.workunit.client.1.smithi138.stdout:2/755: stat d3/l70 0 2024-05-17T21:53:09.508 INFO:tasks.workunit.client.0.smithi107.stdout:7/880: link d5/d16/ca3 d5/d16/d3b/ddf/d12e/dc1/c133 0 2024-05-17T21:53:09.510 INFO:tasks.workunit.client.0.smithi107.stdout:5/924: creat d5/f133 x:0 0 0 2024-05-17T21:53:09.510 INFO:tasks.workunit.client.0.smithi107.stdout:7/881: chown d5/df/d11/d1c/d31/d39/d92/d9a/fdc 753 1 2024-05-17T21:53:09.511 INFO:tasks.workunit.client.0.smithi107.stdout:5/925: readlink d5/db/d16/d2a/l2c 0 2024-05-17T21:53:09.511 INFO:tasks.workunit.client.0.smithi107.stdout:1/966: fdatasync d4/fa 0 2024-05-17T21:53:09.512 INFO:tasks.workunit.client.0.smithi107.stdout:5/926: fsync d5/d75/fae 0 2024-05-17T21:53:09.513 INFO:tasks.workunit.client.1.smithi138.stdout:1/693: mknod d4/dd/d3b/d9f/db9/dae/ce1 0 2024-05-17T21:53:09.514 INFO:tasks.workunit.client.1.smithi138.stdout:9/650: mknod d6/cce 0 2024-05-17T21:53:09.517 INFO:tasks.workunit.client.1.smithi138.stdout:2/756: rename d3/d21/d2f/d3e/d58/dc3 to d3/d21/d2f/d3e/d55/d99/d84/d9f/dfd 0 2024-05-17T21:53:09.519 INFO:tasks.workunit.client.0.smithi107.stdout:3/959: creat d6/d8/d10/d35/d36/d4c/dcc/d111/f13a x:0 0 0 2024-05-17T21:53:09.521 INFO:tasks.workunit.client.0.smithi107.stdout:2/752: getdents d0 0 2024-05-17T21:53:09.523 INFO:tasks.workunit.client.0.smithi107.stdout:1/967: unlink d4/df/d1a/d26/d72/daa/deb/f120 0 2024-05-17T21:53:09.525 INFO:tasks.workunit.client.0.smithi107.stdout:5/927: rename d5/db/d16/d11d/f132 to d5/db/dc/f134 0 2024-05-17T21:53:09.526 INFO:tasks.workunit.client.1.smithi138.stdout:9/651: symlink d6/d7/d16/d29/d48/d67/lcf 0 2024-05-17T21:53:09.529 INFO:tasks.workunit.client.0.smithi107.stdout:4/971: creat d3/d6/d26/d36/d8b/d95/dee/dfb/d128/f14f x:0 0 0 2024-05-17T21:53:09.530 INFO:tasks.workunit.client.1.smithi138.stdout:2/757: mkdir d3/da0/dfe 0 2024-05-17T21:53:09.530 INFO:tasks.workunit.client.0.smithi107.stdout:4/972: write d3/d6/d26/d36/d37/d85/fec [944687,20564] 0 2024-05-17T21:53:09.531 INFO:tasks.workunit.client.0.smithi107.stdout:3/960: creat d6/d8/d3e/d3f/dbe/f13b x:0 0 0 2024-05-17T21:53:09.533 INFO:tasks.workunit.client.1.smithi138.stdout:1/694: creat d4/db/fe2 x:0 0 0 2024-05-17T21:53:09.534 INFO:tasks.workunit.client.1.smithi138.stdout:9/652: dwrite d6/d7/d16/d29/d38/d8b/fc6 [0,4194304] 0 2024-05-17T21:53:09.534 INFO:tasks.workunit.client.1.smithi138.stdout:1/695: chown d4/db/d29/l2b 1646397573 1 2024-05-17T21:53:09.535 INFO:tasks.workunit.client.1.smithi138.stdout:4/753: dread d0/d3/d17/d5e/fbf [0,4194304] 0 2024-05-17T21:53:09.536 INFO:tasks.workunit.client.1.smithi138.stdout:1/696: dread - d4/d2e/d36/fac zero size 2024-05-17T21:53:09.536 INFO:tasks.workunit.client.0.smithi107.stdout:2/753: mknod d0/d9/d11/df/d5a/d6d/cf6 0 2024-05-17T21:53:09.542 INFO:tasks.workunit.client.1.smithi138.stdout:2/758: mkdir d3/d21/d2f/d3e/d58/d77/da4/dff 0 2024-05-17T21:53:09.546 INFO:tasks.workunit.client.1.smithi138.stdout:1/697: rmdir d4 39 2024-05-17T21:53:09.548 INFO:tasks.workunit.client.1.smithi138.stdout:7/705: dwrite d0/f11 [0,4194304] 0 2024-05-17T21:53:09.550 INFO:tasks.workunit.client.0.smithi107.stdout:3/961: creat d6/d8/d3e/d3f/d90/da7/ddc/f13c x:0 0 0 2024-05-17T21:53:09.550 INFO:tasks.workunit.client.1.smithi138.stdout:4/754: truncate d0/d3/d1d/d5b/fb6 557057 0 2024-05-17T21:53:09.551 INFO:tasks.workunit.client.0.smithi107.stdout:3/962: write d6/d8/d10/d25/f99 [2722537,119525] 0 2024-05-17T21:53:09.560 INFO:tasks.workunit.client.1.smithi138.stdout:7/706: stat d0/d18/d9c/cb1 0 2024-05-17T21:53:09.562 INFO:tasks.workunit.client.1.smithi138.stdout:4/755: readlink d0/d3/d1d/d5b/l9b 0 2024-05-17T21:53:09.564 INFO:tasks.workunit.client.0.smithi107.stdout:3/963: symlink d6/d2a/d2d/da8/daa/df5/d88/l13d 0 2024-05-17T21:53:09.569 INFO:tasks.workunit.client.1.smithi138.stdout:2/759: dread d3/fe [0,4194304] 0 2024-05-17T21:53:09.569 INFO:tasks.workunit.client.0.smithi107.stdout:3/964: rename d6/d2a/d2d/d37/dc6/c12a to d6/d2a/d2d/d37/d138/c13e 0 2024-05-17T21:53:09.570 INFO:tasks.workunit.client.1.smithi138.stdout:7/707: symlink d0/d18/d9c/le3 0 2024-05-17T21:53:09.571 INFO:tasks.workunit.client.0.smithi107.stdout:2/754: link d0/d9/d11/c5 d0/d26/d8e/dc4/cf7 0 2024-05-17T21:53:09.573 INFO:tasks.workunit.client.1.smithi138.stdout:4/756: symlink d0/d3/d17/d21/d78/d9f/l106 0 2024-05-17T21:53:09.573 INFO:tasks.workunit.client.0.smithi107.stdout:2/755: write d0/d9/d11/d15/d99/fbb [1038417,107211] 0 2024-05-17T21:53:09.576 INFO:tasks.workunit.client.0.smithi107.stdout:3/965: dwrite d6/d1a/d57/f73 [0,4194304] 0 2024-05-17T21:53:09.577 INFO:tasks.workunit.client.0.smithi107.stdout:1/968: dread d4/df/f13 [0,4194304] 0 2024-05-17T21:53:09.577 INFO:tasks.workunit.client.1.smithi138.stdout:7/708: mknod d0/d4c/ce4 0 2024-05-17T21:53:09.578 INFO:tasks.workunit.client.1.smithi138.stdout:9/653: write d6/d7/d16/d29/d38/f81 [1765161,200] 0 2024-05-17T21:53:09.581 INFO:tasks.workunit.client.1.smithi138.stdout:4/757: fsync d0/d3/d1d/d38/d8a/fab 0 2024-05-17T21:53:09.581 INFO:tasks.workunit.client.0.smithi107.stdout:4/973: write d3/db/f69 [2501288,90612] 0 2024-05-17T21:53:09.583 INFO:tasks.workunit.client.1.smithi138.stdout:4/758: write d0/d3/db/da0/fa8 [743587,130246] 0 2024-05-17T21:53:09.585 INFO:tasks.workunit.client.0.smithi107.stdout:2/756: mkdir d0/d9/d1c/d37/df8 0 2024-05-17T21:53:09.585 INFO:tasks.workunit.client.1.smithi138.stdout:4/759: write d0/d1b/d2b/d55/d79/f7b [1664536,7307] 0 2024-05-17T21:53:09.586 INFO:tasks.workunit.client.0.smithi107.stdout:2/757: write d0/d9/d11/d27/dd3/fc1 [4113396,111058] 0 2024-05-17T21:53:09.587 INFO:tasks.workunit.client.1.smithi138.stdout:1/698: dread d4/d2e/f54 [0,4194304] 0 2024-05-17T21:53:09.587 INFO:tasks.workunit.client.1.smithi138.stdout:7/709: fsync d0/d39/f80 0 2024-05-17T21:53:09.588 INFO:tasks.workunit.client.0.smithi107.stdout:2/758: readlink d0/d9/d11/d18/dad/db8/lf1 0 2024-05-17T21:53:09.591 INFO:tasks.workunit.client.0.smithi107.stdout:4/974: mkdir d3/d4a/d47/d60/d3a/ddc/d150 0 2024-05-17T21:53:09.592 INFO:tasks.workunit.client.0.smithi107.stdout:4/975: write d3/ddf/f103 [121665,92996] 0 2024-05-17T21:53:09.593 INFO:tasks.workunit.client.0.smithi107.stdout:4/976: dread - d3/db/d35/d55/d8a/f137 zero size 2024-05-17T21:53:09.595 INFO:tasks.workunit.client.0.smithi107.stdout:4/977: chown d3/d6/d26/d36/d7e/d98/cc9 65 1 2024-05-17T21:53:09.596 INFO:tasks.workunit.client.0.smithi107.stdout:3/966: mknod d6/d8/d3e/d3f/d90/da7/c13f 0 2024-05-17T21:53:09.597 INFO:tasks.workunit.client.0.smithi107.stdout:1/969: dread d4/df/d1a/d26/f60 [0,4194304] 0 2024-05-17T21:53:09.602 INFO:tasks.workunit.client.0.smithi107.stdout:1/970: symlink d4/de/l137 0 2024-05-17T21:53:09.605 INFO:tasks.workunit.client.1.smithi138.stdout:9/654: write d6/d7/d16/d29/d38/d40/f6b [920834,18339] 0 2024-05-17T21:53:09.606 INFO:tasks.workunit.client.1.smithi138.stdout:7/710: rename d0/d18/fd2 to d0/d18/dbf/fe5 0 2024-05-17T21:53:09.608 INFO:tasks.workunit.client.0.smithi107.stdout:1/971: truncate d4/df/d1a/f58 170746 0 2024-05-17T21:53:09.609 INFO:tasks.workunit.client.1.smithi138.stdout:2/760: getdents d3/d21/d2f/d3e/d58/d77/da4/df2/dd0 0 2024-05-17T21:53:09.610 INFO:tasks.workunit.client.1.smithi138.stdout:4/760: symlink d0/d3/d1d/d38/d8a/dbd/l107 0 2024-05-17T21:53:09.611 INFO:tasks.workunit.client.0.smithi107.stdout:2/759: write d0/d9/d11/d15/f81 [354184,59051] 0 2024-05-17T21:53:09.613 INFO:tasks.workunit.client.0.smithi107.stdout:3/967: rename d6/d2a/d2d/da8/daa/df5/f7b to d6/d2a/d2d/da8/f140 0 2024-05-17T21:53:09.614 INFO:tasks.workunit.client.0.smithi107.stdout:1/972: chown d4/df/d1a/d26/d72/daa/deb/d104/l11a 847 1 2024-05-17T21:53:09.615 INFO:tasks.workunit.client.0.smithi107.stdout:4/978: dread d3/db/d35/d5e/dff/f51 [0,4194304] 0 2024-05-17T21:53:09.616 INFO:tasks.workunit.client.0.smithi107.stdout:2/760: rmdir d0/d9/d11/d18/d9e 39 2024-05-17T21:53:09.618 INFO:tasks.workunit.client.1.smithi138.stdout:1/699: dwrite d4/dd/d3b/d6c/dcf/f85 [0,4194304] 0 2024-05-17T21:53:09.618 INFO:tasks.workunit.client.0.smithi107.stdout:3/968: creat d6/d2a/d2d/da8/daa/df5/d88/f141 x:0 0 0 2024-05-17T21:53:09.623 INFO:tasks.workunit.client.0.smithi107.stdout:1/973: unlink d4/df/d14/d2e/d107/l88 0 2024-05-17T21:53:09.624 INFO:tasks.workunit.client.1.smithi138.stdout:2/761: truncate d3/d21/d2f/d3e/d55/d99/d93/f79 3050182 0 2024-05-17T21:53:09.624 INFO:tasks.workunit.client.1.smithi138.stdout:2/762: readlink d3/d21/d2f/l3b 0 2024-05-17T21:53:09.626 INFO:tasks.workunit.client.1.smithi138.stdout:9/655: write d6/d7/d16/d29/d48/d6f/f71 [3285728,17215] 0 2024-05-17T21:53:09.627 INFO:tasks.workunit.client.0.smithi107.stdout:2/761: creat d0/d9/d11/d27/dd3/ff9 x:0 0 0 2024-05-17T21:53:09.628 INFO:tasks.workunit.client.1.smithi138.stdout:4/761: write d0/d3/d17/d21/d78/fac [891874,107984] 0 2024-05-17T21:53:09.629 INFO:tasks.workunit.client.1.smithi138.stdout:7/711: creat d0/d18/dbf/dd1/fe6 x:0 0 0 2024-05-17T21:53:09.629 INFO:tasks.workunit.client.0.smithi107.stdout:3/969: rmdir d6/d8/d10/d35/d91 39 2024-05-17T21:53:09.630 INFO:tasks.workunit.client.1.smithi138.stdout:1/700: creat d4/df/d3a/fe3 x:0 0 0 2024-05-17T21:53:09.631 INFO:tasks.workunit.client.0.smithi107.stdout:1/974: mknod d4/da7/daf/c138 0 2024-05-17T21:53:09.632 INFO:tasks.workunit.client.1.smithi138.stdout:2/763: symlink d3/d21/d2f/d3e/d55/d99/l100 0 2024-05-17T21:53:09.633 INFO:tasks.workunit.client.0.smithi107.stdout:4/979: mkdir d3/d4a/d47/d60/d3a/ddc/d150/d151 0 2024-05-17T21:53:09.636 INFO:tasks.workunit.client.1.smithi138.stdout:4/762: unlink d0/d1b/d2b/lff 0 2024-05-17T21:53:09.636 INFO:tasks.workunit.client.1.smithi138.stdout:2/764: dread d3/d21/d2f/d3e/d55/d99/d93/fca [0,4194304] 0 2024-05-17T21:53:09.637 INFO:tasks.workunit.client.1.smithi138.stdout:4/763: readlink d0/d3/d17/d5e/d7f/lc4 0 2024-05-17T21:53:09.639 INFO:tasks.workunit.client.1.smithi138.stdout:2/765: dread - d3/d21/d2f/d3e/d55/d99/d84/d9f/dfd/ff5 zero size 2024-05-17T21:53:09.640 INFO:tasks.workunit.client.1.smithi138.stdout:7/712: truncate d0/f12 7350319 0 2024-05-17T21:53:09.641 INFO:tasks.workunit.client.1.smithi138.stdout:1/701: truncate d4/db/d29/f5b 411172 0 2024-05-17T21:53:09.645 INFO:tasks.workunit.client.1.smithi138.stdout:9/656: write d6/d7/f2c [1052800,92817] 0 2024-05-17T21:53:09.646 INFO:tasks.workunit.client.1.smithi138.stdout:4/764: mkdir d0/d3/d1d/d108 0 2024-05-17T21:53:09.647 INFO:tasks.workunit.client.1.smithi138.stdout:4/765: chown d0/d3/db/da0/lb5 131358 1 2024-05-17T21:53:09.648 INFO:tasks.workunit.client.0.smithi107.stdout:3/970: write d6/d8/d10/d25/f49 [639897,16103] 0 2024-05-17T21:53:09.650 INFO:tasks.workunit.client.0.smithi107.stdout:2/762: rename d0/d9/d11/df/lae to d0/d26/lfa 0 2024-05-17T21:53:09.653 INFO:tasks.workunit.client.0.smithi107.stdout:1/975: dread d4/f8 [0,4194304] 0 2024-05-17T21:53:09.654 INFO:tasks.workunit.client.1.smithi138.stdout:9/657: dread d6/d7/f15 [0,4194304] 0 2024-05-17T21:53:09.654 INFO:tasks.workunit.client.1.smithi138.stdout:7/713: symlink d0/d9d/dd7/le7 0 2024-05-17T21:53:09.656 INFO:tasks.workunit.client.1.smithi138.stdout:2/766: dwrite d3/fe [4194304,4194304] 0 2024-05-17T21:53:09.656 INFO:tasks.workunit.client.0.smithi107.stdout:4/980: write d3/ddf/fe5 [243402,8048] 0 2024-05-17T21:53:09.657 INFO:tasks.workunit.client.1.smithi138.stdout:1/702: creat d4/dd/db4/fe4 x:0 0 0 2024-05-17T21:53:09.659 INFO:tasks.workunit.client.0.smithi107.stdout:2/763: dread d0/d9/d11/d18/f79 [0,4194304] 0 2024-05-17T21:53:09.662 INFO:tasks.workunit.client.1.smithi138.stdout:4/766: fdatasync d0/d3/d17/fa9 0 2024-05-17T21:53:09.665 INFO:tasks.workunit.client.0.smithi107.stdout:4/981: creat d3/db/d1e/dc6/f152 x:0 0 0 2024-05-17T21:53:09.666 INFO:tasks.workunit.client.0.smithi107.stdout:2/764: creat d0/d9/d11/d18/dad/db8/ffb x:0 0 0 2024-05-17T21:53:09.667 INFO:tasks.workunit.client.0.smithi107.stdout:1/976: dread d4/df/d1a/fbd [0,4194304] 0 2024-05-17T21:53:09.668 INFO:tasks.workunit.client.0.smithi107.stdout:3/971: creat d6/d8/d10/d35/d91/d139/f142 x:0 0 0 2024-05-17T21:53:09.670 INFO:tasks.workunit.client.1.smithi138.stdout:2/767: creat d3/d21/d2f/d3e/d55/d99/dd2/f101 x:0 0 0 2024-05-17T21:53:09.670 INFO:tasks.workunit.client.0.smithi107.stdout:2/765: creat d0/d9/d11/df/d5a/d6d/ffc x:0 0 0 2024-05-17T21:53:09.675 INFO:tasks.workunit.client.1.smithi138.stdout:4/767: unlink d0/d3/d17/d5e/d7f/l92 0 2024-05-17T21:53:09.678 INFO:tasks.workunit.client.1.smithi138.stdout:9/658: dread d6/d7/f99 [0,4194304] 0 2024-05-17T21:53:09.679 INFO:tasks.workunit.client.1.smithi138.stdout:9/659: dread - d6/d7/d16/d29/d38/d40/d44/f90 zero size 2024-05-17T21:53:09.685 INFO:tasks.workunit.client.0.smithi107.stdout:1/977: getdents d4/de 0 2024-05-17T21:53:09.685 INFO:tasks.workunit.client.1.smithi138.stdout:1/703: unlink d4/df/d3a/lbb 0 2024-05-17T21:53:09.686 INFO:tasks.workunit.client.1.smithi138.stdout:1/704: dread - d4/dd/d3b/d6c/dc1/fce zero size 2024-05-17T21:53:09.687 INFO:tasks.workunit.client.0.smithi107.stdout:3/972: dread d6/d8/d10/d35/d36/d4c/dcc/f102 [0,4194304] 0 2024-05-17T21:53:09.688 INFO:tasks.workunit.client.1.smithi138.stdout:9/660: creat d6/d7/d16/d29/d6c/d9c/fd0 x:0 0 0 2024-05-17T21:53:09.689 INFO:tasks.workunit.client.1.smithi138.stdout:9/661: chown d6/d7/d16/d29/d38/d40/d44/ca1 187515 1 2024-05-17T21:53:09.691 INFO:tasks.workunit.client.0.smithi107.stdout:4/982: dwrite d3/db/d35/f81 [4194304,4194304] 0 2024-05-17T21:53:09.691 INFO:tasks.workunit.client.1.smithi138.stdout:1/705: dread d4/db/d29/f5b [0,4194304] 0 2024-05-17T21:53:09.692 INFO:tasks.workunit.client.0.smithi107.stdout:1/978: truncate d4/df/d4e/d59/fe3 9076704 0 2024-05-17T21:53:09.693 INFO:tasks.workunit.client.1.smithi138.stdout:7/714: truncate d0/d39/f85 877294 0 2024-05-17T21:53:09.693 INFO:tasks.workunit.client.0.smithi107.stdout:2/766: dread d0/f7d [0,4194304] 0 2024-05-17T21:53:09.694 INFO:tasks.workunit.client.0.smithi107.stdout:3/973: creat d6/d8/db6/f143 x:0 0 0 2024-05-17T21:53:09.695 INFO:tasks.workunit.client.1.smithi138.stdout:7/715: read d0/d18/d64/d7c/f9f [600533,17099] 0 2024-05-17T21:53:09.698 INFO:tasks.workunit.client.1.smithi138.stdout:9/662: dwrite d6/d7/d16/d29/d38/d40/d44/fa4 [0,4194304] 0 2024-05-17T21:53:09.699 INFO:tasks.workunit.client.0.smithi107.stdout:4/983: unlink d3/db/f33 0 2024-05-17T21:53:09.701 INFO:tasks.workunit.client.0.smithi107.stdout:1/979: fdatasync d4/df/d1a/d26/d72/d85/fc0 0 2024-05-17T21:53:09.703 INFO:tasks.workunit.client.1.smithi138.stdout:2/768: dwrite d3/f4a [0,4194304] 0 2024-05-17T21:53:09.704 INFO:tasks.workunit.client.1.smithi138.stdout:4/768: write d0/d2f/d49/d60/f51 [1787827,65256] 0 2024-05-17T21:53:09.709 INFO:tasks.workunit.client.1.smithi138.stdout:7/716: fsync d0/d18/d40/d41/d50/f53 0 2024-05-17T21:53:09.711 INFO:tasks.workunit.client.0.smithi107.stdout:2/767: dwrite d0/d9/d11/d18/d9e/fe1 [0,4194304] 0 2024-05-17T21:53:09.712 INFO:tasks.workunit.client.1.smithi138.stdout:7/717: truncate d0/d18/d64/fca 559017 0 2024-05-17T21:53:09.713 INFO:tasks.workunit.client.0.smithi107.stdout:4/984: rename d3/d4a/d47 to d3/d6/d26/d36/d37/d153 0 2024-05-17T21:53:09.717 INFO:tasks.workunit.client.1.smithi138.stdout:9/663: dread d6/d7/d16/d29/fca [0,4194304] 0 2024-05-17T21:53:09.718 INFO:tasks.workunit.client.0.smithi107.stdout:1/980: mknod d4/df/d1a/d26/d72/d85/c139 0 2024-05-17T21:53:09.720 INFO:tasks.workunit.client.0.smithi107.stdout:8/893: sync 2024-05-17T21:53:09.720 INFO:tasks.workunit.client.0.smithi107.stdout:9/931: sync 2024-05-17T21:53:09.722 INFO:tasks.workunit.client.1.smithi138.stdout:1/706: dread d4/df/f1f [0,4194304] 0 2024-05-17T21:53:09.723 INFO:tasks.workunit.client.1.smithi138.stdout:1/707: stat d4/df/d3a/d3c/d63/d6d/fdc 0 2024-05-17T21:53:09.724 INFO:tasks.workunit.client.1.smithi138.stdout:1/708: readlink d4/db/l32 0 2024-05-17T21:53:09.725 INFO:tasks.workunit.client.1.smithi138.stdout:2/769: unlink d3/d21/d2f/d3e/d55/d73/ca6 0 2024-05-17T21:53:09.727 INFO:tasks.workunit.client.0.smithi107.stdout:1/981: dwrite d4/df/d1a/d26/d72/daa/f116 [4194304,4194304] 0 2024-05-17T21:53:09.727 INFO:tasks.workunit.client.1.smithi138.stdout:2/770: dread d3/d21/d2f/d3e/d55/d99/d84/d9f/fcb [0,4194304] 0 2024-05-17T21:53:09.727 INFO:tasks.workunit.client.0.smithi107.stdout:1/982: chown d4/c49 472301818 1 2024-05-17T21:53:09.728 INFO:tasks.workunit.client.0.smithi107.stdout:4/985: rename d3/db/d35/d5e/dff/cda to d3/d6/d26/d36/d37/d153/d60/d3c/d12d/c154 0 2024-05-17T21:53:09.730 INFO:tasks.workunit.client.1.smithi138.stdout:9/664: creat d6/d7/d16/d29/d48/d6e/d74/d75/d76/d8e/dba/fd1 x:0 0 0 2024-05-17T21:53:09.733 INFO:tasks.workunit.client.1.smithi138.stdout:2/771: mknod d3/d21/d2f/d3e/d55/d99/dd2/c102 0 2024-05-17T21:53:09.734 INFO:tasks.workunit.client.0.smithi107.stdout:1/983: rmdir d4/df/d14/d2e/ddf 39 2024-05-17T21:53:09.737 INFO:tasks.workunit.client.0.smithi107.stdout:4/986: rename d3/db/d35/d55/lb6 to d3/d6/d26/d36/d37/d153/da3/l155 0 2024-05-17T21:53:09.740 INFO:tasks.workunit.client.0.smithi107.stdout:9/932: getdents d0/dd/d21/d29/dd4/d11b 0 2024-05-17T21:53:09.741 INFO:tasks.workunit.client.0.smithi107.stdout:2/768: creat d0/d26/ffd x:0 0 0 2024-05-17T21:53:09.743 INFO:tasks.workunit.client.1.smithi138.stdout:7/718: getdents d0/d18/d40/d9a 0 2024-05-17T21:53:09.745 INFO:tasks.workunit.client.0.smithi107.stdout:8/894: creat dc/d21/d27/f13a x:0 0 0 2024-05-17T21:53:09.746 INFO:tasks.workunit.client.0.smithi107.stdout:3/974: dwrite d6/d2a/f78 [0,4194304] 0 2024-05-17T21:53:09.748 INFO:tasks.workunit.client.0.smithi107.stdout:4/987: unlink d3/db/d35/d5e/dff/d80/d43/f62 0 2024-05-17T21:53:09.748 INFO:tasks.workunit.client.1.smithi138.stdout:7/719: rmdir d0/d18/d40/d41/d51 39 2024-05-17T21:53:09.749 INFO:tasks.workunit.client.1.smithi138.stdout:4/769: write d0/d2f/d49/d60/d1f/d7e/f83 [3504697,14447] 0 2024-05-17T21:53:09.751 INFO:tasks.workunit.client.1.smithi138.stdout:4/770: dread - d0/d3/d17/d5e/d7f/ff0 zero size 2024-05-17T21:53:09.757 INFO:tasks.workunit.client.1.smithi138.stdout:7/720: rename d0/d18/d64/d7c/f9f to d0/d18/d64/d7c/fe8 0 2024-05-17T21:53:09.757 INFO:tasks.workunit.client.1.smithi138.stdout:9/665: dread d6/d7/d16/d29/f37 [0,4194304] 0 2024-05-17T21:53:09.759 INFO:tasks.workunit.client.1.smithi138.stdout:4/771: fsync d0/d3/d17/fa9 0 2024-05-17T21:53:09.761 INFO:tasks.workunit.client.1.smithi138.stdout:9/666: creat d6/d7/d16/d29/d6c/d9c/fd2 x:0 0 0 2024-05-17T21:53:09.762 INFO:tasks.workunit.client.1.smithi138.stdout:9/667: stat d6/f33 0 2024-05-17T21:53:09.763 INFO:tasks.workunit.client.1.smithi138.stdout:7/721: symlink d0/d18/d64/d7c/da2/dc5/le9 0 2024-05-17T21:53:09.764 INFO:tasks.workunit.client.1.smithi138.stdout:7/722: readlink d0/d18/d64/d7c/la4 0 2024-05-17T21:53:09.766 INFO:tasks.workunit.client.1.smithi138.stdout:2/772: dwrite d3/da0/daa/fee [0,4194304] 0 2024-05-17T21:53:09.767 INFO:tasks.workunit.client.1.smithi138.stdout:9/668: fdatasync d6/d7/d16/f6d 0 2024-05-17T21:53:09.768 INFO:tasks.workunit.client.1.smithi138.stdout:2/773: readlink d3/d21/d2f/d3e/d55/l89 0 2024-05-17T21:53:09.768 INFO:tasks.workunit.client.0.smithi107.stdout:1/984: link d4/df/d4e/l5d d4/df/dc1/l13a 0 2024-05-17T21:53:09.769 INFO:tasks.workunit.client.0.smithi107.stdout:3/975: dread d6/d8/d10/f23 [0,4194304] 0 2024-05-17T21:53:09.772 INFO:tasks.workunit.client.1.smithi138.stdout:2/774: rename d3/d21/d2f/d3e to d3/d21/d2f/d3e/d58/d77/da4/dff/d103 22 2024-05-17T21:53:09.773 INFO:tasks.workunit.client.1.smithi138.stdout:2/775: stat d3/d21/d2f/d3e/d55/d99/d93/ff9 0 2024-05-17T21:53:09.773 INFO:tasks.workunit.client.0.smithi107.stdout:9/933: write d0/dd/d2f/d87/d89/fae [1588052,41776] 0 2024-05-17T21:53:09.775 INFO:tasks.workunit.client.1.smithi138.stdout:9/669: creat d6/d7/d16/d29/d38/d3c/fd3 x:0 0 0 2024-05-17T21:53:09.778 INFO:tasks.workunit.client.1.smithi138.stdout:7/723: symlink d0/d18/d40/d41/d51/lea 0 2024-05-17T21:53:09.779 INFO:tasks.workunit.client.1.smithi138.stdout:7/724: read d0/d18/d40/f58 [789913,47999] 0 2024-05-17T21:53:09.780 INFO:tasks.workunit.client.1.smithi138.stdout:2/776: rename d3/d21/d2f/d3e/d63/da3/fb6 to d3/d21/d2f/dd4/f104 0 2024-05-17T21:53:09.784 INFO:tasks.workunit.client.1.smithi138.stdout:7/725: symlink d0/d4c/leb 0 2024-05-17T21:53:09.787 INFO:tasks.workunit.client.1.smithi138.stdout:4/772: dread d0/d3/f12 [0,4194304] 0 2024-05-17T21:53:09.789 INFO:tasks.workunit.client.0.smithi107.stdout:4/988: dwrite d3/d6/d26/d36/d37/d153/d60/d3c/d9d/fe7 [0,4194304] 0 2024-05-17T21:53:09.789 INFO:tasks.workunit.client.1.smithi138.stdout:7/726: truncate d0/d18/d40/d5f/f96 644205 0 2024-05-17T21:53:09.790 INFO:tasks.workunit.client.0.smithi107.stdout:4/989: stat d3/db/d1e/db5 0 2024-05-17T21:53:09.795 INFO:tasks.workunit.client.0.smithi107.stdout:7/882: sync 2024-05-17T21:53:09.795 INFO:tasks.workunit.client.0.smithi107.stdout:5/928: sync 2024-05-17T21:53:09.795 INFO:tasks.workunit.client.0.smithi107.stdout:6/864: sync 2024-05-17T21:53:09.795 INFO:tasks.workunit.client.1.smithi138.stdout:7/727: mkdir d0/d4c/dec 0 2024-05-17T21:53:09.796 INFO:tasks.workunit.client.1.smithi138.stdout:7/728: dread - d0/d18/d40/d9a/fbb zero size 2024-05-17T21:53:09.803 INFO:tasks.workunit.client.1.smithi138.stdout:2/777: getdents d3/d21/d2f/d3e/d55/d99 0 2024-05-17T21:53:09.805 INFO:tasks.workunit.client.1.smithi138.stdout:9/670: write d6/d7/d16/d29/d48/f98 [3500853,77516] 0 2024-05-17T21:53:09.807 INFO:tasks.workunit.client.1.smithi138.stdout:7/729: read d0/d18/d64/d7c/fe8 [3225847,108889] 0 2024-05-17T21:53:09.816 INFO:tasks.workunit.client.0.smithi107.stdout:8/895: link dc/d21/d35/fda dc/d21/d68/f13b 0 2024-05-17T21:53:09.821 INFO:tasks.workunit.client.0.smithi107.stdout:3/976: unlink d6/d8/d10/d35/d36/d4c/f97 0 2024-05-17T21:53:09.825 INFO:tasks.workunit.client.0.smithi107.stdout:4/990: creat d3/d6/d26/d36/d8b/d95/dee/f156 x:0 0 0 2024-05-17T21:53:09.827 INFO:tasks.workunit.client.0.smithi107.stdout:7/883: stat d5/d16/d3b/ddf/d12e/db3/lca 0 2024-05-17T21:53:09.829 INFO:tasks.workunit.client.0.smithi107.stdout:5/929: creat d5/db/d16/d2d/d40/d5a/d7c/d8e/f135 x:0 0 0 2024-05-17T21:53:09.834 INFO:tasks.workunit.client.0.smithi107.stdout:1/985: unlink d4/df/f4d 0 2024-05-17T21:53:09.836 INFO:tasks.workunit.client.0.smithi107.stdout:3/977: rename d6/d8/d10/c20 to d6/d8/d10/d35/d36/d4c/c144 0 2024-05-17T21:53:09.838 INFO:tasks.workunit.client.0.smithi107.stdout:2/769: dwrite d0/d9/d11/f5c [0,4194304] 0 2024-05-17T21:53:09.840 INFO:tasks.workunit.client.0.smithi107.stdout:6/865: symlink d9/d10e/d7b/d114/l11c 0 2024-05-17T21:53:09.841 INFO:tasks.workunit.client.0.smithi107.stdout:4/991: symlink d3/db/d1e/d99/dd6/l157 0 2024-05-17T21:53:09.841 INFO:tasks.workunit.client.0.smithi107.stdout:4/992: chown d3/d4a/c67 23937 1 2024-05-17T21:53:09.849 INFO:tasks.workunit.client.1.smithi138.stdout:2/778: write d3/d21/d2f/dd4/f104 [4471732,105591] 0 2024-05-17T21:53:09.852 INFO:tasks.workunit.client.0.smithi107.stdout:2/770: symlink d0/d9/d11/d15/lfe 0 2024-05-17T21:53:09.853 INFO:tasks.workunit.client.1.smithi138.stdout:7/730: creat d0/d18/d40/d5f/dbe/fed x:0 0 0 2024-05-17T21:53:09.855 INFO:tasks.workunit.client.1.smithi138.stdout:7/731: read d0/d18/d40/d41/f5b [4188607,28377] 0 2024-05-17T21:53:09.856 INFO:tasks.workunit.client.1.smithi138.stdout:2/779: rename d3/da0/daa/fee to d3/d21/d2f/dd4/f105 0 2024-05-17T21:53:09.857 INFO:tasks.workunit.client.0.smithi107.stdout:8/896: dwrite dc/d21/d27/d2f/f2c [0,4194304] 0 2024-05-17T21:53:09.858 INFO:tasks.workunit.client.0.smithi107.stdout:9/934: write d0/dd/d21/d29/dd4/fc5 [2367324,32973] 0 2024-05-17T21:53:09.859 INFO:tasks.workunit.client.0.smithi107.stdout:8/897: chown dc/d21/d27/d2f/dba/d104 22 1 2024-05-17T21:53:09.859 INFO:tasks.workunit.client.0.smithi107.stdout:4/993: write d3/d6/f5c [2560241,116254] 0 2024-05-17T21:53:09.860 INFO:tasks.workunit.client.0.smithi107.stdout:8/898: stat dc/d44/d4b/d52/dcb/d12b 0 2024-05-17T21:53:09.863 INFO:tasks.workunit.client.0.smithi107.stdout:1/986: creat d4/de/d12b/f13b x:0 0 0 2024-05-17T21:53:09.864 INFO:tasks.workunit.client.0.smithi107.stdout:1/987: stat d4/df/d14/d2e/dd3/c134 0 2024-05-17T21:53:09.864 INFO:tasks.workunit.client.1.smithi138.stdout:2/780: creat d3/d81/d96/da8/f106 x:0 0 0 2024-05-17T21:53:09.865 INFO:tasks.workunit.client.0.smithi107.stdout:7/884: write d5/df/d11/d1d/f96 [256229,79972] 0 2024-05-17T21:53:09.876 INFO:tasks.workunit.client.1.smithi138.stdout:7/732: write d0/f20 [9402117,26887] 0 2024-05-17T21:53:09.879 INFO:tasks.workunit.client.1.smithi138.stdout:2/781: write d3/f9b [4423602,56607] 0 2024-05-17T21:53:09.881 INFO:tasks.workunit.client.0.smithi107.stdout:9/935: fdatasync d0/dd/f78 0 2024-05-17T21:53:09.885 INFO:tasks.workunit.client.0.smithi107.stdout:8/899: rename dc/d21/db8/ld1 to dc/d44/d4b/d52/dcb/d11c/l13c 0 2024-05-17T21:53:09.886 INFO:tasks.workunit.client.1.smithi138.stdout:2/782: write d3/d21/f38 [2202934,75053] 0 2024-05-17T21:53:09.886 INFO:tasks.workunit.client.0.smithi107.stdout:2/771: write d0/d9/d11/f5f [189376,55649] 0 2024-05-17T21:53:09.887 INFO:tasks.workunit.client.0.smithi107.stdout:3/978: dwrite d6/d2a/d2d/f43 [0,4194304] 0 2024-05-17T21:53:09.887 INFO:tasks.workunit.client.0.smithi107.stdout:6/866: dread d9/d16/d22/f28 [0,4194304] 0 2024-05-17T21:53:09.887 INFO:tasks.workunit.client.1.smithi138.stdout:2/783: chown d3/d21/d2f/d3e/d58/f71 97721 1 2024-05-17T21:53:09.888 INFO:tasks.workunit.client.0.smithi107.stdout:4/994: creat d3/d6/d26/d3b/f158 x:0 0 0 2024-05-17T21:53:09.892 INFO:tasks.workunit.client.0.smithi107.stdout:2/772: write d0/f6f [494541,12615] 0 2024-05-17T21:53:09.903 INFO:tasks.workunit.client.1.smithi138.stdout:7/733: dread d0/d18/d64/f6e [0,4194304] 0 2024-05-17T21:53:09.924 INFO:tasks.workunit.client.1.smithi138.stdout:3/912: sync 2024-05-17T21:53:09.924 INFO:tasks.workunit.client.1.smithi138.stdout:6/757: sync 2024-05-17T21:53:09.924 INFO:tasks.workunit.client.1.smithi138.stdout:5/719: sync 2024-05-17T21:53:09.926 INFO:tasks.workunit.client.1.smithi138.stdout:5/720: chown d1/d53/d88/db9/d39/d4e/fb4 244 1 2024-05-17T21:53:09.932 INFO:tasks.workunit.client.0.smithi107.stdout:7/885: dread d5/df/d11/d1c/d31/d39/d92/d9a/fdc [0,4194304] 0 2024-05-17T21:53:09.933 INFO:tasks.workunit.client.0.smithi107.stdout:3/979: mknod d6/d8/d3e/d3f/d90/da7/ddc/c145 0 2024-05-17T21:53:09.935 INFO:tasks.workunit.client.0.smithi107.stdout:4/995: symlink d3/db/d35/d4f/l159 0 2024-05-17T21:53:09.935 INFO:tasks.workunit.client.0.smithi107.stdout:3/980: dread - d6/d2a/d2d/d37/d108/f11b zero size 2024-05-17T21:53:09.935 INFO:tasks.workunit.client.1.smithi138.stdout:2/784: dwrite d3/d21/d2f/f5f [0,4194304] 0 2024-05-17T21:53:09.935 INFO:tasks.workunit.client.1.smithi138.stdout:6/758: read d7/d69/db5/dc8/d43/d3f/db6/fc5 [584373,63613] 0 2024-05-17T21:53:09.937 INFO:tasks.workunit.client.1.smithi138.stdout:7/734: write d0/f4b [4784092,125413] 0 2024-05-17T21:53:09.938 INFO:tasks.workunit.client.1.smithi138.stdout:6/759: fsync d7/d69/db5/dc8/d43/d5f/da4/fee 0 2024-05-17T21:53:09.940 INFO:tasks.workunit.client.1.smithi138.stdout:6/760: chown d7/d69/db5/dc8/d43/d5f/d6a/d9b/db2/fb9 158836103 1 2024-05-17T21:53:09.941 INFO:tasks.workunit.client.0.smithi107.stdout:6/867: truncate d9/d16/f41 101322 0 2024-05-17T21:53:09.942 INFO:tasks.workunit.client.0.smithi107.stdout:2/773: mknod d0/d9/d1c/dd4/cff 0 2024-05-17T21:53:09.943 INFO:tasks.workunit.client.0.smithi107.stdout:2/774: write d0/d9/d11/df/d5a/d6d/f74 [1006267,81224] 0 2024-05-17T21:53:09.944 INFO:tasks.workunit.client.1.smithi138.stdout:3/913: dread d1/d3/d16/d3e/f4a [0,4194304] 0 2024-05-17T21:53:09.945 INFO:tasks.workunit.client.1.smithi138.stdout:3/914: chown d1/d4f/d80/lca 24155507 1 2024-05-17T21:53:09.946 INFO:tasks.workunit.client.1.smithi138.stdout:3/915: stat d1/d3/d16/d3e/f73 0 2024-05-17T21:53:09.947 INFO:tasks.workunit.client.1.smithi138.stdout:3/916: chown d1/d3/d16/d2d/fd1 1 1 2024-05-17T21:53:09.948 INFO:tasks.workunit.client.0.smithi107.stdout:7/886: dread - d5/df/d11/d1c/d31/d39/d92/d9a/ffd zero size 2024-05-17T21:53:09.948 INFO:tasks.workunit.client.1.smithi138.stdout:3/917: fsync d1/d2e/f98 0 2024-05-17T21:53:09.950 INFO:tasks.workunit.client.0.smithi107.stdout:7/887: chown d5/df/d11/d1d/d33/f87 47288 1 2024-05-17T21:53:09.950 INFO:tasks.workunit.client.0.smithi107.stdout:4/996: creat d3/db/d35/d4f/d93/f15a x:0 0 0 2024-05-17T21:53:09.953 INFO:tasks.workunit.client.0.smithi107.stdout:4/997: stat d3/d6/d26/d36/d7e/l13d 0 2024-05-17T21:53:09.955 INFO:tasks.workunit.client.0.smithi107.stdout:7/888: dread d5/f22 [0,4194304] 0 2024-05-17T21:53:09.956 INFO:tasks.workunit.client.1.smithi138.stdout:6/761: mknod d7/d69/c105 0 2024-05-17T21:53:09.958 INFO:tasks.workunit.client.0.smithi107.stdout:8/900: rmdir dc/d21/d97/d11d 0 2024-05-17T21:53:09.958 INFO:tasks.workunit.client.1.smithi138.stdout:5/721: write d1/d53/d88/db9/d39/d4e/d58/d5b/f9e [301870,9603] 0 2024-05-17T21:53:09.959 INFO:tasks.workunit.client.0.smithi107.stdout:7/889: read f4 [1614454,95085] 0 2024-05-17T21:53:09.959 INFO:tasks.workunit.client.0.smithi107.stdout:8/901: chown dc/d44/d4b/d52/f96 0 1 2024-05-17T21:53:09.960 INFO:tasks.workunit.client.1.smithi138.stdout:5/722: write d1/f13 [1093095,91082] 0 2024-05-17T21:53:09.960 INFO:tasks.workunit.client.1.smithi138.stdout:3/918: creat d1/d3/d16/d2d/df6/f135 x:0 0 0 2024-05-17T21:53:09.961 INFO:tasks.workunit.client.0.smithi107.stdout:8/902: stat dc/d21/d27/d34/c53 0 2024-05-17T21:53:09.961 INFO:tasks.workunit.client.0.smithi107.stdout:2/775: dread - d0/d26/ddd/f95 zero size 2024-05-17T21:53:09.963 INFO:tasks.workunit.client.0.smithi107.stdout:5/930: sync 2024-05-17T21:53:09.963 INFO:tasks.workunit.client.1.smithi138.stdout:2/785: write d3/d21/d2f/d3e/d58/d77/da4/dc5/fbc [488812,27501] 0 2024-05-17T21:53:09.963 INFO:tasks.workunit.client.0.smithi107.stdout:8/903: chown dc/d21/d27/d34/cc7 25940 1 2024-05-17T21:53:09.965 INFO:tasks.workunit.client.0.smithi107.stdout:9/936: link d0/dd/d2f/d87/cdf d0/dd/d21/d29/d57/d5f/d112/d128/c132 0 2024-05-17T21:53:09.965 INFO:tasks.workunit.client.0.smithi107.stdout:3/981: write d6/d2a/d2d/da8/fb8 [411326,112232] 0 2024-05-17T21:53:09.967 INFO:tasks.workunit.client.0.smithi107.stdout:6/868: write d9/d10e/d66/de2/f109 [1905683,104395] 0 2024-05-17T21:53:09.968 INFO:tasks.workunit.client.1.smithi138.stdout:6/762: symlink d7/l106 0 2024-05-17T21:53:09.969 INFO:tasks.workunit.client.0.smithi107.stdout:9/937: chown d0/dd/d21/d29/dca/ff8 440 1 2024-05-17T21:53:09.969 INFO:tasks.workunit.client.1.smithi138.stdout:6/763: write d7/f22 [2012779,11928] 0 2024-05-17T21:53:09.970 INFO:tasks.workunit.client.1.smithi138.stdout:2/786: creat d3/d21/d2f/f107 x:0 0 0 2024-05-17T21:53:09.972 INFO:tasks.workunit.client.0.smithi107.stdout:2/776: symlink d0/d9/d11/df/l100 0 2024-05-17T21:53:09.973 INFO:tasks.workunit.client.1.smithi138.stdout:5/723: dread d1/d2c/d63/db5/d19/f3a [0,4194304] 0 2024-05-17T21:53:09.974 INFO:tasks.workunit.client.1.smithi138.stdout:3/919: creat d1/d62/f136 x:0 0 0 2024-05-17T21:53:09.976 INFO:tasks.workunit.client.1.smithi138.stdout:3/920: stat d1/d4f/d80/de5/f130 0 2024-05-17T21:53:09.976 INFO:tasks.workunit.client.1.smithi138.stdout:6/764: dwrite d7/d69/db5/dc8/d43/d5f/da4/fee [0,4194304] 0 2024-05-17T21:53:09.977 INFO:tasks.workunit.client.0.smithi107.stdout:4/998: dread d3/d6/d26/d36/f116 [0,4194304] 0 2024-05-17T21:53:09.978 INFO:tasks.workunit.client.1.smithi138.stdout:5/724: mknod d1/d53/d88/db9/cea 0 2024-05-17T21:53:09.978 INFO:tasks.workunit.client.1.smithi138.stdout:3/921: dread d1/d3/d16/fa2 [4194304,4194304] 0 2024-05-17T21:53:09.980 INFO:tasks.workunit.client.0.smithi107.stdout:8/904: dread dc/d21/d69/f77 [4194304,4194304] 0 2024-05-17T21:53:09.982 INFO:tasks.workunit.client.0.smithi107.stdout:7/890: dread f4 [0,4194304] 0 2024-05-17T21:53:09.984 INFO:tasks.workunit.client.0.smithi107.stdout:3/982: rename f3 to d6/d8/d3e/dc9/d11f/d125/f146 0 2024-05-17T21:53:09.988 INFO:tasks.workunit.client.0.smithi107.stdout:5/931: truncate d5/db/d16/d2d/d40/d5a/d7c/fb7 584858 0 2024-05-17T21:53:09.990 INFO:tasks.workunit.client.0.smithi107.stdout:2/777: mkdir d0/d9/d11/df/d5a/d6d/d101 0 2024-05-17T21:53:09.991 INFO:tasks.workunit.client.0.smithi107.stdout:2/778: readlink d0/d26/ddd/ld2 0 2024-05-17T21:53:09.994 INFO:tasks.workunit.client.1.smithi138.stdout:3/922: dread d1/d62/dad/de/fb5 [0,4194304] 0 2024-05-17T21:53:09.996 INFO:tasks.workunit.client.0.smithi107.stdout:6/869: unlink d9/d16/d4d/f60 0 2024-05-17T21:53:09.998 INFO:tasks.workunit.client.0.smithi107.stdout:6/870: chown d9/d16/d4d/db8/dc9/dd0/f54 1476643 1 2024-05-17T21:53:09.998 INFO:tasks.workunit.client.0.smithi107.stdout:8/905: truncate dc/f43 395208 0 2024-05-17T21:53:09.998 INFO:tasks.workunit.client.1.smithi138.stdout:2/787: getdents d3/d21/d2f/d3e/d55/d99/d84/dbe 0 2024-05-17T21:53:10.000 INFO:tasks.workunit.client.1.smithi138.stdout:3/923: write d1/d4f/d80/de5/f130 [1823739,106750] 0 2024-05-17T21:53:10.002 INFO:tasks.workunit.client.0.smithi107.stdout:7/891: symlink d5/df/d11/d1c/d31/d39/d92/l134 0 2024-05-17T21:53:10.004 INFO:tasks.workunit.client.1.smithi138.stdout:2/788: creat d3/d21/d2f/d3e/d58/f108 x:0 0 0 2024-05-17T21:53:10.005 INFO:tasks.workunit.client.0.smithi107.stdout:1/988: sync 2024-05-17T21:53:10.005 INFO:tasks.workunit.client.0.smithi107.stdout:9/938: dread d0/f7 [0,4194304] 0 2024-05-17T21:53:10.005 INFO:tasks.workunit.client.0.smithi107.stdout:1/989: chown d4/df/fa8 109682 1 2024-05-17T21:53:10.006 INFO:tasks.workunit.client.0.smithi107.stdout:2/779: unlink d0/d9/d1c/dd4/cff 0 2024-05-17T21:53:10.007 INFO:tasks.workunit.client.1.smithi138.stdout:2/789: chown d3/d21/d2f/d3e/d55/d73/l75 0 1 2024-05-17T21:53:10.008 INFO:tasks.workunit.client.1.smithi138.stdout:3/924: dwrite d1/d4f/d87/ff3 [0,4194304] 0 2024-05-17T21:53:10.008 INFO:tasks.workunit.client.1.smithi138.stdout:2/790: chown d3/l2a 82627 1 2024-05-17T21:53:10.011 INFO:tasks.workunit.client.0.smithi107.stdout:5/932: dread d5/db/d16/d54/d59/f7f [0,4194304] 0 2024-05-17T21:53:10.013 INFO:tasks.workunit.client.0.smithi107.stdout:6/871: creat d9/d10e/d66/dd5/df3/f11d x:0 0 0 2024-05-17T21:53:10.015 INFO:tasks.workunit.client.1.smithi138.stdout:2/791: symlink d3/d21/d2f/dd4/l109 0 2024-05-17T21:53:10.017 INFO:tasks.workunit.client.1.smithi138.stdout:3/925: truncate d1/d3/d16/d5d/fb4 1028763 0 2024-05-17T21:53:10.017 INFO:tasks.workunit.client.0.smithi107.stdout:7/892: unlink d5/f18 0 2024-05-17T21:53:10.018 INFO:tasks.workunit.client.1.smithi138.stdout:3/926: fsync d1/d3/d49/ddb/f6e 0 2024-05-17T21:53:10.018 INFO:tasks.workunit.client.1.smithi138.stdout:3/927: write d1/d4f/d87/f88 [4837373,42546] 0 2024-05-17T21:53:10.024 INFO:tasks.workunit.client.1.smithi138.stdout:6/765: dwrite d7/d69/db5/dc8/f52 [4194304,4194304] 0 2024-05-17T21:53:10.029 INFO:tasks.workunit.client.0.smithi107.stdout:2/780: rename d0/d9/d11/d27/c80 to d0/d9/d11/d18/d9e/deb/c102 0 2024-05-17T21:53:10.033 INFO:tasks.workunit.client.0.smithi107.stdout:4/999: creat d3/d6/d26/f15b x:0 0 0 2024-05-17T21:53:10.033 INFO:tasks.workunit.client.0.smithi107.stdout:5/933: truncate d5/db/f25 3502291 0 2024-05-17T21:53:10.034 INFO:tasks.workunit.client.0.smithi107.stdout:8/906: write dc/d21/d97/db7/ff9 [277796,92550] 0 2024-05-17T21:53:10.035 INFO:tasks.workunit.client.1.smithi138.stdout:3/928: mknod d1/d3/d16/d3e/d71/c137 0 2024-05-17T21:53:10.035 INFO:tasks.workunit.client.1.smithi138.stdout:3/929: stat d1/d3/d49/ddb/l103 0 2024-05-17T21:53:10.036 INFO:tasks.workunit.client.1.smithi138.stdout:2/792: symlink d3/da0/df6/l10a 0 2024-05-17T21:53:10.037 INFO:tasks.workunit.client.0.smithi107.stdout:2/781: dwrite d0/d9/d11/df/d5a/d6d/dc8/de8/f8a [0,4194304] 0 2024-05-17T21:53:10.038 INFO:tasks.workunit.client.1.smithi138.stdout:2/793: readlink d3/d81/d96/da8/lcf 0 2024-05-17T21:53:10.038 INFO:tasks.workunit.client.0.smithi107.stdout:2/782: dread - d0/d26/ffd zero size 2024-05-17T21:53:10.039 INFO:tasks.workunit.client.1.smithi138.stdout:6/766: read - d7/d69/db5/dc8/f9e zero size 2024-05-17T21:53:10.041 INFO:tasks.workunit.client.1.smithi138.stdout:6/767: stat d7/d69/db5/dc8/d34/cf8 0 2024-05-17T21:53:10.044 INFO:tasks.workunit.client.0.smithi107.stdout:9/939: dwrite d0/dd/d1f/f82 [0,4194304] 0 2024-05-17T21:53:10.045 INFO:tasks.workunit.client.1.smithi138.stdout:5/725: dwrite d1/d53/d88/db9/f26 [0,4194304] 0 2024-05-17T21:53:10.050 INFO:tasks.workunit.client.0.smithi107.stdout:3/983: link d6/d2a/f5b d6/d8/d3e/d3f/f147 0 2024-05-17T21:53:10.051 INFO:tasks.workunit.client.0.smithi107.stdout:3/984: chown d6/d8/d3e/d3f/d90/ff4 3094 1 2024-05-17T21:53:10.053 INFO:tasks.workunit.client.0.smithi107.stdout:3/985: chown d6/d8/d10/d35/d91/de4/d122 0 1 2024-05-17T21:53:10.053 INFO:tasks.workunit.client.0.smithi107.stdout:3/986: stat d6/d1a/d57/lc2 0 2024-05-17T21:53:10.053 INFO:tasks.workunit.client.1.smithi138.stdout:6/768: dwrite d7/d8/f5d [0,4194304] 0 2024-05-17T21:53:10.058 INFO:tasks.workunit.client.0.smithi107.stdout:1/990: fdatasync d4/df/d1a/d41/db2/dc8/ff2 0 2024-05-17T21:53:10.058 INFO:tasks.workunit.client.1.smithi138.stdout:5/726: unlink d1/d46/f9c 0 2024-05-17T21:53:10.059 INFO:tasks.workunit.client.0.smithi107.stdout:3/987: dread d6/d8/d10/d35/d36/f8e [0,4194304] 0 2024-05-17T21:53:10.060 INFO:tasks.workunit.client.1.smithi138.stdout:6/769: unlink d7/d8/f24 0 2024-05-17T21:53:10.061 INFO:tasks.workunit.client.0.smithi107.stdout:5/934: mkdir d5/db/d16/d54/da0/d136 0 2024-05-17T21:53:10.062 INFO:tasks.workunit.client.1.smithi138.stdout:5/727: mknod d1/dd1/dca/ceb 0 2024-05-17T21:53:10.064 INFO:tasks.workunit.client.0.smithi107.stdout:8/907: rename dc/d21/d35/d134 to dc/ddb/df2/d13d 0 2024-05-17T21:53:10.066 INFO:tasks.workunit.client.1.smithi138.stdout:5/728: read d1/d2c/d63/db5/d1b/dc5/f67 [6678258,63369] 0 2024-05-17T21:53:10.068 INFO:tasks.workunit.client.0.smithi107.stdout:6/872: link d9/d16/d4d/d85/d96/ffb d9/d16/d22/d5f/f11e 0 2024-05-17T21:53:10.069 INFO:tasks.workunit.client.1.smithi138.stdout:5/729: write d1/d53/d96/f9b [76040,75521] 0 2024-05-17T21:53:10.073 INFO:tasks.workunit.client.0.smithi107.stdout:3/988: symlink d6/d8/d3e/l148 0 2024-05-17T21:53:10.076 INFO:tasks.workunit.client.1.smithi138.stdout:5/730: mknod d1/d60/d2f/d5e/de4/d90/cec 0 2024-05-17T21:53:10.078 INFO:tasks.workunit.client.0.smithi107.stdout:2/783: mkdir d0/d9/d11/d18/d103 0 2024-05-17T21:53:10.079 INFO:tasks.workunit.client.0.smithi107.stdout:7/893: fdatasync d5/f70 0 2024-05-17T21:53:10.081 INFO:tasks.workunit.client.0.smithi107.stdout:1/991: creat d4/df/d1a/d26/d72/d85/df9/f13c x:0 0 0 2024-05-17T21:53:10.082 INFO:tasks.workunit.client.0.smithi107.stdout:3/989: rmdir d6/d8/d3e/dc9/d11f/d125 39 2024-05-17T21:53:10.085 INFO:tasks.workunit.client.1.smithi138.stdout:5/731: getdents d1/d2c/d63/db5/d1b/db7 0 2024-05-17T21:53:10.085 INFO:tasks.workunit.client.1.smithi138.stdout:5/732: chown d1/dd1/dca/ceb 307456566 1 2024-05-17T21:53:10.091 INFO:tasks.workunit.client.0.smithi107.stdout:7/894: dread - d5/d16/d3b/f115 zero size 2024-05-17T21:53:10.093 INFO:tasks.workunit.client.1.smithi138.stdout:5/733: rename d1/d53/d88/db9/d39/d69 to d1/d53/d88/de5/ded 0 2024-05-17T21:53:10.095 INFO:tasks.workunit.client.0.smithi107.stdout:1/992: unlink d4/df/d14/d2e/d43/d47/lb0 0 2024-05-17T21:53:10.095 INFO:tasks.workunit.client.0.smithi107.stdout:1/993: chown d4/f21 1 1 2024-05-17T21:53:10.097 INFO:tasks.workunit.client.1.smithi138.stdout:5/734: dread - d1/d60/faf zero size 2024-05-17T21:53:10.100 INFO:tasks.workunit.client.1.smithi138.stdout:0/680: sync 2024-05-17T21:53:10.100 INFO:tasks.workunit.client.1.smithi138.stdout:8/654: sync 2024-05-17T21:53:10.100 INFO:tasks.workunit.client.0.smithi107.stdout:5/935: rename d5/db/d16/d5b/f3f to d5/d75/dc7/f137 0 2024-05-17T21:53:10.101 INFO:tasks.workunit.client.1.smithi138.stdout:8/655: write d0/d63/fbb [52723,61880] 0 2024-05-17T21:53:10.103 INFO:tasks.workunit.client.1.smithi138.stdout:5/735: rename d1/d53/db1/db8 to d1/d53/d88/db9/d39/d4e/d58/d5b/dee 0 2024-05-17T21:53:10.105 INFO:tasks.workunit.client.1.smithi138.stdout:0/681: symlink d3/d10/d39/le5 0 2024-05-17T21:53:10.106 INFO:tasks.workunit.client.0.smithi107.stdout:1/994: dwrite d4/df/d1a/d26/d72/daa/f116 [0,4194304] 0 2024-05-17T21:53:10.109 INFO:tasks.workunit.client.0.smithi107.stdout:5/936: dwrite d5/db/d16/d2d/d40/d9f/f10c [0,4194304] 0 2024-05-17T21:53:10.114 INFO:tasks.workunit.client.0.smithi107.stdout:5/937: read d5/db/d16/d2d/d40/f130 [2673746,28799] 0 2024-05-17T21:53:10.207 INFO:tasks.workunit.client.0.smithi107.stdout:7/895: sync 2024-05-17T21:53:10.219 INFO:tasks.workunit.client.0.smithi107.stdout:3/990: mknod d6/d2a/d2d/da8/c149 0 2024-05-17T21:53:10.220 INFO:tasks.workunit.client.1.smithi138.stdout:6/770: dwrite d7/da0/fbf [0,4194304] 0 2024-05-17T21:53:10.222 INFO:tasks.workunit.client.1.smithi138.stdout:6/771: write d7/d69/db5/dc8/f82 [1310622,89015] 0 2024-05-17T21:53:10.222 INFO:tasks.workunit.client.0.smithi107.stdout:9/940: write d0/dd/d2f/f64 [930185,19891] 0 2024-05-17T21:53:10.224 INFO:tasks.workunit.client.1.smithi138.stdout:0/682: unlink d3/d10/d38/l52 0 2024-05-17T21:53:10.227 INFO:tasks.workunit.client.0.smithi107.stdout:9/941: read d0/dd/d21/d29/d57/f8a [3357420,121336] 0 2024-05-17T21:53:10.231 INFO:tasks.workunit.client.0.smithi107.stdout:1/995: truncate d4/df/d1a/d26/f99 219164 0 2024-05-17T21:53:10.233 INFO:tasks.workunit.client.0.smithi107.stdout:2/784: write d0/d9/d1c/d22/fb1 [753186,80824] 0 2024-05-17T21:53:10.238 INFO:tasks.workunit.client.0.smithi107.stdout:6/873: truncate d9/d16/d4d/db8/fbb 1944641 0 2024-05-17T21:53:10.239 INFO:tasks.workunit.client.0.smithi107.stdout:8/908: dwrite dc/d21/daa/dc5/ff0 [0,4194304] 0 2024-05-17T21:53:10.241 INFO:tasks.workunit.client.0.smithi107.stdout:6/874: chown d9/d16/f4a 36 1 2024-05-17T21:53:10.244 INFO:tasks.workunit.client.0.smithi107.stdout:1/996: dwrite d4/df/d1a/d26/f126 [0,4194304] 0 2024-05-17T21:53:10.244 INFO:tasks.workunit.client.1.smithi138.stdout:6/772: mkdir d7/d69/db5/dc8/d43/d5f/d6a/d77/d107 0 2024-05-17T21:53:10.246 INFO:tasks.workunit.client.0.smithi107.stdout:8/909: chown dc/d21/d97/l109 18501 1 2024-05-17T21:53:10.249 INFO:tasks.workunit.client.1.smithi138.stdout:0/683: rename d3/fa1 to d3/d10/d6d/d9f/fe6 0 2024-05-17T21:53:10.250 INFO:tasks.workunit.client.1.smithi138.stdout:0/684: write d3/d3d/d51/f6e [4086933,121787] 0 2024-05-17T21:53:10.251 INFO:tasks.workunit.client.1.smithi138.stdout:5/736: link d1/d60/d2f/c56 d1/d2c/d63/db5/d1b/db7/cef 0 2024-05-17T21:53:10.252 INFO:tasks.workunit.client.1.smithi138.stdout:5/737: chown d1/d60/d2f/d5e/de4/fbc 7733 1 2024-05-17T21:53:10.253 INFO:tasks.workunit.client.1.smithi138.stdout:6/773: fsync d7/d69/db5/dc8/d34/da6/f21 0 2024-05-17T21:53:10.256 INFO:tasks.workunit.client.0.smithi107.stdout:9/942: dread d0/dd/d21/d29/d3e/dfe/f103 [0,4194304] 0 2024-05-17T21:53:10.258 INFO:tasks.workunit.client.1.smithi138.stdout:0/685: fdatasync d3/d10/f20 0 2024-05-17T21:53:10.259 INFO:tasks.workunit.client.0.smithi107.stdout:3/991: dwrite d6/d1a/d57/f73 [0,4194304] 0 2024-05-17T21:53:10.261 INFO:tasks.workunit.client.1.smithi138.stdout:5/738: chown d1/d53/d96/fc6 84021 1 2024-05-17T21:53:10.262 INFO:tasks.workunit.client.1.smithi138.stdout:8/656: truncate d0/dc/d3b/d4d/d7d/f7f 3304567 0 2024-05-17T21:53:10.263 INFO:tasks.workunit.client.0.smithi107.stdout:2/785: mknod d0/d93/df5/c104 0 2024-05-17T21:53:10.263 INFO:tasks.workunit.client.0.smithi107.stdout:8/910: dwrite dc/ddb/f11b [0,4194304] 0 2024-05-17T21:53:10.267 INFO:tasks.workunit.client.1.smithi138.stdout:6/774: creat d7/d69/db5/dc8/d34/f108 x:0 0 0 2024-05-17T21:53:10.272 INFO:tasks.workunit.client.1.smithi138.stdout:0/686: readlink d3/d10/d6d/d81/l89 0 2024-05-17T21:53:10.275 INFO:tasks.workunit.client.0.smithi107.stdout:5/938: dwrite d5/db/d16/d2d/d32/f91 [4194304,4194304] 0 2024-05-17T21:53:10.279 INFO:tasks.workunit.client.0.smithi107.stdout:5/939: chown d5/db/dc/c98 14 1 2024-05-17T21:53:10.281 INFO:tasks.workunit.client.0.smithi107.stdout:3/992: creat d6/d2a/d2d/d48/d4f/f14a x:0 0 0 2024-05-17T21:53:10.282 INFO:tasks.workunit.client.0.smithi107.stdout:3/993: chown d6/d2a/l2b 1599 1 2024-05-17T21:53:10.282 INFO:tasks.workunit.client.0.smithi107.stdout:6/875: dread d9/d16/f4a [0,4194304] 0 2024-05-17T21:53:10.282 INFO:tasks.workunit.client.1.smithi138.stdout:5/739: mkdir d1/d8a/df0 0 2024-05-17T21:53:10.285 INFO:tasks.workunit.client.1.smithi138.stdout:1/709: sync 2024-05-17T21:53:10.286 INFO:tasks.workunit.client.0.smithi107.stdout:1/997: dread d4/f1b [0,4194304] 0 2024-05-17T21:53:10.286 INFO:tasks.workunit.client.0.smithi107.stdout:3/994: truncate d6/d2a/d2d/d37/f115 980770 0 2024-05-17T21:53:10.288 INFO:tasks.workunit.client.0.smithi107.stdout:8/911: mkdir dc/ddb/df2/d13d/d13e 0 2024-05-17T21:53:10.291 INFO:tasks.workunit.client.1.smithi138.stdout:0/687: creat d3/d10/d6d/d9f/fe7 x:0 0 0 2024-05-17T21:53:10.292 INFO:tasks.workunit.client.1.smithi138.stdout:5/740: symlink d1/d2c/d63/db5/d1b/dc5/d36/d68/dad/lf1 0 2024-05-17T21:53:10.294 INFO:tasks.workunit.client.1.smithi138.stdout:1/710: mkdir d4/db/d29/d5d/de5 0 2024-05-17T21:53:10.295 INFO:tasks.workunit.client.0.smithi107.stdout:7/896: dwrite d5/df/d11/d1c/d2f/d65/da7/fba [0,4194304] 0 2024-05-17T21:53:10.296 INFO:tasks.workunit.client.1.smithi138.stdout:8/657: symlink d0/dc/d3b/da4/dbc/lc2 0 2024-05-17T21:53:10.301 INFO:tasks.workunit.client.0.smithi107.stdout:1/998: fsync d4/df/d14/d1d/f27 0 2024-05-17T21:53:10.302 INFO:tasks.workunit.client.0.smithi107.stdout:1/999: chown d4/c1c 2365612 1 2024-05-17T21:53:10.305 INFO:tasks.workunit.client.0.smithi107.stdout:6/876: rename d9/d16/d22/d5f/dd8 to d9/d16/d22/d5f/d6e/dd6/d11f 0 2024-05-17T21:53:10.306 INFO:tasks.workunit.client.1.smithi138.stdout:5/741: creat d1/d53/d88/ff2 x:0 0 0 2024-05-17T21:53:10.307 INFO:tasks.workunit.client.0.smithi107.stdout:3/995: creat d6/d8/d10/d25/d80/f14b x:0 0 0 2024-05-17T21:53:10.313 INFO:tasks.workunit.client.0.smithi107.stdout:8/912: symlink dc/d21/d27/d34/df5/l13f 0 2024-05-17T21:53:10.313 INFO:tasks.workunit.client.0.smithi107.stdout:6/877: dwrite d9/d16/f3e [0,4194304] 0 2024-05-17T21:53:10.313 INFO:tasks.workunit.client.1.smithi138.stdout:9/671: sync 2024-05-17T21:53:10.314 INFO:tasks.workunit.client.1.smithi138.stdout:4/773: sync 2024-05-17T21:53:10.315 INFO:tasks.workunit.client.1.smithi138.stdout:9/672: chown d6/d7/d16/d29/d48/d6e/d74 442 1 2024-05-17T21:53:10.319 INFO:tasks.workunit.client.0.smithi107.stdout:3/996: dwrite d6/d2a/d2d/d37/f115 [0,4194304] 0 2024-05-17T21:53:10.323 INFO:tasks.workunit.client.1.smithi138.stdout:8/658: dwrite d0/dc/d1a/d33/d4a/d68/fac [0,4194304] 0 2024-05-17T21:53:10.324 INFO:tasks.workunit.client.0.smithi107.stdout:5/940: write d5/db/dc/d1e/fb5 [1187423,123124] 0 2024-05-17T21:53:10.325 INFO:tasks.workunit.client.1.smithi138.stdout:8/659: chown d0/d63/f9c 28641851 1 2024-05-17T21:53:10.326 INFO:tasks.workunit.client.1.smithi138.stdout:0/688: unlink d3/d10/d38/l77 0 2024-05-17T21:53:10.330 INFO:tasks.workunit.client.0.smithi107.stdout:8/913: dwrite dc/d21/daa/db4/f120 [0,4194304] 0 2024-05-17T21:53:10.330 INFO:tasks.workunit.client.1.smithi138.stdout:0/689: stat d3/d3d/d48/dbc 0 2024-05-17T21:53:10.331 INFO:tasks.workunit.client.0.smithi107.stdout:5/941: fsync d5/db/d16/f4b 0 2024-05-17T21:53:10.332 INFO:tasks.workunit.client.1.smithi138.stdout:5/742: dread d1/d2c/d63/db5/d1b/dc5/f3b [0,4194304] 0 2024-05-17T21:53:10.333 INFO:tasks.workunit.client.1.smithi138.stdout:6/775: write d7/d69/db5/dc8/f11 [2107591,19524] 0 2024-05-17T21:53:10.336 INFO:tasks.workunit.client.0.smithi107.stdout:2/786: creat d0/d26/f105 x:0 0 0 2024-05-17T21:53:10.340 INFO:tasks.workunit.client.1.smithi138.stdout:8/660: dread - d0/dc/d1a/fa1 zero size 2024-05-17T21:53:10.342 INFO:tasks.workunit.client.0.smithi107.stdout:9/943: dwrite d0/dd/d21/d29/d3e/d45/d68/f8c [0,4194304] 0 2024-05-17T21:53:10.343 INFO:tasks.workunit.client.0.smithi107.stdout:2/787: readlink d0/d9/d11/d15/l7e 0 2024-05-17T21:53:10.344 INFO:tasks.workunit.client.1.smithi138.stdout:1/711: dwrite d4/dd/f2f [4194304,4194304] 0 2024-05-17T21:53:10.345 INFO:tasks.workunit.client.0.smithi107.stdout:6/878: creat d9/d81/f120 x:0 0 0 2024-05-17T21:53:10.347 INFO:tasks.workunit.client.0.smithi107.stdout:2/788: write d0/d9/d11/d15/f57 [494334,8588] 0 2024-05-17T21:53:10.351 INFO:tasks.workunit.client.1.smithi138.stdout:9/673: symlink d6/d7/d16/d29/d38/d40/ld4 0 2024-05-17T21:53:10.357 INFO:tasks.workunit.client.1.smithi138.stdout:1/712: creat d4/df/d5f/fe6 x:0 0 0 2024-05-17T21:53:10.359 INFO:tasks.workunit.client.0.smithi107.stdout:9/944: dwrite d0/dd/d2f/f64 [0,4194304] 0 2024-05-17T21:53:10.359 INFO:tasks.workunit.client.1.smithi138.stdout:9/674: readlink d6/d7/d16/d29/d38/d40/l4f 0 2024-05-17T21:53:10.360 INFO:tasks.workunit.client.1.smithi138.stdout:9/675: chown d6/d7/d16/d29/d38/d3c/l89 51 1 2024-05-17T21:53:10.362 INFO:tasks.workunit.client.0.smithi107.stdout:6/879: fsync d9/f8d 0 2024-05-17T21:53:10.368 INFO:tasks.workunit.client.1.smithi138.stdout:1/713: link d4/df/d3a/d3c/d63/d6d/f87 d4/db/d60/dcc/fe7 0 2024-05-17T21:53:10.369 INFO:tasks.workunit.client.1.smithi138.stdout:1/714: write d4/db/d29/d5d/f86 [4690863,71710] 0 2024-05-17T21:53:10.370 INFO:tasks.workunit.client.1.smithi138.stdout:9/676: mknod d6/cd5 0 2024-05-17T21:53:10.371 INFO:tasks.workunit.client.1.smithi138.stdout:1/715: write d4/db/d29/d5d/f6f [1627750,54338] 0 2024-05-17T21:53:10.372 INFO:tasks.workunit.client.0.smithi107.stdout:5/942: getdents d5/db/d16/d54/d59/d114 0 2024-05-17T21:53:10.375 INFO:tasks.workunit.client.1.smithi138.stdout:6/776: dread d7/f22 [0,4194304] 0 2024-05-17T21:53:10.377 INFO:tasks.workunit.client.0.smithi107.stdout:9/945: dread d0/dd/d21/d29/d3e/fc7 [0,4194304] 0 2024-05-17T21:53:10.377 INFO:tasks.workunit.client.0.smithi107.stdout:6/880: rename d9/d16/d4d/db8/dc9/dd0/c51 to d9/d81/c121 0 2024-05-17T21:53:10.378 INFO:tasks.workunit.client.1.smithi138.stdout:1/716: mknod d4/dd/d3b/d9f/db9/dae/ce8 0 2024-05-17T21:53:10.382 INFO:tasks.workunit.client.0.smithi107.stdout:8/914: creat dc/d21/d27/f140 x:0 0 0 2024-05-17T21:53:10.384 INFO:tasks.workunit.client.0.smithi107.stdout:3/997: getdents d6/dfd 0 2024-05-17T21:53:10.385 INFO:tasks.workunit.client.1.smithi138.stdout:9/677: link d6/d7/l12 d6/d7/d16/d29/d6c/d9c/ld6 0 2024-05-17T21:53:10.387 INFO:tasks.workunit.client.0.smithi107.stdout:5/943: read d5/db/d16/d2a/f64 [1905367,122270] 0 2024-05-17T21:53:10.389 INFO:tasks.workunit.client.0.smithi107.stdout:3/998: write d6/d8/d10/d25/d80/f10a [2266498,39806] 0 2024-05-17T21:53:10.390 INFO:tasks.workunit.client.1.smithi138.stdout:1/717: dwrite d4/fc [4194304,4194304] 0 2024-05-17T21:53:10.390 INFO:tasks.workunit.client.0.smithi107.stdout:7/897: dwrite d5/df/d11/d1c/d31/d39/d92/f98 [0,4194304] 0 2024-05-17T21:53:10.391 INFO:tasks.workunit.client.0.smithi107.stdout:3/999: chown d6/d2a/d2d/d69/dad/fdf 23424494 1 2024-05-17T21:53:10.393 INFO:tasks.workunit.client.0.smithi107.stdout:9/946: truncate d0/dd/d21/d29/d3e/fc7 1978052 0 2024-05-17T21:53:10.393 INFO:tasks.workunit.client.0.smithi107.stdout:9/947: chown d0/dd/d21/daf/ffa 0 1 2024-05-17T21:53:10.394 INFO:tasks.workunit.client.0.smithi107.stdout:9/948: readlink d0/dd/d2f/l79 0 2024-05-17T21:53:10.395 INFO:tasks.workunit.client.0.smithi107.stdout:8/915: symlink dc/d44/d4b/d52/dcb/d11c/l141 0 2024-05-17T21:53:10.400 INFO:tasks.workunit.client.1.smithi138.stdout:9/678: dread d6/d7/d16/d29/d38/d40/d44/d5c/f8f [0,4194304] 0 2024-05-17T21:53:10.401 INFO:tasks.workunit.client.1.smithi138.stdout:9/679: readlink d6/d7/d16/d29/d38/d40/l4f 0 2024-05-17T21:53:10.402 INFO:tasks.workunit.client.0.smithi107.stdout:8/916: dread dc/ddb/f11b [0,4194304] 0 2024-05-17T21:53:10.405 INFO:tasks.workunit.client.1.smithi138.stdout:8/661: write d0/dc/d1a/d51/f80 [199409,1991] 0 2024-05-17T21:53:10.406 INFO:tasks.workunit.client.0.smithi107.stdout:5/944: symlink d5/db/d16/d2d/d32/l138 0 2024-05-17T21:53:10.406 INFO:tasks.workunit.client.1.smithi138.stdout:5/743: dwrite d1/d60/d2f/d5e/de4/fbc [0,4194304] 0 2024-05-17T21:53:10.407 INFO:tasks.workunit.client.0.smithi107.stdout:5/945: chown d5/db/dc/d1e 0 1 2024-05-17T21:53:10.409 INFO:tasks.workunit.client.0.smithi107.stdout:7/898: read d5/df/d11/d1c/d31/d56/f9c [413289,69881] 0 2024-05-17T21:53:10.413 INFO:tasks.workunit.client.1.smithi138.stdout:0/690: dwrite d3/d3d/f6a [0,4194304] 0 2024-05-17T21:53:10.417 INFO:tasks.workunit.client.0.smithi107.stdout:8/917: fdatasync dc/d21/d97/db7/fcc 0 2024-05-17T21:53:10.418 INFO:tasks.workunit.client.0.smithi107.stdout:8/918: chown dc/d21/d97/d63 2 1 2024-05-17T21:53:10.420 INFO:tasks.workunit.client.1.smithi138.stdout:8/662: fsync d0/dc/d1a/d33/d4a/d4e/f7c 0 2024-05-17T21:53:10.424 INFO:tasks.workunit.client.1.smithi138.stdout:0/691: creat d3/d10/d38/fe8 x:0 0 0 2024-05-17T21:53:10.424 INFO:tasks.workunit.client.0.smithi107.stdout:2/789: write d0/d9/f60 [4226442,33894] 0 2024-05-17T21:53:10.425 INFO:tasks.workunit.client.0.smithi107.stdout:7/899: truncate d5/df/d11/d1c/d31/d39/f94 447337 0 2024-05-17T21:53:10.427 INFO:tasks.workunit.client.1.smithi138.stdout:9/680: link d6/d7/d16/d29/da6/fae d6/d7/d16/d29/d48/fd7 0 2024-05-17T21:53:10.430 INFO:tasks.workunit.client.1.smithi138.stdout:6/777: write d7/d8/f50 [1053351,73514] 0 2024-05-17T21:53:10.432 INFO:tasks.workunit.client.1.smithi138.stdout:5/744: link d1/d53/d88/de5/ded/fe1 d1/d2c/d63/db5/d1b/ddb/ff3 0 2024-05-17T21:53:10.434 INFO:tasks.workunit.client.0.smithi107.stdout:5/946: dread d5/db/d16/d54/d59/d65/fac [0,4194304] 0 2024-05-17T21:53:10.435 INFO:tasks.workunit.client.1.smithi138.stdout:5/745: write d1/d2c/d63/f9d [1728785,23980] 0 2024-05-17T21:53:10.436 INFO:tasks.workunit.client.0.smithi107.stdout:6/881: truncate d9/d16/d22/d5f/fdc 884842 0 2024-05-17T21:53:10.437 INFO:tasks.workunit.client.1.smithi138.stdout:0/692: mknod d3/d10/d39/dd9/ce9 0 2024-05-17T21:53:10.439 INFO:tasks.workunit.client.1.smithi138.stdout:1/718: dwrite d4/d2e/da0/daa/fad [0,4194304] 0 2024-05-17T21:53:10.442 INFO:tasks.workunit.client.0.smithi107.stdout:2/790: creat d0/d9/d11/df/d5a/d6d/d82/f106 x:0 0 0 2024-05-17T21:53:10.442 INFO:tasks.workunit.client.0.smithi107.stdout:9/949: write d0/d80/fcf [372995,38429] 0 2024-05-17T21:53:10.443 INFO:tasks.workunit.client.1.smithi138.stdout:5/746: dread d1/d2c/d63/db5/d1b/dc5/f3b [0,4194304] 0 2024-05-17T21:53:10.443 INFO:tasks.workunit.client.0.smithi107.stdout:2/791: stat d0/d9/d11/f9b 0 2024-05-17T21:53:10.444 INFO:tasks.workunit.client.0.smithi107.stdout:9/950: chown d0/dd/d21/l106 51127 1 2024-05-17T21:53:10.445 INFO:tasks.workunit.client.0.smithi107.stdout:2/792: chown d0/d9/d11/d18/dad/db8/lc2 48 1 2024-05-17T21:53:10.445 INFO:tasks.workunit.client.1.smithi138.stdout:6/778: fsync d7/d69/f80 0 2024-05-17T21:53:10.447 INFO:tasks.workunit.client.0.smithi107.stdout:8/919: symlink dc/d21/d27/l142 0 2024-05-17T21:53:10.454 INFO:tasks.workunit.client.1.smithi138.stdout:1/719: mknod d4/df/d4e/ce9 0 2024-05-17T21:53:10.456 INFO:tasks.workunit.client.0.smithi107.stdout:9/951: mkdir d0/d80/dad/d133 0 2024-05-17T21:53:10.456 INFO:tasks.workunit.client.0.smithi107.stdout:9/952: readlink d0/dd/d21/d29/d57/d5f/l86 0 2024-05-17T21:53:10.457 INFO:tasks.workunit.client.1.smithi138.stdout:9/681: symlink d6/ld8 0 2024-05-17T21:53:10.458 INFO:tasks.workunit.client.1.smithi138.stdout:8/663: write d0/dc/d3b/d4d/f77 [693171,28767] 0 2024-05-17T21:53:10.459 INFO:tasks.workunit.client.1.smithi138.stdout:8/664: readlink d0/d2f/l39 0 2024-05-17T21:53:10.460 INFO:tasks.workunit.client.1.smithi138.stdout:9/682: truncate d6/d7/d16/d29/d38/f97 1290564 0 2024-05-17T21:53:10.462 INFO:tasks.workunit.client.0.smithi107.stdout:7/900: write d5/df/f20 [4850884,37735] 0 2024-05-17T21:53:10.463 INFO:tasks.workunit.client.0.smithi107.stdout:7/901: readlink d5/df/d11/d1c/l71 0 2024-05-17T21:53:10.464 INFO:tasks.workunit.client.0.smithi107.stdout:7/902: dread - d5/d16/d3b/d45/f108 zero size 2024-05-17T21:53:10.465 INFO:tasks.workunit.client.1.smithi138.stdout:6/779: mknod d7/d69/db5/dc8/d43/d5f/d94/c109 0 2024-05-17T21:53:10.466 INFO:tasks.workunit.client.1.smithi138.stdout:0/693: symlink d3/d3d/d51/dba/lea 0 2024-05-17T21:53:10.468 INFO:tasks.workunit.client.0.smithi107.stdout:5/947: write d5/db/d16/d2d/d40/d5a/d7c/d8e/fb0 [22335,42846] 0 2024-05-17T21:53:10.469 INFO:tasks.workunit.client.1.smithi138.stdout:8/665: mkdir d0/dc/d3b/da4/dbc/dc3 0 2024-05-17T21:53:10.470 INFO:tasks.workunit.client.0.smithi107.stdout:9/953: fsync d0/dd/d21/d29/dca/ff0 0 2024-05-17T21:53:10.473 INFO:tasks.workunit.client.1.smithi138.stdout:9/683: fdatasync d6/d7/d16/d29/d48/d6e/f56 0 2024-05-17T21:53:10.474 INFO:tasks.workunit.client.1.smithi138.stdout:9/684: chown d6/d7/d16/d29/d48/d6e/d74/d75/d76/f7f 15 1 2024-05-17T21:53:10.475 INFO:tasks.workunit.client.0.smithi107.stdout:8/920: fdatasync dc/d21/d27/d34/d4c/f50 0 2024-05-17T21:53:10.476 INFO:tasks.workunit.client.1.smithi138.stdout:0/694: unlink d3/d10/l13 0 2024-05-17T21:53:10.476 INFO:tasks.workunit.client.1.smithi138.stdout:0/695: chown d3/d3d/d48/l8c 16537419 1 2024-05-17T21:53:10.477 INFO:tasks.workunit.client.0.smithi107.stdout:5/948: dwrite d5/db/d16/d2a/f94 [0,4194304] 0 2024-05-17T21:53:10.478 INFO:tasks.workunit.client.0.smithi107.stdout:7/903: read d5/f42 [867448,63500] 0 2024-05-17T21:53:10.479 INFO:tasks.workunit.client.0.smithi107.stdout:6/882: creat d9/d16/d4d/d85/f122 x:0 0 0 2024-05-17T21:53:10.479 INFO:tasks.workunit.client.1.smithi138.stdout:1/720: write d4/db/f2a [550586,89076] 0 2024-05-17T21:53:10.484 INFO:tasks.workunit.client.0.smithi107.stdout:8/921: dread dc/d21/fd3 [0,4194304] 0 2024-05-17T21:53:10.484 INFO:tasks.workunit.client.0.smithi107.stdout:6/883: write d9/d16/d22/daf/dc7/f101 [1564140,11147] 0 2024-05-17T21:53:10.485 INFO:tasks.workunit.client.0.smithi107.stdout:2/793: creat d0/d26/f107 x:0 0 0 2024-05-17T21:53:10.489 INFO:tasks.workunit.client.1.smithi138.stdout:6/780: link d7/d69/db5/dc8/d43/d5f/d6a/d77/f85 d7/d69/db5/dc8/d60/f10a 0 2024-05-17T21:53:10.491 INFO:tasks.workunit.client.1.smithi138.stdout:5/747: getdents d1/d2c/d63/db5/d1b/dc5/d36/d68 0 2024-05-17T21:53:10.497 INFO:tasks.workunit.client.1.smithi138.stdout:6/781: write d7/da0/dbe/fe2 [709346,63159] 0 2024-05-17T21:53:10.498 INFO:tasks.workunit.client.0.smithi107.stdout:8/922: creat dc/d21/d68/f143 x:0 0 0 2024-05-17T21:53:10.500 INFO:tasks.workunit.client.0.smithi107.stdout:7/904: dread d5/df/d11/d1c/d2f/d53/fa9 [0,4194304] 0 2024-05-17T21:53:10.500 INFO:tasks.workunit.client.0.smithi107.stdout:2/794: creat d0/d9/d11/d18/d9e/f108 x:0 0 0 2024-05-17T21:53:10.501 INFO:tasks.workunit.client.1.smithi138.stdout:5/748: stat d1/l14 0 2024-05-17T21:53:10.502 INFO:tasks.workunit.client.0.smithi107.stdout:7/905: write d5/d16/d3b/ddf/d12e/fc7 [1915113,83492] 0 2024-05-17T21:53:10.502 INFO:tasks.workunit.client.0.smithi107.stdout:2/795: write d0/d9/d11/d18/dad/fc3 [1775015,105188] 0 2024-05-17T21:53:10.503 INFO:tasks.workunit.client.0.smithi107.stdout:7/906: dread - d5/d16/d3b/f115 zero size 2024-05-17T21:53:10.504 INFO:tasks.workunit.client.1.smithi138.stdout:0/696: mknod d3/d10/d38/d3e/d59/ceb 0 2024-05-17T21:53:10.507 INFO:tasks.workunit.client.1.smithi138.stdout:9/685: dwrite d6/d7/d16/d29/d6c/d9c/dc1/f92 [0,4194304] 0 2024-05-17T21:53:10.508 INFO:tasks.workunit.client.1.smithi138.stdout:6/782: symlink d7/d8/def/l10b 0 2024-05-17T21:53:10.508 INFO:tasks.workunit.client.0.smithi107.stdout:5/949: mkdir d5/d11e/d139 0 2024-05-17T21:53:10.511 INFO:tasks.workunit.client.0.smithi107.stdout:6/884: dread d9/d81/f82 [0,4194304] 0 2024-05-17T21:53:10.513 INFO:tasks.workunit.client.1.smithi138.stdout:0/697: write d3/d10/d38/d70/d80/f8b [1373239,92018] 0 2024-05-17T21:53:10.513 INFO:tasks.workunit.client.1.smithi138.stdout:9/686: stat d6/cce 0 2024-05-17T21:53:10.514 INFO:tasks.workunit.client.1.smithi138.stdout:8/666: dread d0/dc/d1a/f1d [0,4194304] 0 2024-05-17T21:53:10.516 INFO:tasks.workunit.client.1.smithi138.stdout:5/749: dwrite d1/d2c/d63/db5/d19/f20 [0,4194304] 0 2024-05-17T21:53:10.518 INFO:tasks.workunit.client.1.smithi138.stdout:6/783: mkdir d7/d8/d10c 0 2024-05-17T21:53:10.519 INFO:tasks.workunit.client.0.smithi107.stdout:5/950: fdatasync d5/db/d16/d54/d59/f7f 0 2024-05-17T21:53:10.520 INFO:tasks.workunit.client.1.smithi138.stdout:1/721: dwrite d4/df/d4e/f84 [0,4194304] 0 2024-05-17T21:53:10.520 INFO:tasks.workunit.client.0.smithi107.stdout:9/954: dwrite d0/dd/ff6 [0,4194304] 0 2024-05-17T21:53:10.521 INFO:tasks.workunit.client.1.smithi138.stdout:0/698: chown d3/d10/d6d/fa5 1897351 1 2024-05-17T21:53:10.522 INFO:tasks.workunit.client.1.smithi138.stdout:6/784: dread - d7/d8/def/fd1 zero size 2024-05-17T21:53:10.522 INFO:tasks.workunit.client.1.smithi138.stdout:7/735: sync 2024-05-17T21:53:10.523 INFO:tasks.workunit.client.1.smithi138.stdout:3/930: sync 2024-05-17T21:53:10.523 INFO:tasks.workunit.client.1.smithi138.stdout:2/794: sync 2024-05-17T21:53:10.524 INFO:tasks.workunit.client.1.smithi138.stdout:2/795: readlink d3/l51 0 2024-05-17T21:53:10.525 INFO:tasks.workunit.client.0.smithi107.stdout:8/923: rename dc/d21/d27/f37 to dc/d21/d97/f144 0 2024-05-17T21:53:10.525 INFO:tasks.workunit.client.1.smithi138.stdout:8/667: mknod d0/d11/d6a/cc4 0 2024-05-17T21:53:10.526 INFO:tasks.workunit.client.0.smithi107.stdout:6/885: dread d9/d16/d22/d5f/d6e/f87 [0,4194304] 0 2024-05-17T21:53:10.534 INFO:tasks.workunit.client.1.smithi138.stdout:5/750: mknod d1/d2c/d63/db5/cf4 0 2024-05-17T21:53:10.536 INFO:tasks.workunit.client.1.smithi138.stdout:7/736: truncate d0/d18/d40/d41/d51/d66/d9e/fa1 1277818 0 2024-05-17T21:53:10.537 INFO:tasks.workunit.client.1.smithi138.stdout:1/722: dwrite d4/df/d5f/fe6 [0,4194304] 0 2024-05-17T21:53:10.537 INFO:tasks.workunit.client.1.smithi138.stdout:3/931: creat d1/d3/d16/dd7/f138 x:0 0 0 2024-05-17T21:53:10.538 INFO:tasks.workunit.client.0.smithi107.stdout:9/955: read - d0/d80/dc9/f10a zero size 2024-05-17T21:53:10.539 INFO:tasks.workunit.client.0.smithi107.stdout:6/886: dwrite d9/d16/d4d/d85/d88/db1/ffe [0,4194304] 0 2024-05-17T21:53:10.540 INFO:tasks.workunit.client.0.smithi107.stdout:6/887: write d9/d16/f1e [3452095,111483] 0 2024-05-17T21:53:10.541 INFO:tasks.workunit.client.1.smithi138.stdout:9/687: creat d6/fd9 x:0 0 0 2024-05-17T21:53:10.546 INFO:tasks.workunit.client.0.smithi107.stdout:9/956: read d0/dd/d21/d29/d3e/d45/f6b [5593317,37265] 0 2024-05-17T21:53:10.549 INFO:tasks.workunit.client.1.smithi138.stdout:8/668: mknod d0/d31/cc5 0 2024-05-17T21:53:10.551 INFO:tasks.workunit.client.0.smithi107.stdout:2/796: truncate d0/d9/d11/df/d5a/d6d/dc8/de8/ddb/fe3 3708749 0 2024-05-17T21:53:10.553 INFO:tasks.workunit.client.1.smithi138.stdout:8/669: dread d0/dc/d1a/d33/d4a/d68/fac [0,4194304] 0 2024-05-17T21:53:10.554 INFO:tasks.workunit.client.0.smithi107.stdout:2/797: readlink d0/d9/d11/d27/l56 0 2024-05-17T21:53:10.556 INFO:tasks.workunit.client.1.smithi138.stdout:6/785: truncate d7/d69/db5/dc8/d43/d3f/f46 223318 0 2024-05-17T21:53:10.558 INFO:tasks.workunit.client.1.smithi138.stdout:6/786: write d7/d69/db5/dc8/d43/d5f/d6a/d9b/db2/fe7 [1006001,117597] 0 2024-05-17T21:53:10.559 INFO:tasks.workunit.client.0.smithi107.stdout:6/888: unlink d9/d16/c46 0 2024-05-17T21:53:10.561 INFO:tasks.workunit.client.0.smithi107.stdout:6/889: write d9/d16/d22/daf/dc7/f101 [1779091,12854] 0 2024-05-17T21:53:10.561 INFO:tasks.workunit.client.1.smithi138.stdout:8/670: dwrite d0/d71/fa5 [0,4194304] 0 2024-05-17T21:53:10.562 INFO:tasks.workunit.client.0.smithi107.stdout:2/798: mkdir d0/d9/d11/d18/d9e/deb/d109 0 2024-05-17T21:53:10.563 INFO:tasks.workunit.client.1.smithi138.stdout:8/671: fsync d0/ff 0 2024-05-17T21:53:10.563 INFO:tasks.workunit.client.1.smithi138.stdout:6/787: write d7/d69/db5/dc8/d43/d5f/d6a/fd3 [3199468,80835] 0 2024-05-17T21:53:10.564 INFO:tasks.workunit.client.0.smithi107.stdout:8/924: link dc/d21/d97/d63/f79 dc/d44/d4b/d121/f145 0 2024-05-17T21:53:10.564 INFO:tasks.workunit.client.1.smithi138.stdout:2/796: symlink d3/d21/d2f/de1/l10b 0 2024-05-17T21:53:10.564 INFO:tasks.workunit.client.1.smithi138.stdout:4/774: sync 2024-05-17T21:53:10.568 INFO:tasks.workunit.client.0.smithi107.stdout:6/890: rmdir d9/d16/d22 39 2024-05-17T21:53:10.571 INFO:tasks.workunit.client.1.smithi138.stdout:7/737: mknod d0/d18/cee 0 2024-05-17T21:53:10.573 INFO:tasks.workunit.client.0.smithi107.stdout:9/957: link d0/dd/d3d/f92 d0/dd/d21/d29/dd4/f134 0 2024-05-17T21:53:10.574 INFO:tasks.workunit.client.1.smithi138.stdout:4/775: dwrite d0/d3/d17/d5e/d7f/fe7 [0,4194304] 0 2024-05-17T21:53:10.576 INFO:tasks.workunit.client.0.smithi107.stdout:5/951: dwrite d5/fb8 [0,4194304] 0 2024-05-17T21:53:10.578 INFO:tasks.workunit.client.0.smithi107.stdout:5/952: chown d5/db/dc/d1e/l46 1 1 2024-05-17T21:53:10.579 INFO:tasks.workunit.client.0.smithi107.stdout:8/925: rename dc/d21/d69/f8e to dc/d21/d97/db7/dcd/f146 0 2024-05-17T21:53:10.580 INFO:tasks.workunit.client.1.smithi138.stdout:3/932: symlink d1/d62/l139 0 2024-05-17T21:53:10.580 INFO:tasks.workunit.client.1.smithi138.stdout:4/776: read d0/d2f/d49/d60/d1f/f4d [407462,44009] 0 2024-05-17T21:53:10.583 INFO:tasks.workunit.client.1.smithi138.stdout:3/933: write d1/d4f/fd5 [343636,6126] 0 2024-05-17T21:53:10.586 INFO:tasks.workunit.client.1.smithi138.stdout:1/723: creat d4/dd/db4/dba/fea x:0 0 0 2024-05-17T21:53:10.588 INFO:tasks.workunit.client.0.smithi107.stdout:8/926: dwrite dc/d21/d27/d34/f6e [0,4194304] 0 2024-05-17T21:53:10.594 INFO:tasks.workunit.client.1.smithi138.stdout:0/699: dwrite d3/fd [0,4194304] 0 2024-05-17T21:53:10.596 INFO:tasks.workunit.client.0.smithi107.stdout:9/958: unlink d0/dd/d2f/d4a/c8b 0 2024-05-17T21:53:10.604 INFO:tasks.workunit.client.1.smithi138.stdout:8/672: unlink d0/d2f/l3d 0 2024-05-17T21:53:10.605 INFO:tasks.workunit.client.1.smithi138.stdout:9/688: dwrite d6/d7/d16/d29/d38/d40/d44/f78 [0,4194304] 0 2024-05-17T21:53:10.605 INFO:tasks.workunit.client.1.smithi138.stdout:8/673: stat d0/f17 0 2024-05-17T21:53:10.609 INFO:tasks.workunit.client.1.smithi138.stdout:9/689: fsync d6/d7/d16/d29/d48/d6e/d74/d75/d76/fbf 0 2024-05-17T21:53:10.611 INFO:tasks.workunit.client.1.smithi138.stdout:6/788: mkdir d7/d69/db5/dc8/d43/d5f/d6a/d9b/d10d 0 2024-05-17T21:53:10.614 INFO:tasks.workunit.client.0.smithi107.stdout:2/799: write d0/d9/d11/df/d5a/d6d/dc8/de8/fa9 [1358684,128618] 0 2024-05-17T21:53:10.618 INFO:tasks.workunit.client.1.smithi138.stdout:2/797: rename d3/d21/d2f/d3e/d55/d99/d93/lc6 to d3/d21/d2f/d3e/d63/l10c 0 2024-05-17T21:53:10.620 INFO:tasks.workunit.client.1.smithi138.stdout:2/798: truncate d3/d21/d2f/d3e/d55/d99/dd2/f101 620414 0 2024-05-17T21:53:10.620 INFO:tasks.workunit.client.0.smithi107.stdout:5/953: getdents d5/d106 0 2024-05-17T21:53:10.622 INFO:tasks.workunit.client.1.smithi138.stdout:7/738: unlink d0/d18/d64/fca 0 2024-05-17T21:53:10.622 INFO:tasks.workunit.client.0.smithi107.stdout:6/891: link d9/d16/d4d/db8/dc9/dd0/l7d d9/d16/d4d/d67/l123 0 2024-05-17T21:53:10.623 INFO:tasks.workunit.client.0.smithi107.stdout:8/927: dwrite dc/d21/d27/d2f/dba/dee/fff [0,4194304] 0 2024-05-17T21:53:10.624 INFO:tasks.workunit.client.1.smithi138.stdout:7/739: fsync d0/d18/d40/d41/d50/f59 0 2024-05-17T21:53:10.624 INFO:tasks.workunit.client.1.smithi138.stdout:7/740: write d0/d9d/fb5 [2495856,116046] 0 2024-05-17T21:53:10.626 INFO:tasks.workunit.client.0.smithi107.stdout:8/928: write dc/d21/d27/d2f/fe9 [788413,109729] 0 2024-05-17T21:53:10.626 INFO:tasks.workunit.client.0.smithi107.stdout:9/959: dread d0/dd/d21/d29/d57/fe2 [0,4194304] 0 2024-05-17T21:53:10.628 INFO:tasks.workunit.client.0.smithi107.stdout:8/929: chown dc/f67 22083 1 2024-05-17T21:53:10.633 INFO:tasks.workunit.client.0.smithi107.stdout:8/930: dread dc/d21/d27/d2f/dba/d104/f117 [0,4194304] 0 2024-05-17T21:53:10.635 INFO:tasks.workunit.client.0.smithi107.stdout:8/931: fsync dc/d21/d27/d34/f38 0 2024-05-17T21:53:10.636 INFO:tasks.workunit.client.0.smithi107.stdout:8/932: stat dc/d21/d27/d34/f6e 0 2024-05-17T21:53:10.637 INFO:tasks.workunit.client.0.smithi107.stdout:5/954: mknod d5/db/d16/d2d/d40/d5a/db4/df4/c13a 0 2024-05-17T21:53:10.640 INFO:tasks.workunit.client.0.smithi107.stdout:9/960: creat d0/dd/d21/d29/d57/d5f/dc6/f135 x:0 0 0 2024-05-17T21:53:10.644 INFO:tasks.workunit.client.1.smithi138.stdout:5/751: getdents d1/d53/d96/da0 0 2024-05-17T21:53:10.648 INFO:tasks.workunit.client.0.smithi107.stdout:6/892: dread d9/d16/d4d/db8/dc9/dd0/f63 [0,4194304] 0 2024-05-17T21:53:10.651 INFO:tasks.workunit.client.0.smithi107.stdout:5/955: rename d5/db/c15 to d5/db/d16/d2d/d51/c13b 0 2024-05-17T21:53:10.652 INFO:tasks.workunit.client.0.smithi107.stdout:5/956: write d5/f133 [800269,21415] 0 2024-05-17T21:53:10.653 INFO:tasks.workunit.client.1.smithi138.stdout:2/799: truncate d3/d21/d2f/d3e/d58/d77/da4/dc5/f8b 427737 0 2024-05-17T21:53:10.653 INFO:tasks.workunit.client.1.smithi138.stdout:1/724: dwrite d4/df/d3a/d3c/f99 [0,4194304] 0 2024-05-17T21:53:10.659 INFO:tasks.workunit.client.1.smithi138.stdout:9/690: dread d6/d7/d16/d29/d48/d6e/f56 [0,4194304] 0 2024-05-17T21:53:10.660 INFO:tasks.workunit.client.0.smithi107.stdout:5/957: dwrite d5/db/d16/d54/d59/d65/d48/d89/fa8 [4194304,4194304] 0 2024-05-17T21:53:10.663 INFO:tasks.workunit.client.0.smithi107.stdout:9/961: link d0/dd/d69/fc1 d0/d80/dad/d133/f136 0 2024-05-17T21:53:10.663 INFO:tasks.workunit.client.1.smithi138.stdout:3/934: symlink d1/d3/d16/d3e/l13a 0 2024-05-17T21:53:10.668 INFO:tasks.workunit.client.0.smithi107.stdout:5/958: creat d5/dec/f13c x:0 0 0 2024-05-17T21:53:10.669 INFO:tasks.workunit.client.1.smithi138.stdout:6/789: mkdir d7/d69/db5/dc8/d43/d5f/d6a/d77/de5/d10e 0 2024-05-17T21:53:10.670 INFO:tasks.workunit.client.0.smithi107.stdout:8/933: link dc/d21/d97/d63/d71/fbb dc/d21/d27/d34/d4c/f147 0 2024-05-17T21:53:10.670 INFO:tasks.workunit.client.0.smithi107.stdout:6/893: dread d9/d16/d22/daf/dc7/f101 [0,4194304] 0 2024-05-17T21:53:10.671 INFO:tasks.workunit.client.0.smithi107.stdout:9/962: truncate d0/dd/d2f/d87/d89/de0/f47 3984474 0 2024-05-17T21:53:10.676 INFO:tasks.workunit.client.1.smithi138.stdout:9/691: fsync d6/d7/d16/f8a 0 2024-05-17T21:53:10.680 INFO:tasks.workunit.client.0.smithi107.stdout:9/963: dwrite d0/dd/d2f/d87/fea [4194304,4194304] 0 2024-05-17T21:53:10.682 INFO:tasks.workunit.client.0.smithi107.stdout:5/959: rename d5/db/d16/d54/da0/laf to d5/db/d16/d2d/d40/d5a/d7c/dee/l13d 0 2024-05-17T21:53:10.684 INFO:tasks.workunit.client.1.smithi138.stdout:8/674: creat d0/dc/fc6 x:0 0 0 2024-05-17T21:53:10.690 INFO:tasks.workunit.client.0.smithi107.stdout:6/894: creat d9/d16/d4d/d85/d96/d104/f124 x:0 0 0 2024-05-17T21:53:10.693 INFO:tasks.workunit.client.1.smithi138.stdout:8/675: dwrite d0/d11/f69 [4194304,4194304] 0 2024-05-17T21:53:10.693 INFO:tasks.workunit.client.1.smithi138.stdout:5/752: dread d1/d2c/d63/db5/f72 [0,4194304] 0 2024-05-17T21:53:10.694 INFO:tasks.workunit.client.1.smithi138.stdout:4/777: rename d0/d3/db/c19 to d0/d2f/d49/d60/c109 0 2024-05-17T21:53:10.696 INFO:tasks.workunit.client.0.smithi107.stdout:9/964: fsync d0/fc 0 2024-05-17T21:53:10.697 INFO:tasks.workunit.client.0.smithi107.stdout:6/895: dwrite d9/d16/f1e [0,4194304] 0 2024-05-17T21:53:10.697 INFO:tasks.workunit.client.1.smithi138.stdout:6/790: dread d7/d69/db5/dc8/f11 [0,4194304] 0 2024-05-17T21:53:10.697 INFO:tasks.workunit.client.0.smithi107.stdout:9/965: chown d0/dd/d21/d29/d57/d5f 227479 1 2024-05-17T21:53:10.700 INFO:tasks.workunit.client.1.smithi138.stdout:1/725: symlink d4/leb 0 2024-05-17T21:53:10.701 INFO:tasks.workunit.client.0.smithi107.stdout:9/966: chown d0/dd/d69/dee/d116/f11d 1304 1 2024-05-17T21:53:10.703 INFO:tasks.workunit.client.1.smithi138.stdout:4/778: dwrite d0/d2f/d49/d60/f41 [0,4194304] 0 2024-05-17T21:53:10.703 INFO:tasks.workunit.client.0.smithi107.stdout:5/960: symlink d5/db/d16/d2d/d32/l13e 0 2024-05-17T21:53:10.704 INFO:tasks.workunit.client.0.smithi107.stdout:5/961: stat d5/db/d16/d2a/ce2 0 2024-05-17T21:53:10.705 INFO:tasks.workunit.client.0.smithi107.stdout:8/934: link dc/d21/d27/d34/d4c/l5a dc/d21/d27/d2f/dba/dee/l148 0 2024-05-17T21:53:10.712 INFO:tasks.workunit.client.1.smithi138.stdout:7/741: creat d0/d18/fef x:0 0 0 2024-05-17T21:53:10.713 INFO:tasks.workunit.client.1.smithi138.stdout:2/800: write d3/d21/d2f/d3e/d63/f6e [180353,125528] 0 2024-05-17T21:53:10.715 INFO:tasks.workunit.client.1.smithi138.stdout:1/726: dread d4/df/d3a/d3c/d63/d6d/de0/fcb [0,4194304] 0 2024-05-17T21:53:10.716 INFO:tasks.workunit.client.1.smithi138.stdout:3/935: dread d1/d62/dad/f34 [0,4194304] 0 2024-05-17T21:53:10.716 INFO:tasks.workunit.client.1.smithi138.stdout:2/801: readlink d3/d21/d2f/l4b 0 2024-05-17T21:53:10.717 INFO:tasks.workunit.client.0.smithi107.stdout:6/896: mknod d9/d16/d4d/d85/d88/c125 0 2024-05-17T21:53:10.717 INFO:tasks.workunit.client.1.smithi138.stdout:9/692: mkdir d6/d7/d16/d29/d38/d40/d44/d5c/dda 0 2024-05-17T21:53:10.719 INFO:tasks.workunit.client.1.smithi138.stdout:0/700: creat d3/d10/d38/d3e/d59/fec x:0 0 0 2024-05-17T21:53:10.720 INFO:tasks.workunit.client.0.smithi107.stdout:9/967: symlink d0/dd/d69/dee/l137 0 2024-05-17T21:53:10.720 INFO:tasks.workunit.client.0.smithi107.stdout:5/962: dread d5/db/d39/f70 [0,4194304] 0 2024-05-17T21:53:10.721 INFO:tasks.workunit.client.0.smithi107.stdout:5/963: stat d5/db/fc6 0 2024-05-17T21:53:10.721 INFO:tasks.workunit.client.0.smithi107.stdout:8/935: mkdir dc/d21/daa/db4/d149 0 2024-05-17T21:53:10.721 INFO:tasks.workunit.client.1.smithi138.stdout:0/701: chown d3/d3d/d51/f6e 65259416 1 2024-05-17T21:53:10.723 INFO:tasks.workunit.client.1.smithi138.stdout:7/742: dwrite d0/d18/d40/d5f/f8f [0,4194304] 0 2024-05-17T21:53:10.724 INFO:tasks.workunit.client.0.smithi107.stdout:6/897: unlink d9/d81/fcd 0 2024-05-17T21:53:10.725 INFO:tasks.workunit.client.0.smithi107.stdout:6/898: dread - d9/d16/d4d/d85/d96/ffb zero size 2024-05-17T21:53:10.728 INFO:tasks.workunit.client.1.smithi138.stdout:8/676: creat d0/d71/fc7 x:0 0 0 2024-05-17T21:53:10.730 INFO:tasks.workunit.client.0.smithi107.stdout:5/964: rmdir d5/d5f 39 2024-05-17T21:53:10.738 INFO:tasks.workunit.client.0.smithi107.stdout:5/965: dread d5/db/dc/d1e/fb5 [0,4194304] 0 2024-05-17T21:53:10.741 INFO:tasks.workunit.client.1.smithi138.stdout:4/779: symlink d0/d3/db/da0/l10a 0 2024-05-17T21:53:10.741 INFO:tasks.workunit.client.0.smithi107.stdout:9/968: unlink d0/dd/d1f/l129 0 2024-05-17T21:53:10.742 INFO:tasks.workunit.client.1.smithi138.stdout:4/780: write d0/d3/d1d/d38/d8a/dbd/ff5 [472944,59487] 0 2024-05-17T21:53:10.742 INFO:tasks.workunit.client.1.smithi138.stdout:1/727: dread d4/d2e/f9d [0,4194304] 0 2024-05-17T21:53:10.743 INFO:tasks.workunit.client.0.smithi107.stdout:7/907: sync 2024-05-17T21:53:10.745 INFO:tasks.workunit.client.0.smithi107.stdout:6/899: rename d9/d16/d4d/db8/dc9/dd0/f9a to d9/d16/d4d/d85/d96/d117/f126 0 2024-05-17T21:53:10.749 INFO:tasks.workunit.client.1.smithi138.stdout:3/936: rename d1/d3/d16/d3e/d71/c137 to d1/d3/d16/d2d/c13b 0 2024-05-17T21:53:10.749 INFO:tasks.workunit.client.0.smithi107.stdout:7/908: dwrite d5/df/d11/d1c/d31/f48 [4194304,4194304] 0 2024-05-17T21:53:10.756 INFO:tasks.workunit.client.0.smithi107.stdout:5/966: creat d5/db/d16/d2d/d40/d5a/d7c/f13f x:0 0 0 2024-05-17T21:53:10.757 INFO:tasks.workunit.client.1.smithi138.stdout:7/743: symlink d0/d18/dbf/lf0 0 2024-05-17T21:53:10.761 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:10 smithi107 ceph-mon[28434]: pgmap v43: 65 pgs: 65 active+clean; 3.0 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 88 MiB/s rd, 202 MiB/s wr, 449 op/s 2024-05-17T21:53:10.761 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:10 smithi107 ceph-mon[28434]: from='mgr.14636 ' entity='mgr.smithi138.napnll' 2024-05-17T21:53:10.761 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:10 smithi107 ceph-mon[28434]: from='mgr.14636 ' entity='mgr.smithi138.napnll' 2024-05-17T21:53:10.761 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:10 smithi107 ceph-mon[28434]: from='mgr.14636 172.21.15.138:0/3120106723' entity='mgr.smithi138.napnll' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:53:10.762 INFO:tasks.workunit.client.1.smithi138.stdout:5/753: mkdir d1/d2c/d63/db5/df5 0 2024-05-17T21:53:10.763 INFO:tasks.workunit.client.1.smithi138.stdout:5/754: chown d1/d60/d2f/d5e/de4/fac 1994 1 2024-05-17T21:53:10.765 INFO:tasks.workunit.client.1.smithi138.stdout:4/781: mknod d0/d1b/dd4/c10b 0 2024-05-17T21:53:10.768 INFO:tasks.workunit.client.1.smithi138.stdout:1/728: creat d4/dd/d3b/d9f/fec x:0 0 0 2024-05-17T21:53:10.768 INFO:tasks.workunit.client.1.smithi138.stdout:2/802: write d3/d21/d2f/d3e/d63/f80 [5209545,58909] 0 2024-05-17T21:53:10.769 INFO:tasks.workunit.client.1.smithi138.stdout:2/803: fsync d3/f4a 0 2024-05-17T21:53:10.776 INFO:tasks.workunit.client.1.smithi138.stdout:8/677: write d0/dc/d1a/f1d [4304364,107669] 0 2024-05-17T21:53:10.777 INFO:tasks.workunit.client.1.smithi138.stdout:9/693: dwrite d6/d7/d16/d29/da6/fae [0,4194304] 0 2024-05-17T21:53:10.781 INFO:tasks.workunit.client.1.smithi138.stdout:5/755: fdatasync d1/d2c/f35 0 2024-05-17T21:53:10.785 INFO:tasks.workunit.client.1.smithi138.stdout:3/937: write d1/d4f/d50/f54 [511476,9488] 0 2024-05-17T21:53:10.786 INFO:tasks.workunit.client.1.smithi138.stdout:3/938: fdatasync d1/d4f/d80/fe6 0 2024-05-17T21:53:10.796 INFO:tasks.workunit.client.1.smithi138.stdout:0/702: unlink d3/d10/d38/d3e/d59/f68 0 2024-05-17T21:53:10.798 INFO:tasks.workunit.client.1.smithi138.stdout:0/703: chown d3/d10/d38/d3e/d59/fec 51134924 1 2024-05-17T21:53:10.798 INFO:tasks.workunit.client.1.smithi138.stdout:7/744: creat d0/d18/d9c/dd4/ff1 x:0 0 0 2024-05-17T21:53:10.804 INFO:tasks.workunit.client.0.smithi107.stdout:2/800: sync 2024-05-17T21:53:10.808 INFO:tasks.workunit.client.1.smithi138.stdout:5/756: truncate d1/d2c/d63/db5/d19/f3a 1743932 0 2024-05-17T21:53:10.813 INFO:tasks.workunit.client.1.smithi138.stdout:3/939: truncate d1/fa5 217540 0 2024-05-17T21:53:10.815 INFO:tasks.workunit.client.0.smithi107.stdout:6/900: creat d9/d16/d4d/d85/d88/db1/f127 x:0 0 0 2024-05-17T21:53:10.819 INFO:tasks.workunit.client.1.smithi138.stdout:2/804: mknod d3/da0/dfe/c10d 0 2024-05-17T21:53:10.820 INFO:tasks.workunit.client.1.smithi138.stdout:0/704: creat d3/d10/d6d/d9f/fed x:0 0 0 2024-05-17T21:53:10.822 INFO:tasks.workunit.client.0.smithi107.stdout:5/967: dwrite d5/db/d16/d5b/f2f [0,4194304] 0 2024-05-17T21:53:10.822 INFO:tasks.workunit.client.1.smithi138.stdout:7/745: symlink d0/d4c/lf2 0 2024-05-17T21:53:10.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:10 smithi138 ceph-mon[35948]: pgmap v43: 65 pgs: 65 active+clean; 3.0 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 88 MiB/s rd, 202 MiB/s wr, 449 op/s 2024-05-17T21:53:10.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:10 smithi138 ceph-mon[35948]: from='mgr.14636 ' entity='mgr.smithi138.napnll' 2024-05-17T21:53:10.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:10 smithi138 ceph-mon[35948]: from='mgr.14636 ' entity='mgr.smithi138.napnll' 2024-05-17T21:53:10.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:10 smithi138 ceph-mon[35948]: from='mgr.14636 172.21.15.138:0/3120106723' entity='mgr.smithi138.napnll' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:53:10.832 INFO:tasks.workunit.client.1.smithi138.stdout:4/782: dwrite d0/d2f/d49/d60/d1f/f73 [0,4194304] 0 2024-05-17T21:53:10.835 INFO:tasks.workunit.client.1.smithi138.stdout:1/729: write d4/dd/d3b/f73 [1113291,81707] 0 2024-05-17T21:53:10.836 INFO:tasks.workunit.client.1.smithi138.stdout:4/783: truncate d0/d3/d1d/d38/d8a/dbd/ddc/ffe 633361 0 2024-05-17T21:53:10.851 INFO:tasks.workunit.client.0.smithi107.stdout:5/968: dread d5/db/d16/d54/d59/d92/fe5 [0,4194304] 0 2024-05-17T21:53:10.853 INFO:tasks.workunit.client.0.smithi107.stdout:5/969: dread - d5/db/d16/d2d/d40/d5a/d7c/d8e/f135 zero size 2024-05-17T21:53:10.868 INFO:tasks.workunit.client.0.smithi107.stdout:8/936: getdents dc/d21/d97/db7 0 2024-05-17T21:53:10.871 INFO:tasks.workunit.client.0.smithi107.stdout:7/909: fdatasync d5/df/d11/d1c/d2f/d53/fb2 0 2024-05-17T21:53:10.874 INFO:tasks.workunit.client.0.smithi107.stdout:8/937: rmdir dc/d21/d97/db7/dcd 39 2024-05-17T21:53:10.875 INFO:tasks.workunit.client.0.smithi107.stdout:6/901: dread d9/d16/d22/d5f/fdc [0,4194304] 0 2024-05-17T21:53:10.875 INFO:tasks.workunit.client.0.smithi107.stdout:8/938: readlink dc/d21/d27/d34/df5/l13f 0 2024-05-17T21:53:10.877 INFO:tasks.workunit.client.0.smithi107.stdout:2/801: mkdir d0/d9/d1c/d37/df8/d10a 0 2024-05-17T21:53:10.883 INFO:tasks.workunit.client.0.smithi107.stdout:5/970: getdents d5/db/d16/d2d/d123 0 2024-05-17T21:53:10.884 INFO:tasks.workunit.client.0.smithi107.stdout:9/969: sync 2024-05-17T21:53:10.884 INFO:tasks.workunit.client.1.smithi138.stdout:5/757: creat d1/d53/ff6 x:0 0 0 2024-05-17T21:53:10.887 INFO:tasks.workunit.client.0.smithi107.stdout:8/939: creat dc/d44/d4b/d52/dcb/d11c/f14a x:0 0 0 2024-05-17T21:53:10.888 INFO:tasks.workunit.client.1.smithi138.stdout:3/940: readlink d1/d62/dad/d72/l9d 0 2024-05-17T21:53:10.889 INFO:tasks.workunit.client.0.smithi107.stdout:2/802: creat d0/d26/d8e/dc4/f10b x:0 0 0 2024-05-17T21:53:10.891 INFO:tasks.workunit.client.0.smithi107.stdout:7/910: mknod d5/df/d11/d1c/d2f/d65/da7/d112/c135 0 2024-05-17T21:53:10.891 INFO:tasks.workunit.client.0.smithi107.stdout:7/911: readlink d5/df/d11/d1d/l74 0 2024-05-17T21:53:10.892 INFO:tasks.workunit.client.1.smithi138.stdout:2/805: creat d3/d81/d96/f10e x:0 0 0 2024-05-17T21:53:10.893 INFO:tasks.workunit.client.0.smithi107.stdout:5/971: creat d5/db/d16/d54/d59/d65/d48/f140 x:0 0 0 2024-05-17T21:53:10.895 INFO:tasks.workunit.client.1.smithi138.stdout:0/705: read d3/d10/f31 [2723083,55755] 0 2024-05-17T21:53:10.900 INFO:tasks.workunit.client.1.smithi138.stdout:8/678: creat d0/dc/d1a/d33/d4a/fc8 x:0 0 0 2024-05-17T21:53:10.901 INFO:tasks.workunit.client.0.smithi107.stdout:2/803: write d0/fa [3814102,80111] 0 2024-05-17T21:53:10.904 INFO:tasks.workunit.client.0.smithi107.stdout:7/912: mknod d5/df/d11/d1c/d31/d56/c136 0 2024-05-17T21:53:10.907 INFO:tasks.workunit.client.1.smithi138.stdout:4/784: rename d0/d3/db/da0/l10a to d0/d3/d1d/d5b/l10c 0 2024-05-17T21:53:10.912 INFO:tasks.workunit.client.0.smithi107.stdout:2/804: symlink d0/d9/d11/df/d5a/d6d/d82/l10c 0 2024-05-17T21:53:10.914 INFO:tasks.workunit.client.1.smithi138.stdout:5/758: write d1/d2c/d63/db5/d1b/f94 [258044,77116] 0 2024-05-17T21:53:10.917 INFO:tasks.workunit.client.1.smithi138.stdout:3/941: mkdir d1/d4f/d50/dbb/d13c 0 2024-05-17T21:53:10.918 INFO:tasks.workunit.client.0.smithi107.stdout:6/902: getdents d9/d16/d4d/d85/d96/d104 0 2024-05-17T21:53:10.924 INFO:tasks.workunit.client.1.smithi138.stdout:7/746: creat d0/d18/d64/d7c/da2/de0/ff3 x:0 0 0 2024-05-17T21:53:10.925 INFO:tasks.workunit.client.0.smithi107.stdout:7/913: dread d5/df/d78/f111 [0,4194304] 0 2024-05-17T21:53:10.926 INFO:tasks.workunit.client.0.smithi107.stdout:8/940: getdents dc/d21/d27/d2f/dba 0 2024-05-17T21:53:10.929 INFO:tasks.workunit.client.1.smithi138.stdout:4/785: creat d0/d3/d1d/d38/f10d x:0 0 0 2024-05-17T21:53:10.930 INFO:tasks.workunit.client.0.smithi107.stdout:6/903: write d9/f15 [1982146,86754] 0 2024-05-17T21:53:10.932 INFO:tasks.workunit.client.1.smithi138.stdout:5/759: creat d1/d53/d96/da0/ff7 x:0 0 0 2024-05-17T21:53:10.933 INFO:tasks.workunit.client.1.smithi138.stdout:7/747: dwrite d0/d18/d40/d41/d51/fce [0,4194304] 0 2024-05-17T21:53:10.933 INFO:tasks.workunit.client.1.smithi138.stdout:8/679: dwrite d0/f40 [0,4194304] 0 2024-05-17T21:53:10.934 INFO:tasks.workunit.client.0.smithi107.stdout:7/914: mkdir d5/df/d11/d1c/d31/d39/d137 0 2024-05-17T21:53:10.937 INFO:tasks.workunit.client.0.smithi107.stdout:8/941: fdatasync dc/d21/d97/d63/d71/fea 0 2024-05-17T21:53:10.937 INFO:tasks.workunit.client.1.smithi138.stdout:5/760: chown d1/d2c/d63/db5/d1b/dc5/d36/d68/f6f 0 1 2024-05-17T21:53:10.942 INFO:tasks.workunit.client.1.smithi138.stdout:0/706: unlink d3/d10/d38/d3e/c49 0 2024-05-17T21:53:10.946 INFO:tasks.workunit.client.1.smithi138.stdout:1/730: creat d4/dd/d3b/fed x:0 0 0 2024-05-17T21:53:10.947 INFO:tasks.workunit.client.1.smithi138.stdout:1/731: write d4/df/d3a/fe3 [21112,43624] 0 2024-05-17T21:53:10.948 INFO:tasks.workunit.client.1.smithi138.stdout:1/732: chown d4/d2e/da0/daa 805285 1 2024-05-17T21:53:10.950 INFO:tasks.workunit.client.1.smithi138.stdout:9/694: write d6/d7/d16/d29/f94 [127510,66204] 0 2024-05-17T21:53:10.950 INFO:tasks.workunit.client.1.smithi138.stdout:1/733: chown d4/df/d3a/d3c/d63/d6d/f87 4082900 1 2024-05-17T21:53:10.952 INFO:tasks.workunit.client.0.smithi107.stdout:8/942: creat dc/d21/daa/f14b x:0 0 0 2024-05-17T21:53:10.953 INFO:tasks.workunit.client.1.smithi138.stdout:8/680: truncate d0/d71/d92/f9d 1037814 0 2024-05-17T21:53:10.954 INFO:tasks.workunit.client.0.smithi107.stdout:2/805: link d0/d9/d11/d18/c1a d0/d9/d11/df/d5a/d6d/d101/c10d 0 2024-05-17T21:53:10.955 INFO:tasks.workunit.client.1.smithi138.stdout:3/942: mknod d1/d4f/d50/c13d 0 2024-05-17T21:53:10.959 INFO:tasks.workunit.client.1.smithi138.stdout:5/761: rename d1/d53/d88/db9/d39/d4e/d58/c7d to d1/d53/d88/de5/ded/cf8 0 2024-05-17T21:53:10.961 INFO:tasks.workunit.client.0.smithi107.stdout:8/943: mkdir dc/ddb/d14c 0 2024-05-17T21:53:10.964 INFO:tasks.workunit.client.0.smithi107.stdout:6/904: creat d9/d16/d4d/f128 x:0 0 0 2024-05-17T21:53:10.965 INFO:tasks.workunit.client.0.smithi107.stdout:9/970: dwrite d0/dd/d21/d29/d57/fe2 [0,4194304] 0 2024-05-17T21:53:10.968 INFO:tasks.workunit.client.1.smithi138.stdout:7/748: dread d0/d18/d40/d41/d50/f59 [0,4194304] 0 2024-05-17T21:53:10.969 INFO:tasks.workunit.client.1.smithi138.stdout:1/734: dread d4/d2e/f54 [0,4194304] 0 2024-05-17T21:53:10.970 INFO:tasks.workunit.client.1.smithi138.stdout:3/943: symlink d1/d3/d16/d3e/d71/d10d/l13e 0 2024-05-17T21:53:10.974 INFO:tasks.workunit.client.0.smithi107.stdout:8/944: unlink dc/d21/d97/d63/f99 0 2024-05-17T21:53:10.979 INFO:tasks.workunit.client.0.smithi107.stdout:9/971: symlink d0/dd/d21/d29/d57/d5f/d112/l138 0 2024-05-17T21:53:10.980 INFO:tasks.workunit.client.1.smithi138.stdout:5/762: stat d1/d2c/d63/db5/d19/f3a 0 2024-05-17T21:53:10.982 INFO:tasks.workunit.client.1.smithi138.stdout:2/806: dwrite d3/d21/d2f/d3e/d58/fb1 [0,4194304] 0 2024-05-17T21:53:10.987 INFO:tasks.workunit.client.1.smithi138.stdout:9/695: creat d6/d7/d16/d29/d48/d3b/fdb x:0 0 0 2024-05-17T21:53:10.988 INFO:tasks.workunit.client.0.smithi107.stdout:8/945: dread - dc/d21/d69/ffc zero size 2024-05-17T21:53:10.989 INFO:tasks.workunit.client.1.smithi138.stdout:7/749: readlink d0/d18/d40/d41/d50/l8a 0 2024-05-17T21:53:10.990 INFO:tasks.workunit.client.0.smithi107.stdout:8/946: write dc/d21/d27/d2f/dba/dee/fff [1702959,57778] 0 2024-05-17T21:53:10.991 INFO:tasks.workunit.client.0.smithi107.stdout:9/972: creat d0/dd/d2f/d4a/f139 x:0 0 0 2024-05-17T21:53:10.993 INFO:tasks.workunit.client.0.smithi107.stdout:7/915: link d5/d16/dd3/cf2 d5/df/d11/c138 0 2024-05-17T21:53:10.993 INFO:tasks.workunit.client.0.smithi107.stdout:9/973: truncate d0/dd/d21/d29/dca/f130 228807 0 2024-05-17T21:53:10.997 INFO:tasks.workunit.client.1.smithi138.stdout:0/707: rmdir d3/d3d/d48/dbc 0 2024-05-17T21:53:11.002 INFO:tasks.workunit.client.1.smithi138.stdout:5/763: mknod d1/d53/d96/da0/cf9 0 2024-05-17T21:53:11.005 INFO:tasks.workunit.client.1.smithi138.stdout:2/807: mkdir d3/d21/d2f/d3e/d58/d77/da4/df2/d10f 0 2024-05-17T21:53:11.008 INFO:tasks.workunit.client.0.smithi107.stdout:8/947: unlink dc/d21/d97/d63/d71/fbb 0 2024-05-17T21:53:11.009 INFO:tasks.workunit.client.1.smithi138.stdout:4/786: getdents d0/d3/d17/d21/d78/d9f 0 2024-05-17T21:53:11.013 INFO:tasks.workunit.client.1.smithi138.stdout:3/944: dread d1/d2e/fa1 [0,4194304] 0 2024-05-17T21:53:11.013 INFO:tasks.workunit.client.0.smithi107.stdout:9/974: creat d0/dd/d21/d29/dca/df3/f13a x:0 0 0 2024-05-17T21:53:11.014 INFO:tasks.workunit.client.1.smithi138.stdout:7/750: creat d0/d18/d40/d41/d51/ff4 x:0 0 0 2024-05-17T21:53:11.014 INFO:tasks.workunit.client.1.smithi138.stdout:7/751: stat d0/d18/cee 0 2024-05-17T21:53:11.017 INFO:tasks.workunit.client.1.smithi138.stdout:1/735: link d4/db/d29/d5d/f6f d4/d2e/d36/fee 0 2024-05-17T21:53:11.017 INFO:tasks.workunit.client.0.smithi107.stdout:9/975: dread d0/dd/d21/d29/d3e/d45/fec [0,4194304] 0 2024-05-17T21:53:11.017 INFO:tasks.workunit.client.0.smithi107.stdout:7/916: write d5/d16/d3b/ddf/d12e/db3/f103 [128503,98464] 0 2024-05-17T21:53:11.018 INFO:tasks.workunit.client.0.smithi107.stdout:9/976: chown d0/dd/d2f/f5b 239445 1 2024-05-17T21:53:11.019 INFO:tasks.workunit.client.0.smithi107.stdout:9/977: stat d0/d80/f105 0 2024-05-17T21:53:11.020 INFO:tasks.workunit.client.1.smithi138.stdout:0/708: creat d3/d10/d38/d70/d80/d94/fee x:0 0 0 2024-05-17T21:53:11.020 INFO:tasks.workunit.client.1.smithi138.stdout:7/752: dwrite d0/f91 [0,4194304] 0 2024-05-17T21:53:11.025 INFO:tasks.workunit.client.1.smithi138.stdout:2/808: rmdir d3/d21/d2f/d3e/d58/d77/da4/dc5 39 2024-05-17T21:53:11.028 INFO:tasks.workunit.client.1.smithi138.stdout:4/787: chown d0/d3/d17/c66 1364895 1 2024-05-17T21:53:11.029 INFO:tasks.workunit.client.1.smithi138.stdout:4/788: readlink d0/d2f/d49/d60/d1f/d32/d71/l76 0 2024-05-17T21:53:11.030 INFO:tasks.workunit.client.0.smithi107.stdout:9/978: creat d0/dd/d21/d29/d57/d5f/dc6/f13b x:0 0 0 2024-05-17T21:53:11.030 INFO:tasks.workunit.client.0.smithi107.stdout:9/979: stat d0/dd/d1f/f82 0 2024-05-17T21:53:11.033 INFO:tasks.workunit.client.0.smithi107.stdout:9/980: dread d0/d80/fcf [0,4194304] 0 2024-05-17T21:53:11.033 INFO:tasks.workunit.client.1.smithi138.stdout:3/945: rename d1/d4f/d80/de7 to d1/d62/db0/d13f 0 2024-05-17T21:53:11.034 INFO:tasks.workunit.client.0.smithi107.stdout:8/948: getdents dc/d44/d4b/d121/de5 0 2024-05-17T21:53:11.035 INFO:tasks.workunit.client.1.smithi138.stdout:3/946: chown d1/d3/d8d/d92/f96 253120090 1 2024-05-17T21:53:11.037 INFO:tasks.workunit.client.1.smithi138.stdout:3/947: truncate d1/d3/d16/d3e/d71/d101/f115 1323080 0 2024-05-17T21:53:11.037 INFO:tasks.workunit.client.1.smithi138.stdout:1/736: creat d4/df/d3a/d3c/d63/d6d/fef x:0 0 0 2024-05-17T21:53:11.039 INFO:tasks.workunit.client.1.smithi138.stdout:1/737: fdatasync d4/db/d29/d5d/f86 0 2024-05-17T21:53:11.041 INFO:tasks.workunit.client.1.smithi138.stdout:9/696: write f5 [1359451,72601] 0 2024-05-17T21:53:11.048 INFO:tasks.workunit.client.1.smithi138.stdout:9/697: dwrite d6/d7/d16/d29/da6/fae [0,4194304] 0 2024-05-17T21:53:11.049 INFO:tasks.workunit.client.1.smithi138.stdout:9/698: chown d6/d7/f99 34838990 1 2024-05-17T21:53:11.049 INFO:tasks.workunit.client.1.smithi138.stdout:7/753: rmdir d0/d18 39 2024-05-17T21:53:11.050 INFO:tasks.workunit.client.0.smithi107.stdout:7/917: dread d5/d16/dd3/fdd [0,4194304] 0 2024-05-17T21:53:11.059 INFO:tasks.workunit.client.1.smithi138.stdout:5/764: creat d1/d60/ffa x:0 0 0 2024-05-17T21:53:11.061 INFO:tasks.workunit.client.0.smithi107.stdout:7/918: write d5/d16/d3b/d4f/fe2 [320516,56153] 0 2024-05-17T21:53:11.062 INFO:tasks.workunit.client.1.smithi138.stdout:1/738: write d4/db/f21 [2432350,127334] 0 2024-05-17T21:53:11.063 INFO:tasks.workunit.client.0.smithi107.stdout:9/981: dwrite d0/dd/d1b/f65 [0,4194304] 0 2024-05-17T21:53:11.063 INFO:tasks.workunit.client.1.smithi138.stdout:5/765: fsync d1/d53/f80 0 2024-05-17T21:53:11.064 INFO:tasks.workunit.client.1.smithi138.stdout:1/739: chown d4/f9 19940941 1 2024-05-17T21:53:11.066 INFO:tasks.workunit.client.1.smithi138.stdout:2/809: dwrite d3/d21/f31 [4194304,4194304] 0 2024-05-17T21:53:11.069 INFO:tasks.workunit.client.0.smithi107.stdout:9/982: rename d0/dd/d1b/l109 to d0/d80/l13c 0 2024-05-17T21:53:11.073 INFO:tasks.workunit.client.1.smithi138.stdout:3/948: creat d1/d62/dad/d5c/f140 x:0 0 0 2024-05-17T21:53:11.073 INFO:tasks.workunit.client.1.smithi138.stdout:9/699: link d6/d7/d16/d29/d48/d6e/d74/d75/la8 d6/d7/d16/d29/d48/d67/ldc 0 2024-05-17T21:53:11.076 INFO:tasks.workunit.client.0.smithi107.stdout:9/983: unlink d0/dd/d1f/f82 0 2024-05-17T21:53:11.083 INFO:tasks.workunit.client.0.smithi107.stdout:7/919: dwrite d5/df/d11/d1d/d33/f64 [0,4194304] 0 2024-05-17T21:53:11.084 INFO:tasks.workunit.client.0.smithi107.stdout:9/984: mkdir d0/dd/d21/d13d 0 2024-05-17T21:53:11.085 INFO:tasks.workunit.client.1.smithi138.stdout:9/700: rmdir d6/d7/d16/d29/d6c/d9c/dc1/d7e 39 2024-05-17T21:53:11.086 INFO:tasks.workunit.client.1.smithi138.stdout:7/754: dwrite d0/d18/d40/d41/d51/f69 [0,4194304] 0 2024-05-17T21:53:11.087 INFO:tasks.workunit.client.0.smithi107.stdout:7/920: rmdir d5/d16/d3b/ddf 39 2024-05-17T21:53:11.088 INFO:tasks.workunit.client.0.smithi107.stdout:7/921: fsync d5/df/d11/d1c/d31/f48 0 2024-05-17T21:53:11.088 INFO:tasks.workunit.client.0.smithi107.stdout:9/985: mknod d0/dd/d69/dee/c13e 0 2024-05-17T21:53:11.090 INFO:tasks.workunit.client.0.smithi107.stdout:9/986: fsync d0/dd/d69/dee/d116/f11d 0 2024-05-17T21:53:11.091 INFO:tasks.workunit.client.0.smithi107.stdout:9/987: chown d0/dd/d69/f9e 5446759 1 2024-05-17T21:53:11.095 INFO:tasks.workunit.client.1.smithi138.stdout:3/949: getdents d1/dc0/ddd 0 2024-05-17T21:53:11.097 INFO:tasks.workunit.client.0.smithi107.stdout:5/972: sync 2024-05-17T21:53:11.097 INFO:tasks.workunit.client.1.smithi138.stdout:9/701: getdents d6/d7/dc0 0 2024-05-17T21:53:11.097 INFO:tasks.workunit.client.1.smithi138.stdout:3/950: truncate d1/d3/d16/f29 1726667 0 2024-05-17T21:53:11.100 INFO:tasks.workunit.client.1.smithi138.stdout:6/791: sync 2024-05-17T21:53:11.103 INFO:tasks.workunit.client.1.smithi138.stdout:5/766: write d1/d53/d88/de5/ded/dc2/fb6 [550536,63181] 0 2024-05-17T21:53:11.104 INFO:tasks.workunit.client.0.smithi107.stdout:5/973: creat d5/db/d16/d2d/d40/d5a/d7c/d103/f141 x:0 0 0 2024-05-17T21:53:11.105 INFO:tasks.workunit.client.1.smithi138.stdout:0/709: dread d3/f5e [0,4194304] 0 2024-05-17T21:53:11.105 INFO:tasks.workunit.client.1.smithi138.stdout:7/755: creat d0/d18/ff5 x:0 0 0 2024-05-17T21:53:11.106 INFO:tasks.workunit.client.0.smithi107.stdout:5/974: mknod d5/db/dc/dc5/c142 0 2024-05-17T21:53:11.107 INFO:tasks.workunit.client.1.smithi138.stdout:1/740: dwrite d4/db/f4b [0,4194304] 0 2024-05-17T21:53:11.110 INFO:tasks.workunit.client.1.smithi138.stdout:0/710: truncate d3/d10/d6d/d81/fd3 968434 0 2024-05-17T21:53:11.110 INFO:tasks.workunit.client.1.smithi138.stdout:5/767: dread - d1/d2c/d63/db5/d1b/dc5/d36/f98 zero size 2024-05-17T21:53:11.111 INFO:tasks.workunit.client.1.smithi138.stdout:3/951: rename d1/d3/d49/ddb to d1/d3/d16/d3e/d141 0 2024-05-17T21:53:11.113 INFO:tasks.workunit.client.1.smithi138.stdout:7/756: mkdir d0/d4c/df6 0 2024-05-17T21:53:11.114 INFO:tasks.workunit.client.1.smithi138.stdout:3/952: stat d1/d3/d49/fff 0 2024-05-17T21:53:11.115 INFO:tasks.workunit.client.1.smithi138.stdout:2/810: dread d3/f35 [0,4194304] 0 2024-05-17T21:53:11.115 INFO:tasks.workunit.client.1.smithi138.stdout:5/768: chown d1/d53/d88/lc7 292 1 2024-05-17T21:53:11.119 INFO:tasks.workunit.client.1.smithi138.stdout:3/953: creat d1/d4f/d80/f142 x:0 0 0 2024-05-17T21:53:11.119 INFO:tasks.workunit.client.1.smithi138.stdout:1/741: dwrite d4/d2e/da0/daa/fad [0,4194304] 0 2024-05-17T21:53:11.120 INFO:tasks.workunit.client.1.smithi138.stdout:6/792: link d7/d69/db5/dc8/d43/d3f/c40 d7/d69/db5/dc8/d34/da6/dfa/d101/c10f 0 2024-05-17T21:53:11.121 INFO:tasks.workunit.client.1.smithi138.stdout:5/769: truncate d1/d2c/dbf/fd3 3887092 0 2024-05-17T21:53:11.124 INFO:tasks.workunit.client.1.smithi138.stdout:5/770: chown d1/d2c/d63/db5/d1b/dc5/l40 21046650 1 2024-05-17T21:53:11.127 INFO:tasks.workunit.client.1.smithi138.stdout:1/742: link d4/dd/d3b/f73 d4/df/d3a/ff0 0 2024-05-17T21:53:11.127 INFO:tasks.workunit.client.1.smithi138.stdout:1/743: readlink d4/dd/l7a 0 2024-05-17T21:53:11.130 INFO:tasks.workunit.client.1.smithi138.stdout:1/744: mknod d4/dd/d3b/d6c/dc1/cf1 0 2024-05-17T21:53:11.130 INFO:tasks.workunit.client.1.smithi138.stdout:1/745: write d4/dd/db4/dba/fea [915682,118649] 0 2024-05-17T21:53:11.132 INFO:tasks.workunit.client.1.smithi138.stdout:1/746: symlink d4/df/d3a/dc4/lf2 0 2024-05-17T21:53:11.133 INFO:tasks.workunit.client.1.smithi138.stdout:1/747: readlink d4/df/l23 0 2024-05-17T21:53:11.135 INFO:tasks.workunit.client.1.smithi138.stdout:1/748: mkdir d4/df/d35/df3 0 2024-05-17T21:53:11.138 INFO:tasks.workunit.client.1.smithi138.stdout:1/749: read d4/db/d60/dcc/fe7 [859295,86158] 0 2024-05-17T21:53:11.140 INFO:tasks.workunit.client.0.smithi107.stdout:9/988: write d0/dd/d2f/d81/dc3/f104 [1042319,125271] 0 2024-05-17T21:53:11.146 INFO:tasks.workunit.client.0.smithi107.stdout:7/922: dwrite d5/df/d11/d1d/d33/f99 [0,4194304] 0 2024-05-17T21:53:11.146 INFO:tasks.workunit.client.1.smithi138.stdout:9/702: truncate d6/d7/d16/d29/d48/d6f/f71 794946 0 2024-05-17T21:53:11.146 INFO:tasks.workunit.client.1.smithi138.stdout:1/750: dwrite d4/fc [0,4194304] 0 2024-05-17T21:53:11.148 INFO:tasks.workunit.client.1.smithi138.stdout:0/711: dread d3/d10/d6d/d81/fbd [0,4194304] 0 2024-05-17T21:53:11.148 INFO:tasks.workunit.client.0.smithi107.stdout:5/975: truncate d5/db/d16/d2d/f4a 911975 0 2024-05-17T21:53:11.149 INFO:tasks.workunit.client.1.smithi138.stdout:9/703: mkdir d6/d7/d16/d29/d48/d6e/d74/d75/d76/d8e/ddd 0 2024-05-17T21:53:11.150 INFO:tasks.workunit.client.1.smithi138.stdout:0/712: dread d3/d10/d6d/d81/fd3 [0,4194304] 0 2024-05-17T21:53:11.150 INFO:tasks.workunit.client.1.smithi138.stdout:9/704: chown d6/d7/d16/d29/d48/d6f 871660344 1 2024-05-17T21:53:11.160 INFO:tasks.workunit.client.1.smithi138.stdout:2/811: dwrite f2 [0,4194304] 0 2024-05-17T21:53:11.161 INFO:tasks.workunit.client.1.smithi138.stdout:1/751: link d4/d93/faf d4/dd/d3b/da7/ff4 0 2024-05-17T21:53:11.163 INFO:tasks.workunit.client.1.smithi138.stdout:0/713: chown d3/d3d/f58 30855112 1 2024-05-17T21:53:11.163 INFO:tasks.workunit.client.1.smithi138.stdout:1/752: chown d4/df/d4e/f84 1765002 1 2024-05-17T21:53:11.164 INFO:tasks.workunit.client.1.smithi138.stdout:6/793: dwrite d7/d8/f96 [0,4194304] 0 2024-05-17T21:53:11.168 INFO:tasks.workunit.client.0.smithi107.stdout:5/976: getdents d5/db/d16/d2d/d40/d5a/d7c 0 2024-05-17T21:53:11.170 INFO:tasks.workunit.client.1.smithi138.stdout:2/812: mkdir d3/d21/d2f/d3e/d55/d99/d110 0 2024-05-17T21:53:11.171 INFO:tasks.workunit.client.1.smithi138.stdout:2/813: chown d3/d21/d2f/d3e/d55/d99/d93/cc2 28508774 1 2024-05-17T21:53:11.173 INFO:tasks.workunit.client.0.smithi107.stdout:9/989: write d0/dd/d69/fde [286946,68165] 0 2024-05-17T21:53:11.173 INFO:tasks.workunit.client.0.smithi107.stdout:5/977: rmdir d5/db/d16/d2a/d73/dd0 39 2024-05-17T21:53:11.174 INFO:tasks.workunit.client.1.smithi138.stdout:0/714: mknod d3/d3d/d48/cef 0 2024-05-17T21:53:11.175 INFO:tasks.workunit.client.0.smithi107.stdout:9/990: write d0/dd/d2f/d87/fb6 [175876,124581] 0 2024-05-17T21:53:11.176 INFO:tasks.workunit.client.1.smithi138.stdout:1/753: mknod d4/db/d29/d89/cf5 0 2024-05-17T21:53:11.177 INFO:tasks.workunit.client.1.smithi138.stdout:6/794: chown d7/d69/db5/dc8/d34/da6/l2c 464 1 2024-05-17T21:53:11.179 INFO:tasks.workunit.client.0.smithi107.stdout:7/923: dread d5/df/d11/d1d/f10a [0,4194304] 0 2024-05-17T21:53:11.180 INFO:tasks.workunit.client.0.smithi107.stdout:2/806: sync 2024-05-17T21:53:11.181 INFO:tasks.workunit.client.1.smithi138.stdout:1/754: creat d4/dd/d3b/d9f/da9/ff6 x:0 0 0 2024-05-17T21:53:11.181 INFO:tasks.workunit.client.0.smithi107.stdout:7/924: chown d5/df/d11/d1c/d2f/f104 295803398 1 2024-05-17T21:53:11.181 INFO:tasks.workunit.client.0.smithi107.stdout:9/991: symlink d0/dd/d21/d29/d3e/l13f 0 2024-05-17T21:53:11.182 INFO:tasks.workunit.client.1.smithi138.stdout:6/795: rmdir d7/d69/db5/dc8 39 2024-05-17T21:53:11.182 INFO:tasks.workunit.client.0.smithi107.stdout:5/978: dwrite d5/db/d16/d54/d59/d65/fac [0,4194304] 0 2024-05-17T21:53:11.184 INFO:tasks.workunit.client.1.smithi138.stdout:0/715: read d3/d10/d38/d3e/d59/f9d [3292646,100150] 0 2024-05-17T21:53:11.184 INFO:tasks.workunit.client.0.smithi107.stdout:7/925: dread - d5/df/d11/d1d/dc4/f12c zero size 2024-05-17T21:53:11.185 INFO:tasks.workunit.client.1.smithi138.stdout:2/814: creat d3/d21/d2f/d3e/d55/f111 x:0 0 0 2024-05-17T21:53:11.187 INFO:tasks.workunit.client.1.smithi138.stdout:6/796: fsync d7/d69/db5/dc8/d43/d5f/d6a/fd3 0 2024-05-17T21:53:11.187 INFO:tasks.workunit.client.0.smithi107.stdout:5/979: creat d5/db/d16/d2d/d40/d5a/d7c/dee/f143 x:0 0 0 2024-05-17T21:53:11.188 INFO:tasks.workunit.client.1.smithi138.stdout:1/755: dread d4/fa [0,4194304] 0 2024-05-17T21:53:11.189 INFO:tasks.workunit.client.0.smithi107.stdout:9/992: mknod d0/dd/d21/d29/c140 0 2024-05-17T21:53:11.190 INFO:tasks.workunit.client.1.smithi138.stdout:1/756: write d4/d2e/da0/daa/fad [3289148,97586] 0 2024-05-17T21:53:11.193 INFO:tasks.workunit.client.0.smithi107.stdout:7/926: rename d5/df/d11/d1c/d31/d39/d92/lb5 to d5/df/dd7/d118/l139 0 2024-05-17T21:53:11.193 INFO:tasks.workunit.client.0.smithi107.stdout:2/807: dread d0/d9/d1c/d37/f50 [0,4194304] 0 2024-05-17T21:53:11.194 INFO:tasks.workunit.client.0.smithi107.stdout:5/980: creat d5/db/d16/d2a/f144 x:0 0 0 2024-05-17T21:53:11.197 INFO:tasks.workunit.client.0.smithi107.stdout:2/808: dread - d0/d9/d11/df/d5a/d6d/ffc zero size 2024-05-17T21:53:11.198 INFO:tasks.workunit.client.0.smithi107.stdout:2/809: read d0/d9/d1c/d37/f50 [723236,80581] 0 2024-05-17T21:53:11.202 INFO:tasks.workunit.client.1.smithi138.stdout:2/815: rename d3/d21/d2f/d3e/d55/d99/d93 to d3/d21/d2f/d3e/d55/d99/d112 0 2024-05-17T21:53:11.202 INFO:tasks.workunit.client.1.smithi138.stdout:2/816: fdatasync d3/d21/d2f/d3e/d58/d77/da4/df2/dd0/fb5 0 2024-05-17T21:53:11.206 INFO:tasks.workunit.client.0.smithi107.stdout:2/810: dwrite d0/d9/d11/fda [0,4194304] 0 2024-05-17T21:53:11.206 INFO:tasks.workunit.client.1.smithi138.stdout:2/817: chown d3/d21/d2f/d3e/d55/d99/d84/lab 5829 1 2024-05-17T21:53:11.207 INFO:tasks.workunit.client.0.smithi107.stdout:2/811: stat d0/d9/d11/d15/lfe 0 2024-05-17T21:53:11.207 INFO:tasks.workunit.client.0.smithi107.stdout:5/981: read d5/db/d16/f81 [128413,63511] 0 2024-05-17T21:53:11.209 INFO:tasks.workunit.client.1.smithi138.stdout:9/705: dwrite d6/d7/d16/d29/d48/f7d [0,4194304] 0 2024-05-17T21:53:11.212 INFO:tasks.workunit.client.0.smithi107.stdout:9/993: creat d0/dd/d21/d29/f141 x:0 0 0 2024-05-17T21:53:11.213 INFO:tasks.workunit.client.0.smithi107.stdout:9/994: read d0/dd/d21/d29/d57/fe2 [846302,102190] 0 2024-05-17T21:53:11.214 INFO:tasks.workunit.client.0.smithi107.stdout:9/995: fsync d0/dd/d69/dee/d116/f11d 0 2024-05-17T21:53:11.219 INFO:tasks.workunit.client.0.smithi107.stdout:7/927: link d5/d16/d3b/d45/cbd d5/df/d11/d1d/dc4/c13a 0 2024-05-17T21:53:11.220 INFO:tasks.workunit.client.1.smithi138.stdout:9/706: creat d6/d7/d16/dc2/fde x:0 0 0 2024-05-17T21:53:11.221 INFO:tasks.workunit.client.1.smithi138.stdout:9/707: write f5 [439687,2626] 0 2024-05-17T21:53:11.223 INFO:tasks.workunit.client.0.smithi107.stdout:7/928: mkdir d5/df/d11/d1c/d31/d56/d13b 0 2024-05-17T21:53:11.224 INFO:tasks.workunit.client.0.smithi107.stdout:9/996: mkdir d0/dd/d21/d29/d142 0 2024-05-17T21:53:11.225 INFO:tasks.workunit.client.0.smithi107.stdout:2/812: creat d0/d9/d1c/f10e x:0 0 0 2024-05-17T21:53:11.226 INFO:tasks.workunit.client.1.smithi138.stdout:2/818: getdents d3/d21 0 2024-05-17T21:53:11.226 INFO:tasks.workunit.client.0.smithi107.stdout:7/929: dread - d5/df/d11/d1c/d2f/f104 zero size 2024-05-17T21:53:11.229 INFO:tasks.workunit.client.1.smithi138.stdout:2/819: dread d3/d21/d2f/d3e/d55/d99/d84/d9f/fcb [0,4194304] 0 2024-05-17T21:53:11.230 INFO:tasks.workunit.client.0.smithi107.stdout:7/930: mknod d5/d16/d3b/d4f/c13c 0 2024-05-17T21:53:11.233 INFO:tasks.workunit.client.0.smithi107.stdout:7/931: readlink d5/df/d11/d1d/l80 0 2024-05-17T21:53:11.237 INFO:tasks.workunit.client.1.smithi138.stdout:9/708: getdents d6/d7/d16 0 2024-05-17T21:53:11.238 INFO:tasks.workunit.client.1.smithi138.stdout:9/709: fsync d6/d7/d16/d29/da6/fae 0 2024-05-17T21:53:11.240 INFO:tasks.workunit.client.1.smithi138.stdout:5/771: dread d1/d53/d88/de5/ded/dc2/fb6 [0,4194304] 0 2024-05-17T21:53:11.242 INFO:tasks.workunit.client.1.smithi138.stdout:5/772: mknod d1/d2c/d63/db5/d1b/cfb 0 2024-05-17T21:53:11.245 INFO:tasks.workunit.client.1.smithi138.stdout:5/773: link d1/d2c/d63/db5/f72 d1/d53/ffc 0 2024-05-17T21:53:11.254 INFO:tasks.workunit.client.1.smithi138.stdout:0/716: write d3/d10/d6d/d9f/fe6 [3251646,101901] 0 2024-05-17T21:53:11.254 INFO:tasks.workunit.client.1.smithi138.stdout:6/797: write d7/d69/db5/dc8/d43/f64 [353912,113610] 0 2024-05-17T21:53:11.255 INFO:tasks.workunit.client.0.smithi107.stdout:9/997: dread d0/dd/d21/dcd/fe5 [0,4194304] 0 2024-05-17T21:53:11.259 INFO:tasks.workunit.client.0.smithi107.stdout:9/998: fsync d0/dd/d21/daf/fe1 0 2024-05-17T21:53:11.259 INFO:tasks.workunit.client.1.smithi138.stdout:1/757: dwrite d4/df/d35/f37 [0,4194304] 0 2024-05-17T21:53:11.266 INFO:tasks.workunit.client.1.smithi138.stdout:6/798: rename d7/d69/db5/dc8/l38 to d7/da0/dbe/l110 0 2024-05-17T21:53:11.267 INFO:tasks.workunit.client.0.smithi107.stdout:9/999: chown d0/dd/d21/d29/d57/l73 0 1 2024-05-17T21:53:11.268 INFO:tasks.workunit.client.1.smithi138.stdout:6/799: read d7/d8/f50 [1116401,71236] 0 2024-05-17T21:53:11.269 INFO:tasks.workunit.client.1.smithi138.stdout:1/758: fsync d4/f3f 0 2024-05-17T21:53:11.269 INFO:tasks.workunit.client.1.smithi138.stdout:6/800: chown d7/d69/db5/dc8/d43/d3f/d79/c5e 135 1 2024-05-17T21:53:11.270 INFO:tasks.workunit.client.1.smithi138.stdout:1/759: write d4/df/d4e/f55 [722237,68283] 0 2024-05-17T21:53:11.271 INFO:tasks.workunit.client.1.smithi138.stdout:1/760: chown d4/dd/db4/fe4 679 1 2024-05-17T21:53:11.274 INFO:tasks.workunit.client.1.smithi138.stdout:6/801: symlink d7/d69/db5/dc8/d43/d5f/da4/db0/l111 0 2024-05-17T21:53:11.278 INFO:tasks.workunit.client.1.smithi138.stdout:1/761: dwrite d4/df/f2c [8388608,4194304] 0 2024-05-17T21:53:11.282 INFO:tasks.workunit.client.1.smithi138.stdout:1/762: dread f2 [0,4194304] 0 2024-05-17T21:53:11.283 INFO:tasks.workunit.client.1.smithi138.stdout:1/763: write d4/db/d29/d5d/f86 [4916419,24444] 0 2024-05-17T21:53:11.285 INFO:tasks.workunit.client.1.smithi138.stdout:5/774: write d1/d2c/d63/db5/d19/fa3 [4541051,97204] 0 2024-05-17T21:53:11.289 INFO:tasks.workunit.client.1.smithi138.stdout:2/820: dwrite d3/d21/d2f/d3e/fad [0,4194304] 0 2024-05-17T21:53:11.290 INFO:tasks.workunit.client.1.smithi138.stdout:5/775: mknod d1/d53/d96/cfd 0 2024-05-17T21:53:11.290 INFO:tasks.workunit.client.0.smithi107.stdout:2/813: dwrite d0/d9/d11/d18/dad/fb7 [0,4194304] 0 2024-05-17T21:53:11.294 INFO:tasks.workunit.client.1.smithi138.stdout:2/821: creat d3/d21/f113 x:0 0 0 2024-05-17T21:53:11.294 INFO:tasks.workunit.client.0.smithi107.stdout:7/932: dwrite d5/df/d11/d1d/d33/d82/ffc [0,4194304] 0 2024-05-17T21:53:11.296 INFO:tasks.workunit.client.1.smithi138.stdout:5/776: rename d1/d2c/d63/db5/ca9 to d1/d53/db1/cfe 0 2024-05-17T21:53:11.297 INFO:tasks.workunit.client.1.smithi138.stdout:0/717: write d3/d10/d38/d70/d80/d94/fcd [297481,113253] 0 2024-05-17T21:53:11.299 INFO:tasks.workunit.client.0.smithi107.stdout:2/814: rename d0/d9/d11/df/f35 to d0/d9/d1c/d37/f10f 0 2024-05-17T21:53:11.300 INFO:tasks.workunit.client.1.smithi138.stdout:2/822: unlink d3/l51 0 2024-05-17T21:53:11.301 INFO:tasks.workunit.client.1.smithi138.stdout:0/718: dread d3/d10/d6d/d81/fbd [0,4194304] 0 2024-05-17T21:53:11.302 INFO:tasks.workunit.client.1.smithi138.stdout:2/823: readlink d3/d21/d2f/d3e/d55/d99/dd2/lda 0 2024-05-17T21:53:11.303 INFO:tasks.workunit.client.1.smithi138.stdout:5/777: creat d1/d2c/d63/db5/d19/fff x:0 0 0 2024-05-17T21:53:11.306 INFO:tasks.workunit.client.0.smithi107.stdout:6/905: sync 2024-05-17T21:53:11.306 INFO:tasks.workunit.client.0.smithi107.stdout:8/949: sync 2024-05-17T21:53:11.306 INFO:tasks.workunit.client.0.smithi107.stdout:2/815: creat d0/d9/d1c/d37/f110 x:0 0 0 2024-05-17T21:53:11.308 INFO:tasks.workunit.client.0.smithi107.stdout:7/933: unlink d5/d16/d3b/ddf/d12e/dc1/f106 0 2024-05-17T21:53:11.309 INFO:tasks.workunit.client.0.smithi107.stdout:6/906: chown d9/d16/d22/d5f/d6e/dd6/d11f/lda 1 1 2024-05-17T21:53:11.310 INFO:tasks.workunit.client.0.smithi107.stdout:6/907: stat d9/d16/d4d/d85/d88/l8e 0 2024-05-17T21:53:11.311 INFO:tasks.workunit.client.0.smithi107.stdout:8/950: creat dc/d21/d27/d83/f14d x:0 0 0 2024-05-17T21:53:11.312 INFO:tasks.workunit.client.0.smithi107.stdout:6/908: dread - d9/f8d zero size 2024-05-17T21:53:11.317 INFO:tasks.workunit.client.0.smithi107.stdout:6/909: unlink d9/d10e/d66/dd5/l100 0 2024-05-17T21:53:11.318 INFO:tasks.workunit.client.0.smithi107.stdout:7/934: link d5/d16/d3b/ddf/d12e/db3/fed d5/df/d11/d1c/d31/d39/dc0/f13d 0 2024-05-17T21:53:11.320 INFO:tasks.workunit.client.0.smithi107.stdout:7/935: mknod d5/df/d11/d1c/d31/d39/d137/c13e 0 2024-05-17T21:53:11.323 INFO:tasks.workunit.client.0.smithi107.stdout:6/910: link d9/f1f d9/d10e/d7b/d114/f129 0 2024-05-17T21:53:11.324 INFO:tasks.workunit.client.0.smithi107.stdout:7/936: creat d5/df/d11/df6/f13f x:0 0 0 2024-05-17T21:53:11.326 INFO:tasks.workunit.client.0.smithi107.stdout:6/911: mknod d9/d16/d22/d5f/d6e/dd6/c12a 0 2024-05-17T21:53:11.328 INFO:tasks.workunit.client.0.smithi107.stdout:7/937: getdents d5/df/d11/d2a 0 2024-05-17T21:53:11.330 INFO:tasks.workunit.client.0.smithi107.stdout:6/912: symlink d9/d16/d4d/d85/d96/d117/d119/l12b 0 2024-05-17T21:53:11.331 INFO:tasks.workunit.client.0.smithi107.stdout:7/938: dread d5/f42 [0,4194304] 0 2024-05-17T21:53:11.334 INFO:tasks.workunit.client.0.smithi107.stdout:6/913: write d9/d16/d4d/db8/dc9/dd0/ffc [171831,539] 0 2024-05-17T21:53:11.334 INFO:tasks.workunit.client.0.smithi107.stdout:7/939: mkdir d5/df/d11/d140 0 2024-05-17T21:53:11.335 INFO:tasks.workunit.client.1.smithi138.stdout:1/764: write d4/d2e/f9d [3406330,78148] 0 2024-05-17T21:53:11.337 INFO:tasks.workunit.client.1.smithi138.stdout:1/765: write d4/d2e/da0/fbf [250694,35169] 0 2024-05-17T21:53:11.337 INFO:tasks.workunit.client.0.smithi107.stdout:5/982: sync 2024-05-17T21:53:11.339 INFO:tasks.workunit.client.0.smithi107.stdout:6/914: creat d9/d10e/f12c x:0 0 0 2024-05-17T21:53:11.340 INFO:tasks.workunit.client.0.smithi107.stdout:6/915: chown d9/d16/d4d/d85/l94 3 1 2024-05-17T21:53:11.344 INFO:tasks.workunit.client.1.smithi138.stdout:1/766: dwrite d4/df/d3a/d3c/f99 [4194304,4194304] 0 2024-05-17T21:53:11.344 INFO:tasks.workunit.client.0.smithi107.stdout:5/983: readlink d5/db/d16/d2a/d73/dd0/le6 0 2024-05-17T21:53:11.349 INFO:tasks.workunit.client.0.smithi107.stdout:6/916: getdents d9 0 2024-05-17T21:53:11.349 INFO:tasks.workunit.client.0.smithi107.stdout:2/816: truncate d0/d9/f77 4083272 0 2024-05-17T21:53:11.379 INFO:tasks.workunit.client.1.smithi138.stdout:8/681: sync 2024-05-17T21:53:11.381 INFO:tasks.workunit.client.1.smithi138.stdout:0/719: dread d3/f7 [4194304,4194304] 0 2024-05-17T21:53:11.381 INFO:tasks.workunit.client.1.smithi138.stdout:8/682: chown d0/d71/f9f 27 1 2024-05-17T21:53:11.382 INFO:tasks.workunit.client.1.smithi138.stdout:2/824: dread d3/d21/d2f/f36 [0,4194304] 0 2024-05-17T21:53:11.382 INFO:tasks.workunit.client.0.smithi107.stdout:2/817: write d0/d9/d11/d18/dad/fc3 [140121,59585] 0 2024-05-17T21:53:11.385 INFO:tasks.workunit.client.1.smithi138.stdout:2/825: dread - d3/d81/d96/da8/fe0 zero size 2024-05-17T21:53:11.385 INFO:tasks.workunit.client.0.smithi107.stdout:6/917: readlink d9/d16/d22/l57 0 2024-05-17T21:53:11.385 INFO:tasks.workunit.client.1.smithi138.stdout:1/767: truncate d4/df/fa8 207639 0 2024-05-17T21:53:11.387 INFO:tasks.workunit.client.0.smithi107.stdout:8/951: dwrite dc/d21/d97/feb [0,4194304] 0 2024-05-17T21:53:11.388 INFO:tasks.workunit.client.1.smithi138.stdout:5/778: dwrite d1/d2c/d63/db5/d1b/f54 [0,4194304] 0 2024-05-17T21:53:11.389 INFO:tasks.workunit.client.1.smithi138.stdout:1/768: dread - d4/dd/d3b/da7/ff4 zero size 2024-05-17T21:53:11.389 INFO:tasks.workunit.client.1.smithi138.stdout:0/720: truncate d3/d10/d38/d70/d80/fc2 324803 0 2024-05-17T21:53:11.390 INFO:tasks.workunit.client.1.smithi138.stdout:1/769: stat d4/df/d3a/fbe 0 2024-05-17T21:53:11.394 INFO:tasks.workunit.client.0.smithi107.stdout:6/918: rmdir d9/d16/d22/d5f 39 2024-05-17T21:53:11.397 INFO:tasks.workunit.client.0.smithi107.stdout:5/984: dwrite d5/db/d39/f70 [0,4194304] 0 2024-05-17T21:53:11.398 INFO:tasks.workunit.client.1.smithi138.stdout:1/770: dread d4/df/d4e/f84 [0,4194304] 0 2024-05-17T21:53:11.399 INFO:tasks.workunit.client.0.smithi107.stdout:8/952: creat dc/d44/d4b/d52/d70/d76/f14e x:0 0 0 2024-05-17T21:53:11.400 INFO:tasks.workunit.client.1.smithi138.stdout:2/826: dwrite d3/d21/d2f/d3e/d55/d99/f8f [4194304,4194304] 0 2024-05-17T21:53:11.401 INFO:tasks.workunit.client.1.smithi138.stdout:2/827: stat d3/d21/d2f/f33 0 2024-05-17T21:53:11.401 INFO:tasks.workunit.client.0.smithi107.stdout:8/953: write dc/d44/fbe [405798,85552] 0 2024-05-17T21:53:11.402 INFO:tasks.workunit.client.1.smithi138.stdout:1/771: rename d4/db/d29/d5d to d4/db/d29/d5d/df7 22 2024-05-17T21:53:11.406 INFO:tasks.workunit.client.0.smithi107.stdout:6/919: dwrite d9/d10e/d66/f93 [0,4194304] 0 2024-05-17T21:53:11.413 INFO:tasks.workunit.client.1.smithi138.stdout:1/772: dwrite d4/df/d3a/fc5 [0,4194304] 0 2024-05-17T21:53:11.413 INFO:tasks.workunit.client.0.smithi107.stdout:8/954: rename dc/dd5 to dc/d21/d69/d11e/d14f 0 2024-05-17T21:53:11.413 INFO:tasks.workunit.client.0.smithi107.stdout:2/818: write d0/d9/d11/d18/fd5 [639395,3948] 0 2024-05-17T21:53:11.414 INFO:tasks.workunit.client.1.smithi138.stdout:1/773: chown d4/dd/d3b/f9e 12728 1 2024-05-17T21:53:11.416 INFO:tasks.workunit.client.0.smithi107.stdout:5/985: mknod d5/d11e/d139/c145 0 2024-05-17T21:53:11.420 INFO:tasks.workunit.client.0.smithi107.stdout:6/920: unlink d9/d10e/d66/dd5/ce3 0 2024-05-17T21:53:11.422 INFO:tasks.workunit.client.1.smithi138.stdout:1/774: creat d4/df/d3a/d3c/ff8 x:0 0 0 2024-05-17T21:53:11.425 INFO:tasks.workunit.client.0.smithi107.stdout:2/819: creat d0/d9/d1c/d22/db0/f111 x:0 0 0 2024-05-17T21:53:11.426 INFO:tasks.workunit.client.1.smithi138.stdout:2/828: rmdir d3/d21/d2f/d3e/d55/d99/d84/dbe/df0 0 2024-05-17T21:53:11.427 INFO:tasks.workunit.client.1.smithi138.stdout:8/683: write d0/f2c [5013008,66180] 0 2024-05-17T21:53:11.428 INFO:tasks.workunit.client.1.smithi138.stdout:8/684: chown d0/dc/d3b/fba 60578783 1 2024-05-17T21:53:11.429 INFO:tasks.workunit.client.1.smithi138.stdout:1/775: rename d4/df/d5f/fe6 to d4/dd/d3b/d6c/dcf/ff9 0 2024-05-17T21:53:11.429 INFO:tasks.workunit.client.1.smithi138.stdout:8/685: dread - d0/dc/d3b/fba zero size 2024-05-17T21:53:11.430 INFO:tasks.workunit.client.0.smithi107.stdout:5/986: chown d5/db/dc/d1e/d6e/le1 1082853 1 2024-05-17T21:53:11.432 INFO:tasks.workunit.client.1.smithi138.stdout:2/829: mkdir d3/d21/d2f/d3e/d55/d99/d84/d9f/dfd/ddf/d114 0 2024-05-17T21:53:11.434 INFO:tasks.workunit.client.0.smithi107.stdout:2/820: creat d0/d9/d1c/d37/df8/f112 x:0 0 0 2024-05-17T21:53:11.435 INFO:tasks.workunit.client.0.smithi107.stdout:5/987: rmdir d5/db/d16/d2d 39 2024-05-17T21:53:11.437 INFO:tasks.workunit.client.0.smithi107.stdout:8/955: link dc/d21/d27/d2f/l54 dc/d44/d4b/d52/d70/d76/l150 0 2024-05-17T21:53:11.442 INFO:tasks.workunit.client.0.smithi107.stdout:8/956: creat dc/d44/d4b/d52/f151 x:0 0 0 2024-05-17T21:53:11.443 INFO:tasks.workunit.client.1.smithi138.stdout:1/776: link d4/dd/d3b/lc8 d4/db/d60/lfa 0 2024-05-17T21:53:11.444 INFO:tasks.workunit.client.1.smithi138.stdout:8/686: getdents d0/d71 0 2024-05-17T21:53:11.448 INFO:tasks.workunit.client.0.smithi107.stdout:8/957: getdents dc/d21/d27/d2f 0 2024-05-17T21:53:11.448 INFO:tasks.workunit.client.0.smithi107.stdout:8/958: truncate dc/f18 9295031 0 2024-05-17T21:53:11.449 INFO:tasks.workunit.client.1.smithi138.stdout:1/777: dread d4/df/d4e/fd7 [0,4194304] 0 2024-05-17T21:53:11.450 INFO:tasks.workunit.client.0.smithi107.stdout:8/959: symlink dc/d44/d4b/d52/d70/d76/l152 0 2024-05-17T21:53:11.451 INFO:tasks.workunit.client.1.smithi138.stdout:1/778: creat d4/df/d5f/ffb x:0 0 0 2024-05-17T21:53:11.452 INFO:tasks.workunit.client.0.smithi107.stdout:8/960: write dc/d21/f126 [4751839,117588] 0 2024-05-17T21:53:11.453 INFO:tasks.workunit.client.1.smithi138.stdout:1/779: creat d4/df/d3a/d3c/d63/d6d/ffc x:0 0 0 2024-05-17T21:53:11.453 INFO:tasks.workunit.client.0.smithi107.stdout:8/961: write dc/d21/d27/d83/f14d [178489,58671] 0 2024-05-17T21:53:11.455 INFO:tasks.workunit.client.1.smithi138.stdout:1/780: creat d4/db/d29/d89/ffd x:0 0 0 2024-05-17T21:53:11.457 INFO:tasks.workunit.client.0.smithi107.stdout:8/962: unlink dc/d44/d4b/d52/d70/l125 0 2024-05-17T21:53:11.462 INFO:tasks.workunit.client.1.smithi138.stdout:5/779: dread d1/d60/d2f/d5e/f79 [0,4194304] 0 2024-05-17T21:53:11.462 INFO:tasks.workunit.client.1.smithi138.stdout:5/780: chown d1/d60/d2f 228001 1 2024-05-17T21:53:11.470 INFO:tasks.workunit.client.1.smithi138.stdout:0/721: truncate d3/d10/d38/d3e/fb7 2893215 0 2024-05-17T21:53:11.477 INFO:tasks.workunit.client.1.smithi138.stdout:2/830: write d3/f66 [19528,99849] 0 2024-05-17T21:53:11.477 INFO:tasks.workunit.client.0.smithi107.stdout:5/988: write d5/db/d16/d54/d59/f87 [590074,15678] 0 2024-05-17T21:53:11.479 INFO:tasks.workunit.client.0.smithi107.stdout:2/821: truncate d0/d9/d1c/d22/f65 3815310 0 2024-05-17T21:53:11.481 INFO:tasks.workunit.client.1.smithi138.stdout:8/687: truncate d0/dc/d3b/f99 634684 0 2024-05-17T21:53:11.483 INFO:tasks.workunit.client.0.smithi107.stdout:6/921: dwrite d9/d16/d22/d5f/f11e [0,4194304] 0 2024-05-17T21:53:11.485 INFO:tasks.workunit.client.0.smithi107.stdout:2/822: symlink d0/d9/d11/d18/d9e/l113 0 2024-05-17T21:53:11.485 INFO:tasks.workunit.client.1.smithi138.stdout:8/688: dread - d0/dc/d8d/fad zero size 2024-05-17T21:53:11.488 INFO:tasks.workunit.client.1.smithi138.stdout:8/689: rename d0/fb6 to d0/d31/dbe/fc9 0 2024-05-17T21:53:11.490 INFO:tasks.workunit.client.1.smithi138.stdout:5/781: write d1/d2c/d63/db5/d73/fb0 [3069952,25067] 0 2024-05-17T21:53:11.490 INFO:tasks.workunit.client.0.smithi107.stdout:2/823: read d0/d9/d11/d18/dad/fc3 [264868,47752] 0 2024-05-17T21:53:11.490 INFO:tasks.workunit.client.1.smithi138.stdout:8/690: chown d0/dc/d1a/f4b 62851168 1 2024-05-17T21:53:11.491 INFO:tasks.workunit.client.0.smithi107.stdout:5/989: dwrite d5/db/d16/d2d/d40/d5a/d7c/f13f [0,4194304] 0 2024-05-17T21:53:11.494 INFO:tasks.workunit.client.1.smithi138.stdout:5/782: readlink d1/d53/d88/de5/ded/dc2/lbd 0 2024-05-17T21:53:11.496 INFO:tasks.workunit.client.1.smithi138.stdout:4/789: sync 2024-05-17T21:53:11.497 INFO:tasks.workunit.client.1.smithi138.stdout:1/781: dwrite d4/db/d29/f5b [0,4194304] 0 2024-05-17T21:53:11.508 INFO:tasks.workunit.client.1.smithi138.stdout:8/691: mkdir d0/d11/d3e/dca 0 2024-05-17T21:53:11.513 INFO:tasks.workunit.client.0.smithi107.stdout:6/922: link d9/d16/d22/daf/dc7/f101 d9/d16/d22/d5f/d6e/dd6/d11f/d108/f12d 0 2024-05-17T21:53:11.515 INFO:tasks.workunit.client.1.smithi138.stdout:4/790: creat d0/d1b/d2b/f10e x:0 0 0 2024-05-17T21:53:11.515 INFO:tasks.workunit.client.0.smithi107.stdout:6/923: truncate d9/d16/d4d/d85/f122 1031289 0 2024-05-17T21:53:11.517 INFO:tasks.workunit.client.0.smithi107.stdout:5/990: creat d5/db/d16/d107/f146 x:0 0 0 2024-05-17T21:53:11.518 INFO:tasks.workunit.client.0.smithi107.stdout:6/924: symlink d9/d16/d4d/db8/dc9/l12e 0 2024-05-17T21:53:11.519 INFO:tasks.workunit.client.1.smithi138.stdout:8/692: rmdir d0/dc/d3b 39 2024-05-17T21:53:11.521 INFO:tasks.workunit.client.1.smithi138.stdout:2/831: dread d3/d21/d2f/fc0 [0,4194304] 0 2024-05-17T21:53:11.521 INFO:tasks.workunit.client.1.smithi138.stdout:4/791: mkdir d0/d3/d17/d21/d78/d10f 0 2024-05-17T21:53:11.523 INFO:tasks.workunit.client.0.smithi107.stdout:6/925: fdatasync d9/d16/d4d/db8/dc9/dd0/f5c 0 2024-05-17T21:53:11.523 INFO:tasks.workunit.client.1.smithi138.stdout:1/782: creat d4/dd/d3b/d9f/da9/dd9/ffe x:0 0 0 2024-05-17T21:53:11.525 INFO:tasks.workunit.client.1.smithi138.stdout:1/783: dread - d4/df/d3a/d3c/d63/d6d/ffc zero size 2024-05-17T21:53:11.526 INFO:tasks.workunit.client.1.smithi138.stdout:4/792: creat d0/d1b/d42/f110 x:0 0 0 2024-05-17T21:53:11.528 INFO:tasks.workunit.client.0.smithi107.stdout:6/926: creat d9/d16/d4d/d85/d88/f12f x:0 0 0 2024-05-17T21:53:11.529 INFO:tasks.workunit.client.0.smithi107.stdout:2/824: write d0/d9/d1c/d22/f65 [4652195,57652] 0 2024-05-17T21:53:11.530 INFO:tasks.workunit.client.0.smithi107.stdout:2/825: chown d0/d9/d11/d18/dad/db8 39973 1 2024-05-17T21:53:11.531 INFO:tasks.workunit.client.0.smithi107.stdout:6/927: creat d9/d10e/d66/dd5/df1/f130 x:0 0 0 2024-05-17T21:53:11.534 INFO:tasks.workunit.client.0.smithi107.stdout:6/928: getdents d9/d16/d4d/d85/d96/d104 0 2024-05-17T21:53:11.535 INFO:tasks.workunit.client.0.smithi107.stdout:2/826: getdents d0/d9/d11/df/d5a/d6d/dc8/de8/ddb 0 2024-05-17T21:53:11.536 INFO:tasks.workunit.client.0.smithi107.stdout:6/929: creat d9/f131 x:0 0 0 2024-05-17T21:53:11.544 INFO:tasks.workunit.client.0.smithi107.stdout:2/827: dread d0/f6f [0,4194304] 0 2024-05-17T21:53:11.546 INFO:tasks.workunit.client.1.smithi138.stdout:5/783: dwrite d1/d2c/d63/f6d [0,4194304] 0 2024-05-17T21:53:11.548 INFO:tasks.workunit.client.1.smithi138.stdout:2/832: dread d3/f4a [0,4194304] 0 2024-05-17T21:53:11.549 INFO:tasks.workunit.client.1.smithi138.stdout:5/784: write d1/d2c/d63/db5/d73/fb0 [650194,95043] 0 2024-05-17T21:53:11.552 INFO:tasks.workunit.client.0.smithi107.stdout:2/828: dwrite d0/d26/ffd [0,4194304] 0 2024-05-17T21:53:11.558 INFO:tasks.workunit.client.1.smithi138.stdout:2/833: dwrite d3/d21/d2f/d3e/d55/d99/dd2/f101 [0,4194304] 0 2024-05-17T21:53:11.559 INFO:tasks.workunit.client.1.smithi138.stdout:1/784: dread d4/df/d3a/f81 [0,4194304] 0 2024-05-17T21:53:11.560 INFO:tasks.workunit.client.0.smithi107.stdout:6/930: readlink d9/d16/d4d/l99 0 2024-05-17T21:53:11.561 INFO:tasks.workunit.client.1.smithi138.stdout:2/834: write d3/d21/d2f/d3e/d63/f80 [6170683,103222] 0 2024-05-17T21:53:11.562 INFO:tasks.workunit.client.1.smithi138.stdout:1/785: symlink d4/dd/d3b/d6c/dcf/lff 0 2024-05-17T21:53:11.568 INFO:tasks.workunit.client.0.smithi107.stdout:6/931: rename d9/l40 to d9/d16/d22/daf/l132 0 2024-05-17T21:53:11.576 INFO:tasks.workunit.client.0.smithi107.stdout:7/940: sync 2024-05-17T21:53:11.578 INFO:tasks.workunit.client.0.smithi107.stdout:7/941: write d5/d16/d3b/ddf/d12e/db3/f103 [33326,39506] 0 2024-05-17T21:53:11.580 INFO:tasks.workunit.client.0.smithi107.stdout:7/942: rmdir d5/df/d11/d1c/d2f/dfe 39 2024-05-17T21:53:11.583 INFO:tasks.workunit.client.0.smithi107.stdout:2/829: dread d0/d9/d11/df/f47 [0,4194304] 0 2024-05-17T21:53:11.585 INFO:tasks.workunit.client.0.smithi107.stdout:2/830: truncate d0/d9/d1c/d37/df8/f112 369452 0 2024-05-17T21:53:11.586 INFO:tasks.workunit.client.0.smithi107.stdout:2/831: stat d0/d9/f60 0 2024-05-17T21:53:11.597 INFO:tasks.workunit.client.1.smithi138.stdout:1/786: write d4/db/d29/d5d/f6f [2459894,39081] 0 2024-05-17T21:53:11.600 INFO:tasks.workunit.client.0.smithi107.stdout:6/932: rename d9/d16/d4d/d85/d96 to d9/d16/d4d/d85/d96/d117/d133 22 2024-05-17T21:53:11.601 INFO:tasks.workunit.client.1.smithi138.stdout:1/787: chown d4/dd/d3b/l70 11 1 2024-05-17T21:53:11.601 INFO:tasks.workunit.client.1.smithi138.stdout:5/785: dwrite d1/d2c/d63/db5/d19/f3a [0,4194304] 0 2024-05-17T21:53:11.604 INFO:tasks.workunit.client.0.smithi107.stdout:7/943: dread d5/df/d11/d1d/d33/d82/fa0 [0,4194304] 0 2024-05-17T21:53:11.605 INFO:tasks.workunit.client.1.smithi138.stdout:1/788: dread d4/dd/f2d [0,4194304] 0 2024-05-17T21:53:11.606 INFO:tasks.workunit.client.1.smithi138.stdout:1/789: dread - d4/df/d3a/d3c/d63/d6d/fda zero size 2024-05-17T21:53:11.610 INFO:tasks.workunit.client.0.smithi107.stdout:2/832: dwrite d0/d9/d11/df/d5a/d6d/fb6 [4194304,4194304] 0 2024-05-17T21:53:11.612 INFO:tasks.workunit.client.0.smithi107.stdout:7/944: creat d5/df/d11/d1c/d2f/dfe/f141 x:0 0 0 2024-05-17T21:53:11.615 INFO:tasks.workunit.client.0.smithi107.stdout:2/833: creat d0/d9/d11/d15/df2/f114 x:0 0 0 2024-05-17T21:53:11.617 INFO:tasks.workunit.client.1.smithi138.stdout:5/786: dwrite d1/d2c/d63/db5/f41 [0,4194304] 0 2024-05-17T21:53:11.620 INFO:tasks.workunit.client.0.smithi107.stdout:2/834: unlink d0/d26/d8e/dc4/cf7 0 2024-05-17T21:53:11.620 INFO:tasks.workunit.client.0.smithi107.stdout:7/945: dwrite d5/d16/d3b/ddf/d12e/dc1/de6/feb [0,4194304] 0 2024-05-17T21:53:11.629 INFO:tasks.workunit.client.0.smithi107.stdout:7/946: unlink d5/df/d11/d1c/d31/d39/d92/d9a/f105 0 2024-05-17T21:53:11.635 INFO:tasks.workunit.client.0.smithi107.stdout:7/947: getdents d5/df/d11/d1c/d2f/d53 0 2024-05-17T21:53:11.640 INFO:tasks.workunit.client.0.smithi107.stdout:6/933: dread d9/d10e/d66/dd5/de9/f115 [0,4194304] 0 2024-05-17T21:53:11.642 INFO:tasks.workunit.client.0.smithi107.stdout:6/934: truncate d9/d16/f7f 955213 0 2024-05-17T21:53:11.642 INFO:tasks.workunit.client.0.smithi107.stdout:8/963: sync 2024-05-17T21:53:11.644 INFO:tasks.workunit.client.0.smithi107.stdout:6/935: mkdir d9/d10e/d66/dd5/de9/d134 0 2024-05-17T21:53:11.645 INFO:tasks.workunit.client.0.smithi107.stdout:8/964: chown dc/d21/d27/d34/d4c/f147 458798 1 2024-05-17T21:53:11.649 INFO:tasks.workunit.client.0.smithi107.stdout:6/936: rename d9/d16/d4d/d85/d96/d117/d119 to d9/d16/d22/d5f/d6e/dd6/d11f/d135 0 2024-05-17T21:53:11.652 INFO:tasks.workunit.client.0.smithi107.stdout:8/965: dwrite dc/d21/db8/f138 [0,4194304] 0 2024-05-17T21:53:11.654 INFO:tasks.workunit.client.0.smithi107.stdout:6/937: link d9/d10e/f9b d9/d10e/d7b/d106/f136 0 2024-05-17T21:53:11.655 INFO:tasks.workunit.client.1.smithi138.stdout:1/790: dread d4/db/d29/fa4 [0,4194304] 0 2024-05-17T21:53:11.655 INFO:tasks.workunit.client.0.smithi107.stdout:6/938: read d9/d16/d4d/db8/dc9/dd0/f63 [1153329,30336] 0 2024-05-17T21:53:11.656 INFO:tasks.workunit.client.1.smithi138.stdout:1/791: write d4/d2e/d36/fac [460130,88930] 0 2024-05-17T21:53:11.657 INFO:tasks.workunit.client.0.smithi107.stdout:6/939: truncate d9/d10e/d7b/da3/fd7 494560 0 2024-05-17T21:53:11.659 INFO:tasks.workunit.client.1.smithi138.stdout:1/792: creat d4/df/d5f/f100 x:0 0 0 2024-05-17T21:53:11.660 INFO:tasks.workunit.client.1.smithi138.stdout:1/793: chown d4/df/d3a/d3c/d63/d6d/fef 61627913 1 2024-05-17T21:53:11.662 INFO:tasks.workunit.client.0.smithi107.stdout:6/940: symlink d9/d16/d4d/l137 0 2024-05-17T21:53:11.662 INFO:tasks.workunit.client.1.smithi138.stdout:1/794: chown d4/ldb 14846 1 2024-05-17T21:53:11.663 INFO:tasks.workunit.client.0.smithi107.stdout:6/941: creat d9/d10e/d66/dd5/df1/f138 x:0 0 0 2024-05-17T21:53:11.664 INFO:tasks.workunit.client.1.smithi138.stdout:1/795: readlink d4/df/d35/l45 0 2024-05-17T21:53:11.666 INFO:tasks.workunit.client.0.smithi107.stdout:6/942: creat d9/d16/d22/de6/f139 x:0 0 0 2024-05-17T21:53:11.667 INFO:tasks.workunit.client.0.smithi107.stdout:6/943: chown d9/d16/d4d/db8/dc9/dd0/cb4 1 1 2024-05-17T21:53:11.669 INFO:tasks.workunit.client.0.smithi107.stdout:6/944: mknod d9/d10e/d66/dd5/c13a 0 2024-05-17T21:53:11.672 INFO:tasks.workunit.client.0.smithi107.stdout:2/835: truncate d0/d26/ffd 2027336 0 2024-05-17T21:53:11.674 INFO:tasks.workunit.client.1.smithi138.stdout:5/787: dwrite d1/d53/ffc [0,4194304] 0 2024-05-17T21:53:11.676 INFO:tasks.workunit.client.0.smithi107.stdout:6/945: rmdir d9/d16/d4d/db8/d110 0 2024-05-17T21:53:11.677 INFO:tasks.workunit.client.0.smithi107.stdout:7/948: dwrite d5/df/d11/d1d/d44/f4b [4194304,4194304] 0 2024-05-17T21:53:11.685 INFO:tasks.workunit.client.0.smithi107.stdout:5/991: sync 2024-05-17T21:53:11.686 INFO:tasks.workunit.client.0.smithi107.stdout:2/836: symlink d0/d9/d11/d18/l115 0 2024-05-17T21:53:11.686 INFO:tasks.workunit.client.0.smithi107.stdout:8/966: write dc/d21/d27/d2f/f4a [524881,118912] 0 2024-05-17T21:53:11.691 INFO:tasks.workunit.client.0.smithi107.stdout:8/967: rmdir dc/d21/d27/d2f/dba/dee 39 2024-05-17T21:53:11.693 INFO:tasks.workunit.client.1.smithi138.stdout:1/796: dwrite d4/df/d4e/f5c [0,4194304] 0 2024-05-17T21:53:11.697 INFO:tasks.workunit.client.0.smithi107.stdout:6/946: write f5 [562409,118366] 0 2024-05-17T21:53:11.698 INFO:tasks.workunit.client.0.smithi107.stdout:8/968: creat dc/d21/daa/db4/d149/f153 x:0 0 0 2024-05-17T21:53:11.700 INFO:tasks.workunit.client.1.smithi138.stdout:5/788: dwrite d1/d60/fcb [0,4194304] 0 2024-05-17T21:53:11.701 INFO:tasks.workunit.client.1.smithi138.stdout:5/789: chown d1/d2c/d63/db5/d1b/dc5/l40 86 1 2024-05-17T21:53:11.701 INFO:tasks.workunit.client.0.smithi107.stdout:2/837: dwrite d0/d9/d11/df/d5a/d6d/ffc [0,4194304] 0 2024-05-17T21:53:11.702 INFO:tasks.workunit.client.0.smithi107.stdout:6/947: fsync d9/d81/f82 0 2024-05-17T21:53:11.705 INFO:tasks.workunit.client.0.smithi107.stdout:8/969: fsync dc/d21/d35/f36 0 2024-05-17T21:53:11.706 INFO:tasks.workunit.client.0.smithi107.stdout:6/948: creat d9/d16/d22/d5f/d6e/dd6/f13b x:0 0 0 2024-05-17T21:53:11.717 INFO:tasks.workunit.client.0.smithi107.stdout:7/949: dread d5/df/d11/d1c/d2f/fd2 [0,4194304] 0 2024-05-17T21:53:11.717 INFO:tasks.workunit.client.0.smithi107.stdout:6/949: rename d9/d16/d22/d5f/fe7 to d9/d81/f13c 0 2024-05-17T21:53:11.719 INFO:tasks.workunit.client.0.smithi107.stdout:8/970: mkdir dc/d21/d27/d83/d124/d154 0 2024-05-17T21:53:11.721 INFO:tasks.workunit.client.0.smithi107.stdout:6/950: mkdir d9/d10e/d7b/d106/d13d 0 2024-05-17T21:53:11.722 INFO:tasks.workunit.client.0.smithi107.stdout:7/950: creat d5/df/d11/d1c/d31/d56/d13b/f142 x:0 0 0 2024-05-17T21:53:11.723 INFO:tasks.workunit.client.0.smithi107.stdout:6/951: fsync d9/d16/d4d/d85/d88/db1/f127 0 2024-05-17T21:53:11.724 INFO:tasks.workunit.client.0.smithi107.stdout:7/951: rmdir d5/d16/d3b/ddf/d12e/dc1 39 2024-05-17T21:53:11.726 INFO:tasks.workunit.client.0.smithi107.stdout:7/952: creat d5/d16/d3b/ddf/d12e/dac/d107/f143 x:0 0 0 2024-05-17T21:53:11.729 INFO:tasks.workunit.client.0.smithi107.stdout:7/953: dread d5/df/d11/d1c/d31/d39/d92/d9a/fa1 [0,4194304] 0 2024-05-17T21:53:11.730 INFO:tasks.workunit.client.0.smithi107.stdout:7/954: read - d5/df/d11/d1c/d31/d39/fcc zero size 2024-05-17T21:53:11.732 INFO:tasks.workunit.client.0.smithi107.stdout:2/838: truncate d0/d9/d11/fda 2551288 0 2024-05-17T21:53:11.733 INFO:tasks.workunit.client.0.smithi107.stdout:7/955: truncate d5/d16/d3b/ddf/d12e/dac/fef 913680 0 2024-05-17T21:53:11.734 INFO:tasks.workunit.client.0.smithi107.stdout:2/839: mknod d0/d9/d1c/d37/df8/c116 0 2024-05-17T21:53:11.734 INFO:tasks.workunit.client.0.smithi107.stdout:7/956: rmdir d5 39 2024-05-17T21:53:11.735 INFO:tasks.workunit.client.0.smithi107.stdout:2/840: write d0/d9/d1c/d22/fb1 [1509077,40104] 0 2024-05-17T21:53:11.737 INFO:tasks.workunit.client.0.smithi107.stdout:7/957: mknod d5/df/d11/d2a/c144 0 2024-05-17T21:53:11.739 INFO:tasks.workunit.client.0.smithi107.stdout:2/841: creat d0/d9/f117 x:0 0 0 2024-05-17T21:53:11.740 INFO:tasks.workunit.client.0.smithi107.stdout:7/958: rmdir d5/df/d11/df6 39 2024-05-17T21:53:11.740 INFO:tasks.workunit.client.0.smithi107.stdout:2/842: dread - d0/d93/fb3 zero size 2024-05-17T21:53:11.742 INFO:tasks.workunit.client.0.smithi107.stdout:6/952: dwrite d9/d16/d4d/db8/dc9/dd0/f70 [0,4194304] 0 2024-05-17T21:53:11.743 INFO:tasks.workunit.client.0.smithi107.stdout:6/953: write d9/d81/f120 [159783,3116] 0 2024-05-17T21:53:11.743 INFO:tasks.workunit.client.0.smithi107.stdout:7/959: rmdir d5/df/d11/d140 0 2024-05-17T21:53:11.748 INFO:tasks.workunit.client.0.smithi107.stdout:6/954: creat d9/d16/d22/de6/f13e x:0 0 0 2024-05-17T21:53:11.750 INFO:tasks.workunit.client.0.smithi107.stdout:2/843: getdents d0/d9/d11/d18/d103 0 2024-05-17T21:53:11.751 INFO:tasks.workunit.client.0.smithi107.stdout:6/955: dread d9/d10e/d7b/da3/fd7 [0,4194304] 0 2024-05-17T21:53:11.752 INFO:tasks.workunit.client.0.smithi107.stdout:7/960: rename d5/df/d11/d1c/d31/d39/fcc to d5/df/d11/df6/f145 0 2024-05-17T21:53:11.756 INFO:tasks.workunit.client.0.smithi107.stdout:7/961: symlink d5/df/d11/d1d/d33/l146 0 2024-05-17T21:53:11.757 INFO:tasks.workunit.client.0.smithi107.stdout:2/844: mknod d0/d9/c118 0 2024-05-17T21:53:11.758 INFO:tasks.workunit.client.0.smithi107.stdout:2/845: chown d0/d9/d11/df/c1e 263326 1 2024-05-17T21:53:11.758 INFO:tasks.workunit.client.0.smithi107.stdout:6/956: link d9/d10e/d7b/da3/fd7 d9/d16/d4d/d67/f13f 0 2024-05-17T21:53:11.759 INFO:tasks.workunit.client.0.smithi107.stdout:2/846: truncate d0/d9/d1c/d22/db0/f111 607543 0 2024-05-17T21:53:11.760 INFO:tasks.workunit.client.0.smithi107.stdout:7/962: read d5/df/d11/d1c/d2f/fec [2888704,106960] 0 2024-05-17T21:53:11.762 INFO:tasks.workunit.client.0.smithi107.stdout:6/957: unlink d9/d16/d22/d5f/d6e/l9c 0 2024-05-17T21:53:11.762 INFO:tasks.workunit.client.0.smithi107.stdout:2/847: dread - d0/d9/d11/df/fb4 zero size 2024-05-17T21:53:11.766 INFO:tasks.workunit.client.0.smithi107.stdout:7/963: rename d5/d16/d3b/d45/l93 to d5/df/d11/d1c/d31/d56/d13b/l147 0 2024-05-17T21:53:11.766 INFO:tasks.workunit.client.0.smithi107.stdout:7/964: chown d5/df 3153371 1 2024-05-17T21:53:11.767 INFO:tasks.workunit.client.0.smithi107.stdout:6/958: dread d9/d16/d22/d5f/d6e/dd6/d11f/d108/f12d [0,4194304] 0 2024-05-17T21:53:11.767 INFO:tasks.workunit.client.0.smithi107.stdout:2/848: unlink d0/d9/d11/d27/dd3/ff9 0 2024-05-17T21:53:11.769 INFO:tasks.workunit.client.0.smithi107.stdout:7/965: truncate d5/d16/d3b/d45/faf 806444 0 2024-05-17T21:53:11.771 INFO:tasks.workunit.client.0.smithi107.stdout:7/966: chown d5/df/d11/d1c/d2f/l57 13814501 1 2024-05-17T21:53:11.772 INFO:tasks.workunit.client.0.smithi107.stdout:2/849: rmdir d0/d9/d1c/dd4/dec 0 2024-05-17T21:53:11.774 INFO:tasks.workunit.client.0.smithi107.stdout:6/959: dwrite d9/d10e/d66/de2/f105 [0,4194304] 0 2024-05-17T21:53:11.777 INFO:tasks.workunit.client.0.smithi107.stdout:2/850: mknod d0/d9/d11/df/d5a/d6d/dc8/de8/c119 0 2024-05-17T21:53:11.777 INFO:tasks.workunit.client.0.smithi107.stdout:2/851: write d0/d9/d11/df/d5a/d6d/d82/f106 [26802,55246] 0 2024-05-17T21:53:11.779 INFO:tasks.workunit.client.0.smithi107.stdout:7/967: truncate d5/df/d11/d1d/d33/d82/ffb 164042 0 2024-05-17T21:53:11.784 INFO:tasks.workunit.client.0.smithi107.stdout:2/852: unlink d0/d9/d11/df/d5a/d6d/d82/fb9 0 2024-05-17T21:53:11.786 INFO:tasks.workunit.client.0.smithi107.stdout:2/853: write d0/d9/d11/d18/dad/db8/ffb [846080,26463] 0 2024-05-17T21:53:11.789 INFO:tasks.workunit.client.0.smithi107.stdout:7/968: creat d5/d16/d3b/ddf/d12e/dc1/de6/d116/f148 x:0 0 0 2024-05-17T21:53:11.790 INFO:tasks.workunit.client.0.smithi107.stdout:6/960: write d9/d10e/f9b [970258,56869] 0 2024-05-17T21:53:11.791 INFO:tasks.workunit.client.0.smithi107.stdout:2/854: link d0/d9/d11/d18/dad/fe6 d0/d9/d11/df/d5a/d6d/f11a 0 2024-05-17T21:53:11.792 INFO:tasks.workunit.client.0.smithi107.stdout:6/961: write d9/d16/d22/daf/dc7/ff0 [258974,15507] 0 2024-05-17T21:53:11.794 INFO:tasks.workunit.client.0.smithi107.stdout:7/969: getdents d5/df/d11/d1c/d31/d56 0 2024-05-17T21:53:11.798 INFO:tasks.workunit.client.0.smithi107.stdout:2/855: link d0/d9/d11/d27/c4d d0/d9/d1c/dd4/c11b 0 2024-05-17T21:53:11.799 INFO:tasks.workunit.client.0.smithi107.stdout:5/992: sync 2024-05-17T21:53:11.800 INFO:tasks.workunit.client.0.smithi107.stdout:6/962: rename d9/d16/d22/d5f/d6e/dd6/d11f to d9/d16/d4d/d85/d140 0 2024-05-17T21:53:11.803 INFO:tasks.workunit.client.0.smithi107.stdout:5/993: rename d5/dec/f13c to d5/db/d16/d54/d59/d92/f147 0 2024-05-17T21:53:11.805 INFO:tasks.workunit.client.0.smithi107.stdout:7/970: write d5/df/d11/d1c/d31/d39/d92/f113 [4970919,126193] 0 2024-05-17T21:53:11.805 INFO:tasks.workunit.client.0.smithi107.stdout:2/856: getdents d0/d9/d1c/d37/df8 0 2024-05-17T21:53:11.808 INFO:tasks.workunit.client.0.smithi107.stdout:6/963: dwrite d9/d16/d4d/d85/d88/db1/f127 [0,4194304] 0 2024-05-17T21:53:11.866 INFO:tasks.workunit.client.0.smithi107.stdout:2/857: creat d0/d9/d1c/d22/db0/f11c x:0 0 0 2024-05-17T21:53:11.867 INFO:tasks.workunit.client.0.smithi107.stdout:2/858: write d0/d9/d11/df/d5a/d6d/fca [330601,126579] 0 2024-05-17T21:53:11.885 INFO:tasks.workunit.client.0.smithi107.stdout:8/971: sync 2024-05-17T21:53:11.919 INFO:tasks.workunit.client.0.smithi107.stdout:7/971: sync 2024-05-17T21:53:11.944 INFO:tasks.workunit.client.0.smithi107.stdout:6/964: mknod d9/d16/d4d/d85/d88/db1/c141 0 2024-05-17T21:53:11.979 INFO:tasks.workunit.client.0.smithi107.stdout:5/994: creat d5/db/dc/d1e/d125/f148 x:0 0 0 2024-05-17T21:53:11.980 INFO:tasks.workunit.client.0.smithi107.stdout:5/995: readlink d5/db/d16/d54/d59/d65/d48/d89/lf9 0 2024-05-17T21:53:11.981 INFO:tasks.workunit.client.0.smithi107.stdout:2/859: mknod d0/d9/d1c/d22/db0/c11d 0 2024-05-17T21:53:11.983 INFO:tasks.workunit.client.0.smithi107.stdout:8/972: unlink dc/d21/f92 0 2024-05-17T21:53:11.984 INFO:tasks.workunit.client.0.smithi107.stdout:6/965: creat d9/d16/d22/d5f/d6e/dd6/f142 x:0 0 0 2024-05-17T21:53:11.987 INFO:tasks.workunit.client.0.smithi107.stdout:2/860: dwrite d0/d9/d11/d27/dd3/fc1 [0,4194304] 0 2024-05-17T21:53:11.989 INFO:tasks.workunit.client.0.smithi107.stdout:8/973: mkdir dc/d44/d4b/d121/d155 0 2024-05-17T21:53:11.990 INFO:tasks.workunit.client.0.smithi107.stdout:6/966: fsync d9/d16/d4d/db8/dc9/dd0/f54 0 2024-05-17T21:53:11.991 INFO:tasks.workunit.client.0.smithi107.stdout:5/996: getdents d5/db/d16/d54/d59/d65/d12c 0 2024-05-17T21:53:11.994 INFO:tasks.workunit.client.0.smithi107.stdout:6/967: fdatasync d9/d10e/d66/f6c 0 2024-05-17T21:53:11.996 INFO:tasks.workunit.client.0.smithi107.stdout:2/861: symlink d0/d93/dc5/l11e 0 2024-05-17T21:53:11.998 INFO:tasks.workunit.client.0.smithi107.stdout:8/974: symlink dc/d21/d27/d34/l156 0 2024-05-17T21:53:12.000 INFO:tasks.workunit.client.1.smithi138.stdout:3/954: sync 2024-05-17T21:53:12.000 INFO:tasks.workunit.client.1.smithi138.stdout:7/757: sync 2024-05-17T21:53:12.000 INFO:tasks.workunit.client.0.smithi107.stdout:6/968: dread d9/d16/d22/d5f/d6f/f91 [0,4194304] 0 2024-05-17T21:53:12.001 INFO:tasks.workunit.client.0.smithi107.stdout:8/975: mkdir dc/d44/d4b/d121/de5/d157 0 2024-05-17T21:53:12.001 INFO:tasks.workunit.client.0.smithi107.stdout:6/969: chown d9/d16/d22/d5f/d6e/c7a 36731048 1 2024-05-17T21:53:12.002 INFO:tasks.workunit.client.1.smithi138.stdout:3/955: creat d1/d62/db0/d13f/f143 x:0 0 0 2024-05-17T21:53:12.003 INFO:tasks.workunit.client.1.smithi138.stdout:3/956: chown d1/d4f/d50/f54 0 1 2024-05-17T21:53:12.004 INFO:tasks.workunit.client.1.smithi138.stdout:7/758: creat d0/d18/d9c/dd4/ff7 x:0 0 0 2024-05-17T21:53:12.006 INFO:tasks.workunit.client.0.smithi107.stdout:6/970: stat d9/d16/d22/d5f/d6e/dd6/cf7 0 2024-05-17T21:53:12.008 INFO:tasks.workunit.client.0.smithi107.stdout:8/976: getdents dc/d44/d4b/d52/dcb/d12b 0 2024-05-17T21:53:12.010 INFO:tasks.workunit.client.0.smithi107.stdout:6/971: fsync d9/f7c 0 2024-05-17T21:53:12.011 INFO:tasks.workunit.client.0.smithi107.stdout:8/977: creat dc/d44/d4b/d52/dcb/d11c/f158 x:0 0 0 2024-05-17T21:53:12.014 INFO:tasks.workunit.client.1.smithi138.stdout:7/759: creat d0/ff8 x:0 0 0 2024-05-17T21:53:12.016 INFO:tasks.workunit.client.0.smithi107.stdout:8/978: mknod dc/d21/d69/d11e/d14f/c159 0 2024-05-17T21:53:12.016 INFO:tasks.workunit.client.1.smithi138.stdout:7/760: readlink d0/d18/d40/d41/d51/la0 0 2024-05-17T21:53:12.019 INFO:tasks.workunit.client.1.smithi138.stdout:7/761: getdents d0/d39/d78 0 2024-05-17T21:53:12.021 INFO:tasks.workunit.client.1.smithi138.stdout:3/957: dread d1/d4f/d87/ff3 [0,4194304] 0 2024-05-17T21:53:12.022 INFO:tasks.workunit.client.1.smithi138.stdout:7/762: mkdir d0/d18/d40/d41/d51/d66/d9e/df9 0 2024-05-17T21:53:12.025 INFO:tasks.workunit.client.1.smithi138.stdout:3/958: symlink d1/d4f/d50/dbb/d13c/l144 0 2024-05-17T21:53:12.026 INFO:tasks.workunit.client.1.smithi138.stdout:3/959: readlink d1/d3/d16/d3e/d141/l94 0 2024-05-17T21:53:12.028 INFO:tasks.workunit.client.1.smithi138.stdout:3/960: mknod d1/d3/c145 0 2024-05-17T21:53:12.034 INFO:tasks.workunit.client.1.smithi138.stdout:0/722: sync 2024-05-17T21:53:12.034 INFO:tasks.workunit.client.1.smithi138.stdout:9/710: sync 2024-05-17T21:53:12.034 INFO:tasks.workunit.client.1.smithi138.stdout:8/693: sync 2024-05-17T21:53:12.034 INFO:tasks.workunit.client.1.smithi138.stdout:6/802: sync 2024-05-17T21:53:12.034 INFO:tasks.workunit.client.1.smithi138.stdout:4/793: sync 2024-05-17T21:53:12.035 INFO:tasks.workunit.client.1.smithi138.stdout:5/790: sync 2024-05-17T21:53:12.035 INFO:tasks.workunit.client.1.smithi138.stdout:2/835: sync 2024-05-17T21:53:12.035 INFO:tasks.workunit.client.1.smithi138.stdout:1/797: sync 2024-05-17T21:53:12.038 INFO:tasks.workunit.client.1.smithi138.stdout:5/791: chown d1/d2c/d63/db5/d1b/dc5/d36/d68/dad/cdc 6660 1 2024-05-17T21:53:12.038 INFO:tasks.workunit.client.1.smithi138.stdout:2/836: write d3/d21/d2f/d3e/d55/d99/d84/d9f/dfd/ff5 [686655,64479] 0 2024-05-17T21:53:12.042 INFO:tasks.workunit.client.1.smithi138.stdout:6/803: creat d7/d69/db5/dc8/d43/d3f/d79/f112 x:0 0 0 2024-05-17T21:53:12.044 INFO:tasks.workunit.client.1.smithi138.stdout:2/837: dread d3/d21/d2f/d3e/d63/fcd [4194304,4194304] 0 2024-05-17T21:53:12.044 INFO:tasks.workunit.client.1.smithi138.stdout:0/723: creat d3/d10/d38/d3e/d59/d9b/ff0 x:0 0 0 2024-05-17T21:53:12.046 INFO:tasks.workunit.client.0.smithi107.stdout:5/997: write d5/db/fc6 [266678,10192] 0 2024-05-17T21:53:12.046 INFO:tasks.workunit.client.1.smithi138.stdout:5/792: read d1/d2c/d63/db5/d1b/f33 [1254603,7237] 0 2024-05-17T21:53:12.048 INFO:tasks.workunit.client.1.smithi138.stdout:2/838: dread d3/f10 [0,4194304] 0 2024-05-17T21:53:12.051 INFO:tasks.workunit.client.0.smithi107.stdout:5/998: mkdir d5/db/d16/d2d/d40/dc2/d149 0 2024-05-17T21:53:12.052 INFO:tasks.workunit.client.0.smithi107.stdout:5/999: read d5/db/d16/f21 [3979139,44130] 0 2024-05-17T21:53:12.055 INFO:tasks.workunit.client.1.smithi138.stdout:0/724: mkdir d3/d10/d6d/d9f/df1 0 2024-05-17T21:53:12.055 INFO:tasks.workunit.client.1.smithi138.stdout:0/725: chown d3/d3d/d51 1423422 1 2024-05-17T21:53:12.056 INFO:tasks.workunit.client.0.smithi107.stdout:6/972: dwrite d9/d10e/fa1 [0,4194304] 0 2024-05-17T21:53:12.059 INFO:tasks.workunit.client.1.smithi138.stdout:5/793: read d1/d2c/d63/db5/d1b/dc5/f7c [1204749,109756] 0 2024-05-17T21:53:12.059 INFO:tasks.workunit.client.1.smithi138.stdout:7/763: write d0/f31 [1708353,110020] 0 2024-05-17T21:53:12.064 INFO:tasks.workunit.client.1.smithi138.stdout:8/694: rename d0/dc/d1a/d33/d4a/d4e/f7c to d0/dc/fcb 0 2024-05-17T21:53:12.064 INFO:tasks.workunit.client.1.smithi138.stdout:5/794: read d1/d2c/d63/db5/d1b/dc5/f3b [114907,116984] 0 2024-05-17T21:53:12.066 INFO:tasks.workunit.client.1.smithi138.stdout:3/961: write d1/d3/d8d/fc7 [2670989,65491] 0 2024-05-17T21:53:12.066 INFO:tasks.workunit.client.1.smithi138.stdout:5/795: write d1/f4f [1794222,22377] 0 2024-05-17T21:53:12.072 INFO:tasks.workunit.client.1.smithi138.stdout:1/798: dwrite d4/df/d3a/f81 [0,4194304] 0 2024-05-17T21:53:12.074 INFO:tasks.workunit.client.1.smithi138.stdout:0/726: mknod d3/d10/d38/d70/d80/cf2 0 2024-05-17T21:53:12.075 INFO:tasks.workunit.client.1.smithi138.stdout:9/711: link d6/d7/d16/d29/d48/lac d6/d7/d16/d29/da6/ldf 0 2024-05-17T21:53:12.077 INFO:tasks.workunit.client.0.smithi107.stdout:6/973: link d9/d16/f41 d9/d16/d4d/d85/d88/f143 0 2024-05-17T21:53:12.082 INFO:tasks.workunit.client.0.smithi107.stdout:6/974: write d9/d16/d22/f56 [1302866,107165] 0 2024-05-17T21:53:12.082 INFO:tasks.workunit.client.1.smithi138.stdout:2/839: symlink d3/d21/d2f/d3e/d55/d99/d84/dbe/dd8/l115 0 2024-05-17T21:53:12.083 INFO:tasks.workunit.client.0.smithi107.stdout:6/975: chown d9/d10e/d66/dd5/df3/f11d 202 1 2024-05-17T21:53:12.084 INFO:tasks.workunit.client.1.smithi138.stdout:5/796: read - d1/d2c/d63/db5/d1b/db7/fda zero size 2024-05-17T21:53:12.084 INFO:tasks.workunit.client.1.smithi138.stdout:4/794: getdents d0/d2f/d49/d60/d1f/d7e/de8 0 2024-05-17T21:53:12.085 INFO:tasks.workunit.client.0.smithi107.stdout:7/972: sync 2024-05-17T21:53:12.086 INFO:tasks.workunit.client.1.smithi138.stdout:9/712: dwrite d6/d7/d16/d29/d6c/d9c/fd0 [0,4194304] 0 2024-05-17T21:53:12.087 INFO:tasks.workunit.client.0.smithi107.stdout:7/973: write d5/d16/d3b/ddf/d12e/db3/fff [354392,86201] 0 2024-05-17T21:53:12.089 INFO:tasks.workunit.client.1.smithi138.stdout:7/764: getdents d0/d18/d40/d41/d51/d66/d9e/df9 0 2024-05-17T21:53:12.090 INFO:tasks.workunit.client.1.smithi138.stdout:7/765: dread - d0/d18/d40/d41/d51/d66/d9e/fde zero size 2024-05-17T21:53:12.091 INFO:tasks.workunit.client.1.smithi138.stdout:3/962: mknod d1/d3/d16/d5d/c146 0 2024-05-17T21:53:12.092 INFO:tasks.workunit.client.1.smithi138.stdout:7/766: write d0/d18/d64/fcd [279314,91474] 0 2024-05-17T21:53:12.092 INFO:tasks.workunit.client.1.smithi138.stdout:6/804: getdents d7/d69/db5/dc8/d34/da6/dfa 0 2024-05-17T21:53:12.092 INFO:tasks.workunit.client.0.smithi107.stdout:7/974: read d5/df/d11/d1d/f55 [1289978,98864] 0 2024-05-17T21:53:12.094 INFO:tasks.workunit.client.1.smithi138.stdout:7/767: truncate d0/d18/d9c/fe1 714159 0 2024-05-17T21:53:12.095 INFO:tasks.workunit.client.1.smithi138.stdout:8/695: write d0/dc/d1a/f2a [3814075,57345] 0 2024-05-17T21:53:12.097 INFO:tasks.workunit.client.0.smithi107.stdout:6/976: write d9/d16/d4d/f5b [1913353,20642] 0 2024-05-17T21:53:12.099 INFO:tasks.workunit.client.1.smithi138.stdout:0/727: symlink d3/d3d/lf3 0 2024-05-17T21:53:12.102 INFO:tasks.workunit.client.0.smithi107.stdout:7/975: rename d5/f83 to d5/df/d11/d1c/d31/d39/f149 0 2024-05-17T21:53:12.103 INFO:tasks.workunit.client.1.smithi138.stdout:7/768: dwrite d0/fd6 [0,4194304] 0 2024-05-17T21:53:12.104 INFO:tasks.workunit.client.0.smithi107.stdout:6/977: dwrite d9/d16/d4d/f52 [4194304,4194304] 0 2024-05-17T21:53:12.104 INFO:tasks.workunit.client.1.smithi138.stdout:1/799: truncate d4/df/d4e/f55 941238 0 2024-05-17T21:53:12.106 INFO:tasks.workunit.client.1.smithi138.stdout:1/800: write d4/db/d29/d5d/f86 [4703176,13340] 0 2024-05-17T21:53:12.107 INFO:tasks.workunit.client.1.smithi138.stdout:4/795: truncate d0/d1b/d88/f8b 953424 0 2024-05-17T21:53:12.110 INFO:tasks.workunit.client.1.smithi138.stdout:4/796: chown d0/d2f/d49/d60/d1f/d7e/f93 109 1 2024-05-17T21:53:12.112 INFO:tasks.workunit.client.1.smithi138.stdout:6/805: rename d7/d8/def to d7/d69/db5/dc8/d43/d5f/da4/db0/d113 0 2024-05-17T21:53:12.114 INFO:tasks.workunit.client.1.smithi138.stdout:6/806: chown d7/d69/db5/dc8/d43/d5f/fd5 3 1 2024-05-17T21:53:12.114 INFO:tasks.workunit.client.0.smithi107.stdout:6/978: symlink d9/d10e/d66/l144 0 2024-05-17T21:53:12.115 INFO:tasks.workunit.client.1.smithi138.stdout:9/713: dwrite d6/d7/d16/f6d [0,4194304] 0 2024-05-17T21:53:12.116 INFO:tasks.workunit.client.0.smithi107.stdout:6/979: chown d9/d16/d4d/db8/dc9/dd0/l4c 3537 1 2024-05-17T21:53:12.122 INFO:tasks.workunit.client.1.smithi138.stdout:8/696: dread d0/d2f/f6e [0,4194304] 0 2024-05-17T21:53:12.125 INFO:tasks.workunit.client.0.smithi107.stdout:7/976: write d5/d16/f1e [524897,2284] 0 2024-05-17T21:53:12.126 INFO:tasks.workunit.client.0.smithi107.stdout:7/977: dread - d5/df/d11/d1c/d2f/d53/fe0 zero size 2024-05-17T21:53:12.127 INFO:tasks.workunit.client.0.smithi107.stdout:6/980: fdatasync d9/d10e/d7b/da3/fc3 0 2024-05-17T21:53:12.128 INFO:tasks.workunit.client.0.smithi107.stdout:6/981: fsync d9/d16/d4d/f52 0 2024-05-17T21:53:12.128 INFO:tasks.workunit.client.1.smithi138.stdout:0/728: truncate f0 1601587 0 2024-05-17T21:53:12.129 INFO:tasks.workunit.client.0.smithi107.stdout:6/982: chown d9/d16/d22/d5f/cd3 1 1 2024-05-17T21:53:12.129 INFO:tasks.workunit.client.1.smithi138.stdout:5/797: mknod d1/d2c/d63/db5/d1b/ddb/c100 0 2024-05-17T21:53:12.130 INFO:tasks.workunit.client.1.smithi138.stdout:5/798: chown d1/d2c/dbf/fd3 3 1 2024-05-17T21:53:12.134 INFO:tasks.workunit.client.1.smithi138.stdout:3/963: mknod d1/d62/c147 0 2024-05-17T21:53:12.134 INFO:tasks.workunit.client.0.smithi107.stdout:7/978: dwrite d5/df/d11/d1c/d2f/d65/da7/fba [0,4194304] 0 2024-05-17T21:53:12.135 INFO:tasks.workunit.client.1.smithi138.stdout:4/797: mkdir d0/d1b/d2b/dca/d111 0 2024-05-17T21:53:12.135 INFO:tasks.workunit.client.0.smithi107.stdout:2/862: sync 2024-05-17T21:53:12.135 INFO:tasks.workunit.client.0.smithi107.stdout:8/979: sync 2024-05-17T21:53:12.137 INFO:tasks.workunit.client.1.smithi138.stdout:9/714: creat d6/d7/d16/dc2/fe0 x:0 0 0 2024-05-17T21:53:12.138 INFO:tasks.workunit.client.0.smithi107.stdout:6/983: dwrite d9/d16/d4d/db8/dc9/f112 [0,4194304] 0 2024-05-17T21:53:12.139 INFO:tasks.workunit.client.0.smithi107.stdout:8/980: chown dc/d44/d4b/l72 863771882 1 2024-05-17T21:53:12.140 INFO:tasks.workunit.client.1.smithi138.stdout:1/801: creat d4/df/d35/df3/f101 x:0 0 0 2024-05-17T21:53:12.143 INFO:tasks.workunit.client.1.smithi138.stdout:2/840: getdents d3/d21/d2f/d3e/d55/d99/d84/dbe/dd8 0 2024-05-17T21:53:12.146 INFO:tasks.workunit.client.0.smithi107.stdout:8/981: creat dc/d21/d69/f15a x:0 0 0 2024-05-17T21:53:12.147 INFO:tasks.workunit.client.0.smithi107.stdout:8/982: truncate dc/d21/d97/d63/f133 1255907 0 2024-05-17T21:53:12.148 INFO:tasks.workunit.client.0.smithi107.stdout:8/983: stat dc/d44/d4b 0 2024-05-17T21:53:12.149 INFO:tasks.workunit.client.0.smithi107.stdout:7/979: fdatasync d5/df/d11/d1c/d31/d39/d92/d9a/fdc 0 2024-05-17T21:53:12.151 INFO:tasks.workunit.client.0.smithi107.stdout:7/980: dread d5/f42 [0,4194304] 0 2024-05-17T21:53:12.152 INFO:tasks.workunit.client.0.smithi107.stdout:6/984: mknod d9/d16/d4d/d67/c145 0 2024-05-17T21:53:12.153 INFO:tasks.workunit.client.1.smithi138.stdout:0/729: mknod d3/d10/d6d/d9f/df1/cf4 0 2024-05-17T21:53:12.154 INFO:tasks.workunit.client.0.smithi107.stdout:2/863: mkdir d0/d9/d11/d18/d9e/deb/d109/d11f 0 2024-05-17T21:53:12.157 INFO:tasks.workunit.client.1.smithi138.stdout:8/697: write d0/dc/f2b [4712625,81049] 0 2024-05-17T21:53:12.159 INFO:tasks.workunit.client.0.smithi107.stdout:7/981: stat d5/df/d11/d1c/d2f/d65/da7/cda 0 2024-05-17T21:53:12.160 INFO:tasks.workunit.client.0.smithi107.stdout:6/985: mkdir d9/d16/d4d/d85/d88/db1/ddd/d146 0 2024-05-17T21:53:12.162 INFO:tasks.workunit.client.1.smithi138.stdout:7/769: rename d0/d18/d9c/cac to d0/cfa 0 2024-05-17T21:53:12.163 INFO:tasks.workunit.client.1.smithi138.stdout:1/802: dread d4/df/f97 [0,4194304] 0 2024-05-17T21:53:12.163 INFO:tasks.workunit.client.0.smithi107.stdout:2/864: mknod d0/d9/d11/df/d5a/d6d/c120 0 2024-05-17T21:53:12.164 INFO:tasks.workunit.client.1.smithi138.stdout:1/803: dread - d4/dd/d3b/d9f/da9/dd9/ffe zero size 2024-05-17T21:53:12.165 INFO:tasks.workunit.client.1.smithi138.stdout:2/841: unlink d3/d21/d2f/d3e/d58/d77/da4/df2/dd0/l76 0 2024-05-17T21:53:12.165 INFO:tasks.workunit.client.0.smithi107.stdout:6/986: dwrite d9/d16/d4d/d85/d88/db1/ffe [0,4194304] 0 2024-05-17T21:53:12.167 INFO:tasks.workunit.client.1.smithi138.stdout:4/798: creat d0/d3/d1d/d38/d8a/dbd/ddc/d101/f112 x:0 0 0 2024-05-17T21:53:12.168 INFO:tasks.workunit.client.0.smithi107.stdout:7/982: creat d5/df/d11/df6/f14a x:0 0 0 2024-05-17T21:53:12.170 INFO:tasks.workunit.client.0.smithi107.stdout:8/984: write dc/ddb/f110 [153936,121840] 0 2024-05-17T21:53:12.171 INFO:tasks.workunit.client.0.smithi107.stdout:7/983: chown d5/df/dd7/ld8 580779 1 2024-05-17T21:53:12.173 INFO:tasks.workunit.client.0.smithi107.stdout:2/865: mknod d0/d9/d11/df/d5a/d6d/d82/c121 0 2024-05-17T21:53:12.174 INFO:tasks.workunit.client.1.smithi138.stdout:9/715: mknod d6/d7/d16/d29/d38/d40/d44/d5c/db4/ce1 0 2024-05-17T21:53:12.175 INFO:tasks.workunit.client.0.smithi107.stdout:6/987: dwrite d9/d16/d22/d5f/f9d [0,4194304] 0 2024-05-17T21:53:12.178 INFO:tasks.workunit.client.1.smithi138.stdout:3/964: creat d1/d62/dad/f148 x:0 0 0 2024-05-17T21:53:12.178 INFO:tasks.workunit.client.0.smithi107.stdout:6/988: dread - d9/d16/d4d/d85/d96/d104/f124 zero size 2024-05-17T21:53:12.179 INFO:tasks.workunit.client.0.smithi107.stdout:8/985: rename dc/d21/d97/d63 to dc/d21/daa/db4/d15b 0 2024-05-17T21:53:12.182 INFO:tasks.workunit.client.0.smithi107.stdout:6/989: creat d9/d16/d4d/d85/d140/d135/f147 x:0 0 0 2024-05-17T21:53:12.182 INFO:tasks.workunit.client.1.smithi138.stdout:1/804: mknod d4/dd/d3b/d9f/db9/dae/c102 0 2024-05-17T21:53:12.183 INFO:tasks.workunit.client.1.smithi138.stdout:1/805: write d4/df/d35/f37 [1278656,10903] 0 2024-05-17T21:53:12.209 INFO:tasks.workunit.client.1.smithi138.stdout:4/799: creat d0/d1b/dd4/f113 x:0 0 0 2024-05-17T21:53:12.210 INFO:tasks.workunit.client.0.smithi107.stdout:6/990: symlink d9/d16/d4d/d85/de8/l148 0 2024-05-17T21:53:12.211 INFO:tasks.workunit.client.1.smithi138.stdout:6/807: link d7/d69/db5/dc8/d43/d3f/d79/ffe d7/da0/dbe/f114 0 2024-05-17T21:53:12.213 INFO:tasks.workunit.client.1.smithi138.stdout:0/730: truncate d3/f29 6697004 0 2024-05-17T21:53:12.217 INFO:tasks.workunit.client.0.smithi107.stdout:7/984: rename d5/df/d11/d1c/d31/d39/dc0/l10c to d5/d16/l14b 0 2024-05-17T21:53:12.222 INFO:tasks.workunit.client.1.smithi138.stdout:3/965: rename d1/d3/d8d/d92/f96 to d1/d4f/d50/dbb/f149 0 2024-05-17T21:53:12.223 INFO:tasks.workunit.client.1.smithi138.stdout:3/966: fdatasync d1/d4f/d87/f11b 0 2024-05-17T21:53:12.224 INFO:tasks.workunit.client.0.smithi107.stdout:2/866: write d0/d9/d1c/d37/f50 [125020,101087] 0 2024-05-17T21:53:12.225 INFO:tasks.workunit.client.1.smithi138.stdout:7/770: write d0/d18/d40/f4e [4608055,24212] 0 2024-05-17T21:53:12.225 INFO:tasks.workunit.client.0.smithi107.stdout:6/991: creat d9/d16/d22/f149 x:0 0 0 2024-05-17T21:53:12.227 INFO:tasks.workunit.client.0.smithi107.stdout:6/992: fdatasync d9/d10e/f9b 0 2024-05-17T21:53:12.228 INFO:tasks.workunit.client.1.smithi138.stdout:2/842: truncate d3/f27 933701 0 2024-05-17T21:53:12.230 INFO:tasks.workunit.client.0.smithi107.stdout:2/867: dwrite d0/d9/f60 [0,4194304] 0 2024-05-17T21:53:12.231 INFO:tasks.workunit.client.1.smithi138.stdout:4/800: creat d0/d3/d17/f114 x:0 0 0 2024-05-17T21:53:12.234 INFO:tasks.workunit.client.1.smithi138.stdout:6/808: creat d7/d69/db5/dc8/d43/d63/f115 x:0 0 0 2024-05-17T21:53:12.236 INFO:tasks.workunit.client.0.smithi107.stdout:2/868: read d0/d9/d1c/d22/f5b [3412731,14600] 0 2024-05-17T21:53:12.240 INFO:tasks.workunit.client.1.smithi138.stdout:9/716: mkdir d6/d7/d16/d29/d38/d40/d44/d5c/dc8/de2 0 2024-05-17T21:53:12.242 INFO:tasks.workunit.client.0.smithi107.stdout:7/985: dwrite d5/df/d11/d1c/d31/d39/f5e [4194304,4194304] 0 2024-05-17T21:53:12.242 INFO:tasks.workunit.client.1.smithi138.stdout:9/717: dread - d6/d7/d16/d29/d48/d6e/d74/d75/d76/fbf zero size 2024-05-17T21:53:12.243 INFO:tasks.workunit.client.1.smithi138.stdout:9/718: stat d6/d7/d16/d29/d38/d40/ld4 0 2024-05-17T21:53:12.244 INFO:tasks.workunit.client.1.smithi138.stdout:3/967: mknod d1/dc0/ddd/c14a 0 2024-05-17T21:53:12.247 INFO:tasks.workunit.client.1.smithi138.stdout:2/843: fdatasync d3/d21/d2f/d3e/f6d 0 2024-05-17T21:53:12.252 INFO:tasks.workunit.client.0.smithi107.stdout:7/986: unlink d5/df/d11/d2a/ca6 0 2024-05-17T21:53:12.254 INFO:tasks.workunit.client.0.smithi107.stdout:6/993: dread d9/d16/f4a [0,4194304] 0 2024-05-17T21:53:12.254 INFO:tasks.workunit.client.1.smithi138.stdout:4/801: rmdir d0/d1b/d36 39 2024-05-17T21:53:12.256 INFO:tasks.workunit.client.1.smithi138.stdout:6/809: creat d7/d69/db5/dc8/d43/d5f/d94/dc9/f116 x:0 0 0 2024-05-17T21:53:12.257 INFO:tasks.workunit.client.0.smithi107.stdout:7/987: mkdir d5/df/dd7/d118/d14c 0 2024-05-17T21:53:12.258 INFO:tasks.workunit.client.1.smithi138.stdout:0/731: read - d3/d10/d38/d3e/d59/dad/fd1 zero size 2024-05-17T21:53:12.262 INFO:tasks.workunit.client.0.smithi107.stdout:7/988: dread d5/d16/d3b/ddf/d12e/dc1/fea [0,4194304] 0 2024-05-17T21:53:12.263 INFO:tasks.workunit.client.1.smithi138.stdout:8/698: write d0/dc/d3b/d4d/d7d/f7f [2438805,113712] 0 2024-05-17T21:53:12.264 INFO:tasks.workunit.client.1.smithi138.stdout:2/844: mkdir d3/d21/d2f/d3e/d55/d99/d84/d9f/d116 0 2024-05-17T21:53:12.265 INFO:tasks.workunit.client.0.smithi107.stdout:7/989: rmdir d5/df/dd7 39 2024-05-17T21:53:12.266 INFO:tasks.workunit.client.0.smithi107.stdout:7/990: write d5/d16/d3b/ddf/d12e/dac/d107/f143 [1039535,73271] 0 2024-05-17T21:53:12.266 INFO:tasks.workunit.client.1.smithi138.stdout:7/771: write d0/f10 [8351552,75763] 0 2024-05-17T21:53:12.269 INFO:tasks.workunit.client.0.smithi107.stdout:2/869: truncate d0/d9/d11/d27/dd3/fc1 3072391 0 2024-05-17T21:53:12.270 INFO:tasks.workunit.client.1.smithi138.stdout:6/810: truncate d7/d8/f42 1848369 0 2024-05-17T21:53:12.270 INFO:tasks.workunit.client.1.smithi138.stdout:1/806: truncate d4/d2e/d36/f91 10252 0 2024-05-17T21:53:12.271 INFO:tasks.workunit.client.1.smithi138.stdout:6/811: chown d7/d69/db5/dc8/d43/d5f/d6a/d9b/db2 56 1 2024-05-17T21:53:12.273 INFO:tasks.workunit.client.0.smithi107.stdout:2/870: dread d0/d9/d11/df/d5a/d6d/d82/f92 [0,4194304] 0 2024-05-17T21:53:12.274 INFO:tasks.workunit.client.1.smithi138.stdout:9/719: mknod d6/ce3 0 2024-05-17T21:53:12.275 INFO:tasks.workunit.client.0.smithi107.stdout:6/994: write d9/d16/d22/d5f/d6e/f87 [1499699,89394] 0 2024-05-17T21:53:12.276 INFO:tasks.workunit.client.1.smithi138.stdout:9/720: truncate d6/d7/d16/dc2/fe0 209404 0 2024-05-17T21:53:12.276 INFO:tasks.workunit.client.0.smithi107.stdout:6/995: write d9/d81/f82 [3959157,125364] 0 2024-05-17T21:53:12.278 INFO:tasks.workunit.client.1.smithi138.stdout:2/845: dread - d3/d21/d2f/d3e/d58/d77/fd1 zero size 2024-05-17T21:53:12.278 INFO:tasks.workunit.client.1.smithi138.stdout:9/721: chown d6/d7/d16/d29/d6c/d9c/dc1/f92 28 1 2024-05-17T21:53:12.279 INFO:tasks.workunit.client.1.smithi138.stdout:2/846: readlink d3/d21/d2f/l4b 0 2024-05-17T21:53:12.279 INFO:tasks.workunit.client.1.smithi138.stdout:9/722: chown d6/d7/d16/f1a 46016618 1 2024-05-17T21:53:12.280 INFO:tasks.workunit.client.1.smithi138.stdout:4/802: dread d0/fcb [0,4194304] 0 2024-05-17T21:53:12.282 INFO:tasks.workunit.client.1.smithi138.stdout:9/723: write d6/d7/d16/d29/d6c/d9c/fd0 [929422,55971] 0 2024-05-17T21:53:12.282 INFO:tasks.workunit.client.1.smithi138.stdout:2/847: read d3/d21/d2f/d3e/d58/f71 [101893,44603] 0 2024-05-17T21:53:12.288 INFO:tasks.workunit.client.1.smithi138.stdout:0/732: rename d3/cc8 to d3/d10/d38/d3e/d59/cf5 0 2024-05-17T21:53:12.292 INFO:tasks.workunit.client.1.smithi138.stdout:4/803: truncate d0/d1b/dd4/fed 99046 0 2024-05-17T21:53:12.293 INFO:tasks.workunit.client.1.smithi138.stdout:3/968: getdents d1/d3/d16/d2d/df6 0 2024-05-17T21:53:12.295 INFO:tasks.workunit.client.1.smithi138.stdout:8/699: truncate d0/dc/fbf 4901248 0 2024-05-17T21:53:12.297 INFO:tasks.workunit.client.0.smithi107.stdout:2/871: dwrite d0/d9/d11/d18/dad/db8/fbd [0,4194304] 0 2024-05-17T21:53:12.300 INFO:tasks.workunit.client.0.smithi107.stdout:6/996: dwrite d9/d16/f24 [4194304,4194304] 0 2024-05-17T21:53:12.300 INFO:tasks.workunit.client.1.smithi138.stdout:6/812: dwrite d7/d69/db5/dc8/f25 [0,4194304] 0 2024-05-17T21:53:12.301 INFO:tasks.workunit.client.0.smithi107.stdout:2/872: dread d0/d9/d1c/d37/f50 [0,4194304] 0 2024-05-17T21:53:12.302 INFO:tasks.workunit.client.1.smithi138.stdout:0/733: write d3/d10/d38/d3e/d59/f9d [2879038,57753] 0 2024-05-17T21:53:12.302 INFO:tasks.workunit.client.0.smithi107.stdout:6/997: read d9/d16/d4d/f52 [6462616,103135] 0 2024-05-17T21:53:12.303 INFO:tasks.workunit.client.1.smithi138.stdout:1/807: link d4/df/l23 d4/dd/d3b/d6c/dc1/l103 0 2024-05-17T21:53:12.305 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:11 smithi107 ceph-mon[28434]: from='mgr.14636 ' entity='mgr.smithi138.napnll' 2024-05-17T21:53:12.305 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:11 smithi107 ceph-mon[28434]: from='mgr.14636 ' entity='mgr.smithi138.napnll' 2024-05-17T21:53:12.305 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:11 smithi107 ceph-mon[28434]: pgmap v44: 65 pgs: 65 active+clean; 3.5 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 133 MiB/s rd, 288 MiB/s wr, 672 op/s 2024-05-17T21:53:12.313 INFO:tasks.workunit.client.1.smithi138.stdout:6/813: dwrite d7/d69/db5/dc8/d43/d5f/da4/dcc/fe6 [0,4194304] 0 2024-05-17T21:53:12.314 INFO:tasks.workunit.client.1.smithi138.stdout:6/814: chown d7/d69/db5/dc8/d60/ce0 21 1 2024-05-17T21:53:12.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:11 smithi138 ceph-mon[35948]: from='mgr.14636 ' entity='mgr.smithi138.napnll' 2024-05-17T21:53:12.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:11 smithi138 ceph-mon[35948]: from='mgr.14636 ' entity='mgr.smithi138.napnll' 2024-05-17T21:53:12.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:11 smithi138 ceph-mon[35948]: pgmap v44: 65 pgs: 65 active+clean; 3.5 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 133 MiB/s rd, 288 MiB/s wr, 672 op/s 2024-05-17T21:53:12.328 INFO:tasks.workunit.client.1.smithi138.stdout:8/700: dread d0/d71/fa5 [0,4194304] 0 2024-05-17T21:53:12.329 INFO:tasks.workunit.client.1.smithi138.stdout:7/772: rename d0/d18/d40/d5f/f8d to d0/d18/d40/d41/d51/d81/ffb 0 2024-05-17T21:53:12.329 INFO:tasks.workunit.client.0.smithi107.stdout:8/986: sync 2024-05-17T21:53:12.330 INFO:tasks.workunit.client.0.smithi107.stdout:6/998: getdents d9/d16/d22/de6 0 2024-05-17T21:53:12.330 INFO:tasks.workunit.client.0.smithi107.stdout:8/987: fsync dc/d44/f131 0 2024-05-17T21:53:12.331 INFO:tasks.workunit.client.0.smithi107.stdout:8/988: write dc/d21/daa/dc5/f111 [1478961,112344] 0 2024-05-17T21:53:12.332 INFO:tasks.workunit.client.1.smithi138.stdout:0/734: creat d3/d10/d6d/db6/db9/ff6 x:0 0 0 2024-05-17T21:53:12.334 INFO:tasks.workunit.client.1.smithi138.stdout:4/804: symlink d0/d1b/d2b/d55/l115 0 2024-05-17T21:53:12.336 INFO:tasks.workunit.client.1.smithi138.stdout:3/969: creat d1/d12f/f14b x:0 0 0 2024-05-17T21:53:12.336 INFO:tasks.workunit.client.1.smithi138.stdout:4/805: fsync d0/d3/d17/d21/d78/fcd 0 2024-05-17T21:53:12.340 INFO:tasks.workunit.client.0.smithi107.stdout:2/873: dread d0/d9/d11/f5c [0,4194304] 0 2024-05-17T21:53:12.340 INFO:tasks.workunit.client.1.smithi138.stdout:2/848: dwrite d3/d21/d2f/d3e/d55/d73/f97 [0,4194304] 0 2024-05-17T21:53:12.341 INFO:tasks.workunit.client.0.smithi107.stdout:6/999: mknod d9/d16/d22/c14a 0 2024-05-17T21:53:12.346 INFO:tasks.workunit.client.0.smithi107.stdout:2/874: symlink d0/d9/d1c/d37/l122 0 2024-05-17T21:53:12.350 INFO:tasks.workunit.client.1.smithi138.stdout:5/799: sync 2024-05-17T21:53:12.352 INFO:tasks.workunit.client.1.smithi138.stdout:8/701: symlink d0/d63/lcc 0 2024-05-17T21:53:12.355 INFO:tasks.workunit.client.1.smithi138.stdout:1/808: creat d4/db/d60/dcc/f104 x:0 0 0 2024-05-17T21:53:12.356 INFO:tasks.workunit.client.1.smithi138.stdout:1/809: read - d4/db/d29/d89/ffd zero size 2024-05-17T21:53:12.357 INFO:tasks.workunit.client.1.smithi138.stdout:1/810: chown d4/dd/d3b/d6c/dc1/cf1 43431894 1 2024-05-17T21:53:12.358 INFO:tasks.workunit.client.0.smithi107.stdout:8/989: write dc/d21/d27/d34/d4c/d7e/f45 [3696523,5834] 0 2024-05-17T21:53:12.358 INFO:tasks.workunit.client.1.smithi138.stdout:1/811: fsync d4/df/d3a/fc5 0 2024-05-17T21:53:12.360 INFO:tasks.workunit.client.1.smithi138.stdout:4/806: fdatasync d0/d1b/d42/fd7 0 2024-05-17T21:53:12.361 INFO:tasks.workunit.client.0.smithi107.stdout:8/990: stat dc/d21/daa/db4/d15b/c65 0 2024-05-17T21:53:12.363 INFO:tasks.workunit.client.0.smithi107.stdout:2/875: dwrite d0/d9/d1c/d37/f98 [0,4194304] 0 2024-05-17T21:53:12.364 INFO:tasks.workunit.client.1.smithi138.stdout:2/849: unlink d3/d81/cbf 0 2024-05-17T21:53:12.365 INFO:tasks.workunit.client.0.smithi107.stdout:2/876: truncate d0/d9/d11/d18/dad/fe6 699494 0 2024-05-17T21:53:12.365 INFO:tasks.workunit.client.0.smithi107.stdout:8/991: fsync dc/d21/db8/fc4 0 2024-05-17T21:53:12.367 INFO:tasks.workunit.client.1.smithi138.stdout:6/815: unlink d7/d69/db5/dc8/ce 0 2024-05-17T21:53:12.368 INFO:tasks.workunit.client.1.smithi138.stdout:9/724: getdents d6/d7/d16/d29/d6c/d9c/dc1 0 2024-05-17T21:53:12.370 INFO:tasks.workunit.client.1.smithi138.stdout:5/800: rename d1/d46/d95/cd6 to d1/d53/d88/de5/ded/dc2/c101 0 2024-05-17T21:53:12.372 INFO:tasks.workunit.client.0.smithi107.stdout:2/877: fsync d0/d9/d11/df/d5a/fc0 0 2024-05-17T21:53:12.372 INFO:tasks.workunit.client.1.smithi138.stdout:5/801: write d1/d2c/d63/db5/d1b/f54 [8562028,21921] 0 2024-05-17T21:53:12.373 INFO:tasks.workunit.client.0.smithi107.stdout:2/878: readlink d0/d9/d11/d18/dad/db8/lf1 0 2024-05-17T21:53:12.374 INFO:tasks.workunit.client.1.smithi138.stdout:8/702: creat d0/dc/d1a/d78/fcd x:0 0 0 2024-05-17T21:53:12.376 INFO:tasks.workunit.client.1.smithi138.stdout:7/773: mkdir d0/d18/d40/d41/d51/d66/d9e/df9/dfc 0 2024-05-17T21:53:12.376 INFO:tasks.workunit.client.0.smithi107.stdout:2/879: fdatasync d0/d9/d11/df/f47 0 2024-05-17T21:53:12.377 INFO:tasks.workunit.client.1.smithi138.stdout:7/774: fsync d0/d18/d40/d5f/dbe/fed 0 2024-05-17T21:53:12.377 INFO:tasks.workunit.client.1.smithi138.stdout:0/735: truncate d3/d3d/f7f 153320 0 2024-05-17T21:53:12.379 INFO:tasks.workunit.client.1.smithi138.stdout:8/703: read d0/d63/fbb [45954,78684] 0 2024-05-17T21:53:12.380 INFO:tasks.workunit.client.0.smithi107.stdout:2/880: rename d0/l69 to d0/d9/d11/d18/d9e/l123 0 2024-05-17T21:53:12.381 INFO:tasks.workunit.client.0.smithi107.stdout:7/991: sync 2024-05-17T21:53:12.381 INFO:tasks.workunit.client.0.smithi107.stdout:7/992: dread - d5/df/d11/d1c/d2f/f104 zero size 2024-05-17T21:53:12.386 INFO:tasks.workunit.client.1.smithi138.stdout:4/807: creat d0/d2f/d49/d60/d1f/d32/d71/d95/f116 x:0 0 0 2024-05-17T21:53:12.386 INFO:tasks.workunit.client.0.smithi107.stdout:8/992: getdents dc/d21/d27/d2f/dba/dee 0 2024-05-17T21:53:12.387 INFO:tasks.workunit.client.0.smithi107.stdout:7/993: truncate d5/df/d11/d1c/d31/d39/dc0/fe9 410020 0 2024-05-17T21:53:12.390 INFO:tasks.workunit.client.1.smithi138.stdout:1/812: rename d4/db/f2a to d4/df/d3a/d3c/d63/d6d/de0/f105 0 2024-05-17T21:53:12.392 INFO:tasks.workunit.client.1.smithi138.stdout:5/802: chown d1/d53/d88/de5/ded/cf8 1180252 1 2024-05-17T21:53:12.394 INFO:tasks.workunit.client.1.smithi138.stdout:0/736: write d3/fe3 [983477,26144] 0 2024-05-17T21:53:12.395 INFO:tasks.workunit.client.1.smithi138.stdout:0/737: chown d3/d10/d6d/le0 1623843 1 2024-05-17T21:53:12.396 INFO:tasks.workunit.client.1.smithi138.stdout:3/970: rmdir d1/d62/dad/de/d120 0 2024-05-17T21:53:12.397 INFO:tasks.workunit.client.1.smithi138.stdout:8/704: chown d0/dc/d3b/l95 1775 1 2024-05-17T21:53:12.399 INFO:tasks.workunit.client.0.smithi107.stdout:2/881: rename d0/d9/d11/d27/c90 to d0/d9/d1c/d37/df8/d10a/c124 0 2024-05-17T21:53:12.400 INFO:tasks.workunit.client.0.smithi107.stdout:8/993: fsync dc/d21/daa/db4/d15b/f133 0 2024-05-17T21:53:12.401 INFO:tasks.workunit.client.0.smithi107.stdout:7/994: truncate d5/d16/f1e 2551837 0 2024-05-17T21:53:12.402 INFO:tasks.workunit.client.1.smithi138.stdout:2/850: creat d3/d21/d2f/d3e/d55/d99/d84/d9f/d116/f117 x:0 0 0 2024-05-17T21:53:12.403 INFO:tasks.workunit.client.0.smithi107.stdout:7/995: stat d5/df/d11/d1c/d31/d39/d137 0 2024-05-17T21:53:12.403 INFO:tasks.workunit.client.0.smithi107.stdout:2/882: fdatasync d0/d93/fb3 0 2024-05-17T21:53:12.404 INFO:tasks.workunit.client.0.smithi107.stdout:8/994: mkdir dc/d44/d4b/d52/d15c 0 2024-05-17T21:53:12.404 INFO:tasks.workunit.client.0.smithi107.stdout:2/883: chown d0/d93/dba 62635 1 2024-05-17T21:53:12.405 INFO:tasks.workunit.client.0.smithi107.stdout:7/996: read d5/df/d11/d1c/d31/d39/d92/d9a/fa1 [291553,96200] 0 2024-05-17T21:53:12.405 INFO:tasks.workunit.client.0.smithi107.stdout:8/995: truncate dc/d44/f131 1586825 0 2024-05-17T21:53:12.406 INFO:tasks.workunit.client.1.smithi138.stdout:7/775: rename d0/d18/d40/d41/c4f to d0/d18/d40/d41/d51/d66/d9e/cfd 0 2024-05-17T21:53:12.408 INFO:tasks.workunit.client.1.smithi138.stdout:5/803: mknod d1/d2c/d63/db5/d1b/c102 0 2024-05-17T21:53:12.412 INFO:tasks.workunit.client.0.smithi107.stdout:8/996: truncate dc/f4f 2252935 0 2024-05-17T21:53:12.413 INFO:tasks.workunit.client.1.smithi138.stdout:9/725: write d6/d7/fc [1074196,60428] 0 2024-05-17T21:53:12.414 INFO:tasks.workunit.client.0.smithi107.stdout:7/997: mknod d5/df/d11/d1c/d31/d56/d13b/c14d 0 2024-05-17T21:53:12.415 INFO:tasks.workunit.client.1.smithi138.stdout:9/726: truncate d6/fd9 282481 0 2024-05-17T21:53:12.416 INFO:tasks.workunit.client.0.smithi107.stdout:7/998: chown d5/df/d11/d1c/d31/d39/d92/d9a/l10f 18227296 1 2024-05-17T21:53:12.416 INFO:tasks.workunit.client.1.smithi138.stdout:5/804: dwrite d1/d53/d96/da0/ff7 [0,4194304] 0 2024-05-17T21:53:12.418 INFO:tasks.workunit.client.0.smithi107.stdout:7/999: dread d5/df/d11/d1c/d31/d39/d92/d9a/fa1 [0,4194304] 0 2024-05-17T21:53:12.423 INFO:tasks.workunit.client.0.smithi107.stdout:2/884: link d0/d9/d11/df/d5a/d6d/d82/l86 d0/d9/d1c/dd4/l125 0 2024-05-17T21:53:12.424 INFO:tasks.workunit.client.1.smithi138.stdout:2/851: mkdir d3/d21/d2f/dd4/d118 0 2024-05-17T21:53:12.428 INFO:tasks.workunit.client.1.smithi138.stdout:7/776: mkdir d0/d18/d40/d41/d51/d66/d9e/df9/dfe 0 2024-05-17T21:53:12.431 INFO:tasks.workunit.client.1.smithi138.stdout:5/805: creat d1/d53/d88/de5/f103 x:0 0 0 2024-05-17T21:53:12.433 INFO:tasks.workunit.client.1.smithi138.stdout:8/705: symlink d0/dc/lce 0 2024-05-17T21:53:12.434 INFO:tasks.workunit.client.0.smithi107.stdout:8/997: dread dc/d21/d35/f2e [0,4194304] 0 2024-05-17T21:53:12.436 INFO:tasks.workunit.client.1.smithi138.stdout:6/816: getdents d7/d69/db5/dc8/d43/d3f/d79 0 2024-05-17T21:53:12.440 INFO:tasks.workunit.client.1.smithi138.stdout:1/813: creat d4/d2e/f106 x:0 0 0 2024-05-17T21:53:12.443 INFO:tasks.workunit.client.1.smithi138.stdout:6/817: dwrite d7/d69/db5/dc8/d43/d3f/f56 [0,4194304] 0 2024-05-17T21:53:12.443 INFO:tasks.workunit.client.1.smithi138.stdout:9/727: mkdir d6/de4 0 2024-05-17T21:53:12.444 INFO:tasks.workunit.client.1.smithi138.stdout:6/818: readlink d7/d69/lab 0 2024-05-17T21:53:12.445 INFO:tasks.workunit.client.0.smithi107.stdout:2/885: dread d0/d9/d1c/d37/f10f [0,4194304] 0 2024-05-17T21:53:12.445 INFO:tasks.workunit.client.0.smithi107.stdout:8/998: getdents dc/d21/d97/db7/dcd/d106 0 2024-05-17T21:53:12.447 INFO:tasks.workunit.client.1.smithi138.stdout:3/971: write d1/d3/d49/fff [795401,113800] 0 2024-05-17T21:53:12.448 INFO:tasks.workunit.client.1.smithi138.stdout:3/972: chown d1/d3 0 1 2024-05-17T21:53:12.449 INFO:tasks.workunit.client.1.smithi138.stdout:8/706: rename d0/dc/d1a/c53 to d0/d11/d3e/ccf 0 2024-05-17T21:53:12.451 INFO:tasks.workunit.client.1.smithi138.stdout:4/808: getdents d0/d3/d1d/d38/d9c 0 2024-05-17T21:53:12.452 INFO:tasks.workunit.client.0.smithi107.stdout:2/886: creat d0/d9/d1c/d22/f126 x:0 0 0 2024-05-17T21:53:12.452 INFO:tasks.workunit.client.1.smithi138.stdout:0/738: dwrite d3/f8e [0,4194304] 0 2024-05-17T21:53:12.453 INFO:tasks.workunit.client.0.smithi107.stdout:8/999: dwrite dc/d21/d27/d34/f38 [0,4194304] 0 2024-05-17T21:53:12.457 INFO:tasks.workunit.client.0.smithi107.stdout:2/887: creat d0/d9/d11/d18/d9e/deb/d109/f127 x:0 0 0 2024-05-17T21:53:12.457 INFO:tasks.workunit.client.1.smithi138.stdout:7/777: getdents d0/d4c/dec 0 2024-05-17T21:53:12.458 INFO:tasks.workunit.client.1.smithi138.stdout:7/778: chown d0/d18/d40 1 1 2024-05-17T21:53:12.459 INFO:tasks.workunit.client.1.smithi138.stdout:0/739: dread d3/fe3 [0,4194304] 0 2024-05-17T21:53:12.459 INFO:tasks.workunit.client.1.smithi138.stdout:1/814: symlink d4/df/d3a/d3c/d63/l107 0 2024-05-17T21:53:12.459 INFO:tasks.workunit.client.1.smithi138.stdout:7/779: chown d0/d18/d40/f58 34102 1 2024-05-17T21:53:12.462 INFO:tasks.workunit.client.1.smithi138.stdout:1/815: write d4/df/d3a/d3c/d63/d6d/fdc [478823,17305] 0 2024-05-17T21:53:12.464 INFO:tasks.workunit.client.1.smithi138.stdout:3/973: fsync d1/d62/dad/d5c/f81 0 2024-05-17T21:53:12.540 INFO:tasks.workunit.client.1.smithi138.stdout:2/852: link d3/d21/d2f/d3e/d55/d99/d84/dbe/dd8/l115 d3/d21/d2f/d3e/d55/d99/dd2/l119 0 2024-05-17T21:53:12.540 INFO:tasks.workunit.client.1.smithi138.stdout:7/780: dread - d0/d39/f9b zero size 2024-05-17T21:53:12.543 INFO:tasks.workunit.client.1.smithi138.stdout:4/809: mkdir d0/d3/d1d/d38/de3/d117 0 2024-05-17T21:53:12.543 INFO:tasks.workunit.client.1.smithi138.stdout:7/781: chown d0/d18/d40/d41/cba 56 1 2024-05-17T21:53:12.543 INFO:tasks.workunit.client.1.smithi138.stdout:1/816: stat d4/dd/d3b/d6c/dc1/l103 0 2024-05-17T21:53:12.547 INFO:tasks.workunit.client.1.smithi138.stdout:1/817: write d4/dd/d3b/d6c/dcf/f85 [5391530,52190] 0 2024-05-17T21:53:12.547 INFO:tasks.workunit.client.1.smithi138.stdout:3/974: dwrite d1/d3/d16/d3e/d141/f6e [0,4194304] 0 2024-05-17T21:53:12.548 INFO:tasks.workunit.client.1.smithi138.stdout:7/782: write d0/d18/d40/d41/d51/ff4 [473488,54850] 0 2024-05-17T21:53:12.549 INFO:tasks.workunit.client.1.smithi138.stdout:4/810: mknod d0/d2f/d49/d60/d1f/d7e/c118 0 2024-05-17T21:53:12.549 INFO:tasks.workunit.client.1.smithi138.stdout:1/818: readlink d4/d2e/d36/l71 0 2024-05-17T21:53:12.553 INFO:tasks.workunit.client.1.smithi138.stdout:3/975: fdatasync d1/d62/dad/de/f3f 0 2024-05-17T21:53:12.555 INFO:tasks.workunit.client.1.smithi138.stdout:4/811: mknod d0/d1b/d42/d45/d84/c119 0 2024-05-17T21:53:12.556 INFO:tasks.workunit.client.1.smithi138.stdout:1/819: mkdir d4/dd/d3b/da7/d108 0 2024-05-17T21:53:12.558 INFO:tasks.workunit.client.1.smithi138.stdout:1/820: stat d4/db/d29/d89/c77 0 2024-05-17T21:53:12.559 INFO:tasks.workunit.client.1.smithi138.stdout:4/812: mknod d0/d3/db/da0/c11a 0 2024-05-17T21:53:12.561 INFO:tasks.workunit.client.1.smithi138.stdout:1/821: mknod d4/dd/d3b/d6c/c109 0 2024-05-17T21:53:12.565 INFO:tasks.workunit.client.1.smithi138.stdout:4/813: fsync d0/d3/d17/d5e/f8e 0 2024-05-17T21:53:12.567 INFO:tasks.workunit.client.1.smithi138.stdout:4/814: mknod d0/d1b/d88/c11b 0 2024-05-17T21:53:12.569 INFO:tasks.workunit.client.1.smithi138.stdout:1/822: rename d4/df/d4e/f55 to d4/dd/d3b/d9f/da9/f10a 0 2024-05-17T21:53:12.578 INFO:tasks.workunit.client.0.smithi107.stdout:2/888: dwrite d0/d9/d1c/d37/d9f/fa7 [4194304,4194304] 0 2024-05-17T21:53:12.581 INFO:tasks.workunit.client.1.smithi138.stdout:3/976: write d1/d3/f25 [2095281,5662] 0 2024-05-17T21:53:12.583 INFO:tasks.workunit.client.0.smithi107.stdout:2/889: creat d0/d26/ddd/f128 x:0 0 0 2024-05-17T21:53:12.586 INFO:tasks.workunit.client.1.smithi138.stdout:1/823: dwrite d4/df/d3a/d3c/f88 [4194304,4194304] 0 2024-05-17T21:53:12.588 INFO:tasks.workunit.client.0.smithi107.stdout:2/890: rename d0/d9/d11/d18/d9e/ld0 to d0/d9/d11/d18/d9e/deb/l129 0 2024-05-17T21:53:12.591 INFO:tasks.workunit.client.1.smithi138.stdout:4/815: write d0/d2f/f82 [1438471,20871] 0 2024-05-17T21:53:12.597 INFO:tasks.workunit.client.0.smithi107.stdout:2/891: dwrite d0/d9/d11/df/d5a/d6d/ffc [4194304,4194304] 0 2024-05-17T21:53:12.598 INFO:tasks.workunit.client.1.smithi138.stdout:6/819: dwrite d7/d69/db5/dc8/d43/da1/fcb [0,4194304] 0 2024-05-17T21:53:12.602 INFO:tasks.workunit.client.1.smithi138.stdout:0/740: dwrite d3/d10/f25 [4194304,4194304] 0 2024-05-17T21:53:12.603 INFO:tasks.workunit.client.1.smithi138.stdout:9/728: dwrite d6/d7/d16/d29/d48/d6e/d74/d75/d76/f7f [4194304,4194304] 0 2024-05-17T21:53:12.608 INFO:tasks.workunit.client.1.smithi138.stdout:1/824: fdatasync d4/dd/db4/fe4 0 2024-05-17T21:53:12.609 INFO:tasks.workunit.client.1.smithi138.stdout:2/853: dwrite d3/d21/d2f/f39 [0,4194304] 0 2024-05-17T21:53:12.610 INFO:tasks.workunit.client.1.smithi138.stdout:8/707: dwrite d0/d2f/f45 [0,4194304] 0 2024-05-17T21:53:12.615 INFO:tasks.workunit.client.1.smithi138.stdout:7/783: dwrite d0/d18/d40/d9a/fb8 [0,4194304] 0 2024-05-17T21:53:12.617 INFO:tasks.workunit.client.1.smithi138.stdout:1/825: dread - d4/dd/d3b/d9f/da9/dd9/ffe zero size 2024-05-17T21:53:12.625 INFO:tasks.workunit.client.1.smithi138.stdout:8/708: fdatasync d0/dc/d1a/d51/f73 0 2024-05-17T21:53:12.625 INFO:tasks.workunit.client.1.smithi138.stdout:0/741: creat d3/d3d/ff7 x:0 0 0 2024-05-17T21:53:12.625 INFO:tasks.workunit.client.1.smithi138.stdout:6/820: write d7/d69/db5/dc8/d43/d3f/f5a [181973,53604] 0 2024-05-17T21:53:12.625 INFO:tasks.workunit.client.1.smithi138.stdout:3/977: dwrite d1/d3/d16/d3e/d71/f9a [0,4194304] 0 2024-05-17T21:53:12.634 INFO:tasks.workunit.client.1.smithi138.stdout:6/821: write d7/d69/db5/dc8/d43/d5f/d94/dc9/f116 [365407,83922] 0 2024-05-17T21:53:12.634 INFO:tasks.workunit.client.1.smithi138.stdout:4/816: link d0/d3/d1d/f97 d0/d3/d17/d21/d78/d10f/f11c 0 2024-05-17T21:53:12.635 INFO:tasks.workunit.client.1.smithi138.stdout:9/729: write d6/d7/d16/f1a [1291705,35578] 0 2024-05-17T21:53:12.637 INFO:tasks.workunit.client.1.smithi138.stdout:1/826: rename d4/la2 to d4/dd/d3b/l10b 0 2024-05-17T21:53:12.639 INFO:tasks.workunit.client.1.smithi138.stdout:7/784: dwrite d0/d18/d40/d41/d51/d81/fa3 [0,4194304] 0 2024-05-17T21:53:12.640 INFO:tasks.workunit.client.1.smithi138.stdout:8/709: creat d0/dc/d1a/d78/fd0 x:0 0 0 2024-05-17T21:53:12.641 INFO:tasks.workunit.client.1.smithi138.stdout:1/827: stat d4/db/d29/d89/c7f 0 2024-05-17T21:53:12.645 INFO:tasks.workunit.client.1.smithi138.stdout:0/742: link d3/d10/f15 d3/d10/d6d/db6/db9/ff8 0 2024-05-17T21:53:12.646 INFO:tasks.workunit.client.1.smithi138.stdout:9/730: chown d6/d7/d16/d29/d48/d6e/d74/d75/d76/d8e/fa7 111767 1 2024-05-17T21:53:12.649 INFO:tasks.workunit.client.1.smithi138.stdout:3/978: dwrite d1/d3/d16/d3e/f43 [0,4194304] 0 2024-05-17T21:53:12.649 INFO:tasks.workunit.client.1.smithi138.stdout:6/822: truncate d7/d69/db5/dc8/d34/da6/f27 1525441 0 2024-05-17T21:53:12.650 INFO:tasks.workunit.client.1.smithi138.stdout:9/731: fsync d6/d7/d16/d29/d38/d3c/fd3 0 2024-05-17T21:53:12.655 INFO:tasks.workunit.client.1.smithi138.stdout:4/817: dread d0/d3/d1d/d38/d8a/dbd/ff5 [0,4194304] 0 2024-05-17T21:53:12.657 INFO:tasks.workunit.client.1.smithi138.stdout:1/828: write d4/df/d4e/f84 [5831667,124159] 0 2024-05-17T21:53:12.658 INFO:tasks.workunit.client.1.smithi138.stdout:6/823: chown d7/d69/db5/dc8/d43/d63/f87 658777 1 2024-05-17T21:53:12.658 INFO:tasks.workunit.client.1.smithi138.stdout:2/854: dread d3/d21/f31 [0,4194304] 0 2024-05-17T21:53:12.661 INFO:tasks.workunit.client.1.smithi138.stdout:3/979: rename d1/d2e to d1/d62/db0/d13f/d14c 0 2024-05-17T21:53:12.663 INFO:tasks.workunit.client.1.smithi138.stdout:8/710: dwrite d0/d2f/f6e [0,4194304] 0 2024-05-17T21:53:12.666 INFO:tasks.workunit.client.1.smithi138.stdout:6/824: chown d7/d69/db5/dc8/d43/c30 11 1 2024-05-17T21:53:12.669 INFO:tasks.workunit.client.1.smithi138.stdout:4/818: creat d0/d1b/d2b/dca/d111/f11d x:0 0 0 2024-05-17T21:53:12.670 INFO:tasks.workunit.client.1.smithi138.stdout:2/855: symlink d3/d81/d96/da8/l11a 0 2024-05-17T21:53:12.670 INFO:tasks.workunit.client.1.smithi138.stdout:9/732: dwrite d6/d7/f99 [0,4194304] 0 2024-05-17T21:53:12.673 INFO:tasks.workunit.client.1.smithi138.stdout:8/711: creat d0/dc/d1a/d78/fd1 x:0 0 0 2024-05-17T21:53:12.675 INFO:tasks.workunit.client.1.smithi138.stdout:3/980: getdents d1/d4f/d50/dbb/dde 0 2024-05-17T21:53:12.676 INFO:tasks.workunit.client.1.smithi138.stdout:7/785: dread d0/d39/f80 [0,4194304] 0 2024-05-17T21:53:12.678 INFO:tasks.workunit.client.1.smithi138.stdout:4/819: unlink d0/d2f/d49/d60/d1f/d32/d71/l76 0 2024-05-17T21:53:12.680 INFO:tasks.workunit.client.1.smithi138.stdout:6/825: symlink d7/d69/db5/dc8/d43/d5f/d94/dc9/dcf/l117 0 2024-05-17T21:53:12.681 INFO:tasks.workunit.client.1.smithi138.stdout:0/743: write f0 [729388,98105] 0 2024-05-17T21:53:12.683 INFO:tasks.workunit.client.1.smithi138.stdout:0/744: dread - d3/d10/d6d/db6/db9/ff6 zero size 2024-05-17T21:53:12.683 INFO:tasks.workunit.client.1.smithi138.stdout:6/826: write d7/d69/db5/dc8/d43/d3f/d79/f93 [1072932,49205] 0 2024-05-17T21:53:12.686 INFO:tasks.workunit.client.1.smithi138.stdout:5/806: sync 2024-05-17T21:53:12.691 INFO:tasks.workunit.client.1.smithi138.stdout:1/829: dread d4/df/f2c [8388608,4194304] 0 2024-05-17T21:53:12.702 INFO:tasks.workunit.client.1.smithi138.stdout:3/981: symlink d1/d3/d8d/l14d 0 2024-05-17T21:53:12.706 INFO:tasks.workunit.client.1.smithi138.stdout:9/733: creat d6/fe5 x:0 0 0 2024-05-17T21:53:12.707 INFO:tasks.workunit.client.1.smithi138.stdout:9/734: write d6/fe5 [102104,119414] 0 2024-05-17T21:53:12.707 INFO:tasks.workunit.client.1.smithi138.stdout:4/820: unlink d0/d2f/d49/c54 0 2024-05-17T21:53:12.710 INFO:tasks.workunit.client.1.smithi138.stdout:5/807: fsync d1/d2c/d63/db5/d1b/fba 0 2024-05-17T21:53:12.711 INFO:tasks.workunit.client.1.smithi138.stdout:1/830: creat d4/df/d35/df3/f10c x:0 0 0 2024-05-17T21:53:12.712 INFO:tasks.workunit.client.1.smithi138.stdout:3/982: chown d1/d4f/d80/de5/l10a 9681 1 2024-05-17T21:53:12.716 INFO:tasks.workunit.client.1.smithi138.stdout:4/821: mkdir d0/d3/d17/d21/d78/d9f/d11e 0 2024-05-17T21:53:12.717 INFO:tasks.workunit.client.1.smithi138.stdout:4/822: chown d0/d2f/d49/d60 54287172 1 2024-05-17T21:53:12.718 INFO:tasks.workunit.client.1.smithi138.stdout:1/831: dwrite d4/db/d29/d89/ffd [0,4194304] 0 2024-05-17T21:53:12.719 INFO:tasks.workunit.client.1.smithi138.stdout:0/745: mknod d3/d10/d38/d3e/d59/cf9 0 2024-05-17T21:53:12.720 INFO:tasks.workunit.client.1.smithi138.stdout:1/832: rename d4/dd/d64 to d4/dd/d64/d10d 22 2024-05-17T21:53:12.721 INFO:tasks.workunit.client.1.smithi138.stdout:5/808: mknod d1/d53/d96/da0/c104 0 2024-05-17T21:53:12.721 INFO:tasks.workunit.client.1.smithi138.stdout:2/856: link d3/d21/l3d d3/d21/d2f/d3e/d58/d77/da4/df2/dd0/l11b 0 2024-05-17T21:53:12.722 INFO:tasks.workunit.client.1.smithi138.stdout:5/809: stat d1/d60/d2f/d5e/de4/l97 0 2024-05-17T21:53:12.723 INFO:tasks.workunit.client.1.smithi138.stdout:2/857: stat d3/d81/d96/da8/lcf 0 2024-05-17T21:53:12.723 INFO:tasks.workunit.client.1.smithi138.stdout:3/983: symlink d1/d3/d16/d3e/d71/db2/l14e 0 2024-05-17T21:53:12.724 INFO:tasks.workunit.client.1.smithi138.stdout:5/810: chown d1/d53/d88/lce 27 1 2024-05-17T21:53:12.726 INFO:tasks.workunit.client.1.smithi138.stdout:9/735: fdatasync d6/d7/d16/d29/f3a 0 2024-05-17T21:53:12.726 INFO:tasks.workunit.client.1.smithi138.stdout:7/786: dwrite d0/d18/d64/d7c/fe8 [0,4194304] 0 2024-05-17T21:53:12.728 INFO:tasks.workunit.client.1.smithi138.stdout:5/811: chown d1/dd1/c86 8498661 1 2024-05-17T21:53:12.729 INFO:tasks.workunit.client.1.smithi138.stdout:6/827: write d7/d69/fa7 [4320701,11077] 0 2024-05-17T21:53:12.733 INFO:tasks.workunit.client.1.smithi138.stdout:9/736: read d6/d7/d16/dc2/fe0 [98161,93270] 0 2024-05-17T21:53:12.734 INFO:tasks.workunit.client.1.smithi138.stdout:2/858: dread d3/d21/d2f/d3e/d58/f71 [0,4194304] 0 2024-05-17T21:53:12.738 INFO:tasks.workunit.client.1.smithi138.stdout:4/823: fsync d0/d3/d17/d21/d78/d10f/f11c 0 2024-05-17T21:53:12.741 INFO:tasks.workunit.client.1.smithi138.stdout:6/828: creat d7/d69/db5/dc8/d43/d5f/da4/db0/f118 x:0 0 0 2024-05-17T21:53:12.747 INFO:tasks.workunit.client.1.smithi138.stdout:9/737: rename d6/d7/d16/d29/d48/d3b/fdb to d6/d7/d16/d29/d38/d40/d44/d5c/dc8/de2/fe6 0 2024-05-17T21:53:12.748 INFO:tasks.workunit.client.1.smithi138.stdout:6/829: dread d7/d69/db5/dc8/d43/d3f/d79/f93 [0,4194304] 0 2024-05-17T21:53:12.748 INFO:tasks.workunit.client.1.smithi138.stdout:0/746: write d3/f7 [1913354,64007] 0 2024-05-17T21:53:12.750 INFO:tasks.workunit.client.1.smithi138.stdout:0/747: chown d3/d10/d6d/d9f/lcf 11598003 1 2024-05-17T21:53:12.750 INFO:tasks.workunit.client.1.smithi138.stdout:5/812: creat d1/f105 x:0 0 0 2024-05-17T21:53:12.752 INFO:tasks.workunit.client.1.smithi138.stdout:4/824: mknod d0/d1b/d42/c11f 0 2024-05-17T21:53:12.752 INFO:tasks.workunit.client.0.smithi107.stdout:2/892: sync 2024-05-17T21:53:12.754 INFO:tasks.workunit.client.1.smithi138.stdout:7/787: dread d0/d18/d40/d41/d50/f61 [0,4194304] 0 2024-05-17T21:53:12.755 INFO:tasks.workunit.client.1.smithi138.stdout:3/984: dread d1/d3/d16/d3e/f113 [0,4194304] 0 2024-05-17T21:53:12.755 INFO:tasks.workunit.client.0.smithi107.stdout:2/893: chown d0/d9/d11/d18/c1a 1664930940 1 2024-05-17T21:53:12.756 INFO:tasks.workunit.client.1.smithi138.stdout:7/788: readlink d0/d18/d64/d7c/da2/dc5/le9 0 2024-05-17T21:53:12.756 INFO:tasks.workunit.client.0.smithi107.stdout:2/894: read d0/d9/d11/d18/dad/db8/fdc [198078,22548] 0 2024-05-17T21:53:12.756 INFO:tasks.workunit.client.1.smithi138.stdout:3/985: chown d1/d62/dad/de/l32 17869 1 2024-05-17T21:53:12.757 INFO:tasks.workunit.client.0.smithi107.stdout:2/895: chown d0/d9/d1c/d37/d9f/fa7 26 1 2024-05-17T21:53:12.757 INFO:tasks.workunit.client.1.smithi138.stdout:0/748: rmdir d3/d3d/d48 39 2024-05-17T21:53:12.758 INFO:tasks.workunit.client.1.smithi138.stdout:3/986: write d1/d62/dad/d5c/f140 [23069,80453] 0 2024-05-17T21:53:12.758 INFO:tasks.workunit.client.1.smithi138.stdout:0/749: chown d3/d10/d6d/c76 3852 1 2024-05-17T21:53:12.761 INFO:tasks.workunit.client.1.smithi138.stdout:4/825: mkdir d0/d3/d1d/d38/d8a/dbd/ddc/d101/d120 0 2024-05-17T21:53:12.765 INFO:tasks.workunit.client.1.smithi138.stdout:6/830: mknod d7/c119 0 2024-05-17T21:53:12.766 INFO:tasks.workunit.client.1.smithi138.stdout:7/789: mkdir d0/d18/d64/d7c/da2/dff 0 2024-05-17T21:53:12.766 INFO:tasks.workunit.client.1.smithi138.stdout:9/738: dread d6/d7/d16/d29/d48/d6e/f56 [0,4194304] 0 2024-05-17T21:53:12.769 INFO:tasks.workunit.client.1.smithi138.stdout:5/813: write d1/d2c/d63/db5/d19/fab [158515,89595] 0 2024-05-17T21:53:12.769 INFO:tasks.workunit.client.1.smithi138.stdout:4/826: chown d0/d2f/l59 24841790 1 2024-05-17T21:53:12.769 INFO:tasks.workunit.client.1.smithi138.stdout:2/859: link d3/d21/d2f/d3e/d63/c90 d3/d21/d2f/d3e/d55/d99/dd2/c11c 0 2024-05-17T21:53:12.769 INFO:tasks.workunit.client.1.smithi138.stdout:2/860: chown d3/da0/daa 1885 1 2024-05-17T21:53:12.772 INFO:tasks.workunit.client.1.smithi138.stdout:6/831: unlink d7/d69/db5/dc8/d43/d3f/d79/f93 0 2024-05-17T21:53:12.772 INFO:tasks.workunit.client.0.smithi107.stdout:2/896: dread d0/d9/d11/df/f47 [12582912,4194304] 0 2024-05-17T21:53:12.773 INFO:tasks.workunit.client.1.smithi138.stdout:7/790: mkdir d0/d18/d40/d41/d50/d100 0 2024-05-17T21:53:12.773 INFO:tasks.workunit.client.1.smithi138.stdout:7/791: dread - d0/d18/fef zero size 2024-05-17T21:53:12.775 INFO:tasks.workunit.client.1.smithi138.stdout:4/827: dread - d0/d1b/d2b/d55/d79/f96 zero size 2024-05-17T21:53:12.775 INFO:tasks.workunit.client.1.smithi138.stdout:4/828: chown d0/d1b/d2b/d55 9994129 1 2024-05-17T21:53:12.778 INFO:tasks.workunit.client.1.smithi138.stdout:5/814: mkdir d1/d53/d88/dc9/d106 0 2024-05-17T21:53:12.779 INFO:tasks.workunit.client.1.smithi138.stdout:7/792: mkdir d0/d39/d101 0 2024-05-17T21:53:12.781 INFO:tasks.workunit.client.1.smithi138.stdout:3/987: getdents d1 0 2024-05-17T21:53:12.783 INFO:tasks.workunit.client.1.smithi138.stdout:9/739: dread d6/d7/d16/d29/d6c/d9c/dc1/f92 [0,4194304] 0 2024-05-17T21:53:12.783 INFO:tasks.workunit.client.1.smithi138.stdout:4/829: dwrite d0/d1b/d2b/f65 [0,4194304] 0 2024-05-17T21:53:12.788 INFO:tasks.workunit.client.1.smithi138.stdout:2/861: link d3/d21/d2f/d3e/d55/d99/dd2/cd5 d3/d21/d2f/d3e/d55/d73/c11d 0 2024-05-17T21:53:12.790 INFO:tasks.workunit.client.1.smithi138.stdout:6/832: write d7/d8/fb4 [1117380,101867] 0 2024-05-17T21:53:12.793 INFO:tasks.workunit.client.1.smithi138.stdout:9/740: mknod d6/d7/d16/d29/d48/d6e/d74/d75/d76/ce7 0 2024-05-17T21:53:12.794 INFO:tasks.workunit.client.1.smithi138.stdout:2/862: symlink d3/da0/df6/l11e 0 2024-05-17T21:53:12.796 INFO:tasks.workunit.client.1.smithi138.stdout:3/988: write d1/d62/dad/f34 [1813136,93491] 0 2024-05-17T21:53:12.797 INFO:tasks.workunit.client.1.smithi138.stdout:2/863: chown d3/d21/d2f/d3e/d55/d99/d84/d9f/dfd 461676799 1 2024-05-17T21:53:12.800 INFO:tasks.workunit.client.1.smithi138.stdout:2/864: rename d3/d21/d2f/d3e/d55/d99/d84 to d3/d21/d2f/d3e/d55/d99/d84/ded/d11f 22 2024-05-17T21:53:12.800 INFO:tasks.workunit.client.1.smithi138.stdout:3/989: chown d1/d62/c127 390982125 1 2024-05-17T21:53:12.801 INFO:tasks.workunit.client.1.smithi138.stdout:9/741: mkdir d6/d7/d16/d29/d48/d6e/d74/d75/d76/de8 0 2024-05-17T21:53:12.802 INFO:tasks.workunit.client.1.smithi138.stdout:5/815: getdents d1/d2c 0 2024-05-17T21:53:12.803 INFO:tasks.workunit.client.1.smithi138.stdout:7/793: getdents d0/d18/d64/d7c/da2 0 2024-05-17T21:53:12.803 INFO:tasks.workunit.client.1.smithi138.stdout:2/865: unlink d3/l70 0 2024-05-17T21:53:12.804 INFO:tasks.workunit.client.1.smithi138.stdout:4/830: getdents d0/d3/d17 0 2024-05-17T21:53:12.807 INFO:tasks.workunit.client.1.smithi138.stdout:9/742: write d6/d7/f15 [3286153,120963] 0 2024-05-17T21:53:12.808 INFO:tasks.workunit.client.1.smithi138.stdout:2/866: chown d3/c8 14118 1 2024-05-17T21:53:12.809 INFO:tasks.workunit.client.1.smithi138.stdout:7/794: symlink d0/d4c/dec/l102 0 2024-05-17T21:53:12.811 INFO:tasks.workunit.client.1.smithi138.stdout:6/833: dread d7/d69/db5/dc8/d34/f53 [0,4194304] 0 2024-05-17T21:53:12.817 INFO:tasks.workunit.client.1.smithi138.stdout:5/816: mknod d1/d2c/d63/db5/d1b/dc5/c107 0 2024-05-17T21:53:12.822 INFO:tasks.workunit.client.1.smithi138.stdout:3/990: dwrite d1/d62/db0/d13f/d14c/fa1 [0,4194304] 0 2024-05-17T21:53:12.825 INFO:tasks.workunit.client.1.smithi138.stdout:3/991: write d1/d62/dad/f148 [360643,76528] 0 2024-05-17T21:53:12.828 INFO:tasks.workunit.client.1.smithi138.stdout:2/867: dwrite d3/d21/d2f/d3e/d55/d99/d84/f91 [0,4194304] 0 2024-05-17T21:53:12.847 INFO:tasks.workunit.client.1.smithi138.stdout:9/743: unlink d6/d7/d16/l28 0 2024-05-17T21:53:12.853 INFO:tasks.workunit.client.1.smithi138.stdout:3/992: read - d1/dee/f112 zero size 2024-05-17T21:53:12.854 INFO:tasks.workunit.client.1.smithi138.stdout:4/831: getdents d0/d2f 0 2024-05-17T21:53:12.866 INFO:tasks.workunit.client.1.smithi138.stdout:9/744: rename d6/d7/d16/d29/d48/d6f/f71 to d6/d7/d16/fe9 0 2024-05-17T21:53:12.866 INFO:tasks.workunit.client.1.smithi138.stdout:9/745: chown d6/d7/f15 15 1 2024-05-17T21:53:12.869 INFO:tasks.workunit.client.1.smithi138.stdout:4/832: mknod d0/d1b/c121 0 2024-05-17T21:53:12.870 INFO:tasks.workunit.client.1.smithi138.stdout:9/746: unlink f5 0 2024-05-17T21:53:12.871 INFO:tasks.workunit.client.1.smithi138.stdout:9/747: stat d6/d7/d16/f21 0 2024-05-17T21:53:12.873 INFO:tasks.workunit.client.1.smithi138.stdout:9/748: write d6/d7/d16/d29/d6c/d9c/dc1/d7e/db7/fc4 [300544,7608] 0 2024-05-17T21:53:12.875 INFO:tasks.workunit.client.1.smithi138.stdout:4/833: rmdir d0/d1b/d36 39 2024-05-17T21:53:12.881 INFO:tasks.workunit.client.1.smithi138.stdout:4/834: dread d0/d3/d17/d21/d78/d10f/f11c [0,4194304] 0 2024-05-17T21:53:12.885 INFO:tasks.workunit.client.1.smithi138.stdout:4/835: fdatasync d0/d2f/d49/d60/d1f/d32/fb2 0 2024-05-17T21:53:12.886 INFO:tasks.workunit.client.1.smithi138.stdout:4/836: chown d0/d2f/d49/fc2 422215 1 2024-05-17T21:53:12.886 INFO:tasks.workunit.client.1.smithi138.stdout:4/837: readlink d0/d3/l8 0 2024-05-17T21:53:12.887 INFO:tasks.workunit.client.1.smithi138.stdout:4/838: write d0/d2f/d49/d60/d1f/d7e/f83 [2650677,121210] 0 2024-05-17T21:53:12.896 INFO:tasks.workunit.client.1.smithi138.stdout:4/839: rename d0/d3/d17/d21/d78/d10f to d0/d122 0 2024-05-17T21:53:12.900 INFO:tasks.workunit.client.1.smithi138.stdout:7/795: dwrite d0/d18/d40/d41/d51/d81/ffb [0,4194304] 0 2024-05-17T21:53:12.900 INFO:tasks.workunit.client.1.smithi138.stdout:5/817: write d1/d53/d96/fc6 [4275010,46983] 0 2024-05-17T21:53:12.903 INFO:tasks.workunit.client.1.smithi138.stdout:5/818: stat d1/d2c/d63/db5/d1b/c102 0 2024-05-17T21:53:12.904 INFO:tasks.workunit.client.1.smithi138.stdout:6/834: dwrite d7/d69/db5/dc8/d43/d5f/fd5 [4194304,4194304] 0 2024-05-17T21:53:12.909 INFO:tasks.workunit.client.0.smithi107.stdout:2/897: sync 2024-05-17T21:53:12.909 INFO:tasks.workunit.client.0.smithi107.stdout:2/898: stat d0/d9/c118 0 2024-05-17T21:53:12.911 INFO:tasks.workunit.client.1.smithi138.stdout:7/796: rename d0/d18/f92 to d0/d39/f103 0 2024-05-17T21:53:12.915 INFO:tasks.workunit.client.1.smithi138.stdout:3/993: write d1/f65 [4256672,67883] 0 2024-05-17T21:53:12.916 INFO:tasks.workunit.client.1.smithi138.stdout:7/797: rmdir d0/d18/dbf/dd1 39 2024-05-17T21:53:12.918 INFO:tasks.workunit.client.1.smithi138.stdout:7/798: chown d0/d18/d64/d7c/da2/de0 1890497 1 2024-05-17T21:53:12.919 INFO:tasks.workunit.client.1.smithi138.stdout:7/799: dread - d0/d18/d64/d7c/da2/de0/ff3 zero size 2024-05-17T21:53:12.919 INFO:tasks.workunit.client.1.smithi138.stdout:4/840: link d0/d3/d17/d21/d78/fcd d0/d2f/d49/f123 0 2024-05-17T21:53:12.925 INFO:tasks.workunit.client.1.smithi138.stdout:4/841: mkdir d0/d1b/dd4/d124 0 2024-05-17T21:53:12.927 INFO:tasks.workunit.client.1.smithi138.stdout:3/994: mkdir d1/d4f/d50/d14f 0 2024-05-17T21:53:12.929 INFO:tasks.workunit.client.1.smithi138.stdout:9/749: write d6/d7/d16/d29/d38/d3c/f4e [805377,64636] 0 2024-05-17T21:53:12.930 INFO:tasks.workunit.client.1.smithi138.stdout:4/842: creat d0/d1b/d42/f125 x:0 0 0 2024-05-17T21:53:12.931 INFO:tasks.workunit.client.1.smithi138.stdout:6/835: dread d7/d69/db5/dc8/d34/f7c [0,4194304] 0 2024-05-17T21:53:12.931 INFO:tasks.workunit.client.1.smithi138.stdout:9/750: chown d6/d7/d16/d29/d48/f7d 2846091 1 2024-05-17T21:53:12.934 INFO:tasks.workunit.client.0.smithi107.stdout:2/899: link d0/d9/d1c/d22/f65 d0/d9/d1c/d37/df8/f12a 0 2024-05-17T21:53:12.937 INFO:tasks.workunit.client.1.smithi138.stdout:6/836: symlink d7/l11a 0 2024-05-17T21:53:12.938 INFO:tasks.workunit.client.1.smithi138.stdout:9/751: dwrite d6/d7/fc [0,4194304] 0 2024-05-17T21:53:12.938 INFO:tasks.workunit.client.1.smithi138.stdout:4/843: getdents d0/d3/d17/d21/d78/d9f/d104 0 2024-05-17T21:53:12.939 INFO:tasks.workunit.client.0.smithi107.stdout:2/900: dwrite d0/d9/d11/d15/df2/f114 [0,4194304] 0 2024-05-17T21:53:12.940 INFO:tasks.workunit.client.1.smithi138.stdout:4/844: write d0/d2f/d49/d60/d1f/f4d [2432758,68993] 0 2024-05-17T21:53:12.941 INFO:tasks.workunit.client.1.smithi138.stdout:6/837: fdatasync d7/d69/db5/dc8/d43/d3f/db6/fd8 0 2024-05-17T21:53:12.942 INFO:tasks.workunit.client.1.smithi138.stdout:4/845: chown d0/d2f/d49/d60/d1f/d7e/lf9 678 1 2024-05-17T21:53:12.946 INFO:tasks.workunit.client.1.smithi138.stdout:9/752: mknod d6/d7/d16/d29/da6/cea 0 2024-05-17T21:53:12.947 INFO:tasks.workunit.client.1.smithi138.stdout:9/753: write d6/d7/d16/d29/d6c/d9c/fd0 [664419,129523] 0 2024-05-17T21:53:12.953 INFO:tasks.workunit.client.0.smithi107.stdout:2/901: creat d0/d9/d11/f12b x:0 0 0 2024-05-17T21:53:13.014 INFO:tasks.workunit.client.1.smithi138.stdout:4/846: mkdir d0/d3/db/da0/d126 0 2024-05-17T21:53:13.017 INFO:tasks.workunit.client.1.smithi138.stdout:4/847: stat d0/d2f/d49/d60/f51 0 2024-05-17T21:53:13.018 INFO:tasks.workunit.client.1.smithi138.stdout:6/838: symlink d7/d69/db5/dc8/d60/dfc/l11b 0 2024-05-17T21:53:13.021 INFO:tasks.workunit.client.1.smithi138.stdout:9/754: rename d6/d7/d16/d29/d48/d6e/d74/l9a to d6/d7/d16/d29/d6c/d9c/leb 0 2024-05-17T21:53:13.021 INFO:tasks.workunit.client.1.smithi138.stdout:4/848: read - d0/d3/d1d/d38/f10d zero size 2024-05-17T21:53:13.021 INFO:tasks.workunit.client.1.smithi138.stdout:6/839: chown d7/d69/fa7 464 1 2024-05-17T21:53:13.024 INFO:tasks.workunit.client.1.smithi138.stdout:7/800: dwrite d0/d39/f3d [0,4194304] 0 2024-05-17T21:53:13.025 INFO:tasks.workunit.client.1.smithi138.stdout:3/995: dwrite d1/d62/d9b/d107/fc6 [4194304,4194304] 0 2024-05-17T21:53:13.032 INFO:tasks.workunit.client.1.smithi138.stdout:6/840: rename d7/d69/db5/dc8/d43/da1 to d7/d8/de9/d11c 0 2024-05-17T21:53:13.034 INFO:tasks.workunit.client.1.smithi138.stdout:3/996: dread - d1/d62/dad/d5c/d9c/fa3 zero size 2024-05-17T21:53:13.035 INFO:tasks.workunit.client.1.smithi138.stdout:7/801: dwrite d0/d9d/fb5 [4194304,4194304] 0 2024-05-17T21:53:13.036 INFO:tasks.workunit.client.1.smithi138.stdout:9/755: creat d6/d7/d16/d29/d48/d6e/d74/d75/d76/d8e/ddd/fec x:0 0 0 2024-05-17T21:53:13.038 INFO:tasks.workunit.client.1.smithi138.stdout:3/997: chown d1/d3/d16/d3e/d71/d10d 2692 1 2024-05-17T21:53:13.039 INFO:tasks.workunit.client.1.smithi138.stdout:9/756: stat d6/d7/d16/d29/d38 0 2024-05-17T21:53:13.040 INFO:tasks.workunit.client.1.smithi138.stdout:3/998: dread - d1/d62/d9b/d107/df2/f12d zero size 2024-05-17T21:53:13.043 INFO:tasks.workunit.client.1.smithi138.stdout:6/841: link d7/d69/db5/dc8/d43/d5f/da4/db0/d113/f100 d7/f11d 0 2024-05-17T21:53:13.043 INFO:tasks.workunit.client.0.smithi107.stdout:2/902: dwrite d0/d9/d11/f52 [0,4194304] 0 2024-05-17T21:53:13.044 INFO:tasks.workunit.client.1.smithi138.stdout:9/757: mkdir d6/d7/d16/d29/d38/d40/d44/d5c/dc8/ded 0 2024-05-17T21:53:13.047 INFO:tasks.workunit.client.1.smithi138.stdout:3/999: truncate d1/d62/d9b/ffc 292554 0 2024-05-17T21:53:13.048 INFO:tasks.workunit.client.0.smithi107.stdout:2/903: read d0/d26/f6c [3633165,128344] 0 2024-05-17T21:53:13.049 INFO:tasks.workunit.client.0.smithi107.stdout:2/904: fdatasync d0/d9/f46 0 2024-05-17T21:53:13.050 INFO:tasks.workunit.client.1.smithi138.stdout:6/842: symlink d7/d69/db5/dc8/d43/d5f/d6a/d9b/db2/l11e 0 2024-05-17T21:53:13.052 INFO:tasks.workunit.client.1.smithi138.stdout:9/758: creat d6/d7/d16/d29/d38/d40/d44/fee x:0 0 0 2024-05-17T21:53:13.053 INFO:tasks.workunit.client.1.smithi138.stdout:7/802: link d0/d18/d40/d41/d51/d66/f6f d0/d18/d64/d7c/f104 0 2024-05-17T21:53:13.055 INFO:tasks.workunit.client.1.smithi138.stdout:6/843: chown d7/l16 19 1 2024-05-17T21:53:13.057 INFO:tasks.workunit.client.0.smithi107.stdout:2/905: mknod d0/d9/d11/d15/c12c 0 2024-05-17T21:53:13.057 INFO:tasks.workunit.client.1.smithi138.stdout:4/849: dread d0/d3/d17/d21/d78/fa5 [0,4194304] 0 2024-05-17T21:53:13.065 INFO:tasks.workunit.client.1.smithi138.stdout:4/850: fdatasync d0/d3/d1d/d38/d8a/dbd/fc0 0 2024-05-17T21:53:13.067 INFO:tasks.workunit.client.1.smithi138.stdout:6/844: creat d7/d69/db5/f11f x:0 0 0 2024-05-17T21:53:13.070 INFO:tasks.workunit.client.0.smithi107.stdout:2/906: truncate d0/d9/d11/d15/f57 363994 0 2024-05-17T21:53:13.071 INFO:tasks.workunit.client.1.smithi138.stdout:6/845: mknod d7/d69/db5/dc8/d43/d5f/da4/c120 0 2024-05-17T21:53:13.072 INFO:tasks.workunit.client.1.smithi138.stdout:7/803: getdents d0/d18/dbf/dd1 0 2024-05-17T21:53:13.072 INFO:tasks.workunit.client.0.smithi107.stdout:2/907: read d0/d9/d1c/f73 [3996367,73723] 0 2024-05-17T21:53:13.072 INFO:tasks.workunit.client.1.smithi138.stdout:4/851: symlink d0/d3/db/da0/d126/l127 0 2024-05-17T21:53:13.073 INFO:tasks.workunit.client.0.smithi107.stdout:2/908: chown d0/d9/d11/df/d5a/c91 6667 1 2024-05-17T21:53:13.073 INFO:tasks.workunit.client.1.smithi138.stdout:7/804: read - d0/d9d/dd7/fdb zero size 2024-05-17T21:53:13.074 INFO:tasks.workunit.client.1.smithi138.stdout:4/852: chown d0/d2f/d49/d60/d1f/d7e 12110 1 2024-05-17T21:53:13.079 INFO:tasks.workunit.client.0.smithi107.stdout:2/909: dwrite d0/fa [4194304,4194304] 0 2024-05-17T21:53:13.080 INFO:tasks.workunit.client.1.smithi138.stdout:7/805: link d0/d18/d40/d41/d51/d66/c79 d0/d18/dbf/dd1/c105 0 2024-05-17T21:53:13.082 INFO:tasks.workunit.client.0.smithi107.stdout:2/910: rmdir d0/d9/d11/df/d5a/d6d/dc8/de8 39 2024-05-17T21:53:13.083 INFO:tasks.workunit.client.0.smithi107.stdout:2/911: chown d0/d9/d1c/fb5 53 1 2024-05-17T21:53:13.084 INFO:tasks.workunit.client.1.smithi138.stdout:7/806: creat d0/d18/d64/d7c/da2/dc5/dcf/f106 x:0 0 0 2024-05-17T21:53:13.086 INFO:tasks.workunit.client.0.smithi107.stdout:2/912: symlink d0/d9/d1c/d37/l12d 0 2024-05-17T21:53:13.088 INFO:tasks.workunit.client.1.smithi138.stdout:6/846: dwrite d7/d69/db5/dc8/d43/d3f/f46 [0,4194304] 0 2024-05-17T21:53:13.090 INFO:tasks.workunit.client.1.smithi138.stdout:7/807: rmdir d0/d18/d40/d41/d51/d66/d9e/df9 39 2024-05-17T21:53:13.092 INFO:tasks.workunit.client.0.smithi107.stdout:2/913: unlink d0/d9/d11/d18/fce 0 2024-05-17T21:53:13.095 INFO:tasks.workunit.client.1.smithi138.stdout:7/808: fdatasync d0/f4d 0 2024-05-17T21:53:13.096 INFO:tasks.workunit.client.1.smithi138.stdout:6/847: rmdir d7/d69/db5/dc8/d43/d5f/d6a/d77/de5 39 2024-05-17T21:53:13.097 INFO:tasks.workunit.client.1.smithi138.stdout:7/809: dread d0/d18/d40/f58 [0,4194304] 0 2024-05-17T21:53:13.100 INFO:tasks.workunit.client.1.smithi138.stdout:7/810: fdatasync d0/f1e 0 2024-05-17T21:53:13.102 INFO:tasks.workunit.client.1.smithi138.stdout:4/853: dread d0/d3/d17/d21/fa7 [0,4194304] 0 2024-05-17T21:53:13.103 INFO:tasks.workunit.client.0.smithi107.stdout:2/914: link d0/d9/d11/df/d5a/d6d/dc8/de8/fa9 d0/d9/d11/df/d5a/d6d/dc8/de8/ddb/f12e 0 2024-05-17T21:53:13.104 INFO:tasks.workunit.client.1.smithi138.stdout:7/811: mknod d0/d18/d40/d41/d51/d66/c107 0 2024-05-17T21:53:13.106 INFO:tasks.workunit.client.0.smithi107.stdout:2/915: rename d0/d9/d11/d15/f81 to d0/d26/d8e/dc4/f12f 0 2024-05-17T21:53:13.107 INFO:tasks.workunit.client.1.smithi138.stdout:4/854: dwrite d0/d2f/d49/d60/f3c [0,4194304] 0 2024-05-17T21:53:13.108 INFO:tasks.workunit.client.1.smithi138.stdout:7/812: unlink d0/d39/c42 0 2024-05-17T21:53:13.115 INFO:tasks.workunit.client.1.smithi138.stdout:4/855: creat d0/d3/f128 x:0 0 0 2024-05-17T21:53:13.118 INFO:tasks.workunit.client.1.smithi138.stdout:4/856: creat d0/d2f/d49/d60/d1f/d32/d71/f129 x:0 0 0 2024-05-17T21:53:13.120 INFO:tasks.workunit.client.1.smithi138.stdout:7/813: rename d0/d18/d40/d5f/f73 to d0/d18/d40/d41/d51/d66/d9e/df9/dfe/f108 0 2024-05-17T21:53:13.122 INFO:tasks.workunit.client.1.smithi138.stdout:6/848: dwrite d7/d69/db5/dc8/d43/f98 [0,4194304] 0 2024-05-17T21:53:13.124 INFO:tasks.workunit.client.1.smithi138.stdout:4/857: rmdir d0/d1b/d2b/d55/d79 39 2024-05-17T21:53:13.125 INFO:tasks.workunit.client.1.smithi138.stdout:4/858: write d0/d2f/d49/d60/d1f/d7e/f83 [6347576,58288] 0 2024-05-17T21:53:13.127 INFO:tasks.workunit.client.1.smithi138.stdout:7/814: fsync d0/d18/d40/d41/d51/d66/d9e/fd5 0 2024-05-17T21:53:13.130 INFO:tasks.workunit.client.1.smithi138.stdout:7/815: fdatasync d0/d18/d40/f4e 0 2024-05-17T21:53:13.130 INFO:tasks.workunit.client.1.smithi138.stdout:4/859: rmdir d0 39 2024-05-17T21:53:13.131 INFO:tasks.workunit.client.0.smithi107.stdout:2/916: write d0/d93/fcf [666255,1797] 0 2024-05-17T21:53:13.132 INFO:tasks.workunit.client.1.smithi138.stdout:6/849: rmdir d7/d69/db5/dc8/d43/d5f/d6a/d77/de5 39 2024-05-17T21:53:13.137 INFO:tasks.workunit.client.1.smithi138.stdout:7/816: mkdir d0/d109 0 2024-05-17T21:53:13.137 INFO:tasks.workunit.client.0.smithi107.stdout:2/917: dwrite d0/d9/d1c/d3e/f3f [4194304,4194304] 0 2024-05-17T21:53:13.138 INFO:tasks.workunit.client.1.smithi138.stdout:6/850: creat d7/d8/d10c/f121 x:0 0 0 2024-05-17T21:53:13.138 INFO:tasks.workunit.client.1.smithi138.stdout:7/817: dread - d0/d18/d40/d5f/dbe/fed zero size 2024-05-17T21:53:13.139 INFO:tasks.workunit.client.1.smithi138.stdout:6/851: fsync d7/d69/db5/dc8/f11 0 2024-05-17T21:53:13.142 INFO:tasks.workunit.client.0.smithi107.stdout:2/918: rename d0/d26/ddd/f128 to d0/d9/d11/df/d5a/dc7/f130 0 2024-05-17T21:53:13.145 INFO:tasks.workunit.client.0.smithi107.stdout:2/919: creat d0/d9/d1c/d37/df8/d10a/f131 x:0 0 0 2024-05-17T21:53:13.147 INFO:tasks.workunit.client.0.smithi107.stdout:2/920: mkdir d0/d9/d1c/d37/d9f/d132 0 2024-05-17T21:53:13.148 INFO:tasks.workunit.client.0.smithi107.stdout:2/921: write d0/d26/f107 [939845,112239] 0 2024-05-17T21:53:13.149 INFO:tasks.workunit.client.0.smithi107.stdout:2/922: read d0/d9/d11/f5c [1596911,13821] 0 2024-05-17T21:53:13.150 INFO:tasks.workunit.client.0.smithi107.stdout:2/923: fsync d0/d9/d11/df/d5a/d6d/d82/f106 0 2024-05-17T21:53:13.153 INFO:tasks.workunit.client.1.smithi138.stdout:4/860: dwrite d0/d2f/d49/d60/d1f/d7e/fce [0,4194304] 0 2024-05-17T21:53:13.155 INFO:tasks.workunit.client.0.smithi107.stdout:2/924: rename d0/d9/d11/df/d5a/c91 to d0/d9/d11/df/d5a/d6d/dc8/de8/c133 0 2024-05-17T21:53:13.159 INFO:tasks.workunit.client.1.smithi138.stdout:4/861: mknod d0/d1b/d2b/dca/c12a 0 2024-05-17T21:53:13.161 INFO:tasks.workunit.client.1.smithi138.stdout:4/862: stat d0/d1b/d2b/d55/d79/f7b 0 2024-05-17T21:53:13.162 INFO:tasks.workunit.client.0.smithi107.stdout:2/925: dread d0/f6f [0,4194304] 0 2024-05-17T21:53:13.163 INFO:tasks.workunit.client.1.smithi138.stdout:6/852: dread d7/d69/fa7 [0,4194304] 0 2024-05-17T21:53:13.164 INFO:tasks.workunit.client.1.smithi138.stdout:4/863: symlink d0/d3/d17/l12b 0 2024-05-17T21:53:13.165 INFO:tasks.workunit.client.1.smithi138.stdout:7/818: dread d0/d18/d64/f6e [0,4194304] 0 2024-05-17T21:53:13.165 INFO:tasks.workunit.client.1.smithi138.stdout:4/864: chown d0/d3/d17/d21/d78/d9f/d104 722733 1 2024-05-17T21:53:13.166 INFO:tasks.workunit.client.0.smithi107.stdout:2/926: symlink d0/d9/l134 0 2024-05-17T21:53:13.166 INFO:tasks.workunit.client.1.smithi138.stdout:7/819: dread - d0/d18/d64/d7c/da2/dc5/dcf/f106 zero size 2024-05-17T21:53:13.167 INFO:tasks.workunit.client.1.smithi138.stdout:6/853: unlink d7/d69/l76 0 2024-05-17T21:53:13.170 INFO:tasks.workunit.client.0.smithi107.stdout:2/927: rename d0/d26/d8e/dc4/f10b to d0/d9/d1c/d22/f135 0 2024-05-17T21:53:13.174 INFO:tasks.workunit.client.1.smithi138.stdout:6/854: read d7/d69/db5/dc8/d43/d3f/d79/ffe [365203,60706] 0 2024-05-17T21:53:13.175 INFO:tasks.workunit.client.1.smithi138.stdout:6/855: dread - d7/d69/db5/dc8/d43/d63/f115 zero size 2024-05-17T21:53:13.175 INFO:tasks.workunit.client.1.smithi138.stdout:8/712: sync 2024-05-17T21:53:13.177 INFO:tasks.workunit.client.1.smithi138.stdout:8/713: write d0/dc/fc6 [378809,42634] 0 2024-05-17T21:53:13.179 INFO:tasks.workunit.client.1.smithi138.stdout:8/714: write d0/dc/d1a/f1d [4045634,24943] 0 2024-05-17T21:53:13.182 INFO:tasks.workunit.client.1.smithi138.stdout:8/715: chown d0/dc/d1a/d51 150 1 2024-05-17T21:53:13.183 INFO:tasks.workunit.client.1.smithi138.stdout:4/865: dwrite d0/d122/f11c [0,4194304] 0 2024-05-17T21:53:13.183 INFO:tasks.workunit.client.1.smithi138.stdout:6/856: dread d7/f68 [0,4194304] 0 2024-05-17T21:53:13.188 INFO:tasks.workunit.client.1.smithi138.stdout:7/820: dwrite d0/d18/d40/d41/d51/d66/d9e/df9/dfe/f108 [0,4194304] 0 2024-05-17T21:53:13.188 INFO:tasks.workunit.client.1.smithi138.stdout:4/866: symlink d0/d1b/d2b/dca/d111/l12c 0 2024-05-17T21:53:13.190 INFO:tasks.workunit.client.1.smithi138.stdout:6/857: stat d7/d69/db5/dc8/d43/cf1 0 2024-05-17T21:53:13.193 INFO:tasks.workunit.client.1.smithi138.stdout:4/867: fdatasync d0/fe2 0 2024-05-17T21:53:13.194 INFO:tasks.workunit.client.1.smithi138.stdout:4/868: chown d0/d1b/d42/d45 10264999 1 2024-05-17T21:53:13.195 INFO:tasks.workunit.client.1.smithi138.stdout:4/869: dread - d0/d1b/dd4/f113 zero size 2024-05-17T21:53:13.197 INFO:tasks.workunit.client.1.smithi138.stdout:7/821: rename d0/d18/ff5 to d0/d18/d40/d41/d51/d66/d9e/df9/dfe/f10a 0 2024-05-17T21:53:13.198 INFO:tasks.workunit.client.1.smithi138.stdout:8/716: getdents d0/dc 0 2024-05-17T21:53:13.203 INFO:tasks.workunit.client.1.smithi138.stdout:8/717: creat d0/dc/d3b/d4d/d7d/d97/dc1/fd2 x:0 0 0 2024-05-17T21:53:13.206 INFO:tasks.workunit.client.1.smithi138.stdout:6/858: write d7/d8/fba [2388107,116205] 0 2024-05-17T21:53:13.209 INFO:tasks.workunit.client.1.smithi138.stdout:4/870: write d0/d2f/d49/d60/d1f/d32/fe1 [560571,42220] 0 2024-05-17T21:53:13.212 INFO:tasks.workunit.client.1.smithi138.stdout:8/718: dwrite d0/dc/d1a/d51/d82/f8e [0,4194304] 0 2024-05-17T21:53:13.213 INFO:tasks.workunit.client.1.smithi138.stdout:4/871: creat d0/d3/d17/d21/d78/d9f/d11e/f12d x:0 0 0 2024-05-17T21:53:13.217 INFO:tasks.workunit.client.1.smithi138.stdout:6/859: dwrite d7/d69/db5/dc8/f82 [0,4194304] 0 2024-05-17T21:53:13.223 INFO:tasks.workunit.client.1.smithi138.stdout:7/822: dread d0/d18/d40/d41/d51/fce [0,4194304] 0 2024-05-17T21:53:13.223 INFO:tasks.workunit.client.1.smithi138.stdout:8/719: dwrite d0/d2f/f6e [4194304,4194304] 0 2024-05-17T21:53:13.227 INFO:tasks.workunit.client.1.smithi138.stdout:7/823: creat d0/d18/d64/d7c/da2/dc5/f10b x:0 0 0 2024-05-17T21:53:13.229 INFO:tasks.workunit.client.1.smithi138.stdout:6/860: getdents d7/d69/db5/dc8/d43/d5f/da4/db0/d113 0 2024-05-17T21:53:13.229 INFO:tasks.workunit.client.1.smithi138.stdout:4/872: write d0/d1b/fc8 [33415,17596] 0 2024-05-17T21:53:13.230 INFO:tasks.workunit.client.1.smithi138.stdout:7/824: symlink d0/d18/d64/d7c/da2/dc5/dcf/l10c 0 2024-05-17T21:53:13.233 INFO:tasks.workunit.client.1.smithi138.stdout:4/873: truncate d0/d1b/fc8 353823 0 2024-05-17T21:53:13.233 INFO:tasks.workunit.client.1.smithi138.stdout:6/861: mkdir d7/d8/d10c/d122 0 2024-05-17T21:53:13.234 INFO:tasks.workunit.client.1.smithi138.stdout:7/825: creat d0/d4c/dec/f10d x:0 0 0 2024-05-17T21:53:13.242 INFO:tasks.workunit.client.1.smithi138.stdout:7/826: dwrite d0/d18/d40/d9a/fa5 [0,4194304] 0 2024-05-17T21:53:13.248 INFO:tasks.workunit.client.1.smithi138.stdout:7/827: link d0/d18/d64/d7c/da2/dc5/dcf/f106 d0/d18/d40/d5f/f10e 0 2024-05-17T21:53:13.251 INFO:tasks.workunit.client.1.smithi138.stdout:6/862: write d7/d8/d78/fbd [4459439,67344] 0 2024-05-17T21:53:13.251 INFO:tasks.workunit.client.1.smithi138.stdout:4/874: write d0/d2f/d49/d60/d1f/d32/f64 [1913388,95353] 0 2024-05-17T21:53:13.257 INFO:tasks.workunit.client.1.smithi138.stdout:4/875: dwrite d0/d3/d17/d5e/f8e [0,4194304] 0 2024-05-17T21:53:13.261 INFO:tasks.workunit.client.1.smithi138.stdout:4/876: mkdir d0/d2f/d49/d60/d1f/d32/d71/d95/d12e 0 2024-05-17T21:53:13.261 INFO:tasks.workunit.client.1.smithi138.stdout:7/828: fsync d0/d18/d64/d7c/da2/dc5/dcf/f106 0 2024-05-17T21:53:13.263 INFO:tasks.workunit.client.1.smithi138.stdout:6/863: truncate d7/d69/db5/dc8/d34/da6/f26 2556673 0 2024-05-17T21:53:13.265 INFO:tasks.workunit.client.1.smithi138.stdout:7/829: creat d0/d39/f10f x:0 0 0 2024-05-17T21:53:13.267 INFO:tasks.workunit.client.1.smithi138.stdout:6/864: rmdir d7/d69/db5/dc8/d43/d3f/d79/ddf 39 2024-05-17T21:53:13.269 INFO:tasks.workunit.client.1.smithi138.stdout:4/877: getdents d0/d3 0 2024-05-17T21:53:13.270 INFO:tasks.workunit.client.1.smithi138.stdout:4/878: chown d0/d3/d1d/c77 0 1 2024-05-17T21:53:13.272 INFO:tasks.workunit.client.1.smithi138.stdout:4/879: write d0/d2f/d49/fc2 [145894,82193] 0 2024-05-17T21:53:13.275 INFO:tasks.workunit.client.1.smithi138.stdout:4/880: creat d0/d2f/d49/d60/d1f/da1/f12f x:0 0 0 2024-05-17T21:53:13.279 INFO:tasks.workunit.client.1.smithi138.stdout:4/881: creat d0/d1b/d2b/d55/f130 x:0 0 0 2024-05-17T21:53:13.280 INFO:tasks.workunit.client.1.smithi138.stdout:6/865: dwrite d7/da0/fbf [0,4194304] 0 2024-05-17T21:53:13.286 INFO:tasks.workunit.client.1.smithi138.stdout:4/882: write d0/d2f/d49/d60/d1f/d32/fb2 [376629,93587] 0 2024-05-17T21:53:13.288 INFO:tasks.workunit.client.1.smithi138.stdout:7/830: dread d0/d4c/f6d [0,4194304] 0 2024-05-17T21:53:13.289 INFO:tasks.workunit.client.1.smithi138.stdout:6/866: dwrite d7/d69/db5/dc8/f25 [0,4194304] 0 2024-05-17T21:53:13.290 INFO:tasks.workunit.client.1.smithi138.stdout:4/883: creat d0/d1b/dd4/f131 x:0 0 0 2024-05-17T21:53:13.295 INFO:tasks.workunit.client.0.smithi107.stdout:2/928: sync 2024-05-17T21:53:13.296 INFO:tasks.workunit.client.0.smithi107.stdout:2/929: readlink d0/d26/ddd/lbc 0 2024-05-17T21:53:13.297 INFO:tasks.workunit.client.0.smithi107.stdout:2/930: write d0/fa [8119809,18167] 0 2024-05-17T21:53:13.298 INFO:tasks.workunit.client.1.smithi138.stdout:4/884: dwrite d0/d3/d17/d5e/f8e [0,4194304] 0 2024-05-17T21:53:13.306 INFO:tasks.workunit.client.0.smithi107.stdout:2/931: dwrite d0/d9/f46 [4194304,4194304] 0 2024-05-17T21:53:13.316 INFO:tasks.workunit.client.1.smithi138.stdout:6/867: dread d7/d8/f96 [0,4194304] 0 2024-05-17T21:53:13.325 INFO:tasks.workunit.client.1.smithi138.stdout:7/831: dread d0/d39/fc6 [0,4194304] 0 2024-05-17T21:53:13.329 INFO:tasks.workunit.client.1.smithi138.stdout:4/885: dread d0/d3/db/fd3 [0,4194304] 0 2024-05-17T21:53:13.332 INFO:tasks.workunit.client.1.smithi138.stdout:4/886: link d0/d3/d17/d21/d78/cdb d0/d3/d17/d21/d78/c132 0 2024-05-17T21:53:13.334 INFO:tasks.workunit.client.1.smithi138.stdout:4/887: unlink d0/d2f/cea 0 2024-05-17T21:53:13.337 INFO:tasks.workunit.client.1.smithi138.stdout:6/868: dread d7/d69/db5/dc8/d43/d3f/f56 [0,4194304] 0 2024-05-17T21:53:13.363 INFO:tasks.workunit.client.1.smithi138.stdout:4/888: dread - d0/d2f/d49/d60/d1f/d32/d71/d95/f116 zero size 2024-05-17T21:53:13.366 INFO:tasks.workunit.client.1.smithi138.stdout:4/889: truncate d0/d3/d17/d21/fa7 3669251 0 2024-05-17T21:53:13.367 INFO:tasks.workunit.client.1.smithi138.stdout:6/869: dwrite d7/d69/db5/dc8/d43/d5f/da4/dcc/ff0 [0,4194304] 0 2024-05-17T21:53:13.369 INFO:tasks.workunit.client.1.smithi138.stdout:4/890: mkdir d0/d1b/d133 0 2024-05-17T21:53:13.370 INFO:tasks.workunit.client.1.smithi138.stdout:4/891: truncate d0/d1b/f100 124896 0 2024-05-17T21:53:13.376 INFO:tasks.workunit.client.1.smithi138.stdout:6/870: write d7/d69/db5/dc8/d43/d3f/d79/f9f [3176211,61580] 0 2024-05-17T21:53:13.377 INFO:tasks.workunit.client.1.smithi138.stdout:4/892: dwrite d0/d2f/d49/d60/d1f/d7e/f103 [0,4194304] 0 2024-05-17T21:53:13.382 INFO:tasks.workunit.client.1.smithi138.stdout:4/893: symlink d0/d3/d1d/d38/de3/d117/l134 0 2024-05-17T21:53:13.383 INFO:tasks.workunit.client.1.smithi138.stdout:4/894: chown d0/d3/d1d/d5b 191 1 2024-05-17T21:53:13.384 INFO:tasks.workunit.client.1.smithi138.stdout:6/871: dwrite d7/d69/db5/dc8/f52 [0,4194304] 0 2024-05-17T21:53:13.386 INFO:tasks.workunit.client.1.smithi138.stdout:4/895: fdatasync d0/d3/d17/d21/d78/fcd 0 2024-05-17T21:53:13.389 INFO:tasks.workunit.client.1.smithi138.stdout:6/872: mknod d7/d69/db5/dc8/d43/d5f/d6a/d9b/db2/c123 0 2024-05-17T21:53:13.389 INFO:tasks.workunit.client.1.smithi138.stdout:4/896: symlink d0/d3/d17/d21/d78/d9f/d11e/l135 0 2024-05-17T21:53:13.390 INFO:tasks.workunit.client.1.smithi138.stdout:6/873: chown d7/d69/db5/dc8/d43/f64 198729508 1 2024-05-17T21:53:13.390 INFO:tasks.workunit.client.1.smithi138.stdout:4/897: readlink d0/d3/d1d/d38/d9c/lc3 0 2024-05-17T21:53:13.398 INFO:tasks.workunit.client.1.smithi138.stdout:4/898: dwrite d0/d2f/d49/d60/d1f/d32/f64 [0,4194304] 0 2024-05-17T21:53:13.401 INFO:tasks.workunit.client.1.smithi138.stdout:4/899: mknod d0/d3/d17/d5e/d7f/c136 0 2024-05-17T21:53:13.404 INFO:tasks.workunit.client.1.smithi138.stdout:6/874: dwrite d7/f11d [0,4194304] 0 2024-05-17T21:53:13.404 INFO:tasks.workunit.client.1.smithi138.stdout:4/900: write d0/d2f/d49/d60/d1f/d32/d71/f129 [630361,31476] 0 2024-05-17T21:53:13.407 INFO:tasks.workunit.client.1.smithi138.stdout:4/901: mknod d0/d3/d1d/d38/d9c/c137 0 2024-05-17T21:53:13.411 INFO:tasks.workunit.client.1.smithi138.stdout:6/875: rmdir d7/d69/db5/dc8/d43/d5f/d6a/d77/de5/d10e 0 2024-05-17T21:53:13.412 INFO:tasks.workunit.client.1.smithi138.stdout:4/902: getdents d0 0 2024-05-17T21:53:13.413 INFO:tasks.workunit.client.1.smithi138.stdout:4/903: read - d0/d1b/d42/f125 zero size 2024-05-17T21:53:13.420 INFO:tasks.workunit.client.1.smithi138.stdout:4/904: dread d0/d2f/d49/f123 [0,4194304] 0 2024-05-17T21:53:13.425 INFO:tasks.workunit.client.1.smithi138.stdout:4/905: dwrite d0/d2f/d49/d60/d1f/d7e/f103 [0,4194304] 0 2024-05-17T21:53:13.426 INFO:tasks.workunit.client.1.smithi138.stdout:4/906: write d0/f34 [914655,18310] 0 2024-05-17T21:53:13.431 INFO:tasks.workunit.client.1.smithi138.stdout:4/907: write d0/f7 [4856305,41499] 0 2024-05-17T21:53:13.433 INFO:tasks.workunit.client.1.smithi138.stdout:4/908: truncate d0/d1b/d42/f7c 959023 0 2024-05-17T21:53:13.435 INFO:tasks.workunit.client.1.smithi138.stdout:4/909: unlink d0/d3/d1d/d38/d9c/cda 0 2024-05-17T21:53:13.436 INFO:tasks.workunit.client.1.smithi138.stdout:4/910: truncate d0/d3/d17/d5e/fb4 4756740 0 2024-05-17T21:53:13.437 INFO:tasks.workunit.client.1.smithi138.stdout:4/911: rmdir d0/d3/d1d 39 2024-05-17T21:53:13.440 INFO:tasks.workunit.client.1.smithi138.stdout:4/912: unlink d0/d2f/l52 0 2024-05-17T21:53:13.441 INFO:tasks.workunit.client.1.smithi138.stdout:4/913: creat d0/d3/d1d/d5b/f138 x:0 0 0 2024-05-17T21:53:13.444 INFO:tasks.workunit.client.0.smithi107.stdout:2/932: sync 2024-05-17T21:53:13.444 INFO:tasks.workunit.client.1.smithi138.stdout:4/914: getdents d0/d1b/d2b/d55/d79 0 2024-05-17T21:53:13.445 INFO:tasks.workunit.client.1.smithi138.stdout:4/915: write d0/d1b/d2b/f10e [266372,106817] 0 2024-05-17T21:53:13.446 INFO:tasks.workunit.client.1.smithi138.stdout:4/916: write d0/d3/d17/f114 [318075,60356] 0 2024-05-17T21:53:13.455 INFO:tasks.workunit.client.1.smithi138.stdout:4/917: dwrite d0/fe2 [0,4194304] 0 2024-05-17T21:53:13.457 INFO:tasks.workunit.client.1.smithi138.stdout:4/918: fdatasync d0/d1b/d88/fc6 0 2024-05-17T21:53:13.459 INFO:tasks.workunit.client.1.smithi138.stdout:4/919: creat d0/d2f/d49/d60/d1f/da1/f139 x:0 0 0 2024-05-17T21:53:13.463 INFO:tasks.workunit.client.1.smithi138.stdout:4/920: rename d0/d3/d17/fa9 to d0/d2f/d49/d60/d1f/d32/d71/d95/f13a 0 2024-05-17T21:53:13.464 INFO:tasks.workunit.client.1.smithi138.stdout:4/921: truncate d0/d1b/d42/f125 1042031 0 2024-05-17T21:53:13.466 INFO:tasks.workunit.client.1.smithi138.stdout:4/922: symlink d0/d2f/d49/d60/d1f/d32/d71/l13b 0 2024-05-17T21:53:13.467 INFO:tasks.workunit.client.1.smithi138.stdout:4/923: dread - d0/d3/f128 zero size 2024-05-17T21:53:13.468 INFO:tasks.workunit.client.1.smithi138.stdout:4/924: chown d0/d1b/c6e 438621368 1 2024-05-17T21:53:13.469 INFO:tasks.workunit.client.1.smithi138.stdout:4/925: chown d0/d2f/d49/d60/d1f/d32/fe1 62935424 1 2024-05-17T21:53:13.471 INFO:tasks.workunit.client.1.smithi138.stdout:4/926: mknod d0/d3/db/da0/d126/c13c 0 2024-05-17T21:53:13.476 INFO:tasks.workunit.client.1.smithi138.stdout:4/927: truncate d0/d1b/dd4/fed 373017 0 2024-05-17T21:53:13.480 INFO:tasks.workunit.client.1.smithi138.stdout:4/928: fdatasync d0/d1b/d42/f7c 0 2024-05-17T21:53:13.481 INFO:tasks.workunit.client.1.smithi138.stdout:4/929: read - d0/d1b/dd4/f131 zero size 2024-05-17T21:53:13.496 INFO:tasks.workunit.client.1.smithi138.stdout:4/930: dread d0/d1b/d42/f4f [0,4194304] 0 2024-05-17T21:53:13.498 INFO:tasks.workunit.client.1.smithi138.stdout:4/931: mknod d0/d3/d17/d21/d78/d9f/c13d 0 2024-05-17T21:53:13.499 INFO:tasks.workunit.client.1.smithi138.stdout:4/932: symlink d0/d1b/d42/l13e 0 2024-05-17T21:53:13.500 INFO:tasks.workunit.client.0.smithi107.stdout:2/933: dread d0/d9/d1c/d37/df8/f12a [0,4194304] 0 2024-05-17T21:53:13.501 INFO:tasks.workunit.client.0.smithi107.stdout:2/934: chown d0/d26/f6c 909541738 1 2024-05-17T21:53:13.504 INFO:tasks.workunit.client.1.smithi138.stdout:4/933: write d0/d2f/d49/d60/d1f/d7e/f93 [108679,993] 0 2024-05-17T21:53:13.508 INFO:tasks.workunit.client.0.smithi107.stdout:2/935: dwrite d0/d9/d1c/d22/db0/f11c [0,4194304] 0 2024-05-17T21:53:13.511 INFO:tasks.workunit.client.0.smithi107.stdout:2/936: write d0/f7d [9160874,13874] 0 2024-05-17T21:53:13.512 INFO:tasks.workunit.client.0.smithi107.stdout:2/937: chown d0/d9/d11/d15/lfe 75304 1 2024-05-17T21:53:13.521 INFO:tasks.workunit.client.0.smithi107.stdout:2/938: dwrite d0/d9/d11/f8b [4194304,4194304] 0 2024-05-17T21:53:13.526 INFO:tasks.workunit.client.1.smithi138.stdout:1/833: sync 2024-05-17T21:53:13.529 INFO:tasks.workunit.client.1.smithi138.stdout:1/834: unlink d4/df/d3a/fe3 0 2024-05-17T21:53:13.531 INFO:tasks.workunit.client.1.smithi138.stdout:1/835: mkdir d4/df/d3a/d10e 0 2024-05-17T21:53:13.532 INFO:tasks.workunit.client.1.smithi138.stdout:1/836: write d4/dd/f2f [946864,25598] 0 2024-05-17T21:53:13.534 INFO:tasks.workunit.client.0.smithi107.stdout:2/939: write d0/d9/d11/d27/dd3/fc1 [626373,90955] 0 2024-05-17T21:53:13.535 INFO:tasks.workunit.client.1.smithi138.stdout:1/837: truncate d4/df/d3a/fc5 3535909 0 2024-05-17T21:53:13.536 INFO:tasks.workunit.client.0.smithi107.stdout:2/940: write d0/d9/d11/d18/d9e/deb/ff0 [7169493,1473] 0 2024-05-17T21:53:13.541 INFO:tasks.workunit.client.0.smithi107.stdout:2/941: dread d0/f7d [8388608,4194304] 0 2024-05-17T21:53:13.542 INFO:tasks.workunit.client.1.smithi138.stdout:1/838: write d4/df/d3a/ff0 [1395760,96292] 0 2024-05-17T21:53:13.543 INFO:tasks.workunit.client.0.smithi107.stdout:2/942: write d0/d9/f77 [413498,74673] 0 2024-05-17T21:53:13.545 INFO:tasks.workunit.client.1.smithi138.stdout:1/839: creat d4/df/d3a/d3c/d63/d6d/f10f x:0 0 0 2024-05-17T21:53:13.546 INFO:tasks.workunit.client.0.smithi107.stdout:2/943: mkdir d0/d9/d11/d18/d9e/deb/d109/d136 0 2024-05-17T21:53:13.549 INFO:tasks.workunit.client.0.smithi107.stdout:2/944: creat d0/d9/d11/df/d5a/d6d/d82/f137 x:0 0 0 2024-05-17T21:53:13.550 INFO:tasks.workunit.client.0.smithi107.stdout:2/945: chown d0/d9/d1c/f96 1 1 2024-05-17T21:53:13.550 INFO:tasks.workunit.client.1.smithi138.stdout:1/840: dread d4/dd/d3b/d9f/da9/f10a [0,4194304] 0 2024-05-17T21:53:13.552 INFO:tasks.workunit.client.1.smithi138.stdout:1/841: unlink d4/df/c79 0 2024-05-17T21:53:13.560 INFO:tasks.workunit.client.1.smithi138.stdout:1/842: dread d4/db/d29/d5d/f6e [0,4194304] 0 2024-05-17T21:53:13.561 INFO:tasks.workunit.client.1.smithi138.stdout:1/843: dread - d4/db/f7b zero size 2024-05-17T21:53:13.561 INFO:tasks.workunit.client.0.smithi107.stdout:2/946: dwrite d0/d9/d11/f5c [0,4194304] 0 2024-05-17T21:53:13.562 INFO:tasks.workunit.client.0.smithi107.stdout:2/947: chown d0/d9/d11/df/fb4 88087514 1 2024-05-17T21:53:13.563 INFO:tasks.workunit.client.0.smithi107.stdout:2/948: chown d0/d26/d8e/dc4/f12f 281712 1 2024-05-17T21:53:13.564 INFO:tasks.workunit.client.1.smithi138.stdout:1/844: creat d4/db/f110 x:0 0 0 2024-05-17T21:53:13.573 INFO:tasks.workunit.client.1.smithi138.stdout:1/845: dwrite d4/df/d3a/fbe [0,4194304] 0 2024-05-17T21:53:13.574 INFO:tasks.workunit.client.0.smithi107.stdout:2/949: dwrite d0/d9/d1c/d37/df8/f12a [0,4194304] 0 2024-05-17T21:53:13.575 INFO:tasks.workunit.client.0.smithi107.stdout:2/950: write d0/d9/d1c/d37/d9f/fa7 [832166,52466] 0 2024-05-17T21:53:13.577 INFO:tasks.workunit.client.1.smithi138.stdout:1/846: readlink d4/df/d3a/d3c/lc2 0 2024-05-17T21:53:13.580 INFO:tasks.workunit.client.0.smithi107.stdout:2/951: mkdir d0/d9/d1c/d37/d138 0 2024-05-17T21:53:13.580 INFO:tasks.workunit.client.1.smithi138.stdout:1/847: mkdir d4/dd/d3b/da7/d111 0 2024-05-17T21:53:13.581 INFO:tasks.workunit.client.0.smithi107.stdout:2/952: write d0/d9/f60 [222177,63744] 0 2024-05-17T21:53:13.582 INFO:tasks.workunit.client.0.smithi107.stdout:2/953: stat d0/d9/d1c/d22/db0/f111 0 2024-05-17T21:53:13.585 INFO:tasks.workunit.client.1.smithi138.stdout:1/848: unlink d4/d2e/da0/lab 0 2024-05-17T21:53:13.587 INFO:tasks.workunit.client.0.smithi107.stdout:2/954: rmdir d0/d9/d11/d18/dad/db8 39 2024-05-17T21:53:13.588 INFO:tasks.workunit.client.1.smithi138.stdout:1/849: mknod d4/df/d3a/d3c/d63/d6d/de0/c112 0 2024-05-17T21:53:13.589 INFO:tasks.workunit.client.1.smithi138.stdout:1/850: symlink d4/dd/d3b/da7/d111/l113 0 2024-05-17T21:53:13.590 INFO:tasks.workunit.client.1.smithi138.stdout:1/851: write d4/dd/d3b/d6c/dc1/fce [160399,9820] 0 2024-05-17T21:53:13.597 INFO:tasks.workunit.client.0.smithi107.stdout:2/955: dread d0/d9/d11/df/d5a/d6d/dc8/de8/fa9 [0,4194304] 0 2024-05-17T21:53:13.599 INFO:tasks.workunit.client.0.smithi107.stdout:2/956: rmdir d0/d9/d11/d15 39 2024-05-17T21:53:13.601 INFO:tasks.workunit.client.0.smithi107.stdout:2/957: mknod d0/d93/df5/c139 0 2024-05-17T21:53:13.601 INFO:tasks.workunit.client.0.smithi107.stdout:2/958: chown d0/d9/l134 1187769196 1 2024-05-17T21:53:13.601 INFO:tasks.workunit.client.0.smithi107.stdout:2/959: chown d0/d9/d11/d18/d103 15135307 1 2024-05-17T21:53:13.605 INFO:tasks.workunit.client.0.smithi107.stdout:2/960: dread d0/d9/d11/d18/d9e/deb/ff0 [4194304,4194304] 0 2024-05-17T21:53:13.606 INFO:tasks.workunit.client.1.smithi138.stdout:1/852: dwrite d4/dd/f90 [0,4194304] 0 2024-05-17T21:53:13.607 INFO:tasks.workunit.client.1.smithi138.stdout:1/853: stat d4/df/f1f 0 2024-05-17T21:53:13.609 INFO:tasks.workunit.client.0.smithi107.stdout:2/961: mkdir d0/d9/d11/d18/d9e/deb/d109/d13a 0 2024-05-17T21:53:13.614 INFO:tasks.workunit.client.1.smithi138.stdout:1/854: dwrite d4/db/f110 [0,4194304] 0 2024-05-17T21:53:13.616 INFO:tasks.workunit.client.0.smithi107.stdout:2/962: dwrite d0/d9/d1c/d3e/f3f [0,4194304] 0 2024-05-17T21:53:13.617 INFO:tasks.workunit.client.1.smithi138.stdout:1/855: symlink d4/df/d35/df3/l114 0 2024-05-17T21:53:13.618 INFO:tasks.workunit.client.0.smithi107.stdout:2/963: dread d0/d9/d1c/d22/db0/f111 [0,4194304] 0 2024-05-17T21:53:13.620 INFO:tasks.workunit.client.1.smithi138.stdout:1/856: creat d4/d2e/da0/daa/f115 x:0 0 0 2024-05-17T21:53:13.640 INFO:tasks.workunit.client.1.smithi138.stdout:1/857: dwrite d4/dd/d3b/d9f/da9/f10a [0,4194304] 0 2024-05-17T21:53:13.643 INFO:tasks.workunit.client.1.smithi138.stdout:1/858: symlink d4/db/d29/d5d/l116 0 2024-05-17T21:53:13.645 INFO:tasks.workunit.client.1.smithi138.stdout:1/859: creat d4/d2e/f117 x:0 0 0 2024-05-17T21:53:13.648 INFO:tasks.workunit.client.1.smithi138.stdout:1/860: rename d4/d2e/d36/fee to d4/db/d60/f118 0 2024-05-17T21:53:13.651 INFO:tasks.workunit.client.1.smithi138.stdout:1/861: creat d4/dd/f119 x:0 0 0 2024-05-17T21:53:13.652 INFO:tasks.workunit.client.1.smithi138.stdout:1/862: mknod d4/dd/d64/c11a 0 2024-05-17T21:53:13.655 INFO:tasks.workunit.client.1.smithi138.stdout:1/863: creat d4/dd/d3b/d9f/db9/dae/f11b x:0 0 0 2024-05-17T21:53:13.663 INFO:tasks.workunit.client.1.smithi138.stdout:1/864: dwrite d4/db/f6a [0,4194304] 0 2024-05-17T21:53:13.667 INFO:tasks.workunit.client.1.smithi138.stdout:1/865: getdents d4/db/d29/d89 0 2024-05-17T21:53:13.671 INFO:tasks.workunit.client.1.smithi138.stdout:1/866: dread d4/f3f [0,4194304] 0 2024-05-17T21:53:13.675 INFO:tasks.workunit.client.1.smithi138.stdout:1/867: dread d4/d2e/d36/f91 [0,4194304] 0 2024-05-17T21:53:13.678 INFO:tasks.workunit.client.1.smithi138.stdout:1/868: creat d4/db/d29/d5d/de5/f11c x:0 0 0 2024-05-17T21:53:13.680 INFO:tasks.workunit.client.1.smithi138.stdout:1/869: mkdir d4/df/d3a/dc4/d11d 0 2024-05-17T21:53:13.681 INFO:tasks.workunit.client.1.smithi138.stdout:1/870: write d4/df/d5f/f100 [325922,128588] 0 2024-05-17T21:53:13.683 INFO:tasks.workunit.client.1.smithi138.stdout:1/871: creat d4/dd/d3b/d9f/db9/f11e x:0 0 0 2024-05-17T21:53:13.684 INFO:tasks.workunit.client.1.smithi138.stdout:1/872: mknod d4/db/d29/d5d/de5/c11f 0 2024-05-17T21:53:13.685 INFO:tasks.workunit.client.1.smithi138.stdout:1/873: chown d4/leb 26 1 2024-05-17T21:53:13.763 INFO:tasks.workunit.client.0.smithi107.stdout:2/964: sync 2024-05-17T21:53:13.766 INFO:tasks.workunit.client.0.smithi107.stdout:2/965: fsync d0/d9/d11/d15/f57 0 2024-05-17T21:53:13.768 INFO:tasks.workunit.client.0.smithi107.stdout:2/966: write d0/d9/d11/d18/fd5 [487242,79349] 0 2024-05-17T21:53:13.769 INFO:tasks.workunit.client.0.smithi107.stdout:2/967: truncate d0/d9/d1c/f10e 522263 0 2024-05-17T21:53:13.770 INFO:tasks.workunit.client.0.smithi107.stdout:2/968: fdatasync d0/d9/d11/d18/d9e/deb/d109/f127 0 2024-05-17T21:53:13.773 INFO:tasks.workunit.client.0.smithi107.stdout:2/969: link d0/d9/d11/df/d5a/d6d/l78 d0/d9/d11/df/d5a/d6d/l13b 0 2024-05-17T21:53:13.777 INFO:tasks.workunit.client.0.smithi107.stdout:2/970: dread d0/d9/d1c/d22/db0/f11c [0,4194304] 0 2024-05-17T21:53:13.783 INFO:tasks.workunit.client.0.smithi107.stdout:2/971: dwrite d0/d9/d11/df/d5a/d6d/d82/f137 [0,4194304] 0 2024-05-17T21:53:13.787 INFO:tasks.workunit.client.0.smithi107.stdout:2/972: mknod d0/d9/d11/df/d5a/d6d/dc8/c13c 0 2024-05-17T21:53:13.789 INFO:tasks.workunit.client.0.smithi107.stdout:2/973: fsync d0/d9/d11/df/d5a/d6d/dc8/de8/ddb/fe3 0 2024-05-17T21:53:13.792 INFO:tasks.workunit.client.0.smithi107.stdout:2/974: mknod d0/d9/d11/d18/dad/db8/c13d 0 2024-05-17T21:53:13.793 INFO:tasks.workunit.client.0.smithi107.stdout:2/975: chown d0/d9/d11/df/d5a/d6d/dc8/de8/l76 9343 1 2024-05-17T21:53:13.795 INFO:tasks.workunit.client.0.smithi107.stdout:2/976: mkdir d0/d9/d11/df/d5a/d13e 0 2024-05-17T21:53:13.796 INFO:tasks.workunit.client.0.smithi107.stdout:2/977: read d0/d9/d11/df/d5a/d6d/d82/f106 [47954,21674] 0 2024-05-17T21:53:13.796 INFO:tasks.workunit.client.1.smithi138.stdout:2/868: sync 2024-05-17T21:53:13.796 INFO:tasks.workunit.client.1.smithi138.stdout:0/750: sync 2024-05-17T21:53:13.797 INFO:tasks.workunit.client.0.smithi107.stdout:2/978: dread - d0/d9/f117 zero size 2024-05-17T21:53:13.797 INFO:tasks.workunit.client.1.smithi138.stdout:2/869: fdatasync d3/d21/d2f/fc0 0 2024-05-17T21:53:13.799 INFO:tasks.workunit.client.1.smithi138.stdout:0/751: unlink d3/d3d/f58 0 2024-05-17T21:53:13.804 INFO:tasks.workunit.client.1.smithi138.stdout:2/870: dread d3/d21/d2f/d3e/d58/d77/da4/dc5/fbc [0,4194304] 0 2024-05-17T21:53:13.806 INFO:tasks.workunit.client.0.smithi107.stdout:2/979: write d0/d9/d1c/d22/db0/f111 [829308,23735] 0 2024-05-17T21:53:13.809 INFO:tasks.workunit.client.0.smithi107.stdout:2/980: truncate d0/d9/d11/d18/f79 64804 0 2024-05-17T21:53:13.811 INFO:tasks.workunit.client.0.smithi107.stdout:2/981: truncate d0/d9/d1c/d37/f50 872351 0 2024-05-17T21:53:13.812 INFO:tasks.workunit.client.1.smithi138.stdout:0/752: dwrite d3/d10/d6d/db6/fc1 [0,4194304] 0 2024-05-17T21:53:13.813 INFO:tasks.workunit.client.1.smithi138.stdout:0/753: stat d3/d3d/f53 0 2024-05-17T21:53:13.815 INFO:tasks.workunit.client.0.smithi107.stdout:2/982: truncate d0/d9/d11/df/d5a/d6d/dc8/de8/f8d 389195 0 2024-05-17T21:53:13.815 INFO:tasks.workunit.client.1.smithi138.stdout:2/871: dwrite d3/d21/d2f/d3e/d58/d77/f86 [0,4194304] 0 2024-05-17T21:53:13.817 INFO:tasks.workunit.client.1.smithi138.stdout:2/872: chown d3/d21/d2f/d3e/d58/d77/da4/df2/dd0 36 1 2024-05-17T21:53:13.855 INFO:tasks.workunit.client.1.smithi138.stdout:9/759: sync 2024-05-17T21:53:13.856 INFO:tasks.workunit.client.1.smithi138.stdout:5/819: sync 2024-05-17T21:53:13.856 INFO:tasks.workunit.client.1.smithi138.stdout:8/720: sync 2024-05-17T21:53:13.856 INFO:tasks.workunit.client.1.smithi138.stdout:7/832: sync 2024-05-17T21:53:13.856 INFO:tasks.workunit.client.1.smithi138.stdout:4/934: sync 2024-05-17T21:53:13.856 INFO:tasks.workunit.client.1.smithi138.stdout:6/876: sync 2024-05-17T21:53:13.861 INFO:tasks.workunit.client.1.smithi138.stdout:5/820: dwrite d1/d2c/d63/db5/d19/fab [0,4194304] 0 2024-05-17T21:53:13.888 INFO:tasks.workunit.client.1.smithi138.stdout:0/754: link d3/fe3 d3/d10/d6d/db6/db9/ffa 0 2024-05-17T21:53:13.888 INFO:tasks.workunit.client.1.smithi138.stdout:2/873: read d3/d81/d96/fb4 [972776,50629] 0 2024-05-17T21:53:13.888 INFO:tasks.workunit.client.1.smithi138.stdout:0/755: chown d3/d3d/ff7 13453454 1 2024-05-17T21:53:13.890 INFO:tasks.workunit.client.1.smithi138.stdout:8/721: mknod d0/dc/d3b/d87/cd3 0 2024-05-17T21:53:13.890 INFO:tasks.workunit.client.1.smithi138.stdout:8/722: chown d0/d11/d6a/f91 29775 1 2024-05-17T21:53:13.891 INFO:tasks.workunit.client.1.smithi138.stdout:4/935: symlink d0/d3/d1d/d38/l13f 0 2024-05-17T21:53:13.892 INFO:tasks.workunit.client.1.smithi138.stdout:9/760: creat d6/d7/d16/fef x:0 0 0 2024-05-17T21:53:13.897 INFO:tasks.workunit.client.1.smithi138.stdout:6/877: rename d7/d8/c33 to d7/d69/db5/dc8/d43/d5f/da4/c124 0 2024-05-17T21:53:13.899 INFO:tasks.workunit.client.1.smithi138.stdout:5/821: mknod d1/d2c/d63/db5/d1b/dc5/d36/d68/dad/c108 0 2024-05-17T21:53:13.900 INFO:tasks.workunit.client.1.smithi138.stdout:7/833: dwrite d0/d18/d64/d7c/da2/dc5/dcf/fd8 [0,4194304] 0 2024-05-17T21:53:13.902 INFO:tasks.workunit.client.1.smithi138.stdout:7/834: write d0/d18/d40/d41/d51/d66/d9e/fa1 [468893,10814] 0 2024-05-17T21:53:13.904 INFO:tasks.workunit.client.1.smithi138.stdout:2/874: dread - d3/d21/d2f/d3e/d63/da3/fd3 zero size 2024-05-17T21:53:13.907 INFO:tasks.workunit.client.1.smithi138.stdout:8/723: creat d0/dc/d8d/fd4 x:0 0 0 2024-05-17T21:53:13.911 INFO:tasks.workunit.client.1.smithi138.stdout:9/761: chown d6/d7/d16/d29/d48/f23 699605035 1 2024-05-17T21:53:13.912 INFO:tasks.workunit.client.1.smithi138.stdout:9/762: chown d6/de4 1 1 2024-05-17T21:53:13.914 INFO:tasks.workunit.client.1.smithi138.stdout:4/936: dwrite d0/d3/d17/f63 [0,4194304] 0 2024-05-17T21:53:13.916 INFO:tasks.workunit.client.1.smithi138.stdout:4/937: readlink d0/d1b/d2b/dca/d111/l12c 0 2024-05-17T21:53:13.917 INFO:tasks.workunit.client.1.smithi138.stdout:5/822: mkdir d1/d2c/d63/db5/d1b/d109 0 2024-05-17T21:53:13.920 INFO:tasks.workunit.client.1.smithi138.stdout:7/835: symlink d0/d18/d40/d41/d51/l110 0 2024-05-17T21:53:13.921 INFO:tasks.workunit.client.0.smithi107.stdout:2/983: sync 2024-05-17T21:53:13.921 INFO:tasks.workunit.client.1.smithi138.stdout:4/938: dread d0/d1b/d88/f8b [0,4194304] 0 2024-05-17T21:53:13.924 INFO:tasks.workunit.client.1.smithi138.stdout:8/724: mkdir d0/dc/d3b/d4d/d7d/dd5 0 2024-05-17T21:53:13.926 INFO:tasks.workunit.client.0.smithi107.stdout:2/984: creat d0/d93/f13f x:0 0 0 2024-05-17T21:53:13.927 INFO:tasks.workunit.client.1.smithi138.stdout:9/763: creat d6/d7/d16/d29/d38/d40/d44/d5c/ff0 x:0 0 0 2024-05-17T21:53:13.930 INFO:tasks.workunit.client.1.smithi138.stdout:5/823: creat d1/d46/d95/f10a x:0 0 0 2024-05-17T21:53:13.937 INFO:tasks.workunit.client.1.smithi138.stdout:1/874: sync 2024-05-17T21:53:13.943 INFO:tasks.workunit.client.1.smithi138.stdout:8/725: dread d0/dc/f22 [4194304,4194304] 0 2024-05-17T21:53:14.027 INFO:tasks.workunit.client.0.smithi107.stdout:2/985: getdents d0/d9/d11/df/d5a/dc7 0 2024-05-17T21:53:14.029 INFO:tasks.workunit.client.1.smithi138.stdout:0/756: link d3/d10/f2c d3/d3d/da2/ffb 0 2024-05-17T21:53:14.034 INFO:tasks.workunit.client.0.smithi107.stdout:2/986: write d0/d9/d11/df/d5a/d6d/d82/f137 [1226128,43837] 0 2024-05-17T21:53:14.034 INFO:tasks.workunit.client.0.smithi107.stdout:2/987: fdatasync d0/d9/d11/d18/dad/fb7 0 2024-05-17T21:53:14.034 INFO:tasks.workunit.client.0.smithi107.stdout:2/988: chown d0/d9/d1c/l3c 46020845 1 2024-05-17T21:53:14.035 INFO:tasks.workunit.client.1.smithi138.stdout:6/878: truncate d7/d69/db5/dc8/d43/d63/fce 5164685 0 2024-05-17T21:53:14.035 INFO:tasks.workunit.client.0.smithi107.stdout:2/989: fdatasync d0/d9/d11/d18/d9e/fe1 0 2024-05-17T21:53:14.042 INFO:tasks.workunit.client.1.smithi138.stdout:4/939: link d0/d2f/d49/f123 d0/d1b/d2b/d55/d79/f140 0 2024-05-17T21:53:14.051 INFO:tasks.workunit.client.1.smithi138.stdout:8/726: dread d0/d11/d6a/f91 [0,4194304] 0 2024-05-17T21:53:14.052 INFO:tasks.workunit.client.1.smithi138.stdout:8/727: readlink d0/dc/d3b/laa 0 2024-05-17T21:53:14.056 INFO:tasks.workunit.client.1.smithi138.stdout:1/875: link d4/d2e/da0/daa/cc3 d4/df/d3a/d10e/c120 0 2024-05-17T21:53:14.059 INFO:tasks.workunit.client.1.smithi138.stdout:6/879: creat d7/d69/db5/dc8/d43/d5f/da4/db0/dc1/f125 x:0 0 0 2024-05-17T21:53:14.060 INFO:tasks.workunit.client.1.smithi138.stdout:4/940: dread d0/d2f/d49/d60/f51 [0,4194304] 0 2024-05-17T21:53:14.063 INFO:tasks.workunit.client.1.smithi138.stdout:6/880: chown d7/d69/db5/dc8/d34/da6/c92 3319341 1 2024-05-17T21:53:14.065 INFO:tasks.workunit.client.1.smithi138.stdout:1/876: dwrite d4/dd/d3b/d9f/da9/f10a [0,4194304] 0 2024-05-17T21:53:14.068 INFO:tasks.workunit.client.1.smithi138.stdout:4/941: dread d0/d2f/d49/d60/d1f/d32/d71/f129 [0,4194304] 0 2024-05-17T21:53:14.072 INFO:tasks.workunit.client.1.smithi138.stdout:0/757: dread d3/d10/f2c [0,4194304] 0 2024-05-17T21:53:14.074 INFO:tasks.workunit.client.1.smithi138.stdout:1/877: link d4/db/d29/d5d/de5/f11c d4/d2e/da0/daa/f121 0 2024-05-17T21:53:14.078 INFO:tasks.workunit.client.1.smithi138.stdout:4/942: creat d0/d1b/f141 x:0 0 0 2024-05-17T21:53:14.080 INFO:tasks.workunit.client.1.smithi138.stdout:1/878: truncate d4/dd/d3b/d6c/dcf/fd5 736655 0 2024-05-17T21:53:14.080 INFO:tasks.workunit.client.1.smithi138.stdout:4/943: write d0/d2f/d49/d60/d1f/f73 [3072535,119539] 0 2024-05-17T21:53:14.089 INFO:tasks.workunit.client.1.smithi138.stdout:1/879: getdents d4/dd/d64 0 2024-05-17T21:53:14.090 INFO:tasks.workunit.client.1.smithi138.stdout:1/880: write d4/df/d35/df3/f10c [88613,23794] 0 2024-05-17T21:53:14.091 INFO:tasks.workunit.client.1.smithi138.stdout:1/881: fsync d4/db/d29/f5b 0 2024-05-17T21:53:14.127 INFO:tasks.workunit.client.1.smithi138.stdout:2/875: dwrite d3/d21/d2f/d3e/d55/d99/d112/fe6 [0,4194304] 0 2024-05-17T21:53:14.129 INFO:tasks.workunit.client.1.smithi138.stdout:7/836: write d0/d18/d40/d41/f43 [3794840,99969] 0 2024-05-17T21:53:14.131 INFO:tasks.workunit.client.1.smithi138.stdout:7/837: truncate d0/d18/d40/d9a/fbb 417548 0 2024-05-17T21:53:14.133 INFO:tasks.workunit.client.1.smithi138.stdout:7/838: chown d0/d18/d64/d7c/fad 56058415 1 2024-05-17T21:53:14.137 INFO:tasks.workunit.client.1.smithi138.stdout:7/839: mknod d0/d18/d64/d7c/da2/dc5/dcf/c111 0 2024-05-17T21:53:14.141 INFO:tasks.workunit.client.1.smithi138.stdout:7/840: unlink d0/d18/dbf/fe5 0 2024-05-17T21:53:14.142 INFO:tasks.workunit.client.1.smithi138.stdout:7/841: stat d0/c35 0 2024-05-17T21:53:14.142 INFO:tasks.workunit.client.1.smithi138.stdout:7/842: readlink d0/d18/d64/l67 0 2024-05-17T21:53:14.145 INFO:tasks.workunit.client.1.smithi138.stdout:7/843: mkdir d0/d18/d40/d41/d51/d66/d9e/df9/dfe/d112 0 2024-05-17T21:53:14.148 INFO:tasks.workunit.client.1.smithi138.stdout:7/844: mknod d0/d18/d9c/c113 0 2024-05-17T21:53:14.149 INFO:tasks.workunit.client.1.smithi138.stdout:2/876: dread d3/d21/d2f/d3e/d55/d73/f97 [0,4194304] 0 2024-05-17T21:53:14.150 INFO:tasks.workunit.client.1.smithi138.stdout:7/845: creat d0/d109/f114 x:0 0 0 2024-05-17T21:53:14.152 INFO:tasks.workunit.client.1.smithi138.stdout:2/877: creat d3/d21/d2f/d3e/d55/d99/d112/f120 x:0 0 0 2024-05-17T21:53:14.153 INFO:tasks.workunit.client.1.smithi138.stdout:7/846: symlink d0/d18/d64/d7c/l115 0 2024-05-17T21:53:14.154 INFO:tasks.workunit.client.1.smithi138.stdout:7/847: chown d0/d18/d64/fcd 2588 1 2024-05-17T21:53:14.156 INFO:tasks.workunit.client.1.smithi138.stdout:7/848: rmdir d0/d18/d40/d41/d51/d66/da8 39 2024-05-17T21:53:14.162 INFO:tasks.workunit.client.1.smithi138.stdout:0/758: rmdir d3/d10/d38/d3e/d59/d9b/dc3/dd5 39 2024-05-17T21:53:14.163 INFO:tasks.workunit.client.1.smithi138.stdout:7/849: dread d0/d39/f85 [0,4194304] 0 2024-05-17T21:53:14.164 INFO:tasks.workunit.client.1.smithi138.stdout:7/850: stat d0/d9d/ld9 0 2024-05-17T21:53:14.165 INFO:tasks.workunit.client.1.smithi138.stdout:0/759: creat d3/d10/d6d/db6/db9/ffc x:0 0 0 2024-05-17T21:53:14.166 INFO:tasks.workunit.client.1.smithi138.stdout:0/760: write d3/d3d/f6a [1176555,35304] 0 2024-05-17T21:53:14.191 INFO:tasks.workunit.client.1.smithi138.stdout:0/761: rmdir d3/d10/d39/dd9 39 2024-05-17T21:53:14.192 INFO:tasks.workunit.client.1.smithi138.stdout:0/762: chown d3/f18 1610536280 1 2024-05-17T21:53:14.194 INFO:tasks.workunit.client.1.smithi138.stdout:0/763: unlink d3/d10/d39/c7b 0 2024-05-17T21:53:14.200 INFO:tasks.workunit.client.1.smithi138.stdout:9/764: write d6/d7/d16/d29/d38/f73 [3092604,82108] 0 2024-05-17T21:53:14.213 INFO:tasks.workunit.client.1.smithi138.stdout:8/728: write d0/dc/d1a/f4b [426355,70697] 0 2024-05-17T21:53:14.216 INFO:tasks.workunit.client.1.smithi138.stdout:8/729: rename d0/d63/l7b to d0/dc/d3b/d4d/d7d/d97/ld6 0 2024-05-17T21:53:14.217 INFO:tasks.workunit.client.1.smithi138.stdout:8/730: symlink d0/dc/d8d/ld7 0 2024-05-17T21:53:14.331 INFO:tasks.workunit.client.1.smithi138.stdout:4/944: write d0/d2f/d49/f123 [718791,10227] 0 2024-05-17T21:53:14.332 INFO:tasks.workunit.client.1.smithi138.stdout:8/731: symlink d0/dc/d1a/d51/d82/ld8 0 2024-05-17T21:53:14.333 INFO:tasks.workunit.client.1.smithi138.stdout:8/732: readlink d0/dc/d8d/la2 0 2024-05-17T21:53:14.335 INFO:tasks.workunit.client.1.smithi138.stdout:4/945: creat d0/d3/d1d/d5b/f142 x:0 0 0 2024-05-17T21:53:14.337 INFO:tasks.workunit.client.1.smithi138.stdout:8/733: mknod d0/d2f/cd9 0 2024-05-17T21:53:14.340 INFO:tasks.workunit.client.1.smithi138.stdout:4/946: creat d0/d2f/d49/d60/d1f/f143 x:0 0 0 2024-05-17T21:53:14.341 INFO:tasks.workunit.client.1.smithi138.stdout:8/734: truncate d0/dc/d1a/f3f 3173673 0 2024-05-17T21:53:14.345 INFO:tasks.workunit.client.1.smithi138.stdout:5/824: creat d1/d2c/d63/db5/f10b x:0 0 0 2024-05-17T21:53:14.347 INFO:tasks.workunit.client.1.smithi138.stdout:4/947: dread d0/d3/d17/d21/fa7 [0,4194304] 0 2024-05-17T21:53:14.353 INFO:tasks.workunit.client.1.smithi138.stdout:4/948: getdents d0/d3/d17/d21/ddd 0 2024-05-17T21:53:14.354 INFO:tasks.workunit.client.1.smithi138.stdout:4/949: readlink d0/d2f/d49/d60/l28 0 2024-05-17T21:53:14.358 INFO:tasks.workunit.client.1.smithi138.stdout:4/950: symlink d0/d1b/d133/l144 0 2024-05-17T21:53:14.361 INFO:tasks.workunit.client.1.smithi138.stdout:4/951: creat d0/d3/d17/d21/ddd/f145 x:0 0 0 2024-05-17T21:53:14.362 INFO:tasks.workunit.client.1.smithi138.stdout:4/952: readlink d0/d3/d17/d5e/d7f/lc4 0 2024-05-17T21:53:14.365 INFO:tasks.workunit.client.1.smithi138.stdout:4/953: creat d0/d1b/d42/d45/d84/f146 x:0 0 0 2024-05-17T21:53:14.367 INFO:tasks.workunit.client.1.smithi138.stdout:4/954: mknod d0/d3/d17/d21/d78/d9f/de4/c147 0 2024-05-17T21:53:14.375 INFO:tasks.workunit.client.1.smithi138.stdout:7/851: write d0/d39/f103 [1420729,78274] 0 2024-05-17T21:53:14.377 INFO:tasks.workunit.client.1.smithi138.stdout:1/882: mkdir d4/d2e/da0/d122 0 2024-05-17T21:53:14.378 INFO:tasks.workunit.client.1.smithi138.stdout:1/883: stat d4/dd/d3b/d9f/da9/f10a 0 2024-05-17T21:53:14.379 INFO:tasks.workunit.client.1.smithi138.stdout:7/852: mkdir d0/d18/dbf/d116 0 2024-05-17T21:53:14.380 INFO:tasks.workunit.client.1.smithi138.stdout:1/884: mkdir d4/dd/d3b/d6c/dcf/d123 0 2024-05-17T21:53:14.382 INFO:tasks.workunit.client.1.smithi138.stdout:7/853: rename d0/d18/d40/d41/d51/d66/d9e/df9/dfe/d112 to d0/d18/d40/d41/d51/d66/d9e/df9/dfe/d112/d117 22 2024-05-17T21:53:14.382 INFO:tasks.workunit.client.1.smithi138.stdout:7/854: stat d0/d18/d9c/dd4/ff1 0 2024-05-17T21:53:14.383 INFO:tasks.workunit.client.1.smithi138.stdout:6/881: mknod d7/d69/db5/dc8/d34/c126 0 2024-05-17T21:53:14.384 INFO:tasks.workunit.client.1.smithi138.stdout:7/855: rmdir d0/d18/d40/d41/d51/d66/d9e/df9/dfe 39 2024-05-17T21:53:14.386 INFO:tasks.workunit.client.1.smithi138.stdout:6/882: mknod d7/d69/db5/dc8/d43/d5f/da4/db0/dc1/c127 0 2024-05-17T21:53:14.389 INFO:tasks.workunit.client.1.smithi138.stdout:7/856: creat d0/d18/d40/d41/d51/d66/da8/f118 x:0 0 0 2024-05-17T21:53:14.391 INFO:tasks.workunit.client.1.smithi138.stdout:7/857: fdatasync d0/d39/fc6 0 2024-05-17T21:53:14.393 INFO:tasks.workunit.client.1.smithi138.stdout:7/858: dread - d0/d4c/fc0 zero size 2024-05-17T21:53:14.397 INFO:tasks.workunit.client.1.smithi138.stdout:9/765: creat d6/ff1 x:0 0 0 2024-05-17T21:53:14.401 INFO:tasks.workunit.client.1.smithi138.stdout:9/766: mkdir d6/d7/d16/d29/d48/d6e/d74/d75/d76/d8e/dba/df2 0 2024-05-17T21:53:14.401 INFO:tasks.workunit.client.1.smithi138.stdout:0/764: link d3/cf d3/d10/cfd 0 2024-05-17T21:53:14.402 INFO:tasks.workunit.client.1.smithi138.stdout:0/765: write d3/d3d/ff7 [759863,7780] 0 2024-05-17T21:53:14.405 INFO:tasks.workunit.client.1.smithi138.stdout:9/767: mkdir d6/d7/d16/d29/d38/d40/d44/d5c/dc8/ded/df3 0 2024-05-17T21:53:14.409 INFO:tasks.workunit.client.1.smithi138.stdout:9/768: rename d6/d7/d16/d29/da6 to d6/d7/d16/d29/d38/df4 0 2024-05-17T21:53:14.421 INFO:tasks.workunit.client.1.smithi138.stdout:9/769: dread d6/d7/d16/d29/d38/d40/d44/fa4 [0,4194304] 0 2024-05-17T21:53:14.422 INFO:tasks.workunit.client.1.smithi138.stdout:8/735: write d0/dc/d3b/f89 [8456161,363] 0 2024-05-17T21:53:14.423 INFO:tasks.workunit.client.1.smithi138.stdout:8/736: chown d0/c8a 7379 1 2024-05-17T21:53:14.424 INFO:tasks.workunit.client.1.smithi138.stdout:9/770: read - d6/d7/d16/d29/d48/d6e/d74/d75/d76/d8e/ddd/fec zero size 2024-05-17T21:53:14.424 INFO:tasks.workunit.client.0.smithi107.stdout:2/990: link d0/d9/d1c/d37/df8/d10a/c124 d0/d9/d1c/c140 0 2024-05-17T21:53:14.425 INFO:tasks.workunit.client.1.smithi138.stdout:9/771: stat d6/d7/d16/d29/d6c/d9c/dc1/f53 0 2024-05-17T21:53:14.426 INFO:tasks.workunit.client.1.smithi138.stdout:0/766: symlink d3/d3d/lfe 0 2024-05-17T21:53:14.429 INFO:tasks.workunit.client.0.smithi107.stdout:2/991: dwrite d0/d9/d1c/d3e/f3f [4194304,4194304] 0 2024-05-17T21:53:14.430 INFO:tasks.workunit.client.1.smithi138.stdout:9/772: creat d6/d7/d16/d29/d38/d40/d44/d5c/dda/ff5 x:0 0 0 2024-05-17T21:53:14.432 INFO:tasks.workunit.client.1.smithi138.stdout:0/767: dwrite d3/f7 [4194304,4194304] 0 2024-05-17T21:53:14.434 INFO:tasks.workunit.client.1.smithi138.stdout:9/773: rmdir d6/d7/d16/d29/d48/d6e 39 2024-05-17T21:53:14.442 INFO:tasks.workunit.client.1.smithi138.stdout:4/955: dwrite d0/d3/d1d/d38/d8a/dbd/fc0 [0,4194304] 0 2024-05-17T21:53:14.442 INFO:tasks.workunit.client.1.smithi138.stdout:1/885: truncate f2 5171053 0 2024-05-17T21:53:14.443 INFO:tasks.workunit.client.0.smithi107.stdout:2/992: getdents d0/d9/d11/d18/d9e/deb/d109 0 2024-05-17T21:53:14.445 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:14 smithi107 ceph-mon[28434]: pgmap v45: 65 pgs: 65 active+clean; 3.5 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 83 MiB/s rd, 179 MiB/s wr, 432 op/s 2024-05-17T21:53:14.446 INFO:tasks.workunit.client.1.smithi138.stdout:4/956: dread d0/d3/d1d/d5b/fb6 [0,4194304] 0 2024-05-17T21:53:14.447 INFO:tasks.workunit.client.0.smithi107.stdout:2/993: dread d0/d9/d1c/d37/d9f/fa7 [4194304,4194304] 0 2024-05-17T21:53:14.448 INFO:tasks.workunit.client.1.smithi138.stdout:4/957: fsync d0/d3/d17/d5e/d7f/fe7 0 2024-05-17T21:53:14.450 INFO:tasks.workunit.client.1.smithi138.stdout:7/859: write d0/d18/d40/d5f/f10e [528543,44490] 0 2024-05-17T21:53:14.454 INFO:tasks.workunit.client.0.smithi107.stdout:2/994: dwrite d0/d9/d11/df/d5a/d6d/fca [4194304,4194304] 0 2024-05-17T21:53:14.455 INFO:tasks.workunit.client.1.smithi138.stdout:6/883: dwrite d7/d69/db5/dc8/d34/da6/f21 [0,4194304] 0 2024-05-17T21:53:14.462 INFO:tasks.workunit.client.0.smithi107.stdout:2/995: creat d0/d9/d11/df/d5a/d6d/dc8/de8/ddb/f141 x:0 0 0 2024-05-17T21:53:14.463 INFO:tasks.workunit.client.0.smithi107.stdout:2/996: stat d0/d9/d11/df/d5a/dc7/le2 0 2024-05-17T21:53:14.463 INFO:tasks.workunit.client.1.smithi138.stdout:0/768: dread d3/d3d/f78 [0,4194304] 0 2024-05-17T21:53:14.467 INFO:tasks.workunit.client.1.smithi138.stdout:6/884: creat d7/d69/db5/dc8/d43/d5f/d6a/d77/de5/f128 x:0 0 0 2024-05-17T21:53:14.475 INFO:tasks.workunit.client.1.smithi138.stdout:1/886: dread d4/df/f1b [0,4194304] 0 2024-05-17T21:53:14.477 INFO:tasks.workunit.client.1.smithi138.stdout:8/737: dwrite d0/dc/d1a/d51/d82/f8e [4194304,4194304] 0 2024-05-17T21:53:14.477 INFO:tasks.workunit.client.1.smithi138.stdout:1/887: dread - d4/dd/d3b/d9f/da9/ff6 zero size 2024-05-17T21:53:14.480 INFO:tasks.workunit.client.1.smithi138.stdout:1/888: rmdir d4 39 2024-05-17T21:53:14.483 INFO:tasks.workunit.client.1.smithi138.stdout:8/738: creat d0/dc/d3b/d4d/fda x:0 0 0 2024-05-17T21:53:14.485 INFO:tasks.workunit.client.1.smithi138.stdout:8/739: chown d0/dc/d1a/d78/fd1 924743719 1 2024-05-17T21:53:14.487 INFO:tasks.workunit.client.1.smithi138.stdout:7/860: truncate d0/f24 4116934 0 2024-05-17T21:53:14.489 INFO:tasks.workunit.client.0.smithi107.stdout:2/997: write d0/d9/d11/df/d5a/d6d/fa0 [310035,126766] 0 2024-05-17T21:53:14.489 INFO:tasks.workunit.client.1.smithi138.stdout:4/958: dwrite d0/d2f/d49/d60/f25 [0,4194304] 0 2024-05-17T21:53:14.489 INFO:tasks.workunit.client.1.smithi138.stdout:6/885: write d7/fa [2523963,60155] 0 2024-05-17T21:53:14.490 INFO:tasks.workunit.client.1.smithi138.stdout:8/740: read d0/ff [6270022,33135] 0 2024-05-17T21:53:14.491 INFO:tasks.workunit.client.1.smithi138.stdout:0/769: truncate d3/d3d/ff7 78723 0 2024-05-17T21:53:14.498 INFO:tasks.workunit.client.1.smithi138.stdout:8/741: mkdir d0/dc/d1a/d51/d82/ddb 0 2024-05-17T21:53:14.500 INFO:tasks.workunit.client.1.smithi138.stdout:0/770: mknod d3/d10/d6d/d81/cff 0 2024-05-17T21:53:14.504 INFO:tasks.workunit.client.1.smithi138.stdout:4/959: mkdir d0/d3/d17/d21/d78/d9f/d104/d148 0 2024-05-17T21:53:14.504 INFO:tasks.workunit.client.0.smithi107.stdout:2/998: write d0/d9/d1c/d37/f10f [2034848,41828] 0 2024-05-17T21:53:14.506 INFO:tasks.workunit.client.1.smithi138.stdout:8/742: dread d0/dc/d3b/f43 [0,4194304] 0 2024-05-17T21:53:14.507 INFO:tasks.workunit.client.1.smithi138.stdout:1/889: dread d4/db/f4b [0,4194304] 0 2024-05-17T21:53:14.511 INFO:tasks.workunit.client.1.smithi138.stdout:6/886: rename d7/f2e to d7/d69/db5/dc8/d43/d3f/d79/f129 0 2024-05-17T21:53:14.513 INFO:tasks.workunit.client.1.smithi138.stdout:0/771: symlink d3/d10/d39/dd9/l100 0 2024-05-17T21:53:14.513 INFO:tasks.workunit.client.1.smithi138.stdout:1/890: dwrite d4/db/f110 [0,4194304] 0 2024-05-17T21:53:14.516 INFO:tasks.workunit.client.1.smithi138.stdout:7/861: getdents d0/d18/d64/d7c/da2/dc5/dcf 0 2024-05-17T21:53:14.519 INFO:tasks.workunit.client.1.smithi138.stdout:1/891: read d4/db/d29/d89/ffd [1734789,93740] 0 2024-05-17T21:53:14.521 INFO:tasks.workunit.client.1.smithi138.stdout:6/887: mknod d7/d69/db5/dc8/d43/d5f/da4/c12a 0 2024-05-17T21:53:14.523 INFO:tasks.workunit.client.1.smithi138.stdout:7/862: creat d0/d9d/f119 x:0 0 0 2024-05-17T21:53:14.529 INFO:tasks.workunit.client.1.smithi138.stdout:8/743: getdents d0/d2f 0 2024-05-17T21:53:14.530 INFO:tasks.workunit.client.1.smithi138.stdout:7/863: rename d0/d18/d64/d7c/da2/fc1 to d0/d18/d64/f11a 0 2024-05-17T21:53:14.531 INFO:tasks.workunit.client.1.smithi138.stdout:7/864: write d0/d18/d9c/dd4/ff1 [233793,50118] 0 2024-05-17T21:53:14.533 INFO:tasks.workunit.client.1.smithi138.stdout:6/888: creat d7/d69/db5/dc8/d43/d5f/f12b x:0 0 0 2024-05-17T21:53:14.535 INFO:tasks.workunit.client.1.smithi138.stdout:4/960: dread d0/d3/db/f20 [0,4194304] 0 2024-05-17T21:53:14.537 INFO:tasks.workunit.client.1.smithi138.stdout:0/772: write d3/fe3 [518026,20295] 0 2024-05-17T21:53:14.540 INFO:tasks.workunit.client.1.smithi138.stdout:4/961: fdatasync d0/d3/d1d/d38/d8a/fab 0 2024-05-17T21:53:14.541 INFO:tasks.workunit.client.1.smithi138.stdout:0/773: fsync d3/d10/f22 0 2024-05-17T21:53:14.544 INFO:tasks.workunit.client.1.smithi138.stdout:6/889: dwrite d7/d69/db5/dc8/d43/d5f/da4/db0/d113/fd1 [0,4194304] 0 2024-05-17T21:53:14.591 INFO:tasks.workunit.client.1.smithi138.stdout:0/774: mkdir d3/d10/d6d/d9f/da8/d101 0 2024-05-17T21:53:14.591 INFO:tasks.workunit.client.1.smithi138.stdout:6/890: chown d7/d69/db5/dc8/d43/d5f/da4/fee 122058220 1 2024-05-17T21:53:14.592 INFO:tasks.workunit.client.1.smithi138.stdout:1/892: dwrite d4/f9 [0,4194304] 0 2024-05-17T21:53:14.599 INFO:tasks.workunit.client.1.smithi138.stdout:8/744: dwrite d0/dc/d3b/fba [0,4194304] 0 2024-05-17T21:53:14.605 INFO:tasks.workunit.client.1.smithi138.stdout:4/962: dwrite d0/d3/d1d/d38/d8a/dbd/ff5 [0,4194304] 0 2024-05-17T21:53:14.609 INFO:tasks.workunit.client.1.smithi138.stdout:0/775: rename d3/d10/d38/d70/d80/f8b to d3/d10/d6d/d9f/f102 0 2024-05-17T21:53:14.609 INFO:tasks.workunit.client.1.smithi138.stdout:6/891: truncate d7/d69/db5/f11f 774549 0 2024-05-17T21:53:14.614 INFO:tasks.workunit.client.1.smithi138.stdout:7/865: dwrite d0/f2a [0,4194304] 0 2024-05-17T21:53:14.615 INFO:tasks.workunit.client.1.smithi138.stdout:8/745: creat d0/dc/d1a/fdc x:0 0 0 2024-05-17T21:53:14.615 INFO:tasks.workunit.client.1.smithi138.stdout:4/963: symlink d0/d1b/d88/l149 0 2024-05-17T21:53:14.620 INFO:tasks.workunit.client.1.smithi138.stdout:1/893: dwrite d4/dd/d3b/d6c/dcf/ff9 [0,4194304] 0 2024-05-17T21:53:14.620 INFO:tasks.workunit.client.1.smithi138.stdout:6/892: creat d7/d8/d10c/d122/f12c x:0 0 0 2024-05-17T21:53:14.623 INFO:tasks.workunit.client.1.smithi138.stdout:8/746: mknod d0/d11/d3e/dca/cdd 0 2024-05-17T21:53:14.626 INFO:tasks.workunit.client.1.smithi138.stdout:1/894: chown d4/df/d3a/d3c/d63/d6d/de0/f105 176762416 1 2024-05-17T21:53:14.628 INFO:tasks.workunit.client.1.smithi138.stdout:6/893: creat d7/d8/de9/d11c/f12d x:0 0 0 2024-05-17T21:53:14.630 INFO:tasks.workunit.client.1.smithi138.stdout:4/964: dwrite d0/d2f/d49/d60/f3c [0,4194304] 0 2024-05-17T21:53:14.632 INFO:tasks.workunit.client.1.smithi138.stdout:8/747: rmdir d0/dc/d3b/d87 39 2024-05-17T21:53:14.633 INFO:tasks.workunit.client.1.smithi138.stdout:6/894: write d7/d69/db5/dc8/d34/da6/f26 [1038408,69477] 0 2024-05-17T21:53:14.635 INFO:tasks.workunit.client.1.smithi138.stdout:6/895: fsync d7/d8/fb4 0 2024-05-17T21:53:14.635 INFO:tasks.workunit.client.1.smithi138.stdout:0/776: write d3/d10/d39/f72 [1020781,7381] 0 2024-05-17T21:53:14.636 INFO:tasks.workunit.client.1.smithi138.stdout:6/896: stat d7/l16 0 2024-05-17T21:53:14.636 INFO:tasks.workunit.client.1.smithi138.stdout:1/895: rename d4/dd/d3b/d6c/dc1/l103 to d4/df/d3a/d3c/d63/l124 0 2024-05-17T21:53:14.640 INFO:tasks.workunit.client.1.smithi138.stdout:8/748: unlink d0/dc/fcb 0 2024-05-17T21:53:14.640 INFO:tasks.workunit.client.1.smithi138.stdout:6/897: fdatasync d7/d69/db5/dc8/d43/d63/f115 0 2024-05-17T21:53:14.642 INFO:tasks.workunit.client.1.smithi138.stdout:7/866: dwrite d0/d18/d64/d7c/da2/f8b [0,4194304] 0 2024-05-17T21:53:14.645 INFO:tasks.workunit.client.1.smithi138.stdout:0/777: symlink d3/d3d/da2/l103 0 2024-05-17T21:53:14.646 INFO:tasks.workunit.client.1.smithi138.stdout:1/896: creat d4/df/d3a/d3c/d63/d6d/de0/f125 x:0 0 0 2024-05-17T21:53:14.648 INFO:tasks.workunit.client.1.smithi138.stdout:6/898: creat d7/d69/db5/dc8/d43/d5f/da4/db0/f12e x:0 0 0 2024-05-17T21:53:14.649 INFO:tasks.workunit.client.1.smithi138.stdout:4/965: dread d0/d2f/d49/d60/d1f/ffb [0,4194304] 0 2024-05-17T21:53:14.651 INFO:tasks.workunit.client.1.smithi138.stdout:0/778: write d3/d3d/d48/fe4 [276309,23621] 0 2024-05-17T21:53:14.652 INFO:tasks.workunit.client.1.smithi138.stdout:1/897: mknod d4/d93/c126 0 2024-05-17T21:53:14.654 INFO:tasks.workunit.client.1.smithi138.stdout:7/867: creat d0/d18/d40/d41/d51/d66/d9e/df9/dfe/f11b x:0 0 0 2024-05-17T21:53:14.655 INFO:tasks.workunit.client.1.smithi138.stdout:6/899: mknod d7/d69/db5/dc8/d43/d5f/da4/db0/dc1/c12f 0 2024-05-17T21:53:14.657 INFO:tasks.workunit.client.1.smithi138.stdout:4/966: symlink d0/d3/d17/d21/d78/d9f/d11e/l14a 0 2024-05-17T21:53:14.658 INFO:tasks.workunit.client.1.smithi138.stdout:0/779: chown d3/d10/d39/d97/lae 565257 1 2024-05-17T21:53:14.662 INFO:tasks.workunit.client.1.smithi138.stdout:6/900: mknod d7/d69/db5/dc8/d43/d3f/db6/c130 0 2024-05-17T21:53:14.668 INFO:tasks.workunit.client.1.smithi138.stdout:0/780: unlink d3/l42 0 2024-05-17T21:53:14.670 INFO:tasks.workunit.client.1.smithi138.stdout:6/901: write d7/d69/db5/dc8/d43/d3f/f4a [2406143,60940] 0 2024-05-17T21:53:14.674 INFO:tasks.workunit.client.1.smithi138.stdout:8/749: rename d0/d11/d6a/f91 to d0/dc/d3b/d4d/fde 0 2024-05-17T21:53:14.677 INFO:tasks.workunit.client.1.smithi138.stdout:1/898: truncate d4/db/d60/f118 2240231 0 2024-05-17T21:53:14.680 INFO:tasks.workunit.client.1.smithi138.stdout:6/902: rename d7/d69/db5/dc8/d43/d5f/da4/db0/d113 to d7/d69/db5/dc8/d43/d5f/d6a/d131 0 2024-05-17T21:53:14.680 INFO:tasks.workunit.client.1.smithi138.stdout:7/868: dread d0/d18/d40/d5f/f8f [0,4194304] 0 2024-05-17T21:53:14.681 INFO:tasks.workunit.client.1.smithi138.stdout:8/750: truncate d0/dc/d3b/d4d/fbd 517265 0 2024-05-17T21:53:14.762 INFO:tasks.workunit.client.1.smithi138.stdout:2/878: sync 2024-05-17T21:53:14.763 INFO:tasks.workunit.client.1.smithi138.stdout:1/899: rename d4/dd/f15 to d4/df/d3a/f127 0 2024-05-17T21:53:14.766 INFO:tasks.workunit.client.1.smithi138.stdout:8/751: write d0/dc/d1a/d33/d4a/fc8 [955198,75189] 0 2024-05-17T21:53:14.766 INFO:tasks.workunit.client.1.smithi138.stdout:1/900: rmdir d4/d93 39 2024-05-17T21:53:14.768 INFO:tasks.workunit.client.1.smithi138.stdout:6/903: mknod d7/d69/db5/dc8/d43/d5f/da4/c132 0 2024-05-17T21:53:14.769 INFO:tasks.workunit.client.1.smithi138.stdout:2/879: creat d3/d21/d2f/d3e/d58/f121 x:0 0 0 2024-05-17T21:53:14.771 INFO:tasks.workunit.client.1.smithi138.stdout:0/781: dread d3/d10/d38/d70/d80/d94/fa4 [0,4194304] 0 2024-05-17T21:53:14.771 INFO:tasks.workunit.client.1.smithi138.stdout:2/880: fsync d3/d21/d2f/f107 0 2024-05-17T21:53:14.772 INFO:tasks.workunit.client.1.smithi138.stdout:8/752: dread - d0/dc/d1a/d33/d4a/fae zero size 2024-05-17T21:53:14.774 INFO:tasks.workunit.client.1.smithi138.stdout:0/782: chown d3/d10/d6d/d9f/da8/le2 2559618 1 2024-05-17T21:53:14.776 INFO:tasks.workunit.client.1.smithi138.stdout:1/901: mkdir d4/dd/d3b/da7/d108/d128 0 2024-05-17T21:53:14.778 INFO:tasks.workunit.client.1.smithi138.stdout:1/902: read - d4/d2e/f106 zero size 2024-05-17T21:53:14.778 INFO:tasks.workunit.client.1.smithi138.stdout:2/881: rename d3/d21/d2f/d3e/l52 to d3/da0/daa/l122 0 2024-05-17T21:53:14.781 INFO:tasks.workunit.client.1.smithi138.stdout:0/783: mkdir d3/d10/d6d/d81/d104 0 2024-05-17T21:53:14.782 INFO:tasks.workunit.client.1.smithi138.stdout:2/882: read - d3/d21/d2f/d3e/d58/d77/fd1 zero size 2024-05-17T21:53:14.782 INFO:tasks.workunit.client.1.smithi138.stdout:6/904: dwrite d7/d8/d10c/d122/f12c [0,4194304] 0 2024-05-17T21:53:14.782 INFO:tasks.workunit.client.1.smithi138.stdout:1/903: write d4/db/d29/d5d/f86 [1487669,86150] 0 2024-05-17T21:53:14.784 INFO:tasks.workunit.client.1.smithi138.stdout:8/753: rmdir d0/dc/d3b/d4d/d7d/d97/dc1 39 2024-05-17T21:53:14.787 INFO:tasks.workunit.client.0.smithi107.stdout:2/999: sync 2024-05-17T21:53:14.789 INFO:tasks.workunit.client.1.smithi138.stdout:6/905: write d7/d8/d10c/d122/f12c [1429039,89376] 0 2024-05-17T21:53:14.789 INFO:tasks.workunit.client.1.smithi138.stdout:7/869: dread d0/d18/d64/d7c/fad [0,4194304] 0 2024-05-17T21:53:14.791 INFO:tasks.workunit.client.0.smithi107.stderr:+ rm -rf -- ./tmp.8ZNzSwV3Np 2024-05-17T21:53:14.792 INFO:tasks.workunit.client.1.smithi138.stdout:0/784: stat d3/d10/d38/d3e/d59/fd0 0 2024-05-17T21:53:14.793 INFO:tasks.workunit.client.1.smithi138.stdout:7/870: read - d0/d18/d40/d41/d51/d66/d9e/fde zero size 2024-05-17T21:53:14.794 INFO:tasks.workunit.client.1.smithi138.stdout:2/883: dwrite d3/d21/d2f/f5f [0,4194304] 0 2024-05-17T21:53:14.795 INFO:tasks.workunit.client.1.smithi138.stdout:6/906: read d7/d69/db5/dc8/d43/d5f/d6a/d131/f100 [113328,61393] 0 2024-05-17T21:53:14.796 INFO:tasks.workunit.client.1.smithi138.stdout:8/754: rename d0/l93 to d0/dc/d1a/d33/d4a/d4e/ldf 0 2024-05-17T21:53:14.801 INFO:tasks.workunit.client.1.smithi138.stdout:6/907: truncate d7/d69/db5/dc8/d43/d63/f104 4947732 0 2024-05-17T21:53:14.803 INFO:tasks.workunit.client.1.smithi138.stdout:2/884: mkdir d3/d21/d2f/d3e/d55/d73/d123 0 2024-05-17T21:53:14.804 INFO:tasks.workunit.client.1.smithi138.stdout:1/904: link d4/dd/d3b/lb3 d4/dd/d3b/d6c/dc1/l129 0 2024-05-17T21:53:14.807 INFO:tasks.workunit.client.1.smithi138.stdout:7/871: write d0/d18/d40/d5f/f8f [4429654,90476] 0 2024-05-17T21:53:14.812 INFO:tasks.workunit.client.1.smithi138.stdout:0/785: link d3/f18 d3/d10/d6d/db6/db9/f105 0 2024-05-17T21:53:14.813 INFO:tasks.workunit.client.1.smithi138.stdout:0/786: chown d3/d10/l4c 276 1 2024-05-17T21:53:14.813 INFO:tasks.workunit.client.1.smithi138.stdout:1/905: getdents d4/dd/d3b/da7/d108/d128 0 2024-05-17T21:53:14.815 INFO:tasks.workunit.client.1.smithi138.stdout:0/787: stat d3/d10/d38/d70/d80/lb5 0 2024-05-17T21:53:14.816 INFO:tasks.workunit.client.1.smithi138.stdout:6/908: getdents d7/d69/db5/dc8/d43/d3f 0 2024-05-17T21:53:14.818 INFO:tasks.workunit.client.1.smithi138.stdout:2/885: dwrite d3/d21/d2f/d3e/d55/d99/d112/fca [0,4194304] 0 2024-05-17T21:53:14.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:14 smithi138 ceph-mon[35948]: pgmap v45: 65 pgs: 65 active+clean; 3.5 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 83 MiB/s rd, 179 MiB/s wr, 432 op/s 2024-05-17T21:53:14.827 INFO:tasks.workunit.client.1.smithi138.stdout:0/788: unlink d3/d10/d6d/db6/db9/ffc 0 2024-05-17T21:53:14.827 INFO:tasks.workunit.client.1.smithi138.stdout:6/909: fsync d7/d69/db5/dc8/f9e 0 2024-05-17T21:53:14.830 INFO:tasks.workunit.client.1.smithi138.stdout:8/755: dread d0/d11/f69 [4194304,4194304] 0 2024-05-17T21:53:14.831 INFO:tasks.workunit.client.1.smithi138.stdout:1/906: dwrite d4/d2e/da0/daa/f115 [0,4194304] 0 2024-05-17T21:53:14.836 INFO:tasks.workunit.client.1.smithi138.stdout:0/789: dread d3/d3d/d48/fe4 [0,4194304] 0 2024-05-17T21:53:14.841 INFO:tasks.workunit.client.1.smithi138.stdout:1/907: dwrite d4/dd/d3b/d9f/db9/f11e [0,4194304] 0 2024-05-17T21:53:14.846 INFO:tasks.workunit.client.1.smithi138.stdout:2/886: dread d3/d21/d2f/f33 [4194304,4194304] 0 2024-05-17T21:53:14.869 INFO:tasks.workunit.client.1.smithi138.stdout:6/910: fsync d7/d8/f102 0 2024-05-17T21:53:14.871 INFO:tasks.workunit.client.1.smithi138.stdout:8/756: mkdir d0/dc/d3b/da4/dbc/de0 0 2024-05-17T21:53:14.872 INFO:tasks.workunit.client.1.smithi138.stdout:0/790: chown d3/d10/cfd 19687764 1 2024-05-17T21:53:14.883 INFO:tasks.workunit.client.1.smithi138.stdout:6/911: creat d7/d69/db5/dc8/d43/d5f/d94/f133 x:0 0 0 2024-05-17T21:53:14.888 INFO:tasks.workunit.client.1.smithi138.stdout:1/908: getdents d4/dd/d64 0 2024-05-17T21:53:14.889 INFO:tasks.workunit.client.1.smithi138.stdout:2/887: dread d3/d21/d2f/d3e/d55/d99/f8f [0,4194304] 0 2024-05-17T21:53:14.891 INFO:tasks.workunit.client.1.smithi138.stdout:6/912: chown d7/d69/db5/dc8/d43/fc4 13098444 1 2024-05-17T21:53:14.898 INFO:tasks.workunit.client.1.smithi138.stdout:2/888: symlink d3/d21/d2f/d3e/d63/da3/l124 0 2024-05-17T21:53:14.901 INFO:tasks.workunit.client.1.smithi138.stdout:8/757: link d0/dc/fc6 d0/dc/d1a/d78/fe1 0 2024-05-17T21:53:14.906 INFO:tasks.workunit.client.1.smithi138.stdout:2/889: unlink d3/d21/f2b 0 2024-05-17T21:53:14.908 INFO:tasks.workunit.client.1.smithi138.stdout:8/758: creat d0/d31/fe2 x:0 0 0 2024-05-17T21:53:14.910 INFO:tasks.workunit.client.1.smithi138.stdout:2/890: write d3/d21/d2f/d3e/d55/d73/f97 [1031497,9745] 0 2024-05-17T21:53:14.911 INFO:tasks.workunit.client.1.smithi138.stdout:8/759: symlink d0/dc/d3b/d4d/le3 0 2024-05-17T21:53:14.912 INFO:tasks.workunit.client.1.smithi138.stdout:8/760: read - d0/dc/d3b/d4d/fda zero size 2024-05-17T21:53:14.914 INFO:tasks.workunit.client.1.smithi138.stdout:2/891: creat d3/d21/d2f/d3e/d55/d99/d84/d9f/dfd/f125 x:0 0 0 2024-05-17T21:53:14.917 INFO:tasks.workunit.client.1.smithi138.stdout:2/892: truncate d3/d21/d2f/d3e/d55/f111 306431 0 2024-05-17T21:53:14.919 INFO:tasks.workunit.client.1.smithi138.stdout:8/761: dwrite d0/dc/d1a/f2a [4194304,4194304] 0 2024-05-17T21:53:14.925 INFO:tasks.workunit.client.1.smithi138.stdout:2/893: dread d3/f20 [0,4194304] 0 2024-05-17T21:53:14.942 INFO:tasks.workunit.client.1.smithi138.stdout:0/791: dwrite d3/d10/f22 [0,4194304] 0 2024-05-17T21:53:14.946 INFO:tasks.workunit.client.1.smithi138.stdout:0/792: chown d3/f21 3505 1 2024-05-17T21:53:14.953 INFO:tasks.workunit.client.1.smithi138.stdout:1/909: dwrite d4/db/d29/d5d/f6f [0,4194304] 0 2024-05-17T21:53:14.955 INFO:tasks.workunit.client.1.smithi138.stdout:1/910: truncate d4/d2e/f106 306452 0 2024-05-17T21:53:14.965 INFO:tasks.workunit.client.1.smithi138.stdout:8/762: write d0/dc/d1a/d33/d4a/fa8 [200954,27885] 0 2024-05-17T21:53:14.965 INFO:tasks.workunit.client.1.smithi138.stdout:1/911: dwrite d4/dd/db4/dba/fea [0,4194304] 0 2024-05-17T21:53:14.966 INFO:tasks.workunit.client.1.smithi138.stdout:8/763: dread - d0/d71/fc7 zero size 2024-05-17T21:53:14.973 INFO:tasks.workunit.client.1.smithi138.stdout:8/764: symlink d0/dc/d1a/d33/le4 0 2024-05-17T21:53:14.974 INFO:tasks.workunit.client.1.smithi138.stdout:2/894: dwrite d3/d21/d2f/d3e/d55/d99/dd2/fd7 [0,4194304] 0 2024-05-17T21:53:14.976 INFO:tasks.workunit.client.1.smithi138.stdout:1/912: dread d4/db/f4b [4194304,4194304] 0 2024-05-17T21:53:14.981 INFO:tasks.workunit.client.1.smithi138.stdout:1/913: dread - d4/dd/d3b/d9f/db9/dae/f11b zero size 2024-05-17T21:53:14.982 INFO:tasks.workunit.client.1.smithi138.stdout:0/793: dread d3/fd [0,4194304] 0 2024-05-17T21:53:14.984 INFO:tasks.workunit.client.1.smithi138.stdout:0/794: read - d3/d10/d6d/d9f/fed zero size 2024-05-17T21:53:14.984 INFO:tasks.workunit.client.1.smithi138.stdout:8/765: dwrite d0/dc/d1a/f4b [0,4194304] 0 2024-05-17T21:53:14.985 INFO:tasks.workunit.client.1.smithi138.stdout:2/895: creat d3/da0/df6/f126 x:0 0 0 2024-05-17T21:53:14.987 INFO:tasks.workunit.client.1.smithi138.stdout:2/896: chown d3/d81/d96/fb4 435 1 2024-05-17T21:53:14.991 INFO:tasks.workunit.client.1.smithi138.stdout:8/766: unlink d0/dc/d1a/f4b 0 2024-05-17T21:53:14.997 INFO:tasks.workunit.client.1.smithi138.stdout:8/767: mknod d0/dc/d3b/da0/ce5 0 2024-05-17T21:53:15.000 INFO:tasks.workunit.client.1.smithi138.stdout:2/897: creat d3/da0/f127 x:0 0 0 2024-05-17T21:53:15.002 INFO:tasks.workunit.client.1.smithi138.stdout:0/795: link d3/d10/d38/d70/d80/lb5 d3/d10/d39/l106 0 2024-05-17T21:53:15.003 INFO:tasks.workunit.client.1.smithi138.stdout:1/914: write d4/df/d4e/fc7 [396949,96342] 0 2024-05-17T21:53:15.004 INFO:tasks.workunit.client.1.smithi138.stdout:0/796: dread - d3/d10/fd8 zero size 2024-05-17T21:53:15.008 INFO:tasks.workunit.client.1.smithi138.stdout:2/898: dwrite d3/da0/f127 [0,4194304] 0 2024-05-17T21:53:15.013 INFO:tasks.workunit.client.1.smithi138.stdout:0/797: chown d3/d10/d38/d3e/d59/fda 42 1 2024-05-17T21:53:15.015 INFO:tasks.workunit.client.1.smithi138.stdout:8/768: dread d0/dc/d1a/d51/f73 [0,4194304] 0 2024-05-17T21:53:15.015 INFO:tasks.workunit.client.1.smithi138.stdout:2/899: dread - d3/da0/daa/feb zero size 2024-05-17T21:53:15.017 INFO:tasks.workunit.client.1.smithi138.stdout:0/798: chown d3/d3d/ff7 189248 1 2024-05-17T21:53:15.019 INFO:tasks.workunit.client.1.smithi138.stdout:8/769: creat d0/dc/d3b/d4d/d7d/fe6 x:0 0 0 2024-05-17T21:53:15.020 INFO:tasks.workunit.client.1.smithi138.stdout:8/770: chown d0/dc/d1a/d33 4 1 2024-05-17T21:53:15.021 INFO:tasks.workunit.client.1.smithi138.stdout:2/900: creat d3/d21/d2f/d3e/d58/d77/da4/df2/dd0/f128 x:0 0 0 2024-05-17T21:53:15.025 INFO:tasks.workunit.client.1.smithi138.stdout:2/901: getdents d3/d21/d2f/d3e/d58/d77/da4/dff 0 2024-05-17T21:53:15.027 INFO:tasks.workunit.client.1.smithi138.stdout:2/902: stat d3/d21/d2f/d3e/d58/d77/da4/dc5 0 2024-05-17T21:53:15.027 INFO:tasks.workunit.client.1.smithi138.stdout:8/771: truncate d0/dc/d3b/d4d/fbd 95765 0 2024-05-17T21:53:15.029 INFO:tasks.workunit.client.1.smithi138.stdout:1/915: dread d4/dd/f90 [0,4194304] 0 2024-05-17T21:53:15.030 INFO:tasks.workunit.client.1.smithi138.stdout:0/799: write d3/d3d/f78 [124327,116344] 0 2024-05-17T21:53:15.032 INFO:tasks.workunit.client.1.smithi138.stdout:8/772: symlink d0/d31/le7 0 2024-05-17T21:53:15.036 INFO:tasks.workunit.client.1.smithi138.stdout:2/903: dwrite d3/d21/d2f/dd4/f104 [0,4194304] 0 2024-05-17T21:53:15.038 INFO:tasks.workunit.client.1.smithi138.stdout:8/773: dread d0/dc/fc6 [0,4194304] 0 2024-05-17T21:53:15.043 INFO:tasks.workunit.client.1.smithi138.stdout:1/916: dread d4/df/d3a/d3c/d63/d6d/de0/f105 [0,4194304] 0 2024-05-17T21:53:15.046 INFO:tasks.workunit.client.1.smithi138.stdout:0/800: write d3/f83 [884279,41227] 0 2024-05-17T21:53:15.049 INFO:tasks.workunit.client.1.smithi138.stdout:1/917: rmdir d4/df 39 2024-05-17T21:53:15.050 INFO:tasks.workunit.client.1.smithi138.stdout:0/801: dread d3/d10/d6d/d81/fbd [0,4194304] 0 2024-05-17T21:53:15.050 INFO:tasks.workunit.client.1.smithi138.stdout:1/918: stat d4/dd/d3b/d6c/dc1/fdf 0 2024-05-17T21:53:15.052 INFO:tasks.workunit.client.1.smithi138.stdout:0/802: read d3/fe3 [781649,127381] 0 2024-05-17T21:53:15.053 INFO:tasks.workunit.client.1.smithi138.stdout:8/774: truncate d0/f2 2773064 0 2024-05-17T21:53:15.053 INFO:tasks.workunit.client.1.smithi138.stdout:5/825: sync 2024-05-17T21:53:15.054 INFO:tasks.workunit.client.1.smithi138.stdout:5/826: stat d1/d53/f80 0 2024-05-17T21:53:15.057 INFO:tasks.workunit.client.1.smithi138.stdout:2/904: rename d3/d21/d2f/d3e/d58/fae to d3/f129 0 2024-05-17T21:53:15.062 INFO:tasks.workunit.client.1.smithi138.stdout:1/919: mknod d4/dd/d3b/da7/d108/c12a 0 2024-05-17T21:53:15.062 INFO:tasks.workunit.client.1.smithi138.stdout:0/803: creat d3/d10/d6d/d9f/da8/f107 x:0 0 0 2024-05-17T21:53:15.071 INFO:tasks.workunit.client.1.smithi138.stdout:0/804: mknod d3/d10/d6d/d9f/da8/daa/c108 0 2024-05-17T21:53:15.073 INFO:tasks.workunit.client.1.smithi138.stdout:5/827: link d1/d46/d95/f10a d1/dd1/dca/dd0/f10c 0 2024-05-17T21:53:15.074 INFO:tasks.workunit.client.1.smithi138.stdout:7/872: sync 2024-05-17T21:53:15.074 INFO:tasks.workunit.client.1.smithi138.stdout:4/967: sync 2024-05-17T21:53:15.074 INFO:tasks.workunit.client.1.smithi138.stdout:9/774: sync 2024-05-17T21:53:15.076 INFO:tasks.workunit.client.1.smithi138.stdout:9/775: chown d6/d7/d16/d29/f94 18 1 2024-05-17T21:53:15.080 INFO:tasks.workunit.client.1.smithi138.stdout:5/828: symlink d1/d8a/l10d 0 2024-05-17T21:53:15.086 INFO:tasks.workunit.client.1.smithi138.stdout:8/775: write d0/dc/d8d/faf [967436,63643] 0 2024-05-17T21:53:15.086 INFO:tasks.workunit.client.1.smithi138.stdout:2/905: dwrite d3/d21/d2f/d3e/d55/d99/f8f [0,4194304] 0 2024-05-17T21:53:15.092 INFO:tasks.workunit.client.1.smithi138.stdout:1/920: write d4/dd/d3b/d6c/dcf/fd5 [1187181,51385] 0 2024-05-17T21:53:15.094 INFO:tasks.workunit.client.1.smithi138.stdout:5/829: unlink d1/d2c/f7b 0 2024-05-17T21:53:15.095 INFO:tasks.workunit.client.1.smithi138.stdout:7/873: getdents d0/d18/d64/d7c/da2/dff 0 2024-05-17T21:53:15.097 INFO:tasks.workunit.client.1.smithi138.stdout:2/906: creat d3/d21/f12a x:0 0 0 2024-05-17T21:53:15.099 INFO:tasks.workunit.client.1.smithi138.stdout:9/776: fdatasync d6/d7/d16/d29/d48/d6e/d74/d75/fbb 0 2024-05-17T21:53:15.101 INFO:tasks.workunit.client.1.smithi138.stdout:0/805: link d3/d10/c1c d3/d3d/d51/c109 0 2024-05-17T21:53:15.102 INFO:tasks.workunit.client.1.smithi138.stdout:4/968: dwrite d0/d3/d17/d21/f75 [0,4194304] 0 2024-05-17T21:53:15.103 INFO:tasks.workunit.client.1.smithi138.stdout:1/921: stat d4/df/d5f/ffb 0 2024-05-17T21:53:15.103 INFO:tasks.workunit.client.1.smithi138.stdout:0/806: dread - d3/d10/d6d/f74 zero size 2024-05-17T21:53:15.106 INFO:tasks.workunit.client.1.smithi138.stdout:5/830: creat d1/d53/d88/de5/ded/dc2/f10e x:0 0 0 2024-05-17T21:53:15.106 INFO:tasks.workunit.client.1.smithi138.stdout:0/807: chown d3/d10/d6d/db6/db9/cbf 100 1 2024-05-17T21:53:15.107 INFO:tasks.workunit.client.1.smithi138.stdout:7/874: mknod d0/d39/d78/c11c 0 2024-05-17T21:53:15.114 INFO:tasks.workunit.client.1.smithi138.stdout:7/875: dwrite d0/d18/d64/d7c/da2/f8b [0,4194304] 0 2024-05-17T21:53:15.114 INFO:tasks.workunit.client.1.smithi138.stdout:4/969: creat d0/d3/d17/d21/d78/d9f/d11e/f14b x:0 0 0 2024-05-17T21:53:15.116 INFO:tasks.workunit.client.1.smithi138.stdout:4/970: fsync d0/d1b/d42/f125 0 2024-05-17T21:53:15.116 INFO:tasks.workunit.client.1.smithi138.stdout:6/913: sync 2024-05-17T21:53:15.118 INFO:tasks.workunit.client.1.smithi138.stdout:0/808: creat d3/d10/d6d/d9f/da8/daa/f10a x:0 0 0 2024-05-17T21:53:15.118 INFO:tasks.workunit.client.1.smithi138.stdout:6/914: chown d7/c6e 1 1 2024-05-17T21:53:15.123 INFO:tasks.workunit.client.1.smithi138.stdout:8/776: rename d0/d2f/cd9 to d0/dc/d3b/d4d/d7d/d97/ce8 0 2024-05-17T21:53:15.123 INFO:tasks.workunit.client.1.smithi138.stdout:9/777: write d6/d7/d16/d29/d6c/d9c/dc1/f95 [1395773,53727] 0 2024-05-17T21:53:15.127 INFO:tasks.workunit.client.1.smithi138.stdout:7/876: rmdir d0/d18/d64/d7c/da2/de0 39 2024-05-17T21:53:15.127 INFO:tasks.workunit.client.1.smithi138.stdout:6/915: dwrite d7/d69/db5/dc8/d43/d5f/d6a/d77/de5/f128 [0,4194304] 0 2024-05-17T21:53:15.129 INFO:tasks.workunit.client.1.smithi138.stdout:4/971: mknod d0/d3/db/c14c 0 2024-05-17T21:53:15.130 INFO:tasks.workunit.client.1.smithi138.stdout:4/972: readlink d0/d1b/dd4/lf7 0 2024-05-17T21:53:15.132 INFO:tasks.workunit.client.1.smithi138.stdout:4/973: chown d0/d3/d17/d21/d78/d9f 3372 1 2024-05-17T21:53:15.136 INFO:tasks.workunit.client.1.smithi138.stdout:4/974: chown d0/d3/d1d/d38/de3/d117 136 1 2024-05-17T21:53:15.137 INFO:tasks.workunit.client.1.smithi138.stdout:1/922: rmdir d4/dd/d3b/da7/d108/d128 0 2024-05-17T21:53:15.138 INFO:tasks.workunit.client.1.smithi138.stdout:9/778: stat d6/d7/d16/d29/d38/d3c/fad 0 2024-05-17T21:53:15.138 INFO:tasks.workunit.client.1.smithi138.stdout:6/916: mknod d7/d69/db5/dc8/d34/da6/c134 0 2024-05-17T21:53:15.141 INFO:tasks.workunit.client.1.smithi138.stdout:7/877: unlink d0/d39/d78/c11c 0 2024-05-17T21:53:15.142 INFO:tasks.workunit.client.1.smithi138.stdout:5/831: fsync d1/d2c/d63/db5/d1b/ddb/ff3 0 2024-05-17T21:53:15.145 INFO:tasks.workunit.client.1.smithi138.stdout:1/923: dwrite f0 [0,4194304] 0 2024-05-17T21:53:15.151 INFO:tasks.workunit.client.1.smithi138.stdout:6/917: symlink d7/d69/db5/dc8/d43/d5f/d6a/d9b/l135 0 2024-05-17T21:53:15.154 INFO:tasks.workunit.client.1.smithi138.stdout:7/878: fsync d0/f19 0 2024-05-17T21:53:15.155 INFO:tasks.workunit.client.1.smithi138.stdout:8/777: dwrite d0/d11/f42 [0,4194304] 0 2024-05-17T21:53:15.161 INFO:tasks.workunit.client.1.smithi138.stdout:0/809: write d3/d10/d38/d3e/f5b [1986422,50176] 0 2024-05-17T21:53:15.161 INFO:tasks.workunit.client.1.smithi138.stdout:2/907: rename d3/d21/d2f/d3e/d58/d77/da4/df2/d10f to d3/d21/d2f/d3e/d58/d77/da4/d12b 0 2024-05-17T21:53:15.163 INFO:tasks.workunit.client.1.smithi138.stdout:2/908: stat d3/d21/d2f/d3e/d58/d77/da4/dff 0 2024-05-17T21:53:15.164 INFO:tasks.workunit.client.1.smithi138.stdout:1/924: creat d4/dd/d3b/d9f/f12b x:0 0 0 2024-05-17T21:53:15.165 INFO:tasks.workunit.client.1.smithi138.stdout:1/925: chown f0 11 1 2024-05-17T21:53:15.166 INFO:tasks.workunit.client.1.smithi138.stdout:5/832: write d1/d46/d95/f10a [130911,109008] 0 2024-05-17T21:53:15.169 INFO:tasks.workunit.client.1.smithi138.stdout:9/779: mkdir d6/d7/d16/d29/d38/d40/d44/d5c/dc8/ded/df3/df6 0 2024-05-17T21:53:15.171 INFO:tasks.workunit.client.1.smithi138.stdout:1/926: dwrite f0 [0,4194304] 0 2024-05-17T21:53:15.173 INFO:tasks.workunit.client.1.smithi138.stdout:1/927: read - d4/db/f7b zero size 2024-05-17T21:53:15.177 INFO:tasks.workunit.client.1.smithi138.stdout:9/780: dwrite d6/d7/f99 [0,4194304] 0 2024-05-17T21:53:15.178 INFO:tasks.workunit.client.1.smithi138.stdout:6/918: rename d7/d69/db5/dc8/d43/d63/l95 to d7/d8/d10c/l136 0 2024-05-17T21:53:15.181 INFO:tasks.workunit.client.1.smithi138.stdout:4/975: link d0/d3/d1d/d38/d8a/dbd/l107 d0/d2f/d49/d60/d1f/d32/d71/d95/l14d 0 2024-05-17T21:53:15.182 INFO:tasks.workunit.client.1.smithi138.stdout:2/909: dread d3/d21/d2f/d3e/d58/d77/da4/df2/fdc [0,4194304] 0 2024-05-17T21:53:15.223 INFO:tasks.workunit.client.1.smithi138.stdout:9/781: creat d6/d7/d16/d29/d38/d40/d44/d5c/dc8/ff7 x:0 0 0 2024-05-17T21:53:15.225 INFO:tasks.workunit.client.1.smithi138.stdout:6/919: creat d7/d69/db5/dc8/d43/d3f/f137 x:0 0 0 2024-05-17T21:53:15.254 INFO:tasks.workunit.client.1.smithi138.stdout:4/976: mknod d0/c14e 0 2024-05-17T21:53:15.258 INFO:tasks.workunit.client.1.smithi138.stdout:5/833: dread d1/d60/d2f/d5e/de4/fac [0,4194304] 0 2024-05-17T21:53:15.259 INFO:tasks.workunit.client.1.smithi138.stdout:2/910: rename d3/d21/f38 to d3/d21/d2f/d3e/d63/da3/f12c 0 2024-05-17T21:53:15.267 INFO:tasks.workunit.client.1.smithi138.stdout:0/810: creat d3/d10/f10b x:0 0 0 2024-05-17T21:53:15.268 INFO:tasks.workunit.client.1.smithi138.stdout:8/778: write d0/dc/d3b/d4d/f8c [584880,105384] 0 2024-05-17T21:53:15.270 INFO:tasks.workunit.client.1.smithi138.stdout:8/779: stat d0/dc/d3b/d4d/d7d/dd5 0 2024-05-17T21:53:15.276 INFO:tasks.workunit.client.1.smithi138.stdout:9/782: symlink d6/d7/d16/d29/d38/d40/d44/d5c/db4/lf8 0 2024-05-17T21:53:15.278 INFO:tasks.workunit.client.1.smithi138.stdout:6/920: chown d7/c1b 239 1 2024-05-17T21:53:15.286 INFO:tasks.workunit.client.1.smithi138.stdout:2/911: rmdir d3/d21/d2f/d3e/d55/d99/d84/d9f 39 2024-05-17T21:53:15.287 INFO:tasks.workunit.client.1.smithi138.stdout:7/879: getdents d0/d18/d40/d41 0 2024-05-17T21:53:15.291 INFO:tasks.workunit.client.1.smithi138.stdout:0/811: mkdir d3/d10/d39/d97/d10c 0 2024-05-17T21:53:15.301 INFO:tasks.workunit.client.1.smithi138.stdout:4/977: chown d0/d3/d17/d21/c4a 1370 1 2024-05-17T21:53:15.335 INFO:tasks.workunit.client.1.smithi138.stdout:0/812: dread d3/f8e [0,4194304] 0 2024-05-17T21:53:15.335 INFO:tasks.workunit.client.1.smithi138.stdout:5/834: creat d1/d2c/d63/db5/df5/f10f x:0 0 0 2024-05-17T21:53:15.335 INFO:tasks.workunit.client.1.smithi138.stdout:9/783: write d6/d7/d16/d29/d48/f23 [1436817,9013] 0 2024-05-17T21:53:15.335 INFO:tasks.workunit.client.1.smithi138.stdout:7/880: creat d0/d18/d64/d7c/da2/dc5/dcf/f11d x:0 0 0 2024-05-17T21:53:15.335 INFO:tasks.workunit.client.1.smithi138.stdout:0/813: dread - d3/d10/d38/d3e/d59/d9b/ff0 zero size 2024-05-17T21:53:15.335 INFO:tasks.workunit.client.1.smithi138.stdout:7/881: dread - d0/d4c/dec/f10d zero size 2024-05-17T21:53:15.336 INFO:tasks.workunit.client.1.smithi138.stdout:8/780: mkdir d0/dc/d3b/d4d/d7d/d97/de9 0 2024-05-17T21:53:15.336 INFO:tasks.workunit.client.1.smithi138.stdout:6/921: dwrite d7/d69/f8e [0,4194304] 0 2024-05-17T21:53:15.336 INFO:tasks.workunit.client.1.smithi138.stdout:8/781: fdatasync d0/dc/d3b/fba 0 2024-05-17T21:53:15.336 INFO:tasks.workunit.client.1.smithi138.stdout:6/922: readlink d7/d69/db5/dc8/d43/d5f/d6a/d77/lea 0 2024-05-17T21:53:15.336 INFO:tasks.workunit.client.1.smithi138.stdout:6/923: readlink d7/d69/db5/dc8/d43/d5f/d6a/d131/l10b 0 2024-05-17T21:53:15.336 INFO:tasks.workunit.client.1.smithi138.stdout:2/912: unlink d3/c1e 0 2024-05-17T21:53:15.336 INFO:tasks.workunit.client.1.smithi138.stdout:9/784: creat d6/d7/d16/d29/d38/d3c/ff9 x:0 0 0 2024-05-17T21:53:15.337 INFO:tasks.workunit.client.1.smithi138.stdout:0/814: symlink d3/d3d/d48/l10d 0 2024-05-17T21:53:15.337 INFO:tasks.workunit.client.1.smithi138.stdout:7/882: mknod d0/d18/d64/d7c/da2/dc5/c11e 0 2024-05-17T21:53:15.337 INFO:tasks.workunit.client.1.smithi138.stdout:4/978: mknod d0/d3/d1d/d108/c14f 0 2024-05-17T21:53:15.337 INFO:tasks.workunit.client.1.smithi138.stdout:8/782: mkdir d0/d63/dea 0 2024-05-17T21:53:15.337 INFO:tasks.workunit.client.1.smithi138.stdout:7/883: mknod d0/d18/d64/c11f 0 2024-05-17T21:53:15.341 INFO:tasks.workunit.client.1.smithi138.stdout:8/783: rmdir d0/dc/d1a/d33/d4a/d4e 39 2024-05-17T21:53:15.343 INFO:tasks.workunit.client.1.smithi138.stdout:4/979: dwrite d0/d2f/d49/d60/d1f/d7e/f83 [0,4194304] 0 2024-05-17T21:53:15.343 INFO:tasks.workunit.client.1.smithi138.stdout:6/924: mknod d7/c138 0 2024-05-17T21:53:15.347 INFO:tasks.workunit.client.1.smithi138.stdout:2/913: unlink d3/d21/d2f/d3e/d58/d77/c8d 0 2024-05-17T21:53:15.348 INFO:tasks.workunit.client.1.smithi138.stdout:6/925: dread d7/d69/db5/dc8/d43/d5f/d6a/d77/de5/f128 [0,4194304] 0 2024-05-17T21:53:15.352 INFO:tasks.workunit.client.1.smithi138.stdout:5/835: rename d1/d2c/d63/db5/d1b/fba to d1/d53/d88/db9/f110 0 2024-05-17T21:53:15.358 INFO:tasks.workunit.client.1.smithi138.stdout:4/980: truncate d0/d3/d17/d21/d78/fac 1032881 0 2024-05-17T21:53:15.359 INFO:tasks.workunit.client.1.smithi138.stdout:4/981: chown d0/d2f/d49/d60/d1f/d32/fe1 52898275 1 2024-05-17T21:53:15.364 INFO:tasks.workunit.client.1.smithi138.stdout:6/926: truncate d7/d69/ffd 36905 0 2024-05-17T21:53:15.365 INFO:tasks.workunit.client.1.smithi138.stdout:6/927: write d7/d69/db5/dc8/d60/f10a [1457937,4842] 0 2024-05-17T21:53:15.367 INFO:tasks.workunit.client.1.smithi138.stdout:2/914: write d3/d21/d2f/d3e/d63/da3/fd3 [607738,94933] 0 2024-05-17T21:53:15.371 INFO:tasks.workunit.client.1.smithi138.stdout:9/785: getdents d6/d7/d16/d29/d48/d6e/d74/d75/d76/d8e/dba 0 2024-05-17T21:53:15.372 INFO:tasks.workunit.client.1.smithi138.stdout:9/786: readlink d6/d7/d16/l79 0 2024-05-17T21:53:15.373 INFO:tasks.workunit.client.1.smithi138.stdout:9/787: chown d6/d7/d16/c43 11313 1 2024-05-17T21:53:15.374 INFO:tasks.workunit.client.1.smithi138.stdout:9/788: stat d6/d7/d16/d29/d48/d3b/ca9 0 2024-05-17T21:53:15.375 INFO:tasks.workunit.client.1.smithi138.stdout:9/789: write d6/d7/d16/fef [945124,5928] 0 2024-05-17T21:53:15.377 INFO:tasks.workunit.client.1.smithi138.stdout:4/982: write d0/d2f/d49/d60/d1f/f70 [3599730,38627] 0 2024-05-17T21:53:15.378 INFO:tasks.workunit.client.1.smithi138.stdout:9/790: readlink d6/d7/d16/d29/d38/d40/d44/d5c/db4/lf8 0 2024-05-17T21:53:15.379 INFO:tasks.workunit.client.1.smithi138.stdout:0/815: rename d3/f29 to d3/d10/d38/d70/d80/d94/f10e 0 2024-05-17T21:53:15.380 INFO:tasks.workunit.client.1.smithi138.stdout:8/784: dwrite d0/f2c [0,4194304] 0 2024-05-17T21:53:15.381 INFO:tasks.workunit.client.1.smithi138.stdout:6/928: creat d7/d69/db5/dc8/d43/d5f/d6a/d131/f139 x:0 0 0 2024-05-17T21:53:15.381 INFO:tasks.workunit.client.1.smithi138.stdout:5/836: getdents d1/d8a/df0 0 2024-05-17T21:53:15.382 INFO:tasks.workunit.client.1.smithi138.stdout:8/785: write d0/d11/f42 [541686,103659] 0 2024-05-17T21:53:15.384 INFO:tasks.workunit.client.1.smithi138.stdout:7/884: creat d0/d18/d40/d41/d51/d66/d9e/f120 x:0 0 0 2024-05-17T21:53:15.388 INFO:tasks.workunit.client.1.smithi138.stdout:4/983: chown d0/d1b/d42/c11f 12746 1 2024-05-17T21:53:15.389 INFO:tasks.workunit.client.1.smithi138.stdout:0/816: mkdir d3/d3d/d51/d10f 0 2024-05-17T21:53:15.391 INFO:tasks.workunit.client.1.smithi138.stdout:9/791: rename d6/c14 to d6/d7/d16/d29/d48/d67/d86/cfa 0 2024-05-17T21:53:15.391 INFO:tasks.workunit.client.1.smithi138.stdout:4/984: write d0/d2f/d49/d60/d1f/da1/f12f [569583,15535] 0 2024-05-17T21:53:15.393 INFO:tasks.workunit.client.1.smithi138.stdout:6/929: dwrite d7/d69/db5/f11f [0,4194304] 0 2024-05-17T21:53:15.394 INFO:tasks.workunit.client.1.smithi138.stdout:7/885: mknod d0/d109/c121 0 2024-05-17T21:53:15.394 INFO:tasks.workunit.client.1.smithi138.stdout:9/792: chown d6/d7/d16/d29/d38/fa3 1707 1 2024-05-17T21:53:15.398 INFO:tasks.workunit.client.1.smithi138.stdout:5/837: mknod d1/d2c/d63/db5/d1b/ddb/c111 0 2024-05-17T21:53:15.399 INFO:tasks.workunit.client.1.smithi138.stdout:8/786: chown d0/dc/d3b/d4d/fbd 2 1 2024-05-17T21:53:15.399 INFO:tasks.workunit.client.1.smithi138.stdout:2/915: write d3/d81/d96/fb4 [162024,8700] 0 2024-05-17T21:53:15.400 INFO:tasks.workunit.client.1.smithi138.stdout:8/787: stat d0/d11/l54 0 2024-05-17T21:53:15.400 INFO:tasks.workunit.client.1.smithi138.stdout:8/788: dread - d0/dc/d8d/fad zero size 2024-05-17T21:53:15.403 INFO:tasks.workunit.client.1.smithi138.stdout:0/817: mknod d3/d3d/d48/c110 0 2024-05-17T21:53:15.406 INFO:tasks.workunit.client.1.smithi138.stdout:7/886: unlink d0/d18/d64/l67 0 2024-05-17T21:53:15.408 INFO:tasks.workunit.client.1.smithi138.stdout:2/916: rmdir d3/d21/d2f/d3e/d55/d99 39 2024-05-17T21:53:15.414 INFO:tasks.workunit.client.1.smithi138.stdout:7/887: dwrite d0/d18/d40/d41/d50/f53 [4194304,4194304] 0 2024-05-17T21:53:15.414 INFO:tasks.workunit.client.1.smithi138.stdout:8/789: creat d0/dc/d3b/d4d/d7d/d97/de9/feb x:0 0 0 2024-05-17T21:53:15.415 INFO:tasks.workunit.client.1.smithi138.stdout:7/888: fdatasync d0/d18/d64/d7c/da2/dc5/f10b 0 2024-05-17T21:53:15.421 INFO:tasks.workunit.client.1.smithi138.stdout:4/985: rename d0/d3/d17/d21/d78/d9f/d11e/f12d to d0/d2f/f150 0 2024-05-17T21:53:15.422 INFO:tasks.workunit.client.1.smithi138.stdout:5/838: link d1/d2c/d63/db5/d1b/f1d d1/dd1/dca/f112 0 2024-05-17T21:53:15.423 INFO:tasks.workunit.client.1.smithi138.stdout:5/839: write d1/d2c/d63/db5/d1b/f54 [6533072,23446] 0 2024-05-17T21:53:15.425 INFO:tasks.workunit.client.1.smithi138.stdout:8/790: symlink d0/d71/d92/lec 0 2024-05-17T21:53:15.425 INFO:tasks.workunit.client.1.smithi138.stdout:7/889: creat d0/d9d/f122 x:0 0 0 2024-05-17T21:53:15.426 INFO:tasks.workunit.client.1.smithi138.stdout:7/890: readlink d0/d4c/dec/l102 0 2024-05-17T21:53:15.427 INFO:tasks.workunit.client.1.smithi138.stdout:1/928: sync 2024-05-17T21:53:15.427 INFO:tasks.workunit.client.1.smithi138.stdout:1/929: stat d4/dd/f119 0 2024-05-17T21:53:15.429 INFO:tasks.workunit.client.1.smithi138.stdout:6/930: getdents d7/d69 0 2024-05-17T21:53:15.431 INFO:tasks.workunit.client.1.smithi138.stdout:6/931: write d7/d69/db5/dc8/d43/d5f/fd5 [5739127,34365] 0 2024-05-17T21:53:15.435 INFO:tasks.workunit.client.1.smithi138.stdout:9/793: rename d6/d7/d16/d29/d38/c47 to d6/d7/d16/d29/d48/d6f/cfb 0 2024-05-17T21:53:15.436 INFO:tasks.workunit.client.1.smithi138.stdout:2/917: write d3/da0/fbb [154272,99531] 0 2024-05-17T21:53:15.437 INFO:tasks.workunit.client.1.smithi138.stdout:4/986: chown d0/d2f/d49/d60/d1f/d7e/l99 209059 1 2024-05-17T21:53:15.438 INFO:tasks.workunit.client.1.smithi138.stdout:2/918: write d3/d21/d2f/d3e/d63/f80 [5198472,436] 0 2024-05-17T21:53:15.439 INFO:tasks.workunit.client.1.smithi138.stdout:4/987: chown d0/d3/d17/d21/d78/f9a 323014 1 2024-05-17T21:53:15.440 INFO:tasks.workunit.client.1.smithi138.stdout:4/988: readlink d0/d2f/d49/d60/d1f/d7e/l99 0 2024-05-17T21:53:15.452 INFO:tasks.workunit.client.1.smithi138.stdout:9/794: mkdir d6/dbc/dfc 0 2024-05-17T21:53:15.453 INFO:tasks.workunit.client.1.smithi138.stdout:9/795: write d6/d7/fc [3452429,60348] 0 2024-05-17T21:53:15.457 INFO:tasks.workunit.client.1.smithi138.stdout:1/930: symlink d4/df/d3a/dc4/d11d/l12c 0 2024-05-17T21:53:15.458 INFO:tasks.workunit.client.1.smithi138.stdout:8/791: write d0/dc/d1a/d33/d4a/fae [128354,103051] 0 2024-05-17T21:53:15.461 INFO:tasks.workunit.client.1.smithi138.stdout:0/818: rename d3/d3d/f7f to d3/d10/d38/d70/d80/d94/f111 0 2024-05-17T21:53:15.462 INFO:tasks.workunit.client.1.smithi138.stdout:5/840: dwrite d1/d2c/d63/db5/d1b/db7/fda [0,4194304] 0 2024-05-17T21:53:15.470 INFO:tasks.workunit.client.1.smithi138.stdout:9/796: mkdir d6/d7/d16/d29/d48/d6e/d74/d75/dfd 0 2024-05-17T21:53:15.470 INFO:tasks.workunit.client.1.smithi138.stdout:4/989: mkdir d0/d122/d151 0 2024-05-17T21:53:15.471 INFO:tasks.workunit.client.1.smithi138.stdout:7/891: dwrite d0/d39/f9b [0,4194304] 0 2024-05-17T21:53:15.473 INFO:tasks.workunit.client.1.smithi138.stdout:1/931: rmdir d4/dd/d3b/da7 39 2024-05-17T21:53:15.474 INFO:tasks.workunit.client.1.smithi138.stdout:6/932: dwrite d7/d69/db5/dc8/d43/d3f/d79/f6c [0,4194304] 0 2024-05-17T21:53:15.474 INFO:tasks.workunit.client.1.smithi138.stdout:6/933: chown f1 190584854 1 2024-05-17T21:53:15.483 INFO:tasks.workunit.client.1.smithi138.stdout:4/990: stat d0/c2c 0 2024-05-17T21:53:15.484 INFO:tasks.workunit.client.1.smithi138.stdout:4/991: chown d0/d3/d1d/d38/d9c/c137 42785757 1 2024-05-17T21:53:15.484 INFO:tasks.workunit.client.1.smithi138.stdout:9/797: symlink d6/d7/d16/d29/d38/lfe 0 2024-05-17T21:53:15.486 INFO:tasks.workunit.client.1.smithi138.stdout:1/932: rename d4/df/d3a/dc4/lf2 to d4/df/d3a/d3c/d63/d6d/de0/l12d 0 2024-05-17T21:53:15.487 INFO:tasks.workunit.client.1.smithi138.stdout:1/933: chown d4/df/l1a 49635 1 2024-05-17T21:53:15.488 INFO:tasks.workunit.client.1.smithi138.stdout:5/841: creat d1/d8a/df0/f113 x:0 0 0 2024-05-17T21:53:15.488 INFO:tasks.workunit.client.1.smithi138.stdout:2/919: getdents d3/d21/d2f/d3e/d58/d77/da4 0 2024-05-17T21:53:15.491 INFO:tasks.workunit.client.1.smithi138.stdout:7/892: symlink d0/d18/d40/l123 0 2024-05-17T21:53:15.493 INFO:tasks.workunit.client.1.smithi138.stdout:6/934: rename d7/d8/c2a to d7/d69/db5/dc8/d43/d5f/d6a/d77/de5/c13a 0 2024-05-17T21:53:15.495 INFO:tasks.workunit.client.1.smithi138.stdout:1/934: creat d4/d2e/d36/f12e x:0 0 0 2024-05-17T21:53:15.497 INFO:tasks.workunit.client.1.smithi138.stdout:5/842: unlink d1/d2c/d63/db5/d1b/dc5/l40 0 2024-05-17T21:53:15.501 INFO:tasks.workunit.client.1.smithi138.stdout:9/798: link d6/d7/d16/d29/d48/d6e/d74/d75/d76/f7f d6/d7/d16/d29/d48/d3b/fff 0 2024-05-17T21:53:15.502 INFO:tasks.workunit.client.1.smithi138.stdout:7/893: symlink d0/d4c/dec/l124 0 2024-05-17T21:53:15.503 INFO:tasks.workunit.client.1.smithi138.stdout:9/799: chown d6/d7/d16/d29/d38/d40/d44/d5c 52 1 2024-05-17T21:53:15.504 INFO:tasks.workunit.client.1.smithi138.stdout:8/792: dwrite d0/dc/d3b/d4d/d7d/d97/f9a [0,4194304] 0 2024-05-17T21:53:15.508 INFO:tasks.workunit.client.1.smithi138.stdout:8/793: write d0/dc/d3b/d4d/d7d/fe6 [258483,69808] 0 2024-05-17T21:53:15.513 INFO:tasks.workunit.client.1.smithi138.stdout:7/894: dwrite d0/d109/f114 [0,4194304] 0 2024-05-17T21:53:15.515 INFO:tasks.workunit.client.1.smithi138.stdout:4/992: write d0/d2f/d49/d60/d1f/d32/feb [1244054,27516] 0 2024-05-17T21:53:15.521 INFO:tasks.workunit.client.1.smithi138.stdout:6/935: rename d7/d69/db5/dc8/d43/d5f/d6a/l83 to d7/d69/db5/dc8/d43/l13b 0 2024-05-17T21:53:15.525 INFO:tasks.workunit.client.1.smithi138.stdout:4/993: creat d0/d1b/d42/d45/f152 x:0 0 0 2024-05-17T21:53:15.527 INFO:tasks.workunit.client.1.smithi138.stdout:7/895: truncate d0/d18/d40/d41/d51/d66/d9e/fd5 423537 0 2024-05-17T21:53:15.527 INFO:tasks.workunit.client.1.smithi138.stdout:8/794: write d0/fb [7794825,32073] 0 2024-05-17T21:53:15.530 INFO:tasks.workunit.client.1.smithi138.stdout:8/795: fdatasync d0/dc/d8d/fd4 0 2024-05-17T21:53:15.532 INFO:tasks.workunit.client.1.smithi138.stdout:5/843: dwrite d1/d53/d88/db9/d39/d4e/d58/fa4 [0,4194304] 0 2024-05-17T21:53:15.533 INFO:tasks.workunit.client.1.smithi138.stdout:5/844: chown d1/d2c/d63/db5/d1b/ddb/c100 65597457 1 2024-05-17T21:53:15.534 INFO:tasks.workunit.client.1.smithi138.stdout:4/994: creat d0/d3/d17/f153 x:0 0 0 2024-05-17T21:53:15.537 INFO:tasks.workunit.client.1.smithi138.stdout:7/896: creat d0/d18/d40/d9a/f125 x:0 0 0 2024-05-17T21:53:15.538 INFO:tasks.workunit.client.1.smithi138.stdout:8/796: mkdir d0/dc/d3b/d4d/ded 0 2024-05-17T21:53:15.541 INFO:tasks.workunit.client.1.smithi138.stdout:7/897: fdatasync d0/d18/d64/d7c/da2/dc5/dcf/f106 0 2024-05-17T21:53:15.544 INFO:tasks.workunit.client.1.smithi138.stdout:5/845: dwrite d1/d53/d88/db9/d39/d4e/d58/d5b/f9e [0,4194304] 0 2024-05-17T21:53:15.544 INFO:tasks.workunit.client.1.smithi138.stdout:1/935: rename d4/dd/d3b to d4/db/d12f 0 2024-05-17T21:53:15.546 INFO:tasks.workunit.client.1.smithi138.stdout:4/995: chown d0/d2f/d49/d60/d1f/d32/d71/d95/l14d 28030 1 2024-05-17T21:53:15.549 INFO:tasks.workunit.client.1.smithi138.stdout:8/797: creat d0/dc/d3b/da4/fee x:0 0 0 2024-05-17T21:53:15.551 INFO:tasks.workunit.client.1.smithi138.stdout:6/936: write d7/d69/f80 [458608,71586] 0 2024-05-17T21:53:15.553 INFO:tasks.workunit.client.1.smithi138.stdout:1/936: rmdir d4/dd/db4 39 2024-05-17T21:53:15.557 INFO:tasks.workunit.client.1.smithi138.stdout:4/996: mknod d0/d3/db/da0/c154 0 2024-05-17T21:53:15.560 INFO:tasks.workunit.client.1.smithi138.stdout:8/798: mknod d0/d2f/cef 0 2024-05-17T21:53:15.562 INFO:tasks.workunit.client.1.smithi138.stdout:1/937: rmdir d4/df/d3a/d3c/d63 39 2024-05-17T21:53:15.564 INFO:tasks.workunit.client.1.smithi138.stdout:4/997: unlink d0/d3/l1e 0 2024-05-17T21:53:15.566 INFO:tasks.workunit.client.1.smithi138.stdout:9/800: rename d6/d7/f2c to d6/d7/d16/d29/f100 0 2024-05-17T21:53:15.572 INFO:tasks.workunit.client.1.smithi138.stdout:5/846: getdents d1/d53/d88/db9/d39/d4e 0 2024-05-17T21:53:15.575 INFO:tasks.workunit.client.1.smithi138.stdout:8/799: creat d0/ff0 x:0 0 0 2024-05-17T21:53:15.576 INFO:tasks.workunit.client.1.smithi138.stdout:6/937: write d7/d69/db5/dc8/d43/d3f/db6/fe8 [379497,14172] 0 2024-05-17T21:53:15.576 INFO:tasks.workunit.client.1.smithi138.stdout:7/898: dread d0/f4a [0,4194304] 0 2024-05-17T21:53:15.578 INFO:tasks.workunit.client.1.smithi138.stdout:5/847: dread - d1/d53/db1/fdd zero size 2024-05-17T21:53:15.580 INFO:tasks.workunit.client.1.smithi138.stdout:9/801: write d6/d7/f26 [1102846,8017] 0 2024-05-17T21:53:15.585 INFO:tasks.workunit.client.1.smithi138.stdout:1/938: dread d4/fc [0,4194304] 0 2024-05-17T21:53:15.586 INFO:tasks.workunit.client.1.smithi138.stdout:1/939: readlink d4/leb 0 2024-05-17T21:53:15.590 INFO:tasks.workunit.client.1.smithi138.stdout:9/802: dwrite d6/d7/d16/d29/d6c/d9c/dc1/d7e/db7/fc4 [0,4194304] 0 2024-05-17T21:53:15.591 INFO:tasks.workunit.client.1.smithi138.stdout:8/800: dwrite d0/d63/fbb [0,4194304] 0 2024-05-17T21:53:15.595 INFO:tasks.workunit.client.1.smithi138.stdout:6/938: unlink d7/d69/db5/dc8/d43/d5f/d94/lac 0 2024-05-17T21:53:15.596 INFO:tasks.workunit.client.1.smithi138.stdout:7/899: symlink d0/d18/d40/d41/d50/d100/l126 0 2024-05-17T21:53:15.597 INFO:tasks.workunit.client.1.smithi138.stdout:1/940: rmdir d4/db/d60/dcc 39 2024-05-17T21:53:15.598 INFO:tasks.workunit.client.1.smithi138.stdout:7/900: chown d0 6 1 2024-05-17T21:53:15.599 INFO:tasks.workunit.client.1.smithi138.stdout:5/848: mkdir d1/d53/d88/db9/d39/d4e/d58/d114 0 2024-05-17T21:53:15.602 INFO:tasks.workunit.client.1.smithi138.stdout:8/801: mknod d0/dc/d1a/d51/cf1 0 2024-05-17T21:53:15.604 INFO:tasks.workunit.client.1.smithi138.stdout:6/939: symlink d7/d8/de9/d11c/l13c 0 2024-05-17T21:53:15.604 INFO:tasks.workunit.client.1.smithi138.stdout:1/941: dread d4/df/d35/df3/f10c [0,4194304] 0 2024-05-17T21:53:15.604 INFO:tasks.workunit.client.1.smithi138.stdout:9/803: dwrite d6/d7/d16/d29/d38/d40/d44/d5c/dda/ff5 [0,4194304] 0 2024-05-17T21:53:15.607 INFO:tasks.workunit.client.1.smithi138.stdout:9/804: dread - d6/d7/d16/d29/d48/d6e/d74/d75/d76/d8e/dba/fd1 zero size 2024-05-17T21:53:15.610 INFO:tasks.workunit.client.1.smithi138.stdout:9/805: dread - d6/d7/d16/d29/d48/d6e/d74/d75/d76/d8e/dba/fc9 zero size 2024-05-17T21:53:15.612 INFO:tasks.workunit.client.1.smithi138.stdout:9/806: dread - d6/d7/d16/d29/d38/d40/d44/d5c/dc8/ff7 zero size 2024-05-17T21:53:15.613 INFO:tasks.workunit.client.1.smithi138.stdout:8/802: fsync d0/f17 0 2024-05-17T21:53:15.615 INFO:tasks.workunit.client.1.smithi138.stdout:6/940: rmdir d7/da0 39 2024-05-17T21:53:15.615 INFO:tasks.workunit.client.1.smithi138.stdout:8/803: truncate d0/dc/d3b/d4d/fda 808218 0 2024-05-17T21:53:15.617 INFO:tasks.workunit.client.1.smithi138.stdout:1/942: creat d4/db/d29/d5d/f130 x:0 0 0 2024-05-17T21:53:15.618 INFO:tasks.workunit.client.1.smithi138.stdout:1/943: write d4/db/f6a [6212890,67821] 0 2024-05-17T21:53:15.622 INFO:tasks.workunit.client.1.smithi138.stdout:6/941: dwrite d7/d69/db5/dc8/d43/d5f/d6a/d77/f85 [0,4194304] 0 2024-05-17T21:53:15.624 INFO:tasks.workunit.client.1.smithi138.stdout:8/804: chown d0/dc/fa7 0 1 2024-05-17T21:53:15.630 INFO:tasks.workunit.client.1.smithi138.stdout:1/944: fsync d4/db/d12f/d6c/dc1/fdf 0 2024-05-17T21:53:15.632 INFO:tasks.workunit.client.1.smithi138.stdout:9/807: creat d6/d7/d16/d29/d48/d6e/d74/d75/d76/d8e/dba/df2/f101 x:0 0 0 2024-05-17T21:53:15.632 INFO:tasks.workunit.client.1.smithi138.stdout:1/945: write d4/df/d3a/ff0 [688457,53924] 0 2024-05-17T21:53:15.636 INFO:tasks.workunit.client.1.smithi138.stdout:6/942: fsync d7/d69/db5/dc8/d43/d3f/f4c 0 2024-05-17T21:53:15.645 INFO:tasks.workunit.client.1.smithi138.stdout:5/849: write d1/d60/d2f/d5e/f79 [1312246,67662] 0 2024-05-17T21:53:15.647 INFO:tasks.workunit.client.1.smithi138.stdout:5/850: dread - d1/d2c/d63/db5/d19/fff zero size 2024-05-17T21:53:15.649 INFO:tasks.workunit.client.1.smithi138.stdout:7/901: dwrite d0/d18/dbf/dd1/fe6 [0,4194304] 0 2024-05-17T21:53:15.651 INFO:tasks.workunit.client.1.smithi138.stdout:8/805: dread d0/dc/d1a/d51/d82/f8e [4194304,4194304] 0 2024-05-17T21:53:15.652 INFO:tasks.workunit.client.1.smithi138.stdout:7/902: dread - d0/d18/d40/d41/d51/d66/d9e/fde zero size 2024-05-17T21:53:15.653 INFO:tasks.workunit.client.1.smithi138.stdout:1/946: truncate d4/df/d3a/d3c/d63/d6d/f10f 700344 0 2024-05-17T21:53:15.660 INFO:tasks.workunit.client.1.smithi138.stdout:5/851: read d1/d60/f87 [168749,6867] 0 2024-05-17T21:53:15.661 INFO:tasks.workunit.client.1.smithi138.stdout:5/852: dread d1/d60/f87 [0,4194304] 0 2024-05-17T21:53:15.664 INFO:tasks.workunit.client.1.smithi138.stdout:8/806: fsync d0/d71/fa5 0 2024-05-17T21:53:15.666 INFO:tasks.workunit.client.1.smithi138.stdout:9/808: write d6/d7/d16/d29/d48/d3b/fff [3125468,94186] 0 2024-05-17T21:53:15.668 INFO:tasks.workunit.client.1.smithi138.stdout:1/947: unlink d4/db/d29/d89/ffd 0 2024-05-17T21:53:15.672 INFO:tasks.workunit.client.1.smithi138.stdout:5/853: chown d1/d2c/d63/db5/c2d 267 1 2024-05-17T21:53:15.673 INFO:tasks.workunit.client.1.smithi138.stdout:6/943: write d7/da0/fd2 [2674072,15088] 0 2024-05-17T21:53:15.678 INFO:tasks.workunit.client.1.smithi138.stdout:1/948: creat d4/dd/d64/f131 x:0 0 0 2024-05-17T21:53:15.678 INFO:tasks.workunit.client.1.smithi138.stdout:1/949: chown d4/db/f39 103173 1 2024-05-17T21:53:15.683 INFO:tasks.workunit.client.1.smithi138.stdout:1/950: dread d4/d2e/da0/daa/f115 [0,4194304] 0 2024-05-17T21:53:15.686 INFO:tasks.workunit.client.1.smithi138.stdout:6/944: stat d7/d69/db5/dc8/d43/d5f/da4/dcc 0 2024-05-17T21:53:15.687 INFO:tasks.workunit.client.1.smithi138.stdout:8/807: truncate d0/d11/f42 2459407 0 2024-05-17T21:53:15.689 INFO:tasks.workunit.client.1.smithi138.stdout:7/903: link d0/d18/d9c/c113 d0/d18/c127 0 2024-05-17T21:53:15.690 INFO:tasks.workunit.client.1.smithi138.stdout:9/809: dread d6/d7/d16/d29/d38/d8b/fc6 [0,4194304] 0 2024-05-17T21:53:15.690 INFO:tasks.workunit.client.1.smithi138.stdout:1/951: fdatasync f2 0 2024-05-17T21:53:15.695 INFO:tasks.workunit.client.1.smithi138.stdout:6/945: truncate d7/d69/db5/dc8/d43/d3f/d79/f112 464583 0 2024-05-17T21:53:15.696 INFO:tasks.workunit.client.1.smithi138.stdout:8/808: rename d0/d31/fe2 to d0/dc/d3b/da0/ff2 0 2024-05-17T21:53:15.698 INFO:tasks.workunit.client.1.smithi138.stdout:7/904: mkdir d0/d18/d64/d7c/d128 0 2024-05-17T21:53:15.701 INFO:tasks.workunit.client.1.smithi138.stdout:1/952: mkdir d4/dd/d64/d132 0 2024-05-17T21:53:15.703 INFO:tasks.workunit.client.1.smithi138.stdout:9/810: read d6/d7/d16/d29/d38/f81 [732051,57205] 0 2024-05-17T21:53:15.705 INFO:tasks.workunit.client.1.smithi138.stdout:9/811: chown d6/d7/d16/d29/d38/d40/d44/d5c/l69 195770576 1 2024-05-17T21:53:15.707 INFO:tasks.workunit.client.1.smithi138.stdout:1/953: creat d4/db/d29/d5d/f133 x:0 0 0 2024-05-17T21:53:15.711 INFO:tasks.workunit.client.1.smithi138.stdout:9/812: readlink d6/d7/l12 0 2024-05-17T21:53:15.712 INFO:tasks.workunit.client.1.smithi138.stdout:1/954: rmdir d4/db/d12f 39 2024-05-17T21:53:15.713 INFO:tasks.workunit.client.1.smithi138.stdout:9/813: truncate d6/d7/d16/d29/d38/d3c/ff9 154781 0 2024-05-17T21:53:15.714 INFO:tasks.workunit.client.1.smithi138.stdout:8/809: rmdir d0/dc/d3b/d4d/ded 0 2024-05-17T21:53:15.715 INFO:tasks.workunit.client.1.smithi138.stdout:7/905: truncate d0/fd6 1321675 0 2024-05-17T21:53:15.717 INFO:tasks.workunit.client.1.smithi138.stdout:9/814: unlink d6/d7/d16/d29/d38/d3c/l6a 0 2024-05-17T21:53:15.718 INFO:tasks.workunit.client.1.smithi138.stdout:8/810: creat d0/dc/d1a/d44/ff3 x:0 0 0 2024-05-17T21:53:15.719 INFO:tasks.workunit.client.1.smithi138.stdout:8/811: chown d0 25 1 2024-05-17T21:53:15.719 INFO:tasks.workunit.client.1.smithi138.stdout:9/815: chown d6/d7/d16/d29/d38/d40/d44/ca1 638 1 2024-05-17T21:53:15.721 INFO:tasks.workunit.client.1.smithi138.stdout:7/906: mknod d0/d18/d64/d7c/c129 0 2024-05-17T21:53:15.722 INFO:tasks.workunit.client.1.smithi138.stdout:7/907: write d0/d18/d40/d41/d51/d81/ffb [937587,93720] 0 2024-05-17T21:53:15.727 INFO:tasks.workunit.client.1.smithi138.stdout:1/955: creat d4/db/d12f/da7/d111/f134 x:0 0 0 2024-05-17T21:53:15.728 INFO:tasks.workunit.client.1.smithi138.stdout:1/956: chown d4/db/d60/ca1 145523784 1 2024-05-17T21:53:15.730 INFO:tasks.workunit.client.1.smithi138.stdout:9/816: mknod d6/d7/d16/d29/c102 0 2024-05-17T21:53:15.734 INFO:tasks.workunit.client.1.smithi138.stdout:1/957: dwrite d4/dd/d64/f131 [0,4194304] 0 2024-05-17T21:53:15.736 INFO:tasks.workunit.client.1.smithi138.stdout:8/812: rename d0/dc/f22 to d0/dc/d3b/da4/dbc/de0/ff4 0 2024-05-17T21:53:15.740 INFO:tasks.workunit.client.1.smithi138.stdout:9/817: symlink d6/d7/d16/d29/d6c/d9c/l103 0 2024-05-17T21:53:15.740 INFO:tasks.workunit.client.1.smithi138.stdout:9/818: chown d6/c8d 8164 1 2024-05-17T21:53:15.741 INFO:tasks.workunit.client.1.smithi138.stdout:9/819: stat d6/d7/d16/d29/d6c/d9c 0 2024-05-17T21:53:15.747 INFO:tasks.workunit.client.1.smithi138.stdout:7/908: creat d0/d18/d40/d41/d51/d66/f12a x:0 0 0 2024-05-17T21:53:15.748 INFO:tasks.workunit.client.1.smithi138.stdout:9/820: unlink d6/d7/d16/d29/d6c/d9c/ld6 0 2024-05-17T21:53:15.750 INFO:tasks.workunit.client.1.smithi138.stdout:7/909: fsync d0/d18/f45 0 2024-05-17T21:53:15.756 INFO:tasks.workunit.client.1.smithi138.stdout:7/910: readlink d0/la6 0 2024-05-17T21:53:15.760 INFO:tasks.workunit.client.1.smithi138.stdout:1/958: dwrite d4/db/d29/fa4 [0,4194304] 0 2024-05-17T21:53:15.761 INFO:tasks.workunit.client.1.smithi138.stdout:9/821: write d6/d7/d16/d29/f37 [4643212,62756] 0 2024-05-17T21:53:15.767 INFO:tasks.workunit.client.1.smithi138.stdout:9/822: symlink d6/d7/dc0/l104 0 2024-05-17T21:53:15.769 INFO:tasks.workunit.client.1.smithi138.stdout:1/959: link d4/df/d5f/ffb d4/db/d12f/d6c/dcf/d123/f135 0 2024-05-17T21:53:15.772 INFO:tasks.workunit.client.1.smithi138.stdout:9/823: rename d6/d7/d16/d29/d48/d6e/d74/d75/d76/ca5 to d6/d7/d16/d29/d38/df4/c105 0 2024-05-17T21:53:15.774 INFO:tasks.workunit.client.1.smithi138.stdout:1/960: creat d4/db/d12f/d9f/f136 x:0 0 0 2024-05-17T21:53:15.776 INFO:tasks.workunit.client.1.smithi138.stdout:8/813: write d0/dc/fa7 [340205,46841] 0 2024-05-17T21:53:15.778 INFO:tasks.workunit.client.1.smithi138.stdout:1/961: rename d4/f3f to d4/df/d3a/dc4/f137 0 2024-05-17T21:53:15.780 INFO:tasks.workunit.client.1.smithi138.stdout:7/911: dread d0/f19 [0,4194304] 0 2024-05-17T21:53:15.786 INFO:tasks.workunit.client.1.smithi138.stdout:8/814: unlink d0/dc/d3b/d4d/le3 0 2024-05-17T21:53:15.790 INFO:tasks.workunit.client.1.smithi138.stdout:1/962: rename d4/df/l23 to d4/db/d12f/d9f/da9/dd9/l138 0 2024-05-17T21:53:15.791 INFO:tasks.workunit.client.1.smithi138.stdout:7/912: mkdir d0/d18/d40/d12b 0 2024-05-17T21:53:15.792 INFO:tasks.workunit.client.1.smithi138.stdout:8/815: dwrite d0/dc/d3b/d4d/d7d/d97/f9a [0,4194304] 0 2024-05-17T21:53:15.792 INFO:tasks.workunit.client.1.smithi138.stdout:7/913: write d0/d18/d9c/dd4/ff1 [990037,76833] 0 2024-05-17T21:53:15.794 INFO:tasks.workunit.client.1.smithi138.stdout:9/824: link d6/d7/d16/d29/cb1 d6/d7/d16/d29/d6c/d9c/dc1/d7e/db7/c106 0 2024-05-17T21:53:15.796 INFO:tasks.workunit.client.1.smithi138.stdout:1/963: symlink d4/df/d5f/l139 0 2024-05-17T21:53:15.799 INFO:tasks.workunit.client.1.smithi138.stdout:7/914: unlink d0/d9d/f119 0 2024-05-17T21:53:15.801 INFO:tasks.workunit.client.1.smithi138.stdout:1/964: symlink d4/db/d12f/d9f/da9/dd9/l13a 0 2024-05-17T21:53:15.816 INFO:tasks.workunit.client.1.smithi138.stdout:8/816: dread d0/d11/f69 [4194304,4194304] 0 2024-05-17T21:53:15.818 INFO:tasks.workunit.client.1.smithi138.stdout:9/825: dread d6/d7/fc [0,4194304] 0 2024-05-17T21:53:15.819 INFO:tasks.workunit.client.1.smithi138.stdout:8/817: creat d0/d71/d92/ff5 x:0 0 0 2024-05-17T21:53:15.824 INFO:tasks.workunit.client.1.smithi138.stdout:8/818: write d0/dc/d1a/d51/f80 [379598,81607] 0 2024-05-17T21:53:15.834 INFO:tasks.workunit.client.1.smithi138.stdout:7/915: write d0/d9d/fab [1648273,54944] 0 2024-05-17T21:53:15.836 INFO:tasks.workunit.client.1.smithi138.stdout:8/819: dwrite d0/dc/d1a/d51/f80 [0,4194304] 0 2024-05-17T21:53:15.836 INFO:tasks.workunit.client.1.smithi138.stdout:7/916: readlink d0/d18/d40/d41/d50/l7d 0 2024-05-17T21:53:15.851 INFO:tasks.workunit.client.1.smithi138.stdout:9/826: dwrite d6/d7/d16/d29/d48/d6e/d74/d75/d76/d8e/dba/fd1 [0,4194304] 0 2024-05-17T21:53:15.853 INFO:tasks.workunit.client.1.smithi138.stdout:9/827: chown d6/d7/d16/d29/d6c/d9c/dc1/c4a 6769737 1 2024-05-17T21:53:15.881 INFO:tasks.workunit.client.1.smithi138.stdout:7/917: truncate d0/d39/f48 5169834 0 2024-05-17T21:53:15.883 INFO:tasks.workunit.client.1.smithi138.stdout:8/820: mkdir d0/dc/d3b/d4d/d7d/d97/de9/df6 0 2024-05-17T21:53:15.886 INFO:tasks.workunit.client.1.smithi138.stdout:9/828: symlink d6/d7/d16/d29/d48/d6e/d74/d75/d76/l107 0 2024-05-17T21:53:15.890 INFO:tasks.workunit.client.1.smithi138.stdout:7/918: rename d0/d18/d64 to d0/d18/d40/d41/d51/d81/d12c 0 2024-05-17T21:53:15.896 INFO:tasks.workunit.client.1.smithi138.stdout:9/829: rename d6/d7/d16/d29/d38/d3c to d6/d7/d16/d29/d48/d6e/d74/d75/d76/de8/d108 0 2024-05-17T21:53:15.897 INFO:tasks.workunit.client.1.smithi138.stdout:8/821: dread d0/dc/d3b/fba [0,4194304] 0 2024-05-17T21:53:15.901 INFO:tasks.workunit.client.1.smithi138.stdout:9/830: fsync d6/d7/d16/d29/d48/f98 0 2024-05-17T21:53:15.905 INFO:tasks.workunit.client.1.smithi138.stdout:7/919: dread d0/d18/d40/d41/d51/d81/d12c/d7c/fad [0,4194304] 0 2024-05-17T21:53:15.906 INFO:tasks.workunit.client.1.smithi138.stdout:8/822: dwrite d0/dc/d1a/d78/fcd [0,4194304] 0 2024-05-17T21:53:15.910 INFO:tasks.workunit.client.1.smithi138.stdout:9/831: mkdir d6/d7/d16/d29/d38/d109 0 2024-05-17T21:53:15.914 INFO:tasks.workunit.client.1.smithi138.stdout:8/823: symlink d0/d11/d3e/lf7 0 2024-05-17T21:53:15.917 INFO:tasks.workunit.client.1.smithi138.stdout:9/832: rename d6/d7/dc0/l104 to d6/d7/d16/d29/d38/d40/d44/d5c/dc8/ded/df3/l10a 0 2024-05-17T21:53:15.919 INFO:tasks.workunit.client.1.smithi138.stdout:9/833: readlink d6/d7/d16/d29/d48/d6e/d74/d75/d76/de8/d108/l89 0 2024-05-17T21:53:15.920 INFO:tasks.workunit.client.1.smithi138.stdout:9/834: chown d6/d7/d16/d29/d38/c55 16 1 2024-05-17T21:53:15.922 INFO:tasks.workunit.client.1.smithi138.stdout:8/824: creat d0/dc/d1a/d33/ff8 x:0 0 0 2024-05-17T21:53:15.924 INFO:tasks.workunit.client.1.smithi138.stdout:9/835: creat d6/d7/d16/d29/d48/d6e/d74/d75/d76/de8/f10b x:0 0 0 2024-05-17T21:53:15.927 INFO:tasks.workunit.client.1.smithi138.stdout:9/836: fsync d6/d7/d16/f8a 0 2024-05-17T21:53:15.932 INFO:tasks.workunit.client.1.smithi138.stdout:8/825: creat d0/dc/d1a/d33/d4a/ff9 x:0 0 0 2024-05-17T21:53:15.933 INFO:tasks.workunit.client.1.smithi138.stdout:9/837: symlink d6/de4/l10c 0 2024-05-17T21:53:15.936 INFO:tasks.workunit.client.1.smithi138.stdout:9/838: unlink d6/d7/d16/fef 0 2024-05-17T21:53:15.937 INFO:tasks.workunit.client.1.smithi138.stdout:9/839: dread - d6/d7/d16/d29/d48/d6e/d74/d75/fbb zero size 2024-05-17T21:53:15.938 INFO:tasks.workunit.client.1.smithi138.stdout:8/826: creat d0/dc/d3b/da4/dbc/dc3/ffa x:0 0 0 2024-05-17T21:53:15.941 INFO:tasks.workunit.client.1.smithi138.stdout:8/827: creat d0/dc/d1a/d44/ffb x:0 0 0 2024-05-17T21:53:15.944 INFO:tasks.workunit.client.1.smithi138.stdout:8/828: creat d0/dc/d1a/d51/d82/ffc x:0 0 0 2024-05-17T21:53:15.948 INFO:tasks.workunit.client.1.smithi138.stdout:8/829: dread d0/d63/fbb [0,4194304] 0 2024-05-17T21:53:15.949 INFO:tasks.workunit.client.1.smithi138.stdout:8/830: write d0/d2f/f6e [7305157,105994] 0 2024-05-17T21:53:15.952 INFO:tasks.workunit.client.1.smithi138.stdout:8/831: creat d0/dc/d3b/da4/dbc/de0/ffd x:0 0 0 2024-05-17T21:53:15.979 INFO:tasks.workunit.client.1.smithi138.stdout:7/920: write d0/d4c/f6d [1496753,59808] 0 2024-05-17T21:53:15.982 INFO:tasks.workunit.client.1.smithi138.stdout:7/921: symlink d0/d18/d9c/l12d 0 2024-05-17T21:53:15.983 INFO:tasks.workunit.client.1.smithi138.stdout:7/922: write d0/d18/d40/d9a/f125 [996386,35706] 0 2024-05-17T21:53:15.985 INFO:tasks.workunit.client.1.smithi138.stdout:7/923: creat d0/d18/d40/d5f/dbe/f12e x:0 0 0 2024-05-17T21:53:15.987 INFO:tasks.workunit.client.1.smithi138.stdout:8/832: getdents d0/dc/d1a/d33/d4a 0 2024-05-17T21:53:15.991 INFO:tasks.workunit.client.1.smithi138.stdout:8/833: symlink d0/dc/d3b/d87/lfe 0 2024-05-17T21:53:15.991 INFO:tasks.workunit.client.1.smithi138.stdout:7/924: dwrite d0/d18/d40/d5f/f8f [4194304,4194304] 0 2024-05-17T21:53:15.994 INFO:tasks.workunit.client.1.smithi138.stdout:9/840: write d6/d7/d16/d29/d38/d40/d44/f87 [1775217,25462] 0 2024-05-17T21:53:15.997 INFO:tasks.workunit.client.1.smithi138.stdout:8/834: dread d0/dc/d3b/fba [0,4194304] 0 2024-05-17T21:53:16.002 INFO:tasks.workunit.client.1.smithi138.stdout:9/841: link d6/d7/fc d6/d7/d16/d29/d48/d3b/f10d 0 2024-05-17T21:53:16.004 INFO:tasks.workunit.client.1.smithi138.stdout:8/835: dwrite d0/dc/d1a/d51/f80 [0,4194304] 0 2024-05-17T21:53:16.009 INFO:tasks.workunit.client.1.smithi138.stdout:9/842: mkdir d6/dbc/d10e 0 2024-05-17T21:53:16.012 INFO:tasks.workunit.client.1.smithi138.stdout:8/836: dwrite d0/dc/d1a/d51/f80 [0,4194304] 0 2024-05-17T21:53:16.018 INFO:tasks.workunit.client.1.smithi138.stdout:8/837: unlink d0/dc/d1a/d51/cf1 0 2024-05-17T21:53:16.020 INFO:tasks.workunit.client.1.smithi138.stdout:9/843: dwrite d6/d7/d16/d29/d38/d40/d44/d5c/ff0 [0,4194304] 0 2024-05-17T21:53:16.025 INFO:tasks.workunit.client.1.smithi138.stdout:7/925: write d0/d18/d40/d41/d50/f59 [6938054,62621] 0 2024-05-17T21:53:16.030 INFO:tasks.workunit.client.1.smithi138.stdout:8/838: dwrite d0/dc/d1a/d78/fcd [0,4194304] 0 2024-05-17T21:53:16.042 INFO:tasks.workunit.client.1.smithi138.stdout:7/926: write d0/f4b [1709163,93484] 0 2024-05-17T21:53:16.043 INFO:tasks.workunit.client.1.smithi138.stdout:8/839: rmdir d0/dc/d3b/d4d/d7d/dd5 0 2024-05-17T21:53:16.044 INFO:tasks.workunit.client.1.smithi138.stdout:9/844: write d6/d7/d16/dc2/fe0 [914869,66601] 0 2024-05-17T21:53:16.046 INFO:tasks.workunit.client.1.smithi138.stdout:9/845: chown d6/d7/d16/d29/d6c/d9c/dc1/c4a 1465442 1 2024-05-17T21:53:16.047 INFO:tasks.workunit.client.1.smithi138.stdout:8/840: creat d0/d63/fff x:0 0 0 2024-05-17T21:53:16.048 INFO:tasks.workunit.client.1.smithi138.stdout:7/927: mknod d0/d18/d40/c12f 0 2024-05-17T21:53:16.048 INFO:tasks.workunit.client.1.smithi138.stdout:8/841: chown d0/dc/d1a/d51/d82 0 1 2024-05-17T21:53:16.050 INFO:tasks.workunit.client.1.smithi138.stdout:9/846: creat d6/d7/d16/d29/d48/d6e/d74/d75/d76/d8e/ddd/f10f x:0 0 0 2024-05-17T21:53:16.054 INFO:tasks.workunit.client.1.smithi138.stdout:9/847: write d6/d7/d16/d29/d38/d40/d44/d5c/dda/ff5 [1646805,92275] 0 2024-05-17T21:53:16.056 INFO:tasks.workunit.client.1.smithi138.stdout:7/928: dwrite d0/f24 [4194304,4194304] 0 2024-05-17T21:53:16.057 INFO:tasks.workunit.client.1.smithi138.stdout:9/848: fdatasync d6/d7/d16/d29/d48/d6e/d74/d75/d76/de8/f10b 0 2024-05-17T21:53:16.062 INFO:tasks.workunit.client.1.smithi138.stdout:7/929: creat d0/d18/d40/d41/d51/d81/f130 x:0 0 0 2024-05-17T21:53:16.063 INFO:tasks.workunit.client.1.smithi138.stdout:7/930: truncate d0/ff8 500829 0 2024-05-17T21:53:16.063 INFO:tasks.workunit.client.1.smithi138.stdout:8/842: rmdir d0/dc/d3b/d4d/d7d/d97/de9/df6 0 2024-05-17T21:53:16.065 INFO:tasks.workunit.client.1.smithi138.stdout:8/843: write d0/dc/d3b/fba [1539865,49231] 0 2024-05-17T21:53:16.066 INFO:tasks.workunit.client.1.smithi138.stdout:7/931: symlink d0/d18/d40/d41/d51/l131 0 2024-05-17T21:53:16.072 INFO:tasks.workunit.client.1.smithi138.stdout:9/849: dread d6/d7/d16/f8a [0,4194304] 0 2024-05-17T21:53:16.158 INFO:tasks.workunit.client.1.smithi138.stdout:8/844: mknod d0/d71/c100 0 2024-05-17T21:53:16.161 INFO:tasks.workunit.client.1.smithi138.stdout:8/845: creat d0/dc/d3b/d4d/d7d/d97/de9/f101 x:0 0 0 2024-05-17T21:53:16.162 INFO:tasks.workunit.client.1.smithi138.stdout:9/850: getdents d6/d7/d16/d29 0 2024-05-17T21:53:16.163 INFO:tasks.workunit.client.1.smithi138.stdout:8/846: truncate d0/dc/d1a/d33/d4a/ff9 666711 0 2024-05-17T21:53:16.166 INFO:tasks.workunit.client.1.smithi138.stdout:8/847: mkdir d0/dc/d3b/da4/dbc/de0/d102 0 2024-05-17T21:53:16.169 INFO:tasks.workunit.client.1.smithi138.stdout:8/848: mknod d0/dc/d3b/d4d/d7d/d97/dc1/c103 0 2024-05-17T21:53:16.170 INFO:tasks.workunit.client.1.smithi138.stdout:8/849: chown d0/dc/d8d/cb5 5792 1 2024-05-17T21:53:16.173 INFO:tasks.workunit.client.1.smithi138.stdout:8/850: symlink d0/dc/d1a/d51/d82/ddb/l104 0 2024-05-17T21:53:16.175 INFO:tasks.workunit.client.1.smithi138.stdout:8/851: symlink d0/dc/d3b/l105 0 2024-05-17T21:53:16.186 INFO:tasks.workunit.client.1.smithi138.stdout:8/852: dread d0/dc/d1a/d33/d4a/d68/fac [0,4194304] 0 2024-05-17T21:53:16.189 INFO:tasks.workunit.client.1.smithi138.stdout:8/853: dread d0/dc/d3b/d4d/d7d/fe6 [0,4194304] 0 2024-05-17T21:53:16.194 INFO:tasks.workunit.client.1.smithi138.stdout:8/854: dwrite d0/dc/d1a/d51/f80 [0,4194304] 0 2024-05-17T21:53:16.197 INFO:tasks.workunit.client.1.smithi138.stdout:8/855: creat d0/dc/d1a/d44/f106 x:0 0 0 2024-05-17T21:53:16.205 INFO:tasks.workunit.client.1.smithi138.stdout:8/856: dwrite d0/dc/d1a/d78/fd1 [0,4194304] 0 2024-05-17T21:53:16.207 INFO:tasks.workunit.client.1.smithi138.stdout:8/857: mkdir d0/d2f/d107 0 2024-05-17T21:53:16.240 INFO:tasks.workunit.client.1.smithi138.stdout:8/858: truncate d0/dc/d3b/da4/dbc/de0/ff4 3466763 0 2024-05-17T21:53:16.382 INFO:tasks.workunit.client.1.smithi138.stdout:0/819: sync 2024-05-17T21:53:16.387 INFO:tasks.workunit.client.1.smithi138.stdout:0/820: write d3/d10/d6d/d9f/da8/daa/fcb [458210,46793] 0 2024-05-17T21:53:16.390 INFO:tasks.workunit.client.1.smithi138.stdout:0/821: dread d3/d10/f17 [0,4194304] 0 2024-05-17T21:53:16.392 INFO:tasks.workunit.client.1.smithi138.stdout:0/822: mknod d3/d3d/d51/dba/c112 0 2024-05-17T21:53:16.395 INFO:tasks.workunit.client.1.smithi138.stdout:0/823: getdents d3/d10/d38/d3e/d59/dad 0 2024-05-17T21:53:16.398 INFO:tasks.workunit.client.1.smithi138.stdout:0/824: creat d3/d10/d6d/db6/db9/f113 x:0 0 0 2024-05-17T21:53:16.400 INFO:tasks.workunit.client.1.smithi138.stdout:0/825: rmdir d3/d10/d38/d70 39 2024-05-17T21:53:16.403 INFO:tasks.workunit.client.1.smithi138.stdout:0/826: creat d3/d3d/f114 x:0 0 0 2024-05-17T21:53:16.406 INFO:tasks.workunit.client.1.smithi138.stdout:0/827: truncate d3/fc 1675737 0 2024-05-17T21:53:16.411 INFO:tasks.workunit.client.1.smithi138.stdout:0/828: fdatasync d3/d10/d38/d70/d80/d94/fcd 0 2024-05-17T21:53:16.416 INFO:tasks.workunit.client.1.smithi138.stdout:0/829: mkdir d3/d10/d6d/d9f/d115 0 2024-05-17T21:53:16.420 INFO:tasks.workunit.client.1.smithi138.stdout:0/830: creat d3/d10/f116 x:0 0 0 2024-05-17T21:53:16.422 INFO:tasks.workunit.client.1.smithi138.stdout:0/831: creat d3/d10/d38/d3e/d59/d9b/f117 x:0 0 0 2024-05-17T21:53:16.423 INFO:tasks.workunit.client.1.smithi138.stdout:0/832: rmdir d3/d10/d39/d97 39 2024-05-17T21:53:16.427 INFO:tasks.workunit.client.1.smithi138.stdout:0/833: creat d3/d10/d38/d3e/d59/f118 x:0 0 0 2024-05-17T21:53:16.429 INFO:tasks.workunit.client.1.smithi138.stdout:0/834: creat d3/d10/d6d/d9f/da8/d101/f119 x:0 0 0 2024-05-17T21:53:16.431 INFO:tasks.workunit.client.1.smithi138.stdout:0/835: fdatasync d3/d10/d38/d70/d80/d94/fcd 0 2024-05-17T21:53:16.438 INFO:tasks.workunit.client.1.smithi138.stdout:0/836: creat d3/d10/d38/d3e/f11a x:0 0 0 2024-05-17T21:53:16.440 INFO:tasks.workunit.client.1.smithi138.stdout:0/837: chown d3/d10/c90 1810 1 2024-05-17T21:53:16.440 INFO:tasks.workunit.client.1.smithi138.stdout:0/838: readlink d3/d10/d38/d70/d80/l9a 0 2024-05-17T21:53:16.442 INFO:tasks.workunit.client.1.smithi138.stdout:0/839: mknod d3/d10/d6d/d9f/da8/daa/c11b 0 2024-05-17T21:53:16.443 INFO:tasks.workunit.client.1.smithi138.stdout:0/840: creat d3/d10/d6d/d9f/f11c x:0 0 0 2024-05-17T21:53:16.456 INFO:tasks.workunit.client.1.smithi138.stdout:0/841: dwrite d3/d10/d38/d3e/d59/dad/fbb [0,4194304] 0 2024-05-17T21:53:16.460 INFO:tasks.workunit.client.1.smithi138.stdout:0/842: mkdir d3/d3d/da2/d11d 0 2024-05-17T21:53:16.462 INFO:tasks.workunit.client.1.smithi138.stdout:0/843: creat d3/d10/d6d/d9f/f11e x:0 0 0 2024-05-17T21:53:16.482 INFO:tasks.workunit.client.1.smithi138.stdout:0/844: dread d3/d3d/f6a [0,4194304] 0 2024-05-17T21:53:16.482 INFO:tasks.workunit.client.1.smithi138.stdout:0/845: fsync d3/d10/d6d/d9f/fe6 0 2024-05-17T21:53:16.483 INFO:tasks.workunit.client.1.smithi138.stdout:0/846: chown d3/d10/d38/d3e/d59/dad 61 1 2024-05-17T21:53:16.494 INFO:tasks.workunit.client.1.smithi138.stdout:0/847: dread d3/d10/d6d/f75 [0,4194304] 0 2024-05-17T21:53:16.496 INFO:tasks.workunit.client.1.smithi138.stdout:0/848: read d3/d10/f7d [286911,30903] 0 2024-05-17T21:53:16.501 INFO:tasks.workunit.client.1.smithi138.stdout:0/849: dwrite d3/d10/d38/d3e/d59/f9d [0,4194304] 0 2024-05-17T21:53:16.524 INFO:tasks.workunit.client.1.smithi138.stdout:0/850: dread d3/d10/f32 [0,4194304] 0 2024-05-17T21:53:16.529 INFO:tasks.workunit.client.1.smithi138.stdout:0/851: dwrite f0 [0,4194304] 0 2024-05-17T21:53:16.535 INFO:tasks.workunit.client.1.smithi138.stdout:0/852: link d3/d10/d39/lb8 d3/d10/d6d/l11f 0 2024-05-17T21:53:16.536 INFO:tasks.workunit.client.1.smithi138.stdout:0/853: chown d3/d10/d6d/db6/db9/f105 29521 1 2024-05-17T21:53:16.537 INFO:tasks.workunit.client.1.smithi138.stdout:0/854: chown d3/d10/d38/c86 100076 1 2024-05-17T21:53:16.540 INFO:tasks.workunit.client.1.smithi138.stdout:0/855: unlink d3/d3d/d51/f54 0 2024-05-17T21:53:16.541 INFO:tasks.workunit.client.1.smithi138.stdout:2/920: sync 2024-05-17T21:53:16.542 INFO:tasks.workunit.client.1.smithi138.stdout:0/856: mknod d3/d10/d38/d70/c120 0 2024-05-17T21:53:16.560 INFO:tasks.workunit.client.1.smithi138.stdout:0/857: dread d3/d3d/da2/ffb [0,4194304] 0 2024-05-17T21:53:16.561 INFO:tasks.workunit.client.1.smithi138.stdout:2/921: truncate d3/f9 77117 0 2024-05-17T21:53:16.568 INFO:tasks.workunit.client.1.smithi138.stdout:2/922: dwrite d3/d21/d2f/dd4/f105 [0,4194304] 0 2024-05-17T21:53:16.572 INFO:tasks.workunit.client.1.smithi138.stdout:2/923: chown d3/d21/d2f/d3e/d58/d77/da4/df2/dd0/l11b 4928018 1 2024-05-17T21:53:16.578 INFO:tasks.workunit.client.1.smithi138.stdout:2/924: dread - d3/d81/d96/da8/f106 zero size 2024-05-17T21:53:16.592 INFO:tasks.workunit.client.1.smithi138.stdout:2/925: dwrite d3/f20 [0,4194304] 0 2024-05-17T21:53:16.601 INFO:tasks.workunit.client.1.smithi138.stdout:2/926: rmdir d3/d21/d2f/d3e/d55/d99/d84/d9f/d116 39 2024-05-17T21:53:16.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:16 smithi107 ceph-mon[28434]: pgmap v46: 65 pgs: 65 active+clean; 3.8 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 107 MiB/s rd, 235 MiB/s wr, 577 op/s 2024-05-17T21:53:16.603 INFO:tasks.workunit.client.1.smithi138.stdout:2/927: creat d3/d21/d2f/f12d x:0 0 0 2024-05-17T21:53:16.606 INFO:tasks.workunit.client.1.smithi138.stdout:2/928: creat d3/d21/d2f/d3e/d63/da3/f12e x:0 0 0 2024-05-17T21:53:16.608 INFO:tasks.workunit.client.1.smithi138.stdout:2/929: unlink d3/da0/df6/l11e 0 2024-05-17T21:53:16.620 INFO:tasks.workunit.client.1.smithi138.stdout:2/930: dwrite d3/d21/d2f/d3e/d58/d77/da4/df2/fdc [0,4194304] 0 2024-05-17T21:53:16.623 INFO:tasks.workunit.client.1.smithi138.stdout:2/931: fsync d3/d21/d2f/d3e/d58/f71 0 2024-05-17T21:53:16.625 INFO:tasks.workunit.client.1.smithi138.stdout:2/932: creat d3/d21/d2f/f12f x:0 0 0 2024-05-17T21:53:16.629 INFO:tasks.workunit.client.1.smithi138.stdout:2/933: mkdir d3/da0/d130 0 2024-05-17T21:53:16.631 INFO:tasks.workunit.client.1.smithi138.stdout:2/934: mkdir d3/d21/d2f/d3e/d58/d77/da4/df2/dd0/d131 0 2024-05-17T21:53:16.636 INFO:tasks.workunit.client.1.smithi138.stdout:2/935: rename d3/f129 to d3/d21/d2f/d3e/d63/da3/f132 0 2024-05-17T21:53:16.638 INFO:tasks.workunit.client.1.smithi138.stdout:2/936: truncate d3/d21/d2f/d3e/d58/d77/da4/dc5/fbc 1557905 0 2024-05-17T21:53:16.648 INFO:tasks.workunit.client.1.smithi138.stdout:2/937: truncate d3/fe 211565 0 2024-05-17T21:53:16.652 INFO:tasks.workunit.client.1.smithi138.stdout:2/938: unlink d3/d21/d2f/d3e/d58/d77/da4/df2/dd0/fb0 0 2024-05-17T21:53:16.653 INFO:tasks.workunit.client.1.smithi138.stdout:2/939: write d3/da0/df6/f126 [878900,9284] 0 2024-05-17T21:53:16.737 INFO:tasks.workunit.client.1.smithi138.stdout:5/854: sync 2024-05-17T21:53:16.737 INFO:tasks.workunit.client.1.smithi138.stdout:4/998: sync 2024-05-17T21:53:16.738 INFO:tasks.workunit.client.1.smithi138.stdout:5/855: stat d1/d2c/d63/db5/d1b/dc5/f7c 0 2024-05-17T21:53:16.739 INFO:tasks.workunit.client.1.smithi138.stdout:4/999: rmdir d0/d1b/d2b/dca 39 2024-05-17T21:53:16.747 INFO:tasks.workunit.client.1.smithi138.stdout:5/856: dwrite d1/d2c/f35 [0,4194304] 0 2024-05-17T21:53:16.752 INFO:tasks.workunit.client.1.smithi138.stdout:5/857: rename d1/d53/d88/de5/ded/fe1 to d1/d2c/d63/db5/d19/f115 0 2024-05-17T21:53:16.756 INFO:tasks.workunit.client.1.smithi138.stdout:5/858: rename d1/d53/d88/db9/d39/d4e/d58/fa4 to d1/d53/d88/f116 0 2024-05-17T21:53:16.758 INFO:tasks.workunit.client.1.smithi138.stdout:5/859: mknod d1/dd1/c117 0 2024-05-17T21:53:16.759 INFO:tasks.workunit.client.1.smithi138.stdout:5/860: rmdir d1/d53/d88/db9 39 2024-05-17T21:53:16.763 INFO:tasks.workunit.client.1.smithi138.stdout:5/861: rename d1/d60/d2f/c56 to d1/dd1/dca/dd0/c118 0 2024-05-17T21:53:16.767 INFO:tasks.workunit.client.1.smithi138.stdout:5/862: truncate d1/d46/d95/fae 743817 0 2024-05-17T21:53:16.773 INFO:tasks.workunit.client.1.smithi138.stdout:5/863: dwrite d1/d2c/d63/db5/d19/f3a [0,4194304] 0 2024-05-17T21:53:16.798 INFO:tasks.workunit.client.1.smithi138.stdout:5/864: dread d1/f84 [0,4194304] 0 2024-05-17T21:53:16.801 INFO:tasks.workunit.client.1.smithi138.stdout:5/865: getdents d1/d53/db1 0 2024-05-17T21:53:16.801 INFO:tasks.workunit.client.1.smithi138.stdout:5/866: readlink d1/d53/d88/lce 0 2024-05-17T21:53:16.802 INFO:tasks.workunit.client.1.smithi138.stdout:5/867: write d1/d2c/d63/db5/d19/f3a [2789714,113859] 0 2024-05-17T21:53:16.806 INFO:tasks.workunit.client.1.smithi138.stdout:5/868: dread d1/f84 [0,4194304] 0 2024-05-17T21:53:16.808 INFO:tasks.workunit.client.1.smithi138.stdout:5/869: mkdir d1/d60/d2f/d119 0 2024-05-17T21:53:16.813 INFO:tasks.workunit.client.1.smithi138.stdout:5/870: write d1/d60/d2f/d5e/de4/fbc [4633405,102916] 0 2024-05-17T21:53:16.822 INFO:tasks.workunit.client.1.smithi138.stdout:5/871: dwrite d1/dd1/f8f [0,4194304] 0 2024-05-17T21:53:16.823 INFO:tasks.workunit.client.1.smithi138.stdout:5/872: chown d1/d53/d88/lce 97 1 2024-05-17T21:53:16.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:16 smithi138 ceph-mon[35948]: pgmap v46: 65 pgs: 65 active+clean; 3.8 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 107 MiB/s rd, 235 MiB/s wr, 577 op/s 2024-05-17T21:53:16.830 INFO:tasks.workunit.client.1.smithi138.stdout:5/873: dwrite d1/d2c/d63/db5/d1b/f54 [0,4194304] 0 2024-05-17T21:53:16.859 INFO:tasks.workunit.client.1.smithi138.stdout:5/874: dread d1/d2c/d63/db5/d1b/dc5/f3b [0,4194304] 0 2024-05-17T21:53:16.872 INFO:tasks.workunit.client.1.smithi138.stdout:6/946: sync 2024-05-17T21:53:16.887 INFO:tasks.workunit.client.1.smithi138.stdout:6/947: mkdir d7/d69/db5/dc8/d43/d5f/da4/dcc/d13d 0 2024-05-17T21:53:16.893 INFO:tasks.workunit.client.1.smithi138.stdout:5/875: getdents d1 0 2024-05-17T21:53:16.893 INFO:tasks.workunit.client.1.smithi138.stdout:5/876: chown d1/d2c/d63/db5/d73/fe9 0 1 2024-05-17T21:53:16.894 INFO:tasks.workunit.client.1.smithi138.stdout:6/948: creat d7/d69/db5/dc8/f13e x:0 0 0 2024-05-17T21:53:16.900 INFO:tasks.workunit.client.1.smithi138.stdout:6/949: dread d7/da0/dbe/fd6 [0,4194304] 0 2024-05-17T21:53:16.904 INFO:tasks.workunit.client.1.smithi138.stdout:6/950: truncate d7/d69/db5/dc8/d43/d3f/db6/fd8 390641 0 2024-05-17T21:53:16.912 INFO:tasks.workunit.client.1.smithi138.stdout:6/951: creat d7/d69/db5/dc8/d43/d63/f13f x:0 0 0 2024-05-17T21:53:16.912 INFO:tasks.workunit.client.1.smithi138.stdout:6/952: truncate d7/da0/dbe/f114 975935 0 2024-05-17T21:53:16.914 INFO:tasks.workunit.client.1.smithi138.stdout:6/953: creat d7/d8/d10c/f140 x:0 0 0 2024-05-17T21:53:16.917 INFO:tasks.workunit.client.1.smithi138.stdout:6/954: creat d7/d69/db5/dc8/d43/d5f/d94/dc9/f141 x:0 0 0 2024-05-17T21:53:16.920 INFO:tasks.workunit.client.1.smithi138.stdout:6/955: mkdir d7/d69/db5/dc8/d43/d5f/da4/d142 0 2024-05-17T21:53:16.922 INFO:tasks.workunit.client.1.smithi138.stdout:6/956: truncate d7/d69/db5/dc8/d43/d5f/da4/db0/f12e 636364 0 2024-05-17T21:53:16.924 INFO:tasks.workunit.client.1.smithi138.stdout:6/957: read d7/d69/db5/dc8/d43/d5f/d6a/d9b/faf [306891,41682] 0 2024-05-17T21:53:16.927 INFO:tasks.workunit.client.1.smithi138.stdout:6/958: mkdir d7/d69/db5/dc8/d43/d5f/da4/db0/dc1/d143 0 2024-05-17T21:53:16.930 INFO:tasks.workunit.client.1.smithi138.stdout:6/959: mkdir d7/d69/db5/dc8/d43/d5f/d6a/d9b/d10d/d144 0 2024-05-17T21:53:16.932 INFO:tasks.workunit.client.1.smithi138.stdout:6/960: mknod d7/d69/db5/c145 0 2024-05-17T21:53:16.945 INFO:tasks.workunit.client.1.smithi138.stdout:5/877: write d1/d60/f5c [3762108,47748] 0 2024-05-17T21:53:16.958 INFO:tasks.workunit.client.1.smithi138.stdout:6/961: write d7/d8/f50 [2096516,40048] 0 2024-05-17T21:53:16.963 INFO:tasks.workunit.client.1.smithi138.stdout:1/965: sync 2024-05-17T21:53:16.963 INFO:tasks.workunit.client.1.smithi138.stdout:9/851: sync 2024-05-17T21:53:16.963 INFO:tasks.workunit.client.1.smithi138.stdout:7/932: sync 2024-05-17T21:53:16.963 INFO:tasks.workunit.client.1.smithi138.stdout:8/859: sync 2024-05-17T21:53:16.963 INFO:tasks.workunit.client.1.smithi138.stdout:0/858: sync 2024-05-17T21:53:16.963 INFO:tasks.workunit.client.1.smithi138.stdout:2/940: sync 2024-05-17T21:53:16.963 INFO:tasks.workunit.client.1.smithi138.stdout:5/878: dwrite d1/d2c/d63/db5/d1b/dc5/d36/f7a [0,4194304] 0 2024-05-17T21:53:16.966 INFO:tasks.workunit.client.1.smithi138.stdout:0/859: chown d3/d10/d38/d3e/d59/dad/fbb 0 1 2024-05-17T21:53:16.967 INFO:tasks.workunit.client.1.smithi138.stdout:8/860: dread - d0/dc/d3b/da4/dbc/de0/ffd zero size 2024-05-17T21:53:16.969 INFO:tasks.workunit.client.1.smithi138.stdout:6/962: truncate d7/d69/db5/dc8/d43/d3f/d79/ffe 1474506 0 2024-05-17T21:53:16.969 INFO:tasks.workunit.client.1.smithi138.stdout:8/861: write d0/dc/d1a/fdc [573547,9466] 0 2024-05-17T21:53:16.975 INFO:tasks.workunit.client.1.smithi138.stdout:7/933: symlink d0/d18/d40/d41/d51/d81/d12c/d7c/da2/dc5/l132 0 2024-05-17T21:53:16.976 INFO:tasks.workunit.client.1.smithi138.stdout:0/860: dread d3/d3d/d48/fe4 [0,4194304] 0 2024-05-17T21:53:16.977 INFO:tasks.workunit.client.1.smithi138.stdout:1/966: symlink d4/df/d3a/d3c/l13b 0 2024-05-17T21:53:16.978 INFO:tasks.workunit.client.1.smithi138.stdout:5/879: creat d1/d2c/d63/db5/d1b/d109/f11a x:0 0 0 2024-05-17T21:53:16.980 INFO:tasks.workunit.client.1.smithi138.stdout:9/852: truncate d6/d7/d16/fe9 1204777 0 2024-05-17T21:53:16.982 INFO:tasks.workunit.client.1.smithi138.stdout:6/963: unlink d7/d69/db5/dc8/d43/d5f/d6a/f89 0 2024-05-17T21:53:16.984 INFO:tasks.workunit.client.1.smithi138.stdout:2/941: dwrite d3/d21/d2f/d3e/d55/d99/d112/ff9 [0,4194304] 0 2024-05-17T21:53:16.985 INFO:tasks.workunit.client.1.smithi138.stdout:2/942: readlink d3/d81/d96/da8/lcf 0 2024-05-17T21:53:16.986 INFO:tasks.workunit.client.1.smithi138.stdout:0/861: write d3/fd [431809,37080] 0 2024-05-17T21:53:17.002 INFO:tasks.workunit.client.1.smithi138.stdout:8/862: creat d0/dc/f108 x:0 0 0 2024-05-17T21:53:17.004 INFO:tasks.workunit.client.1.smithi138.stdout:7/934: link d0/d18/d40/d41/d51/d81/d12c/d7c/da2/dc5/dcf/l10c d0/d18/d40/d41/d50/l133 0 2024-05-17T21:53:17.006 INFO:tasks.workunit.client.1.smithi138.stdout:0/862: mknod d3/d10/d6d/d81/d104/c121 0 2024-05-17T21:53:17.008 INFO:tasks.workunit.client.1.smithi138.stdout:5/880: creat d1/d53/d88/db9/d39/d4e/d58/d114/f11b x:0 0 0 2024-05-17T21:53:17.011 INFO:tasks.workunit.client.1.smithi138.stdout:6/964: link d7/d8/d78/f7f d7/d69/db5/dc8/d60/f146 0 2024-05-17T21:53:17.017 INFO:tasks.workunit.client.1.smithi138.stdout:5/881: unlink d1/d2c/d63/db5/fe 0 2024-05-17T21:53:17.018 INFO:tasks.workunit.client.1.smithi138.stdout:9/853: dwrite d6/d7/d16/d29/d38/d40/d44/d5c/f7a [0,4194304] 0 2024-05-17T21:53:17.019 INFO:tasks.workunit.client.1.smithi138.stdout:5/882: dread - d1/d2c/d63/db5/d1b/d109/f11a zero size 2024-05-17T21:53:17.019 INFO:tasks.workunit.client.1.smithi138.stdout:8/863: mknod d0/dc/d1a/d51/c109 0 2024-05-17T21:53:17.021 INFO:tasks.workunit.client.1.smithi138.stdout:1/967: write d4/d93/faf [353386,25332] 0 2024-05-17T21:53:17.023 INFO:tasks.workunit.client.1.smithi138.stdout:8/864: write d0/dc/d1a/d44/ff3 [103915,67406] 0 2024-05-17T21:53:17.025 INFO:tasks.workunit.client.1.smithi138.stdout:6/965: rename d7/da0/fbf to d7/d69/db5/dc8/d34/da6/dfa/d101/f147 0 2024-05-17T21:53:17.026 INFO:tasks.workunit.client.1.smithi138.stdout:2/943: link d3/d21/d2f/d3e/d58/d77/da4/dc5/f8b d3/d21/d2f/d3e/d63/da3/f133 0 2024-05-17T21:53:17.029 INFO:tasks.workunit.client.1.smithi138.stdout:5/883: creat d1/d2c/d63/db5/df5/f11c x:0 0 0 2024-05-17T21:53:17.032 INFO:tasks.workunit.client.1.smithi138.stdout:0/863: link d3/d10/d6d/d9f/lcf d3/d10/d6d/d81/d104/l122 0 2024-05-17T21:53:17.035 INFO:tasks.workunit.client.1.smithi138.stdout:0/864: truncate d3/d10/d6d/d9f/da8/d101/f119 726860 0 2024-05-17T21:53:17.035 INFO:tasks.workunit.client.1.smithi138.stdout:1/968: rename d4/df/d35/l45 to d4/df/d3a/d3c/d63/l13c 0 2024-05-17T21:53:17.037 INFO:tasks.workunit.client.1.smithi138.stdout:6/966: mkdir d7/d69/db5/dc8/d60/d148 0 2024-05-17T21:53:17.041 INFO:tasks.workunit.client.1.smithi138.stdout:9/854: unlink d6/d7/d16/fe9 0 2024-05-17T21:53:17.048 INFO:tasks.workunit.client.1.smithi138.stdout:0/865: truncate d3/d10/d38/fe8 434061 0 2024-05-17T21:53:17.049 INFO:tasks.workunit.client.1.smithi138.stdout:2/944: symlink d3/d81/l134 0 2024-05-17T21:53:17.052 INFO:tasks.workunit.client.1.smithi138.stdout:9/855: readlink d6/d7/d16/d29/d48/d6e/d74/d75/la8 0 2024-05-17T21:53:17.052 INFO:tasks.workunit.client.1.smithi138.stdout:9/856: stat d6/d7/d16/d29/d38/d40/d44/d5c 0 2024-05-17T21:53:17.053 INFO:tasks.workunit.client.1.smithi138.stdout:8/865: link d0/f2 d0/dc/d3b/da0/f10a 0 2024-05-17T21:53:17.060 INFO:tasks.workunit.client.1.smithi138.stdout:5/884: creat d1/d53/d88/db9/d39/d4e/d58/f11d x:0 0 0 2024-05-17T21:53:17.061 INFO:tasks.workunit.client.1.smithi138.stdout:5/885: write d1/d2c/d63/db5/df5/f11c [369135,111669] 0 2024-05-17T21:53:17.063 INFO:tasks.workunit.client.1.smithi138.stdout:1/969: rename d4/d2e/f106 to d4/dd/f13d 0 2024-05-17T21:53:17.065 INFO:tasks.workunit.client.1.smithi138.stdout:9/857: mknod d6/d7/d16/d29/d38/d40/d44/d5c/dda/c110 0 2024-05-17T21:53:17.066 INFO:tasks.workunit.client.1.smithi138.stdout:9/858: fsync d6/fd9 0 2024-05-17T21:53:17.073 INFO:tasks.workunit.client.1.smithi138.stdout:6/967: write d7/d69/db5/dc8/d43/f64 [2241649,88158] 0 2024-05-17T21:53:17.077 INFO:tasks.workunit.client.1.smithi138.stdout:2/945: dwrite d3/d21/d2f/d3e/d55/d99/fa9 [0,4194304] 0 2024-05-17T21:53:17.078 INFO:tasks.workunit.client.1.smithi138.stdout:2/946: write d3/d21/d2f/d3e/d55/d99/d112/fe6 [4464779,88624] 0 2024-05-17T21:53:17.079 INFO:tasks.workunit.client.1.smithi138.stdout:5/886: mknod d1/d53/d88/db9/d39/d4e/d58/d5b/dee/c11e 0 2024-05-17T21:53:17.080 INFO:tasks.workunit.client.1.smithi138.stdout:2/947: fdatasync d3/d21/d2f/d3e/d58/d77/da4/df2/dd0/f128 0 2024-05-17T21:53:17.081 INFO:tasks.workunit.client.1.smithi138.stdout:1/970: write d4/df/d3a/d3c/f7c [4650659,108921] 0 2024-05-17T21:53:17.082 INFO:tasks.workunit.client.1.smithi138.stdout:0/866: write d3/d3d/d48/f5f [1721533,64902] 0 2024-05-17T21:53:17.086 INFO:tasks.workunit.client.1.smithi138.stdout:8/866: mkdir d0/d63/dea/d10b 0 2024-05-17T21:53:17.087 INFO:tasks.workunit.client.1.smithi138.stdout:9/859: truncate d6/f33 1161307 0 2024-05-17T21:53:17.090 INFO:tasks.workunit.client.1.smithi138.stdout:9/860: write d6/d7/f99 [3155980,112579] 0 2024-05-17T21:53:17.091 INFO:tasks.workunit.client.1.smithi138.stdout:6/968: creat d7/d69/db5/dc8/d60/f149 x:0 0 0 2024-05-17T21:53:17.094 INFO:tasks.workunit.client.1.smithi138.stdout:5/887: truncate d1/d46/f78 43750 0 2024-05-17T21:53:17.095 INFO:tasks.workunit.client.1.smithi138.stdout:1/971: dwrite d4/db/d12f/da7/ff4 [0,4194304] 0 2024-05-17T21:53:17.097 INFO:tasks.workunit.client.1.smithi138.stdout:2/948: dwrite d3/d21/d2f/d3e/d55/f111 [0,4194304] 0 2024-05-17T21:53:17.103 INFO:tasks.workunit.client.1.smithi138.stdout:0/867: symlink d3/d10/d38/d3e/d59/d9b/dc3/l123 0 2024-05-17T21:53:17.109 INFO:tasks.workunit.client.1.smithi138.stdout:8/867: dread d0/dc/fc6 [0,4194304] 0 2024-05-17T21:53:17.109 INFO:tasks.workunit.client.1.smithi138.stdout:5/888: unlink d1/d53/d96/da0/c104 0 2024-05-17T21:53:17.110 INFO:tasks.workunit.client.1.smithi138.stdout:2/949: stat d3/fe 0 2024-05-17T21:53:17.114 INFO:tasks.workunit.client.1.smithi138.stdout:1/972: dwrite d4/df/d3a/ff0 [0,4194304] 0 2024-05-17T21:53:17.117 INFO:tasks.workunit.client.1.smithi138.stdout:9/861: mkdir d6/d7/d16/d29/d48/d6e/d74/d75/d76/d8e/ddd/d111 0 2024-05-17T21:53:17.121 INFO:tasks.workunit.client.1.smithi138.stdout:8/868: fdatasync d0/d63/f9c 0 2024-05-17T21:53:17.122 INFO:tasks.workunit.client.1.smithi138.stdout:6/969: dwrite d7/d69/db5/dc8/d43/d5f/d6a/d131/f139 [0,4194304] 0 2024-05-17T21:53:17.128 INFO:tasks.workunit.client.1.smithi138.stdout:2/950: dwrite d3/d21/d2f/d3e/d63/fcd [4194304,4194304] 0 2024-05-17T21:53:17.130 INFO:tasks.workunit.client.1.smithi138.stdout:5/889: mknod d1/d2c/d63/db5/d1b/dc5/d36/d68/c11f 0 2024-05-17T21:53:17.133 INFO:tasks.workunit.client.1.smithi138.stdout:1/973: rmdir d4/db/d29/d5d/de5 39 2024-05-17T21:53:17.139 INFO:tasks.workunit.client.1.smithi138.stdout:8/869: mkdir d0/dc/d3b/d4d/d7d/d97/de9/d10c 0 2024-05-17T21:53:17.144 INFO:tasks.workunit.client.1.smithi138.stdout:1/974: mknod d4/df/d3a/c13e 0 2024-05-17T21:53:17.147 INFO:tasks.workunit.client.1.smithi138.stdout:0/868: rename d3/d3d/d51 to d3/d10/d38/d124 0 2024-05-17T21:53:17.150 INFO:tasks.workunit.client.1.smithi138.stdout:8/870: fdatasync d0/dc/d3b/da0/ff2 0 2024-05-17T21:53:17.152 INFO:tasks.workunit.client.1.smithi138.stdout:5/890: symlink d1/d53/d88/dc9/d106/l120 0 2024-05-17T21:53:17.154 INFO:tasks.workunit.client.1.smithi138.stdout:1/975: dread - d4/db/d12f/fed zero size 2024-05-17T21:53:17.156 INFO:tasks.workunit.client.1.smithi138.stdout:1/976: read d4/dd/f2f [7514609,115954] 0 2024-05-17T21:53:17.156 INFO:tasks.workunit.client.1.smithi138.stdout:9/862: rename d6/d7/d16/d29/d48/d3b/f10d to d6/d7/d16/d29/d48/d6e/d74/d75/d76/de8/f112 0 2024-05-17T21:53:17.161 INFO:tasks.workunit.client.1.smithi138.stdout:2/951: dwrite d3/d21/d2f/d3e/d63/f6e [0,4194304] 0 2024-05-17T21:53:17.163 INFO:tasks.workunit.client.1.smithi138.stdout:8/871: rmdir d0/d11/d3e 39 2024-05-17T21:53:17.165 INFO:tasks.workunit.client.1.smithi138.stdout:5/891: rmdir d1/d2c/d63/db5/d1b/db7 39 2024-05-17T21:53:17.169 INFO:tasks.workunit.client.1.smithi138.stdout:0/869: write d3/d10/d6d/fa5 [4228900,13186] 0 2024-05-17T21:53:17.181 INFO:tasks.workunit.client.1.smithi138.stdout:1/977: write d4/db/fe2 [369360,114171] 0 2024-05-17T21:53:17.186 INFO:tasks.workunit.client.1.smithi138.stdout:9/863: mknod d6/dbc/d10e/c113 0 2024-05-17T21:53:17.187 INFO:tasks.workunit.client.1.smithi138.stdout:6/970: link d7/d69/db5/dc8/d34/da6/cd4 d7/d69/db5/dc8/d43/d3f/d79/ddf/c14a 0 2024-05-17T21:53:17.189 INFO:tasks.workunit.client.1.smithi138.stdout:8/872: unlink d0/dc/d1a/f3f 0 2024-05-17T21:53:17.189 INFO:tasks.workunit.client.1.smithi138.stdout:8/873: chown d0/d2f/cef 5979502 1 2024-05-17T21:53:17.191 INFO:tasks.workunit.client.1.smithi138.stdout:1/978: unlink d4/db/d12f/f73 0 2024-05-17T21:53:17.192 INFO:tasks.workunit.client.1.smithi138.stdout:0/870: write d3/d10/d38/d3e/fb7 [2928110,19452] 0 2024-05-17T21:53:17.195 INFO:tasks.workunit.client.1.smithi138.stdout:9/864: rmdir d6/d7/d16/d29/d6c/d9c 39 2024-05-17T21:53:17.195 INFO:tasks.workunit.client.1.smithi138.stdout:5/892: dwrite d1/d53/ff6 [0,4194304] 0 2024-05-17T21:53:17.196 INFO:tasks.workunit.client.1.smithi138.stdout:9/865: write d6/ff1 [692382,112037] 0 2024-05-17T21:53:17.203 INFO:tasks.workunit.client.1.smithi138.stdout:8/874: mkdir d0/dc/d1a/d51/d82/d10d 0 2024-05-17T21:53:17.204 INFO:tasks.workunit.client.1.smithi138.stdout:6/971: write d7/d69/db5/dc8/d60/fda [337650,90348] 0 2024-05-17T21:53:17.204 INFO:tasks.workunit.client.1.smithi138.stdout:8/875: chown d0/c8 895404443 1 2024-05-17T21:53:17.205 INFO:tasks.workunit.client.1.smithi138.stdout:6/972: chown d7/da0/dbe/lec 52567 1 2024-05-17T21:53:17.206 INFO:tasks.workunit.client.1.smithi138.stdout:8/876: chown d0/dc/d3b/d4d/d7d/d97/de9/d10c 16 1 2024-05-17T21:53:17.208 INFO:tasks.workunit.client.1.smithi138.stdout:0/871: dread d3/d10/d6d/d81/fd3 [0,4194304] 0 2024-05-17T21:53:17.209 INFO:tasks.workunit.client.1.smithi138.stdout:9/866: symlink d6/d7/d16/d29/d48/d6e/d74/d75/d76/d8e/dba/l114 0 2024-05-17T21:53:17.217 INFO:tasks.workunit.client.1.smithi138.stdout:1/979: rename d4/dd/d64/l72 to d4/dd/db4/l13f 0 2024-05-17T21:53:17.218 INFO:tasks.workunit.client.1.smithi138.stdout:6/973: write d7/d8/f102 [5419,64703] 0 2024-05-17T21:53:17.223 INFO:tasks.workunit.client.1.smithi138.stdout:0/872: mkdir d3/d10/d38/d124/dba/d125 0 2024-05-17T21:53:17.226 INFO:tasks.workunit.client.1.smithi138.stdout:8/877: dwrite d0/dc/d3b/d4d/f8c [0,4194304] 0 2024-05-17T21:53:17.231 INFO:tasks.workunit.client.1.smithi138.stdout:1/980: rename d4/db/d12f/d9f/f136 to d4/dd/d64/f140 0 2024-05-17T21:53:17.234 INFO:tasks.workunit.client.1.smithi138.stdout:0/873: mkdir d3/d10/d38/d124/dba/d126 0 2024-05-17T21:53:17.235 INFO:tasks.workunit.client.1.smithi138.stdout:9/867: fdatasync d6/d7/d16/d29/d48/f34 0 2024-05-17T21:53:17.235 INFO:tasks.workunit.client.1.smithi138.stdout:5/893: link d1/d2c/d63/db5/d1b/ddb/ff3 d1/d2c/d63/db5/df5/f121 0 2024-05-17T21:53:17.236 INFO:tasks.workunit.client.1.smithi138.stdout:9/868: stat d6/dbc/dfc 0 2024-05-17T21:53:17.236 INFO:tasks.workunit.client.1.smithi138.stdout:0/874: chown d3/d10/d38/d3e/d59/fd0 0 1 2024-05-17T21:53:17.237 INFO:tasks.workunit.client.1.smithi138.stdout:9/869: fsync d6/dbc/f9e 0 2024-05-17T21:53:17.237 INFO:tasks.workunit.client.1.smithi138.stdout:0/875: stat d3/d10/d38/d3e/d59 0 2024-05-17T21:53:17.240 INFO:tasks.workunit.client.1.smithi138.stdout:1/981: creat d4/db/d12f/da7/d111/f141 x:0 0 0 2024-05-17T21:53:17.244 INFO:tasks.workunit.client.1.smithi138.stdout:5/894: creat d1/d53/d88/dc9/d106/f122 x:0 0 0 2024-05-17T21:53:17.250 INFO:tasks.workunit.client.1.smithi138.stdout:0/876: rename d3/d10/c33 to d3/d10/d6d/d9f/da8/daa/c127 0 2024-05-17T21:53:17.251 INFO:tasks.workunit.client.1.smithi138.stdout:1/982: unlink d4/db/d12f/d9f/da9/cc6 0 2024-05-17T21:53:17.252 INFO:tasks.workunit.client.1.smithi138.stdout:1/983: readlink d4/d2e/d36/l71 0 2024-05-17T21:53:17.253 INFO:tasks.workunit.client.1.smithi138.stdout:5/895: fdatasync d1/d2c/d63/db5/f72 0 2024-05-17T21:53:17.259 INFO:tasks.workunit.client.1.smithi138.stdout:1/984: mkdir d4/d2e/da0/d142 0 2024-05-17T21:53:17.260 INFO:tasks.workunit.client.1.smithi138.stdout:5/896: dwrite d1/d53/d96/fc6 [0,4194304] 0 2024-05-17T21:53:17.267 INFO:tasks.workunit.client.1.smithi138.stdout:5/897: creat d1/d2c/d63/db5/d73/f123 x:0 0 0 2024-05-17T21:53:17.268 INFO:tasks.workunit.client.1.smithi138.stdout:5/898: chown d1/d2c/d63/db5/d19/f3a 200065810 1 2024-05-17T21:53:17.270 INFO:tasks.workunit.client.1.smithi138.stdout:6/974: truncate d7/d8/f96 1258626 0 2024-05-17T21:53:17.271 INFO:tasks.workunit.client.1.smithi138.stdout:6/975: chown d7/d8/fb4 223 1 2024-05-17T21:53:17.306 INFO:tasks.workunit.client.1.smithi138.stdout:1/985: symlink d4/df/d3a/d10e/l143 0 2024-05-17T21:53:17.307 INFO:tasks.workunit.client.1.smithi138.stdout:6/976: write d7/d69/db5/dc8/d43/d3f/d79/f129 [3671260,58910] 0 2024-05-17T21:53:17.308 INFO:tasks.workunit.client.1.smithi138.stdout:8/878: truncate d0/d2f/f6e 1038119 0 2024-05-17T21:53:17.308 INFO:tasks.workunit.client.1.smithi138.stdout:1/986: chown d4/df/d35/df3/f101 85979 1 2024-05-17T21:53:17.312 INFO:tasks.workunit.client.1.smithi138.stdout:5/899: rmdir d1/d53/d88/db9/dcc 0 2024-05-17T21:53:17.316 INFO:tasks.workunit.client.1.smithi138.stdout:8/879: chown d0/c3 2 1 2024-05-17T21:53:17.318 INFO:tasks.workunit.client.1.smithi138.stdout:0/877: dwrite d3/d3d/ff7 [0,4194304] 0 2024-05-17T21:53:17.318 INFO:tasks.workunit.client.1.smithi138.stdout:5/900: mknod d1/dd1/c124 0 2024-05-17T21:53:17.320 INFO:tasks.workunit.client.1.smithi138.stdout:9/870: dwrite d6/d7/d16/d29/d6c/d9c/dc1/f53 [0,4194304] 0 2024-05-17T21:53:17.326 INFO:tasks.workunit.client.1.smithi138.stdout:8/880: creat d0/d63/f10e x:0 0 0 2024-05-17T21:53:17.328 INFO:tasks.workunit.client.1.smithi138.stdout:8/881: chown d0/dc/d1a/d33/d4a/l84 13 1 2024-05-17T21:53:17.331 INFO:tasks.workunit.client.1.smithi138.stdout:0/878: rename d3/d10/f2c to d3/d10/d6d/db6/f128 0 2024-05-17T21:53:17.332 INFO:tasks.workunit.client.1.smithi138.stdout:1/987: truncate d4/df/d3a/fc5 80722 0 2024-05-17T21:53:17.333 INFO:tasks.workunit.client.1.smithi138.stdout:0/879: read - d3/d10/d38/d3e/d59/d9b/f117 zero size 2024-05-17T21:53:17.336 INFO:tasks.workunit.client.1.smithi138.stdout:9/871: creat d6/d7/d16/d29/d48/d6e/d74/d75/d76/d8e/dba/f115 x:0 0 0 2024-05-17T21:53:17.336 INFO:tasks.workunit.client.1.smithi138.stdout:6/977: getdents d7/d8/de9 0 2024-05-17T21:53:17.341 INFO:tasks.workunit.client.1.smithi138.stdout:8/882: creat d0/d71/d92/f10f x:0 0 0 2024-05-17T21:53:17.347 INFO:tasks.workunit.client.1.smithi138.stdout:9/872: creat d6/d7/d16/d29/f116 x:0 0 0 2024-05-17T21:53:17.353 INFO:tasks.workunit.client.1.smithi138.stdout:8/883: dread - d0/dc/d8d/fad zero size 2024-05-17T21:53:17.357 INFO:tasks.workunit.client.1.smithi138.stdout:1/988: rename d4/db/d12f/d9f/db9/dae to d4/db/d60/dcc/d144 0 2024-05-17T21:53:17.362 INFO:tasks.workunit.client.1.smithi138.stdout:8/884: mkdir d0/d31/d110 0 2024-05-17T21:53:17.368 INFO:tasks.workunit.client.1.smithi138.stdout:8/885: creat d0/dc/d1a/d33/d4a/d68/f111 x:0 0 0 2024-05-17T21:53:17.369 INFO:tasks.workunit.client.1.smithi138.stdout:8/886: chown d0/dc/d1a/d33/l5e 82279 1 2024-05-17T21:53:17.370 INFO:tasks.workunit.client.1.smithi138.stdout:0/880: link d3/d10/f7d d3/d10/d39/f129 0 2024-05-17T21:53:17.373 INFO:tasks.workunit.client.1.smithi138.stdout:8/887: write d0/dc/f2b [1446269,14699] 0 2024-05-17T21:53:17.373 INFO:tasks.workunit.client.1.smithi138.stdout:0/881: symlink d3/d10/d6d/d9f/l12a 0 2024-05-17T21:53:17.374 INFO:tasks.workunit.client.1.smithi138.stdout:8/888: chown d0/f40 1458 1 2024-05-17T21:53:17.380 INFO:tasks.workunit.client.1.smithi138.stdout:8/889: truncate d0/dc/fa7 1709570 0 2024-05-17T21:53:17.381 INFO:tasks.workunit.client.1.smithi138.stdout:1/989: link d4/fc d4/db/d29/d5d/de5/f145 0 2024-05-17T21:53:17.381 INFO:tasks.workunit.client.1.smithi138.stdout:8/890: read d0/dc/f2b [3799164,72912] 0 2024-05-17T21:53:17.384 INFO:tasks.workunit.client.1.smithi138.stdout:1/990: truncate d4/db/d12f/fed 192098 0 2024-05-17T21:53:17.389 INFO:tasks.workunit.client.1.smithi138.stdout:1/991: dwrite d4/dd/db4/dba/fea [0,4194304] 0 2024-05-17T21:53:17.390 INFO:tasks.workunit.client.1.smithi138.stdout:1/992: chown d4/db/d29/f5b 9 1 2024-05-17T21:53:17.391 INFO:tasks.workunit.client.1.smithi138.stdout:1/993: stat d4/dd/d64/c11a 0 2024-05-17T21:53:17.403 INFO:tasks.workunit.client.1.smithi138.stdout:5/901: write d1/d2c/d63/db5/d1b/db7/fdf [537805,23454] 0 2024-05-17T21:53:17.408 INFO:tasks.workunit.client.1.smithi138.stdout:5/902: rename d1/d2c/d63/db5/d1b/db7/fdf to d1/f125 0 2024-05-17T21:53:17.410 INFO:tasks.workunit.client.1.smithi138.stdout:6/978: dwrite d7/d69/db5/dc8/d43/d3f/db6/fc5 [0,4194304] 0 2024-05-17T21:53:17.423 INFO:tasks.workunit.client.1.smithi138.stdout:9/873: link d6/d7/d16/d29/d38/d40/d44/f90 d6/d7/d16/d29/d48/d6f/f117 0 2024-05-17T21:53:17.424 INFO:tasks.workunit.client.1.smithi138.stdout:0/882: write d3/d10/d6d/fc7 [405121,70677] 0 2024-05-17T21:53:17.426 INFO:tasks.workunit.client.1.smithi138.stdout:9/874: truncate d6/d7/d16/d29/d38/d40/d44/fcb 300207 0 2024-05-17T21:53:17.429 INFO:tasks.workunit.client.1.smithi138.stdout:9/875: fdatasync d6/d7/d16/d29/d38/d40/d44/d5c/dda/ff5 0 2024-05-17T21:53:17.431 INFO:tasks.workunit.client.1.smithi138.stdout:0/883: dwrite d3/d10/d39/f72 [0,4194304] 0 2024-05-17T21:53:17.440 INFO:tasks.workunit.client.1.smithi138.stdout:1/994: dwrite d4/d2e/da0/daa/fad [4194304,4194304] 0 2024-05-17T21:53:17.444 INFO:tasks.workunit.client.1.smithi138.stdout:0/884: dwrite d3/d10/d6d/db6/fc1 [0,4194304] 0 2024-05-17T21:53:17.451 INFO:tasks.workunit.client.1.smithi138.stdout:9/876: dread d6/d7/d16/d29/d48/f34 [0,4194304] 0 2024-05-17T21:53:17.452 INFO:tasks.workunit.client.1.smithi138.stdout:0/885: read d3/d10/d38/d70/fa9 [98680,21791] 0 2024-05-17T21:53:17.459 INFO:tasks.workunit.client.1.smithi138.stdout:5/903: dwrite d1/f105 [0,4194304] 0 2024-05-17T21:53:17.463 INFO:tasks.workunit.client.1.smithi138.stdout:5/904: stat d1/d2c/d63/db5/d1b/c102 0 2024-05-17T21:53:17.463 INFO:tasks.workunit.client.1.smithi138.stdout:6/979: dwrite d7/d69/db5/dc8/d43/d5f/d6a/fde [0,4194304] 0 2024-05-17T21:53:17.467 INFO:tasks.workunit.client.1.smithi138.stdout:5/905: write d1/d2c/d63/db5/d1b/dc5/d36/f7a [4737164,103555] 0 2024-05-17T21:53:17.471 INFO:tasks.workunit.client.1.smithi138.stdout:9/877: rename d6/d7/d16/d29/d38/d40/d44/d5c/dc8 to d6/d7/d16/d29/d118 0 2024-05-17T21:53:17.475 INFO:tasks.workunit.client.1.smithi138.stdout:6/980: dwrite d7/d69/db5/dc8/d43/f64 [0,4194304] 0 2024-05-17T21:53:17.477 INFO:tasks.workunit.client.1.smithi138.stdout:1/995: dread d4/df/d4e/f84 [0,4194304] 0 2024-05-17T21:53:17.484 INFO:tasks.workunit.client.1.smithi138.stdout:1/996: fdatasync d4/df/d3a/fbe 0 2024-05-17T21:53:17.485 INFO:tasks.workunit.client.1.smithi138.stdout:6/981: write d7/d69/db5/dc8/d43/d3f/db6/fe8 [336789,121654] 0 2024-05-17T21:53:17.486 INFO:tasks.workunit.client.1.smithi138.stdout:0/886: dwrite d3/d10/d38/d70/d80/f93 [0,4194304] 0 2024-05-17T21:53:17.489 INFO:tasks.workunit.client.1.smithi138.stdout:5/906: rename d1/d53/d88/db9/d39/d4e/d58/d5b/f9e to d1/dd1/f126 0 2024-05-17T21:53:17.492 INFO:tasks.workunit.client.1.smithi138.stdout:0/887: mkdir d3/d10/d38/d3e/d59/dad/d12b 0 2024-05-17T21:53:17.495 INFO:tasks.workunit.client.1.smithi138.stdout:6/982: rename d7/d8/la9 to d7/d69/db5/dc8/d43/d5f/da4/l14b 0 2024-05-17T21:53:17.496 INFO:tasks.workunit.client.1.smithi138.stdout:5/907: rmdir d1/d53/d96/da0 39 2024-05-17T21:53:17.498 INFO:tasks.workunit.client.1.smithi138.stdout:1/997: creat d4/f146 x:0 0 0 2024-05-17T21:53:17.499 INFO:tasks.workunit.client.1.smithi138.stdout:0/888: unlink d3/d10/f17 0 2024-05-17T21:53:17.500 INFO:tasks.workunit.client.1.smithi138.stdout:6/983: mkdir d7/d69/db5/dc8/d43/d5f/da4/dcc/d14c 0 2024-05-17T21:53:17.503 INFO:tasks.workunit.client.1.smithi138.stdout:0/889: creat d3/d3d/d48/f12c x:0 0 0 2024-05-17T21:53:17.508 INFO:tasks.workunit.client.1.smithi138.stdout:0/890: truncate d3/d3d/f78 2335818 0 2024-05-17T21:53:17.509 INFO:tasks.workunit.client.1.smithi138.stdout:0/891: write d3/d10/d38/d3e/f11a [957821,17213] 0 2024-05-17T21:53:17.510 INFO:tasks.workunit.client.1.smithi138.stdout:6/984: creat d7/d69/db5/dc8/d43/d5f/d6a/d77/d107/f14d x:0 0 0 2024-05-17T21:53:17.511 INFO:tasks.workunit.client.1.smithi138.stdout:0/892: write d3/d10/f22 [3216650,118148] 0 2024-05-17T21:53:17.511 INFO:tasks.workunit.client.1.smithi138.stdout:5/908: truncate d1/d60/faf 694332 0 2024-05-17T21:53:17.517 INFO:tasks.workunit.client.1.smithi138.stdout:0/893: unlink d3/d10/d6d/d9f/f102 0 2024-05-17T21:53:17.519 INFO:tasks.workunit.client.1.smithi138.stdout:0/894: dread - d3/d10/d6d/d9f/f11c zero size 2024-05-17T21:53:17.521 INFO:tasks.workunit.client.1.smithi138.stdout:9/878: dwrite d6/d7/d16/d29/f100 [0,4194304] 0 2024-05-17T21:53:17.530 INFO:tasks.workunit.client.1.smithi138.stdout:6/985: getdents d7/d69/db5/dc8/d43/d3f/d79 0 2024-05-17T21:53:17.532 INFO:tasks.workunit.client.1.smithi138.stdout:1/998: dwrite d4/df/f12 [0,4194304] 0 2024-05-17T21:53:17.533 INFO:tasks.workunit.client.1.smithi138.stdout:9/879: dread d6/d7/d16/d29/d38/d40/d44/d5c/f8f [0,4194304] 0 2024-05-17T21:53:17.534 INFO:tasks.workunit.client.1.smithi138.stdout:9/880: chown d6/dbc/f9e 109 1 2024-05-17T21:53:17.535 INFO:tasks.workunit.client.1.smithi138.stdout:0/895: creat d3/d10/d38/d3e/d59/d9b/dc3/dd5/f12d x:0 0 0 2024-05-17T21:53:17.548 INFO:tasks.workunit.client.1.smithi138.stdout:1/999: fdatasync d4/df/f12 0 2024-05-17T21:53:17.551 INFO:tasks.workunit.client.1.smithi138.stdout:5/909: dwrite d1/d2c/d63/db5/fa [4194304,4194304] 0 2024-05-17T21:53:17.551 INFO:tasks.workunit.client.1.smithi138.stdout:6/986: unlink d7/d69/db5/dc8/d43/ldd 0 2024-05-17T21:53:17.556 INFO:tasks.workunit.client.1.smithi138.stdout:0/896: rename d3/d3d/f53 to d3/d10/d38/d3e/d59/dad/f12e 0 2024-05-17T21:53:17.561 INFO:tasks.workunit.client.1.smithi138.stdout:6/987: unlink d7/d8/f50 0 2024-05-17T21:53:17.571 INFO:tasks.workunit.client.1.smithi138.stdout:5/910: dwrite d1/d2c/d63/f9d [0,4194304] 0 2024-05-17T21:53:17.573 INFO:tasks.workunit.client.1.smithi138.stdout:6/988: chown d7/d69/db5/dc8/d43/l3d 390 1 2024-05-17T21:53:17.578 INFO:tasks.workunit.client.1.smithi138.stdout:9/881: rename d6/d7/d16/d29/d38/df4/ldf to d6/d7/d16/d29/d118/ded/df3/df6/l119 0 2024-05-17T21:53:17.578 INFO:tasks.workunit.client.1.smithi138.stdout:0/897: dwrite d3/d10/d38/d124/f6e [0,4194304] 0 2024-05-17T21:53:17.584 INFO:tasks.workunit.client.1.smithi138.stdout:5/911: rmdir d1/dd1/dca/dd0 39 2024-05-17T21:53:17.590 INFO:tasks.workunit.client.1.smithi138.stdout:5/912: dwrite d1/d53/d96/fc6 [0,4194304] 0 2024-05-17T21:53:17.592 INFO:tasks.workunit.client.1.smithi138.stdout:5/913: stat d1/d2c/d63/db5/df5/f10f 0 2024-05-17T21:53:17.603 INFO:tasks.workunit.client.1.smithi138.stdout:0/898: dread d3/d10/d39/f129 [0,4194304] 0 2024-05-17T21:53:17.604 INFO:tasks.workunit.client.1.smithi138.stdout:0/899: chown d3/d3d/da2/ccc 3 1 2024-05-17T21:53:17.611 INFO:tasks.workunit.client.1.smithi138.stdout:0/900: dwrite d3/d10/d38/d3e/f11a [0,4194304] 0 2024-05-17T21:53:17.611 INFO:tasks.workunit.client.1.smithi138.stdout:6/989: link d7/d69/db5/dc8/d43/d5f/da4/dcc/fe6 d7/d8/d10c/d122/f14e 0 2024-05-17T21:53:17.613 INFO:tasks.workunit.client.1.smithi138.stdout:0/901: chown d3/d10/d38/d3e/fb7 3 1 2024-05-17T21:53:17.615 INFO:tasks.workunit.client.1.smithi138.stdout:6/990: mkdir d7/d69/db5/dc8/d43/d3f/d14f 0 2024-05-17T21:53:17.616 INFO:tasks.workunit.client.1.smithi138.stdout:9/882: dread d6/d7/d16/d29/d38/d40/d44/fa4 [0,4194304] 0 2024-05-17T21:53:17.619 INFO:tasks.workunit.client.1.smithi138.stdout:5/914: dwrite d1/d2c/d63/db5/d1b/dc5/d36/f98 [0,4194304] 0 2024-05-17T21:53:17.619 INFO:tasks.workunit.client.1.smithi138.stdout:0/902: symlink d3/d10/d38/d3e/d59/dad/l12f 0 2024-05-17T21:53:17.623 INFO:tasks.workunit.client.1.smithi138.stdout:5/915: readlink d1/d2c/d63/db5/d19/l2a 0 2024-05-17T21:53:17.627 INFO:tasks.workunit.client.1.smithi138.stdout:6/991: dwrite d7/d69/db5/dc8/d43/d3f/db6/fc5 [0,4194304] 0 2024-05-17T21:53:17.627 INFO:tasks.workunit.client.1.smithi138.stdout:9/883: dread d6/d7/d16/d29/d38/d40/d44/fcb [0,4194304] 0 2024-05-17T21:53:17.629 INFO:tasks.workunit.client.1.smithi138.stdout:6/992: chown d7/d8/d78/l7a 3584 1 2024-05-17T21:53:17.632 INFO:tasks.workunit.client.1.smithi138.stdout:5/916: creat d1/d2c/d63/db5/d1b/db7/f127 x:0 0 0 2024-05-17T21:53:17.636 INFO:tasks.workunit.client.1.smithi138.stdout:6/993: fdatasync d7/d69/db5/dc8/d43/d5f/d6a/d9b/db2/fb9 0 2024-05-17T21:53:17.639 INFO:tasks.workunit.client.1.smithi138.stdout:7/935: sync 2024-05-17T21:53:17.639 INFO:tasks.workunit.client.1.smithi138.stdout:9/884: mkdir d6/d7/d16/d29/d48/d6e/d74/d75/d11a 0 2024-05-17T21:53:17.644 INFO:tasks.workunit.client.1.smithi138.stdout:9/885: stat d6/dbc/c65 0 2024-05-17T21:53:17.644 INFO:tasks.workunit.client.1.smithi138.stdout:5/917: dread d1/d2c/d63/db5/d1b/dc5/d36/f4b [0,4194304] 0 2024-05-17T21:53:17.645 INFO:tasks.workunit.client.1.smithi138.stdout:0/903: write d3/d10/d38/d3e/fc6 [327678,78844] 0 2024-05-17T21:53:17.649 INFO:tasks.workunit.client.1.smithi138.stdout:7/936: symlink d0/d18/d40/d41/d51/d66/d9e/l134 0 2024-05-17T21:53:17.649 INFO:tasks.workunit.client.1.smithi138.stdout:0/904: chown d3/d10/d38/d3e/d59/ceb 931452237 1 2024-05-17T21:53:17.650 INFO:tasks.workunit.client.1.smithi138.stdout:6/994: dwrite d7/d69/db5/dc8/d43/f35 [4194304,4194304] 0 2024-05-17T21:53:17.659 INFO:tasks.workunit.client.1.smithi138.stdout:7/937: rename d0/d18/d40/d9a/fa5 to d0/d18/d40/d9a/f135 0 2024-05-17T21:53:17.663 INFO:tasks.workunit.client.1.smithi138.stdout:6/995: read d7/da0/dbe/f114 [359221,14258] 0 2024-05-17T21:53:17.664 INFO:tasks.workunit.client.1.smithi138.stdout:9/886: dread d6/d7/d16/d29/d48/d6e/d74/d75/d76/de8/f112 [0,4194304] 0 2024-05-17T21:53:17.668 INFO:tasks.workunit.client.1.smithi138.stdout:5/918: write d1/d53/d88/db9/fa7 [3365,76885] 0 2024-05-17T21:53:17.673 INFO:tasks.workunit.client.1.smithi138.stdout:6/996: symlink d7/d69/db5/dc8/d43/d3f/d79/l150 0 2024-05-17T21:53:17.673 INFO:tasks.workunit.client.1.smithi138.stdout:0/905: dwrite d3/d10/d38/d3e/d59/fec [0,4194304] 0 2024-05-17T21:53:17.676 INFO:tasks.workunit.client.1.smithi138.stdout:7/938: link d0/d39/f10f d0/d18/d40/d41/d51/d81/f136 0 2024-05-17T21:53:17.681 INFO:tasks.workunit.client.1.smithi138.stdout:9/887: symlink d6/l11b 0 2024-05-17T21:53:17.682 INFO:tasks.workunit.client.1.smithi138.stdout:9/888: stat d6/d7/d16/l52 0 2024-05-17T21:53:17.685 INFO:tasks.workunit.client.1.smithi138.stdout:6/997: read d7/fc [7953615,125272] 0 2024-05-17T21:53:17.687 INFO:tasks.workunit.client.1.smithi138.stdout:5/919: dread d1/d2c/d63/db5/d1b/f54 [4194304,4194304] 0 2024-05-17T21:53:17.688 INFO:tasks.workunit.client.1.smithi138.stdout:9/889: dread d6/fe5 [0,4194304] 0 2024-05-17T21:53:17.809 INFO:tasks.workunit.client.1.smithi138.stdout:5/920: creat d1/d53/d88/dc9/d106/f128 x:0 0 0 2024-05-17T21:53:17.811 INFO:tasks.workunit.client.1.smithi138.stdout:6/998: mknod d7/d69/db5/dc8/d43/d3f/dd7/c151 0 2024-05-17T21:53:17.812 INFO:tasks.workunit.client.1.smithi138.stdout:6/999: readlink d7/da0/dbe/l110 0 2024-05-17T21:53:17.812 INFO:tasks.workunit.client.1.smithi138.stdout:9/890: symlink d6/d7/d16/d29/d38/df4/l11c 0 2024-05-17T21:53:17.815 INFO:tasks.workunit.client.1.smithi138.stdout:7/939: getdents d0/d18/d40/d41/d51/d81/d12c/d7c/da2/de0 0 2024-05-17T21:53:17.818 INFO:tasks.workunit.client.1.smithi138.stdout:5/921: dread - d1/d2c/d63/db5/d1b/dc5/d36/d68/f6c zero size 2024-05-17T21:53:17.823 INFO:tasks.workunit.client.1.smithi138.stdout:9/891: getdents d6/d7/d16/d29/d38/d40/d44/d5c 0 2024-05-17T21:53:17.825 INFO:tasks.workunit.client.1.smithi138.stdout:5/922: dwrite d1/d2c/d63/db5/f10b [0,4194304] 0 2024-05-17T21:53:17.827 INFO:tasks.workunit.client.1.smithi138.stdout:7/940: write d0/d18/d40/d41/d51/f69 [3951381,66859] 0 2024-05-17T21:53:17.832 INFO:tasks.workunit.client.1.smithi138.stdout:9/892: write d6/d7/d16/f21 [2660483,23562] 0 2024-05-17T21:53:17.836 INFO:tasks.workunit.client.1.smithi138.stdout:5/923: dread d1/d2c/d63/db5/d1b/f94 [0,4194304] 0 2024-05-17T21:53:17.844 INFO:tasks.workunit.client.1.smithi138.stdout:9/893: dread d6/d7/fc [0,4194304] 0 2024-05-17T21:53:17.849 INFO:tasks.workunit.client.1.smithi138.stdout:7/941: dread d0/f2a [4194304,4194304] 0 2024-05-17T21:53:17.850 INFO:tasks.workunit.client.1.smithi138.stdout:7/942: readlink d0/d9d/ld9 0 2024-05-17T21:53:17.855 INFO:tasks.workunit.client.1.smithi138.stdout:7/943: dwrite d0/f4b [4194304,4194304] 0 2024-05-17T21:53:17.908 INFO:tasks.workunit.client.1.smithi138.stdout:7/944: unlink d0/d39/f80 0 2024-05-17T21:53:17.911 INFO:tasks.workunit.client.1.smithi138.stdout:7/945: creat d0/d39/d101/f137 x:0 0 0 2024-05-17T21:53:17.928 INFO:tasks.workunit.client.1.smithi138.stdout:8/891: sync 2024-05-17T21:53:17.928 INFO:tasks.workunit.client.1.smithi138.stdout:2/952: sync 2024-05-17T21:53:17.932 INFO:tasks.workunit.client.1.smithi138.stdout:8/892: creat d0/dc/d3b/da4/dbc/de0/f112 x:0 0 0 2024-05-17T21:53:17.944 INFO:tasks.workunit.client.1.smithi138.stdout:8/893: truncate d0/d11/f69 2128716 0 2024-05-17T21:53:17.949 INFO:tasks.workunit.client.1.smithi138.stdout:9/894: dwrite d6/f33 [0,4194304] 0 2024-05-17T21:53:17.949 INFO:tasks.workunit.client.1.smithi138.stdout:7/946: dwrite d0/d18/d40/d41/d51/d66/f88 [0,4194304] 0 2024-05-17T21:53:17.953 INFO:tasks.workunit.client.1.smithi138.stdout:2/953: dwrite d3/da0/daa/feb [0,4194304] 0 2024-05-17T21:53:17.953 INFO:tasks.workunit.client.1.smithi138.stdout:2/954: stat d3/da0/daa/cb8 0 2024-05-17T21:53:17.954 INFO:tasks.workunit.client.1.smithi138.stdout:2/955: chown d3/d81 23160 1 2024-05-17T21:53:17.958 INFO:tasks.workunit.client.1.smithi138.stdout:8/894: symlink d0/dc/l113 0 2024-05-17T21:53:17.966 INFO:tasks.workunit.client.1.smithi138.stdout:7/947: dread d0/d18/d40/d41/d51/d81/d12c/d7c/da2/dc5/dcf/f106 [0,4194304] 0 2024-05-17T21:53:17.967 INFO:tasks.workunit.client.1.smithi138.stdout:8/895: symlink d0/dc/d8d/l114 0 2024-05-17T21:53:17.970 INFO:tasks.workunit.client.1.smithi138.stdout:8/896: symlink d0/d31/dbe/l115 0 2024-05-17T21:53:17.975 INFO:tasks.workunit.client.1.smithi138.stdout:8/897: creat d0/dc/d3b/da4/dbc/de0/d102/f116 x:0 0 0 2024-05-17T21:53:17.976 INFO:tasks.workunit.client.1.smithi138.stdout:8/898: chown d0/dc/d3b/da4/dbc/de0/f112 6 1 2024-05-17T21:53:17.977 INFO:tasks.workunit.client.1.smithi138.stdout:8/899: creat d0/dc/d3b/d4d/d7d/f117 x:0 0 0 2024-05-17T21:53:17.979 INFO:tasks.workunit.client.1.smithi138.stdout:2/956: dread d3/d21/d2f/d3e/d63/f80 [4194304,4194304] 0 2024-05-17T21:53:17.980 INFO:tasks.workunit.client.1.smithi138.stdout:2/957: stat d3/d81/d96/c9e 0 2024-05-17T21:53:17.984 INFO:tasks.workunit.client.1.smithi138.stdout:2/958: creat d3/d21/d2f/d3e/d58/d77/f135 x:0 0 0 2024-05-17T21:53:17.987 INFO:tasks.workunit.client.1.smithi138.stdout:2/959: read d3/d21/d2f/d3e/d58/fb1 [3175726,27120] 0 2024-05-17T21:53:17.991 INFO:tasks.workunit.client.1.smithi138.stdout:9/895: dwrite d6/d7/f77 [0,4194304] 0 2024-05-17T21:53:17.993 INFO:tasks.workunit.client.1.smithi138.stdout:2/960: fsync d3/f4 0 2024-05-17T21:53:17.996 INFO:tasks.workunit.client.1.smithi138.stdout:7/948: dwrite d0/d18/d40/fc7 [0,4194304] 0 2024-05-17T21:53:17.999 INFO:tasks.workunit.client.1.smithi138.stdout:7/949: truncate d0/d9d/fb5 10958385 0 2024-05-17T21:53:18.007 INFO:tasks.workunit.client.1.smithi138.stdout:7/950: mknod d0/d18/d40/d41/c138 0 2024-05-17T21:53:18.008 INFO:tasks.workunit.client.1.smithi138.stdout:8/900: dwrite d0/dc/d1a/d78/fe1 [0,4194304] 0 2024-05-17T21:53:18.010 INFO:tasks.workunit.client.1.smithi138.stdout:8/901: write d0/dc/d3b/d4d/d7d/d97/de9/f101 [739097,48542] 0 2024-05-17T21:53:18.018 INFO:tasks.workunit.client.1.smithi138.stdout:8/902: mkdir d0/dc/d3b/d87/d118 0 2024-05-17T21:53:18.020 INFO:tasks.workunit.client.1.smithi138.stdout:9/896: dwrite d6/d7/d16/f6d [0,4194304] 0 2024-05-17T21:53:18.020 INFO:tasks.workunit.client.1.smithi138.stdout:7/951: unlink d0/l7 0 2024-05-17T21:53:18.021 INFO:tasks.workunit.client.1.smithi138.stdout:2/961: write d3/d21/d2f/f33 [880083,8355] 0 2024-05-17T21:53:18.023 INFO:tasks.workunit.client.1.smithi138.stdout:8/903: fsync d0/dc/d8d/fad 0 2024-05-17T21:53:18.025 INFO:tasks.workunit.client.1.smithi138.stdout:7/952: rmdir d0/d18 39 2024-05-17T21:53:18.033 INFO:tasks.workunit.client.1.smithi138.stdout:9/897: dwrite d6/d7/d16/d29/d6c/d9c/dc1/f95 [0,4194304] 0 2024-05-17T21:53:18.033 INFO:tasks.workunit.client.1.smithi138.stdout:2/962: creat d3/d21/d2f/d3e/d58/d77/da4/dc5/f136 x:0 0 0 2024-05-17T21:53:18.038 INFO:tasks.workunit.client.1.smithi138.stdout:9/898: truncate d6/d7/d16/f1a 638404 0 2024-05-17T21:53:18.041 INFO:tasks.workunit.client.1.smithi138.stdout:8/904: dread d0/dc/d3b/f99 [0,4194304] 0 2024-05-17T21:53:18.043 INFO:tasks.workunit.client.1.smithi138.stdout:8/905: dread - d0/d71/fc7 zero size 2024-05-17T21:53:18.046 INFO:tasks.workunit.client.1.smithi138.stdout:7/953: dwrite d0/d18/d9c/dd4/ff1 [0,4194304] 0 2024-05-17T21:53:18.048 INFO:tasks.workunit.client.1.smithi138.stdout:2/963: dwrite d3/d21/d2f/d3e/d63/f6e [0,4194304] 0 2024-05-17T21:53:18.049 INFO:tasks.workunit.client.1.smithi138.stdout:7/954: write d0/d39/f9b [1289676,3943] 0 2024-05-17T21:53:18.049 INFO:tasks.workunit.client.1.smithi138.stdout:2/964: stat f2 0 2024-05-17T21:53:18.050 INFO:tasks.workunit.client.1.smithi138.stdout:9/899: truncate d6/d7/d16/d29/d48/d6e/d74/d75/d76/d8e/dba/df2/f101 581176 0 2024-05-17T21:53:18.054 INFO:tasks.workunit.client.1.smithi138.stdout:8/906: creat d0/d63/dea/f119 x:0 0 0 2024-05-17T21:53:18.054 INFO:tasks.workunit.client.1.smithi138.stdout:2/965: dread d3/d21/d2f/d3e/d55/d99/d112/ff9 [0,4194304] 0 2024-05-17T21:53:18.067 INFO:tasks.workunit.client.1.smithi138.stdout:8/907: dread d0/d11/f28 [0,4194304] 0 2024-05-17T21:53:18.068 INFO:tasks.workunit.client.1.smithi138.stdout:2/966: symlink d3/d21/d2f/d3e/d55/d99/dd2/l137 0 2024-05-17T21:53:18.070 INFO:tasks.workunit.client.1.smithi138.stdout:9/900: write d6/d7/d16/d29/d48/d6e/d74/d75/d76/d8e/dba/fc9 [49440,81045] 0 2024-05-17T21:53:18.073 INFO:tasks.workunit.client.1.smithi138.stdout:8/908: mkdir d0/d71/d92/d11a 0 2024-05-17T21:53:18.074 INFO:tasks.workunit.client.1.smithi138.stdout:2/967: readlink d3/d21/d2f/d3e/d55/d99/d84/le8 0 2024-05-17T21:53:18.075 INFO:tasks.workunit.client.1.smithi138.stdout:2/968: chown d3/d21/d2f/d3e/d55/d99/fa9 1 1 2024-05-17T21:53:18.076 INFO:tasks.workunit.client.1.smithi138.stdout:9/901: unlink d6/ff1 0 2024-05-17T21:53:18.076 INFO:tasks.workunit.client.1.smithi138.stdout:2/969: write d3/d21/d2f/f107 [1031244,66144] 0 2024-05-17T21:53:18.078 INFO:tasks.workunit.client.1.smithi138.stdout:9/902: write d6/d7/d16/d29/d6c/d9c/dc1/f53 [2412249,49671] 0 2024-05-17T21:53:18.079 INFO:tasks.workunit.client.1.smithi138.stdout:8/909: mknod d0/dc/d3b/d4d/d7d/c11b 0 2024-05-17T21:53:18.082 INFO:tasks.workunit.client.1.smithi138.stdout:0/906: sync 2024-05-17T21:53:18.082 INFO:tasks.workunit.client.1.smithi138.stdout:5/924: sync 2024-05-17T21:53:18.083 INFO:tasks.workunit.client.1.smithi138.stdout:7/955: rename d0/d18/d40/d41/d51/d81/d12c/d7c/da2/dc5/dcf/f11d to d0/d18/d40/d41/d50/d100/f139 0 2024-05-17T21:53:18.085 INFO:tasks.workunit.client.1.smithi138.stdout:2/970: unlink d3/d21/d2f/d3e/d58/d77/f135 0 2024-05-17T21:53:18.087 INFO:tasks.workunit.client.1.smithi138.stdout:8/910: mkdir d0/dc/d1a/d78/d11c 0 2024-05-17T21:53:18.089 INFO:tasks.workunit.client.1.smithi138.stdout:9/903: mknod d6/d7/d16/d29/d38/d40/d44/d5c/db4/c11d 0 2024-05-17T21:53:18.090 INFO:tasks.workunit.client.1.smithi138.stdout:5/925: dwrite d1/d2c/d63/db5/d1b/d109/f11a [0,4194304] 0 2024-05-17T21:53:18.093 INFO:tasks.workunit.client.1.smithi138.stdout:0/907: fdatasync d3/d3d/da2/ffb 0 2024-05-17T21:53:18.104 INFO:tasks.workunit.client.1.smithi138.stdout:2/971: mkdir d3/d21/d2f/d3e/d55/d99/d84/d9f/dfd/ddf/d114/d138 0 2024-05-17T21:53:18.105 INFO:tasks.workunit.client.1.smithi138.stdout:5/926: dread - d1/d46/fe7 zero size 2024-05-17T21:53:18.106 INFO:tasks.workunit.client.1.smithi138.stdout:5/927: write d1/d2c/d63/db5/d1b/f1d [5323284,37497] 0 2024-05-17T21:53:18.109 INFO:tasks.workunit.client.1.smithi138.stdout:0/908: unlink d3/d10/c1c 0 2024-05-17T21:53:18.110 INFO:tasks.workunit.client.1.smithi138.stdout:2/972: dwrite d3/f15 [0,4194304] 0 2024-05-17T21:53:18.113 INFO:tasks.workunit.client.1.smithi138.stdout:0/909: dread d3/d3d/ff7 [0,4194304] 0 2024-05-17T21:53:18.117 INFO:tasks.workunit.client.1.smithi138.stdout:8/911: rmdir d0/dc 39 2024-05-17T21:53:18.125 INFO:tasks.workunit.client.1.smithi138.stdout:2/973: creat d3/d21/d2f/d3e/d63/da3/f139 x:0 0 0 2024-05-17T21:53:18.127 INFO:tasks.workunit.client.1.smithi138.stdout:7/956: dwrite d0/f19 [0,4194304] 0 2024-05-17T21:53:18.130 INFO:tasks.workunit.client.1.smithi138.stdout:7/957: fsync d0/d18/d40/d41/d51/d81/d12c/d7c/da2/dc5/dcf/fd8 0 2024-05-17T21:53:18.137 INFO:tasks.workunit.client.1.smithi138.stdout:9/904: getdents d6/d7/d16/d29/d118/de2 0 2024-05-17T21:53:18.142 INFO:tasks.workunit.client.1.smithi138.stdout:5/928: getdents d1/d60/d2f/d119 0 2024-05-17T21:53:18.163 INFO:tasks.workunit.client.1.smithi138.stdout:8/912: rename d0/dc/d1a/d33/d4a to d0/dc/d1a/d51/d11d 0 2024-05-17T21:53:18.168 INFO:tasks.workunit.client.1.smithi138.stdout:2/974: mknod d3/c13a 0 2024-05-17T21:53:18.170 INFO:tasks.workunit.client.1.smithi138.stdout:7/958: symlink d0/d18/d40/d41/d51/d66/d9e/df9/l13a 0 2024-05-17T21:53:18.174 INFO:tasks.workunit.client.1.smithi138.stdout:9/905: creat d6/d7/d16/d29/d48/d6e/d74/d75/d76/d8e/ddd/d111/f11e x:0 0 0 2024-05-17T21:53:18.177 INFO:tasks.workunit.client.1.smithi138.stdout:7/959: dwrite d0/d18/d40/d9a/f125 [0,4194304] 0 2024-05-17T21:53:18.179 INFO:tasks.workunit.client.1.smithi138.stdout:7/960: write d0/d18/d40/d41/d51/d66/f88 [1697498,4627] 0 2024-05-17T21:53:18.181 INFO:tasks.workunit.client.1.smithi138.stdout:7/961: dread - d0/d18/d40/d41/d51/d66/da8/f118 zero size 2024-05-17T21:53:18.182 INFO:tasks.workunit.client.1.smithi138.stdout:2/975: creat d3/d81/d96/da8/f13b x:0 0 0 2024-05-17T21:53:18.187 INFO:tasks.workunit.client.1.smithi138.stdout:0/910: getdents d3/d10/d38 0 2024-05-17T21:53:18.190 INFO:tasks.workunit.client.1.smithi138.stdout:8/913: symlink d0/dc/d3b/d4d/d7d/d97/l11e 0 2024-05-17T21:53:18.191 INFO:tasks.workunit.client.1.smithi138.stdout:9/906: symlink d6/d7/d16/d29/d6c/d9c/dc1/d7e/db7/l11f 0 2024-05-17T21:53:18.193 INFO:tasks.workunit.client.1.smithi138.stdout:5/929: dwrite d1/d53/d88/de5/ded/f7f [0,4194304] 0 2024-05-17T21:53:18.195 INFO:tasks.workunit.client.1.smithi138.stdout:7/962: mkdir d0/d18/d40/d5f/d13b 0 2024-05-17T21:53:18.200 INFO:tasks.workunit.client.1.smithi138.stdout:8/914: creat d0/d63/dea/f11f x:0 0 0 2024-05-17T21:53:18.202 INFO:tasks.workunit.client.1.smithi138.stdout:8/915: fdatasync d0/dc/d3b/fba 0 2024-05-17T21:53:18.202 INFO:tasks.workunit.client.1.smithi138.stdout:5/930: chown d1/d2c/d63/db5/d1b/dc5/d36/d68/dad/c108 15 1 2024-05-17T21:53:18.203 INFO:tasks.workunit.client.1.smithi138.stdout:9/907: read d6/d7/d16/f1a [536571,20488] 0 2024-05-17T21:53:18.204 INFO:tasks.workunit.client.1.smithi138.stdout:0/911: creat d3/d10/d39/dd9/f130 x:0 0 0 2024-05-17T21:53:18.209 INFO:tasks.workunit.client.1.smithi138.stdout:8/916: creat d0/dc/d3b/da4/dbc/f120 x:0 0 0 2024-05-17T21:53:18.209 INFO:tasks.workunit.client.1.smithi138.stdout:8/917: chown d0/d63/f9c 4438 1 2024-05-17T21:53:18.210 INFO:tasks.workunit.client.1.smithi138.stdout:9/908: symlink d6/dbc/d10e/l120 0 2024-05-17T21:53:18.216 INFO:tasks.workunit.client.1.smithi138.stdout:2/976: dwrite d3/d21/d2f/d3e/d58/d77/da4/df2/fd6 [0,4194304] 0 2024-05-17T21:53:18.217 INFO:tasks.workunit.client.1.smithi138.stdout:2/977: chown d3/d21/d2f/d3e/d58/d77/da4/dc5/f136 346050 1 2024-05-17T21:53:18.218 INFO:tasks.workunit.client.1.smithi138.stdout:0/912: read d3/d10/d38/d70/d80/d94/f111 [86372,746] 0 2024-05-17T21:53:18.218 INFO:tasks.workunit.client.1.smithi138.stdout:9/909: mknod d6/d7/d16/d29/d118/de2/c121 0 2024-05-17T21:53:18.219 INFO:tasks.workunit.client.1.smithi138.stdout:2/978: stat d3/d21/d2f/d3e/d55/d99/dd2/lda 0 2024-05-17T21:53:18.222 INFO:tasks.workunit.client.1.smithi138.stdout:8/918: stat d0/d11/d3e/ccf 0 2024-05-17T21:53:18.225 INFO:tasks.workunit.client.1.smithi138.stdout:0/913: mkdir d3/d10/d38/d70/d80/d131 0 2024-05-17T21:53:18.228 INFO:tasks.workunit.client.1.smithi138.stdout:9/910: dwrite d6/d7/d16/d29/d6c/d9c/dc1/f95 [0,4194304] 0 2024-05-17T21:53:18.229 INFO:tasks.workunit.client.1.smithi138.stdout:2/979: rename d3/da0/daa to d3/d21/d2f/d3e/d55/d99/d84/dbe/dd8/d13c 0 2024-05-17T21:53:18.230 INFO:tasks.workunit.client.1.smithi138.stdout:5/931: dwrite d1/d60/faf [0,4194304] 0 2024-05-17T21:53:18.240 INFO:tasks.workunit.client.1.smithi138.stdout:0/914: rmdir d3/d10/d6d/d81 39 2024-05-17T21:53:18.241 INFO:tasks.workunit.client.1.smithi138.stdout:0/915: chown d3/d10/d6d/d9f/fed 1510 1 2024-05-17T21:53:18.243 INFO:tasks.workunit.client.1.smithi138.stdout:8/919: fdatasync d0/d11/f69 0 2024-05-17T21:53:18.248 INFO:tasks.workunit.client.1.smithi138.stdout:0/916: creat d3/d10/d6d/db6/db9/f132 x:0 0 0 2024-05-17T21:53:18.253 INFO:tasks.workunit.client.1.smithi138.stdout:2/980: dwrite d3/d21/d2f/fc0 [4194304,4194304] 0 2024-05-17T21:53:18.254 INFO:tasks.workunit.client.1.smithi138.stdout:9/911: creat d6/d7/d16/d29/d48/d6e/d74/d75/d11a/f122 x:0 0 0 2024-05-17T21:53:18.256 INFO:tasks.workunit.client.1.smithi138.stdout:9/912: dread d6/fe5 [0,4194304] 0 2024-05-17T21:53:18.259 INFO:tasks.workunit.client.1.smithi138.stdout:8/920: mkdir d0/dc/d1a/d51/d82/d10d/d121 0 2024-05-17T21:53:18.261 INFO:tasks.workunit.client.1.smithi138.stdout:5/932: write d1/d2c/d63/db5/d1b/dc5/d36/d68/fb3 [171676,111054] 0 2024-05-17T21:53:18.262 INFO:tasks.workunit.client.1.smithi138.stdout:2/981: mkdir d3/d21/d2f/d3e/d55/d99/d84/d9f/dfd/d13d 0 2024-05-17T21:53:18.263 INFO:tasks.workunit.client.1.smithi138.stdout:2/982: fsync d3/d21/d2f/d3e/d55/f111 0 2024-05-17T21:53:18.263 INFO:tasks.workunit.client.1.smithi138.stdout:0/917: truncate d3/d10/d39/f129 775383 0 2024-05-17T21:53:18.265 INFO:tasks.workunit.client.1.smithi138.stdout:2/983: read d3/f6 [2891721,114472] 0 2024-05-17T21:53:18.270 INFO:tasks.workunit.client.1.smithi138.stdout:0/918: creat d3/d10/d6d/db6/f133 x:0 0 0 2024-05-17T21:53:18.271 INFO:tasks.workunit.client.1.smithi138.stdout:0/919: fsync d3/d10/d6d/db6/db9/f113 0 2024-05-17T21:53:18.272 INFO:tasks.workunit.client.1.smithi138.stdout:0/920: chown d3/d10/d6d/d9f/da8/d101/f119 11379449 1 2024-05-17T21:53:18.272 INFO:tasks.workunit.client.1.smithi138.stdout:0/921: chown d3/d3d/da2/ffb 0 1 2024-05-17T21:53:18.277 INFO:tasks.workunit.client.1.smithi138.stdout:5/933: unlink d1/f84 0 2024-05-17T21:53:18.284 INFO:tasks.workunit.client.1.smithi138.stdout:2/984: link d3/d21/d2f/d3e/d58/f6b d3/d21/d2f/d3e/d55/d99/d84/d9f/d116/f13e 0 2024-05-17T21:53:18.285 INFO:tasks.workunit.client.1.smithi138.stdout:9/913: write d6/d7/d16/d29/d48/d3b/f88 [1434142,17501] 0 2024-05-17T21:53:18.288 INFO:tasks.workunit.client.1.smithi138.stdout:0/922: creat d3/d10/d38/d70/d80/d94/f134 x:0 0 0 2024-05-17T21:53:18.289 INFO:tasks.workunit.client.1.smithi138.stdout:5/934: rename d1/d2c/laa to d1/d53/d88/de5/ded/l129 0 2024-05-17T21:53:18.291 INFO:tasks.workunit.client.1.smithi138.stdout:0/923: truncate d3/d10/d38/d3e/d59/d9b/ff0 216743 0 2024-05-17T21:53:18.293 INFO:tasks.workunit.client.1.smithi138.stdout:9/914: dwrite d6/d7/d16/d29/d38/d40/d44/d5c/dda/ff5 [0,4194304] 0 2024-05-17T21:53:18.293 INFO:tasks.workunit.client.1.smithi138.stdout:2/985: mkdir d3/d21/d2f/d3e/d58/d13f 0 2024-05-17T21:53:18.301 INFO:tasks.workunit.client.1.smithi138.stdout:2/986: dwrite d3/d21/d2f/dd4/fdb [0,4194304] 0 2024-05-17T21:53:18.310 INFO:tasks.workunit.client.1.smithi138.stdout:8/921: truncate d0/d63/fbb 1668164 0 2024-05-17T21:53:18.317 INFO:tasks.workunit.client.1.smithi138.stdout:8/922: dwrite d0/dc/d1a/d51/d82/ffc [0,4194304] 0 2024-05-17T21:53:18.322 INFO:tasks.workunit.client.1.smithi138.stdout:0/924: fdatasync d3/d10/d39/f129 0 2024-05-17T21:53:18.324 INFO:tasks.workunit.client.1.smithi138.stdout:8/923: mkdir d0/dc/d3b/da4/dbc/d122 0 2024-05-17T21:53:18.328 INFO:tasks.workunit.client.1.smithi138.stdout:9/915: dread d6/d7/d16/d29/d38/d40/f6b [0,4194304] 0 2024-05-17T21:53:18.328 INFO:tasks.workunit.client.1.smithi138.stdout:8/924: mkdir d0/dc/d1a/d51/d82/d10d/d123 0 2024-05-17T21:53:18.330 INFO:tasks.workunit.client.1.smithi138.stdout:0/925: rename d3/d10/d38/d3e/d59/cce to d3/d10/d38/d3e/d59/d9b/dc3/dd5/c135 0 2024-05-17T21:53:18.332 INFO:tasks.workunit.client.1.smithi138.stdout:9/916: truncate d6/d7/d16/d29/d38/d40/d44/d5c/dda/ff5 5076641 0 2024-05-17T21:53:18.333 INFO:tasks.workunit.client.1.smithi138.stdout:8/925: mknod d0/dc/d1a/d51/d82/d10d/d123/c124 0 2024-05-17T21:53:18.334 INFO:tasks.workunit.client.1.smithi138.stdout:9/917: chown d6/d7/d16/d29/d48/fd7 3 1 2024-05-17T21:53:18.334 INFO:tasks.workunit.client.1.smithi138.stdout:0/926: mknod d3/d10/d6d/d9f/da8/d101/c136 0 2024-05-17T21:53:18.335 INFO:tasks.workunit.client.1.smithi138.stdout:8/926: creat d0/dc/d1a/d44/f125 x:0 0 0 2024-05-17T21:53:18.335 INFO:tasks.workunit.client.1.smithi138.stdout:0/927: chown d3/d10/d38/d3e/d59/f9d 25 1 2024-05-17T21:53:18.336 INFO:tasks.workunit.client.1.smithi138.stdout:9/918: chown d6/d7/d16/d29/d38/d40/d44/d5c/dda/c110 2503450 1 2024-05-17T21:53:18.340 INFO:tasks.workunit.client.1.smithi138.stdout:0/928: symlink d3/d10/d38/d70/l137 0 2024-05-17T21:53:18.341 INFO:tasks.workunit.client.1.smithi138.stdout:9/919: mkdir d6/d7/d16/d29/d48/d6e/d74/d75/d76/d8e/dba/df2/d123 0 2024-05-17T21:53:18.342 INFO:tasks.workunit.client.1.smithi138.stdout:8/927: dwrite d0/dc/d3b/da4/dbc/de0/f112 [0,4194304] 0 2024-05-17T21:53:18.344 INFO:tasks.workunit.client.1.smithi138.stdout:5/935: rename d1/d2c/d63/db5/df5/f11c to d1/d53/db1/f12a 0 2024-05-17T21:53:18.348 INFO:tasks.workunit.client.1.smithi138.stdout:8/928: creat d0/d63/f126 x:0 0 0 2024-05-17T21:53:18.352 INFO:tasks.workunit.client.1.smithi138.stdout:5/936: mkdir d1/d2c/d63/db5/d1b/dc5/d36/d68/d12b 0 2024-05-17T21:53:18.354 INFO:tasks.workunit.client.1.smithi138.stdout:8/929: creat d0/dc/d3b/da4/dbc/dc3/f127 x:0 0 0 2024-05-17T21:53:18.356 INFO:tasks.workunit.client.1.smithi138.stdout:5/937: mknod d1/d53/d88/db9/d39/d4e/c12c 0 2024-05-17T21:53:18.358 INFO:tasks.workunit.client.1.smithi138.stdout:9/920: dread d6/d7/d16/d29/d38/f81 [0,4194304] 0 2024-05-17T21:53:18.359 INFO:tasks.workunit.client.1.smithi138.stdout:9/921: chown d6/d7/d16/d29/d38/fa3 904 1 2024-05-17T21:53:18.363 INFO:tasks.workunit.client.1.smithi138.stdout:8/930: mknod d0/d63/dea/d10b/c128 0 2024-05-17T21:53:18.363 INFO:tasks.workunit.client.1.smithi138.stdout:8/931: readlink d0/dc/d3b/laa 0 2024-05-17T21:53:18.364 INFO:tasks.workunit.client.1.smithi138.stdout:5/938: truncate d1/f4f 549154 0 2024-05-17T21:53:18.365 INFO:tasks.workunit.client.1.smithi138.stdout:9/922: truncate d6/d7/d16/d29/d38/d40/d44/f78 4241696 0 2024-05-17T21:53:18.367 INFO:tasks.workunit.client.1.smithi138.stdout:8/932: symlink d0/d31/l129 0 2024-05-17T21:53:18.369 INFO:tasks.workunit.client.1.smithi138.stdout:9/923: mkdir d6/d7/d16/d29/d118/ded/df3/df6/d124 0 2024-05-17T21:53:18.372 INFO:tasks.workunit.client.1.smithi138.stdout:8/933: creat d0/dc/d3b/d87/d118/f12a x:0 0 0 2024-05-17T21:53:18.374 INFO:tasks.workunit.client.1.smithi138.stdout:9/924: creat d6/d7/d16/d29/d38/df4/f125 x:0 0 0 2024-05-17T21:53:18.385 INFO:tasks.workunit.client.1.smithi138.stdout:9/925: rename d6/d7/d16/d29/f100 to d6/d7/d16/d29/d48/d6e/f126 0 2024-05-17T21:53:18.405 INFO:tasks.workunit.client.1.smithi138.stdout:0/929: dwrite d3/f5e [0,4194304] 0 2024-05-17T21:53:18.412 INFO:tasks.workunit.client.1.smithi138.stdout:0/930: chown d3/d10/d38/d3e/la6 3 1 2024-05-17T21:53:18.422 INFO:tasks.workunit.client.1.smithi138.stdout:0/931: rename d3/d10/c69 to d3/d10/d38/d70/d80/d94/c138 0 2024-05-17T21:53:18.432 INFO:tasks.workunit.client.1.smithi138.stdout:0/932: creat d3/d3d/f139 x:0 0 0 2024-05-17T21:53:18.433 INFO:tasks.workunit.client.1.smithi138.stdout:0/933: fdatasync d3/d10/d38/d3e/f11a 0 2024-05-17T21:53:18.438 INFO:tasks.workunit.client.1.smithi138.stdout:0/934: dread - d3/d3d/da2/fd7 zero size 2024-05-17T21:53:18.440 INFO:tasks.workunit.client.1.smithi138.stdout:0/935: chown d3/d10/d38/d3e/d59/cf9 2987201 1 2024-05-17T21:53:18.444 INFO:tasks.workunit.client.1.smithi138.stdout:8/934: truncate d0/f2 4159527 0 2024-05-17T21:53:18.445 INFO:tasks.workunit.client.1.smithi138.stdout:8/935: dread - d0/dc/d1a/d33/ff8 zero size 2024-05-17T21:53:18.457 INFO:tasks.workunit.client.1.smithi138.stdout:0/936: creat d3/d10/d38/d70/d80/d94/f13a x:0 0 0 2024-05-17T21:53:18.459 INFO:tasks.workunit.client.1.smithi138.stdout:0/937: write d3/d10/d39/dd9/f130 [386716,130966] 0 2024-05-17T21:53:18.466 INFO:tasks.workunit.client.1.smithi138.stdout:9/926: truncate d6/d7/d16/d29/d6c/d9c/dc1/d7e/db7/fc4 830473 0 2024-05-17T21:53:18.467 INFO:tasks.workunit.client.1.smithi138.stdout:0/938: dread d3/d10/d39/dd9/f130 [0,4194304] 0 2024-05-17T21:53:18.468 INFO:tasks.workunit.client.1.smithi138.stdout:9/927: chown d6/d7/d16/d29/d6c/d9c/laa 10494 1 2024-05-17T21:53:18.474 INFO:tasks.workunit.client.1.smithi138.stdout:0/939: unlink d3/f8e 0 2024-05-17T21:53:18.481 INFO:tasks.workunit.client.1.smithi138.stdout:9/928: mknod d6/c127 0 2024-05-17T21:53:18.482 INFO:tasks.workunit.client.1.smithi138.stdout:0/940: dread d3/d10/f1a [0,4194304] 0 2024-05-17T21:53:18.485 INFO:tasks.workunit.client.1.smithi138.stdout:0/941: write d3/d10/d6d/fa5 [4917446,85473] 0 2024-05-17T21:53:18.487 INFO:tasks.workunit.client.1.smithi138.stdout:8/936: dwrite d0/dc/d3b/d4d/f77 [0,4194304] 0 2024-05-17T21:53:18.497 INFO:tasks.workunit.client.1.smithi138.stdout:0/942: creat d3/d10/d6d/d9f/da8/d101/f13b x:0 0 0 2024-05-17T21:53:18.502 INFO:tasks.workunit.client.1.smithi138.stdout:9/929: dread d6/d7/d16/f20 [0,4194304] 0 2024-05-17T21:53:18.509 INFO:tasks.workunit.client.1.smithi138.stdout:8/937: dread d0/f2c [0,4194304] 0 2024-05-17T21:53:18.511 INFO:tasks.workunit.client.1.smithi138.stdout:9/930: mknod d6/d7/d16/d29/d48/d6e/d74/d75/d76/d8e/dba/df2/d123/c128 0 2024-05-17T21:53:18.512 INFO:tasks.workunit.client.1.smithi138.stdout:0/943: dread d3/d10/f31 [0,4194304] 0 2024-05-17T21:53:18.513 INFO:tasks.workunit.client.1.smithi138.stdout:0/944: chown d3/d10/d38/d3e/d59/dad/fd1 117019578 1 2024-05-17T21:53:18.515 INFO:tasks.workunit.client.1.smithi138.stdout:8/938: mkdir d0/d2f/d107/d12b 0 2024-05-17T21:53:18.516 INFO:tasks.workunit.client.1.smithi138.stdout:8/939: fsync d0/dc/d1a/f2a 0 2024-05-17T21:53:18.517 INFO:tasks.workunit.client.1.smithi138.stdout:9/931: rename d6/d7/d16/f1a to d6/d7/d16/d29/d48/d6e/d74/d75/d76/d8e/ddd/f129 0 2024-05-17T21:53:18.519 INFO:tasks.workunit.client.1.smithi138.stdout:0/945: read d3/d10/d39/f72 [1386556,117703] 0 2024-05-17T21:53:18.520 INFO:tasks.workunit.client.1.smithi138.stdout:0/946: read d3/d10/f7d [374326,91372] 0 2024-05-17T21:53:18.527 INFO:tasks.workunit.client.1.smithi138.stdout:0/947: creat d3/d3d/d48/f13c x:0 0 0 2024-05-17T21:53:18.527 INFO:tasks.workunit.client.1.smithi138.stdout:0/948: stat d3/d3d/d48/fe4 0 2024-05-17T21:53:18.530 INFO:tasks.workunit.client.1.smithi138.stdout:9/932: mkdir d6/d7/d16/dc2/d12a 0 2024-05-17T21:53:18.533 INFO:tasks.workunit.client.1.smithi138.stdout:8/940: getdents d0/dc/d3b/da4/dbc/de0/d102 0 2024-05-17T21:53:18.533 INFO:tasks.workunit.client.1.smithi138.stdout:8/941: chown d0/dc/d1a/d33/l66 1508 1 2024-05-17T21:53:18.535 INFO:tasks.workunit.client.1.smithi138.stdout:9/933: rename d6/l72 to d6/d7/d16/d29/d48/d6e/d74/d75/d76/d8e/ddd/l12b 0 2024-05-17T21:53:18.535 INFO:tasks.workunit.client.1.smithi138.stdout:0/949: mkdir d3/d10/d38/d3e/d13d 0 2024-05-17T21:53:18.538 INFO:tasks.workunit.client.1.smithi138.stdout:9/934: mkdir d6/d7/d16/d29/d48/d6e/d74/d75/d76/de8/d12c 0 2024-05-17T21:53:18.539 INFO:tasks.workunit.client.1.smithi138.stdout:8/942: link d0/d71/d92/f10f d0/dc/d3b/da4/dbc/d122/f12c 0 2024-05-17T21:53:18.541 INFO:tasks.workunit.client.1.smithi138.stdout:0/950: rename d3/d10/d6d/d9f/f11c to d3/d10/d6d/d9f/f13e 0 2024-05-17T21:53:18.543 INFO:tasks.workunit.client.1.smithi138.stdout:8/943: mknod d0/dc/d3b/da4/dbc/de0/c12d 0 2024-05-17T21:53:18.545 INFO:tasks.workunit.client.1.smithi138.stdout:0/951: creat d3/d10/d38/d70/d80/f13f x:0 0 0 2024-05-17T21:53:18.549 INFO:tasks.workunit.client.1.smithi138.stdout:9/935: rename d6/d7/d16/d29/d48/d6e/f56 to d6/d7/d16/dc2/f12d 0 2024-05-17T21:53:18.551 INFO:tasks.workunit.client.1.smithi138.stdout:9/936: write d6/d7/d16/d29/d48/d6e/d74/d75/d76/d8e/dba/fd1 [2759166,64394] 0 2024-05-17T21:53:18.551 INFO:tasks.workunit.client.1.smithi138.stdout:9/937: readlink d6/d7/d16/l79 0 2024-05-17T21:53:18.552 INFO:tasks.workunit.client.1.smithi138.stdout:0/952: fsync d3/d10/d6d/db6/db9/ff6 0 2024-05-17T21:53:18.555 INFO:tasks.workunit.client.1.smithi138.stdout:0/953: dread d3/d10/d38/d70/d80/d94/f111 [0,4194304] 0 2024-05-17T21:53:18.564 INFO:tasks.workunit.client.1.smithi138.stdout:8/944: write d0/d2f/f41 [171983,96935] 0 2024-05-17T21:53:18.565 INFO:tasks.workunit.client.1.smithi138.stdout:9/938: link d6/d7/d16/c19 d6/d7/d16/d29/d48/d6e/d74/d75/d11a/c12e 0 2024-05-17T21:53:18.570 INFO:tasks.workunit.client.1.smithi138.stdout:0/954: getdents d3/d10/d38/d124 0 2024-05-17T21:53:18.571 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:18 smithi107 ceph-mon[28434]: pgmap v47: 65 pgs: 65 active+clean; 3.9 GiB data, 13 GiB used, 524 GiB / 536 GiB avail; 100 MiB/s rd, 215 MiB/s wr, 483 op/s 2024-05-17T21:53:18.572 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:18 smithi107 ceph-mon[28434]: from='mgr.14636 ' entity='mgr.smithi138.napnll' 2024-05-17T21:53:18.572 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:18 smithi107 ceph-mon[28434]: from='mgr.14636 ' entity='mgr.smithi138.napnll' 2024-05-17T21:53:18.575 INFO:tasks.workunit.client.1.smithi138.stdout:9/939: getdents d6/d7/d16/d29/d48/d6e/d74/d75 0 2024-05-17T21:53:18.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:18 smithi138 ceph-mon[35948]: pgmap v47: 65 pgs: 65 active+clean; 3.9 GiB data, 13 GiB used, 524 GiB / 536 GiB avail; 100 MiB/s rd, 215 MiB/s wr, 483 op/s 2024-05-17T21:53:18.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:18 smithi138 ceph-mon[35948]: from='mgr.14636 ' entity='mgr.smithi138.napnll' 2024-05-17T21:53:18.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:18 smithi138 ceph-mon[35948]: from='mgr.14636 ' entity='mgr.smithi138.napnll' 2024-05-17T21:53:18.578 INFO:tasks.workunit.client.1.smithi138.stdout:8/945: dread d0/dc/d3b/d4d/d7d/d97/f9a [0,4194304] 0 2024-05-17T21:53:18.579 INFO:tasks.workunit.client.1.smithi138.stdout:9/940: creat d6/d7/d16/d29/d48/d6e/d74/d75/d76/f12f x:0 0 0 2024-05-17T21:53:18.581 INFO:tasks.workunit.client.1.smithi138.stdout:8/946: truncate d0/d11/f28 1028616 0 2024-05-17T21:53:18.581 INFO:tasks.workunit.client.1.smithi138.stdout:0/955: write d3/d10/d38/d70/d80/d94/fa4 [4634168,110821] 0 2024-05-17T21:53:18.584 INFO:tasks.workunit.client.1.smithi138.stdout:8/947: mknod d0/dc/d3b/d4d/d7d/c12e 0 2024-05-17T21:53:18.588 INFO:tasks.workunit.client.1.smithi138.stdout:8/948: creat d0/d71/d92/f12f x:0 0 0 2024-05-17T21:53:18.589 INFO:tasks.workunit.client.1.smithi138.stdout:0/956: rmdir d3/d10/d39/d97 39 2024-05-17T21:53:18.590 INFO:tasks.workunit.client.1.smithi138.stdout:9/941: getdents d6/d7/d16/d29/d48/d6e/d74/d75/d76/d8e/dba/df2 0 2024-05-17T21:53:18.591 INFO:tasks.workunit.client.1.smithi138.stdout:9/942: dread - d6/d7/d16/d29/d38/df4/f125 zero size 2024-05-17T21:53:18.592 INFO:tasks.workunit.client.1.smithi138.stdout:8/949: dread d0/dc/d1a/d51/d11d/fc8 [0,4194304] 0 2024-05-17T21:53:18.595 INFO:tasks.workunit.client.1.smithi138.stdout:0/957: symlink d3/d10/d38/d70/d80/d131/l140 0 2024-05-17T21:53:18.598 INFO:tasks.workunit.client.1.smithi138.stdout:0/958: creat d3/d10/d6d/d9f/d115/f141 x:0 0 0 2024-05-17T21:53:18.602 INFO:tasks.workunit.client.1.smithi138.stdout:8/950: link d0/dc/d1a/d33/c83 d0/dc/d1a/c130 0 2024-05-17T21:53:18.605 INFO:tasks.workunit.client.1.smithi138.stdout:0/959: dread d3/d10/d38/d3e/d59/dad/f12e [4194304,4194304] 0 2024-05-17T21:53:18.606 INFO:tasks.workunit.client.1.smithi138.stdout:0/960: fdatasync d3/f83 0 2024-05-17T21:53:18.609 INFO:tasks.workunit.client.1.smithi138.stdout:9/943: dread d6/d7/d16/d29/d48/d6e/d74/d75/d76/de8/d108/fbd [0,4194304] 0 2024-05-17T21:53:18.610 INFO:tasks.workunit.client.1.smithi138.stdout:0/961: fdatasync d3/d10/d39/f72 0 2024-05-17T21:53:18.613 INFO:tasks.workunit.client.1.smithi138.stdout:8/951: write d0/dc/d3b/d4d/fbd [860830,53093] 0 2024-05-17T21:53:18.613 INFO:tasks.workunit.client.1.smithi138.stdout:9/944: mknod d6/d7/d16/d29/d48/d6e/d74/d75/d76/c130 0 2024-05-17T21:53:18.614 INFO:tasks.workunit.client.1.smithi138.stdout:8/952: fdatasync d0/d63/fff 0 2024-05-17T21:53:18.615 INFO:tasks.workunit.client.1.smithi138.stdout:0/962: mkdir d3/d10/d6d/d9f/d115/d142 0 2024-05-17T21:53:18.618 INFO:tasks.workunit.client.1.smithi138.stdout:8/953: mkdir d0/dc/d1a/d51/d82/d131 0 2024-05-17T21:53:18.620 INFO:tasks.workunit.client.1.smithi138.stdout:9/945: truncate d6/d7/d16/d29/d48/d6e/d74/d75/d76/de8/d108/fbd 3412683 0 2024-05-17T21:53:18.621 INFO:tasks.workunit.client.1.smithi138.stdout:0/963: dwrite d3/d10/d38/d124/f6e [0,4194304] 0 2024-05-17T21:53:18.625 INFO:tasks.workunit.client.1.smithi138.stdout:9/946: readlink d6/d7/d16/d29/d48/d6e/d74/d75/lcd 0 2024-05-17T21:53:18.628 INFO:tasks.workunit.client.1.smithi138.stdout:0/964: creat d3/d3d/da2/d11d/f143 x:0 0 0 2024-05-17T21:53:18.632 INFO:tasks.workunit.client.1.smithi138.stdout:0/965: fdatasync d3/d10/fd8 0 2024-05-17T21:53:18.638 INFO:tasks.workunit.client.1.smithi138.stdout:0/966: mknod d3/d10/d38/d70/d80/c144 0 2024-05-17T21:53:18.641 INFO:tasks.workunit.client.1.smithi138.stdout:0/967: mknod d3/d10/d39/c145 0 2024-05-17T21:53:18.644 INFO:tasks.workunit.client.1.smithi138.stdout:8/954: dwrite d0/dc/f10 [4194304,4194304] 0 2024-05-17T21:53:18.648 INFO:tasks.workunit.client.1.smithi138.stdout:0/968: symlink d3/d10/d39/d97/l146 0 2024-05-17T21:53:18.649 INFO:tasks.workunit.client.1.smithi138.stdout:9/947: dread d6/d7/d16/d29/d48/d3b/f88 [0,4194304] 0 2024-05-17T21:53:18.661 INFO:tasks.workunit.client.1.smithi138.stdout:0/969: getdents d3/d10/d38/d3e 0 2024-05-17T21:53:18.668 INFO:tasks.workunit.client.1.smithi138.stdout:8/955: dread d0/ff [0,4194304] 0 2024-05-17T21:53:18.669 INFO:tasks.workunit.client.1.smithi138.stdout:0/970: mkdir d3/d10/d38/d3e/d59/d147 0 2024-05-17T21:53:18.670 INFO:tasks.workunit.client.1.smithi138.stdout:0/971: stat d3/d10/d38/d3e/d59/d9b/f117 0 2024-05-17T21:53:18.671 INFO:tasks.workunit.client.1.smithi138.stdout:8/956: mknod d0/dc/d1a/c132 0 2024-05-17T21:53:18.672 INFO:tasks.workunit.client.1.smithi138.stdout:0/972: rmdir d3/d3d/d48 39 2024-05-17T21:53:18.675 INFO:tasks.workunit.client.1.smithi138.stdout:8/957: truncate d0/d2f/f45 2686922 0 2024-05-17T21:53:18.676 INFO:tasks.workunit.client.1.smithi138.stdout:9/948: dwrite d6/d7/d16/d29/fca [0,4194304] 0 2024-05-17T21:53:18.678 INFO:tasks.workunit.client.1.smithi138.stdout:9/949: chown d6/d7/d16/d29/d6c/d9c/dc1/d7e/lb5 30927253 1 2024-05-17T21:53:18.680 INFO:tasks.workunit.client.1.smithi138.stdout:8/958: rmdir d0/dc/d3b 39 2024-05-17T21:53:18.684 INFO:tasks.workunit.client.1.smithi138.stdout:9/950: write d6/d7/d16/d29/d38/d40/d44/fa4 [2771186,115448] 0 2024-05-17T21:53:18.686 INFO:tasks.workunit.client.1.smithi138.stdout:0/973: creat d3/d10/d38/d3e/d59/f148 x:0 0 0 2024-05-17T21:53:18.687 INFO:tasks.workunit.client.1.smithi138.stdout:8/959: rename d0/dc/d1a/d33/ff8 to d0/d31/d110/f133 0 2024-05-17T21:53:18.690 INFO:tasks.workunit.client.1.smithi138.stdout:9/951: creat d6/d7/d16/d29/d118/ded/df3/f131 x:0 0 0 2024-05-17T21:53:18.693 INFO:tasks.workunit.client.1.smithi138.stdout:0/974: fsync d3/d10/f20 0 2024-05-17T21:53:18.695 INFO:tasks.workunit.client.1.smithi138.stdout:0/975: chown d3/d10/d38/d70/d80/d94/f13a 1298 1 2024-05-17T21:53:18.696 INFO:tasks.workunit.client.1.smithi138.stdout:8/960: symlink d0/dc/d1a/d51/l134 0 2024-05-17T21:53:18.697 INFO:tasks.workunit.client.1.smithi138.stdout:9/952: mknod d6/d7/d16/d29/d48/d6f/c132 0 2024-05-17T21:53:18.699 INFO:tasks.workunit.client.1.smithi138.stdout:8/961: dread d0/dc/d1a/d51/d11d/fa8 [0,4194304] 0 2024-05-17T21:53:18.705 INFO:tasks.workunit.client.1.smithi138.stdout:0/976: symlink d3/d10/d6d/d9f/d115/d142/l149 0 2024-05-17T21:53:18.706 INFO:tasks.workunit.client.1.smithi138.stdout:0/977: read d3/d3d/da2/ffb [2395125,58769] 0 2024-05-17T21:53:18.707 INFO:tasks.workunit.client.1.smithi138.stdout:0/978: dread - d3/d10/d6d/d9f/da8/daa/f10a zero size 2024-05-17T21:53:18.708 INFO:tasks.workunit.client.1.smithi138.stdout:0/979: chown d3/d10/f31 0 1 2024-05-17T21:53:18.708 INFO:tasks.workunit.client.1.smithi138.stdout:9/953: mkdir d6/d7/d16/d29/d48/d6e/d74/d75/d76/de8/d12c/d133 0 2024-05-17T21:53:18.709 INFO:tasks.workunit.client.1.smithi138.stdout:8/962: mkdir d0/d71/d92/d11a/d135 0 2024-05-17T21:53:18.709 INFO:tasks.workunit.client.1.smithi138.stdout:8/963: fdatasync d0/ff0 0 2024-05-17T21:53:18.713 INFO:tasks.workunit.client.1.smithi138.stdout:9/954: write d6/d7/d16/d29/f37 [3017672,67327] 0 2024-05-17T21:53:18.713 INFO:tasks.workunit.client.1.smithi138.stdout:8/964: chown d0/dc/fb9 557016 1 2024-05-17T21:53:18.715 INFO:tasks.workunit.client.1.smithi138.stdout:9/955: read d6/d7/d16/d29/d38/d40/d44/d5c/dda/ff5 [1842499,88813] 0 2024-05-17T21:53:18.719 INFO:tasks.workunit.client.1.smithi138.stdout:0/980: creat d3/d10/d38/d70/f14a x:0 0 0 2024-05-17T21:53:18.721 INFO:tasks.workunit.client.1.smithi138.stdout:8/965: dwrite d0/d63/dea/f11f [0,4194304] 0 2024-05-17T21:53:18.725 INFO:tasks.workunit.client.1.smithi138.stdout:9/956: dwrite d6/d7/d16/d29/d38/d40/d44/d5c/ff0 [0,4194304] 0 2024-05-17T21:53:18.727 INFO:tasks.workunit.client.1.smithi138.stdout:0/981: unlink d3/d10/d38/d70/d80/f93 0 2024-05-17T21:53:18.728 INFO:tasks.workunit.client.1.smithi138.stdout:0/982: fsync d3/d10/d6d/d9f/d115/f141 0 2024-05-17T21:53:18.734 INFO:tasks.workunit.client.1.smithi138.stdout:9/957: unlink d6/d7/d16/d29/d48/d6e/d74/d75/d76/d8e/ddd/fec 0 2024-05-17T21:53:18.735 INFO:tasks.workunit.client.1.smithi138.stdout:0/983: mkdir d3/d10/d38/d70/d80/d14b 0 2024-05-17T21:53:18.742 INFO:tasks.workunit.client.1.smithi138.stdout:8/966: getdents d0/dc/d3b/d87 0 2024-05-17T21:53:18.755 INFO:tasks.workunit.client.1.smithi138.stdout:8/967: write d0/d71/d92/f10f [928531,50833] 0 2024-05-17T21:53:18.757 INFO:tasks.workunit.client.1.smithi138.stdout:9/958: dwrite d6/d7/d16/d29/d48/f34 [4194304,4194304] 0 2024-05-17T21:53:18.762 INFO:tasks.workunit.client.1.smithi138.stdout:8/968: rename d0/ff to d0/dc/d3b/da4/dbc/de0/f136 0 2024-05-17T21:53:18.767 INFO:tasks.workunit.client.1.smithi138.stdout:8/969: creat d0/d2f/d107/d12b/f137 x:0 0 0 2024-05-17T21:53:18.768 INFO:tasks.workunit.client.1.smithi138.stdout:9/959: creat d6/d7/d16/dc2/f134 x:0 0 0 2024-05-17T21:53:18.771 INFO:tasks.workunit.client.1.smithi138.stdout:8/970: mkdir d0/dc/d1a/d33/d138 0 2024-05-17T21:53:18.772 INFO:tasks.workunit.client.1.smithi138.stdout:8/971: chown d0/d71 869349 1 2024-05-17T21:53:18.772 INFO:tasks.workunit.client.1.smithi138.stdout:9/960: creat d6/d7/d16/d29/d48/d6e/d74/d75/d76/d8e/dba/df2/d123/f135 x:0 0 0 2024-05-17T21:53:18.773 INFO:tasks.workunit.client.1.smithi138.stdout:9/961: chown d6/d7/d16/c43 483051592 1 2024-05-17T21:53:18.773 INFO:tasks.workunit.client.1.smithi138.stdout:8/972: stat d0/dc/lce 0 2024-05-17T21:53:18.774 INFO:tasks.workunit.client.1.smithi138.stdout:8/973: dread - d0/dc/d1a/d51/d11d/d68/f111 zero size 2024-05-17T21:53:18.775 INFO:tasks.workunit.client.1.smithi138.stdout:9/962: creat d6/d7/d16/d29/d118/f136 x:0 0 0 2024-05-17T21:53:18.778 INFO:tasks.workunit.client.1.smithi138.stdout:9/963: dread - d6/d7/d16/d29/d48/d6e/d74/d75/d76/fbf zero size 2024-05-17T21:53:18.784 INFO:tasks.workunit.client.1.smithi138.stdout:9/964: dwrite d6/d7/d16/d29/d38/d40/d44/f87 [0,4194304] 0 2024-05-17T21:53:18.787 INFO:tasks.workunit.client.1.smithi138.stdout:9/965: write d6/d7/d16/d29/d118/ff7 [624310,115924] 0 2024-05-17T21:53:18.791 INFO:tasks.workunit.client.1.smithi138.stdout:9/966: rename d6/d7/d16/d29/d118/de2 to d6/d7/d16/d29/d38/d8b/d137 0 2024-05-17T21:53:18.792 INFO:tasks.workunit.client.1.smithi138.stdout:8/974: truncate d0/dc/d3b/d4d/d7d/d97/f9a 3569733 0 2024-05-17T21:53:18.794 INFO:tasks.workunit.client.1.smithi138.stdout:8/975: fdatasync d0/dc/fbf 0 2024-05-17T21:53:18.795 INFO:tasks.workunit.client.1.smithi138.stdout:9/967: mknod d6/d7/d16/d29/d48/d6e/d74/d75/dfd/c138 0 2024-05-17T21:53:18.796 INFO:tasks.workunit.client.1.smithi138.stdout:9/968: write d6/d7/d16/d29/d38/f81 [1234987,42908] 0 2024-05-17T21:53:18.799 INFO:tasks.workunit.client.1.smithi138.stdout:9/969: chown d6/d7/d16/d29/d48/d6e/d74/d75/d76/d8e/dba/df2/d123 71 1 2024-05-17T21:53:18.801 INFO:tasks.workunit.client.1.smithi138.stdout:8/976: getdents d0/dc/d3b/d87/d118 0 2024-05-17T21:53:18.805 INFO:tasks.workunit.client.1.smithi138.stdout:8/977: fdatasync d0/d11/f28 0 2024-05-17T21:53:18.812 INFO:tasks.workunit.client.1.smithi138.stdout:9/970: write d6/d7/d16/d29/d38/d40/f6b [2201011,102422] 0 2024-05-17T21:53:18.814 INFO:tasks.workunit.client.1.smithi138.stdout:9/971: chown d6/d7/d16/d29/d48/d6e/d74/d75/d76/de8/d12c/d133 419 1 2024-05-17T21:53:18.821 INFO:tasks.workunit.client.1.smithi138.stdout:8/978: dwrite d0/d63/fbb [0,4194304] 0 2024-05-17T21:53:18.825 INFO:tasks.workunit.client.1.smithi138.stdout:9/972: write d6/d7/d16/d29/d38/d40/d44/d5c/f8f [146296,130723] 0 2024-05-17T21:53:18.829 INFO:tasks.workunit.client.1.smithi138.stdout:8/979: rename d0/dc/d1a/d51/d11d/fa8 to d0/dc/d1a/d51/d82/d10d/d121/f139 0 2024-05-17T21:53:18.833 INFO:tasks.workunit.client.1.smithi138.stdout:8/980: mkdir d0/dc/d1a/d51/d82/ddb/d13a 0 2024-05-17T21:53:18.835 INFO:tasks.workunit.client.1.smithi138.stdout:8/981: truncate d0/dc/d3b/da4/dbc/dc3/f127 827018 0 2024-05-17T21:53:18.836 INFO:tasks.workunit.client.1.smithi138.stdout:8/982: write d0/d63/dea/f119 [693648,56338] 0 2024-05-17T21:53:18.836 INFO:tasks.workunit.client.1.smithi138.stdout:8/983: dread - d0/dc/d3b/da0/ff2 zero size 2024-05-17T21:53:18.837 INFO:tasks.workunit.client.1.smithi138.stdout:8/984: chown d0/d63/l81 13 1 2024-05-17T21:53:18.838 INFO:tasks.workunit.client.1.smithi138.stdout:8/985: fsync d0/dc/d3b/da0/ff2 0 2024-05-17T21:53:18.839 INFO:tasks.workunit.client.1.smithi138.stdout:8/986: write d0/dc/d3b/f89 [7634879,116975] 0 2024-05-17T21:53:18.840 INFO:tasks.workunit.client.1.smithi138.stdout:8/987: write d0/dc/d1a/d44/ff3 [506003,109103] 0 2024-05-17T21:53:18.895 INFO:tasks.workunit.client.1.smithi138.stdout:8/988: mkdir d0/d13b 0 2024-05-17T21:53:18.896 INFO:tasks.workunit.client.1.smithi138.stdout:8/989: mknod d0/d63/dea/c13c 0 2024-05-17T21:53:18.897 INFO:tasks.workunit.client.1.smithi138.stdout:8/990: readlink d0/dc/d1a/d51/d82/ld8 0 2024-05-17T21:53:18.898 INFO:tasks.workunit.client.1.smithi138.stdout:8/991: symlink d0/d71/l13d 0 2024-05-17T21:53:18.901 INFO:tasks.workunit.client.1.smithi138.stdout:8/992: unlink d0/dc/d1a/d51/d82/ffc 0 2024-05-17T21:53:18.969 INFO:tasks.workunit.client.1.smithi138.stdout:7/963: sync 2024-05-17T21:53:19.080 INFO:tasks.workunit.client.1.smithi138.stdout:7/964: truncate d0/f1e 1610635 0 2024-05-17T21:53:19.081 INFO:tasks.workunit.client.1.smithi138.stdout:7/965: chown d0/f2a 396 1 2024-05-17T21:53:19.087 INFO:tasks.workunit.client.1.smithi138.stdout:7/966: rename d0/la6 to d0/d18/d40/d41/d51/d81/l13c 0 2024-05-17T21:53:19.099 INFO:tasks.workunit.client.1.smithi138.stdout:8/993: dwrite d0/d71/d92/f9d [0,4194304] 0 2024-05-17T21:53:19.100 INFO:tasks.workunit.client.1.smithi138.stdout:8/994: fdatasync d0/dc/d3b/d4d/d7d/d97/de9/f101 0 2024-05-17T21:53:19.125 INFO:tasks.workunit.client.1.smithi138.stdout:8/995: dread d0/f40 [0,4194304] 0 2024-05-17T21:53:19.127 INFO:tasks.workunit.client.1.smithi138.stdout:8/996: write d0/dc/d3b/da4/dbc/de0/ffd [1011183,56200] 0 2024-05-17T21:53:19.135 INFO:tasks.workunit.client.1.smithi138.stdout:8/997: creat d0/d71/d92/d11a/d135/f13e x:0 0 0 2024-05-17T21:53:19.135 INFO:tasks.workunit.client.1.smithi138.stdout:0/984: sync 2024-05-17T21:53:19.135 INFO:tasks.workunit.client.1.smithi138.stdout:9/973: sync 2024-05-17T21:53:19.135 INFO:tasks.workunit.client.1.smithi138.stdout:2/987: sync 2024-05-17T21:53:19.135 INFO:tasks.workunit.client.1.smithi138.stdout:5/939: sync 2024-05-17T21:53:19.142 INFO:tasks.workunit.client.1.smithi138.stdout:8/998: dread d0/dc/f10 [4194304,4194304] 0 2024-05-17T21:53:19.144 INFO:tasks.workunit.client.1.smithi138.stdout:8/999: write d0/d63/dea/f119 [179782,94444] 0 2024-05-17T21:53:19.150 INFO:tasks.workunit.client.1.smithi138.stdout:9/974: mkdir d6/d7/d16/d29/d48/d6e/d74/d75/dfd/d139 0 2024-05-17T21:53:19.160 INFO:tasks.workunit.client.1.smithi138.stdout:0/985: rename d3/d10/f31 to d3/d3d/f14c 0 2024-05-17T21:53:19.163 INFO:tasks.workunit.client.1.smithi138.stdout:9/975: fsync d6/d7/d16/d29/d48/d6e/d74/d75/d76/de8/d108/fbd 0 2024-05-17T21:53:19.170 INFO:tasks.workunit.client.1.smithi138.stdout:9/976: symlink d6/d7/d16/d29/d48/d6e/d74/d75/dfd/l13a 0 2024-05-17T21:53:19.170 INFO:tasks.workunit.client.1.smithi138.stdout:5/940: getdents d1/dd1/dca/dd0 0 2024-05-17T21:53:19.173 INFO:tasks.workunit.client.1.smithi138.stdout:9/977: mknod d6/d7/d16/d29/d6c/d9c/dc1/c13b 0 2024-05-17T21:53:19.180 INFO:tasks.workunit.client.1.smithi138.stdout:9/978: getdents d6/d7/d16/d29/d48/d6e/d74/d75/d76/d8e/dba/df2/d123 0 2024-05-17T21:53:19.205 INFO:tasks.workunit.client.1.smithi138.stdout:7/967: symlink d0/d18/d40/d41/d51/d81/d12c/d7c/l13d 0 2024-05-17T21:53:19.279 INFO:tasks.workunit.client.1.smithi138.stdout:2/988: dwrite d3/d21/d2f/d3e/d58/f71 [0,4194304] 0 2024-05-17T21:53:19.294 INFO:tasks.workunit.client.1.smithi138.stdout:9/979: chown d6/d7/d16/d29/d6c/d9c/dc1/ccc 3 1 2024-05-17T21:53:19.295 INFO:tasks.workunit.client.1.smithi138.stdout:9/980: write d6/d7/d16/d29/d38/f81 [3457482,32064] 0 2024-05-17T21:53:19.307 INFO:tasks.workunit.client.1.smithi138.stdout:7/968: dwrite d0/f12 [4194304,4194304] 0 2024-05-17T21:53:19.308 INFO:tasks.workunit.client.1.smithi138.stdout:0/986: rename d3/cd2 to d3/d10/d6d/d9f/c14d 0 2024-05-17T21:53:19.311 INFO:tasks.workunit.client.1.smithi138.stdout:2/989: rename d3/d21/d2f/d3e/l6c to d3/d21/d2f/d3e/d58/d77/da4/d12b/l140 0 2024-05-17T21:53:19.312 INFO:tasks.workunit.client.1.smithi138.stdout:5/941: mkdir d1/d53/d88/d12d 0 2024-05-17T21:53:19.312 INFO:tasks.workunit.client.1.smithi138.stdout:2/990: write d3/d21/d2f/f33 [280445,43323] 0 2024-05-17T21:53:19.313 INFO:tasks.workunit.client.1.smithi138.stdout:7/969: mknod d0/d18/d40/d9a/c13e 0 2024-05-17T21:53:19.313 INFO:tasks.workunit.client.1.smithi138.stdout:2/991: write d3/fe [1488783,55758] 0 2024-05-17T21:53:19.314 INFO:tasks.workunit.client.1.smithi138.stdout:0/987: symlink d3/d3d/da2/l14e 0 2024-05-17T21:53:19.316 INFO:tasks.workunit.client.1.smithi138.stdout:0/988: write d3/d10/d38/d124/f6e [3612018,53016] 0 2024-05-17T21:53:19.327 INFO:tasks.workunit.client.1.smithi138.stdout:2/992: creat d3/d21/d2f/d3e/d55/d73/f141 x:0 0 0 2024-05-17T21:53:19.328 INFO:tasks.workunit.client.1.smithi138.stdout:7/970: mkdir d0/d18/d40/d41/d51/d81/d12c/d7c/d13f 0 2024-05-17T21:53:19.328 INFO:tasks.workunit.client.1.smithi138.stdout:2/993: chown d3/d21/d2f/d3e/d55/d99/d84/d9f/dfd/ddf/d114 221810 1 2024-05-17T21:53:19.328 INFO:tasks.workunit.client.1.smithi138.stdout:2/994: chown d3/d21/f67 11419711 1 2024-05-17T21:53:19.329 INFO:tasks.workunit.client.1.smithi138.stdout:0/989: symlink d3/d10/d6d/db6/db9/l14f 0 2024-05-17T21:53:19.419 INFO:tasks.workunit.client.1.smithi138.stdout:0/990: mkdir d3/d10/d6d/db6/d150 0 2024-05-17T21:53:19.420 INFO:tasks.workunit.client.1.smithi138.stdout:0/991: write d3/d3d/f114 [391869,30508] 0 2024-05-17T21:53:19.421 INFO:tasks.workunit.client.1.smithi138.stdout:2/995: unlink d3/d21/d2f/d3e/d55/d99/dd2/ca5 0 2024-05-17T21:53:19.439 INFO:tasks.workunit.client.1.smithi138.stdout:2/996: symlink d3/d21/d2f/d3e/d55/d99/d84/d9f/d116/l142 0 2024-05-17T21:53:19.444 INFO:tasks.workunit.client.1.smithi138.stdout:2/997: creat d3/da0/d130/f143 x:0 0 0 2024-05-17T21:53:19.446 INFO:tasks.workunit.client.1.smithi138.stdout:2/998: creat d3/d21/d2f/d3e/d58/d77/da4/d12b/f144 x:0 0 0 2024-05-17T21:53:19.449 INFO:tasks.workunit.client.1.smithi138.stdout:2/999: creat d3/d81/f145 x:0 0 0 2024-05-17T21:53:19.476 INFO:tasks.workunit.client.1.smithi138.stdout:0/992: write d3/d10/d6d/fa5 [2618495,35775] 0 2024-05-17T21:53:19.482 INFO:tasks.workunit.client.1.smithi138.stdout:7/971: dwrite d0/d18/d9c/fe1 [0,4194304] 0 2024-05-17T21:53:19.482 INFO:tasks.workunit.client.1.smithi138.stdout:5/942: dwrite d1/d2c/dbf/fc3 [0,4194304] 0 2024-05-17T21:53:19.485 INFO:tasks.workunit.client.1.smithi138.stdout:0/993: unlink d3/d10/d6d/d9f/da8/daa/c108 0 2024-05-17T21:53:19.486 INFO:tasks.workunit.client.1.smithi138.stdout:0/994: dread - d3/d3d/f139 zero size 2024-05-17T21:53:19.488 INFO:tasks.workunit.client.1.smithi138.stdout:5/943: fsync d1/d2c/d63/db5/d19/f3a 0 2024-05-17T21:53:19.489 INFO:tasks.workunit.client.1.smithi138.stdout:0/995: fdatasync d3/d10/d6d/d9f/d115/f141 0 2024-05-17T21:53:19.493 INFO:tasks.workunit.client.1.smithi138.stdout:7/972: dwrite d0/d18/d40/d41/d51/d66/d9e/fa1 [0,4194304] 0 2024-05-17T21:53:19.504 INFO:tasks.workunit.client.1.smithi138.stdout:7/973: rename d0/d18/d40/d41/d51/d66/da8/f118 to d0/d18/d40/d41/d51/d81/d12c/d7c/da2/dff/f140 0 2024-05-17T21:53:19.512 INFO:tasks.workunit.client.1.smithi138.stdout:0/996: creat d3/d10/d38/d3e/d59/f151 x:0 0 0 2024-05-17T21:53:19.513 INFO:tasks.workunit.client.1.smithi138.stdout:5/944: dread d1/d53/ff6 [0,4194304] 0 2024-05-17T21:53:19.518 INFO:tasks.workunit.client.1.smithi138.stdout:0/997: creat d3/d10/d6d/d9f/d115/d142/f152 x:0 0 0 2024-05-17T21:53:19.519 INFO:tasks.workunit.client.1.smithi138.stdout:5/945: truncate d1/d2c/d63/db5/d73/fe9 204311 0 2024-05-17T21:53:19.520 INFO:tasks.workunit.client.1.smithi138.stdout:5/946: stat d1/d53/db1/f12a 0 2024-05-17T21:53:19.522 INFO:tasks.workunit.client.1.smithi138.stdout:0/998: unlink d3/d10/d38/d3e/d59/dad/l12f 0 2024-05-17T21:53:19.523 INFO:tasks.workunit.client.1.smithi138.stdout:7/974: rename d0/d18/dbf/d116 to d0/d18/d40/d41/d141 0 2024-05-17T21:53:19.527 INFO:tasks.workunit.client.1.smithi138.stdout:0/999: write d3/d3d/d48/f12c [422140,48350] 0 2024-05-17T21:53:19.530 INFO:tasks.workunit.client.1.smithi138.stdout:5/947: creat d1/d60/d2f/d5e/de4/f12e x:0 0 0 2024-05-17T21:53:19.537 INFO:tasks.workunit.client.1.smithi138.stdout:5/948: symlink d1/d2c/d63/db5/d1b/l12f 0 2024-05-17T21:53:19.538 INFO:tasks.workunit.client.1.smithi138.stdout:5/949: fdatasync d1/d2c/d63/db5/d1b/db7/fda 0 2024-05-17T21:53:19.542 INFO:tasks.workunit.client.1.smithi138.stdout:5/950: creat d1/d2c/d63/db5/d1b/ddb/f130 x:0 0 0 2024-05-17T21:53:19.543 INFO:tasks.workunit.client.1.smithi138.stdout:7/975: dwrite d0/d18/d40/f82 [0,4194304] 0 2024-05-17T21:53:19.545 INFO:tasks.workunit.client.1.smithi138.stdout:5/951: read d1/d2c/d63/db5/d1b/dc5/f7c [2353293,87628] 0 2024-05-17T21:53:19.546 INFO:tasks.workunit.client.1.smithi138.stdout:5/952: readlink d1/d2c/d63/db5/d1b/l12f 0 2024-05-17T21:53:19.548 INFO:tasks.workunit.client.1.smithi138.stdout:7/976: rmdir d0/d39/d78 39 2024-05-17T21:53:19.552 INFO:tasks.workunit.client.1.smithi138.stdout:7/977: truncate d0/d39/f3d 1062905 0 2024-05-17T21:53:19.555 INFO:tasks.workunit.client.1.smithi138.stdout:7/978: symlink d0/d18/d40/d41/d51/d66/d9e/df9/dfe/d112/l142 0 2024-05-17T21:53:19.556 INFO:tasks.workunit.client.1.smithi138.stdout:7/979: stat d0/d18/d40/d41/d51/c62 0 2024-05-17T21:53:19.561 INFO:tasks.workunit.client.1.smithi138.stdout:7/980: getdents d0/d18/d40/d5f/dbe 0 2024-05-17T21:53:19.564 INFO:tasks.workunit.client.1.smithi138.stdout:7/981: mknod d0/d18/d40/d41/d51/d81/d12c/d7c/da2/c143 0 2024-05-17T21:53:19.574 INFO:tasks.workunit.client.1.smithi138.stdout:7/982: write d0/f27 [140038,14700] 0 2024-05-17T21:53:19.575 INFO:tasks.workunit.client.1.smithi138.stdout:5/953: dread d1/d2c/d63/db5/f31 [0,4194304] 0 2024-05-17T21:53:19.576 INFO:tasks.workunit.client.1.smithi138.stdout:7/983: unlink d0/f4b 0 2024-05-17T21:53:19.577 INFO:tasks.workunit.client.1.smithi138.stdout:7/984: truncate d0/d18/d40/d9a/f125 5200310 0 2024-05-17T21:53:19.580 INFO:tasks.workunit.client.1.smithi138.stdout:5/954: symlink d1/d53/d88/l131 0 2024-05-17T21:53:19.580 INFO:tasks.workunit.client.1.smithi138.stdout:7/985: mknod d0/c144 0 2024-05-17T21:53:19.581 INFO:tasks.workunit.client.1.smithi138.stdout:5/955: chown d1/d46/cd7 111 1 2024-05-17T21:53:19.586 INFO:tasks.workunit.client.1.smithi138.stdout:7/986: dwrite d0/d39/f48 [4194304,4194304] 0 2024-05-17T21:53:19.593 INFO:tasks.workunit.client.1.smithi138.stdout:7/987: mkdir d0/d18/d40/d41/d50/d145 0 2024-05-17T21:53:19.596 INFO:tasks.workunit.client.1.smithi138.stdout:5/956: write d1/d2c/d63/db5/d1b/ddb/ff3 [316822,99769] 0 2024-05-17T21:53:19.596 INFO:tasks.workunit.client.1.smithi138.stdout:5/957: dread - d1/d53/db1/fdd zero size 2024-05-17T21:53:19.599 INFO:tasks.workunit.client.1.smithi138.stdout:7/988: link d0/d18/d40/d41/d51/d81/d12c/d7c/da2/dff/f140 d0/d18/dbf/dd1/f146 0 2024-05-17T21:53:19.611 INFO:tasks.workunit.client.1.smithi138.stdout:7/989: write d0/d18/d40/d41/f77 [2045050,85609] 0 2024-05-17T21:53:19.612 INFO:tasks.workunit.client.1.smithi138.stdout:5/958: dread d1/dd1/f8f [0,4194304] 0 2024-05-17T21:53:19.620 INFO:tasks.workunit.client.1.smithi138.stdout:5/959: creat d1/d53/d96/f132 x:0 0 0 2024-05-17T21:53:19.620 INFO:tasks.workunit.client.1.smithi138.stdout:9/981: sync 2024-05-17T21:53:19.621 INFO:tasks.workunit.client.1.smithi138.stdout:7/990: truncate d0/d18/d40/d41/d51/f69 3983791 0 2024-05-17T21:53:19.623 INFO:tasks.workunit.client.1.smithi138.stdout:5/960: rename d1/d46/d95/f10a to d1/dd1/f133 0 2024-05-17T21:53:19.626 INFO:tasks.workunit.client.1.smithi138.stdout:9/982: mknod d6/d7/d16/d29/d48/d67/d86/c13c 0 2024-05-17T21:53:19.629 INFO:tasks.workunit.client.1.smithi138.stdout:7/991: creat d0/d18/d40/d5f/dbe/f147 x:0 0 0 2024-05-17T21:53:19.639 INFO:tasks.workunit.client.1.smithi138.stdout:9/983: symlink d6/d7/d16/d29/d6c/d9c/dc1/d7e/db7/l13d 0 2024-05-17T21:53:19.642 INFO:tasks.workunit.client.1.smithi138.stdout:9/984: fdatasync d6/fe5 0 2024-05-17T21:53:19.643 INFO:tasks.workunit.client.1.smithi138.stdout:5/961: write d1/d53/d96/f9b [2215906,46449] 0 2024-05-17T21:53:19.644 INFO:tasks.workunit.client.1.smithi138.stdout:9/985: write d6/d7/d16/d29/d38/d40/d44/fa4 [2670441,83176] 0 2024-05-17T21:53:19.645 INFO:tasks.workunit.client.1.smithi138.stdout:7/992: creat d0/d18/d40/f148 x:0 0 0 2024-05-17T21:53:19.650 INFO:tasks.workunit.client.1.smithi138.stdout:5/962: write d1/dd1/dca/f112 [4067131,106732] 0 2024-05-17T21:53:19.654 INFO:tasks.workunit.client.1.smithi138.stdout:9/986: rmdir d6/d7/d16/d29/d48/d6e/d74/d75/d76 39 2024-05-17T21:53:19.655 INFO:tasks.workunit.client.1.smithi138.stdout:7/993: unlink d0/d18/d40/d41/d51/d66/d9e/df9/dfe/d112/l142 0 2024-05-17T21:53:19.656 INFO:tasks.workunit.client.1.smithi138.stdout:7/994: chown d0/d18/d40/f148 2 1 2024-05-17T21:53:19.657 INFO:tasks.workunit.client.1.smithi138.stdout:5/963: rmdir d1/d60/d2f/d5e/de4/d90 39 2024-05-17T21:53:19.662 INFO:tasks.workunit.client.1.smithi138.stdout:7/995: rename d0/d18/d40/d5f/ccb to d0/d18/d40/d41/d51/d81/d12c/d7c/c149 0 2024-05-17T21:53:19.664 INFO:tasks.workunit.client.1.smithi138.stdout:5/964: mknod d1/d53/d88/db9/c134 0 2024-05-17T21:53:19.665 INFO:tasks.workunit.client.1.smithi138.stdout:5/965: fsync d1/d2c/d63/db5/d1b/ddb/f130 0 2024-05-17T21:53:19.666 INFO:tasks.workunit.client.1.smithi138.stdout:9/987: getdents d6/d7/dc0 0 2024-05-17T21:53:19.667 INFO:tasks.workunit.client.1.smithi138.stdout:5/966: fdatasync d1/d2c/d63/db5/d1b/f1d 0 2024-05-17T21:53:19.669 INFO:tasks.workunit.client.1.smithi138.stdout:9/988: truncate d6/d7/d16/d29/d48/d3b/fff 8767347 0 2024-05-17T21:53:19.673 INFO:tasks.workunit.client.1.smithi138.stdout:5/967: symlink d1/d2c/d63/db5/d1b/dc5/d36/d68/l135 0 2024-05-17T21:53:19.674 INFO:tasks.workunit.client.1.smithi138.stdout:5/968: write d1/d2c/d63/db5/d1b/d109/f11a [2034820,1879] 0 2024-05-17T21:53:19.675 INFO:tasks.workunit.client.1.smithi138.stdout:9/989: mkdir d6/d7/d16/d29/d118/ded/df3/df6/d13e 0 2024-05-17T21:53:19.677 INFO:tasks.workunit.client.1.smithi138.stdout:7/996: rmdir d0/d18/d40/d41/d51/d66/d9e/df9/dfe/d112 0 2024-05-17T21:53:19.686 INFO:tasks.workunit.client.1.smithi138.stdout:7/997: dwrite d0/f2a [4194304,4194304] 0 2024-05-17T21:53:19.688 INFO:tasks.workunit.client.1.smithi138.stdout:9/990: dwrite d6/d7/d16/d29/d48/d6e/d74/d75/d76/d8e/dba/fd1 [0,4194304] 0 2024-05-17T21:53:19.692 INFO:tasks.workunit.client.1.smithi138.stdout:7/998: mknod d0/d18/d40/d5f/dbe/c14a 0 2024-05-17T21:53:19.695 INFO:tasks.workunit.client.1.smithi138.stdout:9/991: mknod d6/dbc/d10e/c13f 0 2024-05-17T21:53:19.696 INFO:tasks.workunit.client.1.smithi138.stdout:5/969: link d1/d60/l44 d1/d2c/d63/db5/d1b/ddb/l136 0 2024-05-17T21:53:19.705 INFO:tasks.workunit.client.1.smithi138.stdout:9/992: rename d6/d7/d16/d29/d48/d6e/d74/d75/d76/de8/d108/fd3 to d6/d7/d16/d29/d6c/d9c/dc1/f140 0 2024-05-17T21:53:19.707 INFO:tasks.workunit.client.1.smithi138.stdout:5/970: getdents d1/d60/d2f/d5e 0 2024-05-17T21:53:19.709 INFO:tasks.workunit.client.1.smithi138.stdout:9/993: symlink d6/d7/d16/d29/d38/l141 0 2024-05-17T21:53:19.713 INFO:tasks.workunit.client.1.smithi138.stdout:7/999: write d0/d18/d40/d9a/fb8 [3816891,12587] 0 2024-05-17T21:53:19.714 INFO:tasks.workunit.client.1.smithi138.stdout:9/994: truncate d6/d7/d16/d29/d48/d6e/d74/d75/d76/d8e/ddd/f10f 766528 0 2024-05-17T21:53:19.722 INFO:tasks.workunit.client.1.smithi138.stdout:9/995: dwrite d6/d7/d16/d29/d48/d6e/f126 [0,4194304] 0 2024-05-17T21:53:19.722 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:19 smithi138 ceph-mon[35948]: Standby manager daemon smithi107.xhhyiw restarted 2024-05-17T21:53:19.723 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:19 smithi138 ceph-mon[35948]: Standby manager daemon smithi107.xhhyiw started 2024-05-17T21:53:19.723 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:19 smithi138 ceph-mon[35948]: from='mgr.? 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi107.xhhyiw/crt"}]: dispatch 2024-05-17T21:53:19.723 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:19 smithi138 ceph-mon[35948]: from='mgr.? 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-17T21:53:19.723 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:19 smithi138 ceph-mon[35948]: from='mgr.? 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi107.xhhyiw/key"}]: dispatch 2024-05-17T21:53:19.723 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:19 smithi138 ceph-mon[35948]: from='mgr.? 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-17T21:53:19.723 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:19 smithi138 ceph-mon[35948]: from='mgr.14636 172.21.15.138:0/3120106723' entity='mgr.smithi138.napnll' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:53:19.726 INFO:tasks.workunit.client.1.smithi138.stdout:9/996: creat d6/d7/d16/d29/d6c/d9c/f142 x:0 0 0 2024-05-17T21:53:19.730 INFO:tasks.workunit.client.1.smithi138.stdout:5/971: dread d1/d2c/d63/f6d [0,4194304] 0 2024-05-17T21:53:19.731 INFO:tasks.workunit.client.1.smithi138.stdout:5/972: read - d1/d2c/d63/db5/d1b/db7/f127 zero size 2024-05-17T21:53:19.736 INFO:tasks.workunit.client.1.smithi138.stdout:5/973: rmdir d1/d60/d2f/d5e/de4 39 2024-05-17T21:53:19.745 INFO:tasks.workunit.client.1.smithi138.stdout:9/997: dread d6/d7/d16/d29/d38/f73 [0,4194304] 0 2024-05-17T21:53:19.749 INFO:tasks.workunit.client.1.smithi138.stdout:9/998: mkdir d6/de4/d143 0 2024-05-17T21:53:19.750 INFO:tasks.workunit.client.1.smithi138.stdout:9/999: chown d6/d7/d16/d29/d48/d6e/d74/d75/d76/de8/d108/l62 697 1 2024-05-17T21:53:19.752 INFO:tasks.workunit.client.1.smithi138.stdout:5/974: write d1/d2c/d63/db5/d1b/f94 [1221994,130369] 0 2024-05-17T21:53:19.752 INFO:tasks.workunit.client.1.smithi138.stdout:5/975: stat d1/d53/d88/fd9 0 2024-05-17T21:53:19.764 INFO:tasks.workunit.client.1.smithi138.stdout:5/976: dread d1/d53/d96/fc6 [0,4194304] 0 2024-05-17T21:53:19.769 INFO:tasks.workunit.client.1.smithi138.stdout:5/977: creat d1/d53/d96/f137 x:0 0 0 2024-05-17T21:53:19.770 INFO:tasks.workunit.client.1.smithi138.stdout:5/978: stat d1/d53/d96/da0/fa1 0 2024-05-17T21:53:19.773 INFO:tasks.workunit.client.1.smithi138.stdout:5/979: stat d1/d46/d95/fae 0 2024-05-17T21:53:19.780 INFO:tasks.workunit.client.1.smithi138.stdout:5/980: dwrite d1/d2c/d63/db5/d73/f123 [0,4194304] 0 2024-05-17T21:53:19.784 INFO:tasks.workunit.client.1.smithi138.stdout:5/981: creat d1/d53/d88/dc9/d106/f138 x:0 0 0 2024-05-17T21:53:19.788 INFO:tasks.workunit.client.1.smithi138.stdout:5/982: truncate d1/d2c/dbf/fd3 4920913 0 2024-05-17T21:53:19.791 INFO:tasks.workunit.client.1.smithi138.stdout:5/983: unlink d1/d53/ff6 0 2024-05-17T21:53:19.807 INFO:tasks.workunit.client.1.smithi138.stdout:5/984: dwrite d1/d60/fcb [4194304,4194304] 0 2024-05-17T21:53:19.817 INFO:tasks.workunit.client.1.smithi138.stdout:5/985: dwrite d1/d2c/d63/db5/f41 [4194304,4194304] 0 2024-05-17T21:53:19.847 INFO:tasks.workunit.client.1.smithi138.stdout:5/986: dread d1/d60/d2f/d5e/de4/fac [0,4194304] 0 2024-05-17T21:53:19.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:19 smithi107 ceph-mon[28434]: Standby manager daemon smithi107.xhhyiw restarted 2024-05-17T21:53:19.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:19 smithi107 ceph-mon[28434]: Standby manager daemon smithi107.xhhyiw started 2024-05-17T21:53:19.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:19 smithi107 ceph-mon[28434]: from='mgr.? 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi107.xhhyiw/crt"}]: dispatch 2024-05-17T21:53:19.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:19 smithi107 ceph-mon[28434]: from='mgr.? 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-17T21:53:19.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:19 smithi107 ceph-mon[28434]: from='mgr.? 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi107.xhhyiw/key"}]: dispatch 2024-05-17T21:53:19.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:19 smithi107 ceph-mon[28434]: from='mgr.? 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-17T21:53:19.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:19 smithi107 ceph-mon[28434]: from='mgr.14636 172.21.15.138:0/3120106723' entity='mgr.smithi138.napnll' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:53:19.854 INFO:tasks.workunit.client.1.smithi138.stdout:5/987: creat d1/dd1/dca/dd0/f139 x:0 0 0 2024-05-17T21:53:19.897 INFO:tasks.workunit.client.1.smithi138.stdout:5/988: mknod d1/d46/d95/c13a 0 2024-05-17T21:53:19.902 INFO:tasks.workunit.client.1.smithi138.stdout:5/989: link d1/d53/d88/db9/d39/d4e/d58/f11d d1/d2c/d63/db5/d1b/dc5/d36/d68/d12b/f13b 0 2024-05-17T21:53:19.909 INFO:tasks.workunit.client.1.smithi138.stdout:5/990: link d1/d53/f80 d1/d8a/df0/f13c 0 2024-05-17T21:53:19.932 INFO:tasks.workunit.client.1.smithi138.stdout:5/991: dwrite d1/d2c/f52 [0,4194304] 0 2024-05-17T21:53:19.939 INFO:tasks.workunit.client.1.smithi138.stdout:5/992: link d1/d2c/d63/f9d d1/d2c/d63/db5/f13d 0 2024-05-17T21:53:19.942 INFO:tasks.workunit.client.1.smithi138.stdout:5/993: creat d1/d60/d2f/d5e/f13e x:0 0 0 2024-05-17T21:53:19.946 INFO:tasks.workunit.client.1.smithi138.stdout:5/994: creat d1/d2c/d63/db5/f13f x:0 0 0 2024-05-17T21:53:19.949 INFO:tasks.workunit.client.1.smithi138.stdout:5/995: creat d1/d53/d88/db9/d39/d4e/d58/d5b/f140 x:0 0 0 2024-05-17T21:53:19.953 INFO:tasks.workunit.client.1.smithi138.stdout:5/996: link d1/d60/d2f/d5e/de4/ccd d1/d53/d88/de5/ded/dc2/c141 0 2024-05-17T21:53:19.956 INFO:tasks.workunit.client.1.smithi138.stdout:5/997: creat d1/dd1/dca/f142 x:0 0 0 2024-05-17T21:53:19.957 INFO:tasks.workunit.client.1.smithi138.stdout:5/998: write d1/d2c/d63/db5/d19/f115 [889020,87260] 0 2024-05-17T21:53:19.961 INFO:tasks.workunit.client.1.smithi138.stdout:5/999: dread d1/d2c/f52 [0,4194304] 0 2024-05-17T21:53:19.965 INFO:tasks.workunit.client.1.smithi138.stderr:+ rm -rf -- ./tmp.T32dW2XgB2 2024-05-17T21:53:20.493 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:20 smithi107 ceph-mon[28434]: pgmap v48: 65 pgs: 65 active+clean; 3.9 GiB data, 13 GiB used, 524 GiB / 536 GiB avail; 78 MiB/s rd, 160 MiB/s wr, 402 op/s 2024-05-17T21:53:20.494 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:20 smithi107 ceph-mon[28434]: mgrmap e26: smithi138.napnll(active, since 90s), standbys: smithi107.xhhyiw 2024-05-17T21:53:20.494 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:20 smithi107 ceph-mon[28434]: from='mgr.14636 ' entity='mgr.smithi138.napnll' 2024-05-17T21:53:20.494 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:20 smithi107 ceph-mon[28434]: from='mgr.14636 ' entity='mgr.smithi138.napnll' 2024-05-17T21:53:20.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:20 smithi138 ceph-mon[35948]: pgmap v48: 65 pgs: 65 active+clean; 3.9 GiB data, 13 GiB used, 524 GiB / 536 GiB avail; 78 MiB/s rd, 160 MiB/s wr, 402 op/s 2024-05-17T21:53:20.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:20 smithi138 ceph-mon[35948]: mgrmap e26: smithi138.napnll(active, since 90s), standbys: smithi107.xhhyiw 2024-05-17T21:53:20.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:20 smithi138 ceph-mon[35948]: from='mgr.14636 ' entity='mgr.smithi138.napnll' 2024-05-17T21:53:20.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:20 smithi138 ceph-mon[35948]: from='mgr.14636 ' entity='mgr.smithi138.napnll' 2024-05-17T21:53:21.920 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:21 smithi107 ceph-mon[28434]: from='mgr.14636 ' entity='mgr.smithi138.napnll' 2024-05-17T21:53:21.920 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:21 smithi107 ceph-mon[28434]: from='mgr.14636 ' entity='mgr.smithi138.napnll' 2024-05-17T21:53:22.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:21 smithi138 ceph-mon[35948]: from='mgr.14636 ' entity='mgr.smithi138.napnll' 2024-05-17T21:53:22.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:21 smithi138 ceph-mon[35948]: from='mgr.14636 ' entity='mgr.smithi138.napnll' 2024-05-17T21:53:23.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:22 smithi138 ceph-mon[35948]: pgmap v49: 65 pgs: 65 active+clean; 3.9 GiB data, 13 GiB used, 524 GiB / 536 GiB avail; 93 MiB/s rd, 203 MiB/s wr, 532 op/s 2024-05-17T21:53:23.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:22 smithi138 ceph-mon[35948]: mgrmap e27: smithi138.napnll(active, since 92s), standbys: smithi107.xhhyiw 2024-05-17T21:53:23.098 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:22 smithi107 ceph-mon[28434]: pgmap v49: 65 pgs: 65 active+clean; 3.9 GiB data, 13 GiB used, 524 GiB / 536 GiB avail; 93 MiB/s rd, 203 MiB/s wr, 532 op/s 2024-05-17T21:53:23.099 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:22 smithi107 ceph-mon[28434]: mgrmap e27: smithi138.napnll(active, since 92s), standbys: smithi107.xhhyiw 2024-05-17T21:53:24.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:24 smithi107 ceph-mon[28434]: from='mgr.14636 ' entity='mgr.smithi138.napnll' 2024-05-17T21:53:24.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:24 smithi107 ceph-mon[28434]: from='mgr.14636 ' entity='mgr.smithi138.napnll' 2024-05-17T21:53:24.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:24 smithi107 ceph-mon[28434]: from='mgr.14636 172.21.15.138:0/3120106723' entity='mgr.smithi138.napnll' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:53:24.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:24 smithi107 ceph-mon[28434]: from='mgr.14636 172.21.15.138:0/3120106723' entity='mgr.smithi138.napnll' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T21:53:24.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:24 smithi107 ceph-mon[28434]: from='mgr.14636 ' entity='mgr.smithi138.napnll' 2024-05-17T21:53:24.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:24 smithi107 ceph-mon[28434]: from='mgr.14636 172.21.15.138:0/3120106723' entity='mgr.smithi138.napnll' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:53:24.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:24 smithi107 ceph-mon[28434]: from='mgr.14636 172.21.15.138:0/3120106723' entity='mgr.smithi138.napnll' cmd=[{"prefix": "mgr fail", "who": "smithi138.napnll"}]: dispatch 2024-05-17T21:53:24.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:24 smithi107 ceph-mon[28434]: from='mgr.14636 ' entity='mgr.smithi138.napnll' cmd=[{"prefix": "mgr fail", "who": "smithi138.napnll"}]: dispatch 2024-05-17T21:53:24.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:24 smithi107 ceph-mon[28434]: Activating manager daemon smithi107.xhhyiw 2024-05-17T21:53:24.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:24 smithi107 ceph-mon[28434]: osdmap e48: 6 total, 6 up, 6 in 2024-05-17T21:53:24.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:24 smithi107 ceph-mon[28434]: from='mgr.14636 ' entity='mgr.smithi138.napnll' cmd='[{"prefix": "mgr fail", "who": "smithi138.napnll"}]': finished 2024-05-17T21:53:24.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:24 smithi107 ceph-mon[28434]: mgrmap e28: smithi107.xhhyiw(active, starting, since 0.0101182s) 2024-05-17T21:53:24.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:24 smithi107 ceph-mon[28434]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mon metadata", "id": "smithi107"}]: dispatch 2024-05-17T21:53:24.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:24 smithi107 ceph-mon[28434]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mon metadata", "id": "smithi138"}]: dispatch 2024-05-17T21:53:24.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:24 smithi107 ceph-mon[28434]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi107.drqody"}]: dispatch 2024-05-17T21:53:24.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:24 smithi107 ceph-mon[28434]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi107.idmbgj"}]: dispatch 2024-05-17T21:53:24.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:24 smithi107 ceph-mon[28434]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi138.gwbtyd"}]: dispatch 2024-05-17T21:53:24.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:24 smithi107 ceph-mon[28434]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi138.zckrvs"}]: dispatch 2024-05-17T21:53:24.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:24 smithi107 ceph-mon[28434]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mgr metadata", "who": "smithi107.xhhyiw", "id": "smithi107.xhhyiw"}]: dispatch 2024-05-17T21:53:24.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:24 smithi107 ceph-mon[28434]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-17T21:53:24.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:24 smithi107 ceph-mon[28434]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-17T21:53:24.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:24 smithi107 ceph-mon[28434]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-17T21:53:24.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:24 smithi107 ceph-mon[28434]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-17T21:53:24.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:24 smithi107 ceph-mon[28434]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-17T21:53:24.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:24 smithi107 ceph-mon[28434]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-17T21:53:24.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:24 smithi107 ceph-mon[28434]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-17T21:53:24.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:24 smithi107 ceph-mon[28434]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-17T21:53:24.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:24 smithi107 ceph-mon[28434]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-17T21:53:24.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:24 smithi107 ceph-mon[28434]: Manager daemon smithi107.xhhyiw is now available 2024-05-17T21:53:24.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:24 smithi107 ceph-mon[28434]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:53:24.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:24 smithi107 ceph-mon[28434]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:53:24.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:24 smithi107 ceph-mon[28434]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi107.xhhyiw/mirror_snapshot_schedule"}]: dispatch 2024-05-17T21:53:24.855 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:24 smithi107 ceph-mon[28434]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi107.xhhyiw/trash_purge_schedule"}]: dispatch 2024-05-17T21:53:24.855 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:24 smithi107 ceph-mon[28434]: [17/May/2024:21:53:24] ENGINE Bus STARTING 2024-05-17T21:53:24.855 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:24 smithi107 ceph-mon[28434]: [17/May/2024:21:53:24] ENGINE Serving on https://172.21.15.107:7150 2024-05-17T21:53:24.985 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:24 smithi138 ceph-mon[35948]: from='mgr.14636 ' entity='mgr.smithi138.napnll' 2024-05-17T21:53:24.985 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:24 smithi138 ceph-mon[35948]: from='mgr.14636 ' entity='mgr.smithi138.napnll' 2024-05-17T21:53:24.985 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:24 smithi138 ceph-mon[35948]: from='mgr.14636 172.21.15.138:0/3120106723' entity='mgr.smithi138.napnll' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:53:24.985 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:24 smithi138 ceph-mon[35948]: from='mgr.14636 172.21.15.138:0/3120106723' entity='mgr.smithi138.napnll' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T21:53:24.985 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:24 smithi138 ceph-mon[35948]: from='mgr.14636 ' entity='mgr.smithi138.napnll' 2024-05-17T21:53:24.985 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:24 smithi138 ceph-mon[35948]: from='mgr.14636 172.21.15.138:0/3120106723' entity='mgr.smithi138.napnll' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:53:24.986 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:24 smithi138 ceph-mon[35948]: from='mgr.14636 172.21.15.138:0/3120106723' entity='mgr.smithi138.napnll' cmd=[{"prefix": "mgr fail", "who": "smithi138.napnll"}]: dispatch 2024-05-17T21:53:24.986 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:24 smithi138 ceph-mon[35948]: from='mgr.14636 ' entity='mgr.smithi138.napnll' cmd=[{"prefix": "mgr fail", "who": "smithi138.napnll"}]: dispatch 2024-05-17T21:53:24.986 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:24 smithi138 ceph-mon[35948]: Activating manager daemon smithi107.xhhyiw 2024-05-17T21:53:24.986 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:24 smithi138 ceph-mon[35948]: osdmap e48: 6 total, 6 up, 6 in 2024-05-17T21:53:24.986 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:24 smithi138 ceph-mon[35948]: from='mgr.14636 ' entity='mgr.smithi138.napnll' cmd='[{"prefix": "mgr fail", "who": "smithi138.napnll"}]': finished 2024-05-17T21:53:24.986 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:24 smithi138 ceph-mon[35948]: mgrmap e28: smithi107.xhhyiw(active, starting, since 0.0101182s) 2024-05-17T21:53:24.986 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:24 smithi138 ceph-mon[35948]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mon metadata", "id": "smithi107"}]: dispatch 2024-05-17T21:53:24.986 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:24 smithi138 ceph-mon[35948]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mon metadata", "id": "smithi138"}]: dispatch 2024-05-17T21:53:24.986 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:24 smithi138 ceph-mon[35948]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi107.drqody"}]: dispatch 2024-05-17T21:53:24.986 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:24 smithi138 ceph-mon[35948]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi107.idmbgj"}]: dispatch 2024-05-17T21:53:24.987 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:24 smithi138 ceph-mon[35948]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi138.gwbtyd"}]: dispatch 2024-05-17T21:53:24.987 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:24 smithi138 ceph-mon[35948]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi138.zckrvs"}]: dispatch 2024-05-17T21:53:24.987 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:24 smithi138 ceph-mon[35948]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mgr metadata", "who": "smithi107.xhhyiw", "id": "smithi107.xhhyiw"}]: dispatch 2024-05-17T21:53:24.987 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:24 smithi138 ceph-mon[35948]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-17T21:53:24.987 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:24 smithi138 ceph-mon[35948]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-17T21:53:24.987 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:24 smithi138 ceph-mon[35948]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-17T21:53:24.987 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:24 smithi138 ceph-mon[35948]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-17T21:53:24.988 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:24 smithi138 ceph-mon[35948]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-17T21:53:24.988 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:24 smithi138 ceph-mon[35948]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-17T21:53:24.988 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:24 smithi138 ceph-mon[35948]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-17T21:53:24.988 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:24 smithi138 ceph-mon[35948]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-17T21:53:24.988 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:24 smithi138 ceph-mon[35948]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-17T21:53:24.988 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:24 smithi138 ceph-mon[35948]: Manager daemon smithi107.xhhyiw is now available 2024-05-17T21:53:24.988 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:24 smithi138 ceph-mon[35948]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:53:24.988 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:24 smithi138 ceph-mon[35948]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:53:24.988 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:24 smithi138 ceph-mon[35948]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi107.xhhyiw/mirror_snapshot_schedule"}]: dispatch 2024-05-17T21:53:24.988 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:24 smithi138 ceph-mon[35948]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi107.xhhyiw/trash_purge_schedule"}]: dispatch 2024-05-17T21:53:24.988 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:24 smithi138 ceph-mon[35948]: [17/May/2024:21:53:24] ENGINE Bus STARTING 2024-05-17T21:53:24.988 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:24 smithi138 ceph-mon[35948]: [17/May/2024:21:53:24] ENGINE Serving on https://172.21.15.107:7150 2024-05-17T21:53:26.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:25 smithi138 ceph-mon[35948]: [17/May/2024:21:53:24] ENGINE Serving on http://172.21.15.107:8765 2024-05-17T21:53:26.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:25 smithi138 ceph-mon[35948]: [17/May/2024:21:53:24] ENGINE Bus STARTED 2024-05-17T21:53:26.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:25 smithi138 ceph-mon[35948]: mgrmap e29: smithi107.xhhyiw(active, since 1.03767s) 2024-05-17T21:53:26.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:25 smithi138 ceph-mon[35948]: pgmap v3: 65 pgs: 65 active+clean; 3.1 GiB data, 11 GiB used, 525 GiB / 536 GiB avail 2024-05-17T21:53:26.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:25 smithi107 ceph-mon[28434]: [17/May/2024:21:53:24] ENGINE Serving on http://172.21.15.107:8765 2024-05-17T21:53:26.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:25 smithi107 ceph-mon[28434]: [17/May/2024:21:53:24] ENGINE Bus STARTED 2024-05-17T21:53:26.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:25 smithi107 ceph-mon[28434]: mgrmap e29: smithi107.xhhyiw(active, since 1.03767s) 2024-05-17T21:53:26.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:25 smithi107 ceph-mon[28434]: pgmap v3: 65 pgs: 65 active+clean; 3.1 GiB data, 11 GiB used, 525 GiB / 536 GiB avail 2024-05-17T21:53:27.040 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:26 smithi138 ceph-mon[35948]: pgmap v4: 65 pgs: 65 active+clean; 3.1 GiB data, 11 GiB used, 525 GiB / 536 GiB avail 2024-05-17T21:53:27.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:26 smithi107 ceph-mon[28434]: pgmap v4: 65 pgs: 65 active+clean; 3.1 GiB data, 11 GiB used, 525 GiB / 536 GiB avail 2024-05-17T21:53:28.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:27 smithi138 ceph-mon[35948]: mgrmap e30: smithi107.xhhyiw(active, since 3s) 2024-05-17T21:53:28.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:27 smithi138 ceph-mon[35948]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:53:28.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:27 smithi138 ceph-mon[35948]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:53:28.083 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:27 smithi107 ceph-mon[28434]: mgrmap e30: smithi107.xhhyiw(active, since 3s) 2024-05-17T21:53:28.083 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:27 smithi107 ceph-mon[28434]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:53:28.083 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:27 smithi107 ceph-mon[28434]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:53:29.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:28 smithi138 ceph-mon[35948]: pgmap v5: 65 pgs: 65 active+clean; 3.1 GiB data, 11 GiB used, 525 GiB / 536 GiB avail 2024-05-17T21:53:29.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:28 smithi107 ceph-mon[28434]: pgmap v5: 65 pgs: 65 active+clean; 3.1 GiB data, 11 GiB used, 525 GiB / 536 GiB avail 2024-05-17T21:53:30.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:30 smithi138 ceph-mon[35948]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:53:30.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:30 smithi138 ceph-mon[35948]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:53:30.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:30 smithi138 ceph-mon[35948]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:53:30.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:30 smithi138 ceph-mon[35948]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:53:30.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:30 smithi107 ceph-mon[28434]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:53:30.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:30 smithi107 ceph-mon[28434]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:53:30.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:30 smithi107 ceph-mon[28434]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:53:30.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:30 smithi107 ceph-mon[28434]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:53:31.525 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:31 smithi107 ceph-mon[28434]: pgmap v6: 65 pgs: 65 active+clean; 3.1 GiB data, 11 GiB used, 525 GiB / 536 GiB avail 2024-05-17T21:53:31.525 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:31 smithi107 ceph-mon[28434]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:53:31.526 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:31 smithi107 ceph-mon[28434]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:53:31.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:31 smithi138 ceph-mon[35948]: pgmap v6: 65 pgs: 65 active+clean; 3.1 GiB data, 11 GiB used, 525 GiB / 536 GiB avail 2024-05-17T21:53:31.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:31 smithi138 ceph-mon[35948]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:53:31.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:31 smithi138 ceph-mon[35948]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:53:32.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:32.577+0000 7fc29e753700 1 -- 172.21.15.107:0/3718246080 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc298107880 msgr2=0x7fc298109c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:53:32.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:32.577+0000 7fc29e753700 1 --2- 172.21.15.107:0/3718246080 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc298107880 0x7fc298109c60 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fc28c004970 tx=0x7fc28c009e30 comp rx=0 tx=0).stop 2024-05-17T21:53:32.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:32.577+0000 7fc29e753700 1 -- 172.21.15.107:0/3718246080 shutdown_connections 2024-05-17T21:53:32.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:32.577+0000 7fc29e753700 1 --2- 172.21.15.107:0/3718246080 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc298107880 0x7fc298109c60 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:53:32.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:32.577+0000 7fc29e753700 1 --2- 172.21.15.107:0/3718246080 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc298104f60 0x7fc298107340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:53:32.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:32.577+0000 7fc29e753700 1 -- 172.21.15.107:0/3718246080 >> 172.21.15.107:0/3718246080 conn(0x7fc29806a360 msgr2=0x7fc29806c790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:53:32.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:32.577+0000 7fc29e753700 1 -- 172.21.15.107:0/3718246080 shutdown_connections 2024-05-17T21:53:32.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:32.577+0000 7fc29e753700 1 -- 172.21.15.107:0/3718246080 wait complete. 2024-05-17T21:53:32.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:32.578+0000 7fc29e753700 1 Processor -- start 2024-05-17T21:53:32.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:32.578+0000 7fc29e753700 1 -- start start 2024-05-17T21:53:32.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:32.578+0000 7fc29e753700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc298104f60 0x7fc29811c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:53:32.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:32.578+0000 7fc29e753700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc298107880 0x7fc29811ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:53:32.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:32.578+0000 7fc29e753700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc29811d000 con 0x7fc298107880 2024-05-17T21:53:32.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:32.578+0000 7fc29e753700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc29811d170 con 0x7fc298104f60 2024-05-17T21:53:32.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:32.578+0000 7fc2977fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc298107880 0x7fc29811ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:53:32.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:32.578+0000 7fc2977fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc298107880 0x7fc29811ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52202/0 (socket says 172.21.15.107:52202) 2024-05-17T21:53:32.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:32.578+0000 7fc2977fe700 1 -- 172.21.15.107:0/2076961902 learned_addr learned my addr 172.21.15.107:0/2076961902 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:53:32.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:32.578+0000 7fc297fff700 1 --2- 172.21.15.107:0/2076961902 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc298104f60 0x7fc29811c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:53:32.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:32.579+0000 7fc297fff700 1 -- 172.21.15.107:0/2076961902 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc298107880 msgr2=0x7fc29811ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:53:32.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:32.579+0000 7fc297fff700 1 --2- 172.21.15.107:0/2076961902 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc298107880 0x7fc29811ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:53:32.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:32.579+0000 7fc297fff700 1 -- 172.21.15.107:0/2076961902 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc28c005040 con 0x7fc298104f60 2024-05-17T21:53:32.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:32.579+0000 7fc2977fe700 1 --2- 172.21.15.107:0/2076961902 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc298107880 0x7fc29811ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T21:53:32.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:32.579+0000 7fc297fff700 1 --2- 172.21.15.107:0/2076961902 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc298104f60 0x7fc29811c4c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc29806e870 tx=0x7fc28800b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:53:32.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:32.579+0000 7fc294ff9700 1 -- 172.21.15.107:0/2076961902 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc288004e90 con 0x7fc298104f60 2024-05-17T21:53:32.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:32.580+0000 7fc294ff9700 1 -- 172.21.15.107:0/2076961902 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc28800f040 con 0x7fc298104f60 2024-05-17T21:53:32.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:32.580+0000 7fc294ff9700 1 -- 172.21.15.107:0/2076961902 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc28800ca20 con 0x7fc298104f60 2024-05-17T21:53:32.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:32.580+0000 7fc29e753700 1 -- 172.21.15.107:0/2076961902 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc298121b70 con 0x7fc298104f60 2024-05-17T21:53:32.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:32.580+0000 7fc29e753700 1 -- 172.21.15.107:0/2076961902 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2981220b0 con 0x7fc298104f60 2024-05-17T21:53:32.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:32.581+0000 7fc29e753700 1 -- 172.21.15.107:0/2076961902 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc29804c710 con 0x7fc298104f60 2024-05-17T21:53:32.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:32.582+0000 7fc294ff9700 1 -- 172.21.15.107:0/2076961902 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 30) v1 ==== 50232+0+0 (secure 0 0 0) 0x7fc28800cb80 con 0x7fc298104f60 2024-05-17T21:53:32.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:32.582+0000 7fc294ff9700 1 --2- 172.21.15.107:0/2076961902 >> [v2:172.21.15.107:6800/416442229,v1:172.21.15.107:6801/416442229] conn(0x7fc28003e370 0x7fc280040820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:53:32.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:32.582+0000 7fc294ff9700 1 -- 172.21.15.107:0/2076961902 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fc288052c00 con 0x7fc298104f60 2024-05-17T21:53:32.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:32.582+0000 7fc2977fe700 1 --2- 172.21.15.107:0/2076961902 >> [v2:172.21.15.107:6800/416442229,v1:172.21.15.107:6801/416442229] conn(0x7fc28003e370 0x7fc280040820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:53:32.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:32.583+0000 7fc2977fe700 1 --2- 172.21.15.107:0/2076961902 >> [v2:172.21.15.107:6800/416442229,v1:172.21.15.107:6801/416442229] conn(0x7fc28003e370 0x7fc280040820 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc28c00b8c0 tx=0x7fc28c009e30 comp rx=0 tx=0).ready entity=mgr.14750 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:53:32.584 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:32.591+0000 7fc294ff9700 1 -- 172.21.15.107:0/2076961902 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+190121 (secure 0 0 0) 0x7fc28801b2c0 con 0x7fc298104f60 2024-05-17T21:53:32.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:32.864+0000 7fc29e753700 1 -- 172.21.15.107:0/2076961902 --> [v2:172.21.15.107:6800/416442229,v1:172.21.15.107:6801/416442229] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc2981083b0 con 0x7fc28003e370 2024-05-17T21:53:32.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:32.866+0000 7fc294ff9700 1 -- 172.21.15.107:0/2076961902 <== mgr.14750 v2:172.21.15.107:6800/416442229 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7fc2981083b0 con 0x7fc28003e370 2024-05-17T21:53:32.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:32.869+0000 7fc29e753700 1 -- 172.21.15.107:0/2076961902 >> [v2:172.21.15.107:6800/416442229,v1:172.21.15.107:6801/416442229] conn(0x7fc28003e370 msgr2=0x7fc280040820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:53:32.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:32.869+0000 7fc29e753700 1 --2- 172.21.15.107:0/2076961902 >> [v2:172.21.15.107:6800/416442229,v1:172.21.15.107:6801/416442229] conn(0x7fc28003e370 0x7fc280040820 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc28c00b8c0 tx=0x7fc28c009e30 comp rx=0 tx=0).stop 2024-05-17T21:53:32.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:32.869+0000 7fc29e753700 1 -- 172.21.15.107:0/2076961902 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc298104f60 msgr2=0x7fc29811c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:53:32.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:32.869+0000 7fc29e753700 1 --2- 172.21.15.107:0/2076961902 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc298104f60 0x7fc29811c4c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc29806e870 tx=0x7fc28800b040 comp rx=0 tx=0).stop 2024-05-17T21:53:32.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:32.869+0000 7fc29e753700 1 -- 172.21.15.107:0/2076961902 shutdown_connections 2024-05-17T21:53:32.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:32.869+0000 7fc29e753700 1 --2- 172.21.15.107:0/2076961902 >> [v2:172.21.15.107:6800/416442229,v1:172.21.15.107:6801/416442229] conn(0x7fc28003e370 0x7fc280040820 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:53:32.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:32.869+0000 7fc29e753700 1 --2- 172.21.15.107:0/2076961902 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc298104f60 0x7fc29811c4c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:53:32.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:32.869+0000 7fc29e753700 1 --2- 172.21.15.107:0/2076961902 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc298107880 0x7fc29811ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:53:32.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:32.869+0000 7fc29e753700 1 -- 172.21.15.107:0/2076961902 >> 172.21.15.107:0/2076961902 conn(0x7fc29806a360 msgr2=0x7fc298107ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:53:32.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:32.870+0000 7fc29e753700 1 -- 172.21.15.107:0/2076961902 shutdown_connections 2024-05-17T21:53:32.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:32.870+0000 7fc29e753700 1 -- 172.21.15.107:0/2076961902 wait complete. 2024-05-17T21:53:32.872 INFO:teuthology.orchestra.run.smithi107.stdout:true 2024-05-17T21:53:32.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:32.984+0000 7fef7342c700 1 -- 172.21.15.107:0/1739799984 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef6c066260 msgr2=0x7fef6c0ffa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:53:32.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:32.984+0000 7fef7342c700 1 --2- 172.21.15.107:0/1739799984 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef6c066260 0x7fef6c0ffa00 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fef5c004940 tx=0x7fef5c009e30 comp rx=0 tx=0).stop 2024-05-17T21:53:32.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:32.984+0000 7fef7342c700 1 -- 172.21.15.107:0/1739799984 shutdown_connections 2024-05-17T21:53:32.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:32.984+0000 7fef7342c700 1 --2- 172.21.15.107:0/1739799984 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef6c0fff40 0x7fef6c102320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:53:32.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:32.984+0000 7fef7342c700 1 --2- 172.21.15.107:0/1739799984 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef6c066260 0x7fef6c0ffa00 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:53:32.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:32.984+0000 7fef7342c700 1 -- 172.21.15.107:0/1739799984 >> 172.21.15.107:0/1739799984 conn(0x7fef6c0f7430 msgr2=0x7fef6c0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:53:32.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:32.984+0000 7fef7342c700 1 -- 172.21.15.107:0/1739799984 shutdown_connections 2024-05-17T21:53:32.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:32.984+0000 7fef7342c700 1 -- 172.21.15.107:0/1739799984 wait complete. 2024-05-17T21:53:32.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:32.985+0000 7fef7342c700 1 Processor -- start 2024-05-17T21:53:32.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:32.985+0000 7fef7342c700 1 -- start start 2024-05-17T21:53:32.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:32.985+0000 7fef7342c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef6c066260 0x7fef6c0fd430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:53:32.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:32.985+0000 7fef7342c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef6c0fff40 0x7fef6c0fb900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:53:32.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:32.985+0000 7fef7342c700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef6c0fbeb0 con 0x7fef6c066260 2024-05-17T21:53:32.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:32.985+0000 7fef7342c700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef6c0fc020 con 0x7fef6c0fff40 2024-05-17T21:53:32.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:32.985+0000 7fef709c7700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef6c0fff40 0x7fef6c0fb900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:53:32.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:32.985+0000 7fef711c8700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef6c066260 0x7fef6c0fd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:53:32.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:32.985+0000 7fef709c7700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef6c0fff40 0x7fef6c0fb900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.107:35316/0 (socket says 172.21.15.107:35316) 2024-05-17T21:53:32.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:32.985+0000 7fef711c8700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef6c066260 0x7fef6c0fd430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52218/0 (socket says 172.21.15.107:52218) 2024-05-17T21:53:32.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:32.985+0000 7fef709c7700 1 -- 172.21.15.107:0/4099933760 learned_addr learned my addr 172.21.15.107:0/4099933760 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:53:32.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:32.986+0000 7fef711c8700 1 -- 172.21.15.107:0/4099933760 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef6c0fff40 msgr2=0x7fef6c0fb900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:53:32.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:32.986+0000 7fef711c8700 1 --2- 172.21.15.107:0/4099933760 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef6c0fff40 0x7fef6c0fb900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:53:32.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:32.986+0000 7fef711c8700 1 -- 172.21.15.107:0/4099933760 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef5c005040 con 0x7fef6c066260 2024-05-17T21:53:32.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:32.986+0000 7fef711c8700 1 --2- 172.21.15.107:0/4099933760 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef6c066260 0x7fef6c0fd430 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fef5c000c00 tx=0x7fef5c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:53:32.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:32.986+0000 7fef61ffb700 1 -- 172.21.15.107:0/4099933760 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fef5c00e040 con 0x7fef6c066260 2024-05-17T21:53:32.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:32.986+0000 7fef61ffb700 1 -- 172.21.15.107:0/4099933760 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fef5c00bbf0 con 0x7fef6c066260 2024-05-17T21:53:32.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:32.986+0000 7fef7342c700 1 -- 172.21.15.107:0/4099933760 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef6c0fc2a0 con 0x7fef6c066260 2024-05-17T21:53:32.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:32.986+0000 7fef61ffb700 1 -- 172.21.15.107:0/4099933760 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fef5c01c650 con 0x7fef6c066260 2024-05-17T21:53:32.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:32.986+0000 7fef7342c700 1 -- 172.21.15.107:0/4099933760 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef6c115a60 con 0x7fef6c066260 2024-05-17T21:53:32.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:32.987+0000 7fef7342c700 1 -- 172.21.15.107:0/4099933760 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef6c05f5c0 con 0x7fef6c066260 2024-05-17T21:53:32.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:32.987+0000 7fef61ffb700 1 -- 172.21.15.107:0/4099933760 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 30) v1 ==== 50232+0+0 (secure 0 0 0) 0x7fef5c01c8c0 con 0x7fef6c066260 2024-05-17T21:53:32.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:32.988+0000 7fef61ffb700 1 --2- 172.21.15.107:0/4099933760 >> [v2:172.21.15.107:6800/416442229,v1:172.21.15.107:6801/416442229] conn(0x7fef5803e330 0x7fef580407e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:53:32.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:32.988+0000 7fef61ffb700 1 -- 172.21.15.107:0/4099933760 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fef5c054530 con 0x7fef6c066260 2024-05-17T21:53:32.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:32.988+0000 7fef709c7700 1 --2- 172.21.15.107:0/4099933760 >> [v2:172.21.15.107:6800/416442229,v1:172.21.15.107:6801/416442229] conn(0x7fef5803e330 0x7fef580407e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:53:32.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:32.989+0000 7fef709c7700 1 --2- 172.21.15.107:0/4099933760 >> [v2:172.21.15.107:6800/416442229,v1:172.21.15.107:6801/416442229] conn(0x7fef5803e330 0x7fef580407e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fef680041d0 tx=0x7fef6800b040 comp rx=0 tx=0).ready entity=mgr.14750 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:53:32.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:32.995+0000 7fef61ffb700 1 -- 172.21.15.107:0/4099933760 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+190121 (secure 0 0 0) 0x7fef5c011e80 con 0x7fef6c066260 2024-05-17T21:53:33.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:33.241+0000 7fef7342c700 1 -- 172.21.15.107:0/4099933760 --> [v2:172.21.15.107:6800/416442229,v1:172.21.15.107:6801/416442229] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fef6c0f8f20 con 0x7fef5803e330 2024-05-17T21:53:33.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:33.242+0000 7fef61ffb700 1 -- 172.21.15.107:0/4099933760 <== mgr.14750 v2:172.21.15.107:6800/416442229 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7fef6c0f8f20 con 0x7fef5803e330 2024-05-17T21:53:33.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:33.244+0000 7fef7342c700 1 -- 172.21.15.107:0/4099933760 >> [v2:172.21.15.107:6800/416442229,v1:172.21.15.107:6801/416442229] conn(0x7fef5803e330 msgr2=0x7fef580407e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:53:33.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:33.244+0000 7fef7342c700 1 --2- 172.21.15.107:0/4099933760 >> [v2:172.21.15.107:6800/416442229,v1:172.21.15.107:6801/416442229] conn(0x7fef5803e330 0x7fef580407e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fef680041d0 tx=0x7fef6800b040 comp rx=0 tx=0).stop 2024-05-17T21:53:33.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:33.244+0000 7fef7342c700 1 -- 172.21.15.107:0/4099933760 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef6c066260 msgr2=0x7fef6c0fd430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:53:33.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:33.244+0000 7fef7342c700 1 --2- 172.21.15.107:0/4099933760 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef6c066260 0x7fef6c0fd430 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fef5c000c00 tx=0x7fef5c009e30 comp rx=0 tx=0).stop 2024-05-17T21:53:33.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:33.245+0000 7fef7342c700 1 -- 172.21.15.107:0/4099933760 shutdown_connections 2024-05-17T21:53:33.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:33.245+0000 7fef7342c700 1 --2- 172.21.15.107:0/4099933760 >> [v2:172.21.15.107:6800/416442229,v1:172.21.15.107:6801/416442229] conn(0x7fef5803e330 0x7fef580407e0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:53:33.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:33.245+0000 7fef7342c700 1 --2- 172.21.15.107:0/4099933760 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef6c066260 0x7fef6c0fd430 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:53:33.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:33.245+0000 7fef7342c700 1 --2- 172.21.15.107:0/4099933760 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef6c0fff40 0x7fef6c0fb900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:53:33.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:33.245+0000 7fef7342c700 1 -- 172.21.15.107:0/4099933760 >> 172.21.15.107:0/4099933760 conn(0x7fef6c0f7430 msgr2=0x7fef6c0f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:53:33.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:33.245+0000 7fef7342c700 1 -- 172.21.15.107:0/4099933760 shutdown_connections 2024-05-17T21:53:33.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:33.245+0000 7fef7342c700 1 -- 172.21.15.107:0/4099933760 wait complete. 2024-05-17T21:53:33.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:33.352+0000 7f20b92dc700 1 -- 172.21.15.107:0/2257672935 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20b40fcf60 msgr2=0x7f20b40fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:53:33.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:33.352+0000 7f20b92dc700 1 --2- 172.21.15.107:0/2257672935 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20b40fcf60 0x7f20b40fd3d0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f209c004940 tx=0x7f209c009e30 comp rx=0 tx=0).stop 2024-05-17T21:53:33.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:33.352+0000 7f20b92dc700 1 -- 172.21.15.107:0/2257672935 shutdown_connections 2024-05-17T21:53:33.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:33.352+0000 7f20b92dc700 1 --2- 172.21.15.107:0/2257672935 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20b40fcf60 0x7f20b40fd3d0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:53:33.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:33.352+0000 7f20b92dc700 1 --2- 172.21.15.107:0/2257672935 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f20b40fbcb0 0x7f20b40fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:53:33.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:33.352+0000 7f20b92dc700 1 -- 172.21.15.107:0/2257672935 >> 172.21.15.107:0/2257672935 conn(0x7f20b40f7430 msgr2=0x7f20b40f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:53:33.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:33.352+0000 7f20b92dc700 1 -- 172.21.15.107:0/2257672935 shutdown_connections 2024-05-17T21:53:33.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:33.352+0000 7f20b92dc700 1 -- 172.21.15.107:0/2257672935 wait complete. 2024-05-17T21:53:33.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:33.353+0000 7f20b92dc700 1 Processor -- start 2024-05-17T21:53:33.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:33.353+0000 7f20b92dc700 1 -- start start 2024-05-17T21:53:33.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:33.353+0000 7f20b92dc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20b40fbcb0 0x7f20b40ffdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:53:33.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:33.353+0000 7f20b92dc700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f20b40fcf60 0x7f20b40fe2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:53:33.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:33.353+0000 7f20b92dc700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20b40fe850 con 0x7f20b40fbcb0 2024-05-17T21:53:33.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:33.353+0000 7f20b92dc700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20b40fe9c0 con 0x7f20b40fcf60 2024-05-17T21:53:33.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:33.353+0000 7f20b2ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20b40fbcb0 0x7f20b40ffdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:53:33.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:33.353+0000 7f20b2ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20b40fbcb0 0x7f20b40ffdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52244/0 (socket says 172.21.15.107:52244) 2024-05-17T21:53:33.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:33.353+0000 7f20b2ffd700 1 -- 172.21.15.107:0/2517804438 learned_addr learned my addr 172.21.15.107:0/2517804438 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:53:33.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:33.353+0000 7f20b27fc700 1 --2- 172.21.15.107:0/2517804438 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f20b40fcf60 0x7f20b40fe2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:53:33.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:33.354+0000 7f20b2ffd700 1 -- 172.21.15.107:0/2517804438 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f20b40fcf60 msgr2=0x7f20b40fe2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:53:33.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:33.354+0000 7f20b2ffd700 1 --2- 172.21.15.107:0/2517804438 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f20b40fcf60 0x7f20b40fe2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:53:33.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:33.354+0000 7f20b2ffd700 1 -- 172.21.15.107:0/2517804438 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f209c005040 con 0x7f20b40fbcb0 2024-05-17T21:53:33.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:33.354+0000 7f20b2ffd700 1 --2- 172.21.15.107:0/2517804438 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20b40fbcb0 0x7f20b40ffdd0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f20a400eb40 tx=0x7f20a400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:53:33.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:33.354+0000 7f20ab7fe700 1 -- 172.21.15.107:0/2517804438 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f20a4007cc0 con 0x7f20b40fbcb0 2024-05-17T21:53:33.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:33.354+0000 7f20ab7fe700 1 -- 172.21.15.107:0/2517804438 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f20a400f040 con 0x7f20b40fbcb0 2024-05-17T21:53:33.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:33.354+0000 7f20b92dc700 1 -- 172.21.15.107:0/2517804438 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20b40fec40 con 0x7f20b40fbcb0 2024-05-17T21:53:33.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:33.354+0000 7f20ab7fe700 1 -- 172.21.15.107:0/2517804438 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f20a400ca10 con 0x7f20b40fbcb0 2024-05-17T21:53:33.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:33.354+0000 7f20b92dc700 1 -- 172.21.15.107:0/2517804438 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20b41159f0 con 0x7f20b40fbcb0 2024-05-17T21:53:33.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:33.355+0000 7f20ab7fe700 1 -- 172.21.15.107:0/2517804438 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 30) v1 ==== 50232+0+0 (secure 0 0 0) 0x7f20a400cc80 con 0x7f20b40fbcb0 2024-05-17T21:53:33.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:33.355+0000 7f20ab7fe700 1 --2- 172.21.15.107:0/2517804438 >> [v2:172.21.15.107:6800/416442229,v1:172.21.15.107:6801/416442229] conn(0x7f20a003e330 0x7f20a00407e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:53:33.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:33.355+0000 7f20ab7fe700 1 -- 172.21.15.107:0/2517804438 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f20a4057170 con 0x7f20b40fbcb0 2024-05-17T21:53:33.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:33.356+0000 7f20b27fc700 1 --2- 172.21.15.107:0/2517804438 >> [v2:172.21.15.107:6800/416442229,v1:172.21.15.107:6801/416442229] conn(0x7f20a003e330 0x7f20a00407e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:53:33.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:33.356+0000 7f20b92dc700 1 -- 172.21.15.107:0/2517804438 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20b405f5c0 con 0x7f20b40fbcb0 2024-05-17T21:53:33.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:33.356+0000 7f20b27fc700 1 --2- 172.21.15.107:0/2517804438 >> [v2:172.21.15.107:6800/416442229,v1:172.21.15.107:6801/416442229] conn(0x7f20a003e330 0x7f20a00407e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f209c009b30 tx=0x7f209c009e30 comp rx=0 tx=0).ready entity=mgr.14750 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:53:33.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:33 smithi107 ceph-mon[28434]: pgmap v7: 65 pgs: 65 active+clean; 2.1 GiB data, 8.1 GiB used, 528 GiB / 536 GiB avail; 956 KiB/s rd, 894 KiB/s wr, 133 op/s 2024-05-17T21:53:33.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:33.361+0000 7f20ab7fe700 1 -- 172.21.15.107:0/2517804438 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+190121 (secure 0 0 0) 0x7f20a4019070 con 0x7f20b40fbcb0 2024-05-17T21:53:33.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:33 smithi138 ceph-mon[35948]: pgmap v7: 65 pgs: 65 active+clean; 2.1 GiB data, 8.1 GiB used, 528 GiB / 536 GiB avail; 956 KiB/s rd, 894 KiB/s wr, 133 op/s 2024-05-17T21:53:33.657 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:33.664+0000 7f20b92dc700 1 -- 172.21.15.107:0/2517804438 --> [v2:172.21.15.107:6800/416442229,v1:172.21.15.107:6801/416442229] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f20b405e3a0 con 0x7f20a003e330 2024-05-17T21:53:33.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:33.674+0000 7f20ab7fe700 1 -- 172.21.15.107:0/2517804438 <== mgr.14750 v2:172.21.15.107:6800/416442229 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f20b405e3a0 con 0x7f20a003e330 2024-05-17T21:53:33.667 INFO:teuthology.orchestra.run.smithi107.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-17T21:53:33.667 INFO:teuthology.orchestra.run.smithi107.stdout:alertmanager.smithi107 smithi107 *:9093,9094 running (67s) 4s ago 9m 17.2M - 0.25.0 c8568f914cd2 bac2a944ae42 2024-05-17T21:53:33.667 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi107 smithi107 running (9m) 4s ago 9m 6660k - 17.2.7-952-g9903f776 d15a36853796 828741c3a89c 2024-05-17T21:53:33.667 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi138 smithi138 running (8m) 5s ago 8m 6660k - 17.2.7-952-g9903f776 d15a36853796 20e9ccf1fadd 2024-05-17T21:53:33.667 INFO:teuthology.orchestra.run.smithi107.stdout:grafana.smithi107 smithi107 *:3000 running (61s) 4s ago 9m 79.5M - 9.4.7 954c08fa6188 8c747e777bc5 2024-05-17T21:53:33.667 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.drqody smithi107 running (4m) 4s ago 4m 261M - 17.2.7-952-g9903f776 d15a36853796 0d851bc912f0 2024-05-17T21:53:33.667 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.idmbgj smithi107 running (4m) 4s ago 4m 12.4M - 17.2.7-952-g9903f776 d15a36853796 1348b09ab29e 2024-05-17T21:53:33.667 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi138.gwbtyd smithi138 running (4m) 5s ago 4m 252M - 17.2.7-952-g9903f776 d15a36853796 fd2ba75354c0 2024-05-17T21:53:33.668 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi138.zckrvs smithi138 running (4m) 5s ago 4m 13.1M - 17.2.7-952-g9903f776 d15a36853796 a1cfd2fe4cda 2024-05-17T21:53:33.668 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi107.xhhyiw smithi107 *:8443,9283,8765 running (25s) 4s ago 11m 556M - 19.0.0-3794-g931809a9 8880f7613d02 5acf3e511342 2024-05-17T21:53:33.668 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi138.napnll smithi138 *:8443,9283 running (2m) 5s ago 8m 130M - 19.0.0-3794-g931809a9 8880f7613d02 3f320d02d8e2 2024-05-17T21:53:33.668 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi107 smithi107 running (11m) 4s ago 11m 57.4M 2048M 17.2.7-952-g9903f776 d15a36853796 7de9c23eee97 2024-05-17T21:53:33.668 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi138 smithi138 running (8m) 5s ago 8m 43.7M 2048M 17.2.7-952-g9903f776 d15a36853796 ee61a065778d 2024-05-17T21:53:33.668 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi107 smithi107 *:9100 running (73s) 4s ago 9m 8292k - 1.5.0 0da6a335fe13 49bec71f5487 2024-05-17T21:53:33.668 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi138 smithi138 *:9100 running (42s) 5s ago 8m 8354k - 1.5.0 0da6a335fe13 e52824376658 2024-05-17T21:53:33.668 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0 smithi107 running (7m) 4s ago 7m 180M 1557M 17.2.7-952-g9903f776 d15a36853796 f2d20009361d 2024-05-17T21:53:33.668 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1 smithi107 running (7m) 4s ago 7m 187M 1557M 17.2.7-952-g9903f776 d15a36853796 e5d3e99844a9 2024-05-17T21:53:33.668 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2 smithi107 running (6m) 4s ago 6m 153M 1557M 17.2.7-952-g9903f776 d15a36853796 e044d7b78088 2024-05-17T21:53:33.668 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3 smithi138 running (6m) 5s ago 6m 201M 2581M 17.2.7-952-g9903f776 d15a36853796 8c980cc8af87 2024-05-17T21:53:33.668 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4 smithi138 running (6m) 5s ago 6m 185M 2581M 17.2.7-952-g9903f776 d15a36853796 b306a36a744e 2024-05-17T21:53:33.668 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5 smithi138 running (5m) 5s ago 5m 172M 2581M 17.2.7-952-g9903f776 d15a36853796 7c8d8ed74974 2024-05-17T21:53:33.668 INFO:teuthology.orchestra.run.smithi107.stdout:prometheus.smithi107 smithi107 *:9095 running (54s) 4s ago 9m 47.3M - 2.43.0 a07b618ecd1d 5f44e3b1144e 2024-05-17T21:53:33.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:33.678+0000 7f20b92dc700 1 -- 172.21.15.107:0/2517804438 >> [v2:172.21.15.107:6800/416442229,v1:172.21.15.107:6801/416442229] conn(0x7f20a003e330 msgr2=0x7f20a00407e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:53:33.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:33.678+0000 7f20b92dc700 1 --2- 172.21.15.107:0/2517804438 >> [v2:172.21.15.107:6800/416442229,v1:172.21.15.107:6801/416442229] conn(0x7f20a003e330 0x7f20a00407e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f209c009b30 tx=0x7f209c009e30 comp rx=0 tx=0).stop 2024-05-17T21:53:33.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:33.678+0000 7f20b92dc700 1 -- 172.21.15.107:0/2517804438 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20b40fbcb0 msgr2=0x7f20b40ffdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:53:33.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:33.678+0000 7f20b92dc700 1 --2- 172.21.15.107:0/2517804438 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20b40fbcb0 0x7f20b40ffdd0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f20a400eb40 tx=0x7f20a400b040 comp rx=0 tx=0).stop 2024-05-17T21:53:33.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:33.678+0000 7f20b92dc700 1 -- 172.21.15.107:0/2517804438 shutdown_connections 2024-05-17T21:53:33.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:33.678+0000 7f20b92dc700 1 --2- 172.21.15.107:0/2517804438 >> [v2:172.21.15.107:6800/416442229,v1:172.21.15.107:6801/416442229] conn(0x7f20a003e330 0x7f20a00407e0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:53:33.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:33.678+0000 7f20b92dc700 1 --2- 172.21.15.107:0/2517804438 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20b40fbcb0 0x7f20b40ffdd0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:53:33.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:33.678+0000 7f20b92dc700 1 --2- 172.21.15.107:0/2517804438 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f20b40fcf60 0x7f20b40fe2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:53:33.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:33.678+0000 7f20b92dc700 1 -- 172.21.15.107:0/2517804438 >> 172.21.15.107:0/2517804438 conn(0x7f20b40f7430 msgr2=0x7f20b40f8a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:53:33.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:33.678+0000 7f20b92dc700 1 -- 172.21.15.107:0/2517804438 shutdown_connections 2024-05-17T21:53:33.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:33.678+0000 7f20b92dc700 1 -- 172.21.15.107:0/2517804438 wait complete. 2024-05-17T21:53:33.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:33.796+0000 7f4ff715f700 1 -- 172.21.15.107:0/3327028200 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ff00ffee0 msgr2=0x7f4ff0102300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:53:33.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:33.796+0000 7f4ff715f700 1 --2- 172.21.15.107:0/3327028200 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ff00ffee0 0x7f4ff0102300 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc004940 tx=0x7f4fdc009e30 comp rx=0 tx=0).stop 2024-05-17T21:53:33.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:33.796+0000 7f4ff715f700 1 -- 172.21.15.107:0/3327028200 shutdown_connections 2024-05-17T21:53:33.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:33.796+0000 7f4ff715f700 1 --2- 172.21.15.107:0/3327028200 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ff00ffee0 0x7f4ff0102300 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:53:33.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:33.796+0000 7f4ff715f700 1 --2- 172.21.15.107:0/3327028200 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ff00662c0 0x7f4ff00ff9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:53:33.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:33.796+0000 7f4ff715f700 1 -- 172.21.15.107:0/3327028200 >> 172.21.15.107:0/3327028200 conn(0x7f4ff00f7430 msgr2=0x7f4ff00f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:53:33.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:33.796+0000 7f4ff715f700 1 -- 172.21.15.107:0/3327028200 shutdown_connections 2024-05-17T21:53:33.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:33.796+0000 7f4ff715f700 1 -- 172.21.15.107:0/3327028200 wait complete. 2024-05-17T21:53:33.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:33.796+0000 7f4ff715f700 1 Processor -- start 2024-05-17T21:53:33.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:33.797+0000 7f4ff715f700 1 -- start start 2024-05-17T21:53:33.790 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:33.797+0000 7f4ff715f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ff00662c0 0x7f4ff00fd380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:53:33.790 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:33.797+0000 7f4ff715f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ff00ffee0 0x7f4ff00fb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:53:33.791 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:33.797+0000 7f4ff715f700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff00fbe30 con 0x7f4ff00ffee0 2024-05-17T21:53:33.791 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:33.797+0000 7f4ff715f700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff00fbf70 con 0x7f4ff00662c0 2024-05-17T21:53:33.791 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:33.797+0000 7f4ff4efb700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ff00662c0 0x7f4ff00fd380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:53:33.791 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:33.797+0000 7f4feffff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ff00ffee0 0x7f4ff00fb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:53:33.791 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:33.797+0000 7f4ff4efb700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ff00662c0 0x7f4ff00fd380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.107:35350/0 (socket says 172.21.15.107:35350) 2024-05-17T21:53:33.791 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:33.797+0000 7f4ff4efb700 1 -- 172.21.15.107:0/4274652449 learned_addr learned my addr 172.21.15.107:0/4274652449 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:53:33.791 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:33.798+0000 7f4feffff700 1 -- 172.21.15.107:0/4274652449 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ff00662c0 msgr2=0x7f4ff00fd380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:53:33.791 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:33.798+0000 7f4feffff700 1 --2- 172.21.15.107:0/4274652449 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ff00662c0 0x7f4ff00fd380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:53:33.791 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:33.798+0000 7f4feffff700 1 -- 172.21.15.107:0/4274652449 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fdc005040 con 0x7f4ff00ffee0 2024-05-17T21:53:33.791 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:33.798+0000 7f4feffff700 1 --2- 172.21.15.107:0/4274652449 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ff00ffee0 0x7f4ff00fb8a0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc000c00 tx=0x7f4fdc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:53:33.791 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:33.798+0000 7f4fed7fa700 1 -- 172.21.15.107:0/4274652449 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4fdc00e030 con 0x7f4ff00ffee0 2024-05-17T21:53:33.791 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:33.798+0000 7f4fed7fa700 1 -- 172.21.15.107:0/4274652449 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4fdc013de0 con 0x7f4ff00ffee0 2024-05-17T21:53:33.791 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:33.798+0000 7f4fed7fa700 1 -- 172.21.15.107:0/4274652449 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4fdc01ce50 con 0x7f4ff00ffee0 2024-05-17T21:53:33.792 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:33.798+0000 7f4ff715f700 1 -- 172.21.15.107:0/4274652449 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ff00fc1f0 con 0x7f4ff00ffee0 2024-05-17T21:53:33.792 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:33.798+0000 7f4ff715f700 1 -- 172.21.15.107:0/4274652449 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ff0115a00 con 0x7f4ff00ffee0 2024-05-17T21:53:33.792 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:33.799+0000 7f4ff715f700 1 -- 172.21.15.107:0/4274652449 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ff005f5c0 con 0x7f4ff00ffee0 2024-05-17T21:53:33.792 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:33.799+0000 7f4fed7fa700 1 -- 172.21.15.107:0/4274652449 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 30) v1 ==== 50232+0+0 (secure 0 0 0) 0x7f4fdc00b460 con 0x7f4ff00ffee0 2024-05-17T21:53:33.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:33.799+0000 7f4fed7fa700 1 --2- 172.21.15.107:0/4274652449 >> [v2:172.21.15.107:6800/416442229,v1:172.21.15.107:6801/416442229] conn(0x7f4fe003e330 0x7f4fe00407e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:53:33.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:33.799+0000 7f4fed7fa700 1 -- 172.21.15.107:0/4274652449 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f4fdc054ac0 con 0x7f4ff00ffee0 2024-05-17T21:53:33.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:33.800+0000 7f4ff4efb700 1 --2- 172.21.15.107:0/4274652449 >> [v2:172.21.15.107:6800/416442229,v1:172.21.15.107:6801/416442229] conn(0x7f4fe003e330 0x7f4fe00407e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:53:33.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:33.800+0000 7f4ff4efb700 1 --2- 172.21.15.107:0/4274652449 >> [v2:172.21.15.107:6800/416442229,v1:172.21.15.107:6801/416442229] conn(0x7f4fe003e330 0x7f4fe00407e0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f4ff00fd110 tx=0x7f4fe400b040 comp rx=0 tx=0).ready entity=mgr.14750 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:53:33.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:33.804+0000 7f4fed7fa700 1 -- 172.21.15.107:0/4274652449 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+190121 (secure 0 0 0) 0x7f4fdc025400 con 0x7f4ff00ffee0 2024-05-17T21:53:34.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:34.174+0000 7f4ff715f700 1 -- 172.21.15.107:0/4274652449 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f4ff004bed0 con 0x7f4ff00ffee0 2024-05-17T21:53:34.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:34.175+0000 7f4fed7fa700 1 -- 172.21.15.107:0/4274652449 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+739 (secure 0 0 0) 0x7f4fdc011c00 con 0x7f4ff00ffee0 2024-05-17T21:53:34.168 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-05-17T21:53:34.168 INFO:teuthology.orchestra.run.smithi107.stdout: "mon": { 2024-05-17T21:53:34.168 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-952-g9903f776 (9903f7767386a9d9efadb7c1effdcd3064d71804) quincy (stable)": 2 2024-05-17T21:53:34.168 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-05-17T21:53:34.168 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr": { 2024-05-17T21:53:34.168 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev)": 1 2024-05-17T21:53:34.168 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-05-17T21:53:34.168 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": { 2024-05-17T21:53:34.169 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-952-g9903f776 (9903f7767386a9d9efadb7c1effdcd3064d71804) quincy (stable)": 6 2024-05-17T21:53:34.169 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-05-17T21:53:34.169 INFO:teuthology.orchestra.run.smithi107.stdout: "mds": { 2024-05-17T21:53:34.169 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-952-g9903f776 (9903f7767386a9d9efadb7c1effdcd3064d71804) quincy (stable)": 4 2024-05-17T21:53:34.169 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-05-17T21:53:34.169 INFO:teuthology.orchestra.run.smithi107.stdout: "overall": { 2024-05-17T21:53:34.169 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-952-g9903f776 (9903f7767386a9d9efadb7c1effdcd3064d71804) quincy (stable)": 12, 2024-05-17T21:53:34.169 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev)": 1 2024-05-17T21:53:34.169 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-05-17T21:53:34.169 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-05-17T21:53:34.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:34.177+0000 7f4ff715f700 1 -- 172.21.15.107:0/4274652449 >> [v2:172.21.15.107:6800/416442229,v1:172.21.15.107:6801/416442229] conn(0x7f4fe003e330 msgr2=0x7f4fe00407e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:53:34.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:34.177+0000 7f4ff715f700 1 --2- 172.21.15.107:0/4274652449 >> [v2:172.21.15.107:6800/416442229,v1:172.21.15.107:6801/416442229] conn(0x7f4fe003e330 0x7f4fe00407e0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f4ff00fd110 tx=0x7f4fe400b040 comp rx=0 tx=0).stop 2024-05-17T21:53:34.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:34.177+0000 7f4ff715f700 1 -- 172.21.15.107:0/4274652449 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ff00ffee0 msgr2=0x7f4ff00fb8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:53:34.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:34.177+0000 7f4ff715f700 1 --2- 172.21.15.107:0/4274652449 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ff00ffee0 0x7f4ff00fb8a0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc000c00 tx=0x7f4fdc009e30 comp rx=0 tx=0).stop 2024-05-17T21:53:34.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:34.177+0000 7f4ff715f700 1 -- 172.21.15.107:0/4274652449 shutdown_connections 2024-05-17T21:53:34.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:34.177+0000 7f4ff715f700 1 --2- 172.21.15.107:0/4274652449 >> [v2:172.21.15.107:6800/416442229,v1:172.21.15.107:6801/416442229] conn(0x7f4fe003e330 0x7f4fe00407e0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:53:34.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:34.177+0000 7f4ff715f700 1 --2- 172.21.15.107:0/4274652449 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ff00662c0 0x7f4ff00fd380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:53:34.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:34.177+0000 7f4ff715f700 1 --2- 172.21.15.107:0/4274652449 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ff00ffee0 0x7f4ff00fb8a0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:53:34.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:34.177+0000 7f4ff715f700 1 -- 172.21.15.107:0/4274652449 >> 172.21.15.107:0/4274652449 conn(0x7f4ff00f7430 msgr2=0x7f4ff00f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:53:34.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:34.177+0000 7f4ff715f700 1 -- 172.21.15.107:0/4274652449 shutdown_connections 2024-05-17T21:53:34.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:34.177+0000 7f4ff715f700 1 -- 172.21.15.107:0/4274652449 wait complete. 2024-05-17T21:53:34.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:34.292+0000 7f32c4b57700 1 -- 172.21.15.107:0/1996337010 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32c00fce60 msgr2=0x7f32c00fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:53:34.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:34.292+0000 7f32c4b57700 1 --2- 172.21.15.107:0/1996337010 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32c00fce60 0x7f32c00fd270 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f32b0004940 tx=0x7f32b0009e30 comp rx=0 tx=0).stop 2024-05-17T21:53:34.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:34.292+0000 7f32c4b57700 1 -- 172.21.15.107:0/1996337010 shutdown_connections 2024-05-17T21:53:34.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:34.292+0000 7f32c4b57700 1 --2- 172.21.15.107:0/1996337010 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32c00fdfd0 0x7f32c00fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:53:34.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:34.292+0000 7f32c4b57700 1 --2- 172.21.15.107:0/1996337010 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32c00fce60 0x7f32c00fd270 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:53:34.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:34.292+0000 7f32c4b57700 1 -- 172.21.15.107:0/1996337010 >> 172.21.15.107:0/1996337010 conn(0x7f32c00f8540 msgr2=0x7f32c00fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:53:34.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:34.292+0000 7f32c4b57700 1 -- 172.21.15.107:0/1996337010 shutdown_connections 2024-05-17T21:53:34.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:34.292+0000 7f32c4b57700 1 -- 172.21.15.107:0/1996337010 wait complete. 2024-05-17T21:53:34.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:34.292+0000 7f32c4b57700 1 Processor -- start 2024-05-17T21:53:34.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:34.293+0000 7f32c4b57700 1 -- start start 2024-05-17T21:53:34.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:34.293+0000 7f32c4b57700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32c00fce60 0x7f32c0071de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:53:34.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:34.293+0000 7f32c4b57700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32c00fdfd0 0x7f32c00702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:53:34.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:34.293+0000 7f32c4b57700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32c0070860 con 0x7f32c00fdfd0 2024-05-17T21:53:34.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:34.293+0000 7f32c4b57700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32c00709d0 con 0x7f32c00fce60 2024-05-17T21:53:34.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:34.293+0000 7f32bdd9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32c00fdfd0 0x7f32c00702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:53:34.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:34.293+0000 7f32bdd9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32c00fdfd0 0x7f32c00702b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52264/0 (socket says 172.21.15.107:52264) 2024-05-17T21:53:34.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:34.293+0000 7f32bdd9b700 1 -- 172.21.15.107:0/2797373876 learned_addr learned my addr 172.21.15.107:0/2797373876 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:53:34.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:34.293+0000 7f32be59c700 1 --2- 172.21.15.107:0/2797373876 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32c00fce60 0x7f32c0071de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:53:34.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:34.293+0000 7f32bdd9b700 1 -- 172.21.15.107:0/2797373876 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32c00fce60 msgr2=0x7f32c0071de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:53:34.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:34.293+0000 7f32bdd9b700 1 --2- 172.21.15.107:0/2797373876 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32c00fce60 0x7f32c0071de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:53:34.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:34.293+0000 7f32bdd9b700 1 -- 172.21.15.107:0/2797373876 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32b0005040 con 0x7f32c00fdfd0 2024-05-17T21:53:34.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:34.293+0000 7f32bdd9b700 1 --2- 172.21.15.107:0/2797373876 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32c00fdfd0 0x7f32c00702b0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f32b4002a50 tx=0x7f32b40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:53:34.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:34.293+0000 7f32aeffd700 1 -- 172.21.15.107:0/2797373876 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f32b4007cc0 con 0x7f32c00fdfd0 2024-05-17T21:53:34.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:34.293+0000 7f32aeffd700 1 -- 172.21.15.107:0/2797373876 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f32b400f040 con 0x7f32c00fdfd0 2024-05-17T21:53:34.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:34.293+0000 7f32c4b57700 1 -- 172.21.15.107:0/2797373876 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32c0070cb0 con 0x7f32c00fdfd0 2024-05-17T21:53:34.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:34.294+0000 7f32aeffd700 1 -- 172.21.15.107:0/2797373876 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f32b400da30 con 0x7f32c00fdfd0 2024-05-17T21:53:34.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:34.294+0000 7f32c4b57700 1 -- 172.21.15.107:0/2797373876 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32c0115a20 con 0x7f32c00fdfd0 2024-05-17T21:53:34.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:34.295+0000 7f32c4b57700 1 -- 172.21.15.107:0/2797373876 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32c005f5c0 con 0x7f32c00fdfd0 2024-05-17T21:53:34.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:34.296+0000 7f32aeffd700 1 -- 172.21.15.107:0/2797373876 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 31) v1 ==== 99906+0+0 (secure 0 0 0) 0x7f32b400dbb0 con 0x7f32c00fdfd0 2024-05-17T21:53:34.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:34.297+0000 7f32aeffd700 1 --2- 172.21.15.107:0/2797373876 >> [v2:172.21.15.107:6800/416442229,v1:172.21.15.107:6801/416442229] conn(0x7f32a80783f0 0x7f32a807a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:53:34.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:34.297+0000 7f32aeffd700 1 -- 172.21.15.107:0/2797373876 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f32b409a4c0 con 0x7f32c00fdfd0 2024-05-17T21:53:34.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:34.297+0000 7f32be59c700 1 --2- 172.21.15.107:0/2797373876 >> [v2:172.21.15.107:6800/416442229,v1:172.21.15.107:6801/416442229] conn(0x7f32a80783f0 0x7f32a807a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:53:34.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:34.298+0000 7f32be59c700 1 --2- 172.21.15.107:0/2797373876 >> [v2:172.21.15.107:6800/416442229,v1:172.21.15.107:6801/416442229] conn(0x7f32a80783f0 0x7f32a807a8a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f32b0004890 tx=0x7f32b0009e30 comp rx=0 tx=0).ready entity=mgr.14750 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:53:34.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:34.302+0000 7f32aeffd700 1 -- 172.21.15.107:0/2797373876 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+190121 (secure 0 0 0) 0x7f32b4062740 con 0x7f32c00fdfd0 2024-05-17T21:53:34.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:34 smithi138 ceph-mon[35948]: from='client.24511 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:53:34.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:34 smithi138 ceph-mon[35948]: from='client.14786 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:53:34.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:34 smithi138 ceph-mon[35948]: Standby manager daemon smithi138.napnll started 2024-05-17T21:53:34.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:34 smithi138 ceph-mon[35948]: from='mgr.? 172.21.15.138:0/2270662160' entity='mgr.smithi138.napnll' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi138.napnll/crt"}]: dispatch 2024-05-17T21:53:34.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:34 smithi138 ceph-mon[35948]: from='mgr.? 172.21.15.138:0/2270662160' entity='mgr.smithi138.napnll' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-17T21:53:34.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:34 smithi138 ceph-mon[35948]: from='mgr.? 172.21.15.138:0/2270662160' entity='mgr.smithi138.napnll' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi138.napnll/key"}]: dispatch 2024-05-17T21:53:34.577 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:34 smithi138 ceph-mon[35948]: from='mgr.? 172.21.15.138:0/2270662160' entity='mgr.smithi138.napnll' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-17T21:53:34.577 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:34 smithi138 ceph-mon[35948]: from='client.? 172.21.15.107:0/4274652449' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:53:34.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:34 smithi107 ceph-mon[28434]: from='client.24511 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:53:34.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:34 smithi107 ceph-mon[28434]: from='client.14786 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:53:34.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:34 smithi107 ceph-mon[28434]: Standby manager daemon smithi138.napnll started 2024-05-17T21:53:34.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:34 smithi107 ceph-mon[28434]: from='mgr.? 172.21.15.138:0/2270662160' entity='mgr.smithi138.napnll' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi138.napnll/crt"}]: dispatch 2024-05-17T21:53:34.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:34 smithi107 ceph-mon[28434]: from='mgr.? 172.21.15.138:0/2270662160' entity='mgr.smithi138.napnll' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-17T21:53:34.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:34 smithi107 ceph-mon[28434]: from='mgr.? 172.21.15.138:0/2270662160' entity='mgr.smithi138.napnll' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi138.napnll/key"}]: dispatch 2024-05-17T21:53:34.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:34 smithi107 ceph-mon[28434]: from='mgr.? 172.21.15.138:0/2270662160' entity='mgr.smithi138.napnll' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-17T21:53:34.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:34 smithi107 ceph-mon[28434]: from='client.? 172.21.15.107:0/4274652449' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:53:34.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:34.614+0000 7f32c4b57700 1 -- 172.21.15.107:0/2797373876 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f32c004bed0 con 0x7f32c00fdfd0 2024-05-17T21:53:34.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:34.614+0000 7f32aeffd700 1 -- 172.21.15.107:0/2797373876 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1840 (secure 0 0 0) 0x7f32b40620e0 con 0x7f32c00fdfd0 2024-05-17T21:53:34.607 INFO:teuthology.orchestra.run.smithi107.stdout:e15 2024-05-17T21:53:34.607 INFO:teuthology.orchestra.run.smithi107.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-17T21:53:34.607 INFO:teuthology.orchestra.run.smithi107.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-17T21:53:34.607 INFO:teuthology.orchestra.run.smithi107.stdout:legacy client fscid: 1 2024-05-17T21:53:34.607 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:53:34.607 INFO:teuthology.orchestra.run.smithi107.stdout:Filesystem 'cephfs' (1) 2024-05-17T21:53:34.607 INFO:teuthology.orchestra.run.smithi107.stdout:fs_name cephfs 2024-05-17T21:53:34.607 INFO:teuthology.orchestra.run.smithi107.stdout:epoch 15 2024-05-17T21:53:34.607 INFO:teuthology.orchestra.run.smithi107.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-17T21:53:34.608 INFO:teuthology.orchestra.run.smithi107.stdout:created 2024-05-17T21:49:16.836333+0000 2024-05-17T21:53:34.608 INFO:teuthology.orchestra.run.smithi107.stdout:modified 2024-05-17T21:49:34.873628+0000 2024-05-17T21:53:34.608 INFO:teuthology.orchestra.run.smithi107.stdout:tableserver 0 2024-05-17T21:53:34.608 INFO:teuthology.orchestra.run.smithi107.stdout:root 0 2024-05-17T21:53:34.608 INFO:teuthology.orchestra.run.smithi107.stdout:session_timeout 60 2024-05-17T21:53:34.608 INFO:teuthology.orchestra.run.smithi107.stdout:session_autoclose 300 2024-05-17T21:53:34.608 INFO:teuthology.orchestra.run.smithi107.stdout:max_file_size 1099511627776 2024-05-17T21:53:34.608 INFO:teuthology.orchestra.run.smithi107.stdout:required_client_features {} 2024-05-17T21:53:34.608 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure 0 2024-05-17T21:53:34.608 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure_osd_epoch 46 2024-05-17T21:53:34.608 INFO:teuthology.orchestra.run.smithi107.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-17T21:53:34.608 INFO:teuthology.orchestra.run.smithi107.stdout:max_mds 1 2024-05-17T21:53:34.608 INFO:teuthology.orchestra.run.smithi107.stdout:in 0 2024-05-17T21:53:34.608 INFO:teuthology.orchestra.run.smithi107.stdout:up {0=24259} 2024-05-17T21:53:34.609 INFO:teuthology.orchestra.run.smithi107.stdout:failed 2024-05-17T21:53:34.609 INFO:teuthology.orchestra.run.smithi107.stdout:damaged 2024-05-17T21:53:34.609 INFO:teuthology.orchestra.run.smithi107.stdout:stopped 2024-05-17T21:53:34.609 INFO:teuthology.orchestra.run.smithi107.stdout:data_pools [3] 2024-05-17T21:53:34.609 INFO:teuthology.orchestra.run.smithi107.stdout:metadata_pool 2 2024-05-17T21:53:34.609 INFO:teuthology.orchestra.run.smithi107.stdout:inline_data enabled 2024-05-17T21:53:34.609 INFO:teuthology.orchestra.run.smithi107.stdout:balancer 2024-05-17T21:53:34.609 INFO:teuthology.orchestra.run.smithi107.stdout:standby_count_wanted 1 2024-05-17T21:53:34.609 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi138.gwbtyd{0:24259} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.138:6824/2779634919,v1:172.21.15.138:6825/2779634919] compat {c=[1],r=[1],i=[7ff]}] 2024-05-17T21:53:34.609 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.drqody{0:14486} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.107:6828/2275807100,v1:172.21.15.107:6829/2275807100] compat {c=[1],r=[1],i=[7ff]}] 2024-05-17T21:53:34.609 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:53:34.609 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:53:34.609 INFO:teuthology.orchestra.run.smithi107.stdout:Standby daemons: 2024-05-17T21:53:34.610 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:53:34.610 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.idmbgj{-1:14498} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.107:6826/1460186079,v1:172.21.15.107:6827/1460186079] compat {c=[1],r=[1],i=[7ff]}] 2024-05-17T21:53:34.610 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi138.zckrvs{-1:24267} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.138:6826/170628661,v1:172.21.15.138:6827/170628661] compat {c=[1],r=[1],i=[7ff]}] 2024-05-17T21:53:34.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:34.618+0000 7f32c4b57700 1 -- 172.21.15.107:0/2797373876 >> [v2:172.21.15.107:6800/416442229,v1:172.21.15.107:6801/416442229] conn(0x7f32a80783f0 msgr2=0x7f32a807a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:53:34.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:34.618+0000 7f32c4b57700 1 --2- 172.21.15.107:0/2797373876 >> [v2:172.21.15.107:6800/416442229,v1:172.21.15.107:6801/416442229] conn(0x7f32a80783f0 0x7f32a807a8a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f32b0004890 tx=0x7f32b0009e30 comp rx=0 tx=0).stop 2024-05-17T21:53:34.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:34.618+0000 7f32c4b57700 1 -- 172.21.15.107:0/2797373876 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32c00fdfd0 msgr2=0x7f32c00702b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:53:34.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:34.618+0000 7f32c4b57700 1 --2- 172.21.15.107:0/2797373876 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32c00fdfd0 0x7f32c00702b0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f32b4002a50 tx=0x7f32b40046b0 comp rx=0 tx=0).stop 2024-05-17T21:53:34.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:34.618+0000 7f32c4b57700 1 -- 172.21.15.107:0/2797373876 shutdown_connections 2024-05-17T21:53:34.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:34.618+0000 7f32c4b57700 1 --2- 172.21.15.107:0/2797373876 >> [v2:172.21.15.107:6800/416442229,v1:172.21.15.107:6801/416442229] conn(0x7f32a80783f0 0x7f32a807a8a0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:53:34.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:34.618+0000 7f32c4b57700 1 --2- 172.21.15.107:0/2797373876 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32c00fce60 0x7f32c0071de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:53:34.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:34.618+0000 7f32c4b57700 1 --2- 172.21.15.107:0/2797373876 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32c00fdfd0 0x7f32c00702b0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:53:34.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:34.618+0000 7f32c4b57700 1 -- 172.21.15.107:0/2797373876 >> 172.21.15.107:0/2797373876 conn(0x7f32c00f8540 msgr2=0x7f32c00f9c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:53:34.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:34.618+0000 7f32c4b57700 1 -- 172.21.15.107:0/2797373876 shutdown_connections 2024-05-17T21:53:34.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:34.618+0000 7f32c4b57700 1 -- 172.21.15.107:0/2797373876 wait complete. 2024-05-17T21:53:34.611 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 15 2024-05-17T21:53:34.725 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:34.732+0000 7f64dc14e700 1 -- 172.21.15.107:0/1296120880 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f64d40fcf90 msgr2=0x7f64d40fd400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:53:34.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:34.732+0000 7f64dc14e700 1 --2- 172.21.15.107:0/1296120880 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f64d40fcf90 0x7f64d40fd400 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f64c4004940 tx=0x7f64c4009e30 comp rx=0 tx=0).stop 2024-05-17T21:53:34.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:34.732+0000 7f64dc14e700 1 -- 172.21.15.107:0/1296120880 shutdown_connections 2024-05-17T21:53:34.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:34.732+0000 7f64dc14e700 1 --2- 172.21.15.107:0/1296120880 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f64d40fcf90 0x7f64d40fd400 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:53:34.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:34.732+0000 7f64dc14e700 1 --2- 172.21.15.107:0/1296120880 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f64d40fbce0 0x7f64d40fc0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:53:34.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:34.732+0000 7f64dc14e700 1 -- 172.21.15.107:0/1296120880 >> 172.21.15.107:0/1296120880 conn(0x7f64d40f7440 msgr2=0x7f64d40f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:53:34.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:34.732+0000 7f64dc14e700 1 -- 172.21.15.107:0/1296120880 shutdown_connections 2024-05-17T21:53:34.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:34.732+0000 7f64dc14e700 1 -- 172.21.15.107:0/1296120880 wait complete. 2024-05-17T21:53:34.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:34.733+0000 7f64dc14e700 1 Processor -- start 2024-05-17T21:53:34.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:34.733+0000 7f64dc14e700 1 -- start start 2024-05-17T21:53:34.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:34.733+0000 7f64dc14e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f64d40fbce0 0x7f64d4110710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:53:34.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:34.733+0000 7f64dc14e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f64d40fcf90 0x7f64d4112c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:53:34.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:34.733+0000 7f64dc14e700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64d4066b70 con 0x7f64d40fcf90 2024-05-17T21:53:34.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:34.733+0000 7f64dc14e700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64d4066cb0 con 0x7f64d40fbce0 2024-05-17T21:53:34.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:34.733+0000 7f64d96e9700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f64d40fcf90 0x7f64d4112c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:53:34.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:34.733+0000 7f64d96e9700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f64d40fcf90 0x7f64d4112c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52286/0 (socket says 172.21.15.107:52286) 2024-05-17T21:53:34.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:34.733+0000 7f64d96e9700 1 -- 172.21.15.107:0/1341961156 learned_addr learned my addr 172.21.15.107:0/1341961156 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:53:34.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:34.733+0000 7f64d9eea700 1 --2- 172.21.15.107:0/1341961156 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f64d40fbce0 0x7f64d4110710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:53:34.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:34.733+0000 7f64d96e9700 1 -- 172.21.15.107:0/1341961156 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f64d40fbce0 msgr2=0x7f64d4110710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:53:34.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:34.733+0000 7f64d96e9700 1 --2- 172.21.15.107:0/1341961156 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f64d40fbce0 0x7f64d4110710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:53:34.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:34.733+0000 7f64d96e9700 1 -- 172.21.15.107:0/1341961156 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64d0005040 con 0x7f64d40fcf90 2024-05-17T21:53:34.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:34.733+0000 7f64d96e9700 1 --2- 172.21.15.107:0/1341961156 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f64d40fcf90 0x7f64d4112c60 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f64c400b0d0 tx=0x7f64c4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:53:34.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:34.734+0000 7f64ca7fc700 1 -- 172.21.15.107:0/1341961156 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f64c400e030 con 0x7f64d40fcf90 2024-05-17T21:53:34.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:34.734+0000 7f64ca7fc700 1 -- 172.21.15.107:0/1341961156 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f64c401c400 con 0x7f64d40fcf90 2024-05-17T21:53:34.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:34.734+0000 7f64dc14e700 1 -- 172.21.15.107:0/1341961156 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64c4005040 con 0x7f64d40fcf90 2024-05-17T21:53:34.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:34.734+0000 7f64ca7fc700 1 -- 172.21.15.107:0/1341961156 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f64c400b630 con 0x7f64d40fcf90 2024-05-17T21:53:34.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:34.734+0000 7f64dc14e700 1 -- 172.21.15.107:0/1341961156 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64d4113580 con 0x7f64d40fcf90 2024-05-17T21:53:34.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:34.735+0000 7f64dc14e700 1 -- 172.21.15.107:0/1341961156 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64d405f5c0 con 0x7f64d40fcf90 2024-05-17T21:53:34.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:34.735+0000 7f64ca7fc700 1 -- 172.21.15.107:0/1341961156 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 31) v1 ==== 99906+0+0 (secure 0 0 0) 0x7f64c401c570 con 0x7f64d40fcf90 2024-05-17T21:53:34.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:34.736+0000 7f64ca7fc700 1 --2- 172.21.15.107:0/1341961156 >> [v2:172.21.15.107:6800/416442229,v1:172.21.15.107:6801/416442229] conn(0x7f64c00783a0 0x7f64c007a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:53:34.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:34.736+0000 7f64ca7fc700 1 -- 172.21.15.107:0/1341961156 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f64c409b910 con 0x7f64d40fcf90 2024-05-17T21:53:34.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:34.736+0000 7f64d9eea700 1 --2- 172.21.15.107:0/1341961156 >> [v2:172.21.15.107:6800/416442229,v1:172.21.15.107:6801/416442229] conn(0x7f64c00783a0 0x7f64c007a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:53:34.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:34.736+0000 7f64d9eea700 1 --2- 172.21.15.107:0/1341961156 >> [v2:172.21.15.107:6800/416442229,v1:172.21.15.107:6801/416442229] conn(0x7f64c00783a0 0x7f64c007a850 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f64d40667f0 tx=0x7f64d000b040 comp rx=0 tx=0).ready entity=mgr.14750 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:53:34.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:34.741+0000 7f64ca7fc700 1 -- 172.21.15.107:0/1341961156 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+190121 (secure 0 0 0) 0x7f64c4063c30 con 0x7f64d40fcf90 2024-05-17T21:53:35.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:35.013+0000 7f64dc14e700 1 -- 172.21.15.107:0/1341961156 --> [v2:172.21.15.107:6800/416442229,v1:172.21.15.107:6801/416442229] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f64d4101280 con 0x7f64c00783a0 2024-05-17T21:53:35.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:35.015+0000 7f64ca7fc700 1 -- 172.21.15.107:0/1341961156 <== mgr.14750 v2:172.21.15.107:6800/416442229 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7f64d4101280 con 0x7f64c00783a0 2024-05-17T21:53:35.007 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-05-17T21:53:35.007 INFO:teuthology.orchestra.run.smithi107.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b", 2024-05-17T21:53:35.007 INFO:teuthology.orchestra.run.smithi107.stdout: "in_progress": true, 2024-05-17T21:53:35.007 INFO:teuthology.orchestra.run.smithi107.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-17T21:53:35.007 INFO:teuthology.orchestra.run.smithi107.stdout: "services_complete": [ 2024-05-17T21:53:35.008 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr" 2024-05-17T21:53:35.008 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-05-17T21:53:35.008 INFO:teuthology.orchestra.run.smithi107.stdout: "progress": "2/21 daemons upgraded", 2024-05-17T21:53:35.008 INFO:teuthology.orchestra.run.smithi107.stdout: "message": "", 2024-05-17T21:53:35.008 INFO:teuthology.orchestra.run.smithi107.stdout: "is_paused": false 2024-05-17T21:53:35.008 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-05-17T21:53:35.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:35.019+0000 7f64dc14e700 1 -- 172.21.15.107:0/1341961156 >> [v2:172.21.15.107:6800/416442229,v1:172.21.15.107:6801/416442229] conn(0x7f64c00783a0 msgr2=0x7f64c007a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:53:35.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:35.019+0000 7f64dc14e700 1 --2- 172.21.15.107:0/1341961156 >> [v2:172.21.15.107:6800/416442229,v1:172.21.15.107:6801/416442229] conn(0x7f64c00783a0 0x7f64c007a850 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f64d40667f0 tx=0x7f64d000b040 comp rx=0 tx=0).stop 2024-05-17T21:53:35.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:35.019+0000 7f64dc14e700 1 -- 172.21.15.107:0/1341961156 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f64d40fcf90 msgr2=0x7f64d4112c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:53:35.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:35.019+0000 7f64dc14e700 1 --2- 172.21.15.107:0/1341961156 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f64d40fcf90 0x7f64d4112c60 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f64c400b0d0 tx=0x7f64c4009e30 comp rx=0 tx=0).stop 2024-05-17T21:53:35.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:35.019+0000 7f64dc14e700 1 -- 172.21.15.107:0/1341961156 shutdown_connections 2024-05-17T21:53:35.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:35.019+0000 7f64dc14e700 1 --2- 172.21.15.107:0/1341961156 >> [v2:172.21.15.107:6800/416442229,v1:172.21.15.107:6801/416442229] conn(0x7f64c00783a0 0x7f64c007a850 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:53:35.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:35.019+0000 7f64dc14e700 1 --2- 172.21.15.107:0/1341961156 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f64d40fbce0 0x7f64d4110710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:53:35.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:35.019+0000 7f64dc14e700 1 --2- 172.21.15.107:0/1341961156 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f64d40fcf90 0x7f64d4112c60 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:53:35.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:35.019+0000 7f64dc14e700 1 -- 172.21.15.107:0/1341961156 >> 172.21.15.107:0/1341961156 conn(0x7f64d40f7440 msgr2=0x7f64d40f8aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:53:35.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:35.019+0000 7f64dc14e700 1 -- 172.21.15.107:0/1341961156 shutdown_connections 2024-05-17T21:53:35.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:35.019+0000 7f64dc14e700 1 -- 172.21.15.107:0/1341961156 wait complete. 2024-05-17T21:53:35.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:35.142+0000 7f9b83ab5700 1 -- 172.21.15.107:0/1240404773 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b7c0fff10 msgr2=0x7f9b7c102330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:53:35.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:35.142+0000 7f9b83ab5700 1 --2- 172.21.15.107:0/1240404773 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b7c0fff10 0x7f9b7c102330 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c004940 tx=0x7f9b6c009e30 comp rx=0 tx=0).stop 2024-05-17T21:53:35.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:35.142+0000 7f9b83ab5700 1 -- 172.21.15.107:0/1240404773 shutdown_connections 2024-05-17T21:53:35.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:35.142+0000 7f9b83ab5700 1 --2- 172.21.15.107:0/1240404773 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b7c0fff10 0x7f9b7c102330 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:53:35.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:35.142+0000 7f9b83ab5700 1 --2- 172.21.15.107:0/1240404773 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9b7c066b50 0x7f9b7c0ff9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:53:35.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:35.142+0000 7f9b83ab5700 1 -- 172.21.15.107:0/1240404773 >> 172.21.15.107:0/1240404773 conn(0x7f9b7c0f7440 msgr2=0x7f9b7c0f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:53:35.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:35.142+0000 7f9b83ab5700 1 -- 172.21.15.107:0/1240404773 shutdown_connections 2024-05-17T21:53:35.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:35.142+0000 7f9b83ab5700 1 -- 172.21.15.107:0/1240404773 wait complete. 2024-05-17T21:53:35.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:35.142+0000 7f9b83ab5700 1 Processor -- start 2024-05-17T21:53:35.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:35.142+0000 7f9b83ab5700 1 -- start start 2024-05-17T21:53:35.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:35.143+0000 7f9b83ab5700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9b7c066b50 0x7f9b7c10da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:53:35.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:35.143+0000 7f9b83ab5700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b7c0fff10 0x7f9b7c10df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:53:35.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:35.143+0000 7f9b83ab5700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b7c10e560 con 0x7f9b7c0fff10 2024-05-17T21:53:35.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:35.143+0000 7f9b83ab5700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b7c10ed60 con 0x7f9b7c066b50 2024-05-17T21:53:35.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:35.143+0000 7f9b81050700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b7c0fff10 0x7f9b7c10df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:53:35.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:35.143+0000 7f9b81050700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b7c0fff10 0x7f9b7c10df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52306/0 (socket says 172.21.15.107:52306) 2024-05-17T21:53:35.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:35.143+0000 7f9b81050700 1 -- 172.21.15.107:0/857526149 learned_addr learned my addr 172.21.15.107:0/857526149 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:53:35.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:35.143+0000 7f9b81851700 1 --2- 172.21.15.107:0/857526149 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9b7c066b50 0x7f9b7c10da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:53:35.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:35.143+0000 7f9b81050700 1 -- 172.21.15.107:0/857526149 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9b7c066b50 msgr2=0x7f9b7c10da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:53:35.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:35.143+0000 7f9b81050700 1 --2- 172.21.15.107:0/857526149 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9b7c066b50 0x7f9b7c10da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:53:35.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:35.143+0000 7f9b81050700 1 -- 172.21.15.107:0/857526149 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b6c005040 con 0x7f9b7c0fff10 2024-05-17T21:53:35.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:35.143+0000 7f9b81050700 1 --2- 172.21.15.107:0/857526149 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b7c0fff10 0x7f9b7c10df60 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c013870 tx=0x7f9b6c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:53:35.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:35.143+0000 7f9b727fc700 1 -- 172.21.15.107:0/857526149 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b6c021070 con 0x7f9b7c0fff10 2024-05-17T21:53:35.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:35.143+0000 7f9b83ab5700 1 -- 172.21.15.107:0/857526149 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b7c10efe0 con 0x7f9b7c0fff10 2024-05-17T21:53:35.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:35.143+0000 7f9b727fc700 1 -- 172.21.15.107:0/857526149 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9b6c009430 con 0x7f9b7c0fff10 2024-05-17T21:53:35.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:35.143+0000 7f9b83ab5700 1 -- 172.21.15.107:0/857526149 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b7c10f520 con 0x7f9b7c0fff10 2024-05-17T21:53:35.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:35.144+0000 7f9b727fc700 1 -- 172.21.15.107:0/857526149 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b6c00e070 con 0x7f9b7c0fff10 2024-05-17T21:53:35.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:35.144+0000 7f9b83ab5700 1 -- 172.21.15.107:0/857526149 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b7c05f5c0 con 0x7f9b7c0fff10 2024-05-17T21:53:35.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:35.145+0000 7f9b727fc700 1 -- 172.21.15.107:0/857526149 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 31) v1 ==== 99906+0+0 (secure 0 0 0) 0x7f9b6c01c750 con 0x7f9b7c0fff10 2024-05-17T21:53:35.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:35.146+0000 7f9b727fc700 1 --2- 172.21.15.107:0/857526149 >> [v2:172.21.15.107:6800/416442229,v1:172.21.15.107:6801/416442229] conn(0x7f9b680783a0 0x7f9b6807a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:53:35.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:35.146+0000 7f9b727fc700 1 -- 172.21.15.107:0/857526149 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f9b6c09b060 con 0x7f9b7c0fff10 2024-05-17T21:53:35.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:35.146+0000 7f9b81851700 1 --2- 172.21.15.107:0/857526149 >> [v2:172.21.15.107:6800/416442229,v1:172.21.15.107:6801/416442229] conn(0x7f9b680783a0 0x7f9b6807a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:53:35.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:35.147+0000 7f9b81851700 1 --2- 172.21.15.107:0/857526149 >> [v2:172.21.15.107:6800/416442229,v1:172.21.15.107:6801/416442229] conn(0x7f9b680783a0 0x7f9b6807a850 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9b78009b80 tx=0x7f9b7800b040 comp rx=0 tx=0).ready entity=mgr.14750 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:53:35.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:35.154+0000 7f9b727fc700 1 -- 172.21.15.107:0/857526149 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+190121 (secure 0 0 0) 0x7f9b6c063420 con 0x7f9b7c0fff10 2024-05-17T21:53:35.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:35 smithi107 ceph-mon[28434]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:53:35.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:35 smithi107 ceph-mon[28434]: pgmap v8: 65 pgs: 65 active+clean; 2.1 GiB data, 8.1 GiB used, 528 GiB / 536 GiB avail; 743 KiB/s rd, 695 KiB/s wr, 103 op/s 2024-05-17T21:53:35.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:35 smithi107 ceph-mon[28434]: mgrmap e31: smithi107.xhhyiw(active, since 10s), standbys: smithi138.napnll 2024-05-17T21:53:35.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:35 smithi107 ceph-mon[28434]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mgr metadata", "who": "smithi138.napnll", "id": "smithi138.napnll"}]: dispatch 2024-05-17T21:53:35.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:35 smithi107 ceph-mon[28434]: from='client.? 172.21.15.107:0/2797373876' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-17T21:53:35.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:35 smithi107 ceph-mon[28434]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:53:35.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:35 smithi107 ceph-mon[28434]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:53:35.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:35 smithi107 ceph-mon[28434]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:53:35.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:35 smithi107 ceph-mon[28434]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T21:53:35.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:35.465+0000 7f9b83ab5700 1 -- 172.21.15.107:0/857526149 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f9b7c1058e0 con 0x7f9b7c0fff10 2024-05-17T21:53:35.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:35.465+0000 7f9b727fc700 1 -- 172.21.15.107:0/857526149 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f9b6c004490 con 0x7f9b7c0fff10 2024-05-17T21:53:35.457 INFO:teuthology.orchestra.run.smithi107.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-17T21:53:35.458 INFO:teuthology.orchestra.run.smithi107.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-17T21:53:35.458 INFO:teuthology.orchestra.run.smithi107.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-17T21:53:35.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:35.468+0000 7f9b83ab5700 1 -- 172.21.15.107:0/857526149 >> [v2:172.21.15.107:6800/416442229,v1:172.21.15.107:6801/416442229] conn(0x7f9b680783a0 msgr2=0x7f9b6807a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:53:35.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:35.468+0000 7f9b83ab5700 1 --2- 172.21.15.107:0/857526149 >> [v2:172.21.15.107:6800/416442229,v1:172.21.15.107:6801/416442229] conn(0x7f9b680783a0 0x7f9b6807a850 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9b78009b80 tx=0x7f9b7800b040 comp rx=0 tx=0).stop 2024-05-17T21:53:35.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:35.468+0000 7f9b83ab5700 1 -- 172.21.15.107:0/857526149 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b7c0fff10 msgr2=0x7f9b7c10df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:53:35.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:35.468+0000 7f9b83ab5700 1 --2- 172.21.15.107:0/857526149 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b7c0fff10 0x7f9b7c10df60 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c013870 tx=0x7f9b6c009e30 comp rx=0 tx=0).stop 2024-05-17T21:53:35.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:35.468+0000 7f9b83ab5700 1 -- 172.21.15.107:0/857526149 shutdown_connections 2024-05-17T21:53:35.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:35.468+0000 7f9b83ab5700 1 --2- 172.21.15.107:0/857526149 >> [v2:172.21.15.107:6800/416442229,v1:172.21.15.107:6801/416442229] conn(0x7f9b680783a0 0x7f9b6807a850 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:53:35.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:35.468+0000 7f9b83ab5700 1 --2- 172.21.15.107:0/857526149 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9b7c066b50 0x7f9b7c10da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:53:35.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:35.468+0000 7f9b83ab5700 1 --2- 172.21.15.107:0/857526149 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b7c0fff10 0x7f9b7c10df60 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:53:35.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:35.468+0000 7f9b83ab5700 1 -- 172.21.15.107:0/857526149 >> 172.21.15.107:0/857526149 conn(0x7f9b7c0f7440 msgr2=0x7f9b7c0f7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:53:35.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:35.468+0000 7f9b83ab5700 1 -- 172.21.15.107:0/857526149 shutdown_connections 2024-05-17T21:53:35.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:53:35.468+0000 7f9b83ab5700 1 -- 172.21.15.107:0/857526149 wait complete. 2024-05-17T21:53:35.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:35 smithi138 ceph-mon[35948]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:53:35.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:35 smithi138 ceph-mon[35948]: pgmap v8: 65 pgs: 65 active+clean; 2.1 GiB data, 8.1 GiB used, 528 GiB / 536 GiB avail; 743 KiB/s rd, 695 KiB/s wr, 103 op/s 2024-05-17T21:53:35.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:35 smithi138 ceph-mon[35948]: mgrmap e31: smithi107.xhhyiw(active, since 10s), standbys: smithi138.napnll 2024-05-17T21:53:35.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:35 smithi138 ceph-mon[35948]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mgr metadata", "who": "smithi138.napnll", "id": "smithi138.napnll"}]: dispatch 2024-05-17T21:53:35.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:35 smithi138 ceph-mon[35948]: from='client.? 172.21.15.107:0/2797373876' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-17T21:53:35.577 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:35 smithi138 ceph-mon[35948]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:53:35.577 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:35 smithi138 ceph-mon[35948]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:53:35.577 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:35 smithi138 ceph-mon[35948]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:53:35.577 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:35 smithi138 ceph-mon[35948]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T21:53:35.721 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-17T21:53:35.722 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-17T21:53:36.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:36 smithi138 ceph-mon[35948]: Updating smithi107:/etc/ceph/ceph.conf 2024-05-17T21:53:36.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:36 smithi138 ceph-mon[35948]: Updating smithi138:/etc/ceph/ceph.conf 2024-05-17T21:53:36.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:36 smithi138 ceph-mon[35948]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:53:36.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:36 smithi138 ceph-mon[35948]: Updating smithi107:/var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-17T21:53:36.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:36 smithi138 ceph-mon[35948]: Updating smithi138:/var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-17T21:53:36.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:36 smithi138 ceph-mon[35948]: from='client.? 172.21.15.107:0/857526149' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-17T21:53:36.577 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:36 smithi138 ceph-mon[35948]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:53:36.577 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:36 smithi138 ceph-mon[35948]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:53:36.577 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:36 smithi138 ceph-mon[35948]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:53:36.577 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:36 smithi138 ceph-mon[35948]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:53:36.577 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:36 smithi138 ceph-mon[35948]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:53:36.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:36 smithi107 ceph-mon[28434]: Updating smithi107:/etc/ceph/ceph.conf 2024-05-17T21:53:36.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:36 smithi107 ceph-mon[28434]: Updating smithi138:/etc/ceph/ceph.conf 2024-05-17T21:53:36.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:36 smithi107 ceph-mon[28434]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:53:36.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:36 smithi107 ceph-mon[28434]: Updating smithi107:/var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-17T21:53:36.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:36 smithi107 ceph-mon[28434]: Updating smithi138:/var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-17T21:53:36.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:36 smithi107 ceph-mon[28434]: from='client.? 172.21.15.107:0/857526149' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-17T21:53:36.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:36 smithi107 ceph-mon[28434]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:53:36.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:36 smithi107 ceph-mon[28434]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:53:36.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:36 smithi107 ceph-mon[28434]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:53:36.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:36 smithi107 ceph-mon[28434]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:53:36.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:36 smithi107 ceph-mon[28434]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:53:37.418 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-17T21:53:37.418 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1/tmp 2024-05-17T21:53:37.495 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:37 smithi138 ceph-mon[35948]: Updating smithi107:/etc/ceph/ceph.client.admin.keyring 2024-05-17T21:53:37.495 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:37 smithi138 ceph-mon[35948]: Updating smithi138:/etc/ceph/ceph.client.admin.keyring 2024-05-17T21:53:37.496 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:37 smithi138 ceph-mon[35948]: Updating smithi107:/var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-17T21:53:37.496 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:37 smithi138 ceph-mon[35948]: pgmap v9: 65 pgs: 65 active+clean; 1.0 GiB data, 5.2 GiB used, 531 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 160 op/s 2024-05-17T21:53:37.496 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:37 smithi138 ceph-mon[35948]: Updating smithi138:/var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-17T21:53:37.496 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:37 smithi138 ceph-mon[35948]: Reconfiguring prometheus.smithi107 (dependencies changed)... 2024-05-17T21:53:37.559 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:37 smithi107 ceph-mon[28434]: Updating smithi107:/etc/ceph/ceph.client.admin.keyring 2024-05-17T21:53:37.559 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:37 smithi107 ceph-mon[28434]: Updating smithi138:/etc/ceph/ceph.client.admin.keyring 2024-05-17T21:53:37.559 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:37 smithi107 ceph-mon[28434]: Updating smithi107:/var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-17T21:53:37.559 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:37 smithi107 ceph-mon[28434]: pgmap v9: 65 pgs: 65 active+clean; 1.0 GiB data, 5.2 GiB used, 531 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 160 op/s 2024-05-17T21:53:37.559 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:37 smithi107 ceph-mon[28434]: Updating smithi138:/var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-17T21:53:37.559 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:37 smithi107 ceph-mon[28434]: Reconfiguring prometheus.smithi107 (dependencies changed)... 2024-05-17T21:53:38.507 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:38 smithi107 ceph-mon[28434]: Reconfiguring daemon prometheus.smithi107 on smithi107 2024-05-17T21:53:38.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:38 smithi138 ceph-mon[35948]: Reconfiguring daemon prometheus.smithi107 on smithi107 2024-05-17T21:53:39.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:39 smithi138 ceph-mon[35948]: pgmap v10: 65 pgs: 65 active+clean; 1.0 GiB data, 5.2 GiB used, 531 GiB / 536 GiB avail; 1009 KiB/s rd, 1.0 MiB/s wr, 146 op/s 2024-05-17T21:53:39.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:39 smithi138 ceph-mon[35948]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:53:39.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:39 smithi107 ceph-mon[28434]: pgmap v10: 65 pgs: 65 active+clean; 1.0 GiB data, 5.2 GiB used, 531 GiB / 536 GiB avail; 1009 KiB/s rd, 1.0 MiB/s wr, 146 op/s 2024-05-17T21:53:39.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:39 smithi107 ceph-mon[28434]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:53:41.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:41 smithi138 ceph-mon[35948]: pgmap v11: 65 pgs: 65 active+clean; 1.0 GiB data, 5.2 GiB used, 531 GiB / 536 GiB avail; 1009 KiB/s rd, 1.0 MiB/s wr, 146 op/s 2024-05-17T21:53:41.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:41 smithi107 ceph-mon[28434]: pgmap v11: 65 pgs: 65 active+clean; 1.0 GiB data, 5.2 GiB used, 531 GiB / 536 GiB avail; 1009 KiB/s rd, 1.0 MiB/s wr, 146 op/s 2024-05-17T21:53:43.527 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:43 smithi107 ceph-mon[28434]: pgmap v12: 65 pgs: 65 active+clean; 280 MiB data, 2.9 GiB used, 534 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 217 op/s 2024-05-17T21:53:43.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:43 smithi138 ceph-mon[35948]: pgmap v12: 65 pgs: 65 active+clean; 280 MiB data, 2.9 GiB used, 534 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 217 op/s 2024-05-17T21:53:45.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:45 smithi138 ceph-mon[35948]: pgmap v13: 65 pgs: 65 active+clean; 280 MiB data, 2.9 GiB used, 534 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 140 op/s 2024-05-17T21:53:45.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:45 smithi138 ceph-mon[35948]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:53:45.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:45 smithi138 ceph-mon[35948]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:53:45.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:45 smithi138 ceph-mon[35948]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-17T21:53:45.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:45 smithi138 ceph-mon[35948]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:53:45.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:45 smithi107 ceph-mon[28434]: pgmap v13: 65 pgs: 65 active+clean; 280 MiB data, 2.9 GiB used, 534 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 140 op/s 2024-05-17T21:53:45.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:45 smithi107 ceph-mon[28434]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:53:45.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:45 smithi107 ceph-mon[28434]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:53:45.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:45 smithi107 ceph-mon[28434]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-17T21:53:45.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:45 smithi107 ceph-mon[28434]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:53:46.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:46 smithi138 ceph-mon[35948]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-17T21:53:46.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:46 smithi107 ceph-mon[28434]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-17T21:53:47.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:47 smithi138 ceph-mon[35948]: pgmap v14: 65 pgs: 65 active+clean; 287 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 190 op/s 2024-05-17T21:53:47.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:47 smithi107 ceph-mon[28434]: pgmap v14: 65 pgs: 65 active+clean; 287 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 190 op/s 2024-05-17T21:53:49.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:49 smithi138 ceph-mon[35948]: pgmap v15: 65 pgs: 65 active+clean; 287 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 121 op/s 2024-05-17T21:53:49.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:49 smithi107 ceph-mon[28434]: pgmap v15: 65 pgs: 65 active+clean; 287 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 121 op/s 2024-05-17T21:53:51.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:51 smithi138 ceph-mon[35948]: pgmap v16: 65 pgs: 65 active+clean; 287 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 121 op/s 2024-05-17T21:53:51.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:51 smithi107 ceph-mon[28434]: pgmap v16: 65 pgs: 65 active+clean; 287 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 121 op/s 2024-05-17T21:53:53.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:53 smithi138 ceph-mon[35948]: pgmap v17: 65 pgs: 65 active+clean; 294 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 163 op/s 2024-05-17T21:53:53.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:53 smithi107 ceph-mon[28434]: pgmap v17: 65 pgs: 65 active+clean; 294 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 163 op/s 2024-05-17T21:53:54.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:54 smithi138 ceph-mon[35948]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:53:54.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:54 smithi107 ceph-mon[28434]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:53:55.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:55 smithi138 ceph-mon[35948]: pgmap v18: 65 pgs: 65 active+clean; 294 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 92 op/s 2024-05-17T21:53:55.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:55 smithi107 ceph-mon[28434]: pgmap v18: 65 pgs: 65 active+clean; 294 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 92 op/s 2024-05-17T21:53:57.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:57 smithi138 ceph-mon[35948]: pgmap v19: 65 pgs: 65 active+clean; 302 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 131 op/s 2024-05-17T21:53:57.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:57 smithi107 ceph-mon[28434]: pgmap v19: 65 pgs: 65 active+clean; 302 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 131 op/s 2024-05-17T21:53:58.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:58 smithi138 ceph-mon[35948]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi138.napnll", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-17T21:53:58.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:58 smithi138 ceph-mon[35948]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-17T21:53:58.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:58 smithi138 ceph-mon[35948]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:53:58.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:58 smithi107 ceph-mon[28434]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi138.napnll", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-17T21:53:58.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:58 smithi107 ceph-mon[28434]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-17T21:53:58.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:58 smithi107 ceph-mon[28434]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:53:59.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:59 smithi138 ceph-mon[35948]: pgmap v20: 65 pgs: 65 active+clean; 302 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 81 op/s 2024-05-17T21:53:59.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:59 smithi138 ceph-mon[35948]: Upgrade: Updating mgr.smithi138.napnll 2024-05-17T21:53:59.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:53:59 smithi138 ceph-mon[35948]: Deploying daemon mgr.smithi138.napnll on smithi138 2024-05-17T21:53:59.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:59 smithi107 ceph-mon[28434]: pgmap v20: 65 pgs: 65 active+clean; 302 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 81 op/s 2024-05-17T21:53:59.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:59 smithi107 ceph-mon[28434]: Upgrade: Updating mgr.smithi138.napnll 2024-05-17T21:53:59.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:53:59 smithi107 ceph-mon[28434]: Deploying daemon mgr.smithi138.napnll on smithi138 2024-05-17T21:54:01.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:01 smithi138 ceph-mon[35948]: pgmap v21: 65 pgs: 65 active+clean; 302 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 81 op/s 2024-05-17T21:54:01.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:01 smithi107 ceph-mon[28434]: pgmap v21: 65 pgs: 65 active+clean; 302 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 81 op/s 2024-05-17T21:54:03.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:03 smithi138 ceph-mon[35948]: pgmap v22: 65 pgs: 65 active+clean; 308 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 125 op/s 2024-05-17T21:54:03.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:03 smithi107 ceph-mon[28434]: pgmap v22: 65 pgs: 65 active+clean; 308 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 125 op/s 2024-05-17T21:54:04.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:04 smithi107 ceph-mon[28434]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:54:04.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:04 smithi107 ceph-mon[28434]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:54:04.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:04 smithi107 ceph-mon[28434]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:54:04.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:04 smithi107 ceph-mon[28434]: pgmap v23: 65 pgs: 65 active+clean; 308 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 82 op/s 2024-05-17T21:54:04.719 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:04 smithi138 ceph-mon[35948]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:54:04.719 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:04 smithi138 ceph-mon[35948]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:54:04.719 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:04 smithi138 ceph-mon[35948]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:54:04.719 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:04 smithi138 ceph-mon[35948]: pgmap v23: 65 pgs: 65 active+clean; 308 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 82 op/s 2024-05-17T21:54:05.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:05.586+0000 7f4ae8ef6700 1 -- 172.21.15.107:0/3775721187 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ae40fed30 msgr2=0x7f4ae40fae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:54:05.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:05.586+0000 7f4ae8ef6700 1 --2- 172.21.15.107:0/3775721187 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ae40fed30 0x7f4ae40fae70 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f4ad8004940 tx=0x7f4ad8009e30 comp rx=0 tx=0).stop 2024-05-17T21:54:05.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:05.586+0000 7f4ae8ef6700 1 -- 172.21.15.107:0/3775721187 shutdown_connections 2024-05-17T21:54:05.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:05.586+0000 7f4ae8ef6700 1 --2- 172.21.15.107:0/3775721187 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ae40fed30 0x7f4ae40fae70 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:54:05.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:05.586+0000 7f4ae8ef6700 1 --2- 172.21.15.107:0/3775721187 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ae40fe3e0 0x7f4ae40fe7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:54:05.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:05.586+0000 7f4ae8ef6700 1 -- 172.21.15.107:0/3775721187 >> 172.21.15.107:0/3775721187 conn(0x7f4ae40f6b80 msgr2=0x7f4ae40f8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:54:05.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:05.586+0000 7f4ae8ef6700 1 -- 172.21.15.107:0/3775721187 shutdown_connections 2024-05-17T21:54:05.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:05.586+0000 7f4ae8ef6700 1 -- 172.21.15.107:0/3775721187 wait complete. 2024-05-17T21:54:05.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:05.587+0000 7f4ae8ef6700 1 Processor -- start 2024-05-17T21:54:05.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:05.587+0000 7f4ae8ef6700 1 -- start start 2024-05-17T21:54:05.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:05.587+0000 7f4ae8ef6700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ae40fe3e0 0x7f4ae4113f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:54:05.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:05.587+0000 7f4ae8ef6700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ae40fed30 0x7f4ae41144c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:54:05.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:05.587+0000 7f4ae8ef6700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ae4114ac0 con 0x7f4ae40fed30 2024-05-17T21:54:05.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:05.587+0000 7f4ae8ef6700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ae4115310 con 0x7f4ae40fe3e0 2024-05-17T21:54:05.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:05.587+0000 7f4ae1d9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ae40fed30 0x7f4ae41144c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:54:05.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:05.587+0000 7f4ae1d9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ae40fed30 0x7f4ae41144c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51860/0 (socket says 172.21.15.107:51860) 2024-05-17T21:54:05.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:05.587+0000 7f4ae1d9b700 1 -- 172.21.15.107:0/1048221923 learned_addr learned my addr 172.21.15.107:0/1048221923 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:54:05.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:05.587+0000 7f4ae259c700 1 --2- 172.21.15.107:0/1048221923 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ae40fe3e0 0x7f4ae4113f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:54:05.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:05.588+0000 7f4ae259c700 1 -- 172.21.15.107:0/1048221923 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ae40fed30 msgr2=0x7f4ae41144c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:54:05.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:05.588+0000 7f4ae259c700 1 --2- 172.21.15.107:0/1048221923 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ae40fed30 0x7f4ae41144c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:54:05.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:05.588+0000 7f4ae259c700 1 -- 172.21.15.107:0/1048221923 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ad8005040 con 0x7f4ae40fe3e0 2024-05-17T21:54:05.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:05.588+0000 7f4ae1d9b700 1 --2- 172.21.15.107:0/1048221923 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ae40fed30 0x7f4ae41144c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T21:54:05.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:05.588+0000 7f4ae259c700 1 --2- 172.21.15.107:0/1048221923 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ae40fe3e0 0x7f4ae4113f80 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4ad4009fd0 tx=0x7f4ad400b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:54:05.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:05.588+0000 7f4ad2ffd700 1 -- 172.21.15.107:0/1048221923 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ad4004e90 con 0x7f4ae40fe3e0 2024-05-17T21:54:05.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:05.588+0000 7f4ae8ef6700 1 -- 172.21.15.107:0/1048221923 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ae4115590 con 0x7f4ae40fe3e0 2024-05-17T21:54:05.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:05.588+0000 7f4ad2ffd700 1 -- 172.21.15.107:0/1048221923 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4ad400f040 con 0x7f4ae40fe3e0 2024-05-17T21:54:05.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:05.588+0000 7f4ad2ffd700 1 -- 172.21.15.107:0/1048221923 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ad400ca50 con 0x7f4ae40fe3e0 2024-05-17T21:54:05.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:05.588+0000 7f4ae8ef6700 1 -- 172.21.15.107:0/1048221923 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ae4115ad0 con 0x7f4ae40fe3e0 2024-05-17T21:54:05.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:05.589+0000 7f4ae8ef6700 1 -- 172.21.15.107:0/1048221923 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ae4101b20 con 0x7f4ae40fe3e0 2024-05-17T21:54:05.584 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:05.590+0000 7f4ad2ffd700 1 -- 172.21.15.107:0/1048221923 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 31) v1 ==== 99906+0+0 (secure 0 0 0) 0x7f4ad4007cc0 con 0x7f4ae40fe3e0 2024-05-17T21:54:05.584 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:05.591+0000 7f4ad2ffd700 1 --2- 172.21.15.107:0/1048221923 >> [v2:172.21.15.107:6800/416442229,v1:172.21.15.107:6801/416442229] conn(0x7f4acc0783a0 0x7f4acc07a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:54:05.584 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:05.591+0000 7f4ad2ffd700 1 -- 172.21.15.107:0/1048221923 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f4ad409a320 con 0x7f4ae40fe3e0 2024-05-17T21:54:05.584 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:05.591+0000 7f4ae1d9b700 1 --2- 172.21.15.107:0/1048221923 >> [v2:172.21.15.107:6800/416442229,v1:172.21.15.107:6801/416442229] conn(0x7f4acc0783a0 0x7f4acc07a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:54:05.584 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:05.591+0000 7f4ae1d9b700 1 --2- 172.21.15.107:0/1048221923 >> [v2:172.21.15.107:6800/416442229,v1:172.21.15.107:6801/416442229] conn(0x7f4acc0783a0 0x7f4acc07a850 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4ad800b0d0 tx=0x7f4ad8009e30 comp rx=0 tx=0).ready entity=mgr.14750 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:54:05.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:05.596+0000 7f4ad2ffd700 1 -- 172.21.15.107:0/1048221923 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+190121 (secure 0 0 0) 0x7f4ad4062520 con 0x7f4ae40fe3e0 2024-05-17T21:54:05.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:05.843+0000 7f4ae8ef6700 1 -- 172.21.15.107:0/1048221923 --> [v2:172.21.15.107:6800/416442229,v1:172.21.15.107:6801/416442229] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4ae41059a0 con 0x7f4acc0783a0 2024-05-17T21:54:05.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:05.844+0000 7f4ad2ffd700 1 -- 172.21.15.107:0/1048221923 <== mgr.14750 v2:172.21.15.107:6800/416442229 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+328 (secure 0 0 0) 0x7f4ae41059a0 con 0x7f4acc0783a0 2024-05-17T21:54:05.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:05.847+0000 7f4ae8ef6700 1 -- 172.21.15.107:0/1048221923 >> [v2:172.21.15.107:6800/416442229,v1:172.21.15.107:6801/416442229] conn(0x7f4acc0783a0 msgr2=0x7f4acc07a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:54:05.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:05.847+0000 7f4ae8ef6700 1 --2- 172.21.15.107:0/1048221923 >> [v2:172.21.15.107:6800/416442229,v1:172.21.15.107:6801/416442229] conn(0x7f4acc0783a0 0x7f4acc07a850 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4ad800b0d0 tx=0x7f4ad8009e30 comp rx=0 tx=0).stop 2024-05-17T21:54:05.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:05.847+0000 7f4ae8ef6700 1 -- 172.21.15.107:0/1048221923 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ae40fe3e0 msgr2=0x7f4ae4113f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:54:05.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:05.847+0000 7f4ae8ef6700 1 --2- 172.21.15.107:0/1048221923 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ae40fe3e0 0x7f4ae4113f80 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4ad4009fd0 tx=0x7f4ad400b040 comp rx=0 tx=0).stop 2024-05-17T21:54:05.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:05.847+0000 7f4ae8ef6700 1 -- 172.21.15.107:0/1048221923 shutdown_connections 2024-05-17T21:54:05.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:05.847+0000 7f4ae8ef6700 1 --2- 172.21.15.107:0/1048221923 >> [v2:172.21.15.107:6800/416442229,v1:172.21.15.107:6801/416442229] conn(0x7f4acc0783a0 0x7f4acc07a850 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:54:05.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:05.847+0000 7f4ae8ef6700 1 --2- 172.21.15.107:0/1048221923 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ae40fe3e0 0x7f4ae4113f80 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:54:05.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:05.847+0000 7f4ae8ef6700 1 --2- 172.21.15.107:0/1048221923 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ae40fed30 0x7f4ae41144c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:54:05.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:05.847+0000 7f4ae8ef6700 1 -- 172.21.15.107:0/1048221923 >> 172.21.15.107:0/1048221923 conn(0x7f4ae40f6b80 msgr2=0x7f4ae40f81b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:54:05.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:05.847+0000 7f4ae8ef6700 1 -- 172.21.15.107:0/1048221923 shutdown_connections 2024-05-17T21:54:05.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:05.847+0000 7f4ae8ef6700 1 -- 172.21.15.107:0/1048221923 wait complete. 2024-05-17T21:54:05.850 INFO:teuthology.orchestra.run.smithi107.stdout:true 2024-05-17T21:54:05.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:05.962+0000 7f1b3b52d700 1 -- 172.21.15.107:0/1088735294 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b340fcf60 msgr2=0x7f1b340fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:54:05.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:05.962+0000 7f1b3b52d700 1 --2- 172.21.15.107:0/1088735294 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b340fcf60 0x7f1b340fd3d0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f1b24004940 tx=0x7f1b24009e30 comp rx=0 tx=0).stop 2024-05-17T21:54:05.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:05.962+0000 7f1b3b52d700 1 -- 172.21.15.107:0/1088735294 shutdown_connections 2024-05-17T21:54:05.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:05.962+0000 7f1b3b52d700 1 --2- 172.21.15.107:0/1088735294 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b340fcf60 0x7f1b340fd3d0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:54:05.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:05.962+0000 7f1b3b52d700 1 --2- 172.21.15.107:0/1088735294 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1b340fbcb0 0x7f1b340fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:54:05.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:05.962+0000 7f1b3b52d700 1 -- 172.21.15.107:0/1088735294 >> 172.21.15.107:0/1088735294 conn(0x7f1b340f7430 msgr2=0x7f1b340f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:54:05.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:05.963+0000 7f1b3b52d700 1 -- 172.21.15.107:0/1088735294 shutdown_connections 2024-05-17T21:54:05.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:05.963+0000 7f1b3b52d700 1 -- 172.21.15.107:0/1088735294 wait complete. 2024-05-17T21:54:05.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:05.963+0000 7f1b3b52d700 1 Processor -- start 2024-05-17T21:54:05.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:05.963+0000 7f1b3b52d700 1 -- start start 2024-05-17T21:54:05.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:05.963+0000 7f1b3b52d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b340fbcb0 0x7f1b3410d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:54:05.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:05.963+0000 7f1b3b52d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1b340fcf60 0x7f1b3410df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:54:05.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:05.963+0000 7f1b3b52d700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b3410e500 con 0x7f1b340fbcb0 2024-05-17T21:54:05.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:05.963+0000 7f1b3b52d700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b3410ed00 con 0x7f1b340fcf60 2024-05-17T21:54:05.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:05.963+0000 7f1b392c9700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b340fbcb0 0x7f1b3410d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:54:05.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:05.964+0000 7f1b392c9700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b340fbcb0 0x7f1b3410d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51882/0 (socket says 172.21.15.107:51882) 2024-05-17T21:54:05.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:05.964+0000 7f1b392c9700 1 -- 172.21.15.107:0/4288667117 learned_addr learned my addr 172.21.15.107:0/4288667117 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:54:05.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:05.964+0000 7f1b38ac8700 1 --2- 172.21.15.107:0/4288667117 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1b340fcf60 0x7f1b3410df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:54:05.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:05.964+0000 7f1b392c9700 1 -- 172.21.15.107:0/4288667117 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1b340fcf60 msgr2=0x7f1b3410df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:54:05.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:05.964+0000 7f1b392c9700 1 --2- 172.21.15.107:0/4288667117 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1b340fcf60 0x7f1b3410df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:54:05.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:05.964+0000 7f1b392c9700 1 -- 172.21.15.107:0/4288667117 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b24005040 con 0x7f1b340fbcb0 2024-05-17T21:54:05.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:05.964+0000 7f1b392c9700 1 --2- 172.21.15.107:0/4288667117 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b340fbcb0 0x7f1b3410d9c0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f1b3000eb40 tx=0x7f1b3000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:54:05.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:05.964+0000 7f1b29ffb700 1 -- 172.21.15.107:0/4288667117 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1b30007cc0 con 0x7f1b340fbcb0 2024-05-17T21:54:05.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:05.964+0000 7f1b3b52d700 1 -- 172.21.15.107:0/4288667117 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b3410ef80 con 0x7f1b340fbcb0 2024-05-17T21:54:05.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:05.964+0000 7f1b29ffb700 1 -- 172.21.15.107:0/4288667117 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1b3000f040 con 0x7f1b340fbcb0 2024-05-17T21:54:05.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:05.964+0000 7f1b29ffb700 1 -- 172.21.15.107:0/4288667117 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1b3000ca10 con 0x7f1b340fbcb0 2024-05-17T21:54:05.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:05.964+0000 7f1b3b52d700 1 -- 172.21.15.107:0/4288667117 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b3410f4c0 con 0x7f1b340fbcb0 2024-05-17T21:54:05.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:05.965+0000 7f1b3b52d700 1 -- 172.21.15.107:0/4288667117 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b34101af0 con 0x7f1b340fbcb0 2024-05-17T21:54:05.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:05.966+0000 7f1b29ffb700 1 -- 172.21.15.107:0/4288667117 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 31) v1 ==== 99906+0+0 (secure 0 0 0) 0x7f1b3000cc80 con 0x7f1b340fbcb0 2024-05-17T21:54:05.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:05.966+0000 7f1b29ffb700 1 --2- 172.21.15.107:0/4288667117 >> [v2:172.21.15.107:6800/416442229,v1:172.21.15.107:6801/416442229] conn(0x7f1b200783a0 0x7f1b2007a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:54:05.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:05.967+0000 7f1b29ffb700 1 -- 172.21.15.107:0/4288667117 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f1b30099be0 con 0x7f1b340fbcb0 2024-05-17T21:54:05.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:05.967+0000 7f1b38ac8700 1 --2- 172.21.15.107:0/4288667117 >> [v2:172.21.15.107:6800/416442229,v1:172.21.15.107:6801/416442229] conn(0x7f1b200783a0 0x7f1b2007a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:54:05.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:05.967+0000 7f1b38ac8700 1 --2- 172.21.15.107:0/4288667117 >> [v2:172.21.15.107:6800/416442229,v1:172.21.15.107:6801/416442229] conn(0x7f1b200783a0 0x7f1b2007a850 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f1b24009b30 tx=0x7f1b24009e30 comp rx=0 tx=0).ready entity=mgr.14750 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:54:05.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:05.972+0000 7f1b29ffb700 1 -- 172.21.15.107:0/4288667117 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+190121 (secure 0 0 0) 0x7f1b30061f30 con 0x7f1b340fbcb0 2024-05-17T21:54:06.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:06.215+0000 7f1b3b52d700 1 -- 172.21.15.107:0/4288667117 --> [v2:172.21.15.107:6800/416442229,v1:172.21.15.107:6801/416442229] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1b34104910 con 0x7f1b200783a0 2024-05-17T21:54:06.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:06.216+0000 7f1b29ffb700 1 -- 172.21.15.107:0/4288667117 <== mgr.14750 v2:172.21.15.107:6800/416442229 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+328 (secure 0 0 0) 0x7f1b34104910 con 0x7f1b200783a0 2024-05-17T21:54:06.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:06.219+0000 7f1b3b52d700 1 -- 172.21.15.107:0/4288667117 >> [v2:172.21.15.107:6800/416442229,v1:172.21.15.107:6801/416442229] conn(0x7f1b200783a0 msgr2=0x7f1b2007a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:54:06.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:06.219+0000 7f1b3b52d700 1 --2- 172.21.15.107:0/4288667117 >> [v2:172.21.15.107:6800/416442229,v1:172.21.15.107:6801/416442229] conn(0x7f1b200783a0 0x7f1b2007a850 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f1b24009b30 tx=0x7f1b24009e30 comp rx=0 tx=0).stop 2024-05-17T21:54:06.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:06.219+0000 7f1b3b52d700 1 -- 172.21.15.107:0/4288667117 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b340fbcb0 msgr2=0x7f1b3410d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:54:06.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:06.219+0000 7f1b3b52d700 1 --2- 172.21.15.107:0/4288667117 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b340fbcb0 0x7f1b3410d9c0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f1b3000eb40 tx=0x7f1b3000b040 comp rx=0 tx=0).stop 2024-05-17T21:54:06.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:06.219+0000 7f1b3b52d700 1 -- 172.21.15.107:0/4288667117 shutdown_connections 2024-05-17T21:54:06.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:06.219+0000 7f1b3b52d700 1 --2- 172.21.15.107:0/4288667117 >> [v2:172.21.15.107:6800/416442229,v1:172.21.15.107:6801/416442229] conn(0x7f1b200783a0 0x7f1b2007a850 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:54:06.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:06.219+0000 7f1b3b52d700 1 --2- 172.21.15.107:0/4288667117 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b340fbcb0 0x7f1b3410d9c0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:54:06.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:06.219+0000 7f1b3b52d700 1 --2- 172.21.15.107:0/4288667117 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1b340fcf60 0x7f1b3410df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:54:06.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:06.219+0000 7f1b3b52d700 1 -- 172.21.15.107:0/4288667117 >> 172.21.15.107:0/4288667117 conn(0x7f1b340f7430 msgr2=0x7f1b34065b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:54:06.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:06.219+0000 7f1b3b52d700 1 -- 172.21.15.107:0/4288667117 shutdown_connections 2024-05-17T21:54:06.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:06.219+0000 7f1b3b52d700 1 -- 172.21.15.107:0/4288667117 wait complete. 2024-05-17T21:54:06.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:06.329+0000 7f4417e7f700 1 -- 172.21.15.107:0/3700621619 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44100fed30 msgr2=0x7f44100fae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:54:06.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:06.329+0000 7f4417e7f700 1 --2- 172.21.15.107:0/3700621619 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44100fed30 0x7f44100fae70 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f4400004940 tx=0x7f4400009e30 comp rx=0 tx=0).stop 2024-05-17T21:54:06.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:06.329+0000 7f4417e7f700 1 -- 172.21.15.107:0/3700621619 shutdown_connections 2024-05-17T21:54:06.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:06.329+0000 7f4417e7f700 1 --2- 172.21.15.107:0/3700621619 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44100fed30 0x7f44100fae70 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:54:06.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:06.329+0000 7f4417e7f700 1 --2- 172.21.15.107:0/3700621619 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f44100fe3e0 0x7f44100fe7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:54:06.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:06.329+0000 7f4417e7f700 1 -- 172.21.15.107:0/3700621619 >> 172.21.15.107:0/3700621619 conn(0x7f44100f6b80 msgr2=0x7f44100f8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:54:06.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:06.330+0000 7f4417e7f700 1 -- 172.21.15.107:0/3700621619 shutdown_connections 2024-05-17T21:54:06.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:06.330+0000 7f4417e7f700 1 -- 172.21.15.107:0/3700621619 wait complete. 2024-05-17T21:54:06.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:06.331+0000 7f4417e7f700 1 Processor -- start 2024-05-17T21:54:06.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:06.331+0000 7f4417e7f700 1 -- start start 2024-05-17T21:54:06.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:06.331+0000 7f4417e7f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44100fe3e0 0x7f4410114000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:54:06.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:06.331+0000 7f4417e7f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f44100fed30 0x7f4410114540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:54:06.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:06.331+0000 7f4417e7f700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4410114b40 con 0x7f44100fe3e0 2024-05-17T21:54:06.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:06.331+0000 7f4417e7f700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4410115340 con 0x7f44100fed30 2024-05-17T21:54:06.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:06.331+0000 7f4415c1b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44100fe3e0 0x7f4410114000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:54:06.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:06.331+0000 7f4415c1b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44100fe3e0 0x7f4410114000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51900/0 (socket says 172.21.15.107:51900) 2024-05-17T21:54:06.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:06.331+0000 7f4415c1b700 1 -- 172.21.15.107:0/456917012 learned_addr learned my addr 172.21.15.107:0/456917012 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:54:06.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:06.332+0000 7f441541a700 1 --2- 172.21.15.107:0/456917012 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f44100fed30 0x7f4410114540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:54:06.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:06.332+0000 7f4415c1b700 1 -- 172.21.15.107:0/456917012 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f44100fed30 msgr2=0x7f4410114540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:54:06.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:06.332+0000 7f4415c1b700 1 --2- 172.21.15.107:0/456917012 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f44100fed30 0x7f4410114540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:54:06.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:06.332+0000 7f4415c1b700 1 -- 172.21.15.107:0/456917012 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4400005040 con 0x7f44100fe3e0 2024-05-17T21:54:06.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:06.332+0000 7f4415c1b700 1 --2- 172.21.15.107:0/456917012 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44100fe3e0 0x7f4410114000 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f440c002a50 tx=0x7f440c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:54:06.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:06.332+0000 7f44067fc700 1 -- 172.21.15.107:0/456917012 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f440c007cc0 con 0x7f44100fe3e0 2024-05-17T21:54:06.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:06.332+0000 7f44067fc700 1 -- 172.21.15.107:0/456917012 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f440c00f040 con 0x7f44100fe3e0 2024-05-17T21:54:06.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:06.332+0000 7f4417e7f700 1 -- 172.21.15.107:0/456917012 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44101155c0 con 0x7f44100fe3e0 2024-05-17T21:54:06.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:06.332+0000 7f44067fc700 1 -- 172.21.15.107:0/456917012 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f440c00da30 con 0x7f44100fe3e0 2024-05-17T21:54:06.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:06.332+0000 7f4417e7f700 1 -- 172.21.15.107:0/456917012 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4410115b80 con 0x7f44100fe3e0 2024-05-17T21:54:06.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:06.334+0000 7f4417e7f700 1 -- 172.21.15.107:0/456917012 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4410101b20 con 0x7f44100fe3e0 2024-05-17T21:54:06.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:06.335+0000 7f44067fc700 1 -- 172.21.15.107:0/456917012 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 31) v1 ==== 99906+0+0 (secure 0 0 0) 0x7f440c00db90 con 0x7f44100fe3e0 2024-05-17T21:54:06.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:06.336+0000 7f44067fc700 1 --2- 172.21.15.107:0/456917012 >> [v2:172.21.15.107:6800/416442229,v1:172.21.15.107:6801/416442229] conn(0x7f43fc0783a0 0x7f43fc07a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:54:06.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:06.336+0000 7f44067fc700 1 -- 172.21.15.107:0/456917012 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f440c099cf0 con 0x7f44100fe3e0 2024-05-17T21:54:06.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:06.337+0000 7f441541a700 1 --2- 172.21.15.107:0/456917012 >> [v2:172.21.15.107:6800/416442229,v1:172.21.15.107:6801/416442229] conn(0x7f43fc0783a0 0x7f43fc07a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:54:06.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:06.338+0000 7f441541a700 1 --2- 172.21.15.107:0/456917012 >> [v2:172.21.15.107:6800/416442229,v1:172.21.15.107:6801/416442229] conn(0x7f43fc0783a0 0x7f43fc07a850 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f4400009fd0 tx=0x7f4400009e30 comp rx=0 tx=0).ready entity=mgr.14750 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:54:06.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:06.344+0000 7f44067fc700 1 -- 172.21.15.107:0/456917012 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+190121 (secure 0 0 0) 0x7f440c062010 con 0x7f44100fe3e0 2024-05-17T21:54:06.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:06.634+0000 7f4417e7f700 1 -- 172.21.15.107:0/456917012 --> [v2:172.21.15.107:6800/416442229,v1:172.21.15.107:6801/416442229] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f4410115df0 con 0x7f43fc0783a0 2024-05-17T21:54:06.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:06.644+0000 7f44067fc700 1 -- 172.21.15.107:0/456917012 <== mgr.14750 v2:172.21.15.107:6800/416442229 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f4410115df0 con 0x7f43fc0783a0 2024-05-17T21:54:06.637 INFO:teuthology.orchestra.run.smithi107.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-17T21:54:06.637 INFO:teuthology.orchestra.run.smithi107.stdout:alertmanager.smithi107 smithi107 *:9093,9094 running (100s) 37s ago 10m 17.2M - 0.25.0 c8568f914cd2 bac2a944ae42 2024-05-17T21:54:06.637 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi107 smithi107 running (10m) 37s ago 10m 6660k - 17.2.7-952-g9903f776 d15a36853796 828741c3a89c 2024-05-17T21:54:06.637 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi138 smithi138 running (9m) 38s ago 9m 6660k - 17.2.7-952-g9903f776 d15a36853796 20e9ccf1fadd 2024-05-17T21:54:06.637 INFO:teuthology.orchestra.run.smithi107.stdout:grafana.smithi107 smithi107 *:3000 running (94s) 37s ago 9m 79.5M - 9.4.7 954c08fa6188 8c747e777bc5 2024-05-17T21:54:06.637 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.drqody smithi107 running (4m) 37s ago 4m 261M - 17.2.7-952-g9903f776 d15a36853796 0d851bc912f0 2024-05-17T21:54:06.637 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.idmbgj smithi107 running (4m) 37s ago 4m 12.4M - 17.2.7-952-g9903f776 d15a36853796 1348b09ab29e 2024-05-17T21:54:06.637 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi138.gwbtyd smithi138 running (4m) 38s ago 4m 252M - 17.2.7-952-g9903f776 d15a36853796 fd2ba75354c0 2024-05-17T21:54:06.637 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi138.zckrvs smithi138 running (4m) 38s ago 4m 13.1M - 17.2.7-952-g9903f776 d15a36853796 a1cfd2fe4cda 2024-05-17T21:54:06.637 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi107.xhhyiw smithi107 *:8443,9283,8765 running (58s) 37s ago 11m 556M - 19.0.0-3794-g931809a9 8880f7613d02 5acf3e511342 2024-05-17T21:54:06.638 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi138.napnll smithi138 *:8443,9283,8765 starting - - - - 2024-05-17T21:54:06.638 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi107 smithi107 running (11m) 37s ago 11m 57.4M 2048M 17.2.7-952-g9903f776 d15a36853796 7de9c23eee97 2024-05-17T21:54:06.638 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi138 smithi138 running (9m) 38s ago 9m 43.7M 2048M 17.2.7-952-g9903f776 d15a36853796 ee61a065778d 2024-05-17T21:54:06.638 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi107 smithi107 *:9100 running (106s) 37s ago 10m 8292k - 1.5.0 0da6a335fe13 49bec71f5487 2024-05-17T21:54:06.638 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi138 smithi138 *:9100 running (75s) 38s ago 9m 8354k - 1.5.0 0da6a335fe13 e52824376658 2024-05-17T21:54:06.638 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0 smithi107 running (8m) 37s ago 8m 180M 1557M 17.2.7-952-g9903f776 d15a36853796 f2d20009361d 2024-05-17T21:54:06.638 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1 smithi107 running (8m) 37s ago 8m 187M 1557M 17.2.7-952-g9903f776 d15a36853796 e5d3e99844a9 2024-05-17T21:54:06.638 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2 smithi107 running (7m) 37s ago 7m 153M 1557M 17.2.7-952-g9903f776 d15a36853796 e044d7b78088 2024-05-17T21:54:06.638 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3 smithi138 running (7m) 38s ago 7m 201M 2581M 17.2.7-952-g9903f776 d15a36853796 8c980cc8af87 2024-05-17T21:54:06.638 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4 smithi138 running (6m) 38s ago 6m 185M 2581M 17.2.7-952-g9903f776 d15a36853796 b306a36a744e 2024-05-17T21:54:06.638 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5 smithi138 running (6m) 38s ago 6m 172M 2581M 17.2.7-952-g9903f776 d15a36853796 7c8d8ed74974 2024-05-17T21:54:06.638 INFO:teuthology.orchestra.run.smithi107.stdout:prometheus.smithi107 smithi107 *:9095 starting - - - - 2024-05-17T21:54:06.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:06.647+0000 7f4417e7f700 1 -- 172.21.15.107:0/456917012 >> [v2:172.21.15.107:6800/416442229,v1:172.21.15.107:6801/416442229] conn(0x7f43fc0783a0 msgr2=0x7f43fc07a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:54:06.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:06.647+0000 7f4417e7f700 1 --2- 172.21.15.107:0/456917012 >> [v2:172.21.15.107:6800/416442229,v1:172.21.15.107:6801/416442229] conn(0x7f43fc0783a0 0x7f43fc07a850 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f4400009fd0 tx=0x7f4400009e30 comp rx=0 tx=0).stop 2024-05-17T21:54:06.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:06.647+0000 7f4417e7f700 1 -- 172.21.15.107:0/456917012 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44100fe3e0 msgr2=0x7f4410114000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:54:06.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:06.647+0000 7f4417e7f700 1 --2- 172.21.15.107:0/456917012 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44100fe3e0 0x7f4410114000 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f440c002a50 tx=0x7f440c0046b0 comp rx=0 tx=0).stop 2024-05-17T21:54:06.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:06.647+0000 7f4417e7f700 1 -- 172.21.15.107:0/456917012 shutdown_connections 2024-05-17T21:54:06.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:06.647+0000 7f4417e7f700 1 --2- 172.21.15.107:0/456917012 >> [v2:172.21.15.107:6800/416442229,v1:172.21.15.107:6801/416442229] conn(0x7f43fc0783a0 0x7f43fc07a850 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:54:06.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:06.647+0000 7f4417e7f700 1 --2- 172.21.15.107:0/456917012 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44100fe3e0 0x7f4410114000 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:54:06.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:06.647+0000 7f4417e7f700 1 --2- 172.21.15.107:0/456917012 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f44100fed30 0x7f4410114540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:54:06.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:06.647+0000 7f4417e7f700 1 -- 172.21.15.107:0/456917012 >> 172.21.15.107:0/456917012 conn(0x7f44100f6b80 msgr2=0x7f44100f8220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:54:06.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:06.647+0000 7f4417e7f700 1 -- 172.21.15.107:0/456917012 shutdown_connections 2024-05-17T21:54:06.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:06.647+0000 7f4417e7f700 1 -- 172.21.15.107:0/456917012 wait complete. 2024-05-17T21:54:06.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:06.755+0000 7f33a98f4700 1 -- 172.21.15.107:0/3976888129 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33a40faa10 msgr2=0x7f33a40fae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:54:06.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:06.755+0000 7f33a98f4700 1 --2- 172.21.15.107:0/3976888129 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33a40faa10 0x7f33a40fae20 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f3394004940 tx=0x7f3394009e30 comp rx=0 tx=0).stop 2024-05-17T21:54:06.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:06.755+0000 7f33a98f4700 1 -- 172.21.15.107:0/3976888129 shutdown_connections 2024-05-17T21:54:06.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:06.755+0000 7f33a98f4700 1 --2- 172.21.15.107:0/3976888129 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f33a40fb360 0x7f33a41025e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:54:06.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:06.755+0000 7f33a98f4700 1 --2- 172.21.15.107:0/3976888129 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33a40faa10 0x7f33a40fae20 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:54:06.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:06.755+0000 7f33a98f4700 1 -- 172.21.15.107:0/3976888129 >> 172.21.15.107:0/3976888129 conn(0x7f33a40f6740 msgr2=0x7f33a40f8b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:54:06.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:06.755+0000 7f33a98f4700 1 -- 172.21.15.107:0/3976888129 shutdown_connections 2024-05-17T21:54:06.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:06.755+0000 7f33a98f4700 1 -- 172.21.15.107:0/3976888129 wait complete. 2024-05-17T21:54:06.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:06.756+0000 7f33a98f4700 1 Processor -- start 2024-05-17T21:54:06.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:06.756+0000 7f33a98f4700 1 -- start start 2024-05-17T21:54:06.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:06.756+0000 7f33a98f4700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f33a40faa10 0x7f33a40fde20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:54:06.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:06.756+0000 7f33a98f4700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33a40fb360 0x7f33a40fc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:54:06.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:06.756+0000 7f33a98f4700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a40fe420 con 0x7f33a40fb360 2024-05-17T21:54:06.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:06.756+0000 7f33a98f4700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a41115b0 con 0x7f33a40faa10 2024-05-17T21:54:06.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:06.756+0000 7f339bfff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33a40fb360 0x7f33a40fc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:54:06.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:06.756+0000 7f339bfff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33a40fb360 0x7f33a40fc4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51922/0 (socket says 172.21.15.107:51922) 2024-05-17T21:54:06.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:06.756+0000 7f339bfff700 1 -- 172.21.15.107:0/3201027630 learned_addr learned my addr 172.21.15.107:0/3201027630 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:54:06.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:06.756+0000 7f33a2ffd700 1 --2- 172.21.15.107:0/3201027630 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f33a40faa10 0x7f33a40fde20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:54:06.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:06.757+0000 7f339bfff700 1 -- 172.21.15.107:0/3201027630 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f33a40faa10 msgr2=0x7f33a40fde20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:54:06.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:06.757+0000 7f339bfff700 1 --2- 172.21.15.107:0/3201027630 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f33a40faa10 0x7f33a40fde20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:54:06.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:06.757+0000 7f339bfff700 1 -- 172.21.15.107:0/3201027630 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3394005040 con 0x7f33a40fb360 2024-05-17T21:54:06.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:06.757+0000 7f33a2ffd700 1 --2- 172.21.15.107:0/3201027630 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f33a40faa10 0x7f33a40fde20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T21:54:06.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:06.757+0000 7f339bfff700 1 --2- 172.21.15.107:0/3201027630 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33a40fb360 0x7f33a40fc4d0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f338c00eb40 tx=0x7f338c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:54:06.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:06.757+0000 7f339b7fe700 1 -- 172.21.15.107:0/3201027630 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f338c007cc0 con 0x7f33a40fb360 2024-05-17T21:54:06.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:06.757+0000 7f339b7fe700 1 -- 172.21.15.107:0/3201027630 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f338c00f040 con 0x7f33a40fb360 2024-05-17T21:54:06.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:06.757+0000 7f33a98f4700 1 -- 172.21.15.107:0/3201027630 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33a4111800 con 0x7f33a40fb360 2024-05-17T21:54:06.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:06.757+0000 7f339b7fe700 1 -- 172.21.15.107:0/3201027630 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f338c00ca10 con 0x7f33a40fb360 2024-05-17T21:54:06.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:06.757+0000 7f33a98f4700 1 -- 172.21.15.107:0/3201027630 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33a4111d40 con 0x7f33a40fb360 2024-05-17T21:54:06.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:06.759+0000 7f33a98f4700 1 -- 172.21.15.107:0/3201027630 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33a404c710 con 0x7f33a40fb360 2024-05-17T21:54:06.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:06.760+0000 7f339b7fe700 1 -- 172.21.15.107:0/3201027630 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 31) v1 ==== 99906+0+0 (secure 0 0 0) 0x7f338c00cc80 con 0x7f33a40fb360 2024-05-17T21:54:06.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:06.760+0000 7f339b7fe700 1 --2- 172.21.15.107:0/3201027630 >> [v2:172.21.15.107:6800/416442229,v1:172.21.15.107:6801/416442229] conn(0x7f33900783a0 0x7f339007a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:54:06.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:06.760+0000 7f339b7fe700 1 -- 172.21.15.107:0/3201027630 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f338c099be0 con 0x7f33a40fb360 2024-05-17T21:54:06.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:06.760+0000 7f33a2ffd700 1 --2- 172.21.15.107:0/3201027630 >> [v2:172.21.15.107:6800/416442229,v1:172.21.15.107:6801/416442229] conn(0x7f33900783a0 0x7f339007a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:54:06.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:06.761+0000 7f33a2ffd700 1 --2- 172.21.15.107:0/3201027630 >> [v2:172.21.15.107:6800/416442229,v1:172.21.15.107:6801/416442229] conn(0x7f33900783a0 0x7f339007a850 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f3394004890 tx=0x7f3394009e30 comp rx=0 tx=0).ready entity=mgr.14750 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:54:06.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:06.767+0000 7f339b7fe700 1 -- 172.21.15.107:0/3201027630 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+190121 (secure 0 0 0) 0x7f338c061f30 con 0x7f33a40fb360 2024-05-17T21:54:06.911 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:06 smithi107 ceph-mon[28434]: pgmap v24: 65 pgs: 65 active+clean; 303 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 134 op/s 2024-05-17T21:54:06.911 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:06 smithi107 ceph-mon[28434]: from='client.24543 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:54:06.911 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:06 smithi107 ceph-mon[28434]: from='client.14818 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:54:07.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:07.112+0000 7f33a98f4700 1 -- 172.21.15.107:0/3201027630 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f33a40fca10 con 0x7f33a40fb360 2024-05-17T21:54:07.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:07.112+0000 7f339b7fe700 1 -- 172.21.15.107:0/3201027630 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+739 (secure 0 0 0) 0x7f338c0618d0 con 0x7f33a40fb360 2024-05-17T21:54:07.104 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-05-17T21:54:07.105 INFO:teuthology.orchestra.run.smithi107.stdout: "mon": { 2024-05-17T21:54:07.105 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-952-g9903f776 (9903f7767386a9d9efadb7c1effdcd3064d71804) quincy (stable)": 2 2024-05-17T21:54:07.105 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-05-17T21:54:07.105 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr": { 2024-05-17T21:54:07.105 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev)": 2 2024-05-17T21:54:07.105 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-05-17T21:54:07.105 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": { 2024-05-17T21:54:07.105 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-952-g9903f776 (9903f7767386a9d9efadb7c1effdcd3064d71804) quincy (stable)": 6 2024-05-17T21:54:07.105 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-05-17T21:54:07.105 INFO:teuthology.orchestra.run.smithi107.stdout: "mds": { 2024-05-17T21:54:07.105 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-952-g9903f776 (9903f7767386a9d9efadb7c1effdcd3064d71804) quincy (stable)": 4 2024-05-17T21:54:07.105 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-05-17T21:54:07.105 INFO:teuthology.orchestra.run.smithi107.stdout: "overall": { 2024-05-17T21:54:07.106 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-952-g9903f776 (9903f7767386a9d9efadb7c1effdcd3064d71804) quincy (stable)": 12, 2024-05-17T21:54:07.106 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev)": 2 2024-05-17T21:54:07.106 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-05-17T21:54:07.106 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-05-17T21:54:07.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:07.114+0000 7f33a98f4700 1 -- 172.21.15.107:0/3201027630 >> [v2:172.21.15.107:6800/416442229,v1:172.21.15.107:6801/416442229] conn(0x7f33900783a0 msgr2=0x7f339007a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:54:07.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:07.114+0000 7f33a98f4700 1 --2- 172.21.15.107:0/3201027630 >> [v2:172.21.15.107:6800/416442229,v1:172.21.15.107:6801/416442229] conn(0x7f33900783a0 0x7f339007a850 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f3394004890 tx=0x7f3394009e30 comp rx=0 tx=0).stop 2024-05-17T21:54:07.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:07.114+0000 7f33a98f4700 1 -- 172.21.15.107:0/3201027630 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33a40fb360 msgr2=0x7f33a40fc4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:54:07.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:07.114+0000 7f33a98f4700 1 --2- 172.21.15.107:0/3201027630 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33a40fb360 0x7f33a40fc4d0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f338c00eb40 tx=0x7f338c00b040 comp rx=0 tx=0).stop 2024-05-17T21:54:07.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:07.114+0000 7f33a98f4700 1 -- 172.21.15.107:0/3201027630 shutdown_connections 2024-05-17T21:54:07.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:07.114+0000 7f33a98f4700 1 --2- 172.21.15.107:0/3201027630 >> [v2:172.21.15.107:6800/416442229,v1:172.21.15.107:6801/416442229] conn(0x7f33900783a0 0x7f339007a850 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:54:07.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:07.114+0000 7f33a98f4700 1 --2- 172.21.15.107:0/3201027630 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f33a40faa10 0x7f33a40fde20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:54:07.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:07.114+0000 7f33a98f4700 1 --2- 172.21.15.107:0/3201027630 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33a40fb360 0x7f33a40fc4d0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:54:07.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:07.114+0000 7f33a98f4700 1 -- 172.21.15.107:0/3201027630 >> 172.21.15.107:0/3201027630 conn(0x7f33a40f6740 msgr2=0x7f33a41006a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:54:07.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:07.115+0000 7f33a98f4700 1 -- 172.21.15.107:0/3201027630 shutdown_connections 2024-05-17T21:54:07.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:07.115+0000 7f33a98f4700 1 -- 172.21.15.107:0/3201027630 wait complete. 2024-05-17T21:54:07.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:07.261+0000 7fb11503b700 1 -- 172.21.15.107:0/412604161 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb1100fc2f0 msgr2=0x7fb1100fc700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:54:07.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:07.261+0000 7fb11503b700 1 --2- 172.21.15.107:0/412604161 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb1100fc2f0 0x7fb1100fc700 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fb100004940 tx=0x7fb100009e30 comp rx=0 tx=0).stop 2024-05-17T21:54:07.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:07.261+0000 7fb11503b700 1 -- 172.21.15.107:0/412604161 shutdown_connections 2024-05-17T21:54:07.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:07.261+0000 7fb11503b700 1 --2- 172.21.15.107:0/412604161 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1100fcc40 0x7fb1100fd0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:54:07.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:07.261+0000 7fb11503b700 1 --2- 172.21.15.107:0/412604161 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb1100fc2f0 0x7fb1100fc700 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:54:07.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:07.261+0000 7fb11503b700 1 -- 172.21.15.107:0/412604161 >> 172.21.15.107:0/412604161 conn(0x7fb1100f6b80 msgr2=0x7fb1100f8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:54:07.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:07.262+0000 7fb11503b700 1 -- 172.21.15.107:0/412604161 shutdown_connections 2024-05-17T21:54:07.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:07.262+0000 7fb11503b700 1 -- 172.21.15.107:0/412604161 wait complete. 2024-05-17T21:54:07.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:07.262+0000 7fb11503b700 1 Processor -- start 2024-05-17T21:54:07.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:07.262+0000 7fb11503b700 1 -- start start 2024-05-17T21:54:07.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:07.263+0000 7fb11503b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1100fc2f0 0x7fb110110690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:54:07.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:07.263+0000 7fb11503b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb1100fcc40 0x7fb110112be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:54:07.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:07.263+0000 7fb11503b700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1100fbce0 con 0x7fb1100fcc40 2024-05-17T21:54:07.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:07.263+0000 7fb11503b700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1100fbe20 con 0x7fb1100fc2f0 2024-05-17T21:54:07.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:07.263+0000 7fb10e59c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb1100fcc40 0x7fb110112be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:54:07.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:07.263+0000 7fb10e59c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb1100fcc40 0x7fb110112be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51950/0 (socket says 172.21.15.107:51950) 2024-05-17T21:54:07.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:07.263+0000 7fb10e59c700 1 -- 172.21.15.107:0/3125368557 learned_addr learned my addr 172.21.15.107:0/3125368557 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:54:07.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:07.263+0000 7fb10ed9d700 1 --2- 172.21.15.107:0/3125368557 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1100fc2f0 0x7fb110110690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:54:07.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:07.263+0000 7fb10e59c700 1 -- 172.21.15.107:0/3125368557 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1100fc2f0 msgr2=0x7fb110110690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:54:07.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:07.263+0000 7fb10e59c700 1 --2- 172.21.15.107:0/3125368557 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1100fc2f0 0x7fb110110690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:54:07.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:07.263+0000 7fb10e59c700 1 -- 172.21.15.107:0/3125368557 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb100005040 con 0x7fb1100fcc40 2024-05-17T21:54:07.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:07.263+0000 7fb10e59c700 1 --2- 172.21.15.107:0/3125368557 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb1100fcc40 0x7fb110112be0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fb10400eb40 tx=0x7fb10400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:54:07.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:07.264+0000 7fb0f77fe700 1 -- 172.21.15.107:0/3125368557 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb104007cc0 con 0x7fb1100fcc40 2024-05-17T21:54:07.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:07.264+0000 7fb0f77fe700 1 -- 172.21.15.107:0/3125368557 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb10400f040 con 0x7fb1100fcc40 2024-05-17T21:54:07.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:07.264+0000 7fb11503b700 1 -- 172.21.15.107:0/3125368557 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb110113120 con 0x7fb1100fcc40 2024-05-17T21:54:07.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:07.264+0000 7fb0f77fe700 1 -- 172.21.15.107:0/3125368557 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb10400ca10 con 0x7fb1100fcc40 2024-05-17T21:54:07.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:07.264+0000 7fb11503b700 1 -- 172.21.15.107:0/3125368557 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb110113660 con 0x7fb1100fcc40 2024-05-17T21:54:07.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:07.265+0000 7fb11503b700 1 -- 172.21.15.107:0/3125368557 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0fc0052e0 con 0x7fb1100fcc40 2024-05-17T21:54:07.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:07.266+0000 7fb0f77fe700 1 -- 172.21.15.107:0/3125368557 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 31) v1 ==== 99906+0+0 (secure 0 0 0) 0x7fb10400cb70 con 0x7fb1100fcc40 2024-05-17T21:54:07.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:07.267+0000 7fb0f77fe700 1 --2- 172.21.15.107:0/3125368557 >> [v2:172.21.15.107:6800/416442229,v1:172.21.15.107:6801/416442229] conn(0x7fb0f80783a0 0x7fb0f807a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:54:07.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:07.267+0000 7fb0f77fe700 1 -- 172.21.15.107:0/3125368557 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fb104099c60 con 0x7fb1100fcc40 2024-05-17T21:54:07.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:07.267+0000 7fb10ed9d700 1 --2- 172.21.15.107:0/3125368557 >> [v2:172.21.15.107:6800/416442229,v1:172.21.15.107:6801/416442229] conn(0x7fb0f80783a0 0x7fb0f807a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:54:07.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:07.267+0000 7fb10ed9d700 1 --2- 172.21.15.107:0/3125368557 >> [v2:172.21.15.107:6800/416442229,v1:172.21.15.107:6801/416442229] conn(0x7fb0f80783a0 0x7fb0f807a850 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fb100004890 tx=0x7fb100009e30 comp rx=0 tx=0).ready entity=mgr.14750 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:54:07.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:07.271+0000 7fb0f77fe700 1 -- 172.21.15.107:0/3125368557 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+190121 (secure 0 0 0) 0x7fb1040620d0 con 0x7fb1100fcc40 2024-05-17T21:54:07.294 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:06 smithi138 ceph-mon[35948]: pgmap v24: 65 pgs: 65 active+clean; 303 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 134 op/s 2024-05-17T21:54:07.294 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:06 smithi138 ceph-mon[35948]: from='client.24543 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:54:07.294 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:06 smithi138 ceph-mon[35948]: from='client.14818 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:54:07.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:07.543+0000 7fb11503b700 1 -- 172.21.15.107:0/3125368557 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fb0fc005860 con 0x7fb1100fcc40 2024-05-17T21:54:07.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:07.543+0000 7fb0f77fe700 1 -- 172.21.15.107:0/3125368557 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1840 (secure 0 0 0) 0x7fb104061a70 con 0x7fb1100fcc40 2024-05-17T21:54:07.534 INFO:teuthology.orchestra.run.smithi107.stdout:e15 2024-05-17T21:54:07.535 INFO:teuthology.orchestra.run.smithi107.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-17T21:54:07.535 INFO:teuthology.orchestra.run.smithi107.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-17T21:54:07.535 INFO:teuthology.orchestra.run.smithi107.stdout:legacy client fscid: 1 2024-05-17T21:54:07.535 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:54:07.535 INFO:teuthology.orchestra.run.smithi107.stdout:Filesystem 'cephfs' (1) 2024-05-17T21:54:07.535 INFO:teuthology.orchestra.run.smithi107.stdout:fs_name cephfs 2024-05-17T21:54:07.535 INFO:teuthology.orchestra.run.smithi107.stdout:epoch 15 2024-05-17T21:54:07.535 INFO:teuthology.orchestra.run.smithi107.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-17T21:54:07.535 INFO:teuthology.orchestra.run.smithi107.stdout:created 2024-05-17T21:49:16.836333+0000 2024-05-17T21:54:07.535 INFO:teuthology.orchestra.run.smithi107.stdout:modified 2024-05-17T21:49:34.873628+0000 2024-05-17T21:54:07.535 INFO:teuthology.orchestra.run.smithi107.stdout:tableserver 0 2024-05-17T21:54:07.535 INFO:teuthology.orchestra.run.smithi107.stdout:root 0 2024-05-17T21:54:07.535 INFO:teuthology.orchestra.run.smithi107.stdout:session_timeout 60 2024-05-17T21:54:07.536 INFO:teuthology.orchestra.run.smithi107.stdout:session_autoclose 300 2024-05-17T21:54:07.536 INFO:teuthology.orchestra.run.smithi107.stdout:max_file_size 1099511627776 2024-05-17T21:54:07.536 INFO:teuthology.orchestra.run.smithi107.stdout:required_client_features {} 2024-05-17T21:54:07.536 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure 0 2024-05-17T21:54:07.536 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure_osd_epoch 46 2024-05-17T21:54:07.536 INFO:teuthology.orchestra.run.smithi107.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-17T21:54:07.536 INFO:teuthology.orchestra.run.smithi107.stdout:max_mds 1 2024-05-17T21:54:07.536 INFO:teuthology.orchestra.run.smithi107.stdout:in 0 2024-05-17T21:54:07.536 INFO:teuthology.orchestra.run.smithi107.stdout:up {0=24259} 2024-05-17T21:54:07.536 INFO:teuthology.orchestra.run.smithi107.stdout:failed 2024-05-17T21:54:07.536 INFO:teuthology.orchestra.run.smithi107.stdout:damaged 2024-05-17T21:54:07.537 INFO:teuthology.orchestra.run.smithi107.stdout:stopped 2024-05-17T21:54:07.537 INFO:teuthology.orchestra.run.smithi107.stdout:data_pools [3] 2024-05-17T21:54:07.537 INFO:teuthology.orchestra.run.smithi107.stdout:metadata_pool 2 2024-05-17T21:54:07.537 INFO:teuthology.orchestra.run.smithi107.stdout:inline_data enabled 2024-05-17T21:54:07.537 INFO:teuthology.orchestra.run.smithi107.stdout:balancer 2024-05-17T21:54:07.537 INFO:teuthology.orchestra.run.smithi107.stdout:standby_count_wanted 1 2024-05-17T21:54:07.537 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi138.gwbtyd{0:24259} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.138:6824/2779634919,v1:172.21.15.138:6825/2779634919] compat {c=[1],r=[1],i=[7ff]}] 2024-05-17T21:54:07.537 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.drqody{0:14486} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.107:6828/2275807100,v1:172.21.15.107:6829/2275807100] compat {c=[1],r=[1],i=[7ff]}] 2024-05-17T21:54:07.537 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:54:07.537 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:54:07.537 INFO:teuthology.orchestra.run.smithi107.stdout:Standby daemons: 2024-05-17T21:54:07.537 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:54:07.537 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.idmbgj{-1:14498} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.107:6826/1460186079,v1:172.21.15.107:6827/1460186079] compat {c=[1],r=[1],i=[7ff]}] 2024-05-17T21:54:07.537 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi138.zckrvs{-1:24267} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.138:6826/170628661,v1:172.21.15.138:6827/170628661] compat {c=[1],r=[1],i=[7ff]}] 2024-05-17T21:54:07.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:07.545+0000 7fb11503b700 1 -- 172.21.15.107:0/3125368557 >> [v2:172.21.15.107:6800/416442229,v1:172.21.15.107:6801/416442229] conn(0x7fb0f80783a0 msgr2=0x7fb0f807a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:54:07.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:07.545+0000 7fb11503b700 1 --2- 172.21.15.107:0/3125368557 >> [v2:172.21.15.107:6800/416442229,v1:172.21.15.107:6801/416442229] conn(0x7fb0f80783a0 0x7fb0f807a850 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fb100004890 tx=0x7fb100009e30 comp rx=0 tx=0).stop 2024-05-17T21:54:07.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:07.545+0000 7fb11503b700 1 -- 172.21.15.107:0/3125368557 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb1100fcc40 msgr2=0x7fb110112be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:54:07.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:07.545+0000 7fb11503b700 1 --2- 172.21.15.107:0/3125368557 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb1100fcc40 0x7fb110112be0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fb10400eb40 tx=0x7fb10400b040 comp rx=0 tx=0).stop 2024-05-17T21:54:07.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:07.546+0000 7fb11503b700 1 -- 172.21.15.107:0/3125368557 shutdown_connections 2024-05-17T21:54:07.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:07.546+0000 7fb11503b700 1 --2- 172.21.15.107:0/3125368557 >> [v2:172.21.15.107:6800/416442229,v1:172.21.15.107:6801/416442229] conn(0x7fb0f80783a0 0x7fb0f807a850 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:54:07.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:07.546+0000 7fb11503b700 1 --2- 172.21.15.107:0/3125368557 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1100fc2f0 0x7fb110110690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:54:07.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:07.546+0000 7fb11503b700 1 --2- 172.21.15.107:0/3125368557 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb1100fcc40 0x7fb110112be0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:54:07.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:07.546+0000 7fb11503b700 1 -- 172.21.15.107:0/3125368557 >> 172.21.15.107:0/3125368557 conn(0x7fb1100f6b80 msgr2=0x7fb1100f8200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:54:07.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:07.546+0000 7fb11503b700 1 -- 172.21.15.107:0/3125368557 shutdown_connections 2024-05-17T21:54:07.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:07.546+0000 7fb11503b700 1 -- 172.21.15.107:0/3125368557 wait complete. 2024-05-17T21:54:07.539 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 15 2024-05-17T21:54:07.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:07.664+0000 7fdf15906700 1 -- 172.21.15.107:0/4258887166 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf100fe090 msgr2=0x7fdf100fe500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:54:07.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:07.664+0000 7fdf15906700 1 --2- 172.21.15.107:0/4258887166 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf100fe090 0x7fdf100fe500 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fdf04004940 tx=0x7fdf04009e30 comp rx=0 tx=0).stop 2024-05-17T21:54:07.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:07.664+0000 7fdf15906700 1 -- 172.21.15.107:0/4258887166 shutdown_connections 2024-05-17T21:54:07.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:07.664+0000 7fdf15906700 1 --2- 172.21.15.107:0/4258887166 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf100fe090 0x7fdf100fe500 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:54:07.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:07.664+0000 7fdf15906700 1 --2- 172.21.15.107:0/4258887166 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf100fce90 0x7fdf100fd2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:54:07.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:07.664+0000 7fdf15906700 1 -- 172.21.15.107:0/4258887166 >> 172.21.15.107:0/4258887166 conn(0x7fdf100f85c0 msgr2=0x7fdf100fa9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:54:07.657 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:07.665+0000 7fdf15906700 1 -- 172.21.15.107:0/4258887166 shutdown_connections 2024-05-17T21:54:07.657 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:07.665+0000 7fdf15906700 1 -- 172.21.15.107:0/4258887166 wait complete. 2024-05-17T21:54:07.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:07.665+0000 7fdf15906700 1 Processor -- start 2024-05-17T21:54:07.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:07.665+0000 7fdf15906700 1 -- start start 2024-05-17T21:54:07.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:07.665+0000 7fdf15906700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf100fce90 0x7fdf1010d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:54:07.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:07.665+0000 7fdf15906700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf100fe090 0x7fdf1010def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:54:07.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:07.665+0000 7fdf15906700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf1010e4f0 con 0x7fdf100fce90 2024-05-17T21:54:07.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:07.665+0000 7fdf15906700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf1010ed40 con 0x7fdf100fe090 2024-05-17T21:54:07.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:07.666+0000 7fdf0effd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf100fce90 0x7fdf1010d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:54:07.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:07.666+0000 7fdf0effd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf100fce90 0x7fdf1010d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51968/0 (socket says 172.21.15.107:51968) 2024-05-17T21:54:07.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:07.666+0000 7fdf0effd700 1 -- 172.21.15.107:0/2652787994 learned_addr learned my addr 172.21.15.107:0/2652787994 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:54:07.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:07.666+0000 7fdf0e7fc700 1 --2- 172.21.15.107:0/2652787994 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf100fe090 0x7fdf1010def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:54:07.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:07.666+0000 7fdf0effd700 1 -- 172.21.15.107:0/2652787994 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf100fe090 msgr2=0x7fdf1010def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:54:07.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:07.666+0000 7fdf0effd700 1 --2- 172.21.15.107:0/2652787994 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf100fe090 0x7fdf1010def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:54:07.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:07.666+0000 7fdf0effd700 1 -- 172.21.15.107:0/2652787994 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf04005040 con 0x7fdf100fce90 2024-05-17T21:54:07.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:07.666+0000 7fdf0effd700 1 --2- 172.21.15.107:0/2652787994 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf100fce90 0x7fdf1010d9b0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fdf0000eb70 tx=0x7fdf0000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:54:07.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:07.666+0000 7fdef77fe700 1 -- 172.21.15.107:0/2652787994 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf00007cc0 con 0x7fdf100fce90 2024-05-17T21:54:07.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:07.666+0000 7fdef77fe700 1 -- 172.21.15.107:0/2652787994 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdf0000f040 con 0x7fdf100fce90 2024-05-17T21:54:07.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:07.666+0000 7fdf15906700 1 -- 172.21.15.107:0/2652787994 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf1010efc0 con 0x7fdf100fce90 2024-05-17T21:54:07.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:07.666+0000 7fdef77fe700 1 -- 172.21.15.107:0/2652787994 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf0000cb20 con 0x7fdf100fce90 2024-05-17T21:54:07.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:07.666+0000 7fdf15906700 1 -- 172.21.15.107:0/2652787994 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf1010f500 con 0x7fdf100fce90 2024-05-17T21:54:07.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:07.667+0000 7fdf15906700 1 -- 172.21.15.107:0/2652787994 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf1005f5c0 con 0x7fdf100fce90 2024-05-17T21:54:07.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:07.669+0000 7fdef77fe700 1 -- 172.21.15.107:0/2652787994 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 31) v1 ==== 99906+0+0 (secure 0 0 0) 0x7fdf0000cc80 con 0x7fdf100fce90 2024-05-17T21:54:07.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:07.669+0000 7fdef77fe700 1 --2- 172.21.15.107:0/2652787994 >> [v2:172.21.15.107:6800/416442229,v1:172.21.15.107:6801/416442229] conn(0x7fdef80783a0 0x7fdef807a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:54:07.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:07.669+0000 7fdef77fe700 1 -- 172.21.15.107:0/2652787994 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fdf00099980 con 0x7fdf100fce90 2024-05-17T21:54:07.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:07.669+0000 7fdf0e7fc700 1 --2- 172.21.15.107:0/2652787994 >> [v2:172.21.15.107:6800/416442229,v1:172.21.15.107:6801/416442229] conn(0x7fdef80783a0 0x7fdef807a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:54:07.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:07.670+0000 7fdf0e7fc700 1 --2- 172.21.15.107:0/2652787994 >> [v2:172.21.15.107:6800/416442229,v1:172.21.15.107:6801/416442229] conn(0x7fdef80783a0 0x7fdef807a850 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fdf04009b30 tx=0x7fdf04009e30 comp rx=0 tx=0).ready entity=mgr.14750 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:54:07.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:07.675+0000 7fdef77fe700 1 -- 172.21.15.107:0/2652787994 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+190121 (secure 0 0 0) 0x7fdf00061d50 con 0x7fdf100fce90 2024-05-17T21:54:07.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:07.939+0000 7fdf15906700 1 -- 172.21.15.107:0/2652787994 --> [v2:172.21.15.107:6800/416442229,v1:172.21.15.107:6801/416442229] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdf1010f8f0 con 0x7fdef80783a0 2024-05-17T21:54:07.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:07.941+0000 7fdef77fe700 1 -- 172.21.15.107:0/2652787994 <== mgr.14750 v2:172.21.15.107:6800/416442229 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+328 (secure 0 0 0) 0x7fdf1010f8f0 con 0x7fdef80783a0 2024-05-17T21:54:07.932 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-05-17T21:54:07.932 INFO:teuthology.orchestra.run.smithi107.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b", 2024-05-17T21:54:07.932 INFO:teuthology.orchestra.run.smithi107.stdout: "in_progress": true, 2024-05-17T21:54:07.932 INFO:teuthology.orchestra.run.smithi107.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-17T21:54:07.932 INFO:teuthology.orchestra.run.smithi107.stdout: "services_complete": [], 2024-05-17T21:54:07.932 INFO:teuthology.orchestra.run.smithi107.stdout: "progress": "", 2024-05-17T21:54:07.932 INFO:teuthology.orchestra.run.smithi107.stdout: "message": "Currently upgrading mgr daemons", 2024-05-17T21:54:07.933 INFO:teuthology.orchestra.run.smithi107.stdout: "is_paused": false 2024-05-17T21:54:07.933 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-05-17T21:54:07.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:07.944+0000 7fdf15906700 1 -- 172.21.15.107:0/2652787994 >> [v2:172.21.15.107:6800/416442229,v1:172.21.15.107:6801/416442229] conn(0x7fdef80783a0 msgr2=0x7fdef807a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:54:07.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:07.944+0000 7fdf15906700 1 --2- 172.21.15.107:0/2652787994 >> [v2:172.21.15.107:6800/416442229,v1:172.21.15.107:6801/416442229] conn(0x7fdef80783a0 0x7fdef807a850 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fdf04009b30 tx=0x7fdf04009e30 comp rx=0 tx=0).stop 2024-05-17T21:54:07.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:07.944+0000 7fdf15906700 1 -- 172.21.15.107:0/2652787994 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf100fce90 msgr2=0x7fdf1010d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:54:07.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:07.944+0000 7fdf15906700 1 --2- 172.21.15.107:0/2652787994 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf100fce90 0x7fdf1010d9b0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fdf0000eb70 tx=0x7fdf0000b040 comp rx=0 tx=0).stop 2024-05-17T21:54:07.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:07.944+0000 7fdf15906700 1 -- 172.21.15.107:0/2652787994 shutdown_connections 2024-05-17T21:54:07.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:07.944+0000 7fdf15906700 1 --2- 172.21.15.107:0/2652787994 >> [v2:172.21.15.107:6800/416442229,v1:172.21.15.107:6801/416442229] conn(0x7fdef80783a0 0x7fdef807a850 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:54:07.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:07.944+0000 7fdf15906700 1 --2- 172.21.15.107:0/2652787994 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf100fce90 0x7fdf1010d9b0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:54:07.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:07.944+0000 7fdf15906700 1 --2- 172.21.15.107:0/2652787994 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf100fe090 0x7fdf1010def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:54:07.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:07.944+0000 7fdf15906700 1 -- 172.21.15.107:0/2652787994 >> 172.21.15.107:0/2652787994 conn(0x7fdf100f85c0 msgr2=0x7fdf100f9c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:54:07.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:07.944+0000 7fdf15906700 1 -- 172.21.15.107:0/2652787994 shutdown_connections 2024-05-17T21:54:07.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:07.944+0000 7fdf15906700 1 -- 172.21.15.107:0/2652787994 wait complete. 2024-05-17T21:54:08.048 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:08.055+0000 7f8279936700 1 -- 172.21.15.107:0/1217491174 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82740fcf60 msgr2=0x7f82740fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:54:08.048 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:08.055+0000 7f8279936700 1 --2- 172.21.15.107:0/1217491174 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82740fcf60 0x7f82740fd3d0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f8268004940 tx=0x7f8268009e30 comp rx=0 tx=0).stop 2024-05-17T21:54:08.048 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:08.055+0000 7f8279936700 1 -- 172.21.15.107:0/1217491174 shutdown_connections 2024-05-17T21:54:08.048 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:08.055+0000 7f8279936700 1 --2- 172.21.15.107:0/1217491174 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82740fcf60 0x7f82740fd3d0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:54:08.048 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:08.055+0000 7f8279936700 1 --2- 172.21.15.107:0/1217491174 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82740fbcb0 0x7f82740fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:54:08.048 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:08.055+0000 7f8279936700 1 -- 172.21.15.107:0/1217491174 >> 172.21.15.107:0/1217491174 conn(0x7f82740f7430 msgr2=0x7f82740f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:54:08.048 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:08.055+0000 7f8279936700 1 -- 172.21.15.107:0/1217491174 shutdown_connections 2024-05-17T21:54:08.048 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:08.055+0000 7f8279936700 1 -- 172.21.15.107:0/1217491174 wait complete. 2024-05-17T21:54:08.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:08.056+0000 7f8279936700 1 Processor -- start 2024-05-17T21:54:08.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:08.056+0000 7f8279936700 1 -- start start 2024-05-17T21:54:08.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:08.056+0000 7f8279936700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82740fbcb0 0x7f82741106a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:54:08.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:08.056+0000 7f8279936700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82740fcf60 0x7f8274112bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:54:08.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:08.056+0000 7f8279936700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8274066330 con 0x7f82740fbcb0 2024-05-17T21:54:08.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:08.056+0000 7f8279936700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8274066470 con 0x7f82740fcf60 2024-05-17T21:54:08.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:08.056+0000 7f8272ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82740fbcb0 0x7f82741106a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:54:08.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:08.056+0000 7f8272ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82740fbcb0 0x7f82741106a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51986/0 (socket says 172.21.15.107:51986) 2024-05-17T21:54:08.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:08.056+0000 7f8272ffd700 1 -- 172.21.15.107:0/2767394782 learned_addr learned my addr 172.21.15.107:0/2767394782 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:54:08.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:08.056+0000 7f82727fc700 1 --2- 172.21.15.107:0/2767394782 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82740fcf60 0x7f8274112bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:54:08.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:08.056+0000 7f8272ffd700 1 -- 172.21.15.107:0/2767394782 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82740fcf60 msgr2=0x7f8274112bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:54:08.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:08.056+0000 7f8272ffd700 1 --2- 172.21.15.107:0/2767394782 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82740fcf60 0x7f8274112bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:54:08.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:08.056+0000 7f8272ffd700 1 -- 172.21.15.107:0/2767394782 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8268005040 con 0x7f82740fbcb0 2024-05-17T21:54:08.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:08.056+0000 7f8272ffd700 1 --2- 172.21.15.107:0/2767394782 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82740fbcb0 0x7f82741106a0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f826400eaf0 tx=0x7f826400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:54:08.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:08.056+0000 7f825b7fe700 1 -- 172.21.15.107:0/2767394782 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8264007cc0 con 0x7f82740fbcb0 2024-05-17T21:54:08.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:08.057+0000 7f825b7fe700 1 -- 172.21.15.107:0/2767394782 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f826400f040 con 0x7f82740fbcb0 2024-05-17T21:54:08.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:08.057+0000 7f8279936700 1 -- 172.21.15.107:0/2767394782 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8274113130 con 0x7f82740fbcb0 2024-05-17T21:54:08.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:08.057+0000 7f825b7fe700 1 -- 172.21.15.107:0/2767394782 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f826400cb90 con 0x7f82740fbcb0 2024-05-17T21:54:08.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:08.057+0000 7f8279936700 1 -- 172.21.15.107:0/2767394782 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8274113670 con 0x7f82740fbcb0 2024-05-17T21:54:08.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:08.058+0000 7f8279936700 1 -- 172.21.15.107:0/2767394782 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f827405f5c0 con 0x7f82740fbcb0 2024-05-17T21:54:08.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:08.058+0000 7f825b7fe700 1 -- 172.21.15.107:0/2767394782 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 31) v1 ==== 99906+0+0 (secure 0 0 0) 0x7f826400ccf0 con 0x7f82740fbcb0 2024-05-17T21:54:08.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:08.059+0000 7f825b7fe700 1 --2- 172.21.15.107:0/2767394782 >> [v2:172.21.15.107:6800/416442229,v1:172.21.15.107:6801/416442229] conn(0x7f825c0783a0 0x7f825c07a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:54:08.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:08.059+0000 7f825b7fe700 1 -- 172.21.15.107:0/2767394782 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f8264099c00 con 0x7f82740fbcb0 2024-05-17T21:54:08.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:08.059+0000 7f82727fc700 1 --2- 172.21.15.107:0/2767394782 >> [v2:172.21.15.107:6800/416442229,v1:172.21.15.107:6801/416442229] conn(0x7f825c0783a0 0x7f825c07a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:54:08.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:08.059+0000 7f82727fc700 1 --2- 172.21.15.107:0/2767394782 >> [v2:172.21.15.107:6800/416442229,v1:172.21.15.107:6801/416442229] conn(0x7f825c0783a0 0x7f825c07a850 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f8268009b30 tx=0x7f8268009e30 comp rx=0 tx=0).ready entity=mgr.14750 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:54:08.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:08.063+0000 7f825b7fe700 1 -- 172.21.15.107:0/2767394782 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+190121 (secure 0 0 0) 0x7f826409f020 con 0x7f82740fbcb0 2024-05-17T21:54:08.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:07 smithi107 ceph-mon[28434]: from='client.14822 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:54:08.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:07 smithi107 ceph-mon[28434]: from='client.? 172.21.15.107:0/3201027630' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:54:08.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:07 smithi107 ceph-mon[28434]: from='client.? 172.21.15.107:0/3125368557' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-17T21:54:08.254 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:07 smithi138 ceph-mon[35948]: from='client.14822 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:54:08.254 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:07 smithi138 ceph-mon[35948]: from='client.? 172.21.15.107:0/3201027630' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:54:08.254 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:07 smithi138 ceph-mon[35948]: from='client.? 172.21.15.107:0/3125368557' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-17T21:54:08.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:08.421+0000 7f8279936700 1 -- 172.21.15.107:0/2767394782 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f827404bed0 con 0x7f82740fbcb0 2024-05-17T21:54:08.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:08.421+0000 7f825b7fe700 1 -- 172.21.15.107:0/2767394782 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f8264061f20 con 0x7f82740fbcb0 2024-05-17T21:54:08.412 INFO:teuthology.orchestra.run.smithi107.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-17T21:54:08.413 INFO:teuthology.orchestra.run.smithi107.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-17T21:54:08.413 INFO:teuthology.orchestra.run.smithi107.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-17T21:54:08.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:08.423+0000 7f8279936700 1 -- 172.21.15.107:0/2767394782 >> [v2:172.21.15.107:6800/416442229,v1:172.21.15.107:6801/416442229] conn(0x7f825c0783a0 msgr2=0x7f825c07a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:54:08.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:08.423+0000 7f8279936700 1 --2- 172.21.15.107:0/2767394782 >> [v2:172.21.15.107:6800/416442229,v1:172.21.15.107:6801/416442229] conn(0x7f825c0783a0 0x7f825c07a850 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f8268009b30 tx=0x7f8268009e30 comp rx=0 tx=0).stop 2024-05-17T21:54:08.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:08.423+0000 7f8279936700 1 -- 172.21.15.107:0/2767394782 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82740fbcb0 msgr2=0x7f82741106a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:54:08.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:08.423+0000 7f8279936700 1 --2- 172.21.15.107:0/2767394782 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82740fbcb0 0x7f82741106a0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f826400eaf0 tx=0x7f826400b040 comp rx=0 tx=0).stop 2024-05-17T21:54:08.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:08.423+0000 7f8279936700 1 -- 172.21.15.107:0/2767394782 shutdown_connections 2024-05-17T21:54:08.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:08.423+0000 7f8279936700 1 --2- 172.21.15.107:0/2767394782 >> [v2:172.21.15.107:6800/416442229,v1:172.21.15.107:6801/416442229] conn(0x7f825c0783a0 0x7f825c07a850 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:54:08.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:08.423+0000 7f8279936700 1 --2- 172.21.15.107:0/2767394782 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82740fbcb0 0x7f82741106a0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:54:08.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:08.423+0000 7f8279936700 1 --2- 172.21.15.107:0/2767394782 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82740fcf60 0x7f8274112bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:54:08.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:08.423+0000 7f8279936700 1 -- 172.21.15.107:0/2767394782 >> 172.21.15.107:0/2767394782 conn(0x7f82740f7430 msgr2=0x7f82740f8a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:54:08.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:08.423+0000 7f8279936700 1 -- 172.21.15.107:0/2767394782 shutdown_connections 2024-05-17T21:54:08.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:08.423+0000 7f8279936700 1 -- 172.21.15.107:0/2767394782 wait complete. 2024-05-17T21:54:09.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:08 smithi107 ceph-mon[28434]: pgmap v25: 65 pgs: 65 active+clean; 303 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 95 op/s 2024-05-17T21:54:09.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:08 smithi107 ceph-mon[28434]: from='client.14834 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:54:09.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:08 smithi107 ceph-mon[28434]: from='client.? 172.21.15.107:0/2767394782' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-17T21:54:09.167 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:08 smithi138 ceph-mon[35948]: pgmap v25: 65 pgs: 65 active+clean; 303 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 95 op/s 2024-05-17T21:54:09.167 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:08 smithi138 ceph-mon[35948]: from='client.14834 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:54:09.167 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:08 smithi138 ceph-mon[35948]: from='client.? 172.21.15.107:0/2767394782' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-17T21:54:10.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:09 smithi138 ceph-mon[35948]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:54:10.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:09 smithi107 ceph-mon[28434]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:54:11.375 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:11 smithi138 ceph-mon[35948]: pgmap v26: 65 pgs: 65 active+clean; 303 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 95 op/s 2024-05-17T21:54:11.375 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:11 smithi138 ceph-mon[35948]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:54:11.375 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:11 smithi138 ceph-mon[35948]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:54:11.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:11 smithi107 ceph-mon[28434]: pgmap v26: 65 pgs: 65 active+clean; 303 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 95 op/s 2024-05-17T21:54:11.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:11 smithi107 ceph-mon[28434]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:54:11.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:11 smithi107 ceph-mon[28434]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:54:13.496 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:13 smithi107 ceph-mon[28434]: pgmap v27: 65 pgs: 65 active+clean; 299 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 143 op/s 2024-05-17T21:54:13.496 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:13 smithi107 ceph-mon[28434]: Standby manager daemon smithi138.napnll restarted 2024-05-17T21:54:13.496 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:13 smithi107 ceph-mon[28434]: Standby manager daemon smithi138.napnll started 2024-05-17T21:54:13.496 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:13 smithi107 ceph-mon[28434]: from='mgr.? 172.21.15.138:0/2667864806' entity='mgr.smithi138.napnll' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi138.napnll/crt"}]: dispatch 2024-05-17T21:54:13.497 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:13 smithi107 ceph-mon[28434]: from='mgr.? 172.21.15.138:0/2667864806' entity='mgr.smithi138.napnll' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-17T21:54:13.497 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:13 smithi107 ceph-mon[28434]: from='mgr.? 172.21.15.138:0/2667864806' entity='mgr.smithi138.napnll' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi138.napnll/key"}]: dispatch 2024-05-17T21:54:13.497 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:13 smithi107 ceph-mon[28434]: from='mgr.? 172.21.15.138:0/2667864806' entity='mgr.smithi138.napnll' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-17T21:54:13.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:13 smithi138 ceph-mon[35948]: pgmap v27: 65 pgs: 65 active+clean; 299 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 143 op/s 2024-05-17T21:54:13.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:13 smithi138 ceph-mon[35948]: Standby manager daemon smithi138.napnll restarted 2024-05-17T21:54:13.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:13 smithi138 ceph-mon[35948]: Standby manager daemon smithi138.napnll started 2024-05-17T21:54:13.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:13 smithi138 ceph-mon[35948]: from='mgr.? 172.21.15.138:0/2667864806' entity='mgr.smithi138.napnll' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi138.napnll/crt"}]: dispatch 2024-05-17T21:54:13.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:13 smithi138 ceph-mon[35948]: from='mgr.? 172.21.15.138:0/2667864806' entity='mgr.smithi138.napnll' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-17T21:54:13.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:13 smithi138 ceph-mon[35948]: from='mgr.? 172.21.15.138:0/2667864806' entity='mgr.smithi138.napnll' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi138.napnll/key"}]: dispatch 2024-05-17T21:54:13.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:13 smithi138 ceph-mon[35948]: from='mgr.? 172.21.15.138:0/2667864806' entity='mgr.smithi138.napnll' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-17T21:54:14.445 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:14 smithi107 ceph-mon[28434]: mgrmap e32: smithi107.xhhyiw(active, since 49s), standbys: smithi138.napnll 2024-05-17T21:54:14.445 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:14 smithi107 ceph-mon[28434]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:54:14.445 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:14 smithi107 ceph-mon[28434]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:54:14.578 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:14 smithi138 ceph-mon[35948]: mgrmap e32: smithi107.xhhyiw(active, since 49s), standbys: smithi138.napnll 2024-05-17T21:54:14.578 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:14 smithi138 ceph-mon[35948]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:54:14.578 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:14 smithi138 ceph-mon[35948]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:54:15.486 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:15 smithi138 ceph-mon[35948]: pgmap v28: 65 pgs: 65 active+clean; 299 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 99 op/s 2024-05-17T21:54:15.487 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:15 smithi138 ceph-mon[35948]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:54:15.487 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:15 smithi138 ceph-mon[35948]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:54:15.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:15 smithi107 ceph-mon[28434]: pgmap v28: 65 pgs: 65 active+clean; 299 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 99 op/s 2024-05-17T21:54:15.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:15 smithi107 ceph-mon[28434]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:54:15.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:15 smithi107 ceph-mon[28434]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:54:17.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:17 smithi138 ceph-mon[35948]: pgmap v29: 65 pgs: 65 active+clean; 302 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 236 op/s 2024-05-17T21:54:17.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:17 smithi107 ceph-mon[28434]: pgmap v29: 65 pgs: 65 active+clean; 302 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 236 op/s 2024-05-17T21:54:19.469 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:19 smithi107 ceph-mon[28434]: pgmap v30: 65 pgs: 65 active+clean; 302 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 185 op/s 2024-05-17T21:54:19.469 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:19 smithi107 ceph-mon[28434]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:54:19.469 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:19 smithi107 ceph-mon[28434]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:54:19.469 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:19 smithi107 ceph-mon[28434]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:54:19.469 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:19 smithi107 ceph-mon[28434]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T21:54:19.469 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:19 smithi107 ceph-mon[28434]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:54:19.469 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:19 smithi107 ceph-mon[28434]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:54:19.470 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:19 smithi107 ceph-mon[28434]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:54:19.470 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:19 smithi107 ceph-mon[28434]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:54:19.470 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:19 smithi107 ceph-mon[28434]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi107.xhhyiw"}]: dispatch 2024-05-17T21:54:19.470 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:19 smithi107 ceph-mon[28434]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi107.xhhyiw"}]': finished 2024-05-17T21:54:19.470 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:19 smithi107 ceph-mon[28434]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi138.napnll"}]: dispatch 2024-05-17T21:54:19.470 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:19 smithi107 ceph-mon[28434]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi138.napnll"}]': finished 2024-05-17T21:54:19.470 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:19 smithi107 ceph-mon[28434]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-05-17T21:54:19.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:19 smithi138 ceph-mon[35948]: pgmap v30: 65 pgs: 65 active+clean; 302 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 185 op/s 2024-05-17T21:54:19.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:19 smithi138 ceph-mon[35948]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:54:19.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:19 smithi138 ceph-mon[35948]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:54:19.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:19 smithi138 ceph-mon[35948]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:54:19.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:19 smithi138 ceph-mon[35948]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T21:54:19.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:19 smithi138 ceph-mon[35948]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:54:19.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:19 smithi138 ceph-mon[35948]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:54:19.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:19 smithi138 ceph-mon[35948]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:54:19.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:19 smithi138 ceph-mon[35948]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:54:19.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:19 smithi138 ceph-mon[35948]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi107.xhhyiw"}]: dispatch 2024-05-17T21:54:19.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:19 smithi138 ceph-mon[35948]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi107.xhhyiw"}]': finished 2024-05-17T21:54:19.577 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:19 smithi138 ceph-mon[35948]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi138.napnll"}]: dispatch 2024-05-17T21:54:19.577 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:19 smithi138 ceph-mon[35948]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi138.napnll"}]': finished 2024-05-17T21:54:19.577 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:19 smithi138 ceph-mon[35948]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-05-17T21:54:20.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:20 smithi138 ceph-mon[35948]: Upgrade: Setting container_image for all mgr 2024-05-17T21:54:20.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:20 smithi107 ceph-mon[28434]: Upgrade: Setting container_image for all mgr 2024-05-17T21:54:21.436 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:21 smithi107 ceph-mon[28434]: pgmap v31: 65 pgs: 65 active+clean; 302 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 185 op/s 2024-05-17T21:54:21.436 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:21 smithi107 ceph-mon[28434]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:54:21.436 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:21 smithi107 ceph-mon[28434]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-17T21:54:21.436 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:21 smithi107 ceph-mon[28434]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-17T21:54:21.437 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:21 smithi107 ceph-mon[28434]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:54:21.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:21 smithi138 ceph-mon[35948]: pgmap v31: 65 pgs: 65 active+clean; 302 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 185 op/s 2024-05-17T21:54:21.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:21 smithi138 ceph-mon[35948]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:54:21.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:21 smithi138 ceph-mon[35948]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-17T21:54:21.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:21 smithi138 ceph-mon[35948]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-17T21:54:21.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:21 smithi138 ceph-mon[35948]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:54:22.431 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:22 smithi107 ceph-mon[28434]: Upgrade: Updating mon.smithi107 2024-05-17T21:54:22.432 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:22 smithi107 ceph-mon[28434]: Deploying daemon mon.smithi107 on smithi107 2024-05-17T21:54:22.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:22 smithi138 ceph-mon[35948]: Upgrade: Updating mon.smithi107 2024-05-17T21:54:22.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:22 smithi138 ceph-mon[35948]: Deploying daemon mon.smithi107 on smithi107 2024-05-17T21:54:22.995 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:22 smithi107 systemd[1]: Stopping Ceph mon.smithi107 for 3722731a-1496-11ef-bc9a-c7b262605968... 2024-05-17T21:54:23.320 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:23 smithi107 ceph-3722731a-1496-11ef-bc9a-c7b262605968-mon-smithi107[28430]: 2024-05-17T21:54:23.003+0000 7f8990ffc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi107 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-17T21:54:23.321 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:23 smithi107 ceph-3722731a-1496-11ef-bc9a-c7b262605968-mon-smithi107[28430]: 2024-05-17T21:54:23.003+0000 7f8990ffc700 -1 mon.smithi107@0(leader) e2 *** Got Signal Terminated *** 2024-05-17T21:54:23.321 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:23 smithi107 podman[93055]: 2024-05-17 21:54:23.033616431 +0000 UTC m=+0.153714551 container died 7de9c23eee97e9e9fa855317247a2c2bcabb0b393bf44e87c3ae8c2e6ef63941 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-mon-smithi107, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=quincy-9903f77, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240514) 2024-05-17T21:54:23.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:23 smithi107 podman[93055]: 2024-05-17 21:54:23.42791988 +0000 UTC m=+0.548017999 container cleanup 7de9c23eee97e9e9fa855317247a2c2bcabb0b393bf44e87c3ae8c2e6ef63941 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-mon-smithi107, ceph=True, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, RELEASE=quincy-9903f77) 2024-05-17T21:54:23.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:23 smithi107 bash[93055]: ceph-3722731a-1496-11ef-bc9a-c7b262605968-mon-smithi107 2024-05-17T21:54:24.023 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:23 smithi107 podman[93069]: 2024-05-17 21:54:23.671253731 +0000 UTC m=+0.641033264 container remove 7de9c23eee97e9e9fa855317247a2c2bcabb0b393bf44e87c3ae8c2e6ef63941 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-mon-smithi107, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=quincy-9903f77, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-05-17T21:54:24.023 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:23 smithi107 systemd[1]: ceph-3722731a-1496-11ef-bc9a-c7b262605968@mon.smithi107.service: Deactivated successfully. 2024-05-17T21:54:24.024 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:23 smithi107 systemd[1]: Stopped Ceph mon.smithi107 for 3722731a-1496-11ef-bc9a-c7b262605968. 2024-05-17T21:54:24.024 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:23 smithi107 systemd[1]: ceph-3722731a-1496-11ef-bc9a-c7b262605968@mon.smithi107.service: Consumed 11.049s CPU time. 2024-05-17T21:54:24.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 systemd[1]: Starting Ceph mon.smithi107 for 3722731a-1496-11ef-bc9a-c7b262605968... 2024-05-17T21:54:24.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 podman[93206]: 2024-05-17 21:54:24.153665216 +0000 UTC m=+0.018369776 image pull 8880f7613d0283efa7002505fdccdbc3e37d3b5da31c57fa22d62262c5dab66c quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b 2024-05-17T21:54:24.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 podman[93206]: 2024-05-17 21:54:24.281034056 +0000 UTC m=+0.145738603 container create 9717f3169f2f33e9afca7a12e2206b211d65c2fdda90950f2c75eb685a3f1bd8 (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-mon-smithi107, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-05-17T21:54:24.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 podman[93206]: 2024-05-17 21:54:24.550419998 +0000 UTC m=+0.415124546 container init 9717f3169f2f33e9afca7a12e2206b211d65c2fdda90950f2c75eb685a3f1bd8 (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-mon-smithi107, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-05-17T21:54:24.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 podman[93206]: 2024-05-17 21:54:24.554042577 +0000 UTC m=+0.418747124 container start 9717f3169f2f33e9afca7a12e2206b211d65c2fdda90950f2c75eb685a3f1bd8 (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-mon-smithi107, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-17T21:54:24.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: set uid:gid to 167:167 (ceph:ceph) 2024-05-17T21:54:24.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev), process ceph-mon, pid 2 2024-05-17T21:54:24.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: pidfile_write: ignore empty --pid-file 2024-05-17T21:54:24.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: load: jerasure load: lrc 2024-05-17T21:54:24.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: RocksDB version: 7.9.2 2024-05-17T21:54:24.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Git sha 0 2024-05-17T21:54:24.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Compile date 2024-05-17 01:53:57 2024-05-17T21:54:24.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: DB SUMMARY 2024-05-17T21:54:24.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: DB Session ID: GR3AZQO5S9NMPACFNA76 2024-05-17T21:54:24.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: CURRENT file: CURRENT 2024-05-17T21:54:24.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: IDENTITY file: IDENTITY 2024-05-17T21:54:24.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: MANIFEST file: MANIFEST-000015 size: 972 Bytes 2024-05-17T21:54:24.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi107/store.db dir, Total Num: 1, files: 000030.sst 2024-05-17T21:54:24.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi107/store.db: 000028.log size: 2400050 ; 2024-05-17T21:54:24.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.error_if_exists: 0 2024-05-17T21:54:24.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.create_if_missing: 0 2024-05-17T21:54:24.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.paranoid_checks: 1 2024-05-17T21:54:24.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-17T21:54:24.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-17T21:54:24.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-17T21:54:24.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.env: 0x560253fe2500 2024-05-17T21:54:24.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.fs: PosixFileSystem 2024-05-17T21:54:24.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.info_log: 0x560255cc3f20 2024-05-17T21:54:24.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.max_file_opening_threads: 16 2024-05-17T21:54:24.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.statistics: (nil) 2024-05-17T21:54:24.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.use_fsync: 0 2024-05-17T21:54:24.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.max_log_file_size: 0 2024-05-17T21:54:24.855 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-17T21:54:24.855 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-17T21:54:24.855 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.keep_log_file_num: 1000 2024-05-17T21:54:24.855 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.recycle_log_file_num: 0 2024-05-17T21:54:24.855 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.allow_fallocate: 1 2024-05-17T21:54:24.855 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.allow_mmap_reads: 0 2024-05-17T21:54:24.855 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.allow_mmap_writes: 0 2024-05-17T21:54:24.855 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.use_direct_reads: 0 2024-05-17T21:54:24.855 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-17T21:54:24.855 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.create_missing_column_families: 0 2024-05-17T21:54:24.855 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.db_log_dir: 2024-05-17T21:54:24.855 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.wal_dir: 2024-05-17T21:54:24.855 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-17T21:54:24.855 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-17T21:54:24.856 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-17T21:54:24.856 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-17T21:54:24.856 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-17T21:54:24.856 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-17T21:54:24.856 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.advise_random_on_open: 1 2024-05-17T21:54:24.856 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.db_write_buffer_size: 0 2024-05-17T21:54:24.856 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.write_buffer_manager: 0x560255c7eb40 2024-05-17T21:54:24.856 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-17T21:54:24.856 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-17T21:54:24.856 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-17T21:54:24.856 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.rate_limiter: (nil) 2024-05-17T21:54:24.856 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-17T21:54:24.856 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.wal_recovery_mode: 2 2024-05-17T21:54:24.856 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.enable_thread_tracking: 0 2024-05-17T21:54:24.857 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.enable_pipelined_write: 0 2024-05-17T21:54:24.857 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.unordered_write: 0 2024-05-17T21:54:24.857 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-17T21:54:24.857 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-17T21:54:24.857 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-17T21:54:24.857 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-17T21:54:24.857 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.row_cache: None 2024-05-17T21:54:24.857 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.wal_filter: None 2024-05-17T21:54:24.858 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-17T21:54:24.858 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.allow_ingest_behind: 0 2024-05-17T21:54:24.859 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.two_write_queues: 0 2024-05-17T21:54:24.859 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.manual_wal_flush: 0 2024-05-17T21:54:24.859 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.wal_compression: 0 2024-05-17T21:54:24.859 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.atomic_flush: 0 2024-05-17T21:54:24.859 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-17T21:54:24.859 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-17T21:54:24.859 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-17T21:54:24.859 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.log_readahead_size: 0 2024-05-17T21:54:24.859 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-17T21:54:24.859 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.best_efforts_recovery: 0 2024-05-17T21:54:24.859 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-17T21:54:24.859 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-17T21:54:24.859 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.allow_data_in_errors: 0 2024-05-17T21:54:24.860 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.db_host_id: __hostname__ 2024-05-17T21:54:24.860 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-17T21:54:24.860 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.max_background_jobs: 2 2024-05-17T21:54:24.860 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.max_background_compactions: -1 2024-05-17T21:54:24.860 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.max_subcompactions: 1 2024-05-17T21:54:24.860 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-17T21:54:24.860 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-17T21:54:24.860 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-17T21:54:24.860 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.max_total_wal_size: 0 2024-05-17T21:54:24.860 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-17T21:54:24.860 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-17T21:54:24.860 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-17T21:54:24.860 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-17T21:54:24.860 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.max_open_files: -1 2024-05-17T21:54:24.861 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.bytes_per_sync: 0 2024-05-17T21:54:24.861 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-17T21:54:24.861 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-17T21:54:24.861 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.compaction_readahead_size: 0 2024-05-17T21:54:24.861 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.max_background_flushes: -1 2024-05-17T21:54:24.861 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Compression algorithms supported: 2024-05-17T21:54:24.861 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-17T21:54:24.861 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: kZSTD supported: 0 2024-05-17T21:54:24.861 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: kXpressCompression supported: 0 2024-05-17T21:54:24.861 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: kLZ4HCCompression supported: 1 2024-05-17T21:54:24.861 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: kZlibCompression supported: 1 2024-05-17T21:54:24.861 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: kSnappyCompression supported: 1 2024-05-17T21:54:24.861 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: kLZ4Compression supported: 1 2024-05-17T21:54:24.861 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: kBZip2Compression supported: 0 2024-05-17T21:54:24.862 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-17T21:54:24.862 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-17T21:54:24.862 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi107/store.db/MANIFEST-000015 2024-05-17T21:54:24.862 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-17T21:54:24.862 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-17T21:54:24.862 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.merge_operator: 2024-05-17T21:54:24.862 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.compaction_filter: None 2024-05-17T21:54:24.862 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.compaction_filter_factory: None 2024-05-17T21:54:24.862 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.sst_partitioner_factory: None 2024-05-17T21:54:24.862 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-17T21:54:24.862 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-17T21:54:24.862 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x560255dca100) 2024-05-17T21:54:24.862 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: cache_index_and_filter_blocks: 1 2024-05-17T21:54:24.863 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-17T21:54:24.863 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-17T21:54:24.863 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: pin_top_level_index_and_filter: 1 2024-05-17T21:54:24.863 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: index_type: 0 2024-05-17T21:54:24.863 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: data_block_index_type: 0 2024-05-17T21:54:24.863 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: index_shortening: 1 2024-05-17T21:54:24.863 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-17T21:54:24.863 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: checksum: 4 2024-05-17T21:54:24.863 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: no_block_cache: 0 2024-05-17T21:54:24.863 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: block_cache: 0x560255cc71f0 2024-05-17T21:54:24.863 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: block_cache_name: BinnedLRUCache 2024-05-17T21:54:24.863 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: block_cache_options: 2024-05-17T21:54:24.863 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: capacity : 536870912 2024-05-17T21:54:24.863 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: num_shard_bits : 4 2024-05-17T21:54:24.864 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: strict_capacity_limit : 0 2024-05-17T21:54:24.864 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: high_pri_pool_ratio: 0.000 2024-05-17T21:54:24.864 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: block_cache_compressed: (nil) 2024-05-17T21:54:24.864 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: persistent_cache: (nil) 2024-05-17T21:54:24.864 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: block_size: 4096 2024-05-17T21:54:24.864 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: block_size_deviation: 10 2024-05-17T21:54:24.864 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: block_restart_interval: 16 2024-05-17T21:54:24.864 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: index_block_restart_interval: 1 2024-05-17T21:54:24.864 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: metadata_block_size: 4096 2024-05-17T21:54:24.864 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: partition_filters: 0 2024-05-17T21:54:24.864 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: use_delta_encoding: 1 2024-05-17T21:54:24.864 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: filter_policy: bloomfilter 2024-05-17T21:54:24.864 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: whole_key_filtering: 1 2024-05-17T21:54:24.864 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: verify_compression: 0 2024-05-17T21:54:24.865 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: read_amp_bytes_per_bit: 0 2024-05-17T21:54:24.865 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: format_version: 5 2024-05-17T21:54:24.865 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: enable_index_compression: 1 2024-05-17T21:54:24.865 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: block_align: 0 2024-05-17T21:54:24.865 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: max_auto_readahead_size: 262144 2024-05-17T21:54:24.865 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: prepopulate_block_cache: 0 2024-05-17T21:54:24.865 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: initial_auto_readahead_size: 8192 2024-05-17T21:54:24.865 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: num_file_reads_for_auto_readahead: 2 2024-05-17T21:54:24.865 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.write_buffer_size: 33554432 2024-05-17T21:54:24.865 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.max_write_buffer_number: 2 2024-05-17T21:54:24.865 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.compression: NoCompression 2024-05-17T21:54:24.865 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.bottommost_compression: Disabled 2024-05-17T21:54:24.865 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.prefix_extractor: nullptr 2024-05-17T21:54:24.866 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-17T21:54:24.866 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.num_levels: 7 2024-05-17T21:54:24.866 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-17T21:54:24.866 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-17T21:54:24.866 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-17T21:54:24.866 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-17T21:54:24.866 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-17T21:54:24.866 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-17T21:54:24.866 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-17T21:54:24.866 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-17T21:54:24.866 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-17T21:54:24.866 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-17T21:54:24.866 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-17T21:54:24.866 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-17T21:54:24.867 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-17T21:54:24.867 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.compression_opts.level: 32767 2024-05-17T21:54:24.867 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.compression_opts.strategy: 0 2024-05-17T21:54:24.867 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-17T21:54:24.867 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-17T21:54:24.867 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-17T21:54:24.867 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-17T21:54:24.867 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.compression_opts.enabled: false 2024-05-17T21:54:24.867 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-17T21:54:24.867 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-17T21:54:24.867 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-17T21:54:24.867 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-17T21:54:24.867 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.target_file_size_base: 67108864 2024-05-17T21:54:24.867 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-17T21:54:24.868 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-17T21:54:24.868 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-17T21:54:24.868 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-17T21:54:24.868 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-17T21:54:24.868 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-17T21:54:24.868 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-17T21:54:24.868 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-17T21:54:24.868 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-17T21:54:24.868 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-17T21:54:24.868 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-17T21:54:24.868 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-17T21:54:24.868 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-17T21:54:24.868 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-17T21:54:24.869 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.arena_block_size: 1048576 2024-05-17T21:54:24.869 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-17T21:54:24.869 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-17T21:54:24.869 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.disable_auto_compactions: 0 2024-05-17T21:54:24.869 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-17T21:54:24.869 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-17T21:54:24.869 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-17T21:54:24.869 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-17T21:54:24.869 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-17T21:54:24.869 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-17T21:54:24.869 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-17T21:54:24.869 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-17T21:54:24.869 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-17T21:54:24.869 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-17T21:54:24.870 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-05-17T21:54:24.870 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.inplace_update_support: 0 2024-05-17T21:54:24.870 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-17T21:54:24.870 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-17T21:54:24.870 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-17T21:54:24.870 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-17T21:54:24.870 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.bloom_locality: 0 2024-05-17T21:54:24.870 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.max_successive_merges: 0 2024-05-17T21:54:24.870 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-17T21:54:24.870 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.paranoid_file_checks: 0 2024-05-17T21:54:24.870 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.force_consistency_checks: 1 2024-05-17T21:54:24.870 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.report_bg_io_stats: 0 2024-05-17T21:54:24.871 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.ttl: 2592000 2024-05-17T21:54:24.871 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-17T21:54:24.871 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-17T21:54:24.871 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-17T21:54:24.871 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.enable_blob_files: false 2024-05-17T21:54:24.871 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.min_blob_size: 0 2024-05-17T21:54:24.871 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.blob_file_size: 268435456 2024-05-17T21:54:24.871 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-17T21:54:24.871 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-17T21:54:24.871 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-17T21:54:24.871 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-17T21:54:24.871 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-17T21:54:24.871 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.blob_file_starting_level: 0 2024-05-17T21:54:24.872 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-17T21:54:24.872 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: [table/block_based/block_based_table_reader.cc:721] At least one SST file opened without unique ID to verify: 30.sst 2024-05-17T21:54:24.872 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-05-17T21:54:24.872 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi107/store.db/MANIFEST-000015 succeeded,manifest_file_number is 15, next_file_number is 32, last_sequence is 10425, log_number is 28,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-05-17T21:54:24.872 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 28 2024-05-17T21:54:24.872 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 550fdaca-bfae-486e-8611-6ce40e96b2ee 2024-05-17T21:54:24.872 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715982864596639, "job": 1, "event": "recovery_started", "wal_files": [28]} 2024-05-17T21:54:24.872 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #28 mode 2 2024-05-17T21:54:24.872 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715982864604764, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 33, "file_size": 2219223, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 10426, "largest_seqno": 11433, "table_properties": {"data_size": 2213974, "index_size": 3052, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 1157, "raw_key_size": 10504, "raw_average_key_size": 23, "raw_value_size": 2204154, "raw_average_value_size": 4833, "num_data_blocks": 139, "num_entries": 456, "num_filter_entries": 456, "num_deletions": 2, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1715982864, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "550fdaca-bfae-486e-8611-6ce40e96b2ee", "db_session_id": "GR3AZQO5S9NMPACFNA76", "orig_file_number": 33, "seqno_to_time_mapping": "N/A"}} 2024-05-17T21:54:24.872 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715982864604868, "job": 1, "event": "recovery_finished"} 2024-05-17T21:54:24.872 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: [db/version_set.cc:5047] Creating manifest 35 2024-05-17T21:54:24.872 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-05-17T21:54:24.872 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi107/store.db/000028.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-17T21:54:24.873 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x560255ca9c00 2024-05-17T21:54:24.873 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: DB pointer 0x56025aa9e000 2024-05-17T21:54:24.873 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-17T21:54:24.873 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-17T21:54:24.873 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: ** DB Stats ** 2024-05-17T21:54:24.873 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-17T21:54:24.873 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-17T21:54:24.873 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-17T21:54:24.873 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-17T21:54:24.873 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-17T21:54:24.873 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-17T21:54:24.873 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-17T21:54:24.873 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: 2024-05-17T21:54:24.874 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: ** Compaction Stats [default] ** 2024-05-17T21:54:24.874 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-17T21:54:24.874 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-17T21:54:24.874 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: L0 1/0 2.12 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 542.4 0.00 0.00 1 0.004 0 0 0.0 0.0 2024-05-17T21:54:24.874 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: L6 1/0 5.57 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-05-17T21:54:24.874 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Sum 2/0 7.68 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 542.4 0.00 0.00 1 0.004 0 0 0.0 0.0 2024-05-17T21:54:24.874 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 542.4 0.00 0.00 1 0.004 0 0 0.0 0.0 2024-05-17T21:54:24.874 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: 2024-05-17T21:54:24.874 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: ** Compaction Stats [default] ** 2024-05-17T21:54:24.874 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-17T21:54:24.874 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-17T21:54:24.874 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 542.4 0.00 0.00 1 0.004 0 0 0.0 0.0 2024-05-17T21:54:24.874 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: 2024-05-17T21:54:24.875 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-17T21:54:24.875 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: 2024-05-17T21:54:24.875 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-17T21:54:24.875 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Flush(GB): cumulative 0.002, interval 0.002 2024-05-17T21:54:24.875 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-17T21:54:24.875 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-17T21:54:24.875 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-17T21:54:24.875 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-17T21:54:24.875 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Cumulative compaction: 0.00 GB write, 168.17 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-17T21:54:24.875 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Interval compaction: 0.00 GB write, 168.17 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-17T21:54:24.875 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-17T21:54:24.875 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Block cache BinnedLRUCache@0x560255cc71f0#2 capacity: 512.00 MB usage: 4.37 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-05-17T21:54:24.875 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,1.29 KB,0.000245869%) IndexBlock(1,3.08 KB,0.000587106%) Misc(1,0.00 KB,0%) 2024-05-17T21:54:24.875 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: 2024-05-17T21:54:24.876 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-17T21:54:24.876 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: starting mon.smithi107 rank 0 at public addrs [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] at bind addrs [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi107 fsid 3722731a-1496-11ef-bc9a-c7b262605968 2024-05-17T21:54:24.876 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: mon.smithi107@-1(???) e2 preinit fsid 3722731a-1496-11ef-bc9a-c7b262605968 2024-05-17T21:54:24.876 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: mon.smithi107@-1(???).mds e15 new map 2024-05-17T21:54:24.876 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: mon.smithi107@-1(???).mds e15 print_map 2024-05-17T21:54:24.876 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: e15 2024-05-17T21:54:24.876 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-17T21:54:24.876 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-17T21:54:24.876 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: legacy client fscid: 1 2024-05-17T21:54:24.876 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: 2024-05-17T21:54:24.876 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Filesystem 'cephfs' (1) 2024-05-17T21:54:24.876 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: fs_name cephfs 2024-05-17T21:54:24.876 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: epoch 15 2024-05-17T21:54:24.876 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-17T21:54:24.877 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: created 2024-05-17T21:49:16.836333+0000 2024-05-17T21:54:24.877 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: modified 2024-05-17T21:49:34.873628+0000 2024-05-17T21:54:24.877 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: tableserver 0 2024-05-17T21:54:24.877 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: root 0 2024-05-17T21:54:24.877 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: session_timeout 60 2024-05-17T21:54:24.877 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: session_autoclose 300 2024-05-17T21:54:24.877 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: max_file_size 1099511627776 2024-05-17T21:54:24.877 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: max_xattr_size 65536 2024-05-17T21:54:24.877 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: required_client_features {} 2024-05-17T21:54:24.877 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: last_failure 0 2024-05-17T21:54:24.877 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: last_failure_osd_epoch 46 2024-05-17T21:54:24.877 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-17T21:54:24.877 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: max_mds 1 2024-05-17T21:54:24.877 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: in 0 2024-05-17T21:54:24.878 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: up {0=24259} 2024-05-17T21:54:24.878 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: failed 2024-05-17T21:54:24.878 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: damaged 2024-05-17T21:54:24.878 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: stopped 2024-05-17T21:54:24.878 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: data_pools [3] 2024-05-17T21:54:24.878 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: metadata_pool 2 2024-05-17T21:54:24.878 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: inline_data enabled 2024-05-17T21:54:24.878 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: balancer 2024-05-17T21:54:24.878 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: bal_rank_mask -1 2024-05-17T21:54:24.878 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: standby_count_wanted 1 2024-05-17T21:54:24.878 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: qdb_cluster leader: 0 members: 2024-05-17T21:54:24.878 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: [mds.cephfs.smithi138.gwbtyd{0:24259} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.138:6824/2779634919,v1:172.21.15.138:6825/2779634919] compat {c=[1],r=[1],i=[7ff]}] 2024-05-17T21:54:24.878 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: [mds.cephfs.smithi107.drqody{0:14486} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.107:6828/2275807100,v1:172.21.15.107:6829/2275807100] compat {c=[1],r=[1],i=[7ff]}] 2024-05-17T21:54:24.879 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: 2024-05-17T21:54:24.879 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: 2024-05-17T21:54:24.879 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Standby daemons: 2024-05-17T21:54:24.879 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: 2024-05-17T21:54:24.879 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: [mds.cephfs.smithi107.idmbgj{-1:14498} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.107:6826/1460186079,v1:172.21.15.107:6827/1460186079] compat {c=[1],r=[1],i=[7ff]}] 2024-05-17T21:54:24.879 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: [mds.cephfs.smithi138.zckrvs{-1:24267} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.138:6826/170628661,v1:172.21.15.138:6827/170628661] compat {c=[1],r=[1],i=[7ff]}] 2024-05-17T21:54:24.879 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: mon.smithi107@-1(???).osd e48 crush map has features 3314933000852226048, adjusting msgr requires 2024-05-17T21:54:24.879 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: mon.smithi107@-1(???).osd e48 crush map has features 288514051259236352, adjusting msgr requires 2024-05-17T21:54:24.879 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: mon.smithi107@-1(???).osd e48 crush map has features 288514051259236352, adjusting msgr requires 2024-05-17T21:54:24.879 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: mon.smithi107@-1(???).osd e48 crush map has features 288514051259236352, adjusting msgr requires 2024-05-17T21:54:24.879 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 ceph-mon[93224]: mon.smithi107@-1(???).paxosservice(auth 1..21) refresh upgraded, format 0 -> 3 2024-05-17T21:54:24.879 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 bash[93206]: 9717f3169f2f33e9afca7a12e2206b211d65c2fdda90950f2c75eb685a3f1bd8 2024-05-17T21:54:24.879 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:24 smithi107 systemd[1]: Started Ceph mon.smithi107 for 3722731a-1496-11ef-bc9a-c7b262605968. 2024-05-17T21:54:26.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:26 smithi138 ceph-mon[35948]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mon metadata", "id": "smithi107"}]: dispatch 2024-05-17T21:54:26.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:26 smithi138 ceph-mon[35948]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mon metadata", "id": "smithi138"}]: dispatch 2024-05-17T21:54:26.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:26 smithi138 ceph-mon[35948]: pgmap v33: 65 pgs: 65 active+clean; 304 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 316 op/s 2024-05-17T21:54:26.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:26 smithi138 ceph-mon[35948]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:54:26.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:26 smithi138 ceph-mon[35948]: mon.smithi107 calling monitor election 2024-05-17T21:54:26.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:26 smithi138 ceph-mon[35948]: mon.smithi107 is new leader, mons smithi107,smithi138 in quorum (ranks 0,1) 2024-05-17T21:54:26.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:26 smithi138 ceph-mon[35948]: monmap epoch 2 2024-05-17T21:54:26.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:26 smithi138 ceph-mon[35948]: fsid 3722731a-1496-11ef-bc9a-c7b262605968 2024-05-17T21:54:26.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:26 smithi138 ceph-mon[35948]: last_changed 2024-05-17T21:44:56.895553+0000 2024-05-17T21:54:26.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:26 smithi138 ceph-mon[35948]: created 2024-05-17T21:42:10.472726+0000 2024-05-17T21:54:26.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:26 smithi138 ceph-mon[35948]: min_mon_release 17 (quincy) 2024-05-17T21:54:26.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:26 smithi138 ceph-mon[35948]: election_strategy: 1 2024-05-17T21:54:26.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:26 smithi138 ceph-mon[35948]: 0: [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] mon.smithi107 2024-05-17T21:54:26.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:26 smithi138 ceph-mon[35948]: 1: [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] mon.smithi138 2024-05-17T21:54:26.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:26 smithi138 ceph-mon[35948]: fsmap cephfs:1 {0=cephfs.smithi138.gwbtyd=up:active} 1 up:standby-replay 2 up:standby 2024-05-17T21:54:26.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:26 smithi138 ceph-mon[35948]: osdmap e48: 6 total, 6 up, 6 in 2024-05-17T21:54:26.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:26 smithi138 ceph-mon[35948]: mgrmap e32: smithi107.xhhyiw(active, since 62s), standbys: smithi138.napnll 2024-05-17T21:54:26.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:26 smithi138 ceph-mon[35948]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-17T21:54:26.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:26 smithi138 ceph-mon[35948]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-17T21:54:26.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:26 smithi138 ceph-mon[35948]: fs cephfs has deprecated feature inline_data enabled. 2024-05-17T21:54:26.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:26 smithi138 ceph-mon[35948]: from='mgr.14750 ' entity='' 2024-05-17T21:54:26.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:26 smithi138 ceph-mon[35948]: mgrmap e33: smithi107.xhhyiw(active, since 62s), standbys: smithi138.napnll 2024-05-17T21:54:26.348 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:26 smithi107 ceph-mon[93224]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mon metadata", "id": "smithi107"}]: dispatch 2024-05-17T21:54:26.348 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:26 smithi107 ceph-mon[93224]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mon metadata", "id": "smithi138"}]: dispatch 2024-05-17T21:54:26.348 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:26 smithi107 ceph-mon[93224]: pgmap v33: 65 pgs: 65 active+clean; 304 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 316 op/s 2024-05-17T21:54:26.349 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:26 smithi107 ceph-mon[93224]: from='mgr.14750 172.21.15.107:0/3018950947' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:54:26.349 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:26 smithi107 ceph-mon[93224]: mon.smithi107 calling monitor election 2024-05-17T21:54:26.349 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:26 smithi107 ceph-mon[93224]: mon.smithi107 is new leader, mons smithi107,smithi138 in quorum (ranks 0,1) 2024-05-17T21:54:26.349 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:26 smithi107 ceph-mon[93224]: monmap epoch 2 2024-05-17T21:54:26.349 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:26 smithi107 ceph-mon[93224]: fsid 3722731a-1496-11ef-bc9a-c7b262605968 2024-05-17T21:54:26.349 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:26 smithi107 ceph-mon[93224]: last_changed 2024-05-17T21:44:56.895553+0000 2024-05-17T21:54:26.349 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:26 smithi107 ceph-mon[93224]: created 2024-05-17T21:42:10.472726+0000 2024-05-17T21:54:26.349 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:26 smithi107 ceph-mon[93224]: min_mon_release 17 (quincy) 2024-05-17T21:54:26.349 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:26 smithi107 ceph-mon[93224]: election_strategy: 1 2024-05-17T21:54:26.349 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:26 smithi107 ceph-mon[93224]: 0: [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] mon.smithi107 2024-05-17T21:54:26.349 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:26 smithi107 ceph-mon[93224]: 1: [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] mon.smithi138 2024-05-17T21:54:26.349 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:26 smithi107 ceph-mon[93224]: fsmap cephfs:1 {0=cephfs.smithi138.gwbtyd=up:active} 1 up:standby-replay 2 up:standby 2024-05-17T21:54:26.349 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:26 smithi107 ceph-mon[93224]: osdmap e48: 6 total, 6 up, 6 in 2024-05-17T21:54:26.350 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:26 smithi107 ceph-mon[93224]: mgrmap e32: smithi107.xhhyiw(active, since 62s), standbys: smithi138.napnll 2024-05-17T21:54:26.350 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:26 smithi107 ceph-mon[93224]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-17T21:54:26.350 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:26 smithi107 ceph-mon[93224]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-17T21:54:26.350 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:26 smithi107 ceph-mon[93224]: fs cephfs has deprecated feature inline_data enabled. 2024-05-17T21:54:26.350 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:26 smithi107 ceph-mon[93224]: from='mgr.14750 ' entity='' 2024-05-17T21:54:26.350 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:26 smithi107 ceph-mon[93224]: mgrmap e33: smithi107.xhhyiw(active, since 62s), standbys: smithi138.napnll 2024-05-17T21:54:34.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:34 smithi107 ceph-mon[93224]: Active manager daemon smithi107.xhhyiw restarted 2024-05-17T21:54:34.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:34 smithi107 ceph-mon[93224]: Activating manager daemon smithi107.xhhyiw 2024-05-17T21:54:34.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:34 smithi107 ceph-mon[93224]: osdmap e49: 6 total, 6 up, 6 in 2024-05-17T21:54:34.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:34 smithi107 ceph-mon[93224]: mgrmap e34: smithi107.xhhyiw(active, starting, since 0.0103253s), standbys: smithi138.napnll 2024-05-17T21:54:34.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:34 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mon metadata", "id": "smithi107"}]: dispatch 2024-05-17T21:54:34.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:34 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mon metadata", "id": "smithi138"}]: dispatch 2024-05-17T21:54:34.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:34 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi107.drqody"}]: dispatch 2024-05-17T21:54:34.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:34 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi107.idmbgj"}]: dispatch 2024-05-17T21:54:34.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:34 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi138.gwbtyd"}]: dispatch 2024-05-17T21:54:34.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:34 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi138.zckrvs"}]: dispatch 2024-05-17T21:54:34.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:34 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mgr metadata", "who": "smithi107.xhhyiw", "id": "smithi107.xhhyiw"}]: dispatch 2024-05-17T21:54:34.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:34 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mgr metadata", "who": "smithi138.napnll", "id": "smithi138.napnll"}]: dispatch 2024-05-17T21:54:34.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:34 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-17T21:54:34.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:34 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-17T21:54:34.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:34 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-17T21:54:34.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:34 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-17T21:54:34.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:34 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-17T21:54:34.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:34 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-17T21:54:34.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:34 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-17T21:54:34.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:34 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-17T21:54:34.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:34 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-17T21:54:34.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:34 smithi107 ceph-mon[93224]: Manager daemon smithi107.xhhyiw is now available 2024-05-17T21:54:34.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:34 smithi138 ceph-mon[35948]: Active manager daemon smithi107.xhhyiw restarted 2024-05-17T21:54:34.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:34 smithi138 ceph-mon[35948]: Activating manager daemon smithi107.xhhyiw 2024-05-17T21:54:34.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:34 smithi138 ceph-mon[35948]: osdmap e49: 6 total, 6 up, 6 in 2024-05-17T21:54:34.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:34 smithi138 ceph-mon[35948]: mgrmap e34: smithi107.xhhyiw(active, starting, since 0.0103253s), standbys: smithi138.napnll 2024-05-17T21:54:34.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:34 smithi138 ceph-mon[35948]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mon metadata", "id": "smithi107"}]: dispatch 2024-05-17T21:54:34.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:34 smithi138 ceph-mon[35948]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mon metadata", "id": "smithi138"}]: dispatch 2024-05-17T21:54:34.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:34 smithi138 ceph-mon[35948]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi107.drqody"}]: dispatch 2024-05-17T21:54:34.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:34 smithi138 ceph-mon[35948]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi107.idmbgj"}]: dispatch 2024-05-17T21:54:34.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:34 smithi138 ceph-mon[35948]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi138.gwbtyd"}]: dispatch 2024-05-17T21:54:34.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:34 smithi138 ceph-mon[35948]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi138.zckrvs"}]: dispatch 2024-05-17T21:54:34.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:34 smithi138 ceph-mon[35948]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mgr metadata", "who": "smithi107.xhhyiw", "id": "smithi107.xhhyiw"}]: dispatch 2024-05-17T21:54:34.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:34 smithi138 ceph-mon[35948]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mgr metadata", "who": "smithi138.napnll", "id": "smithi138.napnll"}]: dispatch 2024-05-17T21:54:34.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:34 smithi138 ceph-mon[35948]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-17T21:54:34.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:34 smithi138 ceph-mon[35948]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-17T21:54:34.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:34 smithi138 ceph-mon[35948]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-17T21:54:34.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:34 smithi138 ceph-mon[35948]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-17T21:54:34.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:34 smithi138 ceph-mon[35948]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-17T21:54:34.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:34 smithi138 ceph-mon[35948]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-17T21:54:34.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:34 smithi138 ceph-mon[35948]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-17T21:54:34.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:34 smithi138 ceph-mon[35948]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-17T21:54:34.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:34 smithi138 ceph-mon[35948]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-17T21:54:34.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:34 smithi138 ceph-mon[35948]: Manager daemon smithi107.xhhyiw is now available 2024-05-17T21:54:35.711 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:35 smithi138 ceph-mon[35948]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:54:35.711 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:35 smithi138 ceph-mon[35948]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:54:35.711 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:35 smithi138 ceph-mon[35948]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi107.xhhyiw/mirror_snapshot_schedule"}]: dispatch 2024-05-17T21:54:35.711 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:35 smithi138 ceph-mon[35948]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi107.xhhyiw/trash_purge_schedule"}]: dispatch 2024-05-17T21:54:35.711 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:35 smithi138 ceph-mon[35948]: [17/May/2024:21:54:34] ENGINE Bus STARTING 2024-05-17T21:54:35.711 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:35 smithi138 ceph-mon[35948]: [17/May/2024:21:54:35] ENGINE Serving on https://172.21.15.107:7150 2024-05-17T21:54:35.711 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:35 smithi138 ceph-mon[35948]: mgrmap e35: smithi107.xhhyiw(active, since 1.01107s), standbys: smithi138.napnll 2024-05-17T21:54:35.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:35 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:54:35.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:35 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:54:35.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:35 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi107.xhhyiw/mirror_snapshot_schedule"}]: dispatch 2024-05-17T21:54:35.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:35 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi107.xhhyiw/trash_purge_schedule"}]: dispatch 2024-05-17T21:54:35.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:35 smithi107 ceph-mon[93224]: [17/May/2024:21:54:34] ENGINE Bus STARTING 2024-05-17T21:54:35.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:35 smithi107 ceph-mon[93224]: [17/May/2024:21:54:35] ENGINE Serving on https://172.21.15.107:7150 2024-05-17T21:54:35.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:35 smithi107 ceph-mon[93224]: mgrmap e35: smithi107.xhhyiw(active, since 1.01107s), standbys: smithi138.napnll 2024-05-17T21:54:36.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:36 smithi138 ceph-mon[35948]: [17/May/2024:21:54:35] ENGINE Serving on http://172.21.15.107:8765 2024-05-17T21:54:36.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:36 smithi138 ceph-mon[35948]: [17/May/2024:21:54:35] ENGINE Bus STARTED 2024-05-17T21:54:36.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:36 smithi138 ceph-mon[35948]: pgmap v3: 65 pgs: 65 active+clean; 281 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-05-17T21:54:36.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:36 smithi138 ceph-mon[35948]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:54:36.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:36 smithi138 ceph-mon[35948]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:54:36.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:36 smithi138 ceph-mon[35948]: Standby manager daemon smithi138.napnll restarted 2024-05-17T21:54:36.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:36 smithi138 ceph-mon[35948]: Standby manager daemon smithi138.napnll started 2024-05-17T21:54:36.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:36 smithi138 ceph-mon[35948]: from='mgr.? 172.21.15.138:0/1304368261' entity='mgr.smithi138.napnll' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi138.napnll/crt"}]: dispatch 2024-05-17T21:54:36.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:36 smithi138 ceph-mon[35948]: from='mgr.? 172.21.15.138:0/1304368261' entity='mgr.smithi138.napnll' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-17T21:54:36.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:36 smithi138 ceph-mon[35948]: from='mgr.? 172.21.15.138:0/1304368261' entity='mgr.smithi138.napnll' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi138.napnll/key"}]: dispatch 2024-05-17T21:54:36.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:36 smithi138 ceph-mon[35948]: from='mgr.? 172.21.15.138:0/1304368261' entity='mgr.smithi138.napnll' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-17T21:54:36.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:36 smithi107 ceph-mon[93224]: [17/May/2024:21:54:35] ENGINE Serving on http://172.21.15.107:8765 2024-05-17T21:54:36.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:36 smithi107 ceph-mon[93224]: [17/May/2024:21:54:35] ENGINE Bus STARTED 2024-05-17T21:54:36.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:36 smithi107 ceph-mon[93224]: pgmap v3: 65 pgs: 65 active+clean; 281 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-05-17T21:54:36.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:36 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:54:36.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:36 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:54:36.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:36 smithi107 ceph-mon[93224]: Standby manager daemon smithi138.napnll restarted 2024-05-17T21:54:36.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:36 smithi107 ceph-mon[93224]: Standby manager daemon smithi138.napnll started 2024-05-17T21:54:36.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:36 smithi107 ceph-mon[93224]: from='mgr.? 172.21.15.138:0/1304368261' entity='mgr.smithi138.napnll' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi138.napnll/crt"}]: dispatch 2024-05-17T21:54:36.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:36 smithi107 ceph-mon[93224]: from='mgr.? 172.21.15.138:0/1304368261' entity='mgr.smithi138.napnll' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-17T21:54:36.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:36 smithi107 ceph-mon[93224]: from='mgr.? 172.21.15.138:0/1304368261' entity='mgr.smithi138.napnll' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi138.napnll/key"}]: dispatch 2024-05-17T21:54:36.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:36 smithi107 ceph-mon[93224]: from='mgr.? 172.21.15.138:0/1304368261' entity='mgr.smithi138.napnll' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-17T21:54:37.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:37 smithi107 ceph-mon[93224]: pgmap v4: 65 pgs: 65 active+clean; 281 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-05-17T21:54:37.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:37 smithi107 ceph-mon[93224]: mgrmap e36: smithi107.xhhyiw(active, since 2s), standbys: smithi138.napnll 2024-05-17T21:54:38.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:37 smithi138 ceph-mon[35948]: pgmap v4: 65 pgs: 65 active+clean; 281 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-05-17T21:54:38.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:37 smithi138 ceph-mon[35948]: mgrmap e36: smithi107.xhhyiw(active, since 2s), standbys: smithi138.napnll 2024-05-17T21:54:38.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:38.578+0000 7f94c2571700 1 -- 172.21.15.107:0/2907788702 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94bc0fbcb0 msgr2=0x7f94bc0fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:54:38.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:38.578+0000 7f94c2571700 1 --2- 172.21.15.107:0/2907788702 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94bc0fbcb0 0x7f94bc0fc0c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f94ac004940 tx=0x7f94ac009e30 comp rx=0 tx=0).stop 2024-05-17T21:54:38.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:38.578+0000 7f94c2571700 1 -- 172.21.15.107:0/2907788702 shutdown_connections 2024-05-17T21:54:38.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:38.578+0000 7f94c2571700 1 --2- 172.21.15.107:0/2907788702 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f94bc0fcf60 0x7f94bc0fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:54:38.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:38.578+0000 7f94c2571700 1 --2- 172.21.15.107:0/2907788702 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94bc0fbcb0 0x7f94bc0fc0c0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:54:38.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:38.578+0000 7f94c2571700 1 -- 172.21.15.107:0/2907788702 >> 172.21.15.107:0/2907788702 conn(0x7f94bc0f7430 msgr2=0x7f94bc0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:54:38.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:38.578+0000 7f94c2571700 1 -- 172.21.15.107:0/2907788702 shutdown_connections 2024-05-17T21:54:38.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:38.578+0000 7f94c2571700 1 -- 172.21.15.107:0/2907788702 wait complete. 2024-05-17T21:54:38.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:38.579+0000 7f94c2571700 1 Processor -- start 2024-05-17T21:54:38.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:38.579+0000 7f94c2571700 1 -- start start 2024-05-17T21:54:38.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:38.579+0000 7f94c2571700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f94bc0fbcb0 0x7f94bc10c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:54:38.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:38.579+0000 7f94c2571700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94bc0fcf60 0x7f94bc10ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:54:38.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:38.579+0000 7f94c2571700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94bc10d400 con 0x7f94bc0fcf60 2024-05-17T21:54:38.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:38.579+0000 7f94c2571700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94bc10dc00 con 0x7f94bc0fbcb0 2024-05-17T21:54:38.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:38.579+0000 7f94bbfff700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f94bc0fbcb0 0x7f94bc10c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:54:38.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:38.579+0000 7f94bbfff700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f94bc0fbcb0 0x7f94bc10c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.107:44104/0 (socket says 172.21.15.107:44104) 2024-05-17T21:54:38.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:38.579+0000 7f94bbfff700 1 -- 172.21.15.107:0/3159620506 learned_addr learned my addr 172.21.15.107:0/3159620506 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:54:38.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:38.579+0000 7f94bb7fe700 1 --2- 172.21.15.107:0/3159620506 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94bc0fcf60 0x7f94bc10ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:54:38.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:38.580+0000 7f94bbfff700 1 -- 172.21.15.107:0/3159620506 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94bc0fcf60 msgr2=0x7f94bc10ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:54:38.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:38.580+0000 7f94bbfff700 1 --2- 172.21.15.107:0/3159620506 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94bc0fcf60 0x7f94bc10ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:54:38.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:38.580+0000 7f94bbfff700 1 -- 172.21.15.107:0/3159620506 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94ac005040 con 0x7f94bc0fbcb0 2024-05-17T21:54:38.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:38.580+0000 7f94bbfff700 1 --2- 172.21.15.107:0/3159620506 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f94bc0fbcb0 0x7f94bc10c8c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f94ac013870 tx=0x7f94ac009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:54:38.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:38.580+0000 7f94b8ff9700 1 -- 172.21.15.107:0/3159620506 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f94ac00e070 con 0x7f94bc0fbcb0 2024-05-17T21:54:38.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:38.580+0000 7f94b8ff9700 1 -- 172.21.15.107:0/3159620506 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f94ac01e070 con 0x7f94bc0fbcb0 2024-05-17T21:54:38.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:38.580+0000 7f94b8ff9700 1 -- 172.21.15.107:0/3159620506 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f94ac0193e0 con 0x7f94bc0fbcb0 2024-05-17T21:54:38.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:38.580+0000 7f94c2571700 1 -- 172.21.15.107:0/3159620506 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94bc10de80 con 0x7f94bc0fbcb0 2024-05-17T21:54:38.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:38.580+0000 7f94c2571700 1 -- 172.21.15.107:0/3159620506 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94bc10e300 con 0x7f94bc0fbcb0 2024-05-17T21:54:38.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:38.581+0000 7f94c2571700 1 -- 172.21.15.107:0/3159620506 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94bc05f5c0 con 0x7f94bc0fbcb0 2024-05-17T21:54:38.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:38.582+0000 7f94b8ff9700 1 -- 172.21.15.107:0/3159620506 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 36) v1 ==== 100128+0+0 (secure 0 0 0) 0x7f94ac01a830 con 0x7f94bc0fbcb0 2024-05-17T21:54:38.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:38.582+0000 7f94b8ff9700 1 --2- 172.21.15.107:0/3159620506 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f94a4078a10 0x7f94a407aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:54:38.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:38.582+0000 7f94b8ff9700 1 -- 172.21.15.107:0/3159620506 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f94ac09bc80 con 0x7f94bc0fbcb0 2024-05-17T21:54:38.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:38.582+0000 7f94bb7fe700 1 --2- 172.21.15.107:0/3159620506 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f94a4078a10 0x7f94a407aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:54:38.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:38.583+0000 7f94bb7fe700 1 --2- 172.21.15.107:0/3159620506 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f94a4078a10 0x7f94a407aec0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f94b0009b80 tx=0x7f94b0014040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:54:38.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:38.587+0000 7f94b8ff9700 1 -- 172.21.15.107:0/3159620506 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f94ac063900 con 0x7f94bc0fbcb0 2024-05-17T21:54:38.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:38.599+0000 7f94b8ff9700 1 -- 172.21.15.107:0/3159620506 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mgrmap(e 37) v1 ==== 100184+0+0 (secure 0 0 0) 0x7f94ac05fdc0 con 0x7f94bc0fbcb0 2024-05-17T21:54:38.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:38.841+0000 7f94c2571700 1 -- 172.21.15.107:0/3159620506 --> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f94bc10e740 con 0x7f94a4078a10 2024-05-17T21:54:38.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:38.843+0000 7f94b8ff9700 1 -- 172.21.15.107:0/3159620506 <== mgr.34100 v2:172.21.15.107:6800/3205268226 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7f94bc10e740 con 0x7f94a4078a10 2024-05-17T21:54:38.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:38.846+0000 7f94c2571700 1 -- 172.21.15.107:0/3159620506 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f94a4078a10 msgr2=0x7f94a407aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:54:38.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:38.846+0000 7f94c2571700 1 --2- 172.21.15.107:0/3159620506 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f94a4078a10 0x7f94a407aec0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f94b0009b80 tx=0x7f94b0014040 comp rx=0 tx=0).stop 2024-05-17T21:54:38.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:38.846+0000 7f94c2571700 1 -- 172.21.15.107:0/3159620506 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f94bc0fbcb0 msgr2=0x7f94bc10c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:54:38.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:38.846+0000 7f94c2571700 1 --2- 172.21.15.107:0/3159620506 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f94bc0fbcb0 0x7f94bc10c8c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f94ac013870 tx=0x7f94ac009e30 comp rx=0 tx=0).stop 2024-05-17T21:54:38.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:38.846+0000 7f94c2571700 1 -- 172.21.15.107:0/3159620506 shutdown_connections 2024-05-17T21:54:38.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:38.846+0000 7f94c2571700 1 --2- 172.21.15.107:0/3159620506 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f94a4078a10 0x7f94a407aec0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:54:38.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:38.846+0000 7f94c2571700 1 --2- 172.21.15.107:0/3159620506 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f94bc0fbcb0 0x7f94bc10c8c0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:54:38.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:38.846+0000 7f94c2571700 1 --2- 172.21.15.107:0/3159620506 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94bc0fcf60 0x7f94bc10ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:54:38.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:38.846+0000 7f94c2571700 1 -- 172.21.15.107:0/3159620506 >> 172.21.15.107:0/3159620506 conn(0x7f94bc0f7430 msgr2=0x7f94bc0f8ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:54:38.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:38.846+0000 7f94c2571700 1 -- 172.21.15.107:0/3159620506 shutdown_connections 2024-05-17T21:54:38.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:38.846+0000 7f94c2571700 1 -- 172.21.15.107:0/3159620506 wait complete. 2024-05-17T21:54:38.849 INFO:teuthology.orchestra.run.smithi107.stdout:true 2024-05-17T21:54:38.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:38.962+0000 7f4c93d25700 1 -- 172.21.15.107:0/4014210634 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c8c0fbd40 msgr2=0x7f4c8c0fc150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:54:38.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:38.962+0000 7f4c93d25700 1 --2- 172.21.15.107:0/4014210634 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c8c0fbd40 0x7f4c8c0fc150 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f4c88004940 tx=0x7f4c88009e30 comp rx=0 tx=0).stop 2024-05-17T21:54:38.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:38.962+0000 7f4c93d25700 1 -- 172.21.15.107:0/4014210634 shutdown_connections 2024-05-17T21:54:38.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:38.962+0000 7f4c93d25700 1 --2- 172.21.15.107:0/4014210634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c8c0fcff0 0x7f4c8c0fd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:54:38.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:38.962+0000 7f4c93d25700 1 --2- 172.21.15.107:0/4014210634 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c8c0fbd40 0x7f4c8c0fc150 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:54:38.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:38.962+0000 7f4c93d25700 1 -- 172.21.15.107:0/4014210634 >> 172.21.15.107:0/4014210634 conn(0x7f4c8c0f74c0 msgr2=0x7f4c8c0f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:54:38.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:38.962+0000 7f4c93d25700 1 -- 172.21.15.107:0/4014210634 shutdown_connections 2024-05-17T21:54:38.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:38.962+0000 7f4c93d25700 1 -- 172.21.15.107:0/4014210634 wait complete. 2024-05-17T21:54:38.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:38.963+0000 7f4c93d25700 1 Processor -- start 2024-05-17T21:54:38.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:38.963+0000 7f4c93d25700 1 -- start start 2024-05-17T21:54:38.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:38.963+0000 7f4c93d25700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c8c0fbd40 0x7f4c8c1106b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:54:38.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:38.963+0000 7f4c93d25700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c8c0fcff0 0x7f4c8c112c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:54:38.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:38.963+0000 7f4c93d25700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c8c066bc0 con 0x7f4c8c0fcff0 2024-05-17T21:54:38.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:38.963+0000 7f4c93d25700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c8c066d00 con 0x7f4c8c0fbd40 2024-05-17T21:54:38.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:38.963+0000 7f4c912c0700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c8c0fcff0 0x7f4c8c112c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:54:38.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:38.963+0000 7f4c912c0700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c8c0fcff0 0x7f4c8c112c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48420/0 (socket says 172.21.15.107:48420) 2024-05-17T21:54:38.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:38.963+0000 7f4c912c0700 1 -- 172.21.15.107:0/2442465578 learned_addr learned my addr 172.21.15.107:0/2442465578 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:54:38.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:38.963+0000 7f4c91ac1700 1 --2- 172.21.15.107:0/2442465578 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c8c0fbd40 0x7f4c8c1106b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:54:38.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:38.964+0000 7f4c912c0700 1 -- 172.21.15.107:0/2442465578 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c8c0fbd40 msgr2=0x7f4c8c1106b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:54:38.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:38.964+0000 7f4c912c0700 1 --2- 172.21.15.107:0/2442465578 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c8c0fbd40 0x7f4c8c1106b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:54:38.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:38.964+0000 7f4c912c0700 1 -- 172.21.15.107:0/2442465578 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c88005040 con 0x7f4c8c0fcff0 2024-05-17T21:54:38.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:38.964+0000 7f4c912c0700 1 --2- 172.21.15.107:0/2442465578 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c8c0fcff0 0x7f4c8c112c00 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c004f20 tx=0x7f4c7c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:54:38.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:38.964+0000 7f4c827fc700 1 -- 172.21.15.107:0/2442465578 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c7c007ea0 con 0x7f4c8c0fcff0 2024-05-17T21:54:38.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:38.964+0000 7f4c93d25700 1 -- 172.21.15.107:0/2442465578 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c8c10c490 con 0x7f4c8c0fcff0 2024-05-17T21:54:38.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:38.964+0000 7f4c827fc700 1 -- 172.21.15.107:0/2442465578 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4c7c00f040 con 0x7f4c8c0fcff0 2024-05-17T21:54:38.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:38.964+0000 7f4c827fc700 1 -- 172.21.15.107:0/2442465578 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c7c013be0 con 0x7f4c8c0fcff0 2024-05-17T21:54:38.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:38.965+0000 7f4c93d25700 1 -- 172.21.15.107:0/2442465578 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c8c10c910 con 0x7f4c8c0fcff0 2024-05-17T21:54:38.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:38.966+0000 7f4c93d25700 1 -- 172.21.15.107:0/2442465578 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c8c05f5c0 con 0x7f4c8c0fcff0 2024-05-17T21:54:38.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:38.967+0000 7f4c827fc700 1 -- 172.21.15.107:0/2442465578 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 100184+0+0 (secure 0 0 0) 0x7f4c7c013d40 con 0x7f4c8c0fcff0 2024-05-17T21:54:38.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:38.967+0000 7f4c827fc700 1 --2- 172.21.15.107:0/2442465578 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f4c78078aa0 0x7f4c7807af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:54:38.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:38.967+0000 7f4c827fc700 1 -- 172.21.15.107:0/2442465578 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f4c7c09dfd0 con 0x7f4c8c0fcff0 2024-05-17T21:54:38.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:38.968+0000 7f4c91ac1700 1 --2- 172.21.15.107:0/2442465578 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f4c78078aa0 0x7f4c7807af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:54:38.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:38.968+0000 7f4c91ac1700 1 --2- 172.21.15.107:0/2442465578 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f4c78078aa0 0x7f4c7807af50 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f4c88004890 tx=0x7f4c88009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:54:38.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:38.973+0000 7f4c827fc700 1 -- 172.21.15.107:0/2442465578 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f4c7c0a3060 con 0x7f4c8c0fcff0 2024-05-17T21:54:39.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:39.232+0000 7f4c93d25700 1 -- 172.21.15.107:0/2442465578 --> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4c8c101290 con 0x7f4c78078aa0 2024-05-17T21:54:39.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:39.233+0000 7f4c827fc700 1 -- 172.21.15.107:0/2442465578 <== mgr.34100 v2:172.21.15.107:6800/3205268226 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7f4c8c101290 con 0x7f4c78078aa0 2024-05-17T21:54:39.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:39.236+0000 7f4c93d25700 1 -- 172.21.15.107:0/2442465578 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f4c78078aa0 msgr2=0x7f4c7807af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:54:39.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:39.236+0000 7f4c93d25700 1 --2- 172.21.15.107:0/2442465578 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f4c78078aa0 0x7f4c7807af50 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f4c88004890 tx=0x7f4c88009e30 comp rx=0 tx=0).stop 2024-05-17T21:54:39.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:39.236+0000 7f4c93d25700 1 -- 172.21.15.107:0/2442465578 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c8c0fcff0 msgr2=0x7f4c8c112c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:54:39.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:39.236+0000 7f4c93d25700 1 --2- 172.21.15.107:0/2442465578 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c8c0fcff0 0x7f4c8c112c00 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c004f20 tx=0x7f4c7c00b040 comp rx=0 tx=0).stop 2024-05-17T21:54:39.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:39.236+0000 7f4c93d25700 1 -- 172.21.15.107:0/2442465578 shutdown_connections 2024-05-17T21:54:39.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:39.236+0000 7f4c93d25700 1 --2- 172.21.15.107:0/2442465578 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f4c78078aa0 0x7f4c7807af50 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:54:39.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:39.236+0000 7f4c93d25700 1 --2- 172.21.15.107:0/2442465578 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c8c0fbd40 0x7f4c8c1106b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:54:39.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:39.236+0000 7f4c93d25700 1 --2- 172.21.15.107:0/2442465578 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c8c0fcff0 0x7f4c8c112c00 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:54:39.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:39.236+0000 7f4c93d25700 1 -- 172.21.15.107:0/2442465578 >> 172.21.15.107:0/2442465578 conn(0x7f4c8c0f74c0 msgr2=0x7f4c8c0f8b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:54:39.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:39.236+0000 7f4c93d25700 1 -- 172.21.15.107:0/2442465578 shutdown_connections 2024-05-17T21:54:39.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:39.236+0000 7f4c93d25700 1 -- 172.21.15.107:0/2442465578 wait complete. 2024-05-17T21:54:39.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:39.347+0000 7ff916a2c700 1 -- 172.21.15.107:0/19761429 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff9100fbcb0 msgr2=0x7ff9100fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:54:39.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:39.347+0000 7ff916a2c700 1 --2- 172.21.15.107:0/19761429 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff9100fbcb0 0x7ff9100fc0c0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff900004940 tx=0x7ff900009e30 comp rx=0 tx=0).stop 2024-05-17T21:54:39.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:39.347+0000 7ff916a2c700 1 -- 172.21.15.107:0/19761429 shutdown_connections 2024-05-17T21:54:39.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:39.347+0000 7ff916a2c700 1 --2- 172.21.15.107:0/19761429 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9100fcf60 0x7ff9100fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:54:39.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:39.347+0000 7ff916a2c700 1 --2- 172.21.15.107:0/19761429 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff9100fbcb0 0x7ff9100fc0c0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:54:39.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:39.347+0000 7ff916a2c700 1 -- 172.21.15.107:0/19761429 >> 172.21.15.107:0/19761429 conn(0x7ff9100f7430 msgr2=0x7ff9100f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:54:39.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:39.348+0000 7ff916a2c700 1 -- 172.21.15.107:0/19761429 shutdown_connections 2024-05-17T21:54:39.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:39.348+0000 7ff916a2c700 1 -- 172.21.15.107:0/19761429 wait complete. 2024-05-17T21:54:39.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:39.348+0000 7ff916a2c700 1 Processor -- start 2024-05-17T21:54:39.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:39.348+0000 7ff916a2c700 1 -- start start 2024-05-17T21:54:39.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:39.349+0000 7ff916a2c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff9100fbcb0 0x7ff91010d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:54:39.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:39.349+0000 7ff916a2c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9100fcf60 0x7ff91010ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:54:39.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:39.349+0000 7ff916a2c700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff91010e4d0 con 0x7ff9100fbcb0 2024-05-17T21:54:39.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:39.349+0000 7ff916a2c700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff91010ed20 con 0x7ff9100fcf60 2024-05-17T21:54:39.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:39.349+0000 7ff90ffff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff9100fbcb0 0x7ff91010d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:54:39.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:39.349+0000 7ff90ffff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff9100fbcb0 0x7ff91010d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48434/0 (socket says 172.21.15.107:48434) 2024-05-17T21:54:39.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:39.349+0000 7ff90ffff700 1 -- 172.21.15.107:0/2418348719 learned_addr learned my addr 172.21.15.107:0/2418348719 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:54:39.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:39.349+0000 7ff907fff700 1 --2- 172.21.15.107:0/2418348719 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9100fcf60 0x7ff91010ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:54:39.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:39.349+0000 7ff90ffff700 1 -- 172.21.15.107:0/2418348719 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9100fcf60 msgr2=0x7ff91010ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:54:39.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:39.349+0000 7ff90ffff700 1 --2- 172.21.15.107:0/2418348719 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9100fcf60 0x7ff91010ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:54:39.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:39.349+0000 7ff90ffff700 1 -- 172.21.15.107:0/2418348719 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff900005040 con 0x7ff9100fbcb0 2024-05-17T21:54:39.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:39.349+0000 7ff90ffff700 1 --2- 172.21.15.107:0/2418348719 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff9100fbcb0 0x7ff91010d990 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff900013870 tx=0x7ff900009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:54:39.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:39.349+0000 7ff90d7fa700 1 -- 172.21.15.107:0/2418348719 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff900021070 con 0x7ff9100fbcb0 2024-05-17T21:54:39.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:39.349+0000 7ff916a2c700 1 -- 172.21.15.107:0/2418348719 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff91010efa0 con 0x7ff9100fbcb0 2024-05-17T21:54:39.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:39.349+0000 7ff90d7fa700 1 -- 172.21.15.107:0/2418348719 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff900009430 con 0x7ff9100fbcb0 2024-05-17T21:54:39.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:39.349+0000 7ff90d7fa700 1 -- 172.21.15.107:0/2418348719 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff90000e070 con 0x7ff9100fbcb0 2024-05-17T21:54:39.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:39.350+0000 7ff916a2c700 1 -- 172.21.15.107:0/2418348719 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff91010f4e0 con 0x7ff9100fbcb0 2024-05-17T21:54:39.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:39.350+0000 7ff916a2c700 1 -- 172.21.15.107:0/2418348719 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff910101af0 con 0x7ff9100fbcb0 2024-05-17T21:54:39.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:39.351+0000 7ff90d7fa700 1 -- 172.21.15.107:0/2418348719 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 100184+0+0 (secure 0 0 0) 0x7ff90001c750 con 0x7ff9100fbcb0 2024-05-17T21:54:39.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:39.352+0000 7ff90d7fa700 1 --2- 172.21.15.107:0/2418348719 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7ff8fc078a30 0x7ff8fc07aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:54:39.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:39.352+0000 7ff90d7fa700 1 -- 172.21.15.107:0/2418348719 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6410+0+0 (secure 0 0 0) 0x7ff90009bb50 con 0x7ff9100fbcb0 2024-05-17T21:54:39.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:39.352+0000 7ff907fff700 1 --2- 172.21.15.107:0/2418348719 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7ff8fc078a30 0x7ff8fc07aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:54:39.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:39.352+0000 7ff907fff700 1 --2- 172.21.15.107:0/2418348719 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7ff8fc078a30 0x7ff8fc07aee0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff8f8009b80 tx=0x7ff8f800b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:54:39.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:39.357+0000 7ff90d7fa700 1 -- 172.21.15.107:0/2418348719 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7ff900063620 con 0x7ff9100fbcb0 2024-05-17T21:54:39.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:39.640+0000 7ff916a2c700 1 -- 172.21.15.107:0/2418348719 --> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7ff91010f750 con 0x7ff8fc078a30 2024-05-17T21:54:39.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:39.659+0000 7ff90d7fa700 1 -- 172.21.15.107:0/2418348719 <== mgr.34100 v2:172.21.15.107:6800/3205268226 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7ff91010f750 con 0x7ff8fc078a30 2024-05-17T21:54:39.652 INFO:teuthology.orchestra.run.smithi107.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-17T21:54:39.652 INFO:teuthology.orchestra.run.smithi107.stdout:alertmanager.smithi107 smithi107 *:9093,9094 running (2m) 29s ago 10m 17.2M - 0.25.0 c8568f914cd2 bac2a944ae42 2024-05-17T21:54:39.652 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi107 smithi107 running (11m) 29s ago 11m 6660k - 17.2.7-952-g9903f776 d15a36853796 828741c3a89c 2024-05-17T21:54:39.652 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi138 smithi138 running (9m) 26s ago 9m 6660k - 17.2.7-952-g9903f776 d15a36853796 20e9ccf1fadd 2024-05-17T21:54:39.652 INFO:teuthology.orchestra.run.smithi107.stdout:grafana.smithi107 smithi107 *:3000 running (2m) 29s ago 10m 79.6M - 9.4.7 954c08fa6188 8c747e777bc5 2024-05-17T21:54:39.652 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.drqody smithi107 running (5m) 29s ago 5m 253M - 17.2.7-952-g9903f776 d15a36853796 0d851bc912f0 2024-05-17T21:54:39.652 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.idmbgj smithi107 running (5m) 29s ago 5m 12.6M - 17.2.7-952-g9903f776 d15a36853796 1348b09ab29e 2024-05-17T21:54:39.652 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi138.gwbtyd smithi138 running (5m) 26s ago 5m 245M - 17.2.7-952-g9903f776 d15a36853796 fd2ba75354c0 2024-05-17T21:54:39.652 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi138.zckrvs smithi138 running (5m) 26s ago 5m 13.4M - 17.2.7-952-g9903f776 d15a36853796 a1cfd2fe4cda 2024-05-17T21:54:39.652 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi107.xhhyiw smithi107 *:8443,9283,8765 running (91s) 29s ago 12m 575M - 19.0.0-3794-g931809a9 8880f7613d02 5acf3e511342 2024-05-17T21:54:39.652 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi138.napnll smithi138 *:8443,9283,8765 running (36s) 26s ago 9m 129M - 19.0.0-3794-g931809a9 8880f7613d02 d627761dbd0d 2024-05-17T21:54:39.653 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi107 smithi107 running (12m) 29s ago 12m 57.1M 2048M 17.2.7-952-g9903f776 d15a36853796 7de9c23eee97 2024-05-17T21:54:39.653 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi138 smithi138 running (9m) 26s ago 9m 42.7M 2048M 17.2.7-952-g9903f776 d15a36853796 ee61a065778d 2024-05-17T21:54:39.653 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi107 smithi107 *:9100 running (2m) 29s ago 10m 8140k - 1.5.0 0da6a335fe13 49bec71f5487 2024-05-17T21:54:39.653 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi138 smithi138 *:9100 running (108s) 26s ago 9m 13.8M - 1.5.0 0da6a335fe13 e52824376658 2024-05-17T21:54:39.653 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0 smithi107 running (9m) 29s ago 9m 177M 1557M 17.2.7-952-g9903f776 d15a36853796 f2d20009361d 2024-05-17T21:54:39.653 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1 smithi107 running (8m) 29s ago 8m 194M 1557M 17.2.7-952-g9903f776 d15a36853796 e5d3e99844a9 2024-05-17T21:54:39.653 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2 smithi107 running (8m) 29s ago 8m 155M 1557M 17.2.7-952-g9903f776 d15a36853796 e044d7b78088 2024-05-17T21:54:39.653 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3 smithi138 running (7m) 26s ago 7m 205M 2581M 17.2.7-952-g9903f776 d15a36853796 8c980cc8af87 2024-05-17T21:54:39.653 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4 smithi138 running (7m) 26s ago 7m 182M 2581M 17.2.7-952-g9903f776 d15a36853796 b306a36a744e 2024-05-17T21:54:39.653 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5 smithi138 running (6m) 26s ago 6m 168M 2581M 17.2.7-952-g9903f776 d15a36853796 7c8d8ed74974 2024-05-17T21:54:39.653 INFO:teuthology.orchestra.run.smithi107.stdout:prometheus.smithi107 smithi107 *:9095 running (55s) 29s ago 10m 44.7M - 2.43.0 a07b618ecd1d bf5f609bbd6e 2024-05-17T21:54:39.655 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:39.663+0000 7ff916a2c700 1 -- 172.21.15.107:0/2418348719 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7ff8fc078a30 msgr2=0x7ff8fc07aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:54:39.655 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:39.663+0000 7ff916a2c700 1 --2- 172.21.15.107:0/2418348719 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7ff8fc078a30 0x7ff8fc07aee0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff8f8009b80 tx=0x7ff8f800b040 comp rx=0 tx=0).stop 2024-05-17T21:54:39.655 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:39.663+0000 7ff916a2c700 1 -- 172.21.15.107:0/2418348719 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff9100fbcb0 msgr2=0x7ff91010d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:54:39.655 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:39.663+0000 7ff916a2c700 1 --2- 172.21.15.107:0/2418348719 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff9100fbcb0 0x7ff91010d990 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff900013870 tx=0x7ff900009e30 comp rx=0 tx=0).stop 2024-05-17T21:54:39.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:39.663+0000 7ff916a2c700 1 -- 172.21.15.107:0/2418348719 shutdown_connections 2024-05-17T21:54:39.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:39.663+0000 7ff916a2c700 1 --2- 172.21.15.107:0/2418348719 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7ff8fc078a30 0x7ff8fc07aee0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:54:39.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:39.663+0000 7ff916a2c700 1 --2- 172.21.15.107:0/2418348719 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff9100fbcb0 0x7ff91010d990 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:54:39.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:39.663+0000 7ff916a2c700 1 --2- 172.21.15.107:0/2418348719 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9100fcf60 0x7ff91010ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:54:39.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:39.663+0000 7ff916a2c700 1 -- 172.21.15.107:0/2418348719 >> 172.21.15.107:0/2418348719 conn(0x7ff9100f7430 msgr2=0x7ff9100f8ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:54:39.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:39.663+0000 7ff916a2c700 1 -- 172.21.15.107:0/2418348719 shutdown_connections 2024-05-17T21:54:39.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:39.663+0000 7ff916a2c700 1 -- 172.21.15.107:0/2418348719 wait complete. 2024-05-17T21:54:39.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:39.787+0000 7fcdbce8d700 1 -- 172.21.15.107:0/2845727738 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcdb8066e90 msgr2=0x7fcdb80ffa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:54:39.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:39.787+0000 7fcdbce8d700 1 --2- 172.21.15.107:0/2845727738 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcdb8066e90 0x7fcdb80ffa10 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fcda8004940 tx=0x7fcda8009e30 comp rx=0 tx=0).stop 2024-05-17T21:54:39.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:39.787+0000 7fcdbce8d700 1 -- 172.21.15.107:0/2845727738 shutdown_connections 2024-05-17T21:54:39.780 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:39 smithi107 ceph-mon[93224]: pgmap v5: 65 pgs: 65 active+clean; 281 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-05-17T21:54:39.780 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:39 smithi107 ceph-mon[93224]: mgrmap e37: smithi107.xhhyiw(active, since 4s), standbys: smithi138.napnll 2024-05-17T21:54:39.781 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:39 smithi107 ceph-mon[93224]: from='client.24597 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:54:39.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:39.788+0000 7fcdbce8d700 1 --2- 172.21.15.107:0/2845727738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcdb80fff50 0x7fcdb8102330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:54:39.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:39.788+0000 7fcdbce8d700 1 --2- 172.21.15.107:0/2845727738 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcdb8066e90 0x7fcdb80ffa10 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:54:39.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:39.788+0000 7fcdbce8d700 1 -- 172.21.15.107:0/2845727738 >> 172.21.15.107:0/2845727738 conn(0x7fcdb80f7420 msgr2=0x7fcdb80f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:54:39.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:39.788+0000 7fcdbce8d700 1 -- 172.21.15.107:0/2845727738 shutdown_connections 2024-05-17T21:54:39.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:39.788+0000 7fcdbce8d700 1 -- 172.21.15.107:0/2845727738 wait complete. 2024-05-17T21:54:39.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:39.788+0000 7fcdbce8d700 1 Processor -- start 2024-05-17T21:54:39.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:39.788+0000 7fcdbce8d700 1 -- start start 2024-05-17T21:54:39.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:39.789+0000 7fcdbce8d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcdb8066e90 0x7fcdb80fd210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:54:39.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:39.789+0000 7fcdbce8d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcdb80fff50 0x7fcdb80fb910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:54:39.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:39.789+0000 7fcdbce8d700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb80fd810 con 0x7fcdb80fff50 2024-05-17T21:54:39.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:39.789+0000 7fcdb659c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcdb8066e90 0x7fcdb80fd210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:54:39.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:39.789+0000 7fcdbce8d700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb80fbe50 con 0x7fcdb8066e90 2024-05-17T21:54:39.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:39.789+0000 7fcdb659c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcdb8066e90 0x7fcdb80fd210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.107:44158/0 (socket says 172.21.15.107:44158) 2024-05-17T21:54:39.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:39.789+0000 7fcdb659c700 1 -- 172.21.15.107:0/3950559156 learned_addr learned my addr 172.21.15.107:0/3950559156 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:54:39.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:39.789+0000 7fcdb5d9b700 1 --2- 172.21.15.107:0/3950559156 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcdb80fff50 0x7fcdb80fb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:54:39.784 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:39.789+0000 7fcdb5d9b700 1 -- 172.21.15.107:0/3950559156 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcdb8066e90 msgr2=0x7fcdb80fd210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:54:39.784 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:39.789+0000 7fcdb5d9b700 1 --2- 172.21.15.107:0/3950559156 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcdb8066e90 0x7fcdb80fd210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:54:39.784 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:39.789+0000 7fcdb5d9b700 1 -- 172.21.15.107:0/3950559156 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcda8005040 con 0x7fcdb80fff50 2024-05-17T21:54:39.784 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:39.789+0000 7fcdb659c700 1 --2- 172.21.15.107:0/3950559156 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcdb8066e90 0x7fcdb80fd210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T21:54:39.784 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:39.789+0000 7fcdb5d9b700 1 --2- 172.21.15.107:0/3950559156 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcdb80fff50 0x7fcdb80fb910 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fcdac009fd0 tx=0x7fcdac00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:54:39.784 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:39.790+0000 7fcda6ffd700 1 -- 172.21.15.107:0/3950559156 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcdac004e90 con 0x7fcdb80fff50 2024-05-17T21:54:39.784 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:39.790+0000 7fcda6ffd700 1 -- 172.21.15.107:0/3950559156 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcdac00f040 con 0x7fcdb80fff50 2024-05-17T21:54:39.784 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:39.790+0000 7fcdbce8d700 1 -- 172.21.15.107:0/3950559156 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdb80fc0d0 con 0x7fcdb80fff50 2024-05-17T21:54:39.784 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:39.790+0000 7fcda6ffd700 1 -- 172.21.15.107:0/3950559156 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcdac00ca50 con 0x7fcdb80fff50 2024-05-17T21:54:39.784 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:39.790+0000 7fcdbce8d700 1 -- 172.21.15.107:0/3950559156 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdb80fc610 con 0x7fcdb80fff50 2024-05-17T21:54:39.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:39.791+0000 7fcdbce8d700 1 -- 172.21.15.107:0/3950559156 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdb804c710 con 0x7fcdb80fff50 2024-05-17T21:54:39.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:39.791+0000 7fcda6ffd700 1 -- 172.21.15.107:0/3950559156 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 100184+0+0 (secure 0 0 0) 0x7fcdac00cbb0 con 0x7fcdb80fff50 2024-05-17T21:54:39.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:39.792+0000 7fcda6ffd700 1 --2- 172.21.15.107:0/3950559156 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fcda0078aa0 0x7fcda007af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:54:39.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:39.792+0000 7fcda6ffd700 1 -- 172.21.15.107:0/3950559156 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fcdac09a300 con 0x7fcdb80fff50 2024-05-17T21:54:39.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:39.792+0000 7fcdb659c700 1 --2- 172.21.15.107:0/3950559156 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fcda0078aa0 0x7fcda007af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:54:39.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:39.792+0000 7fcdb659c700 1 --2- 172.21.15.107:0/3950559156 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fcda0078aa0 0x7fcda007af50 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fcda8009fd0 tx=0x7fcda8009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:54:39.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:39.797+0000 7fcda6ffd700 1 -- 172.21.15.107:0/3950559156 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7fcdac061e80 con 0x7fcdb80fff50 2024-05-17T21:54:39.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:39 smithi138 ceph-mon[35948]: pgmap v5: 65 pgs: 65 active+clean; 281 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-05-17T21:54:39.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:39 smithi138 ceph-mon[35948]: mgrmap e37: smithi107.xhhyiw(active, since 4s), standbys: smithi138.napnll 2024-05-17T21:54:39.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:39 smithi138 ceph-mon[35948]: from='client.24597 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:54:40.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.120+0000 7fcdbce8d700 1 -- 172.21.15.107:0/3950559156 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fcdb8115c80 con 0x7fcdb80fff50 2024-05-17T21:54:40.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.121+0000 7fcda6ffd700 1 -- 172.21.15.107:0/3950559156 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+843 (secure 0 0 0) 0x7fcdac00ce60 con 0x7fcdb80fff50 2024-05-17T21:54:40.113 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-05-17T21:54:40.113 INFO:teuthology.orchestra.run.smithi107.stdout: "mon": { 2024-05-17T21:54:40.113 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-952-g9903f776 (9903f7767386a9d9efadb7c1effdcd3064d71804) quincy (stable)": 1, 2024-05-17T21:54:40.113 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev)": 1 2024-05-17T21:54:40.113 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-05-17T21:54:40.113 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr": { 2024-05-17T21:54:40.113 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev)": 2 2024-05-17T21:54:40.114 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-05-17T21:54:40.114 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": { 2024-05-17T21:54:40.114 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-952-g9903f776 (9903f7767386a9d9efadb7c1effdcd3064d71804) quincy (stable)": 6 2024-05-17T21:54:40.114 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-05-17T21:54:40.114 INFO:teuthology.orchestra.run.smithi107.stdout: "mds": { 2024-05-17T21:54:40.114 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-952-g9903f776 (9903f7767386a9d9efadb7c1effdcd3064d71804) quincy (stable)": 4 2024-05-17T21:54:40.114 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-05-17T21:54:40.114 INFO:teuthology.orchestra.run.smithi107.stdout: "overall": { 2024-05-17T21:54:40.114 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-952-g9903f776 (9903f7767386a9d9efadb7c1effdcd3064d71804) quincy (stable)": 11, 2024-05-17T21:54:40.114 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev)": 3 2024-05-17T21:54:40.114 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-05-17T21:54:40.114 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-05-17T21:54:40.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.123+0000 7fcdbce8d700 1 -- 172.21.15.107:0/3950559156 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fcda0078aa0 msgr2=0x7fcda007af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:54:40.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.123+0000 7fcdbce8d700 1 --2- 172.21.15.107:0/3950559156 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fcda0078aa0 0x7fcda007af50 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fcda8009fd0 tx=0x7fcda8009e30 comp rx=0 tx=0).stop 2024-05-17T21:54:40.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.123+0000 7fcdbce8d700 1 -- 172.21.15.107:0/3950559156 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcdb80fff50 msgr2=0x7fcdb80fb910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:54:40.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.123+0000 7fcdbce8d700 1 --2- 172.21.15.107:0/3950559156 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcdb80fff50 0x7fcdb80fb910 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fcdac009fd0 tx=0x7fcdac00b040 comp rx=0 tx=0).stop 2024-05-17T21:54:40.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.124+0000 7fcdbce8d700 1 -- 172.21.15.107:0/3950559156 shutdown_connections 2024-05-17T21:54:40.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.124+0000 7fcdbce8d700 1 --2- 172.21.15.107:0/3950559156 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fcda0078aa0 0x7fcda007af50 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:54:40.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.124+0000 7fcdbce8d700 1 --2- 172.21.15.107:0/3950559156 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcdb8066e90 0x7fcdb80fd210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:54:40.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.124+0000 7fcdbce8d700 1 --2- 172.21.15.107:0/3950559156 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcdb80fff50 0x7fcdb80fb910 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:54:40.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.124+0000 7fcdbce8d700 1 -- 172.21.15.107:0/3950559156 >> 172.21.15.107:0/3950559156 conn(0x7fcdb80f7420 msgr2=0x7fcdb80f7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:54:40.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.124+0000 7fcdbce8d700 1 -- 172.21.15.107:0/3950559156 shutdown_connections 2024-05-17T21:54:40.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.124+0000 7fcdbce8d700 1 -- 172.21.15.107:0/3950559156 wait complete. 2024-05-17T21:54:40.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.236+0000 7f904e113700 1 -- 172.21.15.107:0/4249205421 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f90480fbcb0 msgr2=0x7f90480fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:54:40.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.236+0000 7f904e113700 1 --2- 172.21.15.107:0/4249205421 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f90480fbcb0 0x7f90480fc0c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9038004940 tx=0x7f9038009e30 comp rx=0 tx=0).stop 2024-05-17T21:54:40.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.236+0000 7f904e113700 1 -- 172.21.15.107:0/4249205421 shutdown_connections 2024-05-17T21:54:40.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.236+0000 7f904e113700 1 --2- 172.21.15.107:0/4249205421 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90480fcf60 0x7f90480fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:54:40.230 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.237+0000 7f904e113700 1 --2- 172.21.15.107:0/4249205421 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f90480fbcb0 0x7f90480fc0c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:54:40.230 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.237+0000 7f904e113700 1 -- 172.21.15.107:0/4249205421 >> 172.21.15.107:0/4249205421 conn(0x7f90480f7430 msgr2=0x7f90480f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:54:40.230 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.237+0000 7f904e113700 1 -- 172.21.15.107:0/4249205421 shutdown_connections 2024-05-17T21:54:40.230 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.237+0000 7f904e113700 1 -- 172.21.15.107:0/4249205421 wait complete. 2024-05-17T21:54:40.230 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.237+0000 7f904e113700 1 Processor -- start 2024-05-17T21:54:40.230 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.237+0000 7f904e113700 1 -- start start 2024-05-17T21:54:40.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.237+0000 7f904e113700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f90480fbcb0 0x7f90480ffd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:54:40.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.237+0000 7f904e113700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90480fcf60 0x7f90480fe2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:54:40.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.237+0000 7f904e113700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90480fe7e0 con 0x7f90480fbcb0 2024-05-17T21:54:40.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.237+0000 7f904e113700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90480fe920 con 0x7f90480fcf60 2024-05-17T21:54:40.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.238+0000 7f90477fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f90480fbcb0 0x7f90480ffd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:54:40.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.238+0000 7f90477fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f90480fbcb0 0x7f90480ffd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48466/0 (socket says 172.21.15.107:48466) 2024-05-17T21:54:40.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.238+0000 7f90477fe700 1 -- 172.21.15.107:0/3746675533 learned_addr learned my addr 172.21.15.107:0/3746675533 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:54:40.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.238+0000 7f9046ffd700 1 --2- 172.21.15.107:0/3746675533 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90480fcf60 0x7f90480fe2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:54:40.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.238+0000 7f90477fe700 1 -- 172.21.15.107:0/3746675533 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90480fcf60 msgr2=0x7f90480fe2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:54:40.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.238+0000 7f90477fe700 1 --2- 172.21.15.107:0/3746675533 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90480fcf60 0x7f90480fe2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:54:40.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.238+0000 7f90477fe700 1 -- 172.21.15.107:0/3746675533 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9038005040 con 0x7f90480fbcb0 2024-05-17T21:54:40.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.238+0000 7f90477fe700 1 --2- 172.21.15.107:0/3746675533 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f90480fbcb0 0x7f90480ffd30 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f9038013870 tx=0x7f9038009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:54:40.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.238+0000 7f902ffff700 1 -- 172.21.15.107:0/3746675533 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9038021070 con 0x7f90480fbcb0 2024-05-17T21:54:40.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.238+0000 7f904e113700 1 -- 172.21.15.107:0/3746675533 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90480feba0 con 0x7f90480fbcb0 2024-05-17T21:54:40.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.238+0000 7f902ffff700 1 -- 172.21.15.107:0/3746675533 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9038009430 con 0x7f90480fbcb0 2024-05-17T21:54:40.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.238+0000 7f902ffff700 1 -- 172.21.15.107:0/3746675533 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f903800e070 con 0x7f90480fbcb0 2024-05-17T21:54:40.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.238+0000 7f904e113700 1 -- 172.21.15.107:0/3746675533 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90480ff0e0 con 0x7f90480fbcb0 2024-05-17T21:54:40.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.239+0000 7f904e113700 1 -- 172.21.15.107:0/3746675533 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9048101af0 con 0x7f90480fbcb0 2024-05-17T21:54:40.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.240+0000 7f902ffff700 1 -- 172.21.15.107:0/3746675533 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 100184+0+0 (secure 0 0 0) 0x7f903801c750 con 0x7f90480fbcb0 2024-05-17T21:54:40.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.240+0000 7f902ffff700 1 --2- 172.21.15.107:0/3746675533 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f9030078a30 0x7f903007aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:54:40.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.240+0000 7f902ffff700 1 -- 172.21.15.107:0/3746675533 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f903809bb30 con 0x7f90480fbcb0 2024-05-17T21:54:40.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.241+0000 7f9046ffd700 1 --2- 172.21.15.107:0/3746675533 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f9030078a30 0x7f903007aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:54:40.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.241+0000 7f9046ffd700 1 --2- 172.21.15.107:0/3746675533 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f9030078a30 0x7f903007aee0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f903c009b80 tx=0x7f903c00b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:54:40.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.245+0000 7f902ffff700 1 -- 172.21.15.107:0/3746675533 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f9038063600 con 0x7f90480fbcb0 2024-05-17T21:54:40.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.522+0000 7f904e113700 1 -- 172.21.15.107:0/3746675533 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f904804bed0 con 0x7f90480fbcb0 2024-05-17T21:54:40.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.522+0000 7f902ffff700 1 -- 172.21.15.107:0/3746675533 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1910 (secure 0 0 0) 0x7f9038062fa0 con 0x7f90480fbcb0 2024-05-17T21:54:40.515 INFO:teuthology.orchestra.run.smithi107.stdout:e15 2024-05-17T21:54:40.515 INFO:teuthology.orchestra.run.smithi107.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-17T21:54:40.515 INFO:teuthology.orchestra.run.smithi107.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-17T21:54:40.515 INFO:teuthology.orchestra.run.smithi107.stdout:legacy client fscid: 1 2024-05-17T21:54:40.515 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:54:40.515 INFO:teuthology.orchestra.run.smithi107.stdout:Filesystem 'cephfs' (1) 2024-05-17T21:54:40.515 INFO:teuthology.orchestra.run.smithi107.stdout:fs_name cephfs 2024-05-17T21:54:40.515 INFO:teuthology.orchestra.run.smithi107.stdout:epoch 15 2024-05-17T21:54:40.515 INFO:teuthology.orchestra.run.smithi107.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-17T21:54:40.515 INFO:teuthology.orchestra.run.smithi107.stdout:created 2024-05-17T21:49:16.836333+0000 2024-05-17T21:54:40.515 INFO:teuthology.orchestra.run.smithi107.stdout:modified 2024-05-17T21:49:34.873628+0000 2024-05-17T21:54:40.516 INFO:teuthology.orchestra.run.smithi107.stdout:tableserver 0 2024-05-17T21:54:40.516 INFO:teuthology.orchestra.run.smithi107.stdout:root 0 2024-05-17T21:54:40.516 INFO:teuthology.orchestra.run.smithi107.stdout:session_timeout 60 2024-05-17T21:54:40.516 INFO:teuthology.orchestra.run.smithi107.stdout:session_autoclose 300 2024-05-17T21:54:40.516 INFO:teuthology.orchestra.run.smithi107.stdout:max_file_size 1099511627776 2024-05-17T21:54:40.516 INFO:teuthology.orchestra.run.smithi107.stdout:max_xattr_size 65536 2024-05-17T21:54:40.516 INFO:teuthology.orchestra.run.smithi107.stdout:required_client_features {} 2024-05-17T21:54:40.516 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure 0 2024-05-17T21:54:40.516 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure_osd_epoch 46 2024-05-17T21:54:40.516 INFO:teuthology.orchestra.run.smithi107.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-17T21:54:40.516 INFO:teuthology.orchestra.run.smithi107.stdout:max_mds 1 2024-05-17T21:54:40.516 INFO:teuthology.orchestra.run.smithi107.stdout:in 0 2024-05-17T21:54:40.517 INFO:teuthology.orchestra.run.smithi107.stdout:up {0=24259} 2024-05-17T21:54:40.517 INFO:teuthology.orchestra.run.smithi107.stdout:failed 2024-05-17T21:54:40.517 INFO:teuthology.orchestra.run.smithi107.stdout:damaged 2024-05-17T21:54:40.517 INFO:teuthology.orchestra.run.smithi107.stdout:stopped 2024-05-17T21:54:40.517 INFO:teuthology.orchestra.run.smithi107.stdout:data_pools [3] 2024-05-17T21:54:40.517 INFO:teuthology.orchestra.run.smithi107.stdout:metadata_pool 2 2024-05-17T21:54:40.517 INFO:teuthology.orchestra.run.smithi107.stdout:inline_data enabled 2024-05-17T21:54:40.517 INFO:teuthology.orchestra.run.smithi107.stdout:balancer 2024-05-17T21:54:40.517 INFO:teuthology.orchestra.run.smithi107.stdout:bal_rank_mask -1 2024-05-17T21:54:40.517 INFO:teuthology.orchestra.run.smithi107.stdout:standby_count_wanted 1 2024-05-17T21:54:40.517 INFO:teuthology.orchestra.run.smithi107.stdout:qdb_cluster leader: 0 members: 2024-05-17T21:54:40.517 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi138.gwbtyd{0:24259} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.138:6824/2779634919,v1:172.21.15.138:6825/2779634919] compat {c=[1],r=[1],i=[7ff]}] 2024-05-17T21:54:40.517 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.drqody{0:14486} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.107:6828/2275807100,v1:172.21.15.107:6829/2275807100] compat {c=[1],r=[1],i=[7ff]}] 2024-05-17T21:54:40.517 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:54:40.518 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:54:40.518 INFO:teuthology.orchestra.run.smithi107.stdout:Standby daemons: 2024-05-17T21:54:40.518 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:54:40.518 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.idmbgj{-1:14498} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.107:6826/1460186079,v1:172.21.15.107:6827/1460186079] compat {c=[1],r=[1],i=[7ff]}] 2024-05-17T21:54:40.518 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi138.zckrvs{-1:24267} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.138:6826/170628661,v1:172.21.15.138:6827/170628661] compat {c=[1],r=[1],i=[7ff]}] 2024-05-17T21:54:40.519 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.525+0000 7f904e113700 1 -- 172.21.15.107:0/3746675533 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f9030078a30 msgr2=0x7f903007aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:54:40.519 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.525+0000 7f904e113700 1 --2- 172.21.15.107:0/3746675533 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f9030078a30 0x7f903007aee0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f903c009b80 tx=0x7f903c00b040 comp rx=0 tx=0).stop 2024-05-17T21:54:40.519 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.525+0000 7f904e113700 1 -- 172.21.15.107:0/3746675533 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f90480fbcb0 msgr2=0x7f90480ffd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:54:40.519 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.525+0000 7f904e113700 1 --2- 172.21.15.107:0/3746675533 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f90480fbcb0 0x7f90480ffd30 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f9038013870 tx=0x7f9038009e30 comp rx=0 tx=0).stop 2024-05-17T21:54:40.519 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.525+0000 7f904e113700 1 -- 172.21.15.107:0/3746675533 shutdown_connections 2024-05-17T21:54:40.519 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.525+0000 7f904e113700 1 --2- 172.21.15.107:0/3746675533 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f9030078a30 0x7f903007aee0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:54:40.519 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.525+0000 7f904e113700 1 --2- 172.21.15.107:0/3746675533 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f90480fbcb0 0x7f90480ffd30 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:54:40.519 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.525+0000 7f904e113700 1 --2- 172.21.15.107:0/3746675533 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90480fcf60 0x7f90480fe2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:54:40.519 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.525+0000 7f904e113700 1 -- 172.21.15.107:0/3746675533 >> 172.21.15.107:0/3746675533 conn(0x7f90480f7430 msgr2=0x7f9048065b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:54:40.519 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.525+0000 7f904e113700 1 -- 172.21.15.107:0/3746675533 shutdown_connections 2024-05-17T21:54:40.519 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.525+0000 7f904e113700 1 -- 172.21.15.107:0/3746675533 wait complete. 2024-05-17T21:54:40.520 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 15 2024-05-17T21:54:40.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.679+0000 7facf7454700 1 -- 172.21.15.107:0/3505887207 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7facf0066e50 msgr2=0x7facf00ffa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:54:40.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.679+0000 7facf7454700 1 --2- 172.21.15.107:0/3505887207 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7facf0066e50 0x7facf00ffa20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7facec004940 tx=0x7facec009e30 comp rx=0 tx=0).stop 2024-05-17T21:54:40.672 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:40 smithi107 ceph-mon[93224]: from='client.34136 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:54:40.672 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:40 smithi107 ceph-mon[93224]: from='client.34140 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:54:40.672 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:40 smithi107 ceph-mon[93224]: from='client.? 172.21.15.107:0/3950559156' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:54:40.672 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:40 smithi107 ceph-mon[93224]: from='client.? 172.21.15.107:0/3746675533' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-17T21:54:40.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.679+0000 7facf7454700 1 -- 172.21.15.107:0/3505887207 shutdown_connections 2024-05-17T21:54:40.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.679+0000 7facf7454700 1 --2- 172.21.15.107:0/3505887207 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7facf00fff60 0x7facf0102340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:54:40.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.679+0000 7facf7454700 1 --2- 172.21.15.107:0/3505887207 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7facf0066e50 0x7facf00ffa20 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:54:40.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.679+0000 7facf7454700 1 -- 172.21.15.107:0/3505887207 >> 172.21.15.107:0/3505887207 conn(0x7facf00f7430 msgr2=0x7facf00f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:54:40.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.679+0000 7facf7454700 1 -- 172.21.15.107:0/3505887207 shutdown_connections 2024-05-17T21:54:40.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.679+0000 7facf7454700 1 -- 172.21.15.107:0/3505887207 wait complete. 2024-05-17T21:54:40.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.680+0000 7facf7454700 1 Processor -- start 2024-05-17T21:54:40.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.680+0000 7facf7454700 1 -- start start 2024-05-17T21:54:40.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.680+0000 7facf7454700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7facf0066e50 0x7facf00fd220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:54:40.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.680+0000 7facf7454700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7facf00fff60 0x7facf00fb920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:54:40.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.680+0000 7facf7454700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facf00fd820 con 0x7facf00fff60 2024-05-17T21:54:40.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.680+0000 7facf7454700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facf00fbe60 con 0x7facf0066e50 2024-05-17T21:54:40.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.680+0000 7facf49ef700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7facf00fff60 0x7facf00fb920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:54:40.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.680+0000 7facf51f0700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7facf0066e50 0x7facf00fd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:54:40.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.680+0000 7facf51f0700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7facf0066e50 0x7facf00fd220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.107:44186/0 (socket says 172.21.15.107:44186) 2024-05-17T21:54:40.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.680+0000 7facf49ef700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7facf00fff60 0x7facf00fb920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48480/0 (socket says 172.21.15.107:48480) 2024-05-17T21:54:40.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.681+0000 7facf51f0700 1 -- 172.21.15.107:0/2907164901 learned_addr learned my addr 172.21.15.107:0/2907164901 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:54:40.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.681+0000 7facf51f0700 1 -- 172.21.15.107:0/2907164901 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7facf00fff60 msgr2=0x7facf00fb920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:54:40.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.681+0000 7facf51f0700 1 --2- 172.21.15.107:0/2907164901 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7facf00fff60 0x7facf00fb920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:54:40.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.681+0000 7facf51f0700 1 -- 172.21.15.107:0/2907164901 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facec005040 con 0x7facf0066e50 2024-05-17T21:54:40.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.681+0000 7facf49ef700 1 --2- 172.21.15.107:0/2907164901 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7facf00fff60 0x7facf00fb920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T21:54:40.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.681+0000 7facf51f0700 1 --2- 172.21.15.107:0/2907164901 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7facf0066e50 0x7facf00fd220 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7facec004090 tx=0x7facec009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:54:40.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.681+0000 7face5ffb700 1 -- 172.21.15.107:0/2907164901 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7facec021070 con 0x7facf0066e50 2024-05-17T21:54:40.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.681+0000 7face5ffb700 1 -- 172.21.15.107:0/2907164901 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7facec01c400 con 0x7facf0066e50 2024-05-17T21:54:40.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.682+0000 7facf7454700 1 -- 172.21.15.107:0/2907164901 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facf00fc0e0 con 0x7facf0066e50 2024-05-17T21:54:40.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.682+0000 7facf7454700 1 -- 172.21.15.107:0/2907164901 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facf00fc5c0 con 0x7facf0066e50 2024-05-17T21:54:40.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.682+0000 7face5ffb700 1 -- 172.21.15.107:0/2907164901 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7facec00e070 con 0x7facf0066e50 2024-05-17T21:54:40.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.683+0000 7facf7454700 1 -- 172.21.15.107:0/2907164901 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facf004c710 con 0x7facf0066e50 2024-05-17T21:54:40.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.684+0000 7face5ffb700 1 -- 172.21.15.107:0/2907164901 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 37) v1 ==== 100184+0+0 (secure 0 0 0) 0x7facec01ca50 con 0x7facf0066e50 2024-05-17T21:54:40.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.685+0000 7face5ffb700 1 --2- 172.21.15.107:0/2907164901 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7facdc078aa0 0x7facdc07af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:54:40.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.685+0000 7face5ffb700 1 -- 172.21.15.107:0/2907164901 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6410+0+0 (secure 0 0 0) 0x7facec09bb10 con 0x7facf0066e50 2024-05-17T21:54:40.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.685+0000 7facf49ef700 1 --2- 172.21.15.107:0/2907164901 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7facdc078aa0 0x7facdc07af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:54:40.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.686+0000 7facf49ef700 1 --2- 172.21.15.107:0/2907164901 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7facdc078aa0 0x7facdc07af50 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7face0009b80 tx=0x7face000b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:54:40.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.691+0000 7face5ffb700 1 -- 172.21.15.107:0/2907164901 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7facec0635c0 con 0x7facf0066e50 2024-05-17T21:54:40.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.950+0000 7facf7454700 1 -- 172.21.15.107:0/2907164901 --> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7facf00f8f20 con 0x7facdc078aa0 2024-05-17T21:54:40.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.951+0000 7face5ffb700 1 -- 172.21.15.107:0/2907164901 <== mgr.34100 v2:172.21.15.107:6800/3205268226 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7facf00f8f20 con 0x7facdc078aa0 2024-05-17T21:54:40.943 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-05-17T21:54:40.943 INFO:teuthology.orchestra.run.smithi107.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b", 2024-05-17T21:54:40.944 INFO:teuthology.orchestra.run.smithi107.stdout: "in_progress": true, 2024-05-17T21:54:40.944 INFO:teuthology.orchestra.run.smithi107.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-17T21:54:40.944 INFO:teuthology.orchestra.run.smithi107.stdout: "services_complete": [ 2024-05-17T21:54:40.944 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr" 2024-05-17T21:54:40.944 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-05-17T21:54:40.944 INFO:teuthology.orchestra.run.smithi107.stdout: "progress": "2/21 daemons upgraded", 2024-05-17T21:54:40.944 INFO:teuthology.orchestra.run.smithi107.stdout: "message": "", 2024-05-17T21:54:40.944 INFO:teuthology.orchestra.run.smithi107.stdout: "is_paused": false 2024-05-17T21:54:40.944 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-05-17T21:54:40.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.953+0000 7facf7454700 1 -- 172.21.15.107:0/2907164901 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7facdc078aa0 msgr2=0x7facdc07af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:54:40.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.953+0000 7facf7454700 1 --2- 172.21.15.107:0/2907164901 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7facdc078aa0 0x7facdc07af50 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7face0009b80 tx=0x7face000b040 comp rx=0 tx=0).stop 2024-05-17T21:54:40.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.953+0000 7facf7454700 1 -- 172.21.15.107:0/2907164901 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7facf0066e50 msgr2=0x7facf00fd220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:54:40.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.953+0000 7facf7454700 1 --2- 172.21.15.107:0/2907164901 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7facf0066e50 0x7facf00fd220 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7facec004090 tx=0x7facec009e30 comp rx=0 tx=0).stop 2024-05-17T21:54:40.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.953+0000 7facf7454700 1 -- 172.21.15.107:0/2907164901 shutdown_connections 2024-05-17T21:54:40.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.953+0000 7facf7454700 1 --2- 172.21.15.107:0/2907164901 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7facdc078aa0 0x7facdc07af50 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:54:40.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.953+0000 7facf7454700 1 --2- 172.21.15.107:0/2907164901 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7facf0066e50 0x7facf00fd220 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:54:40.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.953+0000 7facf7454700 1 --2- 172.21.15.107:0/2907164901 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7facf00fff60 0x7facf00fb920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:54:40.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.953+0000 7facf7454700 1 -- 172.21.15.107:0/2907164901 >> 172.21.15.107:0/2907164901 conn(0x7facf00f7430 msgr2=0x7facf00f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:54:40.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.954+0000 7facf7454700 1 -- 172.21.15.107:0/2907164901 shutdown_connections 2024-05-17T21:54:40.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:40.954+0000 7facf7454700 1 -- 172.21.15.107:0/2907164901 wait complete. 2024-05-17T21:54:41.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:41.072+0000 7f0f2d32d700 1 -- 172.21.15.107:0/879510844 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f280fce80 msgr2=0x7f0f280fd290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:54:41.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:41.072+0000 7f0f2d32d700 1 --2- 172.21.15.107:0/879510844 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f280fce80 0x7f0f280fd290 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0f18004940 tx=0x7f0f18009e30 comp rx=0 tx=0).stop 2024-05-17T21:54:41.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:41.072+0000 7f0f2d32d700 1 -- 172.21.15.107:0/879510844 shutdown_connections 2024-05-17T21:54:41.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:41.072+0000 7f0f2d32d700 1 --2- 172.21.15.107:0/879510844 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f280fe080 0x7f0f280fe4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:54:41.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:41.072+0000 7f0f2d32d700 1 --2- 172.21.15.107:0/879510844 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f280fce80 0x7f0f280fd290 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:54:41.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:41.072+0000 7f0f2d32d700 1 -- 172.21.15.107:0/879510844 >> 172.21.15.107:0/879510844 conn(0x7f0f280f8590 msgr2=0x7f0f280fa9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:54:41.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:41.072+0000 7f0f2d32d700 1 -- 172.21.15.107:0/879510844 shutdown_connections 2024-05-17T21:54:41.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:41.072+0000 7f0f2d32d700 1 -- 172.21.15.107:0/879510844 wait complete. 2024-05-17T21:54:41.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:41.073+0000 7f0f2d32d700 1 Processor -- start 2024-05-17T21:54:41.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:41.073+0000 7f0f2d32d700 1 -- start start 2024-05-17T21:54:41.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:41.073+0000 7f0f2d32d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f280fce80 0x7f0f28071de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:54:41.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:41.073+0000 7f0f2d32d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f280fe080 0x7f0f280702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:54:41.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:41.073+0000 7f0f2d32d700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f28070860 con 0x7f0f280fce80 2024-05-17T21:54:41.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:41.073+0000 7f0f2d32d700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f280709d0 con 0x7f0f280fe080 2024-05-17T21:54:41.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:41.074+0000 7f0f26ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f280fce80 0x7f0f28071de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:54:41.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:41.074+0000 7f0f26ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f280fce80 0x7f0f28071de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48508/0 (socket says 172.21.15.107:48508) 2024-05-17T21:54:41.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:41.074+0000 7f0f26ffd700 1 -- 172.21.15.107:0/2884042768 learned_addr learned my addr 172.21.15.107:0/2884042768 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:54:41.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:41.074+0000 7f0f267fc700 1 --2- 172.21.15.107:0/2884042768 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f280fe080 0x7f0f280702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:54:41.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:41.074+0000 7f0f26ffd700 1 -- 172.21.15.107:0/2884042768 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f280fe080 msgr2=0x7f0f280702b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:54:41.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:41.074+0000 7f0f26ffd700 1 --2- 172.21.15.107:0/2884042768 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f280fe080 0x7f0f280702b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:54:41.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:41.074+0000 7f0f26ffd700 1 -- 172.21.15.107:0/2884042768 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f18005040 con 0x7f0f280fce80 2024-05-17T21:54:41.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:41.074+0000 7f0f26ffd700 1 --2- 172.21.15.107:0/2884042768 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f280fce80 0x7f0f28071de0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f0f1800b970 tx=0x7f0f18009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:54:41.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:41.074+0000 7f0f0f7fe700 1 -- 172.21.15.107:0/2884042768 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f1800e040 con 0x7f0f280fce80 2024-05-17T21:54:41.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:41.074+0000 7f0f0f7fe700 1 -- 172.21.15.107:0/2884042768 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0f1800bbf0 con 0x7f0f280fce80 2024-05-17T21:54:41.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:41.074+0000 7f0f2d32d700 1 -- 172.21.15.107:0/2884042768 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f28070c50 con 0x7f0f280fce80 2024-05-17T21:54:41.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:41.074+0000 7f0f0f7fe700 1 -- 172.21.15.107:0/2884042768 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f1801c3f0 con 0x7f0f280fce80 2024-05-17T21:54:41.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:41.074+0000 7f0f2d32d700 1 -- 172.21.15.107:0/2884042768 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f281159d0 con 0x7f0f280fce80 2024-05-17T21:54:41.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:41.075+0000 7f0f2d32d700 1 -- 172.21.15.107:0/2884042768 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f2805f5c0 con 0x7f0f280fce80 2024-05-17T21:54:41.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:41.076+0000 7f0f0f7fe700 1 -- 172.21.15.107:0/2884042768 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 100184+0+0 (secure 0 0 0) 0x7f0f1800bd60 con 0x7f0f280fce80 2024-05-17T21:54:41.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:41.077+0000 7f0f0f7fe700 1 --2- 172.21.15.107:0/2884042768 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f0f10078aa0 0x7f0f1007af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:54:41.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:41.077+0000 7f0f267fc700 1 --2- 172.21.15.107:0/2884042768 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f0f10078aa0 0x7f0f1007af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:54:41.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:41.077+0000 7f0f0f7fe700 1 -- 172.21.15.107:0/2884042768 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f0f1809bc80 con 0x7f0f280fce80 2024-05-17T21:54:41.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:41.077+0000 7f0f267fc700 1 --2- 172.21.15.107:0/2884042768 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f0f10078aa0 0x7f0f1007af50 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c009fd0 tx=0x7f0f1c00b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:54:41.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:41.082+0000 7f0f0f7fe700 1 -- 172.21.15.107:0/2884042768 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f0f18063750 con 0x7f0f280fce80 2024-05-17T21:54:41.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:40 smithi138 ceph-mon[35948]: from='client.34136 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:54:41.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:40 smithi138 ceph-mon[35948]: from='client.34140 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:54:41.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:40 smithi138 ceph-mon[35948]: from='client.? 172.21.15.107:0/3950559156' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:54:41.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:40 smithi138 ceph-mon[35948]: from='client.? 172.21.15.107:0/3746675533' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-17T21:54:41.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:41.390+0000 7f0f2d32d700 1 -- 172.21.15.107:0/2884042768 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f0f28105a30 con 0x7f0f280fce80 2024-05-17T21:54:41.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:41.390+0000 7f0f0f7fe700 1 -- 172.21.15.107:0/2884042768 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f0f28105a30 con 0x7f0f280fce80 2024-05-17T21:54:41.383 INFO:teuthology.orchestra.run.smithi107.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-17T21:54:41.383 INFO:teuthology.orchestra.run.smithi107.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-17T21:54:41.383 INFO:teuthology.orchestra.run.smithi107.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-17T21:54:41.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:41.393+0000 7f0f2d32d700 1 -- 172.21.15.107:0/2884042768 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f0f10078aa0 msgr2=0x7f0f1007af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:54:41.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:41.393+0000 7f0f2d32d700 1 --2- 172.21.15.107:0/2884042768 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f0f10078aa0 0x7f0f1007af50 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c009fd0 tx=0x7f0f1c00b040 comp rx=0 tx=0).stop 2024-05-17T21:54:41.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:41.393+0000 7f0f2d32d700 1 -- 172.21.15.107:0/2884042768 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f280fce80 msgr2=0x7f0f28071de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:54:41.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:41.393+0000 7f0f2d32d700 1 --2- 172.21.15.107:0/2884042768 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f280fce80 0x7f0f28071de0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f0f1800b970 tx=0x7f0f18009e30 comp rx=0 tx=0).stop 2024-05-17T21:54:41.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:41.393+0000 7f0f2d32d700 1 -- 172.21.15.107:0/2884042768 shutdown_connections 2024-05-17T21:54:41.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:41.393+0000 7f0f2d32d700 1 --2- 172.21.15.107:0/2884042768 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f0f10078aa0 0x7f0f1007af50 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:54:41.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:41.393+0000 7f0f2d32d700 1 --2- 172.21.15.107:0/2884042768 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f280fce80 0x7f0f28071de0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:54:41.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:41.393+0000 7f0f2d32d700 1 --2- 172.21.15.107:0/2884042768 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f280fe080 0x7f0f280702b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:54:41.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:41.393+0000 7f0f2d32d700 1 -- 172.21.15.107:0/2884042768 >> 172.21.15.107:0/2884042768 conn(0x7f0f280f8590 msgr2=0x7f0f280f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:54:41.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:41.393+0000 7f0f2d32d700 1 -- 172.21.15.107:0/2884042768 shutdown_connections 2024-05-17T21:54:41.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:54:41.393+0000 7f0f2d32d700 1 -- 172.21.15.107:0/2884042768 wait complete. 2024-05-17T21:54:41.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:41 smithi107 ceph-mon[93224]: pgmap v6: 65 pgs: 65 active+clean; 279 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 895 KiB/s rd, 1.1 MiB/s wr, 196 op/s 2024-05-17T21:54:41.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:41 smithi107 ceph-mon[93224]: from='client.24617 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:54:41.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:41 smithi107 ceph-mon[93224]: from='client.? 172.21.15.107:0/2884042768' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-17T21:54:42.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:41 smithi138 ceph-mon[35948]: pgmap v6: 65 pgs: 65 active+clean; 279 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 895 KiB/s rd, 1.1 MiB/s wr, 196 op/s 2024-05-17T21:54:42.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:41 smithi138 ceph-mon[35948]: from='client.24617 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:54:42.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:41 smithi138 ceph-mon[35948]: from='client.? 172.21.15.107:0/2884042768' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-17T21:54:43.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:42 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:54:43.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:42 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:54:43.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:42 smithi138 ceph-mon[35948]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:54:43.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:42 smithi138 ceph-mon[35948]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:54:44.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:44 smithi138 ceph-mon[35948]: pgmap v7: 65 pgs: 65 active+clean; 278 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 773 KiB/s rd, 882 KiB/s wr, 182 op/s 2024-05-17T21:54:44.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:44 smithi138 ceph-mon[35948]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:54:44.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:44 smithi138 ceph-mon[35948]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:54:44.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:44 smithi107 ceph-mon[93224]: pgmap v7: 65 pgs: 65 active+clean; 278 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 773 KiB/s rd, 882 KiB/s wr, 182 op/s 2024-05-17T21:54:44.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:44 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:54:44.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:44 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:54:46.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:46 smithi138 ceph-mon[35948]: pgmap v8: 65 pgs: 65 active+clean; 278 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 601 KiB/s rd, 686 KiB/s wr, 141 op/s 2024-05-17T21:54:46.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:46 smithi107 ceph-mon[93224]: pgmap v8: 65 pgs: 65 active+clean; 278 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 601 KiB/s rd, 686 KiB/s wr, 141 op/s 2024-05-17T21:54:47.390 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.0... 2024-05-17T21:54:47.391 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-05-17T21:54:47.391 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.1... 2024-05-17T21:54:47.392 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.1 /home/ubuntu/cephtest/clone.client.1 2024-05-17T21:54:47.748 DEBUG:teuthology.parallel:result is None 2024-05-17T21:54:47.756 DEBUG:teuthology.parallel:result is None 2024-05-17T21:54:47.756 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-05-17T21:54:47.791 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-05-17T21:54:47.791 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1 2024-05-17T21:54:47.865 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.1/client.1 2024-05-17T21:54:47.866 DEBUG:teuthology.parallel:result is None 2024-05-17T21:54:48.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:48 smithi107 ceph-mon[93224]: pgmap v9: 65 pgs: 65 active+clean; 269 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 220 op/s 2024-05-17T21:54:48.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:48 smithi138 ceph-mon[35948]: pgmap v9: 65 pgs: 65 active+clean; 269 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 220 op/s 2024-05-17T21:54:50.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:50 smithi107 ceph-mon[93224]: pgmap v10: 65 pgs: 65 active+clean; 269 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1003 KiB/s rd, 1.1 MiB/s wr, 202 op/s 2024-05-17T21:54:50.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:50 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:54:50.468 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:50 smithi138 ceph-mon[35948]: pgmap v10: 65 pgs: 65 active+clean; 269 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1003 KiB/s rd, 1.1 MiB/s wr, 202 op/s 2024-05-17T21:54:50.468 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:50 smithi138 ceph-mon[35948]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:54:52.305 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:51 smithi107 ceph-mon[93224]: pgmap v11: 65 pgs: 65 active+clean; 264 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 245 op/s 2024-05-17T21:54:52.305 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:51 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:54:52.306 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:51 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:54:52.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:51 smithi138 ceph-mon[35948]: pgmap v11: 65 pgs: 65 active+clean; 264 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 245 op/s 2024-05-17T21:54:52.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:51 smithi138 ceph-mon[35948]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:54:52.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:51 smithi138 ceph-mon[35948]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:54:53.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:53 smithi138 ceph-mon[35948]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:54:53.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:53 smithi138 ceph-mon[35948]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:54:53.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:53 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:54:53.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:53 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:54:54.222 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:54 smithi138 ceph-mon[35948]: pgmap v12: 65 pgs: 65 active+clean; 262 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1001 KiB/s rd, 942 KiB/s wr, 182 op/s 2024-05-17T21:54:54.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:54 smithi107 ceph-mon[93224]: pgmap v12: 65 pgs: 65 active+clean; 262 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1001 KiB/s rd, 942 KiB/s wr, 182 op/s 2024-05-17T21:54:56.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:56 smithi107 ceph-mon[93224]: pgmap v13: 65 pgs: 65 active+clean; 262 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 924 KiB/s rd, 878 KiB/s wr, 157 op/s 2024-05-17T21:54:56.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:56 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:54:56.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:56 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:54:56.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:56 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:54:56.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:56 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T21:54:56.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:56 smithi138 ceph-mon[35948]: pgmap v13: 65 pgs: 65 active+clean; 262 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 924 KiB/s rd, 878 KiB/s wr, 157 op/s 2024-05-17T21:54:56.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:56 smithi138 ceph-mon[35948]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:54:56.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:56 smithi138 ceph-mon[35948]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:54:56.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:56 smithi138 ceph-mon[35948]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:54:56.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:56 smithi138 ceph-mon[35948]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T21:54:57.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:57 smithi107 ceph-mon[93224]: Updating smithi107:/etc/ceph/ceph.conf 2024-05-17T21:54:57.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:57 smithi107 ceph-mon[93224]: Updating smithi138:/etc/ceph/ceph.conf 2024-05-17T21:54:57.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:57 smithi107 ceph-mon[93224]: Updating smithi138:/var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-17T21:54:57.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:57 smithi107 ceph-mon[93224]: Updating smithi107:/var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-17T21:54:57.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:57 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:54:57.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:57 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:54:57.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:57 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:54:57.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:57 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:54:57.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:57 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:54:57.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:57 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:54:57.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:57 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:54:57.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:57 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-05-17T21:54:57.535 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:57 smithi138 ceph-mon[35948]: Updating smithi107:/etc/ceph/ceph.conf 2024-05-17T21:54:57.535 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:57 smithi138 ceph-mon[35948]: Updating smithi138:/etc/ceph/ceph.conf 2024-05-17T21:54:57.535 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:57 smithi138 ceph-mon[35948]: Updating smithi138:/var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-17T21:54:57.535 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:57 smithi138 ceph-mon[35948]: Updating smithi107:/var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-17T21:54:57.536 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:57 smithi138 ceph-mon[35948]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:54:57.536 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:57 smithi138 ceph-mon[35948]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:54:57.536 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:57 smithi138 ceph-mon[35948]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:54:57.536 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:57 smithi138 ceph-mon[35948]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:54:57.536 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:57 smithi138 ceph-mon[35948]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:54:57.536 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:57 smithi138 ceph-mon[35948]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:54:57.536 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:57 smithi138 ceph-mon[35948]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:54:57.536 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:57 smithi138 ceph-mon[35948]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-05-17T21:54:58.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:58 smithi138 ceph-mon[35948]: Updating smithi138:/etc/ceph/ceph.client.admin.keyring 2024-05-17T21:54:58.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:58 smithi138 ceph-mon[35948]: Updating smithi107:/etc/ceph/ceph.client.admin.keyring 2024-05-17T21:54:58.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:58 smithi138 ceph-mon[35948]: pgmap v14: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 929 KiB/s rd, 883 KiB/s wr, 159 op/s 2024-05-17T21:54:58.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:58 smithi138 ceph-mon[35948]: Updating smithi138:/var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-17T21:54:58.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:58 smithi138 ceph-mon[35948]: Updating smithi107:/var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-17T21:54:58.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:58 smithi107 ceph-mon[93224]: Updating smithi138:/etc/ceph/ceph.client.admin.keyring 2024-05-17T21:54:58.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:58 smithi107 ceph-mon[93224]: Updating smithi107:/etc/ceph/ceph.client.admin.keyring 2024-05-17T21:54:58.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:58 smithi107 ceph-mon[93224]: pgmap v14: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 929 KiB/s rd, 883 KiB/s wr, 159 op/s 2024-05-17T21:54:58.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:58 smithi107 ceph-mon[93224]: Updating smithi138:/var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-17T21:54:58.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:58 smithi107 ceph-mon[93224]: Updating smithi107:/var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-17T21:54:59.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:59 smithi138 ceph-mon[35948]: pgmap v15: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 376 KiB/s rd, 317 KiB/s wr, 64 op/s 2024-05-17T21:54:59.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:59 smithi138 ceph-mon[35948]: Upgrade: Updating mon.smithi138 2024-05-17T21:54:59.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:59 smithi138 ceph-mon[35948]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:54:59.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:59 smithi138 ceph-mon[35948]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-17T21:54:59.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:59 smithi138 ceph-mon[35948]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-17T21:54:59.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:59 smithi138 ceph-mon[35948]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:54:59.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:54:59 smithi138 ceph-mon[35948]: Deploying daemon mon.smithi138 on smithi138 2024-05-17T21:54:59.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:59 smithi107 ceph-mon[93224]: pgmap v15: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 376 KiB/s rd, 317 KiB/s wr, 64 op/s 2024-05-17T21:54:59.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:59 smithi107 ceph-mon[93224]: Upgrade: Updating mon.smithi138 2024-05-17T21:54:59.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:59 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:54:59.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:59 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-17T21:54:59.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:59 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-17T21:54:59.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:59 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:54:59.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:54:59 smithi107 ceph-mon[93224]: Deploying daemon mon.smithi138 on smithi138 2024-05-17T21:55:00.725 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:00 smithi138 systemd[1]: Stopping Ceph mon.smithi138 for 3722731a-1496-11ef-bc9a-c7b262605968... 2024-05-17T21:55:01.042 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:00 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-mon-smithi138[35934]: 2024-05-17T21:55:00.733+0000 7ffba3a7b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi138 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-17T21:55:01.042 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:00 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-mon-smithi138[35934]: 2024-05-17T21:55:00.733+0000 7ffba3a7b700 -1 mon.smithi138@1(peon) e2 *** Got Signal Terminated *** 2024-05-17T21:55:01.042 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:00 smithi138 podman[81217]: 2024-05-17 21:55:00.785307186 +0000 UTC m=+0.168997362 container died ee61a065778d248677a0f5a17cdd2e879d3fc7b25128d9700cc17c88f16c00c2 (image=quay.ceph.io/ceph-ci/ceph@sha256:c200327b094672e93680375e82d2de1a08f5881ef304978f791cce179f847d60, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-mon-smithi138, RELEASE=quincy-9903f77, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-05-17T21:55:01.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:01 smithi138 podman[81217]: 2024-05-17 21:55:01.159012701 +0000 UTC m=+0.542702868 container cleanup ee61a065778d248677a0f5a17cdd2e879d3fc7b25128d9700cc17c88f16c00c2 (image=quay.ceph.io/ceph-ci/ceph@sha256:c200327b094672e93680375e82d2de1a08f5881ef304978f791cce179f847d60, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-mon-smithi138, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, RELEASE=quincy-9903f77, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-05-17T21:55:01.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:01 smithi138 bash[81217]: ceph-3722731a-1496-11ef-bc9a-c7b262605968-mon-smithi138 2024-05-17T21:55:01.779 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:01 smithi138 podman[81230]: 2024-05-17 21:55:01.452235268 +0000 UTC m=+0.662099781 container remove ee61a065778d248677a0f5a17cdd2e879d3fc7b25128d9700cc17c88f16c00c2 (image=quay.ceph.io/ceph-ci/ceph@sha256:c200327b094672e93680375e82d2de1a08f5881ef304978f791cce179f847d60, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-mon-smithi138, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=quincy-9903f77, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-05-17T21:55:01.779 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:01 smithi138 systemd[1]: ceph-3722731a-1496-11ef-bc9a-c7b262605968@mon.smithi138.service: Deactivated successfully. 2024-05-17T21:55:01.779 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:01 smithi138 systemd[1]: Stopped Ceph mon.smithi138 for 3722731a-1496-11ef-bc9a-c7b262605968. 2024-05-17T21:55:01.779 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:01 smithi138 systemd[1]: ceph-3722731a-1496-11ef-bc9a-c7b262605968@mon.smithi138.service: Consumed 6.103s CPU time. 2024-05-17T21:55:02.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:01 smithi138 systemd[1]: Starting Ceph mon.smithi138 for 3722731a-1496-11ef-bc9a-c7b262605968... 2024-05-17T21:55:02.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 podman[81358]: 2024-05-17 21:55:01.907523233 +0000 UTC m=+0.017301383 image pull 8880f7613d0283efa7002505fdccdbc3e37d3b5da31c57fa22d62262c5dab66c quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b 2024-05-17T21:55:02.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 podman[81358]: 2024-05-17 21:55:02.011581925 +0000 UTC m=+0.121360085 container create 55aeed62bf4d04dc3383ddd42054643c62cfda1d91c3cb4983c52feb5b0ce1b9 (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-mon-smithi138, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-05-17T21:55:02.437 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 podman[81358]: 2024-05-17 21:55:02.239512926 +0000 UTC m=+0.349291075 container init 55aeed62bf4d04dc3383ddd42054643c62cfda1d91c3cb4983c52feb5b0ce1b9 (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-mon-smithi138, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-05-17T21:55:02.438 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 podman[81358]: 2024-05-17 21:55:02.242719119 +0000 UTC m=+0.352497262 container start 55aeed62bf4d04dc3383ddd42054643c62cfda1d91c3cb4983c52feb5b0ce1b9 (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-mon-smithi138, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-05-17T21:55:02.438 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: set uid:gid to 167:167 (ceph:ceph) 2024-05-17T21:55:02.438 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev), process ceph-mon, pid 2 2024-05-17T21:55:02.438 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: pidfile_write: ignore empty --pid-file 2024-05-17T21:55:02.438 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: load: jerasure load: lrc 2024-05-17T21:55:02.438 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: RocksDB version: 7.9.2 2024-05-17T21:55:02.438 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Git sha 0 2024-05-17T21:55:02.438 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Compile date 2024-05-17 01:53:57 2024-05-17T21:55:02.438 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: DB SUMMARY 2024-05-17T21:55:02.438 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: DB Session ID: 5XK2BU5UFQBWU158G81V 2024-05-17T21:55:02.439 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: CURRENT file: CURRENT 2024-05-17T21:55:02.439 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: IDENTITY file: IDENTITY 2024-05-17T21:55:02.439 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: MANIFEST file: MANIFEST-000009 size: 1084 Bytes 2024-05-17T21:55:02.439 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi138/store.db dir, Total Num: 1, files: 000027.sst 2024-05-17T21:55:02.439 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi138/store.db: 000025.log size: 427711 ; 2024-05-17T21:55:02.439 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.error_if_exists: 0 2024-05-17T21:55:02.439 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.create_if_missing: 0 2024-05-17T21:55:02.439 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.paranoid_checks: 1 2024-05-17T21:55:02.439 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-17T21:55:02.439 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-17T21:55:02.439 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-17T21:55:02.439 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.env: 0x55eafde94500 2024-05-17T21:55:02.439 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.fs: PosixFileSystem 2024-05-17T21:55:02.439 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.info_log: 0x55eaffd35f20 2024-05-17T21:55:02.440 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.max_file_opening_threads: 16 2024-05-17T21:55:02.440 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.statistics: (nil) 2024-05-17T21:55:02.441 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.use_fsync: 0 2024-05-17T21:55:02.441 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.max_log_file_size: 0 2024-05-17T21:55:02.441 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-17T21:55:02.441 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-17T21:55:02.441 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.keep_log_file_num: 1000 2024-05-17T21:55:02.441 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.recycle_log_file_num: 0 2024-05-17T21:55:02.441 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.allow_fallocate: 1 2024-05-17T21:55:02.441 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.allow_mmap_reads: 0 2024-05-17T21:55:02.441 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.allow_mmap_writes: 0 2024-05-17T21:55:02.441 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.use_direct_reads: 0 2024-05-17T21:55:02.441 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-17T21:55:02.441 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.create_missing_column_families: 0 2024-05-17T21:55:02.441 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.db_log_dir: 2024-05-17T21:55:02.442 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.wal_dir: 2024-05-17T21:55:02.442 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-17T21:55:02.442 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-17T21:55:02.442 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-17T21:55:02.442 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-17T21:55:02.442 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-17T21:55:02.442 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-17T21:55:02.442 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.advise_random_on_open: 1 2024-05-17T21:55:02.442 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.db_write_buffer_size: 0 2024-05-17T21:55:02.442 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.write_buffer_manager: 0x55eaffcf0b40 2024-05-17T21:55:02.442 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-17T21:55:02.442 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-17T21:55:02.443 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-17T21:55:02.443 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.rate_limiter: (nil) 2024-05-17T21:55:02.443 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-17T21:55:02.443 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.wal_recovery_mode: 2 2024-05-17T21:55:02.443 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.enable_thread_tracking: 0 2024-05-17T21:55:02.443 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.enable_pipelined_write: 0 2024-05-17T21:55:02.443 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.unordered_write: 0 2024-05-17T21:55:02.443 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-17T21:55:02.443 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-17T21:55:02.443 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-17T21:55:02.443 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-17T21:55:02.443 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.row_cache: None 2024-05-17T21:55:02.443 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.wal_filter: None 2024-05-17T21:55:02.443 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-17T21:55:02.444 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.allow_ingest_behind: 0 2024-05-17T21:55:02.444 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.two_write_queues: 0 2024-05-17T21:55:02.444 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.manual_wal_flush: 0 2024-05-17T21:55:02.444 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.wal_compression: 0 2024-05-17T21:55:02.444 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.atomic_flush: 0 2024-05-17T21:55:02.444 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-17T21:55:02.444 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-17T21:55:02.444 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-17T21:55:02.444 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.log_readahead_size: 0 2024-05-17T21:55:02.444 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-17T21:55:02.444 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.best_efforts_recovery: 0 2024-05-17T21:55:02.444 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-17T21:55:02.444 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-17T21:55:02.445 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.allow_data_in_errors: 0 2024-05-17T21:55:02.445 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.db_host_id: __hostname__ 2024-05-17T21:55:02.445 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-17T21:55:02.445 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.max_background_jobs: 2 2024-05-17T21:55:02.445 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.max_background_compactions: -1 2024-05-17T21:55:02.445 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.max_subcompactions: 1 2024-05-17T21:55:02.445 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-17T21:55:02.445 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-17T21:55:02.445 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-17T21:55:02.445 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.max_total_wal_size: 0 2024-05-17T21:55:02.445 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-17T21:55:02.445 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-17T21:55:02.445 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-17T21:55:02.445 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-17T21:55:02.445 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.max_open_files: -1 2024-05-17T21:55:02.446 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.bytes_per_sync: 0 2024-05-17T21:55:02.446 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-17T21:55:02.446 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-17T21:55:02.446 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.compaction_readahead_size: 0 2024-05-17T21:55:02.446 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.max_background_flushes: -1 2024-05-17T21:55:02.446 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Compression algorithms supported: 2024-05-17T21:55:02.446 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-17T21:55:02.446 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: kZSTD supported: 0 2024-05-17T21:55:02.446 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: kXpressCompression supported: 0 2024-05-17T21:55:02.446 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: kLZ4HCCompression supported: 1 2024-05-17T21:55:02.446 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: kZlibCompression supported: 1 2024-05-17T21:55:02.446 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: kSnappyCompression supported: 1 2024-05-17T21:55:02.446 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: kLZ4Compression supported: 1 2024-05-17T21:55:02.447 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: kBZip2Compression supported: 0 2024-05-17T21:55:02.447 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-17T21:55:02.447 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-17T21:55:02.447 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi138/store.db/MANIFEST-000009 2024-05-17T21:55:02.447 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-17T21:55:02.447 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-17T21:55:02.447 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.merge_operator: 2024-05-17T21:55:02.447 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.compaction_filter: None 2024-05-17T21:55:02.447 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.compaction_filter_factory: None 2024-05-17T21:55:02.447 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.sst_partitioner_factory: None 2024-05-17T21:55:02.447 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-17T21:55:02.447 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-17T21:55:02.447 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55eaffe3c100) 2024-05-17T21:55:02.447 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: cache_index_and_filter_blocks: 1 2024-05-17T21:55:02.447 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-17T21:55:02.448 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-17T21:55:02.448 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: pin_top_level_index_and_filter: 1 2024-05-17T21:55:02.448 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: index_type: 0 2024-05-17T21:55:02.448 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: data_block_index_type: 0 2024-05-17T21:55:02.448 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: index_shortening: 1 2024-05-17T21:55:02.448 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-17T21:55:02.448 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: checksum: 4 2024-05-17T21:55:02.448 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: no_block_cache: 0 2024-05-17T21:55:02.448 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: block_cache: 0x55eaffd391f0 2024-05-17T21:55:02.448 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: block_cache_name: BinnedLRUCache 2024-05-17T21:55:02.448 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: block_cache_options: 2024-05-17T21:55:02.448 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: capacity : 536870912 2024-05-17T21:55:02.448 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: num_shard_bits : 4 2024-05-17T21:55:02.449 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: strict_capacity_limit : 0 2024-05-17T21:55:02.449 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: high_pri_pool_ratio: 0.000 2024-05-17T21:55:02.449 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: block_cache_compressed: (nil) 2024-05-17T21:55:02.449 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: persistent_cache: (nil) 2024-05-17T21:55:02.449 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: block_size: 4096 2024-05-17T21:55:02.449 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: block_size_deviation: 10 2024-05-17T21:55:02.449 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: block_restart_interval: 16 2024-05-17T21:55:02.449 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: index_block_restart_interval: 1 2024-05-17T21:55:02.449 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: metadata_block_size: 4096 2024-05-17T21:55:02.449 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: partition_filters: 0 2024-05-17T21:55:02.449 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: use_delta_encoding: 1 2024-05-17T21:55:02.449 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: filter_policy: bloomfilter 2024-05-17T21:55:02.449 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: whole_key_filtering: 1 2024-05-17T21:55:02.449 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: verify_compression: 0 2024-05-17T21:55:02.449 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: read_amp_bytes_per_bit: 0 2024-05-17T21:55:02.450 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: format_version: 5 2024-05-17T21:55:02.450 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: enable_index_compression: 1 2024-05-17T21:55:02.450 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: block_align: 0 2024-05-17T21:55:02.450 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: max_auto_readahead_size: 262144 2024-05-17T21:55:02.450 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: prepopulate_block_cache: 0 2024-05-17T21:55:02.450 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: initial_auto_readahead_size: 8192 2024-05-17T21:55:02.450 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: num_file_reads_for_auto_readahead: 2 2024-05-17T21:55:02.450 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.write_buffer_size: 33554432 2024-05-17T21:55:02.450 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.max_write_buffer_number: 2 2024-05-17T21:55:02.450 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.compression: NoCompression 2024-05-17T21:55:02.450 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.bottommost_compression: Disabled 2024-05-17T21:55:02.450 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.prefix_extractor: nullptr 2024-05-17T21:55:02.450 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-17T21:55:02.450 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.num_levels: 7 2024-05-17T21:55:02.451 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-17T21:55:02.451 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-17T21:55:02.451 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-17T21:55:02.451 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-17T21:55:02.451 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-17T21:55:02.451 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-17T21:55:02.451 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-17T21:55:02.451 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-17T21:55:02.451 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-17T21:55:02.451 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-17T21:55:02.451 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-17T21:55:02.451 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-17T21:55:02.451 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-17T21:55:02.451 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.compression_opts.level: 32767 2024-05-17T21:55:02.451 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.compression_opts.strategy: 0 2024-05-17T21:55:02.452 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-17T21:55:02.452 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-17T21:55:02.452 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-17T21:55:02.452 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-17T21:55:02.452 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.compression_opts.enabled: false 2024-05-17T21:55:02.452 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-17T21:55:02.452 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-17T21:55:02.452 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-17T21:55:02.452 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-17T21:55:02.452 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.target_file_size_base: 67108864 2024-05-17T21:55:02.452 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-17T21:55:02.452 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-17T21:55:02.452 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-17T21:55:02.452 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-17T21:55:02.452 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-17T21:55:02.453 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-17T21:55:02.453 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-17T21:55:02.453 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-17T21:55:02.453 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-17T21:55:02.454 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-17T21:55:02.454 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-17T21:55:02.454 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-17T21:55:02.454 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-17T21:55:02.454 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-17T21:55:02.454 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.arena_block_size: 1048576 2024-05-17T21:55:02.454 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-17T21:55:02.454 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-17T21:55:02.454 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.disable_auto_compactions: 0 2024-05-17T21:55:02.454 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-17T21:55:02.454 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-17T21:55:02.454 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-17T21:55:02.454 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-17T21:55:02.455 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-17T21:55:02.455 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-17T21:55:02.455 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-17T21:55:02.455 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-17T21:55:02.455 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-17T21:55:02.455 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-17T21:55:02.455 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-05-17T21:55:02.455 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.inplace_update_support: 0 2024-05-17T21:55:02.455 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-17T21:55:02.455 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-17T21:55:02.455 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-17T21:55:02.455 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-17T21:55:02.455 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.bloom_locality: 0 2024-05-17T21:55:02.455 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.max_successive_merges: 0 2024-05-17T21:55:02.456 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-17T21:55:02.456 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.paranoid_file_checks: 0 2024-05-17T21:55:02.456 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.force_consistency_checks: 1 2024-05-17T21:55:02.456 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.report_bg_io_stats: 0 2024-05-17T21:55:02.456 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.ttl: 2592000 2024-05-17T21:55:02.456 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-17T21:55:02.456 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-17T21:55:02.456 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-17T21:55:02.456 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.enable_blob_files: false 2024-05-17T21:55:02.456 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.min_blob_size: 0 2024-05-17T21:55:02.456 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.blob_file_size: 268435456 2024-05-17T21:55:02.456 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-17T21:55:02.456 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-17T21:55:02.456 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-17T21:55:02.456 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-17T21:55:02.457 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-17T21:55:02.457 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.blob_file_starting_level: 0 2024-05-17T21:55:02.457 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-17T21:55:02.457 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: [table/block_based/block_based_table_reader.cc:721] At least one SST file opened without unique ID to verify: 27.sst 2024-05-17T21:55:02.457 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-05-17T21:55:02.457 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi138/store.db/MANIFEST-000009 succeeded,manifest_file_number is 9, next_file_number is 29, last_sequence is 12284, log_number is 25,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-05-17T21:55:02.457 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 25 2024-05-17T21:55:02.457 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: bd3e77f4-6fcc-4a21-b404-5693d34983e5 2024-05-17T21:55:02.457 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715982902283557, "job": 1, "event": "recovery_started", "wal_files": [25]} 2024-05-17T21:55:02.457 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #25 mode 2 2024-05-17T21:55:02.457 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715982902285345, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 30, "file_size": 261184, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 12289, "largest_seqno": 12492, "table_properties": {"data_size": 259246, "index_size": 644, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 261, "raw_key_size": 2399, "raw_average_key_size": 24, "raw_value_size": 257012, "raw_average_value_size": 2677, "num_data_blocks": 27, "num_entries": 96, "num_filter_entries": 96, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1715982902, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "bd3e77f4-6fcc-4a21-b404-5693d34983e5", "db_session_id": "5XK2BU5UFQBWU158G81V", "orig_file_number": 30, "seqno_to_time_mapping": "N/A"}} 2024-05-17T21:55:02.457 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715982902285432, "job": 1, "event": "recovery_finished"} 2024-05-17T21:55:02.458 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: [db/version_set.cc:5047] Creating manifest 32 2024-05-17T21:55:02.458 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-05-17T21:55:02.458 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi138/store.db/000025.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-17T21:55:02.458 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55eaffd1bc00 2024-05-17T21:55:02.458 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: DB pointer 0x55eb04b10000 2024-05-17T21:55:02.458 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-17T21:55:02.458 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-17T21:55:02.458 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: ** DB Stats ** 2024-05-17T21:55:02.458 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-17T21:55:02.459 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-17T21:55:02.459 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-17T21:55:02.459 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-17T21:55:02.459 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-17T21:55:02.459 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-17T21:55:02.459 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-17T21:55:02.459 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: 2024-05-17T21:55:02.459 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: ** Compaction Stats [default] ** 2024-05-17T21:55:02.459 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-17T21:55:02.459 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-17T21:55:02.459 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: L0 1/0 255.06 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 274.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-17T21:55:02.459 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: L6 1/0 8.13 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-05-17T21:55:02.460 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: Sum 2/0 8.38 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 274.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-17T21:55:02.460 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 274.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-17T21:55:02.460 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: 2024-05-17T21:55:02.460 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: ** Compaction Stats [default] ** 2024-05-17T21:55:02.460 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-17T21:55:02.460 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-17T21:55:02.460 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 274.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-17T21:55:02.460 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: 2024-05-17T21:55:02.460 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-17T21:55:02.460 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: 2024-05-17T21:55:02.460 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-17T21:55:02.460 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-17T21:55:02.460 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-17T21:55:02.461 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-17T21:55:02.461 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-17T21:55:02.461 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-17T21:55:02.461 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: Cumulative compaction: 0.00 GB write, 61.16 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-17T21:55:02.461 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: Interval compaction: 0.00 GB write, 61.16 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-17T21:55:02.461 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-17T21:55:02.461 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: Block cache BinnedLRUCache@0x55eaffd391f0#2 capacity: 512.00 MB usage: 1.09 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-05-17T21:55:02.461 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.32 KB,6.10948e-05%) IndexBlock(1,0.77 KB,0.000146031%) Misc(1,0.00 KB,0%) 2024-05-17T21:55:02.461 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: 2024-05-17T21:55:02.461 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-17T21:55:02.461 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: starting mon.smithi138 rank 1 at public addrs [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] at bind addrs [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi138 fsid 3722731a-1496-11ef-bc9a-c7b262605968 2024-05-17T21:55:02.461 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: mon.smithi138@-1(???) e2 preinit fsid 3722731a-1496-11ef-bc9a-c7b262605968 2024-05-17T21:55:02.461 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: mon.smithi138@-1(???).mds e15 new map 2024-05-17T21:55:02.462 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: mon.smithi138@-1(???).mds e15 print_map 2024-05-17T21:55:02.462 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: e15 2024-05-17T21:55:02.462 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-17T21:55:02.462 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-17T21:55:02.462 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: legacy client fscid: 1 2024-05-17T21:55:02.462 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: 2024-05-17T21:55:02.462 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: Filesystem 'cephfs' (1) 2024-05-17T21:55:02.462 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: fs_name cephfs 2024-05-17T21:55:02.462 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: epoch 15 2024-05-17T21:55:02.462 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-17T21:55:02.462 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: created 2024-05-17T21:49:16.836333+0000 2024-05-17T21:55:02.462 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: modified 2024-05-17T21:49:34.873628+0000 2024-05-17T21:55:02.462 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: tableserver 0 2024-05-17T21:55:02.463 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: root 0 2024-05-17T21:55:02.463 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: session_timeout 60 2024-05-17T21:55:02.463 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: session_autoclose 300 2024-05-17T21:55:02.463 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: max_file_size 1099511627776 2024-05-17T21:55:02.463 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: max_xattr_size 65536 2024-05-17T21:55:02.463 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: required_client_features {} 2024-05-17T21:55:02.463 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: last_failure 0 2024-05-17T21:55:02.463 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: last_failure_osd_epoch 46 2024-05-17T21:55:02.463 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-17T21:55:02.463 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: max_mds 1 2024-05-17T21:55:02.463 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: in 0 2024-05-17T21:55:02.463 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: up {0=24259} 2024-05-17T21:55:02.463 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: failed 2024-05-17T21:55:02.463 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: damaged 2024-05-17T21:55:02.464 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: stopped 2024-05-17T21:55:02.464 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: data_pools [3] 2024-05-17T21:55:02.464 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: metadata_pool 2 2024-05-17T21:55:02.464 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: inline_data enabled 2024-05-17T21:55:02.464 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: balancer 2024-05-17T21:55:02.464 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: bal_rank_mask -1 2024-05-17T21:55:02.464 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: standby_count_wanted 1 2024-05-17T21:55:02.464 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: qdb_cluster leader: 0 members: 2024-05-17T21:55:02.464 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: [mds.cephfs.smithi138.gwbtyd{0:24259} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.138:6824/2779634919,v1:172.21.15.138:6825/2779634919] compat {c=[1],r=[1],i=[7ff]}] 2024-05-17T21:55:02.464 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: [mds.cephfs.smithi107.drqody{0:14486} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.107:6828/2275807100,v1:172.21.15.107:6829/2275807100] compat {c=[1],r=[1],i=[7ff]}] 2024-05-17T21:55:02.464 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: 2024-05-17T21:55:02.464 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: 2024-05-17T21:55:02.464 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: Standby daemons: 2024-05-17T21:55:02.464 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: 2024-05-17T21:55:02.465 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: [mds.cephfs.smithi107.idmbgj{-1:14498} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.107:6826/1460186079,v1:172.21.15.107:6827/1460186079] compat {c=[1],r=[1],i=[7ff]}] 2024-05-17T21:55:02.465 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: [mds.cephfs.smithi138.zckrvs{-1:24267} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.138:6826/170628661,v1:172.21.15.138:6827/170628661] compat {c=[1],r=[1],i=[7ff]}] 2024-05-17T21:55:02.465 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: mon.smithi138@-1(???).osd e49 crush map has features 3314933000852226048, adjusting msgr requires 2024-05-17T21:55:02.465 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: mon.smithi138@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-05-17T21:55:02.465 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: mon.smithi138@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-05-17T21:55:02.465 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: mon.smithi138@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-05-17T21:55:02.465 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: mon.smithi138@-1(???).paxosservice(auth 1..22) refresh upgraded, format 0 -> 3 2024-05-17T21:55:02.465 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 bash[81358]: 55aeed62bf4d04dc3383ddd42054643c62cfda1d91c3cb4983c52feb5b0ce1b9 2024-05-17T21:55:02.465 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 systemd[1]: Started Ceph mon.smithi138 for 3722731a-1496-11ef-bc9a-c7b262605968. 2024-05-17T21:55:02.465 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:02 smithi138 ceph-mon[81376]: pgmap v16: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 376 KiB/s rd, 317 KiB/s wr, 65 op/s 2024-05-17T21:55:02.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:02 smithi107 ceph-mon[93224]: pgmap v16: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 376 KiB/s rd, 317 KiB/s wr, 65 op/s 2024-05-17T21:55:03.567 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:03 smithi138 ceph-mon[81376]: pgmap v17: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 142 KiB/s rd, 79 KiB/s wr, 21 op/s 2024-05-17T21:55:03.567 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:03 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mon metadata", "id": "smithi107"}]: dispatch 2024-05-17T21:55:03.567 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:03 smithi138 ceph-mon[81376]: mon.smithi107 calling monitor election 2024-05-17T21:55:03.567 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:03 smithi138 ceph-mon[81376]: mon.smithi138 calling monitor election 2024-05-17T21:55:03.567 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:03 smithi138 ceph-mon[81376]: mon.smithi107 is new leader, mons smithi107,smithi138 in quorum (ranks 0,1) 2024-05-17T21:55:03.567 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:03 smithi138 ceph-mon[81376]: monmap epoch 3 2024-05-17T21:55:03.567 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:03 smithi138 ceph-mon[81376]: fsid 3722731a-1496-11ef-bc9a-c7b262605968 2024-05-17T21:55:03.567 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:03 smithi138 ceph-mon[81376]: last_changed 2024-05-17T21:55:02.451243+0000 2024-05-17T21:55:03.567 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:03 smithi138 ceph-mon[81376]: created 2024-05-17T21:42:10.472726+0000 2024-05-17T21:55:03.567 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:03 smithi138 ceph-mon[81376]: min_mon_release 19 (squid) 2024-05-17T21:55:03.567 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:03 smithi138 ceph-mon[81376]: election_strategy: 1 2024-05-17T21:55:03.567 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:03 smithi138 ceph-mon[81376]: 0: [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] mon.smithi107 2024-05-17T21:55:03.568 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:03 smithi138 ceph-mon[81376]: 1: [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] mon.smithi138 2024-05-17T21:55:03.568 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:03 smithi138 ceph-mon[81376]: fsmap cephfs:1 {0=cephfs.smithi138.gwbtyd=up:active} 1 up:standby-replay 2 up:standby 2024-05-17T21:55:03.568 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:03 smithi138 ceph-mon[81376]: osdmap e49: 6 total, 6 up, 6 in 2024-05-17T21:55:03.568 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:03 smithi138 ceph-mon[81376]: mgrmap e37: smithi107.xhhyiw(active, since 28s), standbys: smithi138.napnll 2024-05-17T21:55:03.568 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:03 smithi138 ceph-mon[81376]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-17T21:55:03.568 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:03 smithi138 ceph-mon[81376]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-17T21:55:03.568 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:03 smithi138 ceph-mon[81376]: fs cephfs has deprecated feature inline_data enabled. 2024-05-17T21:55:03.568 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:03 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mon metadata", "id": "smithi138"}]: dispatch 2024-05-17T21:55:03.568 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:03 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:03.568 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:03 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:03.568 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:03 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:55:03.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:03 smithi107 ceph-mon[93224]: pgmap v17: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 142 KiB/s rd, 79 KiB/s wr, 21 op/s 2024-05-17T21:55:03.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:03 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mon metadata", "id": "smithi107"}]: dispatch 2024-05-17T21:55:03.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:03 smithi107 ceph-mon[93224]: mon.smithi107 calling monitor election 2024-05-17T21:55:03.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:03 smithi107 ceph-mon[93224]: mon.smithi138 calling monitor election 2024-05-17T21:55:03.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:03 smithi107 ceph-mon[93224]: mon.smithi107 is new leader, mons smithi107,smithi138 in quorum (ranks 0,1) 2024-05-17T21:55:03.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:03 smithi107 ceph-mon[93224]: monmap epoch 3 2024-05-17T21:55:03.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:03 smithi107 ceph-mon[93224]: fsid 3722731a-1496-11ef-bc9a-c7b262605968 2024-05-17T21:55:03.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:03 smithi107 ceph-mon[93224]: last_changed 2024-05-17T21:55:02.451243+0000 2024-05-17T21:55:03.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:03 smithi107 ceph-mon[93224]: created 2024-05-17T21:42:10.472726+0000 2024-05-17T21:55:03.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:03 smithi107 ceph-mon[93224]: min_mon_release 19 (squid) 2024-05-17T21:55:03.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:03 smithi107 ceph-mon[93224]: election_strategy: 1 2024-05-17T21:55:03.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:03 smithi107 ceph-mon[93224]: 0: [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] mon.smithi107 2024-05-17T21:55:03.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:03 smithi107 ceph-mon[93224]: 1: [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] mon.smithi138 2024-05-17T21:55:03.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:03 smithi107 ceph-mon[93224]: fsmap cephfs:1 {0=cephfs.smithi138.gwbtyd=up:active} 1 up:standby-replay 2 up:standby 2024-05-17T21:55:03.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:03 smithi107 ceph-mon[93224]: osdmap e49: 6 total, 6 up, 6 in 2024-05-17T21:55:03.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:03 smithi107 ceph-mon[93224]: mgrmap e37: smithi107.xhhyiw(active, since 28s), standbys: smithi138.napnll 2024-05-17T21:55:03.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:03 smithi107 ceph-mon[93224]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-17T21:55:03.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:03 smithi107 ceph-mon[93224]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-17T21:55:03.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:03 smithi107 ceph-mon[93224]: fs cephfs has deprecated feature inline_data enabled. 2024-05-17T21:55:03.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:03 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mon metadata", "id": "smithi138"}]: dispatch 2024-05-17T21:55:03.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:03 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:03.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:03 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:03.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:03 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:55:04.761 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:04 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:55:04.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:04 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:55:05.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:05 smithi138 ceph-mon[81376]: pgmap v18: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 5.5 KiB/s rd, 4.9 KiB/s wr, 2 op/s 2024-05-17T21:55:05.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:05 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:05.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:05 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:05.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:05 smithi107 ceph-mon[93224]: pgmap v18: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 5.5 KiB/s rd, 4.9 KiB/s wr, 2 op/s 2024-05-17T21:55:05.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:05 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:05.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:05 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:07.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:07 smithi107 ceph-mon[93224]: pgmap v19: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 5.9 KiB/s rd, 4.9 KiB/s wr, 3 op/s 2024-05-17T21:55:07.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:07 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:07.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:07 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:07.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:07 smithi138 ceph-mon[81376]: pgmap v19: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 5.9 KiB/s rd, 4.9 KiB/s wr, 3 op/s 2024-05-17T21:55:07.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:07 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:07.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:07 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:09.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:09 smithi138 ceph-mon[81376]: pgmap v20: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T21:55:09.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:09 smithi107 ceph-mon[93224]: pgmap v20: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T21:55:11.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:11 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:11.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:11 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:11.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:11 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:55:11.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:11 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T21:55:11.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:11 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:11.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:11 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-17T21:55:11.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:11 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-17T21:55:11.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:11 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:55:11.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:11 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:11.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:11 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:11.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:11 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:55:11.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:11 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T21:55:11.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:11 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:11.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:11 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-17T21:55:11.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:11 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-17T21:55:11.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:11 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:55:11.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:11.503+0000 7fd5123f0700 1 -- 172.21.15.107:0/455899100 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd50c0fc710 msgr2=0x7fd50c0fcb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:55:11.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:11.503+0000 7fd5123f0700 1 --2- 172.21.15.107:0/455899100 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd50c0fc710 0x7fd50c0fcb80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd4f4004a50 tx=0x7fd4f400af40 comp rx=0 tx=0).stop 2024-05-17T21:55:11.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:11.503+0000 7fd5123f0700 1 -- 172.21.15.107:0/455899100 shutdown_connections 2024-05-17T21:55:11.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:11.503+0000 7fd5123f0700 1 --2- 172.21.15.107:0/455899100 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd50c0fc710 0x7fd50c0fcb80 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:55:11.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:11.503+0000 7fd5123f0700 1 --2- 172.21.15.107:0/455899100 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd50c0fb460 0x7fd50c0fb870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:55:11.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:11.503+0000 7fd5123f0700 1 -- 172.21.15.107:0/455899100 >> 172.21.15.107:0/455899100 conn(0x7fd50c0f6be0 msgr2=0x7fd50c0f8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:55:11.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:11.503+0000 7fd5123f0700 1 -- 172.21.15.107:0/455899100 shutdown_connections 2024-05-17T21:55:11.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:11.503+0000 7fd5123f0700 1 -- 172.21.15.107:0/455899100 wait complete. 2024-05-17T21:55:11.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:11.503+0000 7fd5123f0700 1 Processor -- start 2024-05-17T21:55:11.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:11.503+0000 7fd5123f0700 1 -- start start 2024-05-17T21:55:11.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:11.504+0000 7fd5123f0700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd50c0fb460 0x7fd50c0ff3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:55:11.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:11.504+0000 7fd5123f0700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd50c0fc710 0x7fd50c0fda50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:55:11.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:11.504+0000 7fd5123f0700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd50c0ff9a0 con 0x7fd50c0fc710 2024-05-17T21:55:11.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:11.504+0000 7fd5123f0700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd50c0fdf90 con 0x7fd50c0fb460 2024-05-17T21:55:11.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:11.504+0000 7fd503fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd50c0fc710 0x7fd50c0fda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:55:11.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:11.504+0000 7fd503fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd50c0fc710 0x7fd50c0fda50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42376/0 (socket says 172.21.15.107:42376) 2024-05-17T21:55:11.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:11.504+0000 7fd503fff700 1 -- 172.21.15.107:0/3839426866 learned_addr learned my addr 172.21.15.107:0/3839426866 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:55:11.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:11.504+0000 7fd50bfff700 1 --2- 172.21.15.107:0/3839426866 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd50c0fb460 0x7fd50c0ff3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:55:11.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:11.504+0000 7fd503fff700 1 -- 172.21.15.107:0/3839426866 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd50c0fb460 msgr2=0x7fd50c0ff3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:55:11.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:11.504+0000 7fd503fff700 1 --2- 172.21.15.107:0/3839426866 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd50c0fb460 0x7fd50c0ff3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:55:11.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:11.504+0000 7fd503fff700 1 -- 172.21.15.107:0/3839426866 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4f4005040 con 0x7fd50c0fc710 2024-05-17T21:55:11.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:11.504+0000 7fd50bfff700 1 --2- 172.21.15.107:0/3839426866 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd50c0fb460 0x7fd50c0ff3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T21:55:11.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:11.504+0000 7fd503fff700 1 --2- 172.21.15.107:0/3839426866 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd50c0fc710 0x7fd50c0fda50 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd4f40043d0 tx=0x7fd4f400af40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:55:11.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:11.504+0000 7fd5097fa700 1 -- 172.21.15.107:0/3839426866 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd4f400f050 con 0x7fd50c0fc710 2024-05-17T21:55:11.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:11.504+0000 7fd5097fa700 1 -- 172.21.15.107:0/3839426866 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd4f401d950 con 0x7fd50c0fc710 2024-05-17T21:55:11.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:11.504+0000 7fd5123f0700 1 -- 172.21.15.107:0/3839426866 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd50c0fe1e0 con 0x7fd50c0fc710 2024-05-17T21:55:11.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:11.505+0000 7fd5097fa700 1 -- 172.21.15.107:0/3839426866 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd4f4014a00 con 0x7fd50c0fc710 2024-05-17T21:55:11.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:11.505+0000 7fd5123f0700 1 -- 172.21.15.107:0/3839426866 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd50c0fe740 con 0x7fd50c0fc710 2024-05-17T21:55:11.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:11.505+0000 7fd5123f0700 1 -- 172.21.15.107:0/3839426866 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd50c04c710 con 0x7fd50c0fc710 2024-05-17T21:55:11.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:11.506+0000 7fd5097fa700 1 -- 172.21.15.107:0/3839426866 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 100184+0+0 (secure 0 0 0) 0x7fd4f4014cc0 con 0x7fd50c0fc710 2024-05-17T21:55:11.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:11.507+0000 7fd5097fa700 1 --2- 172.21.15.107:0/3839426866 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fd4f8078aa0 0x7fd4f807af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:55:11.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:11.507+0000 7fd5097fa700 1 -- 172.21.15.107:0/3839426866 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fd4f409c780 con 0x7fd50c0fc710 2024-05-17T21:55:11.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:11.507+0000 7fd50bfff700 1 --2- 172.21.15.107:0/3839426866 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fd4f8078aa0 0x7fd4f807af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:55:11.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:11.507+0000 7fd50bfff700 1 --2- 172.21.15.107:0/3839426866 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fd4f8078aa0 0x7fd4f807af50 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd4fc009fd0 tx=0x7fd4fc009ee0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:55:11.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:11.511+0000 7fd5097fa700 1 -- 172.21.15.107:0/3839426866 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7fd4f4064250 con 0x7fd50c0fc710 2024-05-17T21:55:11.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:11.742+0000 7fd5123f0700 1 -- 172.21.15.107:0/3839426866 --> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd50c100bf0 con 0x7fd4f8078aa0 2024-05-17T21:55:11.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:11.743+0000 7fd5097fa700 1 -- 172.21.15.107:0/3839426866 <== mgr.34100 v2:172.21.15.107:6800/3205268226 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7fd50c100bf0 con 0x7fd4f8078aa0 2024-05-17T21:55:11.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:11.746+0000 7fd5123f0700 1 -- 172.21.15.107:0/3839426866 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fd4f8078aa0 msgr2=0x7fd4f807af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:55:11.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:11.746+0000 7fd5123f0700 1 --2- 172.21.15.107:0/3839426866 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fd4f8078aa0 0x7fd4f807af50 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd4fc009fd0 tx=0x7fd4fc009ee0 comp rx=0 tx=0).stop 2024-05-17T21:55:11.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:11.746+0000 7fd5123f0700 1 -- 172.21.15.107:0/3839426866 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd50c0fc710 msgr2=0x7fd50c0fda50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:55:11.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:11.746+0000 7fd5123f0700 1 --2- 172.21.15.107:0/3839426866 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd50c0fc710 0x7fd50c0fda50 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd4f40043d0 tx=0x7fd4f400af40 comp rx=0 tx=0).stop 2024-05-17T21:55:11.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:11.746+0000 7fd5123f0700 1 -- 172.21.15.107:0/3839426866 shutdown_connections 2024-05-17T21:55:11.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:11.746+0000 7fd5123f0700 1 --2- 172.21.15.107:0/3839426866 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fd4f8078aa0 0x7fd4f807af50 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:55:11.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:11.746+0000 7fd5123f0700 1 --2- 172.21.15.107:0/3839426866 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd50c0fb460 0x7fd50c0ff3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:55:11.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:11.746+0000 7fd5123f0700 1 --2- 172.21.15.107:0/3839426866 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd50c0fc710 0x7fd50c0fda50 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:55:11.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:11.746+0000 7fd5123f0700 1 -- 172.21.15.107:0/3839426866 >> 172.21.15.107:0/3839426866 conn(0x7fd50c0f6be0 msgr2=0x7fd50c0f8230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:55:11.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:11.746+0000 7fd5123f0700 1 -- 172.21.15.107:0/3839426866 shutdown_connections 2024-05-17T21:55:11.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:11.746+0000 7fd5123f0700 1 -- 172.21.15.107:0/3839426866 wait complete. 2024-05-17T21:55:11.749 INFO:teuthology.orchestra.run.smithi107.stdout:true 2024-05-17T21:55:11.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:11.857+0000 7ff89ff88700 1 -- 172.21.15.107:0/2621064031 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8980fe3e0 msgr2=0x7ff8980fe7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:55:11.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:11.857+0000 7ff89ff88700 1 --2- 172.21.15.107:0/2621064031 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8980fe3e0 0x7ff8980fe7f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff88c004940 tx=0x7ff88c009e30 comp rx=0 tx=0).stop 2024-05-17T21:55:11.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:11.857+0000 7ff89ff88700 1 -- 172.21.15.107:0/2621064031 shutdown_connections 2024-05-17T21:55:11.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:11.857+0000 7ff89ff88700 1 --2- 172.21.15.107:0/2621064031 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8980fed30 0x7ff8980fae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:55:11.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:11.857+0000 7ff89ff88700 1 --2- 172.21.15.107:0/2621064031 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8980fe3e0 0x7ff8980fe7f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:55:11.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:11.857+0000 7ff89ff88700 1 -- 172.21.15.107:0/2621064031 >> 172.21.15.107:0/2621064031 conn(0x7ff8980f6b80 msgr2=0x7ff8980f8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:55:11.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:11.857+0000 7ff89ff88700 1 -- 172.21.15.107:0/2621064031 shutdown_connections 2024-05-17T21:55:11.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:11.857+0000 7ff89ff88700 1 -- 172.21.15.107:0/2621064031 wait complete. 2024-05-17T21:55:11.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:11.858+0000 7ff89ff88700 1 Processor -- start 2024-05-17T21:55:11.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:11.858+0000 7ff89ff88700 1 -- start start 2024-05-17T21:55:11.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:11.858+0000 7ff89ff88700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8980fe3e0 0x7ff89810d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:55:11.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:11.858+0000 7ff89ff88700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8980fed30 0x7ff89810df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:55:11.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:11.858+0000 7ff89ff88700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff89810e510 con 0x7ff8980fed30 2024-05-17T21:55:11.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:11.858+0000 7ff89ff88700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff89810ed60 con 0x7ff8980fe3e0 2024-05-17T21:55:11.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:11.858+0000 7ff89d523700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8980fed30 0x7ff89810df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:55:11.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:11.858+0000 7ff89d523700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8980fed30 0x7ff89810df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42408/0 (socket says 172.21.15.107:42408) 2024-05-17T21:55:11.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:11.858+0000 7ff89d523700 1 -- 172.21.15.107:0/2322534359 learned_addr learned my addr 172.21.15.107:0/2322534359 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:55:11.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:11.858+0000 7ff89dd24700 1 --2- 172.21.15.107:0/2322534359 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8980fe3e0 0x7ff89810d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:55:11.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:11.858+0000 7ff89d523700 1 -- 172.21.15.107:0/2322534359 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8980fe3e0 msgr2=0x7ff89810d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:55:11.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:11.858+0000 7ff89d523700 1 --2- 172.21.15.107:0/2322534359 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8980fe3e0 0x7ff89810d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:55:11.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:11.858+0000 7ff89d523700 1 -- 172.21.15.107:0/2322534359 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff88c005040 con 0x7ff8980fed30 2024-05-17T21:55:11.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:11.858+0000 7ff89dd24700 1 --2- 172.21.15.107:0/2322534359 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8980fe3e0 0x7ff89810d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T21:55:11.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:11.858+0000 7ff89d523700 1 --2- 172.21.15.107:0/2322534359 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8980fed30 0x7ff89810df10 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff894002a50 tx=0x7ff8940046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:55:11.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:11.859+0000 7ff88a7fc700 1 -- 172.21.15.107:0/2322534359 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff894007cc0 con 0x7ff8980fed30 2024-05-17T21:55:11.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:11.859+0000 7ff88a7fc700 1 -- 172.21.15.107:0/2322534359 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff89400f040 con 0x7ff8980fed30 2024-05-17T21:55:11.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:11.859+0000 7ff89ff88700 1 -- 172.21.15.107:0/2322534359 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff89810f040 con 0x7ff8980fed30 2024-05-17T21:55:11.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:11.859+0000 7ff88a7fc700 1 -- 172.21.15.107:0/2322534359 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff894009640 con 0x7ff8980fed30 2024-05-17T21:55:11.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:11.859+0000 7ff89ff88700 1 -- 172.21.15.107:0/2322534359 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff89810f580 con 0x7ff8980fed30 2024-05-17T21:55:11.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:11.860+0000 7ff89ff88700 1 -- 172.21.15.107:0/2322534359 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff89805f5c0 con 0x7ff8980fed30 2024-05-17T21:55:11.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:11.861+0000 7ff88a7fc700 1 -- 172.21.15.107:0/2322534359 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 100184+0+0 (secure 0 0 0) 0x7ff8940097a0 con 0x7ff8980fed30 2024-05-17T21:55:11.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:11.861+0000 7ff88a7fc700 1 --2- 172.21.15.107:0/2322534359 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7ff884078aa0 0x7ff88407af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:55:11.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:11.861+0000 7ff88a7fc700 1 -- 172.21.15.107:0/2322534359 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6410+0+0 (secure 0 0 0) 0x7ff89409a680 con 0x7ff8980fed30 2024-05-17T21:55:11.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:11.861+0000 7ff89dd24700 1 --2- 172.21.15.107:0/2322534359 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7ff884078aa0 0x7ff88407af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:55:11.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:11.862+0000 7ff89dd24700 1 --2- 172.21.15.107:0/2322534359 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7ff884078aa0 0x7ff88407af50 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff88c00b3b0 tx=0x7ff88c009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:55:11.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:11.870+0000 7ff88a7fc700 1 -- 172.21.15.107:0/2322534359 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7ff894062150 con 0x7ff8980fed30 2024-05-17T21:55:12.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.116+0000 7ff89ff88700 1 -- 172.21.15.107:0/2322534359 --> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff89810f9b0 con 0x7ff884078aa0 2024-05-17T21:55:12.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.118+0000 7ff88a7fc700 1 -- 172.21.15.107:0/2322534359 <== mgr.34100 v2:172.21.15.107:6800/3205268226 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7ff89810f9b0 con 0x7ff884078aa0 2024-05-17T21:55:12.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.120+0000 7ff89ff88700 1 -- 172.21.15.107:0/2322534359 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7ff884078aa0 msgr2=0x7ff88407af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:55:12.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.120+0000 7ff89ff88700 1 --2- 172.21.15.107:0/2322534359 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7ff884078aa0 0x7ff88407af50 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff88c00b3b0 tx=0x7ff88c009e30 comp rx=0 tx=0).stop 2024-05-17T21:55:12.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.120+0000 7ff89ff88700 1 -- 172.21.15.107:0/2322534359 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8980fed30 msgr2=0x7ff89810df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:55:12.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.120+0000 7ff89ff88700 1 --2- 172.21.15.107:0/2322534359 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8980fed30 0x7ff89810df10 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff894002a50 tx=0x7ff8940046b0 comp rx=0 tx=0).stop 2024-05-17T21:55:12.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.120+0000 7ff89ff88700 1 -- 172.21.15.107:0/2322534359 shutdown_connections 2024-05-17T21:55:12.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.120+0000 7ff89ff88700 1 --2- 172.21.15.107:0/2322534359 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7ff884078aa0 0x7ff88407af50 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:55:12.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.120+0000 7ff89ff88700 1 --2- 172.21.15.107:0/2322534359 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8980fe3e0 0x7ff89810d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:55:12.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.120+0000 7ff89ff88700 1 --2- 172.21.15.107:0/2322534359 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8980fed30 0x7ff89810df10 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:55:12.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.120+0000 7ff89ff88700 1 -- 172.21.15.107:0/2322534359 >> 172.21.15.107:0/2322534359 conn(0x7ff8980f6b80 msgr2=0x7ff8980f8260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:55:12.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.120+0000 7ff89ff88700 1 -- 172.21.15.107:0/2322534359 shutdown_connections 2024-05-17T21:55:12.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.120+0000 7ff89ff88700 1 -- 172.21.15.107:0/2322534359 wait complete. 2024-05-17T21:55:12.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.231+0000 7fe120202700 1 -- 172.21.15.107:0/1746681886 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe1180fc270 msgr2=0x7fe1180fc680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:55:12.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.231+0000 7fe120202700 1 --2- 172.21.15.107:0/1746681886 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe1180fc270 0x7fe1180fc680 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe114004940 tx=0x7fe114009e30 comp rx=0 tx=0).stop 2024-05-17T21:55:12.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.232+0000 7fe120202700 1 -- 172.21.15.107:0/1746681886 shutdown_connections 2024-05-17T21:55:12.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.232+0000 7fe120202700 1 --2- 172.21.15.107:0/1746681886 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1180fcbc0 0x7fe1180fd030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:55:12.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.232+0000 7fe120202700 1 --2- 172.21.15.107:0/1746681886 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe1180fc270 0x7fe1180fc680 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:55:12.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.232+0000 7fe120202700 1 -- 172.21.15.107:0/1746681886 >> 172.21.15.107:0/1746681886 conn(0x7fe1180f6b80 msgr2=0x7fe1180f8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:55:12.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.232+0000 7fe120202700 1 -- 172.21.15.107:0/1746681886 shutdown_connections 2024-05-17T21:55:12.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.232+0000 7fe120202700 1 -- 172.21.15.107:0/1746681886 wait complete. 2024-05-17T21:55:12.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.232+0000 7fe120202700 1 Processor -- start 2024-05-17T21:55:12.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.232+0000 7fe120202700 1 -- start start 2024-05-17T21:55:12.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.233+0000 7fe120202700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe1180fc270 0x7fe11810c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:55:12.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.233+0000 7fe120202700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1180fcbc0 0x7fe11810ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:55:12.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.233+0000 7fe120202700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe11810d420 con 0x7fe1180fc270 2024-05-17T21:55:12.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.233+0000 7fe120202700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe11810dc70 con 0x7fe1180fcbc0 2024-05-17T21:55:12.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.233+0000 7fe11df9e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe1180fc270 0x7fe11810c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:55:12.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.233+0000 7fe11df9e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe1180fc270 0x7fe11810c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42418/0 (socket says 172.21.15.107:42418) 2024-05-17T21:55:12.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.233+0000 7fe11df9e700 1 -- 172.21.15.107:0/2096893952 learned_addr learned my addr 172.21.15.107:0/2096893952 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:55:12.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.233+0000 7fe11d79d700 1 --2- 172.21.15.107:0/2096893952 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1180fcbc0 0x7fe11810ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:55:12.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.233+0000 7fe11df9e700 1 -- 172.21.15.107:0/2096893952 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1180fcbc0 msgr2=0x7fe11810ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:55:12.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.233+0000 7fe11df9e700 1 --2- 172.21.15.107:0/2096893952 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1180fcbc0 0x7fe11810ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:55:12.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.233+0000 7fe11df9e700 1 -- 172.21.15.107:0/2096893952 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe114005040 con 0x7fe1180fc270 2024-05-17T21:55:12.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.233+0000 7fe11df9e700 1 --2- 172.21.15.107:0/2096893952 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe1180fc270 0x7fe11810c8e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe11400b3b0 tx=0x7fe114009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:55:12.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.233+0000 7fe10e7fc700 1 -- 172.21.15.107:0/2096893952 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe11400e040 con 0x7fe1180fc270 2024-05-17T21:55:12.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.234+0000 7fe120202700 1 -- 172.21.15.107:0/2096893952 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe11810def0 con 0x7fe1180fc270 2024-05-17T21:55:12.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.234+0000 7fe10e7fc700 1 -- 172.21.15.107:0/2096893952 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe11400bcc0 con 0x7fe1180fc270 2024-05-17T21:55:12.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.234+0000 7fe10e7fc700 1 -- 172.21.15.107:0/2096893952 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe11401c630 con 0x7fe1180fc270 2024-05-17T21:55:12.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.234+0000 7fe120202700 1 -- 172.21.15.107:0/2096893952 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe11810e430 con 0x7fe1180fc270 2024-05-17T21:55:12.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.235+0000 7fe120202700 1 -- 172.21.15.107:0/2096893952 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe11805f5c0 con 0x7fe1180fc270 2024-05-17T21:55:12.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.235+0000 7fe10e7fc700 1 -- 172.21.15.107:0/2096893952 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 100184+0+0 (secure 0 0 0) 0x7fe114013980 con 0x7fe1180fc270 2024-05-17T21:55:12.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.236+0000 7fe10e7fc700 1 --2- 172.21.15.107:0/2096893952 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fe104078aa0 0x7fe10407af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:55:12.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.236+0000 7fe11d79d700 1 --2- 172.21.15.107:0/2096893952 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fe104078aa0 0x7fe10407af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:55:12.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.236+0000 7fe10e7fc700 1 -- 172.21.15.107:0/2096893952 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fe11409b6c0 con 0x7fe1180fc270 2024-05-17T21:55:12.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.236+0000 7fe11d79d700 1 --2- 172.21.15.107:0/2096893952 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fe104078aa0 0x7fe10407af50 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe108004990 tx=0x7fe108009ee0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:55:12.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.241+0000 7fe10e7fc700 1 -- 172.21.15.107:0/2096893952 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7fe114063110 con 0x7fe1180fc270 2024-05-17T21:55:12.304 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:12 smithi107 ceph-mon[93224]: Reconfiguring mon.smithi107 (monmap changed)... 2024-05-17T21:55:12.304 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:12 smithi107 ceph-mon[93224]: Reconfiguring daemon mon.smithi107 on smithi107 2024-05-17T21:55:12.305 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:12 smithi107 ceph-mon[93224]: pgmap v21: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T21:55:12.305 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:12 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:12.305 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:12 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:12.305 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:12 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi107.xhhyiw", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-17T21:55:12.305 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:12 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-17T21:55:12.305 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:12 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:55:12.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:12 smithi138 ceph-mon[81376]: Reconfiguring mon.smithi107 (monmap changed)... 2024-05-17T21:55:12.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:12 smithi138 ceph-mon[81376]: Reconfiguring daemon mon.smithi107 on smithi107 2024-05-17T21:55:12.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:12 smithi138 ceph-mon[81376]: pgmap v21: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T21:55:12.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:12 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:12.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:12 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:12.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:12 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi107.xhhyiw", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-17T21:55:12.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:12 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-17T21:55:12.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:12 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:55:12.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.460+0000 7fe120202700 1 -- 172.21.15.107:0/2096893952 --> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fe11810e710 con 0x7fe104078aa0 2024-05-17T21:55:12.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.470+0000 7fe10e7fc700 1 -- 172.21.15.107:0/2096893952 <== mgr.34100 v2:172.21.15.107:6800/3205268226 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fe11810e710 con 0x7fe104078aa0 2024-05-17T21:55:12.461 INFO:teuthology.orchestra.run.smithi107.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-17T21:55:12.462 INFO:teuthology.orchestra.run.smithi107.stdout:alertmanager.smithi107 smithi107 *:9093,9094 running (2m) 30s ago 11m 17.2M - 0.25.0 c8568f914cd2 bac2a944ae42 2024-05-17T21:55:12.462 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi107 smithi107 running (11m) 30s ago 11m 6660k - 17.2.7-952-g9903f776 d15a36853796 828741c3a89c 2024-05-17T21:55:12.462 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi138 smithi138 running (10m) 7s ago 10m 6660k - 17.2.7-952-g9903f776 d15a36853796 20e9ccf1fadd 2024-05-17T21:55:12.462 INFO:teuthology.orchestra.run.smithi107.stdout:grafana.smithi107 smithi107 *:3000 running (2m) 30s ago 11m 79.6M - 9.4.7 954c08fa6188 8c747e777bc5 2024-05-17T21:55:12.462 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.drqody smithi107 running (5m) 30s ago 5m 170M - 17.2.7-952-g9903f776 d15a36853796 0d851bc912f0 2024-05-17T21:55:12.462 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.idmbgj smithi107 running (5m) 30s ago 5m 13.1M - 17.2.7-952-g9903f776 d15a36853796 1348b09ab29e 2024-05-17T21:55:12.462 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi138.gwbtyd smithi138 running (5m) 7s ago 5m 153M - 17.2.7-952-g9903f776 d15a36853796 fd2ba75354c0 2024-05-17T21:55:12.462 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi138.zckrvs smithi138 running (5m) 7s ago 5m 13.9M - 17.2.7-952-g9903f776 d15a36853796 a1cfd2fe4cda 2024-05-17T21:55:12.462 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi107.xhhyiw smithi107 *:8443,9283,8765 running (2m) 30s ago 12m 556M - 19.0.0-3794-g931809a9 8880f7613d02 5acf3e511342 2024-05-17T21:55:12.462 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi138.napnll smithi138 *:8443,9283,8765 running (69s) 7s ago 10m 531M - 19.0.0-3794-g931809a9 8880f7613d02 d627761dbd0d 2024-05-17T21:55:12.463 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi107 smithi107 running (48s) 30s ago 12m 96.4M 2048M 19.0.0-3794-g931809a9 8880f7613d02 9717f3169f2f 2024-05-17T21:55:12.463 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi138 smithi138 running (10s) 7s ago 10m 87.1M 2048M 19.0.0-3794-g931809a9 8880f7613d02 55aeed62bf4d 2024-05-17T21:55:12.463 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi107 smithi107 *:9100 running (2m) 30s ago 11m 17.2M - 1.5.0 0da6a335fe13 49bec71f5487 2024-05-17T21:55:12.463 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi138 smithi138 *:9100 running (2m) 7s ago 10m 16.2M - 1.5.0 0da6a335fe13 e52824376658 2024-05-17T21:55:12.463 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0 smithi107 running (9m) 30s ago 9m 198M 1557M 17.2.7-952-g9903f776 d15a36853796 f2d20009361d 2024-05-17T21:55:12.463 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1 smithi107 running (9m) 30s ago 9m 214M 1557M 17.2.7-952-g9903f776 d15a36853796 e5d3e99844a9 2024-05-17T21:55:12.463 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2 smithi107 running (8m) 30s ago 8m 175M 1557M 17.2.7-952-g9903f776 d15a36853796 e044d7b78088 2024-05-17T21:55:12.463 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3 smithi138 running (8m) 7s ago 8m 227M 2581M 17.2.7-952-g9903f776 d15a36853796 8c980cc8af87 2024-05-17T21:55:12.463 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4 smithi138 running (7m) 7s ago 7m 204M 2581M 17.2.7-952-g9903f776 d15a36853796 b306a36a744e 2024-05-17T21:55:12.463 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5 smithi138 running (7m) 7s ago 7m 189M 2581M 17.2.7-952-g9903f776 d15a36853796 7c8d8ed74974 2024-05-17T21:55:12.463 INFO:teuthology.orchestra.run.smithi107.stdout:prometheus.smithi107 smithi107 *:9095 running (88s) 30s ago 10m 51.5M - 2.43.0 a07b618ecd1d bf5f609bbd6e 2024-05-17T21:55:12.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.472+0000 7fe120202700 1 -- 172.21.15.107:0/2096893952 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fe104078aa0 msgr2=0x7fe10407af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:55:12.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.472+0000 7fe120202700 1 --2- 172.21.15.107:0/2096893952 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fe104078aa0 0x7fe10407af50 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe108004990 tx=0x7fe108009ee0 comp rx=0 tx=0).stop 2024-05-17T21:55:12.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.472+0000 7fe120202700 1 -- 172.21.15.107:0/2096893952 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe1180fc270 msgr2=0x7fe11810c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:55:12.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.472+0000 7fe120202700 1 --2- 172.21.15.107:0/2096893952 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe1180fc270 0x7fe11810c8e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe11400b3b0 tx=0x7fe114009e30 comp rx=0 tx=0).stop 2024-05-17T21:55:12.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.472+0000 7fe120202700 1 -- 172.21.15.107:0/2096893952 shutdown_connections 2024-05-17T21:55:12.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.473+0000 7fe120202700 1 --2- 172.21.15.107:0/2096893952 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fe104078aa0 0x7fe10407af50 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:55:12.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.473+0000 7fe120202700 1 --2- 172.21.15.107:0/2096893952 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe1180fc270 0x7fe11810c8e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:55:12.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.473+0000 7fe120202700 1 --2- 172.21.15.107:0/2096893952 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1180fcbc0 0x7fe11810ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:55:12.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.473+0000 7fe120202700 1 -- 172.21.15.107:0/2096893952 >> 172.21.15.107:0/2096893952 conn(0x7fe1180f6b80 msgr2=0x7fe1180f8270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:55:12.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.473+0000 7fe120202700 1 -- 172.21.15.107:0/2096893952 shutdown_connections 2024-05-17T21:55:12.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.473+0000 7fe120202700 1 -- 172.21.15.107:0/2096893952 wait complete. 2024-05-17T21:55:12.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.577+0000 7f10c30d7700 1 -- 172.21.15.107:0/4017112325 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f10bc0fe410 msgr2=0x7f10bc0fe820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:55:12.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.577+0000 7f10c30d7700 1 --2- 172.21.15.107:0/4017112325 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f10bc0fe410 0x7f10bc0fe820 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f10b0004940 tx=0x7f10b0009e30 comp rx=0 tx=0).stop 2024-05-17T21:55:12.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.577+0000 7f10c30d7700 1 -- 172.21.15.107:0/4017112325 shutdown_connections 2024-05-17T21:55:12.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.577+0000 7f10c30d7700 1 --2- 172.21.15.107:0/4017112325 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10bc0fed60 0x7f10bc0faea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:55:12.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.577+0000 7f10c30d7700 1 --2- 172.21.15.107:0/4017112325 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f10bc0fe410 0x7f10bc0fe820 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:55:12.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.577+0000 7f10c30d7700 1 -- 172.21.15.107:0/4017112325 >> 172.21.15.107:0/4017112325 conn(0x7f10bc0f6bd0 msgr2=0x7f10bc0f8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:55:12.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.578+0000 7f10c30d7700 1 -- 172.21.15.107:0/4017112325 shutdown_connections 2024-05-17T21:55:12.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.578+0000 7f10c30d7700 1 -- 172.21.15.107:0/4017112325 wait complete. 2024-05-17T21:55:12.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.578+0000 7f10c30d7700 1 Processor -- start 2024-05-17T21:55:12.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.578+0000 7f10c30d7700 1 -- start start 2024-05-17T21:55:12.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.579+0000 7f10c30d7700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10bc0fe410 0x7f10bc10d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:55:12.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.579+0000 7f10c30d7700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f10bc0fed60 0x7f10bc10de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:55:12.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.579+0000 7f10c30d7700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10bc10e450 con 0x7f10bc0fed60 2024-05-17T21:55:12.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.579+0000 7f10c30d7700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10bc10e5c0 con 0x7f10bc0fe410 2024-05-17T21:55:12.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.579+0000 7f10bbfff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f10bc0fed60 0x7f10bc10de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:55:12.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.579+0000 7f10bbfff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f10bc0fed60 0x7f10bc10de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42428/0 (socket says 172.21.15.107:42428) 2024-05-17T21:55:12.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.579+0000 7f10bbfff700 1 -- 172.21.15.107:0/2772796962 learned_addr learned my addr 172.21.15.107:0/2772796962 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:55:12.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.579+0000 7f10c0e73700 1 --2- 172.21.15.107:0/2772796962 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10bc0fe410 0x7f10bc10d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:55:12.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.579+0000 7f10bbfff700 1 -- 172.21.15.107:0/2772796962 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10bc0fe410 msgr2=0x7f10bc10d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:55:12.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.579+0000 7f10bbfff700 1 --2- 172.21.15.107:0/2772796962 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10bc0fe410 0x7f10bc10d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:55:12.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.579+0000 7f10bbfff700 1 -- 172.21.15.107:0/2772796962 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10b0005040 con 0x7f10bc0fed60 2024-05-17T21:55:12.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.579+0000 7f10bbfff700 1 --2- 172.21.15.107:0/2772796962 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f10bc0fed60 0x7f10bc10de50 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f10a8002a50 tx=0x7f10a80046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:55:12.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.579+0000 7f10b97fa700 1 -- 172.21.15.107:0/2772796962 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f10a8007cc0 con 0x7f10bc0fed60 2024-05-17T21:55:12.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.580+0000 7f10b97fa700 1 -- 172.21.15.107:0/2772796962 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f10a800f040 con 0x7f10bc0fed60 2024-05-17T21:55:12.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.580+0000 7f10c30d7700 1 -- 172.21.15.107:0/2772796962 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10bc113020 con 0x7f10bc0fed60 2024-05-17T21:55:12.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.580+0000 7f10b97fa700 1 -- 172.21.15.107:0/2772796962 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f10a800da30 con 0x7f10bc0fed60 2024-05-17T21:55:12.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.580+0000 7f10c30d7700 1 -- 172.21.15.107:0/2772796962 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10bc1135b0 con 0x7f10bc0fed60 2024-05-17T21:55:12.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.581+0000 7f10c30d7700 1 -- 172.21.15.107:0/2772796962 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10bc04c710 con 0x7f10bc0fed60 2024-05-17T21:55:12.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.581+0000 7f10b97fa700 1 -- 172.21.15.107:0/2772796962 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 100184+0+0 (secure 0 0 0) 0x7f10a800db90 con 0x7f10bc0fed60 2024-05-17T21:55:12.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.582+0000 7f10b97fa700 1 --2- 172.21.15.107:0/2772796962 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f10ac078aa0 0x7f10ac07af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:55:12.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.582+0000 7f10b97fa700 1 -- 172.21.15.107:0/2772796962 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f10a809a700 con 0x7f10bc0fed60 2024-05-17T21:55:12.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.582+0000 7f10c0e73700 1 --2- 172.21.15.107:0/2772796962 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f10ac078aa0 0x7f10ac07af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:55:12.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.583+0000 7f10c0e73700 1 --2- 172.21.15.107:0/2772796962 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f10ac078aa0 0x7f10ac07af50 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f10b000b970 tx=0x7f10b0009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:55:12.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.587+0000 7f10b97fa700 1 -- 172.21.15.107:0/2772796962 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f10a8062150 con 0x7f10bc0fed60 2024-05-17T21:55:12.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.882+0000 7f10c30d7700 1 -- 172.21.15.107:0/2772796962 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f10bc1058c0 con 0x7f10bc0fed60 2024-05-17T21:55:12.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.883+0000 7f10b97fa700 1 -- 172.21.15.107:0/2772796962 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+736 (secure 0 0 0) 0x7f10a8061af0 con 0x7f10bc0fed60 2024-05-17T21:55:12.874 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-05-17T21:55:12.874 INFO:teuthology.orchestra.run.smithi107.stdout: "mon": { 2024-05-17T21:55:12.874 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev)": 2 2024-05-17T21:55:12.874 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-05-17T21:55:12.874 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr": { 2024-05-17T21:55:12.875 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev)": 2 2024-05-17T21:55:12.875 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-05-17T21:55:12.875 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": { 2024-05-17T21:55:12.875 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-952-g9903f776 (9903f7767386a9d9efadb7c1effdcd3064d71804) quincy (stable)": 6 2024-05-17T21:55:12.875 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-05-17T21:55:12.875 INFO:teuthology.orchestra.run.smithi107.stdout: "mds": { 2024-05-17T21:55:12.875 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-952-g9903f776 (9903f7767386a9d9efadb7c1effdcd3064d71804) quincy (stable)": 4 2024-05-17T21:55:12.875 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-05-17T21:55:12.876 INFO:teuthology.orchestra.run.smithi107.stdout: "overall": { 2024-05-17T21:55:12.876 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-952-g9903f776 (9903f7767386a9d9efadb7c1effdcd3064d71804) quincy (stable)": 10, 2024-05-17T21:55:12.876 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev)": 4 2024-05-17T21:55:12.876 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-05-17T21:55:12.876 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-05-17T21:55:12.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.885+0000 7f10c30d7700 1 -- 172.21.15.107:0/2772796962 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f10ac078aa0 msgr2=0x7f10ac07af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:55:12.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.885+0000 7f10c30d7700 1 --2- 172.21.15.107:0/2772796962 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f10ac078aa0 0x7f10ac07af50 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f10b000b970 tx=0x7f10b0009e30 comp rx=0 tx=0).stop 2024-05-17T21:55:12.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.885+0000 7f10c30d7700 1 -- 172.21.15.107:0/2772796962 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f10bc0fed60 msgr2=0x7f10bc10de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:55:12.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.885+0000 7f10c30d7700 1 --2- 172.21.15.107:0/2772796962 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f10bc0fed60 0x7f10bc10de50 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f10a8002a50 tx=0x7f10a80046b0 comp rx=0 tx=0).stop 2024-05-17T21:55:12.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.885+0000 7f10c30d7700 1 -- 172.21.15.107:0/2772796962 shutdown_connections 2024-05-17T21:55:12.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.885+0000 7f10c30d7700 1 --2- 172.21.15.107:0/2772796962 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f10ac078aa0 0x7f10ac07af50 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:55:12.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.885+0000 7f10c30d7700 1 --2- 172.21.15.107:0/2772796962 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10bc0fe410 0x7f10bc10d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:55:12.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.885+0000 7f10c30d7700 1 --2- 172.21.15.107:0/2772796962 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f10bc0fed60 0x7f10bc10de50 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:55:12.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.885+0000 7f10c30d7700 1 -- 172.21.15.107:0/2772796962 >> 172.21.15.107:0/2772796962 conn(0x7f10bc0f6bd0 msgr2=0x7f10bc0f8280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:55:12.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.885+0000 7f10c30d7700 1 -- 172.21.15.107:0/2772796962 shutdown_connections 2024-05-17T21:55:12.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.885+0000 7f10c30d7700 1 -- 172.21.15.107:0/2772796962 wait complete. 2024-05-17T21:55:12.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.990+0000 7f7418aa7700 1 -- 172.21.15.107:0/1435770943 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74140fed30 msgr2=0x7f74140fae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:55:12.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.990+0000 7f7418aa7700 1 --2- 172.21.15.107:0/1435770943 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74140fed30 0x7f74140fae70 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7408004940 tx=0x7f7408009e30 comp rx=0 tx=0).stop 2024-05-17T21:55:12.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.990+0000 7f7418aa7700 1 -- 172.21.15.107:0/1435770943 shutdown_connections 2024-05-17T21:55:12.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.990+0000 7f7418aa7700 1 --2- 172.21.15.107:0/1435770943 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74140fed30 0x7f74140fae70 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:55:12.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.990+0000 7f7418aa7700 1 --2- 172.21.15.107:0/1435770943 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74140fe3e0 0x7f74140fe7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:55:12.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.990+0000 7f7418aa7700 1 -- 172.21.15.107:0/1435770943 >> 172.21.15.107:0/1435770943 conn(0x7f74140f6b80 msgr2=0x7f74140f8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:55:12.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.990+0000 7f7418aa7700 1 -- 172.21.15.107:0/1435770943 shutdown_connections 2024-05-17T21:55:12.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.990+0000 7f7418aa7700 1 -- 172.21.15.107:0/1435770943 wait complete. 2024-05-17T21:55:12.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.991+0000 7f7418aa7700 1 Processor -- start 2024-05-17T21:55:12.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.991+0000 7f7418aa7700 1 -- start start 2024-05-17T21:55:12.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.991+0000 7f7418aa7700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74140fe3e0 0x7f741410da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:55:12.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.991+0000 7f7418aa7700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74140fed30 0x7f741410df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:55:12.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.991+0000 7f7418aa7700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f741410e570 con 0x7f74140fed30 2024-05-17T21:55:12.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.991+0000 7f7418aa7700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f741410ed70 con 0x7f74140fe3e0 2024-05-17T21:55:12.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.991+0000 7f7411d9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74140fed30 0x7f741410df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:55:12.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.991+0000 7f7411d9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74140fed30 0x7f741410df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42444/0 (socket says 172.21.15.107:42444) 2024-05-17T21:55:12.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.991+0000 7f7411d9b700 1 -- 172.21.15.107:0/3850954881 learned_addr learned my addr 172.21.15.107:0/3850954881 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:55:12.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.991+0000 7f741259c700 1 --2- 172.21.15.107:0/3850954881 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74140fe3e0 0x7f741410da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:55:12.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.992+0000 7f7411d9b700 1 -- 172.21.15.107:0/3850954881 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74140fe3e0 msgr2=0x7f741410da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:55:12.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.992+0000 7f7411d9b700 1 --2- 172.21.15.107:0/3850954881 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74140fe3e0 0x7f741410da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:55:12.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.992+0000 7f7411d9b700 1 -- 172.21.15.107:0/3850954881 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7408005040 con 0x7f74140fed30 2024-05-17T21:55:12.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.992+0000 7f7411d9b700 1 --2- 172.21.15.107:0/3850954881 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74140fed30 0x7f741410df70 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7408009150 tx=0x7f7408009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:55:12.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.992+0000 7f7402ffd700 1 -- 172.21.15.107:0/3850954881 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f740800e040 con 0x7f74140fed30 2024-05-17T21:55:12.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.992+0000 7f7402ffd700 1 -- 172.21.15.107:0/3850954881 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f740800bbf0 con 0x7f74140fed30 2024-05-17T21:55:12.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.992+0000 7f7402ffd700 1 -- 172.21.15.107:0/3850954881 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f740801c630 con 0x7f74140fed30 2024-05-17T21:55:12.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.992+0000 7f7418aa7700 1 -- 172.21.15.107:0/3850954881 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f741410eff0 con 0x7f74140fed30 2024-05-17T21:55:12.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.992+0000 7f7418aa7700 1 -- 172.21.15.107:0/3850954881 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f741410f530 con 0x7f74140fed30 2024-05-17T21:55:12.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.993+0000 7f7418aa7700 1 -- 172.21.15.107:0/3850954881 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f741405f5c0 con 0x7f74140fed30 2024-05-17T21:55:12.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.994+0000 7f7402ffd700 1 -- 172.21.15.107:0/3850954881 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 100184+0+0 (secure 0 0 0) 0x7f740800bd60 con 0x7f74140fed30 2024-05-17T21:55:12.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.994+0000 7f7402ffd700 1 --2- 172.21.15.107:0/3850954881 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f73fc078aa0 0x7f73fc07af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:55:12.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.994+0000 7f741259c700 1 --2- 172.21.15.107:0/3850954881 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f73fc078aa0 0x7f73fc07af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:55:12.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.994+0000 7f7402ffd700 1 -- 172.21.15.107:0/3850954881 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f740809bce0 con 0x7f74140fed30 2024-05-17T21:55:12.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.995+0000 7f741259c700 1 --2- 172.21.15.107:0/3850954881 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f73fc078aa0 0x7f73fc07af50 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f7404004f40 tx=0x7f740400b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:55:12.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:12.999+0000 7f7402ffd700 1 -- 172.21.15.107:0/3850954881 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f74080637b0 con 0x7f74140fed30 2024-05-17T21:55:13.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:13.250+0000 7f7418aa7700 1 -- 172.21.15.107:0/3850954881 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f7414105890 con 0x7f74140fed30 2024-05-17T21:55:13.241 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:13 smithi107 ceph-mon[93224]: Reconfiguring mgr.smithi107.xhhyiw (monmap changed)... 2024-05-17T21:55:13.241 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:13 smithi107 ceph-mon[93224]: Reconfiguring daemon mgr.smithi107.xhhyiw on smithi107 2024-05-17T21:55:13.241 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:13 smithi107 ceph-mon[93224]: from='client.34160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:55:13.241 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:13 smithi107 ceph-mon[93224]: from='client.34164 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:55:13.241 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:13 smithi107 ceph-mon[93224]: from='client.? 172.21.15.107:0/2772796962' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:55:13.242 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:13.250+0000 7f7402ffd700 1 -- 172.21.15.107:0/3850954881 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1910 (secure 0 0 0) 0x7f7408013e60 con 0x7f74140fed30 2024-05-17T21:55:13.242 INFO:teuthology.orchestra.run.smithi107.stdout:e15 2024-05-17T21:55:13.242 INFO:teuthology.orchestra.run.smithi107.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-17T21:55:13.242 INFO:teuthology.orchestra.run.smithi107.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-17T21:55:13.242 INFO:teuthology.orchestra.run.smithi107.stdout:legacy client fscid: 1 2024-05-17T21:55:13.242 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:55:13.242 INFO:teuthology.orchestra.run.smithi107.stdout:Filesystem 'cephfs' (1) 2024-05-17T21:55:13.243 INFO:teuthology.orchestra.run.smithi107.stdout:fs_name cephfs 2024-05-17T21:55:13.243 INFO:teuthology.orchestra.run.smithi107.stdout:epoch 15 2024-05-17T21:55:13.243 INFO:teuthology.orchestra.run.smithi107.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-17T21:55:13.243 INFO:teuthology.orchestra.run.smithi107.stdout:created 2024-05-17T21:49:16.836333+0000 2024-05-17T21:55:13.243 INFO:teuthology.orchestra.run.smithi107.stdout:modified 2024-05-17T21:49:34.873628+0000 2024-05-17T21:55:13.243 INFO:teuthology.orchestra.run.smithi107.stdout:tableserver 0 2024-05-17T21:55:13.243 INFO:teuthology.orchestra.run.smithi107.stdout:root 0 2024-05-17T21:55:13.243 INFO:teuthology.orchestra.run.smithi107.stdout:session_timeout 60 2024-05-17T21:55:13.243 INFO:teuthology.orchestra.run.smithi107.stdout:session_autoclose 300 2024-05-17T21:55:13.243 INFO:teuthology.orchestra.run.smithi107.stdout:max_file_size 1099511627776 2024-05-17T21:55:13.243 INFO:teuthology.orchestra.run.smithi107.stdout:max_xattr_size 65536 2024-05-17T21:55:13.243 INFO:teuthology.orchestra.run.smithi107.stdout:required_client_features {} 2024-05-17T21:55:13.243 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure 0 2024-05-17T21:55:13.243 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure_osd_epoch 46 2024-05-17T21:55:13.244 INFO:teuthology.orchestra.run.smithi107.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-17T21:55:13.244 INFO:teuthology.orchestra.run.smithi107.stdout:max_mds 1 2024-05-17T21:55:13.244 INFO:teuthology.orchestra.run.smithi107.stdout:in 0 2024-05-17T21:55:13.244 INFO:teuthology.orchestra.run.smithi107.stdout:up {0=24259} 2024-05-17T21:55:13.244 INFO:teuthology.orchestra.run.smithi107.stdout:failed 2024-05-17T21:55:13.244 INFO:teuthology.orchestra.run.smithi107.stdout:damaged 2024-05-17T21:55:13.244 INFO:teuthology.orchestra.run.smithi107.stdout:stopped 2024-05-17T21:55:13.244 INFO:teuthology.orchestra.run.smithi107.stdout:data_pools [3] 2024-05-17T21:55:13.244 INFO:teuthology.orchestra.run.smithi107.stdout:metadata_pool 2 2024-05-17T21:55:13.244 INFO:teuthology.orchestra.run.smithi107.stdout:inline_data enabled 2024-05-17T21:55:13.244 INFO:teuthology.orchestra.run.smithi107.stdout:balancer 2024-05-17T21:55:13.244 INFO:teuthology.orchestra.run.smithi107.stdout:bal_rank_mask -1 2024-05-17T21:55:13.244 INFO:teuthology.orchestra.run.smithi107.stdout:standby_count_wanted 1 2024-05-17T21:55:13.245 INFO:teuthology.orchestra.run.smithi107.stdout:qdb_cluster leader: 0 members: 2024-05-17T21:55:13.245 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi138.gwbtyd{0:24259} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.138:6824/2779634919,v1:172.21.15.138:6825/2779634919] compat {c=[1],r=[1],i=[7ff]}] 2024-05-17T21:55:13.245 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.drqody{0:14486} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.107:6828/2275807100,v1:172.21.15.107:6829/2275807100] compat {c=[1],r=[1],i=[7ff]}] 2024-05-17T21:55:13.245 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:55:13.245 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:55:13.245 INFO:teuthology.orchestra.run.smithi107.stdout:Standby daemons: 2024-05-17T21:55:13.245 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:55:13.245 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.idmbgj{-1:14498} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.107:6826/1460186079,v1:172.21.15.107:6827/1460186079] compat {c=[1],r=[1],i=[7ff]}] 2024-05-17T21:55:13.245 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi138.zckrvs{-1:24267} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.138:6826/170628661,v1:172.21.15.138:6827/170628661] compat {c=[1],r=[1],i=[7ff]}] 2024-05-17T21:55:13.246 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:13.252+0000 7f7418aa7700 1 -- 172.21.15.107:0/3850954881 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f73fc078aa0 msgr2=0x7f73fc07af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:55:13.246 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:13.252+0000 7f7418aa7700 1 --2- 172.21.15.107:0/3850954881 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f73fc078aa0 0x7f73fc07af50 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f7404004f40 tx=0x7f740400b040 comp rx=0 tx=0).stop 2024-05-17T21:55:13.246 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:13.252+0000 7f7418aa7700 1 -- 172.21.15.107:0/3850954881 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74140fed30 msgr2=0x7f741410df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:55:13.246 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:13.252+0000 7f7418aa7700 1 --2- 172.21.15.107:0/3850954881 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74140fed30 0x7f741410df70 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7408009150 tx=0x7f7408009e30 comp rx=0 tx=0).stop 2024-05-17T21:55:13.246 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:13.252+0000 7f7418aa7700 1 -- 172.21.15.107:0/3850954881 shutdown_connections 2024-05-17T21:55:13.246 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:13.252+0000 7f7418aa7700 1 --2- 172.21.15.107:0/3850954881 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f73fc078aa0 0x7f73fc07af50 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:55:13.246 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:13.252+0000 7f7418aa7700 1 --2- 172.21.15.107:0/3850954881 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74140fe3e0 0x7f741410da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:55:13.246 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:13.252+0000 7f7418aa7700 1 --2- 172.21.15.107:0/3850954881 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74140fed30 0x7f741410df70 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:55:13.247 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:13.252+0000 7f7418aa7700 1 -- 172.21.15.107:0/3850954881 >> 172.21.15.107:0/3850954881 conn(0x7f74140f6b80 msgr2=0x7f74140f8280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:55:13.247 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:13.252+0000 7f7418aa7700 1 -- 172.21.15.107:0/3850954881 shutdown_connections 2024-05-17T21:55:13.247 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:13.252+0000 7f7418aa7700 1 -- 172.21.15.107:0/3850954881 wait complete. 2024-05-17T21:55:13.247 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 15 2024-05-17T21:55:13.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:13 smithi138 ceph-mon[81376]: Reconfiguring mgr.smithi107.xhhyiw (monmap changed)... 2024-05-17T21:55:13.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:13 smithi138 ceph-mon[81376]: Reconfiguring daemon mgr.smithi107.xhhyiw on smithi107 2024-05-17T21:55:13.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:13 smithi138 ceph-mon[81376]: from='client.34160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:55:13.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:13 smithi138 ceph-mon[81376]: from='client.34164 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:55:13.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:13 smithi138 ceph-mon[81376]: from='client.? 172.21.15.107:0/2772796962' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:55:13.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:13.357+0000 7fd07b1b1700 1 -- 172.21.15.107:0/3189634107 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0740fbcb0 msgr2=0x7fd0740fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:55:13.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:13.357+0000 7fd07b1b1700 1 --2- 172.21.15.107:0/3189634107 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0740fbcb0 0x7fd0740fc0c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd068004940 tx=0x7fd068009e30 comp rx=0 tx=0).stop 2024-05-17T21:55:13.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:13.357+0000 7fd07b1b1700 1 -- 172.21.15.107:0/3189634107 shutdown_connections 2024-05-17T21:55:13.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:13.357+0000 7fd07b1b1700 1 --2- 172.21.15.107:0/3189634107 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd0740fcf60 0x7fd0740fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:55:13.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:13.357+0000 7fd07b1b1700 1 --2- 172.21.15.107:0/3189634107 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0740fbcb0 0x7fd0740fc0c0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:55:13.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:13.357+0000 7fd07b1b1700 1 -- 172.21.15.107:0/3189634107 >> 172.21.15.107:0/3189634107 conn(0x7fd0740f7430 msgr2=0x7fd0740f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:55:13.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:13.357+0000 7fd07b1b1700 1 -- 172.21.15.107:0/3189634107 shutdown_connections 2024-05-17T21:55:13.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:13.357+0000 7fd07b1b1700 1 -- 172.21.15.107:0/3189634107 wait complete. 2024-05-17T21:55:13.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:13.358+0000 7fd07b1b1700 1 Processor -- start 2024-05-17T21:55:13.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:13.358+0000 7fd07b1b1700 1 -- start start 2024-05-17T21:55:13.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:13.358+0000 7fd07b1b1700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd0740fbcb0 0x7fd07410daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:55:13.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:13.358+0000 7fd07b1b1700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0740fcf60 0x7fd07410dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:55:13.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:13.358+0000 7fd07b1b1700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd07410e5e0 con 0x7fd0740fcf60 2024-05-17T21:55:13.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:13.358+0000 7fd07b1b1700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd07410ede0 con 0x7fd0740fbcb0 2024-05-17T21:55:13.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:13.358+0000 7fd073fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0740fcf60 0x7fd07410dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:55:13.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:13.358+0000 7fd073fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0740fcf60 0x7fd07410dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42458/0 (socket says 172.21.15.107:42458) 2024-05-17T21:55:13.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:13.358+0000 7fd073fff700 1 -- 172.21.15.107:0/3945327154 learned_addr learned my addr 172.21.15.107:0/3945327154 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:55:13.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:13.358+0000 7fd078f4d700 1 --2- 172.21.15.107:0/3945327154 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd0740fbcb0 0x7fd07410daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:55:13.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:13.358+0000 7fd073fff700 1 -- 172.21.15.107:0/3945327154 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd0740fbcb0 msgr2=0x7fd07410daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:55:13.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:13.358+0000 7fd073fff700 1 --2- 172.21.15.107:0/3945327154 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd0740fbcb0 0x7fd07410daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:55:13.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:13.358+0000 7fd073fff700 1 -- 172.21.15.107:0/3945327154 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd068005040 con 0x7fd0740fcf60 2024-05-17T21:55:13.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:13.359+0000 7fd073fff700 1 --2- 172.21.15.107:0/3945327154 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0740fcf60 0x7fd07410dfe0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd060002a50 tx=0x7fd0600046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:55:13.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:13.359+0000 7fd0717fa700 1 -- 172.21.15.107:0/3945327154 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd060007cc0 con 0x7fd0740fcf60 2024-05-17T21:55:13.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:13.359+0000 7fd07b1b1700 1 -- 172.21.15.107:0/3945327154 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd07410f0c0 con 0x7fd0740fcf60 2024-05-17T21:55:13.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:13.359+0000 7fd0717fa700 1 -- 172.21.15.107:0/3945327154 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd06000f040 con 0x7fd0740fcf60 2024-05-17T21:55:13.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:13.359+0000 7fd0717fa700 1 -- 172.21.15.107:0/3945327154 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd06000da70 con 0x7fd0740fcf60 2024-05-17T21:55:13.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:13.359+0000 7fd07b1b1700 1 -- 172.21.15.107:0/3945327154 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd07410f600 con 0x7fd0740fcf60 2024-05-17T21:55:13.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:13.360+0000 7fd07b1b1700 1 -- 172.21.15.107:0/3945327154 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd074101af0 con 0x7fd0740fcf60 2024-05-17T21:55:13.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:13.361+0000 7fd0717fa700 1 -- 172.21.15.107:0/3945327154 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 100184+0+0 (secure 0 0 0) 0x7fd06000dc10 con 0x7fd0740fcf60 2024-05-17T21:55:13.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:13.361+0000 7fd0717fa700 1 --2- 172.21.15.107:0/3945327154 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fd064078aa0 0x7fd06407af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:55:13.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:13.361+0000 7fd0717fa700 1 -- 172.21.15.107:0/3945327154 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fd060099ff0 con 0x7fd0740fcf60 2024-05-17T21:55:13.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:13.361+0000 7fd078f4d700 1 --2- 172.21.15.107:0/3945327154 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fd064078aa0 0x7fd06407af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:55:13.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:13.362+0000 7fd078f4d700 1 --2- 172.21.15.107:0/3945327154 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fd064078aa0 0x7fd06407af50 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fd06800b3b0 tx=0x7fd068009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:55:13.356 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:13.366+0000 7fd0717fa700 1 -- 172.21.15.107:0/3945327154 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7fd060061a40 con 0x7fd0740fcf60 2024-05-17T21:55:13.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:13.624+0000 7fd07b1b1700 1 -- 172.21.15.107:0/3945327154 --> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd07410f820 con 0x7fd064078aa0 2024-05-17T21:55:13.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:13.625+0000 7fd0717fa700 1 -- 172.21.15.107:0/3945327154 <== mgr.34100 v2:172.21.15.107:6800/3205268226 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7fd07410f820 con 0x7fd064078aa0 2024-05-17T21:55:13.616 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-05-17T21:55:13.616 INFO:teuthology.orchestra.run.smithi107.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b", 2024-05-17T21:55:13.616 INFO:teuthology.orchestra.run.smithi107.stdout: "in_progress": true, 2024-05-17T21:55:13.616 INFO:teuthology.orchestra.run.smithi107.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-17T21:55:13.617 INFO:teuthology.orchestra.run.smithi107.stdout: "services_complete": [ 2024-05-17T21:55:13.617 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr", 2024-05-17T21:55:13.617 INFO:teuthology.orchestra.run.smithi107.stdout: "mon" 2024-05-17T21:55:13.617 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-05-17T21:55:13.617 INFO:teuthology.orchestra.run.smithi107.stdout: "progress": "4/21 daemons upgraded", 2024-05-17T21:55:13.617 INFO:teuthology.orchestra.run.smithi107.stdout: "message": "Currently upgrading mon daemons", 2024-05-17T21:55:13.617 INFO:teuthology.orchestra.run.smithi107.stdout: "is_paused": false 2024-05-17T21:55:13.617 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-05-17T21:55:13.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:13.627+0000 7fd07b1b1700 1 -- 172.21.15.107:0/3945327154 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fd064078aa0 msgr2=0x7fd06407af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:55:13.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:13.627+0000 7fd07b1b1700 1 --2- 172.21.15.107:0/3945327154 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fd064078aa0 0x7fd06407af50 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fd06800b3b0 tx=0x7fd068009e30 comp rx=0 tx=0).stop 2024-05-17T21:55:13.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:13.627+0000 7fd07b1b1700 1 -- 172.21.15.107:0/3945327154 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0740fcf60 msgr2=0x7fd07410dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:55:13.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:13.627+0000 7fd07b1b1700 1 --2- 172.21.15.107:0/3945327154 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0740fcf60 0x7fd07410dfe0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd060002a50 tx=0x7fd0600046b0 comp rx=0 tx=0).stop 2024-05-17T21:55:13.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:13.628+0000 7fd07b1b1700 1 -- 172.21.15.107:0/3945327154 shutdown_connections 2024-05-17T21:55:13.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:13.628+0000 7fd07b1b1700 1 --2- 172.21.15.107:0/3945327154 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fd064078aa0 0x7fd06407af50 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:55:13.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:13.628+0000 7fd07b1b1700 1 --2- 172.21.15.107:0/3945327154 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd0740fbcb0 0x7fd07410daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:55:13.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:13.628+0000 7fd07b1b1700 1 --2- 172.21.15.107:0/3945327154 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0740fcf60 0x7fd07410dfe0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:55:13.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:13.628+0000 7fd07b1b1700 1 -- 172.21.15.107:0/3945327154 >> 172.21.15.107:0/3945327154 conn(0x7fd0740f7430 msgr2=0x7fd074065b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:55:13.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:13.628+0000 7fd07b1b1700 1 -- 172.21.15.107:0/3945327154 shutdown_connections 2024-05-17T21:55:13.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:13.628+0000 7fd07b1b1700 1 -- 172.21.15.107:0/3945327154 wait complete. 2024-05-17T21:55:13.725 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:13.734+0000 7fd432cc2700 1 -- 172.21.15.107:0/3305776045 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd42c0fe080 msgr2=0x7fd42c0fe4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:55:13.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:13.734+0000 7fd432cc2700 1 --2- 172.21.15.107:0/3305776045 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd42c0fe080 0x7fd42c0fe4f0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fd418004880 tx=0x7fd418009e30 comp rx=0 tx=0).stop 2024-05-17T21:55:13.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:13.734+0000 7fd432cc2700 1 -- 172.21.15.107:0/3305776045 shutdown_connections 2024-05-17T21:55:13.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:13.734+0000 7fd432cc2700 1 --2- 172.21.15.107:0/3305776045 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd42c0fe080 0x7fd42c0fe4f0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:55:13.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:13.734+0000 7fd432cc2700 1 --2- 172.21.15.107:0/3305776045 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd42c0fcf10 0x7fd42c0fd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:55:13.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:13.734+0000 7fd432cc2700 1 -- 172.21.15.107:0/3305776045 >> 172.21.15.107:0/3305776045 conn(0x7fd42c0f8560 msgr2=0x7fd42c0fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:55:13.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:13.734+0000 7fd432cc2700 1 -- 172.21.15.107:0/3305776045 shutdown_connections 2024-05-17T21:55:13.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:13.734+0000 7fd432cc2700 1 -- 172.21.15.107:0/3305776045 wait complete. 2024-05-17T21:55:13.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:13.735+0000 7fd432cc2700 1 Processor -- start 2024-05-17T21:55:13.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:13.735+0000 7fd432cc2700 1 -- start start 2024-05-17T21:55:13.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:13.735+0000 7fd432cc2700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd42c0fcf10 0x7fd42c10da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:55:13.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:13.735+0000 7fd432cc2700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd42c0fe080 0x7fd42c10df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:55:13.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:13.735+0000 7fd432cc2700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd42c10e550 con 0x7fd42c0fe080 2024-05-17T21:55:13.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:13.735+0000 7fd432cc2700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd42c10eda0 con 0x7fd42c0fcf10 2024-05-17T21:55:13.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:13.735+0000 7fd42bfff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd42c0fe080 0x7fd42c10df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:55:13.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:13.735+0000 7fd42bfff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd42c0fe080 0x7fd42c10df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42466/0 (socket says 172.21.15.107:42466) 2024-05-17T21:55:13.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:13.735+0000 7fd42bfff700 1 -- 172.21.15.107:0/2261763558 learned_addr learned my addr 172.21.15.107:0/2261763558 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:55:13.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:13.736+0000 7fd430a5e700 1 --2- 172.21.15.107:0/2261763558 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd42c0fcf10 0x7fd42c10da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:55:13.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:13.736+0000 7fd42bfff700 1 -- 172.21.15.107:0/2261763558 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd42c0fcf10 msgr2=0x7fd42c10da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:55:13.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:13.736+0000 7fd42bfff700 1 --2- 172.21.15.107:0/2261763558 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd42c0fcf10 0x7fd42c10da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:55:13.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:13.736+0000 7fd42bfff700 1 -- 172.21.15.107:0/2261763558 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd418005040 con 0x7fd42c0fe080 2024-05-17T21:55:13.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:13.736+0000 7fd42bfff700 1 --2- 172.21.15.107:0/2261763558 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd42c0fe080 0x7fd42c10df50 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd4180138f0 tx=0x7fd418009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:55:13.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:13.736+0000 7fd4297fa700 1 -- 172.21.15.107:0/2261763558 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd41800e030 con 0x7fd42c0fe080 2024-05-17T21:55:13.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:13.736+0000 7fd432cc2700 1 -- 172.21.15.107:0/2261763558 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd42c10f020 con 0x7fd42c0fe080 2024-05-17T21:55:13.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:13.736+0000 7fd4297fa700 1 -- 172.21.15.107:0/2261763558 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd418013e60 con 0x7fd42c0fe080 2024-05-17T21:55:13.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:13.736+0000 7fd4297fa700 1 -- 172.21.15.107:0/2261763558 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd4180123c0 con 0x7fd42c0fe080 2024-05-17T21:55:13.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:13.736+0000 7fd432cc2700 1 -- 172.21.15.107:0/2261763558 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd42c10f560 con 0x7fd42c0fe080 2024-05-17T21:55:13.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:13.737+0000 7fd432cc2700 1 -- 172.21.15.107:0/2261763558 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd42c05f5c0 con 0x7fd42c0fe080 2024-05-17T21:55:13.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:13.738+0000 7fd4297fa700 1 -- 172.21.15.107:0/2261763558 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 100184+0+0 (secure 0 0 0) 0x7fd41801c400 con 0x7fd42c0fe080 2024-05-17T21:55:13.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:13.739+0000 7fd4297fa700 1 --2- 172.21.15.107:0/2261763558 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fd41c078aa0 0x7fd41c07af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:55:13.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:13.739+0000 7fd4297fa700 1 -- 172.21.15.107:0/2261763558 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fd41809c0c0 con 0x7fd42c0fe080 2024-05-17T21:55:13.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:13.739+0000 7fd430a5e700 1 --2- 172.21.15.107:0/2261763558 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fd41c078aa0 0x7fd41c07af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:55:13.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:13.739+0000 7fd430a5e700 1 --2- 172.21.15.107:0/2261763558 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fd41c078aa0 0x7fd41c07af50 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fd4200046c0 tx=0x7fd42000b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:55:13.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:13.743+0000 7fd4297fa700 1 -- 172.21.15.107:0/2261763558 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7fd418063b90 con 0x7fd42c0fe080 2024-05-17T21:55:14.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:14.059+0000 7fd432cc2700 1 -- 172.21.15.107:0/2261763558 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fd42c04bed0 con 0x7fd42c0fe080 2024-05-17T21:55:14.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:14.060+0000 7fd4297fa700 1 -- 172.21.15.107:0/2261763558 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7fd418063530 con 0x7fd42c0fe080 2024-05-17T21:55:14.051 INFO:teuthology.orchestra.run.smithi107.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-17T21:55:14.051 INFO:teuthology.orchestra.run.smithi107.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-17T21:55:14.051 INFO:teuthology.orchestra.run.smithi107.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-17T21:55:14.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:14.062+0000 7fd432cc2700 1 -- 172.21.15.107:0/2261763558 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fd41c078aa0 msgr2=0x7fd41c07af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:55:14.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:14.062+0000 7fd432cc2700 1 --2- 172.21.15.107:0/2261763558 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fd41c078aa0 0x7fd41c07af50 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fd4200046c0 tx=0x7fd42000b040 comp rx=0 tx=0).stop 2024-05-17T21:55:14.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:14.062+0000 7fd432cc2700 1 -- 172.21.15.107:0/2261763558 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd42c0fe080 msgr2=0x7fd42c10df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:55:14.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:14.062+0000 7fd432cc2700 1 --2- 172.21.15.107:0/2261763558 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd42c0fe080 0x7fd42c10df50 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd4180138f0 tx=0x7fd418009e30 comp rx=0 tx=0).stop 2024-05-17T21:55:14.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:14.063+0000 7fd432cc2700 1 -- 172.21.15.107:0/2261763558 shutdown_connections 2024-05-17T21:55:14.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:14.063+0000 7fd432cc2700 1 --2- 172.21.15.107:0/2261763558 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fd41c078aa0 0x7fd41c07af50 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:55:14.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:14.063+0000 7fd432cc2700 1 --2- 172.21.15.107:0/2261763558 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd42c0fcf10 0x7fd42c10da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:55:14.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:14.063+0000 7fd432cc2700 1 --2- 172.21.15.107:0/2261763558 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd42c0fe080 0x7fd42c10df50 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:55:14.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:14.063+0000 7fd432cc2700 1 -- 172.21.15.107:0/2261763558 >> 172.21.15.107:0/2261763558 conn(0x7fd42c0f8560 msgr2=0x7fd42c0f9c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:55:14.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:14.063+0000 7fd432cc2700 1 -- 172.21.15.107:0/2261763558 shutdown_connections 2024-05-17T21:55:14.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:14.063+0000 7fd432cc2700 1 -- 172.21.15.107:0/2261763558 wait complete. 2024-05-17T21:55:14.322 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:14 smithi107 ceph-mon[93224]: pgmap v22: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T21:55:14.322 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:14 smithi107 ceph-mon[93224]: from='client.34168 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:55:14.322 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:14 smithi107 ceph-mon[93224]: from='client.? 172.21.15.107:0/3850954881' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-17T21:55:14.322 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:14 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:14.322 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:14 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:14.322 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:14 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi107", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-17T21:55:14.322 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:14 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:55:14.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:14 smithi138 ceph-mon[81376]: pgmap v22: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T21:55:14.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:14 smithi138 ceph-mon[81376]: from='client.34168 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:55:14.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:14 smithi138 ceph-mon[81376]: from='client.? 172.21.15.107:0/3850954881' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-17T21:55:14.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:14 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:14.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:14 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:14.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:14 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi107", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-17T21:55:14.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:14 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:55:15.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:15 smithi138 ceph-mon[81376]: Reconfiguring crash.smithi107 (monmap changed)... 2024-05-17T21:55:15.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:15 smithi138 ceph-mon[81376]: Reconfiguring daemon crash.smithi107 on smithi107 2024-05-17T21:55:15.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:15 smithi138 ceph-mon[81376]: from='client.34180 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:55:15.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:15 smithi138 ceph-mon[81376]: from='client.? 172.21.15.107:0/2261763558' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-17T21:55:15.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:15 smithi107 ceph-mon[93224]: Reconfiguring crash.smithi107 (monmap changed)... 2024-05-17T21:55:15.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:15 smithi107 ceph-mon[93224]: Reconfiguring daemon crash.smithi107 on smithi107 2024-05-17T21:55:15.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:15 smithi107 ceph-mon[93224]: from='client.34180 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:55:15.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:15 smithi107 ceph-mon[93224]: from='client.? 172.21.15.107:0/2261763558' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-17T21:55:16.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:16 smithi138 ceph-mon[81376]: pgmap v23: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T21:55:16.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:16 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:16.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:16 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:16.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:16 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-17T21:55:16.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:16 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:55:16.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:16 smithi107 ceph-mon[93224]: pgmap v23: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T21:55:16.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:16 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:16.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:16 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:16.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:16 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-17T21:55:16.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:16 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:55:17.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:17 smithi138 ceph-mon[81376]: Reconfiguring osd.0 (monmap changed)... 2024-05-17T21:55:17.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:17 smithi138 ceph-mon[81376]: Reconfiguring daemon osd.0 on smithi107 2024-05-17T21:55:17.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:17 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:17.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:17 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:17.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:17 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-17T21:55:17.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:17 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:55:17.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:17 smithi107 ceph-mon[93224]: Reconfiguring osd.0 (monmap changed)... 2024-05-17T21:55:17.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:17 smithi107 ceph-mon[93224]: Reconfiguring daemon osd.0 on smithi107 2024-05-17T21:55:17.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:17 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:17.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:17 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:17.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:17 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-17T21:55:17.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:17 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:55:18.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:18 smithi138 ceph-mon[81376]: pgmap v24: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T21:55:18.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:18 smithi138 ceph-mon[81376]: Reconfiguring osd.1 (monmap changed)... 2024-05-17T21:55:18.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:18 smithi138 ceph-mon[81376]: Reconfiguring daemon osd.1 on smithi107 2024-05-17T21:55:18.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:18 smithi107 ceph-mon[93224]: pgmap v24: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T21:55:18.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:18 smithi107 ceph-mon[93224]: Reconfiguring osd.1 (monmap changed)... 2024-05-17T21:55:18.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:18 smithi107 ceph-mon[93224]: Reconfiguring daemon osd.1 on smithi107 2024-05-17T21:55:19.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:19 smithi138 ceph-mon[81376]: pgmap v25: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T21:55:19.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:19 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:19.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:19 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:19.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:19 smithi138 ceph-mon[81376]: Reconfiguring osd.2 (monmap changed)... 2024-05-17T21:55:19.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:19 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-17T21:55:19.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:19 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:55:19.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:19 smithi138 ceph-mon[81376]: Reconfiguring daemon osd.2 on smithi107 2024-05-17T21:55:19.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:19 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:55:19.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:19 smithi107 ceph-mon[93224]: pgmap v25: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T21:55:19.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:19 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:19.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:19 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:19.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:19 smithi107 ceph-mon[93224]: Reconfiguring osd.2 (monmap changed)... 2024-05-17T21:55:19.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:19 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-17T21:55:19.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:19 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:55:19.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:19 smithi107 ceph-mon[93224]: Reconfiguring daemon osd.2 on smithi107 2024-05-17T21:55:19.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:19 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:55:21.376 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:21 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:21.376 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:21 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:21.376 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:21 smithi107 ceph-mon[93224]: Reconfiguring mds.cephfs.smithi107.idmbgj (monmap changed)... 2024-05-17T21:55:21.376 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:21 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi107.idmbgj", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T21:55:21.376 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:21 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:55:21.376 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:21 smithi107 ceph-mon[93224]: Reconfiguring daemon mds.cephfs.smithi107.idmbgj on smithi107 2024-05-17T21:55:21.376 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:21 smithi107 ceph-mon[93224]: pgmap v26: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T21:55:21.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:21 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:21.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:21 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:21.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:21 smithi138 ceph-mon[81376]: Reconfiguring mds.cephfs.smithi107.idmbgj (monmap changed)... 2024-05-17T21:55:21.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:21 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi107.idmbgj", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T21:55:21.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:21 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:55:21.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:21 smithi138 ceph-mon[81376]: Reconfiguring daemon mds.cephfs.smithi107.idmbgj on smithi107 2024-05-17T21:55:21.577 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:21 smithi138 ceph-mon[81376]: pgmap v26: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T21:55:22.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:22 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:22.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:22 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:22.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:22 smithi107 ceph-mon[93224]: Reconfiguring mds.cephfs.smithi107.drqody (monmap changed)... 2024-05-17T21:55:22.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:22 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi107.drqody", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T21:55:22.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:22 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:55:22.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:22 smithi107 ceph-mon[93224]: Reconfiguring daemon mds.cephfs.smithi107.drqody on smithi107 2024-05-17T21:55:23.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:22 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:23.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:22 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:23.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:22 smithi138 ceph-mon[81376]: Reconfiguring mds.cephfs.smithi107.drqody (monmap changed)... 2024-05-17T21:55:23.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:22 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi107.drqody", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T21:55:23.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:22 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:55:23.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:22 smithi138 ceph-mon[81376]: Reconfiguring daemon mds.cephfs.smithi107.drqody on smithi107 2024-05-17T21:55:23.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:23 smithi107 ceph-mon[93224]: pgmap v27: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T21:55:23.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:23 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:23.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:23 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:23.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:23 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi138", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-17T21:55:23.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:23 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:55:23.951 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:23 smithi138 ceph-mon[81376]: pgmap v27: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T21:55:23.952 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:23 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:23.952 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:23 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:23.952 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:23 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi138", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-17T21:55:23.952 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:23 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:55:24.813 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:24 smithi138 ceph-mon[81376]: Reconfiguring crash.smithi138 (monmap changed)... 2024-05-17T21:55:24.813 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:24 smithi138 ceph-mon[81376]: Reconfiguring daemon crash.smithi138 on smithi138 2024-05-17T21:55:24.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:24 smithi107 ceph-mon[93224]: Reconfiguring crash.smithi138 (monmap changed)... 2024-05-17T21:55:24.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:24 smithi107 ceph-mon[93224]: Reconfiguring daemon crash.smithi138 on smithi138 2024-05-17T21:55:26.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:25 smithi138 ceph-mon[81376]: pgmap v28: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T21:55:26.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:25 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:26.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:25 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:26.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:25 smithi138 ceph-mon[81376]: Reconfiguring mgr.smithi138.napnll (monmap changed)... 2024-05-17T21:55:26.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:25 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi138.napnll", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-17T21:55:26.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:25 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-17T21:55:26.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:25 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:55:26.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:25 smithi138 ceph-mon[81376]: Reconfiguring daemon mgr.smithi138.napnll on smithi138 2024-05-17T21:55:26.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:25 smithi107 ceph-mon[93224]: pgmap v28: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T21:55:26.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:25 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:26.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:25 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:26.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:25 smithi107 ceph-mon[93224]: Reconfiguring mgr.smithi138.napnll (monmap changed)... 2024-05-17T21:55:26.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:25 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi138.napnll", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-17T21:55:26.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:25 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-17T21:55:26.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:25 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:55:26.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:25 smithi107 ceph-mon[93224]: Reconfiguring daemon mgr.smithi138.napnll on smithi138 2024-05-17T21:55:27.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:27 smithi138 ceph-mon[81376]: pgmap v29: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T21:55:27.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:27 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:27.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:27 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:27.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:27 smithi138 ceph-mon[81376]: Reconfiguring mon.smithi138 (monmap changed)... 2024-05-17T21:55:27.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:27 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-17T21:55:27.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:27 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-17T21:55:27.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:27 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:55:27.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:27 smithi138 ceph-mon[81376]: Reconfiguring daemon mon.smithi138 on smithi138 2024-05-17T21:55:27.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:27 smithi107 ceph-mon[93224]: pgmap v29: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T21:55:27.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:27 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:27.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:27 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:27.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:27 smithi107 ceph-mon[93224]: Reconfiguring mon.smithi138 (monmap changed)... 2024-05-17T21:55:27.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:27 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-17T21:55:27.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:27 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-17T21:55:27.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:27 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:55:27.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:27 smithi107 ceph-mon[93224]: Reconfiguring daemon mon.smithi138 on smithi138 2024-05-17T21:55:29.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:29 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:29.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:29 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:29.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:29 smithi107 ceph-mon[93224]: Reconfiguring osd.3 (monmap changed)... 2024-05-17T21:55:29.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:29 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-17T21:55:29.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:29 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:55:29.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:29 smithi107 ceph-mon[93224]: Reconfiguring daemon osd.3 on smithi138 2024-05-17T21:55:29.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:29 smithi107 ceph-mon[93224]: pgmap v30: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-05-17T21:55:29.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:29 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:29.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:29 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:29.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:29 smithi138 ceph-mon[81376]: Reconfiguring osd.3 (monmap changed)... 2024-05-17T21:55:29.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:29 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-17T21:55:29.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:29 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:55:29.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:29 smithi138 ceph-mon[81376]: Reconfiguring daemon osd.3 on smithi138 2024-05-17T21:55:29.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:29 smithi138 ceph-mon[81376]: pgmap v30: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-05-17T21:55:31.291 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:30 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:31.291 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:30 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:31.291 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:30 smithi138 ceph-mon[81376]: Reconfiguring osd.4 (monmap changed)... 2024-05-17T21:55:31.292 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:30 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-17T21:55:31.292 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:30 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:55:31.292 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:30 smithi138 ceph-mon[81376]: Reconfiguring daemon osd.4 on smithi138 2024-05-17T21:55:31.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:30 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:31.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:30 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:31.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:30 smithi107 ceph-mon[93224]: Reconfiguring osd.4 (monmap changed)... 2024-05-17T21:55:31.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:30 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-17T21:55:31.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:30 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:55:31.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:30 smithi107 ceph-mon[93224]: Reconfiguring daemon osd.4 on smithi138 2024-05-17T21:55:32.271 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:31 smithi138 ceph-mon[81376]: pgmap v31: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T21:55:32.271 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:31 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:32.271 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:31 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:32.271 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:31 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-17T21:55:32.271 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:31 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:55:32.304 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:31 smithi107 ceph-mon[93224]: pgmap v31: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T21:55:32.305 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:31 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:32.305 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:31 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:32.305 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:31 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-17T21:55:32.305 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:31 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:55:33.283 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:32 smithi138 ceph-mon[81376]: Reconfiguring osd.5 (monmap changed)... 2024-05-17T21:55:33.284 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:32 smithi138 ceph-mon[81376]: Reconfiguring daemon osd.5 on smithi138 2024-05-17T21:55:33.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:32 smithi107 ceph-mon[93224]: Reconfiguring osd.5 (monmap changed)... 2024-05-17T21:55:33.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:32 smithi107 ceph-mon[93224]: Reconfiguring daemon osd.5 on smithi138 2024-05-17T21:55:34.196 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:33 smithi138 ceph-mon[81376]: pgmap v32: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-05-17T21:55:34.196 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:33 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:34.196 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:33 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:34.196 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:33 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi138.gwbtyd", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T21:55:34.196 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:33 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:55:34.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:33 smithi107 ceph-mon[93224]: pgmap v32: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-05-17T21:55:34.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:33 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:34.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:33 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:34.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:33 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi138.gwbtyd", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T21:55:34.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:33 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:55:35.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:34 smithi138 ceph-mon[81376]: Reconfiguring mds.cephfs.smithi138.gwbtyd (monmap changed)... 2024-05-17T21:55:35.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:34 smithi138 ceph-mon[81376]: Reconfiguring daemon mds.cephfs.smithi138.gwbtyd on smithi138 2024-05-17T21:55:35.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:34 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:55:35.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:34 smithi107 ceph-mon[93224]: Reconfiguring mds.cephfs.smithi138.gwbtyd (monmap changed)... 2024-05-17T21:55:35.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:34 smithi107 ceph-mon[93224]: Reconfiguring daemon mds.cephfs.smithi138.gwbtyd on smithi138 2024-05-17T21:55:35.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:34 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:55:36.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:35 smithi138 ceph-mon[81376]: pgmap v33: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-05-17T21:55:36.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:35 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:36.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:35 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:36.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:35 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi138.zckrvs", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T21:55:36.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:35 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:55:36.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:35 smithi107 ceph-mon[93224]: pgmap v33: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-05-17T21:55:36.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:35 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:36.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:35 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:36.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:35 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi138.zckrvs", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T21:55:36.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:35 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:55:37.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:36 smithi138 ceph-mon[81376]: Reconfiguring mds.cephfs.smithi138.zckrvs (monmap changed)... 2024-05-17T21:55:37.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:36 smithi138 ceph-mon[81376]: Reconfiguring daemon mds.cephfs.smithi138.zckrvs on smithi138 2024-05-17T21:55:37.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:36 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:37.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:36 smithi107 ceph-mon[93224]: Reconfiguring mds.cephfs.smithi138.zckrvs (monmap changed)... 2024-05-17T21:55:37.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:36 smithi107 ceph-mon[93224]: Reconfiguring daemon mds.cephfs.smithi138.zckrvs on smithi138 2024-05-17T21:55:37.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:36 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:38.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:37 smithi107 ceph-mon[93224]: pgmap v34: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T21:55:38.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:37 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:38.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:37 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:55:38.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:37 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:55:38.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:37 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:55:38.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:37 smithi107 ceph-mon[93224]: Upgrade: Setting container_image for all mon 2024-05-17T21:55:38.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:37 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:38.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:37 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi107"}]: dispatch 2024-05-17T21:55:38.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:37 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi107"}]': finished 2024-05-17T21:55:38.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:37 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi138"}]: dispatch 2024-05-17T21:55:38.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:37 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi138"}]': finished 2024-05-17T21:55:38.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:37 smithi138 ceph-mon[81376]: pgmap v34: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T21:55:38.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:37 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:38.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:37 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:55:38.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:37 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:55:38.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:37 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:55:38.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:37 smithi138 ceph-mon[81376]: Upgrade: Setting container_image for all mon 2024-05-17T21:55:38.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:37 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:38.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:37 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi107"}]: dispatch 2024-05-17T21:55:38.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:37 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi107"}]': finished 2024-05-17T21:55:38.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:37 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi138"}]: dispatch 2024-05-17T21:55:38.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:37 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi138"}]': finished 2024-05-17T21:55:40.055 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:39 smithi107 ceph-mon[93224]: pgmap v35: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T21:55:40.055 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:39 smithi107 ceph-mon[93224]: Upgrade: Updating crash.smithi107 (1/2) 2024-05-17T21:55:40.055 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:39 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:40.055 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:39 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi107", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-17T21:55:40.055 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:39 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:55:40.055 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:39 smithi107 ceph-mon[93224]: Deploying daemon crash.smithi107 on smithi107 2024-05-17T21:55:40.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:39 smithi138 ceph-mon[81376]: pgmap v35: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T21:55:40.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:39 smithi138 ceph-mon[81376]: Upgrade: Updating crash.smithi107 (1/2) 2024-05-17T21:55:40.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:39 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:40.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:39 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi107", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-17T21:55:40.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:39 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:55:40.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:39 smithi138 ceph-mon[81376]: Deploying daemon crash.smithi107 on smithi107 2024-05-17T21:55:42.045 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:41 smithi107 ceph-mon[93224]: pgmap v36: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T21:55:42.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:41 smithi138 ceph-mon[81376]: pgmap v36: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T21:55:43.702 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:43 smithi107 ceph-mon[93224]: pgmap v37: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T21:55:43.702 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:43 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:43.702 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:43 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:43.754 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:43 smithi138 ceph-mon[81376]: pgmap v37: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T21:55:43.754 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:43 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:43.754 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:43 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:44.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.174+0000 7f2bb5ee6700 1 -- 172.21.15.107:0/806846841 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2bb00fbaf0 msgr2=0x7f2bb00fe350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:55:44.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.174+0000 7f2bb5ee6700 1 --2- 172.21.15.107:0/806846841 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2bb00fbaf0 0x7f2bb00fe350 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2b98004940 tx=0x7f2b98009e30 comp rx=0 tx=0).stop 2024-05-17T21:55:44.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.174+0000 7f2bb5ee6700 1 -- 172.21.15.107:0/806846841 shutdown_connections 2024-05-17T21:55:44.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.174+0000 7f2bb5ee6700 1 --2- 172.21.15.107:0/806846841 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2bb00fbaf0 0x7f2bb00fe350 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:55:44.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.174+0000 7f2bb5ee6700 1 --2- 172.21.15.107:0/806846841 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2bb00fb1a0 0x7f2bb00fb5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:55:44.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.174+0000 7f2bb5ee6700 1 -- 172.21.15.107:0/806846841 >> 172.21.15.107:0/806846841 conn(0x7f2bb00f6b80 msgr2=0x7f2bb00f8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:55:44.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.174+0000 7f2bb5ee6700 1 -- 172.21.15.107:0/806846841 shutdown_connections 2024-05-17T21:55:44.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.174+0000 7f2bb5ee6700 1 -- 172.21.15.107:0/806846841 wait complete. 2024-05-17T21:55:44.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.174+0000 7f2bb5ee6700 1 Processor -- start 2024-05-17T21:55:44.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.175+0000 7f2bb5ee6700 1 -- start start 2024-05-17T21:55:44.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.175+0000 7f2bb5ee6700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2bb00fb1a0 0x7f2bb00fdcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:55:44.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.175+0000 7f2bb5ee6700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2bb00fbaf0 0x7f2bb00fc180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:55:44.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.175+0000 7f2bb5ee6700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb00fc730 con 0x7f2bb00fbaf0 2024-05-17T21:55:44.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.175+0000 7f2bb5ee6700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb00fc8a0 con 0x7f2bb00fb1a0 2024-05-17T21:55:44.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.175+0000 7f2baeffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2bb00fbaf0 0x7f2bb00fc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:55:44.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.175+0000 7f2baf7fe700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2bb00fb1a0 0x7f2bb00fdcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:55:44.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.175+0000 7f2baeffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2bb00fbaf0 0x7f2bb00fc180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34604/0 (socket says 172.21.15.107:34604) 2024-05-17T21:55:44.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.175+0000 7f2baf7fe700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2bb00fb1a0 0x7f2bb00fdcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.107:42966/0 (socket says 172.21.15.107:42966) 2024-05-17T21:55:44.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.175+0000 7f2baeffd700 1 -- 172.21.15.107:0/3602135898 learned_addr learned my addr 172.21.15.107:0/3602135898 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:55:44.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.175+0000 7f2baeffd700 1 -- 172.21.15.107:0/3602135898 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2bb00fb1a0 msgr2=0x7f2bb00fdcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:55:44.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.175+0000 7f2baeffd700 1 --2- 172.21.15.107:0/3602135898 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2bb00fb1a0 0x7f2bb00fdcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:55:44.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.175+0000 7f2baeffd700 1 -- 172.21.15.107:0/3602135898 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b98005040 con 0x7f2bb00fbaf0 2024-05-17T21:55:44.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.175+0000 7f2baf7fe700 1 --2- 172.21.15.107:0/3602135898 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2bb00fb1a0 0x7f2bb00fdcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T21:55:44.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.175+0000 7f2baeffd700 1 --2- 172.21.15.107:0/3602135898 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2bb00fbaf0 0x7f2bb00fc180 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2b98013870 tx=0x7f2b98009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:55:44.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.176+0000 7f2ba7fff700 1 -- 172.21.15.107:0/3602135898 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2b98021070 con 0x7f2bb00fbaf0 2024-05-17T21:55:44.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.176+0000 7f2bb5ee6700 1 -- 172.21.15.107:0/3602135898 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bb00fcb20 con 0x7f2bb00fbaf0 2024-05-17T21:55:44.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.176+0000 7f2ba7fff700 1 -- 172.21.15.107:0/3602135898 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2b98009430 con 0x7f2bb00fbaf0 2024-05-17T21:55:44.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.176+0000 7f2bb5ee6700 1 -- 172.21.15.107:0/3602135898 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bb0115bb0 con 0x7f2bb00fbaf0 2024-05-17T21:55:44.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.176+0000 7f2ba7fff700 1 -- 172.21.15.107:0/3602135898 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2b9800e070 con 0x7f2bb00fbaf0 2024-05-17T21:55:44.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.177+0000 7f2bb5ee6700 1 -- 172.21.15.107:0/3602135898 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bb005f5c0 con 0x7f2bb00fbaf0 2024-05-17T21:55:44.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.178+0000 7f2ba7fff700 1 -- 172.21.15.107:0/3602135898 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 100184+0+0 (secure 0 0 0) 0x7f2b9801c750 con 0x7f2bb00fbaf0 2024-05-17T21:55:44.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.178+0000 7f2ba7fff700 1 --2- 172.21.15.107:0/3602135898 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f2b9c078aa0 0x7f2b9c07af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:55:44.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.178+0000 7f2ba7fff700 1 -- 172.21.15.107:0/3602135898 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f2b9809bb80 con 0x7f2bb00fbaf0 2024-05-17T21:55:44.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.178+0000 7f2baf7fe700 1 --2- 172.21.15.107:0/3602135898 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f2b9c078aa0 0x7f2b9c07af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:55:44.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.179+0000 7f2baf7fe700 1 --2- 172.21.15.107:0/3602135898 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f2b9c078aa0 0x7f2b9c07af50 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f2bb00fda40 tx=0x7f2ba000b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:55:44.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.183+0000 7f2ba7fff700 1 -- 172.21.15.107:0/3602135898 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f2b980635b0 con 0x7f2bb00fbaf0 2024-05-17T21:55:44.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.414+0000 7f2bb5ee6700 1 -- 172.21.15.107:0/3602135898 --> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2bb00008d0 con 0x7f2b9c078aa0 2024-05-17T21:55:44.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.416+0000 7f2ba7fff700 1 -- 172.21.15.107:0/3602135898 <== mgr.34100 v2:172.21.15.107:6800/3205268226 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+385 (secure 0 0 0) 0x7f2bb00008d0 con 0x7f2b9c078aa0 2024-05-17T21:55:44.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.418+0000 7f2bb5ee6700 1 -- 172.21.15.107:0/3602135898 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f2b9c078aa0 msgr2=0x7f2b9c07af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:55:44.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.418+0000 7f2bb5ee6700 1 --2- 172.21.15.107:0/3602135898 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f2b9c078aa0 0x7f2b9c07af50 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f2bb00fda40 tx=0x7f2ba000b040 comp rx=0 tx=0).stop 2024-05-17T21:55:44.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.418+0000 7f2bb5ee6700 1 -- 172.21.15.107:0/3602135898 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2bb00fbaf0 msgr2=0x7f2bb00fc180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:55:44.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.418+0000 7f2bb5ee6700 1 --2- 172.21.15.107:0/3602135898 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2bb00fbaf0 0x7f2bb00fc180 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2b98013870 tx=0x7f2b98009e30 comp rx=0 tx=0).stop 2024-05-17T21:55:44.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.418+0000 7f2bb5ee6700 1 -- 172.21.15.107:0/3602135898 shutdown_connections 2024-05-17T21:55:44.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.418+0000 7f2bb5ee6700 1 --2- 172.21.15.107:0/3602135898 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f2b9c078aa0 0x7f2b9c07af50 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:55:44.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.418+0000 7f2bb5ee6700 1 --2- 172.21.15.107:0/3602135898 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2bb00fb1a0 0x7f2bb00fdcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:55:44.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.418+0000 7f2bb5ee6700 1 --2- 172.21.15.107:0/3602135898 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2bb00fbaf0 0x7f2bb00fc180 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:55:44.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.418+0000 7f2bb5ee6700 1 -- 172.21.15.107:0/3602135898 >> 172.21.15.107:0/3602135898 conn(0x7f2bb00f6b80 msgr2=0x7f2bb00f8180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:55:44.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.418+0000 7f2bb5ee6700 1 -- 172.21.15.107:0/3602135898 shutdown_connections 2024-05-17T21:55:44.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.418+0000 7f2bb5ee6700 1 -- 172.21.15.107:0/3602135898 wait complete. 2024-05-17T21:55:44.420 INFO:teuthology.orchestra.run.smithi107.stdout:true 2024-05-17T21:55:44.519 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.526+0000 7f4061aee700 1 -- 172.21.15.107:0/3475129242 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f405c0fe080 msgr2=0x7f405c0fe4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:55:44.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.526+0000 7f4061aee700 1 --2- 172.21.15.107:0/3475129242 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f405c0fe080 0x7f405c0fe4f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f404c004940 tx=0x7f404c009e30 comp rx=0 tx=0).stop 2024-05-17T21:55:44.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.526+0000 7f4061aee700 1 -- 172.21.15.107:0/3475129242 shutdown_connections 2024-05-17T21:55:44.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.526+0000 7f4061aee700 1 --2- 172.21.15.107:0/3475129242 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f405c0fe080 0x7f405c0fe4f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:55:44.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.526+0000 7f4061aee700 1 --2- 172.21.15.107:0/3475129242 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f405c0fce80 0x7f405c0fd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:55:44.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.526+0000 7f4061aee700 1 -- 172.21.15.107:0/3475129242 >> 172.21.15.107:0/3475129242 conn(0x7f405c0f8590 msgr2=0x7f405c0fa9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:55:44.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.526+0000 7f4061aee700 1 -- 172.21.15.107:0/3475129242 shutdown_connections 2024-05-17T21:55:44.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.526+0000 7f4061aee700 1 -- 172.21.15.107:0/3475129242 wait complete. 2024-05-17T21:55:44.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.526+0000 7f4061aee700 1 Processor -- start 2024-05-17T21:55:44.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.526+0000 7f4061aee700 1 -- start start 2024-05-17T21:55:44.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.527+0000 7f4061aee700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f405c0fce80 0x7f405c071de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:55:44.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.527+0000 7f4061aee700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f405c0fe080 0x7f405c0702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:55:44.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.527+0000 7f4061aee700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f405c070860 con 0x7f405c0fce80 2024-05-17T21:55:44.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.527+0000 7f4061aee700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f405c0709d0 con 0x7f405c0fe080 2024-05-17T21:55:44.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.527+0000 7f405b7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f405c0fce80 0x7f405c071de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:55:44.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.527+0000 7f405b7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f405c0fce80 0x7f405c071de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34620/0 (socket says 172.21.15.107:34620) 2024-05-17T21:55:44.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.527+0000 7f405b7fe700 1 -- 172.21.15.107:0/2977077783 learned_addr learned my addr 172.21.15.107:0/2977077783 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:55:44.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.527+0000 7f405affd700 1 --2- 172.21.15.107:0/2977077783 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f405c0fe080 0x7f405c0702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:55:44.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.527+0000 7f405b7fe700 1 -- 172.21.15.107:0/2977077783 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f405c0fe080 msgr2=0x7f405c0702b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:55:44.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.527+0000 7f405b7fe700 1 --2- 172.21.15.107:0/2977077783 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f405c0fe080 0x7f405c0702b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:55:44.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.527+0000 7f405b7fe700 1 -- 172.21.15.107:0/2977077783 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f404c005040 con 0x7f405c0fce80 2024-05-17T21:55:44.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.527+0000 7f405b7fe700 1 --2- 172.21.15.107:0/2977077783 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f405c0fce80 0x7f405c071de0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f405000eb70 tx=0x7f405000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:55:44.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.527+0000 7f4043fff700 1 -- 172.21.15.107:0/2977077783 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4050007cc0 con 0x7f405c0fce80 2024-05-17T21:55:44.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.527+0000 7f4061aee700 1 -- 172.21.15.107:0/2977077783 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f405c070c50 con 0x7f405c0fce80 2024-05-17T21:55:44.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.527+0000 7f4043fff700 1 -- 172.21.15.107:0/2977077783 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f405000f040 con 0x7f405c0fce80 2024-05-17T21:55:44.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.528+0000 7f4043fff700 1 -- 172.21.15.107:0/2977077783 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f405000cb20 con 0x7f405c0fce80 2024-05-17T21:55:44.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.528+0000 7f4061aee700 1 -- 172.21.15.107:0/2977077783 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f405c1159d0 con 0x7f405c0fce80 2024-05-17T21:55:44.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.528+0000 7f4061aee700 1 -- 172.21.15.107:0/2977077783 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40480052e0 con 0x7f405c0fce80 2024-05-17T21:55:44.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.529+0000 7f4043fff700 1 -- 172.21.15.107:0/2977077783 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 100184+0+0 (secure 0 0 0) 0x7f40500097c0 con 0x7f405c0fce80 2024-05-17T21:55:44.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.530+0000 7f4043fff700 1 --2- 172.21.15.107:0/2977077783 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f4044078aa0 0x7f404407af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:55:44.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.530+0000 7f405affd700 1 --2- 172.21.15.107:0/2977077783 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f4044078aa0 0x7f404407af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:55:44.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.530+0000 7f4043fff700 1 -- 172.21.15.107:0/2977077783 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f405009a310 con 0x7f405c0fce80 2024-05-17T21:55:44.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.531+0000 7f405affd700 1 --2- 172.21.15.107:0/2977077783 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f4044078aa0 0x7f404407af50 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f404c00b0d0 tx=0x7f404c009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:55:44.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.534+0000 7f4043fff700 1 -- 172.21.15.107:0/2977077783 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f4050051df0 con 0x7f405c0fce80 2024-05-17T21:55:44.756 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.764+0000 7f4061aee700 1 -- 172.21.15.107:0/2977077783 --> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4048000bf0 con 0x7f4044078aa0 2024-05-17T21:55:44.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.765+0000 7f4043fff700 1 -- 172.21.15.107:0/2977077783 <== mgr.34100 v2:172.21.15.107:6800/3205268226 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+385 (secure 0 0 0) 0x7f4048000bf0 con 0x7f4044078aa0 2024-05-17T21:55:44.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.768+0000 7f4061aee700 1 -- 172.21.15.107:0/2977077783 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f4044078aa0 msgr2=0x7f404407af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:55:44.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.768+0000 7f4061aee700 1 --2- 172.21.15.107:0/2977077783 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f4044078aa0 0x7f404407af50 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f404c00b0d0 tx=0x7f404c009e30 comp rx=0 tx=0).stop 2024-05-17T21:55:44.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.768+0000 7f4061aee700 1 -- 172.21.15.107:0/2977077783 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f405c0fce80 msgr2=0x7f405c071de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:55:44.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.768+0000 7f4061aee700 1 --2- 172.21.15.107:0/2977077783 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f405c0fce80 0x7f405c071de0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f405000eb70 tx=0x7f405000b040 comp rx=0 tx=0).stop 2024-05-17T21:55:44.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.768+0000 7f4061aee700 1 -- 172.21.15.107:0/2977077783 shutdown_connections 2024-05-17T21:55:44.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.768+0000 7f4061aee700 1 --2- 172.21.15.107:0/2977077783 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f4044078aa0 0x7f404407af50 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:55:44.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.768+0000 7f4061aee700 1 --2- 172.21.15.107:0/2977077783 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f405c0fce80 0x7f405c071de0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:55:44.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.768+0000 7f4061aee700 1 --2- 172.21.15.107:0/2977077783 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f405c0fe080 0x7f405c0702b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:55:44.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.768+0000 7f4061aee700 1 -- 172.21.15.107:0/2977077783 >> 172.21.15.107:0/2977077783 conn(0x7f405c0f8590 msgr2=0x7f405c0f9c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:55:44.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.768+0000 7f4061aee700 1 -- 172.21.15.107:0/2977077783 shutdown_connections 2024-05-17T21:55:44.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.768+0000 7f4061aee700 1 -- 172.21.15.107:0/2977077783 wait complete. 2024-05-17T21:55:44.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.870+0000 7fbd69ba3700 1 -- 172.21.15.107:0/2816802545 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbd640fbce0 msgr2=0x7fbd640fc0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:55:44.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.870+0000 7fbd69ba3700 1 --2- 172.21.15.107:0/2816802545 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbd640fbce0 0x7fbd640fc0f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fbd58004940 tx=0x7fbd58009e30 comp rx=0 tx=0).stop 2024-05-17T21:55:44.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.870+0000 7fbd69ba3700 1 -- 172.21.15.107:0/2816802545 shutdown_connections 2024-05-17T21:55:44.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.870+0000 7fbd69ba3700 1 --2- 172.21.15.107:0/2816802545 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd640fcf90 0x7fbd640fd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:55:44.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.870+0000 7fbd69ba3700 1 --2- 172.21.15.107:0/2816802545 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbd640fbce0 0x7fbd640fc0f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:55:44.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.870+0000 7fbd69ba3700 1 -- 172.21.15.107:0/2816802545 >> 172.21.15.107:0/2816802545 conn(0x7fbd640f7440 msgr2=0x7fbd640f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:55:44.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.870+0000 7fbd69ba3700 1 -- 172.21.15.107:0/2816802545 shutdown_connections 2024-05-17T21:55:44.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.870+0000 7fbd69ba3700 1 -- 172.21.15.107:0/2816802545 wait complete. 2024-05-17T21:55:44.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.871+0000 7fbd69ba3700 1 Processor -- start 2024-05-17T21:55:44.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.871+0000 7fbd69ba3700 1 -- start start 2024-05-17T21:55:44.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.871+0000 7fbd69ba3700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd640fbce0 0x7fbd64110680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:55:44.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.871+0000 7fbd69ba3700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbd640fcf90 0x7fbd64112bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:55:44.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.871+0000 7fbd69ba3700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd64066be0 con 0x7fbd640fcf90 2024-05-17T21:55:44.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.871+0000 7fbd69ba3700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd64066d20 con 0x7fbd640fbce0 2024-05-17T21:55:44.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.871+0000 7fbd62ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbd640fcf90 0x7fbd64112bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:55:44.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.871+0000 7fbd62ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbd640fcf90 0x7fbd64112bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34642/0 (socket says 172.21.15.107:34642) 2024-05-17T21:55:44.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.871+0000 7fbd62ffd700 1 -- 172.21.15.107:0/612628398 learned_addr learned my addr 172.21.15.107:0/612628398 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:55:44.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.871+0000 7fbd637fe700 1 --2- 172.21.15.107:0/612628398 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd640fbce0 0x7fbd64110680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:55:44.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.872+0000 7fbd62ffd700 1 -- 172.21.15.107:0/612628398 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd640fbce0 msgr2=0x7fbd64110680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:55:44.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.872+0000 7fbd62ffd700 1 --2- 172.21.15.107:0/612628398 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd640fbce0 0x7fbd64110680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:55:44.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.872+0000 7fbd62ffd700 1 -- 172.21.15.107:0/612628398 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd58005040 con 0x7fbd640fcf90 2024-05-17T21:55:44.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.872+0000 7fbd637fe700 1 --2- 172.21.15.107:0/612628398 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd640fbce0 0x7fbd64110680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T21:55:44.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.872+0000 7fbd62ffd700 1 --2- 172.21.15.107:0/612628398 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbd640fcf90 0x7fbd64112bd0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fbd5400eb40 tx=0x7fbd5400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:55:44.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.872+0000 7fbd4bfff700 1 -- 172.21.15.107:0/612628398 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbd54007cc0 con 0x7fbd640fcf90 2024-05-17T21:55:44.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.872+0000 7fbd69ba3700 1 -- 172.21.15.107:0/612628398 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd64113170 con 0x7fbd640fcf90 2024-05-17T21:55:44.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.872+0000 7fbd4bfff700 1 -- 172.21.15.107:0/612628398 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbd5400f040 con 0x7fbd640fcf90 2024-05-17T21:55:44.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.872+0000 7fbd4bfff700 1 -- 172.21.15.107:0/612628398 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbd54009620 con 0x7fbd640fcf90 2024-05-17T21:55:44.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.872+0000 7fbd69ba3700 1 -- 172.21.15.107:0/612628398 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd64113680 con 0x7fbd640fcf90 2024-05-17T21:55:44.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.873+0000 7fbd69ba3700 1 -- 172.21.15.107:0/612628398 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd6405f5c0 con 0x7fbd640fcf90 2024-05-17T21:55:44.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.874+0000 7fbd4bfff700 1 -- 172.21.15.107:0/612628398 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 100184+0+0 (secure 0 0 0) 0x7fbd54009780 con 0x7fbd640fcf90 2024-05-17T21:55:44.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.874+0000 7fbd4bfff700 1 --2- 172.21.15.107:0/612628398 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fbd4c078aa0 0x7fbd4c07af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:55:44.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.874+0000 7fbd637fe700 1 --2- 172.21.15.107:0/612628398 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fbd4c078aa0 0x7fbd4c07af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:55:44.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.874+0000 7fbd4bfff700 1 -- 172.21.15.107:0/612628398 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fbd5409a5e0 con 0x7fbd640fcf90 2024-05-17T21:55:44.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.875+0000 7fbd637fe700 1 --2- 172.21.15.107:0/612628398 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fbd4c078aa0 0x7fbd4c07af50 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fbd58004890 tx=0x7fbd58009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:55:44.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:44.879+0000 7fbd4bfff700 1 -- 172.21.15.107:0/612628398 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7fbd540620e0 con 0x7fbd640fcf90 2024-05-17T21:55:45.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.097+0000 7fbd69ba3700 1 -- 172.21.15.107:0/612628398 --> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fbd64101300 con 0x7fbd4c078aa0 2024-05-17T21:55:45.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.107+0000 7fbd4bfff700 1 -- 172.21.15.107:0/612628398 <== mgr.34100 v2:172.21.15.107:6800/3205268226 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7fbd64101300 con 0x7fbd4c078aa0 2024-05-17T21:55:45.099 INFO:teuthology.orchestra.run.smithi107.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-17T21:55:45.099 INFO:teuthology.orchestra.run.smithi107.stdout:alertmanager.smithi107 smithi107 *:9093,9094 running (3m) 63s ago 11m 17.2M - 0.25.0 c8568f914cd2 bac2a944ae42 2024-05-17T21:55:45.099 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi107 smithi107 starting - - - - 2024-05-17T21:55:45.100 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi138 smithi138 running (11m) 39s ago 11m 6660k - 17.2.7-952-g9903f776 d15a36853796 20e9ccf1fadd 2024-05-17T21:55:45.100 INFO:teuthology.orchestra.run.smithi107.stdout:grafana.smithi107 smithi107 *:3000 running (3m) 63s ago 11m 79.6M - 9.4.7 954c08fa6188 8c747e777bc5 2024-05-17T21:55:45.100 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.drqody smithi107 running (6m) 63s ago 6m 170M - 17.2.7-952-g9903f776 d15a36853796 0d851bc912f0 2024-05-17T21:55:45.100 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.idmbgj smithi107 running (6m) 63s ago 6m 13.1M - 17.2.7-952-g9903f776 d15a36853796 1348b09ab29e 2024-05-17T21:55:45.100 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi138.gwbtyd smithi138 running (6m) 39s ago 6m 153M - 17.2.7-952-g9903f776 d15a36853796 fd2ba75354c0 2024-05-17T21:55:45.100 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi138.zckrvs smithi138 running (6m) 39s ago 6m 13.9M - 17.2.7-952-g9903f776 d15a36853796 a1cfd2fe4cda 2024-05-17T21:55:45.100 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi107.xhhyiw smithi107 *:8443,9283,8765 running (2m) 63s ago 13m 556M - 19.0.0-3794-g931809a9 8880f7613d02 5acf3e511342 2024-05-17T21:55:45.100 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi138.napnll smithi138 *:8443,9283,8765 running (102s) 39s ago 10m 531M - 19.0.0-3794-g931809a9 8880f7613d02 d627761dbd0d 2024-05-17T21:55:45.100 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi107 smithi107 running (80s) 63s ago 13m 96.4M 2048M 19.0.0-3794-g931809a9 8880f7613d02 9717f3169f2f 2024-05-17T21:55:45.100 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi138 smithi138 running (43s) 39s ago 10m 87.1M 2048M 19.0.0-3794-g931809a9 8880f7613d02 55aeed62bf4d 2024-05-17T21:55:45.100 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi107 smithi107 *:9100 running (3m) 63s ago 12m 17.2M - 1.5.0 0da6a335fe13 49bec71f5487 2024-05-17T21:55:45.100 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi138 smithi138 *:9100 running (2m) 39s ago 10m 16.2M - 1.5.0 0da6a335fe13 e52824376658 2024-05-17T21:55:45.100 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0 smithi107 running (10m) 63s ago 10m 198M 1557M 17.2.7-952-g9903f776 d15a36853796 f2d20009361d 2024-05-17T21:55:45.101 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1 smithi107 running (9m) 63s ago 9m 214M 1557M 17.2.7-952-g9903f776 d15a36853796 e5d3e99844a9 2024-05-17T21:55:45.101 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2 smithi107 running (9m) 63s ago 9m 175M 1557M 17.2.7-952-g9903f776 d15a36853796 e044d7b78088 2024-05-17T21:55:45.101 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3 smithi138 running (8m) 39s ago 8m 227M 2581M 17.2.7-952-g9903f776 d15a36853796 8c980cc8af87 2024-05-17T21:55:45.101 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4 smithi138 running (8m) 39s ago 8m 204M 2581M 17.2.7-952-g9903f776 d15a36853796 b306a36a744e 2024-05-17T21:55:45.101 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5 smithi138 running (7m) 39s ago 7m 189M 2581M 17.2.7-952-g9903f776 d15a36853796 7c8d8ed74974 2024-05-17T21:55:45.101 INFO:teuthology.orchestra.run.smithi107.stdout:prometheus.smithi107 smithi107 *:9095 running (2m) 63s ago 11m 51.5M - 2.43.0 a07b618ecd1d bf5f609bbd6e 2024-05-17T21:55:45.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.109+0000 7fbd69ba3700 1 -- 172.21.15.107:0/612628398 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fbd4c078aa0 msgr2=0x7fbd4c07af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:55:45.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.109+0000 7fbd69ba3700 1 --2- 172.21.15.107:0/612628398 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fbd4c078aa0 0x7fbd4c07af50 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fbd58004890 tx=0x7fbd58009e30 comp rx=0 tx=0).stop 2024-05-17T21:55:45.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.109+0000 7fbd69ba3700 1 -- 172.21.15.107:0/612628398 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbd640fcf90 msgr2=0x7fbd64112bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:55:45.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.109+0000 7fbd69ba3700 1 --2- 172.21.15.107:0/612628398 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbd640fcf90 0x7fbd64112bd0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fbd5400eb40 tx=0x7fbd5400b040 comp rx=0 tx=0).stop 2024-05-17T21:55:45.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.109+0000 7fbd69ba3700 1 -- 172.21.15.107:0/612628398 shutdown_connections 2024-05-17T21:55:45.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.109+0000 7fbd69ba3700 1 --2- 172.21.15.107:0/612628398 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fbd4c078aa0 0x7fbd4c07af50 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:55:45.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.109+0000 7fbd69ba3700 1 --2- 172.21.15.107:0/612628398 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd640fbce0 0x7fbd64110680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:55:45.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.109+0000 7fbd69ba3700 1 --2- 172.21.15.107:0/612628398 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbd640fcf90 0x7fbd64112bd0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:55:45.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.109+0000 7fbd69ba3700 1 -- 172.21.15.107:0/612628398 >> 172.21.15.107:0/612628398 conn(0x7fbd640f7440 msgr2=0x7fbd640f8aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:55:45.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.110+0000 7fbd69ba3700 1 -- 172.21.15.107:0/612628398 shutdown_connections 2024-05-17T21:55:45.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.110+0000 7fbd69ba3700 1 -- 172.21.15.107:0/612628398 wait complete. 2024-05-17T21:55:45.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.212+0000 7fc81e2e5700 1 -- 172.21.15.107:0/3071188596 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8180fce60 msgr2=0x7fc8180fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:55:45.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.212+0000 7fc81e2e5700 1 --2- 172.21.15.107:0/3071188596 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8180fce60 0x7fc8180fd270 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc808004940 tx=0x7fc808009e30 comp rx=0 tx=0).stop 2024-05-17T21:55:45.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.212+0000 7fc81e2e5700 1 -- 172.21.15.107:0/3071188596 shutdown_connections 2024-05-17T21:55:45.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.212+0000 7fc81e2e5700 1 --2- 172.21.15.107:0/3071188596 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8180fdfd0 0x7fc8180fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:55:45.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.212+0000 7fc81e2e5700 1 --2- 172.21.15.107:0/3071188596 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8180fce60 0x7fc8180fd270 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:55:45.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.212+0000 7fc81e2e5700 1 -- 172.21.15.107:0/3071188596 >> 172.21.15.107:0/3071188596 conn(0x7fc8180f8540 msgr2=0x7fc8180fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:55:45.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.212+0000 7fc81e2e5700 1 -- 172.21.15.107:0/3071188596 shutdown_connections 2024-05-17T21:55:45.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.212+0000 7fc81e2e5700 1 -- 172.21.15.107:0/3071188596 wait complete. 2024-05-17T21:55:45.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.213+0000 7fc81e2e5700 1 Processor -- start 2024-05-17T21:55:45.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.213+0000 7fc81e2e5700 1 -- start start 2024-05-17T21:55:45.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.213+0000 7fc81e2e5700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8180fce60 0x7fc81810d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:55:45.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.213+0000 7fc81e2e5700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8180fdfd0 0x7fc81810df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:55:45.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.213+0000 7fc81e2e5700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc81810e530 con 0x7fc8180fce60 2024-05-17T21:55:45.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.213+0000 7fc81e2e5700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc81810ed80 con 0x7fc8180fdfd0 2024-05-17T21:55:45.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.213+0000 7fc817fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8180fce60 0x7fc81810d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:55:45.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.213+0000 7fc817fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8180fce60 0x7fc81810d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34660/0 (socket says 172.21.15.107:34660) 2024-05-17T21:55:45.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.213+0000 7fc817fff700 1 -- 172.21.15.107:0/2723464323 learned_addr learned my addr 172.21.15.107:0/2723464323 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:55:45.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.213+0000 7fc80ffff700 1 --2- 172.21.15.107:0/2723464323 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8180fdfd0 0x7fc81810df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:55:45.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.213+0000 7fc817fff700 1 -- 172.21.15.107:0/2723464323 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8180fdfd0 msgr2=0x7fc81810df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:55:45.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.213+0000 7fc817fff700 1 --2- 172.21.15.107:0/2723464323 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8180fdfd0 0x7fc81810df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:55:45.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.213+0000 7fc817fff700 1 -- 172.21.15.107:0/2723464323 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc808005040 con 0x7fc8180fce60 2024-05-17T21:55:45.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.213+0000 7fc80ffff700 1 --2- 172.21.15.107:0/2723464323 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8180fdfd0 0x7fc81810df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:55:45.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.213+0000 7fc817fff700 1 --2- 172.21.15.107:0/2723464323 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8180fce60 0x7fc81810d9f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc80800b3b0 tx=0x7fc808009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:55:45.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.214+0000 7fc8157fa700 1 -- 172.21.15.107:0/2723464323 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc80800e040 con 0x7fc8180fce60 2024-05-17T21:55:45.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.214+0000 7fc8157fa700 1 -- 172.21.15.107:0/2723464323 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc80800bcc0 con 0x7fc8180fce60 2024-05-17T21:55:45.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.214+0000 7fc81e2e5700 1 -- 172.21.15.107:0/2723464323 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc81810f000 con 0x7fc8180fce60 2024-05-17T21:55:45.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.214+0000 7fc8157fa700 1 -- 172.21.15.107:0/2723464323 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc80801c630 con 0x7fc8180fce60 2024-05-17T21:55:45.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.214+0000 7fc81e2e5700 1 -- 172.21.15.107:0/2723464323 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc81810f540 con 0x7fc8180fce60 2024-05-17T21:55:45.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.215+0000 7fc81e2e5700 1 -- 172.21.15.107:0/2723464323 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc81805f5c0 con 0x7fc8180fce60 2024-05-17T21:55:45.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.215+0000 7fc8157fa700 1 -- 172.21.15.107:0/2723464323 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 100184+0+0 (secure 0 0 0) 0x7fc808013980 con 0x7fc8180fce60 2024-05-17T21:55:45.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.216+0000 7fc8157fa700 1 --2- 172.21.15.107:0/2723464323 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fc804078aa0 0x7fc80407af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:55:45.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.216+0000 7fc8157fa700 1 -- 172.21.15.107:0/2723464323 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fc80809b610 con 0x7fc8180fce60 2024-05-17T21:55:45.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.216+0000 7fc80ffff700 1 --2- 172.21.15.107:0/2723464323 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fc804078aa0 0x7fc80407af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:55:45.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.216+0000 7fc80ffff700 1 --2- 172.21.15.107:0/2723464323 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fc804078aa0 0x7fc80407af50 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc800009fd0 tx=0x7fc80000b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:55:45.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.220+0000 7fc8157fa700 1 -- 172.21.15.107:0/2723464323 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7fc8080630e0 con 0x7fc8180fce60 2024-05-17T21:55:45.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:45 smithi138 ceph-mon[81376]: Upgrade: Updating crash.smithi138 (2/2) 2024-05-17T21:55:45.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:45 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:45.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:45 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi138", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-17T21:55:45.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:45 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:55:45.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:45 smithi138 ceph-mon[81376]: Deploying daemon crash.smithi138 on smithi138 2024-05-17T21:55:45.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:45 smithi138 ceph-mon[81376]: pgmap v38: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T21:55:45.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:45 smithi138 ceph-mon[81376]: from='client.34188 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:55:45.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:45 smithi138 ceph-mon[81376]: from='client.34192 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:55:45.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:45 smithi138 ceph-mon[81376]: from='client.34196 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:55:45.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:45 smithi107 ceph-mon[93224]: Upgrade: Updating crash.smithi138 (2/2) 2024-05-17T21:55:45.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:45 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:45.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:45 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi138", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-17T21:55:45.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:45 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:55:45.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:45 smithi107 ceph-mon[93224]: Deploying daemon crash.smithi138 on smithi138 2024-05-17T21:55:45.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:45 smithi107 ceph-mon[93224]: pgmap v38: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T21:55:45.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:45 smithi107 ceph-mon[93224]: from='client.34188 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:55:45.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:45 smithi107 ceph-mon[93224]: from='client.34192 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:55:45.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:45 smithi107 ceph-mon[93224]: from='client.34196 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:55:45.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.511+0000 7fc81e2e5700 1 -- 172.21.15.107:0/2723464323 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fc81810f8b0 con 0x7fc8180fce60 2024-05-17T21:55:45.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.511+0000 7fc8157fa700 1 -- 172.21.15.107:0/2723464323 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+736 (secure 0 0 0) 0x7fc808062a80 con 0x7fc8180fce60 2024-05-17T21:55:45.504 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-05-17T21:55:45.504 INFO:teuthology.orchestra.run.smithi107.stdout: "mon": { 2024-05-17T21:55:45.504 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev)": 2 2024-05-17T21:55:45.504 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-05-17T21:55:45.504 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr": { 2024-05-17T21:55:45.504 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev)": 2 2024-05-17T21:55:45.504 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-05-17T21:55:45.504 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": { 2024-05-17T21:55:45.504 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-952-g9903f776 (9903f7767386a9d9efadb7c1effdcd3064d71804) quincy (stable)": 6 2024-05-17T21:55:45.504 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-05-17T21:55:45.504 INFO:teuthology.orchestra.run.smithi107.stdout: "mds": { 2024-05-17T21:55:45.504 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-952-g9903f776 (9903f7767386a9d9efadb7c1effdcd3064d71804) quincy (stable)": 4 2024-05-17T21:55:45.504 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-05-17T21:55:45.504 INFO:teuthology.orchestra.run.smithi107.stdout: "overall": { 2024-05-17T21:55:45.505 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-952-g9903f776 (9903f7767386a9d9efadb7c1effdcd3064d71804) quincy (stable)": 10, 2024-05-17T21:55:45.505 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev)": 4 2024-05-17T21:55:45.505 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-05-17T21:55:45.505 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-05-17T21:55:45.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.514+0000 7fc81e2e5700 1 -- 172.21.15.107:0/2723464323 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fc804078aa0 msgr2=0x7fc80407af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:55:45.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.514+0000 7fc81e2e5700 1 --2- 172.21.15.107:0/2723464323 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fc804078aa0 0x7fc80407af50 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc800009fd0 tx=0x7fc80000b040 comp rx=0 tx=0).stop 2024-05-17T21:55:45.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.514+0000 7fc81e2e5700 1 -- 172.21.15.107:0/2723464323 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8180fce60 msgr2=0x7fc81810d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:55:45.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.514+0000 7fc81e2e5700 1 --2- 172.21.15.107:0/2723464323 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8180fce60 0x7fc81810d9f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc80800b3b0 tx=0x7fc808009e30 comp rx=0 tx=0).stop 2024-05-17T21:55:45.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.514+0000 7fc81e2e5700 1 -- 172.21.15.107:0/2723464323 shutdown_connections 2024-05-17T21:55:45.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.514+0000 7fc81e2e5700 1 --2- 172.21.15.107:0/2723464323 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fc804078aa0 0x7fc80407af50 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:55:45.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.514+0000 7fc81e2e5700 1 --2- 172.21.15.107:0/2723464323 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8180fce60 0x7fc81810d9f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:55:45.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.514+0000 7fc81e2e5700 1 --2- 172.21.15.107:0/2723464323 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8180fdfd0 0x7fc81810df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:55:45.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.514+0000 7fc81e2e5700 1 -- 172.21.15.107:0/2723464323 >> 172.21.15.107:0/2723464323 conn(0x7fc8180f8540 msgr2=0x7fc8180f9c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:55:45.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.514+0000 7fc81e2e5700 1 -- 172.21.15.107:0/2723464323 shutdown_connections 2024-05-17T21:55:45.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.514+0000 7fc81e2e5700 1 -- 172.21.15.107:0/2723464323 wait complete. 2024-05-17T21:55:45.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.616+0000 7ffb598ed700 1 -- 172.21.15.107:0/2969372079 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb540fce60 msgr2=0x7ffb540fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:55:45.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.616+0000 7ffb598ed700 1 --2- 172.21.15.107:0/2969372079 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb540fce60 0x7ffb540fd270 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ffb44004940 tx=0x7ffb44009e30 comp rx=0 tx=0).stop 2024-05-17T21:55:45.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.616+0000 7ffb598ed700 1 -- 172.21.15.107:0/2969372079 shutdown_connections 2024-05-17T21:55:45.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.616+0000 7ffb598ed700 1 --2- 172.21.15.107:0/2969372079 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb540fdfd0 0x7ffb540fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:55:45.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.616+0000 7ffb598ed700 1 --2- 172.21.15.107:0/2969372079 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb540fce60 0x7ffb540fd270 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:55:45.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.616+0000 7ffb598ed700 1 -- 172.21.15.107:0/2969372079 >> 172.21.15.107:0/2969372079 conn(0x7ffb540f8520 msgr2=0x7ffb540fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:55:45.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.616+0000 7ffb598ed700 1 -- 172.21.15.107:0/2969372079 shutdown_connections 2024-05-17T21:55:45.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.617+0000 7ffb598ed700 1 -- 172.21.15.107:0/2969372079 wait complete. 2024-05-17T21:55:45.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.617+0000 7ffb598ed700 1 Processor -- start 2024-05-17T21:55:45.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.617+0000 7ffb598ed700 1 -- start start 2024-05-17T21:55:45.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.617+0000 7ffb598ed700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb540fce60 0x7ffb54071c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:55:45.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.617+0000 7ffb598ed700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb540fdfd0 0x7ffb540702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:55:45.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.617+0000 7ffb598ed700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb54072200 con 0x7ffb540fce60 2024-05-17T21:55:45.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.617+0000 7ffb598ed700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb540707f0 con 0x7ffb540fdfd0 2024-05-17T21:55:45.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.618+0000 7ffb52ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb540fce60 0x7ffb54071c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:55:45.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.618+0000 7ffb52ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb540fce60 0x7ffb54071c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34018/0 (socket says 172.21.15.107:34018) 2024-05-17T21:55:45.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.618+0000 7ffb52ffd700 1 -- 172.21.15.107:0/3821141756 learned_addr learned my addr 172.21.15.107:0/3821141756 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:55:45.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.618+0000 7ffb527fc700 1 --2- 172.21.15.107:0/3821141756 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb540fdfd0 0x7ffb540702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:55:45.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.618+0000 7ffb52ffd700 1 -- 172.21.15.107:0/3821141756 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb540fdfd0 msgr2=0x7ffb540702b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:55:45.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.618+0000 7ffb52ffd700 1 --2- 172.21.15.107:0/3821141756 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb540fdfd0 0x7ffb540702b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:55:45.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.618+0000 7ffb52ffd700 1 -- 172.21.15.107:0/3821141756 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb44005040 con 0x7ffb540fce60 2024-05-17T21:55:45.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.618+0000 7ffb527fc700 1 --2- 172.21.15.107:0/3821141756 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb540fdfd0 0x7ffb540702b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:55:45.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.618+0000 7ffb52ffd700 1 --2- 172.21.15.107:0/3821141756 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb540fce60 0x7ffb54071c00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ffb4400b3b0 tx=0x7ffb44009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:55:45.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.618+0000 7ffb3b7fe700 1 -- 172.21.15.107:0/3821141756 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffb4400e040 con 0x7ffb540fce60 2024-05-17T21:55:45.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.618+0000 7ffb598ed700 1 -- 172.21.15.107:0/3821141756 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb54070a40 con 0x7ffb540fce60 2024-05-17T21:55:45.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.618+0000 7ffb3b7fe700 1 -- 172.21.15.107:0/3821141756 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffb4400bcc0 con 0x7ffb540fce60 2024-05-17T21:55:45.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.618+0000 7ffb3b7fe700 1 -- 172.21.15.107:0/3821141756 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffb4401c630 con 0x7ffb540fce60 2024-05-17T21:55:45.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.618+0000 7ffb598ed700 1 -- 172.21.15.107:0/3821141756 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb54071000 con 0x7ffb540fce60 2024-05-17T21:55:45.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.619+0000 7ffb598ed700 1 -- 172.21.15.107:0/3821141756 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb5404c710 con 0x7ffb540fce60 2024-05-17T21:55:45.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.620+0000 7ffb3b7fe700 1 -- 172.21.15.107:0/3821141756 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 100184+0+0 (secure 0 0 0) 0x7ffb44013980 con 0x7ffb540fce60 2024-05-17T21:55:45.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.620+0000 7ffb3b7fe700 1 --2- 172.21.15.107:0/3821141756 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7ffb3c078aa0 0x7ffb3c07af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:55:45.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.621+0000 7ffb3b7fe700 1 -- 172.21.15.107:0/3821141756 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6410+0+0 (secure 0 0 0) 0x7ffb4409bde0 con 0x7ffb540fce60 2024-05-17T21:55:45.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.621+0000 7ffb527fc700 1 --2- 172.21.15.107:0/3821141756 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7ffb3c078aa0 0x7ffb3c07af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:55:45.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.621+0000 7ffb527fc700 1 --2- 172.21.15.107:0/3821141756 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7ffb3c078aa0 0x7ffb3c07af50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ffb48007fd0 tx=0x7ffb48009040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:55:45.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.625+0000 7ffb3b7fe700 1 -- 172.21.15.107:0/3821141756 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7ffb44063830 con 0x7ffb540fce60 2024-05-17T21:55:45.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.878+0000 7ffb598ed700 1 -- 172.21.15.107:0/3821141756 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7ffb54115c30 con 0x7ffb540fce60 2024-05-17T21:55:45.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.879+0000 7ffb3b7fe700 1 -- 172.21.15.107:0/3821141756 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1910 (secure 0 0 0) 0x7ffb440631d0 con 0x7ffb540fce60 2024-05-17T21:55:45.871 INFO:teuthology.orchestra.run.smithi107.stdout:e15 2024-05-17T21:55:45.871 INFO:teuthology.orchestra.run.smithi107.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-17T21:55:45.871 INFO:teuthology.orchestra.run.smithi107.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-17T21:55:45.871 INFO:teuthology.orchestra.run.smithi107.stdout:legacy client fscid: 1 2024-05-17T21:55:45.871 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:55:45.871 INFO:teuthology.orchestra.run.smithi107.stdout:Filesystem 'cephfs' (1) 2024-05-17T21:55:45.871 INFO:teuthology.orchestra.run.smithi107.stdout:fs_name cephfs 2024-05-17T21:55:45.871 INFO:teuthology.orchestra.run.smithi107.stdout:epoch 15 2024-05-17T21:55:45.871 INFO:teuthology.orchestra.run.smithi107.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-17T21:55:45.871 INFO:teuthology.orchestra.run.smithi107.stdout:created 2024-05-17T21:49:16.836333+0000 2024-05-17T21:55:45.872 INFO:teuthology.orchestra.run.smithi107.stdout:modified 2024-05-17T21:49:34.873628+0000 2024-05-17T21:55:45.872 INFO:teuthology.orchestra.run.smithi107.stdout:tableserver 0 2024-05-17T21:55:45.872 INFO:teuthology.orchestra.run.smithi107.stdout:root 0 2024-05-17T21:55:45.872 INFO:teuthology.orchestra.run.smithi107.stdout:session_timeout 60 2024-05-17T21:55:45.872 INFO:teuthology.orchestra.run.smithi107.stdout:session_autoclose 300 2024-05-17T21:55:45.872 INFO:teuthology.orchestra.run.smithi107.stdout:max_file_size 1099511627776 2024-05-17T21:55:45.872 INFO:teuthology.orchestra.run.smithi107.stdout:max_xattr_size 65536 2024-05-17T21:55:45.872 INFO:teuthology.orchestra.run.smithi107.stdout:required_client_features {} 2024-05-17T21:55:45.872 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure 0 2024-05-17T21:55:45.872 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure_osd_epoch 46 2024-05-17T21:55:45.872 INFO:teuthology.orchestra.run.smithi107.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-17T21:55:45.872 INFO:teuthology.orchestra.run.smithi107.stdout:max_mds 1 2024-05-17T21:55:45.872 INFO:teuthology.orchestra.run.smithi107.stdout:in 0 2024-05-17T21:55:45.872 INFO:teuthology.orchestra.run.smithi107.stdout:up {0=24259} 2024-05-17T21:55:45.872 INFO:teuthology.orchestra.run.smithi107.stdout:failed 2024-05-17T21:55:45.873 INFO:teuthology.orchestra.run.smithi107.stdout:damaged 2024-05-17T21:55:45.873 INFO:teuthology.orchestra.run.smithi107.stdout:stopped 2024-05-17T21:55:45.873 INFO:teuthology.orchestra.run.smithi107.stdout:data_pools [3] 2024-05-17T21:55:45.873 INFO:teuthology.orchestra.run.smithi107.stdout:metadata_pool 2 2024-05-17T21:55:45.873 INFO:teuthology.orchestra.run.smithi107.stdout:inline_data enabled 2024-05-17T21:55:45.873 INFO:teuthology.orchestra.run.smithi107.stdout:balancer 2024-05-17T21:55:45.873 INFO:teuthology.orchestra.run.smithi107.stdout:bal_rank_mask -1 2024-05-17T21:55:45.873 INFO:teuthology.orchestra.run.smithi107.stdout:standby_count_wanted 1 2024-05-17T21:55:45.873 INFO:teuthology.orchestra.run.smithi107.stdout:qdb_cluster leader: 0 members: 2024-05-17T21:55:45.873 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi138.gwbtyd{0:24259} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.138:6824/2779634919,v1:172.21.15.138:6825/2779634919] compat {c=[1],r=[1],i=[7ff]}] 2024-05-17T21:55:45.873 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.drqody{0:14486} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.107:6828/2275807100,v1:172.21.15.107:6829/2275807100] compat {c=[1],r=[1],i=[7ff]}] 2024-05-17T21:55:45.873 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:55:45.873 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:55:45.873 INFO:teuthology.orchestra.run.smithi107.stdout:Standby daemons: 2024-05-17T21:55:45.874 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:55:45.874 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.idmbgj{-1:14498} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.107:6826/1460186079,v1:172.21.15.107:6827/1460186079] compat {c=[1],r=[1],i=[7ff]}] 2024-05-17T21:55:45.874 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi138.zckrvs{-1:24267} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.138:6826/170628661,v1:172.21.15.138:6827/170628661] compat {c=[1],r=[1],i=[7ff]}] 2024-05-17T21:55:45.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.881+0000 7ffb598ed700 1 -- 172.21.15.107:0/3821141756 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7ffb3c078aa0 msgr2=0x7ffb3c07af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:55:45.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.881+0000 7ffb598ed700 1 --2- 172.21.15.107:0/3821141756 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7ffb3c078aa0 0x7ffb3c07af50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ffb48007fd0 tx=0x7ffb48009040 comp rx=0 tx=0).stop 2024-05-17T21:55:45.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.881+0000 7ffb598ed700 1 -- 172.21.15.107:0/3821141756 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb540fce60 msgr2=0x7ffb54071c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:55:45.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.881+0000 7ffb598ed700 1 --2- 172.21.15.107:0/3821141756 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb540fce60 0x7ffb54071c00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ffb4400b3b0 tx=0x7ffb44009e30 comp rx=0 tx=0).stop 2024-05-17T21:55:45.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.881+0000 7ffb598ed700 1 -- 172.21.15.107:0/3821141756 shutdown_connections 2024-05-17T21:55:45.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.881+0000 7ffb598ed700 1 --2- 172.21.15.107:0/3821141756 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7ffb3c078aa0 0x7ffb3c07af50 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:55:45.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.881+0000 7ffb598ed700 1 --2- 172.21.15.107:0/3821141756 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb540fce60 0x7ffb54071c00 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:55:45.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.881+0000 7ffb598ed700 1 --2- 172.21.15.107:0/3821141756 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb540fdfd0 0x7ffb540702b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:55:45.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.881+0000 7ffb598ed700 1 -- 172.21.15.107:0/3821141756 >> 172.21.15.107:0/3821141756 conn(0x7ffb540f8520 msgr2=0x7ffb540f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:55:45.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.881+0000 7ffb598ed700 1 -- 172.21.15.107:0/3821141756 shutdown_connections 2024-05-17T21:55:45.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.881+0000 7ffb598ed700 1 -- 172.21.15.107:0/3821141756 wait complete. 2024-05-17T21:55:45.875 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 15 2024-05-17T21:55:45.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.987+0000 7f2dcc6c6700 1 -- 172.21.15.107:0/2655356794 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2dc40fdfd0 msgr2=0x7f2dc40fe440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:55:45.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.987+0000 7f2dcc6c6700 1 --2- 172.21.15.107:0/2655356794 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2dc40fdfd0 0x7f2dc40fe440 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2db4004940 tx=0x7f2db4009e30 comp rx=0 tx=0).stop 2024-05-17T21:55:45.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.987+0000 7f2dcc6c6700 1 -- 172.21.15.107:0/2655356794 shutdown_connections 2024-05-17T21:55:45.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.987+0000 7f2dcc6c6700 1 --2- 172.21.15.107:0/2655356794 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2dc40fdfd0 0x7f2dc40fe440 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:55:45.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.987+0000 7f2dcc6c6700 1 --2- 172.21.15.107:0/2655356794 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2dc40fce60 0x7f2dc40fd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:55:45.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.987+0000 7f2dcc6c6700 1 -- 172.21.15.107:0/2655356794 >> 172.21.15.107:0/2655356794 conn(0x7f2dc40f8540 msgr2=0x7f2dc40fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:55:45.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.987+0000 7f2dcc6c6700 1 -- 172.21.15.107:0/2655356794 shutdown_connections 2024-05-17T21:55:45.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.987+0000 7f2dcc6c6700 1 -- 172.21.15.107:0/2655356794 wait complete. 2024-05-17T21:55:45.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.987+0000 7f2dcc6c6700 1 Processor -- start 2024-05-17T21:55:45.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.987+0000 7f2dcc6c6700 1 -- start start 2024-05-17T21:55:45.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.987+0000 7f2dcc6c6700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2dc40fce60 0x7f2dc4071c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:55:45.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.987+0000 7f2dcc6c6700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2dc40fdfd0 0x7f2dc40702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:55:45.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.987+0000 7f2dcc6c6700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dc4072200 con 0x7f2dc40fdfd0 2024-05-17T21:55:45.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.987+0000 7f2dcc6c6700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dc40707f0 con 0x7f2dc40fce60 2024-05-17T21:55:45.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.988+0000 7f2dc9c61700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2dc40fdfd0 0x7f2dc40702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:55:45.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.988+0000 7f2dc9c61700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2dc40fdfd0 0x7f2dc40702b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34038/0 (socket says 172.21.15.107:34038) 2024-05-17T21:55:45.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.988+0000 7f2dc9c61700 1 -- 172.21.15.107:0/2924722045 learned_addr learned my addr 172.21.15.107:0/2924722045 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:55:45.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.988+0000 7f2dca462700 1 --2- 172.21.15.107:0/2924722045 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2dc40fce60 0x7f2dc4071c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:55:45.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.988+0000 7f2dc9c61700 1 -- 172.21.15.107:0/2924722045 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2dc40fce60 msgr2=0x7f2dc4071c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:55:45.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.988+0000 7f2dc9c61700 1 --2- 172.21.15.107:0/2924722045 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2dc40fce60 0x7f2dc4071c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:55:45.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.988+0000 7f2dc9c61700 1 -- 172.21.15.107:0/2924722045 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2db4005040 con 0x7f2dc40fdfd0 2024-05-17T21:55:45.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.988+0000 7f2dc9c61700 1 --2- 172.21.15.107:0/2924722045 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2dc40fdfd0 0x7f2dc40702b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2db4009150 tx=0x7f2db4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:55:45.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.988+0000 7f2dbaffd700 1 -- 172.21.15.107:0/2924722045 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2db400e040 con 0x7f2dc40fdfd0 2024-05-17T21:55:45.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.988+0000 7f2dbaffd700 1 -- 172.21.15.107:0/2924722045 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2db4004290 con 0x7f2dc40fdfd0 2024-05-17T21:55:45.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.988+0000 7f2dbaffd700 1 -- 172.21.15.107:0/2924722045 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2db401c630 con 0x7f2dc40fdfd0 2024-05-17T21:55:45.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.988+0000 7f2dcc6c6700 1 -- 172.21.15.107:0/2924722045 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dc4070a40 con 0x7f2dc40fdfd0 2024-05-17T21:55:45.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.988+0000 7f2dcc6c6700 1 -- 172.21.15.107:0/2924722045 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dc4070fb0 con 0x7f2dc40fdfd0 2024-05-17T21:55:45.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.989+0000 7f2dcc6c6700 1 -- 172.21.15.107:0/2924722045 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2dc404c710 con 0x7f2dc40fdfd0 2024-05-17T21:55:45.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.990+0000 7f2dbaffd700 1 -- 172.21.15.107:0/2924722045 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 100184+0+0 (secure 0 0 0) 0x7f2db400bbf0 con 0x7f2dc40fdfd0 2024-05-17T21:55:45.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.990+0000 7f2dbaffd700 1 --2- 172.21.15.107:0/2924722045 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f2db0078aa0 0x7f2db007af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:55:45.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.990+0000 7f2dbaffd700 1 -- 172.21.15.107:0/2924722045 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f2db409b520 con 0x7f2dc40fdfd0 2024-05-17T21:55:45.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.990+0000 7f2dca462700 1 --2- 172.21.15.107:0/2924722045 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f2db0078aa0 0x7f2db007af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:55:45.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.991+0000 7f2dca462700 1 --2- 172.21.15.107:0/2924722045 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f2db0078aa0 0x7f2db007af50 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f2dc0007fd0 tx=0x7f2dc0009040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:55:45.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:45.995+0000 7f2dbaffd700 1 -- 172.21.15.107:0/2924722045 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f2db4062ff0 con 0x7f2dc40fdfd0 2024-05-17T21:55:46.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:46.219+0000 7f2dcc6c6700 1 -- 172.21.15.107:0/2924722045 --> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2dc4102390 con 0x7f2db0078aa0 2024-05-17T21:55:46.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:46.220+0000 7f2dbaffd700 1 -- 172.21.15.107:0/2924722045 <== mgr.34100 v2:172.21.15.107:6800/3205268226 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+385 (secure 0 0 0) 0x7f2dc4102390 con 0x7f2db0078aa0 2024-05-17T21:55:46.212 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-05-17T21:55:46.213 INFO:teuthology.orchestra.run.smithi107.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b", 2024-05-17T21:55:46.213 INFO:teuthology.orchestra.run.smithi107.stdout: "in_progress": true, 2024-05-17T21:55:46.213 INFO:teuthology.orchestra.run.smithi107.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-17T21:55:46.213 INFO:teuthology.orchestra.run.smithi107.stdout: "services_complete": [ 2024-05-17T21:55:46.213 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr", 2024-05-17T21:55:46.213 INFO:teuthology.orchestra.run.smithi107.stdout: "mon" 2024-05-17T21:55:46.213 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-05-17T21:55:46.213 INFO:teuthology.orchestra.run.smithi107.stdout: "progress": "4/21 daemons upgraded", 2024-05-17T21:55:46.213 INFO:teuthology.orchestra.run.smithi107.stdout: "message": "Currently upgrading crash daemons", 2024-05-17T21:55:46.213 INFO:teuthology.orchestra.run.smithi107.stdout: "is_paused": false 2024-05-17T21:55:46.213 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-05-17T21:55:46.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:46.223+0000 7f2dcc6c6700 1 -- 172.21.15.107:0/2924722045 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f2db0078aa0 msgr2=0x7f2db007af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:55:46.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:46.223+0000 7f2dcc6c6700 1 --2- 172.21.15.107:0/2924722045 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f2db0078aa0 0x7f2db007af50 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f2dc0007fd0 tx=0x7f2dc0009040 comp rx=0 tx=0).stop 2024-05-17T21:55:46.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:46.223+0000 7f2dcc6c6700 1 -- 172.21.15.107:0/2924722045 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2dc40fdfd0 msgr2=0x7f2dc40702b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:55:46.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:46.223+0000 7f2dcc6c6700 1 --2- 172.21.15.107:0/2924722045 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2dc40fdfd0 0x7f2dc40702b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2db4009150 tx=0x7f2db4009e30 comp rx=0 tx=0).stop 2024-05-17T21:55:46.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:46.223+0000 7f2dcc6c6700 1 -- 172.21.15.107:0/2924722045 shutdown_connections 2024-05-17T21:55:46.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:46.223+0000 7f2dcc6c6700 1 --2- 172.21.15.107:0/2924722045 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f2db0078aa0 0x7f2db007af50 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:55:46.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:46.223+0000 7f2dcc6c6700 1 --2- 172.21.15.107:0/2924722045 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2dc40fce60 0x7f2dc4071c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:55:46.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:46.223+0000 7f2dcc6c6700 1 --2- 172.21.15.107:0/2924722045 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2dc40fdfd0 0x7f2dc40702b0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:55:46.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:46.223+0000 7f2dcc6c6700 1 -- 172.21.15.107:0/2924722045 >> 172.21.15.107:0/2924722045 conn(0x7f2dc40f8540 msgr2=0x7f2dc40f9be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:55:46.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:46.223+0000 7f2dcc6c6700 1 -- 172.21.15.107:0/2924722045 shutdown_connections 2024-05-17T21:55:46.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:46.223+0000 7f2dcc6c6700 1 -- 172.21.15.107:0/2924722045 wait complete. 2024-05-17T21:55:46.257 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:46 smithi138 ceph-mon[81376]: from='client.? 172.21.15.107:0/2723464323' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:55:46.258 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:46 smithi138 ceph-mon[81376]: from='client.? 172.21.15.107:0/3821141756' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-17T21:55:46.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:46.323+0000 7f4a4bad5700 1 -- 172.21.15.107:0/1584235698 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4a44066f50 msgr2=0x7f4a440ff9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:55:46.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:46.323+0000 7f4a4bad5700 1 --2- 172.21.15.107:0/1584235698 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4a44066f50 0x7f4a440ff9d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4a40004940 tx=0x7f4a40009e30 comp rx=0 tx=0).stop 2024-05-17T21:55:46.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:46.324+0000 7f4a4bad5700 1 -- 172.21.15.107:0/1584235698 shutdown_connections 2024-05-17T21:55:46.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:46.324+0000 7f4a4bad5700 1 --2- 172.21.15.107:0/1584235698 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a440fff10 0x7f4a44102330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:55:46.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:46.324+0000 7f4a4bad5700 1 --2- 172.21.15.107:0/1584235698 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4a44066f50 0x7f4a440ff9d0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:55:46.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:46.324+0000 7f4a4bad5700 1 -- 172.21.15.107:0/1584235698 >> 172.21.15.107:0/1584235698 conn(0x7f4a440f7480 msgr2=0x7f4a440f9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:55:46.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:46.324+0000 7f4a4bad5700 1 -- 172.21.15.107:0/1584235698 shutdown_connections 2024-05-17T21:55:46.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:46.324+0000 7f4a4bad5700 1 -- 172.21.15.107:0/1584235698 wait complete. 2024-05-17T21:55:46.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:46.324+0000 7f4a4bad5700 1 Processor -- start 2024-05-17T21:55:46.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:46.324+0000 7f4a4bad5700 1 -- start start 2024-05-17T21:55:46.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:46.324+0000 7f4a4bad5700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4a44066f50 0x7f4a440fd180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:55:46.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:46.324+0000 7f4a4bad5700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a440fff10 0x7f4a440fb8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:55:46.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:46.324+0000 7f4a4bad5700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a440fd810 con 0x7f4a44066f50 2024-05-17T21:55:46.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:46.324+0000 7f4a4bad5700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a440fbe10 con 0x7f4a440fff10 2024-05-17T21:55:46.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:46.324+0000 7f4a49871700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4a44066f50 0x7f4a440fd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:55:46.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:46.324+0000 7f4a49871700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4a44066f50 0x7f4a440fd180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34064/0 (socket says 172.21.15.107:34064) 2024-05-17T21:55:46.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:46.324+0000 7f4a49871700 1 -- 172.21.15.107:0/3144152587 learned_addr learned my addr 172.21.15.107:0/3144152587 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:55:46.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:46.325+0000 7f4a49070700 1 --2- 172.21.15.107:0/3144152587 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a440fff10 0x7f4a440fb8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:55:46.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:46.325+0000 7f4a49871700 1 -- 172.21.15.107:0/3144152587 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a440fff10 msgr2=0x7f4a440fb8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:55:46.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:46.325+0000 7f4a49871700 1 --2- 172.21.15.107:0/3144152587 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a440fff10 0x7f4a440fb8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:55:46.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:46.325+0000 7f4a49871700 1 -- 172.21.15.107:0/3144152587 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a40005040 con 0x7f4a44066f50 2024-05-17T21:55:46.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:46.325+0000 7f4a49070700 1 --2- 172.21.15.107:0/3144152587 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a440fff10 0x7f4a440fb8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T21:55:46.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:46.325+0000 7f4a49871700 1 --2- 172.21.15.107:0/3144152587 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4a44066f50 0x7f4a440fd180 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4a40000c00 tx=0x7f4a40009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:55:46.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:46.325+0000 7f4a3a7fc700 1 -- 172.21.15.107:0/3144152587 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4a4000e040 con 0x7f4a44066f50 2024-05-17T21:55:46.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:46.325+0000 7f4a4bad5700 1 -- 172.21.15.107:0/3144152587 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a440fc090 con 0x7f4a44066f50 2024-05-17T21:55:46.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:46.325+0000 7f4a3a7fc700 1 -- 172.21.15.107:0/3144152587 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4a4000bcc0 con 0x7f4a44066f50 2024-05-17T21:55:46.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:46.325+0000 7f4a3a7fc700 1 -- 172.21.15.107:0/3144152587 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4a4001c630 con 0x7f4a44066f50 2024-05-17T21:55:46.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:46.325+0000 7f4a4bad5700 1 -- 172.21.15.107:0/3144152587 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a440fc650 con 0x7f4a44066f50 2024-05-17T21:55:46.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:46.326+0000 7f4a4bad5700 1 -- 172.21.15.107:0/3144152587 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a4404c710 con 0x7f4a44066f50 2024-05-17T21:55:46.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:46.327+0000 7f4a3a7fc700 1 -- 172.21.15.107:0/3144152587 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 100184+0+0 (secure 0 0 0) 0x7f4a40013980 con 0x7f4a44066f50 2024-05-17T21:55:46.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:46.327+0000 7f4a3a7fc700 1 --2- 172.21.15.107:0/3144152587 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f4a30078aa0 0x7f4a3007af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:55:46.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:46.327+0000 7f4a49070700 1 --2- 172.21.15.107:0/3144152587 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f4a30078aa0 0x7f4a3007af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:55:46.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:46.327+0000 7f4a3a7fc700 1 -- 172.21.15.107:0/3144152587 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f4a4009bd00 con 0x7f4a44066f50 2024-05-17T21:55:46.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:46.328+0000 7f4a49070700 1 --2- 172.21.15.107:0/3144152587 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f4a30078aa0 0x7f4a3007af50 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4a34009b10 tx=0x7f4a34008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:55:46.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:46.332+0000 7f4a3a7fc700 1 -- 172.21.15.107:0/3144152587 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f4a4009f030 con 0x7f4a44066f50 2024-05-17T21:55:46.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:46 smithi107 ceph-mon[93224]: from='client.? 172.21.15.107:0/2723464323' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:55:46.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:46 smithi107 ceph-mon[93224]: from='client.? 172.21.15.107:0/3821141756' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-17T21:55:46.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:46.624+0000 7f4a4bad5700 1 -- 172.21.15.107:0/3144152587 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f4a44115b60 con 0x7f4a44066f50 2024-05-17T21:55:46.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:46.625+0000 7f4a3a7fc700 1 -- 172.21.15.107:0/3144152587 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f4a400637d0 con 0x7f4a44066f50 2024-05-17T21:55:46.617 INFO:teuthology.orchestra.run.smithi107.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-17T21:55:46.617 INFO:teuthology.orchestra.run.smithi107.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-17T21:55:46.617 INFO:teuthology.orchestra.run.smithi107.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-17T21:55:46.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:46.627+0000 7f4a4bad5700 1 -- 172.21.15.107:0/3144152587 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f4a30078aa0 msgr2=0x7f4a3007af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:55:46.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:46.627+0000 7f4a4bad5700 1 --2- 172.21.15.107:0/3144152587 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f4a30078aa0 0x7f4a3007af50 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4a34009b10 tx=0x7f4a34008040 comp rx=0 tx=0).stop 2024-05-17T21:55:46.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:46.627+0000 7f4a4bad5700 1 -- 172.21.15.107:0/3144152587 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4a44066f50 msgr2=0x7f4a440fd180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:55:46.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:46.627+0000 7f4a4bad5700 1 --2- 172.21.15.107:0/3144152587 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4a44066f50 0x7f4a440fd180 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4a40000c00 tx=0x7f4a40009e30 comp rx=0 tx=0).stop 2024-05-17T21:55:46.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:46.627+0000 7f4a4bad5700 1 -- 172.21.15.107:0/3144152587 shutdown_connections 2024-05-17T21:55:46.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:46.627+0000 7f4a4bad5700 1 --2- 172.21.15.107:0/3144152587 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f4a30078aa0 0x7f4a3007af50 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:55:46.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:46.627+0000 7f4a4bad5700 1 --2- 172.21.15.107:0/3144152587 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4a44066f50 0x7f4a440fd180 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:55:46.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:46.627+0000 7f4a4bad5700 1 --2- 172.21.15.107:0/3144152587 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a440fff10 0x7f4a440fb8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:55:46.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:46.627+0000 7f4a4bad5700 1 -- 172.21.15.107:0/3144152587 >> 172.21.15.107:0/3144152587 conn(0x7f4a440f7480 msgr2=0x7f4a440f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:55:46.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:46.627+0000 7f4a4bad5700 1 -- 172.21.15.107:0/3144152587 shutdown_connections 2024-05-17T21:55:46.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:55:46.627+0000 7f4a4bad5700 1 -- 172.21.15.107:0/3144152587 wait complete. 2024-05-17T21:55:47.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:47 smithi138 ceph-mon[81376]: from='client.34208 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:55:47.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:47 smithi138 ceph-mon[81376]: pgmap v39: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T21:55:47.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:47 smithi138 ceph-mon[81376]: from='client.? 172.21.15.107:0/3144152587' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-17T21:55:47.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:47 smithi107 ceph-mon[93224]: from='client.34208 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:55:47.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:47 smithi107 ceph-mon[93224]: pgmap v39: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T21:55:47.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:47 smithi107 ceph-mon[93224]: from='client.? 172.21.15.107:0/3144152587' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-17T21:55:49.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:48 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:49.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:48 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:49.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:48 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:55:49.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:48 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:49.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:48 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:49.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:48 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:55:50.005 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:49 smithi138 ceph-mon[81376]: pgmap v40: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T21:55:50.005 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:49 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:55:50.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:49 smithi107 ceph-mon[93224]: pgmap v40: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T21:55:50.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:49 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:55:52.304 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:51 smithi107 ceph-mon[93224]: pgmap v41: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T21:55:52.304 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:51 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:52.305 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:51 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:52.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:51 smithi138 ceph-mon[81376]: pgmap v41: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T21:55:52.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:51 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:52.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:51 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:53.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:53 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:53.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:53 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:53.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:53 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:53.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:53 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:54.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:54 smithi138 ceph-mon[81376]: pgmap v42: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T21:55:54.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:54 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:54.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:54 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:54.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:54 smithi107 ceph-mon[93224]: pgmap v42: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T21:55:54.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:54 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:54.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:54 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:55.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:55 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:55.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:55 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:55.415 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:55 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:55.415 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:55 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:56.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:56 smithi138 ceph-mon[81376]: pgmap v43: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T21:55:56.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:56 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:56.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:56 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:56.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:56 smithi107 ceph-mon[93224]: pgmap v43: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T21:55:56.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:56 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:56.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:56 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:58.385 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:58 smithi107 ceph-mon[93224]: pgmap v44: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T21:55:58.385 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:58 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:58.385 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:58 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:58.385 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:58 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:55:58.385 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:58 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T21:55:58.385 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:58 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:58.385 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:58 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:55:58.385 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:58 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:55:58.386 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:58 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:55:58.386 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:58 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:55:58.386 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:58 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:58.386 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:58 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi107"}]: dispatch 2024-05-17T21:55:58.386 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:58 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi107"}]': finished 2024-05-17T21:55:58.386 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:58 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi138"}]: dispatch 2024-05-17T21:55:58.386 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:58 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi138"}]': finished 2024-05-17T21:55:58.386 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:58 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-05-17T21:55:58.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:58 smithi138 ceph-mon[81376]: pgmap v44: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T21:55:58.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:58 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:58.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:58 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:58.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:58 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:55:58.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:58 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T21:55:58.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:58 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:58.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:58 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:55:58.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:58 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:55:58.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:58 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:55:58.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:58 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:55:58.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:58 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:55:58.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:58 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi107"}]: dispatch 2024-05-17T21:55:58.577 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:58 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi107"}]': finished 2024-05-17T21:55:58.577 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:58 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi138"}]: dispatch 2024-05-17T21:55:58.577 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:58 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi138"}]': finished 2024-05-17T21:55:58.577 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:58 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-05-17T21:55:59.257 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:59 smithi107 ceph-mon[93224]: Upgrade: Setting container_image for all crash 2024-05-17T21:55:59.257 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:59 smithi107 ceph-mon[93224]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-05-17T21:55:59.257 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:55:59 smithi107 ceph-mon[93224]: Upgrade: osd.0 is safe to restart 2024-05-17T21:55:59.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:59 smithi138 ceph-mon[81376]: Upgrade: Setting container_image for all crash 2024-05-17T21:55:59.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:59 smithi138 ceph-mon[81376]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-05-17T21:55:59.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:55:59 smithi138 ceph-mon[81376]: Upgrade: osd.0 is safe to restart 2024-05-17T21:56:00.152 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:00 smithi107 ceph-mon[93224]: pgmap v45: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T21:56:00.152 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:00 smithi107 ceph-mon[93224]: Upgrade: Updating osd.0 2024-05-17T21:56:00.152 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:00 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:56:00.152 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:00 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-17T21:56:00.153 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:00 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:56:00.153 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:00 smithi107 ceph-mon[93224]: Deploying daemon osd.0 on smithi107 2024-05-17T21:56:00.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:00 smithi138 ceph-mon[81376]: pgmap v45: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T21:56:00.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:00 smithi138 ceph-mon[81376]: Upgrade: Updating osd.0 2024-05-17T21:56:00.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:00 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:56:00.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:00 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-17T21:56:00.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:00 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:56:00.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:00 smithi138 ceph-mon[81376]: Deploying daemon osd.0 on smithi107 2024-05-17T21:56:01.527 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:01 smithi107 ceph-mon[93224]: pgmap v46: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T21:56:01.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:01 smithi138 ceph-mon[81376]: pgmap v46: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T21:56:02.602 INFO:journalctl@ceph.osd.0.smithi107.stdout:May 17 21:56:02 smithi107 systemd[1]: Stopping Ceph osd.0 for 3722731a-1496-11ef-bc9a-c7b262605968... 2024-05-17T21:56:03.101 INFO:journalctl@ceph.osd.0.smithi107.stdout:May 17 21:56:02 smithi107 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-0[47179]: 2024-05-17T21:56:02.784+0000 7fb015203700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-17T21:56:03.101 INFO:journalctl@ceph.osd.0.smithi107.stdout:May 17 21:56:02 smithi107 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-0[47179]: 2024-05-17T21:56:02.784+0000 7fb015203700 -1 osd.0 49 *** Got signal Terminated *** 2024-05-17T21:56:03.101 INFO:journalctl@ceph.osd.0.smithi107.stdout:May 17 21:56:02 smithi107 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-0[47179]: 2024-05-17T21:56:02.784+0000 7fb015203700 -1 osd.0 49 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-17T21:56:03.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:03 smithi138 ceph-mon[81376]: pgmap v47: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T21:56:03.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:03 smithi138 ceph-mon[81376]: osd.0 marked itself down and dead 2024-05-17T21:56:03.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:03 smithi107 ceph-mon[93224]: pgmap v47: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T21:56:03.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:03 smithi107 ceph-mon[93224]: osd.0 marked itself down and dead 2024-05-17T21:56:04.253 INFO:journalctl@ceph.osd.0.smithi107.stdout:May 17 21:56:04 smithi107 podman[101180]: 2024-05-17 21:56:03.999928433 +0000 UTC m=+1.344130113 container died f2d20009361db8dd55ff98b3c06bdaf93dbf7db7159538be16fc7e011dfd5577 (image=quay.ceph.io/ceph-ci/ceph@sha256:c200327b094672e93680375e82d2de1a08f5881ef304978f791cce179f847d60, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-0, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , RELEASE=quincy-9903f77, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-05-17T21:56:04.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:04 smithi107 ceph-mon[93224]: Health check failed: 1 osds down (OSD_DOWN) 2024-05-17T21:56:04.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:04 smithi107 ceph-mon[93224]: osdmap e50: 6 total, 5 up, 6 in 2024-05-17T21:56:04.602 INFO:journalctl@ceph.osd.0.smithi107.stdout:May 17 21:56:04 smithi107 podman[101180]: 2024-05-17 21:56:04.351280251 +0000 UTC m=+1.695481929 container cleanup f2d20009361db8dd55ff98b3c06bdaf93dbf7db7159538be16fc7e011dfd5577 (image=quay.ceph.io/ceph-ci/ceph@sha256:c200327b094672e93680375e82d2de1a08f5881ef304978f791cce179f847d60, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-0, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=quincy-9903f77, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-17T21:56:04.602 INFO:journalctl@ceph.osd.0.smithi107.stdout:May 17 21:56:04 smithi107 bash[101180]: ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-0 2024-05-17T21:56:04.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:04 smithi138 ceph-mon[81376]: Health check failed: 1 osds down (OSD_DOWN) 2024-05-17T21:56:04.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:04 smithi138 ceph-mon[81376]: osdmap e50: 6 total, 5 up, 6 in 2024-05-17T21:56:04.973 INFO:journalctl@ceph.osd.0.smithi107.stdout:May 17 21:56:04 smithi107 podman[101195]: 2024-05-17 21:56:04.65283694 +0000 UTC m=+0.655511078 container remove f2d20009361db8dd55ff98b3c06bdaf93dbf7db7159538be16fc7e011dfd5577 (image=quay.ceph.io/ceph-ci/ceph@sha256:c200327b094672e93680375e82d2de1a08f5881ef304978f791cce179f847d60, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=quincy-9903f77, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514) 2024-05-17T21:56:05.239 INFO:journalctl@ceph.osd.0.smithi107.stdout:May 17 21:56:04 smithi107 podman[101286]: 2024-05-17 21:56:04.885221481 +0000 UTC m=+0.016927538 image pull 8880f7613d0283efa7002505fdccdbc3e37d3b5da31c57fa22d62262c5dab66c quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b 2024-05-17T21:56:05.239 INFO:journalctl@ceph.osd.0.smithi107.stdout:May 17 21:56:05 smithi107 podman[101286]: 2024-05-17 21:56:05.004523535 +0000 UTC m=+0.136229618 container create 4b24975f155cf9b3710b60c0b745438af99b368bfbba68a38116e9b32b63186f (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-0-deactivate, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-05-17T21:56:05.239 INFO:journalctl@ceph.osd.0.smithi107.stdout:May 17 21:56:05 smithi107 podman[101286]: 2024-05-17 21:56:05.172940952 +0000 UTC m=+0.304646992 container init 4b24975f155cf9b3710b60c0b745438af99b368bfbba68a38116e9b32b63186f (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-05-17T21:56:05.239 INFO:journalctl@ceph.osd.0.smithi107.stdout:May 17 21:56:05 smithi107 podman[101286]: 2024-05-17 21:56:05.177594272 +0000 UTC m=+0.309300313 container start 4b24975f155cf9b3710b60c0b745438af99b368bfbba68a38116e9b32b63186f (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-0-deactivate, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.build-date=20240514, GIT_CLEAN=True) 2024-05-17T21:56:05.239 INFO:journalctl@ceph.osd.0.smithi107.stdout:May 17 21:56:05 smithi107 podman[101286]: 2024-05-17 21:56:05.247896771 +0000 UTC m=+0.379602814 container attach 4b24975f155cf9b3710b60c0b745438af99b368bfbba68a38116e9b32b63186f (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-05-17T21:56:05.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:05 smithi107 ceph-mon[93224]: pgmap v49: 65 pgs: 9 stale+active+clean, 56 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s 2024-05-17T21:56:05.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:05 smithi107 ceph-mon[93224]: osdmap e51: 6 total, 5 up, 6 in 2024-05-17T21:56:05.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:05 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:56:05.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:05 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:56:05.602 INFO:journalctl@ceph.osd.0.smithi107.stdout:May 17 21:56:05 smithi107 podman[101286]: 2024-05-17 21:56:05.474573677 +0000 UTC m=+0.606279718 container died 4b24975f155cf9b3710b60c0b745438af99b368bfbba68a38116e9b32b63186f (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-0-deactivate, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-05-17T21:56:05.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:05 smithi138 ceph-mon[81376]: pgmap v49: 65 pgs: 9 stale+active+clean, 56 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s 2024-05-17T21:56:05.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:05 smithi138 ceph-mon[81376]: osdmap e51: 6 total, 5 up, 6 in 2024-05-17T21:56:05.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:05 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:56:05.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:05 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:56:06.190 INFO:journalctl@ceph.osd.0.smithi107.stdout:May 17 21:56:06 smithi107 podman[101313]: 2024-05-17 21:56:06.160433334 +0000 UTC m=+0.679923266 container remove 4b24975f155cf9b3710b60c0b745438af99b368bfbba68a38116e9b32b63186f (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-0-deactivate, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514) 2024-05-17T21:56:06.190 INFO:journalctl@ceph.osd.0.smithi107.stdout:May 17 21:56:06 smithi107 systemd[1]: ceph-3722731a-1496-11ef-bc9a-c7b262605968@osd.0.service: Deactivated successfully. 2024-05-17T21:56:06.191 INFO:journalctl@ceph.osd.0.smithi107.stdout:May 17 21:56:06 smithi107 systemd[1]: Stopped Ceph osd.0 for 3722731a-1496-11ef-bc9a-c7b262605968. 2024-05-17T21:56:06.191 INFO:journalctl@ceph.osd.0.smithi107.stdout:May 17 21:56:06 smithi107 systemd[1]: ceph-3722731a-1496-11ef-bc9a-c7b262605968@osd.0.service: Consumed 49.536s CPU time. 2024-05-17T21:56:06.494 INFO:journalctl@ceph.osd.0.smithi107.stdout:May 17 21:56:06 smithi107 systemd[1]: Starting Ceph osd.0 for 3722731a-1496-11ef-bc9a-c7b262605968... 2024-05-17T21:56:06.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:06 smithi138 ceph-mon[81376]: Health check failed: Degraded data redundancy: 50/267 objects degraded (18.727%), 19 pgs degraded (PG_DEGRADED) 2024-05-17T21:56:06.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:06 smithi138 ceph-mon[81376]: mgrmap e38: smithi107.xhhyiw(active, since 92s), standbys: smithi138.napnll 2024-05-17T21:56:06.830 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:06 smithi107 ceph-mon[93224]: Health check failed: Degraded data redundancy: 50/267 objects degraded (18.727%), 19 pgs degraded (PG_DEGRADED) 2024-05-17T21:56:06.830 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:06 smithi107 ceph-mon[93224]: mgrmap e38: smithi107.xhhyiw(active, since 92s), standbys: smithi138.napnll 2024-05-17T21:56:06.830 INFO:journalctl@ceph.osd.0.smithi107.stdout:May 17 21:56:06 smithi107 podman[101412]: 2024-05-17 21:56:06.611696835 +0000 UTC m=+0.113931833 container create 98ca73994b4f3dd9f6d3a35953cb712fbcd72f6d35ebce97aed312c785173418 (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-0-activate, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-05-17T21:56:06.830 INFO:journalctl@ceph.osd.0.smithi107.stdout:May 17 21:56:06 smithi107 podman[101412]: 2024-05-17 21:56:06.515583026 +0000 UTC m=+0.017818005 image pull 8880f7613d0283efa7002505fdccdbc3e37d3b5da31c57fa22d62262c5dab66c quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b 2024-05-17T21:56:06.830 INFO:journalctl@ceph.osd.0.smithi107.stdout:May 17 21:56:06 smithi107 podman[101412]: 2024-05-17 21:56:06.839467831 +0000 UTC m=+0.341702822 container init 98ca73994b4f3dd9f6d3a35953cb712fbcd72f6d35ebce97aed312c785173418 (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-0-activate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux ) 2024-05-17T21:56:07.101 INFO:journalctl@ceph.osd.0.smithi107.stdout:May 17 21:56:06 smithi107 podman[101412]: 2024-05-17 21:56:06.843189878 +0000 UTC m=+0.345424857 container start 98ca73994b4f3dd9f6d3a35953cb712fbcd72f6d35ebce97aed312c785173418 (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-0-activate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-05-17T21:56:07.101 INFO:journalctl@ceph.osd.0.smithi107.stdout:May 17 21:56:06 smithi107 podman[101412]: 2024-05-17 21:56:06.922358276 +0000 UTC m=+0.424593269 container attach 98ca73994b4f3dd9f6d3a35953cb712fbcd72f6d35ebce97aed312c785173418 (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-0-activate, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240514) 2024-05-17T21:56:07.757 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:07 smithi107 ceph-mon[93224]: pgmap v51: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 383 B/s wr, 2 op/s; 50/267 objects degraded (18.727%) 2024-05-17T21:56:07.758 INFO:journalctl@ceph.osd.0.smithi107.stdout:May 17 21:56:07 smithi107 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-0-activate[101427]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-05-17T21:56:07.758 INFO:journalctl@ceph.osd.0.smithi107.stdout:May 17 21:56:07 smithi107 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-0-activate[101427]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-05-17T21:56:07.758 INFO:journalctl@ceph.osd.0.smithi107.stdout:May 17 21:56:07 smithi107 bash[101412]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-05-17T21:56:07.758 INFO:journalctl@ceph.osd.0.smithi107.stdout:May 17 21:56:07 smithi107 bash[101412]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-05-17T21:56:07.758 INFO:journalctl@ceph.osd.0.smithi107.stdout:May 17 21:56:07 smithi107 bash[101412]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-05-17T21:56:07.758 INFO:journalctl@ceph.osd.0.smithi107.stdout:May 17 21:56:07 smithi107 bash[101412]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-05-17T21:56:07.759 INFO:journalctl@ceph.osd.0.smithi107.stdout:May 17 21:56:07 smithi107 bash[101412]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-05-17T21:56:07.759 INFO:journalctl@ceph.osd.0.smithi107.stdout:May 17 21:56:07 smithi107 bash[101412]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-05-17T21:56:07.759 INFO:journalctl@ceph.osd.0.smithi107.stdout:May 17 21:56:07 smithi107 bash[101412]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-05-17T21:56:07.759 INFO:journalctl@ceph.osd.0.smithi107.stdout:May 17 21:56:07 smithi107 bash[101412]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-05-17T21:56:07.759 INFO:journalctl@ceph.osd.0.smithi107.stdout:May 17 21:56:07 smithi107 bash[101412]: --> ceph-volume raw activate successful for osd ID: 0 2024-05-17T21:56:07.759 INFO:journalctl@ceph.osd.0.smithi107.stdout:May 17 21:56:07 smithi107 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-0-activate[101427]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-05-17T21:56:07.759 INFO:journalctl@ceph.osd.0.smithi107.stdout:May 17 21:56:07 smithi107 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-0-activate[101427]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-05-17T21:56:07.759 INFO:journalctl@ceph.osd.0.smithi107.stdout:May 17 21:56:07 smithi107 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-0-activate[101427]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-05-17T21:56:07.759 INFO:journalctl@ceph.osd.0.smithi107.stdout:May 17 21:56:07 smithi107 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-0-activate[101427]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-05-17T21:56:07.759 INFO:journalctl@ceph.osd.0.smithi107.stdout:May 17 21:56:07 smithi107 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-0-activate[101427]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-05-17T21:56:07.759 INFO:journalctl@ceph.osd.0.smithi107.stdout:May 17 21:56:07 smithi107 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-0-activate[101427]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-05-17T21:56:07.759 INFO:journalctl@ceph.osd.0.smithi107.stdout:May 17 21:56:07 smithi107 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-0-activate[101427]: --> ceph-volume raw activate successful for osd ID: 0 2024-05-17T21:56:07.759 INFO:journalctl@ceph.osd.0.smithi107.stdout:May 17 21:56:07 smithi107 podman[101571]: 2024-05-17 21:56:07.495758795 +0000 UTC m=+0.017899296 container died 98ca73994b4f3dd9f6d3a35953cb712fbcd72f6d35ebce97aed312c785173418 (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-0-activate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-05-17T21:56:07.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:07 smithi138 ceph-mon[81376]: pgmap v51: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 383 B/s wr, 2 op/s; 50/267 objects degraded (18.727%) 2024-05-17T21:56:08.101 INFO:journalctl@ceph.osd.0.smithi107.stdout:May 17 21:56:07 smithi107 podman[101571]: 2024-05-17 21:56:07.992685485 +0000 UTC m=+0.514825987 container remove 98ca73994b4f3dd9f6d3a35953cb712fbcd72f6d35ebce97aed312c785173418 (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-0-activate, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-17T21:56:08.462 INFO:journalctl@ceph.osd.0.smithi107.stdout:May 17 21:56:08 smithi107 podman[101633]: 2024-05-17 21:56:08.108167985 +0000 UTC m=+0.017532871 image pull 8880f7613d0283efa7002505fdccdbc3e37d3b5da31c57fa22d62262c5dab66c quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b 2024-05-17T21:56:08.462 INFO:journalctl@ceph.osd.0.smithi107.stdout:May 17 21:56:08 smithi107 podman[101633]: 2024-05-17 21:56:08.218862233 +0000 UTC m=+0.128227165 container create b319bc93500dc6145afd31b133efe5891067329fb4d6df2ee5208b2921697b56 (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-0, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a) 2024-05-17T21:56:08.462 INFO:journalctl@ceph.osd.0.smithi107.stdout:May 17 21:56:08 smithi107 podman[101633]: 2024-05-17 21:56:08.388108648 +0000 UTC m=+0.297473528 container init b319bc93500dc6145afd31b133efe5891067329fb4d6df2ee5208b2921697b56 (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-17T21:56:08.462 INFO:journalctl@ceph.osd.0.smithi107.stdout:May 17 21:56:08 smithi107 podman[101633]: 2024-05-17 21:56:08.391510403 +0000 UTC m=+0.300875278 container start b319bc93500dc6145afd31b133efe5891067329fb4d6df2ee5208b2921697b56 (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-05-17T21:56:08.462 INFO:journalctl@ceph.osd.0.smithi107.stdout:May 17 21:56:08 smithi107 ceph-osd[101652]: -- 172.21.15.107:0/2406082199 <== mon.0 v2:172.21.15.107:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5557f50b7680 con 0x5557f515ed00 2024-05-17T21:56:08.462 INFO:journalctl@ceph.osd.0.smithi107.stdout:May 17 21:56:08 smithi107 bash[101633]: b319bc93500dc6145afd31b133efe5891067329fb4d6df2ee5208b2921697b56 2024-05-17T21:56:08.851 INFO:journalctl@ceph.osd.0.smithi107.stdout:May 17 21:56:08 smithi107 systemd[1]: Started Ceph osd.0 for 3722731a-1496-11ef-bc9a-c7b262605968. 2024-05-17T21:56:09.326 INFO:journalctl@ceph.osd.0.smithi107.stdout:May 17 21:56:08 smithi107 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-0[101648]: 2024-05-17T21:56:08.947+0000 7fcc7f5967c0 -1 Falling back to public interface 2024-05-17T21:56:09.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:09 smithi107 ceph-mon[93224]: pgmap v52: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 639 B/s rd, 383 B/s wr, 1 op/s; 50/267 objects degraded (18.727%) 2024-05-17T21:56:09.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:09 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:56:09.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:09 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:56:09.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:09 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:56:09.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:09 smithi138 ceph-mon[81376]: pgmap v52: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 639 B/s rd, 383 B/s wr, 1 op/s; 50/267 objects degraded (18.727%) 2024-05-17T21:56:09.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:09 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:56:09.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:09 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:56:09.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:09 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:56:11.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:11 smithi138 ceph-mon[81376]: pgmap v53: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.4 KiB/s rd, 383 B/s wr, 2 op/s; 50/267 objects degraded (18.727%) 2024-05-17T21:56:11.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:11 smithi107 ceph-mon[93224]: pgmap v53: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.4 KiB/s rd, 383 B/s wr, 2 op/s; 50/267 objects degraded (18.727%) 2024-05-17T21:56:13.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:13 smithi138 ceph-mon[81376]: pgmap v54: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 343 B/s wr, 2 op/s; 50/267 objects degraded (18.727%) 2024-05-17T21:56:13.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:13 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:56:13.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:13 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:56:13.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:13 smithi107 ceph-mon[93224]: pgmap v54: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 343 B/s wr, 2 op/s; 50/267 objects degraded (18.727%) 2024-05-17T21:56:13.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:13 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:56:13.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:13 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:56:14.351 INFO:journalctl@ceph.osd.0.smithi107.stdout:May 17 21:56:13 smithi107 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-0[101648]: 2024-05-17T21:56:13.956+0000 7fcc7f5967c0 -1 osd.0 49 log_to_monitors true 2024-05-17T21:56:14.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:14 smithi138 ceph-mon[81376]: from='osd.0 [v2:172.21.15.107:6802/347926010,v1:172.21.15.107:6803/347926010]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-17T21:56:14.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:14 smithi107 ceph-mon[93224]: from='osd.0 [v2:172.21.15.107:6802/347926010,v1:172.21.15.107:6803/347926010]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-17T21:56:15.753 INFO:journalctl@ceph.osd.0.smithi107.stdout:May 17 21:56:15 smithi107 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-0[101648]: 2024-05-17T21:56:15.490+0000 7fcc6c9a7700 -1 osd.0 49 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-17T21:56:15.753 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:15 smithi107 ceph-mon[93224]: pgmap v55: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.1 KiB/s rd, 307 B/s wr, 2 op/s; 50/267 objects degraded (18.727%) 2024-05-17T21:56:15.753 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:15 smithi107 ceph-mon[93224]: from='osd.0 [v2:172.21.15.107:6802/347926010,v1:172.21.15.107:6803/347926010]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-17T21:56:15.753 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:15 smithi107 ceph-mon[93224]: osdmap e52: 6 total, 5 up, 6 in 2024-05-17T21:56:15.754 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:15 smithi107 ceph-mon[93224]: from='osd.0 [v2:172.21.15.107:6802/347926010,v1:172.21.15.107:6803/347926010]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-05-17T21:56:15.754 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:15 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:56:15.754 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:15 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:56:15.754 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:15 smithi107 ceph-mon[93224]: from='osd.0 [v2:172.21.15.107:6802/347926010,v1:172.21.15.107:6803/347926010]' entity='osd.0' 2024-05-17T21:56:15.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:15 smithi138 ceph-mon[81376]: pgmap v55: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.1 KiB/s rd, 307 B/s wr, 2 op/s; 50/267 objects degraded (18.727%) 2024-05-17T21:56:15.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:15 smithi138 ceph-mon[81376]: from='osd.0 [v2:172.21.15.107:6802/347926010,v1:172.21.15.107:6803/347926010]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-17T21:56:15.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:15 smithi138 ceph-mon[81376]: osdmap e52: 6 total, 5 up, 6 in 2024-05-17T21:56:15.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:15 smithi138 ceph-mon[81376]: from='osd.0 [v2:172.21.15.107:6802/347926010,v1:172.21.15.107:6803/347926010]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-05-17T21:56:15.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:15 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:56:15.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:15 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:56:15.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:15 smithi138 ceph-mon[81376]: from='osd.0 [v2:172.21.15.107:6802/347926010,v1:172.21.15.107:6803/347926010]' entity='osd.0' 2024-05-17T21:56:16.725 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:16.734+0000 7fd9e1cd8700 1 -- 172.21.15.107:0/2201448497 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9dc0fe080 msgr2=0x7fd9dc0fe4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:56:16.725 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:16.734+0000 7fd9e1cd8700 1 --2- 172.21.15.107:0/2201448497 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9dc0fe080 0x7fd9dc0fe4f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd9d0004940 tx=0x7fd9d0009e30 comp rx=0 tx=0).stop 2024-05-17T21:56:16.725 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:16.734+0000 7fd9e1cd8700 1 -- 172.21.15.107:0/2201448497 shutdown_connections 2024-05-17T21:56:16.725 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:16.734+0000 7fd9e1cd8700 1 --2- 172.21.15.107:0/2201448497 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9dc0fe080 0x7fd9dc0fe4f0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:56:16.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:16.734+0000 7fd9e1cd8700 1 --2- 172.21.15.107:0/2201448497 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd9dc0fcf10 0x7fd9dc0fd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:56:16.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:16.734+0000 7fd9e1cd8700 1 -- 172.21.15.107:0/2201448497 >> 172.21.15.107:0/2201448497 conn(0x7fd9dc0f8560 msgr2=0x7fd9dc0fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:56:16.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:16.734+0000 7fd9e1cd8700 1 -- 172.21.15.107:0/2201448497 shutdown_connections 2024-05-17T21:56:16.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:16.734+0000 7fd9e1cd8700 1 -- 172.21.15.107:0/2201448497 wait complete. 2024-05-17T21:56:16.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:16.735+0000 7fd9e1cd8700 1 Processor -- start 2024-05-17T21:56:16.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:16.735+0000 7fd9e1cd8700 1 -- start start 2024-05-17T21:56:16.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:16.735+0000 7fd9e1cd8700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9dc0fcf10 0x7fd9dc071de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:56:16.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:16.735+0000 7fd9e1cd8700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd9dc0fe080 0x7fd9dc0702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:56:16.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:16.735+0000 7fd9e1cd8700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9dc0708f0 con 0x7fd9dc0fcf10 2024-05-17T21:56:16.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:16.735+0000 7fd9e1cd8700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9dc070a60 con 0x7fd9dc0fe080 2024-05-17T21:56:16.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:16.735+0000 7fd9db7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9dc0fcf10 0x7fd9dc071de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:56:16.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:16.735+0000 7fd9db7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9dc0fcf10 0x7fd9dc071de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55698/0 (socket says 172.21.15.107:55698) 2024-05-17T21:56:16.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:16.735+0000 7fd9db7fe700 1 -- 172.21.15.107:0/411076669 learned_addr learned my addr 172.21.15.107:0/411076669 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:56:16.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:16.735+0000 7fd9daffd700 1 --2- 172.21.15.107:0/411076669 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd9dc0fe080 0x7fd9dc0702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:56:16.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:16.736+0000 7fd9db7fe700 1 -- 172.21.15.107:0/411076669 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd9dc0fe080 msgr2=0x7fd9dc0702b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:56:16.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:16.736+0000 7fd9db7fe700 1 --2- 172.21.15.107:0/411076669 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd9dc0fe080 0x7fd9dc0702b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:56:16.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:16.736+0000 7fd9db7fe700 1 -- 172.21.15.107:0/411076669 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9d0005040 con 0x7fd9dc0fcf10 2024-05-17T21:56:16.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:16.736+0000 7fd9db7fe700 1 --2- 172.21.15.107:0/411076669 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9dc0fcf10 0x7fd9dc071de0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc00b790 tx=0x7fd9cc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:56:16.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:16.736+0000 7fd9c3fff700 1 -- 172.21.15.107:0/411076669 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd9cc00f6f0 con 0x7fd9dc0fcf10 2024-05-17T21:56:16.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:16.736+0000 7fd9e1cd8700 1 -- 172.21.15.107:0/411076669 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9dc070d40 con 0x7fd9dc0fcf10 2024-05-17T21:56:16.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:16.736+0000 7fd9e1cd8700 1 -- 172.21.15.107:0/411076669 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9dc115ab0 con 0x7fd9dc0fcf10 2024-05-17T21:56:16.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:16.736+0000 7fd9c3fff700 1 -- 172.21.15.107:0/411076669 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd9cc00fd20 con 0x7fd9dc0fcf10 2024-05-17T21:56:16.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:16.736+0000 7fd9c3fff700 1 -- 172.21.15.107:0/411076669 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd9cc011400 con 0x7fd9dc0fcf10 2024-05-17T21:56:16.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:16.737+0000 7fd9e1cd8700 1 -- 172.21.15.107:0/411076669 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9c80052e0 con 0x7fd9dc0fcf10 2024-05-17T21:56:16.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:16.738+0000 7fd9c3fff700 1 -- 172.21.15.107:0/411076669 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7fd9cc00f850 con 0x7fd9dc0fcf10 2024-05-17T21:56:16.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:16.738+0000 7fd9c3fff700 1 --2- 172.21.15.107:0/411076669 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fd9c4078990 0x7fd9c407ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:56:16.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:16.738+0000 7fd9daffd700 1 --2- 172.21.15.107:0/411076669 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fd9c4078990 0x7fd9c407ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:56:16.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:16.738+0000 7fd9c3fff700 1 -- 172.21.15.107:0/411076669 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fd9cc09a400 con 0x7fd9dc0fcf10 2024-05-17T21:56:16.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:16.739+0000 7fd9daffd700 1 --2- 172.21.15.107:0/411076669 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fd9c4078990 0x7fd9c407ae40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd9d000b970 tx=0x7fd9d0009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:56:16.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:16.743+0000 7fd9c3fff700 1 -- 172.21.15.107:0/411076669 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7fd9cc0620f0 con 0x7fd9dc0fcf10 2024-05-17T21:56:16.794 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:16 smithi107 ceph-mon[93224]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-05-17T21:56:16.794 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:16 smithi107 ceph-mon[93224]: osd.0 [v2:172.21.15.107:6802/347926010,v1:172.21.15.107:6803/347926010] boot 2024-05-17T21:56:16.794 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:16 smithi107 ceph-mon[93224]: osdmap e53: 6 total, 6 up, 6 in 2024-05-17T21:56:16.794 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:16 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-17T21:56:16.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:16 smithi138 ceph-mon[81376]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-05-17T21:56:16.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:16 smithi138 ceph-mon[81376]: osd.0 [v2:172.21.15.107:6802/347926010,v1:172.21.15.107:6803/347926010] boot 2024-05-17T21:56:16.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:16 smithi138 ceph-mon[81376]: osdmap e53: 6 total, 6 up, 6 in 2024-05-17T21:56:16.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:16 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-17T21:56:16.970 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:16.979+0000 7fd9e1cd8700 1 -- 172.21.15.107:0/411076669 --> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd9c8000bf0 con 0x7fd9c4078990 2024-05-17T21:56:16.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:16.981+0000 7fd9c3fff700 1 -- 172.21.15.107:0/411076669 <== mgr.34100 v2:172.21.15.107:6800/3205268226 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fd9c8000bf0 con 0x7fd9c4078990 2024-05-17T21:56:16.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:16.983+0000 7fd9e1cd8700 1 -- 172.21.15.107:0/411076669 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fd9c4078990 msgr2=0x7fd9c407ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:56:16.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:16.983+0000 7fd9e1cd8700 1 --2- 172.21.15.107:0/411076669 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fd9c4078990 0x7fd9c407ae40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd9d000b970 tx=0x7fd9d0009e30 comp rx=0 tx=0).stop 2024-05-17T21:56:16.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:16.983+0000 7fd9e1cd8700 1 -- 172.21.15.107:0/411076669 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9dc0fcf10 msgr2=0x7fd9dc071de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:56:16.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:16.983+0000 7fd9e1cd8700 1 --2- 172.21.15.107:0/411076669 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9dc0fcf10 0x7fd9dc071de0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc00b790 tx=0x7fd9cc009250 comp rx=0 tx=0).stop 2024-05-17T21:56:16.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:16.983+0000 7fd9e1cd8700 1 -- 172.21.15.107:0/411076669 shutdown_connections 2024-05-17T21:56:16.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:16.983+0000 7fd9e1cd8700 1 --2- 172.21.15.107:0/411076669 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fd9c4078990 0x7fd9c407ae40 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:56:16.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:16.983+0000 7fd9e1cd8700 1 --2- 172.21.15.107:0/411076669 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9dc0fcf10 0x7fd9dc071de0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:56:16.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:16.983+0000 7fd9e1cd8700 1 --2- 172.21.15.107:0/411076669 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd9dc0fe080 0x7fd9dc0702b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:56:16.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:16.983+0000 7fd9e1cd8700 1 -- 172.21.15.107:0/411076669 >> 172.21.15.107:0/411076669 conn(0x7fd9dc0f8560 msgr2=0x7fd9dc0f9b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:56:16.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:16.984+0000 7fd9e1cd8700 1 -- 172.21.15.107:0/411076669 shutdown_connections 2024-05-17T21:56:16.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:16.984+0000 7fd9e1cd8700 1 -- 172.21.15.107:0/411076669 wait complete. 2024-05-17T21:56:16.985 INFO:teuthology.orchestra.run.smithi107.stdout:true 2024-05-17T21:56:17.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.090+0000 7fd26035a700 1 -- 172.21.15.107:0/1940011024 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2580662c0 msgr2=0x7fd2580ff9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:56:17.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.090+0000 7fd26035a700 1 --2- 172.21.15.107:0/1940011024 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2580662c0 0x7fd2580ff9a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd254004940 tx=0x7fd254009e30 comp rx=0 tx=0).stop 2024-05-17T21:56:17.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.090+0000 7fd26035a700 1 -- 172.21.15.107:0/1940011024 shutdown_connections 2024-05-17T21:56:17.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.090+0000 7fd26035a700 1 --2- 172.21.15.107:0/1940011024 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2580ffee0 0x7fd258102300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:56:17.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.090+0000 7fd26035a700 1 --2- 172.21.15.107:0/1940011024 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2580662c0 0x7fd2580ff9a0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:56:17.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.090+0000 7fd26035a700 1 -- 172.21.15.107:0/1940011024 >> 172.21.15.107:0/1940011024 conn(0x7fd2580f7430 msgr2=0x7fd2580f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:56:17.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.090+0000 7fd26035a700 1 -- 172.21.15.107:0/1940011024 shutdown_connections 2024-05-17T21:56:17.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.090+0000 7fd26035a700 1 -- 172.21.15.107:0/1940011024 wait complete. 2024-05-17T21:56:17.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.091+0000 7fd26035a700 1 Processor -- start 2024-05-17T21:56:17.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.091+0000 7fd26035a700 1 -- start start 2024-05-17T21:56:17.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.091+0000 7fd26035a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2580662c0 0x7fd258107090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:56:17.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.091+0000 7fd26035a700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2580ffee0 0x7fd2581075d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:56:17.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.091+0000 7fd26035a700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd258107c20 con 0x7fd2580ffee0 2024-05-17T21:56:17.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.091+0000 7fd26035a700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd258108470 con 0x7fd2580662c0 2024-05-17T21:56:17.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.091+0000 7fd25d8f5700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2580ffee0 0x7fd2581075d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:56:17.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.091+0000 7fd25d8f5700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2580ffee0 0x7fd2581075d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55718/0 (socket says 172.21.15.107:55718) 2024-05-17T21:56:17.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.091+0000 7fd25d8f5700 1 -- 172.21.15.107:0/1207313345 learned_addr learned my addr 172.21.15.107:0/1207313345 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:56:17.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.091+0000 7fd25e0f6700 1 --2- 172.21.15.107:0/1207313345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2580662c0 0x7fd258107090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:56:17.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.091+0000 7fd25d8f5700 1 -- 172.21.15.107:0/1207313345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2580662c0 msgr2=0x7fd258107090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:56:17.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.091+0000 7fd25d8f5700 1 --2- 172.21.15.107:0/1207313345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2580662c0 0x7fd258107090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:56:17.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.091+0000 7fd25d8f5700 1 -- 172.21.15.107:0/1207313345 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd254005040 con 0x7fd2580ffee0 2024-05-17T21:56:17.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.091+0000 7fd25d8f5700 1 --2- 172.21.15.107:0/1207313345 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2580ffee0 0x7fd2581075d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd248002a50 tx=0x7fd2480046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:56:17.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.092+0000 7fd24effd700 1 -- 172.21.15.107:0/1207313345 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd248007cc0 con 0x7fd2580ffee0 2024-05-17T21:56:17.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.092+0000 7fd26035a700 1 -- 172.21.15.107:0/1207313345 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd258108750 con 0x7fd2580ffee0 2024-05-17T21:56:17.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.092+0000 7fd26035a700 1 -- 172.21.15.107:0/1207313345 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd258108d10 con 0x7fd2580ffee0 2024-05-17T21:56:17.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.092+0000 7fd24effd700 1 -- 172.21.15.107:0/1207313345 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd24800f040 con 0x7fd2580ffee0 2024-05-17T21:56:17.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.092+0000 7fd24effd700 1 -- 172.21.15.107:0/1207313345 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd24800da30 con 0x7fd2580ffee0 2024-05-17T21:56:17.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.093+0000 7fd26035a700 1 -- 172.21.15.107:0/1207313345 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd25805f5c0 con 0x7fd2580ffee0 2024-05-17T21:56:17.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.094+0000 7fd24effd700 1 -- 172.21.15.107:0/1207313345 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7fd24800dc70 con 0x7fd2580ffee0 2024-05-17T21:56:17.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.094+0000 7fd24effd700 1 --2- 172.21.15.107:0/1207313345 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fd244078990 0x7fd24407ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:56:17.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.094+0000 7fd24effd700 1 -- 172.21.15.107:0/1207313345 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fd24809a3b0 con 0x7fd2580ffee0 2024-05-17T21:56:17.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.094+0000 7fd25e0f6700 1 --2- 172.21.15.107:0/1207313345 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fd244078990 0x7fd24407ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:56:17.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.095+0000 7fd25e0f6700 1 --2- 172.21.15.107:0/1207313345 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fd244078990 0x7fd24407ae40 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd258106e20 tx=0x7fd254009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:56:17.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.099+0000 7fd24effd700 1 -- 172.21.15.107:0/1207313345 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7fd2480620a0 con 0x7fd2580ffee0 2024-05-17T21:56:17.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.333+0000 7fd26035a700 1 -- 172.21.15.107:0/1207313345 --> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd258108ff0 con 0x7fd244078990 2024-05-17T21:56:17.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.334+0000 7fd24effd700 1 -- 172.21.15.107:0/1207313345 <== mgr.34100 v2:172.21.15.107:6800/3205268226 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fd258108ff0 con 0x7fd244078990 2024-05-17T21:56:17.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.336+0000 7fd26035a700 1 -- 172.21.15.107:0/1207313345 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fd244078990 msgr2=0x7fd24407ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:56:17.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.336+0000 7fd26035a700 1 --2- 172.21.15.107:0/1207313345 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fd244078990 0x7fd24407ae40 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd258106e20 tx=0x7fd254009e30 comp rx=0 tx=0).stop 2024-05-17T21:56:17.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.336+0000 7fd26035a700 1 -- 172.21.15.107:0/1207313345 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2580ffee0 msgr2=0x7fd2581075d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:56:17.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.336+0000 7fd26035a700 1 --2- 172.21.15.107:0/1207313345 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2580ffee0 0x7fd2581075d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd248002a50 tx=0x7fd2480046b0 comp rx=0 tx=0).stop 2024-05-17T21:56:17.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.336+0000 7fd26035a700 1 -- 172.21.15.107:0/1207313345 shutdown_connections 2024-05-17T21:56:17.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.336+0000 7fd26035a700 1 --2- 172.21.15.107:0/1207313345 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fd244078990 0x7fd24407ae40 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:56:17.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.336+0000 7fd26035a700 1 --2- 172.21.15.107:0/1207313345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2580662c0 0x7fd258107090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:56:17.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.336+0000 7fd26035a700 1 --2- 172.21.15.107:0/1207313345 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2580ffee0 0x7fd2581075d0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:56:17.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.336+0000 7fd26035a700 1 -- 172.21.15.107:0/1207313345 >> 172.21.15.107:0/1207313345 conn(0x7fd2580f7430 msgr2=0x7fd2580fba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:56:17.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.336+0000 7fd26035a700 1 -- 172.21.15.107:0/1207313345 shutdown_connections 2024-05-17T21:56:17.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.336+0000 7fd26035a700 1 -- 172.21.15.107:0/1207313345 wait complete. 2024-05-17T21:56:17.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.442+0000 7f94fdd1e700 1 -- 172.21.15.107:0/4253223185 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94f80fcf60 msgr2=0x7f94f80fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:56:17.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.442+0000 7f94fdd1e700 1 --2- 172.21.15.107:0/4253223185 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94f80fcf60 0x7f94f80fd3d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f94e0004940 tx=0x7f94e0009e30 comp rx=0 tx=0).stop 2024-05-17T21:56:17.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.443+0000 7f94fdd1e700 1 -- 172.21.15.107:0/4253223185 shutdown_connections 2024-05-17T21:56:17.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.443+0000 7f94fdd1e700 1 --2- 172.21.15.107:0/4253223185 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94f80fcf60 0x7f94f80fd3d0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:56:17.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.443+0000 7f94fdd1e700 1 --2- 172.21.15.107:0/4253223185 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f94f80fbcb0 0x7f94f80fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:56:17.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.443+0000 7f94fdd1e700 1 -- 172.21.15.107:0/4253223185 >> 172.21.15.107:0/4253223185 conn(0x7f94f80f7430 msgr2=0x7f94f80f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:56:17.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.443+0000 7f94fdd1e700 1 -- 172.21.15.107:0/4253223185 shutdown_connections 2024-05-17T21:56:17.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.443+0000 7f94fdd1e700 1 -- 172.21.15.107:0/4253223185 wait complete. 2024-05-17T21:56:17.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.443+0000 7f94fdd1e700 1 Processor -- start 2024-05-17T21:56:17.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.443+0000 7f94fdd1e700 1 -- start start 2024-05-17T21:56:17.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.444+0000 7f94fdd1e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94f80fbcb0 0x7f94f810c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:56:17.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.444+0000 7f94fdd1e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f94f80fcf60 0x7f94f810cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:56:17.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.444+0000 7f94fdd1e700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f810d3e0 con 0x7f94f80fbcb0 2024-05-17T21:56:17.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.444+0000 7f94fdd1e700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f810dc30 con 0x7f94f80fcf60 2024-05-17T21:56:17.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.444+0000 7f94ef7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94f80fbcb0 0x7f94f810c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:56:17.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.444+0000 7f94ef7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94f80fbcb0 0x7f94f810c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55730/0 (socket says 172.21.15.107:55730) 2024-05-17T21:56:17.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.444+0000 7f94ef7fe700 1 -- 172.21.15.107:0/4251212873 learned_addr learned my addr 172.21.15.107:0/4251212873 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:56:17.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.444+0000 7f94f77fe700 1 --2- 172.21.15.107:0/4251212873 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f94f80fcf60 0x7f94f810cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:56:17.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.444+0000 7f94ef7fe700 1 -- 172.21.15.107:0/4251212873 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f94f80fcf60 msgr2=0x7f94f810cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:56:17.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.444+0000 7f94ef7fe700 1 --2- 172.21.15.107:0/4251212873 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f94f80fcf60 0x7f94f810cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:56:17.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.444+0000 7f94ef7fe700 1 -- 172.21.15.107:0/4251212873 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94e0005040 con 0x7f94f80fbcb0 2024-05-17T21:56:17.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.445+0000 7f94ef7fe700 1 --2- 172.21.15.107:0/4251212873 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94f80fbcb0 0x7f94f810c8a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f94e800eb40 tx=0x7f94e800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:56:17.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.445+0000 7f94f4ff9700 1 -- 172.21.15.107:0/4251212873 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f94e8007cc0 con 0x7f94f80fbcb0 2024-05-17T21:56:17.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.445+0000 7f94fdd1e700 1 -- 172.21.15.107:0/4251212873 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94f810deb0 con 0x7f94f80fbcb0 2024-05-17T21:56:17.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.445+0000 7f94f4ff9700 1 -- 172.21.15.107:0/4251212873 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f94e800f040 con 0x7f94f80fbcb0 2024-05-17T21:56:17.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.445+0000 7f94fdd1e700 1 -- 172.21.15.107:0/4251212873 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94f810e3f0 con 0x7f94f80fbcb0 2024-05-17T21:56:17.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.445+0000 7f94f4ff9700 1 -- 172.21.15.107:0/4251212873 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f94e800ca10 con 0x7f94f80fbcb0 2024-05-17T21:56:17.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.446+0000 7f94fdd1e700 1 -- 172.21.15.107:0/4251212873 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94d80052e0 con 0x7f94f80fbcb0 2024-05-17T21:56:17.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.447+0000 7f94f4ff9700 1 -- 172.21.15.107:0/4251212873 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f94e800cb70 con 0x7f94f80fbcb0 2024-05-17T21:56:17.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.447+0000 7f94f4ff9700 1 --2- 172.21.15.107:0/4251212873 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f94e4078990 0x7f94e407ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:56:17.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.448+0000 7f94f4ff9700 1 -- 172.21.15.107:0/4251212873 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f94e809a160 con 0x7f94f80fbcb0 2024-05-17T21:56:17.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.448+0000 7f94f77fe700 1 --2- 172.21.15.107:0/4251212873 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f94e4078990 0x7f94e407ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:56:17.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.448+0000 7f94f77fe700 1 --2- 172.21.15.107:0/4251212873 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f94e4078990 0x7f94e407ae40 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f94e0009b30 tx=0x7f94e0009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:56:17.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.453+0000 7f94f4ff9700 1 -- 172.21.15.107:0/4251212873 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f94e8061f00 con 0x7f94f80fbcb0 2024-05-17T21:56:17.704 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:17 smithi107 ceph-mon[93224]: pgmap v57: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 258 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.1 KiB/s rd, 2 op/s; 50/267 objects degraded (18.727%) 2024-05-17T21:56:17.704 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:17 smithi107 ceph-mon[93224]: from='client.34220 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:56:17.704 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:17 smithi107 ceph-mon[93224]: osdmap e54: 6 total, 6 up, 6 in 2024-05-17T21:56:17.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.763+0000 7f94fdd1e700 1 -- 172.21.15.107:0/4251212873 --> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f94d8000bf0 con 0x7f94e4078990 2024-05-17T21:56:17.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.774+0000 7f94f4ff9700 1 -- 172.21.15.107:0/4251212873 <== mgr.34100 v2:172.21.15.107:6800/3205268226 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f94d8000bf0 con 0x7f94e4078990 2024-05-17T21:56:17.765 INFO:teuthology.orchestra.run.smithi107.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-17T21:56:17.765 INFO:teuthology.orchestra.run.smithi107.stdout:alertmanager.smithi107 smithi107 *:9093,9094 running (3m) 4s ago 12m 25.5M - 0.25.0 c8568f914cd2 bac2a944ae42 2024-05-17T21:56:17.765 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi107 smithi107 running (35s) 4s ago 12m 6669k - 19.0.0-3794-g931809a9 8880f7613d02 face35de205b 2024-05-17T21:56:17.765 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi138 smithi138 running (30s) 26s ago 11m 6665k - 19.0.0-3794-g931809a9 8880f7613d02 ad59e33e6d43 2024-05-17T21:56:17.765 INFO:teuthology.orchestra.run.smithi107.stdout:grafana.smithi107 smithi107 *:3000 running (3m) 4s ago 12m 80.3M - 9.4.7 954c08fa6188 8c747e777bc5 2024-05-17T21:56:17.765 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.drqody smithi107 running (6m) 4s ago 6m 103M - 17.2.7-952-g9903f776 d15a36853796 0d851bc912f0 2024-05-17T21:56:17.765 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.idmbgj smithi107 running (6m) 4s ago 6m 13.5M - 17.2.7-952-g9903f776 d15a36853796 1348b09ab29e 2024-05-17T21:56:17.765 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi138.gwbtyd smithi138 running (6m) 26s ago 6m 152M - 17.2.7-952-g9903f776 d15a36853796 fd2ba75354c0 2024-05-17T21:56:17.765 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi138.zckrvs smithi138 running (6m) 26s ago 6m 14.1M - 17.2.7-952-g9903f776 d15a36853796 a1cfd2fe4cda 2024-05-17T21:56:17.766 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi107.xhhyiw smithi107 *:8443,9283,8765 running (3m) 4s ago 13m 581M - 19.0.0-3794-g931809a9 8880f7613d02 5acf3e511342 2024-05-17T21:56:17.766 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi138.napnll smithi138 *:8443,9283,8765 running (2m) 26s ago 11m 531M - 19.0.0-3794-g931809a9 8880f7613d02 d627761dbd0d 2024-05-17T21:56:17.766 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi107 smithi107 running (113s) 4s ago 14m 113M 2048M 19.0.0-3794-g931809a9 8880f7613d02 9717f3169f2f 2024-05-17T21:56:17.766 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi138 smithi138 running (75s) 26s ago 11m 99.9M 2048M 19.0.0-3794-g931809a9 8880f7613d02 55aeed62bf4d 2024-05-17T21:56:17.766 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi107 smithi107 *:9100 running (3m) 4s ago 12m 17.6M - 1.5.0 0da6a335fe13 49bec71f5487 2024-05-17T21:56:17.766 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi138 smithi138 *:9100 running (3m) 26s ago 11m 15.9M - 1.5.0 0da6a335fe13 e52824376658 2024-05-17T21:56:17.766 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0 smithi107 running (9s) 4s ago 10m 79.4M 1557M 19.0.0-3794-g931809a9 8880f7613d02 b319bc93500d 2024-05-17T21:56:17.766 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1 smithi107 running (10m) 4s ago 10m 224M 1557M 17.2.7-952-g9903f776 d15a36853796 e5d3e99844a9 2024-05-17T21:56:17.766 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2 smithi107 running (9m) 4s ago 9m 181M 1557M 17.2.7-952-g9903f776 d15a36853796 e044d7b78088 2024-05-17T21:56:17.766 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3 smithi138 running (9m) 26s ago 9m 228M 2581M 17.2.7-952-g9903f776 d15a36853796 8c980cc8af87 2024-05-17T21:56:17.766 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4 smithi138 running (8m) 26s ago 8m 204M 2581M 17.2.7-952-g9903f776 d15a36853796 b306a36a744e 2024-05-17T21:56:17.766 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5 smithi138 running (8m) 26s ago 8m 190M 2581M 17.2.7-952-g9903f776 d15a36853796 7c8d8ed74974 2024-05-17T21:56:17.766 INFO:teuthology.orchestra.run.smithi107.stdout:prometheus.smithi107 smithi107 *:9095 running (2m) 4s ago 11m 56.3M - 2.43.0 a07b618ecd1d bf5f609bbd6e 2024-05-17T21:56:17.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.776+0000 7f94fdd1e700 1 -- 172.21.15.107:0/4251212873 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f94e4078990 msgr2=0x7f94e407ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:56:17.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.776+0000 7f94fdd1e700 1 --2- 172.21.15.107:0/4251212873 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f94e4078990 0x7f94e407ae40 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f94e0009b30 tx=0x7f94e0009e30 comp rx=0 tx=0).stop 2024-05-17T21:56:17.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.776+0000 7f94fdd1e700 1 -- 172.21.15.107:0/4251212873 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94f80fbcb0 msgr2=0x7f94f810c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:56:17.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.776+0000 7f94fdd1e700 1 --2- 172.21.15.107:0/4251212873 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94f80fbcb0 0x7f94f810c8a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f94e800eb40 tx=0x7f94e800b040 comp rx=0 tx=0).stop 2024-05-17T21:56:17.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.776+0000 7f94fdd1e700 1 -- 172.21.15.107:0/4251212873 shutdown_connections 2024-05-17T21:56:17.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.776+0000 7f94fdd1e700 1 --2- 172.21.15.107:0/4251212873 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f94e4078990 0x7f94e407ae40 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:56:17.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.776+0000 7f94fdd1e700 1 --2- 172.21.15.107:0/4251212873 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94f80fbcb0 0x7f94f810c8a0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:56:17.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.776+0000 7f94fdd1e700 1 --2- 172.21.15.107:0/4251212873 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f94f80fcf60 0x7f94f810cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:56:17.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.776+0000 7f94fdd1e700 1 -- 172.21.15.107:0/4251212873 >> 172.21.15.107:0/4251212873 conn(0x7f94f80f7430 msgr2=0x7f94f80f8a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:56:17.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.776+0000 7f94fdd1e700 1 -- 172.21.15.107:0/4251212873 shutdown_connections 2024-05-17T21:56:17.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.776+0000 7f94fdd1e700 1 -- 172.21.15.107:0/4251212873 wait complete. 2024-05-17T21:56:17.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:17 smithi138 ceph-mon[81376]: pgmap v57: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 258 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.1 KiB/s rd, 2 op/s; 50/267 objects degraded (18.727%) 2024-05-17T21:56:17.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:17 smithi138 ceph-mon[81376]: from='client.34220 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:56:17.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:17 smithi138 ceph-mon[81376]: osdmap e54: 6 total, 6 up, 6 in 2024-05-17T21:56:17.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.880+0000 7f7e2b920700 1 -- 172.21.15.107:0/2466004376 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e240fcf10 msgr2=0x7f7e240fd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:56:17.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.880+0000 7f7e2b920700 1 --2- 172.21.15.107:0/2466004376 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e240fcf10 0x7f7e240fd320 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7e14004940 tx=0x7f7e14009e30 comp rx=0 tx=0).stop 2024-05-17T21:56:17.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.880+0000 7f7e2b920700 1 -- 172.21.15.107:0/2466004376 shutdown_connections 2024-05-17T21:56:17.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.880+0000 7f7e2b920700 1 --2- 172.21.15.107:0/2466004376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e240fe080 0x7f7e240fe4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:56:17.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.880+0000 7f7e2b920700 1 --2- 172.21.15.107:0/2466004376 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e240fcf10 0x7f7e240fd320 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:56:17.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.880+0000 7f7e2b920700 1 -- 172.21.15.107:0/2466004376 >> 172.21.15.107:0/2466004376 conn(0x7f7e240f8560 msgr2=0x7f7e240fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:56:17.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.880+0000 7f7e2b920700 1 -- 172.21.15.107:0/2466004376 shutdown_connections 2024-05-17T21:56:17.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.880+0000 7f7e2b920700 1 -- 172.21.15.107:0/2466004376 wait complete. 2024-05-17T21:56:17.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.881+0000 7f7e2b920700 1 Processor -- start 2024-05-17T21:56:17.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.881+0000 7f7e2b920700 1 -- start start 2024-05-17T21:56:17.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.881+0000 7f7e2b920700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e240fcf10 0x7f7e2410d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:56:17.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.881+0000 7f7e2b920700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e240fe080 0x7f7e2410df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:56:17.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.881+0000 7f7e2b920700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e2410e520 con 0x7f7e240fcf10 2024-05-17T21:56:17.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.881+0000 7f7e2b920700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e2410ed70 con 0x7f7e240fe080 2024-05-17T21:56:17.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.881+0000 7f7e296bc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e240fcf10 0x7f7e2410d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:56:17.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.881+0000 7f7e296bc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e240fcf10 0x7f7e2410d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55744/0 (socket says 172.21.15.107:55744) 2024-05-17T21:56:17.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.881+0000 7f7e296bc700 1 -- 172.21.15.107:0/3986441883 learned_addr learned my addr 172.21.15.107:0/3986441883 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:56:17.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.881+0000 7f7e28ebb700 1 --2- 172.21.15.107:0/3986441883 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e240fe080 0x7f7e2410df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:56:17.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.881+0000 7f7e296bc700 1 -- 172.21.15.107:0/3986441883 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e240fe080 msgr2=0x7f7e2410df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:56:17.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.881+0000 7f7e296bc700 1 --2- 172.21.15.107:0/3986441883 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e240fe080 0x7f7e2410df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:56:17.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.881+0000 7f7e296bc700 1 -- 172.21.15.107:0/3986441883 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e14005040 con 0x7f7e240fcf10 2024-05-17T21:56:17.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.881+0000 7f7e296bc700 1 --2- 172.21.15.107:0/3986441883 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e240fcf10 0x7f7e2410d9e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7e14009150 tx=0x7f7e14009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:56:17.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.882+0000 7f7e19ffb700 1 -- 172.21.15.107:0/3986441883 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7e1400e040 con 0x7f7e240fcf10 2024-05-17T21:56:17.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.882+0000 7f7e2b920700 1 -- 172.21.15.107:0/3986441883 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e2410eff0 con 0x7f7e240fcf10 2024-05-17T21:56:17.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.882+0000 7f7e19ffb700 1 -- 172.21.15.107:0/3986441883 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7e1400bcc0 con 0x7f7e240fcf10 2024-05-17T21:56:17.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.882+0000 7f7e19ffb700 1 -- 172.21.15.107:0/3986441883 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7e1401c630 con 0x7f7e240fcf10 2024-05-17T21:56:17.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.882+0000 7f7e2b920700 1 -- 172.21.15.107:0/3986441883 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e2410f530 con 0x7f7e240fcf10 2024-05-17T21:56:17.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.883+0000 7f7e2b920700 1 -- 172.21.15.107:0/3986441883 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e2405f5c0 con 0x7f7e240fcf10 2024-05-17T21:56:17.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.884+0000 7f7e19ffb700 1 -- 172.21.15.107:0/3986441883 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f7e14013420 con 0x7f7e240fcf10 2024-05-17T21:56:17.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.884+0000 7f7e19ffb700 1 --2- 172.21.15.107:0/3986441883 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f7e10078990 0x7f7e1007ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:56:17.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.884+0000 7f7e19ffb700 1 -- 172.21.15.107:0/3986441883 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f7e1409b2f0 con 0x7f7e240fcf10 2024-05-17T21:56:17.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.884+0000 7f7e28ebb700 1 --2- 172.21.15.107:0/3986441883 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f7e10078990 0x7f7e1007ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:56:17.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.885+0000 7f7e28ebb700 1 --2- 172.21.15.107:0/3986441883 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f7e10078990 0x7f7e1007ae40 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f7e20007fd0 tx=0x7f7e20009040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:56:17.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:17.889+0000 7f7e19ffb700 1 -- 172.21.15.107:0/3986441883 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f7e14063080 con 0x7f7e240fcf10 2024-05-17T21:56:18.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:18.194+0000 7f7e2b920700 1 -- 172.21.15.107:0/3986441883 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f7e2410f810 con 0x7f7e240fcf10 2024-05-17T21:56:18.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:18.195+0000 7f7e19ffb700 1 -- 172.21.15.107:0/3986441883 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+839 (secure 0 0 0) 0x7f7e14062a20 con 0x7f7e240fcf10 2024-05-17T21:56:18.186 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-05-17T21:56:18.187 INFO:teuthology.orchestra.run.smithi107.stdout: "mon": { 2024-05-17T21:56:18.187 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev)": 2 2024-05-17T21:56:18.187 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-05-17T21:56:18.187 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr": { 2024-05-17T21:56:18.187 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev)": 2 2024-05-17T21:56:18.187 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-05-17T21:56:18.187 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": { 2024-05-17T21:56:18.187 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-952-g9903f776 (9903f7767386a9d9efadb7c1effdcd3064d71804) quincy (stable)": 5, 2024-05-17T21:56:18.187 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev)": 1 2024-05-17T21:56:18.187 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-05-17T21:56:18.187 INFO:teuthology.orchestra.run.smithi107.stdout: "mds": { 2024-05-17T21:56:18.187 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-952-g9903f776 (9903f7767386a9d9efadb7c1effdcd3064d71804) quincy (stable)": 4 2024-05-17T21:56:18.187 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-05-17T21:56:18.187 INFO:teuthology.orchestra.run.smithi107.stdout: "overall": { 2024-05-17T21:56:18.187 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-952-g9903f776 (9903f7767386a9d9efadb7c1effdcd3064d71804) quincy (stable)": 9, 2024-05-17T21:56:18.188 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev)": 5 2024-05-17T21:56:18.188 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-05-17T21:56:18.188 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-05-17T21:56:18.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:18.197+0000 7f7e2b920700 1 -- 172.21.15.107:0/3986441883 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f7e10078990 msgr2=0x7f7e1007ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:56:18.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:18.197+0000 7f7e2b920700 1 --2- 172.21.15.107:0/3986441883 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f7e10078990 0x7f7e1007ae40 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f7e20007fd0 tx=0x7f7e20009040 comp rx=0 tx=0).stop 2024-05-17T21:56:18.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:18.197+0000 7f7e2b920700 1 -- 172.21.15.107:0/3986441883 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e240fcf10 msgr2=0x7f7e2410d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:56:18.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:18.197+0000 7f7e2b920700 1 --2- 172.21.15.107:0/3986441883 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e240fcf10 0x7f7e2410d9e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7e14009150 tx=0x7f7e14009e30 comp rx=0 tx=0).stop 2024-05-17T21:56:18.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:18.198+0000 7f7e2b920700 1 -- 172.21.15.107:0/3986441883 shutdown_connections 2024-05-17T21:56:18.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:18.198+0000 7f7e2b920700 1 --2- 172.21.15.107:0/3986441883 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f7e10078990 0x7f7e1007ae40 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:56:18.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:18.198+0000 7f7e2b920700 1 --2- 172.21.15.107:0/3986441883 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e240fcf10 0x7f7e2410d9e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:56:18.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:18.198+0000 7f7e2b920700 1 --2- 172.21.15.107:0/3986441883 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e240fe080 0x7f7e2410df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:56:18.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:18.198+0000 7f7e2b920700 1 -- 172.21.15.107:0/3986441883 >> 172.21.15.107:0/3986441883 conn(0x7f7e240f8560 msgr2=0x7f7e240f9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:56:18.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:18.198+0000 7f7e2b920700 1 -- 172.21.15.107:0/3986441883 shutdown_connections 2024-05-17T21:56:18.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:18.198+0000 7f7e2b920700 1 -- 172.21.15.107:0/3986441883 wait complete. 2024-05-17T21:56:18.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:18.303+0000 7f7711158700 1 -- 172.21.15.107:0/1253467979 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f770c0fce80 msgr2=0x7f770c0fd290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:56:18.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:18.303+0000 7f7711158700 1 --2- 172.21.15.107:0/1253467979 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f770c0fce80 0x7f770c0fd290 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f76fc004940 tx=0x7f76fc009e30 comp rx=0 tx=0).stop 2024-05-17T21:56:18.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:18.303+0000 7f7711158700 1 -- 172.21.15.107:0/1253467979 shutdown_connections 2024-05-17T21:56:18.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:18.303+0000 7f7711158700 1 --2- 172.21.15.107:0/1253467979 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f770c0fdff0 0x7f770c0fe460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:56:18.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:18.303+0000 7f7711158700 1 --2- 172.21.15.107:0/1253467979 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f770c0fce80 0x7f770c0fd290 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:56:18.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:18.303+0000 7f7711158700 1 -- 172.21.15.107:0/1253467979 >> 172.21.15.107:0/1253467979 conn(0x7f770c0f8540 msgr2=0x7f770c0fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:56:18.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:18.303+0000 7f7711158700 1 -- 172.21.15.107:0/1253467979 shutdown_connections 2024-05-17T21:56:18.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:18.303+0000 7f7711158700 1 -- 172.21.15.107:0/1253467979 wait complete. 2024-05-17T21:56:18.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:18.304+0000 7f7711158700 1 Processor -- start 2024-05-17T21:56:18.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:18.304+0000 7f7711158700 1 -- start start 2024-05-17T21:56:18.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:18.305+0000 7f7711158700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f770c0fce80 0x7f770c10d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:56:18.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:18.305+0000 7f7711158700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f770c0fdff0 0x7f770c10def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:56:18.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:18.305+0000 7f7711158700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f770c10e4f0 con 0x7f770c0fdff0 2024-05-17T21:56:18.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:18.305+0000 7f7711158700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f770c10ed40 con 0x7f770c0fce80 2024-05-17T21:56:18.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:18.305+0000 7f770a59c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f770c0fdff0 0x7f770c10def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:56:18.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:18.305+0000 7f770a59c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f770c0fdff0 0x7f770c10def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55766/0 (socket says 172.21.15.107:55766) 2024-05-17T21:56:18.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:18.305+0000 7f770a59c700 1 -- 172.21.15.107:0/1764788317 learned_addr learned my addr 172.21.15.107:0/1764788317 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:56:18.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:18.305+0000 7f770ad9d700 1 --2- 172.21.15.107:0/1764788317 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f770c0fce80 0x7f770c10d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:56:18.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:18.305+0000 7f770a59c700 1 -- 172.21.15.107:0/1764788317 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f770c0fce80 msgr2=0x7f770c10d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:56:18.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:18.305+0000 7f770a59c700 1 --2- 172.21.15.107:0/1764788317 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f770c0fce80 0x7f770c10d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:56:18.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:18.305+0000 7f770a59c700 1 -- 172.21.15.107:0/1764788317 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76fc005040 con 0x7f770c0fdff0 2024-05-17T21:56:18.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:18.305+0000 7f770a59c700 1 --2- 172.21.15.107:0/1764788317 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f770c0fdff0 0x7f770c10def0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f770000eb40 tx=0x7f770000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:56:18.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:18.305+0000 7f76f37fe700 1 -- 172.21.15.107:0/1764788317 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7700007cc0 con 0x7f770c0fdff0 2024-05-17T21:56:18.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:18.306+0000 7f76f37fe700 1 -- 172.21.15.107:0/1764788317 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f770000f040 con 0x7f770c0fdff0 2024-05-17T21:56:18.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:18.306+0000 7f76f37fe700 1 -- 172.21.15.107:0/1764788317 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f770000ca10 con 0x7f770c0fdff0 2024-05-17T21:56:18.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:18.306+0000 7f7711158700 1 -- 172.21.15.107:0/1764788317 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f770c10efc0 con 0x7f770c0fdff0 2024-05-17T21:56:18.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:18.306+0000 7f7711158700 1 -- 172.21.15.107:0/1764788317 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f770c10f4b0 con 0x7f770c0fdff0 2024-05-17T21:56:18.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:18.306+0000 7f7711158700 1 -- 172.21.15.107:0/1764788317 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f770c05f5c0 con 0x7f770c0fdff0 2024-05-17T21:56:18.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:18.307+0000 7f76f37fe700 1 -- 172.21.15.107:0/1764788317 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f770000cc80 con 0x7f770c0fdff0 2024-05-17T21:56:18.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:18.308+0000 7f76f37fe700 1 --2- 172.21.15.107:0/1764788317 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f76f4078990 0x7f76f407ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:56:18.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:18.308+0000 7f76f37fe700 1 -- 172.21.15.107:0/1764788317 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f770009a4f0 con 0x7f770c0fdff0 2024-05-17T21:56:18.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:18.308+0000 7f770ad9d700 1 --2- 172.21.15.107:0/1764788317 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f76f4078990 0x7f76f407ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:56:18.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:18.308+0000 7f770ad9d700 1 --2- 172.21.15.107:0/1764788317 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f76f4078990 0x7f76f407ae40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f76fc009b30 tx=0x7f76fc009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:56:18.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:18.312+0000 7f76f37fe700 1 -- 172.21.15.107:0/1764788317 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f7700062290 con 0x7f770c0fdff0 2024-05-17T21:56:18.557 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:18 smithi107 ceph-mon[93224]: from='client.34224 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:56:18.557 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:18 smithi107 ceph-mon[93224]: from='client.34228 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:56:18.557 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:18 smithi107 ceph-mon[93224]: from='client.? 172.21.15.107:0/3986441883' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:56:18.557 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:18 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:56:18.557 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:18 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:56:18.557 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:18 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:56:18.557 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:18 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T21:56:18.557 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:18 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:56:18.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:18.586+0000 7f7711158700 1 -- 172.21.15.107:0/1764788317 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f770c04bed0 con 0x7f770c0fdff0 2024-05-17T21:56:18.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:18.586+0000 7f76f37fe700 1 -- 172.21.15.107:0/1764788317 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1910 (secure 0 0 0) 0x7f7700061c30 con 0x7f770c0fdff0 2024-05-17T21:56:18.577 INFO:teuthology.orchestra.run.smithi107.stdout:e15 2024-05-17T21:56:18.577 INFO:teuthology.orchestra.run.smithi107.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-17T21:56:18.577 INFO:teuthology.orchestra.run.smithi107.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-17T21:56:18.578 INFO:teuthology.orchestra.run.smithi107.stdout:legacy client fscid: 1 2024-05-17T21:56:18.578 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:56:18.578 INFO:teuthology.orchestra.run.smithi107.stdout:Filesystem 'cephfs' (1) 2024-05-17T21:56:18.578 INFO:teuthology.orchestra.run.smithi107.stdout:fs_name cephfs 2024-05-17T21:56:18.578 INFO:teuthology.orchestra.run.smithi107.stdout:epoch 15 2024-05-17T21:56:18.578 INFO:teuthology.orchestra.run.smithi107.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-17T21:56:18.578 INFO:teuthology.orchestra.run.smithi107.stdout:created 2024-05-17T21:49:16.836333+0000 2024-05-17T21:56:18.578 INFO:teuthology.orchestra.run.smithi107.stdout:modified 2024-05-17T21:49:34.873628+0000 2024-05-17T21:56:18.578 INFO:teuthology.orchestra.run.smithi107.stdout:tableserver 0 2024-05-17T21:56:18.578 INFO:teuthology.orchestra.run.smithi107.stdout:root 0 2024-05-17T21:56:18.578 INFO:teuthology.orchestra.run.smithi107.stdout:session_timeout 60 2024-05-17T21:56:18.578 INFO:teuthology.orchestra.run.smithi107.stdout:session_autoclose 300 2024-05-17T21:56:18.578 INFO:teuthology.orchestra.run.smithi107.stdout:max_file_size 1099511627776 2024-05-17T21:56:18.578 INFO:teuthology.orchestra.run.smithi107.stdout:max_xattr_size 65536 2024-05-17T21:56:18.578 INFO:teuthology.orchestra.run.smithi107.stdout:required_client_features {} 2024-05-17T21:56:18.578 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure 0 2024-05-17T21:56:18.579 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure_osd_epoch 46 2024-05-17T21:56:18.579 INFO:teuthology.orchestra.run.smithi107.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-17T21:56:18.579 INFO:teuthology.orchestra.run.smithi107.stdout:max_mds 1 2024-05-17T21:56:18.579 INFO:teuthology.orchestra.run.smithi107.stdout:in 0 2024-05-17T21:56:18.579 INFO:teuthology.orchestra.run.smithi107.stdout:up {0=24259} 2024-05-17T21:56:18.579 INFO:teuthology.orchestra.run.smithi107.stdout:failed 2024-05-17T21:56:18.579 INFO:teuthology.orchestra.run.smithi107.stdout:damaged 2024-05-17T21:56:18.579 INFO:teuthology.orchestra.run.smithi107.stdout:stopped 2024-05-17T21:56:18.579 INFO:teuthology.orchestra.run.smithi107.stdout:data_pools [3] 2024-05-17T21:56:18.579 INFO:teuthology.orchestra.run.smithi107.stdout:metadata_pool 2 2024-05-17T21:56:18.579 INFO:teuthology.orchestra.run.smithi107.stdout:inline_data enabled 2024-05-17T21:56:18.579 INFO:teuthology.orchestra.run.smithi107.stdout:balancer 2024-05-17T21:56:18.579 INFO:teuthology.orchestra.run.smithi107.stdout:bal_rank_mask -1 2024-05-17T21:56:18.579 INFO:teuthology.orchestra.run.smithi107.stdout:standby_count_wanted 1 2024-05-17T21:56:18.579 INFO:teuthology.orchestra.run.smithi107.stdout:qdb_cluster leader: 0 members: 2024-05-17T21:56:18.579 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi138.gwbtyd{0:24259} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.138:6824/2779634919,v1:172.21.15.138:6825/2779634919] compat {c=[1],r=[1],i=[7ff]}] 2024-05-17T21:56:18.580 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.drqody{0:14486} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.107:6828/2275807100,v1:172.21.15.107:6829/2275807100] compat {c=[1],r=[1],i=[7ff]}] 2024-05-17T21:56:18.580 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:56:18.580 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:56:18.580 INFO:teuthology.orchestra.run.smithi107.stdout:Standby daemons: 2024-05-17T21:56:18.580 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:56:18.580 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.idmbgj{-1:14498} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.107:6826/1460186079,v1:172.21.15.107:6827/1460186079] compat {c=[1],r=[1],i=[7ff]}] 2024-05-17T21:56:18.580 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi138.zckrvs{-1:24267} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.138:6826/170628661,v1:172.21.15.138:6827/170628661] compat {c=[1],r=[1],i=[7ff]}] 2024-05-17T21:56:18.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:18.588+0000 7f7711158700 1 -- 172.21.15.107:0/1764788317 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f76f4078990 msgr2=0x7f76f407ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:56:18.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:18.588+0000 7f7711158700 1 --2- 172.21.15.107:0/1764788317 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f76f4078990 0x7f76f407ae40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f76fc009b30 tx=0x7f76fc009e30 comp rx=0 tx=0).stop 2024-05-17T21:56:18.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:18.589+0000 7f7711158700 1 -- 172.21.15.107:0/1764788317 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f770c0fdff0 msgr2=0x7f770c10def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:56:18.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:18.589+0000 7f7711158700 1 --2- 172.21.15.107:0/1764788317 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f770c0fdff0 0x7f770c10def0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f770000eb40 tx=0x7f770000b040 comp rx=0 tx=0).stop 2024-05-17T21:56:18.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:18.589+0000 7f7711158700 1 -- 172.21.15.107:0/1764788317 shutdown_connections 2024-05-17T21:56:18.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:18.589+0000 7f7711158700 1 --2- 172.21.15.107:0/1764788317 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f76f4078990 0x7f76f407ae40 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:56:18.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:18.589+0000 7f7711158700 1 --2- 172.21.15.107:0/1764788317 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f770c0fce80 0x7f770c10d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:56:18.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:18.589+0000 7f7711158700 1 --2- 172.21.15.107:0/1764788317 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f770c0fdff0 0x7f770c10def0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:56:18.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:18.589+0000 7f7711158700 1 -- 172.21.15.107:0/1764788317 >> 172.21.15.107:0/1764788317 conn(0x7f770c0f8540 msgr2=0x7f770c0f9b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:56:18.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:18.589+0000 7f7711158700 1 -- 172.21.15.107:0/1764788317 shutdown_connections 2024-05-17T21:56:18.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:18.589+0000 7f7711158700 1 -- 172.21.15.107:0/1764788317 wait complete. 2024-05-17T21:56:18.581 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 15 2024-05-17T21:56:18.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:18.693+0000 7fd66b9fa700 1 -- 172.21.15.107:0/1165591043 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd6640fbcb0 msgr2=0x7fd6640fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:56:18.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:18.693+0000 7fd66b9fa700 1 --2- 172.21.15.107:0/1165591043 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd6640fbcb0 0x7fd6640fc0c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd660004940 tx=0x7fd660009e30 comp rx=0 tx=0).stop 2024-05-17T21:56:18.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:18.693+0000 7fd66b9fa700 1 -- 172.21.15.107:0/1165591043 shutdown_connections 2024-05-17T21:56:18.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:18.693+0000 7fd66b9fa700 1 --2- 172.21.15.107:0/1165591043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6640fcf60 0x7fd6640fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:56:18.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:18.693+0000 7fd66b9fa700 1 --2- 172.21.15.107:0/1165591043 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd6640fbcb0 0x7fd6640fc0c0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:56:18.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:18.693+0000 7fd66b9fa700 1 -- 172.21.15.107:0/1165591043 >> 172.21.15.107:0/1165591043 conn(0x7fd6640f7430 msgr2=0x7fd6640f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:56:18.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:18.694+0000 7fd66b9fa700 1 -- 172.21.15.107:0/1165591043 shutdown_connections 2024-05-17T21:56:18.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:18.694+0000 7fd66b9fa700 1 -- 172.21.15.107:0/1165591043 wait complete. 2024-05-17T21:56:18.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:18.694+0000 7fd66b9fa700 1 Processor -- start 2024-05-17T21:56:18.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:18.694+0000 7fd66b9fa700 1 -- start start 2024-05-17T21:56:18.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:18.694+0000 7fd66b9fa700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd6640fbcb0 0x7fd66410c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:56:18.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:18.694+0000 7fd66b9fa700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6640fcf60 0x7fd66410ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:56:18.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:18.694+0000 7fd66b9fa700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd66410d440 con 0x7fd6640fbcb0 2024-05-17T21:56:18.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:18.694+0000 7fd66b9fa700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd66410dc90 con 0x7fd6640fcf60 2024-05-17T21:56:18.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:18.694+0000 7fd669796700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd6640fbcb0 0x7fd66410c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:56:18.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:18.694+0000 7fd669796700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd6640fbcb0 0x7fd66410c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55782/0 (socket says 172.21.15.107:55782) 2024-05-17T21:56:18.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:18.694+0000 7fd669796700 1 -- 172.21.15.107:0/3162679956 learned_addr learned my addr 172.21.15.107:0/3162679956 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:56:18.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:18.695+0000 7fd668f95700 1 --2- 172.21.15.107:0/3162679956 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6640fcf60 0x7fd66410ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:56:18.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:18.695+0000 7fd669796700 1 -- 172.21.15.107:0/3162679956 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6640fcf60 msgr2=0x7fd66410ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:56:18.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:18.695+0000 7fd669796700 1 --2- 172.21.15.107:0/3162679956 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6640fcf60 0x7fd66410ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:56:18.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:18.695+0000 7fd669796700 1 -- 172.21.15.107:0/3162679956 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd660005040 con 0x7fd6640fbcb0 2024-05-17T21:56:18.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:18.695+0000 7fd669796700 1 --2- 172.21.15.107:0/3162679956 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd6640fbcb0 0x7fd66410c900 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd66000b3b0 tx=0x7fd660009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:56:18.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:18.695+0000 7fd659ffb700 1 -- 172.21.15.107:0/3162679956 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd66000e040 con 0x7fd6640fbcb0 2024-05-17T21:56:18.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:18.695+0000 7fd659ffb700 1 -- 172.21.15.107:0/3162679956 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd66000bbf0 con 0x7fd6640fbcb0 2024-05-17T21:56:18.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:18.695+0000 7fd66b9fa700 1 -- 172.21.15.107:0/3162679956 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd66410df10 con 0x7fd6640fbcb0 2024-05-17T21:56:18.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:18.695+0000 7fd659ffb700 1 -- 172.21.15.107:0/3162679956 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd66001c630 con 0x7fd6640fbcb0 2024-05-17T21:56:18.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:18.695+0000 7fd66b9fa700 1 -- 172.21.15.107:0/3162679956 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd66410e450 con 0x7fd6640fbcb0 2024-05-17T21:56:18.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:18.696+0000 7fd66b9fa700 1 -- 172.21.15.107:0/3162679956 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd66405f5c0 con 0x7fd6640fbcb0 2024-05-17T21:56:18.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:18.697+0000 7fd659ffb700 1 -- 172.21.15.107:0/3162679956 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7fd66000bd60 con 0x7fd6640fbcb0 2024-05-17T21:56:18.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:18.698+0000 7fd659ffb700 1 --2- 172.21.15.107:0/3162679956 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fd65007aba0 0x7fd65007d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:56:18.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:18.698+0000 7fd668f95700 1 --2- 172.21.15.107:0/3162679956 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fd65007aba0 0x7fd65007d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:56:18.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:18.698+0000 7fd659ffb700 1 -- 172.21.15.107:0/3162679956 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fd66009b9e0 con 0x7fd6640fbcb0 2024-05-17T21:56:18.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:18.698+0000 7fd668f95700 1 --2- 172.21.15.107:0/3162679956 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fd65007aba0 0x7fd65007d050 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd654004f40 tx=0x7fd65400b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:56:18.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:18.702+0000 7fd659ffb700 1 -- 172.21.15.107:0/3162679956 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7fd660063770 con 0x7fd6640fbcb0 2024-05-17T21:56:18.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:18 smithi138 ceph-mon[81376]: from='client.34224 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:56:18.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:18 smithi138 ceph-mon[81376]: from='client.34228 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:56:18.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:18 smithi138 ceph-mon[81376]: from='client.? 172.21.15.107:0/3986441883' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:56:18.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:18 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:56:18.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:18 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:56:18.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:18 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:56:18.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:18 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T21:56:18.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:18 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:56:18.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:18.932+0000 7fd66b9fa700 1 -- 172.21.15.107:0/3162679956 --> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd66410e8a0 con 0x7fd65007aba0 2024-05-17T21:56:18.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:18.933+0000 7fd659ffb700 1 -- 172.21.15.107:0/3162679956 <== mgr.34100 v2:172.21.15.107:6800/3205268226 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fd66410e8a0 con 0x7fd65007aba0 2024-05-17T21:56:18.924 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-05-17T21:56:18.924 INFO:teuthology.orchestra.run.smithi107.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b", 2024-05-17T21:56:18.924 INFO:teuthology.orchestra.run.smithi107.stdout: "in_progress": true, 2024-05-17T21:56:18.924 INFO:teuthology.orchestra.run.smithi107.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-17T21:56:18.925 INFO:teuthology.orchestra.run.smithi107.stdout: "services_complete": [ 2024-05-17T21:56:18.925 INFO:teuthology.orchestra.run.smithi107.stdout: "crash", 2024-05-17T21:56:18.925 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr", 2024-05-17T21:56:18.925 INFO:teuthology.orchestra.run.smithi107.stdout: "mon" 2024-05-17T21:56:18.925 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-05-17T21:56:18.925 INFO:teuthology.orchestra.run.smithi107.stdout: "progress": "7/21 daemons upgraded", 2024-05-17T21:56:18.925 INFO:teuthology.orchestra.run.smithi107.stdout: "message": "Currently upgrading osd daemons", 2024-05-17T21:56:18.925 INFO:teuthology.orchestra.run.smithi107.stdout: "is_paused": false 2024-05-17T21:56:18.926 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-05-17T21:56:18.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:18.935+0000 7fd66b9fa700 1 -- 172.21.15.107:0/3162679956 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fd65007aba0 msgr2=0x7fd65007d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:56:18.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:18.935+0000 7fd66b9fa700 1 --2- 172.21.15.107:0/3162679956 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fd65007aba0 0x7fd65007d050 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd654004f40 tx=0x7fd65400b040 comp rx=0 tx=0).stop 2024-05-17T21:56:18.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:18.935+0000 7fd66b9fa700 1 -- 172.21.15.107:0/3162679956 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd6640fbcb0 msgr2=0x7fd66410c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:56:18.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:18.935+0000 7fd66b9fa700 1 --2- 172.21.15.107:0/3162679956 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd6640fbcb0 0x7fd66410c900 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd66000b3b0 tx=0x7fd660009e30 comp rx=0 tx=0).stop 2024-05-17T21:56:18.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:18.936+0000 7fd66b9fa700 1 -- 172.21.15.107:0/3162679956 shutdown_connections 2024-05-17T21:56:18.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:18.936+0000 7fd66b9fa700 1 --2- 172.21.15.107:0/3162679956 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fd65007aba0 0x7fd65007d050 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:56:18.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:18.936+0000 7fd66b9fa700 1 --2- 172.21.15.107:0/3162679956 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd6640fbcb0 0x7fd66410c900 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:56:18.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:18.936+0000 7fd66b9fa700 1 --2- 172.21.15.107:0/3162679956 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6640fcf60 0x7fd66410ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:56:18.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:18.936+0000 7fd66b9fa700 1 -- 172.21.15.107:0/3162679956 >> 172.21.15.107:0/3162679956 conn(0x7fd6640f7430 msgr2=0x7fd6640f8b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:56:18.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:18.936+0000 7fd66b9fa700 1 -- 172.21.15.107:0/3162679956 shutdown_connections 2024-05-17T21:56:18.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:18.936+0000 7fd66b9fa700 1 -- 172.21.15.107:0/3162679956 wait complete. 2024-05-17T21:56:19.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:19.039+0000 7fb6dca52700 1 -- 172.21.15.107:0/1129520563 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb6d80fe080 msgr2=0x7fb6d80fe4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:56:19.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:19.039+0000 7fb6dca52700 1 --2- 172.21.15.107:0/1129520563 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb6d80fe080 0x7fb6d80fe4f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb6c0004940 tx=0x7fb6c0009e30 comp rx=0 tx=0).stop 2024-05-17T21:56:19.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:19.039+0000 7fb6dca52700 1 -- 172.21.15.107:0/1129520563 shutdown_connections 2024-05-17T21:56:19.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:19.039+0000 7fb6dca52700 1 --2- 172.21.15.107:0/1129520563 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb6d80fe080 0x7fb6d80fe4f0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:56:19.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:19.039+0000 7fb6dca52700 1 --2- 172.21.15.107:0/1129520563 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6d80fce80 0x7fb6d80fd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:56:19.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:19.039+0000 7fb6dca52700 1 -- 172.21.15.107:0/1129520563 >> 172.21.15.107:0/1129520563 conn(0x7fb6d80f8590 msgr2=0x7fb6d80fa9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:56:19.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:19.039+0000 7fb6dca52700 1 -- 172.21.15.107:0/1129520563 shutdown_connections 2024-05-17T21:56:19.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:19.039+0000 7fb6dca52700 1 -- 172.21.15.107:0/1129520563 wait complete. 2024-05-17T21:56:19.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:19.040+0000 7fb6dca52700 1 Processor -- start 2024-05-17T21:56:19.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:19.040+0000 7fb6dca52700 1 -- start start 2024-05-17T21:56:19.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:19.040+0000 7fb6dca52700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb6d80fce80 0x7fb6d810da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:56:19.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:19.040+0000 7fb6dca52700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6d80fe080 0x7fb6d810df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:56:19.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:19.040+0000 7fb6dca52700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d810e590 con 0x7fb6d80fce80 2024-05-17T21:56:19.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:19.040+0000 7fb6dca52700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d810ede0 con 0x7fb6d80fe080 2024-05-17T21:56:19.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:19.040+0000 7fb6d659c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb6d80fce80 0x7fb6d810da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:56:19.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:19.040+0000 7fb6d659c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb6d80fce80 0x7fb6d810da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55810/0 (socket says 172.21.15.107:55810) 2024-05-17T21:56:19.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:19.040+0000 7fb6d659c700 1 -- 172.21.15.107:0/902629658 learned_addr learned my addr 172.21.15.107:0/902629658 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:56:19.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:19.040+0000 7fb6d5d9b700 1 --2- 172.21.15.107:0/902629658 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6d80fe080 0x7fb6d810df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:56:19.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:19.040+0000 7fb6d659c700 1 -- 172.21.15.107:0/902629658 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6d80fe080 msgr2=0x7fb6d810df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:56:19.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:19.040+0000 7fb6d659c700 1 --2- 172.21.15.107:0/902629658 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6d80fe080 0x7fb6d810df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:56:19.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:19.040+0000 7fb6d659c700 1 -- 172.21.15.107:0/902629658 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6c0005040 con 0x7fb6d80fce80 2024-05-17T21:56:19.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:19.040+0000 7fb6d659c700 1 --2- 172.21.15.107:0/902629658 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb6d80fce80 0x7fb6d810da50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb6d000cae0 tx=0x7fb6d0007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:56:19.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:19.041+0000 7fb6ceffd700 1 -- 172.21.15.107:0/902629658 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb6d000a980 con 0x7fb6d80fce80 2024-05-17T21:56:19.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:19.041+0000 7fb6dca52700 1 -- 172.21.15.107:0/902629658 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6d810f0c0 con 0x7fb6d80fce80 2024-05-17T21:56:19.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:19.041+0000 7fb6ceffd700 1 -- 172.21.15.107:0/902629658 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb6d0012440 con 0x7fb6d80fce80 2024-05-17T21:56:19.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:19.041+0000 7fb6ceffd700 1 -- 172.21.15.107:0/902629658 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb6d00115e0 con 0x7fb6d80fce80 2024-05-17T21:56:19.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:19.041+0000 7fb6dca52700 1 -- 172.21.15.107:0/902629658 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6d810f600 con 0x7fb6d80fce80 2024-05-17T21:56:19.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:19.042+0000 7fb6dca52700 1 -- 172.21.15.107:0/902629658 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6d805f5c0 con 0x7fb6d80fce80 2024-05-17T21:56:19.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:19.042+0000 7fb6ceffd700 1 -- 172.21.15.107:0/902629658 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7fb6d00125b0 con 0x7fb6d80fce80 2024-05-17T21:56:19.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:19.043+0000 7fb6ceffd700 1 --2- 172.21.15.107:0/902629658 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fb6c4078990 0x7fb6c407ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:56:19.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:19.043+0000 7fb6ceffd700 1 -- 172.21.15.107:0/902629658 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fb6d009ad60 con 0x7fb6d80fce80 2024-05-17T21:56:19.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:19.043+0000 7fb6d5d9b700 1 --2- 172.21.15.107:0/902629658 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fb6c4078990 0x7fb6c407ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:56:19.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:19.043+0000 7fb6d5d9b700 1 --2- 172.21.15.107:0/902629658 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fb6c4078990 0x7fb6c407ae40 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb6c000b3b0 tx=0x7fb6c0009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:56:19.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:19.048+0000 7fb6ceffd700 1 -- 172.21.15.107:0/902629658 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7fb6d00629d0 con 0x7fb6d80fce80 2024-05-17T21:56:19.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:19.340+0000 7fb6dca52700 1 -- 172.21.15.107:0/902629658 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fb6d804bed0 con 0x7fb6d80fce80 2024-05-17T21:56:19.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:19.340+0000 7fb6ceffd700 1 -- 172.21.15.107:0/902629658 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+1428 (secure 0 0 0) 0x7fb6d0062370 con 0x7fb6d80fce80 2024-05-17T21:56:19.332 INFO:teuthology.orchestra.run.smithi107.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data; Degraded data redundancy: 50/267 objects degraded (18.727%), 19 pgs degraded 2024-05-17T21:56:19.332 INFO:teuthology.orchestra.run.smithi107.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-17T21:56:19.332 INFO:teuthology.orchestra.run.smithi107.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-17T21:56:19.333 INFO:teuthology.orchestra.run.smithi107.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 50/267 objects degraded (18.727%), 19 pgs degraded 2024-05-17T21:56:19.333 INFO:teuthology.orchestra.run.smithi107.stdout: pg 1.0 is active+undersized+degraded, acting [3,1] 2024-05-17T21:56:19.333 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.0 is active+undersized+degraded, acting [3,1] 2024-05-17T21:56:19.333 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.1 is active+undersized+degraded, acting [2,1] 2024-05-17T21:56:19.333 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.2 is active+undersized+degraded, acting [5,1] 2024-05-17T21:56:19.333 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.4 is active+undersized+degraded, acting [1,4] 2024-05-17T21:56:19.334 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.5 is active+undersized+degraded, acting [3,4] 2024-05-17T21:56:19.334 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.8 is active+undersized+degraded, acting [3,5] 2024-05-17T21:56:19.334 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.9 is active+undersized+degraded, acting [1,4] 2024-05-17T21:56:19.334 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.c is active+undersized+degraded, acting [2,3] 2024-05-17T21:56:19.334 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.e is active+undersized+degraded, acting [2,3] 2024-05-17T21:56:19.335 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.f is active+undersized+degraded, acting [4,5] 2024-05-17T21:56:19.335 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.10 is active+undersized+degraded, acting [2,1] 2024-05-17T21:56:19.335 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.13 is active+undersized+degraded, acting [4,2] 2024-05-17T21:56:19.335 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.15 is active+undersized+degraded, acting [1,3] 2024-05-17T21:56:19.335 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.19 is active+undersized+degraded, acting [4,2] 2024-05-17T21:56:19.335 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.1b is active+undersized+degraded, acting [1,5] 2024-05-17T21:56:19.335 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.1d is active+undersized+degraded, acting [3,5] 2024-05-17T21:56:19.336 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.1e is active+undersized+degraded, acting [2,5] 2024-05-17T21:56:19.336 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.1f is active+undersized+degraded, acting [3,4] 2024-05-17T21:56:19.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:19.343+0000 7fb6dca52700 1 -- 172.21.15.107:0/902629658 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fb6c4078990 msgr2=0x7fb6c407ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:56:19.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:19.343+0000 7fb6dca52700 1 --2- 172.21.15.107:0/902629658 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fb6c4078990 0x7fb6c407ae40 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb6c000b3b0 tx=0x7fb6c0009e30 comp rx=0 tx=0).stop 2024-05-17T21:56:19.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:19.343+0000 7fb6dca52700 1 -- 172.21.15.107:0/902629658 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb6d80fce80 msgr2=0x7fb6d810da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:56:19.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:19.343+0000 7fb6dca52700 1 --2- 172.21.15.107:0/902629658 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb6d80fce80 0x7fb6d810da50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb6d000cae0 tx=0x7fb6d0007f60 comp rx=0 tx=0).stop 2024-05-17T21:56:19.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:19.344+0000 7fb6dca52700 1 -- 172.21.15.107:0/902629658 shutdown_connections 2024-05-17T21:56:19.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:19.344+0000 7fb6dca52700 1 --2- 172.21.15.107:0/902629658 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fb6c4078990 0x7fb6c407ae40 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:56:19.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:19.344+0000 7fb6dca52700 1 --2- 172.21.15.107:0/902629658 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb6d80fce80 0x7fb6d810da50 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:56:19.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:19.344+0000 7fb6dca52700 1 --2- 172.21.15.107:0/902629658 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6d80fe080 0x7fb6d810df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:56:19.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:19.344+0000 7fb6dca52700 1 -- 172.21.15.107:0/902629658 >> 172.21.15.107:0/902629658 conn(0x7fb6d80f8590 msgr2=0x7fb6d80f9ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:56:19.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:19.344+0000 7fb6dca52700 1 -- 172.21.15.107:0/902629658 shutdown_connections 2024-05-17T21:56:19.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:19.344+0000 7fb6dca52700 1 -- 172.21.15.107:0/902629658 wait complete. 2024-05-17T21:56:19.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:19 smithi107 ceph-mon[93224]: pgmap v60: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 258 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 50/267 objects degraded (18.727%) 2024-05-17T21:56:19.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:19 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:56:19.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:19 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:56:19.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:19 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:56:19.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:19 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:56:19.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:19 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-05-17T21:56:19.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:19 smithi107 ceph-mon[93224]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-05-17T21:56:19.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:19 smithi107 ceph-mon[93224]: Upgrade: unsafe to stop osd(s) at this time (15 PGs are or would become offline) 2024-05-17T21:56:19.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:19 smithi107 ceph-mon[93224]: from='client.? 172.21.15.107:0/1764788317' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-17T21:56:19.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:19 smithi107 ceph-mon[93224]: from='client.34240 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:56:19.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:19 smithi107 ceph-mon[93224]: from='client.? 172.21.15.107:0/902629658' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-17T21:56:19.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:19 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:56:19.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:19 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:56:19.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:19 smithi138 ceph-mon[81376]: pgmap v60: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 258 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 50/267 objects degraded (18.727%) 2024-05-17T21:56:19.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:19 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:56:19.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:19 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:56:19.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:19 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:56:19.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:19 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:56:19.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:19 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-05-17T21:56:19.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:19 smithi138 ceph-mon[81376]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-05-17T21:56:19.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:19 smithi138 ceph-mon[81376]: Upgrade: unsafe to stop osd(s) at this time (15 PGs are or would become offline) 2024-05-17T21:56:19.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:19 smithi138 ceph-mon[81376]: from='client.? 172.21.15.107:0/1764788317' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-17T21:56:19.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:19 smithi138 ceph-mon[81376]: from='client.34240 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:56:19.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:19 smithi138 ceph-mon[81376]: from='client.? 172.21.15.107:0/902629658' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-17T21:56:19.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:19 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:56:19.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:19 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:56:20.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:20 smithi107 ceph-mon[93224]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 50/267 objects degraded (18.727%), 19 pgs degraded) 2024-05-17T21:56:21.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:20 smithi138 ceph-mon[81376]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 50/267 objects degraded (18.727%), 19 pgs degraded) 2024-05-17T21:56:21.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:21 smithi107 ceph-mon[93224]: pgmap v61: 65 pgs: 65 active+clean; 258 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-17T21:56:22.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:21 smithi138 ceph-mon[81376]: pgmap v61: 65 pgs: 65 active+clean; 258 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-17T21:56:23.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:23 smithi107 ceph-mon[93224]: pgmap v62: 65 pgs: 65 active+clean; 258 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.3 KiB/s rd, 2 op/s 2024-05-17T21:56:24.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:23 smithi138 ceph-mon[81376]: pgmap v62: 65 pgs: 65 active+clean; 258 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.3 KiB/s rd, 2 op/s 2024-05-17T21:56:25.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:25 smithi107 ceph-mon[93224]: pgmap v63: 65 pgs: 65 active+clean; 258 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 639 B/s rd, 1 op/s 2024-05-17T21:56:26.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:25 smithi138 ceph-mon[81376]: pgmap v63: 65 pgs: 65 active+clean; 258 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 639 B/s rd, 1 op/s 2024-05-17T21:56:28.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:27 smithi138 ceph-mon[81376]: pgmap v64: 65 pgs: 65 active+clean; 258 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.0 KiB/s rd, 2 op/s 2024-05-17T21:56:28.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:27 smithi107 ceph-mon[93224]: pgmap v64: 65 pgs: 65 active+clean; 258 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.0 KiB/s rd, 2 op/s 2024-05-17T21:56:29.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:29 smithi107 ceph-mon[93224]: pgmap v65: 65 pgs: 65 active+clean; 258 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 939 B/s rd, 1 op/s 2024-05-17T21:56:29.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:29 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:56:30.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:29 smithi138 ceph-mon[81376]: pgmap v65: 65 pgs: 65 active+clean; 258 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 939 B/s rd, 1 op/s 2024-05-17T21:56:30.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:29 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:56:32.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:31 smithi138 ceph-mon[81376]: pgmap v66: 65 pgs: 65 active+clean; 258 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T21:56:32.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:31 smithi107 ceph-mon[93224]: pgmap v66: 65 pgs: 65 active+clean; 258 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T21:56:33.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:33 smithi107 ceph-mon[93224]: pgmap v67: 65 pgs: 65 active+clean; 258 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T21:56:33.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:33 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-05-17T21:56:33.878 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:33 smithi138 ceph-mon[81376]: pgmap v67: 65 pgs: 65 active+clean; 258 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T21:56:33.878 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:33 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-05-17T21:56:34.696 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:34 smithi107 ceph-mon[93224]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-05-17T21:56:34.696 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:34 smithi107 ceph-mon[93224]: Upgrade: osd.1 is safe to restart 2024-05-17T21:56:34.696 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:34 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:56:34.696 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:34 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:56:35.073 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:34 smithi138 ceph-mon[81376]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-05-17T21:56:35.073 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:34 smithi138 ceph-mon[81376]: Upgrade: osd.1 is safe to restart 2024-05-17T21:56:35.073 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:34 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:56:35.073 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:34 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:56:35.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:35 smithi107 ceph-mon[93224]: pgmap v68: 65 pgs: 65 active+clean; 258 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T21:56:35.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:35 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:56:35.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:35 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-17T21:56:35.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:35 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:56:36.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:35 smithi138 ceph-mon[81376]: pgmap v68: 65 pgs: 65 active+clean; 258 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T21:56:36.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:35 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:56:36.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:35 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-17T21:56:36.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:35 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:56:37.036 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:36 smithi107 ceph-mon[93224]: Upgrade: Updating osd.1 2024-05-17T21:56:37.036 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:36 smithi107 ceph-mon[93224]: Deploying daemon osd.1 on smithi107 2024-05-17T21:56:37.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:36 smithi138 ceph-mon[81376]: Upgrade: Updating osd.1 2024-05-17T21:56:37.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:36 smithi138 ceph-mon[81376]: Deploying daemon osd.1 on smithi107 2024-05-17T21:56:38.059 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:37 smithi107 ceph-mon[93224]: pgmap v69: 65 pgs: 65 active+clean; 258 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T21:56:38.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:37 smithi138 ceph-mon[81376]: pgmap v69: 65 pgs: 65 active+clean; 258 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T21:56:38.956 INFO:journalctl@ceph.osd.1.smithi107.stdout:May 17 21:56:38 smithi107 systemd[1]: Stopping Ceph osd.1 for 3722731a-1496-11ef-bc9a-c7b262605968... 2024-05-17T21:56:39.351 INFO:journalctl@ceph.osd.1.smithi107.stdout:May 17 21:56:38 smithi107 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-1[54544]: 2024-05-17T21:56:38.964+0000 7f65903b1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-17T21:56:39.351 INFO:journalctl@ceph.osd.1.smithi107.stdout:May 17 21:56:38 smithi107 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-1[54544]: 2024-05-17T21:56:38.964+0000 7f65903b1700 -1 osd.1 54 *** Got signal Terminated *** 2024-05-17T21:56:39.351 INFO:journalctl@ceph.osd.1.smithi107.stdout:May 17 21:56:38 smithi107 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-1[54544]: 2024-05-17T21:56:38.964+0000 7f65903b1700 -1 osd.1 54 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-17T21:56:39.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:39 smithi107 ceph-mon[93224]: pgmap v70: 65 pgs: 65 active+clean; 258 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T21:56:39.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:39 smithi107 ceph-mon[93224]: osd.1 marked itself down and dead 2024-05-17T21:56:40.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:39 smithi138 ceph-mon[81376]: pgmap v70: 65 pgs: 65 active+clean; 258 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T21:56:40.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:39 smithi138 ceph-mon[81376]: osd.1 marked itself down and dead 2024-05-17T21:56:40.351 INFO:journalctl@ceph.osd.1.smithi107.stdout:May 17 21:56:40 smithi107 podman[106506]: 2024-05-17 21:56:40.107499654 +0000 UTC m=+1.245567381 container died e5d3e99844a9fe2041ba5fa1b8579389142e52e55d2348411bc637bf73ad68ed (image=quay.ceph.io/ceph-ci/ceph@sha256:c200327b094672e93680375e82d2de1a08f5881ef304978f791cce179f847d60, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-1, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=quincy-9903f77, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-05-17T21:56:40.628 INFO:journalctl@ceph.osd.1.smithi107.stdout:May 17 21:56:40 smithi107 podman[106506]: 2024-05-17 21:56:40.50615152 +0000 UTC m=+1.644219244 container cleanup e5d3e99844a9fe2041ba5fa1b8579389142e52e55d2348411bc637bf73ad68ed (image=quay.ceph.io/ceph-ci/ceph@sha256:c200327b094672e93680375e82d2de1a08f5881ef304978f791cce179f847d60, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-1, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=quincy-9903f77, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-17T21:56:40.628 INFO:journalctl@ceph.osd.1.smithi107.stdout:May 17 21:56:40 smithi107 bash[106506]: ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-1 2024-05-17T21:56:41.065 INFO:journalctl@ceph.osd.1.smithi107.stdout:May 17 21:56:40 smithi107 podman[106524]: 2024-05-17 21:56:40.749406739 +0000 UTC m=+0.637294802 container remove e5d3e99844a9fe2041ba5fa1b8579389142e52e55d2348411bc637bf73ad68ed (image=quay.ceph.io/ceph-ci/ceph@sha256:c200327b094672e93680375e82d2de1a08f5881ef304978f791cce179f847d60, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-1, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=quincy-9903f77, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-05-17T21:56:41.065 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:40 smithi107 ceph-mon[93224]: Health check failed: 1 osds down (OSD_DOWN) 2024-05-17T21:56:41.065 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:40 smithi107 ceph-mon[93224]: osdmap e55: 6 total, 5 up, 6 in 2024-05-17T21:56:41.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:40 smithi138 ceph-mon[81376]: Health check failed: 1 osds down (OSD_DOWN) 2024-05-17T21:56:41.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:40 smithi138 ceph-mon[81376]: osdmap e55: 6 total, 5 up, 6 in 2024-05-17T21:56:41.351 INFO:journalctl@ceph.osd.1.smithi107.stdout:May 17 21:56:41 smithi107 podman[106612]: 2024-05-17 21:56:40.976102131 +0000 UTC m=+0.017411520 image pull 8880f7613d0283efa7002505fdccdbc3e37d3b5da31c57fa22d62262c5dab66c quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b 2024-05-17T21:56:41.352 INFO:journalctl@ceph.osd.1.smithi107.stdout:May 17 21:56:41 smithi107 podman[106612]: 2024-05-17 21:56:41.075153782 +0000 UTC m=+0.116463180 container create 17d742ee7ce432ba473b8910fdb830f101934fbe192d3857dd49f63b66d38b2f (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.schema-version=1.0) 2024-05-17T21:56:41.352 INFO:journalctl@ceph.osd.1.smithi107.stdout:May 17 21:56:41 smithi107 podman[106612]: 2024-05-17 21:56:41.270003045 +0000 UTC m=+0.311312440 container init 17d742ee7ce432ba473b8910fdb830f101934fbe192d3857dd49f63b66d38b2f (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-1-deactivate, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-05-17T21:56:41.352 INFO:journalctl@ceph.osd.1.smithi107.stdout:May 17 21:56:41 smithi107 podman[106612]: 2024-05-17 21:56:41.273561104 +0000 UTC m=+0.314870481 container start 17d742ee7ce432ba473b8910fdb830f101934fbe192d3857dd49f63b66d38b2f (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240514, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-05-17T21:56:41.629 INFO:journalctl@ceph.osd.1.smithi107.stdout:May 17 21:56:41 smithi107 podman[106612]: 2024-05-17 21:56:41.361117779 +0000 UTC m=+0.402427172 container attach 17d742ee7ce432ba473b8910fdb830f101934fbe192d3857dd49f63b66d38b2f (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.vendor=CentOS) 2024-05-17T21:56:41.629 INFO:journalctl@ceph.osd.1.smithi107.stdout:May 17 21:56:41 smithi107 podman[106612]: 2024-05-17 21:56:41.572198681 +0000 UTC m=+0.613508059 container died 17d742ee7ce432ba473b8910fdb830f101934fbe192d3857dd49f63b66d38b2f (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, ceph=True) 2024-05-17T21:56:42.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:41 smithi138 ceph-mon[81376]: pgmap v72: 65 pgs: 29 peering, 3 stale+active+clean, 33 active+clean; 258 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 1 op/s 2024-05-17T21:56:42.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:41 smithi138 ceph-mon[81376]: Health check failed: Reduced data availability: 9 pgs peering (PG_AVAILABILITY) 2024-05-17T21:56:42.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:41 smithi138 ceph-mon[81376]: osdmap e56: 6 total, 5 up, 6 in 2024-05-17T21:56:42.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:41 smithi107 ceph-mon[93224]: pgmap v72: 65 pgs: 29 peering, 3 stale+active+clean, 33 active+clean; 258 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 1 op/s 2024-05-17T21:56:42.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:41 smithi107 ceph-mon[93224]: Health check failed: Reduced data availability: 9 pgs peering (PG_AVAILABILITY) 2024-05-17T21:56:42.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:41 smithi107 ceph-mon[93224]: osdmap e56: 6 total, 5 up, 6 in 2024-05-17T21:56:42.578 INFO:journalctl@ceph.osd.1.smithi107.stdout:May 17 21:56:42 smithi107 podman[106638]: 2024-05-17 21:56:42.139904731 +0000 UTC m=+0.561778489 container remove 17d742ee7ce432ba473b8910fdb830f101934fbe192d3857dd49f63b66d38b2f (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-05-17T21:56:42.578 INFO:journalctl@ceph.osd.1.smithi107.stdout:May 17 21:56:42 smithi107 systemd[1]: ceph-3722731a-1496-11ef-bc9a-c7b262605968@osd.1.service: Deactivated successfully. 2024-05-17T21:56:42.578 INFO:journalctl@ceph.osd.1.smithi107.stdout:May 17 21:56:42 smithi107 systemd[1]: Stopped Ceph osd.1 for 3722731a-1496-11ef-bc9a-c7b262605968. 2024-05-17T21:56:42.578 INFO:journalctl@ceph.osd.1.smithi107.stdout:May 17 21:56:42 smithi107 systemd[1]: ceph-3722731a-1496-11ef-bc9a-c7b262605968@osd.1.service: Consumed 54.548s CPU time. 2024-05-17T21:56:42.578 INFO:journalctl@ceph.osd.1.smithi107.stdout:May 17 21:56:42 smithi107 systemd[1]: Starting Ceph osd.1 for 3722731a-1496-11ef-bc9a-c7b262605968... 2024-05-17T21:56:42.851 INFO:journalctl@ceph.osd.1.smithi107.stdout:May 17 21:56:42 smithi107 podman[106736]: 2024-05-17 21:56:42.489611467 +0000 UTC m=+0.017148875 image pull 8880f7613d0283efa7002505fdccdbc3e37d3b5da31c57fa22d62262c5dab66c quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b 2024-05-17T21:56:42.851 INFO:journalctl@ceph.osd.1.smithi107.stdout:May 17 21:56:42 smithi107 podman[106736]: 2024-05-17 21:56:42.590715187 +0000 UTC m=+0.118252570 container create a1c58bfd26c40b109c8ab0a6d6b7cf70e326a96fa030dfa19de80d78c079aa40 (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-1-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-05-17T21:56:42.851 INFO:journalctl@ceph.osd.1.smithi107.stdout:May 17 21:56:42 smithi107 podman[106736]: 2024-05-17 21:56:42.818300387 +0000 UTC m=+0.345837789 container init a1c58bfd26c40b109c8ab0a6d6b7cf70e326a96fa030dfa19de80d78c079aa40 (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-1-activate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-17T21:56:42.852 INFO:journalctl@ceph.osd.1.smithi107.stdout:May 17 21:56:42 smithi107 podman[106736]: 2024-05-17 21:56:42.82164667 +0000 UTC m=+0.349184066 container start a1c58bfd26c40b109c8ab0a6d6b7cf70e326a96fa030dfa19de80d78c079aa40 (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-1-activate, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-05-17T21:56:43.351 INFO:journalctl@ceph.osd.1.smithi107.stdout:May 17 21:56:42 smithi107 podman[106736]: 2024-05-17 21:56:42.910388384 +0000 UTC m=+0.437925775 container attach a1c58bfd26c40b109c8ab0a6d6b7cf70e326a96fa030dfa19de80d78c079aa40 (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-1-activate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-05-17T21:56:43.755 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:43 smithi107 ceph-mon[93224]: pgmap v74: 65 pgs: 29 peering, 3 stale+active+clean, 33 active+clean; 258 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 2 op/s 2024-05-17T21:56:43.756 INFO:journalctl@ceph.osd.1.smithi107.stdout:May 17 21:56:43 smithi107 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-1-activate[106751]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-05-17T21:56:43.756 INFO:journalctl@ceph.osd.1.smithi107.stdout:May 17 21:56:43 smithi107 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-1-activate[106751]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-05-17T21:56:43.756 INFO:journalctl@ceph.osd.1.smithi107.stdout:May 17 21:56:43 smithi107 bash[106736]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-05-17T21:56:43.756 INFO:journalctl@ceph.osd.1.smithi107.stdout:May 17 21:56:43 smithi107 bash[106736]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-05-17T21:56:43.756 INFO:journalctl@ceph.osd.1.smithi107.stdout:May 17 21:56:43 smithi107 bash[106736]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-05-17T21:56:43.756 INFO:journalctl@ceph.osd.1.smithi107.stdout:May 17 21:56:43 smithi107 bash[106736]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-05-17T21:56:43.756 INFO:journalctl@ceph.osd.1.smithi107.stdout:May 17 21:56:43 smithi107 bash[106736]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-05-17T21:56:43.756 INFO:journalctl@ceph.osd.1.smithi107.stdout:May 17 21:56:43 smithi107 bash[106736]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-05-17T21:56:43.756 INFO:journalctl@ceph.osd.1.smithi107.stdout:May 17 21:56:43 smithi107 bash[106736]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-05-17T21:56:43.756 INFO:journalctl@ceph.osd.1.smithi107.stdout:May 17 21:56:43 smithi107 bash[106736]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-05-17T21:56:43.756 INFO:journalctl@ceph.osd.1.smithi107.stdout:May 17 21:56:43 smithi107 bash[106736]: --> ceph-volume raw activate successful for osd ID: 1 2024-05-17T21:56:43.756 INFO:journalctl@ceph.osd.1.smithi107.stdout:May 17 21:56:43 smithi107 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-1-activate[106751]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-05-17T21:56:43.756 INFO:journalctl@ceph.osd.1.smithi107.stdout:May 17 21:56:43 smithi107 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-1-activate[106751]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-05-17T21:56:43.757 INFO:journalctl@ceph.osd.1.smithi107.stdout:May 17 21:56:43 smithi107 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-1-activate[106751]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-05-17T21:56:43.757 INFO:journalctl@ceph.osd.1.smithi107.stdout:May 17 21:56:43 smithi107 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-1-activate[106751]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-05-17T21:56:43.757 INFO:journalctl@ceph.osd.1.smithi107.stdout:May 17 21:56:43 smithi107 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-1-activate[106751]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-05-17T21:56:43.757 INFO:journalctl@ceph.osd.1.smithi107.stdout:May 17 21:56:43 smithi107 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-1-activate[106751]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-05-17T21:56:43.757 INFO:journalctl@ceph.osd.1.smithi107.stdout:May 17 21:56:43 smithi107 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-1-activate[106751]: --> ceph-volume raw activate successful for osd ID: 1 2024-05-17T21:56:43.757 INFO:journalctl@ceph.osd.1.smithi107.stdout:May 17 21:56:43 smithi107 podman[106736]: 2024-05-17 21:56:43.446952382 +0000 UTC m=+0.974489772 container died a1c58bfd26c40b109c8ab0a6d6b7cf70e326a96fa030dfa19de80d78c079aa40 (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-1-activate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-05-17T21:56:44.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:43 smithi138 ceph-mon[81376]: pgmap v74: 65 pgs: 29 peering, 3 stale+active+clean, 33 active+clean; 258 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 2 op/s 2024-05-17T21:56:44.351 INFO:journalctl@ceph.osd.1.smithi107.stdout:May 17 21:56:44 smithi107 podman[106890]: 2024-05-17 21:56:44.03090751 +0000 UTC m=+0.577962379 container remove a1c58bfd26c40b109c8ab0a6d6b7cf70e326a96fa030dfa19de80d78c079aa40 (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-1-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.vendor=CentOS, ceph=True) 2024-05-17T21:56:44.351 INFO:journalctl@ceph.osd.1.smithi107.stdout:May 17 21:56:44 smithi107 podman[106951]: 2024-05-17 21:56:44.145883812 +0000 UTC m=+0.017400369 image pull 8880f7613d0283efa7002505fdccdbc3e37d3b5da31c57fa22d62262c5dab66c quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b 2024-05-17T21:56:44.352 INFO:journalctl@ceph.osd.1.smithi107.stdout:May 17 21:56:44 smithi107 podman[106951]: 2024-05-17 21:56:44.248449924 +0000 UTC m=+0.119966462 container create f94b206be6eb344f1eb9d2119362212ebc214e71d5d379083314941a174b708c (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-17T21:56:44.851 INFO:journalctl@ceph.osd.1.smithi107.stdout:May 17 21:56:44 smithi107 podman[106951]: 2024-05-17 21:56:44.509817214 +0000 UTC m=+0.381333762 container init f94b206be6eb344f1eb9d2119362212ebc214e71d5d379083314941a174b708c (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-1, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-17T21:56:44.851 INFO:journalctl@ceph.osd.1.smithi107.stdout:May 17 21:56:44 smithi107 podman[106951]: 2024-05-17 21:56:44.513109529 +0000 UTC m=+0.384626070 container start f94b206be6eb344f1eb9d2119362212ebc214e71d5d379083314941a174b708c (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-05-17T21:56:44.851 INFO:journalctl@ceph.osd.1.smithi107.stdout:May 17 21:56:44 smithi107 ceph-osd[106969]: -- 172.21.15.107:0/4088001651 <== mon.0 v2:172.21.15.107:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f31a965680 con 0x55f31aa0cd00 2024-05-17T21:56:44.852 INFO:journalctl@ceph.osd.1.smithi107.stdout:May 17 21:56:44 smithi107 bash[106951]: f94b206be6eb344f1eb9d2119362212ebc214e71d5d379083314941a174b708c 2024-05-17T21:56:44.852 INFO:journalctl@ceph.osd.1.smithi107.stdout:May 17 21:56:44 smithi107 systemd[1]: Started Ceph osd.1 for 3722731a-1496-11ef-bc9a-c7b262605968. 2024-05-17T21:56:45.351 INFO:journalctl@ceph.osd.1.smithi107.stdout:May 17 21:56:45 smithi107 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-1[106965]: 2024-05-17T21:56:45.068+0000 7fa268ab37c0 -1 Falling back to public interface 2024-05-17T21:56:45.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:45 smithi107 ceph-mon[93224]: pgmap v75: 65 pgs: 29 peering, 3 stale+active+clean, 33 active+clean; 258 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 0 op/s 2024-05-17T21:56:45.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:45 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:56:45.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:45 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:56:45.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:45 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:56:46.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:45 smithi138 ceph-mon[81376]: pgmap v75: 65 pgs: 29 peering, 3 stale+active+clean, 33 active+clean; 258 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 0 op/s 2024-05-17T21:56:46.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:45 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:56:46.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:45 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:56:46.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:45 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:56:47.007 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:46 smithi107 ceph-mon[93224]: Health check failed: Degraded data redundancy: 34/267 objects degraded (12.734%), 13 pgs degraded (PG_DEGRADED) 2024-05-17T21:56:47.008 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:46 smithi107 ceph-mon[93224]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 9 pgs peering) 2024-05-17T21:56:47.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:46 smithi138 ceph-mon[81376]: Health check failed: Degraded data redundancy: 34/267 objects degraded (12.734%), 13 pgs degraded (PG_DEGRADED) 2024-05-17T21:56:47.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:46 smithi138 ceph-mon[81376]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 9 pgs peering) 2024-05-17T21:56:48.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:47 smithi138 ceph-mon[81376]: pgmap v76: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 258 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 34/267 objects degraded (12.734%) 2024-05-17T21:56:48.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:47 smithi107 ceph-mon[93224]: pgmap v76: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 258 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 34/267 objects degraded (12.734%) 2024-05-17T21:56:49.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:49.461+0000 7f9b7f67e700 1 -- 172.21.15.107:0/1989269511 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b780fcf60 msgr2=0x7f9b780fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:56:49.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:49.461+0000 7f9b7f67e700 1 --2- 172.21.15.107:0/1989269511 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b780fcf60 0x7f9b780fd3d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9b68004940 tx=0x7f9b68009e30 comp rx=0 tx=0).stop 2024-05-17T21:56:49.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:49.461+0000 7f9b7f67e700 1 -- 172.21.15.107:0/1989269511 shutdown_connections 2024-05-17T21:56:49.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:49.461+0000 7f9b7f67e700 1 --2- 172.21.15.107:0/1989269511 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b780fcf60 0x7f9b780fd3d0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:56:49.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:49.461+0000 7f9b7f67e700 1 --2- 172.21.15.107:0/1989269511 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9b780fbcb0 0x7f9b780fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:56:49.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:49.461+0000 7f9b7f67e700 1 -- 172.21.15.107:0/1989269511 >> 172.21.15.107:0/1989269511 conn(0x7f9b780f7430 msgr2=0x7f9b780f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:56:49.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:49.461+0000 7f9b7f67e700 1 -- 172.21.15.107:0/1989269511 shutdown_connections 2024-05-17T21:56:49.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:49.461+0000 7f9b7f67e700 1 -- 172.21.15.107:0/1989269511 wait complete. 2024-05-17T21:56:49.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:49.462+0000 7f9b7f67e700 1 Processor -- start 2024-05-17T21:56:49.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:49.462+0000 7f9b7f67e700 1 -- start start 2024-05-17T21:56:49.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:49.462+0000 7f9b7f67e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9b780fbcb0 0x7f9b7810dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:56:49.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:49.462+0000 7f9b7f67e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b780fcf60 0x7f9b7810dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:56:49.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:49.462+0000 7f9b7f67e700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b7810e5f0 con 0x7f9b780fcf60 2024-05-17T21:56:49.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:49.462+0000 7f9b7f67e700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b7810eda0 con 0x7f9b780fbcb0 2024-05-17T21:56:49.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:49.463+0000 7f9b7cc19700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b780fcf60 0x7f9b7810dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:56:49.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:49.463+0000 7f9b7cc19700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b780fcf60 0x7f9b7810dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55874/0 (socket says 172.21.15.107:55874) 2024-05-17T21:56:49.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:49.463+0000 7f9b7cc19700 1 -- 172.21.15.107:0/1079484069 learned_addr learned my addr 172.21.15.107:0/1079484069 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:56:49.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:49.463+0000 7f9b7d41a700 1 --2- 172.21.15.107:0/1079484069 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9b780fbcb0 0x7f9b7810dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:56:49.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:49.463+0000 7f9b7cc19700 1 -- 172.21.15.107:0/1079484069 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9b780fbcb0 msgr2=0x7f9b7810dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:56:49.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:49.463+0000 7f9b7cc19700 1 --2- 172.21.15.107:0/1079484069 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9b780fbcb0 0x7f9b7810dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:56:49.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:49.463+0000 7f9b7cc19700 1 -- 172.21.15.107:0/1079484069 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b68005040 con 0x7f9b780fcf60 2024-05-17T21:56:49.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:49.463+0000 7f9b7cc19700 1 --2- 172.21.15.107:0/1079484069 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b780fcf60 0x7f9b7810dff0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9b68009150 tx=0x7f9b68009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:56:49.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:49.464+0000 7f9b6dffb700 1 -- 172.21.15.107:0/1079484069 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b6800e040 con 0x7f9b780fcf60 2024-05-17T21:56:49.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:49.464+0000 7f9b6dffb700 1 -- 172.21.15.107:0/1079484069 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9b68004290 con 0x7f9b780fcf60 2024-05-17T21:56:49.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:49.464+0000 7f9b7f67e700 1 -- 172.21.15.107:0/1079484069 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b7810f020 con 0x7f9b780fcf60 2024-05-17T21:56:49.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:49.464+0000 7f9b6dffb700 1 -- 172.21.15.107:0/1079484069 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b6801c630 con 0x7f9b780fcf60 2024-05-17T21:56:49.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:49.464+0000 7f9b7f67e700 1 -- 172.21.15.107:0/1079484069 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b7810f560 con 0x7f9b780fcf60 2024-05-17T21:56:49.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:49.465+0000 7f9b7f67e700 1 -- 172.21.15.107:0/1079484069 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b78101af0 con 0x7f9b780fcf60 2024-05-17T21:56:49.458 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:49.466+0000 7f9b6dffb700 1 -- 172.21.15.107:0/1079484069 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f9b6800bbf0 con 0x7f9b780fcf60 2024-05-17T21:56:49.458 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:49.467+0000 7f9b6dffb700 1 --2- 172.21.15.107:0/1079484069 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f9b64078990 0x7f9b6407ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:56:49.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:49.467+0000 7f9b6dffb700 1 -- 172.21.15.107:0/1079484069 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f9b6809b300 con 0x7f9b780fcf60 2024-05-17T21:56:49.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:49.467+0000 7f9b7d41a700 1 --2- 172.21.15.107:0/1079484069 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f9b64078990 0x7f9b6407ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:56:49.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:49.468+0000 7f9b7d41a700 1 --2- 172.21.15.107:0/1079484069 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f9b64078990 0x7f9b6407ae40 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9b74004f40 tx=0x7f9b7400b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:56:49.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:49.475+0000 7f9b6dffb700 1 -- 172.21.15.107:0/1079484069 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f9b68062f90 con 0x7f9b780fcf60 2024-05-17T21:56:49.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:49.738+0000 7f9b7f67e700 1 -- 172.21.15.107:0/1079484069 --> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9b7810f780 con 0x7f9b64078990 2024-05-17T21:56:49.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:49.739+0000 7f9b6dffb700 1 -- 172.21.15.107:0/1079484069 <== mgr.34100 v2:172.21.15.107:6800/3205268226 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f9b7810f780 con 0x7f9b64078990 2024-05-17T21:56:49.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:49.742+0000 7f9b7f67e700 1 -- 172.21.15.107:0/1079484069 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f9b64078990 msgr2=0x7f9b6407ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:56:49.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:49.742+0000 7f9b7f67e700 1 --2- 172.21.15.107:0/1079484069 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f9b64078990 0x7f9b6407ae40 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9b74004f40 tx=0x7f9b7400b040 comp rx=0 tx=0).stop 2024-05-17T21:56:49.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:49.742+0000 7f9b7f67e700 1 -- 172.21.15.107:0/1079484069 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b780fcf60 msgr2=0x7f9b7810dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:56:49.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:49.742+0000 7f9b7f67e700 1 --2- 172.21.15.107:0/1079484069 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b780fcf60 0x7f9b7810dff0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9b68009150 tx=0x7f9b68009e30 comp rx=0 tx=0).stop 2024-05-17T21:56:49.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:49.742+0000 7f9b7f67e700 1 -- 172.21.15.107:0/1079484069 shutdown_connections 2024-05-17T21:56:49.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:49.742+0000 7f9b7f67e700 1 --2- 172.21.15.107:0/1079484069 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f9b64078990 0x7f9b6407ae40 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:56:49.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:49.742+0000 7f9b7f67e700 1 --2- 172.21.15.107:0/1079484069 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9b780fbcb0 0x7f9b7810dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:56:49.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:49.742+0000 7f9b7f67e700 1 --2- 172.21.15.107:0/1079484069 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b780fcf60 0x7f9b7810dff0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:56:49.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:49.742+0000 7f9b7f67e700 1 -- 172.21.15.107:0/1079484069 >> 172.21.15.107:0/1079484069 conn(0x7f9b780f7430 msgr2=0x7f9b78065b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:56:49.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:49.742+0000 7f9b7f67e700 1 -- 172.21.15.107:0/1079484069 shutdown_connections 2024-05-17T21:56:49.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:49.742+0000 7f9b7f67e700 1 -- 172.21.15.107:0/1079484069 wait complete. 2024-05-17T21:56:49.745 INFO:teuthology.orchestra.run.smithi107.stdout:true 2024-05-17T21:56:49.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:49.852+0000 7f7634cb7700 1 -- 172.21.15.107:0/1816234208 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7630070830 msgr2=0x7f7630070ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:56:49.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:49.852+0000 7f7634cb7700 1 --2- 172.21.15.107:0/1816234208 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7630070830 0x7f7630070ca0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7624004940 tx=0x7f7624009e30 comp rx=0 tx=0).stop 2024-05-17T21:56:49.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:49.852+0000 7f7634cb7700 1 -- 172.21.15.107:0/1816234208 shutdown_connections 2024-05-17T21:56:49.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:49.852+0000 7f7634cb7700 1 --2- 172.21.15.107:0/1816234208 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7630070830 0x7f7630070ca0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:56:49.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:49.852+0000 7f7634cb7700 1 --2- 172.21.15.107:0/1816234208 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7630071de0 0x7f76300702f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:56:49.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:49.852+0000 7f7634cb7700 1 -- 172.21.15.107:0/1816234208 >> 172.21.15.107:0/1816234208 conn(0x7f76300f8540 msgr2=0x7f76300fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:56:49.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:49.852+0000 7f7634cb7700 1 -- 172.21.15.107:0/1816234208 shutdown_connections 2024-05-17T21:56:49.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:49.852+0000 7f7634cb7700 1 -- 172.21.15.107:0/1816234208 wait complete. 2024-05-17T21:56:49.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:49.853+0000 7f7634cb7700 1 Processor -- start 2024-05-17T21:56:49.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:49.853+0000 7f7634cb7700 1 -- start start 2024-05-17T21:56:49.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:49.853+0000 7f7634cb7700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7630070830 0x7f763010fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:56:49.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:49.853+0000 7f7634cb7700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7630071de0 0x7f7630110170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:56:49.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:49.853+0000 7f7634cb7700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7630110770 con 0x7f7630070830 2024-05-17T21:56:49.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:49.853+0000 7f7634cb7700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7630110fc0 con 0x7f7630071de0 2024-05-17T21:56:49.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:49.853+0000 7f762e59c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7630070830 0x7f763010fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:56:49.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:49.853+0000 7f762e59c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7630070830 0x7f763010fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55896/0 (socket says 172.21.15.107:55896) 2024-05-17T21:56:49.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:49.853+0000 7f762e59c700 1 -- 172.21.15.107:0/251326663 learned_addr learned my addr 172.21.15.107:0/251326663 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:56:49.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:49.853+0000 7f762dd9b700 1 --2- 172.21.15.107:0/251326663 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7630071de0 0x7f7630110170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:56:49.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:49.853+0000 7f762e59c700 1 -- 172.21.15.107:0/251326663 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7630071de0 msgr2=0x7f7630110170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:56:49.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:49.853+0000 7f762e59c700 1 --2- 172.21.15.107:0/251326663 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7630071de0 0x7f7630110170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:56:49.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:49.853+0000 7f762e59c700 1 -- 172.21.15.107:0/251326663 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7624005040 con 0x7f7630070830 2024-05-17T21:56:49.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:49.853+0000 7f762dd9b700 1 --2- 172.21.15.107:0/251326663 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7630071de0 0x7f7630110170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T21:56:49.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:49.854+0000 7f762e59c700 1 --2- 172.21.15.107:0/251326663 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7630070830 0x7f763010fc30 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f762000c8e0 tx=0x7f76200044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:56:49.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:49.854+0000 7f761effd700 1 -- 172.21.15.107:0/251326663 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f762000a940 con 0x7f7630070830 2024-05-17T21:56:49.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:49.854+0000 7f7634cb7700 1 -- 172.21.15.107:0/251326663 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76301112a0 con 0x7f7630070830 2024-05-17T21:56:49.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:49.854+0000 7f761effd700 1 -- 172.21.15.107:0/251326663 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f762000ce80 con 0x7f7630070830 2024-05-17T21:56:49.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:49.854+0000 7f761effd700 1 -- 172.21.15.107:0/251326663 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f76200113a0 con 0x7f7630070830 2024-05-17T21:56:49.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:49.854+0000 7f7634cb7700 1 -- 172.21.15.107:0/251326663 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76301117e0 con 0x7f7630070830 2024-05-17T21:56:49.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:49.855+0000 7f7634cb7700 1 -- 172.21.15.107:0/251326663 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f763005f5c0 con 0x7f7630070830 2024-05-17T21:56:49.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:49.856+0000 7f761effd700 1 -- 172.21.15.107:0/251326663 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f7620012400 con 0x7f7630070830 2024-05-17T21:56:49.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:49.856+0000 7f761effd700 1 --2- 172.21.15.107:0/251326663 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f7618078990 0x7f761807ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:56:49.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:49.856+0000 7f761effd700 1 -- 172.21.15.107:0/251326663 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f762009a7e0 con 0x7f7630070830 2024-05-17T21:56:49.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:49.856+0000 7f762dd9b700 1 --2- 172.21.15.107:0/251326663 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f7618078990 0x7f761807ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:56:49.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:49.857+0000 7f762dd9b700 1 --2- 172.21.15.107:0/251326663 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f7618078990 0x7f761807ae40 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f762400b3b0 tx=0x7f7624009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:56:49.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:49.861+0000 7f761effd700 1 -- 172.21.15.107:0/251326663 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f76200624d0 con 0x7f7630070830 2024-05-17T21:56:50.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:49 smithi138 ceph-mon[81376]: pgmap v77: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 258 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 34/267 objects degraded (12.734%) 2024-05-17T21:56:50.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:49 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:56:50.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:49 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:56:50.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:49 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:56:50.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:49 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:56:50.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:49 smithi107 ceph-mon[93224]: pgmap v77: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 258 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 34/267 objects degraded (12.734%) 2024-05-17T21:56:50.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:49 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:56:50.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:49 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:56:50.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:49 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:56:50.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:49 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:56:50.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:50.116+0000 7f7634cb7700 1 -- 172.21.15.107:0/251326663 --> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7630111c30 con 0x7f7618078990 2024-05-17T21:56:50.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:50.117+0000 7f761effd700 1 -- 172.21.15.107:0/251326663 <== mgr.34100 v2:172.21.15.107:6800/3205268226 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f7630111c30 con 0x7f7618078990 2024-05-17T21:56:50.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:50.119+0000 7f7634cb7700 1 -- 172.21.15.107:0/251326663 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f7618078990 msgr2=0x7f761807ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:56:50.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:50.119+0000 7f7634cb7700 1 --2- 172.21.15.107:0/251326663 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f7618078990 0x7f761807ae40 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f762400b3b0 tx=0x7f7624009e30 comp rx=0 tx=0).stop 2024-05-17T21:56:50.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:50.119+0000 7f7634cb7700 1 -- 172.21.15.107:0/251326663 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7630070830 msgr2=0x7f763010fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:56:50.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:50.119+0000 7f7634cb7700 1 --2- 172.21.15.107:0/251326663 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7630070830 0x7f763010fc30 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f762000c8e0 tx=0x7f76200044c0 comp rx=0 tx=0).stop 2024-05-17T21:56:50.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:50.119+0000 7f7634cb7700 1 -- 172.21.15.107:0/251326663 shutdown_connections 2024-05-17T21:56:50.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:50.119+0000 7f7634cb7700 1 --2- 172.21.15.107:0/251326663 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f7618078990 0x7f761807ae40 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:56:50.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:50.119+0000 7f7634cb7700 1 --2- 172.21.15.107:0/251326663 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7630070830 0x7f763010fc30 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:56:50.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:50.119+0000 7f7634cb7700 1 --2- 172.21.15.107:0/251326663 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7630071de0 0x7f7630110170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:56:50.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:50.119+0000 7f7634cb7700 1 -- 172.21.15.107:0/251326663 >> 172.21.15.107:0/251326663 conn(0x7f76300f8540 msgr2=0x7f76300f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:56:50.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:50.120+0000 7f7634cb7700 1 -- 172.21.15.107:0/251326663 shutdown_connections 2024-05-17T21:56:50.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:50.120+0000 7f7634cb7700 1 -- 172.21.15.107:0/251326663 wait complete. 2024-05-17T21:56:50.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:50.238+0000 7fc62002a700 1 -- 172.21.15.107:0/2608682534 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6180662c0 msgr2=0x7fc6180ff9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:56:50.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:50.238+0000 7fc62002a700 1 --2- 172.21.15.107:0/2608682534 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6180662c0 0x7fc6180ff9a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc614004940 tx=0x7fc614009e30 comp rx=0 tx=0).stop 2024-05-17T21:56:50.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:50.239+0000 7fc62002a700 1 -- 172.21.15.107:0/2608682534 shutdown_connections 2024-05-17T21:56:50.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:50.239+0000 7fc62002a700 1 --2- 172.21.15.107:0/2608682534 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc6180ffee0 0x7fc618102300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:56:50.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:50.239+0000 7fc62002a700 1 --2- 172.21.15.107:0/2608682534 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6180662c0 0x7fc6180ff9a0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:56:50.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:50.239+0000 7fc62002a700 1 -- 172.21.15.107:0/2608682534 >> 172.21.15.107:0/2608682534 conn(0x7fc6180f7430 msgr2=0x7fc6180f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:56:50.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:50.239+0000 7fc62002a700 1 -- 172.21.15.107:0/2608682534 shutdown_connections 2024-05-17T21:56:50.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:50.239+0000 7fc62002a700 1 -- 172.21.15.107:0/2608682534 wait complete. 2024-05-17T21:56:50.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:50.239+0000 7fc62002a700 1 Processor -- start 2024-05-17T21:56:50.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:50.239+0000 7fc62002a700 1 -- start start 2024-05-17T21:56:50.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:50.239+0000 7fc62002a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc6180662c0 0x7fc6180fd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:56:50.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:50.239+0000 7fc62002a700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6180ffee0 0x7fc6180fb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:56:50.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:50.240+0000 7fc62002a700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6180fbe50 con 0x7fc6180ffee0 2024-05-17T21:56:50.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:50.240+0000 7fc62002a700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6180fbfc0 con 0x7fc6180662c0 2024-05-17T21:56:50.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:50.240+0000 7fc61d5c5700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6180ffee0 0x7fc6180fb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:56:50.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:50.240+0000 7fc61d5c5700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6180ffee0 0x7fc6180fb8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55916/0 (socket says 172.21.15.107:55916) 2024-05-17T21:56:50.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:50.240+0000 7fc61d5c5700 1 -- 172.21.15.107:0/110751100 learned_addr learned my addr 172.21.15.107:0/110751100 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:56:50.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:50.240+0000 7fc61ddc6700 1 --2- 172.21.15.107:0/110751100 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc6180662c0 0x7fc6180fd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:56:50.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:50.240+0000 7fc61d5c5700 1 -- 172.21.15.107:0/110751100 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc6180662c0 msgr2=0x7fc6180fd3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:56:50.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:50.240+0000 7fc61d5c5700 1 --2- 172.21.15.107:0/110751100 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc6180662c0 0x7fc6180fd3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:56:50.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:50.240+0000 7fc61d5c5700 1 -- 172.21.15.107:0/110751100 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc614005040 con 0x7fc6180ffee0 2024-05-17T21:56:50.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:50.240+0000 7fc61d5c5700 1 --2- 172.21.15.107:0/110751100 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6180ffee0 0x7fc6180fb8a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc608002a50 tx=0x7fc6080046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:56:50.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:50.240+0000 7fc60e7fc700 1 -- 172.21.15.107:0/110751100 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc608007cc0 con 0x7fc6180ffee0 2024-05-17T21:56:50.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:50.240+0000 7fc62002a700 1 -- 172.21.15.107:0/110751100 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6180fc2a0 con 0x7fc6180ffee0 2024-05-17T21:56:50.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:50.241+0000 7fc62002a700 1 -- 172.21.15.107:0/110751100 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc618115a50 con 0x7fc6180ffee0 2024-05-17T21:56:50.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:50.241+0000 7fc60e7fc700 1 -- 172.21.15.107:0/110751100 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc60800f040 con 0x7fc6180ffee0 2024-05-17T21:56:50.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:50.241+0000 7fc60e7fc700 1 -- 172.21.15.107:0/110751100 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc60800da30 con 0x7fc6180ffee0 2024-05-17T21:56:50.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:50.242+0000 7fc62002a700 1 -- 172.21.15.107:0/110751100 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5fc0052e0 con 0x7fc6180ffee0 2024-05-17T21:56:50.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:50.242+0000 7fc60e7fc700 1 -- 172.21.15.107:0/110751100 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7fc60800db90 con 0x7fc6180ffee0 2024-05-17T21:56:50.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:50.243+0000 7fc60e7fc700 1 --2- 172.21.15.107:0/110751100 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fc604078990 0x7fc60407ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:56:50.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:50.243+0000 7fc60e7fc700 1 -- 172.21.15.107:0/110751100 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fc60809aa00 con 0x7fc6180ffee0 2024-05-17T21:56:50.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:50.243+0000 7fc61ddc6700 1 --2- 172.21.15.107:0/110751100 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fc604078990 0x7fc60407ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:56:50.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:50.244+0000 7fc61ddc6700 1 --2- 172.21.15.107:0/110751100 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fc604078990 0x7fc60407ae40 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc61400b970 tx=0x7fc614009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:56:50.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:50.247+0000 7fc60e7fc700 1 -- 172.21.15.107:0/110751100 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7fc6080626f0 con 0x7fc6180ffee0 2024-05-17T21:56:50.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:50.547+0000 7fc62002a700 1 -- 172.21.15.107:0/110751100 --> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fc5fc000bf0 con 0x7fc604078990 2024-05-17T21:56:50.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:50.558+0000 7fc60e7fc700 1 -- 172.21.15.107:0/110751100 <== mgr.34100 v2:172.21.15.107:6800/3205268226 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7fc5fc000bf0 con 0x7fc604078990 2024-05-17T21:56:50.550 INFO:teuthology.orchestra.run.smithi107.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-17T21:56:50.550 INFO:teuthology.orchestra.run.smithi107.stdout:alertmanager.smithi107 smithi107 *:9093,9094 running (4m) 0s ago 12m 24.3M - 0.25.0 c8568f914cd2 bac2a944ae42 2024-05-17T21:56:50.550 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi107 smithi107 running (68s) 0s ago 13m 6669k - 19.0.0-3794-g931809a9 8880f7613d02 face35de205b 2024-05-17T21:56:50.550 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi138 smithi138 running (63s) 59s ago 12m 6665k - 19.0.0-3794-g931809a9 8880f7613d02 ad59e33e6d43 2024-05-17T21:56:50.550 INFO:teuthology.orchestra.run.smithi107.stdout:grafana.smithi107 smithi107 *:3000 running (4m) 0s ago 12m 84.6M - 9.4.7 954c08fa6188 8c747e777bc5 2024-05-17T21:56:50.550 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.drqody smithi107 running (7m) 0s ago 7m 103M - 17.2.7-952-g9903f776 d15a36853796 0d851bc912f0 2024-05-17T21:56:50.550 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.idmbgj smithi107 running (7m) 0s ago 7m 13.7M - 17.2.7-952-g9903f776 d15a36853796 1348b09ab29e 2024-05-17T21:56:50.550 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi138.gwbtyd smithi138 running (7m) 59s ago 7m 152M - 17.2.7-952-g9903f776 d15a36853796 fd2ba75354c0 2024-05-17T21:56:50.551 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi138.zckrvs smithi138 running (7m) 59s ago 7m 14.1M - 17.2.7-952-g9903f776 d15a36853796 a1cfd2fe4cda 2024-05-17T21:56:50.551 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi107.xhhyiw smithi107 *:8443,9283,8765 running (3m) 0s ago 14m 586M - 19.0.0-3794-g931809a9 8880f7613d02 5acf3e511342 2024-05-17T21:56:50.551 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi138.napnll smithi138 *:8443,9283,8765 running (2m) 59s ago 11m 531M - 19.0.0-3794-g931809a9 8880f7613d02 d627761dbd0d 2024-05-17T21:56:50.551 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi107 smithi107 running (2m) 0s ago 14m 114M 2048M 19.0.0-3794-g931809a9 8880f7613d02 9717f3169f2f 2024-05-17T21:56:50.551 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi138 smithi138 running (108s) 59s ago 11m 99.9M 2048M 19.0.0-3794-g931809a9 8880f7613d02 55aeed62bf4d 2024-05-17T21:56:50.551 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi107 smithi107 *:9100 running (4m) 0s ago 13m 17.4M - 1.5.0 0da6a335fe13 49bec71f5487 2024-05-17T21:56:50.551 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi138 smithi138 *:9100 running (3m) 59s ago 12m 15.9M - 1.5.0 0da6a335fe13 e52824376658 2024-05-17T21:56:50.551 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0 smithi107 running (42s) 0s ago 11m 156M 1557M 19.0.0-3794-g931809a9 8880f7613d02 b319bc93500d 2024-05-17T21:56:50.551 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1 smithi107 running (6s) 0s ago 10m 80.0M 1557M 19.0.0-3794-g931809a9 8880f7613d02 f94b206be6eb 2024-05-17T21:56:50.551 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2 smithi107 running (10m) 0s ago 10m 180M 1557M 17.2.7-952-g9903f776 d15a36853796 e044d7b78088 2024-05-17T21:56:50.551 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3 smithi138 running (9m) 59s ago 9m 228M 2581M 17.2.7-952-g9903f776 d15a36853796 8c980cc8af87 2024-05-17T21:56:50.551 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4 smithi138 running (9m) 59s ago 9m 204M 2581M 17.2.7-952-g9903f776 d15a36853796 b306a36a744e 2024-05-17T21:56:50.551 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5 smithi138 running (8m) 59s ago 8m 190M 2581M 17.2.7-952-g9903f776 d15a36853796 7c8d8ed74974 2024-05-17T21:56:50.551 INFO:teuthology.orchestra.run.smithi107.stdout:prometheus.smithi107 smithi107 *:9095 running (3m) 0s ago 12m 58.7M - 2.43.0 a07b618ecd1d bf5f609bbd6e 2024-05-17T21:56:50.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:50.561+0000 7fc62002a700 1 -- 172.21.15.107:0/110751100 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fc604078990 msgr2=0x7fc60407ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:56:50.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:50.561+0000 7fc62002a700 1 --2- 172.21.15.107:0/110751100 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fc604078990 0x7fc60407ae40 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc61400b970 tx=0x7fc614009e30 comp rx=0 tx=0).stop 2024-05-17T21:56:50.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:50.561+0000 7fc62002a700 1 -- 172.21.15.107:0/110751100 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6180ffee0 msgr2=0x7fc6180fb8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:56:50.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:50.561+0000 7fc62002a700 1 --2- 172.21.15.107:0/110751100 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6180ffee0 0x7fc6180fb8a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc608002a50 tx=0x7fc6080046b0 comp rx=0 tx=0).stop 2024-05-17T21:56:50.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:50.561+0000 7fc62002a700 1 -- 172.21.15.107:0/110751100 shutdown_connections 2024-05-17T21:56:50.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:50.561+0000 7fc62002a700 1 --2- 172.21.15.107:0/110751100 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fc604078990 0x7fc60407ae40 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:56:50.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:50.561+0000 7fc62002a700 1 --2- 172.21.15.107:0/110751100 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc6180662c0 0x7fc6180fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:56:50.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:50.561+0000 7fc62002a700 1 --2- 172.21.15.107:0/110751100 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6180ffee0 0x7fc6180fb8a0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:56:50.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:50.561+0000 7fc62002a700 1 -- 172.21.15.107:0/110751100 >> 172.21.15.107:0/110751100 conn(0x7fc6180f7430 msgr2=0x7fc6180f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:56:50.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:50.561+0000 7fc62002a700 1 -- 172.21.15.107:0/110751100 shutdown_connections 2024-05-17T21:56:50.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:50.561+0000 7fc62002a700 1 -- 172.21.15.107:0/110751100 wait complete. 2024-05-17T21:56:50.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:50.669+0000 7f6f099fe700 1 -- 172.21.15.107:0/1827093379 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f040fdfd0 msgr2=0x7f6f040fe440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:56:50.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:50.669+0000 7f6f099fe700 1 --2- 172.21.15.107:0/1827093379 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f040fdfd0 0x7f6f040fe440 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f6ef8004940 tx=0x7f6ef8009e30 comp rx=0 tx=0).stop 2024-05-17T21:56:50.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:50.669+0000 7f6f099fe700 1 -- 172.21.15.107:0/1827093379 shutdown_connections 2024-05-17T21:56:50.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:50.669+0000 7f6f099fe700 1 --2- 172.21.15.107:0/1827093379 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f040fdfd0 0x7f6f040fe440 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:56:50.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:50.669+0000 7f6f099fe700 1 --2- 172.21.15.107:0/1827093379 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6f040fce60 0x7f6f040fd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:56:50.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:50.669+0000 7f6f099fe700 1 -- 172.21.15.107:0/1827093379 >> 172.21.15.107:0/1827093379 conn(0x7f6f040f8540 msgr2=0x7f6f040fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:56:50.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:50.669+0000 7f6f099fe700 1 -- 172.21.15.107:0/1827093379 shutdown_connections 2024-05-17T21:56:50.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:50.669+0000 7f6f099fe700 1 -- 172.21.15.107:0/1827093379 wait complete. 2024-05-17T21:56:50.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:50.670+0000 7f6f099fe700 1 Processor -- start 2024-05-17T21:56:50.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:50.670+0000 7f6f099fe700 1 -- start start 2024-05-17T21:56:50.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:50.670+0000 7f6f099fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f040fce60 0x7f6f0410d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:56:50.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:50.670+0000 7f6f099fe700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6f040fdfd0 0x7f6f0410de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:56:50.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:50.670+0000 7f6f099fe700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f0410e450 con 0x7f6f040fce60 2024-05-17T21:56:50.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:50.670+0000 7f6f099fe700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f0410e5c0 con 0x7f6f040fdfd0 2024-05-17T21:56:50.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:50.670+0000 7f6f02ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f040fce60 0x7f6f0410d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:56:50.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:50.670+0000 7f6f02ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f040fce60 0x7f6f0410d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55954/0 (socket says 172.21.15.107:55954) 2024-05-17T21:56:50.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:50.670+0000 7f6f02ffd700 1 -- 172.21.15.107:0/1639933909 learned_addr learned my addr 172.21.15.107:0/1639933909 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:56:50.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:50.671+0000 7f6f027fc700 1 --2- 172.21.15.107:0/1639933909 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6f040fdfd0 0x7f6f0410de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:56:50.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:50.671+0000 7f6f02ffd700 1 -- 172.21.15.107:0/1639933909 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6f040fdfd0 msgr2=0x7f6f0410de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:56:50.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:50.671+0000 7f6f02ffd700 1 --2- 172.21.15.107:0/1639933909 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6f040fdfd0 0x7f6f0410de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:56:50.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:50.671+0000 7f6f02ffd700 1 -- 172.21.15.107:0/1639933909 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ef8005040 con 0x7f6f040fce60 2024-05-17T21:56:50.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:50.671+0000 7f6f02ffd700 1 --2- 172.21.15.107:0/1639933909 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f040fce60 0x7f6f0410d910 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6ef400c8c0 tx=0x7f6ef40044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:56:50.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:50.671+0000 7f6eeb7fe700 1 -- 172.21.15.107:0/1639933909 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ef400a980 con 0x7f6f040fce60 2024-05-17T21:56:50.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:50.671+0000 7f6f099fe700 1 -- 172.21.15.107:0/1639933909 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f04112fc0 con 0x7f6f040fce60 2024-05-17T21:56:50.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:50.671+0000 7f6eeb7fe700 1 -- 172.21.15.107:0/1639933909 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6ef400ce60 con 0x7f6f040fce60 2024-05-17T21:56:50.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:50.671+0000 7f6eeb7fe700 1 -- 172.21.15.107:0/1639933909 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ef40115e0 con 0x7f6f040fce60 2024-05-17T21:56:50.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:50.671+0000 7f6f099fe700 1 -- 172.21.15.107:0/1639933909 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f04113550 con 0x7f6f040fce60 2024-05-17T21:56:50.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:50.672+0000 7f6f099fe700 1 -- 172.21.15.107:0/1639933909 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f0404c710 con 0x7f6f040fce60 2024-05-17T21:56:50.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:50.673+0000 7f6eeb7fe700 1 -- 172.21.15.107:0/1639933909 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f6ef4012400 con 0x7f6f040fce60 2024-05-17T21:56:50.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:50.674+0000 7f6eeb7fe700 1 --2- 172.21.15.107:0/1639933909 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f6eec078990 0x7f6eec07ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:56:50.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:50.674+0000 7f6f027fc700 1 --2- 172.21.15.107:0/1639933909 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f6eec078990 0x7f6eec07ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:56:50.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:50.674+0000 7f6eeb7fe700 1 -- 172.21.15.107:0/1639933909 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f6ef409b3e0 con 0x7f6f040fce60 2024-05-17T21:56:50.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:50.674+0000 7f6f027fc700 1 --2- 172.21.15.107:0/1639933909 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f6eec078990 0x7f6eec07ae40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6ef800b3b0 tx=0x7f6ef8009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:56:50.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:50.678+0000 7f6eeb7fe700 1 -- 172.21.15.107:0/1639933909 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f6ef40630f0 con 0x7f6f040fce60 2024-05-17T21:56:50.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:50 smithi107 ceph-mon[93224]: from='client.34250 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:56:50.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:50 smithi107 ceph-mon[93224]: from='client.34254 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:56:50.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:50 smithi107 ceph-mon[93224]: from='osd.1 [v2:172.21.15.107:6810/748281068,v1:172.21.15.107:6811/748281068]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-17T21:56:50.852 INFO:journalctl@ceph.osd.1.smithi107.stdout:May 17 21:56:50 smithi107 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-1[106965]: 2024-05-17T21:56:50.509+0000 7fa268ab37c0 -1 osd.1 54 log_to_monitors true 2024-05-17T21:56:50.852 INFO:journalctl@ceph.osd.1.smithi107.stdout:May 17 21:56:50 smithi107 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-1[106965]: 2024-05-17T21:56:50.745+0000 7fa259ecc700 -1 osd.1 54 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-17T21:56:50.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:50.991+0000 7f6f099fe700 1 -- 172.21.15.107:0/1639933909 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f6f04113b10 con 0x7f6f040fce60 2024-05-17T21:56:50.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:50.991+0000 7f6eeb7fe700 1 -- 172.21.15.107:0/1639933909 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+839 (secure 0 0 0) 0x7f6ef40126b0 con 0x7f6f040fce60 2024-05-17T21:56:50.983 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-05-17T21:56:50.984 INFO:teuthology.orchestra.run.smithi107.stdout: "mon": { 2024-05-17T21:56:50.984 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev)": 2 2024-05-17T21:56:50.984 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-05-17T21:56:50.984 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr": { 2024-05-17T21:56:50.984 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev)": 2 2024-05-17T21:56:50.984 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-05-17T21:56:50.984 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": { 2024-05-17T21:56:50.984 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-952-g9903f776 (9903f7767386a9d9efadb7c1effdcd3064d71804) quincy (stable)": 4, 2024-05-17T21:56:50.984 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev)": 1 2024-05-17T21:56:50.984 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-05-17T21:56:50.984 INFO:teuthology.orchestra.run.smithi107.stdout: "mds": { 2024-05-17T21:56:50.984 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-952-g9903f776 (9903f7767386a9d9efadb7c1effdcd3064d71804) quincy (stable)": 4 2024-05-17T21:56:50.984 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-05-17T21:56:50.984 INFO:teuthology.orchestra.run.smithi107.stdout: "overall": { 2024-05-17T21:56:50.985 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-952-g9903f776 (9903f7767386a9d9efadb7c1effdcd3064d71804) quincy (stable)": 8, 2024-05-17T21:56:50.985 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev)": 5 2024-05-17T21:56:50.985 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-05-17T21:56:50.985 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-05-17T21:56:50.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:50.994+0000 7f6f099fe700 1 -- 172.21.15.107:0/1639933909 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f6eec078990 msgr2=0x7f6eec07ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:56:50.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:50.994+0000 7f6f099fe700 1 --2- 172.21.15.107:0/1639933909 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f6eec078990 0x7f6eec07ae40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6ef800b3b0 tx=0x7f6ef8009e30 comp rx=0 tx=0).stop 2024-05-17T21:56:50.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:50.994+0000 7f6f099fe700 1 -- 172.21.15.107:0/1639933909 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f040fce60 msgr2=0x7f6f0410d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:56:50.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:50.994+0000 7f6f099fe700 1 --2- 172.21.15.107:0/1639933909 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f040fce60 0x7f6f0410d910 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6ef400c8c0 tx=0x7f6ef40044c0 comp rx=0 tx=0).stop 2024-05-17T21:56:50.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:50.994+0000 7f6f099fe700 1 -- 172.21.15.107:0/1639933909 shutdown_connections 2024-05-17T21:56:50.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:50.994+0000 7f6f099fe700 1 --2- 172.21.15.107:0/1639933909 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f6eec078990 0x7f6eec07ae40 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:56:50.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:50.994+0000 7f6f099fe700 1 --2- 172.21.15.107:0/1639933909 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f040fce60 0x7f6f0410d910 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:56:50.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:50.994+0000 7f6f099fe700 1 --2- 172.21.15.107:0/1639933909 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6f040fdfd0 0x7f6f0410de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:56:50.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:50.994+0000 7f6f099fe700 1 -- 172.21.15.107:0/1639933909 >> 172.21.15.107:0/1639933909 conn(0x7f6f040f8540 msgr2=0x7f6f040f9c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:56:50.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:50.994+0000 7f6f099fe700 1 -- 172.21.15.107:0/1639933909 shutdown_connections 2024-05-17T21:56:50.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:50.994+0000 7f6f099fe700 1 -- 172.21.15.107:0/1639933909 wait complete. 2024-05-17T21:56:51.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:50 smithi138 ceph-mon[81376]: from='client.34250 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:56:51.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:50 smithi138 ceph-mon[81376]: from='client.34254 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:56:51.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:50 smithi138 ceph-mon[81376]: from='osd.1 [v2:172.21.15.107:6810/748281068,v1:172.21.15.107:6811/748281068]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-17T21:56:51.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.101+0000 7f1490a8e700 1 -- 172.21.15.107:0/3702014859 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f148c0fe3e0 msgr2=0x7f148c0fe7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:56:51.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.101+0000 7f1490a8e700 1 --2- 172.21.15.107:0/3702014859 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f148c0fe3e0 0x7f148c0fe7f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1480004940 tx=0x7f1480009e30 comp rx=0 tx=0).stop 2024-05-17T21:56:51.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.101+0000 7f1490a8e700 1 -- 172.21.15.107:0/3702014859 shutdown_connections 2024-05-17T21:56:51.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.101+0000 7f1490a8e700 1 --2- 172.21.15.107:0/3702014859 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f148c0fed30 0x7f148c0fae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:56:51.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.101+0000 7f1490a8e700 1 --2- 172.21.15.107:0/3702014859 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f148c0fe3e0 0x7f148c0fe7f0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:56:51.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.101+0000 7f1490a8e700 1 -- 172.21.15.107:0/3702014859 >> 172.21.15.107:0/3702014859 conn(0x7f148c0f6b80 msgr2=0x7f148c0f8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:56:51.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.101+0000 7f1490a8e700 1 -- 172.21.15.107:0/3702014859 shutdown_connections 2024-05-17T21:56:51.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.101+0000 7f1490a8e700 1 -- 172.21.15.107:0/3702014859 wait complete. 2024-05-17T21:56:51.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.101+0000 7f1490a8e700 1 Processor -- start 2024-05-17T21:56:51.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.102+0000 7f1490a8e700 1 -- start start 2024-05-17T21:56:51.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.102+0000 7f1490a8e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f148c0fe3e0 0x7f148c10fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:56:51.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.102+0000 7f1490a8e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f148c0fed30 0x7f148c110110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:56:51.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.102+0000 7f1490a8e700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f148c110710 con 0x7f148c0fe3e0 2024-05-17T21:56:51.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.102+0000 7f1490a8e700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f148c110f60 con 0x7f148c0fed30 2024-05-17T21:56:51.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.102+0000 7f148a59c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f148c0fe3e0 0x7f148c10fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:56:51.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.102+0000 7f148a59c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f148c0fe3e0 0x7f148c10fbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55968/0 (socket says 172.21.15.107:55968) 2024-05-17T21:56:51.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.102+0000 7f148a59c700 1 -- 172.21.15.107:0/1481612217 learned_addr learned my addr 172.21.15.107:0/1481612217 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:56:51.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.102+0000 7f1489d9b700 1 --2- 172.21.15.107:0/1481612217 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f148c0fed30 0x7f148c110110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:56:51.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.102+0000 7f148a59c700 1 -- 172.21.15.107:0/1481612217 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f148c0fed30 msgr2=0x7f148c110110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:56:51.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.102+0000 7f148a59c700 1 --2- 172.21.15.107:0/1481612217 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f148c0fed30 0x7f148c110110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:56:51.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.102+0000 7f148a59c700 1 -- 172.21.15.107:0/1481612217 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1480005040 con 0x7f148c0fe3e0 2024-05-17T21:56:51.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.102+0000 7f148a59c700 1 --2- 172.21.15.107:0/1481612217 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f148c0fe3e0 0x7f148c10fbd0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1480009150 tx=0x7f1480009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:56:51.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.103+0000 7f147affd700 1 -- 172.21.15.107:0/1481612217 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f148000e040 con 0x7f148c0fe3e0 2024-05-17T21:56:51.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.103+0000 7f1490a8e700 1 -- 172.21.15.107:0/1481612217 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f148c1111e0 con 0x7f148c0fe3e0 2024-05-17T21:56:51.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.103+0000 7f147affd700 1 -- 172.21.15.107:0/1481612217 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1480004290 con 0x7f148c0fe3e0 2024-05-17T21:56:51.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.103+0000 7f147affd700 1 -- 172.21.15.107:0/1481612217 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f148001c630 con 0x7f148c0fe3e0 2024-05-17T21:56:51.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.103+0000 7f1490a8e700 1 -- 172.21.15.107:0/1481612217 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f148c111720 con 0x7f148c0fe3e0 2024-05-17T21:56:51.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.104+0000 7f1490a8e700 1 -- 172.21.15.107:0/1481612217 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f148c101b20 con 0x7f148c0fe3e0 2024-05-17T21:56:51.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.104+0000 7f147affd700 1 -- 172.21.15.107:0/1481612217 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f148000bbf0 con 0x7f148c0fe3e0 2024-05-17T21:56:51.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.105+0000 7f147affd700 1 --2- 172.21.15.107:0/1481612217 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f1474078990 0x7f147407ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:56:51.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.105+0000 7f147affd700 1 -- 172.21.15.107:0/1481612217 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f148009b2f0 con 0x7f148c0fe3e0 2024-05-17T21:56:51.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.105+0000 7f1489d9b700 1 --2- 172.21.15.107:0/1481612217 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f1474078990 0x7f147407ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:56:51.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.105+0000 7f1489d9b700 1 --2- 172.21.15.107:0/1481612217 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f1474078990 0x7f147407ae40 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f147c009fd0 tx=0x7f147c00b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:56:51.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.114+0000 7f147affd700 1 -- 172.21.15.107:0/1481612217 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f1480062fe0 con 0x7f148c0fe3e0 2024-05-17T21:56:51.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.379+0000 7f1490a8e700 1 -- 172.21.15.107:0/1481612217 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f148c105a20 con 0x7f148c0fe3e0 2024-05-17T21:56:51.371 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.379+0000 7f147affd700 1 -- 172.21.15.107:0/1481612217 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1910 (secure 0 0 0) 0x7f148c105a20 con 0x7f148c0fe3e0 2024-05-17T21:56:51.371 INFO:teuthology.orchestra.run.smithi107.stdout:e15 2024-05-17T21:56:51.371 INFO:teuthology.orchestra.run.smithi107.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-17T21:56:51.371 INFO:teuthology.orchestra.run.smithi107.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-17T21:56:51.371 INFO:teuthology.orchestra.run.smithi107.stdout:legacy client fscid: 1 2024-05-17T21:56:51.372 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:56:51.372 INFO:teuthology.orchestra.run.smithi107.stdout:Filesystem 'cephfs' (1) 2024-05-17T21:56:51.372 INFO:teuthology.orchestra.run.smithi107.stdout:fs_name cephfs 2024-05-17T21:56:51.372 INFO:teuthology.orchestra.run.smithi107.stdout:epoch 15 2024-05-17T21:56:51.372 INFO:teuthology.orchestra.run.smithi107.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-17T21:56:51.372 INFO:teuthology.orchestra.run.smithi107.stdout:created 2024-05-17T21:49:16.836333+0000 2024-05-17T21:56:51.372 INFO:teuthology.orchestra.run.smithi107.stdout:modified 2024-05-17T21:49:34.873628+0000 2024-05-17T21:56:51.372 INFO:teuthology.orchestra.run.smithi107.stdout:tableserver 0 2024-05-17T21:56:51.372 INFO:teuthology.orchestra.run.smithi107.stdout:root 0 2024-05-17T21:56:51.372 INFO:teuthology.orchestra.run.smithi107.stdout:session_timeout 60 2024-05-17T21:56:51.372 INFO:teuthology.orchestra.run.smithi107.stdout:session_autoclose 300 2024-05-17T21:56:51.372 INFO:teuthology.orchestra.run.smithi107.stdout:max_file_size 1099511627776 2024-05-17T21:56:51.372 INFO:teuthology.orchestra.run.smithi107.stdout:max_xattr_size 65536 2024-05-17T21:56:51.372 INFO:teuthology.orchestra.run.smithi107.stdout:required_client_features {} 2024-05-17T21:56:51.372 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure 0 2024-05-17T21:56:51.373 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure_osd_epoch 46 2024-05-17T21:56:51.373 INFO:teuthology.orchestra.run.smithi107.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-17T21:56:51.373 INFO:teuthology.orchestra.run.smithi107.stdout:max_mds 1 2024-05-17T21:56:51.373 INFO:teuthology.orchestra.run.smithi107.stdout:in 0 2024-05-17T21:56:51.373 INFO:teuthology.orchestra.run.smithi107.stdout:up {0=24259} 2024-05-17T21:56:51.373 INFO:teuthology.orchestra.run.smithi107.stdout:failed 2024-05-17T21:56:51.373 INFO:teuthology.orchestra.run.smithi107.stdout:damaged 2024-05-17T21:56:51.373 INFO:teuthology.orchestra.run.smithi107.stdout:stopped 2024-05-17T21:56:51.373 INFO:teuthology.orchestra.run.smithi107.stdout:data_pools [3] 2024-05-17T21:56:51.373 INFO:teuthology.orchestra.run.smithi107.stdout:metadata_pool 2 2024-05-17T21:56:51.373 INFO:teuthology.orchestra.run.smithi107.stdout:inline_data enabled 2024-05-17T21:56:51.373 INFO:teuthology.orchestra.run.smithi107.stdout:balancer 2024-05-17T21:56:51.373 INFO:teuthology.orchestra.run.smithi107.stdout:bal_rank_mask -1 2024-05-17T21:56:51.373 INFO:teuthology.orchestra.run.smithi107.stdout:standby_count_wanted 1 2024-05-17T21:56:51.374 INFO:teuthology.orchestra.run.smithi107.stdout:qdb_cluster leader: 0 members: 2024-05-17T21:56:51.374 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi138.gwbtyd{0:24259} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.138:6824/2779634919,v1:172.21.15.138:6825/2779634919] compat {c=[1],r=[1],i=[7ff]}] 2024-05-17T21:56:51.374 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.drqody{0:14486} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.107:6828/2275807100,v1:172.21.15.107:6829/2275807100] compat {c=[1],r=[1],i=[7ff]}] 2024-05-17T21:56:51.374 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:56:51.374 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:56:51.374 INFO:teuthology.orchestra.run.smithi107.stdout:Standby daemons: 2024-05-17T21:56:51.374 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:56:51.374 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.idmbgj{-1:14498} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.107:6826/1460186079,v1:172.21.15.107:6827/1460186079] compat {c=[1],r=[1],i=[7ff]}] 2024-05-17T21:56:51.374 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi138.zckrvs{-1:24267} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.138:6826/170628661,v1:172.21.15.138:6827/170628661] compat {c=[1],r=[1],i=[7ff]}] 2024-05-17T21:56:51.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.381+0000 7f1490a8e700 1 -- 172.21.15.107:0/1481612217 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f1474078990 msgr2=0x7f147407ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:56:51.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.381+0000 7f1490a8e700 1 --2- 172.21.15.107:0/1481612217 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f1474078990 0x7f147407ae40 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f147c009fd0 tx=0x7f147c00b040 comp rx=0 tx=0).stop 2024-05-17T21:56:51.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.381+0000 7f1490a8e700 1 -- 172.21.15.107:0/1481612217 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f148c0fe3e0 msgr2=0x7f148c10fbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:56:51.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.381+0000 7f1490a8e700 1 --2- 172.21.15.107:0/1481612217 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f148c0fe3e0 0x7f148c10fbd0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1480009150 tx=0x7f1480009e30 comp rx=0 tx=0).stop 2024-05-17T21:56:51.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.381+0000 7f1490a8e700 1 -- 172.21.15.107:0/1481612217 shutdown_connections 2024-05-17T21:56:51.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.381+0000 7f1490a8e700 1 --2- 172.21.15.107:0/1481612217 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f1474078990 0x7f147407ae40 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:56:51.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.381+0000 7f1490a8e700 1 --2- 172.21.15.107:0/1481612217 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f148c0fe3e0 0x7f148c10fbd0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:56:51.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.381+0000 7f1490a8e700 1 --2- 172.21.15.107:0/1481612217 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f148c0fed30 0x7f148c110110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:56:51.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.381+0000 7f1490a8e700 1 -- 172.21.15.107:0/1481612217 >> 172.21.15.107:0/1481612217 conn(0x7f148c0f6b80 msgr2=0x7f148c0f8130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:56:51.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.382+0000 7f1490a8e700 1 -- 172.21.15.107:0/1481612217 shutdown_connections 2024-05-17T21:56:51.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.382+0000 7f1490a8e700 1 -- 172.21.15.107:0/1481612217 wait complete. 2024-05-17T21:56:51.376 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 15 2024-05-17T21:56:51.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.490+0000 7f758757a700 1 -- 172.21.15.107:0/2807263665 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f75800fbcc0 msgr2=0x7f75800fc0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:56:51.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.490+0000 7f758757a700 1 --2- 172.21.15.107:0/2807263665 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f75800fbcc0 0x7f75800fc0d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f757c004940 tx=0x7f757c009e30 comp rx=0 tx=0).stop 2024-05-17T21:56:51.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.490+0000 7f758757a700 1 -- 172.21.15.107:0/2807263665 shutdown_connections 2024-05-17T21:56:51.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.490+0000 7f758757a700 1 --2- 172.21.15.107:0/2807263665 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f75800fcf70 0x7f75800fd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:56:51.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.490+0000 7f758757a700 1 --2- 172.21.15.107:0/2807263665 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f75800fbcc0 0x7f75800fc0d0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:56:51.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.490+0000 7f758757a700 1 -- 172.21.15.107:0/2807263665 >> 172.21.15.107:0/2807263665 conn(0x7f75800f7440 msgr2=0x7f75800f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:56:51.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.490+0000 7f758757a700 1 -- 172.21.15.107:0/2807263665 shutdown_connections 2024-05-17T21:56:51.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.490+0000 7f758757a700 1 -- 172.21.15.107:0/2807263665 wait complete. 2024-05-17T21:56:51.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.490+0000 7f758757a700 1 Processor -- start 2024-05-17T21:56:51.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.490+0000 7f758757a700 1 -- start start 2024-05-17T21:56:51.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.491+0000 7f758757a700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f75800fbcc0 0x7f758010c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:56:51.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.491+0000 7f758757a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f75800fcf70 0x7f758010cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:56:51.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.491+0000 7f758757a700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f758010d390 con 0x7f75800fbcc0 2024-05-17T21:56:51.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.491+0000 7f758757a700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f758010d500 con 0x7f75800fcf70 2024-05-17T21:56:51.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.491+0000 7f7585316700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f75800fbcc0 0x7f758010c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:56:51.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.491+0000 7f7585316700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f75800fbcc0 0x7f758010c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55982/0 (socket says 172.21.15.107:55982) 2024-05-17T21:56:51.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.491+0000 7f7585316700 1 -- 172.21.15.107:0/3366232553 learned_addr learned my addr 172.21.15.107:0/3366232553 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:56:51.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.491+0000 7f7584b15700 1 --2- 172.21.15.107:0/3366232553 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f75800fcf70 0x7f758010cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:56:51.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.491+0000 7f7585316700 1 -- 172.21.15.107:0/3366232553 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f75800fcf70 msgr2=0x7f758010cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:56:51.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.491+0000 7f7585316700 1 --2- 172.21.15.107:0/3366232553 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f75800fcf70 0x7f758010cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:56:51.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.491+0000 7f7585316700 1 -- 172.21.15.107:0/3366232553 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f757c005040 con 0x7f75800fbcc0 2024-05-17T21:56:51.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.491+0000 7f7585316700 1 --2- 172.21.15.107:0/3366232553 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f75800fbcc0 0x7f758010c850 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f757c00b3b0 tx=0x7f757c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:56:51.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.491+0000 7f7575ffb700 1 -- 172.21.15.107:0/3366232553 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f757c00e040 con 0x7f75800fbcc0 2024-05-17T21:56:51.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.491+0000 7f758757a700 1 -- 172.21.15.107:0/3366232553 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7580111f00 con 0x7f75800fbcc0 2024-05-17T21:56:51.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.491+0000 7f7575ffb700 1 -- 172.21.15.107:0/3366232553 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f757c00bcc0 con 0x7f75800fbcc0 2024-05-17T21:56:51.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.491+0000 7f7575ffb700 1 -- 172.21.15.107:0/3366232553 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f757c01c630 con 0x7f75800fbcc0 2024-05-17T21:56:51.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.492+0000 7f758757a700 1 -- 172.21.15.107:0/3366232553 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7580112430 con 0x7f75800fbcc0 2024-05-17T21:56:51.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.492+0000 7f758757a700 1 -- 172.21.15.107:0/3366232553 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f758004c710 con 0x7f75800fbcc0 2024-05-17T21:56:51.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.493+0000 7f7575ffb700 1 -- 172.21.15.107:0/3366232553 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f757c013980 con 0x7f75800fbcc0 2024-05-17T21:56:51.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.494+0000 7f7575ffb700 1 --2- 172.21.15.107:0/3366232553 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f756c078990 0x7f756c07ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:56:51.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.494+0000 7f7575ffb700 1 -- 172.21.15.107:0/3366232553 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f757c09b440 con 0x7f75800fbcc0 2024-05-17T21:56:51.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.494+0000 7f7584b15700 1 --2- 172.21.15.107:0/3366232553 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f756c078990 0x7f756c07ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:56:51.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.494+0000 7f7584b15700 1 --2- 172.21.15.107:0/3366232553 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f756c078990 0x7f756c07ae40 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7570004990 tx=0x7f7570009ee0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:56:51.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.498+0000 7f7575ffb700 1 -- 172.21.15.107:0/3366232553 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f757c0630b0 con 0x7f75800fbcc0 2024-05-17T21:56:51.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.739+0000 7f758757a700 1 -- 172.21.15.107:0/3366232553 --> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7580101280 con 0x7f756c078990 2024-05-17T21:56:51.737 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-05-17T21:56:51.737 INFO:teuthology.orchestra.run.smithi107.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b", 2024-05-17T21:56:51.737 INFO:teuthology.orchestra.run.smithi107.stdout: "in_progress": true, 2024-05-17T21:56:51.737 INFO:teuthology.orchestra.run.smithi107.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-17T21:56:51.737 INFO:teuthology.orchestra.run.smithi107.stdout: "services_complete": [ 2024-05-17T21:56:51.737 INFO:teuthology.orchestra.run.smithi107.stdout: "crash", 2024-05-17T21:56:51.737 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr", 2024-05-17T21:56:51.737 INFO:teuthology.orchestra.run.smithi107.stdout: "mon" 2024-05-17T21:56:51.737 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-05-17T21:56:51.737 INFO:teuthology.orchestra.run.smithi107.stdout: "progress": "8/21 daemons upgraded", 2024-05-17T21:56:51.737 INFO:teuthology.orchestra.run.smithi107.stdout: "message": "Currently upgrading osd daemons", 2024-05-17T21:56:51.738 INFO:teuthology.orchestra.run.smithi107.stdout: "is_paused": false 2024-05-17T21:56:51.738 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-05-17T21:56:51.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.745+0000 7f7575ffb700 1 -- 172.21.15.107:0/3366232553 <== mgr.34100 v2:172.21.15.107:6800/3205268226 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f7580101280 con 0x7f756c078990 2024-05-17T21:56:51.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.748+0000 7f758757a700 1 -- 172.21.15.107:0/3366232553 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f756c078990 msgr2=0x7f756c07ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:56:51.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.748+0000 7f758757a700 1 --2- 172.21.15.107:0/3366232553 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f756c078990 0x7f756c07ae40 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7570004990 tx=0x7f7570009ee0 comp rx=0 tx=0).stop 2024-05-17T21:56:51.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.748+0000 7f758757a700 1 -- 172.21.15.107:0/3366232553 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f75800fbcc0 msgr2=0x7f758010c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:56:51.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.748+0000 7f758757a700 1 --2- 172.21.15.107:0/3366232553 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f75800fbcc0 0x7f758010c850 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f757c00b3b0 tx=0x7f757c009e30 comp rx=0 tx=0).stop 2024-05-17T21:56:51.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.748+0000 7f758757a700 1 -- 172.21.15.107:0/3366232553 shutdown_connections 2024-05-17T21:56:51.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.748+0000 7f758757a700 1 --2- 172.21.15.107:0/3366232553 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f756c078990 0x7f756c07ae40 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:56:51.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.748+0000 7f758757a700 1 --2- 172.21.15.107:0/3366232553 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f75800fbcc0 0x7f758010c850 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:56:51.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.748+0000 7f758757a700 1 --2- 172.21.15.107:0/3366232553 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f75800fcf70 0x7f758010cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:56:51.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.748+0000 7f758757a700 1 -- 172.21.15.107:0/3366232553 >> 172.21.15.107:0/3366232553 conn(0x7f75800f7440 msgr2=0x7f75800f8b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:56:51.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.748+0000 7f758757a700 1 -- 172.21.15.107:0/3366232553 shutdown_connections 2024-05-17T21:56:51.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.748+0000 7f758757a700 1 -- 172.21.15.107:0/3366232553 wait complete. 2024-05-17T21:56:51.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.858+0000 7fdab8971700 1 -- 172.21.15.107:0/3931363578 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdab40fcf90 msgr2=0x7fdab40fd400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:56:51.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.858+0000 7fdab8971700 1 --2- 172.21.15.107:0/3931363578 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdab40fcf90 0x7fdab40fd400 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fdaa4004940 tx=0x7fdaa4009e30 comp rx=0 tx=0).stop 2024-05-17T21:56:51.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.858+0000 7fdab8971700 1 -- 172.21.15.107:0/3931363578 shutdown_connections 2024-05-17T21:56:51.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.858+0000 7fdab8971700 1 --2- 172.21.15.107:0/3931363578 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdab40fcf90 0x7fdab40fd400 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:56:51.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.858+0000 7fdab8971700 1 --2- 172.21.15.107:0/3931363578 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdab40fbce0 0x7fdab40fc0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:56:51.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.858+0000 7fdab8971700 1 -- 172.21.15.107:0/3931363578 >> 172.21.15.107:0/3931363578 conn(0x7fdab40f7480 msgr2=0x7fdab40f9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:56:51.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.858+0000 7fdab8971700 1 -- 172.21.15.107:0/3931363578 shutdown_connections 2024-05-17T21:56:51.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.858+0000 7fdab8971700 1 -- 172.21.15.107:0/3931363578 wait complete. 2024-05-17T21:56:51.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.858+0000 7fdab8971700 1 Processor -- start 2024-05-17T21:56:51.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.858+0000 7fdab8971700 1 -- start start 2024-05-17T21:56:51.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.858+0000 7fdab8971700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdab40fbce0 0x7fdab40ffc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:56:51.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.858+0000 7fdab8971700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdab40fcf90 0x7fdab40fe2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:56:51.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.858+0000 7fdab8971700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdab4100220 con 0x7fdab40fbce0 2024-05-17T21:56:51.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.858+0000 7fdab8971700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdab40fe810 con 0x7fdab40fcf90 2024-05-17T21:56:51.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.859+0000 7fdab259c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdab40fbce0 0x7fdab40ffc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:56:51.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.859+0000 7fdab259c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdab40fbce0 0x7fdab40ffc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56008/0 (socket says 172.21.15.107:56008) 2024-05-17T21:56:51.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.859+0000 7fdab259c700 1 -- 172.21.15.107:0/3669727647 learned_addr learned my addr 172.21.15.107:0/3669727647 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:56:51.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.859+0000 7fdab1d9b700 1 --2- 172.21.15.107:0/3669727647 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdab40fcf90 0x7fdab40fe2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:56:51.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.859+0000 7fdab259c700 1 -- 172.21.15.107:0/3669727647 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdab40fcf90 msgr2=0x7fdab40fe2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:56:51.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.859+0000 7fdab259c700 1 --2- 172.21.15.107:0/3669727647 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdab40fcf90 0x7fdab40fe2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:56:51.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.859+0000 7fdab259c700 1 -- 172.21.15.107:0/3669727647 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaa4005040 con 0x7fdab40fbce0 2024-05-17T21:56:51.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.859+0000 7fdab259c700 1 --2- 172.21.15.107:0/3669727647 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdab40fbce0 0x7fdab40ffc20 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fda9c002a50 tx=0x7fda9c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:56:51.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.859+0000 7fdaaaffd700 1 -- 172.21.15.107:0/3669727647 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fda9c007cc0 con 0x7fdab40fbce0 2024-05-17T21:56:51.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.859+0000 7fdaaaffd700 1 -- 172.21.15.107:0/3669727647 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fda9c00f040 con 0x7fdab40fbce0 2024-05-17T21:56:51.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.859+0000 7fdab8971700 1 -- 172.21.15.107:0/3669727647 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdab40feac0 con 0x7fdab40fbce0 2024-05-17T21:56:51.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.859+0000 7fdaaaffd700 1 -- 172.21.15.107:0/3669727647 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fda9c00db70 con 0x7fdab40fbce0 2024-05-17T21:56:51.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.859+0000 7fdab8971700 1 -- 172.21.15.107:0/3669727647 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdab40ff080 con 0x7fdab40fbce0 2024-05-17T21:56:51.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.860+0000 7fdab8971700 1 -- 172.21.15.107:0/3669727647 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdab4101b20 con 0x7fdab40fbce0 2024-05-17T21:56:51.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.861+0000 7fdaaaffd700 1 -- 172.21.15.107:0/3669727647 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7fda9c00dd10 con 0x7fdab40fbce0 2024-05-17T21:56:51.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.861+0000 7fdaaaffd700 1 --2- 172.21.15.107:0/3669727647 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fdaa0078990 0x7fdaa007ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:56:51.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.861+0000 7fdaaaffd700 1 -- 172.21.15.107:0/3669727647 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fda9c099e20 con 0x7fdab40fbce0 2024-05-17T21:56:51.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.861+0000 7fdab1d9b700 1 --2- 172.21.15.107:0/3669727647 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fdaa0078990 0x7fdaa007ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:56:51.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.862+0000 7fdab1d9b700 1 --2- 172.21.15.107:0/3669727647 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fdaa0078990 0x7fdaa007ae40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fdaa400b3b0 tx=0x7fdaa4009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:56:51.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:51.866+0000 7fdaaaffd700 1 -- 172.21.15.107:0/3669727647 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7fda9c061a90 con 0x7fdab40fbce0 2024-05-17T21:56:52.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:51 smithi138 ceph-mon[81376]: pgmap v78: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 258 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 34/267 objects degraded (12.734%) 2024-05-17T21:56:52.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:51 smithi138 ceph-mon[81376]: from='client.34258 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:56:52.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:51 smithi138 ceph-mon[81376]: from='osd.1 [v2:172.21.15.107:6810/748281068,v1:172.21.15.107:6811/748281068]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-17T21:56:52.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:51 smithi138 ceph-mon[81376]: osdmap e57: 6 total, 5 up, 6 in 2024-05-17T21:56:52.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:51 smithi138 ceph-mon[81376]: from='osd.1 [v2:172.21.15.107:6810/748281068,v1:172.21.15.107:6811/748281068]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-05-17T21:56:52.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:51 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:56:52.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:51 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:56:52.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:51 smithi138 ceph-mon[81376]: from='client.? 172.21.15.107:0/1639933909' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:56:52.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:51 smithi138 ceph-mon[81376]: from='client.? 172.21.15.107:0/1481612217' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-17T21:56:52.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:51 smithi107 ceph-mon[93224]: pgmap v78: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 258 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 34/267 objects degraded (12.734%) 2024-05-17T21:56:52.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:51 smithi107 ceph-mon[93224]: from='client.34258 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:56:52.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:51 smithi107 ceph-mon[93224]: from='osd.1 [v2:172.21.15.107:6810/748281068,v1:172.21.15.107:6811/748281068]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-17T21:56:52.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:51 smithi107 ceph-mon[93224]: osdmap e57: 6 total, 5 up, 6 in 2024-05-17T21:56:52.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:51 smithi107 ceph-mon[93224]: from='osd.1 [v2:172.21.15.107:6810/748281068,v1:172.21.15.107:6811/748281068]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-05-17T21:56:52.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:51 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:56:52.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:51 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:56:52.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:51 smithi107 ceph-mon[93224]: from='client.? 172.21.15.107:0/1639933909' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:56:52.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:51 smithi107 ceph-mon[93224]: from='client.? 172.21.15.107:0/1481612217' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-17T21:56:52.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:52.165+0000 7fdab8971700 1 -- 172.21.15.107:0/3669727647 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fdab404c710 con 0x7fdab40fbce0 2024-05-17T21:56:52.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:52.165+0000 7fdaaaffd700 1 -- 172.21.15.107:0/3669727647 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+1093 (secure 0 0 0) 0x7fda9c061430 con 0x7fdab40fbce0 2024-05-17T21:56:52.158 INFO:teuthology.orchestra.run.smithi107.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data; Degraded data redundancy: 34/267 objects degraded (12.734%), 13 pgs degraded 2024-05-17T21:56:52.158 INFO:teuthology.orchestra.run.smithi107.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-17T21:56:52.158 INFO:teuthology.orchestra.run.smithi107.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-17T21:56:52.158 INFO:teuthology.orchestra.run.smithi107.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 34/267 objects degraded (12.734%), 13 pgs degraded 2024-05-17T21:56:52.158 INFO:teuthology.orchestra.run.smithi107.stdout: pg 1.0 is active+undersized+degraded, acting [3,0] 2024-05-17T21:56:52.158 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.0 is active+undersized+degraded, acting [3,0] 2024-05-17T21:56:52.158 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.1 is active+undersized+degraded, acting [2,0] 2024-05-17T21:56:52.158 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.2 is active+undersized+degraded, acting [5,0] 2024-05-17T21:56:52.158 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.3 is active+undersized+degraded, acting [5,2] 2024-05-17T21:56:52.158 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.4 is active+undersized+degraded, acting [0,4] 2024-05-17T21:56:52.159 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.6 is active+undersized+degraded, acting [3,4] 2024-05-17T21:56:52.159 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.9 is active+undersized+degraded, acting [4,0] 2024-05-17T21:56:52.159 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.a is active+undersized+degraded, acting [4,3] 2024-05-17T21:56:52.159 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.d is active+undersized+degraded, acting [3,2] 2024-05-17T21:56:52.159 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.10 is active+undersized+degraded, acting [2,0] 2024-05-17T21:56:52.159 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.15 is active+undersized+degraded, acting [3,0] 2024-05-17T21:56:52.159 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.1b is active+undersized+degraded, acting [0,5] 2024-05-17T21:56:52.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:52.168+0000 7fdab8971700 1 -- 172.21.15.107:0/3669727647 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fdaa0078990 msgr2=0x7fdaa007ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:56:52.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:52.168+0000 7fdab8971700 1 --2- 172.21.15.107:0/3669727647 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fdaa0078990 0x7fdaa007ae40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fdaa400b3b0 tx=0x7fdaa4009e30 comp rx=0 tx=0).stop 2024-05-17T21:56:52.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:52.168+0000 7fdab8971700 1 -- 172.21.15.107:0/3669727647 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdab40fbce0 msgr2=0x7fdab40ffc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:56:52.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:52.168+0000 7fdab8971700 1 --2- 172.21.15.107:0/3669727647 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdab40fbce0 0x7fdab40ffc20 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fda9c002a50 tx=0x7fda9c0046b0 comp rx=0 tx=0).stop 2024-05-17T21:56:52.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:52.168+0000 7fdab8971700 1 -- 172.21.15.107:0/3669727647 shutdown_connections 2024-05-17T21:56:52.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:52.168+0000 7fdab8971700 1 --2- 172.21.15.107:0/3669727647 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fdaa0078990 0x7fdaa007ae40 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:56:52.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:52.168+0000 7fdab8971700 1 --2- 172.21.15.107:0/3669727647 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdab40fbce0 0x7fdab40ffc20 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:56:52.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:52.168+0000 7fdab8971700 1 --2- 172.21.15.107:0/3669727647 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdab40fcf90 0x7fdab40fe2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:56:52.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:52.168+0000 7fdab8971700 1 -- 172.21.15.107:0/3669727647 >> 172.21.15.107:0/3669727647 conn(0x7fdab40f7480 msgr2=0x7fdab40667f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:56:52.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:52.168+0000 7fdab8971700 1 -- 172.21.15.107:0/3669727647 shutdown_connections 2024-05-17T21:56:52.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:56:52.168+0000 7fdab8971700 1 -- 172.21.15.107:0/3669727647 wait complete. 2024-05-17T21:56:53.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:52 smithi138 ceph-mon[81376]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-05-17T21:56:53.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:52 smithi138 ceph-mon[81376]: osd.1 [v2:172.21.15.107:6810/748281068,v1:172.21.15.107:6811/748281068] boot 2024-05-17T21:56:53.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:52 smithi138 ceph-mon[81376]: osdmap e58: 6 total, 6 up, 6 in 2024-05-17T21:56:53.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:52 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-17T21:56:53.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:52 smithi138 ceph-mon[81376]: from='client.34272 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:56:53.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:52 smithi138 ceph-mon[81376]: from='client.? 172.21.15.107:0/3669727647' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-17T21:56:53.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:52 smithi138 ceph-mon[81376]: osdmap e59: 6 total, 6 up, 6 in 2024-05-17T21:56:53.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:52 smithi107 ceph-mon[93224]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-05-17T21:56:53.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:52 smithi107 ceph-mon[93224]: osd.1 [v2:172.21.15.107:6810/748281068,v1:172.21.15.107:6811/748281068] boot 2024-05-17T21:56:53.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:52 smithi107 ceph-mon[93224]: osdmap e58: 6 total, 6 up, 6 in 2024-05-17T21:56:53.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:52 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-17T21:56:53.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:52 smithi107 ceph-mon[93224]: from='client.34272 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:56:53.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:52 smithi107 ceph-mon[93224]: from='client.? 172.21.15.107:0/3669727647' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-17T21:56:53.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:52 smithi107 ceph-mon[93224]: osdmap e59: 6 total, 6 up, 6 in 2024-05-17T21:56:53.813 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:53 smithi107 ceph-mon[93224]: pgmap v81: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 258 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 2 op/s; 34/267 objects degraded (12.734%) 2024-05-17T21:56:54.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:53 smithi138 ceph-mon[81376]: pgmap v81: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 258 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 2 op/s; 34/267 objects degraded (12.734%) 2024-05-17T21:56:55.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:55 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:56:55.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:55 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:56:55.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:55 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:56:55.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:55 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T21:56:55.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:55 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:56:55.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:55 smithi138 ceph-mon[81376]: pgmap v83: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 258 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 34/267 objects degraded (12.734%) 2024-05-17T21:56:55.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:55 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:56:55.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:55 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:56:55.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:55 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:56:55.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:55 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:56:55.577 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:55 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-05-17T21:56:55.577 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:55 smithi138 ceph-mon[81376]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-05-17T21:56:55.577 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:55 smithi138 ceph-mon[81376]: Upgrade: unsafe to stop osd(s) at this time (12 PGs are or would become offline) 2024-05-17T21:56:55.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:55 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:56:55.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:55 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:56:55.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:55 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:56:55.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:55 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T21:56:55.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:55 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:56:55.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:55 smithi107 ceph-mon[93224]: pgmap v83: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 258 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 34/267 objects degraded (12.734%) 2024-05-17T21:56:55.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:55 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:56:55.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:55 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:56:55.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:55 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:56:55.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:55 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:56:55.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:55 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-05-17T21:56:55.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:55 smithi107 ceph-mon[93224]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-05-17T21:56:55.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:55 smithi107 ceph-mon[93224]: Upgrade: unsafe to stop osd(s) at this time (12 PGs are or would become offline) 2024-05-17T21:56:56.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:56 smithi138 ceph-mon[81376]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 34/267 objects degraded (12.734%), 13 pgs degraded) 2024-05-17T21:56:56.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:56 smithi107 ceph-mon[93224]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 34/267 objects degraded (12.734%), 13 pgs degraded) 2024-05-17T21:56:57.753 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:57 smithi107 ceph-mon[93224]: pgmap v84: 65 pgs: 65 active+clean; 258 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T21:56:57.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:57 smithi138 ceph-mon[81376]: pgmap v84: 65 pgs: 65 active+clean; 258 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T21:56:59.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:56:59 smithi138 ceph-mon[81376]: pgmap v85: 65 pgs: 65 active+clean; 258 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 667 B/s rd, 1 op/s 2024-05-17T21:56:59.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:56:59 smithi107 ceph-mon[93224]: pgmap v85: 65 pgs: 65 active+clean; 258 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 667 B/s rd, 1 op/s 2024-05-17T21:57:01.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:01 smithi138 ceph-mon[81376]: pgmap v86: 65 pgs: 65 active+clean; 258 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T21:57:01.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:01 smithi107 ceph-mon[93224]: pgmap v86: 65 pgs: 65 active+clean; 258 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T21:57:03.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:03 smithi138 ceph-mon[81376]: pgmap v87: 65 pgs: 65 active+clean; 258 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s 2024-05-17T21:57:03.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:03 smithi107 ceph-mon[93224]: pgmap v87: 65 pgs: 65 active+clean; 258 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s 2024-05-17T21:57:05.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:05 smithi138 ceph-mon[81376]: pgmap v88: 65 pgs: 65 active+clean; 258 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 877 B/s rd, 1 op/s 2024-05-17T21:57:05.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:05 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:57:05.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:05 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:57:05.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:05 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:57:05.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:05 smithi107 ceph-mon[93224]: pgmap v88: 65 pgs: 65 active+clean; 258 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 877 B/s rd, 1 op/s 2024-05-17T21:57:05.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:05 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:57:05.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:05 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:57:05.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:05 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:57:07.806 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:07 smithi107 ceph-mon[93224]: pgmap v89: 65 pgs: 65 active+clean; 258 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T21:57:07.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:07 smithi138 ceph-mon[81376]: pgmap v89: 65 pgs: 65 active+clean; 258 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T21:57:09.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:09 smithi138 ceph-mon[81376]: pgmap v90: 65 pgs: 65 active+clean; 258 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T21:57:09.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:09 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-05-17T21:57:09.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:09 smithi107 ceph-mon[93224]: pgmap v90: 65 pgs: 65 active+clean; 258 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T21:57:09.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:09 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-05-17T21:57:10.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:10 smithi107 ceph-mon[93224]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-05-17T21:57:10.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:10 smithi107 ceph-mon[93224]: Upgrade: osd.2 is safe to restart 2024-05-17T21:57:10.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:10 smithi138 ceph-mon[81376]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-05-17T21:57:10.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:10 smithi138 ceph-mon[81376]: Upgrade: osd.2 is safe to restart 2024-05-17T21:57:12.249 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:12 smithi107 ceph-mon[93224]: pgmap v91: 65 pgs: 65 active+clean; 258 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T21:57:12.250 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:12 smithi107 ceph-mon[93224]: Upgrade: Updating osd.2 2024-05-17T21:57:12.250 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:12 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:57:12.250 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:12 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-17T21:57:12.250 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:12 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:57:12.250 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:12 smithi107 ceph-mon[93224]: Deploying daemon osd.2 on smithi107 2024-05-17T21:57:12.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:12 smithi138 ceph-mon[81376]: pgmap v91: 65 pgs: 65 active+clean; 258 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T21:57:12.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:12 smithi138 ceph-mon[81376]: Upgrade: Updating osd.2 2024-05-17T21:57:12.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:12 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:57:12.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:12 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-17T21:57:12.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:12 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:57:12.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:12 smithi138 ceph-mon[81376]: Deploying daemon osd.2 on smithi107 2024-05-17T21:57:14.252 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:14 smithi107 ceph-mon[93224]: pgmap v92: 65 pgs: 65 active+clean; 258 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T21:57:14.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:14 smithi138 ceph-mon[81376]: pgmap v92: 65 pgs: 65 active+clean; 258 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T21:57:14.602 INFO:journalctl@ceph.osd.2.smithi107.stdout:May 17 21:57:14 smithi107 systemd[1]: Stopping Ceph osd.2 for 3722731a-1496-11ef-bc9a-c7b262605968... 2024-05-17T21:57:14.602 INFO:journalctl@ceph.osd.2.smithi107.stdout:May 17 21:57:14 smithi107 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-2[61810]: 2024-05-17T21:57:14.496+0000 7f81a9bc0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-17T21:57:14.602 INFO:journalctl@ceph.osd.2.smithi107.stdout:May 17 21:57:14 smithi107 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-2[61810]: 2024-05-17T21:57:14.496+0000 7f81a9bc0700 -1 osd.2 59 *** Got signal Terminated *** 2024-05-17T21:57:14.602 INFO:journalctl@ceph.osd.2.smithi107.stdout:May 17 21:57:14 smithi107 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-2[61810]: 2024-05-17T21:57:14.496+0000 7f81a9bc0700 -1 osd.2 59 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-17T21:57:15.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:15 smithi138 ceph-mon[81376]: osd.2 marked itself down and dead 2024-05-17T21:57:15.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:15 smithi107 ceph-mon[93224]: osd.2 marked itself down and dead 2024-05-17T21:57:15.805 INFO:journalctl@ceph.osd.2.smithi107.stdout:May 17 21:57:15 smithi107 podman[112250]: 2024-05-17 21:57:15.527748631 +0000 UTC m=+1.118613985 container died e044d7b7808804263f6bcd1d9c1750635b6abfda015de740e40b1ac9a18d0c75 (image=quay.ceph.io/ceph-ci/ceph@sha256:c200327b094672e93680375e82d2de1a08f5881ef304978f791cce179f847d60, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=quincy-9903f77, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-17T21:57:16.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:16 smithi107 ceph-mon[93224]: pgmap v93: 65 pgs: 65 active+clean; 258 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T21:57:16.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:16 smithi107 ceph-mon[93224]: Health check failed: 1 osds down (OSD_DOWN) 2024-05-17T21:57:16.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:16 smithi107 ceph-mon[93224]: osdmap e60: 6 total, 5 up, 6 in 2024-05-17T21:57:16.102 INFO:journalctl@ceph.osd.2.smithi107.stdout:May 17 21:57:15 smithi107 podman[112250]: 2024-05-17 21:57:15.912372206 +0000 UTC m=+1.503237554 container cleanup e044d7b7808804263f6bcd1d9c1750635b6abfda015de740e40b1ac9a18d0c75 (image=quay.ceph.io/ceph-ci/ceph@sha256:c200327b094672e93680375e82d2de1a08f5881ef304978f791cce179f847d60, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-2, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=quincy-9903f77, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-05-17T21:57:16.102 INFO:journalctl@ceph.osd.2.smithi107.stdout:May 17 21:57:15 smithi107 bash[112250]: ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-2 2024-05-17T21:57:16.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:16 smithi138 ceph-mon[81376]: pgmap v93: 65 pgs: 65 active+clean; 258 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T21:57:16.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:16 smithi138 ceph-mon[81376]: Health check failed: 1 osds down (OSD_DOWN) 2024-05-17T21:57:16.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:16 smithi138 ceph-mon[81376]: osdmap e60: 6 total, 5 up, 6 in 2024-05-17T21:57:16.489 INFO:journalctl@ceph.osd.2.smithi107.stdout:May 17 21:57:16 smithi107 podman[112268]: 2024-05-17 21:57:16.164113445 +0000 UTC m=+0.631502656 container remove e044d7b7808804263f6bcd1d9c1750635b6abfda015de740e40b1ac9a18d0c75 (image=quay.ceph.io/ceph-ci/ceph@sha256:c200327b094672e93680375e82d2de1a08f5881ef304978f791cce179f847d60, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=quincy-9903f77, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0) 2024-05-17T21:57:16.742 INFO:journalctl@ceph.osd.2.smithi107.stdout:May 17 21:57:16 smithi107 podman[112360]: 2024-05-17 21:57:16.402038511 +0000 UTC m=+0.018055450 image pull 8880f7613d0283efa7002505fdccdbc3e37d3b5da31c57fa22d62262c5dab66c quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b 2024-05-17T21:57:16.742 INFO:journalctl@ceph.osd.2.smithi107.stdout:May 17 21:57:16 smithi107 podman[112360]: 2024-05-17 21:57:16.525017201 +0000 UTC m=+0.141034155 container create 60c7a218273b66c995668d8ee788811f197045f7e6feaa6213bbc1da0385bb1a (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-2-deactivate, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240514) 2024-05-17T21:57:16.742 INFO:journalctl@ceph.osd.2.smithi107.stdout:May 17 21:57:16 smithi107 podman[112360]: 2024-05-17 21:57:16.751051278 +0000 UTC m=+0.367068210 container init 60c7a218273b66c995668d8ee788811f197045f7e6feaa6213bbc1da0385bb1a (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-17T21:57:17.018 INFO:journalctl@ceph.osd.2.smithi107.stdout:May 17 21:57:16 smithi107 podman[112360]: 2024-05-17 21:57:16.755715004 +0000 UTC m=+0.371731939 container start 60c7a218273b66c995668d8ee788811f197045f7e6feaa6213bbc1da0385bb1a (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-2-deactivate, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, CEPH_POINT_RELEASE=) 2024-05-17T21:57:17.018 INFO:journalctl@ceph.osd.2.smithi107.stdout:May 17 21:57:16 smithi107 podman[112360]: 2024-05-17 21:57:16.859583486 +0000 UTC m=+0.475600432 container attach 60c7a218273b66c995668d8ee788811f197045f7e6feaa6213bbc1da0385bb1a (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-2-deactivate, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-05-17T21:57:17.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:17 smithi138 ceph-mon[81376]: osdmap e61: 6 total, 5 up, 6 in 2024-05-17T21:57:17.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:17 smithi107 ceph-mon[93224]: osdmap e61: 6 total, 5 up, 6 in 2024-05-17T21:57:17.351 INFO:journalctl@ceph.osd.2.smithi107.stdout:May 17 21:57:17 smithi107 podman[112360]: 2024-05-17 21:57:17.051109481 +0000 UTC m=+0.667126415 container died 60c7a218273b66c995668d8ee788811f197045f7e6feaa6213bbc1da0385bb1a (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-17T21:57:17.623 INFO:journalctl@ceph.osd.2.smithi107.stdout:May 17 21:57:17 smithi107 podman[112385]: 2024-05-17 21:57:17.596624589 +0000 UTC m=+0.539482942 container remove 60c7a218273b66c995668d8ee788811f197045f7e6feaa6213bbc1da0385bb1a (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240514, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-17T21:57:17.623 INFO:journalctl@ceph.osd.2.smithi107.stdout:May 17 21:57:17 smithi107 systemd[1]: ceph-3722731a-1496-11ef-bc9a-c7b262605968@osd.2.service: Deactivated successfully. 2024-05-17T21:57:17.623 INFO:journalctl@ceph.osd.2.smithi107.stdout:May 17 21:57:17 smithi107 systemd[1]: Stopped Ceph osd.2 for 3722731a-1496-11ef-bc9a-c7b262605968. 2024-05-17T21:57:17.623 INFO:journalctl@ceph.osd.2.smithi107.stdout:May 17 21:57:17 smithi107 systemd[1]: ceph-3722731a-1496-11ef-bc9a-c7b262605968@osd.2.service: Consumed 44.102s CPU time. 2024-05-17T21:57:18.020 INFO:journalctl@ceph.osd.2.smithi107.stdout:May 17 21:57:17 smithi107 systemd[1]: Starting Ceph osd.2 for 3722731a-1496-11ef-bc9a-c7b262605968... 2024-05-17T21:57:18.324 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:18 smithi107 ceph-mon[93224]: pgmap v96: 65 pgs: 19 peering, 3 stale+active+clean, 43 active+clean; 258 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-05-17T21:57:18.324 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:18 smithi107 ceph-mon[93224]: Health check failed: Reduced data availability: 6 pgs peering (PG_AVAILABILITY) 2024-05-17T21:57:18.324 INFO:journalctl@ceph.osd.2.smithi107.stdout:May 17 21:57:18 smithi107 podman[112482]: 2024-05-17 21:57:18.031008238 +0000 UTC m=+0.105669104 container create de33f1b05c971aea376e18be8781fc2fa40699865b067caceb420082d0e7b01d (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-2-activate, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, ceph=True) 2024-05-17T21:57:18.324 INFO:journalctl@ceph.osd.2.smithi107.stdout:May 17 21:57:18 smithi107 podman[112482]: 2024-05-17 21:57:17.942545637 +0000 UTC m=+0.017206512 image pull 8880f7613d0283efa7002505fdccdbc3e37d3b5da31c57fa22d62262c5dab66c quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b 2024-05-17T21:57:18.324 INFO:journalctl@ceph.osd.2.smithi107.stdout:May 17 21:57:18 smithi107 podman[112482]: 2024-05-17 21:57:18.2500754 +0000 UTC m=+0.324736272 container init de33f1b05c971aea376e18be8781fc2fa40699865b067caceb420082d0e7b01d (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-2-activate, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-05-17T21:57:18.325 INFO:journalctl@ceph.osd.2.smithi107.stdout:May 17 21:57:18 smithi107 podman[112482]: 2024-05-17 21:57:18.253740314 +0000 UTC m=+0.328401184 container start de33f1b05c971aea376e18be8781fc2fa40699865b067caceb420082d0e7b01d (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-2-activate, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True) 2024-05-17T21:57:18.325 INFO:journalctl@ceph.osd.2.smithi107.stdout:May 17 21:57:18 smithi107 podman[112482]: 2024-05-17 21:57:18.333624457 +0000 UTC m=+0.408285330 container attach de33f1b05c971aea376e18be8781fc2fa40699865b067caceb420082d0e7b01d (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-2-activate, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-05-17T21:57:18.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:18 smithi138 ceph-mon[81376]: pgmap v96: 65 pgs: 19 peering, 3 stale+active+clean, 43 active+clean; 258 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-05-17T21:57:18.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:18 smithi138 ceph-mon[81376]: Health check failed: Reduced data availability: 6 pgs peering (PG_AVAILABILITY) 2024-05-17T21:57:19.102 INFO:journalctl@ceph.osd.2.smithi107.stdout:May 17 21:57:18 smithi107 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-2-activate[112497]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-05-17T21:57:19.102 INFO:journalctl@ceph.osd.2.smithi107.stdout:May 17 21:57:18 smithi107 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-2-activate[112497]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-05-17T21:57:19.102 INFO:journalctl@ceph.osd.2.smithi107.stdout:May 17 21:57:18 smithi107 bash[112482]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-05-17T21:57:19.102 INFO:journalctl@ceph.osd.2.smithi107.stdout:May 17 21:57:18 smithi107 bash[112482]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-05-17T21:57:19.102 INFO:journalctl@ceph.osd.2.smithi107.stdout:May 17 21:57:18 smithi107 bash[112482]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-05-17T21:57:19.102 INFO:journalctl@ceph.osd.2.smithi107.stdout:May 17 21:57:18 smithi107 bash[112482]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-05-17T21:57:19.103 INFO:journalctl@ceph.osd.2.smithi107.stdout:May 17 21:57:18 smithi107 bash[112482]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-05-17T21:57:19.103 INFO:journalctl@ceph.osd.2.smithi107.stdout:May 17 21:57:18 smithi107 bash[112482]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-05-17T21:57:19.103 INFO:journalctl@ceph.osd.2.smithi107.stdout:May 17 21:57:18 smithi107 bash[112482]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-05-17T21:57:19.103 INFO:journalctl@ceph.osd.2.smithi107.stdout:May 17 21:57:18 smithi107 bash[112482]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-05-17T21:57:19.103 INFO:journalctl@ceph.osd.2.smithi107.stdout:May 17 21:57:18 smithi107 bash[112482]: --> ceph-volume raw activate successful for osd ID: 2 2024-05-17T21:57:19.103 INFO:journalctl@ceph.osd.2.smithi107.stdout:May 17 21:57:18 smithi107 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-2-activate[112497]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-05-17T21:57:19.103 INFO:journalctl@ceph.osd.2.smithi107.stdout:May 17 21:57:18 smithi107 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-2-activate[112497]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-05-17T21:57:19.103 INFO:journalctl@ceph.osd.2.smithi107.stdout:May 17 21:57:18 smithi107 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-2-activate[112497]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-05-17T21:57:19.103 INFO:journalctl@ceph.osd.2.smithi107.stdout:May 17 21:57:18 smithi107 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-2-activate[112497]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-05-17T21:57:19.103 INFO:journalctl@ceph.osd.2.smithi107.stdout:May 17 21:57:18 smithi107 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-2-activate[112497]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-05-17T21:57:19.103 INFO:journalctl@ceph.osd.2.smithi107.stdout:May 17 21:57:18 smithi107 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-2-activate[112497]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-05-17T21:57:19.103 INFO:journalctl@ceph.osd.2.smithi107.stdout:May 17 21:57:18 smithi107 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-2-activate[112497]: --> ceph-volume raw activate successful for osd ID: 2 2024-05-17T21:57:19.103 INFO:journalctl@ceph.osd.2.smithi107.stdout:May 17 21:57:18 smithi107 podman[112482]: 2024-05-17 21:57:18.867602357 +0000 UTC m=+0.942263222 container died de33f1b05c971aea376e18be8781fc2fa40699865b067caceb420082d0e7b01d (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-2-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-17T21:57:19.765 INFO:journalctl@ceph.osd.2.smithi107.stdout:May 17 21:57:19 smithi107 podman[112625]: 2024-05-17 21:57:19.556707916 +0000 UTC m=+0.683414483 container remove de33f1b05c971aea376e18be8781fc2fa40699865b067caceb420082d0e7b01d (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-2-activate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-17T21:57:20.023 INFO:journalctl@ceph.osd.2.smithi107.stdout:May 17 21:57:19 smithi107 podman[112684]: 2024-05-17 21:57:19.77439031 +0000 UTC m=+0.114540419 container create add7015647af5cff02edc5e90d98acaa4c2cc25e1fe4adbda60f6af5e6d9de76 (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-2, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-05-17T21:57:20.023 INFO:journalctl@ceph.osd.2.smithi107.stdout:May 17 21:57:19 smithi107 podman[112684]: 2024-05-17 21:57:19.677776503 +0000 UTC m=+0.017926623 image pull 8880f7613d0283efa7002505fdccdbc3e37d3b5da31c57fa22d62262c5dab66c quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b 2024-05-17T21:57:20.023 INFO:journalctl@ceph.osd.2.smithi107.stdout:May 17 21:57:19 smithi107 podman[112684]: 2024-05-17 21:57:19.984783677 +0000 UTC m=+0.324933786 container init add7015647af5cff02edc5e90d98acaa4c2cc25e1fe4adbda60f6af5e6d9de76 (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-17T21:57:20.023 INFO:journalctl@ceph.osd.2.smithi107.stdout:May 17 21:57:19 smithi107 podman[112684]: 2024-05-17 21:57:19.987960071 +0000 UTC m=+0.328110181 container start add7015647af5cff02edc5e90d98acaa4c2cc25e1fe4adbda60f6af5e6d9de76 (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-2, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, ceph=True) 2024-05-17T21:57:20.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:20 smithi138 ceph-mon[81376]: pgmap v97: 65 pgs: 19 peering, 3 stale+active+clean, 43 active+clean; 258 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 511 B/s rd, 0 op/s 2024-05-17T21:57:20.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:20 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:57:20.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:20 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:57:20.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:20 smithi107 ceph-mon[93224]: pgmap v97: 65 pgs: 19 peering, 3 stale+active+clean, 43 active+clean; 258 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 511 B/s rd, 0 op/s 2024-05-17T21:57:20.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:20 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:57:20.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:20 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:57:20.351 INFO:journalctl@ceph.osd.2.smithi107.stdout:May 17 21:57:20 smithi107 bash[112684]: add7015647af5cff02edc5e90d98acaa4c2cc25e1fe4adbda60f6af5e6d9de76 2024-05-17T21:57:20.352 INFO:journalctl@ceph.osd.2.smithi107.stdout:May 17 21:57:20 smithi107 systemd[1]: Started Ceph osd.2 for 3722731a-1496-11ef-bc9a-c7b262605968. 2024-05-17T21:57:20.851 INFO:journalctl@ceph.osd.2.smithi107.stdout:May 17 21:57:20 smithi107 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-2[112698]: 2024-05-17T21:57:20.537+0000 7fc7133007c0 -1 Falling back to public interface 2024-05-17T21:57:21.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:21 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:57:21.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:21 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:57:21.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:21 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:57:21.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:21 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:57:21.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:21 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:57:21.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:21 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:57:22.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.282+0000 7f102c7fb700 1 -- 172.21.15.107:0/3387991367 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f10240fce80 msgr2=0x7f10240fd290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:22.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.282+0000 7f102c7fb700 1 --2- 172.21.15.107:0/3387991367 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f10240fce80 0x7f10240fd290 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1020004940 tx=0x7f1020009e30 comp rx=0 tx=0).stop 2024-05-17T21:57:22.273 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:22 smithi107 ceph-mon[93224]: pgmap v98: 65 pgs: 9 active+undersized+degraded, 7 peering, 11 active+undersized, 38 active+clean; 258 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 20/267 objects degraded (7.491%) 2024-05-17T21:57:22.273 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:22 smithi107 ceph-mon[93224]: Health check failed: Degraded data redundancy: 20/267 objects degraded (7.491%), 9 pgs degraded (PG_DEGRADED) 2024-05-17T21:57:22.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.282+0000 7f102c7fb700 1 -- 172.21.15.107:0/3387991367 shutdown_connections 2024-05-17T21:57:22.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.282+0000 7f102c7fb700 1 --2- 172.21.15.107:0/3387991367 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10240fdff0 0x7f10240fe460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:22.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.282+0000 7f102c7fb700 1 --2- 172.21.15.107:0/3387991367 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f10240fce80 0x7f10240fd290 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:22.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.282+0000 7f102c7fb700 1 -- 172.21.15.107:0/3387991367 >> 172.21.15.107:0/3387991367 conn(0x7f10240f8540 msgr2=0x7f10240fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:22.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.282+0000 7f102c7fb700 1 -- 172.21.15.107:0/3387991367 shutdown_connections 2024-05-17T21:57:22.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.282+0000 7f102c7fb700 1 -- 172.21.15.107:0/3387991367 wait complete. 2024-05-17T21:57:22.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.283+0000 7f102c7fb700 1 Processor -- start 2024-05-17T21:57:22.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.283+0000 7f102c7fb700 1 -- start start 2024-05-17T21:57:22.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.283+0000 7f102c7fb700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f10240fce80 0x7f102410da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:22.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.283+0000 7f102c7fb700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10240fdff0 0x7f102410df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:22.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.283+0000 7f102c7fb700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f102410e550 con 0x7f10240fce80 2024-05-17T21:57:22.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.283+0000 7f102c7fb700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f102410eda0 con 0x7f10240fdff0 2024-05-17T21:57:22.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.283+0000 7f102a597700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f10240fce80 0x7f102410da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:22.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.283+0000 7f102a597700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f10240fce80 0x7f102410da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45454/0 (socket says 172.21.15.107:45454) 2024-05-17T21:57:22.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.283+0000 7f102a597700 1 -- 172.21.15.107:0/2553809252 learned_addr learned my addr 172.21.15.107:0/2553809252 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:57:22.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.283+0000 7f1029d96700 1 --2- 172.21.15.107:0/2553809252 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10240fdff0 0x7f102410df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:22.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.283+0000 7f102a597700 1 -- 172.21.15.107:0/2553809252 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10240fdff0 msgr2=0x7f102410df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:22.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.283+0000 7f102a597700 1 --2- 172.21.15.107:0/2553809252 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10240fdff0 0x7f102410df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:22.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.283+0000 7f102a597700 1 -- 172.21.15.107:0/2553809252 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1020005040 con 0x7f10240fce80 2024-05-17T21:57:22.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.283+0000 7f102a597700 1 --2- 172.21.15.107:0/2553809252 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f10240fce80 0x7f102410da10 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f102000b3b0 tx=0x7f1020009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:22.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.284+0000 7f101affd700 1 -- 172.21.15.107:0/2553809252 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f102000e040 con 0x7f10240fce80 2024-05-17T21:57:22.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.284+0000 7f102c7fb700 1 -- 172.21.15.107:0/2553809252 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f102410f020 con 0x7f10240fce80 2024-05-17T21:57:22.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.284+0000 7f101affd700 1 -- 172.21.15.107:0/2553809252 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f102000bbf0 con 0x7f10240fce80 2024-05-17T21:57:22.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.284+0000 7f101affd700 1 -- 172.21.15.107:0/2553809252 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f102001c630 con 0x7f10240fce80 2024-05-17T21:57:22.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.284+0000 7f102c7fb700 1 -- 172.21.15.107:0/2553809252 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f102410f560 con 0x7f10240fce80 2024-05-17T21:57:22.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.285+0000 7f102c7fb700 1 -- 172.21.15.107:0/2553809252 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f102405f5c0 con 0x7f10240fce80 2024-05-17T21:57:22.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.286+0000 7f101affd700 1 -- 172.21.15.107:0/2553809252 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f102000bd60 con 0x7f10240fce80 2024-05-17T21:57:22.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.287+0000 7f101affd700 1 --2- 172.21.15.107:0/2553809252 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f1010078990 0x7f101007ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:22.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.287+0000 7f101affd700 1 -- 172.21.15.107:0/2553809252 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f102009ba20 con 0x7f10240fce80 2024-05-17T21:57:22.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.287+0000 7f1029d96700 1 --2- 172.21.15.107:0/2553809252 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f1010078990 0x7f101007ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:22.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.287+0000 7f1029d96700 1 --2- 172.21.15.107:0/2553809252 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f1010078990 0x7f101007ae40 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1014004f40 tx=0x7f101400b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:22.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.291+0000 7f101affd700 1 -- 172.21.15.107:0/2553809252 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f1020063730 con 0x7f10240fce80 2024-05-17T21:57:22.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.522+0000 7f102c7fb700 1 -- 172.21.15.107:0/2553809252 --> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f102410f9a0 con 0x7f1010078990 2024-05-17T21:57:22.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.524+0000 7f101affd700 1 -- 172.21.15.107:0/2553809252 <== mgr.34100 v2:172.21.15.107:6800/3205268226 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f102410f9a0 con 0x7f1010078990 2024-05-17T21:57:22.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.526+0000 7f102c7fb700 1 -- 172.21.15.107:0/2553809252 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f1010078990 msgr2=0x7f101007ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:22.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.526+0000 7f102c7fb700 1 --2- 172.21.15.107:0/2553809252 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f1010078990 0x7f101007ae40 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1014004f40 tx=0x7f101400b040 comp rx=0 tx=0).stop 2024-05-17T21:57:22.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.526+0000 7f102c7fb700 1 -- 172.21.15.107:0/2553809252 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f10240fce80 msgr2=0x7f102410da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:22.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.526+0000 7f102c7fb700 1 --2- 172.21.15.107:0/2553809252 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f10240fce80 0x7f102410da10 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f102000b3b0 tx=0x7f1020009e30 comp rx=0 tx=0).stop 2024-05-17T21:57:22.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.526+0000 7f102c7fb700 1 -- 172.21.15.107:0/2553809252 shutdown_connections 2024-05-17T21:57:22.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.526+0000 7f102c7fb700 1 --2- 172.21.15.107:0/2553809252 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f1010078990 0x7f101007ae40 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:22.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.526+0000 7f102c7fb700 1 --2- 172.21.15.107:0/2553809252 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f10240fce80 0x7f102410da10 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:22.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.526+0000 7f102c7fb700 1 --2- 172.21.15.107:0/2553809252 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10240fdff0 0x7f102410df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:22.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.526+0000 7f102c7fb700 1 -- 172.21.15.107:0/2553809252 >> 172.21.15.107:0/2553809252 conn(0x7f10240f8540 msgr2=0x7f10240f9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:22.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.526+0000 7f102c7fb700 1 -- 172.21.15.107:0/2553809252 shutdown_connections 2024-05-17T21:57:22.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.526+0000 7f102c7fb700 1 -- 172.21.15.107:0/2553809252 wait complete. 2024-05-17T21:57:22.527 INFO:teuthology.orchestra.run.smithi107.stdout:true 2024-05-17T21:57:22.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:22 smithi138 ceph-mon[81376]: pgmap v98: 65 pgs: 9 active+undersized+degraded, 7 peering, 11 active+undersized, 38 active+clean; 258 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 20/267 objects degraded (7.491%) 2024-05-17T21:57:22.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:22 smithi138 ceph-mon[81376]: Health check failed: Degraded data redundancy: 20/267 objects degraded (7.491%), 9 pgs degraded (PG_DEGRADED) 2024-05-17T21:57:22.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.632+0000 7f04d8921700 1 -- 172.21.15.107:0/3244665800 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f04d40fcf60 msgr2=0x7f04d40fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:22.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.632+0000 7f04d8921700 1 --2- 172.21.15.107:0/3244665800 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f04d40fcf60 0x7f04d40fd3d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f04bc004940 tx=0x7f04bc009e30 comp rx=0 tx=0).stop 2024-05-17T21:57:22.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.632+0000 7f04d8921700 1 -- 172.21.15.107:0/3244665800 shutdown_connections 2024-05-17T21:57:22.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.632+0000 7f04d8921700 1 --2- 172.21.15.107:0/3244665800 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f04d40fcf60 0x7f04d40fd3d0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:22.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.632+0000 7f04d8921700 1 --2- 172.21.15.107:0/3244665800 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04d40fbcb0 0x7f04d40fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:22.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.632+0000 7f04d8921700 1 -- 172.21.15.107:0/3244665800 >> 172.21.15.107:0/3244665800 conn(0x7f04d40f7430 msgr2=0x7f04d40f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:22.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.632+0000 7f04d8921700 1 -- 172.21.15.107:0/3244665800 shutdown_connections 2024-05-17T21:57:22.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.632+0000 7f04d8921700 1 -- 172.21.15.107:0/3244665800 wait complete. 2024-05-17T21:57:22.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.633+0000 7f04d8921700 1 Processor -- start 2024-05-17T21:57:22.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.633+0000 7f04d8921700 1 -- start start 2024-05-17T21:57:22.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.633+0000 7f04d8921700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04d40fbcb0 0x7f04d4110750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:22.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.633+0000 7f04d8921700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f04d40fcf60 0x7f04d4112ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:22.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.633+0000 7f04d8921700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04d4066330 con 0x7f04d40fcf60 2024-05-17T21:57:22.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.633+0000 7f04d8921700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04d4066470 con 0x7f04d40fbcb0 2024-05-17T21:57:22.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.633+0000 7f04d1d9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f04d40fcf60 0x7f04d4112ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:22.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.633+0000 7f04d1d9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f04d40fcf60 0x7f04d4112ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45470/0 (socket says 172.21.15.107:45470) 2024-05-17T21:57:22.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.633+0000 7f04d1d9b700 1 -- 172.21.15.107:0/1186602273 learned_addr learned my addr 172.21.15.107:0/1186602273 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:57:22.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.633+0000 7f04d259c700 1 --2- 172.21.15.107:0/1186602273 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04d40fbcb0 0x7f04d4110750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:22.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.633+0000 7f04d1d9b700 1 -- 172.21.15.107:0/1186602273 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04d40fbcb0 msgr2=0x7f04d4110750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:22.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.633+0000 7f04d1d9b700 1 --2- 172.21.15.107:0/1186602273 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04d40fbcb0 0x7f04d4110750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:22.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.633+0000 7f04d1d9b700 1 -- 172.21.15.107:0/1186602273 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04bc005040 con 0x7f04d40fcf60 2024-05-17T21:57:22.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.633+0000 7f04d1d9b700 1 --2- 172.21.15.107:0/1186602273 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f04d40fcf60 0x7f04d4112ca0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f04bc00b3b0 tx=0x7f04bc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:22.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.634+0000 7f04caffd700 1 -- 172.21.15.107:0/1186602273 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f04bc00e040 con 0x7f04d40fcf60 2024-05-17T21:57:22.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.634+0000 7f04caffd700 1 -- 172.21.15.107:0/1186602273 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f04bc00bcc0 con 0x7f04d40fcf60 2024-05-17T21:57:22.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.634+0000 7f04d8921700 1 -- 172.21.15.107:0/1186602273 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04d41131e0 con 0x7f04d40fcf60 2024-05-17T21:57:22.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.634+0000 7f04caffd700 1 -- 172.21.15.107:0/1186602273 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f04bc01c630 con 0x7f04d40fcf60 2024-05-17T21:57:22.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.634+0000 7f04d8921700 1 -- 172.21.15.107:0/1186602273 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04d41137a0 con 0x7f04d40fcf60 2024-05-17T21:57:22.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.635+0000 7f04d8921700 1 -- 172.21.15.107:0/1186602273 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04d405f5c0 con 0x7f04d40fcf60 2024-05-17T21:57:22.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.635+0000 7f04caffd700 1 -- 172.21.15.107:0/1186602273 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f04bc013980 con 0x7f04d40fcf60 2024-05-17T21:57:22.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.636+0000 7f04caffd700 1 --2- 172.21.15.107:0/1186602273 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f04c0078990 0x7f04c007ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:22.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.636+0000 7f04caffd700 1 -- 172.21.15.107:0/1186602273 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f04bc09b440 con 0x7f04d40fcf60 2024-05-17T21:57:22.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.636+0000 7f04d259c700 1 --2- 172.21.15.107:0/1186602273 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f04c0078990 0x7f04c007ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:22.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.636+0000 7f04d259c700 1 --2- 172.21.15.107:0/1186602273 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f04c0078990 0x7f04c007ae40 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f04cc0041d0 tx=0x7f04cc009040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:22.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.640+0000 7f04caffd700 1 -- 172.21.15.107:0/1186602273 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f04bc0630b0 con 0x7f04d40fcf60 2024-05-17T21:57:22.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.867+0000 7f04d8921700 1 -- 172.21.15.107:0/1186602273 --> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f04d41012d0 con 0x7f04c0078990 2024-05-17T21:57:22.859 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.869+0000 7f04caffd700 1 -- 172.21.15.107:0/1186602273 <== mgr.34100 v2:172.21.15.107:6800/3205268226 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f04d41012d0 con 0x7f04c0078990 2024-05-17T21:57:22.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.871+0000 7f04d8921700 1 -- 172.21.15.107:0/1186602273 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f04c0078990 msgr2=0x7f04c007ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:22.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.871+0000 7f04d8921700 1 --2- 172.21.15.107:0/1186602273 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f04c0078990 0x7f04c007ae40 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f04cc0041d0 tx=0x7f04cc009040 comp rx=0 tx=0).stop 2024-05-17T21:57:22.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.871+0000 7f04d8921700 1 -- 172.21.15.107:0/1186602273 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f04d40fcf60 msgr2=0x7f04d4112ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:22.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.871+0000 7f04d8921700 1 --2- 172.21.15.107:0/1186602273 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f04d40fcf60 0x7f04d4112ca0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f04bc00b3b0 tx=0x7f04bc009e30 comp rx=0 tx=0).stop 2024-05-17T21:57:22.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.871+0000 7f04d8921700 1 -- 172.21.15.107:0/1186602273 shutdown_connections 2024-05-17T21:57:22.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.871+0000 7f04d8921700 1 --2- 172.21.15.107:0/1186602273 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f04c0078990 0x7f04c007ae40 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:22.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.871+0000 7f04d8921700 1 --2- 172.21.15.107:0/1186602273 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04d40fbcb0 0x7f04d4110750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:22.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.871+0000 7f04d8921700 1 --2- 172.21.15.107:0/1186602273 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f04d40fcf60 0x7f04d4112ca0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:22.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.871+0000 7f04d8921700 1 -- 172.21.15.107:0/1186602273 >> 172.21.15.107:0/1186602273 conn(0x7f04d40f7430 msgr2=0x7f04d40f8b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:22.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.871+0000 7f04d8921700 1 -- 172.21.15.107:0/1186602273 shutdown_connections 2024-05-17T21:57:22.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.871+0000 7f04d8921700 1 -- 172.21.15.107:0/1186602273 wait complete. 2024-05-17T21:57:22.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.982+0000 7f54991d0700 1 -- 172.21.15.107:0/2766069521 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f54940fcf10 msgr2=0x7f54940fd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:22.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.982+0000 7f54991d0700 1 --2- 172.21.15.107:0/2766069521 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f54940fcf10 0x7f54940fd320 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5484004940 tx=0x7f5484009e30 comp rx=0 tx=0).stop 2024-05-17T21:57:22.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.982+0000 7f54991d0700 1 -- 172.21.15.107:0/2766069521 shutdown_connections 2024-05-17T21:57:22.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.982+0000 7f54991d0700 1 --2- 172.21.15.107:0/2766069521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f54940fe080 0x7f54940fe4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:22.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.982+0000 7f54991d0700 1 --2- 172.21.15.107:0/2766069521 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f54940fcf10 0x7f54940fd320 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:22.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.982+0000 7f54991d0700 1 -- 172.21.15.107:0/2766069521 >> 172.21.15.107:0/2766069521 conn(0x7f54940f8560 msgr2=0x7f54940fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:22.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.982+0000 7f54991d0700 1 -- 172.21.15.107:0/2766069521 shutdown_connections 2024-05-17T21:57:22.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.982+0000 7f54991d0700 1 -- 172.21.15.107:0/2766069521 wait complete. 2024-05-17T21:57:22.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.983+0000 7f54991d0700 1 Processor -- start 2024-05-17T21:57:22.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.983+0000 7f54991d0700 1 -- start start 2024-05-17T21:57:22.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.983+0000 7f54991d0700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f54940fcf10 0x7f5494071de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:22.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.983+0000 7f54991d0700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f54940fe080 0x7f54940702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:22.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.983+0000 7f54991d0700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5494070860 con 0x7f54940fcf10 2024-05-17T21:57:22.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.983+0000 7f54991d0700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54940709d0 con 0x7f54940fe080 2024-05-17T21:57:22.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.983+0000 7f5492d9d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f54940fcf10 0x7f5494071de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:22.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.984+0000 7f5492d9d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f54940fcf10 0x7f5494071de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45484/0 (socket says 172.21.15.107:45484) 2024-05-17T21:57:22.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.984+0000 7f5492d9d700 1 -- 172.21.15.107:0/4146864576 learned_addr learned my addr 172.21.15.107:0/4146864576 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:57:22.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.984+0000 7f549259c700 1 --2- 172.21.15.107:0/4146864576 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f54940fe080 0x7f54940702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:22.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.984+0000 7f5492d9d700 1 -- 172.21.15.107:0/4146864576 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f54940fe080 msgr2=0x7f54940702b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:22.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.984+0000 7f5492d9d700 1 --2- 172.21.15.107:0/4146864576 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f54940fe080 0x7f54940702b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:22.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.984+0000 7f5492d9d700 1 -- 172.21.15.107:0/4146864576 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5484005040 con 0x7f54940fcf10 2024-05-17T21:57:22.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.984+0000 7f5492d9d700 1 --2- 172.21.15.107:0/4146864576 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f54940fcf10 0x7f5494071de0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f548400b3b0 tx=0x7f5484009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:22.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.984+0000 7f547b7fe700 1 -- 172.21.15.107:0/4146864576 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f548400e040 con 0x7f54940fcf10 2024-05-17T21:57:22.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.984+0000 7f547b7fe700 1 -- 172.21.15.107:0/4146864576 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f548400bbf0 con 0x7f54940fcf10 2024-05-17T21:57:22.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.984+0000 7f54991d0700 1 -- 172.21.15.107:0/4146864576 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5494070c50 con 0x7f54940fcf10 2024-05-17T21:57:22.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.984+0000 7f547b7fe700 1 -- 172.21.15.107:0/4146864576 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f548401c630 con 0x7f54940fcf10 2024-05-17T21:57:22.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.984+0000 7f54991d0700 1 -- 172.21.15.107:0/4146864576 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5494115a10 con 0x7f54940fcf10 2024-05-17T21:57:22.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.985+0000 7f54991d0700 1 -- 172.21.15.107:0/4146864576 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f549405f5c0 con 0x7f54940fcf10 2024-05-17T21:57:22.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.986+0000 7f547b7fe700 1 -- 172.21.15.107:0/4146864576 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f548400bd60 con 0x7f54940fcf10 2024-05-17T21:57:22.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.987+0000 7f547b7fe700 1 --2- 172.21.15.107:0/4146864576 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f547c078990 0x7f547c07ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:22.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.987+0000 7f547b7fe700 1 -- 172.21.15.107:0/4146864576 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f548409b320 con 0x7f54940fcf10 2024-05-17T21:57:22.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.987+0000 7f549259c700 1 --2- 172.21.15.107:0/4146864576 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f547c078990 0x7f547c07ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:22.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.987+0000 7f549259c700 1 --2- 172.21.15.107:0/4146864576 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f547c078990 0x7f547c07ae40 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f54880102e0 tx=0x7f548800b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:22.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:22.991+0000 7f547b7fe700 1 -- 172.21.15.107:0/4146864576 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f5484063010 con 0x7f54940fcf10 2024-05-17T21:57:23.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:23.230+0000 7f54991d0700 1 -- 172.21.15.107:0/4146864576 --> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f54941023a0 con 0x7f547c078990 2024-05-17T21:57:23.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:23.240+0000 7f547b7fe700 1 -- 172.21.15.107:0/4146864576 <== mgr.34100 v2:172.21.15.107:6800/3205268226 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f54941023a0 con 0x7f547c078990 2024-05-17T21:57:23.231 INFO:teuthology.orchestra.run.smithi107.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-17T21:57:23.231 INFO:teuthology.orchestra.run.smithi107.stdout:alertmanager.smithi107 smithi107 *:9093,9094 running (4m) 33s ago 13m 24.3M - 0.25.0 c8568f914cd2 bac2a944ae42 2024-05-17T21:57:23.231 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi107 smithi107 running (101s) 33s ago 13m 6669k - 19.0.0-3794-g931809a9 8880f7613d02 face35de205b 2024-05-17T21:57:23.231 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi138 smithi138 running (95s) 92s ago 12m 6665k - 19.0.0-3794-g931809a9 8880f7613d02 ad59e33e6d43 2024-05-17T21:57:23.231 INFO:teuthology.orchestra.run.smithi107.stdout:grafana.smithi107 smithi107 *:3000 running (4m) 33s ago 13m 84.6M - 9.4.7 954c08fa6188 8c747e777bc5 2024-05-17T21:57:23.232 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.drqody smithi107 running (7m) 33s ago 7m 103M - 17.2.7-952-g9903f776 d15a36853796 0d851bc912f0 2024-05-17T21:57:23.232 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.idmbgj smithi107 running (8m) 33s ago 8m 13.7M - 17.2.7-952-g9903f776 d15a36853796 1348b09ab29e 2024-05-17T21:57:23.232 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi138.gwbtyd smithi138 running (7m) 92s ago 7m 152M - 17.2.7-952-g9903f776 d15a36853796 fd2ba75354c0 2024-05-17T21:57:23.232 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi138.zckrvs smithi138 running (7m) 92s ago 7m 14.1M - 17.2.7-952-g9903f776 d15a36853796 a1cfd2fe4cda 2024-05-17T21:57:23.232 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi107.xhhyiw smithi107 *:8443,9283,8765 running (4m) 33s ago 14m 586M - 19.0.0-3794-g931809a9 8880f7613d02 5acf3e511342 2024-05-17T21:57:23.232 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi138.napnll smithi138 *:8443,9283,8765 running (3m) 92s ago 12m 531M - 19.0.0-3794-g931809a9 8880f7613d02 d627761dbd0d 2024-05-17T21:57:23.232 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi107 smithi107 running (2m) 33s ago 15m 114M 2048M 19.0.0-3794-g931809a9 8880f7613d02 9717f3169f2f 2024-05-17T21:57:23.232 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi138 smithi138 running (2m) 92s ago 12m 99.9M 2048M 19.0.0-3794-g931809a9 8880f7613d02 55aeed62bf4d 2024-05-17T21:57:23.232 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi107 smithi107 *:9100 running (5m) 33s ago 13m 17.4M - 1.5.0 0da6a335fe13 49bec71f5487 2024-05-17T21:57:23.232 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi138 smithi138 *:9100 running (4m) 92s ago 12m 15.9M - 1.5.0 0da6a335fe13 e52824376658 2024-05-17T21:57:23.232 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0 smithi107 running (75s) 33s ago 11m 156M 1557M 19.0.0-3794-g931809a9 8880f7613d02 b319bc93500d 2024-05-17T21:57:23.232 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1 smithi107 running (39s) 33s ago 11m 80.0M 1557M 19.0.0-3794-g931809a9 8880f7613d02 f94b206be6eb 2024-05-17T21:57:23.232 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2 smithi107 starting - - - 1557M 2024-05-17T21:57:23.232 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3 smithi138 running (10m) 92s ago 10m 228M 2581M 17.2.7-952-g9903f776 d15a36853796 8c980cc8af87 2024-05-17T21:57:23.232 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4 smithi138 running (9m) 92s ago 9m 204M 2581M 17.2.7-952-g9903f776 d15a36853796 b306a36a744e 2024-05-17T21:57:23.233 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5 smithi138 running (9m) 92s ago 9m 190M 2581M 17.2.7-952-g9903f776 d15a36853796 7c8d8ed74974 2024-05-17T21:57:23.233 INFO:teuthology.orchestra.run.smithi107.stdout:prometheus.smithi107 smithi107 *:9095 running (3m) 33s ago 12m 58.7M - 2.43.0 a07b618ecd1d bf5f609bbd6e 2024-05-17T21:57:23.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:23.244+0000 7f54991d0700 1 -- 172.21.15.107:0/4146864576 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f547c078990 msgr2=0x7f547c07ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:23.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:23.244+0000 7f54991d0700 1 --2- 172.21.15.107:0/4146864576 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f547c078990 0x7f547c07ae40 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f54880102e0 tx=0x7f548800b040 comp rx=0 tx=0).stop 2024-05-17T21:57:23.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:23.244+0000 7f54991d0700 1 -- 172.21.15.107:0/4146864576 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f54940fcf10 msgr2=0x7f5494071de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:23.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:23.244+0000 7f54991d0700 1 --2- 172.21.15.107:0/4146864576 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f54940fcf10 0x7f5494071de0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f548400b3b0 tx=0x7f5484009e30 comp rx=0 tx=0).stop 2024-05-17T21:57:23.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:23.244+0000 7f54991d0700 1 -- 172.21.15.107:0/4146864576 shutdown_connections 2024-05-17T21:57:23.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:23.244+0000 7f54991d0700 1 --2- 172.21.15.107:0/4146864576 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f547c078990 0x7f547c07ae40 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:23.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:23.244+0000 7f54991d0700 1 --2- 172.21.15.107:0/4146864576 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f54940fcf10 0x7f5494071de0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:23.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:23.244+0000 7f54991d0700 1 --2- 172.21.15.107:0/4146864576 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f54940fe080 0x7f54940702b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:23.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:23.244+0000 7f54991d0700 1 -- 172.21.15.107:0/4146864576 >> 172.21.15.107:0/4146864576 conn(0x7f54940f8560 msgr2=0x7f54940f9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:23.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:23.244+0000 7f54991d0700 1 -- 172.21.15.107:0/4146864576 shutdown_connections 2024-05-17T21:57:23.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:23.244+0000 7f54991d0700 1 -- 172.21.15.107:0/4146864576 wait complete. 2024-05-17T21:57:23.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:23.358+0000 7fd03779e700 1 -- 172.21.15.107:0/223616155 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0300662c0 msgr2=0x7fd0300ff9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:23.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:23.358+0000 7fd03779e700 1 --2- 172.21.15.107:0/223616155 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0300662c0 0x7fd0300ff9a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd02c009ce0 tx=0x7fd02c004d10 comp rx=0 tx=0).stop 2024-05-17T21:57:23.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:23.358+0000 7fd03779e700 1 -- 172.21.15.107:0/223616155 shutdown_connections 2024-05-17T21:57:23.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:23.358+0000 7fd03779e700 1 --2- 172.21.15.107:0/223616155 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd0300ffee0 0x7fd030102300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:23.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:23.358+0000 7fd03779e700 1 --2- 172.21.15.107:0/223616155 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0300662c0 0x7fd0300ff9a0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:23.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:23.358+0000 7fd03779e700 1 -- 172.21.15.107:0/223616155 >> 172.21.15.107:0/223616155 conn(0x7fd0300f7430 msgr2=0x7fd0300f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:23.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:23.358+0000 7fd03779e700 1 -- 172.21.15.107:0/223616155 shutdown_connections 2024-05-17T21:57:23.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:23.358+0000 7fd03779e700 1 -- 172.21.15.107:0/223616155 wait complete. 2024-05-17T21:57:23.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:23.359+0000 7fd03779e700 1 Processor -- start 2024-05-17T21:57:23.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:23.359+0000 7fd03779e700 1 -- start start 2024-05-17T21:57:23.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:23.359+0000 7fd03779e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0300662c0 0x7fd0300fd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:23.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:23.359+0000 7fd03779e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd0300ffee0 0x7fd0300fb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:23.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:23.359+0000 7fd03779e700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0300fbe50 con 0x7fd0300662c0 2024-05-17T21:57:23.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:23.359+0000 7fd03779e700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0300fbfc0 con 0x7fd0300ffee0 2024-05-17T21:57:23.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:23.359+0000 7fd03553a700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0300662c0 0x7fd0300fd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:23.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:23.359+0000 7fd03553a700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0300662c0 0x7fd0300fd3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45498/0 (socket says 172.21.15.107:45498) 2024-05-17T21:57:23.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:23.359+0000 7fd03553a700 1 -- 172.21.15.107:0/3892118816 learned_addr learned my addr 172.21.15.107:0/3892118816 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:57:23.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:23.359+0000 7fd03553a700 1 -- 172.21.15.107:0/3892118816 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd0300ffee0 msgr2=0x7fd0300fb8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:23.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:23.359+0000 7fd03553a700 1 --2- 172.21.15.107:0/3892118816 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd0300ffee0 0x7fd0300fb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:23.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:23.359+0000 7fd03553a700 1 -- 172.21.15.107:0/3892118816 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd02c009940 con 0x7fd0300662c0 2024-05-17T21:57:23.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:23.359+0000 7fd03553a700 1 --2- 172.21.15.107:0/3892118816 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0300662c0 0x7fd0300fd3d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd02c009380 tx=0x7fd02c004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:23.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:23.359+0000 7fd025ffb700 1 -- 172.21.15.107:0/3892118816 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd02c00e040 con 0x7fd0300662c0 2024-05-17T21:57:23.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:23.359+0000 7fd03779e700 1 -- 172.21.15.107:0/3892118816 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0300fc240 con 0x7fd0300662c0 2024-05-17T21:57:23.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:23.359+0000 7fd025ffb700 1 -- 172.21.15.107:0/3892118816 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd02c01cd00 con 0x7fd0300662c0 2024-05-17T21:57:23.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:23.359+0000 7fd025ffb700 1 -- 172.21.15.107:0/3892118816 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd02c012bc0 con 0x7fd0300662c0 2024-05-17T21:57:23.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:23.360+0000 7fd03779e700 1 -- 172.21.15.107:0/3892118816 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd030115a50 con 0x7fd0300662c0 2024-05-17T21:57:23.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:23.361+0000 7fd03779e700 1 -- 172.21.15.107:0/3892118816 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd03005f5c0 con 0x7fd0300662c0 2024-05-17T21:57:23.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:23.361+0000 7fd025ffb700 1 -- 172.21.15.107:0/3892118816 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7fd02c012410 con 0x7fd0300662c0 2024-05-17T21:57:23.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:23.362+0000 7fd025ffb700 1 --2- 172.21.15.107:0/3892118816 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fd01c078990 0x7fd01c07ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:23.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:23.362+0000 7fd034d39700 1 --2- 172.21.15.107:0/3892118816 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fd01c078990 0x7fd01c07ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:23.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:23.362+0000 7fd025ffb700 1 -- 172.21.15.107:0/3892118816 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fd02c09ca60 con 0x7fd0300662c0 2024-05-17T21:57:23.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:23.362+0000 7fd034d39700 1 --2- 172.21.15.107:0/3892118816 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fd01c078990 0x7fd01c07ae40 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd020009fd0 tx=0x7fd02000b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:23.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:23.366+0000 7fd025ffb700 1 -- 172.21.15.107:0/3892118816 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7fd02c0646d0 con 0x7fd0300662c0 2024-05-17T21:57:23.496 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:23 smithi107 ceph-mon[93224]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 4 pgs peering) 2024-05-17T21:57:23.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:23 smithi138 ceph-mon[81376]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 4 pgs peering) 2024-05-17T21:57:23.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:23.678+0000 7fd03779e700 1 -- 172.21.15.107:0/3892118816 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fd030115ca0 con 0x7fd0300662c0 2024-05-17T21:57:23.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:23.678+0000 7fd025ffb700 1 -- 172.21.15.107:0/3892118816 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+839 (secure 0 0 0) 0x7fd02c064070 con 0x7fd0300662c0 2024-05-17T21:57:23.669 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-05-17T21:57:23.669 INFO:teuthology.orchestra.run.smithi107.stdout: "mon": { 2024-05-17T21:57:23.669 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev)": 2 2024-05-17T21:57:23.669 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-05-17T21:57:23.669 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr": { 2024-05-17T21:57:23.670 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev)": 2 2024-05-17T21:57:23.670 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-05-17T21:57:23.670 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": { 2024-05-17T21:57:23.670 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-952-g9903f776 (9903f7767386a9d9efadb7c1effdcd3064d71804) quincy (stable)": 3, 2024-05-17T21:57:23.670 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev)": 2 2024-05-17T21:57:23.670 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-05-17T21:57:23.670 INFO:teuthology.orchestra.run.smithi107.stdout: "mds": { 2024-05-17T21:57:23.670 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-952-g9903f776 (9903f7767386a9d9efadb7c1effdcd3064d71804) quincy (stable)": 4 2024-05-17T21:57:23.670 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-05-17T21:57:23.670 INFO:teuthology.orchestra.run.smithi107.stdout: "overall": { 2024-05-17T21:57:23.670 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-952-g9903f776 (9903f7767386a9d9efadb7c1effdcd3064d71804) quincy (stable)": 7, 2024-05-17T21:57:23.670 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev)": 6 2024-05-17T21:57:23.670 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-05-17T21:57:23.670 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-05-17T21:57:23.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:23.681+0000 7fd03779e700 1 -- 172.21.15.107:0/3892118816 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fd01c078990 msgr2=0x7fd01c07ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:23.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:23.682+0000 7fd03779e700 1 --2- 172.21.15.107:0/3892118816 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fd01c078990 0x7fd01c07ae40 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd020009fd0 tx=0x7fd02000b040 comp rx=0 tx=0).stop 2024-05-17T21:57:23.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:23.682+0000 7fd03779e700 1 -- 172.21.15.107:0/3892118816 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0300662c0 msgr2=0x7fd0300fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:23.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:23.682+0000 7fd03779e700 1 --2- 172.21.15.107:0/3892118816 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0300662c0 0x7fd0300fd3d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd02c009380 tx=0x7fd02c004d10 comp rx=0 tx=0).stop 2024-05-17T21:57:23.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:23.682+0000 7fd03779e700 1 -- 172.21.15.107:0/3892118816 shutdown_connections 2024-05-17T21:57:23.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:23.682+0000 7fd03779e700 1 --2- 172.21.15.107:0/3892118816 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fd01c078990 0x7fd01c07ae40 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:23.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:23.682+0000 7fd03779e700 1 --2- 172.21.15.107:0/3892118816 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0300662c0 0x7fd0300fd3d0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:23.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:23.682+0000 7fd03779e700 1 --2- 172.21.15.107:0/3892118816 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd0300ffee0 0x7fd0300fb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:23.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:23.682+0000 7fd03779e700 1 -- 172.21.15.107:0/3892118816 >> 172.21.15.107:0/3892118816 conn(0x7fd0300f7430 msgr2=0x7fd0300f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:23.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:23.682+0000 7fd03779e700 1 -- 172.21.15.107:0/3892118816 shutdown_connections 2024-05-17T21:57:23.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:23.682+0000 7fd03779e700 1 -- 172.21.15.107:0/3892118816 wait complete. 2024-05-17T21:57:23.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:23.805+0000 7f6b65af0700 1 -- 172.21.15.107:0/190904957 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b600fe080 msgr2=0x7f6b600fe4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:23.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:23.805+0000 7f6b65af0700 1 --2- 172.21.15.107:0/190904957 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b600fe080 0x7f6b600fe4f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6b54004940 tx=0x7f6b54009e30 comp rx=0 tx=0).stop 2024-05-17T21:57:23.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:23.805+0000 7f6b65af0700 1 -- 172.21.15.107:0/190904957 shutdown_connections 2024-05-17T21:57:23.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:23.805+0000 7f6b65af0700 1 --2- 172.21.15.107:0/190904957 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b600fe080 0x7f6b600fe4f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:23.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:23.805+0000 7f6b65af0700 1 --2- 172.21.15.107:0/190904957 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b600fcf10 0x7f6b600fd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:23.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:23.805+0000 7f6b65af0700 1 -- 172.21.15.107:0/190904957 >> 172.21.15.107:0/190904957 conn(0x7f6b600f8560 msgr2=0x7f6b600fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:23.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:23.805+0000 7f6b65af0700 1 -- 172.21.15.107:0/190904957 shutdown_connections 2024-05-17T21:57:23.797 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:23.805+0000 7f6b65af0700 1 -- 172.21.15.107:0/190904957 wait complete. 2024-05-17T21:57:23.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:23.806+0000 7f6b65af0700 1 Processor -- start 2024-05-17T21:57:23.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:23.806+0000 7f6b65af0700 1 -- start start 2024-05-17T21:57:23.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:23.806+0000 7f6b65af0700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b600fcf10 0x7f6b60071de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:23.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:23.806+0000 7f6b65af0700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b600fe080 0x7f6b600702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:23.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:23.806+0000 7f6b65af0700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b60070860 con 0x7f6b600fcf10 2024-05-17T21:57:23.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:23.806+0000 7f6b65af0700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b600709d0 con 0x7f6b600fe080 2024-05-17T21:57:23.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:23.806+0000 7f6b5f7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b600fcf10 0x7f6b60071de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:23.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:23.806+0000 7f6b5f7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b600fcf10 0x7f6b60071de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45516/0 (socket says 172.21.15.107:45516) 2024-05-17T21:57:23.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:23.806+0000 7f6b5f7fe700 1 -- 172.21.15.107:0/67438117 learned_addr learned my addr 172.21.15.107:0/67438117 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:57:23.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:23.806+0000 7f6b5effd700 1 --2- 172.21.15.107:0/67438117 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b600fe080 0x7f6b600702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:23.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:23.807+0000 7f6b5f7fe700 1 -- 172.21.15.107:0/67438117 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b600fe080 msgr2=0x7f6b600702b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:23.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:23.807+0000 7f6b5f7fe700 1 --2- 172.21.15.107:0/67438117 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b600fe080 0x7f6b600702b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:23.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:23.807+0000 7f6b5f7fe700 1 -- 172.21.15.107:0/67438117 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b54005040 con 0x7f6b600fcf10 2024-05-17T21:57:23.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:23.807+0000 7f6b5f7fe700 1 --2- 172.21.15.107:0/67438117 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b600fcf10 0x7f6b60071de0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f6b5000eb40 tx=0x7f6b5000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:23.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:23.807+0000 7f6b47fff700 1 -- 172.21.15.107:0/67438117 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b50007cc0 con 0x7f6b600fcf10 2024-05-17T21:57:23.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:23.807+0000 7f6b65af0700 1 -- 172.21.15.107:0/67438117 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b60070c50 con 0x7f6b600fcf10 2024-05-17T21:57:23.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:23.807+0000 7f6b47fff700 1 -- 172.21.15.107:0/67438117 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6b5000f040 con 0x7f6b600fcf10 2024-05-17T21:57:23.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:23.807+0000 7f6b47fff700 1 -- 172.21.15.107:0/67438117 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b5000ca10 con 0x7f6b600fcf10 2024-05-17T21:57:23.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:23.807+0000 7f6b65af0700 1 -- 172.21.15.107:0/67438117 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b601159b0 con 0x7f6b600fcf10 2024-05-17T21:57:23.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:23.808+0000 7f6b65af0700 1 -- 172.21.15.107:0/67438117 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b6005f5c0 con 0x7f6b600fcf10 2024-05-17T21:57:23.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:23.809+0000 7f6b47fff700 1 -- 172.21.15.107:0/67438117 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f6b5000cb90 con 0x7f6b600fcf10 2024-05-17T21:57:23.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:23.809+0000 7f6b47fff700 1 --2- 172.21.15.107:0/67438117 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f6b48078990 0x7f6b4807ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:23.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:23.810+0000 7f6b47fff700 1 -- 172.21.15.107:0/67438117 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f6b5009a250 con 0x7f6b600fcf10 2024-05-17T21:57:23.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:23.810+0000 7f6b5effd700 1 --2- 172.21.15.107:0/67438117 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f6b48078990 0x7f6b4807ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:23.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:23.810+0000 7f6b5effd700 1 --2- 172.21.15.107:0/67438117 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f6b48078990 0x7f6b4807ae40 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6b54009b30 tx=0x7f6b54009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:23.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:23.814+0000 7f6b47fff700 1 -- 172.21.15.107:0/67438117 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f6b50061f90 con 0x7f6b600fcf10 2024-05-17T21:57:24.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:24.091+0000 7f6b65af0700 1 -- 172.21.15.107:0/67438117 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f6b6004bed0 con 0x7f6b600fcf10 2024-05-17T21:57:24.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:24.092+0000 7f6b47fff700 1 -- 172.21.15.107:0/67438117 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1910 (secure 0 0 0) 0x7f6b50061930 con 0x7f6b600fcf10 2024-05-17T21:57:24.083 INFO:teuthology.orchestra.run.smithi107.stdout:e15 2024-05-17T21:57:24.083 INFO:teuthology.orchestra.run.smithi107.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-17T21:57:24.083 INFO:teuthology.orchestra.run.smithi107.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-17T21:57:24.083 INFO:teuthology.orchestra.run.smithi107.stdout:legacy client fscid: 1 2024-05-17T21:57:24.083 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:57:24.083 INFO:teuthology.orchestra.run.smithi107.stdout:Filesystem 'cephfs' (1) 2024-05-17T21:57:24.083 INFO:teuthology.orchestra.run.smithi107.stdout:fs_name cephfs 2024-05-17T21:57:24.083 INFO:teuthology.orchestra.run.smithi107.stdout:epoch 15 2024-05-17T21:57:24.083 INFO:teuthology.orchestra.run.smithi107.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-17T21:57:24.083 INFO:teuthology.orchestra.run.smithi107.stdout:created 2024-05-17T21:49:16.836333+0000 2024-05-17T21:57:24.084 INFO:teuthology.orchestra.run.smithi107.stdout:modified 2024-05-17T21:49:34.873628+0000 2024-05-17T21:57:24.084 INFO:teuthology.orchestra.run.smithi107.stdout:tableserver 0 2024-05-17T21:57:24.084 INFO:teuthology.orchestra.run.smithi107.stdout:root 0 2024-05-17T21:57:24.084 INFO:teuthology.orchestra.run.smithi107.stdout:session_timeout 60 2024-05-17T21:57:24.084 INFO:teuthology.orchestra.run.smithi107.stdout:session_autoclose 300 2024-05-17T21:57:24.084 INFO:teuthology.orchestra.run.smithi107.stdout:max_file_size 1099511627776 2024-05-17T21:57:24.084 INFO:teuthology.orchestra.run.smithi107.stdout:max_xattr_size 65536 2024-05-17T21:57:24.084 INFO:teuthology.orchestra.run.smithi107.stdout:required_client_features {} 2024-05-17T21:57:24.084 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure 0 2024-05-17T21:57:24.084 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure_osd_epoch 46 2024-05-17T21:57:24.084 INFO:teuthology.orchestra.run.smithi107.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-17T21:57:24.084 INFO:teuthology.orchestra.run.smithi107.stdout:max_mds 1 2024-05-17T21:57:24.084 INFO:teuthology.orchestra.run.smithi107.stdout:in 0 2024-05-17T21:57:24.085 INFO:teuthology.orchestra.run.smithi107.stdout:up {0=24259} 2024-05-17T21:57:24.085 INFO:teuthology.orchestra.run.smithi107.stdout:failed 2024-05-17T21:57:24.085 INFO:teuthology.orchestra.run.smithi107.stdout:damaged 2024-05-17T21:57:24.085 INFO:teuthology.orchestra.run.smithi107.stdout:stopped 2024-05-17T21:57:24.085 INFO:teuthology.orchestra.run.smithi107.stdout:data_pools [3] 2024-05-17T21:57:24.085 INFO:teuthology.orchestra.run.smithi107.stdout:metadata_pool 2 2024-05-17T21:57:24.085 INFO:teuthology.orchestra.run.smithi107.stdout:inline_data enabled 2024-05-17T21:57:24.085 INFO:teuthology.orchestra.run.smithi107.stdout:balancer 2024-05-17T21:57:24.085 INFO:teuthology.orchestra.run.smithi107.stdout:bal_rank_mask -1 2024-05-17T21:57:24.085 INFO:teuthology.orchestra.run.smithi107.stdout:standby_count_wanted 1 2024-05-17T21:57:24.085 INFO:teuthology.orchestra.run.smithi107.stdout:qdb_cluster leader: 0 members: 2024-05-17T21:57:24.085 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi138.gwbtyd{0:24259} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.138:6824/2779634919,v1:172.21.15.138:6825/2779634919] compat {c=[1],r=[1],i=[7ff]}] 2024-05-17T21:57:24.085 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.drqody{0:14486} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.107:6828/2275807100,v1:172.21.15.107:6829/2275807100] compat {c=[1],r=[1],i=[7ff]}] 2024-05-17T21:57:24.085 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:57:24.086 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:57:24.086 INFO:teuthology.orchestra.run.smithi107.stdout:Standby daemons: 2024-05-17T21:57:24.086 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:57:24.086 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.idmbgj{-1:14498} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.107:6826/1460186079,v1:172.21.15.107:6827/1460186079] compat {c=[1],r=[1],i=[7ff]}] 2024-05-17T21:57:24.086 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi138.zckrvs{-1:24267} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.138:6826/170628661,v1:172.21.15.138:6827/170628661] compat {c=[1],r=[1],i=[7ff]}] 2024-05-17T21:57:24.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:24.094+0000 7f6b65af0700 1 -- 172.21.15.107:0/67438117 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f6b48078990 msgr2=0x7f6b4807ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:24.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:24.094+0000 7f6b65af0700 1 --2- 172.21.15.107:0/67438117 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f6b48078990 0x7f6b4807ae40 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6b54009b30 tx=0x7f6b54009e30 comp rx=0 tx=0).stop 2024-05-17T21:57:24.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:24.094+0000 7f6b65af0700 1 -- 172.21.15.107:0/67438117 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b600fcf10 msgr2=0x7f6b60071de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:24.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:24.094+0000 7f6b65af0700 1 --2- 172.21.15.107:0/67438117 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b600fcf10 0x7f6b60071de0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f6b5000eb40 tx=0x7f6b5000b040 comp rx=0 tx=0).stop 2024-05-17T21:57:24.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:24.094+0000 7f6b65af0700 1 -- 172.21.15.107:0/67438117 shutdown_connections 2024-05-17T21:57:24.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:24.094+0000 7f6b65af0700 1 --2- 172.21.15.107:0/67438117 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f6b48078990 0x7f6b4807ae40 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:24.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:24.094+0000 7f6b65af0700 1 --2- 172.21.15.107:0/67438117 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b600fcf10 0x7f6b60071de0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:24.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:24.094+0000 7f6b65af0700 1 --2- 172.21.15.107:0/67438117 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b600fe080 0x7f6b600702b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:24.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:24.094+0000 7f6b65af0700 1 -- 172.21.15.107:0/67438117 >> 172.21.15.107:0/67438117 conn(0x7f6b600f8560 msgr2=0x7f6b600f9ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:24.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:24.094+0000 7f6b65af0700 1 -- 172.21.15.107:0/67438117 shutdown_connections 2024-05-17T21:57:24.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:24.094+0000 7f6b65af0700 1 -- 172.21.15.107:0/67438117 wait complete. 2024-05-17T21:57:24.087 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 15 2024-05-17T21:57:24.192 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:24.199+0000 7f9bb8cde700 1 -- 172.21.15.107:0/3260061750 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9bb40fe430 msgr2=0x7f9bb40fe840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:24.192 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:24.199+0000 7f9bb8cde700 1 --2- 172.21.15.107:0/3260061750 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9bb40fe430 0x7f9bb40fe840 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9ba4004940 tx=0x7f9ba4009e30 comp rx=0 tx=0).stop 2024-05-17T21:57:24.192 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:24.200+0000 7f9bb8cde700 1 -- 172.21.15.107:0/3260061750 shutdown_connections 2024-05-17T21:57:24.192 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:24.200+0000 7f9bb8cde700 1 --2- 172.21.15.107:0/3260061750 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9bb40fed80 0x7f9bb40faec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:24.192 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:24.200+0000 7f9bb8cde700 1 --2- 172.21.15.107:0/3260061750 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9bb40fe430 0x7f9bb40fe840 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:24.192 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:24.200+0000 7f9bb8cde700 1 -- 172.21.15.107:0/3260061750 >> 172.21.15.107:0/3260061750 conn(0x7f9bb40f6bb0 msgr2=0x7f9bb40f9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:24.192 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:24.200+0000 7f9bb8cde700 1 -- 172.21.15.107:0/3260061750 shutdown_connections 2024-05-17T21:57:24.192 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:24.200+0000 7f9bb8cde700 1 -- 172.21.15.107:0/3260061750 wait complete. 2024-05-17T21:57:24.192 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:24.200+0000 7f9bb8cde700 1 Processor -- start 2024-05-17T21:57:24.192 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:24.200+0000 7f9bb8cde700 1 -- start start 2024-05-17T21:57:24.193 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:24.200+0000 7f9bb8cde700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9bb40fe430 0x7f9bb410d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:24.193 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:24.200+0000 7f9bb8cde700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9bb40fed80 0x7f9bb410dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:24.193 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:24.200+0000 7f9bb8cde700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb410e4e0 con 0x7f9bb40fe430 2024-05-17T21:57:24.193 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:24.200+0000 7f9bb8cde700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb410e650 con 0x7f9bb40fed80 2024-05-17T21:57:24.193 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:24.200+0000 7f9bb259c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9bb40fe430 0x7f9bb410d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:24.193 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:24.200+0000 7f9bb259c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9bb40fe430 0x7f9bb410d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45538/0 (socket says 172.21.15.107:45538) 2024-05-17T21:57:24.193 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:24.200+0000 7f9bb259c700 1 -- 172.21.15.107:0/1969650523 learned_addr learned my addr 172.21.15.107:0/1969650523 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:57:24.193 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:24.201+0000 7f9bb1d9b700 1 --2- 172.21.15.107:0/1969650523 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9bb40fed80 0x7f9bb410dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:24.193 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:24.201+0000 7f9bb259c700 1 -- 172.21.15.107:0/1969650523 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9bb40fed80 msgr2=0x7f9bb410dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:24.193 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:24.201+0000 7f9bb259c700 1 --2- 172.21.15.107:0/1969650523 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9bb40fed80 0x7f9bb410dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:24.193 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:24.201+0000 7f9bb259c700 1 -- 172.21.15.107:0/1969650523 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ba4005040 con 0x7f9bb40fe430 2024-05-17T21:57:24.193 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:24.201+0000 7f9bb259c700 1 --2- 172.21.15.107:0/1969650523 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9bb40fe430 0x7f9bb410d9a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9ba4009fd0 tx=0x7f9ba4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:24.193 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:24.201+0000 7f9ba2ffd700 1 -- 172.21.15.107:0/1969650523 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9ba400e040 con 0x7f9bb40fe430 2024-05-17T21:57:24.193 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:24.201+0000 7f9ba2ffd700 1 -- 172.21.15.107:0/1969650523 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9ba400bcc0 con 0x7f9bb40fe430 2024-05-17T21:57:24.194 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:24.201+0000 7f9bb8cde700 1 -- 172.21.15.107:0/1969650523 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bb4113050 con 0x7f9bb40fe430 2024-05-17T21:57:24.194 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:24.201+0000 7f9ba2ffd700 1 -- 172.21.15.107:0/1969650523 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9ba401c630 con 0x7f9bb40fe430 2024-05-17T21:57:24.194 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:24.201+0000 7f9bb8cde700 1 -- 172.21.15.107:0/1969650523 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bb4113610 con 0x7f9bb40fe430 2024-05-17T21:57:24.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:24.202+0000 7f9bb8cde700 1 -- 172.21.15.107:0/1969650523 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bb404c710 con 0x7f9bb40fe430 2024-05-17T21:57:24.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:24.203+0000 7f9ba2ffd700 1 -- 172.21.15.107:0/1969650523 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f9ba4013980 con 0x7f9bb40fe430 2024-05-17T21:57:24.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:24.203+0000 7f9ba2ffd700 1 --2- 172.21.15.107:0/1969650523 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f9b9c078990 0x7f9b9c07ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:24.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:24.204+0000 7f9ba2ffd700 1 -- 172.21.15.107:0/1969650523 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f9ba409b990 con 0x7f9bb40fe430 2024-05-17T21:57:24.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:24.204+0000 7f9bb1d9b700 1 --2- 172.21.15.107:0/1969650523 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f9b9c078990 0x7f9b9c07ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:24.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:24.204+0000 7f9bb1d9b700 1 --2- 172.21.15.107:0/1969650523 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f9b9c078990 0x7f9b9c07ae40 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9ba80041d0 tx=0x7f9ba80043c0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:24.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:24.208+0000 7f9ba2ffd700 1 -- 172.21.15.107:0/1969650523 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f9ba4063680 con 0x7f9bb40fe430 2024-05-17T21:57:24.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:24 smithi107 ceph-mon[93224]: pgmap v99: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 258 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 34/267 objects degraded (12.734%) 2024-05-17T21:57:24.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:24 smithi107 ceph-mon[93224]: from='client.34282 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:57:24.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:24 smithi107 ceph-mon[93224]: from='client.34286 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:57:24.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:24 smithi107 ceph-mon[93224]: from='client.? 172.21.15.107:0/3892118816' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:57:24.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:24 smithi107 ceph-mon[93224]: from='client.? 172.21.15.107:0/67438117' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-17T21:57:24.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:24.481+0000 7f9bb8cde700 1 -- 172.21.15.107:0/1969650523 --> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9bb4102540 con 0x7f9b9c078990 2024-05-17T21:57:24.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:24.483+0000 7f9ba2ffd700 1 -- 172.21.15.107:0/1969650523 <== mgr.34100 v2:172.21.15.107:6800/3205268226 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f9bb4102540 con 0x7f9b9c078990 2024-05-17T21:57:24.473 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-05-17T21:57:24.474 INFO:teuthology.orchestra.run.smithi107.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b", 2024-05-17T21:57:24.474 INFO:teuthology.orchestra.run.smithi107.stdout: "in_progress": true, 2024-05-17T21:57:24.474 INFO:teuthology.orchestra.run.smithi107.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-17T21:57:24.474 INFO:teuthology.orchestra.run.smithi107.stdout: "services_complete": [ 2024-05-17T21:57:24.474 INFO:teuthology.orchestra.run.smithi107.stdout: "crash", 2024-05-17T21:57:24.474 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr", 2024-05-17T21:57:24.474 INFO:teuthology.orchestra.run.smithi107.stdout: "mon" 2024-05-17T21:57:24.474 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-05-17T21:57:24.474 INFO:teuthology.orchestra.run.smithi107.stdout: "progress": "8/21 daemons upgraded", 2024-05-17T21:57:24.474 INFO:teuthology.orchestra.run.smithi107.stdout: "message": "Currently upgrading osd daemons", 2024-05-17T21:57:24.474 INFO:teuthology.orchestra.run.smithi107.stdout: "is_paused": false 2024-05-17T21:57:24.474 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-05-17T21:57:24.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:24.486+0000 7f9bb8cde700 1 -- 172.21.15.107:0/1969650523 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f9b9c078990 msgr2=0x7f9b9c07ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:24.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:24.486+0000 7f9bb8cde700 1 --2- 172.21.15.107:0/1969650523 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f9b9c078990 0x7f9b9c07ae40 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9ba80041d0 tx=0x7f9ba80043c0 comp rx=0 tx=0).stop 2024-05-17T21:57:24.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:24.486+0000 7f9bb8cde700 1 -- 172.21.15.107:0/1969650523 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9bb40fe430 msgr2=0x7f9bb410d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:24.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:24.486+0000 7f9bb8cde700 1 --2- 172.21.15.107:0/1969650523 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9bb40fe430 0x7f9bb410d9a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9ba4009fd0 tx=0x7f9ba4009e30 comp rx=0 tx=0).stop 2024-05-17T21:57:24.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:24.486+0000 7f9bb8cde700 1 -- 172.21.15.107:0/1969650523 shutdown_connections 2024-05-17T21:57:24.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:24.486+0000 7f9bb8cde700 1 --2- 172.21.15.107:0/1969650523 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f9b9c078990 0x7f9b9c07ae40 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:24.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:24.486+0000 7f9bb8cde700 1 --2- 172.21.15.107:0/1969650523 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9bb40fe430 0x7f9bb410d9a0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:24.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:24.486+0000 7f9bb8cde700 1 --2- 172.21.15.107:0/1969650523 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9bb40fed80 0x7f9bb410dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:24.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:24.486+0000 7f9bb8cde700 1 -- 172.21.15.107:0/1969650523 >> 172.21.15.107:0/1969650523 conn(0x7f9bb40f6bb0 msgr2=0x7f9bb40f82f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:24.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:24.486+0000 7f9bb8cde700 1 -- 172.21.15.107:0/1969650523 shutdown_connections 2024-05-17T21:57:24.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:24.486+0000 7f9bb8cde700 1 -- 172.21.15.107:0/1969650523 wait complete. 2024-05-17T21:57:24.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:24 smithi138 ceph-mon[81376]: pgmap v99: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 258 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 34/267 objects degraded (12.734%) 2024-05-17T21:57:24.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:24 smithi138 ceph-mon[81376]: from='client.34282 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:57:24.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:24 smithi138 ceph-mon[81376]: from='client.34286 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:57:24.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:24 smithi138 ceph-mon[81376]: from='client.? 172.21.15.107:0/3892118816' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:57:24.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:24 smithi138 ceph-mon[81376]: from='client.? 172.21.15.107:0/67438117' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-17T21:57:24.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:24.629+0000 7f0db5a2b700 1 -- 172.21.15.107:0/786549431 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0db00fce80 msgr2=0x7f0db00fd290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:24.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:24.629+0000 7f0db5a2b700 1 --2- 172.21.15.107:0/786549431 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0db00fce80 0x7f0db00fd290 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0d98004940 tx=0x7f0d98009e30 comp rx=0 tx=0).stop 2024-05-17T21:57:24.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:24.629+0000 7f0db5a2b700 1 -- 172.21.15.107:0/786549431 shutdown_connections 2024-05-17T21:57:24.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:24.629+0000 7f0db5a2b700 1 --2- 172.21.15.107:0/786549431 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0db00fdff0 0x7f0db00fe460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:24.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:24.629+0000 7f0db5a2b700 1 --2- 172.21.15.107:0/786549431 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0db00fce80 0x7f0db00fd290 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:24.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:24.629+0000 7f0db5a2b700 1 -- 172.21.15.107:0/786549431 >> 172.21.15.107:0/786549431 conn(0x7f0db00f8540 msgr2=0x7f0db00fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:24.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:24.629+0000 7f0db5a2b700 1 -- 172.21.15.107:0/786549431 shutdown_connections 2024-05-17T21:57:24.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:24.629+0000 7f0db5a2b700 1 -- 172.21.15.107:0/786549431 wait complete. 2024-05-17T21:57:24.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:24.629+0000 7f0db5a2b700 1 Processor -- start 2024-05-17T21:57:24.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:24.629+0000 7f0db5a2b700 1 -- start start 2024-05-17T21:57:24.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:24.630+0000 7f0db5a2b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0db00fce80 0x7f0db0071de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:24.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:24.630+0000 7f0db5a2b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0db00fdff0 0x7f0db00702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:24.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:24.630+0000 7f0db5a2b700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0db0070860 con 0x7f0db00fce80 2024-05-17T21:57:24.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:24.630+0000 7f0db5a2b700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0db00709d0 con 0x7f0db00fdff0 2024-05-17T21:57:24.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:24.630+0000 7f0daeffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0db00fce80 0x7f0db0071de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:24.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:24.630+0000 7f0daeffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0db00fce80 0x7f0db0071de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45546/0 (socket says 172.21.15.107:45546) 2024-05-17T21:57:24.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:24.630+0000 7f0daeffd700 1 -- 172.21.15.107:0/3601376976 learned_addr learned my addr 172.21.15.107:0/3601376976 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:57:24.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:24.630+0000 7f0dae7fc700 1 --2- 172.21.15.107:0/3601376976 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0db00fdff0 0x7f0db00702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:24.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:24.630+0000 7f0daeffd700 1 -- 172.21.15.107:0/3601376976 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0db00fdff0 msgr2=0x7f0db00702b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:24.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:24.630+0000 7f0daeffd700 1 --2- 172.21.15.107:0/3601376976 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0db00fdff0 0x7f0db00702b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:24.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:24.630+0000 7f0daeffd700 1 -- 172.21.15.107:0/3601376976 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d98005040 con 0x7f0db00fce80 2024-05-17T21:57:24.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:24.630+0000 7f0dae7fc700 1 --2- 172.21.15.107:0/3601376976 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0db00fdff0 0x7f0db00702b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:57:24.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:24.630+0000 7f0daeffd700 1 --2- 172.21.15.107:0/3601376976 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0db00fce80 0x7f0db0071de0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f0d9800b3b0 tx=0x7f0d98009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:24.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:24.631+0000 7f0da77fe700 1 -- 172.21.15.107:0/3601376976 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0d9800e040 con 0x7f0db00fce80 2024-05-17T21:57:24.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:24.631+0000 7f0da77fe700 1 -- 172.21.15.107:0/3601376976 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0d9800bcc0 con 0x7f0db00fce80 2024-05-17T21:57:24.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:24.631+0000 7f0da77fe700 1 -- 172.21.15.107:0/3601376976 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0d9801c630 con 0x7f0db00fce80 2024-05-17T21:57:24.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:24.631+0000 7f0db5a2b700 1 -- 172.21.15.107:0/3601376976 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0db0070c50 con 0x7f0db00fce80 2024-05-17T21:57:24.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:24.631+0000 7f0db5a2b700 1 -- 172.21.15.107:0/3601376976 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0db0115a50 con 0x7f0db00fce80 2024-05-17T21:57:24.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:24.632+0000 7f0db5a2b700 1 -- 172.21.15.107:0/3601376976 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0db005f5c0 con 0x7f0db00fce80 2024-05-17T21:57:24.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:24.633+0000 7f0da77fe700 1 -- 172.21.15.107:0/3601376976 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f0d98013980 con 0x7f0db00fce80 2024-05-17T21:57:24.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:24.633+0000 7f0da77fe700 1 --2- 172.21.15.107:0/3601376976 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f0d9c078990 0x7f0d9c07ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:24.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:24.634+0000 7f0da77fe700 1 -- 172.21.15.107:0/3601376976 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f0d9809b320 con 0x7f0db00fce80 2024-05-17T21:57:24.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:24.634+0000 7f0dae7fc700 1 --2- 172.21.15.107:0/3601376976 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f0d9c078990 0x7f0d9c07ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:24.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:24.634+0000 7f0dae7fc700 1 --2- 172.21.15.107:0/3601376976 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f0d9c078990 0x7f0d9c07ae40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0da0009fd0 tx=0x7f0da000b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:24.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:24.638+0000 7f0da77fe700 1 -- 172.21.15.107:0/3601376976 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f0d98063010 con 0x7f0db00fce80 2024-05-17T21:57:24.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:24.953+0000 7f0db5a2b700 1 -- 172.21.15.107:0/3601376976 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f0db004bed0 con 0x7f0db00fce80 2024-05-17T21:57:24.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:24.953+0000 7f0da77fe700 1 -- 172.21.15.107:0/3601376976 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+1130 (secure 0 0 0) 0x7f0d980629b0 con 0x7f0db00fce80 2024-05-17T21:57:24.944 INFO:teuthology.orchestra.run.smithi107.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data; 1 osds down; Degraded data redundancy: 34/267 objects degraded (12.734%), 12 pgs degraded 2024-05-17T21:57:24.945 INFO:teuthology.orchestra.run.smithi107.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-17T21:57:24.945 INFO:teuthology.orchestra.run.smithi107.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-17T21:57:24.945 INFO:teuthology.orchestra.run.smithi107.stdout:[WRN] OSD_DOWN: 1 osds down 2024-05-17T21:57:24.945 INFO:teuthology.orchestra.run.smithi107.stdout: osd.2 (root=default,host=smithi107) is down 2024-05-17T21:57:24.945 INFO:teuthology.orchestra.run.smithi107.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 34/267 objects degraded (12.734%), 12 pgs degraded 2024-05-17T21:57:24.945 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.1 is active+undersized+degraded, acting [1,0] 2024-05-17T21:57:24.945 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.3 is active+undersized+degraded, acting [5,1] 2024-05-17T21:57:24.945 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.7 is active+undersized+degraded, acting [3,4] 2024-05-17T21:57:24.945 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.c is active+undersized+degraded, acting [3,0] 2024-05-17T21:57:24.945 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.d is active+undersized+degraded, acting [1,3] 2024-05-17T21:57:24.945 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.e is active+undersized+degraded, acting [0,3] 2024-05-17T21:57:24.945 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.10 is active+undersized+degraded, acting [1,0] 2024-05-17T21:57:24.945 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.13 is active+undersized+degraded, acting [0,4] 2024-05-17T21:57:24.946 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.16 is active+undersized+degraded, acting [5,3] 2024-05-17T21:57:24.946 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.19 is active+undersized+degraded, acting [0,4] 2024-05-17T21:57:24.946 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.1c is active+undersized+degraded, acting [4,5] 2024-05-17T21:57:24.946 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.1e is active+undersized+degraded, acting [0,5] 2024-05-17T21:57:24.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:24.955+0000 7f0db5a2b700 1 -- 172.21.15.107:0/3601376976 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f0d9c078990 msgr2=0x7f0d9c07ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:24.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:24.956+0000 7f0db5a2b700 1 --2- 172.21.15.107:0/3601376976 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f0d9c078990 0x7f0d9c07ae40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0da0009fd0 tx=0x7f0da000b040 comp rx=0 tx=0).stop 2024-05-17T21:57:24.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:24.956+0000 7f0db5a2b700 1 -- 172.21.15.107:0/3601376976 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0db00fce80 msgr2=0x7f0db0071de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:24.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:24.956+0000 7f0db5a2b700 1 --2- 172.21.15.107:0/3601376976 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0db00fce80 0x7f0db0071de0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f0d9800b3b0 tx=0x7f0d98009e30 comp rx=0 tx=0).stop 2024-05-17T21:57:24.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:24.956+0000 7f0db5a2b700 1 -- 172.21.15.107:0/3601376976 shutdown_connections 2024-05-17T21:57:24.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:24.956+0000 7f0db5a2b700 1 --2- 172.21.15.107:0/3601376976 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f0d9c078990 0x7f0d9c07ae40 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:24.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:24.956+0000 7f0db5a2b700 1 --2- 172.21.15.107:0/3601376976 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0db00fce80 0x7f0db0071de0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:24.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:24.956+0000 7f0db5a2b700 1 --2- 172.21.15.107:0/3601376976 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0db00fdff0 0x7f0db00702b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:24.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:24.956+0000 7f0db5a2b700 1 -- 172.21.15.107:0/3601376976 >> 172.21.15.107:0/3601376976 conn(0x7f0db00f8540 msgr2=0x7f0db00f9c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:24.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:24.956+0000 7f0db5a2b700 1 -- 172.21.15.107:0/3601376976 shutdown_connections 2024-05-17T21:57:24.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:24.956+0000 7f0db5a2b700 1 -- 172.21.15.107:0/3601376976 wait complete. 2024-05-17T21:57:25.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:25 smithi107 ceph-mon[93224]: from='client.34290 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:57:25.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:25 smithi107 ceph-mon[93224]: from='client.? 172.21.15.107:0/3601376976' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-17T21:57:25.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:25 smithi138 ceph-mon[81376]: from='client.34290 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:57:25.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:25 smithi138 ceph-mon[81376]: from='client.? 172.21.15.107:0/3601376976' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-17T21:57:25.851 INFO:journalctl@ceph.osd.2.smithi107.stdout:May 17 21:57:25 smithi107 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-2[112698]: 2024-05-17T21:57:25.361+0000 7fc7133007c0 -1 osd.2 59 log_to_monitors true 2024-05-17T21:57:26.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:26 smithi138 ceph-mon[81376]: pgmap v100: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 258 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 34/267 objects degraded (12.734%) 2024-05-17T21:57:26.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:26 smithi138 ceph-mon[81376]: from='client.34302 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:57:26.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:26 smithi138 ceph-mon[81376]: from='osd.2 [v2:172.21.15.107:6818/1608136643,v1:172.21.15.107:6819/1608136643]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-17T21:57:26.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:26 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:57:26.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:26 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:57:26.601 INFO:journalctl@ceph.osd.2.smithi107.stdout:May 17 21:57:26 smithi107 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-2[112698]: 2024-05-17T21:57:26.179+0000 7fc704719700 -1 osd.2 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-17T21:57:26.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:26 smithi107 ceph-mon[93224]: pgmap v100: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 258 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 34/267 objects degraded (12.734%) 2024-05-17T21:57:26.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:26 smithi107 ceph-mon[93224]: from='client.34302 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:57:26.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:26 smithi107 ceph-mon[93224]: from='osd.2 [v2:172.21.15.107:6818/1608136643,v1:172.21.15.107:6819/1608136643]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-17T21:57:26.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:26 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:57:26.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:26 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:57:27.495 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:27 smithi107 ceph-mon[93224]: from='osd.2 [v2:172.21.15.107:6818/1608136643,v1:172.21.15.107:6819/1608136643]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-17T21:57:27.496 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:27 smithi107 ceph-mon[93224]: osdmap e62: 6 total, 5 up, 6 in 2024-05-17T21:57:27.496 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:27 smithi107 ceph-mon[93224]: from='osd.2 [v2:172.21.15.107:6818/1608136643,v1:172.21.15.107:6819/1608136643]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-05-17T21:57:27.496 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:27 smithi107 ceph-mon[93224]: pgmap v102: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 258 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 34/267 objects degraded (12.734%) 2024-05-17T21:57:27.496 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:27 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:57:27.496 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:27 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:57:27.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:27 smithi138 ceph-mon[81376]: from='osd.2 [v2:172.21.15.107:6818/1608136643,v1:172.21.15.107:6819/1608136643]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-17T21:57:27.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:27 smithi138 ceph-mon[81376]: osdmap e62: 6 total, 5 up, 6 in 2024-05-17T21:57:27.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:27 smithi138 ceph-mon[81376]: from='osd.2 [v2:172.21.15.107:6818/1608136643,v1:172.21.15.107:6819/1608136643]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-05-17T21:57:27.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:27 smithi138 ceph-mon[81376]: pgmap v102: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 258 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 34/267 objects degraded (12.734%) 2024-05-17T21:57:27.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:27 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:57:27.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:27 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:57:28.483 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:28 smithi107 ceph-mon[93224]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-05-17T21:57:28.483 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:28 smithi107 ceph-mon[93224]: osd.2 [v2:172.21.15.107:6818/1608136643,v1:172.21.15.107:6819/1608136643] boot 2024-05-17T21:57:28.483 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:28 smithi107 ceph-mon[93224]: osdmap e63: 6 total, 6 up, 6 in 2024-05-17T21:57:28.483 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:28 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-17T21:57:28.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:28 smithi138 ceph-mon[81376]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-05-17T21:57:28.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:28 smithi138 ceph-mon[81376]: osd.2 [v2:172.21.15.107:6818/1608136643,v1:172.21.15.107:6819/1608136643] boot 2024-05-17T21:57:28.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:28 smithi138 ceph-mon[81376]: osdmap e63: 6 total, 6 up, 6 in 2024-05-17T21:57:28.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:28 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-17T21:57:29.448 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:29 smithi107 ceph-mon[93224]: osdmap e64: 6 total, 6 up, 6 in 2024-05-17T21:57:29.448 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:29 smithi107 ceph-mon[93224]: pgmap v105: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 258 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 34/267 objects degraded (12.734%) 2024-05-17T21:57:29.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:29 smithi138 ceph-mon[81376]: osdmap e64: 6 total, 6 up, 6 in 2024-05-17T21:57:29.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:29 smithi138 ceph-mon[81376]: pgmap v105: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 258 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 34/267 objects degraded (12.734%) 2024-05-17T21:57:30.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:30 smithi138 ceph-mon[81376]: Health check update: Degraded data redundancy: 34/267 objects degraded (12.734%), 12 pgs degraded (PG_DEGRADED) 2024-05-17T21:57:30.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:30 smithi107 ceph-mon[93224]: Health check update: Degraded data redundancy: 34/267 objects degraded (12.734%), 12 pgs degraded (PG_DEGRADED) 2024-05-17T21:57:31.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:31 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:57:31.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:31 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:57:31.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:31 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:57:31.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:31 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T21:57:31.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:31 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:57:31.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:31 smithi138 ceph-mon[81376]: pgmap v106: 65 pgs: 6 active+undersized+degraded, 6 active+undersized, 53 active+clean; 258 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s; 21/267 objects degraded (7.865%) 2024-05-17T21:57:31.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:31 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:57:31.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:31 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:57:31.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:31 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:57:31.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:31 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:57:31.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:31 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-05-17T21:57:31.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:31 smithi138 ceph-mon[81376]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-05-17T21:57:31.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:31 smithi138 ceph-mon[81376]: Upgrade: unsafe to stop osd(s) at this time (5 PGs are or would become offline) 2024-05-17T21:57:31.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:31 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:57:31.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:31 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:57:31.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:31 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:57:31.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:31 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T21:57:31.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:31 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:57:31.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:31 smithi107 ceph-mon[93224]: pgmap v106: 65 pgs: 6 active+undersized+degraded, 6 active+undersized, 53 active+clean; 258 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s; 21/267 objects degraded (7.865%) 2024-05-17T21:57:31.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:31 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:57:31.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:31 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:57:31.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:31 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:57:31.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:31 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:57:31.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:31 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-05-17T21:57:31.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:31 smithi107 ceph-mon[93224]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-05-17T21:57:31.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:31 smithi107 ceph-mon[93224]: Upgrade: unsafe to stop osd(s) at this time (5 PGs are or would become offline) 2024-05-17T21:57:32.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:32 smithi107 ceph-mon[93224]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 21/267 objects degraded (7.865%), 6 pgs degraded) 2024-05-17T21:57:32.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:32 smithi138 ceph-mon[81376]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 21/267 objects degraded (7.865%), 6 pgs degraded) 2024-05-17T21:57:33.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:33 smithi138 ceph-mon[81376]: pgmap v107: 65 pgs: 65 active+clean; 258 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 819 B/s rd, 1 op/s 2024-05-17T21:57:33.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:33 smithi107 ceph-mon[93224]: pgmap v107: 65 pgs: 65 active+clean; 258 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 819 B/s rd, 1 op/s 2024-05-17T21:57:35.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:35 smithi138 ceph-mon[81376]: pgmap v108: 65 pgs: 65 active+clean; 258 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s 2024-05-17T21:57:35.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:35 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:57:35.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:35 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:57:35.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:35 smithi107 ceph-mon[93224]: pgmap v108: 65 pgs: 65 active+clean; 258 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s 2024-05-17T21:57:35.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:35 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:57:35.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:35 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:57:37.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:37 smithi138 ceph-mon[81376]: pgmap v109: 65 pgs: 65 active+clean; 258 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 2 op/s 2024-05-17T21:57:37.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:37 smithi107 ceph-mon[93224]: pgmap v109: 65 pgs: 65 active+clean; 258 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 2 op/s 2024-05-17T21:57:39.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:39 smithi138 ceph-mon[81376]: pgmap v110: 65 pgs: 65 active+clean; 258 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1000 B/s rd, 1 op/s 2024-05-17T21:57:39.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:39 smithi107 ceph-mon[93224]: pgmap v110: 65 pgs: 65 active+clean; 258 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1000 B/s rd, 1 op/s 2024-05-17T21:57:40.781 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:40 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:57:40.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:40 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:57:41.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:41 smithi138 ceph-mon[81376]: pgmap v111: 65 pgs: 65 active+clean; 258 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T21:57:41.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:41 smithi107 ceph-mon[93224]: pgmap v111: 65 pgs: 65 active+clean; 258 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T21:57:43.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:43 smithi138 ceph-mon[81376]: pgmap v112: 65 pgs: 65 active+clean; 258 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T21:57:43.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:43 smithi107 ceph-mon[93224]: pgmap v112: 65 pgs: 65 active+clean; 258 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T21:57:45.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:45 smithi138 ceph-mon[81376]: pgmap v113: 65 pgs: 65 active+clean; 258 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T21:57:45.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:45 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-05-17T21:57:45.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:45 smithi107 ceph-mon[93224]: pgmap v113: 65 pgs: 65 active+clean; 258 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T21:57:45.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:45 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-05-17T21:57:46.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:46 smithi138 ceph-mon[81376]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-05-17T21:57:46.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:46 smithi138 ceph-mon[81376]: Upgrade: osd.3 is safe to restart 2024-05-17T21:57:46.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:46 smithi107 ceph-mon[93224]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-05-17T21:57:46.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:46 smithi107 ceph-mon[93224]: Upgrade: osd.3 is safe to restart 2024-05-17T21:57:48.241 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:48 smithi138 ceph-mon[81376]: pgmap v114: 65 pgs: 65 active+clean; 258 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T21:57:48.241 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:48 smithi138 ceph-mon[81376]: Upgrade: Updating osd.3 2024-05-17T21:57:48.241 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:48 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:57:48.241 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:48 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-17T21:57:48.241 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:48 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:57:48.241 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:48 smithi138 ceph-mon[81376]: Deploying daemon osd.3 on smithi138 2024-05-17T21:57:48.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:48 smithi107 ceph-mon[93224]: pgmap v114: 65 pgs: 65 active+clean; 258 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T21:57:48.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:48 smithi107 ceph-mon[93224]: Upgrade: Updating osd.3 2024-05-17T21:57:48.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:48 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:57:48.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:48 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-17T21:57:48.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:48 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:57:48.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:48 smithi107 ceph-mon[93224]: Deploying daemon osd.3 on smithi138 2024-05-17T21:57:50.256 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:50 smithi138 ceph-mon[81376]: pgmap v115: 65 pgs: 65 active+clean; 258 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T21:57:50.257 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:50 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:57:50.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:50 smithi107 ceph-mon[93224]: pgmap v115: 65 pgs: 65 active+clean; 258 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T21:57:50.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:50 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:57:50.538 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 17 21:57:50 smithi138 systemd[1]: Stopping Ceph osd.3 for 3722731a-1496-11ef-bc9a-c7b262605968... 2024-05-17T21:57:50.825 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 17 21:57:50 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-3[41471]: 2024-05-17T21:57:50.546+0000 7fbc8fa15700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-17T21:57:50.825 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 17 21:57:50 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-3[41471]: 2024-05-17T21:57:50.546+0000 7fbc8fa15700 -1 osd.3 64 *** Got signal Terminated *** 2024-05-17T21:57:50.825 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 17 21:57:50 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-3[41471]: 2024-05-17T21:57:50.546+0000 7fbc8fa15700 -1 osd.3 64 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-17T21:57:51.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:51 smithi138 ceph-mon[81376]: osd.3 marked itself down and dead 2024-05-17T21:57:51.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:51 smithi107 ceph-mon[93224]: osd.3 marked itself down and dead 2024-05-17T21:57:51.772 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 17 21:57:51 smithi138 podman[90469]: 2024-05-17 21:57:51.476386322 +0000 UTC m=+1.058017919 container died 8c980cc8af876e7d6640653a1089d8874f68604bb91dc1364ffaeade8213ac4d (image=quay.ceph.io/ceph-ci/ceph@sha256:c200327b094672e93680375e82d2de1a08f5881ef304978f791cce179f847d60, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-3, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, RELEASE=quincy-9903f77, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-05-17T21:57:52.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:52 smithi138 ceph-mon[81376]: pgmap v116: 65 pgs: 65 active+clean; 258 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T21:57:52.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:52 smithi138 ceph-mon[81376]: Health check failed: 1 osds down (OSD_DOWN) 2024-05-17T21:57:52.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:52 smithi138 ceph-mon[81376]: osdmap e65: 6 total, 5 up, 6 in 2024-05-17T21:57:52.076 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 17 21:57:51 smithi138 podman[90469]: 2024-05-17 21:57:51.880696525 +0000 UTC m=+1.462328165 container cleanup 8c980cc8af876e7d6640653a1089d8874f68604bb91dc1364ffaeade8213ac4d (image=quay.ceph.io/ceph-ci/ceph@sha256:c200327b094672e93680375e82d2de1a08f5881ef304978f791cce179f847d60, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-3, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, RELEASE=quincy-9903f77, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-17T21:57:52.076 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 17 21:57:51 smithi138 bash[90469]: ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-3 2024-05-17T21:57:52.304 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:52 smithi107 ceph-mon[93224]: pgmap v116: 65 pgs: 65 active+clean; 258 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T21:57:52.304 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:52 smithi107 ceph-mon[93224]: Health check failed: 1 osds down (OSD_DOWN) 2024-05-17T21:57:52.304 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:52 smithi107 ceph-mon[93224]: osdmap e65: 6 total, 5 up, 6 in 2024-05-17T21:57:52.544 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 17 21:57:52 smithi138 podman[90483]: 2024-05-17 21:57:52.123744896 +0000 UTC m=+0.647110637 container remove 8c980cc8af876e7d6640653a1089d8874f68604bb91dc1364ffaeade8213ac4d (image=quay.ceph.io/ceph-ci/ceph@sha256:c200327b094672e93680375e82d2de1a08f5881ef304978f791cce179f847d60, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-3, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=quincy-9903f77, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-05-17T21:57:52.801 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 17 21:57:52 smithi138 podman[90568]: 2024-05-17 21:57:52.456581934 +0000 UTC m=+0.017312164 image pull 8880f7613d0283efa7002505fdccdbc3e37d3b5da31c57fa22d62262c5dab66c quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b 2024-05-17T21:57:52.801 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 17 21:57:52 smithi138 podman[90568]: 2024-05-17 21:57:52.583294369 +0000 UTC m=+0.144024599 container create 9c7604a28f15bbdd1b7aa74c7c3103c3c6a6b64b10ceb43ffcfdca3959523dae (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-3-deactivate, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-05-17T21:57:52.802 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 17 21:57:52 smithi138 podman[90568]: 2024-05-17 21:57:52.810730049 +0000 UTC m=+0.371460267 container init 9c7604a28f15bbdd1b7aa74c7c3103c3c6a6b64b10ceb43ffcfdca3959523dae (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-3-deactivate, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-17T21:57:53.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:53 smithi138 ceph-mon[81376]: osdmap e66: 6 total, 5 up, 6 in 2024-05-17T21:57:53.076 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 17 21:57:52 smithi138 podman[90568]: 2024-05-17 21:57:52.814208914 +0000 UTC m=+0.374939133 container start 9c7604a28f15bbdd1b7aa74c7c3103c3c6a6b64b10ceb43ffcfdca3959523dae (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.vendor=CentOS) 2024-05-17T21:57:53.076 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 17 21:57:52 smithi138 podman[90568]: 2024-05-17 21:57:52.902347323 +0000 UTC m=+0.463077545 container attach 9c7604a28f15bbdd1b7aa74c7c3103c3c6a6b64b10ceb43ffcfdca3959523dae (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-3-deactivate, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS) 2024-05-17T21:57:53.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:53 smithi107 ceph-mon[93224]: osdmap e66: 6 total, 5 up, 6 in 2024-05-17T21:57:53.379 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 17 21:57:53 smithi138 podman[90568]: 2024-05-17 21:57:53.106592995 +0000 UTC m=+0.667323215 container died 9c7604a28f15bbdd1b7aa74c7c3103c3c6a6b64b10ceb43ffcfdca3959523dae (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-17T21:57:54.025 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 17 21:57:53 smithi138 podman[90594]: 2024-05-17 21:57:53.672772167 +0000 UTC m=+0.559979103 container remove 9c7604a28f15bbdd1b7aa74c7c3103c3c6a6b64b10ceb43ffcfdca3959523dae (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-3-deactivate, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-05-17T21:57:54.026 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 17 21:57:53 smithi138 systemd[1]: ceph-3722731a-1496-11ef-bc9a-c7b262605968@osd.3.service: Deactivated successfully. 2024-05-17T21:57:54.026 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 17 21:57:53 smithi138 systemd[1]: Stopped Ceph osd.3 for 3722731a-1496-11ef-bc9a-c7b262605968. 2024-05-17T21:57:54.026 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 17 21:57:53 smithi138 systemd[1]: ceph-3722731a-1496-11ef-bc9a-c7b262605968@osd.3.service: Consumed 1min 1.026s CPU time. 2024-05-17T21:57:54.026 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 17 21:57:53 smithi138 systemd[1]: Starting Ceph osd.3 for 3722731a-1496-11ef-bc9a-c7b262605968... 2024-05-17T21:57:54.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:54 smithi138 ceph-mon[81376]: pgmap v119: 65 pgs: 4 peering, 15 stale+active+clean, 46 active+clean; 258 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T21:57:54.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:54 smithi138 ceph-mon[81376]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-05-17T21:57:54.326 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 17 21:57:54 smithi138 podman[90697]: 2024-05-17 21:57:54.021612201 +0000 UTC m=+0.017102216 image pull 8880f7613d0283efa7002505fdccdbc3e37d3b5da31c57fa22d62262c5dab66c quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b 2024-05-17T21:57:54.326 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 17 21:57:54 smithi138 podman[90697]: 2024-05-17 21:57:54.132132475 +0000 UTC m=+0.127622499 container create 136beb48f7e037277c0d5af8efe44fd37ca3a281fff228f57bbc63f6edb9424d (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-3-activate, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-05-17T21:57:54.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:54 smithi107 ceph-mon[93224]: pgmap v119: 65 pgs: 4 peering, 15 stale+active+clean, 46 active+clean; 258 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T21:57:54.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:54 smithi107 ceph-mon[93224]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-05-17T21:57:54.825 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 17 21:57:54 smithi138 podman[90697]: 2024-05-17 21:57:54.368218822 +0000 UTC m=+0.363708834 container init 136beb48f7e037277c0d5af8efe44fd37ca3a281fff228f57bbc63f6edb9424d (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-3-activate, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.license=GPLv2) 2024-05-17T21:57:54.825 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 17 21:57:54 smithi138 podman[90697]: 2024-05-17 21:57:54.372100963 +0000 UTC m=+0.367590966 container start 136beb48f7e037277c0d5af8efe44fd37ca3a281fff228f57bbc63f6edb9424d (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-3-activate, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-05-17T21:57:54.826 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 17 21:57:54 smithi138 podman[90697]: 2024-05-17 21:57:54.443192261 +0000 UTC m=+0.438682274 container attach 136beb48f7e037277c0d5af8efe44fd37ca3a281fff228f57bbc63f6edb9424d (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-3-activate, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux ) 2024-05-17T21:57:55.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.068+0000 7fe6ecd2a700 1 -- 172.21.15.107:0/2374863376 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe6e80fcf10 msgr2=0x7fe6e80fd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:55.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.068+0000 7fe6ecd2a700 1 --2- 172.21.15.107:0/2374863376 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe6e80fcf10 0x7fe6e80fd320 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe6d8004940 tx=0x7fe6d8009e30 comp rx=0 tx=0).stop 2024-05-17T21:57:55.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.069+0000 7fe6ecd2a700 1 -- 172.21.15.107:0/2374863376 shutdown_connections 2024-05-17T21:57:55.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.069+0000 7fe6ecd2a700 1 --2- 172.21.15.107:0/2374863376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe6e80fe080 0x7fe6e80fe4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:55.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.069+0000 7fe6ecd2a700 1 --2- 172.21.15.107:0/2374863376 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe6e80fcf10 0x7fe6e80fd320 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:55.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.069+0000 7fe6ecd2a700 1 -- 172.21.15.107:0/2374863376 >> 172.21.15.107:0/2374863376 conn(0x7fe6e80f8560 msgr2=0x7fe6e80fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:55.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.069+0000 7fe6ecd2a700 1 -- 172.21.15.107:0/2374863376 shutdown_connections 2024-05-17T21:57:55.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.069+0000 7fe6ecd2a700 1 -- 172.21.15.107:0/2374863376 wait complete. 2024-05-17T21:57:55.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.069+0000 7fe6ecd2a700 1 Processor -- start 2024-05-17T21:57:55.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.069+0000 7fe6ecd2a700 1 -- start start 2024-05-17T21:57:55.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.069+0000 7fe6ecd2a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe6e80fcf10 0x7fe6e810d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:55.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.069+0000 7fe6ecd2a700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe6e80fe080 0x7fe6e810df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:55.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.069+0000 7fe6ecd2a700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e810e520 con 0x7fe6e80fe080 2024-05-17T21:57:55.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.069+0000 7fe6ecd2a700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e810ed70 con 0x7fe6e80fcf10 2024-05-17T21:57:55.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.069+0000 7fe6e5d9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe6e80fe080 0x7fe6e810df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:55.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.070+0000 7fe6e5d9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe6e80fe080 0x7fe6e810df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47596/0 (socket says 172.21.15.107:47596) 2024-05-17T21:57:55.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.070+0000 7fe6e5d9b700 1 -- 172.21.15.107:0/3893795520 learned_addr learned my addr 172.21.15.107:0/3893795520 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:57:55.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.070+0000 7fe6e659c700 1 --2- 172.21.15.107:0/3893795520 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe6e80fcf10 0x7fe6e810d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:55.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.070+0000 7fe6e5d9b700 1 -- 172.21.15.107:0/3893795520 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe6e80fcf10 msgr2=0x7fe6e810d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:55.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.070+0000 7fe6e5d9b700 1 --2- 172.21.15.107:0/3893795520 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe6e80fcf10 0x7fe6e810d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:55.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.070+0000 7fe6e5d9b700 1 -- 172.21.15.107:0/3893795520 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6d8005040 con 0x7fe6e80fe080 2024-05-17T21:57:55.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.070+0000 7fe6e5d9b700 1 --2- 172.21.15.107:0/3893795520 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe6e80fe080 0x7fe6e810df20 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc00c8c0 tx=0x7fe6dc0044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:55.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.070+0000 7fe6d6ffd700 1 -- 172.21.15.107:0/3893795520 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe6dc00a980 con 0x7fe6e80fe080 2024-05-17T21:57:55.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.070+0000 7fe6d6ffd700 1 -- 172.21.15.107:0/3893795520 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe6dc00ce60 con 0x7fe6e80fe080 2024-05-17T21:57:55.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.070+0000 7fe6ecd2a700 1 -- 172.21.15.107:0/3893795520 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6e810f050 con 0x7fe6e80fe080 2024-05-17T21:57:55.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.070+0000 7fe6d6ffd700 1 -- 172.21.15.107:0/3893795520 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe6dc0115e0 con 0x7fe6e80fe080 2024-05-17T21:57:55.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.070+0000 7fe6ecd2a700 1 -- 172.21.15.107:0/3893795520 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6e810f590 con 0x7fe6e80fe080 2024-05-17T21:57:55.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.071+0000 7fe6ecd2a700 1 -- 172.21.15.107:0/3893795520 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6e805f5c0 con 0x7fe6e80fe080 2024-05-17T21:57:55.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.072+0000 7fe6d6ffd700 1 -- 172.21.15.107:0/3893795520 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7fe6dc012400 con 0x7fe6e80fe080 2024-05-17T21:57:55.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.073+0000 7fe6d6ffd700 1 --2- 172.21.15.107:0/3893795520 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fe6d0078990 0x7fe6d007ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:55.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.073+0000 7fe6e659c700 1 --2- 172.21.15.107:0/3893795520 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fe6d0078990 0x7fe6d007ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:55.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.073+0000 7fe6d6ffd700 1 -- 172.21.15.107:0/3893795520 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fe6dc09a750 con 0x7fe6e80fe080 2024-05-17T21:57:55.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.073+0000 7fe6e659c700 1 --2- 172.21.15.107:0/3893795520 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fe6d0078990 0x7fe6d007ae40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe6d800b3b0 tx=0x7fe6d8009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:55.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.077+0000 7fe6d6ffd700 1 -- 172.21.15.107:0/3893795520 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7fe6dc0624e0 con 0x7fe6e80fe080 2024-05-17T21:57:55.263 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 17 21:57:54 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-3-activate[90712]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-05-17T21:57:55.263 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 17 21:57:54 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-3-activate[90712]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-05-17T21:57:55.263 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 17 21:57:54 smithi138 bash[90697]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-05-17T21:57:55.263 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 17 21:57:54 smithi138 bash[90697]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-05-17T21:57:55.263 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 17 21:57:54 smithi138 bash[90697]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-05-17T21:57:55.263 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 17 21:57:54 smithi138 bash[90697]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-05-17T21:57:55.263 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 17 21:57:54 smithi138 bash[90697]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-05-17T21:57:55.263 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 17 21:57:54 smithi138 bash[90697]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-05-17T21:57:55.263 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 17 21:57:54 smithi138 bash[90697]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3/block 2024-05-17T21:57:55.263 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 17 21:57:54 smithi138 bash[90697]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-05-17T21:57:55.264 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 17 21:57:54 smithi138 bash[90697]: --> ceph-volume raw activate successful for osd ID: 3 2024-05-17T21:57:55.264 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 17 21:57:54 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-3-activate[90712]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-05-17T21:57:55.264 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 17 21:57:54 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-3-activate[90712]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-05-17T21:57:55.264 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 17 21:57:54 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-3-activate[90712]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-05-17T21:57:55.264 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 17 21:57:54 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-3-activate[90712]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-05-17T21:57:55.264 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 17 21:57:54 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-3-activate[90712]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3/block 2024-05-17T21:57:55.264 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 17 21:57:54 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-3-activate[90712]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-05-17T21:57:55.264 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 17 21:57:54 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-3-activate[90712]: --> ceph-volume raw activate successful for osd ID: 3 2024-05-17T21:57:55.264 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 17 21:57:54 smithi138 podman[90697]: 2024-05-17 21:57:54.995427828 +0000 UTC m=+0.990917831 container died 136beb48f7e037277c0d5af8efe44fd37ca3a281fff228f57bbc63f6edb9424d (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-3-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS) 2024-05-17T21:57:55.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.300+0000 7fe6ecd2a700 1 -- 172.21.15.107:0/3893795520 --> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe6e810f9d0 con 0x7fe6d0078990 2024-05-17T21:57:55.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.301+0000 7fe6d6ffd700 1 -- 172.21.15.107:0/3893795520 <== mgr.34100 v2:172.21.15.107:6800/3205268226 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fe6e810f9d0 con 0x7fe6d0078990 2024-05-17T21:57:55.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.303+0000 7fe6ecd2a700 1 -- 172.21.15.107:0/3893795520 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fe6d0078990 msgr2=0x7fe6d007ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:55.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.303+0000 7fe6ecd2a700 1 --2- 172.21.15.107:0/3893795520 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fe6d0078990 0x7fe6d007ae40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe6d800b3b0 tx=0x7fe6d8009e30 comp rx=0 tx=0).stop 2024-05-17T21:57:55.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.303+0000 7fe6ecd2a700 1 -- 172.21.15.107:0/3893795520 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe6e80fe080 msgr2=0x7fe6e810df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:55.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.303+0000 7fe6ecd2a700 1 --2- 172.21.15.107:0/3893795520 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe6e80fe080 0x7fe6e810df20 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc00c8c0 tx=0x7fe6dc0044c0 comp rx=0 tx=0).stop 2024-05-17T21:57:55.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.303+0000 7fe6ecd2a700 1 -- 172.21.15.107:0/3893795520 shutdown_connections 2024-05-17T21:57:55.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.303+0000 7fe6ecd2a700 1 --2- 172.21.15.107:0/3893795520 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fe6d0078990 0x7fe6d007ae40 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:55.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.303+0000 7fe6ecd2a700 1 --2- 172.21.15.107:0/3893795520 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe6e80fcf10 0x7fe6e810d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:55.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.303+0000 7fe6ecd2a700 1 --2- 172.21.15.107:0/3893795520 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe6e80fe080 0x7fe6e810df20 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:55.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.303+0000 7fe6ecd2a700 1 -- 172.21.15.107:0/3893795520 >> 172.21.15.107:0/3893795520 conn(0x7fe6e80f8560 msgr2=0x7fe6e80f9c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:55.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.304+0000 7fe6ecd2a700 1 -- 172.21.15.107:0/3893795520 shutdown_connections 2024-05-17T21:57:55.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.304+0000 7fe6ecd2a700 1 -- 172.21.15.107:0/3893795520 wait complete. 2024-05-17T21:57:55.306 INFO:teuthology.orchestra.run.smithi107.stdout:true 2024-05-17T21:57:55.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.407+0000 7f21cc5f3700 1 -- 172.21.15.107:0/1745291166 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f21c40fce60 msgr2=0x7f21c40fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:55.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.407+0000 7f21cc5f3700 1 --2- 172.21.15.107:0/1745291166 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f21c40fce60 0x7f21c40fd270 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f21b4004940 tx=0x7f21b4009e30 comp rx=0 tx=0).stop 2024-05-17T21:57:55.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.407+0000 7f21cc5f3700 1 -- 172.21.15.107:0/1745291166 shutdown_connections 2024-05-17T21:57:55.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.407+0000 7f21cc5f3700 1 --2- 172.21.15.107:0/1745291166 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f21c40fdfd0 0x7f21c40fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:55.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.407+0000 7f21cc5f3700 1 --2- 172.21.15.107:0/1745291166 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f21c40fce60 0x7f21c40fd270 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:55.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.407+0000 7f21cc5f3700 1 -- 172.21.15.107:0/1745291166 >> 172.21.15.107:0/1745291166 conn(0x7f21c40f8540 msgr2=0x7f21c40fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:55.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.407+0000 7f21cc5f3700 1 -- 172.21.15.107:0/1745291166 shutdown_connections 2024-05-17T21:57:55.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.407+0000 7f21cc5f3700 1 -- 172.21.15.107:0/1745291166 wait complete. 2024-05-17T21:57:55.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.408+0000 7f21cc5f3700 1 Processor -- start 2024-05-17T21:57:55.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.408+0000 7f21cc5f3700 1 -- start start 2024-05-17T21:57:55.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.408+0000 7f21cc5f3700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f21c40fce60 0x7f21c410d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:55.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.408+0000 7f21cc5f3700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f21c40fdfd0 0x7f21c410de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:55.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.408+0000 7f21cc5f3700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21c410e480 con 0x7f21c40fdfd0 2024-05-17T21:57:55.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.408+0000 7f21cc5f3700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21c410e5f0 con 0x7f21c40fce60 2024-05-17T21:57:55.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.408+0000 7f21c9b8e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f21c40fdfd0 0x7f21c410de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:55.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.408+0000 7f21c9b8e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f21c40fdfd0 0x7f21c410de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47616/0 (socket says 172.21.15.107:47616) 2024-05-17T21:57:55.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.408+0000 7f21c9b8e700 1 -- 172.21.15.107:0/3270389251 learned_addr learned my addr 172.21.15.107:0/3270389251 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:57:55.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.408+0000 7f21ca38f700 1 --2- 172.21.15.107:0/3270389251 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f21c40fce60 0x7f21c410d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:55.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.409+0000 7f21c9b8e700 1 -- 172.21.15.107:0/3270389251 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f21c40fce60 msgr2=0x7f21c410d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:55.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.409+0000 7f21c9b8e700 1 --2- 172.21.15.107:0/3270389251 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f21c40fce60 0x7f21c410d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:55.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.409+0000 7f21c9b8e700 1 -- 172.21.15.107:0/3270389251 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21b4005040 con 0x7f21c40fdfd0 2024-05-17T21:57:55.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.409+0000 7f21c9b8e700 1 --2- 172.21.15.107:0/3270389251 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f21c40fdfd0 0x7f21c410de80 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f21c0002a50 tx=0x7f21c00046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:55.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.409+0000 7f21baffd700 1 -- 172.21.15.107:0/3270389251 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f21c0007cc0 con 0x7f21c40fdfd0 2024-05-17T21:57:55.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.409+0000 7f21baffd700 1 -- 172.21.15.107:0/3270389251 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f21c000f040 con 0x7f21c40fdfd0 2024-05-17T21:57:55.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.409+0000 7f21cc5f3700 1 -- 172.21.15.107:0/3270389251 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21c4113050 con 0x7f21c40fdfd0 2024-05-17T21:57:55.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.409+0000 7f21baffd700 1 -- 172.21.15.107:0/3270389251 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f21c000da30 con 0x7f21c40fdfd0 2024-05-17T21:57:55.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.409+0000 7f21cc5f3700 1 -- 172.21.15.107:0/3270389251 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21c41135e0 con 0x7f21c40fdfd0 2024-05-17T21:57:55.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.410+0000 7f21cc5f3700 1 -- 172.21.15.107:0/3270389251 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21c404c710 con 0x7f21c40fdfd0 2024-05-17T21:57:55.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.411+0000 7f21baffd700 1 -- 172.21.15.107:0/3270389251 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f21c000db90 con 0x7f21c40fdfd0 2024-05-17T21:57:55.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.411+0000 7f21baffd700 1 --2- 172.21.15.107:0/3270389251 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f21b0078990 0x7f21b007ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:55.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.411+0000 7f21baffd700 1 -- 172.21.15.107:0/3270389251 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f21c009a480 con 0x7f21c40fdfd0 2024-05-17T21:57:55.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.411+0000 7f21ca38f700 1 --2- 172.21.15.107:0/3270389251 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f21b0078990 0x7f21b007ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:55.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.412+0000 7f21ca38f700 1 --2- 172.21.15.107:0/3270389251 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f21b0078990 0x7f21b007ae40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f21b400b970 tx=0x7f21b4009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:55.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.416+0000 7f21baffd700 1 -- 172.21.15.107:0/3270389251 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f21c00620f0 con 0x7f21c40fdfd0 2024-05-17T21:57:55.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.635+0000 7f21cc5f3700 1 -- 172.21.15.107:0/3270389251 --> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f21c4102330 con 0x7f21b0078990 2024-05-17T21:57:55.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.636+0000 7f21baffd700 1 -- 172.21.15.107:0/3270389251 <== mgr.34100 v2:172.21.15.107:6800/3205268226 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f21c4102330 con 0x7f21b0078990 2024-05-17T21:57:55.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.639+0000 7f21cc5f3700 1 -- 172.21.15.107:0/3270389251 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f21b0078990 msgr2=0x7f21b007ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:55.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.639+0000 7f21cc5f3700 1 --2- 172.21.15.107:0/3270389251 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f21b0078990 0x7f21b007ae40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f21b400b970 tx=0x7f21b4009e30 comp rx=0 tx=0).stop 2024-05-17T21:57:55.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.639+0000 7f21cc5f3700 1 -- 172.21.15.107:0/3270389251 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f21c40fdfd0 msgr2=0x7f21c410de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:55.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.639+0000 7f21cc5f3700 1 --2- 172.21.15.107:0/3270389251 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f21c40fdfd0 0x7f21c410de80 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f21c0002a50 tx=0x7f21c00046b0 comp rx=0 tx=0).stop 2024-05-17T21:57:55.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.639+0000 7f21cc5f3700 1 -- 172.21.15.107:0/3270389251 shutdown_connections 2024-05-17T21:57:55.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.639+0000 7f21cc5f3700 1 --2- 172.21.15.107:0/3270389251 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f21b0078990 0x7f21b007ae40 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:55.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.639+0000 7f21cc5f3700 1 --2- 172.21.15.107:0/3270389251 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f21c40fce60 0x7f21c410d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:55.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.639+0000 7f21cc5f3700 1 --2- 172.21.15.107:0/3270389251 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f21c40fdfd0 0x7f21c410de80 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:55.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.639+0000 7f21cc5f3700 1 -- 172.21.15.107:0/3270389251 >> 172.21.15.107:0/3270389251 conn(0x7f21c40f8540 msgr2=0x7f21c40f9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:55.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.639+0000 7f21cc5f3700 1 -- 172.21.15.107:0/3270389251 shutdown_connections 2024-05-17T21:57:55.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.639+0000 7f21cc5f3700 1 -- 172.21.15.107:0/3270389251 wait complete. 2024-05-17T21:57:55.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.741+0000 7f4e6afb6700 1 -- 172.21.15.107:0/2352444101 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4e64066ed0 msgr2=0x7f4e640ffa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:55.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.741+0000 7f4e6afb6700 1 --2- 172.21.15.107:0/2352444101 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4e64066ed0 0x7f4e640ffa10 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4e58004940 tx=0x7f4e58009e30 comp rx=0 tx=0).stop 2024-05-17T21:57:55.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.741+0000 7f4e6afb6700 1 -- 172.21.15.107:0/2352444101 shutdown_connections 2024-05-17T21:57:55.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.741+0000 7f4e6afb6700 1 --2- 172.21.15.107:0/2352444101 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e640fff50 0x7f4e64102330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:55.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.741+0000 7f4e6afb6700 1 --2- 172.21.15.107:0/2352444101 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4e64066ed0 0x7f4e640ffa10 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:55.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.741+0000 7f4e6afb6700 1 -- 172.21.15.107:0/2352444101 >> 172.21.15.107:0/2352444101 conn(0x7f4e640f7440 msgr2=0x7f4e640f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:55.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.741+0000 7f4e6afb6700 1 -- 172.21.15.107:0/2352444101 shutdown_connections 2024-05-17T21:57:55.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.742+0000 7f4e6afb6700 1 -- 172.21.15.107:0/2352444101 wait complete. 2024-05-17T21:57:55.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.742+0000 7f4e6afb6700 1 Processor -- start 2024-05-17T21:57:55.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.742+0000 7f4e6afb6700 1 -- start start 2024-05-17T21:57:55.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.743+0000 7f4e6afb6700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e64066ed0 0x7f4e6410d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:55.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.743+0000 7f4e6afb6700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4e640fff50 0x7f4e6410de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:55.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.743+0000 7f4e6afb6700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e6410e500 con 0x7f4e640fff50 2024-05-17T21:57:55.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.743+0000 7f4e6afb6700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e64112ea0 con 0x7f4e64066ed0 2024-05-17T21:57:55.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.743+0000 7f4e63fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4e640fff50 0x7f4e6410de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:55.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.743+0000 7f4e63fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4e640fff50 0x7f4e6410de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42862/0 (socket says 172.21.15.107:42862) 2024-05-17T21:57:55.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.743+0000 7f4e63fff700 1 -- 172.21.15.107:0/1272755644 learned_addr learned my addr 172.21.15.107:0/1272755644 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:57:55.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.743+0000 7f4e68d52700 1 --2- 172.21.15.107:0/1272755644 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e64066ed0 0x7f4e6410d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:55.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.743+0000 7f4e63fff700 1 -- 172.21.15.107:0/1272755644 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e64066ed0 msgr2=0x7f4e6410d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:55.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.743+0000 7f4e63fff700 1 --2- 172.21.15.107:0/1272755644 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e64066ed0 0x7f4e6410d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:55.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.743+0000 7f4e63fff700 1 -- 172.21.15.107:0/1272755644 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e58005040 con 0x7f4e640fff50 2024-05-17T21:57:55.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.743+0000 7f4e68d52700 1 --2- 172.21.15.107:0/1272755644 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e64066ed0 0x7f4e6410d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T21:57:55.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.743+0000 7f4e63fff700 1 --2- 172.21.15.107:0/1272755644 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4e640fff50 0x7f4e6410de70 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f4e5000b790 tx=0x7f4e50009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:55.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.744+0000 7f4e617fa700 1 -- 172.21.15.107:0/1272755644 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4e5000f770 con 0x7f4e640fff50 2024-05-17T21:57:55.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.744+0000 7f4e617fa700 1 -- 172.21.15.107:0/1272755644 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4e5000fda0 con 0x7f4e640fff50 2024-05-17T21:57:55.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.744+0000 7f4e6afb6700 1 -- 172.21.15.107:0/1272755644 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e64113100 con 0x7f4e640fff50 2024-05-17T21:57:55.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.744+0000 7f4e6afb6700 1 -- 172.21.15.107:0/1272755644 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e64113670 con 0x7f4e640fff50 2024-05-17T21:57:55.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.744+0000 7f4e617fa700 1 -- 172.21.15.107:0/1272755644 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4e5000d310 con 0x7f4e640fff50 2024-05-17T21:57:55.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.745+0000 7f4e6afb6700 1 -- 172.21.15.107:0/1272755644 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e480052e0 con 0x7f4e640fff50 2024-05-17T21:57:55.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.746+0000 7f4e617fa700 1 -- 172.21.15.107:0/1272755644 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f4e5000d4f0 con 0x7f4e640fff50 2024-05-17T21:57:55.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.746+0000 7f4e617fa700 1 --2- 172.21.15.107:0/1272755644 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f4e54078990 0x7f4e5407ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:55.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.746+0000 7f4e68d52700 1 --2- 172.21.15.107:0/1272755644 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f4e54078990 0x7f4e5407ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:55.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.746+0000 7f4e617fa700 1 -- 172.21.15.107:0/1272755644 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f4e5009a550 con 0x7f4e640fff50 2024-05-17T21:57:55.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.747+0000 7f4e68d52700 1 --2- 172.21.15.107:0/1272755644 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f4e54078990 0x7f4e5407ae40 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4e5800b970 tx=0x7f4e58009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:55.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.751+0000 7f4e617fa700 1 -- 172.21.15.107:0/1272755644 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f4e500622e0 con 0x7f4e640fff50 2024-05-17T21:57:55.940 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 17 21:57:55 smithi138 podman[90838]: 2024-05-17 21:57:55.605770214 +0000 UTC m=+0.604934337 container remove 136beb48f7e037277c0d5af8efe44fd37ca3a281fff228f57bbc63f6edb9424d (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-3-activate, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=) 2024-05-17T21:57:55.940 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 17 21:57:55 smithi138 podman[90897]: 2024-05-17 21:57:55.720598673 +0000 UTC m=+0.017292374 image pull 8880f7613d0283efa7002505fdccdbc3e37d3b5da31c57fa22d62262c5dab66c quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b 2024-05-17T21:57:55.940 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 17 21:57:55 smithi138 podman[90897]: 2024-05-17 21:57:55.840137165 +0000 UTC m=+0.136830915 container create 416273d327c766b25b5e166104435102863eb6853763c87d19268c8211466a59 (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-05-17T21:57:55.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.963+0000 7f4e6afb6700 1 -- 172.21.15.107:0/1272755644 --> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f4e48000bf0 con 0x7f4e54078990 2024-05-17T21:57:55.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.973+0000 7f4e617fa700 1 -- 172.21.15.107:0/1272755644 <== mgr.34100 v2:172.21.15.107:6800/3205268226 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f4e48000bf0 con 0x7f4e54078990 2024-05-17T21:57:55.965 INFO:teuthology.orchestra.run.smithi107.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-17T21:57:55.965 INFO:teuthology.orchestra.run.smithi107.stdout:alertmanager.smithi107 smithi107 *:9093,9094 running (5m) 30s ago 14m 24.3M - 0.25.0 c8568f914cd2 bac2a944ae42 2024-05-17T21:57:55.965 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi107 smithi107 running (2m) 30s ago 14m 6669k - 19.0.0-3794-g931809a9 8880f7613d02 face35de205b 2024-05-17T21:57:55.965 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi138 smithi138 running (2m) 2m ago 13m 6665k - 19.0.0-3794-g931809a9 8880f7613d02 ad59e33e6d43 2024-05-17T21:57:55.965 INFO:teuthology.orchestra.run.smithi107.stdout:grafana.smithi107 smithi107 *:3000 running (5m) 30s ago 13m 84.2M - 9.4.7 954c08fa6188 8c747e777bc5 2024-05-17T21:57:55.966 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.drqody smithi107 running (8m) 30s ago 8m 103M - 17.2.7-952-g9903f776 d15a36853796 0d851bc912f0 2024-05-17T21:57:55.966 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.idmbgj smithi107 running (8m) 30s ago 8m 13.8M - 17.2.7-952-g9903f776 d15a36853796 1348b09ab29e 2024-05-17T21:57:55.966 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi138.gwbtyd smithi138 running (8m) 2m ago 8m 152M - 17.2.7-952-g9903f776 d15a36853796 fd2ba75354c0 2024-05-17T21:57:55.966 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi138.zckrvs smithi138 running (8m) 2m ago 8m 14.1M - 17.2.7-952-g9903f776 d15a36853796 a1cfd2fe4cda 2024-05-17T21:57:55.966 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi107.xhhyiw smithi107 *:8443,9283,8765 running (4m) 30s ago 15m 587M - 19.0.0-3794-g931809a9 8880f7613d02 5acf3e511342 2024-05-17T21:57:55.966 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi138.napnll smithi138 *:8443,9283,8765 running (3m) 2m ago 13m 531M - 19.0.0-3794-g931809a9 8880f7613d02 d627761dbd0d 2024-05-17T21:57:55.966 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi107 smithi107 running (3m) 30s ago 15m 115M 2048M 19.0.0-3794-g931809a9 8880f7613d02 9717f3169f2f 2024-05-17T21:57:55.966 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi138 smithi138 running (2m) 2m ago 12m 99.9M 2048M 19.0.0-3794-g931809a9 8880f7613d02 55aeed62bf4d 2024-05-17T21:57:55.966 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi107 smithi107 *:9100 running (5m) 30s ago 14m 17.4M - 1.5.0 0da6a335fe13 49bec71f5487 2024-05-17T21:57:55.966 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi138 smithi138 *:9100 running (5m) 2m ago 13m 15.9M - 1.5.0 0da6a335fe13 e52824376658 2024-05-17T21:57:55.966 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0 smithi107 running (107s) 30s ago 12m 156M 1557M 19.0.0-3794-g931809a9 8880f7613d02 b319bc93500d 2024-05-17T21:57:55.966 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1 smithi107 running (71s) 30s ago 11m 151M 1557M 19.0.0-3794-g931809a9 8880f7613d02 f94b206be6eb 2024-05-17T21:57:55.966 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2 smithi107 running (36s) 30s ago 11m 79.3M 1557M 19.0.0-3794-g931809a9 8880f7613d02 add7015647af 2024-05-17T21:57:55.967 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3 smithi138 running (10m) 2m ago 10m 228M 2581M 17.2.7-952-g9903f776 d15a36853796 8c980cc8af87 2024-05-17T21:57:55.967 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4 smithi138 running (10m) 2m ago 10m 204M 2581M 17.2.7-952-g9903f776 d15a36853796 b306a36a744e 2024-05-17T21:57:55.967 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5 smithi138 running (10m) 2m ago 10m 190M 2581M 17.2.7-952-g9903f776 d15a36853796 7c8d8ed74974 2024-05-17T21:57:55.967 INFO:teuthology.orchestra.run.smithi107.stdout:prometheus.smithi107 smithi107 *:9095 running (4m) 30s ago 13m 59.1M - 2.43.0 a07b618ecd1d bf5f609bbd6e 2024-05-17T21:57:55.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.975+0000 7f4e6afb6700 1 -- 172.21.15.107:0/1272755644 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f4e54078990 msgr2=0x7f4e5407ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:55.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.975+0000 7f4e6afb6700 1 --2- 172.21.15.107:0/1272755644 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f4e54078990 0x7f4e5407ae40 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4e5800b970 tx=0x7f4e58009e30 comp rx=0 tx=0).stop 2024-05-17T21:57:55.968 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.975+0000 7f4e6afb6700 1 -- 172.21.15.107:0/1272755644 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4e640fff50 msgr2=0x7f4e6410de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:55.968 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.975+0000 7f4e6afb6700 1 --2- 172.21.15.107:0/1272755644 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4e640fff50 0x7f4e6410de70 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f4e5000b790 tx=0x7f4e50009250 comp rx=0 tx=0).stop 2024-05-17T21:57:55.968 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.976+0000 7f4e6afb6700 1 -- 172.21.15.107:0/1272755644 shutdown_connections 2024-05-17T21:57:55.968 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.976+0000 7f4e6afb6700 1 --2- 172.21.15.107:0/1272755644 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f4e54078990 0x7f4e5407ae40 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:55.968 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.976+0000 7f4e6afb6700 1 --2- 172.21.15.107:0/1272755644 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e64066ed0 0x7f4e6410d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:55.968 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.976+0000 7f4e6afb6700 1 --2- 172.21.15.107:0/1272755644 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4e640fff50 0x7f4e6410de70 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:55.968 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.976+0000 7f4e6afb6700 1 -- 172.21.15.107:0/1272755644 >> 172.21.15.107:0/1272755644 conn(0x7f4e640f7440 msgr2=0x7f4e640f7810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:55.968 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.976+0000 7f4e6afb6700 1 -- 172.21.15.107:0/1272755644 shutdown_connections 2024-05-17T21:57:55.968 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:55.976+0000 7f4e6afb6700 1 -- 172.21.15.107:0/1272755644 wait complete. 2024-05-17T21:57:56.072 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.078+0000 7ffb2f1ec700 1 -- 172.21.15.107:0/2782212351 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb280fce60 msgr2=0x7ffb280fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:56.072 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.078+0000 7ffb2f1ec700 1 --2- 172.21.15.107:0/2782212351 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb280fce60 0x7ffb280fd270 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c004940 tx=0x7ffb1c009e30 comp rx=0 tx=0).stop 2024-05-17T21:57:56.072 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.079+0000 7ffb2f1ec700 1 -- 172.21.15.107:0/2782212351 shutdown_connections 2024-05-17T21:57:56.072 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.079+0000 7ffb2f1ec700 1 --2- 172.21.15.107:0/2782212351 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb280fdfd0 0x7ffb280fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:56.072 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.079+0000 7ffb2f1ec700 1 --2- 172.21.15.107:0/2782212351 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb280fce60 0x7ffb280fd270 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:56.072 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.079+0000 7ffb2f1ec700 1 -- 172.21.15.107:0/2782212351 >> 172.21.15.107:0/2782212351 conn(0x7ffb280f8540 msgr2=0x7ffb280fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:56.072 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.079+0000 7ffb2f1ec700 1 -- 172.21.15.107:0/2782212351 shutdown_connections 2024-05-17T21:57:56.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.079+0000 7ffb2f1ec700 1 -- 172.21.15.107:0/2782212351 wait complete. 2024-05-17T21:57:56.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.079+0000 7ffb2f1ec700 1 Processor -- start 2024-05-17T21:57:56.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.079+0000 7ffb2f1ec700 1 -- start start 2024-05-17T21:57:56.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.079+0000 7ffb2f1ec700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb280fce60 0x7ffb2810d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:56.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.079+0000 7ffb2f1ec700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb280fdfd0 0x7ffb2810ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:56.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.079+0000 7ffb2f1ec700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb2810e3f0 con 0x7ffb280fce60 2024-05-17T21:57:56.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.079+0000 7ffb2f1ec700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb2810e560 con 0x7ffb280fdfd0 2024-05-17T21:57:56.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.080+0000 7ffb2cf88700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb280fce60 0x7ffb2810d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:56.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.080+0000 7ffb2cf88700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb280fce60 0x7ffb2810d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42882/0 (socket says 172.21.15.107:42882) 2024-05-17T21:57:56.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.080+0000 7ffb2cf88700 1 -- 172.21.15.107:0/3346372326 learned_addr learned my addr 172.21.15.107:0/3346372326 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:57:56.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.080+0000 7ffb27fff700 1 --2- 172.21.15.107:0/3346372326 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb280fdfd0 0x7ffb2810ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:56.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.080+0000 7ffb2cf88700 1 -- 172.21.15.107:0/3346372326 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb280fdfd0 msgr2=0x7ffb2810ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:56.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.080+0000 7ffb2cf88700 1 --2- 172.21.15.107:0/3346372326 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb280fdfd0 0x7ffb2810ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:56.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.080+0000 7ffb2cf88700 1 -- 172.21.15.107:0/3346372326 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb1c005040 con 0x7ffb280fce60 2024-05-17T21:57:56.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.080+0000 7ffb2cf88700 1 --2- 172.21.15.107:0/3346372326 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb280fce60 0x7ffb2810d8b0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c013870 tx=0x7ffb1c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:56.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.080+0000 7ffb257fa700 1 -- 172.21.15.107:0/3346372326 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffb1c021070 con 0x7ffb280fce60 2024-05-17T21:57:56.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.080+0000 7ffb257fa700 1 -- 172.21.15.107:0/3346372326 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffb1c009430 con 0x7ffb280fce60 2024-05-17T21:57:56.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.080+0000 7ffb2f1ec700 1 -- 172.21.15.107:0/3346372326 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb28112f60 con 0x7ffb280fce60 2024-05-17T21:57:56.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.080+0000 7ffb257fa700 1 -- 172.21.15.107:0/3346372326 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffb1c00e070 con 0x7ffb280fce60 2024-05-17T21:57:56.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.080+0000 7ffb2f1ec700 1 -- 172.21.15.107:0/3346372326 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb28113410 con 0x7ffb280fce60 2024-05-17T21:57:56.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.081+0000 7ffb2f1ec700 1 -- 172.21.15.107:0/3346372326 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb2804c710 con 0x7ffb280fce60 2024-05-17T21:57:56.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.082+0000 7ffb257fa700 1 -- 172.21.15.107:0/3346372326 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7ffb1c01c750 con 0x7ffb280fce60 2024-05-17T21:57:56.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.082+0000 7ffb257fa700 1 --2- 172.21.15.107:0/3346372326 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7ffb18078920 0x7ffb1807add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:56.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.082+0000 7ffb257fa700 1 -- 172.21.15.107:0/3346372326 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6410+0+0 (secure 0 0 0) 0x7ffb1c09b8b0 con 0x7ffb280fce60 2024-05-17T21:57:56.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.082+0000 7ffb27fff700 1 --2- 172.21.15.107:0/3346372326 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7ffb18078920 0x7ffb1807add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:56.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.083+0000 7ffb27fff700 1 --2- 172.21.15.107:0/3346372326 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7ffb18078920 0x7ffb1807add0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ffb14009b80 tx=0x7ffb1400b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:56.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.087+0000 7ffb257fa700 1 -- 172.21.15.107:0/3346372326 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7ffb1c0635a0 con 0x7ffb280fce60 2024-05-17T21:57:56.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:56 smithi138 ceph-mon[81376]: pgmap v120: 65 pgs: 4 peering, 15 stale+active+clean, 46 active+clean; 258 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s 2024-05-17T21:57:56.325 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 17 21:57:56 smithi138 podman[90897]: 2024-05-17 21:57:56.025849775 +0000 UTC m=+0.322543476 container init 416273d327c766b25b5e166104435102863eb6853763c87d19268c8211466a59 (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-3, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, ceph=True) 2024-05-17T21:57:56.325 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 17 21:57:56 smithi138 podman[90897]: 2024-05-17 21:57:56.029047126 +0000 UTC m=+0.325740840 container start 416273d327c766b25b5e166104435102863eb6853763c87d19268c8211466a59 (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-3, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514) 2024-05-17T21:57:56.325 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 17 21:57:56 smithi138 bash[90897]: 416273d327c766b25b5e166104435102863eb6853763c87d19268c8211466a59 2024-05-17T21:57:56.326 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 17 21:57:56 smithi138 systemd[1]: Started Ceph osd.3 for 3722731a-1496-11ef-bc9a-c7b262605968. 2024-05-17T21:57:56.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:56 smithi107 ceph-mon[93224]: pgmap v120: 65 pgs: 4 peering, 15 stale+active+clean, 46 active+clean; 258 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s 2024-05-17T21:57:56.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.382+0000 7ffb2f1ec700 1 -- 172.21.15.107:0/3346372326 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ffb28113740 con 0x7ffb280fce60 2024-05-17T21:57:56.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.383+0000 7ffb257fa700 1 -- 172.21.15.107:0/3346372326 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+839 (secure 0 0 0) 0x7ffb1c062f40 con 0x7ffb280fce60 2024-05-17T21:57:56.375 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-05-17T21:57:56.375 INFO:teuthology.orchestra.run.smithi107.stdout: "mon": { 2024-05-17T21:57:56.375 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev)": 2 2024-05-17T21:57:56.375 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-05-17T21:57:56.375 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr": { 2024-05-17T21:57:56.375 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev)": 2 2024-05-17T21:57:56.375 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-05-17T21:57:56.375 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": { 2024-05-17T21:57:56.375 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-952-g9903f776 (9903f7767386a9d9efadb7c1effdcd3064d71804) quincy (stable)": 2, 2024-05-17T21:57:56.375 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev)": 3 2024-05-17T21:57:56.375 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-05-17T21:57:56.376 INFO:teuthology.orchestra.run.smithi107.stdout: "mds": { 2024-05-17T21:57:56.376 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-952-g9903f776 (9903f7767386a9d9efadb7c1effdcd3064d71804) quincy (stable)": 4 2024-05-17T21:57:56.376 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-05-17T21:57:56.376 INFO:teuthology.orchestra.run.smithi107.stdout: "overall": { 2024-05-17T21:57:56.376 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-952-g9903f776 (9903f7767386a9d9efadb7c1effdcd3064d71804) quincy (stable)": 6, 2024-05-17T21:57:56.376 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev)": 7 2024-05-17T21:57:56.376 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-05-17T21:57:56.376 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-05-17T21:57:56.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.385+0000 7ffb2f1ec700 1 -- 172.21.15.107:0/3346372326 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7ffb18078920 msgr2=0x7ffb1807add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:56.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.385+0000 7ffb2f1ec700 1 --2- 172.21.15.107:0/3346372326 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7ffb18078920 0x7ffb1807add0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ffb14009b80 tx=0x7ffb1400b040 comp rx=0 tx=0).stop 2024-05-17T21:57:56.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.385+0000 7ffb2f1ec700 1 -- 172.21.15.107:0/3346372326 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb280fce60 msgr2=0x7ffb2810d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:56.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.385+0000 7ffb2f1ec700 1 --2- 172.21.15.107:0/3346372326 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb280fce60 0x7ffb2810d8b0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c013870 tx=0x7ffb1c009e30 comp rx=0 tx=0).stop 2024-05-17T21:57:56.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.385+0000 7ffb2f1ec700 1 -- 172.21.15.107:0/3346372326 shutdown_connections 2024-05-17T21:57:56.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.385+0000 7ffb2f1ec700 1 --2- 172.21.15.107:0/3346372326 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7ffb18078920 0x7ffb1807add0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:56.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.385+0000 7ffb2f1ec700 1 --2- 172.21.15.107:0/3346372326 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb280fce60 0x7ffb2810d8b0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:56.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.385+0000 7ffb2f1ec700 1 --2- 172.21.15.107:0/3346372326 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb280fdfd0 0x7ffb2810ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:56.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.385+0000 7ffb2f1ec700 1 -- 172.21.15.107:0/3346372326 >> 172.21.15.107:0/3346372326 conn(0x7ffb280f8540 msgr2=0x7ffb280f9ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:56.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.385+0000 7ffb2f1ec700 1 -- 172.21.15.107:0/3346372326 shutdown_connections 2024-05-17T21:57:56.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.385+0000 7ffb2f1ec700 1 -- 172.21.15.107:0/3346372326 wait complete. 2024-05-17T21:57:56.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.488+0000 7fb9c96c9700 1 -- 172.21.15.107:0/4135831210 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9c40fce60 msgr2=0x7fb9c40fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:56.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.488+0000 7fb9c96c9700 1 --2- 172.21.15.107:0/4135831210 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9c40fce60 0x7fb9c40fd270 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb9b4004940 tx=0x7fb9b4009e30 comp rx=0 tx=0).stop 2024-05-17T21:57:56.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.488+0000 7fb9c96c9700 1 -- 172.21.15.107:0/4135831210 shutdown_connections 2024-05-17T21:57:56.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.488+0000 7fb9c96c9700 1 --2- 172.21.15.107:0/4135831210 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9c40fdfd0 0x7fb9c40fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:56.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.488+0000 7fb9c96c9700 1 --2- 172.21.15.107:0/4135831210 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9c40fce60 0x7fb9c40fd270 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:56.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.488+0000 7fb9c96c9700 1 -- 172.21.15.107:0/4135831210 >> 172.21.15.107:0/4135831210 conn(0x7fb9c40f8540 msgr2=0x7fb9c40fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:56.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.488+0000 7fb9c96c9700 1 -- 172.21.15.107:0/4135831210 shutdown_connections 2024-05-17T21:57:56.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.488+0000 7fb9c96c9700 1 -- 172.21.15.107:0/4135831210 wait complete. 2024-05-17T21:57:56.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.489+0000 7fb9c96c9700 1 Processor -- start 2024-05-17T21:57:56.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.489+0000 7fb9c96c9700 1 -- start start 2024-05-17T21:57:56.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.489+0000 7fb9c96c9700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9c40fce60 0x7fb9c4071c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:56.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.489+0000 7fb9c96c9700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9c40fdfd0 0x7fb9c40702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:56.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.489+0000 7fb9c96c9700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c4072200 con 0x7fb9c40fdfd0 2024-05-17T21:57:56.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.489+0000 7fb9c96c9700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c40707f0 con 0x7fb9c40fce60 2024-05-17T21:57:56.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.489+0000 7fb9c27fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9c40fdfd0 0x7fb9c40702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:56.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.489+0000 7fb9c27fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9c40fdfd0 0x7fb9c40702b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42914/0 (socket says 172.21.15.107:42914) 2024-05-17T21:57:56.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.489+0000 7fb9c27fc700 1 -- 172.21.15.107:0/1904042701 learned_addr learned my addr 172.21.15.107:0/1904042701 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:57:56.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.490+0000 7fb9c2ffd700 1 --2- 172.21.15.107:0/1904042701 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9c40fce60 0x7fb9c4071c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:56.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.490+0000 7fb9c27fc700 1 -- 172.21.15.107:0/1904042701 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9c40fce60 msgr2=0x7fb9c4071c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:56.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.490+0000 7fb9c27fc700 1 --2- 172.21.15.107:0/1904042701 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9c40fce60 0x7fb9c4071c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:56.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.490+0000 7fb9c27fc700 1 -- 172.21.15.107:0/1904042701 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9b4005040 con 0x7fb9c40fdfd0 2024-05-17T21:57:56.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.490+0000 7fb9c27fc700 1 --2- 172.21.15.107:0/1904042701 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9c40fdfd0 0x7fb9c40702b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb9b800eb40 tx=0x7fb9b800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:56.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.490+0000 7fb9ab7fe700 1 -- 172.21.15.107:0/1904042701 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb9b8007cc0 con 0x7fb9c40fdfd0 2024-05-17T21:57:56.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.490+0000 7fb9ab7fe700 1 -- 172.21.15.107:0/1904042701 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb9b800f040 con 0x7fb9c40fdfd0 2024-05-17T21:57:56.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.490+0000 7fb9c96c9700 1 -- 172.21.15.107:0/1904042701 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9c4070a40 con 0x7fb9c40fdfd0 2024-05-17T21:57:56.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.490+0000 7fb9ab7fe700 1 -- 172.21.15.107:0/1904042701 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb9b800ca10 con 0x7fb9c40fdfd0 2024-05-17T21:57:56.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.490+0000 7fb9c96c9700 1 -- 172.21.15.107:0/1904042701 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9c4070f80 con 0x7fb9c40fdfd0 2024-05-17T21:57:56.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.491+0000 7fb9c96c9700 1 -- 172.21.15.107:0/1904042701 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9c404c710 con 0x7fb9c40fdfd0 2024-05-17T21:57:56.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.492+0000 7fb9ab7fe700 1 -- 172.21.15.107:0/1904042701 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7fb9b800cc80 con 0x7fb9c40fdfd0 2024-05-17T21:57:56.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.492+0000 7fb9ab7fe700 1 --2- 172.21.15.107:0/1904042701 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fb9ac078990 0x7fb9ac07ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:56.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.493+0000 7fb9c2ffd700 1 --2- 172.21.15.107:0/1904042701 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fb9ac078990 0x7fb9ac07ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:56.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.493+0000 7fb9ab7fe700 1 -- 172.21.15.107:0/1904042701 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fb9b809a320 con 0x7fb9c40fdfd0 2024-05-17T21:57:56.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.493+0000 7fb9c2ffd700 1 --2- 172.21.15.107:0/1904042701 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fb9ac078990 0x7fb9ac07ae40 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb9b4009b30 tx=0x7fb9b4009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:56.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.497+0000 7fb9ab7fe700 1 -- 172.21.15.107:0/1904042701 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7fb9b8061fe0 con 0x7fb9c40fdfd0 2024-05-17T21:57:56.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.748+0000 7fb9c96c9700 1 -- 172.21.15.107:0/1904042701 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fb9c41058c0 con 0x7fb9c40fdfd0 2024-05-17T21:57:56.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.749+0000 7fb9ab7fe700 1 -- 172.21.15.107:0/1904042701 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1910 (secure 0 0 0) 0x7fb9b8061980 con 0x7fb9c40fdfd0 2024-05-17T21:57:56.741 INFO:teuthology.orchestra.run.smithi107.stdout:e15 2024-05-17T21:57:56.741 INFO:teuthology.orchestra.run.smithi107.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-17T21:57:56.741 INFO:teuthology.orchestra.run.smithi107.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-17T21:57:56.741 INFO:teuthology.orchestra.run.smithi107.stdout:legacy client fscid: 1 2024-05-17T21:57:56.741 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:57:56.741 INFO:teuthology.orchestra.run.smithi107.stdout:Filesystem 'cephfs' (1) 2024-05-17T21:57:56.741 INFO:teuthology.orchestra.run.smithi107.stdout:fs_name cephfs 2024-05-17T21:57:56.741 INFO:teuthology.orchestra.run.smithi107.stdout:epoch 15 2024-05-17T21:57:56.742 INFO:teuthology.orchestra.run.smithi107.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-17T21:57:56.742 INFO:teuthology.orchestra.run.smithi107.stdout:created 2024-05-17T21:49:16.836333+0000 2024-05-17T21:57:56.742 INFO:teuthology.orchestra.run.smithi107.stdout:modified 2024-05-17T21:49:34.873628+0000 2024-05-17T21:57:56.742 INFO:teuthology.orchestra.run.smithi107.stdout:tableserver 0 2024-05-17T21:57:56.742 INFO:teuthology.orchestra.run.smithi107.stdout:root 0 2024-05-17T21:57:56.742 INFO:teuthology.orchestra.run.smithi107.stdout:session_timeout 60 2024-05-17T21:57:56.742 INFO:teuthology.orchestra.run.smithi107.stdout:session_autoclose 300 2024-05-17T21:57:56.742 INFO:teuthology.orchestra.run.smithi107.stdout:max_file_size 1099511627776 2024-05-17T21:57:56.742 INFO:teuthology.orchestra.run.smithi107.stdout:max_xattr_size 65536 2024-05-17T21:57:56.742 INFO:teuthology.orchestra.run.smithi107.stdout:required_client_features {} 2024-05-17T21:57:56.742 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure 0 2024-05-17T21:57:56.742 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure_osd_epoch 46 2024-05-17T21:57:56.742 INFO:teuthology.orchestra.run.smithi107.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-17T21:57:56.742 INFO:teuthology.orchestra.run.smithi107.stdout:max_mds 1 2024-05-17T21:57:56.742 INFO:teuthology.orchestra.run.smithi107.stdout:in 0 2024-05-17T21:57:56.743 INFO:teuthology.orchestra.run.smithi107.stdout:up {0=24259} 2024-05-17T21:57:56.743 INFO:teuthology.orchestra.run.smithi107.stdout:failed 2024-05-17T21:57:56.743 INFO:teuthology.orchestra.run.smithi107.stdout:damaged 2024-05-17T21:57:56.743 INFO:teuthology.orchestra.run.smithi107.stdout:stopped 2024-05-17T21:57:56.743 INFO:teuthology.orchestra.run.smithi107.stdout:data_pools [3] 2024-05-17T21:57:56.743 INFO:teuthology.orchestra.run.smithi107.stdout:metadata_pool 2 2024-05-17T21:57:56.743 INFO:teuthology.orchestra.run.smithi107.stdout:inline_data enabled 2024-05-17T21:57:56.743 INFO:teuthology.orchestra.run.smithi107.stdout:balancer 2024-05-17T21:57:56.743 INFO:teuthology.orchestra.run.smithi107.stdout:bal_rank_mask -1 2024-05-17T21:57:56.743 INFO:teuthology.orchestra.run.smithi107.stdout:standby_count_wanted 1 2024-05-17T21:57:56.743 INFO:teuthology.orchestra.run.smithi107.stdout:qdb_cluster leader: 0 members: 2024-05-17T21:57:56.743 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi138.gwbtyd{0:24259} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.138:6824/2779634919,v1:172.21.15.138:6825/2779634919] compat {c=[1],r=[1],i=[7ff]}] 2024-05-17T21:57:56.743 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.drqody{0:14486} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.107:6828/2275807100,v1:172.21.15.107:6829/2275807100] compat {c=[1],r=[1],i=[7ff]}] 2024-05-17T21:57:56.743 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:57:56.743 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:57:56.744 INFO:teuthology.orchestra.run.smithi107.stdout:Standby daemons: 2024-05-17T21:57:56.744 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:57:56.744 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.idmbgj{-1:14498} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.107:6826/1460186079,v1:172.21.15.107:6827/1460186079] compat {c=[1],r=[1],i=[7ff]}] 2024-05-17T21:57:56.744 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi138.zckrvs{-1:24267} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.138:6826/170628661,v1:172.21.15.138:6827/170628661] compat {c=[1],r=[1],i=[7ff]}] 2024-05-17T21:57:56.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.753+0000 7fb9c96c9700 1 -- 172.21.15.107:0/1904042701 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fb9ac078990 msgr2=0x7fb9ac07ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:56.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.753+0000 7fb9c96c9700 1 --2- 172.21.15.107:0/1904042701 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fb9ac078990 0x7fb9ac07ae40 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb9b4009b30 tx=0x7fb9b4009e30 comp rx=0 tx=0).stop 2024-05-17T21:57:56.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.753+0000 7fb9c96c9700 1 -- 172.21.15.107:0/1904042701 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9c40fdfd0 msgr2=0x7fb9c40702b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:56.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.753+0000 7fb9c96c9700 1 --2- 172.21.15.107:0/1904042701 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9c40fdfd0 0x7fb9c40702b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb9b800eb40 tx=0x7fb9b800b040 comp rx=0 tx=0).stop 2024-05-17T21:57:56.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.753+0000 7fb9c96c9700 1 -- 172.21.15.107:0/1904042701 shutdown_connections 2024-05-17T21:57:56.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.753+0000 7fb9c96c9700 1 --2- 172.21.15.107:0/1904042701 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fb9ac078990 0x7fb9ac07ae40 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:56.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.753+0000 7fb9c96c9700 1 --2- 172.21.15.107:0/1904042701 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9c40fce60 0x7fb9c4071c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:56.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.753+0000 7fb9c96c9700 1 --2- 172.21.15.107:0/1904042701 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9c40fdfd0 0x7fb9c40702b0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:56.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.753+0000 7fb9c96c9700 1 -- 172.21.15.107:0/1904042701 >> 172.21.15.107:0/1904042701 conn(0x7fb9c40f8540 msgr2=0x7fb9c40f9ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:56.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.753+0000 7fb9c96c9700 1 -- 172.21.15.107:0/1904042701 shutdown_connections 2024-05-17T21:57:56.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.753+0000 7fb9c96c9700 1 -- 172.21.15.107:0/1904042701 wait complete. 2024-05-17T21:57:56.745 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 15 2024-05-17T21:57:56.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.856+0000 7f134f116700 1 -- 172.21.15.107:0/3558002552 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13480fe090 msgr2=0x7f13480fe500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:56.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.856+0000 7f134f116700 1 --2- 172.21.15.107:0/3558002552 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13480fe090 0x7f13480fe500 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1338004940 tx=0x7f1338009e30 comp rx=0 tx=0).stop 2024-05-17T21:57:56.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.856+0000 7f134f116700 1 -- 172.21.15.107:0/3558002552 shutdown_connections 2024-05-17T21:57:56.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.856+0000 7f134f116700 1 --2- 172.21.15.107:0/3558002552 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13480fe090 0x7f13480fe500 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:56.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.856+0000 7f134f116700 1 --2- 172.21.15.107:0/3558002552 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13480fce90 0x7f13480fd2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:56.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.856+0000 7f134f116700 1 -- 172.21.15.107:0/3558002552 >> 172.21.15.107:0/3558002552 conn(0x7f13480f85c0 msgr2=0x7f13480fa9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:56.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.856+0000 7f134f116700 1 -- 172.21.15.107:0/3558002552 shutdown_connections 2024-05-17T21:57:56.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.856+0000 7f134f116700 1 -- 172.21.15.107:0/3558002552 wait complete. 2024-05-17T21:57:56.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.856+0000 7f134f116700 1 Processor -- start 2024-05-17T21:57:56.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.857+0000 7f134f116700 1 -- start start 2024-05-17T21:57:56.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.857+0000 7f134f116700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13480fce90 0x7f1348071d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:56.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.857+0000 7f134f116700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13480fe090 0x7f13480702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:56.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.857+0000 7f134f116700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1348070840 con 0x7f13480fce90 2024-05-17T21:57:56.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.857+0000 7f134f116700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1348070980 con 0x7f13480fe090 2024-05-17T21:57:56.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.857+0000 7f134ceb2700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13480fce90 0x7f1348071d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:56.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.857+0000 7f134ceb2700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13480fce90 0x7f1348071d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42932/0 (socket says 172.21.15.107:42932) 2024-05-17T21:57:56.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.857+0000 7f134ceb2700 1 -- 172.21.15.107:0/155370584 learned_addr learned my addr 172.21.15.107:0/155370584 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:57:56.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.857+0000 7f133ffff700 1 --2- 172.21.15.107:0/155370584 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13480fe090 0x7f13480702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:56.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.857+0000 7f134ceb2700 1 -- 172.21.15.107:0/155370584 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13480fe090 msgr2=0x7f13480702b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:56.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.857+0000 7f134ceb2700 1 --2- 172.21.15.107:0/155370584 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13480fe090 0x7f13480702b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:56.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.857+0000 7f134ceb2700 1 -- 172.21.15.107:0/155370584 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1338005040 con 0x7f13480fce90 2024-05-17T21:57:56.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.857+0000 7f134ceb2700 1 --2- 172.21.15.107:0/155370584 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13480fce90 0x7f1348071d70 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f1344001ce0 tx=0x7f1344009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:56.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.858+0000 7f133d7fa700 1 -- 172.21.15.107:0/155370584 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1344007b30 con 0x7f13480fce90 2024-05-17T21:57:56.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.858+0000 7f134f116700 1 -- 172.21.15.107:0/155370584 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1348070c30 con 0x7f13480fce90 2024-05-17T21:57:56.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.858+0000 7f133d7fa700 1 -- 172.21.15.107:0/155370584 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f134400f040 con 0x7f13480fce90 2024-05-17T21:57:56.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.858+0000 7f133d7fa700 1 -- 172.21.15.107:0/155370584 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f13440096d0 con 0x7f13480fce90 2024-05-17T21:57:56.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.858+0000 7f134f116700 1 -- 172.21.15.107:0/155370584 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1348115a50 con 0x7f13480fce90 2024-05-17T21:57:56.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.859+0000 7f134f116700 1 -- 172.21.15.107:0/155370584 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f134805f5c0 con 0x7f13480fce90 2024-05-17T21:57:56.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.860+0000 7f133d7fa700 1 -- 172.21.15.107:0/155370584 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f1344007c90 con 0x7f13480fce90 2024-05-17T21:57:56.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.860+0000 7f133d7fa700 1 --2- 172.21.15.107:0/155370584 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f1330078990 0x7f133007ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:56.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.860+0000 7f133ffff700 1 --2- 172.21.15.107:0/155370584 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f1330078990 0x7f133007ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:56.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.860+0000 7f133d7fa700 1 -- 172.21.15.107:0/155370584 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f134409a0f0 con 0x7f13480fce90 2024-05-17T21:57:56.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.861+0000 7f133ffff700 1 --2- 172.21.15.107:0/155370584 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f1330078990 0x7f133007ae40 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f133800b970 tx=0x7f1338009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:56.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:56.865+0000 7f133d7fa700 1 -- 172.21.15.107:0/155370584 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f1344061e10 con 0x7f13480fce90 2024-05-17T21:57:56.973 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 17 21:57:56 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-3[90913]: 2024-05-17T21:57:56.601+0000 7f49865487c0 -1 Falling back to public interface 2024-05-17T21:57:57.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:57.087+0000 7f134f116700 1 -- 172.21.15.107:0/155370584 --> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1348102400 con 0x7f1330078990 2024-05-17T21:57:57.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:57.089+0000 7f133d7fa700 1 -- 172.21.15.107:0/155370584 <== mgr.34100 v2:172.21.15.107:6800/3205268226 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f1348102400 con 0x7f1330078990 2024-05-17T21:57:57.081 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-05-17T21:57:57.081 INFO:teuthology.orchestra.run.smithi107.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b", 2024-05-17T21:57:57.081 INFO:teuthology.orchestra.run.smithi107.stdout: "in_progress": true, 2024-05-17T21:57:57.081 INFO:teuthology.orchestra.run.smithi107.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-17T21:57:57.081 INFO:teuthology.orchestra.run.smithi107.stdout: "services_complete": [ 2024-05-17T21:57:57.081 INFO:teuthology.orchestra.run.smithi107.stdout: "crash", 2024-05-17T21:57:57.081 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr", 2024-05-17T21:57:57.081 INFO:teuthology.orchestra.run.smithi107.stdout: "mon" 2024-05-17T21:57:57.081 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-05-17T21:57:57.081 INFO:teuthology.orchestra.run.smithi107.stdout: "progress": "9/21 daemons upgraded", 2024-05-17T21:57:57.081 INFO:teuthology.orchestra.run.smithi107.stdout: "message": "Currently upgrading osd daemons", 2024-05-17T21:57:57.081 INFO:teuthology.orchestra.run.smithi107.stdout: "is_paused": false 2024-05-17T21:57:57.081 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-05-17T21:57:57.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:57.092+0000 7f134f116700 1 -- 172.21.15.107:0/155370584 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f1330078990 msgr2=0x7f133007ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:57.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:57.092+0000 7f134f116700 1 --2- 172.21.15.107:0/155370584 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f1330078990 0x7f133007ae40 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f133800b970 tx=0x7f1338009e30 comp rx=0 tx=0).stop 2024-05-17T21:57:57.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:57.092+0000 7f134f116700 1 -- 172.21.15.107:0/155370584 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13480fce90 msgr2=0x7f1348071d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:57.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:57.092+0000 7f134f116700 1 --2- 172.21.15.107:0/155370584 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13480fce90 0x7f1348071d70 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f1344001ce0 tx=0x7f1344009e60 comp rx=0 tx=0).stop 2024-05-17T21:57:57.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:57.092+0000 7f134f116700 1 -- 172.21.15.107:0/155370584 shutdown_connections 2024-05-17T21:57:57.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:57.092+0000 7f134f116700 1 --2- 172.21.15.107:0/155370584 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f1330078990 0x7f133007ae40 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:57.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:57.092+0000 7f134f116700 1 --2- 172.21.15.107:0/155370584 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13480fce90 0x7f1348071d70 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:57.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:57.092+0000 7f134f116700 1 --2- 172.21.15.107:0/155370584 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13480fe090 0x7f13480702b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:57.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:57.092+0000 7f134f116700 1 -- 172.21.15.107:0/155370584 >> 172.21.15.107:0/155370584 conn(0x7f13480f85c0 msgr2=0x7f13480f9ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:57.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:57.092+0000 7f134f116700 1 -- 172.21.15.107:0/155370584 shutdown_connections 2024-05-17T21:57:57.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:57.092+0000 7f134f116700 1 -- 172.21.15.107:0/155370584 wait complete. 2024-05-17T21:57:57.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:57.199+0000 7fcc19298700 1 -- 172.21.15.107:0/2150993797 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc140fce80 msgr2=0x7fcc140fd290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:57.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:57.199+0000 7fcc19298700 1 --2- 172.21.15.107:0/2150993797 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc140fce80 0x7fcc140fd290 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fcc04004940 tx=0x7fcc04009e30 comp rx=0 tx=0).stop 2024-05-17T21:57:57.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:57.199+0000 7fcc19298700 1 -- 172.21.15.107:0/2150993797 shutdown_connections 2024-05-17T21:57:57.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:57.199+0000 7fcc19298700 1 --2- 172.21.15.107:0/2150993797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcc140fdff0 0x7fcc140fe460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:57.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:57.199+0000 7fcc19298700 1 --2- 172.21.15.107:0/2150993797 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc140fce80 0x7fcc140fd290 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:57.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:57.199+0000 7fcc19298700 1 -- 172.21.15.107:0/2150993797 >> 172.21.15.107:0/2150993797 conn(0x7fcc140f8540 msgr2=0x7fcc140fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:57.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:57.199+0000 7fcc19298700 1 -- 172.21.15.107:0/2150993797 shutdown_connections 2024-05-17T21:57:57.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:57.199+0000 7fcc19298700 1 -- 172.21.15.107:0/2150993797 wait complete. 2024-05-17T21:57:57.193 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:57.200+0000 7fcc19298700 1 Processor -- start 2024-05-17T21:57:57.193 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:57.200+0000 7fcc19298700 1 -- start start 2024-05-17T21:57:57.193 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:57.200+0000 7fcc19298700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcc140fce80 0x7fcc1410d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:57.193 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:57.200+0000 7fcc19298700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc140fdff0 0x7fcc1410def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:57.193 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:57.200+0000 7fcc19298700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc1410e4f0 con 0x7fcc140fdff0 2024-05-17T21:57:57.193 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:57.200+0000 7fcc19298700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc1410ed40 con 0x7fcc140fce80 2024-05-17T21:57:57.193 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:57.200+0000 7fcc127fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc140fdff0 0x7fcc1410def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:57.193 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:57.200+0000 7fcc127fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc140fdff0 0x7fcc1410def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42956/0 (socket says 172.21.15.107:42956) 2024-05-17T21:57:57.193 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:57.200+0000 7fcc127fc700 1 -- 172.21.15.107:0/2271957634 learned_addr learned my addr 172.21.15.107:0/2271957634 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:57:57.193 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:57.200+0000 7fcc12ffd700 1 --2- 172.21.15.107:0/2271957634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcc140fce80 0x7fcc1410d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:57.193 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:57.201+0000 7fcc127fc700 1 -- 172.21.15.107:0/2271957634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcc140fce80 msgr2=0x7fcc1410d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:57.193 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:57.201+0000 7fcc127fc700 1 --2- 172.21.15.107:0/2271957634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcc140fce80 0x7fcc1410d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:57.193 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:57.201+0000 7fcc127fc700 1 -- 172.21.15.107:0/2271957634 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc04005040 con 0x7fcc140fdff0 2024-05-17T21:57:57.194 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:57.201+0000 7fcc127fc700 1 --2- 172.21.15.107:0/2271957634 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc140fdff0 0x7fcc1410def0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fcc0800eb40 tx=0x7fcc0800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:57.194 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:57.201+0000 7fcbfb7fe700 1 -- 172.21.15.107:0/2271957634 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc08007cc0 con 0x7fcc140fdff0 2024-05-17T21:57:57.194 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:57.201+0000 7fcc19298700 1 -- 172.21.15.107:0/2271957634 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc1410efc0 con 0x7fcc140fdff0 2024-05-17T21:57:57.194 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:57.201+0000 7fcbfb7fe700 1 -- 172.21.15.107:0/2271957634 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcc0800f040 con 0x7fcc140fdff0 2024-05-17T21:57:57.194 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:57.201+0000 7fcbfb7fe700 1 -- 172.21.15.107:0/2271957634 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc0800ca10 con 0x7fcc140fdff0 2024-05-17T21:57:57.194 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:57.201+0000 7fcc19298700 1 -- 172.21.15.107:0/2271957634 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc1410f500 con 0x7fcc140fdff0 2024-05-17T21:57:57.194 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:57.202+0000 7fcc19298700 1 -- 172.21.15.107:0/2271957634 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc1405f5c0 con 0x7fcc140fdff0 2024-05-17T21:57:57.194 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:57.203+0000 7fcbfb7fe700 1 -- 172.21.15.107:0/2271957634 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7fcc0800cc80 con 0x7fcc140fdff0 2024-05-17T21:57:57.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:57.203+0000 7fcbfb7fe700 1 --2- 172.21.15.107:0/2271957634 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fcbfc078990 0x7fcbfc07ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:57.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:57.203+0000 7fcbfb7fe700 1 -- 172.21.15.107:0/2271957634 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fcc0809a2c0 con 0x7fcc140fdff0 2024-05-17T21:57:57.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:57.204+0000 7fcc12ffd700 1 --2- 172.21.15.107:0/2271957634 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fcbfc078990 0x7fcbfc07ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:57.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:57.204+0000 7fcc12ffd700 1 --2- 172.21.15.107:0/2271957634 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fcbfc078990 0x7fcbfc07ae40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fcc04009b30 tx=0x7fcc04009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:57.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:57.208+0000 7fcbfb7fe700 1 -- 172.21.15.107:0/2271957634 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7fcc08061fe0 con 0x7fcc140fdff0 2024-05-17T21:57:57.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:57 smithi138 ceph-mon[81376]: from='client.34312 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:57:57.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:57 smithi138 ceph-mon[81376]: from='client.34316 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:57:57.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:57 smithi138 ceph-mon[81376]: from='client.34320 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:57:57.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:57 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:57:57.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:57 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:57:57.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:57 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:57:57.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:57 smithi138 ceph-mon[81376]: from='client.? 172.21.15.107:0/3346372326' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:57:57.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:57 smithi138 ceph-mon[81376]: from='client.? 172.21.15.107:0/1904042701' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-17T21:57:57.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:57 smithi107 ceph-mon[93224]: from='client.34312 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:57:57.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:57 smithi107 ceph-mon[93224]: from='client.34316 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:57:57.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:57 smithi107 ceph-mon[93224]: from='client.34320 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:57:57.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:57 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:57:57.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:57 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:57:57.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:57 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:57:57.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:57 smithi107 ceph-mon[93224]: from='client.? 172.21.15.107:0/3346372326' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:57:57.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:57 smithi107 ceph-mon[93224]: from='client.? 172.21.15.107:0/1904042701' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-17T21:57:57.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:57.551+0000 7fcc19298700 1 -- 172.21.15.107:0/2271957634 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fcc1410f770 con 0x7fcc140fdff0 2024-05-17T21:57:57.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:57.552+0000 7fcbfb7fe700 1 -- 172.21.15.107:0/2271957634 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+1461 (secure 0 0 0) 0x7fcc08061980 con 0x7fcc140fdff0 2024-05-17T21:57:57.544 INFO:teuthology.orchestra.run.smithi107.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data; 1 osds down; Degraded data redundancy: 60/267 objects degraded (22.472%), 18 pgs degraded 2024-05-17T21:57:57.544 INFO:teuthology.orchestra.run.smithi107.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-17T21:57:57.544 INFO:teuthology.orchestra.run.smithi107.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-17T21:57:57.544 INFO:teuthology.orchestra.run.smithi107.stdout:[WRN] OSD_DOWN: 1 osds down 2024-05-17T21:57:57.544 INFO:teuthology.orchestra.run.smithi107.stdout: osd.3 (root=default,host=smithi138) is down 2024-05-17T21:57:57.544 INFO:teuthology.orchestra.run.smithi107.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 60/267 objects degraded (22.472%), 18 pgs degraded 2024-05-17T21:57:57.544 INFO:teuthology.orchestra.run.smithi107.stdout: pg 1.0 is active+undersized+degraded, acting [0,1] 2024-05-17T21:57:57.544 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.0 is active+undersized+degraded, acting [1,0] 2024-05-17T21:57:57.544 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.5 is active+undersized+degraded, acting [0,4] 2024-05-17T21:57:57.544 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.6 is active+undersized+degraded, acting [1,4] 2024-05-17T21:57:57.544 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.7 is active+undersized+degraded, acting [4,2] 2024-05-17T21:57:57.544 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.8 is active+undersized+degraded, acting [5,0] 2024-05-17T21:57:57.545 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.a is active+undersized+degraded, acting [1,4] 2024-05-17T21:57:57.545 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.b is active+undersized+degraded, acting [4,5] 2024-05-17T21:57:57.545 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.c is active+undersized+degraded, acting [2,0] 2024-05-17T21:57:57.545 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.d is active+undersized+degraded, acting [1,2] 2024-05-17T21:57:57.545 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.e is active+undersized+degraded, acting [2,0] 2024-05-17T21:57:57.545 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.14 is active+undersized+degraded, acting [4,5] 2024-05-17T21:57:57.545 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.15 is active+undersized+degraded, acting [1,0] 2024-05-17T21:57:57.545 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.16 is active+undersized+degraded, acting [5,2] 2024-05-17T21:57:57.545 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.18 is active+undersized+degraded, acting [5,4] 2024-05-17T21:57:57.545 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.1a is active+undersized+degraded, acting [4,5] 2024-05-17T21:57:57.545 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.1d is active+undersized+degraded, acting [5,0] 2024-05-17T21:57:57.545 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.1f is active+undersized+degraded, acting [0,4] 2024-05-17T21:57:57.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:57.554+0000 7fcc19298700 1 -- 172.21.15.107:0/2271957634 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fcbfc078990 msgr2=0x7fcbfc07ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:57.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:57.554+0000 7fcc19298700 1 --2- 172.21.15.107:0/2271957634 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fcbfc078990 0x7fcbfc07ae40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fcc04009b30 tx=0x7fcc04009e30 comp rx=0 tx=0).stop 2024-05-17T21:57:57.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:57.554+0000 7fcc19298700 1 -- 172.21.15.107:0/2271957634 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc140fdff0 msgr2=0x7fcc1410def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:57.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:57.554+0000 7fcc19298700 1 --2- 172.21.15.107:0/2271957634 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc140fdff0 0x7fcc1410def0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fcc0800eb40 tx=0x7fcc0800b040 comp rx=0 tx=0).stop 2024-05-17T21:57:57.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:57.554+0000 7fcc19298700 1 -- 172.21.15.107:0/2271957634 shutdown_connections 2024-05-17T21:57:57.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:57.554+0000 7fcc19298700 1 --2- 172.21.15.107:0/2271957634 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fcbfc078990 0x7fcbfc07ae40 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:57.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:57.554+0000 7fcc19298700 1 --2- 172.21.15.107:0/2271957634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcc140fce80 0x7fcc1410d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:57.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:57.554+0000 7fcc19298700 1 --2- 172.21.15.107:0/2271957634 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc140fdff0 0x7fcc1410def0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:57.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:57.554+0000 7fcc19298700 1 -- 172.21.15.107:0/2271957634 >> 172.21.15.107:0/2271957634 conn(0x7fcc140f8540 msgr2=0x7fcc140f9b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:57.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:57.554+0000 7fcc19298700 1 -- 172.21.15.107:0/2271957634 shutdown_connections 2024-05-17T21:57:57.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:57:57.554+0000 7fcc19298700 1 -- 172.21.15.107:0/2271957634 wait complete. 2024-05-17T21:57:58.492 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:58 smithi138 ceph-mon[81376]: pgmap v121: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 258 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 60/267 objects degraded (22.472%) 2024-05-17T21:57:58.492 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:58 smithi138 ceph-mon[81376]: from='client.34334 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:57:58.492 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:58 smithi138 ceph-mon[81376]: Health check failed: Degraded data redundancy: 60/267 objects degraded (22.472%), 18 pgs degraded (PG_DEGRADED) 2024-05-17T21:57:58.492 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:58 smithi138 ceph-mon[81376]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-05-17T21:57:58.492 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:57:58 smithi138 ceph-mon[81376]: from='client.? 172.21.15.107:0/2271957634' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-17T21:57:58.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:58 smithi107 ceph-mon[93224]: pgmap v121: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 258 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 60/267 objects degraded (22.472%) 2024-05-17T21:57:58.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:58 smithi107 ceph-mon[93224]: from='client.34334 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:57:58.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:58 smithi107 ceph-mon[93224]: Health check failed: Degraded data redundancy: 60/267 objects degraded (22.472%), 18 pgs degraded (PG_DEGRADED) 2024-05-17T21:57:58.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:58 smithi107 ceph-mon[93224]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-05-17T21:57:58.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:57:58 smithi107 ceph-mon[93224]: from='client.? 172.21.15.107:0/2271957634' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-17T21:58:00.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:00 smithi138 ceph-mon[81376]: pgmap v122: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 258 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 60/267 objects degraded (22.472%) 2024-05-17T21:58:00.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:00 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:58:00.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:00 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:58:00.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:00 smithi107 ceph-mon[93224]: pgmap v122: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 258 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 60/267 objects degraded (22.472%) 2024-05-17T21:58:00.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:00 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:58:00.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:00 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:58:01.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:01 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:58:01.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:01 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:58:01.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:01 smithi138 ceph-mon[81376]: pgmap v123: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 258 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 60/267 objects degraded (22.472%) 2024-05-17T21:58:01.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:01 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:58:01.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:01 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:58:01.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:01 smithi107 ceph-mon[93224]: pgmap v123: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 258 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 60/267 objects degraded (22.472%) 2024-05-17T21:58:02.497 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:02 smithi138 ceph-mon[81376]: from='osd.3 [v2:172.21.15.138:6800/1179392099,v1:172.21.15.138:6801/1179392099]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-17T21:58:02.497 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:02 smithi138 ceph-mon[81376]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-17T21:58:02.497 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 17 21:58:02 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-3[90913]: 2024-05-17T21:58:02.252+0000 7f49865487c0 -1 osd.3 64 log_to_monitors true 2024-05-17T21:58:02.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:02 smithi107 ceph-mon[93224]: from='osd.3 [v2:172.21.15.138:6800/1179392099,v1:172.21.15.138:6801/1179392099]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-17T21:58:02.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:02 smithi107 ceph-mon[93224]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-17T21:58:03.559 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:03 smithi138 ceph-mon[81376]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-17T21:58:03.559 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:03 smithi138 ceph-mon[81376]: from='osd.3 [v2:172.21.15.138:6800/1179392099,v1:172.21.15.138:6801/1179392099]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-05-17T21:58:03.559 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:03 smithi138 ceph-mon[81376]: osdmap e67: 6 total, 5 up, 6 in 2024-05-17T21:58:03.560 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:03 smithi138 ceph-mon[81376]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-05-17T21:58:03.560 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:03 smithi138 ceph-mon[81376]: pgmap v125: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 258 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 60/267 objects degraded (22.472%) 2024-05-17T21:58:03.560 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:03 smithi138 ceph-mon[81376]: from='osd.3 ' entity='osd.3' 2024-05-17T21:58:03.560 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 17 21:58:03 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-3[90913]: 2024-05-17T21:58:03.254+0000 7f4973959700 -1 osd.3 64 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-17T21:58:03.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:03 smithi107 ceph-mon[93224]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-17T21:58:03.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:03 smithi107 ceph-mon[93224]: from='osd.3 [v2:172.21.15.138:6800/1179392099,v1:172.21.15.138:6801/1179392099]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-05-17T21:58:03.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:03 smithi107 ceph-mon[93224]: osdmap e67: 6 total, 5 up, 6 in 2024-05-17T21:58:03.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:03 smithi107 ceph-mon[93224]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-05-17T21:58:03.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:03 smithi107 ceph-mon[93224]: pgmap v125: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 258 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 60/267 objects degraded (22.472%) 2024-05-17T21:58:03.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:03 smithi107 ceph-mon[93224]: from='osd.3 ' entity='osd.3' 2024-05-17T21:58:05.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:05 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:58:05.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:05 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:58:05.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:05 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:58:05.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:05 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T21:58:05.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:05 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:58:05.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:05 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:58:05.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:05 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:58:05.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:05 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:58:05.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:05 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:58:05.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:05 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-05-17T21:58:05.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:05 smithi138 ceph-mon[81376]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-05-17T21:58:05.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:05 smithi138 ceph-mon[81376]: Upgrade: unsafe to stop osd(s) at this time (18 PGs are or would become offline) 2024-05-17T21:58:05.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:05 smithi138 ceph-mon[81376]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-05-17T21:58:05.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:05 smithi138 ceph-mon[81376]: osd.3 [v2:172.21.15.138:6800/1179392099,v1:172.21.15.138:6801/1179392099] boot 2024-05-17T21:58:05.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:05 smithi138 ceph-mon[81376]: osdmap e68: 6 total, 6 up, 6 in 2024-05-17T21:58:05.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:05 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-17T21:58:05.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:05 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:58:05.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:05 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:58:05.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:05 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:58:05.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:05 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:58:05.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:05 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:58:05.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:05 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T21:58:05.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:05 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:58:05.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:05 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:58:05.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:05 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:58:05.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:05 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:58:05.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:05 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:58:05.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:05 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-05-17T21:58:05.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:05 smithi107 ceph-mon[93224]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-05-17T21:58:05.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:05 smithi107 ceph-mon[93224]: Upgrade: unsafe to stop osd(s) at this time (18 PGs are or would become offline) 2024-05-17T21:58:05.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:05 smithi107 ceph-mon[93224]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-05-17T21:58:05.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:05 smithi107 ceph-mon[93224]: osd.3 [v2:172.21.15.138:6800/1179392099,v1:172.21.15.138:6801/1179392099] boot 2024-05-17T21:58:05.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:05 smithi107 ceph-mon[93224]: osdmap e68: 6 total, 6 up, 6 in 2024-05-17T21:58:05.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:05 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-17T21:58:05.353 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:05 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:58:05.353 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:05 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:58:06.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:06 smithi138 ceph-mon[81376]: pgmap v127: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 258 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 60/267 objects degraded (22.472%) 2024-05-17T21:58:06.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:06 smithi138 ceph-mon[81376]: osdmap e69: 6 total, 6 up, 6 in 2024-05-17T21:58:06.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:06 smithi107 ceph-mon[93224]: pgmap v127: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 258 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 60/267 objects degraded (22.472%) 2024-05-17T21:58:06.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:06 smithi107 ceph-mon[93224]: osdmap e69: 6 total, 6 up, 6 in 2024-05-17T21:58:07.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:07 smithi138 ceph-mon[81376]: Health check update: Degraded data redundancy: 29/267 objects degraded (10.861%), 9 pgs degraded (PG_DEGRADED) 2024-05-17T21:58:07.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:07 smithi107 ceph-mon[93224]: Health check update: Degraded data redundancy: 29/267 objects degraded (10.861%), 9 pgs degraded (PG_DEGRADED) 2024-05-17T21:58:08.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:08 smithi107 ceph-mon[93224]: pgmap v129: 65 pgs: 4 peering, 9 active+undersized+degraded, 7 active+undersized, 45 active+clean; 258 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 29/267 objects degraded (10.861%) 2024-05-17T21:58:08.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:08 smithi138 ceph-mon[81376]: pgmap v129: 65 pgs: 4 peering, 9 active+undersized+degraded, 7 active+undersized, 45 active+clean; 258 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 29/267 objects degraded (10.861%) 2024-05-17T21:58:09.469 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:09 smithi107 ceph-mon[93224]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 29/267 objects degraded (10.861%), 9 pgs degraded) 2024-05-17T21:58:09.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:09 smithi138 ceph-mon[81376]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 29/267 objects degraded (10.861%), 9 pgs degraded) 2024-05-17T21:58:10.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:10 smithi138 ceph-mon[81376]: pgmap v130: 65 pgs: 4 peering, 61 active+clean; 258 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 842 B/s rd, 1 op/s 2024-05-17T21:58:10.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:10 smithi107 ceph-mon[93224]: pgmap v130: 65 pgs: 4 peering, 61 active+clean; 258 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 842 B/s rd, 1 op/s 2024-05-17T21:58:12.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:12 smithi138 ceph-mon[81376]: pgmap v131: 65 pgs: 65 active+clean; 258 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s 2024-05-17T21:58:12.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:12 smithi107 ceph-mon[93224]: pgmap v131: 65 pgs: 65 active+clean; 258 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s 2024-05-17T21:58:14.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:14 smithi138 ceph-mon[81376]: pgmap v132: 65 pgs: 65 active+clean; 258 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T21:58:14.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:14 smithi107 ceph-mon[93224]: pgmap v132: 65 pgs: 65 active+clean; 258 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T21:58:16.417 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:16 smithi138 ceph-mon[81376]: pgmap v133: 65 pgs: 65 active+clean; 258 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s 2024-05-17T21:58:16.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:16 smithi107 ceph-mon[93224]: pgmap v133: 65 pgs: 65 active+clean; 258 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s 2024-05-17T21:58:18.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:18 smithi138 ceph-mon[81376]: pgmap v134: 65 pgs: 65 active+clean; 258 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 917 B/s rd, 1 op/s 2024-05-17T21:58:18.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:18 smithi107 ceph-mon[93224]: pgmap v134: 65 pgs: 65 active+clean; 258 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 917 B/s rd, 1 op/s 2024-05-17T21:58:20.416 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:20 smithi138 ceph-mon[81376]: pgmap v135: 65 pgs: 65 active+clean; 258 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T21:58:20.416 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:20 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-05-17T21:58:20.417 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:20 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:58:20.417 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:20 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:58:20.417 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:20 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:58:20.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:20 smithi107 ceph-mon[93224]: pgmap v135: 65 pgs: 65 active+clean; 258 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T21:58:20.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:20 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-05-17T21:58:20.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:20 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:58:20.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:20 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:58:20.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:20 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:58:21.401 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:21 smithi138 ceph-mon[81376]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-05-17T21:58:21.401 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:21 smithi138 ceph-mon[81376]: Upgrade: osd.4 is safe to restart 2024-05-17T21:58:21.402 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:21 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:58:21.402 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:21 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-17T21:58:21.402 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:21 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:58:21.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:21 smithi107 ceph-mon[93224]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-05-17T21:58:21.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:21 smithi107 ceph-mon[93224]: Upgrade: osd.4 is safe to restart 2024-05-17T21:58:21.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:21 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:58:21.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:21 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-17T21:58:21.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:21 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:58:22.404 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:22 smithi138 ceph-mon[81376]: pgmap v136: 65 pgs: 65 active+clean; 258 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T21:58:22.404 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:22 smithi138 ceph-mon[81376]: Upgrade: Updating osd.4 2024-05-17T21:58:22.404 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:22 smithi138 ceph-mon[81376]: Deploying daemon osd.4 on smithi138 2024-05-17T21:58:22.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:22 smithi107 ceph-mon[93224]: pgmap v136: 65 pgs: 65 active+clean; 258 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T21:58:22.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:22 smithi107 ceph-mon[93224]: Upgrade: Updating osd.4 2024-05-17T21:58:22.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:22 smithi107 ceph-mon[93224]: Deploying daemon osd.4 on smithi138 2024-05-17T21:58:24.306 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:24 smithi138 ceph-mon[81376]: pgmap v137: 65 pgs: 65 active+clean; 258 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T21:58:24.307 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 17 21:58:24 smithi138 systemd[1]: Stopping Ceph osd.4 for 3722731a-1496-11ef-bc9a-c7b262605968... 2024-05-17T21:58:24.575 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 17 21:58:24 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-4[48290]: 2024-05-17T21:58:24.313+0000 7fce8ab54700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-17T21:58:24.575 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 17 21:58:24 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-4[48290]: 2024-05-17T21:58:24.313+0000 7fce8ab54700 -1 osd.4 69 *** Got signal Terminated *** 2024-05-17T21:58:24.575 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 17 21:58:24 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-4[48290]: 2024-05-17T21:58:24.313+0000 7fce8ab54700 -1 osd.4 69 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-17T21:58:24.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:24 smithi107 ceph-mon[93224]: pgmap v137: 65 pgs: 65 active+clean; 258 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T21:58:25.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:25 smithi138 ceph-mon[81376]: osd.4 marked itself down and dead 2024-05-17T21:58:25.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:25 smithi107 ceph-mon[93224]: osd.4 marked itself down and dead 2024-05-17T21:58:25.940 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 17 21:58:25 smithi138 podman[95584]: 2024-05-17 21:58:25.654636819 +0000 UTC m=+1.445668101 container died b306a36a744eb3a7f4e389716c6a18a056b00f233ed62b8dd1d004a21a619601 (image=quay.ceph.io/ceph-ci/ceph@sha256:c200327b094672e93680375e82d2de1a08f5881ef304978f791cce179f847d60, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-4, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , RELEASE=quincy-9903f77, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-17T21:58:26.276 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:26 smithi138 ceph-mon[81376]: pgmap v138: 65 pgs: 65 active+clean; 258 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T21:58:26.276 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:26 smithi138 ceph-mon[81376]: Health check failed: 1 osds down (OSD_DOWN) 2024-05-17T21:58:26.276 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:26 smithi138 ceph-mon[81376]: osdmap e70: 6 total, 5 up, 6 in 2024-05-17T21:58:26.277 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 17 21:58:26 smithi138 podman[95584]: 2024-05-17 21:58:26.073717326 +0000 UTC m=+1.864748607 container cleanup b306a36a744eb3a7f4e389716c6a18a056b00f233ed62b8dd1d004a21a619601 (image=quay.ceph.io/ceph-ci/ceph@sha256:c200327b094672e93680375e82d2de1a08f5881ef304978f791cce179f847d60, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-4, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=quincy-9903f77, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-17T21:58:26.277 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 17 21:58:26 smithi138 bash[95584]: ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-4 2024-05-17T21:58:26.575 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 17 21:58:26 smithi138 podman[95601]: 2024-05-17 21:58:26.285879251 +0000 UTC m=+0.626927604 container remove b306a36a744eb3a7f4e389716c6a18a056b00f233ed62b8dd1d004a21a619601 (image=quay.ceph.io/ceph-ci/ceph@sha256:c200327b094672e93680375e82d2de1a08f5881ef304978f791cce179f847d60, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-4, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=quincy-9903f77, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-05-17T21:58:26.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:26 smithi107 ceph-mon[93224]: pgmap v138: 65 pgs: 65 active+clean; 258 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T21:58:26.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:26 smithi107 ceph-mon[93224]: Health check failed: 1 osds down (OSD_DOWN) 2024-05-17T21:58:26.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:26 smithi107 ceph-mon[93224]: osdmap e70: 6 total, 5 up, 6 in 2024-05-17T21:58:26.970 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 17 21:58:26 smithi138 podman[95690]: 2024-05-17 21:58:26.603110742 +0000 UTC m=+0.016858548 image pull 8880f7613d0283efa7002505fdccdbc3e37d3b5da31c57fa22d62262c5dab66c quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b 2024-05-17T21:58:26.970 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 17 21:58:26 smithi138 podman[95690]: 2024-05-17 21:58:26.701347751 +0000 UTC m=+0.115095602 container create c46ca9cf64a0bb0445cf0ac38df4ecd529f3cc865728b11972e30759bd625571 (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-4-deactivate, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514) 2024-05-17T21:58:26.970 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 17 21:58:26 smithi138 podman[95690]: 2024-05-17 21:58:26.979377275 +0000 UTC m=+0.393125085 container init c46ca9cf64a0bb0445cf0ac38df4ecd529f3cc865728b11972e30759bd625571 (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-4-deactivate, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-17T21:58:27.272 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:27 smithi138 ceph-mon[81376]: osdmap e71: 6 total, 5 up, 6 in 2024-05-17T21:58:27.272 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 17 21:58:26 smithi138 podman[95690]: 2024-05-17 21:58:26.983015134 +0000 UTC m=+0.396762943 container start c46ca9cf64a0bb0445cf0ac38df4ecd529f3cc865728b11972e30759bd625571 (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-17T21:58:27.272 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 17 21:58:27 smithi138 podman[95690]: 2024-05-17 21:58:27.062222925 +0000 UTC m=+0.475970753 container attach c46ca9cf64a0bb0445cf0ac38df4ecd529f3cc865728b11972e30759bd625571 (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-4-deactivate, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514) 2024-05-17T21:58:27.502 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:27 smithi107 ceph-mon[93224]: osdmap e71: 6 total, 5 up, 6 in 2024-05-17T21:58:27.575 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 17 21:58:27 smithi138 podman[95690]: 2024-05-17 21:58:27.281600198 +0000 UTC m=+0.695348006 container died c46ca9cf64a0bb0445cf0ac38df4ecd529f3cc865728b11972e30759bd625571 (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-4-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a) 2024-05-17T21:58:27.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:27.672+0000 7fb468f58700 1 -- 172.21.15.107:0/585883330 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4640fdfd0 msgr2=0x7fb4640fe440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:27.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:27.672+0000 7fb468f58700 1 --2- 172.21.15.107:0/585883330 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4640fdfd0 0x7fb4640fe440 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb44c004940 tx=0x7fb44c009e30 comp rx=0 tx=0).stop 2024-05-17T21:58:27.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:27.672+0000 7fb468f58700 1 -- 172.21.15.107:0/585883330 shutdown_connections 2024-05-17T21:58:27.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:27.672+0000 7fb468f58700 1 --2- 172.21.15.107:0/585883330 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4640fdfd0 0x7fb4640fe440 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:27.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:27.672+0000 7fb468f58700 1 --2- 172.21.15.107:0/585883330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4640fce60 0x7fb4640fd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:27.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:27.672+0000 7fb468f58700 1 -- 172.21.15.107:0/585883330 >> 172.21.15.107:0/585883330 conn(0x7fb4640f8540 msgr2=0x7fb4640fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:27.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:27.672+0000 7fb468f58700 1 -- 172.21.15.107:0/585883330 shutdown_connections 2024-05-17T21:58:27.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:27.672+0000 7fb468f58700 1 -- 172.21.15.107:0/585883330 wait complete. 2024-05-17T21:58:27.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:27.672+0000 7fb468f58700 1 Processor -- start 2024-05-17T21:58:27.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:27.672+0000 7fb468f58700 1 -- start start 2024-05-17T21:58:27.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:27.673+0000 7fb468f58700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4640fce60 0x7fb46410d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:27.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:27.673+0000 7fb468f58700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4640fdfd0 0x7fb46410ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:27.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:27.673+0000 7fb468f58700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb46410e3c0 con 0x7fb4640fce60 2024-05-17T21:58:27.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:27.673+0000 7fb468f58700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb46410e530 con 0x7fb4640fdfd0 2024-05-17T21:58:27.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:27.673+0000 7fb46259c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4640fce60 0x7fb46410d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:27.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:27.673+0000 7fb46259c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4640fce60 0x7fb46410d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59768/0 (socket says 172.21.15.107:59768) 2024-05-17T21:58:27.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:27.673+0000 7fb46259c700 1 -- 172.21.15.107:0/1240695052 learned_addr learned my addr 172.21.15.107:0/1240695052 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:58:27.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:27.673+0000 7fb461d9b700 1 --2- 172.21.15.107:0/1240695052 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4640fdfd0 0x7fb46410ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:27.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:27.673+0000 7fb46259c700 1 -- 172.21.15.107:0/1240695052 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4640fdfd0 msgr2=0x7fb46410ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:27.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:27.673+0000 7fb46259c700 1 --2- 172.21.15.107:0/1240695052 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4640fdfd0 0x7fb46410ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:27.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:27.673+0000 7fb46259c700 1 -- 172.21.15.107:0/1240695052 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb44c005040 con 0x7fb4640fce60 2024-05-17T21:58:27.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:27.673+0000 7fb46259c700 1 --2- 172.21.15.107:0/1240695052 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4640fce60 0x7fb46410d880 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb454002a80 tx=0x7fb4540046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:27.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:27.673+0000 7fb45affd700 1 -- 172.21.15.107:0/1240695052 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb454007cc0 con 0x7fb4640fce60 2024-05-17T21:58:27.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:27.673+0000 7fb45affd700 1 -- 172.21.15.107:0/1240695052 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb45400f040 con 0x7fb4640fce60 2024-05-17T21:58:27.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:27.673+0000 7fb468f58700 1 -- 172.21.15.107:0/1240695052 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb464112f90 con 0x7fb4640fce60 2024-05-17T21:58:27.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:27.673+0000 7fb45affd700 1 -- 172.21.15.107:0/1240695052 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb45400dac0 con 0x7fb4640fce60 2024-05-17T21:58:27.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:27.673+0000 7fb468f58700 1 -- 172.21.15.107:0/1240695052 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4641134c0 con 0x7fb4640fce60 2024-05-17T21:58:27.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:27.675+0000 7fb468f58700 1 -- 172.21.15.107:0/1240695052 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb46404c710 con 0x7fb4640fce60 2024-05-17T21:58:27.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:27.675+0000 7fb45affd700 1 -- 172.21.15.107:0/1240695052 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7fb45400dc20 con 0x7fb4640fce60 2024-05-17T21:58:27.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:27.676+0000 7fb45affd700 1 --2- 172.21.15.107:0/1240695052 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fb450078990 0x7fb45007ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:27.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:27.676+0000 7fb45affd700 1 -- 172.21.15.107:0/1240695052 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fb45409a470 con 0x7fb4640fce60 2024-05-17T21:58:27.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:27.676+0000 7fb461d9b700 1 --2- 172.21.15.107:0/1240695052 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fb450078990 0x7fb45007ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:27.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:27.676+0000 7fb461d9b700 1 --2- 172.21.15.107:0/1240695052 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fb450078990 0x7fb45007ae40 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb44c00b3b0 tx=0x7fb44c009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:27.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:27.680+0000 7fb45affd700 1 -- 172.21.15.107:0/1240695052 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7fb4540620e0 con 0x7fb4640fce60 2024-05-17T21:58:27.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:27.905+0000 7fb468f58700 1 -- 172.21.15.107:0/1240695052 --> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb4641022a0 con 0x7fb450078990 2024-05-17T21:58:27.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:27.906+0000 7fb45affd700 1 -- 172.21.15.107:0/1240695052 <== mgr.34100 v2:172.21.15.107:6800/3205268226 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fb4641022a0 con 0x7fb450078990 2024-05-17T21:58:27.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:27.909+0000 7fb468f58700 1 -- 172.21.15.107:0/1240695052 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fb450078990 msgr2=0x7fb45007ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:27.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:27.909+0000 7fb468f58700 1 --2- 172.21.15.107:0/1240695052 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fb450078990 0x7fb45007ae40 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb44c00b3b0 tx=0x7fb44c009e30 comp rx=0 tx=0).stop 2024-05-17T21:58:27.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:27.909+0000 7fb468f58700 1 -- 172.21.15.107:0/1240695052 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4640fce60 msgr2=0x7fb46410d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:27.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:27.909+0000 7fb468f58700 1 --2- 172.21.15.107:0/1240695052 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4640fce60 0x7fb46410d880 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb454002a80 tx=0x7fb4540046b0 comp rx=0 tx=0).stop 2024-05-17T21:58:27.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:27.909+0000 7fb468f58700 1 -- 172.21.15.107:0/1240695052 shutdown_connections 2024-05-17T21:58:27.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:27.909+0000 7fb468f58700 1 --2- 172.21.15.107:0/1240695052 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fb450078990 0x7fb45007ae40 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:27.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:27.909+0000 7fb468f58700 1 --2- 172.21.15.107:0/1240695052 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4640fce60 0x7fb46410d880 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:27.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:27.909+0000 7fb468f58700 1 --2- 172.21.15.107:0/1240695052 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4640fdfd0 0x7fb46410ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:27.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:27.909+0000 7fb468f58700 1 -- 172.21.15.107:0/1240695052 >> 172.21.15.107:0/1240695052 conn(0x7fb4640f8540 msgr2=0x7fb4640f9af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:27.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:27.909+0000 7fb468f58700 1 -- 172.21.15.107:0/1240695052 shutdown_connections 2024-05-17T21:58:27.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:27.909+0000 7fb468f58700 1 -- 172.21.15.107:0/1240695052 wait complete. 2024-05-17T21:58:27.910 INFO:teuthology.orchestra.run.smithi107.stdout:true 2024-05-17T21:58:28.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.016+0000 7ff193fdc700 1 -- 172.21.15.107:0/3767517833 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff18c0fbcf0 msgr2=0x7ff18c0fc100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:28.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.016+0000 7ff193fdc700 1 --2- 172.21.15.107:0/3767517833 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff18c0fbcf0 0x7ff18c0fc100 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff184009ce0 tx=0x7ff184004d10 comp rx=0 tx=0).stop 2024-05-17T21:58:28.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.016+0000 7ff193fdc700 1 -- 172.21.15.107:0/3767517833 shutdown_connections 2024-05-17T21:58:28.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.016+0000 7ff193fdc700 1 --2- 172.21.15.107:0/3767517833 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff18c0fcfa0 0x7ff18c0fd410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:28.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.016+0000 7ff193fdc700 1 --2- 172.21.15.107:0/3767517833 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff18c0fbcf0 0x7ff18c0fc100 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:28.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.016+0000 7ff193fdc700 1 -- 172.21.15.107:0/3767517833 >> 172.21.15.107:0/3767517833 conn(0x7ff18c0f7450 msgr2=0x7ff18c0f9880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:28.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.016+0000 7ff193fdc700 1 -- 172.21.15.107:0/3767517833 shutdown_connections 2024-05-17T21:58:28.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.016+0000 7ff193fdc700 1 -- 172.21.15.107:0/3767517833 wait complete. 2024-05-17T21:58:28.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.017+0000 7ff193fdc700 1 Processor -- start 2024-05-17T21:58:28.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.017+0000 7ff193fdc700 1 -- start start 2024-05-17T21:58:28.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.017+0000 7ff193fdc700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff18c0fbcf0 0x7ff18c066cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:28.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.017+0000 7ff193fdc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff18c0fcfa0 0x7ff18c112860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:28.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.017+0000 7ff193fdc700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff18c112da0 con 0x7ff18c0fcfa0 2024-05-17T21:58:28.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.017+0000 7ff193fdc700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff18c112f10 con 0x7ff18c0fbcf0 2024-05-17T21:58:28.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.017+0000 7ff191577700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff18c0fcfa0 0x7ff18c112860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:28.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.017+0000 7ff191d78700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff18c0fbcf0 0x7ff18c066cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:28.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.017+0000 7ff191577700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff18c0fcfa0 0x7ff18c112860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59790/0 (socket says 172.21.15.107:59790) 2024-05-17T21:58:28.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.017+0000 7ff191577700 1 -- 172.21.15.107:0/2270055790 learned_addr learned my addr 172.21.15.107:0/2270055790 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:58:28.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.018+0000 7ff191577700 1 -- 172.21.15.107:0/2270055790 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff18c0fbcf0 msgr2=0x7ff18c066cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:28.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.018+0000 7ff191577700 1 --2- 172.21.15.107:0/2270055790 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff18c0fbcf0 0x7ff18c066cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:28.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.018+0000 7ff191577700 1 -- 172.21.15.107:0/2270055790 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff184009940 con 0x7ff18c0fcfa0 2024-05-17T21:58:28.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.018+0000 7ff191577700 1 --2- 172.21.15.107:0/2270055790 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff18c0fcfa0 0x7ff18c112860 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff18000b790 tx=0x7ff180009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:28.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.018+0000 7ff18a7fc700 1 -- 172.21.15.107:0/2270055790 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff18000f6f0 con 0x7ff18c0fcfa0 2024-05-17T21:58:28.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.018+0000 7ff18a7fc700 1 -- 172.21.15.107:0/2270055790 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff18000fd20 con 0x7ff18c0fcfa0 2024-05-17T21:58:28.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.018+0000 7ff193fdc700 1 -- 172.21.15.107:0/2270055790 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff18c1131f0 con 0x7ff18c0fcfa0 2024-05-17T21:58:28.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.018+0000 7ff18a7fc700 1 -- 172.21.15.107:0/2270055790 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff1800103a0 con 0x7ff18c0fcfa0 2024-05-17T21:58:28.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.018+0000 7ff193fdc700 1 -- 172.21.15.107:0/2270055790 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff18c1137b0 con 0x7ff18c0fcfa0 2024-05-17T21:58:28.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.019+0000 7ff193fdc700 1 -- 172.21.15.107:0/2270055790 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff18c04c710 con 0x7ff18c0fcfa0 2024-05-17T21:58:28.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.020+0000 7ff18a7fc700 1 -- 172.21.15.107:0/2270055790 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7ff180011400 con 0x7ff18c0fcfa0 2024-05-17T21:58:28.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.020+0000 7ff18a7fc700 1 --2- 172.21.15.107:0/2270055790 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7ff174078990 0x7ff17407ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:28.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.020+0000 7ff18a7fc700 1 -- 172.21.15.107:0/2270055790 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6410+0+0 (secure 0 0 0) 0x7ff18009a520 con 0x7ff18c0fcfa0 2024-05-17T21:58:28.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.020+0000 7ff191d78700 1 --2- 172.21.15.107:0/2270055790 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7ff174078990 0x7ff17407ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:28.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.021+0000 7ff191d78700 1 --2- 172.21.15.107:0/2270055790 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7ff174078990 0x7ff17407ae40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff184015040 tx=0x7ff184004d10 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:28.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.025+0000 7ff18a7fc700 1 -- 172.21.15.107:0/2270055790 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7ff180062210 con 0x7ff18c0fcfa0 2024-05-17T21:58:28.166 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 17 21:58:27 smithi138 podman[95717]: 2024-05-17 21:58:27.882994561 +0000 UTC m=+0.591207563 container remove c46ca9cf64a0bb0445cf0ac38df4ecd529f3cc865728b11972e30759bd625571 (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-4-deactivate, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, GIT_CLEAN=True, ceph=True) 2024-05-17T21:58:28.166 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 17 21:58:27 smithi138 systemd[1]: ceph-3722731a-1496-11ef-bc9a-c7b262605968@osd.4.service: Deactivated successfully. 2024-05-17T21:58:28.166 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 17 21:58:27 smithi138 systemd[1]: Stopped Ceph osd.4 for 3722731a-1496-11ef-bc9a-c7b262605968. 2024-05-17T21:58:28.166 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 17 21:58:27 smithi138 systemd[1]: ceph-3722731a-1496-11ef-bc9a-c7b262605968@osd.4.service: Consumed 49.742s CPU time. 2024-05-17T21:58:28.166 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 17 21:58:28 smithi138 systemd[1]: Starting Ceph osd.4 for 3722731a-1496-11ef-bc9a-c7b262605968... 2024-05-17T21:58:28.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.248+0000 7ff193fdc700 1 -- 172.21.15.107:0/2270055790 --> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff18c101240 con 0x7ff174078990 2024-05-17T21:58:28.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.249+0000 7ff18a7fc700 1 -- 172.21.15.107:0/2270055790 <== mgr.34100 v2:172.21.15.107:6800/3205268226 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7ff18c101240 con 0x7ff174078990 2024-05-17T21:58:28.242 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.251+0000 7ff193fdc700 1 -- 172.21.15.107:0/2270055790 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7ff174078990 msgr2=0x7ff17407ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:28.242 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.251+0000 7ff193fdc700 1 --2- 172.21.15.107:0/2270055790 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7ff174078990 0x7ff17407ae40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff184015040 tx=0x7ff184004d10 comp rx=0 tx=0).stop 2024-05-17T21:58:28.242 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.251+0000 7ff193fdc700 1 -- 172.21.15.107:0/2270055790 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff18c0fcfa0 msgr2=0x7ff18c112860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:28.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.251+0000 7ff193fdc700 1 --2- 172.21.15.107:0/2270055790 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff18c0fcfa0 0x7ff18c112860 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff18000b790 tx=0x7ff180009250 comp rx=0 tx=0).stop 2024-05-17T21:58:28.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.251+0000 7ff193fdc700 1 -- 172.21.15.107:0/2270055790 shutdown_connections 2024-05-17T21:58:28.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.251+0000 7ff193fdc700 1 --2- 172.21.15.107:0/2270055790 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7ff174078990 0x7ff17407ae40 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:28.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.251+0000 7ff193fdc700 1 --2- 172.21.15.107:0/2270055790 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff18c0fbcf0 0x7ff18c066cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:28.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.251+0000 7ff193fdc700 1 --2- 172.21.15.107:0/2270055790 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff18c0fcfa0 0x7ff18c112860 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:28.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.251+0000 7ff193fdc700 1 -- 172.21.15.107:0/2270055790 >> 172.21.15.107:0/2270055790 conn(0x7ff18c0f7450 msgr2=0x7ff18c0f8a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:28.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.252+0000 7ff193fdc700 1 -- 172.21.15.107:0/2270055790 shutdown_connections 2024-05-17T21:58:28.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.252+0000 7ff193fdc700 1 -- 172.21.15.107:0/2270055790 wait complete. 2024-05-17T21:58:28.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.354+0000 7ff394119700 1 -- 172.21.15.107:0/1502480347 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff38c0fbcc0 msgr2=0x7ff38c0fc0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:28.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.354+0000 7ff394119700 1 --2- 172.21.15.107:0/1502480347 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff38c0fbcc0 0x7ff38c0fc0d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff37c004940 tx=0x7ff37c009e30 comp rx=0 tx=0).stop 2024-05-17T21:58:28.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.354+0000 7ff394119700 1 -- 172.21.15.107:0/1502480347 shutdown_connections 2024-05-17T21:58:28.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.354+0000 7ff394119700 1 --2- 172.21.15.107:0/1502480347 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff38c0fcf70 0x7ff38c0fd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:28.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.354+0000 7ff394119700 1 --2- 172.21.15.107:0/1502480347 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff38c0fbcc0 0x7ff38c0fc0d0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:28.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.354+0000 7ff394119700 1 -- 172.21.15.107:0/1502480347 >> 172.21.15.107:0/1502480347 conn(0x7ff38c0f7440 msgr2=0x7ff38c0f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:28.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.354+0000 7ff394119700 1 -- 172.21.15.107:0/1502480347 shutdown_connections 2024-05-17T21:58:28.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.354+0000 7ff394119700 1 -- 172.21.15.107:0/1502480347 wait complete. 2024-05-17T21:58:28.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.354+0000 7ff394119700 1 Processor -- start 2024-05-17T21:58:28.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.354+0000 7ff394119700 1 -- start start 2024-05-17T21:58:28.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.355+0000 7ff394119700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff38c0fbcc0 0x7ff38c10c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:28.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.355+0000 7ff394119700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff38c0fcf70 0x7ff38c10cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:28.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.355+0000 7ff394119700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff38c10d3f0 con 0x7ff38c0fbcc0 2024-05-17T21:58:28.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.355+0000 7ff394119700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff38c10d560 con 0x7ff38c0fcf70 2024-05-17T21:58:28.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.355+0000 7ff391eb5700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff38c0fbcc0 0x7ff38c10c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:28.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.355+0000 7ff391eb5700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff38c0fbcc0 0x7ff38c10c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59812/0 (socket says 172.21.15.107:59812) 2024-05-17T21:58:28.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.355+0000 7ff391eb5700 1 -- 172.21.15.107:0/853416595 learned_addr learned my addr 172.21.15.107:0/853416595 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:58:28.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.355+0000 7ff3916b4700 1 --2- 172.21.15.107:0/853416595 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff38c0fcf70 0x7ff38c10cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:28.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.355+0000 7ff391eb5700 1 -- 172.21.15.107:0/853416595 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff38c0fcf70 msgr2=0x7ff38c10cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:28.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.355+0000 7ff391eb5700 1 --2- 172.21.15.107:0/853416595 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff38c0fcf70 0x7ff38c10cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:28.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.355+0000 7ff391eb5700 1 -- 172.21.15.107:0/853416595 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff37c005040 con 0x7ff38c0fbcc0 2024-05-17T21:58:28.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.355+0000 7ff391eb5700 1 --2- 172.21.15.107:0/853416595 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff38c0fbcc0 0x7ff38c10c8b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff37c009380 tx=0x7ff37c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:28.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.355+0000 7ff3827fc700 1 -- 172.21.15.107:0/853416595 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff37c00e040 con 0x7ff38c0fbcc0 2024-05-17T21:58:28.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.355+0000 7ff3827fc700 1 -- 172.21.15.107:0/853416595 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff37c00bd90 con 0x7ff38c0fbcc0 2024-05-17T21:58:28.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.355+0000 7ff394119700 1 -- 172.21.15.107:0/853416595 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff38c111f60 con 0x7ff38c0fbcc0 2024-05-17T21:58:28.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.355+0000 7ff3827fc700 1 -- 172.21.15.107:0/853416595 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff37c01c630 con 0x7ff38c0fbcc0 2024-05-17T21:58:28.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.355+0000 7ff394119700 1 -- 172.21.15.107:0/853416595 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff38c112490 con 0x7ff38c0fbcc0 2024-05-17T21:58:28.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.356+0000 7ff394119700 1 -- 172.21.15.107:0/853416595 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff38c04c710 con 0x7ff38c0fbcc0 2024-05-17T21:58:28.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.357+0000 7ff3827fc700 1 -- 172.21.15.107:0/853416595 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7ff37c0139f0 con 0x7ff38c0fbcc0 2024-05-17T21:58:28.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.357+0000 7ff3827fc700 1 --2- 172.21.15.107:0/853416595 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7ff378078990 0x7ff37807ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:28.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.358+0000 7ff3827fc700 1 -- 172.21.15.107:0/853416595 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6410+0+0 (secure 0 0 0) 0x7ff37c09bb00 con 0x7ff38c0fbcc0 2024-05-17T21:58:28.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.358+0000 7ff3916b4700 1 --2- 172.21.15.107:0/853416595 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7ff378078990 0x7ff37807ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:28.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.358+0000 7ff3916b4700 1 --2- 172.21.15.107:0/853416595 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7ff378078990 0x7ff37807ae40 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff3880041d0 tx=0x7ff388009040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:28.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.362+0000 7ff3827fc700 1 -- 172.21.15.107:0/853416595 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7ff37c0637f0 con 0x7ff38c0fbcc0 2024-05-17T21:58:28.447 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:28 smithi138 ceph-mon[81376]: pgmap v141: 65 pgs: 13 peering, 3 stale+active+clean, 2 active+undersized, 47 active+clean; 258 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s 2024-05-17T21:58:28.447 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:28 smithi138 ceph-mon[81376]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-05-17T21:58:28.448 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 17 21:58:28 smithi138 podman[95816]: 2024-05-17 21:58:28.225034992 +0000 UTC m=+0.017812814 image pull 8880f7613d0283efa7002505fdccdbc3e37d3b5da31c57fa22d62262c5dab66c quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b 2024-05-17T21:58:28.448 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 17 21:58:28 smithi138 podman[95816]: 2024-05-17 21:58:28.334065141 +0000 UTC m=+0.126842938 container create 04f583860839aa780e5f39ad22ead6ab9faf22fb413d356331e4ec701d61999d (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-4-activate, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-05-17T21:58:28.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.574+0000 7ff394119700 1 -- 172.21.15.107:0/853416595 --> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7ff38c112770 con 0x7ff378078990 2024-05-17T21:58:28.565 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:28 smithi107 ceph-mon[93224]: pgmap v141: 65 pgs: 13 peering, 3 stale+active+clean, 2 active+undersized, 47 active+clean; 258 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s 2024-05-17T21:58:28.565 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:28 smithi107 ceph-mon[93224]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-05-17T21:58:28.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.584+0000 7ff3827fc700 1 -- 172.21.15.107:0/853416595 <== mgr.34100 v2:172.21.15.107:6800/3205268226 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7ff38c112770 con 0x7ff378078990 2024-05-17T21:58:28.575 INFO:teuthology.orchestra.run.smithi107.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-17T21:58:28.575 INFO:teuthology.orchestra.run.smithi107.stdout:alertmanager.smithi107 smithi107 *:9093,9094 running (6m) 63s ago 14m 24.3M - 0.25.0 c8568f914cd2 bac2a944ae42 2024-05-17T21:58:28.575 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi107 smithi107 running (2m) 63s ago 14m 6669k - 19.0.0-3794-g931809a9 8880f7613d02 face35de205b 2024-05-17T21:58:28.575 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi138 smithi138 running (2m) 29s ago 13m 6665k - 19.0.0-3794-g931809a9 8880f7613d02 ad59e33e6d43 2024-05-17T21:58:28.575 INFO:teuthology.orchestra.run.smithi107.stdout:grafana.smithi107 smithi107 *:3000 running (5m) 63s ago 14m 84.2M - 9.4.7 954c08fa6188 8c747e777bc5 2024-05-17T21:58:28.575 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.drqody smithi107 running (9m) 63s ago 9m 103M - 17.2.7-952-g9903f776 d15a36853796 0d851bc912f0 2024-05-17T21:58:28.576 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.idmbgj smithi107 running (9m) 63s ago 9m 13.8M - 17.2.7-952-g9903f776 d15a36853796 1348b09ab29e 2024-05-17T21:58:28.576 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi138.gwbtyd smithi138 running (9m) 29s ago 9m 152M - 17.2.7-952-g9903f776 d15a36853796 fd2ba75354c0 2024-05-17T21:58:28.576 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi138.zckrvs smithi138 running (8m) 29s ago 8m 14.6M - 17.2.7-952-g9903f776 d15a36853796 a1cfd2fe4cda 2024-05-17T21:58:28.576 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi107.xhhyiw smithi107 *:8443,9283,8765 running (5m) 63s ago 15m 587M - 19.0.0-3794-g931809a9 8880f7613d02 5acf3e511342 2024-05-17T21:58:28.576 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi138.napnll smithi138 *:8443,9283,8765 running (4m) 29s ago 13m 533M - 19.0.0-3794-g931809a9 8880f7613d02 d627761dbd0d 2024-05-17T21:58:28.576 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi107 smithi107 running (4m) 63s ago 16m 115M 2048M 19.0.0-3794-g931809a9 8880f7613d02 9717f3169f2f 2024-05-17T21:58:28.576 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi138 smithi138 running (3m) 29s ago 13m 105M 2048M 19.0.0-3794-g931809a9 8880f7613d02 55aeed62bf4d 2024-05-17T21:58:28.576 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi107 smithi107 *:9100 running (6m) 63s ago 14m 17.4M - 1.5.0 0da6a335fe13 49bec71f5487 2024-05-17T21:58:28.576 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi138 smithi138 *:9100 running (5m) 29s ago 13m 15.5M - 1.5.0 0da6a335fe13 e52824376658 2024-05-17T21:58:28.576 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0 smithi107 running (2m) 63s ago 12m 156M 1557M 19.0.0-3794-g931809a9 8880f7613d02 b319bc93500d 2024-05-17T21:58:28.576 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1 smithi107 running (104s) 63s ago 12m 151M 1557M 19.0.0-3794-g931809a9 8880f7613d02 f94b206be6eb 2024-05-17T21:58:28.576 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2 smithi107 running (68s) 63s ago 11m 79.3M 1557M 19.0.0-3794-g931809a9 8880f7613d02 add7015647af 2024-05-17T21:58:28.576 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3 smithi138 running (32s) 29s ago 11m 79.7M 2581M 19.0.0-3794-g931809a9 8880f7613d02 416273d327c7 2024-05-17T21:58:28.576 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4 smithi138 running (11m) 29s ago 11m 206M 2581M 17.2.7-952-g9903f776 d15a36853796 b306a36a744e 2024-05-17T21:58:28.576 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5 smithi138 running (10m) 29s ago 10m 191M 2581M 17.2.7-952-g9903f776 d15a36853796 7c8d8ed74974 2024-05-17T21:58:28.576 INFO:teuthology.orchestra.run.smithi107.stdout:prometheus.smithi107 smithi107 *:9095 running (4m) 63s ago 14m 59.1M - 2.43.0 a07b618ecd1d bf5f609bbd6e 2024-05-17T21:58:28.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.587+0000 7ff394119700 1 -- 172.21.15.107:0/853416595 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7ff378078990 msgr2=0x7ff37807ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:28.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.587+0000 7ff394119700 1 --2- 172.21.15.107:0/853416595 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7ff378078990 0x7ff37807ae40 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff3880041d0 tx=0x7ff388009040 comp rx=0 tx=0).stop 2024-05-17T21:58:28.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.587+0000 7ff394119700 1 -- 172.21.15.107:0/853416595 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff38c0fbcc0 msgr2=0x7ff38c10c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:28.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.587+0000 7ff394119700 1 --2- 172.21.15.107:0/853416595 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff38c0fbcc0 0x7ff38c10c8b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff37c009380 tx=0x7ff37c009e30 comp rx=0 tx=0).stop 2024-05-17T21:58:28.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.587+0000 7ff394119700 1 -- 172.21.15.107:0/853416595 shutdown_connections 2024-05-17T21:58:28.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.587+0000 7ff394119700 1 --2- 172.21.15.107:0/853416595 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7ff378078990 0x7ff37807ae40 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:28.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.587+0000 7ff394119700 1 --2- 172.21.15.107:0/853416595 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff38c0fbcc0 0x7ff38c10c8b0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:28.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.587+0000 7ff394119700 1 --2- 172.21.15.107:0/853416595 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff38c0fcf70 0x7ff38c10cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:28.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.587+0000 7ff394119700 1 -- 172.21.15.107:0/853416595 >> 172.21.15.107:0/853416595 conn(0x7ff38c0f7440 msgr2=0x7ff38c0f8bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:28.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.587+0000 7ff394119700 1 -- 172.21.15.107:0/853416595 shutdown_connections 2024-05-17T21:58:28.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.587+0000 7ff394119700 1 -- 172.21.15.107:0/853416595 wait complete. 2024-05-17T21:58:28.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.688+0000 7f996c672700 1 -- 172.21.15.107:0/238934482 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99640fed70 msgr2=0x7f99640faeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:28.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.688+0000 7f996c672700 1 --2- 172.21.15.107:0/238934482 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99640fed70 0x7f99640faeb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9958004940 tx=0x7f9958009e30 comp rx=0 tx=0).stop 2024-05-17T21:58:28.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.688+0000 7f996c672700 1 -- 172.21.15.107:0/238934482 shutdown_connections 2024-05-17T21:58:28.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.688+0000 7f996c672700 1 --2- 172.21.15.107:0/238934482 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99640fed70 0x7f99640faeb0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:28.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.688+0000 7f996c672700 1 --2- 172.21.15.107:0/238934482 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f99640fe420 0x7f99640fe830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:28.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.688+0000 7f996c672700 1 -- 172.21.15.107:0/238934482 >> 172.21.15.107:0/238934482 conn(0x7f99640f6be0 msgr2=0x7f99640f8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:28.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.688+0000 7f996c672700 1 -- 172.21.15.107:0/238934482 shutdown_connections 2024-05-17T21:58:28.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.688+0000 7f996c672700 1 -- 172.21.15.107:0/238934482 wait complete. 2024-05-17T21:58:28.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.689+0000 7f996c672700 1 Processor -- start 2024-05-17T21:58:28.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.689+0000 7f996c672700 1 -- start start 2024-05-17T21:58:28.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.689+0000 7f996c672700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99640fe420 0x7f996410fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:28.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.689+0000 7f996c672700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f99640fed70 0x7f9964110060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:28.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.689+0000 7f996c672700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9964110660 con 0x7f99640fe420 2024-05-17T21:58:28.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.689+0000 7f996c672700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99641107d0 con 0x7f99640fed70 2024-05-17T21:58:28.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.689+0000 7f996a40e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99640fe420 0x7f996410fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:28.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.689+0000 7f996a40e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99640fe420 0x7f996410fb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59824/0 (socket says 172.21.15.107:59824) 2024-05-17T21:58:28.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.689+0000 7f996a40e700 1 -- 172.21.15.107:0/3648703892 learned_addr learned my addr 172.21.15.107:0/3648703892 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:58:28.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.689+0000 7f9969c0d700 1 --2- 172.21.15.107:0/3648703892 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f99640fed70 0x7f9964110060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:28.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.689+0000 7f996a40e700 1 -- 172.21.15.107:0/3648703892 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f99640fed70 msgr2=0x7f9964110060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:28.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.689+0000 7f996a40e700 1 --2- 172.21.15.107:0/3648703892 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f99640fed70 0x7f9964110060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:28.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.689+0000 7f996a40e700 1 -- 172.21.15.107:0/3648703892 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9958005040 con 0x7f99640fe420 2024-05-17T21:58:28.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.690+0000 7f996a40e700 1 --2- 172.21.15.107:0/3648703892 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99640fe420 0x7f996410fb20 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f995c002a50 tx=0x7f995c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:28.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.690+0000 7f9962ffd700 1 -- 172.21.15.107:0/3648703892 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f995c007cc0 con 0x7f99640fe420 2024-05-17T21:58:28.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.690+0000 7f9962ffd700 1 -- 172.21.15.107:0/3648703892 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f995c00f040 con 0x7f99640fe420 2024-05-17T21:58:28.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.690+0000 7f996c672700 1 -- 172.21.15.107:0/3648703892 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9964115230 con 0x7f99640fe420 2024-05-17T21:58:28.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.690+0000 7f9962ffd700 1 -- 172.21.15.107:0/3648703892 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f995c00da70 con 0x7f99640fe420 2024-05-17T21:58:28.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.690+0000 7f996c672700 1 -- 172.21.15.107:0/3648703892 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99641157f0 con 0x7f99640fe420 2024-05-17T21:58:28.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.691+0000 7f996c672700 1 -- 172.21.15.107:0/3648703892 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9964101b60 con 0x7f99640fe420 2024-05-17T21:58:28.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.692+0000 7f9962ffd700 1 -- 172.21.15.107:0/3648703892 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f995c00dc10 con 0x7f99640fe420 2024-05-17T21:58:28.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.692+0000 7f9962ffd700 1 --2- 172.21.15.107:0/3648703892 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f994c078990 0x7f994c07ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:28.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.692+0000 7f9969c0d700 1 --2- 172.21.15.107:0/3648703892 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f994c078990 0x7f994c07ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:28.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.692+0000 7f9962ffd700 1 -- 172.21.15.107:0/3648703892 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f995c099d70 con 0x7f99640fe420 2024-05-17T21:58:28.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.692+0000 7f9969c0d700 1 --2- 172.21.15.107:0/3648703892 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f994c078990 0x7f994c07ae40 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f995800b970 tx=0x7f9958009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:28.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.697+0000 7f9962ffd700 1 -- 172.21.15.107:0/3648703892 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f995c0619e0 con 0x7f99640fe420 2024-05-17T21:58:28.825 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 17 21:58:28 smithi138 podman[95816]: 2024-05-17 21:58:28.570190978 +0000 UTC m=+0.362968791 container init 04f583860839aa780e5f39ad22ead6ab9faf22fb413d356331e4ec701d61999d (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-4-activate, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-17T21:58:28.825 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 17 21:58:28 smithi138 podman[95816]: 2024-05-17 21:58:28.57368841 +0000 UTC m=+0.366466225 container start 04f583860839aa780e5f39ad22ead6ab9faf22fb413d356331e4ec701d61999d (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-4-activate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-17T21:58:28.825 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 17 21:58:28 smithi138 podman[95816]: 2024-05-17 21:58:28.653413921 +0000 UTC m=+0.446191739 container attach 04f583860839aa780e5f39ad22ead6ab9faf22fb413d356331e4ec701d61999d (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-4-activate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-05-17T21:58:28.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.988+0000 7f996c672700 1 -- 172.21.15.107:0/3648703892 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f99641058f0 con 0x7f99640fe420 2024-05-17T21:58:28.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.988+0000 7f9962ffd700 1 -- 172.21.15.107:0/3648703892 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+839 (secure 0 0 0) 0x7f995c061380 con 0x7f99640fe420 2024-05-17T21:58:28.979 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-05-17T21:58:28.979 INFO:teuthology.orchestra.run.smithi107.stdout: "mon": { 2024-05-17T21:58:28.979 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev)": 2 2024-05-17T21:58:28.979 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-05-17T21:58:28.979 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr": { 2024-05-17T21:58:28.979 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev)": 2 2024-05-17T21:58:28.979 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-05-17T21:58:28.980 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": { 2024-05-17T21:58:28.980 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-952-g9903f776 (9903f7767386a9d9efadb7c1effdcd3064d71804) quincy (stable)": 1, 2024-05-17T21:58:28.980 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev)": 4 2024-05-17T21:58:28.980 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-05-17T21:58:28.980 INFO:teuthology.orchestra.run.smithi107.stdout: "mds": { 2024-05-17T21:58:28.980 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-952-g9903f776 (9903f7767386a9d9efadb7c1effdcd3064d71804) quincy (stable)": 4 2024-05-17T21:58:28.980 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-05-17T21:58:28.980 INFO:teuthology.orchestra.run.smithi107.stdout: "overall": { 2024-05-17T21:58:28.980 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-952-g9903f776 (9903f7767386a9d9efadb7c1effdcd3064d71804) quincy (stable)": 5, 2024-05-17T21:58:28.980 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev)": 8 2024-05-17T21:58:28.980 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-05-17T21:58:28.980 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-05-17T21:58:28.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.990+0000 7f996c672700 1 -- 172.21.15.107:0/3648703892 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f994c078990 msgr2=0x7f994c07ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:28.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.990+0000 7f996c672700 1 --2- 172.21.15.107:0/3648703892 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f994c078990 0x7f994c07ae40 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f995800b970 tx=0x7f9958009e30 comp rx=0 tx=0).stop 2024-05-17T21:58:28.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.990+0000 7f996c672700 1 -- 172.21.15.107:0/3648703892 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99640fe420 msgr2=0x7f996410fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:28.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.990+0000 7f996c672700 1 --2- 172.21.15.107:0/3648703892 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99640fe420 0x7f996410fb20 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f995c002a50 tx=0x7f995c0046b0 comp rx=0 tx=0).stop 2024-05-17T21:58:28.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.990+0000 7f996c672700 1 -- 172.21.15.107:0/3648703892 shutdown_connections 2024-05-17T21:58:28.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.990+0000 7f996c672700 1 --2- 172.21.15.107:0/3648703892 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f994c078990 0x7f994c07ae40 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:28.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.990+0000 7f996c672700 1 --2- 172.21.15.107:0/3648703892 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99640fe420 0x7f996410fb20 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:28.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.990+0000 7f996c672700 1 --2- 172.21.15.107:0/3648703892 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f99640fed70 0x7f9964110060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:28.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.990+0000 7f996c672700 1 -- 172.21.15.107:0/3648703892 >> 172.21.15.107:0/3648703892 conn(0x7f99640f6be0 msgr2=0x7f99640f8250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:28.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.991+0000 7f996c672700 1 -- 172.21.15.107:0/3648703892 shutdown_connections 2024-05-17T21:58:28.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:28.991+0000 7f996c672700 1 -- 172.21.15.107:0/3648703892 wait complete. 2024-05-17T21:58:29.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.093+0000 7f8aa9f95700 1 -- 172.21.15.107:0/973279054 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8aa40fce60 msgr2=0x7f8aa40fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:29.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.093+0000 7f8aa9f95700 1 --2- 172.21.15.107:0/973279054 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8aa40fce60 0x7f8aa40fd270 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8a94009ce0 tx=0x7f8a94004d10 comp rx=0 tx=0).stop 2024-05-17T21:58:29.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.093+0000 7f8aa9f95700 1 -- 172.21.15.107:0/973279054 shutdown_connections 2024-05-17T21:58:29.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.093+0000 7f8aa9f95700 1 --2- 172.21.15.107:0/973279054 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8aa40fdfd0 0x7f8aa40fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:29.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.093+0000 7f8aa9f95700 1 --2- 172.21.15.107:0/973279054 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8aa40fce60 0x7f8aa40fd270 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:29.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.093+0000 7f8aa9f95700 1 -- 172.21.15.107:0/973279054 >> 172.21.15.107:0/973279054 conn(0x7f8aa40f8540 msgr2=0x7f8aa40fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:29.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.093+0000 7f8aa9f95700 1 -- 172.21.15.107:0/973279054 shutdown_connections 2024-05-17T21:58:29.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.093+0000 7f8aa9f95700 1 -- 172.21.15.107:0/973279054 wait complete. 2024-05-17T21:58:29.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.093+0000 7f8aa9f95700 1 Processor -- start 2024-05-17T21:58:29.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.093+0000 7f8aa9f95700 1 -- start start 2024-05-17T21:58:29.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.094+0000 7f8aa9f95700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8aa40fce60 0x7f8aa410d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:29.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.094+0000 7f8aa9f95700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8aa40fdfd0 0x7f8aa410de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:29.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.094+0000 7f8aa9f95700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa410e450 con 0x7f8aa40fce60 2024-05-17T21:58:29.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.094+0000 7f8aa9f95700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa410e5c0 con 0x7f8aa40fdfd0 2024-05-17T21:58:29.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.094+0000 7f8aa37fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8aa40fce60 0x7f8aa410d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:29.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.094+0000 7f8aa37fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8aa40fce60 0x7f8aa410d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59842/0 (socket says 172.21.15.107:59842) 2024-05-17T21:58:29.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.094+0000 7f8aa37fe700 1 -- 172.21.15.107:0/3494587088 learned_addr learned my addr 172.21.15.107:0/3494587088 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:58:29.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.094+0000 7f8aa2ffd700 1 --2- 172.21.15.107:0/3494587088 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8aa40fdfd0 0x7f8aa410de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:29.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.094+0000 7f8aa37fe700 1 -- 172.21.15.107:0/3494587088 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8aa40fdfd0 msgr2=0x7f8aa410de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:29.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.094+0000 7f8aa37fe700 1 --2- 172.21.15.107:0/3494587088 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8aa40fdfd0 0x7f8aa410de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:29.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.094+0000 7f8aa37fe700 1 -- 172.21.15.107:0/3494587088 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a94009940 con 0x7f8aa40fce60 2024-05-17T21:58:29.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.094+0000 7f8aa37fe700 1 --2- 172.21.15.107:0/3494587088 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8aa40fce60 0x7f8aa410d910 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8a9400bfd0 tx=0x7f8a94004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:29.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.094+0000 7f8a8bfff700 1 -- 172.21.15.107:0/3494587088 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8a9400e040 con 0x7f8aa40fce60 2024-05-17T21:58:29.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.094+0000 7f8a8bfff700 1 -- 172.21.15.107:0/3494587088 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8a9401cd00 con 0x7f8aa40fce60 2024-05-17T21:58:29.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.094+0000 7f8aa9f95700 1 -- 172.21.15.107:0/3494587088 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8aa4112fc0 con 0x7f8aa40fce60 2024-05-17T21:58:29.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.094+0000 7f8a8bfff700 1 -- 172.21.15.107:0/3494587088 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8a94012c40 con 0x7f8aa40fce60 2024-05-17T21:58:29.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.094+0000 7f8aa9f95700 1 -- 172.21.15.107:0/3494587088 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8aa41134f0 con 0x7f8aa40fce60 2024-05-17T21:58:29.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.095+0000 7f8aa9f95700 1 -- 172.21.15.107:0/3494587088 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8aa404c710 con 0x7f8aa40fce60 2024-05-17T21:58:29.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.096+0000 7f8a8bfff700 1 -- 172.21.15.107:0/3494587088 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f8a940124a0 con 0x7f8aa40fce60 2024-05-17T21:58:29.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.097+0000 7f8a8bfff700 1 --2- 172.21.15.107:0/3494587088 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f8a8c078990 0x7f8a8c07ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:29.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.097+0000 7f8a8bfff700 1 -- 172.21.15.107:0/3494587088 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f8a9409d030 con 0x7f8aa40fce60 2024-05-17T21:58:29.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.097+0000 7f8aa2ffd700 1 --2- 172.21.15.107:0/3494587088 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f8a8c078990 0x7f8a8c07ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:29.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.097+0000 7f8aa2ffd700 1 --2- 172.21.15.107:0/3494587088 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f8a8c078990 0x7f8a8c07ae40 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8a98009b80 tx=0x7f8a9800b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:29.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.101+0000 7f8a8bfff700 1 -- 172.21.15.107:0/3494587088 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f8a94064d20 con 0x7f8aa40fce60 2024-05-17T21:58:29.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.350+0000 7f8aa9f95700 1 -- 172.21.15.107:0/3494587088 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f8aa41058c0 con 0x7f8aa40fce60 2024-05-17T21:58:29.341 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:29 smithi107 ceph-mon[93224]: from='client.34344 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:58:29.341 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:29 smithi107 ceph-mon[93224]: from='client.? 172.21.15.107:0/3648703892' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:58:29.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.351+0000 7f8a8bfff700 1 -- 172.21.15.107:0/3494587088 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1910 (secure 0 0 0) 0x7f8a940646c0 con 0x7f8aa40fce60 2024-05-17T21:58:29.342 INFO:teuthology.orchestra.run.smithi107.stdout:e15 2024-05-17T21:58:29.342 INFO:teuthology.orchestra.run.smithi107.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-17T21:58:29.342 INFO:teuthology.orchestra.run.smithi107.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-17T21:58:29.342 INFO:teuthology.orchestra.run.smithi107.stdout:legacy client fscid: 1 2024-05-17T21:58:29.342 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:58:29.342 INFO:teuthology.orchestra.run.smithi107.stdout:Filesystem 'cephfs' (1) 2024-05-17T21:58:29.342 INFO:teuthology.orchestra.run.smithi107.stdout:fs_name cephfs 2024-05-17T21:58:29.342 INFO:teuthology.orchestra.run.smithi107.stdout:epoch 15 2024-05-17T21:58:29.342 INFO:teuthology.orchestra.run.smithi107.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-17T21:58:29.343 INFO:teuthology.orchestra.run.smithi107.stdout:created 2024-05-17T21:49:16.836333+0000 2024-05-17T21:58:29.343 INFO:teuthology.orchestra.run.smithi107.stdout:modified 2024-05-17T21:49:34.873628+0000 2024-05-17T21:58:29.343 INFO:teuthology.orchestra.run.smithi107.stdout:tableserver 0 2024-05-17T21:58:29.343 INFO:teuthology.orchestra.run.smithi107.stdout:root 0 2024-05-17T21:58:29.343 INFO:teuthology.orchestra.run.smithi107.stdout:session_timeout 60 2024-05-17T21:58:29.343 INFO:teuthology.orchestra.run.smithi107.stdout:session_autoclose 300 2024-05-17T21:58:29.343 INFO:teuthology.orchestra.run.smithi107.stdout:max_file_size 1099511627776 2024-05-17T21:58:29.343 INFO:teuthology.orchestra.run.smithi107.stdout:max_xattr_size 65536 2024-05-17T21:58:29.343 INFO:teuthology.orchestra.run.smithi107.stdout:required_client_features {} 2024-05-17T21:58:29.343 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure 0 2024-05-17T21:58:29.343 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure_osd_epoch 46 2024-05-17T21:58:29.343 INFO:teuthology.orchestra.run.smithi107.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-17T21:58:29.343 INFO:teuthology.orchestra.run.smithi107.stdout:max_mds 1 2024-05-17T21:58:29.343 INFO:teuthology.orchestra.run.smithi107.stdout:in 0 2024-05-17T21:58:29.343 INFO:teuthology.orchestra.run.smithi107.stdout:up {0=24259} 2024-05-17T21:58:29.344 INFO:teuthology.orchestra.run.smithi107.stdout:failed 2024-05-17T21:58:29.344 INFO:teuthology.orchestra.run.smithi107.stdout:damaged 2024-05-17T21:58:29.344 INFO:teuthology.orchestra.run.smithi107.stdout:stopped 2024-05-17T21:58:29.344 INFO:teuthology.orchestra.run.smithi107.stdout:data_pools [3] 2024-05-17T21:58:29.344 INFO:teuthology.orchestra.run.smithi107.stdout:metadata_pool 2 2024-05-17T21:58:29.344 INFO:teuthology.orchestra.run.smithi107.stdout:inline_data enabled 2024-05-17T21:58:29.344 INFO:teuthology.orchestra.run.smithi107.stdout:balancer 2024-05-17T21:58:29.344 INFO:teuthology.orchestra.run.smithi107.stdout:bal_rank_mask -1 2024-05-17T21:58:29.344 INFO:teuthology.orchestra.run.smithi107.stdout:standby_count_wanted 1 2024-05-17T21:58:29.344 INFO:teuthology.orchestra.run.smithi107.stdout:qdb_cluster leader: 0 members: 2024-05-17T21:58:29.344 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi138.gwbtyd{0:24259} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.138:6824/2779634919,v1:172.21.15.138:6825/2779634919] compat {c=[1],r=[1],i=[7ff]}] 2024-05-17T21:58:29.344 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.drqody{0:14486} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.107:6828/2275807100,v1:172.21.15.107:6829/2275807100] compat {c=[1],r=[1],i=[7ff]}] 2024-05-17T21:58:29.345 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:58:29.345 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:58:29.345 INFO:teuthology.orchestra.run.smithi107.stdout:Standby daemons: 2024-05-17T21:58:29.345 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:58:29.345 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.idmbgj{-1:14498} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.107:6826/1460186079,v1:172.21.15.107:6827/1460186079] compat {c=[1],r=[1],i=[7ff]}] 2024-05-17T21:58:29.345 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi138.zckrvs{-1:24267} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.138:6826/170628661,v1:172.21.15.138:6827/170628661] compat {c=[1],r=[1],i=[7ff]}] 2024-05-17T21:58:29.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.354+0000 7f8aa9f95700 1 -- 172.21.15.107:0/3494587088 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f8a8c078990 msgr2=0x7f8a8c07ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:29.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.354+0000 7f8aa9f95700 1 --2- 172.21.15.107:0/3494587088 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f8a8c078990 0x7f8a8c07ae40 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8a98009b80 tx=0x7f8a9800b040 comp rx=0 tx=0).stop 2024-05-17T21:58:29.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.354+0000 7f8aa9f95700 1 -- 172.21.15.107:0/3494587088 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8aa40fce60 msgr2=0x7f8aa410d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:29.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.354+0000 7f8aa9f95700 1 --2- 172.21.15.107:0/3494587088 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8aa40fce60 0x7f8aa410d910 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8a9400bfd0 tx=0x7f8a94004d10 comp rx=0 tx=0).stop 2024-05-17T21:58:29.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.354+0000 7f8aa9f95700 1 -- 172.21.15.107:0/3494587088 shutdown_connections 2024-05-17T21:58:29.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.354+0000 7f8aa9f95700 1 --2- 172.21.15.107:0/3494587088 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f8a8c078990 0x7f8a8c07ae40 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:29.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.354+0000 7f8aa9f95700 1 --2- 172.21.15.107:0/3494587088 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8aa40fce60 0x7f8aa410d910 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:29.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.354+0000 7f8aa9f95700 1 --2- 172.21.15.107:0/3494587088 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8aa40fdfd0 0x7f8aa410de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:29.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.354+0000 7f8aa9f95700 1 -- 172.21.15.107:0/3494587088 >> 172.21.15.107:0/3494587088 conn(0x7f8aa40f8540 msgr2=0x7f8aa40f9c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:29.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.354+0000 7f8aa9f95700 1 -- 172.21.15.107:0/3494587088 shutdown_connections 2024-05-17T21:58:29.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.354+0000 7f8aa9f95700 1 -- 172.21.15.107:0/3494587088 wait complete. 2024-05-17T21:58:29.346 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 15 2024-05-17T21:58:29.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.457+0000 7f6c77dfc700 1 -- 172.21.15.107:0/3059813752 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c700fb1e0 msgr2=0x7f6c700fb5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:29.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.457+0000 7f6c77dfc700 1 --2- 172.21.15.107:0/3059813752 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c700fb1e0 0x7f6c700fb5f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c004940 tx=0x7f6c6c009e30 comp rx=0 tx=0).stop 2024-05-17T21:58:29.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.458+0000 7f6c77dfc700 1 -- 172.21.15.107:0/3059813752 shutdown_connections 2024-05-17T21:58:29.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.458+0000 7f6c77dfc700 1 --2- 172.21.15.107:0/3059813752 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c700fbb30 0x7f6c700fe390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:29.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.458+0000 7f6c77dfc700 1 --2- 172.21.15.107:0/3059813752 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c700fb1e0 0x7f6c700fb5f0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:29.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.458+0000 7f6c77dfc700 1 -- 172.21.15.107:0/3059813752 >> 172.21.15.107:0/3059813752 conn(0x7f6c700f6be0 msgr2=0x7f6c700f8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:29.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.458+0000 7f6c77dfc700 1 -- 172.21.15.107:0/3059813752 shutdown_connections 2024-05-17T21:58:29.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.458+0000 7f6c77dfc700 1 -- 172.21.15.107:0/3059813752 wait complete. 2024-05-17T21:58:29.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.458+0000 7f6c77dfc700 1 Processor -- start 2024-05-17T21:58:29.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.458+0000 7f6c77dfc700 1 -- start start 2024-05-17T21:58:29.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.458+0000 7f6c77dfc700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c700fb1e0 0x7f6c7010db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:29.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.459+0000 7f6c77dfc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c700fbb30 0x7f6c7010e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:29.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.459+0000 7f6c77dfc700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c7010e700 con 0x7f6c700fbb30 2024-05-17T21:58:29.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.459+0000 7f6c77dfc700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c701130a0 con 0x7f6c700fb1e0 2024-05-17T21:58:29.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.459+0000 7f6c75397700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c700fbb30 0x7f6c7010e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:29.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.459+0000 7f6c75397700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c700fbb30 0x7f6c7010e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59846/0 (socket says 172.21.15.107:59846) 2024-05-17T21:58:29.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.459+0000 7f6c75397700 1 -- 172.21.15.107:0/1294231835 learned_addr learned my addr 172.21.15.107:0/1294231835 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:58:29.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.459+0000 7f6c75b98700 1 --2- 172.21.15.107:0/1294231835 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c700fb1e0 0x7f6c7010db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:29.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.459+0000 7f6c75397700 1 -- 172.21.15.107:0/1294231835 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c700fb1e0 msgr2=0x7f6c7010db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:29.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.459+0000 7f6c75397700 1 --2- 172.21.15.107:0/1294231835 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c700fb1e0 0x7f6c7010db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:29.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.459+0000 7f6c75397700 1 -- 172.21.15.107:0/1294231835 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c6c005040 con 0x7f6c700fbb30 2024-05-17T21:58:29.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.459+0000 7f6c75397700 1 --2- 172.21.15.107:0/1294231835 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c700fbb30 0x7f6c7010e070 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6c6000b790 tx=0x7f6c60009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:29.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.459+0000 7f6c667fc700 1 -- 172.21.15.107:0/1294231835 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c6000f6f0 con 0x7f6c700fbb30 2024-05-17T21:58:29.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.459+0000 7f6c77dfc700 1 -- 172.21.15.107:0/1294231835 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c70113300 con 0x7f6c700fbb30 2024-05-17T21:58:29.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.459+0000 7f6c667fc700 1 -- 172.21.15.107:0/1294231835 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6c6000fd20 con 0x7f6c700fbb30 2024-05-17T21:58:29.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.459+0000 7f6c667fc700 1 -- 172.21.15.107:0/1294231835 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c600103a0 con 0x7f6c700fbb30 2024-05-17T21:58:29.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.459+0000 7f6c77dfc700 1 -- 172.21.15.107:0/1294231835 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c70113870 con 0x7f6c700fbb30 2024-05-17T21:58:29.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.460+0000 7f6c77dfc700 1 -- 172.21.15.107:0/1294231835 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c7004c710 con 0x7f6c700fbb30 2024-05-17T21:58:29.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.461+0000 7f6c667fc700 1 -- 172.21.15.107:0/1294231835 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f6c60011400 con 0x7f6c700fbb30 2024-05-17T21:58:29.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.461+0000 7f6c667fc700 1 --2- 172.21.15.107:0/1294231835 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f6c5c078990 0x7f6c5c07ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:29.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.462+0000 7f6c667fc700 1 -- 172.21.15.107:0/1294231835 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f6c6009a520 con 0x7f6c700fbb30 2024-05-17T21:58:29.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.462+0000 7f6c75b98700 1 --2- 172.21.15.107:0/1294231835 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f6c5c078990 0x7f6c5c07ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:29.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.462+0000 7f6c75b98700 1 --2- 172.21.15.107:0/1294231835 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f6c5c078990 0x7f6c5c07ae40 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c00b970 tx=0x7f6c6c009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:29.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.466+0000 7f6c667fc700 1 -- 172.21.15.107:0/1294231835 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f6c60062210 con 0x7f6c700fbb30 2024-05-17T21:58:29.480 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:29 smithi138 ceph-mon[81376]: from='client.34344 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:58:29.480 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:29 smithi138 ceph-mon[81376]: from='client.? 172.21.15.107:0/3648703892' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:58:29.480 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 17 21:58:29 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-4-activate[95831]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-05-17T21:58:29.480 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 17 21:58:29 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-4-activate[95831]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-05-17T21:58:29.481 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 17 21:58:29 smithi138 bash[95816]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-05-17T21:58:29.481 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 17 21:58:29 smithi138 bash[95816]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-05-17T21:58:29.481 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 17 21:58:29 smithi138 bash[95816]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-05-17T21:58:29.481 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 17 21:58:29 smithi138 bash[95816]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-05-17T21:58:29.481 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 17 21:58:29 smithi138 bash[95816]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-05-17T21:58:29.481 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 17 21:58:29 smithi138 bash[95816]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-05-17T21:58:29.481 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 17 21:58:29 smithi138 bash[95816]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-4/block 2024-05-17T21:58:29.481 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 17 21:58:29 smithi138 bash[95816]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-05-17T21:58:29.481 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 17 21:58:29 smithi138 bash[95816]: --> ceph-volume raw activate successful for osd ID: 4 2024-05-17T21:58:29.481 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 17 21:58:29 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-4-activate[95831]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-05-17T21:58:29.481 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 17 21:58:29 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-4-activate[95831]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-05-17T21:58:29.481 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 17 21:58:29 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-4-activate[95831]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-05-17T21:58:29.481 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 17 21:58:29 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-4-activate[95831]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-05-17T21:58:29.481 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 17 21:58:29 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-4-activate[95831]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-4/block 2024-05-17T21:58:29.482 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 17 21:58:29 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-4-activate[95831]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-05-17T21:58:29.482 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 17 21:58:29 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-4-activate[95831]: --> ceph-volume raw activate successful for osd ID: 4 2024-05-17T21:58:29.482 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 17 21:58:29 smithi138 podman[95816]: 2024-05-17 21:58:29.200664245 +0000 UTC m=+0.993442046 container died 04f583860839aa780e5f39ad22ead6ab9faf22fb413d356331e4ec701d61999d (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-4-activate, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-17T21:58:29.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.692+0000 7f6c77dfc700 1 -- 172.21.15.107:0/1294231835 --> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6c70102630 con 0x7f6c5c078990 2024-05-17T21:58:29.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.694+0000 7f6c667fc700 1 -- 172.21.15.107:0/1294231835 <== mgr.34100 v2:172.21.15.107:6800/3205268226 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f6c70102630 con 0x7f6c5c078990 2024-05-17T21:58:29.684 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-05-17T21:58:29.685 INFO:teuthology.orchestra.run.smithi107.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b", 2024-05-17T21:58:29.685 INFO:teuthology.orchestra.run.smithi107.stdout: "in_progress": true, 2024-05-17T21:58:29.685 INFO:teuthology.orchestra.run.smithi107.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-17T21:58:29.685 INFO:teuthology.orchestra.run.smithi107.stdout: "services_complete": [ 2024-05-17T21:58:29.685 INFO:teuthology.orchestra.run.smithi107.stdout: "crash", 2024-05-17T21:58:29.685 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr", 2024-05-17T21:58:29.685 INFO:teuthology.orchestra.run.smithi107.stdout: "mon" 2024-05-17T21:58:29.685 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-05-17T21:58:29.685 INFO:teuthology.orchestra.run.smithi107.stdout: "progress": "10/21 daemons upgraded", 2024-05-17T21:58:29.685 INFO:teuthology.orchestra.run.smithi107.stdout: "message": "Currently upgrading osd daemons", 2024-05-17T21:58:29.685 INFO:teuthology.orchestra.run.smithi107.stdout: "is_paused": false 2024-05-17T21:58:29.685 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-05-17T21:58:29.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.696+0000 7f6c77dfc700 1 -- 172.21.15.107:0/1294231835 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f6c5c078990 msgr2=0x7f6c5c07ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:29.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.696+0000 7f6c77dfc700 1 --2- 172.21.15.107:0/1294231835 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f6c5c078990 0x7f6c5c07ae40 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c00b970 tx=0x7f6c6c009e30 comp rx=0 tx=0).stop 2024-05-17T21:58:29.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.696+0000 7f6c77dfc700 1 -- 172.21.15.107:0/1294231835 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c700fbb30 msgr2=0x7f6c7010e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:29.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.696+0000 7f6c77dfc700 1 --2- 172.21.15.107:0/1294231835 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c700fbb30 0x7f6c7010e070 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6c6000b790 tx=0x7f6c60009250 comp rx=0 tx=0).stop 2024-05-17T21:58:29.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.696+0000 7f6c77dfc700 1 -- 172.21.15.107:0/1294231835 shutdown_connections 2024-05-17T21:58:29.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.696+0000 7f6c77dfc700 1 --2- 172.21.15.107:0/1294231835 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f6c5c078990 0x7f6c5c07ae40 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:29.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.696+0000 7f6c77dfc700 1 --2- 172.21.15.107:0/1294231835 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c700fb1e0 0x7f6c7010db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:29.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.696+0000 7f6c77dfc700 1 --2- 172.21.15.107:0/1294231835 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c700fbb30 0x7f6c7010e070 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:29.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.696+0000 7f6c77dfc700 1 -- 172.21.15.107:0/1294231835 >> 172.21.15.107:0/1294231835 conn(0x7f6c700f6be0 msgr2=0x7f6c700f8240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:29.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.697+0000 7f6c77dfc700 1 -- 172.21.15.107:0/1294231835 shutdown_connections 2024-05-17T21:58:29.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.697+0000 7f6c77dfc700 1 -- 172.21.15.107:0/1294231835 wait complete. 2024-05-17T21:58:29.790 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.799+0000 7f76d2246700 1 -- 172.21.15.107:0/1581917805 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f76cc0fbe30 msgr2=0x7f76cc0fc240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:29.790 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.799+0000 7f76d2246700 1 --2- 172.21.15.107:0/1581917805 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f76cc0fbe30 0x7f76cc0fc240 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f76bc004940 tx=0x7f76bc009e30 comp rx=0 tx=0).stop 2024-05-17T21:58:29.790 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.799+0000 7f76d2246700 1 -- 172.21.15.107:0/1581917805 shutdown_connections 2024-05-17T21:58:29.790 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.799+0000 7f76d2246700 1 --2- 172.21.15.107:0/1581917805 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f76cc0fcfa0 0x7f76cc0fd410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:29.790 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.799+0000 7f76d2246700 1 --2- 172.21.15.107:0/1581917805 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f76cc0fbe30 0x7f76cc0fc240 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:29.790 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.799+0000 7f76d2246700 1 -- 172.21.15.107:0/1581917805 >> 172.21.15.107:0/1581917805 conn(0x7f76cc0f7440 msgr2=0x7f76cc0f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:29.790 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.799+0000 7f76d2246700 1 -- 172.21.15.107:0/1581917805 shutdown_connections 2024-05-17T21:58:29.790 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.799+0000 7f76d2246700 1 -- 172.21.15.107:0/1581917805 wait complete. 2024-05-17T21:58:29.790 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.799+0000 7f76d2246700 1 Processor -- start 2024-05-17T21:58:29.791 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.799+0000 7f76d2246700 1 -- start start 2024-05-17T21:58:29.792 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.800+0000 7f76d2246700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f76cc0fbe30 0x7f76cc110700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:29.792 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.800+0000 7f76d2246700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f76cc0fcfa0 0x7f76cc112c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:29.792 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.800+0000 7f76d2246700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76cc066c20 con 0x7f76cc0fcfa0 2024-05-17T21:58:29.792 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.800+0000 7f76d2246700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76cc066d60 con 0x7f76cc0fbe30 2024-05-17T21:58:29.792 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.800+0000 7f76caffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f76cc0fcfa0 0x7f76cc112c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:29.792 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.800+0000 7f76caffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f76cc0fcfa0 0x7f76cc112c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59856/0 (socket says 172.21.15.107:59856) 2024-05-17T21:58:29.792 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.800+0000 7f76caffd700 1 -- 172.21.15.107:0/3148788227 learned_addr learned my addr 172.21.15.107:0/3148788227 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:58:29.792 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.800+0000 7f76cb7fe700 1 --2- 172.21.15.107:0/3148788227 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f76cc0fbe30 0x7f76cc110700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:29.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.800+0000 7f76caffd700 1 -- 172.21.15.107:0/3148788227 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f76cc0fbe30 msgr2=0x7f76cc110700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:29.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.800+0000 7f76caffd700 1 --2- 172.21.15.107:0/3148788227 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f76cc0fbe30 0x7f76cc110700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:29.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.800+0000 7f76caffd700 1 -- 172.21.15.107:0/3148788227 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76bc005040 con 0x7f76cc0fcfa0 2024-05-17T21:58:29.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.800+0000 7f76caffd700 1 --2- 172.21.15.107:0/3148788227 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f76cc0fcfa0 0x7f76cc112c50 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f76c0002a50 tx=0x7f76c00046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:29.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.800+0000 7f76b3fff700 1 -- 172.21.15.107:0/3148788227 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f76c0007cc0 con 0x7f76cc0fcfa0 2024-05-17T21:58:29.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.801+0000 7f76b3fff700 1 -- 172.21.15.107:0/3148788227 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f76c000f040 con 0x7f76cc0fcfa0 2024-05-17T21:58:29.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.801+0000 7f76d2246700 1 -- 172.21.15.107:0/3148788227 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76cc113250 con 0x7f76cc0fcfa0 2024-05-17T21:58:29.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.801+0000 7f76b3fff700 1 -- 172.21.15.107:0/3148788227 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f76c000da30 con 0x7f76cc0fcfa0 2024-05-17T21:58:29.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.801+0000 7f76d2246700 1 -- 172.21.15.107:0/3148788227 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76cc113810 con 0x7f76cc0fcfa0 2024-05-17T21:58:29.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.802+0000 7f76d2246700 1 -- 172.21.15.107:0/3148788227 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76cc05f5c0 con 0x7f76cc0fcfa0 2024-05-17T21:58:29.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.803+0000 7f76b3fff700 1 -- 172.21.15.107:0/3148788227 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f76c000db90 con 0x7f76cc0fcfa0 2024-05-17T21:58:29.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.803+0000 7f76b3fff700 1 --2- 172.21.15.107:0/3148788227 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f76b4078990 0x7f76b407ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:29.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.803+0000 7f76b3fff700 1 -- 172.21.15.107:0/3148788227 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f76c009a4d0 con 0x7f76cc0fcfa0 2024-05-17T21:58:29.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.803+0000 7f76cb7fe700 1 --2- 172.21.15.107:0/3148788227 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f76b4078990 0x7f76b407ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:29.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.804+0000 7f76cb7fe700 1 --2- 172.21.15.107:0/3148788227 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f76b4078990 0x7f76b407ae40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f76bc004890 tx=0x7f76bc009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:29.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:29.808+0000 7f76b3fff700 1 -- 172.21.15.107:0/3148788227 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f76c0062140 con 0x7f76cc0fcfa0 2024-05-17T21:58:30.075 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 17 21:58:29 smithi138 podman[95961]: 2024-05-17 21:58:29.791095067 +0000 UTC m=+0.584590656 container remove 04f583860839aa780e5f39ad22ead6ab9faf22fb413d356331e4ec701d61999d (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-4-activate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-05-17T21:58:30.075 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 17 21:58:30 smithi138 podman[96021]: 2024-05-17 21:58:29.907586158 +0000 UTC m=+0.017412187 image pull 8880f7613d0283efa7002505fdccdbc3e37d3b5da31c57fa22d62262c5dab66c quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b 2024-05-17T21:58:30.076 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 17 21:58:30 smithi138 podman[96021]: 2024-05-17 21:58:30.008697188 +0000 UTC m=+0.118523198 container create e33f411da8e680c9efc6047708a801968d51f176a8ebed10ed5961165a3f046d (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-4, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-05-17T21:58:30.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:30.096+0000 7f76d2246700 1 -- 172.21.15.107:0/3148788227 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f76cc104890 con 0x7f76cc0fcfa0 2024-05-17T21:58:30.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:30.097+0000 7f76b3fff700 1 -- 172.21.15.107:0/3148788227 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+999 (secure 0 0 0) 0x7f76c0061ae0 con 0x7f76cc0fcfa0 2024-05-17T21:58:30.088 INFO:teuthology.orchestra.run.smithi107.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data; 1 osds down; Reduced data availability: 2 pgs peering; Degraded data redundancy: 22/267 objects degraded (8.240%), 5 pgs degraded 2024-05-17T21:58:30.088 INFO:teuthology.orchestra.run.smithi107.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-17T21:58:30.088 INFO:teuthology.orchestra.run.smithi107.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-17T21:58:30.088 INFO:teuthology.orchestra.run.smithi107.stdout:[WRN] OSD_DOWN: 1 osds down 2024-05-17T21:58:30.088 INFO:teuthology.orchestra.run.smithi107.stdout: osd.4 (root=default,host=smithi138) is down 2024-05-17T21:58:30.088 INFO:teuthology.orchestra.run.smithi107.stdout:[WRN] PG_AVAILABILITY: Reduced data availability: 2 pgs peering 2024-05-17T21:58:30.088 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.1c is stuck peering for 4m, current state peering, last acting [5,2] 2024-05-17T21:58:30.088 INFO:teuthology.orchestra.run.smithi107.stdout: pg 3.1a is stuck peering for 3m, current state peering, last acting [1,2] 2024-05-17T21:58:30.088 INFO:teuthology.orchestra.run.smithi107.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 22/267 objects degraded (8.240%), 5 pgs degraded 2024-05-17T21:58:30.088 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.5 is active+undersized+degraded, acting [3,0] 2024-05-17T21:58:30.088 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.7 is active+undersized+degraded, acting [3,2] 2024-05-17T21:58:30.089 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.b is active+undersized+degraded, acting [3,5] 2024-05-17T21:58:30.089 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.14 is active+undersized+degraded, acting [3,5] 2024-05-17T21:58:30.089 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.1a is active+undersized+degraded, acting [3,5] 2024-05-17T21:58:30.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:30.100+0000 7f76d2246700 1 -- 172.21.15.107:0/3148788227 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f76b4078990 msgr2=0x7f76b407ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:30.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:30.100+0000 7f76d2246700 1 --2- 172.21.15.107:0/3148788227 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f76b4078990 0x7f76b407ae40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f76bc004890 tx=0x7f76bc009e30 comp rx=0 tx=0).stop 2024-05-17T21:58:30.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:30.100+0000 7f76d2246700 1 -- 172.21.15.107:0/3148788227 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f76cc0fcfa0 msgr2=0x7f76cc112c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:30.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:30.100+0000 7f76d2246700 1 --2- 172.21.15.107:0/3148788227 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f76cc0fcfa0 0x7f76cc112c50 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f76c0002a50 tx=0x7f76c00046b0 comp rx=0 tx=0).stop 2024-05-17T21:58:30.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:30.100+0000 7f76d2246700 1 -- 172.21.15.107:0/3148788227 shutdown_connections 2024-05-17T21:58:30.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:30.100+0000 7f76d2246700 1 --2- 172.21.15.107:0/3148788227 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f76b4078990 0x7f76b407ae40 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:30.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:30.100+0000 7f76d2246700 1 --2- 172.21.15.107:0/3148788227 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f76cc0fbe30 0x7f76cc110700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:30.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:30.100+0000 7f76d2246700 1 --2- 172.21.15.107:0/3148788227 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f76cc0fcfa0 0x7f76cc112c50 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:30.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:30.100+0000 7f76d2246700 1 -- 172.21.15.107:0/3148788227 >> 172.21.15.107:0/3148788227 conn(0x7f76cc0f7440 msgr2=0x7f76cc0f8b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:30.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:30.100+0000 7f76d2246700 1 -- 172.21.15.107:0/3148788227 shutdown_connections 2024-05-17T21:58:30.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:58:30.100+0000 7f76d2246700 1 -- 172.21.15.107:0/3148788227 wait complete. 2024-05-17T21:58:30.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:30 smithi107 ceph-mon[93224]: from='client.34348 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:58:30.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:30 smithi107 ceph-mon[93224]: pgmap v142: 65 pgs: 5 active+undersized+degraded, 13 peering, 2 stale+active+clean, 8 active+undersized, 37 active+clean; 258 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 22/267 objects degraded (8.240%) 2024-05-17T21:58:30.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:30 smithi107 ceph-mon[93224]: from='client.34352 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:58:30.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:30 smithi107 ceph-mon[93224]: Health check failed: Degraded data redundancy: 22/267 objects degraded (8.240%), 5 pgs degraded (PG_DEGRADED) 2024-05-17T21:58:30.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:30 smithi107 ceph-mon[93224]: from='client.? 172.21.15.107:0/3494587088' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-17T21:58:30.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:30 smithi107 ceph-mon[93224]: from='client.34364 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:58:30.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:30 smithi107 ceph-mon[93224]: from='client.? 172.21.15.107:0/3148788227' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-17T21:58:30.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:30 smithi138 ceph-mon[81376]: from='client.34348 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:58:30.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:30 smithi138 ceph-mon[81376]: pgmap v142: 65 pgs: 5 active+undersized+degraded, 13 peering, 2 stale+active+clean, 8 active+undersized, 37 active+clean; 258 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 22/267 objects degraded (8.240%) 2024-05-17T21:58:30.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:30 smithi138 ceph-mon[81376]: from='client.34352 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:58:30.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:30 smithi138 ceph-mon[81376]: Health check failed: Degraded data redundancy: 22/267 objects degraded (8.240%), 5 pgs degraded (PG_DEGRADED) 2024-05-17T21:58:30.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:30 smithi138 ceph-mon[81376]: from='client.? 172.21.15.107:0/3494587088' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-17T21:58:30.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:30 smithi138 ceph-mon[81376]: from='client.34364 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:58:30.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:30 smithi138 ceph-mon[81376]: from='client.? 172.21.15.107:0/3148788227' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-17T21:58:30.576 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 17 21:58:30 smithi138 podman[96021]: 2024-05-17 21:58:30.236240942 +0000 UTC m=+0.346066971 container init e33f411da8e680c9efc6047708a801968d51f176a8ebed10ed5961165a3f046d (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-4, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-05-17T21:58:30.576 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 17 21:58:30 smithi138 podman[96021]: 2024-05-17 21:58:30.23957569 +0000 UTC m=+0.349401708 container start e33f411da8e680c9efc6047708a801968d51f176a8ebed10ed5961165a3f046d (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-4, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-17T21:58:30.576 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 17 21:58:30 smithi138 ceph-osd[96041]: -- 172.21.15.138:0/1028076831 <== mon.1 v2:172.21.15.138:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558b9f803680 con 0x558b9f8aad00 2024-05-17T21:58:30.576 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 17 21:58:30 smithi138 bash[96021]: e33f411da8e680c9efc6047708a801968d51f176a8ebed10ed5961165a3f046d 2024-05-17T21:58:30.576 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 17 21:58:30 smithi138 systemd[1]: Started Ceph osd.4 for 3722731a-1496-11ef-bc9a-c7b262605968. 2024-05-17T21:58:31.075 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 17 21:58:30 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-4[96037]: 2024-05-17T21:58:30.798+0000 7f4ff3b457c0 -1 Falling back to public interface 2024-05-17T21:58:31.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:31 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:58:31.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:31 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:58:31.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:31 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:58:31.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:31 smithi138 ceph-mon[81376]: pgmap v143: 65 pgs: 11 active+undersized+degraded, 8 peering, 15 active+undersized, 31 active+clean; 258 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 40/267 objects degraded (14.981%) 2024-05-17T21:58:31.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:31 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:58:31.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:31 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:58:31.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:31 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:58:31.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:31 smithi107 ceph-mon[93224]: pgmap v143: 65 pgs: 11 active+undersized+degraded, 8 peering, 15 active+undersized, 31 active+clean; 258 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 40/267 objects degraded (14.981%) 2024-05-17T21:58:32.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:32 smithi107 ceph-mon[93224]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-05-17T21:58:32.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:32 smithi138 ceph-mon[81376]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-05-17T21:58:33.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:33 smithi138 ceph-mon[81376]: pgmap v144: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 258 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 47/267 objects degraded (17.603%) 2024-05-17T21:58:33.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:33 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:58:33.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:33 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:58:33.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:33 smithi107 ceph-mon[93224]: pgmap v144: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 258 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 47/267 objects degraded (17.603%) 2024-05-17T21:58:33.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:33 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:58:33.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:33 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:58:35.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:35 smithi138 ceph-mon[81376]: pgmap v145: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 258 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 552 B/s rd, 1 op/s; 47/267 objects degraded (17.603%) 2024-05-17T21:58:35.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:35 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:58:35.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:35 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:58:35.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:35 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:58:35.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:35 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:58:35.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:35 smithi138 ceph-mon[81376]: Health check update: Degraded data redundancy: 47/267 objects degraded (17.603%), 15 pgs degraded (PG_DEGRADED) 2024-05-17T21:58:35.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:35 smithi107 ceph-mon[93224]: pgmap v145: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 258 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 552 B/s rd, 1 op/s; 47/267 objects degraded (17.603%) 2024-05-17T21:58:35.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:35 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:58:35.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:35 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:58:35.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:35 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:58:35.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:35 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:58:35.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:35 smithi107 ceph-mon[93224]: Health check update: Degraded data redundancy: 47/267 objects degraded (17.603%), 15 pgs degraded (PG_DEGRADED) 2024-05-17T21:58:36.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:36 smithi138 ceph-mon[81376]: from='osd.4 [v2:172.21.15.138:6808/705568274,v1:172.21.15.138:6809/705568274]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-17T21:58:36.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:36 smithi138 ceph-mon[81376]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-17T21:58:36.576 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 17 21:58:36 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-4[96037]: 2024-05-17T21:58:36.422+0000 7f4ff3b457c0 -1 osd.4 69 log_to_monitors true 2024-05-17T21:58:36.576 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 17 21:58:36 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-4[96037]: 2024-05-17T21:58:36.478+0000 7f4fe4f5e700 -1 osd.4 69 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-17T21:58:36.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:36 smithi107 ceph-mon[93224]: from='osd.4 [v2:172.21.15.138:6808/705568274,v1:172.21.15.138:6809/705568274]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-17T21:58:36.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:36 smithi107 ceph-mon[93224]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-17T21:58:37.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:37 smithi138 ceph-mon[81376]: pgmap v146: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 258 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 498 B/s rd, 0 op/s; 47/267 objects degraded (17.603%) 2024-05-17T21:58:37.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:37 smithi138 ceph-mon[81376]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-17T21:58:37.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:37 smithi138 ceph-mon[81376]: osdmap e72: 6 total, 5 up, 6 in 2024-05-17T21:58:37.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:37 smithi138 ceph-mon[81376]: from='osd.4 [v2:172.21.15.138:6808/705568274,v1:172.21.15.138:6809/705568274]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-05-17T21:58:37.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:37 smithi138 ceph-mon[81376]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-05-17T21:58:37.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:37 smithi107 ceph-mon[93224]: pgmap v146: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 258 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 498 B/s rd, 0 op/s; 47/267 objects degraded (17.603%) 2024-05-17T21:58:37.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:37 smithi107 ceph-mon[93224]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-17T21:58:37.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:37 smithi107 ceph-mon[93224]: osdmap e72: 6 total, 5 up, 6 in 2024-05-17T21:58:37.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:37 smithi107 ceph-mon[93224]: from='osd.4 [v2:172.21.15.138:6808/705568274,v1:172.21.15.138:6809/705568274]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-05-17T21:58:37.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:37 smithi107 ceph-mon[93224]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-05-17T21:58:38.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:38 smithi107 ceph-mon[93224]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-05-17T21:58:38.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:38 smithi107 ceph-mon[93224]: osd.4 [v2:172.21.15.138:6808/705568274,v1:172.21.15.138:6809/705568274] boot 2024-05-17T21:58:38.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:38 smithi107 ceph-mon[93224]: osdmap e73: 6 total, 6 up, 6 in 2024-05-17T21:58:38.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:38 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-17T21:58:38.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:38 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:58:38.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:38 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:58:38.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:38 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:58:38.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:38 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T21:58:38.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:38 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:58:38.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:38 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:58:38.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:38 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:58:38.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:38 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:58:38.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:38 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:58:38.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:38 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-05-17T21:58:38.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:38 smithi107 ceph-mon[93224]: osdmap e74: 6 total, 6 up, 6 in 2024-05-17T21:58:38.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:38 smithi138 ceph-mon[81376]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-05-17T21:58:38.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:38 smithi138 ceph-mon[81376]: osd.4 [v2:172.21.15.138:6808/705568274,v1:172.21.15.138:6809/705568274] boot 2024-05-17T21:58:38.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:38 smithi138 ceph-mon[81376]: osdmap e73: 6 total, 6 up, 6 in 2024-05-17T21:58:38.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:38 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-17T21:58:38.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:38 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:58:38.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:38 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:58:38.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:38 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:58:38.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:38 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T21:58:38.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:38 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:58:38.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:38 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:58:38.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:38 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:58:38.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:38 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:58:38.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:38 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:58:38.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:38 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-05-17T21:58:38.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:38 smithi138 ceph-mon[81376]: osdmap e74: 6 total, 6 up, 6 in 2024-05-17T21:58:40.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:40 smithi138 ceph-mon[81376]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-05-17T21:58:40.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:40 smithi138 ceph-mon[81376]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-05-17T21:58:40.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:40 smithi138 ceph-mon[81376]: pgmap v149: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s; 47/267 objects degraded (17.603%) 2024-05-17T21:58:40.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:40 smithi107 ceph-mon[93224]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-05-17T21:58:40.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:40 smithi107 ceph-mon[93224]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-05-17T21:58:40.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:40 smithi107 ceph-mon[93224]: pgmap v149: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s; 47/267 objects degraded (17.603%) 2024-05-17T21:58:41.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:41 smithi138 ceph-mon[81376]: pgmap v151: 65 pgs: 11 active+undersized+degraded, 13 active+undersized, 41 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 34/267 objects degraded (12.734%) 2024-05-17T21:58:41.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:41 smithi138 ceph-mon[81376]: Health check update: Degraded data redundancy: 34/267 objects degraded (12.734%), 11 pgs degraded (PG_DEGRADED) 2024-05-17T21:58:41.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:41 smithi107 ceph-mon[93224]: pgmap v151: 65 pgs: 11 active+undersized+degraded, 13 active+undersized, 41 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 34/267 objects degraded (12.734%) 2024-05-17T21:58:41.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:41 smithi107 ceph-mon[93224]: Health check update: Degraded data redundancy: 34/267 objects degraded (12.734%), 11 pgs degraded (PG_DEGRADED) 2024-05-17T21:58:42.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:42 smithi107 ceph-mon[93224]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 34/267 objects degraded (12.734%), 11 pgs degraded) 2024-05-17T21:58:42.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:42 smithi138 ceph-mon[81376]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 34/267 objects degraded (12.734%), 11 pgs degraded) 2024-05-17T21:58:43.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:43 smithi138 ceph-mon[81376]: pgmap v152: 65 pgs: 65 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 2 op/s 2024-05-17T21:58:43.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:43 smithi107 ceph-mon[93224]: pgmap v152: 65 pgs: 65 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 2 op/s 2024-05-17T21:58:45.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:45 smithi138 ceph-mon[81376]: pgmap v153: 65 pgs: 65 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.0 KiB/s rd, 2 op/s 2024-05-17T21:58:45.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:45 smithi107 ceph-mon[93224]: pgmap v153: 65 pgs: 65 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.0 KiB/s rd, 2 op/s 2024-05-17T21:58:47.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:47 smithi138 ceph-mon[81376]: pgmap v154: 65 pgs: 65 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 342 B/s rd, 0 op/s 2024-05-17T21:58:47.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:47 smithi107 ceph-mon[93224]: pgmap v154: 65 pgs: 65 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 342 B/s rd, 0 op/s 2024-05-17T21:58:49.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:49 smithi138 ceph-mon[81376]: pgmap v155: 65 pgs: 65 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 818 B/s rd, 1 op/s 2024-05-17T21:58:49.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:49 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:58:49.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:49 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:58:49.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:49 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:58:49.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:49 smithi107 ceph-mon[93224]: pgmap v155: 65 pgs: 65 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 818 B/s rd, 1 op/s 2024-05-17T21:58:49.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:49 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:58:49.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:49 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:58:49.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:49 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:58:51.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:51 smithi138 ceph-mon[81376]: pgmap v156: 65 pgs: 65 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 684 B/s rd, 1 op/s 2024-05-17T21:58:51.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:51 smithi107 ceph-mon[93224]: pgmap v156: 65 pgs: 65 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 684 B/s rd, 1 op/s 2024-05-17T21:58:53.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:53 smithi138 ceph-mon[81376]: pgmap v157: 65 pgs: 65 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 2 op/s 2024-05-17T21:58:53.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:53 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-05-17T21:58:53.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:53 smithi107 ceph-mon[93224]: pgmap v157: 65 pgs: 65 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 2 op/s 2024-05-17T21:58:53.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:53 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-05-17T21:58:54.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:54 smithi138 ceph-mon[81376]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-05-17T21:58:54.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:54 smithi138 ceph-mon[81376]: Upgrade: osd.5 is safe to restart 2024-05-17T21:58:54.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:54 smithi107 ceph-mon[93224]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-05-17T21:58:54.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:54 smithi107 ceph-mon[93224]: Upgrade: osd.5 is safe to restart 2024-05-17T21:58:56.269 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:56 smithi138 ceph-mon[81376]: pgmap v158: 65 pgs: 65 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T21:58:56.269 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:56 smithi138 ceph-mon[81376]: Upgrade: Updating osd.5 2024-05-17T21:58:56.269 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:56 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:58:56.269 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:56 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-17T21:58:56.269 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:56 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:58:56.269 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:56 smithi138 ceph-mon[81376]: Deploying daemon osd.5 on smithi138 2024-05-17T21:58:56.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:56 smithi107 ceph-mon[93224]: pgmap v158: 65 pgs: 65 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T21:58:56.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:56 smithi107 ceph-mon[93224]: Upgrade: Updating osd.5 2024-05-17T21:58:56.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:56 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:58:56.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:56 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-17T21:58:56.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:56 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:58:56.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:56 smithi107 ceph-mon[93224]: Deploying daemon osd.5 on smithi138 2024-05-17T21:58:58.312 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:58 smithi138 ceph-mon[81376]: pgmap v159: 65 pgs: 65 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T21:58:58.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:58 smithi107 ceph-mon[93224]: pgmap v159: 65 pgs: 65 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T21:58:58.575 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 17 21:58:58 smithi138 systemd[1]: Stopping Ceph osd.5 for 3722731a-1496-11ef-bc9a-c7b262605968... 2024-05-17T21:58:59.003 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 17 21:58:58 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-5[55025]: 2024-05-17T21:58:58.759+0000 7f044e270700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-17T21:58:59.003 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 17 21:58:58 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-5[55025]: 2024-05-17T21:58:58.759+0000 7f044e270700 -1 osd.5 74 *** Got signal Terminated *** 2024-05-17T21:58:59.003 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 17 21:58:58 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-5[55025]: 2024-05-17T21:58:58.759+0000 7f044e270700 -1 osd.5 74 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-17T21:58:59.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:58:59 smithi138 ceph-mon[81376]: osd.5 marked itself down and dead 2024-05-17T21:58:59.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:58:59 smithi107 ceph-mon[93224]: osd.5 marked itself down and dead 2024-05-17T21:58:59.777 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 17 21:58:59 smithi138 podman[100690]: 2024-05-17 21:58:59.487713696 +0000 UTC m=+1.004852730 container died 7c8d8ed749741a61a144539f566d13e8c3e3cd73929ee1c69b280b08fbdc2872 (image=quay.ceph.io/ceph-ci/ceph@sha256:c200327b094672e93680375e82d2de1a08f5881ef304978f791cce179f847d60, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, RELEASE=quincy-9903f77, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS) 2024-05-17T21:59:00.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:00 smithi138 ceph-mon[81376]: pgmap v160: 65 pgs: 65 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T21:59:00.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:00 smithi138 ceph-mon[81376]: Health check failed: 1 osds down (OSD_DOWN) 2024-05-17T21:59:00.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:00 smithi138 ceph-mon[81376]: Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-05-17T21:59:00.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:00 smithi138 ceph-mon[81376]: osdmap e75: 6 total, 5 up, 6 in 2024-05-17T21:59:00.076 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 17 21:58:59 smithi138 podman[100690]: 2024-05-17 21:58:59.951674856 +0000 UTC m=+1.468813912 container cleanup 7c8d8ed749741a61a144539f566d13e8c3e3cd73929ee1c69b280b08fbdc2872 (image=quay.ceph.io/ceph-ci/ceph@sha256:c200327b094672e93680375e82d2de1a08f5881ef304978f791cce179f847d60, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-5, RELEASE=quincy-9903f77, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-05-17T21:59:00.076 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 17 21:58:59 smithi138 bash[100690]: ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-5 2024-05-17T21:59:00.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.215+0000 7f2d3754d700 1 -- 172.21.15.107:0/4051682397 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d30066220 msgr2=0x7f2d300ffa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:00.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.215+0000 7f2d3754d700 1 --2- 172.21.15.107:0/4051682397 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d30066220 0x7f2d300ffa40 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2d28009ce0 tx=0x7f2d28004d10 comp rx=0 tx=0).stop 2024-05-17T21:59:00.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.215+0000 7f2d3754d700 1 -- 172.21.15.107:0/4051682397 shutdown_connections 2024-05-17T21:59:00.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.215+0000 7f2d3754d700 1 --2- 172.21.15.107:0/4051682397 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d300fff80 0x7f2d30102360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:00.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.215+0000 7f2d3754d700 1 --2- 172.21.15.107:0/4051682397 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d30066220 0x7f2d300ffa40 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:00.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.215+0000 7f2d3754d700 1 -- 172.21.15.107:0/4051682397 >> 172.21.15.107:0/4051682397 conn(0x7f2d300f7430 msgr2=0x7f2d300f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:00.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.215+0000 7f2d3754d700 1 -- 172.21.15.107:0/4051682397 shutdown_connections 2024-05-17T21:59:00.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.215+0000 7f2d3754d700 1 -- 172.21.15.107:0/4051682397 wait complete. 2024-05-17T21:59:00.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.216+0000 7f2d3754d700 1 Processor -- start 2024-05-17T21:59:00.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.216+0000 7f2d3754d700 1 -- start start 2024-05-17T21:59:00.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.216+0000 7f2d3754d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d30066220 0x7f2d3010da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:00.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.216+0000 7f2d3754d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d300fff80 0x7f2d3010df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:00.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.216+0000 7f2d3754d700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d3010e570 con 0x7f2d300fff80 2024-05-17T21:59:00.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.216+0000 7f2d3754d700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d3010edc0 con 0x7f2d30066220 2024-05-17T21:59:00.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.216+0000 7f2d34ae8700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d300fff80 0x7f2d3010df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:00.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.216+0000 7f2d34ae8700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d300fff80 0x7f2d3010df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59270/0 (socket says 172.21.15.107:59270) 2024-05-17T21:59:00.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.216+0000 7f2d34ae8700 1 -- 172.21.15.107:0/3662471053 learned_addr learned my addr 172.21.15.107:0/3662471053 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:59:00.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.216+0000 7f2d352e9700 1 --2- 172.21.15.107:0/3662471053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d30066220 0x7f2d3010da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:00.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.216+0000 7f2d34ae8700 1 -- 172.21.15.107:0/3662471053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d30066220 msgr2=0x7f2d3010da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:00.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.216+0000 7f2d34ae8700 1 --2- 172.21.15.107:0/3662471053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d30066220 0x7f2d3010da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:00.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.216+0000 7f2d34ae8700 1 -- 172.21.15.107:0/3662471053 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d28009940 con 0x7f2d300fff80 2024-05-17T21:59:00.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.216+0000 7f2d34ae8700 1 --2- 172.21.15.107:0/3662471053 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d300fff80 0x7f2d3010df70 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2d2400eb40 tx=0x7f2d2400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:00.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.217+0000 7f2d2dffb700 1 -- 172.21.15.107:0/3662471053 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d24007cc0 con 0x7f2d300fff80 2024-05-17T21:59:00.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.217+0000 7f2d3754d700 1 -- 172.21.15.107:0/3662471053 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d3010f040 con 0x7f2d300fff80 2024-05-17T21:59:00.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.217+0000 7f2d2dffb700 1 -- 172.21.15.107:0/3662471053 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2d2400f040 con 0x7f2d300fff80 2024-05-17T21:59:00.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.217+0000 7f2d2dffb700 1 -- 172.21.15.107:0/3662471053 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d2400ca10 con 0x7f2d300fff80 2024-05-17T21:59:00.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.217+0000 7f2d3754d700 1 -- 172.21.15.107:0/3662471053 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d3010f580 con 0x7f2d300fff80 2024-05-17T21:59:00.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.218+0000 7f2d3754d700 1 -- 172.21.15.107:0/3662471053 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d3005f5c0 con 0x7f2d300fff80 2024-05-17T21:59:00.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.218+0000 7f2d2dffb700 1 -- 172.21.15.107:0/3662471053 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f2d2400cc80 con 0x7f2d300fff80 2024-05-17T21:59:00.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.219+0000 7f2d2dffb700 1 --2- 172.21.15.107:0/3662471053 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f2d18078990 0x7f2d1807ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:00.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.219+0000 7f2d2dffb700 1 -- 172.21.15.107:0/3662471053 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f2d2409a2c0 con 0x7f2d300fff80 2024-05-17T21:59:00.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.219+0000 7f2d352e9700 1 --2- 172.21.15.107:0/3662471053 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f2d18078990 0x7f2d1807ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:00.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.220+0000 7f2d352e9700 1 --2- 172.21.15.107:0/3662471053 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f2d18078990 0x7f2d1807ae40 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2d28009430 tx=0x7f2d28004d10 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:00.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.224+0000 7f2d2dffb700 1 -- 172.21.15.107:0/3662471053 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f2d24061fe0 con 0x7f2d300fff80 2024-05-17T21:59:00.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:00 smithi107 ceph-mon[93224]: pgmap v160: 65 pgs: 65 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T21:59:00.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:00 smithi107 ceph-mon[93224]: Health check failed: 1 osds down (OSD_DOWN) 2024-05-17T21:59:00.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:00 smithi107 ceph-mon[93224]: Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-05-17T21:59:00.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:00 smithi107 ceph-mon[93224]: osdmap e75: 6 total, 5 up, 6 in 2024-05-17T21:59:00.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.451+0000 7f2d3754d700 1 -- 172.21.15.107:0/3662471053 --> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2d3010f8b0 con 0x7f2d18078990 2024-05-17T21:59:00.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.452+0000 7f2d2dffb700 1 -- 172.21.15.107:0/3662471053 <== mgr.34100 v2:172.21.15.107:6800/3205268226 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f2d3010f8b0 con 0x7f2d18078990 2024-05-17T21:59:00.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.455+0000 7f2d3754d700 1 -- 172.21.15.107:0/3662471053 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f2d18078990 msgr2=0x7f2d1807ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:00.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.455+0000 7f2d3754d700 1 --2- 172.21.15.107:0/3662471053 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f2d18078990 0x7f2d1807ae40 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2d28009430 tx=0x7f2d28004d10 comp rx=0 tx=0).stop 2024-05-17T21:59:00.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.455+0000 7f2d3754d700 1 -- 172.21.15.107:0/3662471053 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d300fff80 msgr2=0x7f2d3010df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:00.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.455+0000 7f2d3754d700 1 --2- 172.21.15.107:0/3662471053 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d300fff80 0x7f2d3010df70 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2d2400eb40 tx=0x7f2d2400b040 comp rx=0 tx=0).stop 2024-05-17T21:59:00.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.455+0000 7f2d3754d700 1 -- 172.21.15.107:0/3662471053 shutdown_connections 2024-05-17T21:59:00.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.455+0000 7f2d3754d700 1 --2- 172.21.15.107:0/3662471053 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f2d18078990 0x7f2d1807ae40 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:00.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.455+0000 7f2d3754d700 1 --2- 172.21.15.107:0/3662471053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d30066220 0x7f2d3010da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:00.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.455+0000 7f2d3754d700 1 --2- 172.21.15.107:0/3662471053 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d300fff80 0x7f2d3010df70 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:00.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.455+0000 7f2d3754d700 1 -- 172.21.15.107:0/3662471053 >> 172.21.15.107:0/3662471053 conn(0x7f2d300f7430 msgr2=0x7f2d300f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:00.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.455+0000 7f2d3754d700 1 -- 172.21.15.107:0/3662471053 shutdown_connections 2024-05-17T21:59:00.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.455+0000 7f2d3754d700 1 -- 172.21.15.107:0/3662471053 wait complete. 2024-05-17T21:59:00.457 INFO:teuthology.orchestra.run.smithi107.stdout:true 2024-05-17T21:59:00.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.559+0000 7f596d269700 1 -- 172.21.15.107:0/3434916450 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59680fcf10 msgr2=0x7f59680fd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:00.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.559+0000 7f596d269700 1 --2- 172.21.15.107:0/3434916450 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59680fcf10 0x7f59680fd320 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5958004940 tx=0x7f5958009e30 comp rx=0 tx=0).stop 2024-05-17T21:59:00.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.559+0000 7f596d269700 1 -- 172.21.15.107:0/3434916450 shutdown_connections 2024-05-17T21:59:00.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.559+0000 7f596d269700 1 --2- 172.21.15.107:0/3434916450 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59680fe080 0x7f59680fe4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:00.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.559+0000 7f596d269700 1 --2- 172.21.15.107:0/3434916450 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59680fcf10 0x7f59680fd320 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:00.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.559+0000 7f596d269700 1 -- 172.21.15.107:0/3434916450 >> 172.21.15.107:0/3434916450 conn(0x7f59680f8560 msgr2=0x7f59680fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:00.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.559+0000 7f596d269700 1 -- 172.21.15.107:0/3434916450 shutdown_connections 2024-05-17T21:59:00.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.559+0000 7f596d269700 1 -- 172.21.15.107:0/3434916450 wait complete. 2024-05-17T21:59:00.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.559+0000 7f596d269700 1 Processor -- start 2024-05-17T21:59:00.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.560+0000 7f596d269700 1 -- start start 2024-05-17T21:59:00.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.560+0000 7f596d269700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59680fcf10 0x7f596810d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:00.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.560+0000 7f596d269700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59680fe080 0x7f596810df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:00.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.560+0000 7f596d269700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f596810e520 con 0x7f59680fe080 2024-05-17T21:59:00.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.560+0000 7f596d269700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f596810ed70 con 0x7f59680fcf10 2024-05-17T21:59:00.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.560+0000 7f595e7fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59680fe080 0x7f596810df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:00.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.560+0000 7f595e7fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59680fe080 0x7f596810df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59276/0 (socket says 172.21.15.107:59276) 2024-05-17T21:59:00.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.560+0000 7f595e7fc700 1 -- 172.21.15.107:0/1696002723 learned_addr learned my addr 172.21.15.107:0/1696002723 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:59:00.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.560+0000 7f5966ffd700 1 --2- 172.21.15.107:0/1696002723 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59680fcf10 0x7f596810d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:00.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.560+0000 7f595e7fc700 1 -- 172.21.15.107:0/1696002723 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59680fcf10 msgr2=0x7f596810d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:00.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.560+0000 7f595e7fc700 1 --2- 172.21.15.107:0/1696002723 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59680fcf10 0x7f596810d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:00.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.560+0000 7f595e7fc700 1 -- 172.21.15.107:0/1696002723 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5958005040 con 0x7f59680fe080 2024-05-17T21:59:00.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.560+0000 7f595e7fc700 1 --2- 172.21.15.107:0/1696002723 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59680fe080 0x7f596810df20 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f595000c830 tx=0x7f5950004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:00.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.560+0000 7f595ffff700 1 -- 172.21.15.107:0/1696002723 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f59500106f0 con 0x7f59680fe080 2024-05-17T21:59:00.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.560+0000 7f595ffff700 1 -- 172.21.15.107:0/1696002723 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5950010d20 con 0x7f59680fe080 2024-05-17T21:59:00.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.560+0000 7f595ffff700 1 -- 172.21.15.107:0/1696002723 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f59500113a0 con 0x7f59680fe080 2024-05-17T21:59:00.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.561+0000 7f596d269700 1 -- 172.21.15.107:0/1696002723 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f596810f050 con 0x7f59680fe080 2024-05-17T21:59:00.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.561+0000 7f596d269700 1 -- 172.21.15.107:0/1696002723 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f596810f590 con 0x7f59680fe080 2024-05-17T21:59:00.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.561+0000 7f596d269700 1 -- 172.21.15.107:0/1696002723 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f596805f5c0 con 0x7f59680fe080 2024-05-17T21:59:00.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.562+0000 7f595ffff700 1 -- 172.21.15.107:0/1696002723 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f5950012400 con 0x7f59680fe080 2024-05-17T21:59:00.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.563+0000 7f595ffff700 1 --2- 172.21.15.107:0/1696002723 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f5954078990 0x7f595407ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:00.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.563+0000 7f595ffff700 1 -- 172.21.15.107:0/1696002723 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f595009b1a0 con 0x7f59680fe080 2024-05-17T21:59:00.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.563+0000 7f5966ffd700 1 --2- 172.21.15.107:0/1696002723 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f5954078990 0x7f595407ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:00.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.563+0000 7f5966ffd700 1 --2- 172.21.15.107:0/1696002723 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f5954078990 0x7f595407ae40 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f595800b3b0 tx=0x7f5958009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:00.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.567+0000 7f595ffff700 1 -- 172.21.15.107:0/1696002723 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f5950062f30 con 0x7f59680fe080 2024-05-17T21:59:00.575 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 17 21:59:00 smithi138 podman[100708]: 2024-05-17 21:59:00.312665582 +0000 UTC m=+0.817675402 container remove 7c8d8ed749741a61a144539f566d13e8c3e3cd73929ee1c69b280b08fbdc2872 (image=quay.ceph.io/ceph-ci/ceph@sha256:c200327b094672e93680375e82d2de1a08f5881ef304978f791cce179f847d60, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , RELEASE=quincy-9903f77, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-05-17T21:59:00.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.787+0000 7f596d269700 1 -- 172.21.15.107:0/1696002723 --> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f596810f9e0 con 0x7f5954078990 2024-05-17T21:59:00.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.788+0000 7f595ffff700 1 -- 172.21.15.107:0/1696002723 <== mgr.34100 v2:172.21.15.107:6800/3205268226 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f596810f9e0 con 0x7f5954078990 2024-05-17T21:59:00.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.791+0000 7f596d269700 1 -- 172.21.15.107:0/1696002723 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f5954078990 msgr2=0x7f595407ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:00.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.791+0000 7f596d269700 1 --2- 172.21.15.107:0/1696002723 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f5954078990 0x7f595407ae40 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f595800b3b0 tx=0x7f5958009e30 comp rx=0 tx=0).stop 2024-05-17T21:59:00.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.791+0000 7f596d269700 1 -- 172.21.15.107:0/1696002723 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59680fe080 msgr2=0x7f596810df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:00.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.791+0000 7f596d269700 1 --2- 172.21.15.107:0/1696002723 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59680fe080 0x7f596810df20 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f595000c830 tx=0x7f5950004470 comp rx=0 tx=0).stop 2024-05-17T21:59:00.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.791+0000 7f596d269700 1 -- 172.21.15.107:0/1696002723 shutdown_connections 2024-05-17T21:59:00.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.791+0000 7f596d269700 1 --2- 172.21.15.107:0/1696002723 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f5954078990 0x7f595407ae40 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:00.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.791+0000 7f596d269700 1 --2- 172.21.15.107:0/1696002723 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59680fcf10 0x7f596810d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:00.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.791+0000 7f596d269700 1 --2- 172.21.15.107:0/1696002723 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59680fe080 0x7f596810df20 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:00.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.791+0000 7f596d269700 1 -- 172.21.15.107:0/1696002723 >> 172.21.15.107:0/1696002723 conn(0x7f59680f8560 msgr2=0x7f59680f9c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:00.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.791+0000 7f596d269700 1 -- 172.21.15.107:0/1696002723 shutdown_connections 2024-05-17T21:59:00.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.791+0000 7f596d269700 1 -- 172.21.15.107:0/1696002723 wait complete. 2024-05-17T21:59:00.873 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 17 21:59:00 smithi138 podman[100796]: 2024-05-17 21:59:00.532367432 +0000 UTC m=+0.017269415 image pull 8880f7613d0283efa7002505fdccdbc3e37d3b5da31c57fa22d62262c5dab66c quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b 2024-05-17T21:59:00.873 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 17 21:59:00 smithi138 podman[100796]: 2024-05-17 21:59:00.66397266 +0000 UTC m=+0.148874630 container create 3f539b9ee7c5b95bb3401a6ccab117651dd2c74693919260fa8564cb3b4609ca (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-5-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, ceph=True) 2024-05-17T21:59:00.873 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 17 21:59:00 smithi138 podman[100796]: 2024-05-17 21:59:00.882979571 +0000 UTC m=+0.367881542 container init 3f539b9ee7c5b95bb3401a6ccab117651dd2c74693919260fa8564cb3b4609ca (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-5-deactivate, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-17T21:59:00.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.892+0000 7f4f8113b700 1 -- 172.21.15.107:0/376710477 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f7c0fce60 msgr2=0x7f4f7c0fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:00.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.892+0000 7f4f8113b700 1 --2- 172.21.15.107:0/376710477 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f7c0fce60 0x7f4f7c0fd270 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4f6c004940 tx=0x7f4f6c009e30 comp rx=0 tx=0).stop 2024-05-17T21:59:00.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.892+0000 7f4f8113b700 1 -- 172.21.15.107:0/376710477 shutdown_connections 2024-05-17T21:59:00.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.892+0000 7f4f8113b700 1 --2- 172.21.15.107:0/376710477 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4f7c0fdfd0 0x7f4f7c0fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:00.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.892+0000 7f4f8113b700 1 --2- 172.21.15.107:0/376710477 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f7c0fce60 0x7f4f7c0fd270 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:00.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.892+0000 7f4f8113b700 1 -- 172.21.15.107:0/376710477 >> 172.21.15.107:0/376710477 conn(0x7f4f7c0f8540 msgr2=0x7f4f7c0fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:00.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.892+0000 7f4f8113b700 1 -- 172.21.15.107:0/376710477 shutdown_connections 2024-05-17T21:59:00.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.892+0000 7f4f8113b700 1 -- 172.21.15.107:0/376710477 wait complete. 2024-05-17T21:59:00.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.893+0000 7f4f8113b700 1 Processor -- start 2024-05-17T21:59:00.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.893+0000 7f4f8113b700 1 -- start start 2024-05-17T21:59:00.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.893+0000 7f4f8113b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4f7c0fce60 0x7f4f7c10d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:00.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.893+0000 7f4f8113b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f7c0fdfd0 0x7f4f7c10de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:00.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.893+0000 7f4f8113b700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f7c10e450 con 0x7f4f7c0fdfd0 2024-05-17T21:59:00.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.893+0000 7f4f8113b700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f7c10e5c0 con 0x7f4f7c0fce60 2024-05-17T21:59:00.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.893+0000 7f4f7a59c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f7c0fdfd0 0x7f4f7c10de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:00.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.893+0000 7f4f7a59c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f7c0fdfd0 0x7f4f7c10de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59286/0 (socket says 172.21.15.107:59286) 2024-05-17T21:59:00.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.893+0000 7f4f7a59c700 1 -- 172.21.15.107:0/3419275729 learned_addr learned my addr 172.21.15.107:0/3419275729 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:59:00.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.893+0000 7f4f7ad9d700 1 --2- 172.21.15.107:0/3419275729 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4f7c0fce60 0x7f4f7c10d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:00.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.894+0000 7f4f7a59c700 1 -- 172.21.15.107:0/3419275729 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4f7c0fce60 msgr2=0x7f4f7c10d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:00.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.894+0000 7f4f7a59c700 1 --2- 172.21.15.107:0/3419275729 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4f7c0fce60 0x7f4f7c10d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:00.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.894+0000 7f4f7a59c700 1 -- 172.21.15.107:0/3419275729 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f6c005040 con 0x7f4f7c0fdfd0 2024-05-17T21:59:00.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.894+0000 7f4f7a59c700 1 --2- 172.21.15.107:0/3419275729 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f7c0fdfd0 0x7f4f7c10de50 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4f70002a50 tx=0x7f4f700046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:00.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.894+0000 7f4f637fe700 1 -- 172.21.15.107:0/3419275729 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4f70007cc0 con 0x7f4f7c0fdfd0 2024-05-17T21:59:00.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.894+0000 7f4f637fe700 1 -- 172.21.15.107:0/3419275729 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4f7000f040 con 0x7f4f7c0fdfd0 2024-05-17T21:59:00.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.894+0000 7f4f8113b700 1 -- 172.21.15.107:0/3419275729 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f7c113020 con 0x7f4f7c0fdfd0 2024-05-17T21:59:00.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.894+0000 7f4f637fe700 1 -- 172.21.15.107:0/3419275729 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4f7000da30 con 0x7f4f7c0fdfd0 2024-05-17T21:59:00.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.894+0000 7f4f8113b700 1 -- 172.21.15.107:0/3419275729 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f7c1135b0 con 0x7f4f7c0fdfd0 2024-05-17T21:59:00.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.895+0000 7f4f8113b700 1 -- 172.21.15.107:0/3419275729 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f7c04c710 con 0x7f4f7c0fdfd0 2024-05-17T21:59:00.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.896+0000 7f4f637fe700 1 -- 172.21.15.107:0/3419275729 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f4f7000dbb0 con 0x7f4f7c0fdfd0 2024-05-17T21:59:00.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.896+0000 7f4f637fe700 1 --2- 172.21.15.107:0/3419275729 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f4f64078990 0x7f4f6407ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:00.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.896+0000 7f4f637fe700 1 -- 172.21.15.107:0/3419275729 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f4f7009a390 con 0x7f4f7c0fdfd0 2024-05-17T21:59:00.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.896+0000 7f4f7ad9d700 1 --2- 172.21.15.107:0/3419275729 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f4f64078990 0x7f4f6407ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:00.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.897+0000 7f4f7ad9d700 1 --2- 172.21.15.107:0/3419275729 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f4f64078990 0x7f4f6407ae40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4f6c00b970 tx=0x7f4f6c009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:00.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:00.901+0000 7f4f637fe700 1 -- 172.21.15.107:0/3419275729 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f4f70062130 con 0x7f4f7c0fdfd0 2024-05-17T21:59:01.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:01 smithi107 ceph-mon[93224]: osdmap e76: 6 total, 5 up, 6 in 2024-05-17T21:59:01.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:01.116+0000 7f4f8113b700 1 -- 172.21.15.107:0/3419275729 --> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f4f7c102360 con 0x7f4f64078990 2024-05-17T21:59:01.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:01.126+0000 7f4f637fe700 1 -- 172.21.15.107:0/3419275729 <== mgr.34100 v2:172.21.15.107:6800/3205268226 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f4f7c102360 con 0x7f4f64078990 2024-05-17T21:59:01.118 INFO:teuthology.orchestra.run.smithi107.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-17T21:59:01.118 INFO:teuthology.orchestra.run.smithi107.stdout:alertmanager.smithi107 smithi107 *:9093,9094 running (6m) 95s ago 15m 24.3M - 0.25.0 c8568f914cd2 bac2a944ae42 2024-05-17T21:59:01.118 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi107 smithi107 running (3m) 95s ago 15m 6669k - 19.0.0-3794-g931809a9 8880f7613d02 face35de205b 2024-05-17T21:59:01.118 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi138 smithi138 running (3m) 27s ago 14m 6665k - 19.0.0-3794-g931809a9 8880f7613d02 ad59e33e6d43 2024-05-17T21:59:01.118 INFO:teuthology.orchestra.run.smithi107.stdout:grafana.smithi107 smithi107 *:3000 running (6m) 95s ago 14m 84.2M - 9.4.7 954c08fa6188 8c747e777bc5 2024-05-17T21:59:01.118 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.drqody smithi107 running (9m) 95s ago 9m 103M - 17.2.7-952-g9903f776 d15a36853796 0d851bc912f0 2024-05-17T21:59:01.118 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.idmbgj smithi107 running (9m) 95s ago 9m 13.8M - 17.2.7-952-g9903f776 d15a36853796 1348b09ab29e 2024-05-17T21:59:01.118 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi138.gwbtyd smithi138 running (9m) 27s ago 9m 152M - 17.2.7-952-g9903f776 d15a36853796 fd2ba75354c0 2024-05-17T21:59:01.118 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi138.zckrvs smithi138 running (9m) 27s ago 9m 14.7M - 17.2.7-952-g9903f776 d15a36853796 a1cfd2fe4cda 2024-05-17T21:59:01.119 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi107.xhhyiw smithi107 *:8443,9283,8765 running (5m) 95s ago 16m 587M - 19.0.0-3794-g931809a9 8880f7613d02 5acf3e511342 2024-05-17T21:59:01.119 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi138.napnll smithi138 *:8443,9283,8765 running (4m) 27s ago 14m 533M - 19.0.0-3794-g931809a9 8880f7613d02 d627761dbd0d 2024-05-17T21:59:01.119 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi107 smithi107 running (4m) 95s ago 16m 115M 2048M 19.0.0-3794-g931809a9 8880f7613d02 9717f3169f2f 2024-05-17T21:59:01.119 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi138 smithi138 running (3m) 27s ago 14m 106M 2048M 19.0.0-3794-g931809a9 8880f7613d02 55aeed62bf4d 2024-05-17T21:59:01.119 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi107 smithi107 *:9100 running (6m) 95s ago 15m 17.4M - 1.5.0 0da6a335fe13 49bec71f5487 2024-05-17T21:59:01.119 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi138 smithi138 *:9100 running (6m) 27s ago 14m 15.9M - 1.5.0 0da6a335fe13 e52824376658 2024-05-17T21:59:01.119 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0 smithi107 running (2m) 95s ago 13m 156M 1557M 19.0.0-3794-g931809a9 8880f7613d02 b319bc93500d 2024-05-17T21:59:01.119 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1 smithi107 running (2m) 95s ago 12m 151M 1557M 19.0.0-3794-g931809a9 8880f7613d02 f94b206be6eb 2024-05-17T21:59:01.119 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2 smithi107 running (101s) 95s ago 12m 79.3M 1557M 19.0.0-3794-g931809a9 8880f7613d02 add7015647af 2024-05-17T21:59:01.119 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3 smithi138 running (65s) 27s ago 12m 160M 2581M 19.0.0-3794-g931809a9 8880f7613d02 416273d327c7 2024-05-17T21:59:01.119 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4 smithi138 running (31s) 27s ago 11m 82.4M 2581M 19.0.0-3794-g931809a9 8880f7613d02 e33f411da8e6 2024-05-17T21:59:01.119 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5 smithi138 running (11m) 27s ago 11m 192M 2581M 17.2.7-952-g9903f776 d15a36853796 7c8d8ed74974 2024-05-17T21:59:01.119 INFO:teuthology.orchestra.run.smithi107.stdout:prometheus.smithi107 smithi107 *:9095 running (5m) 95s ago 14m 59.1M - 2.43.0 a07b618ecd1d bf5f609bbd6e 2024-05-17T21:59:01.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:01.129+0000 7f4f8113b700 1 -- 172.21.15.107:0/3419275729 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f4f64078990 msgr2=0x7f4f6407ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:01.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:01.129+0000 7f4f8113b700 1 --2- 172.21.15.107:0/3419275729 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f4f64078990 0x7f4f6407ae40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4f6c00b970 tx=0x7f4f6c009e30 comp rx=0 tx=0).stop 2024-05-17T21:59:01.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:01.129+0000 7f4f8113b700 1 -- 172.21.15.107:0/3419275729 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f7c0fdfd0 msgr2=0x7f4f7c10de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:01.121 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:01.129+0000 7f4f8113b700 1 --2- 172.21.15.107:0/3419275729 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f7c0fdfd0 0x7f4f7c10de50 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4f70002a50 tx=0x7f4f700046b0 comp rx=0 tx=0).stop 2024-05-17T21:59:01.121 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:01.129+0000 7f4f8113b700 1 -- 172.21.15.107:0/3419275729 shutdown_connections 2024-05-17T21:59:01.121 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:01.129+0000 7f4f8113b700 1 --2- 172.21.15.107:0/3419275729 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f4f64078990 0x7f4f6407ae40 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:01.121 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:01.129+0000 7f4f8113b700 1 --2- 172.21.15.107:0/3419275729 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4f7c0fce60 0x7f4f7c10d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:01.121 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:01.129+0000 7f4f8113b700 1 --2- 172.21.15.107:0/3419275729 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f7c0fdfd0 0x7f4f7c10de50 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:01.121 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:01.129+0000 7f4f8113b700 1 -- 172.21.15.107:0/3419275729 >> 172.21.15.107:0/3419275729 conn(0x7f4f7c0f8540 msgr2=0x7f4f7c0f9c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:01.121 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:01.129+0000 7f4f8113b700 1 -- 172.21.15.107:0/3419275729 shutdown_connections 2024-05-17T21:59:01.121 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:01.129+0000 7f4f8113b700 1 -- 172.21.15.107:0/3419275729 wait complete. 2024-05-17T21:59:01.178 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:01 smithi138 ceph-mon[81376]: osdmap e76: 6 total, 5 up, 6 in 2024-05-17T21:59:01.179 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 17 21:59:00 smithi138 podman[100796]: 2024-05-17 21:59:00.886415546 +0000 UTC m=+0.371317519 container start 3f539b9ee7c5b95bb3401a6ccab117651dd2c74693919260fa8564cb3b4609ca (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-5-deactivate, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a) 2024-05-17T21:59:01.179 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 17 21:59:00 smithi138 podman[100796]: 2024-05-17 21:59:00.991464078 +0000 UTC m=+0.476366078 container attach 3f539b9ee7c5b95bb3401a6ccab117651dd2c74693919260fa8564cb3b4609ca (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-5-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-05-17T21:59:01.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:01.231+0000 7fee03665700 1 -- 172.21.15.107:0/3923277915 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fedfc0fdfd0 msgr2=0x7fedfc0fe440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:01.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:01.231+0000 7fee03665700 1 --2- 172.21.15.107:0/3923277915 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fedfc0fdfd0 0x7fedfc0fe440 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fedf0009ce0 tx=0x7fedf0004d10 comp rx=0 tx=0).stop 2024-05-17T21:59:01.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:01.231+0000 7fee03665700 1 -- 172.21.15.107:0/3923277915 shutdown_connections 2024-05-17T21:59:01.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:01.231+0000 7fee03665700 1 --2- 172.21.15.107:0/3923277915 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fedfc0fdfd0 0x7fedfc0fe440 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:01.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:01.231+0000 7fee03665700 1 --2- 172.21.15.107:0/3923277915 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fedfc0fce60 0x7fedfc0fd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:01.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:01.231+0000 7fee03665700 1 -- 172.21.15.107:0/3923277915 >> 172.21.15.107:0/3923277915 conn(0x7fedfc0f8540 msgr2=0x7fedfc0fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:01.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:01.231+0000 7fee03665700 1 -- 172.21.15.107:0/3923277915 shutdown_connections 2024-05-17T21:59:01.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:01.231+0000 7fee03665700 1 -- 172.21.15.107:0/3923277915 wait complete. 2024-05-17T21:59:01.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:01.231+0000 7fee03665700 1 Processor -- start 2024-05-17T21:59:01.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:01.231+0000 7fee03665700 1 -- start start 2024-05-17T21:59:01.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:01.231+0000 7fee03665700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fedfc0fce60 0x7fedfc071c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:01.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:01.231+0000 7fee03665700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fedfc0fdfd0 0x7fedfc0702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:01.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:01.231+0000 7fee03665700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedfc072200 con 0x7fedfc0fce60 2024-05-17T21:59:01.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:01.231+0000 7fee03665700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedfc0707f0 con 0x7fedfc0fdfd0 2024-05-17T21:59:01.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:01.231+0000 7fee01401700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fedfc0fce60 0x7fedfc071c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:01.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:01.232+0000 7fee01401700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fedfc0fce60 0x7fedfc071c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59296/0 (socket says 172.21.15.107:59296) 2024-05-17T21:59:01.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:01.232+0000 7fee01401700 1 -- 172.21.15.107:0/1482093043 learned_addr learned my addr 172.21.15.107:0/1482093043 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:59:01.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:01.232+0000 7fee00c00700 1 --2- 172.21.15.107:0/1482093043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fedfc0fdfd0 0x7fedfc0702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:01.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:01.232+0000 7fee01401700 1 -- 172.21.15.107:0/1482093043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fedfc0fdfd0 msgr2=0x7fedfc0702b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:01.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:01.232+0000 7fee01401700 1 --2- 172.21.15.107:0/1482093043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fedfc0fdfd0 0x7fedfc0702b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:01.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:01.232+0000 7fee01401700 1 -- 172.21.15.107:0/1482093043 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedf0009940 con 0x7fedfc0fce60 2024-05-17T21:59:01.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:01.232+0000 7fee01401700 1 --2- 172.21.15.107:0/1482093043 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fedfc0fce60 0x7fedfc071c00 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fedec002a80 tx=0x7fedec0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:01.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:01.232+0000 7fedf9ffb700 1 -- 172.21.15.107:0/1482093043 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fedec007cc0 con 0x7fedfc0fce60 2024-05-17T21:59:01.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:01.232+0000 7fee03665700 1 -- 172.21.15.107:0/1482093043 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedfc070aa0 con 0x7fedfc0fce60 2024-05-17T21:59:01.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:01.232+0000 7fedf9ffb700 1 -- 172.21.15.107:0/1482093043 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fedec00f040 con 0x7fedfc0fce60 2024-05-17T21:59:01.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:01.232+0000 7fedf9ffb700 1 -- 172.21.15.107:0/1482093043 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fedec00dac0 con 0x7fedfc0fce60 2024-05-17T21:59:01.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:01.232+0000 7fee03665700 1 -- 172.21.15.107:0/1482093043 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedfc071060 con 0x7fedfc0fce60 2024-05-17T21:59:01.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:01.233+0000 7fee03665700 1 -- 172.21.15.107:0/1482093043 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedfc04c710 con 0x7fedfc0fce60 2024-05-17T21:59:01.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:01.234+0000 7fedf9ffb700 1 -- 172.21.15.107:0/1482093043 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7fedec00dc20 con 0x7fedfc0fce60 2024-05-17T21:59:01.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:01.234+0000 7fedf9ffb700 1 --2- 172.21.15.107:0/1482093043 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fede4078990 0x7fede407ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:01.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:01.235+0000 7fedf9ffb700 1 -- 172.21.15.107:0/1482093043 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fedec09a360 con 0x7fedfc0fce60 2024-05-17T21:59:01.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:01.235+0000 7fee00c00700 1 --2- 172.21.15.107:0/1482093043 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fede4078990 0x7fede407ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:01.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:01.235+0000 7fee00c00700 1 --2- 172.21.15.107:0/1482093043 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fede4078990 0x7fede407ae40 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fedf0015040 tx=0x7fedf0004d10 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:01.230 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:01.239+0000 7fedf9ffb700 1 -- 172.21.15.107:0/1482093043 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7fedec061fd0 con 0x7fedfc0fce60 2024-05-17T21:59:01.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:01.535+0000 7fee03665700 1 -- 172.21.15.107:0/1482093043 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fedfc115d20 con 0x7fedfc0fce60 2024-05-17T21:59:01.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:01.535+0000 7fedf9ffb700 1 -- 172.21.15.107:0/1482093043 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+732 (secure 0 0 0) 0x7fedec061970 con 0x7fedfc0fce60 2024-05-17T21:59:01.527 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-05-17T21:59:01.527 INFO:teuthology.orchestra.run.smithi107.stdout: "mon": { 2024-05-17T21:59:01.527 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev)": 2 2024-05-17T21:59:01.527 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-05-17T21:59:01.527 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr": { 2024-05-17T21:59:01.527 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev)": 2 2024-05-17T21:59:01.528 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-05-17T21:59:01.528 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": { 2024-05-17T21:59:01.528 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev)": 5 2024-05-17T21:59:01.528 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-05-17T21:59:01.528 INFO:teuthology.orchestra.run.smithi107.stdout: "mds": { 2024-05-17T21:59:01.528 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-952-g9903f776 (9903f7767386a9d9efadb7c1effdcd3064d71804) quincy (stable)": 4 2024-05-17T21:59:01.528 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-05-17T21:59:01.528 INFO:teuthology.orchestra.run.smithi107.stdout: "overall": { 2024-05-17T21:59:01.528 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-952-g9903f776 (9903f7767386a9d9efadb7c1effdcd3064d71804) quincy (stable)": 4, 2024-05-17T21:59:01.528 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev)": 9 2024-05-17T21:59:01.528 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-05-17T21:59:01.528 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-05-17T21:59:01.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:01.538+0000 7fee03665700 1 -- 172.21.15.107:0/1482093043 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fede4078990 msgr2=0x7fede407ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:01.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:01.538+0000 7fee03665700 1 --2- 172.21.15.107:0/1482093043 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fede4078990 0x7fede407ae40 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fedf0015040 tx=0x7fedf0004d10 comp rx=0 tx=0).stop 2024-05-17T21:59:01.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:01.538+0000 7fee03665700 1 -- 172.21.15.107:0/1482093043 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fedfc0fce60 msgr2=0x7fedfc071c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:01.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:01.538+0000 7fee03665700 1 --2- 172.21.15.107:0/1482093043 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fedfc0fce60 0x7fedfc071c00 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fedec002a80 tx=0x7fedec0046b0 comp rx=0 tx=0).stop 2024-05-17T21:59:01.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:01.538+0000 7fee03665700 1 -- 172.21.15.107:0/1482093043 shutdown_connections 2024-05-17T21:59:01.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:01.538+0000 7fee03665700 1 --2- 172.21.15.107:0/1482093043 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fede4078990 0x7fede407ae40 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:01.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:01.538+0000 7fee03665700 1 --2- 172.21.15.107:0/1482093043 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fedfc0fce60 0x7fedfc071c00 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:01.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:01.538+0000 7fee03665700 1 --2- 172.21.15.107:0/1482093043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fedfc0fdfd0 0x7fedfc0702b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:01.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:01.538+0000 7fee03665700 1 -- 172.21.15.107:0/1482093043 >> 172.21.15.107:0/1482093043 conn(0x7fedfc0f8540 msgr2=0x7fedfc0f9b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:01.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:01.538+0000 7fee03665700 1 -- 172.21.15.107:0/1482093043 shutdown_connections 2024-05-17T21:59:01.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:01.538+0000 7fee03665700 1 -- 172.21.15.107:0/1482093043 wait complete. 2024-05-17T21:59:01.544 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 17 21:59:01 smithi138 podman[100796]: 2024-05-17 21:59:01.188941377 +0000 UTC m=+0.673843351 container died 3f539b9ee7c5b95bb3401a6ccab117651dd2c74693919260fa8564cb3b4609ca (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-05-17T21:59:01.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:01.640+0000 7fa4c19d5700 1 -- 172.21.15.107:0/3675380498 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4bc0fcf60 msgr2=0x7fa4bc0fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:01.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:01.640+0000 7fa4c19d5700 1 --2- 172.21.15.107:0/3675380498 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4bc0fcf60 0x7fa4bc0fd3d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa4b0004940 tx=0x7fa4b0009e30 comp rx=0 tx=0).stop 2024-05-17T21:59:01.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:01.640+0000 7fa4c19d5700 1 -- 172.21.15.107:0/3675380498 shutdown_connections 2024-05-17T21:59:01.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:01.640+0000 7fa4c19d5700 1 --2- 172.21.15.107:0/3675380498 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4bc0fcf60 0x7fa4bc0fd3d0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:01.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:01.640+0000 7fa4c19d5700 1 --2- 172.21.15.107:0/3675380498 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4bc0fbcb0 0x7fa4bc0fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:01.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:01.640+0000 7fa4c19d5700 1 -- 172.21.15.107:0/3675380498 >> 172.21.15.107:0/3675380498 conn(0x7fa4bc0f7430 msgr2=0x7fa4bc0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:01.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:01.640+0000 7fa4c19d5700 1 -- 172.21.15.107:0/3675380498 shutdown_connections 2024-05-17T21:59:01.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:01.640+0000 7fa4c19d5700 1 -- 172.21.15.107:0/3675380498 wait complete. 2024-05-17T21:59:01.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:01.641+0000 7fa4c19d5700 1 Processor -- start 2024-05-17T21:59:01.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:01.641+0000 7fa4c19d5700 1 -- start start 2024-05-17T21:59:01.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:01.641+0000 7fa4c19d5700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4bc0fbcb0 0x7fa4bc10d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:01.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:01.641+0000 7fa4c19d5700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4bc0fcf60 0x7fa4bc10de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:01.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:01.641+0000 7fa4c19d5700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4bc10e430 con 0x7fa4bc0fbcb0 2024-05-17T21:59:01.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:01.641+0000 7fa4c19d5700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4bc10e5a0 con 0x7fa4bc0fcf60 2024-05-17T21:59:01.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:01.641+0000 7fa4baffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4bc0fbcb0 0x7fa4bc10d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:01.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:01.641+0000 7fa4baffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4bc0fbcb0 0x7fa4bc10d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59318/0 (socket says 172.21.15.107:59318) 2024-05-17T21:59:01.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:01.641+0000 7fa4baffd700 1 -- 172.21.15.107:0/3497503710 learned_addr learned my addr 172.21.15.107:0/3497503710 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:59:01.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:01.641+0000 7fa4ba7fc700 1 --2- 172.21.15.107:0/3497503710 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4bc0fcf60 0x7fa4bc10de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:01.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:01.642+0000 7fa4baffd700 1 -- 172.21.15.107:0/3497503710 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4bc0fcf60 msgr2=0x7fa4bc10de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:01.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:01.642+0000 7fa4baffd700 1 --2- 172.21.15.107:0/3497503710 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4bc0fcf60 0x7fa4bc10de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:01.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:01.642+0000 7fa4baffd700 1 -- 172.21.15.107:0/3497503710 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4b0005040 con 0x7fa4bc0fbcb0 2024-05-17T21:59:01.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:01.642+0000 7fa4baffd700 1 --2- 172.21.15.107:0/3497503710 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4bc0fbcb0 0x7fa4bc10d8f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac002a50 tx=0x7fa4ac0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:01.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:01.642+0000 7fa4a37fe700 1 -- 172.21.15.107:0/3497503710 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa4ac007cc0 con 0x7fa4bc0fbcb0 2024-05-17T21:59:01.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:01.642+0000 7fa4a37fe700 1 -- 172.21.15.107:0/3497503710 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa4ac00f040 con 0x7fa4bc0fbcb0 2024-05-17T21:59:01.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:01.642+0000 7fa4c19d5700 1 -- 172.21.15.107:0/3497503710 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4bc112fa0 con 0x7fa4bc0fbcb0 2024-05-17T21:59:01.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:01.642+0000 7fa4a37fe700 1 -- 172.21.15.107:0/3497503710 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa4ac00da30 con 0x7fa4bc0fbcb0 2024-05-17T21:59:01.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:01.642+0000 7fa4c19d5700 1 -- 172.21.15.107:0/3497503710 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4bc113530 con 0x7fa4bc0fbcb0 2024-05-17T21:59:01.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:01.643+0000 7fa4c19d5700 1 -- 172.21.15.107:0/3497503710 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4bc101af0 con 0x7fa4bc0fbcb0 2024-05-17T21:59:01.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:01.644+0000 7fa4a37fe700 1 -- 172.21.15.107:0/3497503710 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7fa4ac00db90 con 0x7fa4bc0fbcb0 2024-05-17T21:59:01.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:01.644+0000 7fa4a37fe700 1 --2- 172.21.15.107:0/3497503710 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fa4a4078990 0x7fa4a407ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:01.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:01.644+0000 7fa4ba7fc700 1 --2- 172.21.15.107:0/3497503710 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fa4a4078990 0x7fa4a407ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:01.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:01.644+0000 7fa4a37fe700 1 -- 172.21.15.107:0/3497503710 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fa4ac09a480 con 0x7fa4bc0fbcb0 2024-05-17T21:59:01.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:01.645+0000 7fa4ba7fc700 1 --2- 172.21.15.107:0/3497503710 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fa4a4078990 0x7fa4a407ae40 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa4b000b970 tx=0x7fa4b0009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:01.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:01.649+0000 7fa4a37fe700 1 -- 172.21.15.107:0/3497503710 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7fa4ac0620f0 con 0x7fa4bc0fbcb0 2024-05-17T21:59:01.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:01.897+0000 7fa4c19d5700 1 -- 172.21.15.107:0/3497503710 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fa4bc02b510 con 0x7fa4bc0fbcb0 2024-05-17T21:59:01.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:01.898+0000 7fa4a37fe700 1 -- 172.21.15.107:0/3497503710 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1910 (secure 0 0 0) 0x7fa4ac061a90 con 0x7fa4bc0fbcb0 2024-05-17T21:59:01.890 INFO:teuthology.orchestra.run.smithi107.stdout:e15 2024-05-17T21:59:01.890 INFO:teuthology.orchestra.run.smithi107.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-17T21:59:01.890 INFO:teuthology.orchestra.run.smithi107.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-17T21:59:01.890 INFO:teuthology.orchestra.run.smithi107.stdout:legacy client fscid: 1 2024-05-17T21:59:01.890 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:59:01.890 INFO:teuthology.orchestra.run.smithi107.stdout:Filesystem 'cephfs' (1) 2024-05-17T21:59:01.890 INFO:teuthology.orchestra.run.smithi107.stdout:fs_name cephfs 2024-05-17T21:59:01.890 INFO:teuthology.orchestra.run.smithi107.stdout:epoch 15 2024-05-17T21:59:01.890 INFO:teuthology.orchestra.run.smithi107.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-17T21:59:01.890 INFO:teuthology.orchestra.run.smithi107.stdout:created 2024-05-17T21:49:16.836333+0000 2024-05-17T21:59:01.890 INFO:teuthology.orchestra.run.smithi107.stdout:modified 2024-05-17T21:49:34.873628+0000 2024-05-17T21:59:01.891 INFO:teuthology.orchestra.run.smithi107.stdout:tableserver 0 2024-05-17T21:59:01.891 INFO:teuthology.orchestra.run.smithi107.stdout:root 0 2024-05-17T21:59:01.891 INFO:teuthology.orchestra.run.smithi107.stdout:session_timeout 60 2024-05-17T21:59:01.891 INFO:teuthology.orchestra.run.smithi107.stdout:session_autoclose 300 2024-05-17T21:59:01.891 INFO:teuthology.orchestra.run.smithi107.stdout:max_file_size 1099511627776 2024-05-17T21:59:01.891 INFO:teuthology.orchestra.run.smithi107.stdout:max_xattr_size 65536 2024-05-17T21:59:01.891 INFO:teuthology.orchestra.run.smithi107.stdout:required_client_features {} 2024-05-17T21:59:01.891 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure 0 2024-05-17T21:59:01.891 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure_osd_epoch 46 2024-05-17T21:59:01.891 INFO:teuthology.orchestra.run.smithi107.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-17T21:59:01.891 INFO:teuthology.orchestra.run.smithi107.stdout:max_mds 1 2024-05-17T21:59:01.891 INFO:teuthology.orchestra.run.smithi107.stdout:in 0 2024-05-17T21:59:01.891 INFO:teuthology.orchestra.run.smithi107.stdout:up {0=24259} 2024-05-17T21:59:01.891 INFO:teuthology.orchestra.run.smithi107.stdout:failed 2024-05-17T21:59:01.892 INFO:teuthology.orchestra.run.smithi107.stdout:damaged 2024-05-17T21:59:01.892 INFO:teuthology.orchestra.run.smithi107.stdout:stopped 2024-05-17T21:59:01.892 INFO:teuthology.orchestra.run.smithi107.stdout:data_pools [3] 2024-05-17T21:59:01.892 INFO:teuthology.orchestra.run.smithi107.stdout:metadata_pool 2 2024-05-17T21:59:01.892 INFO:teuthology.orchestra.run.smithi107.stdout:inline_data enabled 2024-05-17T21:59:01.892 INFO:teuthology.orchestra.run.smithi107.stdout:balancer 2024-05-17T21:59:01.892 INFO:teuthology.orchestra.run.smithi107.stdout:bal_rank_mask -1 2024-05-17T21:59:01.892 INFO:teuthology.orchestra.run.smithi107.stdout:standby_count_wanted 1 2024-05-17T21:59:01.892 INFO:teuthology.orchestra.run.smithi107.stdout:qdb_cluster leader: 0 members: 2024-05-17T21:59:01.892 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi138.gwbtyd{0:24259} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.138:6824/2779634919,v1:172.21.15.138:6825/2779634919] compat {c=[1],r=[1],i=[7ff]}] 2024-05-17T21:59:01.892 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.drqody{0:14486} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.107:6828/2275807100,v1:172.21.15.107:6829/2275807100] compat {c=[1],r=[1],i=[7ff]}] 2024-05-17T21:59:01.892 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:59:01.892 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:59:01.892 INFO:teuthology.orchestra.run.smithi107.stdout:Standby daemons: 2024-05-17T21:59:01.892 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:59:01.893 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.idmbgj{-1:14498} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.107:6826/1460186079,v1:172.21.15.107:6827/1460186079] compat {c=[1],r=[1],i=[7ff]}] 2024-05-17T21:59:01.893 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi138.zckrvs{-1:24267} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.138:6826/170628661,v1:172.21.15.138:6827/170628661] compat {c=[1],r=[1],i=[7ff]}] 2024-05-17T21:59:01.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:01.900+0000 7fa4c19d5700 1 -- 172.21.15.107:0/3497503710 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fa4a4078990 msgr2=0x7fa4a407ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:01.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:01.900+0000 7fa4c19d5700 1 --2- 172.21.15.107:0/3497503710 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fa4a4078990 0x7fa4a407ae40 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa4b000b970 tx=0x7fa4b0009e30 comp rx=0 tx=0).stop 2024-05-17T21:59:01.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:01.900+0000 7fa4c19d5700 1 -- 172.21.15.107:0/3497503710 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4bc0fbcb0 msgr2=0x7fa4bc10d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:01.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:01.900+0000 7fa4c19d5700 1 --2- 172.21.15.107:0/3497503710 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4bc0fbcb0 0x7fa4bc10d8f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac002a50 tx=0x7fa4ac0046b0 comp rx=0 tx=0).stop 2024-05-17T21:59:01.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:01.900+0000 7fa4c19d5700 1 -- 172.21.15.107:0/3497503710 shutdown_connections 2024-05-17T21:59:01.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:01.900+0000 7fa4c19d5700 1 --2- 172.21.15.107:0/3497503710 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fa4a4078990 0x7fa4a407ae40 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:01.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:01.900+0000 7fa4c19d5700 1 --2- 172.21.15.107:0/3497503710 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4bc0fbcb0 0x7fa4bc10d8f0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:01.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:01.900+0000 7fa4c19d5700 1 --2- 172.21.15.107:0/3497503710 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4bc0fcf60 0x7fa4bc10de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:01.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:01.900+0000 7fa4c19d5700 1 -- 172.21.15.107:0/3497503710 >> 172.21.15.107:0/3497503710 conn(0x7fa4bc0f7430 msgr2=0x7fa4bc066790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:01.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:01.901+0000 7fa4c19d5700 1 -- 172.21.15.107:0/3497503710 shutdown_connections 2024-05-17T21:59:01.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:01.901+0000 7fa4c19d5700 1 -- 172.21.15.107:0/3497503710 wait complete. 2024-05-17T21:59:01.894 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 15 2024-05-17T21:59:01.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:02.003+0000 7f9942c9c700 1 -- 172.21.15.107:0/2210682311 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f993c0fe4e0 msgr2=0x7f993c0fe8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:01.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:02.003+0000 7f9942c9c700 1 --2- 172.21.15.107:0/2210682311 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f993c0fe4e0 0x7f993c0fe8f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9930004940 tx=0x7f9930009e30 comp rx=0 tx=0).stop 2024-05-17T21:59:01.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:02.003+0000 7f9942c9c700 1 -- 172.21.15.107:0/2210682311 shutdown_connections 2024-05-17T21:59:01.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:02.003+0000 7f9942c9c700 1 --2- 172.21.15.107:0/2210682311 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f993c0fee30 0x7f993c0faf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:01.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:02.003+0000 7f9942c9c700 1 --2- 172.21.15.107:0/2210682311 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f993c0fe4e0 0x7f993c0fe8f0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:01.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:02.003+0000 7f9942c9c700 1 -- 172.21.15.107:0/2210682311 >> 172.21.15.107:0/2210682311 conn(0x7f993c0f6be0 msgr2=0x7f993c0f8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:01.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:02.003+0000 7f9942c9c700 1 -- 172.21.15.107:0/2210682311 shutdown_connections 2024-05-17T21:59:01.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:02.003+0000 7f9942c9c700 1 -- 172.21.15.107:0/2210682311 wait complete. 2024-05-17T21:59:01.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:02.003+0000 7f9942c9c700 1 Processor -- start 2024-05-17T21:59:01.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:02.003+0000 7f9942c9c700 1 -- start start 2024-05-17T21:59:01.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:02.004+0000 7f9942c9c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f993c0fe4e0 0x7f993c10d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:01.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:02.004+0000 7f9942c9c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f993c0fee30 0x7f993c10de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:01.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:02.004+0000 7f9942c9c700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f993c10e460 con 0x7f993c0fe4e0 2024-05-17T21:59:01.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:02.004+0000 7f9942c9c700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f993c10e5d0 con 0x7f993c0fee30 2024-05-17T21:59:01.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:02.004+0000 7f9940a38700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f993c0fe4e0 0x7f993c10d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:01.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:02.004+0000 7f9940a38700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f993c0fe4e0 0x7f993c10d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59334/0 (socket says 172.21.15.107:59334) 2024-05-17T21:59:01.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:02.004+0000 7f9940a38700 1 -- 172.21.15.107:0/1303951386 learned_addr learned my addr 172.21.15.107:0/1303951386 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:59:01.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:02.004+0000 7f993bfff700 1 --2- 172.21.15.107:0/1303951386 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f993c0fee30 0x7f993c10de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:01.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:02.004+0000 7f9940a38700 1 -- 172.21.15.107:0/1303951386 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f993c0fee30 msgr2=0x7f993c10de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:01.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:02.004+0000 7f9940a38700 1 --2- 172.21.15.107:0/1303951386 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f993c0fee30 0x7f993c10de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:01.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:02.004+0000 7f9940a38700 1 -- 172.21.15.107:0/1303951386 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9930005040 con 0x7f993c0fe4e0 2024-05-17T21:59:01.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:02.004+0000 7f9940a38700 1 --2- 172.21.15.107:0/1303951386 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f993c0fe4e0 0x7f993c10d920 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f993000b3b0 tx=0x7f9930009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:01.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:02.004+0000 7f99397fa700 1 -- 172.21.15.107:0/1303951386 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f993000e040 con 0x7f993c0fe4e0 2024-05-17T21:59:01.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:02.004+0000 7f99397fa700 1 -- 172.21.15.107:0/1303951386 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f993000bbf0 con 0x7f993c0fe4e0 2024-05-17T21:59:01.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:02.004+0000 7f9942c9c700 1 -- 172.21.15.107:0/1303951386 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f993c112fd0 con 0x7f993c0fe4e0 2024-05-17T21:59:01.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:02.004+0000 7f99397fa700 1 -- 172.21.15.107:0/1303951386 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f993001c630 con 0x7f993c0fe4e0 2024-05-17T21:59:01.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:02.005+0000 7f9942c9c700 1 -- 172.21.15.107:0/1303951386 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f993c113500 con 0x7f993c0fe4e0 2024-05-17T21:59:01.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:02.005+0000 7f9942c9c700 1 -- 172.21.15.107:0/1303951386 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f993c04c710 con 0x7f993c0fe4e0 2024-05-17T21:59:01.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:02.006+0000 7f99397fa700 1 -- 172.21.15.107:0/1303951386 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f993000bd60 con 0x7f993c0fe4e0 2024-05-17T21:59:01.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:02.007+0000 7f99397fa700 1 --2- 172.21.15.107:0/1303951386 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f992c078990 0x7f992c07ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:01.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:02.007+0000 7f993bfff700 1 --2- 172.21.15.107:0/1303951386 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f992c078990 0x7f992c07ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:01.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:02.007+0000 7f99397fa700 1 -- 172.21.15.107:0/1303951386 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f993009b2d0 con 0x7f993c0fe4e0 2024-05-17T21:59:01.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:02.007+0000 7f993bfff700 1 --2- 172.21.15.107:0/1303951386 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f992c078990 0x7f992c07ae40 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9928004f40 tx=0x7f992800b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:02.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:02.012+0000 7f99397fa700 1 -- 172.21.15.107:0/1303951386 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f9930062fc0 con 0x7f993c0fe4e0 2024-05-17T21:59:02.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:02.241+0000 7f9942c9c700 1 -- 172.21.15.107:0/1303951386 --> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f993c102600 con 0x7f992c078990 2024-05-17T21:59:02.233 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:02 smithi107 ceph-mon[93224]: pgmap v163: 65 pgs: 3 peering, 11 stale+active+clean, 51 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s 2024-05-17T21:59:02.233 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:02 smithi107 ceph-mon[93224]: from='client.34376 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:59:02.233 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:02 smithi107 ceph-mon[93224]: from='client.34380 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:59:02.233 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:02 smithi107 ceph-mon[93224]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-05-17T21:59:02.233 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:02 smithi107 ceph-mon[93224]: from='client.34384 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:59:02.233 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:02 smithi107 ceph-mon[93224]: from='client.? 172.21.15.107:0/1482093043' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:59:02.233 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:02 smithi107 ceph-mon[93224]: from='client.? 172.21.15.107:0/3497503710' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-17T21:59:02.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:02.242+0000 7f99397fa700 1 -- 172.21.15.107:0/1303951386 <== mgr.34100 v2:172.21.15.107:6800/3205268226 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f993c102600 con 0x7f992c078990 2024-05-17T21:59:02.234 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-05-17T21:59:02.234 INFO:teuthology.orchestra.run.smithi107.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b", 2024-05-17T21:59:02.234 INFO:teuthology.orchestra.run.smithi107.stdout: "in_progress": true, 2024-05-17T21:59:02.234 INFO:teuthology.orchestra.run.smithi107.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-17T21:59:02.234 INFO:teuthology.orchestra.run.smithi107.stdout: "services_complete": [ 2024-05-17T21:59:02.234 INFO:teuthology.orchestra.run.smithi107.stdout: "crash", 2024-05-17T21:59:02.235 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr", 2024-05-17T21:59:02.235 INFO:teuthology.orchestra.run.smithi107.stdout: "mon" 2024-05-17T21:59:02.235 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-05-17T21:59:02.235 INFO:teuthology.orchestra.run.smithi107.stdout: "progress": "11/21 daemons upgraded", 2024-05-17T21:59:02.235 INFO:teuthology.orchestra.run.smithi107.stdout: "message": "Currently upgrading osd daemons", 2024-05-17T21:59:02.235 INFO:teuthology.orchestra.run.smithi107.stdout: "is_paused": false 2024-05-17T21:59:02.235 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-05-17T21:59:02.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:02.244+0000 7f9942c9c700 1 -- 172.21.15.107:0/1303951386 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f992c078990 msgr2=0x7f992c07ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:02.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:02.244+0000 7f9942c9c700 1 --2- 172.21.15.107:0/1303951386 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f992c078990 0x7f992c07ae40 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9928004f40 tx=0x7f992800b040 comp rx=0 tx=0).stop 2024-05-17T21:59:02.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:02.244+0000 7f9942c9c700 1 -- 172.21.15.107:0/1303951386 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f993c0fe4e0 msgr2=0x7f993c10d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:02.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:02.244+0000 7f9942c9c700 1 --2- 172.21.15.107:0/1303951386 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f993c0fe4e0 0x7f993c10d920 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f993000b3b0 tx=0x7f9930009e30 comp rx=0 tx=0).stop 2024-05-17T21:59:02.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:02.245+0000 7f9942c9c700 1 -- 172.21.15.107:0/1303951386 shutdown_connections 2024-05-17T21:59:02.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:02.245+0000 7f9942c9c700 1 --2- 172.21.15.107:0/1303951386 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f992c078990 0x7f992c07ae40 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:02.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:02.245+0000 7f9942c9c700 1 --2- 172.21.15.107:0/1303951386 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f993c0fe4e0 0x7f993c10d920 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:02.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:02.245+0000 7f9942c9c700 1 --2- 172.21.15.107:0/1303951386 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f993c0fee30 0x7f993c10de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:02.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:02.245+0000 7f9942c9c700 1 -- 172.21.15.107:0/1303951386 >> 172.21.15.107:0/1303951386 conn(0x7f993c0f6be0 msgr2=0x7f993c0f8290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:02.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:02.245+0000 7f9942c9c700 1 -- 172.21.15.107:0/1303951386 shutdown_connections 2024-05-17T21:59:02.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:02.245+0000 7f9942c9c700 1 -- 172.21.15.107:0/1303951386 wait complete. 2024-05-17T21:59:02.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:02 smithi138 ceph-mon[81376]: pgmap v163: 65 pgs: 3 peering, 11 stale+active+clean, 51 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s 2024-05-17T21:59:02.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:02 smithi138 ceph-mon[81376]: from='client.34376 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:59:02.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:02 smithi138 ceph-mon[81376]: from='client.34380 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:59:02.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:02 smithi138 ceph-mon[81376]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-05-17T21:59:02.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:02 smithi138 ceph-mon[81376]: from='client.34384 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:59:02.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:02 smithi138 ceph-mon[81376]: from='client.? 172.21.15.107:0/1482093043' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:59:02.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:02 smithi138 ceph-mon[81376]: from='client.? 172.21.15.107:0/3497503710' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-17T21:59:02.326 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 17 21:59:02 smithi138 podman[100822]: 2024-05-17 21:59:02.020986638 +0000 UTC m=+0.822247225 container remove 3f539b9ee7c5b95bb3401a6ccab117651dd2c74693919260fa8564cb3b4609ca (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-5-deactivate, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS) 2024-05-17T21:59:02.326 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 17 21:59:02 smithi138 systemd[1]: ceph-3722731a-1496-11ef-bc9a-c7b262605968@osd.5.service: Deactivated successfully. 2024-05-17T21:59:02.326 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 17 21:59:02 smithi138 systemd[1]: Stopped Ceph osd.5 for 3722731a-1496-11ef-bc9a-c7b262605968. 2024-05-17T21:59:02.326 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 17 21:59:02 smithi138 systemd[1]: ceph-3722731a-1496-11ef-bc9a-c7b262605968@osd.5.service: Consumed 50.195s CPU time. 2024-05-17T21:59:02.326 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 17 21:59:02 smithi138 systemd[1]: Starting Ceph osd.5 for 3722731a-1496-11ef-bc9a-c7b262605968... 2024-05-17T21:59:02.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:02.347+0000 7f04e950f700 1 -- 172.21.15.107:0/1493754801 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f04e40fe420 msgr2=0x7f04e40fe830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:02.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:02.347+0000 7f04e950f700 1 --2- 172.21.15.107:0/1493754801 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f04e40fe420 0x7f04e40fe830 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f04d4004940 tx=0x7f04d4009e30 comp rx=0 tx=0).stop 2024-05-17T21:59:02.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:02.347+0000 7f04e950f700 1 -- 172.21.15.107:0/1493754801 shutdown_connections 2024-05-17T21:59:02.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:02.347+0000 7f04e950f700 1 --2- 172.21.15.107:0/1493754801 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04e40fed70 0x7f04e40faeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:02.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:02.347+0000 7f04e950f700 1 --2- 172.21.15.107:0/1493754801 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f04e40fe420 0x7f04e40fe830 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:02.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:02.347+0000 7f04e950f700 1 -- 172.21.15.107:0/1493754801 >> 172.21.15.107:0/1493754801 conn(0x7f04e40f6be0 msgr2=0x7f04e40f8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:02.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:02.347+0000 7f04e950f700 1 -- 172.21.15.107:0/1493754801 shutdown_connections 2024-05-17T21:59:02.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:02.347+0000 7f04e950f700 1 -- 172.21.15.107:0/1493754801 wait complete. 2024-05-17T21:59:02.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:02.348+0000 7f04e950f700 1 Processor -- start 2024-05-17T21:59:02.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:02.348+0000 7f04e950f700 1 -- start start 2024-05-17T21:59:02.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:02.348+0000 7f04e950f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f04e40fe420 0x7f04e410fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:02.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:02.348+0000 7f04e950f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04e40fed70 0x7f04e410ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:02.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:02.348+0000 7f04e950f700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04e41105c0 con 0x7f04e40fe420 2024-05-17T21:59:02.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:02.348+0000 7f04e950f700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04e4110730 con 0x7f04e40fed70 2024-05-17T21:59:02.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:02.348+0000 7f04e2ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f04e40fe420 0x7f04e410fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:02.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:02.348+0000 7f04e2ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f04e40fe420 0x7f04e410fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59352/0 (socket says 172.21.15.107:59352) 2024-05-17T21:59:02.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:02.348+0000 7f04e2ffd700 1 -- 172.21.15.107:0/2386945755 learned_addr learned my addr 172.21.15.107:0/2386945755 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:59:02.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:02.348+0000 7f04e27fc700 1 --2- 172.21.15.107:0/2386945755 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04e40fed70 0x7f04e410ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:02.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:02.348+0000 7f04e2ffd700 1 -- 172.21.15.107:0/2386945755 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04e40fed70 msgr2=0x7f04e410ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:02.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:02.348+0000 7f04e2ffd700 1 --2- 172.21.15.107:0/2386945755 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04e40fed70 0x7f04e410ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:02.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:02.348+0000 7f04e2ffd700 1 -- 172.21.15.107:0/2386945755 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04d4005040 con 0x7f04e40fe420 2024-05-17T21:59:02.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:02.349+0000 7f04e2ffd700 1 --2- 172.21.15.107:0/2386945755 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f04e40fe420 0x7f04e410fa80 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f04d4013870 tx=0x7f04d4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:02.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:02.349+0000 7f04db7fe700 1 -- 172.21.15.107:0/2386945755 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f04d4021070 con 0x7f04e40fe420 2024-05-17T21:59:02.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:02.349+0000 7f04db7fe700 1 -- 172.21.15.107:0/2386945755 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f04d4009430 con 0x7f04e40fe420 2024-05-17T21:59:02.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:02.349+0000 7f04e950f700 1 -- 172.21.15.107:0/2386945755 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04e4115130 con 0x7f04e40fe420 2024-05-17T21:59:02.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:02.349+0000 7f04db7fe700 1 -- 172.21.15.107:0/2386945755 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f04d400e070 con 0x7f04e40fe420 2024-05-17T21:59:02.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:02.349+0000 7f04e950f700 1 -- 172.21.15.107:0/2386945755 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04e4115670 con 0x7f04e40fe420 2024-05-17T21:59:02.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:02.350+0000 7f04e950f700 1 -- 172.21.15.107:0/2386945755 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04e4101b60 con 0x7f04e40fe420 2024-05-17T21:59:02.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:02.351+0000 7f04db7fe700 1 -- 172.21.15.107:0/2386945755 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f04d4013af0 con 0x7f04e40fe420 2024-05-17T21:59:02.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:02.351+0000 7f04db7fe700 1 --2- 172.21.15.107:0/2386945755 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f04d0078920 0x7f04d007add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:02.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:02.351+0000 7f04e27fc700 1 --2- 172.21.15.107:0/2386945755 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f04d0078920 0x7f04d007add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:02.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:02.351+0000 7f04db7fe700 1 -- 172.21.15.107:0/2386945755 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f04d409b8a0 con 0x7f04e40fe420 2024-05-17T21:59:02.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:02.352+0000 7f04e27fc700 1 --2- 172.21.15.107:0/2386945755 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f04d0078920 0x7f04d007add0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f04cc009b80 tx=0x7f04cc00b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:02.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:02.356+0000 7f04db7fe700 1 -- 172.21.15.107:0/2386945755 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f04d4063590 con 0x7f04e40fe420 2024-05-17T21:59:02.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:02.649+0000 7f04e950f700 1 -- 172.21.15.107:0/2386945755 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f04e402b400 con 0x7f04e40fe420 2024-05-17T21:59:02.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:02.650+0000 7f04db7fe700 1 -- 172.21.15.107:0/2386945755 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+788 (secure 0 0 0) 0x7f04d406ae60 con 0x7f04e40fe420 2024-05-17T21:59:02.642 INFO:teuthology.orchestra.run.smithi107.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data; 1 osds down; all OSDs are running squid or later but require_osd_release < squid; Reduced data availability: 2 pgs peering 2024-05-17T21:59:02.642 INFO:teuthology.orchestra.run.smithi107.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-17T21:59:02.642 INFO:teuthology.orchestra.run.smithi107.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-17T21:59:02.642 INFO:teuthology.orchestra.run.smithi107.stdout:[WRN] OSD_DOWN: 1 osds down 2024-05-17T21:59:02.642 INFO:teuthology.orchestra.run.smithi107.stdout: osd.5 (root=default,host=smithi138) is down 2024-05-17T21:59:02.642 INFO:teuthology.orchestra.run.smithi107.stdout:[WRN] OSD_UPGRADE_FINISHED: all OSDs are running squid or later but require_osd_release < squid 2024-05-17T21:59:02.643 INFO:teuthology.orchestra.run.smithi107.stdout: all OSDs are running squid or later but require_osd_release < squid 2024-05-17T21:59:02.643 INFO:teuthology.orchestra.run.smithi107.stdout:[WRN] PG_AVAILABILITY: Reduced data availability: 2 pgs peering 2024-05-17T21:59:02.643 INFO:teuthology.orchestra.run.smithi107.stdout: pg 3.c is stuck peering for 4m, current state peering, last acting [0,3] 2024-05-17T21:59:02.643 INFO:teuthology.orchestra.run.smithi107.stdout: pg 3.10 is stuck peering for 4m, current state peering, last acting [0,1] 2024-05-17T21:59:02.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:02.652+0000 7f04e950f700 1 -- 172.21.15.107:0/2386945755 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f04d0078920 msgr2=0x7f04d007add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:02.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:02.652+0000 7f04e950f700 1 --2- 172.21.15.107:0/2386945755 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f04d0078920 0x7f04d007add0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f04cc009b80 tx=0x7f04cc00b040 comp rx=0 tx=0).stop 2024-05-17T21:59:02.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:02.652+0000 7f04e950f700 1 -- 172.21.15.107:0/2386945755 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f04e40fe420 msgr2=0x7f04e410fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:02.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:02.652+0000 7f04e950f700 1 --2- 172.21.15.107:0/2386945755 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f04e40fe420 0x7f04e410fa80 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f04d4013870 tx=0x7f04d4009e30 comp rx=0 tx=0).stop 2024-05-17T21:59:02.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:02.652+0000 7f04e950f700 1 -- 172.21.15.107:0/2386945755 shutdown_connections 2024-05-17T21:59:02.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:02.652+0000 7f04e950f700 1 --2- 172.21.15.107:0/2386945755 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f04d0078920 0x7f04d007add0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:02.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:02.652+0000 7f04e950f700 1 --2- 172.21.15.107:0/2386945755 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f04e40fe420 0x7f04e410fa80 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:02.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:02.652+0000 7f04e950f700 1 --2- 172.21.15.107:0/2386945755 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04e40fed70 0x7f04e410ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:02.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:02.652+0000 7f04e950f700 1 -- 172.21.15.107:0/2386945755 >> 172.21.15.107:0/2386945755 conn(0x7f04e40f6be0 msgr2=0x7f04e40f8140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:02.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:02.652+0000 7f04e950f700 1 -- 172.21.15.107:0/2386945755 shutdown_connections 2024-05-17T21:59:02.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:02.652+0000 7f04e950f700 1 -- 172.21.15.107:0/2386945755 wait complete. 2024-05-17T21:59:02.774 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 17 21:59:02 smithi138 podman[100923]: 2024-05-17 21:59:02.472342695 +0000 UTC m=+0.112861405 container create 7f5ed1d97ac3696d131bcec9aee7eca0117e3032717ba9445bfdde2c6efa6ec7 (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-5-activate, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-17T21:59:02.774 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 17 21:59:02 smithi138 podman[100923]: 2024-05-17 21:59:02.376665665 +0000 UTC m=+0.017184399 image pull 8880f7613d0283efa7002505fdccdbc3e37d3b5da31c57fa22d62262c5dab66c quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b 2024-05-17T21:59:02.774 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 17 21:59:02 smithi138 podman[100923]: 2024-05-17 21:59:02.699933873 +0000 UTC m=+0.340452593 container init 7f5ed1d97ac3696d131bcec9aee7eca0117e3032717ba9445bfdde2c6efa6ec7 (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-5-activate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-05-17T21:59:02.774 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 17 21:59:02 smithi138 podman[100923]: 2024-05-17 21:59:02.703219774 +0000 UTC m=+0.343738496 container start 7f5ed1d97ac3696d131bcec9aee7eca0117e3032717ba9445bfdde2c6efa6ec7 (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-5-activate, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-05-17T21:59:02.774 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 17 21:59:02 smithi138 podman[100923]: 2024-05-17 21:59:02.783237907 +0000 UTC m=+0.423756638 container attach 7f5ed1d97ac3696d131bcec9aee7eca0117e3032717ba9445bfdde2c6efa6ec7 (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-5-activate, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-05-17T21:59:03.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:03 smithi138 ceph-mon[81376]: from='client.? 172.21.15.107:0/2386945755' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-17T21:59:03.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:03 smithi107 ceph-mon[93224]: from='client.? 172.21.15.107:0/2386945755' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-17T21:59:03.576 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 17 21:59:03 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-5-activate[100937]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-05-17T21:59:03.576 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 17 21:59:03 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-5-activate[100937]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-05-17T21:59:03.576 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 17 21:59:03 smithi138 bash[100923]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-05-17T21:59:03.576 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 17 21:59:03 smithi138 bash[100923]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-05-17T21:59:03.576 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 17 21:59:03 smithi138 bash[100923]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-05-17T21:59:03.576 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 17 21:59:03 smithi138 bash[100923]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-05-17T21:59:03.577 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 17 21:59:03 smithi138 bash[100923]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-05-17T21:59:03.577 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 17 21:59:03 smithi138 bash[100923]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-05-17T21:59:03.577 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 17 21:59:03 smithi138 bash[100923]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5/block 2024-05-17T21:59:03.577 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 17 21:59:03 smithi138 bash[100923]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-05-17T21:59:03.577 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 17 21:59:03 smithi138 bash[100923]: --> ceph-volume raw activate successful for osd ID: 5 2024-05-17T21:59:03.577 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 17 21:59:03 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-5-activate[100937]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-05-17T21:59:03.577 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 17 21:59:03 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-5-activate[100937]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-05-17T21:59:03.577 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 17 21:59:03 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-5-activate[100937]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-05-17T21:59:03.577 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 17 21:59:03 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-5-activate[100937]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-05-17T21:59:03.577 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 17 21:59:03 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-5-activate[100937]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5/block 2024-05-17T21:59:03.577 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 17 21:59:03 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-5-activate[100937]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-05-17T21:59:03.577 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 17 21:59:03 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-5-activate[100937]: --> ceph-volume raw activate successful for osd ID: 5 2024-05-17T21:59:03.577 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 17 21:59:03 smithi138 podman[100923]: 2024-05-17 21:59:03.324203215 +0000 UTC m=+0.964721934 container died 7f5ed1d97ac3696d131bcec9aee7eca0117e3032717ba9445bfdde2c6efa6ec7 (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-5-activate, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-05-17T21:59:04.323 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 17 21:59:04 smithi138 podman[101076]: 2024-05-17 21:59:04.112330514 +0000 UTC m=+0.782226599 container remove 7f5ed1d97ac3696d131bcec9aee7eca0117e3032717ba9445bfdde2c6efa6ec7 (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-5-activate, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a) 2024-05-17T21:59:04.323 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:04 smithi138 ceph-mon[81376]: from='client.34396 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:59:04.323 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:04 smithi138 ceph-mon[81376]: pgmap v164: 65 pgs: 13 active+undersized+degraded, 3 peering, 12 active+undersized, 37 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 42/267 objects degraded (15.730%) 2024-05-17T21:59:04.323 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:04 smithi138 ceph-mon[81376]: Health check failed: Degraded data redundancy: 42/267 objects degraded (15.730%), 13 pgs degraded (PG_DEGRADED) 2024-05-17T21:59:04.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:04 smithi107 ceph-mon[93224]: from='client.34396 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:59:04.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:04 smithi107 ceph-mon[93224]: pgmap v164: 65 pgs: 13 active+undersized+degraded, 3 peering, 12 active+undersized, 37 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 42/267 objects degraded (15.730%) 2024-05-17T21:59:04.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:04 smithi107 ceph-mon[93224]: Health check failed: Degraded data redundancy: 42/267 objects degraded (15.730%), 13 pgs degraded (PG_DEGRADED) 2024-05-17T21:59:04.575 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 17 21:59:04 smithi138 podman[101134]: 2024-05-17 21:59:04.235127443 +0000 UTC m=+0.017659437 image pull 8880f7613d0283efa7002505fdccdbc3e37d3b5da31c57fa22d62262c5dab66c quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b 2024-05-17T21:59:04.575 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 17 21:59:04 smithi138 podman[101134]: 2024-05-17 21:59:04.338298697 +0000 UTC m=+0.120830733 container create 63810d431961f18ecc215176a3d29574ca47af1decbb07b62508b551df395075 (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-5, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514) 2024-05-17T21:59:04.576 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 17 21:59:04 smithi138 podman[101134]: 2024-05-17 21:59:04.566463253 +0000 UTC m=+0.348995229 container init 63810d431961f18ecc215176a3d29574ca47af1decbb07b62508b551df395075 (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-5, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-05-17T21:59:04.576 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 17 21:59:04 smithi138 podman[101134]: 2024-05-17 21:59:04.569741954 +0000 UTC m=+0.352273937 container start 63810d431961f18ecc215176a3d29574ca47af1decbb07b62508b551df395075 (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-5, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-17T21:59:05.075 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 17 21:59:04 smithi138 ceph-osd[101153]: -- 172.21.15.138:0/3001838080 <== mon.1 v2:172.21.15.138:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d838b83680 con 0x55d838c2ad00 2024-05-17T21:59:05.075 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 17 21:59:04 smithi138 bash[101134]: 63810d431961f18ecc215176a3d29574ca47af1decbb07b62508b551df395075 2024-05-17T21:59:05.075 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 17 21:59:04 smithi138 systemd[1]: Started Ceph osd.5 for 3722731a-1496-11ef-bc9a-c7b262605968. 2024-05-17T21:59:05.446 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 17 21:59:05 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-5[101149]: 2024-05-17T21:59:05.134+0000 7f0e127637c0 -1 Falling back to public interface 2024-05-17T21:59:05.785 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:05 smithi138 ceph-mon[81376]: pgmap v165: 65 pgs: 13 active+undersized+degraded, 3 peering, 12 active+undersized, 37 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 42/267 objects degraded (15.730%) 2024-05-17T21:59:05.785 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:05 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:59:05.785 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:05 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:59:05.785 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:05 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:59:05.785 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:05 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:59:05.785 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:05 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:59:05.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:05 smithi107 ceph-mon[93224]: pgmap v165: 65 pgs: 13 active+undersized+degraded, 3 peering, 12 active+undersized, 37 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 42/267 objects degraded (15.730%) 2024-05-17T21:59:05.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:05 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:59:05.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:05 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:59:05.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:05 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:59:05.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:05 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:59:05.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:05 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:59:06.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:06 smithi138 ceph-mon[81376]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-05-17T21:59:06.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:06 smithi107 ceph-mon[93224]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-05-17T21:59:07.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:07 smithi138 ceph-mon[81376]: pgmap v166: 65 pgs: 13 active+undersized+degraded, 15 active+undersized, 37 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 42/267 objects degraded (15.730%) 2024-05-17T21:59:07.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:07 smithi107 ceph-mon[93224]: pgmap v166: 65 pgs: 13 active+undersized+degraded, 15 active+undersized, 37 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 42/267 objects degraded (15.730%) 2024-05-17T21:59:09.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:08 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:59:09.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:08 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:59:09.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:08 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:59:09.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:08 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:59:09.966 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:09 smithi138 ceph-mon[81376]: pgmap v167: 65 pgs: 13 active+undersized+degraded, 15 active+undersized, 37 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 543 B/s rd, 0 op/s; 42/267 objects degraded (15.730%) 2024-05-17T21:59:09.966 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:09 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:59:09.966 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:09 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:59:10.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:09 smithi107 ceph-mon[93224]: pgmap v167: 65 pgs: 13 active+undersized+degraded, 15 active+undersized, 37 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 543 B/s rd, 0 op/s; 42/267 objects degraded (15.730%) 2024-05-17T21:59:10.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:09 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:59:10.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:09 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:59:10.325 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 17 21:59:09 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-5[101149]: 2024-05-17T21:59:09.972+0000 7f0e127637c0 -1 osd.5 74 log_to_monitors true 2024-05-17T21:59:11.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:10 smithi138 ceph-mon[81376]: from='osd.5 [v2:172.21.15.138:6816/4099935576,v1:172.21.15.138:6817/4099935576]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-17T21:59:11.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:10 smithi138 ceph-mon[81376]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-17T21:59:11.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:10 smithi107 ceph-mon[93224]: from='osd.5 [v2:172.21.15.138:6816/4099935576,v1:172.21.15.138:6817/4099935576]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-17T21:59:11.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:10 smithi107 ceph-mon[93224]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-17T21:59:12.075 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 17 21:59:11 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-5[101149]: 2024-05-17T21:59:11.792+0000 7f0dffb74700 -1 osd.5 74 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-17T21:59:12.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:11 smithi138 ceph-mon[81376]: pgmap v168: 65 pgs: 13 active+undersized+degraded, 15 active+undersized, 37 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 491 B/s rd, 0 op/s; 42/267 objects degraded (15.730%) 2024-05-17T21:59:12.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:11 smithi138 ceph-mon[81376]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-17T21:59:12.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:11 smithi138 ceph-mon[81376]: osdmap e77: 6 total, 5 up, 6 in 2024-05-17T21:59:12.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:11 smithi138 ceph-mon[81376]: from='osd.5 [v2:172.21.15.138:6816/4099935576,v1:172.21.15.138:6817/4099935576]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-05-17T21:59:12.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:11 smithi138 ceph-mon[81376]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-05-17T21:59:12.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:11 smithi138 ceph-mon[81376]: from='osd.5 ' entity='osd.5' 2024-05-17T21:59:12.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:11 smithi107 ceph-mon[93224]: pgmap v168: 65 pgs: 13 active+undersized+degraded, 15 active+undersized, 37 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 491 B/s rd, 0 op/s; 42/267 objects degraded (15.730%) 2024-05-17T21:59:12.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:11 smithi107 ceph-mon[93224]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-17T21:59:12.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:11 smithi107 ceph-mon[93224]: osdmap e77: 6 total, 5 up, 6 in 2024-05-17T21:59:12.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:11 smithi107 ceph-mon[93224]: from='osd.5 [v2:172.21.15.138:6816/4099935576,v1:172.21.15.138:6817/4099935576]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-05-17T21:59:12.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:11 smithi107 ceph-mon[93224]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-05-17T21:59:12.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:11 smithi107 ceph-mon[93224]: from='osd.5 ' entity='osd.5' 2024-05-17T21:59:13.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:13 smithi138 ceph-mon[81376]: pgmap v170: 65 pgs: 13 active+undersized+degraded, 15 active+undersized, 37 active+clean; 258 MiB data, 976 MiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s; 42/267 objects degraded (15.730%) 2024-05-17T21:59:13.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:13 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:59:13.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:13 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:59:13.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:13 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:59:13.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:13 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T21:59:13.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:13 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:59:13.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:13 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:59:13.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:13 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:59:13.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:13 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:59:13.826 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:13 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:59:13.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:13 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:59:13.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:13 smithi138 ceph-mon[81376]: Upgrade: Setting container_image for all osd 2024-05-17T21:59:13.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:13 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:59:13.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:13 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-05-17T21:59:13.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:13 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-05-17T21:59:13.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:13 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-05-17T21:59:13.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:13 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-05-17T21:59:13.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:13 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-05-17T21:59:13.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:13 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-05-17T21:59:13.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:13 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-05-17T21:59:13.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:13 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-05-17T21:59:13.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:13 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-05-17T21:59:13.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:13 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-05-17T21:59:13.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:13 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-05-17T21:59:13.827 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:13 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-05-17T21:59:13.828 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:13 smithi138 ceph-mon[81376]: Upgrade: Setting require_osd_release to 19 squid 2024-05-17T21:59:13.828 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:13 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd require-osd-release", "release": "squid"}]: dispatch 2024-05-17T21:59:13.828 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:13 smithi138 ceph-mon[81376]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-05-17T21:59:13.828 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:13 smithi138 ceph-mon[81376]: Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-05-17T21:59:13.828 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:13 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-05-17T21:59:13.828 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:13 smithi138 ceph-mon[81376]: osd.5 [v2:172.21.15.138:6816/4099935576,v1:172.21.15.138:6817/4099935576] boot 2024-05-17T21:59:13.828 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:13 smithi138 ceph-mon[81376]: osdmap e78: 6 total, 6 up, 6 in 2024-05-17T21:59:13.828 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:13 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-17T21:59:13.828 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:13 smithi138 ceph-mon[81376]: Upgrade: Disabling standby-replay for filesystem cephfs 2024-05-17T21:59:13.828 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:13 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:59:13.828 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:13 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]: dispatch 2024-05-17T21:59:13.828 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:13 smithi138 ceph-mon[81376]: osdmap e79: 6 total, 6 up, 6 in 2024-05-17T21:59:13.847 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:13 smithi107 ceph-mon[93224]: pgmap v170: 65 pgs: 13 active+undersized+degraded, 15 active+undersized, 37 active+clean; 258 MiB data, 976 MiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s; 42/267 objects degraded (15.730%) 2024-05-17T21:59:13.847 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:13 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:59:13.847 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:13 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:59:13.847 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:13 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:59:13.847 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:13 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T21:59:13.847 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:13 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:59:13.847 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:13 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:59:13.848 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:13 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:59:13.848 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:13 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:59:13.848 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:13 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:59:13.848 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:13 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:59:13.848 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:13 smithi107 ceph-mon[93224]: Upgrade: Setting container_image for all osd 2024-05-17T21:59:13.848 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:13 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:59:13.848 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:13 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-05-17T21:59:13.848 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:13 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-05-17T21:59:13.848 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:13 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-05-17T21:59:13.848 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:13 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-05-17T21:59:13.848 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:13 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-05-17T21:59:13.848 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:13 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-05-17T21:59:13.848 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:13 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-05-17T21:59:13.849 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:13 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-05-17T21:59:13.849 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:13 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-05-17T21:59:13.849 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:13 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-05-17T21:59:13.849 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:13 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-05-17T21:59:13.849 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:13 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-05-17T21:59:13.849 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:13 smithi107 ceph-mon[93224]: Upgrade: Setting require_osd_release to 19 squid 2024-05-17T21:59:13.849 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:13 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd require-osd-release", "release": "squid"}]: dispatch 2024-05-17T21:59:13.849 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:13 smithi107 ceph-mon[93224]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-05-17T21:59:13.849 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:13 smithi107 ceph-mon[93224]: Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-05-17T21:59:13.849 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:13 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-05-17T21:59:13.849 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:13 smithi107 ceph-mon[93224]: osd.5 [v2:172.21.15.138:6816/4099935576,v1:172.21.15.138:6817/4099935576] boot 2024-05-17T21:59:13.849 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:13 smithi107 ceph-mon[93224]: osdmap e78: 6 total, 6 up, 6 in 2024-05-17T21:59:13.849 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:13 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-17T21:59:13.849 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:13 smithi107 ceph-mon[93224]: Upgrade: Disabling standby-replay for filesystem cephfs 2024-05-17T21:59:13.850 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:13 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:59:13.850 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:13 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]: dispatch 2024-05-17T21:59:13.850 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:13 smithi107 ceph-mon[93224]: osdmap e79: 6 total, 6 up, 6 in 2024-05-17T21:59:15.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:14 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]': finished 2024-05-17T21:59:15.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:14 smithi138 ceph-mon[81376]: fsmap cephfs:1 {0=cephfs.smithi138.gwbtyd=up:active} 2 up:standby 2024-05-17T21:59:15.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:14 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:59:15.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:14 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:59:15.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:14 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T21:59:15.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:14 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:59:15.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:14 smithi138 ceph-mon[81376]: osdmap e80: 6 total, 6 up, 6 in 2024-05-17T21:59:15.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:14 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:59:15.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:14 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:59:15.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:14 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:59:15.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:14 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:59:15.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:14 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:59:15.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:14 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi107.idmbgj"]}]: dispatch 2024-05-17T21:59:15.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:14 smithi138 ceph-mon[81376]: Upgrade: It appears safe to stop mds.cephfs.smithi107.idmbgj 2024-05-17T21:59:15.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:14 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]': finished 2024-05-17T21:59:15.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:14 smithi107 ceph-mon[93224]: fsmap cephfs:1 {0=cephfs.smithi138.gwbtyd=up:active} 2 up:standby 2024-05-17T21:59:15.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:14 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:59:15.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:14 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:59:15.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:14 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T21:59:15.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:14 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:59:15.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:14 smithi107 ceph-mon[93224]: osdmap e80: 6 total, 6 up, 6 in 2024-05-17T21:59:15.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:14 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:59:15.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:14 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:59:15.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:14 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:59:15.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:14 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:59:15.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:14 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:59:15.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:14 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi107.idmbgj"]}]: dispatch 2024-05-17T21:59:15.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:14 smithi107 ceph-mon[93224]: Upgrade: It appears safe to stop mds.cephfs.smithi107.idmbgj 2024-05-17T21:59:16.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:15 smithi107 ceph-mon[93224]: pgmap v174: 65 pgs: 13 active+undersized+degraded, 15 active+undersized, 37 active+clean; 258 MiB data, 976 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s; 42/267 objects degraded (15.730%) 2024-05-17T21:59:16.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:15 smithi107 ceph-mon[93224]: mds.? [v2:172.21.15.107:6828/878488679,v1:172.21.15.107:6829/878488679] up:boot 2024-05-17T21:59:16.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:15 smithi107 ceph-mon[93224]: fsmap cephfs:1 {0=cephfs.smithi138.gwbtyd=up:active} 3 up:standby 2024-05-17T21:59:16.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:15 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi107.drqody"}]: dispatch 2024-05-17T21:59:16.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:15 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:59:16.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:15 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi107.idmbgj", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T21:59:16.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:15 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:59:16.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:15 smithi138 ceph-mon[81376]: pgmap v174: 65 pgs: 13 active+undersized+degraded, 15 active+undersized, 37 active+clean; 258 MiB data, 976 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s; 42/267 objects degraded (15.730%) 2024-05-17T21:59:16.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:15 smithi138 ceph-mon[81376]: mds.? [v2:172.21.15.107:6828/878488679,v1:172.21.15.107:6829/878488679] up:boot 2024-05-17T21:59:16.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:15 smithi138 ceph-mon[81376]: fsmap cephfs:1 {0=cephfs.smithi138.gwbtyd=up:active} 3 up:standby 2024-05-17T21:59:16.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:15 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi107.drqody"}]: dispatch 2024-05-17T21:59:16.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:15 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:59:16.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:15 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi107.idmbgj", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T21:59:16.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:15 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:59:17.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:16 smithi107 ceph-mon[93224]: Upgrade: Updating mds.cephfs.smithi107.idmbgj 2024-05-17T21:59:17.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:16 smithi107 ceph-mon[93224]: Deploying daemon mds.cephfs.smithi107.idmbgj on smithi107 2024-05-17T21:59:17.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:16 smithi138 ceph-mon[81376]: Upgrade: Updating mds.cephfs.smithi107.idmbgj 2024-05-17T21:59:17.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:16 smithi138 ceph-mon[81376]: Deploying daemon mds.cephfs.smithi107.idmbgj on smithi107 2024-05-17T21:59:18.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:17 smithi107 ceph-mon[93224]: pgmap v175: 65 pgs: 7 active+undersized+degraded, 3 active+undersized, 55 active+clean; 258 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 913 B/s rd, 1 op/s; 25/267 objects degraded (9.363%) 2024-05-17T21:59:18.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:17 smithi107 ceph-mon[93224]: Health check update: Degraded data redundancy: 25/267 objects degraded (9.363%), 7 pgs degraded (PG_DEGRADED) 2024-05-17T21:59:18.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:17 smithi138 ceph-mon[81376]: pgmap v175: 65 pgs: 7 active+undersized+degraded, 3 active+undersized, 55 active+clean; 258 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 913 B/s rd, 1 op/s; 25/267 objects degraded (9.363%) 2024-05-17T21:59:18.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:17 smithi138 ceph-mon[81376]: Health check update: Degraded data redundancy: 25/267 objects degraded (9.363%), 7 pgs degraded (PG_DEGRADED) 2024-05-17T21:59:19.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:18 smithi138 ceph-mon[81376]: osdmap e81: 6 total, 6 up, 6 in 2024-05-17T21:59:19.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:18 smithi138 ceph-mon[81376]: fsmap cephfs:1 {0=cephfs.smithi138.gwbtyd=up:active} 2 up:standby 2024-05-17T21:59:19.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:18 smithi107 ceph-mon[93224]: osdmap e81: 6 total, 6 up, 6 in 2024-05-17T21:59:19.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:18 smithi107 ceph-mon[93224]: fsmap cephfs:1 {0=cephfs.smithi138.gwbtyd=up:active} 2 up:standby 2024-05-17T21:59:20.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:19 smithi138 ceph-mon[81376]: pgmap v177: 65 pgs: 65 active+clean; 258 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-05-17T21:59:20.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:19 smithi138 ceph-mon[81376]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 25/267 objects degraded (9.363%), 7 pgs degraded) 2024-05-17T21:59:20.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:19 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:59:20.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:19 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:59:20.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:19 smithi107 ceph-mon[93224]: pgmap v177: 65 pgs: 65 active+clean; 258 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-05-17T21:59:20.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:19 smithi107 ceph-mon[93224]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 25/267 objects degraded (9.363%), 7 pgs degraded) 2024-05-17T21:59:20.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:19 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:59:20.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:19 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:59:22.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:21 smithi107 ceph-mon[93224]: pgmap v178: 65 pgs: 65 active+clean; 258 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-05-17T21:59:22.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:21 smithi138 ceph-mon[81376]: pgmap v178: 65 pgs: 65 active+clean; 258 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-05-17T21:59:23.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:22 smithi107 ceph-mon[93224]: mds.? [v2:172.21.15.107:6826/2235359521,v1:172.21.15.107:6827/2235359521] up:boot 2024-05-17T21:59:23.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:22 smithi107 ceph-mon[93224]: fsmap cephfs:1 {0=cephfs.smithi138.gwbtyd=up:active} 3 up:standby 2024-05-17T21:59:23.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:22 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi107.idmbgj"}]: dispatch 2024-05-17T21:59:23.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:22 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:59:23.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:22 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:59:23.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:22 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:59:23.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:22 smithi138 ceph-mon[81376]: mds.? [v2:172.21.15.107:6826/2235359521,v1:172.21.15.107:6827/2235359521] up:boot 2024-05-17T21:59:23.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:22 smithi138 ceph-mon[81376]: fsmap cephfs:1 {0=cephfs.smithi138.gwbtyd=up:active} 3 up:standby 2024-05-17T21:59:23.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:22 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi107.idmbgj"}]: dispatch 2024-05-17T21:59:23.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:22 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:59:23.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:22 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:59:23.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:22 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:59:24.181 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:23 smithi107 ceph-mon[93224]: pgmap v179: 65 pgs: 65 active+clean; 258 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-05-17T21:59:24.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:23 smithi138 ceph-mon[81376]: pgmap v179: 65 pgs: 65 active+clean; 258 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-05-17T21:59:25.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:25 smithi138 ceph-mon[81376]: pgmap v180: 65 pgs: 65 active+clean; 258 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-05-17T21:59:25.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:25 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:59:25.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:25 smithi107 ceph-mon[93224]: pgmap v180: 65 pgs: 65 active+clean; 258 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-05-17T21:59:25.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:25 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:59:28.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:27 smithi138 ceph-mon[81376]: pgmap v181: 65 pgs: 65 active+clean; 258 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-05-17T21:59:28.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:27 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:59:28.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:27 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:59:28.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:27 smithi107 ceph-mon[93224]: pgmap v181: 65 pgs: 65 active+clean; 258 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-05-17T21:59:28.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:27 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:59:28.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:27 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:59:29.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:28 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:59:29.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:28 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:59:29.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:28 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:59:29.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:28 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:59:30.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:29 smithi138 ceph-mon[81376]: pgmap v182: 65 pgs: 65 active+clean; 258 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-05-17T21:59:30.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:29 smithi107 ceph-mon[93224]: pgmap v182: 65 pgs: 65 active+clean; 258 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-05-17T21:59:32.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:32 smithi138 ceph-mon[81376]: pgmap v183: 65 pgs: 65 active+clean; 258 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-05-17T21:59:32.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:32 smithi138 ceph-mon[81376]: Detected new or changed devices on smithi107 2024-05-17T21:59:32.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:32 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:59:32.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:32 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:59:32.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:32 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:59:32.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:32 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T21:59:32.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:32 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:59:32.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:32 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:59:32.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:32 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:59:32.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:32 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:59:32.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:32 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:59:32.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:32 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:59:32.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:32 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi107.drqody"]}]: dispatch 2024-05-17T21:59:32.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:32 smithi107 ceph-mon[93224]: pgmap v183: 65 pgs: 65 active+clean; 258 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-05-17T21:59:32.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:32 smithi107 ceph-mon[93224]: Detected new or changed devices on smithi107 2024-05-17T21:59:32.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:32 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:59:32.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:32 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:59:32.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:32 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:59:32.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:32 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T21:59:32.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:32 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:59:32.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:32 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:59:32.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:32 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:59:32.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:32 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:59:32.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:32 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:59:32.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:32 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:59:32.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:32 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi107.drqody"]}]: dispatch 2024-05-17T21:59:32.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:32.760+0000 7f61f68c2700 1 -- 172.21.15.107:0/3363043845 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61f00fff70 msgr2=0x7f61f01023f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:32.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:32.760+0000 7f61f68c2700 1 --2- 172.21.15.107:0/3363043845 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61f00fff70 0x7f61f01023f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f61d8004940 tx=0x7f61d8009e30 comp rx=0 tx=0).stop 2024-05-17T21:59:32.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:32.761+0000 7f61f68c2700 1 -- 172.21.15.107:0/3363043845 shutdown_connections 2024-05-17T21:59:32.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:32.761+0000 7f61f68c2700 1 --2- 172.21.15.107:0/3363043845 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61f00fff70 0x7f61f01023f0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:32.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:32.761+0000 7f61f68c2700 1 --2- 172.21.15.107:0/3363043845 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61f0066ba0 0x7f61f00ffa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:32.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:32.761+0000 7f61f68c2700 1 -- 172.21.15.107:0/3363043845 >> 172.21.15.107:0/3363043845 conn(0x7f61f00f7440 msgr2=0x7f61f00f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:32.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:32.761+0000 7f61f68c2700 1 -- 172.21.15.107:0/3363043845 shutdown_connections 2024-05-17T21:59:32.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:32.761+0000 7f61f68c2700 1 -- 172.21.15.107:0/3363043845 wait complete. 2024-05-17T21:59:32.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:32.761+0000 7f61f68c2700 1 Processor -- start 2024-05-17T21:59:32.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:32.761+0000 7f61f68c2700 1 -- start start 2024-05-17T21:59:32.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:32.761+0000 7f61f68c2700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61f00fff70 0x7f61f010da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:32.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:32.761+0000 7f61f68c2700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61f010df50 0x7f61f010eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:32.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:32.761+0000 7f61f68c2700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61f010e410 con 0x7f61f010df50 2024-05-17T21:59:32.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:32.761+0000 7f61f68c2700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61f010e550 con 0x7f61f00fff70 2024-05-17T21:59:32.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:32.762+0000 7f61e7fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61f010df50 0x7f61f010eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:32.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:32.762+0000 7f61e7fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61f010df50 0x7f61f010eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56714/0 (socket says 172.21.15.107:56714) 2024-05-17T21:59:32.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:32.762+0000 7f61e7fff700 1 -- 172.21.15.107:0/4273459221 learned_addr learned my addr 172.21.15.107:0/4273459221 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:59:32.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:32.762+0000 7f61effff700 1 --2- 172.21.15.107:0/4273459221 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61f00fff70 0x7f61f010da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:32.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:32.762+0000 7f61e7fff700 1 -- 172.21.15.107:0/4273459221 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61f00fff70 msgr2=0x7f61f010da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:32.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:32.762+0000 7f61e7fff700 1 --2- 172.21.15.107:0/4273459221 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61f00fff70 0x7f61f010da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:32.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:32.762+0000 7f61e7fff700 1 -- 172.21.15.107:0/4273459221 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61d8005040 con 0x7f61f010df50 2024-05-17T21:59:32.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:32.762+0000 7f61effff700 1 --2- 172.21.15.107:0/4273459221 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61f00fff70 0x7f61f010da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:59:32.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:32.762+0000 7f61e7fff700 1 --2- 172.21.15.107:0/4273459221 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61f010df50 0x7f61f010eda0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f61d800b970 tx=0x7f61d8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:32.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:32.762+0000 7f61ed7fa700 1 -- 172.21.15.107:0/4273459221 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f61d800e040 con 0x7f61f010df50 2024-05-17T21:59:32.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:32.762+0000 7f61ed7fa700 1 -- 172.21.15.107:0/4273459221 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f61d800bcc0 con 0x7f61f010df50 2024-05-17T21:59:32.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:32.762+0000 7f61ed7fa700 1 -- 172.21.15.107:0/4273459221 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f61d8013660 con 0x7f61f010df50 2024-05-17T21:59:32.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:32.762+0000 7f61f68c2700 1 -- 172.21.15.107:0/4273459221 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61f010f3a0 con 0x7f61f010df50 2024-05-17T21:59:32.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:32.763+0000 7f61f68c2700 1 -- 172.21.15.107:0/4273459221 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61f010f8e0 con 0x7f61f010df50 2024-05-17T21:59:32.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:32.763+0000 7f61f68c2700 1 -- 172.21.15.107:0/4273459221 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61f005f5c0 con 0x7f61f010df50 2024-05-17T21:59:32.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:32.764+0000 7f61ed7fa700 1 -- 172.21.15.107:0/4273459221 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f61d801ca00 con 0x7f61f010df50 2024-05-17T21:59:32.756 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:32.765+0000 7f61ed7fa700 1 --2- 172.21.15.107:0/4273459221 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f61dc078990 0x7f61dc07ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:32.756 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:32.765+0000 7f61ed7fa700 1 -- 172.21.15.107:0/4273459221 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f61d809b460 con 0x7f61f010df50 2024-05-17T21:59:32.756 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:32.765+0000 7f61effff700 1 --2- 172.21.15.107:0/4273459221 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f61dc078990 0x7f61dc07ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:32.756 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:32.765+0000 7f61effff700 1 --2- 172.21.15.107:0/4273459221 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f61dc078990 0x7f61dc07ae40 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f61e0007fd0 tx=0x7f61e0009040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:32.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:32.770+0000 7f61ed7fa700 1 -- 172.21.15.107:0/4273459221 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f61d80631a0 con 0x7f61f010df50 2024-05-17T21:59:32.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.004+0000 7f61f68c2700 1 -- 172.21.15.107:0/4273459221 --> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f61f0066ba0 con 0x7f61dc078990 2024-05-17T21:59:32.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.006+0000 7f61ed7fa700 1 -- 172.21.15.107:0/4273459221 <== mgr.34100 v2:172.21.15.107:6800/3205268226 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f61f0066ba0 con 0x7f61dc078990 2024-05-17T21:59:32.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.008+0000 7f61f68c2700 1 -- 172.21.15.107:0/4273459221 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f61dc078990 msgr2=0x7f61dc07ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:32.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.008+0000 7f61f68c2700 1 --2- 172.21.15.107:0/4273459221 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f61dc078990 0x7f61dc07ae40 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f61e0007fd0 tx=0x7f61e0009040 comp rx=0 tx=0).stop 2024-05-17T21:59:32.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.008+0000 7f61f68c2700 1 -- 172.21.15.107:0/4273459221 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61f010df50 msgr2=0x7f61f010eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:32.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.008+0000 7f61f68c2700 1 --2- 172.21.15.107:0/4273459221 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61f010df50 0x7f61f010eda0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f61d800b970 tx=0x7f61d8009e30 comp rx=0 tx=0).stop 2024-05-17T21:59:32.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.008+0000 7f61f68c2700 1 -- 172.21.15.107:0/4273459221 shutdown_connections 2024-05-17T21:59:32.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.008+0000 7f61f68c2700 1 --2- 172.21.15.107:0/4273459221 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f61dc078990 0x7f61dc07ae40 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:32.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.008+0000 7f61f68c2700 1 --2- 172.21.15.107:0/4273459221 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61f00fff70 0x7f61f010da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:32.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.008+0000 7f61f68c2700 1 --2- 172.21.15.107:0/4273459221 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61f010df50 0x7f61f010eda0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:32.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.008+0000 7f61f68c2700 1 -- 172.21.15.107:0/4273459221 >> 172.21.15.107:0/4273459221 conn(0x7f61f00f7440 msgr2=0x7f61f00f7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:32.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.008+0000 7f61f68c2700 1 -- 172.21.15.107:0/4273459221 shutdown_connections 2024-05-17T21:59:32.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.008+0000 7f61f68c2700 1 -- 172.21.15.107:0/4273459221 wait complete. 2024-05-17T21:59:33.009 INFO:teuthology.orchestra.run.smithi107.stdout:true 2024-05-17T21:59:33.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.115+0000 7fa1c1dba700 1 -- 172.21.15.107:0/4213460459 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1bc0fcf60 msgr2=0x7fa1bc0fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:33.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.115+0000 7fa1c1dba700 1 --2- 172.21.15.107:0/4213460459 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1bc0fcf60 0x7fa1bc0fd3d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa1a4004940 tx=0x7fa1a4009e30 comp rx=0 tx=0).stop 2024-05-17T21:59:33.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.115+0000 7fa1c1dba700 1 -- 172.21.15.107:0/4213460459 shutdown_connections 2024-05-17T21:59:33.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.115+0000 7fa1c1dba700 1 --2- 172.21.15.107:0/4213460459 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1bc0fcf60 0x7fa1bc0fd3d0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:33.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.115+0000 7fa1c1dba700 1 --2- 172.21.15.107:0/4213460459 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa1bc0fbcb0 0x7fa1bc0fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:33.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.115+0000 7fa1c1dba700 1 -- 172.21.15.107:0/4213460459 >> 172.21.15.107:0/4213460459 conn(0x7fa1bc0f7430 msgr2=0x7fa1bc0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:33.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.115+0000 7fa1c1dba700 1 -- 172.21.15.107:0/4213460459 shutdown_connections 2024-05-17T21:59:33.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.115+0000 7fa1c1dba700 1 -- 172.21.15.107:0/4213460459 wait complete. 2024-05-17T21:59:33.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.115+0000 7fa1c1dba700 1 Processor -- start 2024-05-17T21:59:33.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.116+0000 7fa1c1dba700 1 -- start start 2024-05-17T21:59:33.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.116+0000 7fa1c1dba700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa1bc0fbcb0 0x7fa1bc110720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:33.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.116+0000 7fa1c1dba700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1bc0fcf60 0x7fa1bc112c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:33.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.116+0000 7fa1c1dba700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1bc066330 con 0x7fa1bc0fcf60 2024-05-17T21:59:33.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.116+0000 7fa1c1dba700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1bc066470 con 0x7fa1bc0fbcb0 2024-05-17T21:59:33.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.116+0000 7fa1b2ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1bc0fcf60 0x7fa1bc112c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:33.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.116+0000 7fa1b2ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1bc0fcf60 0x7fa1bc112c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56732/0 (socket says 172.21.15.107:56732) 2024-05-17T21:59:33.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.116+0000 7fa1b2ffd700 1 -- 172.21.15.107:0/3640697621 learned_addr learned my addr 172.21.15.107:0/3640697621 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:59:33.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.116+0000 7fa1bb7fe700 1 --2- 172.21.15.107:0/3640697621 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa1bc0fbcb0 0x7fa1bc110720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:33.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.116+0000 7fa1b2ffd700 1 -- 172.21.15.107:0/3640697621 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa1bc0fbcb0 msgr2=0x7fa1bc110720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:33.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.116+0000 7fa1b2ffd700 1 --2- 172.21.15.107:0/3640697621 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa1bc0fbcb0 0x7fa1bc110720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:33.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.116+0000 7fa1b2ffd700 1 -- 172.21.15.107:0/3640697621 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1a4005040 con 0x7fa1bc0fcf60 2024-05-17T21:59:33.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.116+0000 7fa1b2ffd700 1 --2- 172.21.15.107:0/3640697621 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1bc0fcf60 0x7fa1bc112c70 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa1a400b3b0 tx=0x7fa1a4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:33.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.117+0000 7fa1b8ff9700 1 -- 172.21.15.107:0/3640697621 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa1a400e040 con 0x7fa1bc0fcf60 2024-05-17T21:59:33.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.117+0000 7fa1c1dba700 1 -- 172.21.15.107:0/3640697621 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1bc1131b0 con 0x7fa1bc0fcf60 2024-05-17T21:59:33.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.117+0000 7fa1b8ff9700 1 -- 172.21.15.107:0/3640697621 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa1a400bbf0 con 0x7fa1bc0fcf60 2024-05-17T21:59:33.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.117+0000 7fa1b8ff9700 1 -- 172.21.15.107:0/3640697621 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa1a401c630 con 0x7fa1bc0fcf60 2024-05-17T21:59:33.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.117+0000 7fa1c1dba700 1 -- 172.21.15.107:0/3640697621 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1bc113770 con 0x7fa1bc0fcf60 2024-05-17T21:59:33.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.118+0000 7fa1c1dba700 1 -- 172.21.15.107:0/3640697621 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1bc05f5c0 con 0x7fa1bc0fcf60 2024-05-17T21:59:33.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.119+0000 7fa1b8ff9700 1 -- 172.21.15.107:0/3640697621 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7fa1a400bd60 con 0x7fa1bc0fcf60 2024-05-17T21:59:33.110 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.119+0000 7fa1b8ff9700 1 --2- 172.21.15.107:0/3640697621 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fa1a8078990 0x7fa1a807ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:33.110 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.120+0000 7fa1bb7fe700 1 --2- 172.21.15.107:0/3640697621 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fa1a8078990 0x7fa1a807ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:33.110 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.120+0000 7fa1b8ff9700 1 -- 172.21.15.107:0/3640697621 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fa1a409baa0 con 0x7fa1bc0fcf60 2024-05-17T21:59:33.110 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.120+0000 7fa1bb7fe700 1 --2- 172.21.15.107:0/3640697621 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fa1a8078990 0x7fa1a807ae40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc065fb0 tx=0x7fa1ac00b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:33.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.123+0000 7fa1b8ff9700 1 -- 172.21.15.107:0/3640697621 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7fa1a4063770 con 0x7fa1bc0fcf60 2024-05-17T21:59:33.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.360+0000 7fa1c1dba700 1 -- 172.21.15.107:0/3640697621 --> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa1bc1012c0 con 0x7fa1a8078990 2024-05-17T21:59:33.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:33 smithi107 ceph-mon[93224]: Upgrade: It appears safe to stop mds.cephfs.smithi107.drqody 2024-05-17T21:59:33.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:33 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:59:33.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:33 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi107.drqody", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T21:59:33.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:33 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:59:33.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.361+0000 7fa1b8ff9700 1 -- 172.21.15.107:0/3640697621 <== mgr.34100 v2:172.21.15.107:6800/3205268226 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fa1bc1012c0 con 0x7fa1a8078990 2024-05-17T21:59:33.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.363+0000 7fa1c1dba700 1 -- 172.21.15.107:0/3640697621 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fa1a8078990 msgr2=0x7fa1a807ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:33.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.363+0000 7fa1c1dba700 1 --2- 172.21.15.107:0/3640697621 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fa1a8078990 0x7fa1a807ae40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc065fb0 tx=0x7fa1ac00b040 comp rx=0 tx=0).stop 2024-05-17T21:59:33.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.363+0000 7fa1c1dba700 1 -- 172.21.15.107:0/3640697621 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1bc0fcf60 msgr2=0x7fa1bc112c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:33.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.363+0000 7fa1c1dba700 1 --2- 172.21.15.107:0/3640697621 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1bc0fcf60 0x7fa1bc112c70 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa1a400b3b0 tx=0x7fa1a4009e30 comp rx=0 tx=0).stop 2024-05-17T21:59:33.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.363+0000 7fa1c1dba700 1 -- 172.21.15.107:0/3640697621 shutdown_connections 2024-05-17T21:59:33.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.363+0000 7fa1c1dba700 1 --2- 172.21.15.107:0/3640697621 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fa1a8078990 0x7fa1a807ae40 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:33.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.363+0000 7fa1c1dba700 1 --2- 172.21.15.107:0/3640697621 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa1bc0fbcb0 0x7fa1bc110720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:33.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.363+0000 7fa1c1dba700 1 --2- 172.21.15.107:0/3640697621 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1bc0fcf60 0x7fa1bc112c70 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:33.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.363+0000 7fa1c1dba700 1 -- 172.21.15.107:0/3640697621 >> 172.21.15.107:0/3640697621 conn(0x7fa1bc0f7430 msgr2=0x7fa1bc0f8b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:33.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.363+0000 7fa1c1dba700 1 -- 172.21.15.107:0/3640697621 shutdown_connections 2024-05-17T21:59:33.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.363+0000 7fa1c1dba700 1 -- 172.21.15.107:0/3640697621 wait complete. 2024-05-17T21:59:33.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.468+0000 7f9daeebf700 1 -- 172.21.15.107:0/671148917 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9da80fbcb0 msgr2=0x7f9da80fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:33.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.468+0000 7f9daeebf700 1 --2- 172.21.15.107:0/671148917 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9da80fbcb0 0x7f9da80fc0c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c004940 tx=0x7f9d9c009e30 comp rx=0 tx=0).stop 2024-05-17T21:59:33.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.468+0000 7f9daeebf700 1 -- 172.21.15.107:0/671148917 shutdown_connections 2024-05-17T21:59:33.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.468+0000 7f9daeebf700 1 --2- 172.21.15.107:0/671148917 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9da80fcf60 0x7f9da80fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:33.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.468+0000 7f9daeebf700 1 --2- 172.21.15.107:0/671148917 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9da80fbcb0 0x7f9da80fc0c0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:33.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.468+0000 7f9daeebf700 1 -- 172.21.15.107:0/671148917 >> 172.21.15.107:0/671148917 conn(0x7f9da80f7430 msgr2=0x7f9da80f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:33.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.468+0000 7f9daeebf700 1 -- 172.21.15.107:0/671148917 shutdown_connections 2024-05-17T21:59:33.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.468+0000 7f9daeebf700 1 -- 172.21.15.107:0/671148917 wait complete. 2024-05-17T21:59:33.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.469+0000 7f9daeebf700 1 Processor -- start 2024-05-17T21:59:33.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.469+0000 7f9daeebf700 1 -- start start 2024-05-17T21:59:33.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.469+0000 7f9daeebf700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9da80fbcb0 0x7f9da810d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:33.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.469+0000 7f9daeebf700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9da80fcf60 0x7f9da810df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:33.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.469+0000 7f9daeebf700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9da810e530 con 0x7f9da80fbcb0 2024-05-17T21:59:33.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.469+0000 7f9daeebf700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9da810ed80 con 0x7f9da80fcf60 2024-05-17T21:59:33.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.469+0000 7f9dacc5b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9da80fbcb0 0x7f9da810d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:33.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.469+0000 7f9dacc5b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9da80fbcb0 0x7f9da810d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56746/0 (socket says 172.21.15.107:56746) 2024-05-17T21:59:33.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.469+0000 7f9dacc5b700 1 -- 172.21.15.107:0/2026189339 learned_addr learned my addr 172.21.15.107:0/2026189339 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:59:33.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.469+0000 7f9da7fff700 1 --2- 172.21.15.107:0/2026189339 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9da80fcf60 0x7f9da810df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:33.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.469+0000 7f9dacc5b700 1 -- 172.21.15.107:0/2026189339 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9da80fcf60 msgr2=0x7f9da810df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:33.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.469+0000 7f9dacc5b700 1 --2- 172.21.15.107:0/2026189339 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9da80fcf60 0x7f9da810df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:33.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.469+0000 7f9dacc5b700 1 -- 172.21.15.107:0/2026189339 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d9c005040 con 0x7f9da80fbcb0 2024-05-17T21:59:33.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.470+0000 7f9dacc5b700 1 --2- 172.21.15.107:0/2026189339 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9da80fbcb0 0x7f9da810d9f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c00b3b0 tx=0x7f9d9c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:33.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.470+0000 7f9da57fa700 1 -- 172.21.15.107:0/2026189339 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9d9c00e040 con 0x7f9da80fbcb0 2024-05-17T21:59:33.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.470+0000 7f9daeebf700 1 -- 172.21.15.107:0/2026189339 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9da810f000 con 0x7f9da80fbcb0 2024-05-17T21:59:33.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.470+0000 7f9da57fa700 1 -- 172.21.15.107:0/2026189339 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9d9c00bbf0 con 0x7f9da80fbcb0 2024-05-17T21:59:33.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.470+0000 7f9da57fa700 1 -- 172.21.15.107:0/2026189339 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9d9c01c630 con 0x7f9da80fbcb0 2024-05-17T21:59:33.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.470+0000 7f9daeebf700 1 -- 172.21.15.107:0/2026189339 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9da810f540 con 0x7f9da80fbcb0 2024-05-17T21:59:33.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.471+0000 7f9daeebf700 1 -- 172.21.15.107:0/2026189339 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9da8101af0 con 0x7f9da80fbcb0 2024-05-17T21:59:33.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.472+0000 7f9da57fa700 1 -- 172.21.15.107:0/2026189339 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f9d9c00bd60 con 0x7f9da80fbcb0 2024-05-17T21:59:33.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.472+0000 7f9da57fa700 1 --2- 172.21.15.107:0/2026189339 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f9d98078990 0x7f9d9807ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:33.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.472+0000 7f9da7fff700 1 --2- 172.21.15.107:0/2026189339 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f9d98078990 0x7f9d9807ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:33.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.472+0000 7f9da57fa700 1 -- 172.21.15.107:0/2026189339 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f9d9c09b320 con 0x7f9da80fbcb0 2024-05-17T21:59:33.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.472+0000 7f9da7fff700 1 --2- 172.21.15.107:0/2026189339 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f9d98078990 0x7f9d9807ae40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9d94004f40 tx=0x7f9d9400b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:33.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.477+0000 7f9da57fa700 1 -- 172.21.15.107:0/2026189339 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f9d9c063010 con 0x7f9da80fbcb0 2024-05-17T21:59:33.477 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:33 smithi138 ceph-mon[81376]: Upgrade: It appears safe to stop mds.cephfs.smithi107.drqody 2024-05-17T21:59:33.477 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:33 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:59:33.477 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:33 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi107.drqody", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T21:59:33.477 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:33 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:59:33.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.697+0000 7f9daeebf700 1 -- 172.21.15.107:0/2026189339 --> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f9da8104910 con 0x7f9d98078990 2024-05-17T21:59:33.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.708+0000 7f9da57fa700 1 -- 172.21.15.107:0/2026189339 <== mgr.34100 v2:172.21.15.107:6800/3205268226 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f9da8104910 con 0x7f9d98078990 2024-05-17T21:59:33.698 INFO:teuthology.orchestra.run.smithi107.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-17T21:59:33.698 INFO:teuthology.orchestra.run.smithi107.stdout:alertmanager.smithi107 smithi107 *:9093,9094 running (7m) 7s ago 15m 24.3M - 0.25.0 c8568f914cd2 bac2a944ae42 2024-05-17T21:59:33.699 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi107 smithi107 running (3m) 7s ago 15m 6669k - 19.0.0-3794-g931809a9 8880f7613d02 face35de205b 2024-05-17T21:59:33.699 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi138 smithi138 running (3m) 26s ago 14m 6665k - 19.0.0-3794-g931809a9 8880f7613d02 ad59e33e6d43 2024-05-17T21:59:33.699 INFO:teuthology.orchestra.run.smithi107.stdout:grafana.smithi107 smithi107 *:3000 running (7m) 7s ago 15m 86.4M - 9.4.7 954c08fa6188 8c747e777bc5 2024-05-17T21:59:33.699 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.drqody smithi107 running (10m) 7s ago 10m 10.6M - 17.2.7-952-g9903f776 d15a36853796 0d851bc912f0 2024-05-17T21:59:33.699 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.idmbgj smithi107 running (12s) 7s ago 10m 82.5M - 19.0.0-3794-g931809a9 8880f7613d02 d541bff38965 2024-05-17T21:59:33.699 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi138.gwbtyd smithi138 running (10m) 26s ago 10m 152M - 17.2.7-952-g9903f776 d15a36853796 fd2ba75354c0 2024-05-17T21:59:33.699 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi138.zckrvs smithi138 running (10m) 26s ago 10m 14.9M - 17.2.7-952-g9903f776 d15a36853796 a1cfd2fe4cda 2024-05-17T21:59:33.699 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi107.xhhyiw smithi107 *:8443,9283,8765 running (6m) 7s ago 17m 591M - 19.0.0-3794-g931809a9 8880f7613d02 5acf3e511342 2024-05-17T21:59:33.699 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi138.napnll smithi138 *:8443,9283,8765 running (5m) 26s ago 14m 533M - 19.0.0-3794-g931809a9 8880f7613d02 d627761dbd0d 2024-05-17T21:59:33.699 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi107 smithi107 running (5m) 7s ago 17m 117M 2048M 19.0.0-3794-g931809a9 8880f7613d02 9717f3169f2f 2024-05-17T21:59:33.699 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi138 smithi138 running (4m) 26s ago 14m 107M 2048M 19.0.0-3794-g931809a9 8880f7613d02 55aeed62bf4d 2024-05-17T21:59:33.699 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi107 smithi107 *:9100 running (7m) 7s ago 15m 17.4M - 1.5.0 0da6a335fe13 49bec71f5487 2024-05-17T21:59:33.699 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi138 smithi138 *:9100 running (6m) 26s ago 14m 15.4M - 1.5.0 0da6a335fe13 e52824376658 2024-05-17T21:59:33.700 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0 smithi107 running (3m) 7s ago 13m 160M 1557M 19.0.0-3794-g931809a9 8880f7613d02 b319bc93500d 2024-05-17T21:59:33.700 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1 smithi107 running (2m) 7s ago 13m 154M 1557M 19.0.0-3794-g931809a9 8880f7613d02 f94b206be6eb 2024-05-17T21:59:33.700 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2 smithi107 running (2m) 7s ago 12m 144M 1557M 19.0.0-3794-g931809a9 8880f7613d02 add7015647af 2024-05-17T21:59:33.700 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3 smithi138 running (97s) 26s ago 12m 162M 2581M 19.0.0-3794-g931809a9 8880f7613d02 416273d327c7 2024-05-17T21:59:33.700 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4 smithi138 running (63s) 26s ago 12m 152M 2581M 19.0.0-3794-g931809a9 8880f7613d02 e33f411da8e6 2024-05-17T21:59:33.700 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5 smithi138 running (29s) 26s ago 11m 80.0M 2581M 19.0.0-3794-g931809a9 8880f7613d02 63810d431961 2024-05-17T21:59:33.700 INFO:teuthology.orchestra.run.smithi107.stdout:prometheus.smithi107 smithi107 *:9095 running (5m) 7s ago 15m 67.2M - 2.43.0 a07b618ecd1d bf5f609bbd6e 2024-05-17T21:59:33.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.710+0000 7f9daeebf700 1 -- 172.21.15.107:0/2026189339 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f9d98078990 msgr2=0x7f9d9807ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:33.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.710+0000 7f9daeebf700 1 --2- 172.21.15.107:0/2026189339 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f9d98078990 0x7f9d9807ae40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9d94004f40 tx=0x7f9d9400b040 comp rx=0 tx=0).stop 2024-05-17T21:59:33.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.710+0000 7f9daeebf700 1 -- 172.21.15.107:0/2026189339 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9da80fbcb0 msgr2=0x7f9da810d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:33.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.710+0000 7f9daeebf700 1 --2- 172.21.15.107:0/2026189339 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9da80fbcb0 0x7f9da810d9f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c00b3b0 tx=0x7f9d9c009e30 comp rx=0 tx=0).stop 2024-05-17T21:59:33.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.710+0000 7f9daeebf700 1 -- 172.21.15.107:0/2026189339 shutdown_connections 2024-05-17T21:59:33.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.710+0000 7f9daeebf700 1 --2- 172.21.15.107:0/2026189339 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f9d98078990 0x7f9d9807ae40 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:33.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.710+0000 7f9daeebf700 1 --2- 172.21.15.107:0/2026189339 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9da80fbcb0 0x7f9da810d9f0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:33.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.710+0000 7f9daeebf700 1 --2- 172.21.15.107:0/2026189339 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9da80fcf60 0x7f9da810df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:33.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.710+0000 7f9daeebf700 1 -- 172.21.15.107:0/2026189339 >> 172.21.15.107:0/2026189339 conn(0x7f9da80f7430 msgr2=0x7f9da8065b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:33.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.710+0000 7f9daeebf700 1 -- 172.21.15.107:0/2026189339 shutdown_connections 2024-05-17T21:59:33.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.710+0000 7f9daeebf700 1 -- 172.21.15.107:0/2026189339 wait complete. 2024-05-17T21:59:33.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.812+0000 7f7b3fde7700 1 -- 172.21.15.107:0/1040032275 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b380fff90 msgr2=0x7f7b38102370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:33.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.812+0000 7f7b3fde7700 1 --2- 172.21.15.107:0/1040032275 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b380fff90 0x7f7b38102370 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f7b28004940 tx=0x7f7b28009e30 comp rx=0 tx=0).stop 2024-05-17T21:59:33.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.813+0000 7f7b3fde7700 1 -- 172.21.15.107:0/1040032275 shutdown_connections 2024-05-17T21:59:33.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.813+0000 7f7b3fde7700 1 --2- 172.21.15.107:0/1040032275 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b380fff90 0x7f7b38102370 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:33.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.813+0000 7f7b3fde7700 1 --2- 172.21.15.107:0/1040032275 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b38066ed0 0x7f7b380ffa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:33.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.813+0000 7f7b3fde7700 1 -- 172.21.15.107:0/1040032275 >> 172.21.15.107:0/1040032275 conn(0x7f7b380f7480 msgr2=0x7f7b380f9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:33.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.813+0000 7f7b3fde7700 1 -- 172.21.15.107:0/1040032275 shutdown_connections 2024-05-17T21:59:33.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.813+0000 7f7b3fde7700 1 -- 172.21.15.107:0/1040032275 wait complete. 2024-05-17T21:59:33.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.813+0000 7f7b3fde7700 1 Processor -- start 2024-05-17T21:59:33.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.813+0000 7f7b3fde7700 1 -- start start 2024-05-17T21:59:33.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.813+0000 7f7b3fde7700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b38066ed0 0x7f7b3810d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:33.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.813+0000 7f7b3fde7700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b380fff90 0x7f7b3810ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:33.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.813+0000 7f7b3fde7700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b3810e3f0 con 0x7f7b38066ed0 2024-05-17T21:59:33.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.813+0000 7f7b3fde7700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b3810e560 con 0x7f7b380fff90 2024-05-17T21:59:33.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.814+0000 7f7b3db83700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b38066ed0 0x7f7b3810d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:33.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.814+0000 7f7b3db83700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b38066ed0 0x7f7b3810d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56766/0 (socket says 172.21.15.107:56766) 2024-05-17T21:59:33.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.814+0000 7f7b3db83700 1 -- 172.21.15.107:0/1574959702 learned_addr learned my addr 172.21.15.107:0/1574959702 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:59:33.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.814+0000 7f7b3d382700 1 --2- 172.21.15.107:0/1574959702 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b380fff90 0x7f7b3810ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:33.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.814+0000 7f7b3db83700 1 -- 172.21.15.107:0/1574959702 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b380fff90 msgr2=0x7f7b3810ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:33.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.814+0000 7f7b3db83700 1 --2- 172.21.15.107:0/1574959702 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b380fff90 0x7f7b3810ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:33.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.814+0000 7f7b3db83700 1 -- 172.21.15.107:0/1574959702 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b28005040 con 0x7f7b38066ed0 2024-05-17T21:59:33.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.814+0000 7f7b3d382700 1 --2- 172.21.15.107:0/1574959702 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b380fff90 0x7f7b3810ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T21:59:33.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.814+0000 7f7b3db83700 1 --2- 172.21.15.107:0/1574959702 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b38066ed0 0x7f7b3810d8b0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f7b34002a80 tx=0x7f7b340046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:33.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.814+0000 7f7b2e7fc700 1 -- 172.21.15.107:0/1574959702 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7b34007cc0 con 0x7f7b38066ed0 2024-05-17T21:59:33.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.814+0000 7f7b2e7fc700 1 -- 172.21.15.107:0/1574959702 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7b3400f040 con 0x7f7b38066ed0 2024-05-17T21:59:33.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.814+0000 7f7b3fde7700 1 -- 172.21.15.107:0/1574959702 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b38112fc0 con 0x7f7b38066ed0 2024-05-17T21:59:33.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.814+0000 7f7b2e7fc700 1 -- 172.21.15.107:0/1574959702 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7b34009630 con 0x7f7b38066ed0 2024-05-17T21:59:33.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.814+0000 7f7b3fde7700 1 -- 172.21.15.107:0/1574959702 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b381134f0 con 0x7f7b38066ed0 2024-05-17T21:59:33.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.815+0000 7f7b3fde7700 1 -- 172.21.15.107:0/1574959702 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b3804c710 con 0x7f7b38066ed0 2024-05-17T21:59:33.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.816+0000 7f7b2e7fc700 1 -- 172.21.15.107:0/1574959702 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f7b3400d690 con 0x7f7b38066ed0 2024-05-17T21:59:33.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.816+0000 7f7b2e7fc700 1 --2- 172.21.15.107:0/1574959702 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f7b24078990 0x7f7b2407ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:33.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.816+0000 7f7b2e7fc700 1 -- 172.21.15.107:0/1574959702 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f7b34099f60 con 0x7f7b38066ed0 2024-05-17T21:59:33.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.816+0000 7f7b3d382700 1 --2- 172.21.15.107:0/1574959702 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f7b24078990 0x7f7b2407ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:33.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.817+0000 7f7b3d382700 1 --2- 172.21.15.107:0/1574959702 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f7b24078990 0x7f7b2407ae40 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7b2800b3b0 tx=0x7f7b28009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:33.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:33.821+0000 7f7b2e7fc700 1 -- 172.21.15.107:0/1574959702 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f7b340cfdf0 con 0x7f7b38066ed0 2024-05-17T21:59:34.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.126+0000 7f7b3fde7700 1 -- 172.21.15.107:0/1574959702 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f7b38113b00 con 0x7f7b38066ed0 2024-05-17T21:59:34.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.127+0000 7f7b2e7fc700 1 -- 172.21.15.107:0/1574959702 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7f7b34061bd0 con 0x7f7b38066ed0 2024-05-17T21:59:34.118 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-05-17T21:59:34.118 INFO:teuthology.orchestra.run.smithi107.stdout: "mon": { 2024-05-17T21:59:34.118 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev)": 2 2024-05-17T21:59:34.118 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-05-17T21:59:34.118 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr": { 2024-05-17T21:59:34.118 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev)": 2 2024-05-17T21:59:34.118 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-05-17T21:59:34.118 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": { 2024-05-17T21:59:34.118 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev)": 6 2024-05-17T21:59:34.118 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-05-17T21:59:34.119 INFO:teuthology.orchestra.run.smithi107.stdout: "mds": { 2024-05-17T21:59:34.119 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-952-g9903f776 (9903f7767386a9d9efadb7c1effdcd3064d71804) quincy (stable)": 3, 2024-05-17T21:59:34.119 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev)": 1 2024-05-17T21:59:34.119 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-05-17T21:59:34.119 INFO:teuthology.orchestra.run.smithi107.stdout: "overall": { 2024-05-17T21:59:34.119 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-952-g9903f776 (9903f7767386a9d9efadb7c1effdcd3064d71804) quincy (stable)": 3, 2024-05-17T21:59:34.119 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev)": 11 2024-05-17T21:59:34.119 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-05-17T21:59:34.119 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-05-17T21:59:34.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.129+0000 7f7b3fde7700 1 -- 172.21.15.107:0/1574959702 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f7b24078990 msgr2=0x7f7b2407ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:34.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.129+0000 7f7b3fde7700 1 --2- 172.21.15.107:0/1574959702 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f7b24078990 0x7f7b2407ae40 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7b2800b3b0 tx=0x7f7b28009e30 comp rx=0 tx=0).stop 2024-05-17T21:59:34.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.129+0000 7f7b3fde7700 1 -- 172.21.15.107:0/1574959702 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b38066ed0 msgr2=0x7f7b3810d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:34.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.129+0000 7f7b3fde7700 1 --2- 172.21.15.107:0/1574959702 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b38066ed0 0x7f7b3810d8b0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f7b34002a80 tx=0x7f7b340046b0 comp rx=0 tx=0).stop 2024-05-17T21:59:34.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.129+0000 7f7b3fde7700 1 -- 172.21.15.107:0/1574959702 shutdown_connections 2024-05-17T21:59:34.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.129+0000 7f7b3fde7700 1 --2- 172.21.15.107:0/1574959702 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f7b24078990 0x7f7b2407ae40 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:34.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.129+0000 7f7b3fde7700 1 --2- 172.21.15.107:0/1574959702 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b38066ed0 0x7f7b3810d8b0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:34.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.129+0000 7f7b3fde7700 1 --2- 172.21.15.107:0/1574959702 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b380fff90 0x7f7b3810ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:34.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.129+0000 7f7b3fde7700 1 -- 172.21.15.107:0/1574959702 >> 172.21.15.107:0/1574959702 conn(0x7f7b380f7480 msgr2=0x7f7b380f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:34.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.129+0000 7f7b3fde7700 1 -- 172.21.15.107:0/1574959702 shutdown_connections 2024-05-17T21:59:34.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.129+0000 7f7b3fde7700 1 -- 172.21.15.107:0/1574959702 wait complete. 2024-05-17T21:59:34.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.234+0000 7f6cbc416700 1 -- 172.21.15.107:0/3159974974 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6cb40fff50 msgr2=0x7f6cb4102330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:34.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.234+0000 7f6cbc416700 1 --2- 172.21.15.107:0/3159974974 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6cb40fff50 0x7f6cb4102330 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6ca4004940 tx=0x7f6ca4009e30 comp rx=0 tx=0).stop 2024-05-17T21:59:34.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.234+0000 7f6cbc416700 1 -- 172.21.15.107:0/3159974974 shutdown_connections 2024-05-17T21:59:34.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.234+0000 7f6cbc416700 1 --2- 172.21.15.107:0/3159974974 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6cb40fff50 0x7f6cb4102330 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:34.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.234+0000 7f6cbc416700 1 --2- 172.21.15.107:0/3159974974 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6cb4066ef0 0x7f6cb40ffa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:34.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.234+0000 7f6cbc416700 1 -- 172.21.15.107:0/3159974974 >> 172.21.15.107:0/3159974974 conn(0x7f6cb40f7440 msgr2=0x7f6cb40f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:34.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.234+0000 7f6cbc416700 1 -- 172.21.15.107:0/3159974974 shutdown_connections 2024-05-17T21:59:34.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.234+0000 7f6cbc416700 1 -- 172.21.15.107:0/3159974974 wait complete. 2024-05-17T21:59:34.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.235+0000 7f6cbc416700 1 Processor -- start 2024-05-17T21:59:34.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.235+0000 7f6cbc416700 1 -- start start 2024-05-17T21:59:34.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.235+0000 7f6cbc416700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6cb4066ef0 0x7f6cb40fd210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:34.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.235+0000 7f6cbc416700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6cb40fff50 0x7f6cb40fb910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:34.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.235+0000 7f6cbc416700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb40fd810 con 0x7f6cb4066ef0 2024-05-17T21:59:34.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.235+0000 7f6cbc416700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb40fbe50 con 0x7f6cb40fff50 2024-05-17T21:59:34.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.235+0000 7f6cba1b2700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6cb4066ef0 0x7f6cb40fd210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:34.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.235+0000 7f6cba1b2700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6cb4066ef0 0x7f6cb40fd210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56780/0 (socket says 172.21.15.107:56780) 2024-05-17T21:59:34.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.235+0000 7f6cba1b2700 1 -- 172.21.15.107:0/2294102008 learned_addr learned my addr 172.21.15.107:0/2294102008 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:59:34.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.235+0000 7f6cba1b2700 1 -- 172.21.15.107:0/2294102008 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6cb40fff50 msgr2=0x7f6cb40fb910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:34.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.235+0000 7f6cb99b1700 1 --2- 172.21.15.107:0/2294102008 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6cb40fff50 0x7f6cb40fb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:34.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.235+0000 7f6cba1b2700 1 --2- 172.21.15.107:0/2294102008 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6cb40fff50 0x7f6cb40fb910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:34.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.235+0000 7f6cba1b2700 1 -- 172.21.15.107:0/2294102008 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ca4005040 con 0x7f6cb4066ef0 2024-05-17T21:59:34.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.235+0000 7f6cb99b1700 1 --2- 172.21.15.107:0/2294102008 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6cb40fff50 0x7f6cb40fb910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:59:34.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.235+0000 7f6cba1b2700 1 --2- 172.21.15.107:0/2294102008 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6cb4066ef0 0x7f6cb40fd210 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6cb0002a80 tx=0x7f6cb0004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:34.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.235+0000 7f6caaffd700 1 -- 172.21.15.107:0/2294102008 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6cb0007cc0 con 0x7f6cb4066ef0 2024-05-17T21:59:34.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.236+0000 7f6caaffd700 1 -- 172.21.15.107:0/2294102008 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6cb000f040 con 0x7f6cb4066ef0 2024-05-17T21:59:34.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.236+0000 7f6cbc416700 1 -- 172.21.15.107:0/2294102008 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cb40fc130 con 0x7f6cb4066ef0 2024-05-17T21:59:34.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.236+0000 7f6caaffd700 1 -- 172.21.15.107:0/2294102008 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6cb000dad0 con 0x7f6cb4066ef0 2024-05-17T21:59:34.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.236+0000 7f6cbc416700 1 -- 172.21.15.107:0/2294102008 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cb40fc670 con 0x7f6cb4066ef0 2024-05-17T21:59:34.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.237+0000 7f6cbc416700 1 -- 172.21.15.107:0/2294102008 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cb404c710 con 0x7f6cb4066ef0 2024-05-17T21:59:34.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.237+0000 7f6caaffd700 1 -- 172.21.15.107:0/2294102008 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f6cb000dc30 con 0x7f6cb4066ef0 2024-05-17T21:59:34.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.238+0000 7f6caaffd700 1 --2- 172.21.15.107:0/2294102008 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f6ca0078990 0x7f6ca007ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:34.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.238+0000 7f6cb99b1700 1 --2- 172.21.15.107:0/2294102008 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f6ca0078990 0x7f6ca007ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:34.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.238+0000 7f6caaffd700 1 -- 172.21.15.107:0/2294102008 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f6cb009a4b0 con 0x7f6cb4066ef0 2024-05-17T21:59:34.230 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.239+0000 7f6cb99b1700 1 --2- 172.21.15.107:0/2294102008 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f6ca0078990 0x7f6ca007ae40 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6ca400b970 tx=0x7f6ca4009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:34.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.243+0000 7f6caaffd700 1 -- 172.21.15.107:0/2294102008 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f6cb0062120 con 0x7f6cb4066ef0 2024-05-17T21:59:34.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:34 smithi107 ceph-mon[93224]: pgmap v184: 65 pgs: 65 active+clean; 258 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-05-17T21:59:34.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:34 smithi107 ceph-mon[93224]: Upgrade: Updating mds.cephfs.smithi107.drqody 2024-05-17T21:59:34.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:34 smithi107 ceph-mon[93224]: Deploying daemon mds.cephfs.smithi107.drqody on smithi107 2024-05-17T21:59:34.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:34 smithi107 ceph-mon[93224]: from='client.34412 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:59:34.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:34 smithi107 ceph-mon[93224]: from='client.? 172.21.15.107:0/1574959702' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:59:34.490 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:34 smithi138 ceph-mon[81376]: pgmap v184: 65 pgs: 65 active+clean; 258 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-05-17T21:59:34.490 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:34 smithi138 ceph-mon[81376]: Upgrade: Updating mds.cephfs.smithi107.drqody 2024-05-17T21:59:34.490 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:34 smithi138 ceph-mon[81376]: Deploying daemon mds.cephfs.smithi107.drqody on smithi107 2024-05-17T21:59:34.490 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:34 smithi138 ceph-mon[81376]: from='client.34412 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:59:34.490 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:34 smithi138 ceph-mon[81376]: from='client.? 172.21.15.107:0/1574959702' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:59:34.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.502+0000 7f6cbc416700 1 -- 172.21.15.107:0/2294102008 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f6cb41058c0 con 0x7f6cb4066ef0 2024-05-17T21:59:34.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.502+0000 7f6caaffd700 1 -- 172.21.15.107:0/2294102008 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 19 v19) v1 ==== 76+0+1890 (secure 0 0 0) 0x7f6cb0061ac0 con 0x7f6cb4066ef0 2024-05-17T21:59:34.493 INFO:teuthology.orchestra.run.smithi107.stdout:e19 2024-05-17T21:59:34.493 INFO:teuthology.orchestra.run.smithi107.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-17T21:59:34.493 INFO:teuthology.orchestra.run.smithi107.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-17T21:59:34.493 INFO:teuthology.orchestra.run.smithi107.stdout:legacy client fscid: 1 2024-05-17T21:59:34.493 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:59:34.493 INFO:teuthology.orchestra.run.smithi107.stdout:Filesystem 'cephfs' (1) 2024-05-17T21:59:34.493 INFO:teuthology.orchestra.run.smithi107.stdout:fs_name cephfs 2024-05-17T21:59:34.494 INFO:teuthology.orchestra.run.smithi107.stdout:epoch 16 2024-05-17T21:59:34.494 INFO:teuthology.orchestra.run.smithi107.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-05-17T21:59:34.494 INFO:teuthology.orchestra.run.smithi107.stdout:created 2024-05-17T21:49:16.836333+0000 2024-05-17T21:59:34.494 INFO:teuthology.orchestra.run.smithi107.stdout:modified 2024-05-17T21:59:13.802503+0000 2024-05-17T21:59:34.494 INFO:teuthology.orchestra.run.smithi107.stdout:tableserver 0 2024-05-17T21:59:34.494 INFO:teuthology.orchestra.run.smithi107.stdout:root 0 2024-05-17T21:59:34.494 INFO:teuthology.orchestra.run.smithi107.stdout:session_timeout 60 2024-05-17T21:59:34.494 INFO:teuthology.orchestra.run.smithi107.stdout:session_autoclose 300 2024-05-17T21:59:34.494 INFO:teuthology.orchestra.run.smithi107.stdout:max_file_size 1099511627776 2024-05-17T21:59:34.494 INFO:teuthology.orchestra.run.smithi107.stdout:max_xattr_size 65536 2024-05-17T21:59:34.494 INFO:teuthology.orchestra.run.smithi107.stdout:required_client_features {} 2024-05-17T21:59:34.494 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure 0 2024-05-17T21:59:34.494 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure_osd_epoch 0 2024-05-17T21:59:34.494 INFO:teuthology.orchestra.run.smithi107.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-17T21:59:34.495 INFO:teuthology.orchestra.run.smithi107.stdout:max_mds 1 2024-05-17T21:59:34.495 INFO:teuthology.orchestra.run.smithi107.stdout:in 0 2024-05-17T21:59:34.495 INFO:teuthology.orchestra.run.smithi107.stdout:up {0=24259} 2024-05-17T21:59:34.495 INFO:teuthology.orchestra.run.smithi107.stdout:failed 2024-05-17T21:59:34.495 INFO:teuthology.orchestra.run.smithi107.stdout:damaged 2024-05-17T21:59:34.495 INFO:teuthology.orchestra.run.smithi107.stdout:stopped 2024-05-17T21:59:34.495 INFO:teuthology.orchestra.run.smithi107.stdout:data_pools [3] 2024-05-17T21:59:34.495 INFO:teuthology.orchestra.run.smithi107.stdout:metadata_pool 2 2024-05-17T21:59:34.495 INFO:teuthology.orchestra.run.smithi107.stdout:inline_data enabled 2024-05-17T21:59:34.495 INFO:teuthology.orchestra.run.smithi107.stdout:balancer 2024-05-17T21:59:34.495 INFO:teuthology.orchestra.run.smithi107.stdout:bal_rank_mask -1 2024-05-17T21:59:34.495 INFO:teuthology.orchestra.run.smithi107.stdout:standby_count_wanted 1 2024-05-17T21:59:34.495 INFO:teuthology.orchestra.run.smithi107.stdout:qdb_cluster leader: 24259 members: 24259 2024-05-17T21:59:34.496 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi138.gwbtyd{0:24259} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.138:6824/2779634919,v1:172.21.15.138:6825/2779634919] compat {c=[1],r=[1],i=[7ff]}] 2024-05-17T21:59:34.496 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:59:34.496 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:59:34.496 INFO:teuthology.orchestra.run.smithi107.stdout:Standby daemons: 2024-05-17T21:59:34.496 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T21:59:34.496 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi138.zckrvs{-1:24267} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.138:6826/170628661,v1:172.21.15.138:6827/170628661] compat {c=[1],r=[1],i=[7ff]}] 2024-05-17T21:59:34.496 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.drqody{-1:34404} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.107:6828/878488679,v1:172.21.15.107:6829/878488679] compat {c=[1],r=[1],i=[7ff]}] 2024-05-17T21:59:34.496 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.idmbgj{-1:34408} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.107:6826/2235359521,v1:172.21.15.107:6827/2235359521] compat {c=[1],r=[1],i=[1fff]}] 2024-05-17T21:59:34.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.505+0000 7f6cbc416700 1 -- 172.21.15.107:0/2294102008 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f6ca0078990 msgr2=0x7f6ca007ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:34.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.505+0000 7f6cbc416700 1 --2- 172.21.15.107:0/2294102008 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f6ca0078990 0x7f6ca007ae40 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6ca400b970 tx=0x7f6ca4009e30 comp rx=0 tx=0).stop 2024-05-17T21:59:34.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.505+0000 7f6cbc416700 1 -- 172.21.15.107:0/2294102008 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6cb4066ef0 msgr2=0x7f6cb40fd210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:34.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.505+0000 7f6cbc416700 1 --2- 172.21.15.107:0/2294102008 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6cb4066ef0 0x7f6cb40fd210 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6cb0002a80 tx=0x7f6cb0004600 comp rx=0 tx=0).stop 2024-05-17T21:59:34.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.505+0000 7f6cbc416700 1 -- 172.21.15.107:0/2294102008 shutdown_connections 2024-05-17T21:59:34.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.505+0000 7f6cbc416700 1 --2- 172.21.15.107:0/2294102008 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f6ca0078990 0x7f6ca007ae40 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:34.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.505+0000 7f6cbc416700 1 --2- 172.21.15.107:0/2294102008 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6cb4066ef0 0x7f6cb40fd210 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:34.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.505+0000 7f6cbc416700 1 --2- 172.21.15.107:0/2294102008 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6cb40fff50 0x7f6cb40fb910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:34.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.505+0000 7f6cbc416700 1 -- 172.21.15.107:0/2294102008 >> 172.21.15.107:0/2294102008 conn(0x7f6cb40f7440 msgr2=0x7f6cb40f7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:34.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.505+0000 7f6cbc416700 1 -- 172.21.15.107:0/2294102008 shutdown_connections 2024-05-17T21:59:34.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.505+0000 7f6cbc416700 1 -- 172.21.15.107:0/2294102008 wait complete. 2024-05-17T21:59:34.498 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 19 2024-05-17T21:59:34.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.616+0000 7f9f672a0700 1 -- 172.21.15.107:0/4078787049 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f600fcf60 msgr2=0x7f9f600fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:34.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.616+0000 7f9f672a0700 1 --2- 172.21.15.107:0/4078787049 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f600fcf60 0x7f9f600fd3d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9f50004940 tx=0x7f9f50009e30 comp rx=0 tx=0).stop 2024-05-17T21:59:34.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.616+0000 7f9f672a0700 1 -- 172.21.15.107:0/4078787049 shutdown_connections 2024-05-17T21:59:34.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.616+0000 7f9f672a0700 1 --2- 172.21.15.107:0/4078787049 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f600fcf60 0x7f9f600fd3d0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:34.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.616+0000 7f9f672a0700 1 --2- 172.21.15.107:0/4078787049 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f600fbcb0 0x7f9f600fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:34.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.616+0000 7f9f672a0700 1 -- 172.21.15.107:0/4078787049 >> 172.21.15.107:0/4078787049 conn(0x7f9f600f7430 msgr2=0x7f9f600f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:34.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.616+0000 7f9f672a0700 1 -- 172.21.15.107:0/4078787049 shutdown_connections 2024-05-17T21:59:34.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.616+0000 7f9f672a0700 1 -- 172.21.15.107:0/4078787049 wait complete. 2024-05-17T21:59:34.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.617+0000 7f9f672a0700 1 Processor -- start 2024-05-17T21:59:34.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.617+0000 7f9f672a0700 1 -- start start 2024-05-17T21:59:34.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.617+0000 7f9f672a0700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f600fbcb0 0x7f9f60110700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:34.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.617+0000 7f9f672a0700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f600fcf60 0x7f9f60112c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:34.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.617+0000 7f9f672a0700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f60066350 con 0x7f9f600fbcb0 2024-05-17T21:59:34.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.617+0000 7f9f672a0700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f60066490 con 0x7f9f600fcf60 2024-05-17T21:59:34.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.617+0000 7f9f6503c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f600fbcb0 0x7f9f60110700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:34.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.617+0000 7f9f6503c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f600fbcb0 0x7f9f60110700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56794/0 (socket says 172.21.15.107:56794) 2024-05-17T21:59:34.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.617+0000 7f9f6503c700 1 -- 172.21.15.107:0/2175218913 learned_addr learned my addr 172.21.15.107:0/2175218913 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:59:34.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.617+0000 7f9f6483b700 1 --2- 172.21.15.107:0/2175218913 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f600fcf60 0x7f9f60112c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:34.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.617+0000 7f9f6503c700 1 -- 172.21.15.107:0/2175218913 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f600fcf60 msgr2=0x7f9f60112c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:34.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.617+0000 7f9f6503c700 1 --2- 172.21.15.107:0/2175218913 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f600fcf60 0x7f9f60112c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:34.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.617+0000 7f9f6503c700 1 -- 172.21.15.107:0/2175218913 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f50005040 con 0x7f9f600fbcb0 2024-05-17T21:59:34.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.618+0000 7f9f6503c700 1 --2- 172.21.15.107:0/2175218913 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f600fbcb0 0x7f9f60110700 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c002a50 tx=0x7f9f5c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:34.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.618+0000 7f9f55ffb700 1 -- 172.21.15.107:0/2175218913 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f5c007cc0 con 0x7f9f600fbcb0 2024-05-17T21:59:34.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.618+0000 7f9f672a0700 1 -- 172.21.15.107:0/2175218913 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f60113250 con 0x7f9f600fbcb0 2024-05-17T21:59:34.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.618+0000 7f9f55ffb700 1 -- 172.21.15.107:0/2175218913 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9f5c00f040 con 0x7f9f600fbcb0 2024-05-17T21:59:34.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.618+0000 7f9f55ffb700 1 -- 172.21.15.107:0/2175218913 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f5c00da30 con 0x7f9f600fbcb0 2024-05-17T21:59:34.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.618+0000 7f9f672a0700 1 -- 172.21.15.107:0/2175218913 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f601137e0 con 0x7f9f600fbcb0 2024-05-17T21:59:34.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.619+0000 7f9f672a0700 1 -- 172.21.15.107:0/2175218913 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f6005f5c0 con 0x7f9f600fbcb0 2024-05-17T21:59:34.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.620+0000 7f9f55ffb700 1 -- 172.21.15.107:0/2175218913 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f9f5c00db90 con 0x7f9f600fbcb0 2024-05-17T21:59:34.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.620+0000 7f9f55ffb700 1 --2- 172.21.15.107:0/2175218913 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f9f4c078990 0x7f9f4c07ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:34.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.620+0000 7f9f55ffb700 1 -- 172.21.15.107:0/2175218913 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f9f5c09a480 con 0x7f9f600fbcb0 2024-05-17T21:59:34.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.620+0000 7f9f6483b700 1 --2- 172.21.15.107:0/2175218913 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f9f4c078990 0x7f9f4c07ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:34.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.621+0000 7f9f6483b700 1 --2- 172.21.15.107:0/2175218913 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f9f4c078990 0x7f9f4c07ae40 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9f5000b970 tx=0x7f9f50009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:34.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.629+0000 7f9f55ffb700 1 -- 172.21.15.107:0/2175218913 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f9f5c0620f0 con 0x7f9f600fbcb0 2024-05-17T21:59:34.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.874+0000 7f9f672a0700 1 -- 172.21.15.107:0/2175218913 --> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9f601012c0 con 0x7f9f4c078990 2024-05-17T21:59:34.866 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-05-17T21:59:34.866 INFO:teuthology.orchestra.run.smithi107.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b", 2024-05-17T21:59:34.866 INFO:teuthology.orchestra.run.smithi107.stdout: "in_progress": true, 2024-05-17T21:59:34.866 INFO:teuthology.orchestra.run.smithi107.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-17T21:59:34.866 INFO:teuthology.orchestra.run.smithi107.stdout: "services_complete": [ 2024-05-17T21:59:34.866 INFO:teuthology.orchestra.run.smithi107.stdout: "crash", 2024-05-17T21:59:34.866 INFO:teuthology.orchestra.run.smithi107.stdout: "osd", 2024-05-17T21:59:34.866 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr", 2024-05-17T21:59:34.867 INFO:teuthology.orchestra.run.smithi107.stdout: "mon" 2024-05-17T21:59:34.867 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-05-17T21:59:34.867 INFO:teuthology.orchestra.run.smithi107.stdout: "progress": "13/21 daemons upgraded", 2024-05-17T21:59:34.867 INFO:teuthology.orchestra.run.smithi107.stdout: "message": "Currently upgrading mds daemons", 2024-05-17T21:59:34.867 INFO:teuthology.orchestra.run.smithi107.stdout: "is_paused": false 2024-05-17T21:59:34.867 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-05-17T21:59:34.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.875+0000 7f9f55ffb700 1 -- 172.21.15.107:0/2175218913 <== mgr.34100 v2:172.21.15.107:6800/3205268226 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f9f601012c0 con 0x7f9f4c078990 2024-05-17T21:59:34.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.878+0000 7f9f672a0700 1 -- 172.21.15.107:0/2175218913 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f9f4c078990 msgr2=0x7f9f4c07ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:34.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.878+0000 7f9f672a0700 1 --2- 172.21.15.107:0/2175218913 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f9f4c078990 0x7f9f4c07ae40 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9f5000b970 tx=0x7f9f50009e30 comp rx=0 tx=0).stop 2024-05-17T21:59:34.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.878+0000 7f9f672a0700 1 -- 172.21.15.107:0/2175218913 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f600fbcb0 msgr2=0x7f9f60110700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:34.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.878+0000 7f9f672a0700 1 --2- 172.21.15.107:0/2175218913 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f600fbcb0 0x7f9f60110700 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c002a50 tx=0x7f9f5c0046b0 comp rx=0 tx=0).stop 2024-05-17T21:59:34.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.878+0000 7f9f672a0700 1 -- 172.21.15.107:0/2175218913 shutdown_connections 2024-05-17T21:59:34.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.878+0000 7f9f672a0700 1 --2- 172.21.15.107:0/2175218913 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f9f4c078990 0x7f9f4c07ae40 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:34.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.878+0000 7f9f672a0700 1 --2- 172.21.15.107:0/2175218913 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f600fbcb0 0x7f9f60110700 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:34.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.878+0000 7f9f672a0700 1 --2- 172.21.15.107:0/2175218913 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f600fcf60 0x7f9f60112c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:34.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.878+0000 7f9f672a0700 1 -- 172.21.15.107:0/2175218913 >> 172.21.15.107:0/2175218913 conn(0x7f9f600f7430 msgr2=0x7f9f600f8b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:34.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.878+0000 7f9f672a0700 1 -- 172.21.15.107:0/2175218913 shutdown_connections 2024-05-17T21:59:34.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.878+0000 7f9f672a0700 1 -- 172.21.15.107:0/2175218913 wait complete. 2024-05-17T21:59:34.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.983+0000 7f4a577a8700 1 -- 172.21.15.107:0/4021134805 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4a500fff70 msgr2=0x7f4a50102350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:34.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.983+0000 7f4a577a8700 1 --2- 172.21.15.107:0/4021134805 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4a500fff70 0x7f4a50102350 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4a40004940 tx=0x7f4a40009e30 comp rx=0 tx=0).stop 2024-05-17T21:59:34.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.983+0000 7f4a577a8700 1 -- 172.21.15.107:0/4021134805 shutdown_connections 2024-05-17T21:59:34.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.983+0000 7f4a577a8700 1 --2- 172.21.15.107:0/4021134805 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4a500fff70 0x7f4a50102350 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:34.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.983+0000 7f4a577a8700 1 --2- 172.21.15.107:0/4021134805 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a50066af0 0x7f4a500ffa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:34.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.983+0000 7f4a577a8700 1 -- 172.21.15.107:0/4021134805 >> 172.21.15.107:0/4021134805 conn(0x7f4a500f7440 msgr2=0x7f4a500f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:34.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.983+0000 7f4a577a8700 1 -- 172.21.15.107:0/4021134805 shutdown_connections 2024-05-17T21:59:34.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.983+0000 7f4a577a8700 1 -- 172.21.15.107:0/4021134805 wait complete. 2024-05-17T21:59:34.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.983+0000 7f4a577a8700 1 Processor -- start 2024-05-17T21:59:34.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.984+0000 7f4a577a8700 1 -- start start 2024-05-17T21:59:34.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.984+0000 7f4a577a8700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4a50066af0 0x7f4a5010da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:34.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.984+0000 7f4a577a8700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a500fff70 0x7f4a5010df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:34.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.984+0000 7f4a577a8700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a5010e580 con 0x7f4a50066af0 2024-05-17T21:59:34.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.984+0000 7f4a577a8700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a5010edd0 con 0x7f4a500fff70 2024-05-17T21:59:34.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.984+0000 7f4a55544700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4a50066af0 0x7f4a5010da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:34.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.984+0000 7f4a55544700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4a50066af0 0x7f4a5010da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56802/0 (socket says 172.21.15.107:56802) 2024-05-17T21:59:34.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.984+0000 7f4a55544700 1 -- 172.21.15.107:0/1786710598 learned_addr learned my addr 172.21.15.107:0/1786710598 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T21:59:34.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.984+0000 7f4a54d43700 1 --2- 172.21.15.107:0/1786710598 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a500fff70 0x7f4a5010df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:34.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.984+0000 7f4a55544700 1 -- 172.21.15.107:0/1786710598 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a500fff70 msgr2=0x7f4a5010df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:34.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.984+0000 7f4a55544700 1 --2- 172.21.15.107:0/1786710598 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a500fff70 0x7f4a5010df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:34.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.984+0000 7f4a55544700 1 -- 172.21.15.107:0/1786710598 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a40005040 con 0x7f4a50066af0 2024-05-17T21:59:34.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.984+0000 7f4a55544700 1 --2- 172.21.15.107:0/1786710598 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4a50066af0 0x7f4a5010da40 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c001ce0 tx=0x7f4a4c009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:34.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.984+0000 7f4a45ffb700 1 -- 172.21.15.107:0/1786710598 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4a4c007b30 con 0x7f4a50066af0 2024-05-17T21:59:34.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.984+0000 7f4a577a8700 1 -- 172.21.15.107:0/1786710598 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a5010f0b0 con 0x7f4a50066af0 2024-05-17T21:59:34.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.984+0000 7f4a45ffb700 1 -- 172.21.15.107:0/1786710598 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4a4c00f040 con 0x7f4a50066af0 2024-05-17T21:59:34.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.985+0000 7f4a45ffb700 1 -- 172.21.15.107:0/1786710598 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4a4c0096d0 con 0x7f4a50066af0 2024-05-17T21:59:34.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.985+0000 7f4a577a8700 1 -- 172.21.15.107:0/1786710598 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a5010f5f0 con 0x7f4a50066af0 2024-05-17T21:59:34.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.986+0000 7f4a577a8700 1 -- 172.21.15.107:0/1786710598 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a5005f5c0 con 0x7f4a50066af0 2024-05-17T21:59:34.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.986+0000 7f4a45ffb700 1 -- 172.21.15.107:0/1786710598 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f4a4c007c90 con 0x7f4a50066af0 2024-05-17T21:59:34.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.987+0000 7f4a45ffb700 1 --2- 172.21.15.107:0/1786710598 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f4a3c078990 0x7f4a3c07ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:34.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.987+0000 7f4a45ffb700 1 -- 172.21.15.107:0/1786710598 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f4a4c09a150 con 0x7f4a50066af0 2024-05-17T21:59:34.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.987+0000 7f4a54d43700 1 --2- 172.21.15.107:0/1786710598 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f4a3c078990 0x7f4a3c07ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:34.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.988+0000 7f4a54d43700 1 --2- 172.21.15.107:0/1786710598 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f4a3c078990 0x7f4a3c07ae40 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4a4000b970 tx=0x7f4a40009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:34.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:34.992+0000 7f4a45ffb700 1 -- 172.21.15.107:0/1786710598 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f4a4c061e10 con 0x7f4a50066af0 2024-05-17T21:59:35.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:35.285+0000 7f4a577a8700 1 -- 172.21.15.107:0/1786710598 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f4a5004bed0 con 0x7f4a50066af0 2024-05-17T21:59:35.276 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:35 smithi107 ceph-mon[93224]: from='client.34416 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:59:35.276 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:35 smithi107 ceph-mon[93224]: from='client.34420 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:59:35.276 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:35 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:59:35.277 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:35 smithi107 ceph-mon[93224]: from='client.? 172.21.15.107:0/2294102008' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-17T21:59:35.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:35.286+0000 7f4a45ffb700 1 -- 172.21.15.107:0/1786710598 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f4a4c0617b0 con 0x7f4a50066af0 2024-05-17T21:59:35.277 INFO:teuthology.orchestra.run.smithi107.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-17T21:59:35.277 INFO:teuthology.orchestra.run.smithi107.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-17T21:59:35.277 INFO:teuthology.orchestra.run.smithi107.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-17T21:59:35.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:35.288+0000 7f4a577a8700 1 -- 172.21.15.107:0/1786710598 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f4a3c078990 msgr2=0x7f4a3c07ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:35.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:35.288+0000 7f4a577a8700 1 --2- 172.21.15.107:0/1786710598 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f4a3c078990 0x7f4a3c07ae40 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4a4000b970 tx=0x7f4a40009e30 comp rx=0 tx=0).stop 2024-05-17T21:59:35.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:35.288+0000 7f4a577a8700 1 -- 172.21.15.107:0/1786710598 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4a50066af0 msgr2=0x7f4a5010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:35.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:35.288+0000 7f4a577a8700 1 --2- 172.21.15.107:0/1786710598 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4a50066af0 0x7f4a5010da40 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c001ce0 tx=0x7f4a4c009e60 comp rx=0 tx=0).stop 2024-05-17T21:59:35.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:35.288+0000 7f4a577a8700 1 -- 172.21.15.107:0/1786710598 shutdown_connections 2024-05-17T21:59:35.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:35.288+0000 7f4a577a8700 1 --2- 172.21.15.107:0/1786710598 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f4a3c078990 0x7f4a3c07ae40 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:35.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:35.288+0000 7f4a577a8700 1 --2- 172.21.15.107:0/1786710598 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4a50066af0 0x7f4a5010da40 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:35.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:35.288+0000 7f4a577a8700 1 --2- 172.21.15.107:0/1786710598 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a500fff70 0x7f4a5010df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:35.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:35.288+0000 7f4a577a8700 1 -- 172.21.15.107:0/1786710598 >> 172.21.15.107:0/1786710598 conn(0x7f4a500f7440 msgr2=0x7f4a500f7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:35.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:35.288+0000 7f4a577a8700 1 -- 172.21.15.107:0/1786710598 shutdown_connections 2024-05-17T21:59:35.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T21:59:35.288+0000 7f4a577a8700 1 -- 172.21.15.107:0/1786710598 wait complete. 2024-05-17T21:59:35.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:35 smithi138 ceph-mon[81376]: from='client.34416 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:59:35.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:35 smithi138 ceph-mon[81376]: from='client.34420 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:59:35.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:35 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:59:35.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:35 smithi138 ceph-mon[81376]: from='client.? 172.21.15.107:0/2294102008' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-17T21:59:36.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:36 smithi138 ceph-mon[81376]: pgmap v185: 65 pgs: 65 active+clean; 258 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-05-17T21:59:36.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:36 smithi138 ceph-mon[81376]: from='client.34432 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:59:36.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:36 smithi138 ceph-mon[81376]: osdmap e82: 6 total, 6 up, 6 in 2024-05-17T21:59:36.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:36 smithi138 ceph-mon[81376]: fsmap cephfs:1 {0=cephfs.smithi138.gwbtyd=up:active} 2 up:standby 2024-05-17T21:59:36.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:36 smithi138 ceph-mon[81376]: from='client.? 172.21.15.107:0/1786710598' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-17T21:59:36.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:36 smithi107 ceph-mon[93224]: pgmap v185: 65 pgs: 65 active+clean; 258 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-05-17T21:59:36.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:36 smithi107 ceph-mon[93224]: from='client.34432 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T21:59:36.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:36 smithi107 ceph-mon[93224]: osdmap e82: 6 total, 6 up, 6 in 2024-05-17T21:59:36.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:36 smithi107 ceph-mon[93224]: fsmap cephfs:1 {0=cephfs.smithi138.gwbtyd=up:active} 2 up:standby 2024-05-17T21:59:36.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:36 smithi107 ceph-mon[93224]: from='client.? 172.21.15.107:0/1786710598' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-17T21:59:38.426 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:38 smithi107 ceph-mon[93224]: pgmap v187: 65 pgs: 65 active+clean; 258 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-05-17T21:59:38.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:38 smithi138 ceph-mon[81376]: pgmap v187: 65 pgs: 65 active+clean; 258 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-05-17T21:59:40.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:40 smithi138 ceph-mon[81376]: pgmap v188: 65 pgs: 65 active+clean; 258 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-05-17T21:59:40.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:40 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:59:40.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:40 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:59:40.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:40 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:59:40.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:40 smithi107 ceph-mon[93224]: pgmap v188: 65 pgs: 65 active+clean; 258 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-05-17T21:59:40.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:40 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:59:40.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:40 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:59:40.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:40 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:59:41.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:41 smithi107 ceph-mon[93224]: mds.? [v2:172.21.15.107:6828/369507752,v1:172.21.15.107:6829/369507752] up:boot 2024-05-17T21:59:41.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:41 smithi107 ceph-mon[93224]: fsmap cephfs:1 {0=cephfs.smithi138.gwbtyd=up:active} 3 up:standby 2024-05-17T21:59:41.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:41 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi107.drqody"}]: dispatch 2024-05-17T21:59:41.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:41 smithi138 ceph-mon[81376]: mds.? [v2:172.21.15.107:6828/369507752,v1:172.21.15.107:6829/369507752] up:boot 2024-05-17T21:59:41.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:41 smithi138 ceph-mon[81376]: fsmap cephfs:1 {0=cephfs.smithi138.gwbtyd=up:active} 3 up:standby 2024-05-17T21:59:41.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:41 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi107.drqody"}]: dispatch 2024-05-17T21:59:42.440 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:42 smithi107 ceph-mon[93224]: pgmap v189: 65 pgs: 65 active+clean; 258 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-05-17T21:59:42.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:42 smithi138 ceph-mon[81376]: pgmap v189: 65 pgs: 65 active+clean; 258 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-05-17T21:59:44.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:44 smithi138 ceph-mon[81376]: pgmap v190: 65 pgs: 65 active+clean; 258 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-05-17T21:59:44.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:44 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:59:44.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:44 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:59:44.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:44 smithi107 ceph-mon[93224]: pgmap v190: 65 pgs: 65 active+clean; 258 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-05-17T21:59:44.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:44 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:59:44.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:44 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:59:46.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:45 smithi138 ceph-mon[81376]: pgmap v191: 65 pgs: 65 active+clean; 258 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-05-17T21:59:46.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:45 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:59:46.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:45 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:59:46.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:45 smithi107 ceph-mon[93224]: pgmap v191: 65 pgs: 65 active+clean; 258 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-05-17T21:59:46.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:45 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:59:46.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:45 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:59:48.246 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:47 smithi107 ceph-mon[93224]: pgmap v192: 65 pgs: 65 active+clean; 258 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-05-17T21:59:48.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:47 smithi138 ceph-mon[81376]: pgmap v192: 65 pgs: 65 active+clean; 258 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-05-17T21:59:49.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:49 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:59:49.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:49 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:59:49.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:49 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:59:49.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:49 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T21:59:49.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:49 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:59:49.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:49 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:59:49.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:49 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:59:49.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:49 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:59:49.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:49 smithi138 ceph-mon[81376]: pgmap v193: 65 pgs: 65 active+clean; 258 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-05-17T21:59:49.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:49 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:59:49.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:49 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:59:49.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:49 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi138.gwbtyd"]}]: dispatch 2024-05-17T21:59:49.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:49 smithi138 ceph-mon[81376]: Upgrade: It appears safe to stop mds.cephfs.smithi138.gwbtyd 2024-05-17T21:59:49.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:49 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:59:49.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:49 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:59:49.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:49 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:59:49.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:49 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T21:59:49.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:49 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:59:49.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:49 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T21:59:49.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:49 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:59:49.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:49 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:59:49.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:49 smithi107 ceph-mon[93224]: pgmap v193: 65 pgs: 65 active+clean; 258 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-05-17T21:59:49.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:49 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:59:49.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:49 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T21:59:49.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:49 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi138.gwbtyd"]}]: dispatch 2024-05-17T21:59:49.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:49 smithi107 ceph-mon[93224]: Upgrade: It appears safe to stop mds.cephfs.smithi138.gwbtyd 2024-05-17T21:59:50.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:50 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:59:50.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:50 smithi138 ceph-mon[81376]: Upgrade: Updating mds.cephfs.smithi138.gwbtyd 2024-05-17T21:59:50.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:50 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:59:50.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:50 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi138.gwbtyd", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T21:59:50.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:50 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:59:50.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:50 smithi138 ceph-mon[81376]: Deploying daemon mds.cephfs.smithi138.gwbtyd on smithi138 2024-05-17T21:59:50.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:50 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T21:59:50.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:50 smithi107 ceph-mon[93224]: Upgrade: Updating mds.cephfs.smithi138.gwbtyd 2024-05-17T21:59:50.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:50 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T21:59:50.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:50 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi138.gwbtyd", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T21:59:50.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:50 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T21:59:50.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:50 smithi107 ceph-mon[93224]: Deploying daemon mds.cephfs.smithi138.gwbtyd on smithi138 2024-05-17T21:59:51.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:51 smithi138 ceph-mon[81376]: pgmap v194: 65 pgs: 65 active+clean; 258 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-05-17T21:59:51.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:51 smithi107 ceph-mon[93224]: pgmap v194: 65 pgs: 65 active+clean; 258 MiB data, 985 MiB used, 535 GiB / 536 GiB avail 2024-05-17T21:59:52.578 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:52 smithi138 ceph-mon[81376]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-17T21:59:52.578 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:52 smithi138 ceph-mon[81376]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-17T21:59:52.578 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:52 smithi138 ceph-mon[81376]: osdmap e83: 6 total, 6 up, 6 in 2024-05-17T21:59:52.578 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:52 smithi138 ceph-mon[81376]: Standby daemon mds.cephfs.smithi138.zckrvs assigned to filesystem cephfs as rank 0 2024-05-17T21:59:52.578 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:52 smithi138 ceph-mon[81376]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-17T21:59:52.578 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:52 smithi138 ceph-mon[81376]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-05-17T21:59:52.578 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:52 smithi138 ceph-mon[81376]: fsmap cephfs:1/1 {0=cephfs.smithi138.zckrvs=up:replay} 2 up:standby 2024-05-17T21:59:52.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:52 smithi107 ceph-3722731a-1496-11ef-bc9a-c7b262605968-mon-smithi107[93220]: 2024-05-17T21:59:52.170+0000 7f7d6bd88700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-17T21:59:52.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:52 smithi107 ceph-mon[93224]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-17T21:59:52.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:52 smithi107 ceph-mon[93224]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-17T21:59:52.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:52 smithi107 ceph-mon[93224]: osdmap e83: 6 total, 6 up, 6 in 2024-05-17T21:59:52.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:52 smithi107 ceph-mon[93224]: Standby daemon mds.cephfs.smithi138.zckrvs assigned to filesystem cephfs as rank 0 2024-05-17T21:59:52.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:52 smithi107 ceph-mon[93224]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-17T21:59:52.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:52 smithi107 ceph-mon[93224]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-05-17T21:59:52.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:52 smithi107 ceph-mon[93224]: fsmap cephfs:1/1 {0=cephfs.smithi138.zckrvs=up:replay} 2 up:standby 2024-05-17T21:59:53.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:53 smithi138 ceph-mon[81376]: pgmap v196: 65 pgs: 65 active+clean; 258 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 0 op/s 2024-05-17T21:59:53.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:53 smithi107 ceph-mon[93224]: pgmap v196: 65 pgs: 65 active+clean; 258 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 0 op/s 2024-05-17T21:59:55.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:55 smithi138 ceph-mon[81376]: pgmap v197: 65 pgs: 65 active+clean; 258 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 0 op/s 2024-05-17T21:59:55.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:55 smithi107 ceph-mon[93224]: pgmap v197: 65 pgs: 65 active+clean; 258 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 0 op/s 2024-05-17T21:59:57.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:57 smithi138 ceph-mon[81376]: pgmap v198: 65 pgs: 65 active+clean; 258 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 11 MiB/s rd, 3 op/s 2024-05-17T21:59:57.850 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:57 smithi107 ceph-mon[93224]: pgmap v198: 65 pgs: 65 active+clean; 258 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 11 MiB/s rd, 3 op/s 2024-05-17T21:59:59.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 21:59:59 smithi138 ceph-mon[81376]: pgmap v199: 65 pgs: 65 active+clean; 258 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 17 MiB/s rd, 5 op/s 2024-05-17T21:59:59.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 21:59:59 smithi107 ceph-mon[93224]: pgmap v199: 65 pgs: 65 active+clean; 258 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 17 MiB/s rd, 5 op/s 2024-05-17T22:00:00.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:00 smithi138 ceph-mon[81376]: Health detail: HEALTH_WARN 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data 2024-05-17T22:00:00.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:00 smithi138 ceph-mon[81376]: [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-05-17T22:00:00.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:00 smithi138 ceph-mon[81376]: fs cephfs is degraded 2024-05-17T22:00:00.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:00 smithi138 ceph-mon[81376]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-17T22:00:00.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:00 smithi138 ceph-mon[81376]: fs cephfs has deprecated feature inline_data enabled. 2024-05-17T22:00:00.850 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:00 smithi107 ceph-mon[93224]: Health detail: HEALTH_WARN 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data 2024-05-17T22:00:00.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:00 smithi107 ceph-mon[93224]: [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-05-17T22:00:00.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:00 smithi107 ceph-mon[93224]: fs cephfs is degraded 2024-05-17T22:00:00.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:00 smithi107 ceph-mon[93224]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-17T22:00:00.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:00 smithi107 ceph-mon[93224]: fs cephfs has deprecated feature inline_data enabled. 2024-05-17T22:00:01.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:01 smithi138 ceph-mon[81376]: pgmap v200: 65 pgs: 65 active+clean; 258 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 17 MiB/s rd, 5 op/s 2024-05-17T22:00:01.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:01 smithi107 ceph-mon[93224]: pgmap v200: 65 pgs: 65 active+clean; 258 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 17 MiB/s rd, 5 op/s 2024-05-17T22:00:02.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:02 smithi107 ceph-mon[93224]: reconnect by client.24295 192.168.0.1:0/262157930 after 0.000999978 2024-05-17T22:00:02.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:02 smithi107 ceph-mon[93224]: reconnect by client.14522 192.168.0.1:0/3969818366 after 0.000999978 2024-05-17T22:00:02.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:02 smithi107 ceph-mon[93224]: mds.? [v2:172.21.15.138:6826/170628661,v1:172.21.15.138:6827/170628661] up:reconnect 2024-05-17T22:00:02.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:02 smithi107 ceph-mon[93224]: fsmap cephfs:1/1 {0=cephfs.smithi138.zckrvs=up:reconnect} 2 up:standby 2024-05-17T22:00:02.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:02 smithi138 ceph-mon[81376]: reconnect by client.24295 192.168.0.1:0/262157930 after 0.000999978 2024-05-17T22:00:02.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:02 smithi138 ceph-mon[81376]: reconnect by client.14522 192.168.0.1:0/3969818366 after 0.000999978 2024-05-17T22:00:02.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:02 smithi138 ceph-mon[81376]: mds.? [v2:172.21.15.138:6826/170628661,v1:172.21.15.138:6827/170628661] up:reconnect 2024-05-17T22:00:02.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:02 smithi138 ceph-mon[81376]: fsmap cephfs:1/1 {0=cephfs.smithi138.zckrvs=up:reconnect} 2 up:standby 2024-05-17T22:00:03.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:03 smithi138 ceph-mon[81376]: pgmap v201: 65 pgs: 65 active+clean; 258 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 25 MiB/s rd, 7 op/s 2024-05-17T22:00:03.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:03 smithi138 ceph-mon[81376]: mds.? [v2:172.21.15.138:6826/170628661,v1:172.21.15.138:6827/170628661] up:rejoin 2024-05-17T22:00:03.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:03 smithi138 ceph-mon[81376]: fsmap cephfs:1/1 {0=cephfs.smithi138.zckrvs=up:rejoin} 2 up:standby 2024-05-17T22:00:03.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:03 smithi138 ceph-mon[81376]: daemon mds.cephfs.smithi138.zckrvs is now active in filesystem cephfs as rank 0 2024-05-17T22:00:03.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:03 smithi107 ceph-mon[93224]: pgmap v201: 65 pgs: 65 active+clean; 258 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 25 MiB/s rd, 7 op/s 2024-05-17T22:00:03.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:03 smithi107 ceph-mon[93224]: mds.? [v2:172.21.15.138:6826/170628661,v1:172.21.15.138:6827/170628661] up:rejoin 2024-05-17T22:00:03.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:03 smithi107 ceph-mon[93224]: fsmap cephfs:1/1 {0=cephfs.smithi138.zckrvs=up:rejoin} 2 up:standby 2024-05-17T22:00:03.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:03 smithi107 ceph-mon[93224]: daemon mds.cephfs.smithi138.zckrvs is now active in filesystem cephfs as rank 0 2024-05-17T22:00:04.566 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:04 smithi138 ceph-mon[81376]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-17T22:00:04.566 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:04 smithi138 ceph-mon[81376]: mds.? [v2:172.21.15.138:6826/170628661,v1:172.21.15.138:6827/170628661] up:active 2024-05-17T22:00:04.566 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:04 smithi138 ceph-mon[81376]: fsmap cephfs:1 {0=cephfs.smithi138.zckrvs=up:active} 2 up:standby 2024-05-17T22:00:04.566 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:04 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T22:00:04.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:04 smithi107 ceph-mon[93224]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-17T22:00:04.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:04 smithi107 ceph-mon[93224]: mds.? [v2:172.21.15.138:6826/170628661,v1:172.21.15.138:6827/170628661] up:active 2024-05-17T22:00:04.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:04 smithi107 ceph-mon[93224]: fsmap cephfs:1 {0=cephfs.smithi138.zckrvs=up:active} 2 up:standby 2024-05-17T22:00:04.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:04 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T22:00:05.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:05.405+0000 7faa78c56700 1 -- 172.21.15.107:0/1072751002 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa740fce60 msgr2=0x7faa740fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:05.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:05.405+0000 7faa78c56700 1 --2- 172.21.15.107:0/1072751002 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa740fce60 0x7faa740fd270 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7faa6c009ce0 tx=0x7faa6c004d10 comp rx=0 tx=0).stop 2024-05-17T22:00:05.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:05.405+0000 7faa78c56700 1 -- 172.21.15.107:0/1072751002 shutdown_connections 2024-05-17T22:00:05.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:05.405+0000 7faa78c56700 1 --2- 172.21.15.107:0/1072751002 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa740fdfd0 0x7faa740fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:05.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:05.405+0000 7faa78c56700 1 --2- 172.21.15.107:0/1072751002 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa740fce60 0x7faa740fd270 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:05.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:05.405+0000 7faa78c56700 1 -- 172.21.15.107:0/1072751002 >> 172.21.15.107:0/1072751002 conn(0x7faa740f8540 msgr2=0x7faa740fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:05.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:05.405+0000 7faa78c56700 1 -- 172.21.15.107:0/1072751002 shutdown_connections 2024-05-17T22:00:05.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:05.405+0000 7faa78c56700 1 -- 172.21.15.107:0/1072751002 wait complete. 2024-05-17T22:00:05.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:05.405+0000 7faa78c56700 1 Processor -- start 2024-05-17T22:00:05.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:05.405+0000 7faa78c56700 1 -- start start 2024-05-17T22:00:05.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:05.405+0000 7faa78c56700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa740fce60 0x7faa7410d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:05.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:05.405+0000 7faa78c56700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa740fdfd0 0x7faa7410de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:05.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:05.405+0000 7faa78c56700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa7410e480 con 0x7faa740fce60 2024-05-17T22:00:05.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:05.405+0000 7faa78c56700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa7410e5f0 con 0x7faa740fdfd0 2024-05-17T22:00:05.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:05.406+0000 7faa7259c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa740fce60 0x7faa7410d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:05.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:05.406+0000 7faa7259c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa740fce60 0x7faa7410d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51514/0 (socket says 172.21.15.107:51514) 2024-05-17T22:00:05.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:05.406+0000 7faa7259c700 1 -- 172.21.15.107:0/455304074 learned_addr learned my addr 172.21.15.107:0/455304074 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T22:00:05.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:05.406+0000 7faa71d9b700 1 --2- 172.21.15.107:0/455304074 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa740fdfd0 0x7faa7410de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:05.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:05.406+0000 7faa7259c700 1 -- 172.21.15.107:0/455304074 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa740fdfd0 msgr2=0x7faa7410de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:05.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:05.406+0000 7faa7259c700 1 --2- 172.21.15.107:0/455304074 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa740fdfd0 0x7faa7410de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:05.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:05.406+0000 7faa7259c700 1 -- 172.21.15.107:0/455304074 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa6c009940 con 0x7faa740fce60 2024-05-17T22:00:05.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:05.406+0000 7faa71d9b700 1 --2- 172.21.15.107:0/455304074 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa740fdfd0 0x7faa7410de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:00:05.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:05.406+0000 7faa7259c700 1 --2- 172.21.15.107:0/455304074 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa740fce60 0x7faa7410d940 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7faa6c00bfd0 tx=0x7faa6c004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:05.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:05.406+0000 7faa6affd700 1 -- 172.21.15.107:0/455304074 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa6c00e040 con 0x7faa740fce60 2024-05-17T22:00:05.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:05.406+0000 7faa78c56700 1 -- 172.21.15.107:0/455304074 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa74112ff0 con 0x7faa740fce60 2024-05-17T22:00:05.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:05.406+0000 7faa6affd700 1 -- 172.21.15.107:0/455304074 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faa6c01cd00 con 0x7faa740fce60 2024-05-17T22:00:05.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:05.406+0000 7faa6affd700 1 -- 172.21.15.107:0/455304074 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa6c012c40 con 0x7faa740fce60 2024-05-17T22:00:05.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:05.406+0000 7faa78c56700 1 -- 172.21.15.107:0/455304074 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa74113520 con 0x7faa740fce60 2024-05-17T22:00:05.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:05.407+0000 7faa78c56700 1 -- 172.21.15.107:0/455304074 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa7404c710 con 0x7faa740fce60 2024-05-17T22:00:05.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:05.408+0000 7faa6affd700 1 -- 172.21.15.107:0/455304074 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7faa6c0124a0 con 0x7faa740fce60 2024-05-17T22:00:05.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:05.409+0000 7faa6affd700 1 --2- 172.21.15.107:0/455304074 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7faa60078990 0x7faa6007ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:05.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:05.409+0000 7faa6affd700 1 -- 172.21.15.107:0/455304074 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6496+0+0 (secure 0 0 0) 0x7faa6c09ca10 con 0x7faa740fce60 2024-05-17T22:00:05.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:05.409+0000 7faa71d9b700 1 --2- 172.21.15.107:0/455304074 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7faa60078990 0x7faa6007ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:05.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:05.409+0000 7faa71d9b700 1 --2- 172.21.15.107:0/455304074 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7faa60078990 0x7faa6007ae40 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7faa5c009b80 tx=0x7faa5c00b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:05.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:05.413+0000 7faa6affd700 1 -- 172.21.15.107:0/455304074 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7faa6c064630 con 0x7faa740fce60 2024-05-17T22:00:05.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:05.639+0000 7faa78c56700 1 -- 172.21.15.107:0/455304074 --> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7faa74102330 con 0x7faa60078990 2024-05-17T22:00:05.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:05.641+0000 7faa6affd700 1 -- 172.21.15.107:0/455304074 <== mgr.34100 v2:172.21.15.107:6800/3205268226 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7faa74102330 con 0x7faa60078990 2024-05-17T22:00:05.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:05.643+0000 7faa78c56700 1 -- 172.21.15.107:0/455304074 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7faa60078990 msgr2=0x7faa6007ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:05.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:05.643+0000 7faa78c56700 1 --2- 172.21.15.107:0/455304074 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7faa60078990 0x7faa6007ae40 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7faa5c009b80 tx=0x7faa5c00b040 comp rx=0 tx=0).stop 2024-05-17T22:00:05.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:05.643+0000 7faa78c56700 1 -- 172.21.15.107:0/455304074 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa740fce60 msgr2=0x7faa7410d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:05.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:05.643+0000 7faa78c56700 1 --2- 172.21.15.107:0/455304074 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa740fce60 0x7faa7410d940 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7faa6c00bfd0 tx=0x7faa6c004d10 comp rx=0 tx=0).stop 2024-05-17T22:00:05.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:05.643+0000 7faa78c56700 1 -- 172.21.15.107:0/455304074 shutdown_connections 2024-05-17T22:00:05.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:05.643+0000 7faa78c56700 1 --2- 172.21.15.107:0/455304074 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7faa60078990 0x7faa6007ae40 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:05.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:05.643+0000 7faa78c56700 1 --2- 172.21.15.107:0/455304074 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa740fce60 0x7faa7410d940 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:05.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:05.643+0000 7faa78c56700 1 --2- 172.21.15.107:0/455304074 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa740fdfd0 0x7faa7410de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:05.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:05.643+0000 7faa78c56700 1 -- 172.21.15.107:0/455304074 >> 172.21.15.107:0/455304074 conn(0x7faa740f8540 msgr2=0x7faa740f9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:05.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:05.643+0000 7faa78c56700 1 -- 172.21.15.107:0/455304074 shutdown_connections 2024-05-17T22:00:05.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:05.643+0000 7faa78c56700 1 -- 172.21.15.107:0/455304074 wait complete. 2024-05-17T22:00:05.645 INFO:teuthology.orchestra.run.smithi107.stdout:true 2024-05-17T22:00:05.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:05.746+0000 7f3fd6b63700 1 -- 172.21.15.107:0/159869472 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3fd00fed30 msgr2=0x7f3fd00fae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:05.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:05.746+0000 7f3fd6b63700 1 --2- 172.21.15.107:0/159869472 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3fd00fed30 0x7f3fd00fae70 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc004940 tx=0x7f3fbc009e30 comp rx=0 tx=0).stop 2024-05-17T22:00:05.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:05.746+0000 7f3fd6b63700 1 -- 172.21.15.107:0/159869472 shutdown_connections 2024-05-17T22:00:05.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:05.746+0000 7f3fd6b63700 1 --2- 172.21.15.107:0/159869472 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3fd00fed30 0x7f3fd00fae70 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:05.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:05.746+0000 7f3fd6b63700 1 --2- 172.21.15.107:0/159869472 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3fd00fe3e0 0x7f3fd00fe7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:05.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:05.746+0000 7f3fd6b63700 1 -- 172.21.15.107:0/159869472 >> 172.21.15.107:0/159869472 conn(0x7f3fd00f6b80 msgr2=0x7f3fd00f8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:05.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:05.746+0000 7f3fd6b63700 1 -- 172.21.15.107:0/159869472 shutdown_connections 2024-05-17T22:00:05.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:05.746+0000 7f3fd6b63700 1 -- 172.21.15.107:0/159869472 wait complete. 2024-05-17T22:00:05.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:05.747+0000 7f3fd6b63700 1 Processor -- start 2024-05-17T22:00:05.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:05.747+0000 7f3fd6b63700 1 -- start start 2024-05-17T22:00:05.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:05.747+0000 7f3fd6b63700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3fd00fe3e0 0x7f3fd00fdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:05.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:05.747+0000 7f3fd6b63700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3fd00fed30 0x7f3fd00fc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:05.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:05.747+0000 7f3fd6b63700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd00fc650 con 0x7f3fd00fed30 2024-05-17T22:00:05.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:05.747+0000 7f3fd6b63700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd00fc7c0 con 0x7f3fd00fe3e0 2024-05-17T22:00:05.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:05.747+0000 7f3fcffff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3fd00fed30 0x7f3fd00fc0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:05.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:05.747+0000 7f3fcffff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3fd00fed30 0x7f3fd00fc0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39460/0 (socket says 172.21.15.107:39460) 2024-05-17T22:00:05.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:05.747+0000 7f3fcffff700 1 -- 172.21.15.107:0/2933629974 learned_addr learned my addr 172.21.15.107:0/2933629974 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T22:00:05.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:05.748+0000 7f3fd48ff700 1 --2- 172.21.15.107:0/2933629974 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3fd00fe3e0 0x7f3fd00fdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:05.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:05.748+0000 7f3fcffff700 1 -- 172.21.15.107:0/2933629974 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3fd00fe3e0 msgr2=0x7f3fd00fdbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:05.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:05.748+0000 7f3fcffff700 1 --2- 172.21.15.107:0/2933629974 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3fd00fe3e0 0x7f3fd00fdbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:05.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:05.748+0000 7f3fcffff700 1 -- 172.21.15.107:0/2933629974 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fbc005040 con 0x7f3fd00fed30 2024-05-17T22:00:05.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:05.748+0000 7f3fcffff700 1 --2- 172.21.15.107:0/2933629974 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3fd00fed30 0x7f3fd00fc0a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc00b3b0 tx=0x7f3fbc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:05.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:05.748+0000 7f3fcd7fa700 1 -- 172.21.15.107:0/2933629974 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3fbc00e040 con 0x7f3fd00fed30 2024-05-17T22:00:05.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:05.748+0000 7f3fcd7fa700 1 -- 172.21.15.107:0/2933629974 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3fbc00bcc0 con 0x7f3fd00fed30 2024-05-17T22:00:05.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:05.748+0000 7f3fd6b63700 1 -- 172.21.15.107:0/2933629974 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fd00fca40 con 0x7f3fd00fed30 2024-05-17T22:00:05.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:05.748+0000 7f3fcd7fa700 1 -- 172.21.15.107:0/2933629974 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3fbc01c630 con 0x7f3fd00fed30 2024-05-17T22:00:05.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:05.748+0000 7f3fd6b63700 1 -- 172.21.15.107:0/2933629974 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fd01159f0 con 0x7f3fd00fed30 2024-05-17T22:00:05.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:05.749+0000 7f3fd6b63700 1 -- 172.21.15.107:0/2933629974 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fd005f5c0 con 0x7f3fd00fed30 2024-05-17T22:00:05.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:05.750+0000 7f3fcd7fa700 1 -- 172.21.15.107:0/2933629974 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f3fbc013980 con 0x7f3fd00fed30 2024-05-17T22:00:05.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:05.750+0000 7f3fcd7fa700 1 --2- 172.21.15.107:0/2933629974 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f3fc0078990 0x7f3fc007ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:05.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:05.751+0000 7f3fcd7fa700 1 -- 172.21.15.107:0/2933629974 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f3fbc09b490 con 0x7f3fd00fed30 2024-05-17T22:00:05.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:05.751+0000 7f3fd48ff700 1 --2- 172.21.15.107:0/2933629974 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f3fc0078990 0x7f3fc007ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:05.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:05.751+0000 7f3fd48ff700 1 --2- 172.21.15.107:0/2933629974 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f3fc0078990 0x7f3fc007ae40 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3fd00fd960 tx=0x7f3fc4009ee0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:05.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:05.756+0000 7f3fcd7fa700 1 -- 172.21.15.107:0/2933629974 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f3fbc0630b0 con 0x7f3fd00fed30 2024-05-17T22:00:05.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:05.981+0000 7f3fd6b63700 1 -- 172.21.15.107:0/2933629974 --> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3fd0102500 con 0x7f3fc0078990 2024-05-17T22:00:05.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:05.982+0000 7f3fcd7fa700 1 -- 172.21.15.107:0/2933629974 <== mgr.34100 v2:172.21.15.107:6800/3205268226 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f3fd0102500 con 0x7f3fc0078990 2024-05-17T22:00:05.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:05.984+0000 7f3fd6b63700 1 -- 172.21.15.107:0/2933629974 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f3fc0078990 msgr2=0x7f3fc007ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:05.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:05.984+0000 7f3fd6b63700 1 --2- 172.21.15.107:0/2933629974 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f3fc0078990 0x7f3fc007ae40 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3fd00fd960 tx=0x7f3fc4009ee0 comp rx=0 tx=0).stop 2024-05-17T22:00:05.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:05.984+0000 7f3fd6b63700 1 -- 172.21.15.107:0/2933629974 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3fd00fed30 msgr2=0x7f3fd00fc0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:05.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:05.984+0000 7f3fd6b63700 1 --2- 172.21.15.107:0/2933629974 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3fd00fed30 0x7f3fd00fc0a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc00b3b0 tx=0x7f3fbc009e30 comp rx=0 tx=0).stop 2024-05-17T22:00:05.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:05.984+0000 7f3fd6b63700 1 -- 172.21.15.107:0/2933629974 shutdown_connections 2024-05-17T22:00:05.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:05.984+0000 7f3fd6b63700 1 --2- 172.21.15.107:0/2933629974 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f3fc0078990 0x7f3fc007ae40 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:05.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:05.984+0000 7f3fd6b63700 1 --2- 172.21.15.107:0/2933629974 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3fd00fe3e0 0x7f3fd00fdbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:05.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:05.984+0000 7f3fd6b63700 1 --2- 172.21.15.107:0/2933629974 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3fd00fed30 0x7f3fd00fc0a0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:05.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:05.984+0000 7f3fd6b63700 1 -- 172.21.15.107:0/2933629974 >> 172.21.15.107:0/2933629974 conn(0x7f3fd00f6b80 msgr2=0x7f3fd00f8240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:05.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:05.984+0000 7f3fd6b63700 1 -- 172.21.15.107:0/2933629974 shutdown_connections 2024-05-17T22:00:05.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:05.984+0000 7f3fd6b63700 1 -- 172.21.15.107:0/2933629974 wait complete. 2024-05-17T22:00:06.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:05 smithi138 ceph-mon[81376]: pgmap v202: 65 pgs: 65 active+clean; 258 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 20 MiB/s rd, 5 op/s 2024-05-17T22:00:06.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:05 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:00:06.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:05 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:00:06.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:05 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:00:06.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:05 smithi138 ceph-mon[81376]: mds.? [v2:172.21.15.138:6824/1700112281,v1:172.21.15.138:6825/1700112281] up:boot 2024-05-17T22:00:06.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:05 smithi138 ceph-mon[81376]: fsmap cephfs:1 {0=cephfs.smithi138.zckrvs=up:active} 3 up:standby 2024-05-17T22:00:06.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:05 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi138.gwbtyd"}]: dispatch 2024-05-17T22:00:06.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.087+0000 7f9a3e216700 1 -- 172.21.15.107:0/84067363 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9a380fcca0 msgr2=0x7f9a380fd110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:06.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.087+0000 7f9a3e216700 1 --2- 172.21.15.107:0/84067363 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9a380fcca0 0x7f9a380fd110 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9a20004940 tx=0x7f9a20009e30 comp rx=0 tx=0).stop 2024-05-17T22:00:06.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.087+0000 7f9a3e216700 1 -- 172.21.15.107:0/84067363 shutdown_connections 2024-05-17T22:00:06.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.087+0000 7f9a3e216700 1 --2- 172.21.15.107:0/84067363 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9a380fcca0 0x7f9a380fd110 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:06.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.087+0000 7f9a3e216700 1 --2- 172.21.15.107:0/84067363 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a380fc350 0x7f9a380fc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:06.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.087+0000 7f9a3e216700 1 -- 172.21.15.107:0/84067363 >> 172.21.15.107:0/84067363 conn(0x7f9a380f6be0 msgr2=0x7f9a380f8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:06.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.087+0000 7f9a3e216700 1 -- 172.21.15.107:0/84067363 shutdown_connections 2024-05-17T22:00:06.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.087+0000 7f9a3e216700 1 -- 172.21.15.107:0/84067363 wait complete. 2024-05-17T22:00:06.080 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:05 smithi107 ceph-mon[93224]: pgmap v202: 65 pgs: 65 active+clean; 258 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 20 MiB/s rd, 5 op/s 2024-05-17T22:00:06.080 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:05 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:00:06.080 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:05 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:00:06.080 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:05 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:00:06.080 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:05 smithi107 ceph-mon[93224]: mds.? [v2:172.21.15.138:6824/1700112281,v1:172.21.15.138:6825/1700112281] up:boot 2024-05-17T22:00:06.080 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:05 smithi107 ceph-mon[93224]: fsmap cephfs:1 {0=cephfs.smithi138.zckrvs=up:active} 3 up:standby 2024-05-17T22:00:06.080 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:05 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi138.gwbtyd"}]: dispatch 2024-05-17T22:00:06.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.088+0000 7f9a3e216700 1 Processor -- start 2024-05-17T22:00:06.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.088+0000 7f9a3e216700 1 -- start start 2024-05-17T22:00:06.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.088+0000 7f9a3e216700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a380fc350 0x7f9a3810c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:06.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.088+0000 7f9a3e216700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9a380fcca0 0x7f9a3810ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:06.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.088+0000 7f9a3e216700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a3810d2b0 con 0x7f9a380fcca0 2024-05-17T22:00:06.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.088+0000 7f9a3e216700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a3810d420 con 0x7f9a380fc350 2024-05-17T22:00:06.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.088+0000 7f9a377fe700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a380fc350 0x7f9a3810c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:06.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.088+0000 7f9a2ffff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9a380fcca0 0x7f9a3810ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:06.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.088+0000 7f9a377fe700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a380fc350 0x7f9a3810c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.107:47842/0 (socket says 172.21.15.107:47842) 2024-05-17T22:00:06.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.088+0000 7f9a2ffff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9a380fcca0 0x7f9a3810ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39484/0 (socket says 172.21.15.107:39484) 2024-05-17T22:00:06.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.088+0000 7f9a377fe700 1 -- 172.21.15.107:0/2373514715 learned_addr learned my addr 172.21.15.107:0/2373514715 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T22:00:06.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.089+0000 7f9a377fe700 1 -- 172.21.15.107:0/2373514715 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9a380fcca0 msgr2=0x7f9a3810ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:06.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.089+0000 7f9a377fe700 1 --2- 172.21.15.107:0/2373514715 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9a380fcca0 0x7f9a3810ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:06.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.089+0000 7f9a377fe700 1 -- 172.21.15.107:0/2373514715 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a20005040 con 0x7f9a380fc350 2024-05-17T22:00:06.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.089+0000 7f9a2ffff700 1 --2- 172.21.15.107:0/2373514715 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9a380fcca0 0x7f9a3810ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:00:06.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.089+0000 7f9a377fe700 1 --2- 172.21.15.107:0/2373514715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a380fc350 0x7f9a3810c770 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9a2800eb40 tx=0x7f9a2800b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:06.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.089+0000 7f9a34ff9700 1 -- 172.21.15.107:0/2373514715 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9a28007cc0 con 0x7f9a380fc350 2024-05-17T22:00:06.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.089+0000 7f9a34ff9700 1 -- 172.21.15.107:0/2373514715 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9a2800f040 con 0x7f9a380fc350 2024-05-17T22:00:06.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.089+0000 7f9a34ff9700 1 -- 172.21.15.107:0/2373514715 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9a2800ca10 con 0x7f9a380fc350 2024-05-17T22:00:06.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.089+0000 7f9a3e216700 1 -- 172.21.15.107:0/2373514715 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a380fb130 con 0x7f9a380fc350 2024-05-17T22:00:06.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.089+0000 7f9a3e216700 1 -- 172.21.15.107:0/2373514715 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a380fb620 con 0x7f9a380fc350 2024-05-17T22:00:06.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.090+0000 7f9a3e216700 1 -- 172.21.15.107:0/2373514715 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a3804c710 con 0x7f9a380fc350 2024-05-17T22:00:06.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.091+0000 7f9a34ff9700 1 -- 172.21.15.107:0/2373514715 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f9a2800cc80 con 0x7f9a380fc350 2024-05-17T22:00:06.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.091+0000 7f9a34ff9700 1 --2- 172.21.15.107:0/2373514715 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f9a24078990 0x7f9a2407ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:06.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.091+0000 7f9a34ff9700 1 -- 172.21.15.107:0/2373514715 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f9a2809a390 con 0x7f9a380fc350 2024-05-17T22:00:06.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.091+0000 7f9a2ffff700 1 --2- 172.21.15.107:0/2373514715 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f9a24078990 0x7f9a2407ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:06.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.092+0000 7f9a2ffff700 1 --2- 172.21.15.107:0/2373514715 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f9a24078990 0x7f9a2407ae40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f9a20009b30 tx=0x7f9a20009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:06.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.096+0000 7f9a34ff9700 1 -- 172.21.15.107:0/2373514715 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f9a28061fe0 con 0x7f9a380fc350 2024-05-17T22:00:06.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.308+0000 7f9a3e216700 1 -- 172.21.15.107:0/2373514715 --> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f9a380fbdd0 con 0x7f9a24078990 2024-05-17T22:00:06.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.318+0000 7f9a34ff9700 1 -- 172.21.15.107:0/2373514715 <== mgr.34100 v2:172.21.15.107:6800/3205268226 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f9a380fbdd0 con 0x7f9a24078990 2024-05-17T22:00:06.309 INFO:teuthology.orchestra.run.smithi107.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-17T22:00:06.310 INFO:teuthology.orchestra.run.smithi107.stdout:alertmanager.smithi107 smithi107 *:9093,9094 running (7m) 22s ago 16m 24.3M - 0.25.0 c8568f914cd2 bac2a944ae42 2024-05-17T22:00:06.310 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi107 smithi107 running (4m) 22s ago 16m 6669k - 19.0.0-3794-g931809a9 8880f7613d02 face35de205b 2024-05-17T22:00:06.310 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi138 smithi138 running (4m) 58s ago 15m 6665k - 19.0.0-3794-g931809a9 8880f7613d02 ad59e33e6d43 2024-05-17T22:00:06.310 INFO:teuthology.orchestra.run.smithi107.stdout:grafana.smithi107 smithi107 *:3000 running (7m) 22s ago 15m 86.4M - 9.4.7 954c08fa6188 8c747e777bc5 2024-05-17T22:00:06.310 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.drqody smithi107 running (27s) 22s ago 10m 81.2M - 19.0.0-3794-g931809a9 8880f7613d02 4f02280f8312 2024-05-17T22:00:06.310 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.idmbgj smithi107 running (45s) 22s ago 10m 82.6M - 19.0.0-3794-g931809a9 8880f7613d02 d541bff38965 2024-05-17T22:00:06.310 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi138.gwbtyd smithi138 starting - - - - 2024-05-17T22:00:06.310 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi138.zckrvs smithi138 running (10m) 58s ago 10m 14.9M - 17.2.7-952-g9903f776 d15a36853796 a1cfd2fe4cda 2024-05-17T22:00:06.310 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi107.xhhyiw smithi107 *:8443,9283,8765 running (6m) 22s ago 17m 592M - 19.0.0-3794-g931809a9 8880f7613d02 5acf3e511342 2024-05-17T22:00:06.310 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi138.napnll smithi138 *:8443,9283,8765 running (6m) 58s ago 15m 533M - 19.0.0-3794-g931809a9 8880f7613d02 d627761dbd0d 2024-05-17T22:00:06.310 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi107 smithi107 running (5m) 22s ago 17m 118M 2048M 19.0.0-3794-g931809a9 8880f7613d02 9717f3169f2f 2024-05-17T22:00:06.310 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi138 smithi138 running (5m) 58s ago 15m 107M 2048M 19.0.0-3794-g931809a9 8880f7613d02 55aeed62bf4d 2024-05-17T22:00:06.310 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi107 smithi107 *:9100 running (7m) 22s ago 16m 17.7M - 1.5.0 0da6a335fe13 49bec71f5487 2024-05-17T22:00:06.310 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi138 smithi138 *:9100 running (7m) 58s ago 15m 15.4M - 1.5.0 0da6a335fe13 e52824376658 2024-05-17T22:00:06.311 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0 smithi107 running (3m) 22s ago 14m 161M 1557M 19.0.0-3794-g931809a9 8880f7613d02 b319bc93500d 2024-05-17T22:00:06.311 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1 smithi107 running (3m) 22s ago 13m 154M 1557M 19.0.0-3794-g931809a9 8880f7613d02 f94b206be6eb 2024-05-17T22:00:06.311 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2 smithi107 running (2m) 22s ago 13m 144M 1557M 19.0.0-3794-g931809a9 8880f7613d02 add7015647af 2024-05-17T22:00:06.311 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3 smithi138 running (2m) 58s ago 13m 162M 2581M 19.0.0-3794-g931809a9 8880f7613d02 416273d327c7 2024-05-17T22:00:06.311 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4 smithi138 running (96s) 58s ago 12m 152M 2581M 19.0.0-3794-g931809a9 8880f7613d02 e33f411da8e6 2024-05-17T22:00:06.311 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5 smithi138 running (62s) 58s ago 12m 80.0M 2581M 19.0.0-3794-g931809a9 8880f7613d02 63810d431961 2024-05-17T22:00:06.311 INFO:teuthology.orchestra.run.smithi107.stdout:prometheus.smithi107 smithi107 *:9095 running (6m) 22s ago 15m 67.2M - 2.43.0 a07b618ecd1d bf5f609bbd6e 2024-05-17T22:00:06.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.320+0000 7f9a3e216700 1 -- 172.21.15.107:0/2373514715 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f9a24078990 msgr2=0x7f9a2407ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:06.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.320+0000 7f9a3e216700 1 --2- 172.21.15.107:0/2373514715 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f9a24078990 0x7f9a2407ae40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f9a20009b30 tx=0x7f9a20009e30 comp rx=0 tx=0).stop 2024-05-17T22:00:06.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.320+0000 7f9a3e216700 1 -- 172.21.15.107:0/2373514715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a380fc350 msgr2=0x7f9a3810c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:06.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.320+0000 7f9a3e216700 1 --2- 172.21.15.107:0/2373514715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a380fc350 0x7f9a3810c770 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9a2800eb40 tx=0x7f9a2800b040 comp rx=0 tx=0).stop 2024-05-17T22:00:06.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.320+0000 7f9a3e216700 1 -- 172.21.15.107:0/2373514715 shutdown_connections 2024-05-17T22:00:06.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.320+0000 7f9a3e216700 1 --2- 172.21.15.107:0/2373514715 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f9a24078990 0x7f9a2407ae40 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:06.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.320+0000 7f9a3e216700 1 --2- 172.21.15.107:0/2373514715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a380fc350 0x7f9a3810c770 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:06.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.320+0000 7f9a3e216700 1 --2- 172.21.15.107:0/2373514715 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9a380fcca0 0x7f9a3810ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:06.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.320+0000 7f9a3e216700 1 -- 172.21.15.107:0/2373514715 >> 172.21.15.107:0/2373514715 conn(0x7f9a380f6be0 msgr2=0x7f9a380f81c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:06.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.320+0000 7f9a3e216700 1 -- 172.21.15.107:0/2373514715 shutdown_connections 2024-05-17T22:00:06.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.320+0000 7f9a3e216700 1 -- 172.21.15.107:0/2373514715 wait complete. 2024-05-17T22:00:06.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.422+0000 7f9c77fcb700 1 -- 172.21.15.107:0/1124200699 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9c70070940 msgr2=0x7f9c70070db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:06.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.422+0000 7f9c77fcb700 1 --2- 172.21.15.107:0/1124200699 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9c70070940 0x7f9c70070db0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f9c60004940 tx=0x7f9c60009e30 comp rx=0 tx=0).stop 2024-05-17T22:00:06.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.423+0000 7f9c77fcb700 1 -- 172.21.15.107:0/1124200699 shutdown_connections 2024-05-17T22:00:06.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.423+0000 7f9c77fcb700 1 --2- 172.21.15.107:0/1124200699 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9c70070940 0x7f9c70070db0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:06.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.423+0000 7f9c77fcb700 1 --2- 172.21.15.107:0/1124200699 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c70071ef0 0x7f9c70070400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:06.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.423+0000 7f9c77fcb700 1 -- 172.21.15.107:0/1124200699 >> 172.21.15.107:0/1124200699 conn(0x7f9c700f8560 msgr2=0x7f9c700fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:06.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.423+0000 7f9c77fcb700 1 -- 172.21.15.107:0/1124200699 shutdown_connections 2024-05-17T22:00:06.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.423+0000 7f9c77fcb700 1 -- 172.21.15.107:0/1124200699 wait complete. 2024-05-17T22:00:06.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.423+0000 7f9c77fcb700 1 Processor -- start 2024-05-17T22:00:06.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.423+0000 7f9c77fcb700 1 -- start start 2024-05-17T22:00:06.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.423+0000 7f9c77fcb700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9c70070940 0x7f9c70113fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:06.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.423+0000 7f9c77fcb700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c70071ef0 0x7f9c70114510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:06.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.423+0000 7f9c77fcb700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c70114b10 con 0x7f9c70070940 2024-05-17T22:00:06.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.423+0000 7f9c77fcb700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c70115360 con 0x7f9c70071ef0 2024-05-17T22:00:06.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.423+0000 7f9c75d67700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9c70070940 0x7f9c70113fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:06.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.424+0000 7f9c75d67700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9c70070940 0x7f9c70113fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39502/0 (socket says 172.21.15.107:39502) 2024-05-17T22:00:06.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.424+0000 7f9c75d67700 1 -- 172.21.15.107:0/1340397554 learned_addr learned my addr 172.21.15.107:0/1340397554 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T22:00:06.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.424+0000 7f9c75566700 1 --2- 172.21.15.107:0/1340397554 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c70071ef0 0x7f9c70114510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:06.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.424+0000 7f9c75d67700 1 -- 172.21.15.107:0/1340397554 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c70071ef0 msgr2=0x7f9c70114510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:06.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.424+0000 7f9c75d67700 1 --2- 172.21.15.107:0/1340397554 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c70071ef0 0x7f9c70114510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:06.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.424+0000 7f9c75d67700 1 -- 172.21.15.107:0/1340397554 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c60005040 con 0x7f9c70070940 2024-05-17T22:00:06.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.424+0000 7f9c75d67700 1 --2- 172.21.15.107:0/1340397554 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9c70070940 0x7f9c70113fd0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c002a80 tx=0x7f9c6c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:06.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.424+0000 7f9c667fc700 1 -- 172.21.15.107:0/1340397554 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9c6c007cc0 con 0x7f9c70070940 2024-05-17T22:00:06.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.424+0000 7f9c77fcb700 1 -- 172.21.15.107:0/1340397554 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c70115640 con 0x7f9c70070940 2024-05-17T22:00:06.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.424+0000 7f9c667fc700 1 -- 172.21.15.107:0/1340397554 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9c6c00f040 con 0x7f9c70070940 2024-05-17T22:00:06.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.424+0000 7f9c667fc700 1 -- 172.21.15.107:0/1340397554 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9c6c00da30 con 0x7f9c70070940 2024-05-17T22:00:06.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.424+0000 7f9c77fcb700 1 -- 172.21.15.107:0/1340397554 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c70115b80 con 0x7f9c70070940 2024-05-17T22:00:06.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.425+0000 7f9c77fcb700 1 -- 172.21.15.107:0/1340397554 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c7005f5c0 con 0x7f9c70070940 2024-05-17T22:00:06.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.426+0000 7f9c667fc700 1 -- 172.21.15.107:0/1340397554 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f9c6c00db90 con 0x7f9c70070940 2024-05-17T22:00:06.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.426+0000 7f9c667fc700 1 --2- 172.21.15.107:0/1340397554 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f9c5c078990 0x7f9c5c07ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:06.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.427+0000 7f9c667fc700 1 -- 172.21.15.107:0/1340397554 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f9c6c09a840 con 0x7f9c70070940 2024-05-17T22:00:06.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.427+0000 7f9c75566700 1 --2- 172.21.15.107:0/1340397554 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f9c5c078990 0x7f9c5c07ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:06.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.427+0000 7f9c75566700 1 --2- 172.21.15.107:0/1340397554 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f9c5c078990 0x7f9c5c07ae40 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f9c6000b970 tx=0x7f9c60009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:06.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.430+0000 7f9c667fc700 1 -- 172.21.15.107:0/1340397554 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f9c6c0624e0 con 0x7f9c70070940 2024-05-17T22:00:06.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.727+0000 7f9c77fcb700 1 -- 172.21.15.107:0/1340397554 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f9c701157d0 con 0x7f9c70070940 2024-05-17T22:00:06.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.727+0000 7f9c667fc700 1 -- 172.21.15.107:0/1340397554 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7f9c701157d0 con 0x7f9c70070940 2024-05-17T22:00:06.719 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-05-17T22:00:06.719 INFO:teuthology.orchestra.run.smithi107.stdout: "mon": { 2024-05-17T22:00:06.719 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev)": 2 2024-05-17T22:00:06.719 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-05-17T22:00:06.719 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr": { 2024-05-17T22:00:06.719 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev)": 2 2024-05-17T22:00:06.720 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-05-17T22:00:06.720 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": { 2024-05-17T22:00:06.720 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev)": 6 2024-05-17T22:00:06.720 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-05-17T22:00:06.720 INFO:teuthology.orchestra.run.smithi107.stdout: "mds": { 2024-05-17T22:00:06.720 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-952-g9903f776 (9903f7767386a9d9efadb7c1effdcd3064d71804) quincy (stable)": 1, 2024-05-17T22:00:06.720 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev)": 3 2024-05-17T22:00:06.720 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-05-17T22:00:06.720 INFO:teuthology.orchestra.run.smithi107.stdout: "overall": { 2024-05-17T22:00:06.720 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-952-g9903f776 (9903f7767386a9d9efadb7c1effdcd3064d71804) quincy (stable)": 1, 2024-05-17T22:00:06.720 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev)": 13 2024-05-17T22:00:06.720 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-05-17T22:00:06.720 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-05-17T22:00:06.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.730+0000 7f9c77fcb700 1 -- 172.21.15.107:0/1340397554 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f9c5c078990 msgr2=0x7f9c5c07ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:06.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.730+0000 7f9c77fcb700 1 --2- 172.21.15.107:0/1340397554 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f9c5c078990 0x7f9c5c07ae40 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f9c6000b970 tx=0x7f9c60009e30 comp rx=0 tx=0).stop 2024-05-17T22:00:06.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.730+0000 7f9c77fcb700 1 -- 172.21.15.107:0/1340397554 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9c70070940 msgr2=0x7f9c70113fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:06.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.730+0000 7f9c77fcb700 1 --2- 172.21.15.107:0/1340397554 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9c70070940 0x7f9c70113fd0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c002a80 tx=0x7f9c6c004600 comp rx=0 tx=0).stop 2024-05-17T22:00:06.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.730+0000 7f9c77fcb700 1 -- 172.21.15.107:0/1340397554 shutdown_connections 2024-05-17T22:00:06.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.730+0000 7f9c77fcb700 1 --2- 172.21.15.107:0/1340397554 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f9c5c078990 0x7f9c5c07ae40 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:06.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.730+0000 7f9c77fcb700 1 --2- 172.21.15.107:0/1340397554 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9c70070940 0x7f9c70113fd0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:06.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.730+0000 7f9c77fcb700 1 --2- 172.21.15.107:0/1340397554 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c70071ef0 0x7f9c70114510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:06.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.730+0000 7f9c77fcb700 1 -- 172.21.15.107:0/1340397554 >> 172.21.15.107:0/1340397554 conn(0x7f9c700f8560 msgr2=0x7f9c700f9d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:06.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.730+0000 7f9c77fcb700 1 -- 172.21.15.107:0/1340397554 shutdown_connections 2024-05-17T22:00:06.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.731+0000 7f9c77fcb700 1 -- 172.21.15.107:0/1340397554 wait complete. 2024-05-17T22:00:06.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.832+0000 7f2425b28700 1 -- 172.21.15.107:0/310016326 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24200fdfd0 msgr2=0x7f24200fe440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:06.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.832+0000 7f2425b28700 1 --2- 172.21.15.107:0/310016326 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24200fdfd0 0x7f24200fe440 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2414004940 tx=0x7f2414009e30 comp rx=0 tx=0).stop 2024-05-17T22:00:06.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.832+0000 7f2425b28700 1 -- 172.21.15.107:0/310016326 shutdown_connections 2024-05-17T22:00:06.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.832+0000 7f2425b28700 1 --2- 172.21.15.107:0/310016326 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24200fdfd0 0x7f24200fe440 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:06.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.832+0000 7f2425b28700 1 --2- 172.21.15.107:0/310016326 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f24200fce60 0x7f24200fd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:06.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.832+0000 7f2425b28700 1 -- 172.21.15.107:0/310016326 >> 172.21.15.107:0/310016326 conn(0x7f24200f8540 msgr2=0x7f24200fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:06.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.832+0000 7f2425b28700 1 -- 172.21.15.107:0/310016326 shutdown_connections 2024-05-17T22:00:06.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.832+0000 7f2425b28700 1 -- 172.21.15.107:0/310016326 wait complete. 2024-05-17T22:00:06.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.832+0000 7f2425b28700 1 Processor -- start 2024-05-17T22:00:06.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.832+0000 7f2425b28700 1 -- start start 2024-05-17T22:00:06.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.833+0000 7f2425b28700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24200fce60 0x7f2420071c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:06.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.833+0000 7f2425b28700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f24200fdfd0 0x7f24200702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:06.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.833+0000 7f2425b28700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2420072200 con 0x7f24200fce60 2024-05-17T22:00:06.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.833+0000 7f2425b28700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24200707f0 con 0x7f24200fdfd0 2024-05-17T22:00:06.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.833+0000 7f241f7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24200fce60 0x7f2420071c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:06.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.833+0000 7f241f7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24200fce60 0x7f2420071c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39506/0 (socket says 172.21.15.107:39506) 2024-05-17T22:00:06.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.833+0000 7f241f7fe700 1 -- 172.21.15.107:0/294946432 learned_addr learned my addr 172.21.15.107:0/294946432 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T22:00:06.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.833+0000 7f241effd700 1 --2- 172.21.15.107:0/294946432 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f24200fdfd0 0x7f24200702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:06.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.833+0000 7f241f7fe700 1 -- 172.21.15.107:0/294946432 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f24200fdfd0 msgr2=0x7f24200702b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:06.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.833+0000 7f241f7fe700 1 --2- 172.21.15.107:0/294946432 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f24200fdfd0 0x7f24200702b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:06.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.833+0000 7f241f7fe700 1 -- 172.21.15.107:0/294946432 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2414005040 con 0x7f24200fce60 2024-05-17T22:00:06.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.833+0000 7f241f7fe700 1 --2- 172.21.15.107:0/294946432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24200fce60 0x7f2420071c00 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f241000c830 tx=0x7f2410004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:06.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.833+0000 7f2407fff700 1 -- 172.21.15.107:0/294946432 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f241000a760 con 0x7f24200fce60 2024-05-17T22:00:06.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.833+0000 7f2425b28700 1 -- 172.21.15.107:0/294946432 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2420070aa0 con 0x7f24200fce60 2024-05-17T22:00:06.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.833+0000 7f2407fff700 1 -- 172.21.15.107:0/294946432 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f241000ad90 con 0x7f24200fce60 2024-05-17T22:00:06.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.833+0000 7f2407fff700 1 -- 172.21.15.107:0/294946432 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f24100113a0 con 0x7f24200fce60 2024-05-17T22:00:06.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.833+0000 7f2425b28700 1 -- 172.21.15.107:0/294946432 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2420071060 con 0x7f24200fce60 2024-05-17T22:00:06.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.834+0000 7f2425b28700 1 -- 172.21.15.107:0/294946432 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f242004c710 con 0x7f24200fce60 2024-05-17T22:00:06.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.835+0000 7f2407fff700 1 -- 172.21.15.107:0/294946432 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f2410012400 con 0x7f24200fce60 2024-05-17T22:00:06.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.835+0000 7f2407fff700 1 --2- 172.21.15.107:0/294946432 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f2408078990 0x7f240807ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:06.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.836+0000 7f2407fff700 1 -- 172.21.15.107:0/294946432 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f241009a780 con 0x7f24200fce60 2024-05-17T22:00:06.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.836+0000 7f241effd700 1 --2- 172.21.15.107:0/294946432 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f2408078990 0x7f240807ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:06.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.836+0000 7f241effd700 1 --2- 172.21.15.107:0/294946432 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f2408078990 0x7f240807ae40 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f241400b3b0 tx=0x7f2414009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:06.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:06.840+0000 7f2407fff700 1 -- 172.21.15.107:0/294946432 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f2410062420 con 0x7f24200fce60 2024-05-17T22:00:07.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:06 smithi138 ceph-mon[81376]: from='client.34446 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:00:07.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:06 smithi138 ceph-mon[81376]: from='client.34450 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:00:07.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:06 smithi138 ceph-mon[81376]: from='client.? 172.21.15.107:0/1340397554' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:00:07.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:07.087+0000 7f2425b28700 1 -- 172.21.15.107:0/294946432 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f2420115a80 con 0x7f24200fce60 2024-05-17T22:00:07.079 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:06 smithi107 ceph-mon[93224]: from='client.34446 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:00:07.079 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:06 smithi107 ceph-mon[93224]: from='client.34450 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:00:07.079 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:06 smithi107 ceph-mon[93224]: from='client.? 172.21.15.107:0/1340397554' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:00:07.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:07.088+0000 7f2407fff700 1 -- 172.21.15.107:0/294946432 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 27 v27) v1 ==== 76+0+1895 (secure 0 0 0) 0x7f2420115a80 con 0x7f24200fce60 2024-05-17T22:00:07.079 INFO:teuthology.orchestra.run.smithi107.stdout:e27 2024-05-17T22:00:07.079 INFO:teuthology.orchestra.run.smithi107.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-17T22:00:07.080 INFO:teuthology.orchestra.run.smithi107.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-17T22:00:07.080 INFO:teuthology.orchestra.run.smithi107.stdout:legacy client fscid: 1 2024-05-17T22:00:07.080 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T22:00:07.080 INFO:teuthology.orchestra.run.smithi107.stdout:Filesystem 'cephfs' (1) 2024-05-17T22:00:07.080 INFO:teuthology.orchestra.run.smithi107.stdout:fs_name cephfs 2024-05-17T22:00:07.080 INFO:teuthology.orchestra.run.smithi107.stdout:epoch 26 2024-05-17T22:00:07.080 INFO:teuthology.orchestra.run.smithi107.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-05-17T22:00:07.080 INFO:teuthology.orchestra.run.smithi107.stdout:created 2024-05-17T21:49:16.836333+0000 2024-05-17T22:00:07.080 INFO:teuthology.orchestra.run.smithi107.stdout:modified 2024-05-17T22:00:03.521665+0000 2024-05-17T22:00:07.080 INFO:teuthology.orchestra.run.smithi107.stdout:tableserver 0 2024-05-17T22:00:07.080 INFO:teuthology.orchestra.run.smithi107.stdout:root 0 2024-05-17T22:00:07.080 INFO:teuthology.orchestra.run.smithi107.stdout:session_timeout 60 2024-05-17T22:00:07.080 INFO:teuthology.orchestra.run.smithi107.stdout:session_autoclose 300 2024-05-17T22:00:07.080 INFO:teuthology.orchestra.run.smithi107.stdout:max_file_size 1099511627776 2024-05-17T22:00:07.080 INFO:teuthology.orchestra.run.smithi107.stdout:max_xattr_size 65536 2024-05-17T22:00:07.081 INFO:teuthology.orchestra.run.smithi107.stdout:required_client_features {} 2024-05-17T22:00:07.081 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure 0 2024-05-17T22:00:07.081 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure_osd_epoch 83 2024-05-17T22:00:07.081 INFO:teuthology.orchestra.run.smithi107.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-17T22:00:07.081 INFO:teuthology.orchestra.run.smithi107.stdout:max_mds 1 2024-05-17T22:00:07.081 INFO:teuthology.orchestra.run.smithi107.stdout:in 0 2024-05-17T22:00:07.081 INFO:teuthology.orchestra.run.smithi107.stdout:up {0=24267} 2024-05-17T22:00:07.081 INFO:teuthology.orchestra.run.smithi107.stdout:failed 2024-05-17T22:00:07.081 INFO:teuthology.orchestra.run.smithi107.stdout:damaged 2024-05-17T22:00:07.081 INFO:teuthology.orchestra.run.smithi107.stdout:stopped 2024-05-17T22:00:07.081 INFO:teuthology.orchestra.run.smithi107.stdout:data_pools [3] 2024-05-17T22:00:07.081 INFO:teuthology.orchestra.run.smithi107.stdout:metadata_pool 2 2024-05-17T22:00:07.081 INFO:teuthology.orchestra.run.smithi107.stdout:inline_data enabled 2024-05-17T22:00:07.081 INFO:teuthology.orchestra.run.smithi107.stdout:balancer 2024-05-17T22:00:07.081 INFO:teuthology.orchestra.run.smithi107.stdout:bal_rank_mask -1 2024-05-17T22:00:07.082 INFO:teuthology.orchestra.run.smithi107.stdout:standby_count_wanted 1 2024-05-17T22:00:07.082 INFO:teuthology.orchestra.run.smithi107.stdout:qdb_cluster leader: 24267 members: 24267 2024-05-17T22:00:07.082 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi138.zckrvs{0:24267} state up:active seq 161 join_fscid=1 addr [v2:172.21.15.138:6826/170628661,v1:172.21.15.138:6827/170628661] compat {c=[1],r=[1],i=[7ff]}] 2024-05-17T22:00:07.082 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T22:00:07.082 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T22:00:07.082 INFO:teuthology.orchestra.run.smithi107.stdout:Standby daemons: 2024-05-17T22:00:07.082 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T22:00:07.082 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.idmbgj{-1:34408} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.107:6826/2235359521,v1:172.21.15.107:6827/2235359521] compat {c=[1],r=[1],i=[1fff]}] 2024-05-17T22:00:07.082 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.drqody{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.107:6828/369507752,v1:172.21.15.107:6829/369507752] compat {c=[1],r=[1],i=[1fff]}] 2024-05-17T22:00:07.082 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi138.gwbtyd{-1:44335} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6824/1700112281,v1:172.21.15.138:6825/1700112281] compat {c=[1],r=[1],i=[1fff]}] 2024-05-17T22:00:07.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:07.090+0000 7f2425b28700 1 -- 172.21.15.107:0/294946432 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f2408078990 msgr2=0x7f240807ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:07.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:07.090+0000 7f2425b28700 1 --2- 172.21.15.107:0/294946432 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f2408078990 0x7f240807ae40 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f241400b3b0 tx=0x7f2414009e30 comp rx=0 tx=0).stop 2024-05-17T22:00:07.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:07.090+0000 7f2425b28700 1 -- 172.21.15.107:0/294946432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24200fce60 msgr2=0x7f2420071c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:07.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:07.090+0000 7f2425b28700 1 --2- 172.21.15.107:0/294946432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24200fce60 0x7f2420071c00 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f241000c830 tx=0x7f2410004470 comp rx=0 tx=0).stop 2024-05-17T22:00:07.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:07.090+0000 7f2425b28700 1 -- 172.21.15.107:0/294946432 shutdown_connections 2024-05-17T22:00:07.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:07.090+0000 7f2425b28700 1 --2- 172.21.15.107:0/294946432 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f2408078990 0x7f240807ae40 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:07.090+0000 7f2425b28700 1 --2- 172.21.15.107:0/294946432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24200fce60 0x7f2420071c00 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:07.090+0000 7f2425b28700 1 --2- 172.21.15.107:0/294946432 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f24200fdfd0 0x7f24200702b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:07.090+0000 7f2425b28700 1 -- 172.21.15.107:0/294946432 >> 172.21.15.107:0/294946432 conn(0x7f24200f8540 msgr2=0x7f24200f9c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:07.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:07.090+0000 7f2425b28700 1 -- 172.21.15.107:0/294946432 shutdown_connections 2024-05-17T22:00:07.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:07.090+0000 7f2425b28700 1 -- 172.21.15.107:0/294946432 wait complete. 2024-05-17T22:00:07.084 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 27 2024-05-17T22:00:07.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:07.193+0000 7f05b7936700 1 -- 172.21.15.107:0/3393768725 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05b00fed60 msgr2=0x7f05b00faea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:07.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:07.193+0000 7f05b7936700 1 --2- 172.21.15.107:0/3393768725 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05b00fed60 0x7f05b00faea0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f05a0004940 tx=0x7f05a0009e30 comp rx=0 tx=0).stop 2024-05-17T22:00:07.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:07.193+0000 7f05b7936700 1 -- 172.21.15.107:0/3393768725 shutdown_connections 2024-05-17T22:00:07.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:07.193+0000 7f05b7936700 1 --2- 172.21.15.107:0/3393768725 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05b00fed60 0x7f05b00faea0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:07.193+0000 7f05b7936700 1 --2- 172.21.15.107:0/3393768725 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05b00fe410 0x7f05b00fe820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:07.193+0000 7f05b7936700 1 -- 172.21.15.107:0/3393768725 >> 172.21.15.107:0/3393768725 conn(0x7f05b00f6bd0 msgr2=0x7f05b00f8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:07.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:07.193+0000 7f05b7936700 1 -- 172.21.15.107:0/3393768725 shutdown_connections 2024-05-17T22:00:07.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:07.193+0000 7f05b7936700 1 -- 172.21.15.107:0/3393768725 wait complete. 2024-05-17T22:00:07.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:07.193+0000 7f05b7936700 1 Processor -- start 2024-05-17T22:00:07.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:07.193+0000 7f05b7936700 1 -- start start 2024-05-17T22:00:07.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:07.193+0000 7f05b7936700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05b00fe410 0x7f05b010d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:07.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:07.194+0000 7f05b7936700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05b00fed60 0x7f05b010dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:07.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:07.194+0000 7f05b7936700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05b010e3a0 con 0x7f05b00fe410 2024-05-17T22:00:07.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:07.194+0000 7f05b7936700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05b010e510 con 0x7f05b00fed60 2024-05-17T22:00:07.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:07.194+0000 7f05b56d2700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05b00fe410 0x7f05b010d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:07.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:07.194+0000 7f05b56d2700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05b00fe410 0x7f05b010d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39522/0 (socket says 172.21.15.107:39522) 2024-05-17T22:00:07.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:07.194+0000 7f05b56d2700 1 -- 172.21.15.107:0/1904025206 learned_addr learned my addr 172.21.15.107:0/1904025206 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T22:00:07.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:07.194+0000 7f05b4ed1700 1 --2- 172.21.15.107:0/1904025206 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05b00fed60 0x7f05b010dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:07.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:07.194+0000 7f05b56d2700 1 -- 172.21.15.107:0/1904025206 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05b00fed60 msgr2=0x7f05b010dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:07.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:07.194+0000 7f05b56d2700 1 --2- 172.21.15.107:0/1904025206 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05b00fed60 0x7f05b010dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:07.194+0000 7f05b56d2700 1 -- 172.21.15.107:0/1904025206 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05a0005040 con 0x7f05b00fe410 2024-05-17T22:00:07.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:07.194+0000 7f05b56d2700 1 --2- 172.21.15.107:0/1904025206 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05b00fe410 0x7f05b010d860 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f05ac00eb40 tx=0x7f05ac00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:07.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:07.194+0000 7f05a5ffb700 1 -- 172.21.15.107:0/1904025206 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f05ac007cc0 con 0x7f05b00fe410 2024-05-17T22:00:07.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:07.194+0000 7f05b7936700 1 -- 172.21.15.107:0/1904025206 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05b0112f10 con 0x7f05b00fe410 2024-05-17T22:00:07.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:07.194+0000 7f05a5ffb700 1 -- 172.21.15.107:0/1904025206 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f05ac00f040 con 0x7f05b00fe410 2024-05-17T22:00:07.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:07.194+0000 7f05a5ffb700 1 -- 172.21.15.107:0/1904025206 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f05ac00ca10 con 0x7f05b00fe410 2024-05-17T22:00:07.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:07.194+0000 7f05b7936700 1 -- 172.21.15.107:0/1904025206 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05b01133c0 con 0x7f05b00fe410 2024-05-17T22:00:07.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:07.195+0000 7f05b7936700 1 -- 172.21.15.107:0/1904025206 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05b004c710 con 0x7f05b00fe410 2024-05-17T22:00:07.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:07.196+0000 7f05a5ffb700 1 -- 172.21.15.107:0/1904025206 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f05ac00cc80 con 0x7f05b00fe410 2024-05-17T22:00:07.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:07.196+0000 7f05a5ffb700 1 --2- 172.21.15.107:0/1904025206 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f059c078980 0x7f059c07ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:07.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:07.197+0000 7f05a5ffb700 1 -- 172.21.15.107:0/1904025206 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f05ac09a2f0 con 0x7f05b00fe410 2024-05-17T22:00:07.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:07.197+0000 7f05b4ed1700 1 --2- 172.21.15.107:0/1904025206 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f059c078980 0x7f059c07ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:07.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:07.197+0000 7f05b4ed1700 1 --2- 172.21.15.107:0/1904025206 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f059c078980 0x7f059c07ae30 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f05a000b0d0 tx=0x7f05a0009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:07.192 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:07.201+0000 7f05a5ffb700 1 -- 172.21.15.107:0/1904025206 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f05ac061fc0 con 0x7f05b00fe410 2024-05-17T22:00:07.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:07.424+0000 7f05b7936700 1 -- 172.21.15.107:0/1904025206 --> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f05b0102460 con 0x7f059c078980 2024-05-17T22:00:07.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:07.425+0000 7f05a5ffb700 1 -- 172.21.15.107:0/1904025206 <== mgr.34100 v2:172.21.15.107:6800/3205268226 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f05b0102460 con 0x7f059c078980 2024-05-17T22:00:07.417 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-05-17T22:00:07.417 INFO:teuthology.orchestra.run.smithi107.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b", 2024-05-17T22:00:07.417 INFO:teuthology.orchestra.run.smithi107.stdout: "in_progress": true, 2024-05-17T22:00:07.417 INFO:teuthology.orchestra.run.smithi107.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-17T22:00:07.417 INFO:teuthology.orchestra.run.smithi107.stdout: "services_complete": [ 2024-05-17T22:00:07.417 INFO:teuthology.orchestra.run.smithi107.stdout: "crash", 2024-05-17T22:00:07.417 INFO:teuthology.orchestra.run.smithi107.stdout: "osd", 2024-05-17T22:00:07.417 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr", 2024-05-17T22:00:07.417 INFO:teuthology.orchestra.run.smithi107.stdout: "mon" 2024-05-17T22:00:07.417 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-05-17T22:00:07.417 INFO:teuthology.orchestra.run.smithi107.stdout: "progress": "14/21 daemons upgraded", 2024-05-17T22:00:07.417 INFO:teuthology.orchestra.run.smithi107.stdout: "message": "Currently upgrading mds daemons", 2024-05-17T22:00:07.417 INFO:teuthology.orchestra.run.smithi107.stdout: "is_paused": false 2024-05-17T22:00:07.417 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-05-17T22:00:07.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:07.428+0000 7f05b7936700 1 -- 172.21.15.107:0/1904025206 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f059c078980 msgr2=0x7f059c07ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:07.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:07.428+0000 7f05b7936700 1 --2- 172.21.15.107:0/1904025206 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f059c078980 0x7f059c07ae30 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f05a000b0d0 tx=0x7f05a0009e30 comp rx=0 tx=0).stop 2024-05-17T22:00:07.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:07.428+0000 7f05b7936700 1 -- 172.21.15.107:0/1904025206 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05b00fe410 msgr2=0x7f05b010d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:07.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:07.428+0000 7f05b7936700 1 --2- 172.21.15.107:0/1904025206 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05b00fe410 0x7f05b010d860 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f05ac00eb40 tx=0x7f05ac00b040 comp rx=0 tx=0).stop 2024-05-17T22:00:07.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:07.428+0000 7f05b7936700 1 -- 172.21.15.107:0/1904025206 shutdown_connections 2024-05-17T22:00:07.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:07.428+0000 7f05b7936700 1 --2- 172.21.15.107:0/1904025206 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f059c078980 0x7f059c07ae30 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:07.428+0000 7f05b7936700 1 --2- 172.21.15.107:0/1904025206 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05b00fe410 0x7f05b010d860 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:07.428+0000 7f05b7936700 1 --2- 172.21.15.107:0/1904025206 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05b00fed60 0x7f05b010dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:07.428+0000 7f05b7936700 1 -- 172.21.15.107:0/1904025206 >> 172.21.15.107:0/1904025206 conn(0x7f05b00f6bd0 msgr2=0x7f05b00f8170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:07.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:07.428+0000 7f05b7936700 1 -- 172.21.15.107:0/1904025206 shutdown_connections 2024-05-17T22:00:07.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:07.428+0000 7f05b7936700 1 -- 172.21.15.107:0/1904025206 wait complete. 2024-05-17T22:00:07.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:07.530+0000 7fc85a3c1700 1 -- 172.21.15.107:0/2295216032 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8540fdfd0 msgr2=0x7fc8540fe440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:07.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:07.530+0000 7fc85a3c1700 1 --2- 172.21.15.107:0/2295216032 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8540fdfd0 0x7fc8540fe440 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc83c004940 tx=0x7fc83c009e30 comp rx=0 tx=0).stop 2024-05-17T22:00:07.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:07.530+0000 7fc85a3c1700 1 -- 172.21.15.107:0/2295216032 shutdown_connections 2024-05-17T22:00:07.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:07.530+0000 7fc85a3c1700 1 --2- 172.21.15.107:0/2295216032 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8540fdfd0 0x7fc8540fe440 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:07.530+0000 7fc85a3c1700 1 --2- 172.21.15.107:0/2295216032 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8540fce60 0x7fc8540fd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:07.530+0000 7fc85a3c1700 1 -- 172.21.15.107:0/2295216032 >> 172.21.15.107:0/2295216032 conn(0x7fc8540f8540 msgr2=0x7fc8540fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:07.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:07.530+0000 7fc85a3c1700 1 -- 172.21.15.107:0/2295216032 shutdown_connections 2024-05-17T22:00:07.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:07.530+0000 7fc85a3c1700 1 -- 172.21.15.107:0/2295216032 wait complete. 2024-05-17T22:00:07.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:07.531+0000 7fc85a3c1700 1 Processor -- start 2024-05-17T22:00:07.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:07.531+0000 7fc85a3c1700 1 -- start start 2024-05-17T22:00:07.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:07.531+0000 7fc85a3c1700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8540fce60 0x7fc85410d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:07.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:07.531+0000 7fc85a3c1700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8540fdfd0 0x7fc85410dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:07.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:07.531+0000 7fc85a3c1700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc85410e3e0 con 0x7fc8540fdfd0 2024-05-17T22:00:07.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:07.531+0000 7fc85a3c1700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc85410e550 con 0x7fc8540fce60 2024-05-17T22:00:07.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:07.531+0000 7fc84bfff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8540fdfd0 0x7fc85410dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:07.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:07.531+0000 7fc84bfff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8540fdfd0 0x7fc85410dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39536/0 (socket says 172.21.15.107:39536) 2024-05-17T22:00:07.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:07.531+0000 7fc84bfff700 1 -- 172.21.15.107:0/2361694562 learned_addr learned my addr 172.21.15.107:0/2361694562 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T22:00:07.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:07.531+0000 7fc853fff700 1 --2- 172.21.15.107:0/2361694562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8540fce60 0x7fc85410d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:07.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:07.531+0000 7fc84bfff700 1 -- 172.21.15.107:0/2361694562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8540fce60 msgr2=0x7fc85410d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:07.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:07.531+0000 7fc84bfff700 1 --2- 172.21.15.107:0/2361694562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8540fce60 0x7fc85410d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:07.531+0000 7fc84bfff700 1 -- 172.21.15.107:0/2361694562 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc83c005040 con 0x7fc8540fdfd0 2024-05-17T22:00:07.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:07.531+0000 7fc84bfff700 1 --2- 172.21.15.107:0/2361694562 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8540fdfd0 0x7fc85410dde0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc83c013870 tx=0x7fc83c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:07.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:07.532+0000 7fc8517fa700 1 -- 172.21.15.107:0/2361694562 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc83c021070 con 0x7fc8540fdfd0 2024-05-17T22:00:07.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:07.532+0000 7fc8517fa700 1 -- 172.21.15.107:0/2361694562 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc83c009430 con 0x7fc8540fdfd0 2024-05-17T22:00:07.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:07.532+0000 7fc85a3c1700 1 -- 172.21.15.107:0/2361694562 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc854112ee0 con 0x7fc8540fdfd0 2024-05-17T22:00:07.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:07.532+0000 7fc8517fa700 1 -- 172.21.15.107:0/2361694562 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc83c00e070 con 0x7fc8540fdfd0 2024-05-17T22:00:07.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:07.532+0000 7fc85a3c1700 1 -- 172.21.15.107:0/2361694562 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8541133f0 con 0x7fc8540fdfd0 2024-05-17T22:00:07.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:07.533+0000 7fc85a3c1700 1 -- 172.21.15.107:0/2361694562 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc85404c710 con 0x7fc8540fdfd0 2024-05-17T22:00:07.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:07.533+0000 7fc8517fa700 1 -- 172.21.15.107:0/2361694562 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7fc83c01c750 con 0x7fc8540fdfd0 2024-05-17T22:00:07.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:07.534+0000 7fc8517fa700 1 --2- 172.21.15.107:0/2361694562 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fc840078990 0x7fc84007ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:07.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:07.534+0000 7fc8517fa700 1 -- 172.21.15.107:0/2361694562 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fc83c09b7d0 con 0x7fc8540fdfd0 2024-05-17T22:00:07.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:07.534+0000 7fc853fff700 1 --2- 172.21.15.107:0/2361694562 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fc840078990 0x7fc84007ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:07.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:07.534+0000 7fc853fff700 1 --2- 172.21.15.107:0/2361694562 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fc840078990 0x7fc84007ae40 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc844009b80 tx=0x7fc84400b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:07.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:07.539+0000 7fc8517fa700 1 -- 172.21.15.107:0/2361694562 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7fc83c063530 con 0x7fc8540fdfd0 2024-05-17T22:00:07.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:07.825+0000 7fc85a3c1700 1 -- 172.21.15.107:0/2361694562 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fc8541058e0 con 0x7fc8540fdfd0 2024-05-17T22:00:07.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:07.825+0000 7fc8517fa700 1 -- 172.21.15.107:0/2361694562 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7fc83c062ed0 con 0x7fc8540fdfd0 2024-05-17T22:00:07.817 INFO:teuthology.orchestra.run.smithi107.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-17T22:00:07.817 INFO:teuthology.orchestra.run.smithi107.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-17T22:00:07.817 INFO:teuthology.orchestra.run.smithi107.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-17T22:00:07.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:07.829+0000 7fc85a3c1700 1 -- 172.21.15.107:0/2361694562 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fc840078990 msgr2=0x7fc84007ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:07.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:07.829+0000 7fc85a3c1700 1 --2- 172.21.15.107:0/2361694562 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fc840078990 0x7fc84007ae40 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc844009b80 tx=0x7fc84400b040 comp rx=0 tx=0).stop 2024-05-17T22:00:07.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:07.829+0000 7fc85a3c1700 1 -- 172.21.15.107:0/2361694562 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8540fdfd0 msgr2=0x7fc85410dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:07.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:07.829+0000 7fc85a3c1700 1 --2- 172.21.15.107:0/2361694562 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8540fdfd0 0x7fc85410dde0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc83c013870 tx=0x7fc83c009e30 comp rx=0 tx=0).stop 2024-05-17T22:00:07.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:07.829+0000 7fc85a3c1700 1 -- 172.21.15.107:0/2361694562 shutdown_connections 2024-05-17T22:00:07.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:07.829+0000 7fc85a3c1700 1 --2- 172.21.15.107:0/2361694562 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fc840078990 0x7fc84007ae40 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:07.829+0000 7fc85a3c1700 1 --2- 172.21.15.107:0/2361694562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8540fce60 0x7fc85410d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:07.829+0000 7fc85a3c1700 1 --2- 172.21.15.107:0/2361694562 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8540fdfd0 0x7fc85410dde0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:07.829+0000 7fc85a3c1700 1 -- 172.21.15.107:0/2361694562 >> 172.21.15.107:0/2361694562 conn(0x7fc8540f8540 msgr2=0x7fc8540f9af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:07.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:07.829+0000 7fc85a3c1700 1 -- 172.21.15.107:0/2361694562 shutdown_connections 2024-05-17T22:00:07.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:07.829+0000 7fc85a3c1700 1 -- 172.21.15.107:0/2361694562 wait complete. 2024-05-17T22:00:08.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:07 smithi107 ceph-mon[93224]: from='client.44347 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:00:08.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:07 smithi107 ceph-mon[93224]: pgmap v203: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 20 MiB/s rd, 255 B/s wr, 8 op/s 2024-05-17T22:00:08.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:07 smithi107 ceph-mon[93224]: from='client.? 172.21.15.107:0/294946432' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-17T22:00:08.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:07 smithi107 ceph-mon[93224]: from='client.? 172.21.15.107:0/2361694562' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-17T22:00:08.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:07 smithi138 ceph-mon[81376]: from='client.44347 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:00:08.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:07 smithi138 ceph-mon[81376]: pgmap v203: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 20 MiB/s rd, 255 B/s wr, 8 op/s 2024-05-17T22:00:08.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:07 smithi138 ceph-mon[81376]: from='client.? 172.21.15.107:0/294946432' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-17T22:00:08.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:07 smithi138 ceph-mon[81376]: from='client.? 172.21.15.107:0/2361694562' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-17T22:00:09.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:09 smithi138 ceph-mon[81376]: from='client.34466 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:00:09.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:09 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:00:09.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:09 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:00:09.350 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:09 smithi107 ceph-mon[93224]: from='client.34466 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:00:09.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:09 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:00:09.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:09 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:00:10.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:10 smithi138 ceph-mon[81376]: pgmap v204: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 12 MiB/s rd, 4.7 KiB/s wr, 8 op/s 2024-05-17T22:00:10.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:10 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:00:10.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:10 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:00:10.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:10 smithi107 ceph-mon[93224]: pgmap v204: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 12 MiB/s rd, 4.7 KiB/s wr, 8 op/s 2024-05-17T22:00:10.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:10 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:00:10.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:10 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:00:12.288 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:12 smithi138 ceph-mon[81376]: pgmap v205: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 6.6 MiB/s rd, 4.7 KiB/s wr, 7 op/s 2024-05-17T22:00:12.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:12 smithi107 ceph-mon[93224]: pgmap v205: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 6.6 MiB/s rd, 4.7 KiB/s wr, 7 op/s 2024-05-17T22:00:14.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:13 smithi138 ceph-mon[81376]: pgmap v206: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 6.6 MiB/s rd, 4.7 KiB/s wr, 7 op/s 2024-05-17T22:00:14.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:13 smithi138 ceph-mon[81376]: Detected new or changed devices on smithi138 2024-05-17T22:00:14.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:13 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:00:14.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:13 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:00:14.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:13 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:00:14.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:13 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:00:14.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:13 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:00:14.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:13 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:00:14.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:13 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:00:14.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:13 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:00:14.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:13 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:00:14.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:13 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:00:14.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:13 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi138.zckrvs"]}]: dispatch 2024-05-17T22:00:14.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:13 smithi138 ceph-mon[81376]: Upgrade: It appears safe to stop mds.cephfs.smithi138.zckrvs 2024-05-17T22:00:14.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:13 smithi107 ceph-mon[93224]: pgmap v206: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 6.6 MiB/s rd, 4.7 KiB/s wr, 7 op/s 2024-05-17T22:00:14.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:13 smithi107 ceph-mon[93224]: Detected new or changed devices on smithi138 2024-05-17T22:00:14.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:13 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:00:14.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:13 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:00:14.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:13 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:00:14.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:13 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:00:14.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:13 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:00:14.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:13 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:00:14.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:13 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:00:14.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:13 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:00:14.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:13 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:00:14.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:13 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:00:14.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:13 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi138.zckrvs"]}]: dispatch 2024-05-17T22:00:14.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:13 smithi107 ceph-mon[93224]: Upgrade: It appears safe to stop mds.cephfs.smithi138.zckrvs 2024-05-17T22:00:15.795 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:15 smithi138 ceph-mon[81376]: pgmap v207: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 3.3 KiB/s rd, 4.7 KiB/s wr, 5 op/s 2024-05-17T22:00:15.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:15 smithi138 ceph-mon[81376]: Upgrade: Updating mds.cephfs.smithi138.zckrvs 2024-05-17T22:00:15.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:15 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:00:15.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:15 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi138.zckrvs", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T22:00:15.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:15 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:00:15.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:15 smithi138 ceph-mon[81376]: Deploying daemon mds.cephfs.smithi138.zckrvs on smithi138 2024-05-17T22:00:15.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:15 smithi107 ceph-mon[93224]: pgmap v207: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 3.3 KiB/s rd, 4.7 KiB/s wr, 5 op/s 2024-05-17T22:00:15.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:15 smithi107 ceph-mon[93224]: Upgrade: Updating mds.cephfs.smithi138.zckrvs 2024-05-17T22:00:15.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:15 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:00:15.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:15 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi138.zckrvs", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T22:00:15.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:15 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:00:15.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:15 smithi107 ceph-mon[93224]: Deploying daemon mds.cephfs.smithi138.zckrvs on smithi138 2024-05-17T22:00:17.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:16 smithi107 ceph-3722731a-1496-11ef-bc9a-c7b262605968-mon-smithi107[93220]: 2024-05-17T22:00:16.635+0000 7f7d6bd88700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-17T22:00:17.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:17 smithi138 ceph-mon[81376]: pgmap v208: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 3.3 KiB/s rd, 4.7 KiB/s wr, 5 op/s 2024-05-17T22:00:17.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:17 smithi138 ceph-mon[81376]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-17T22:00:17.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:17 smithi138 ceph-mon[81376]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-17T22:00:17.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:17 smithi138 ceph-mon[81376]: osdmap e84: 6 total, 6 up, 6 in 2024-05-17T22:00:17.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:17 smithi138 ceph-mon[81376]: Standby daemon mds.cephfs.smithi107.idmbgj assigned to filesystem cephfs as rank 0 2024-05-17T22:00:17.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:17 smithi138 ceph-mon[81376]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-17T22:00:17.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:17 smithi138 ceph-mon[81376]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-05-17T22:00:17.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:17 smithi138 ceph-mon[81376]: fsmap cephfs:1/1 {0=cephfs.smithi107.idmbgj=up:replay} 2 up:standby 2024-05-17T22:00:17.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:17 smithi107 ceph-mon[93224]: pgmap v208: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 3.3 KiB/s rd, 4.7 KiB/s wr, 5 op/s 2024-05-17T22:00:17.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:17 smithi107 ceph-mon[93224]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-17T22:00:17.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:17 smithi107 ceph-mon[93224]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-17T22:00:17.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:17 smithi107 ceph-mon[93224]: osdmap e84: 6 total, 6 up, 6 in 2024-05-17T22:00:17.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:17 smithi107 ceph-mon[93224]: Standby daemon mds.cephfs.smithi107.idmbgj assigned to filesystem cephfs as rank 0 2024-05-17T22:00:17.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:17 smithi107 ceph-mon[93224]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-17T22:00:17.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:17 smithi107 ceph-mon[93224]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-05-17T22:00:17.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:17 smithi107 ceph-mon[93224]: fsmap cephfs:1/1 {0=cephfs.smithi107.idmbgj=up:replay} 2 up:standby 2024-05-17T22:00:19.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:19 smithi138 ceph-mon[81376]: pgmap v210: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 2.4 MiB/s rd, 0 B/s wr, 1 op/s 2024-05-17T22:00:19.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:19 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:00:19.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:19 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T22:00:19.850 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:19 smithi107 ceph-mon[93224]: pgmap v210: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 2.4 MiB/s rd, 0 B/s wr, 1 op/s 2024-05-17T22:00:19.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:19 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:00:19.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:19 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T22:00:21.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:21 smithi138 ceph-mon[81376]: pgmap v211: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 3.3 MiB/s rd, 0 B/s wr, 1 op/s 2024-05-17T22:00:21.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:21 smithi107 ceph-mon[93224]: pgmap v211: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 3.3 MiB/s rd, 0 B/s wr, 1 op/s 2024-05-17T22:00:23.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:23 smithi138 ceph-mon[81376]: pgmap v212: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 18 MiB/s rd, 5 op/s 2024-05-17T22:00:23.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:23 smithi107 ceph-mon[93224]: pgmap v212: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 18 MiB/s rd, 5 op/s 2024-05-17T22:00:25.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:25 smithi107 ceph-mon[93224]: pgmap v213: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 18 MiB/s rd, 5 op/s 2024-05-17T22:00:25.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:25 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:00:25.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:25 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:00:25.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:25 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:00:26.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:25 smithi138 ceph-mon[81376]: pgmap v213: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 18 MiB/s rd, 5 op/s 2024-05-17T22:00:26.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:25 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:00:26.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:25 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:00:26.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:25 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:00:26.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:26 smithi107 ceph-mon[93224]: mds.? [v2:172.21.15.138:6826/1801131392,v1:172.21.15.138:6827/1801131392] up:boot 2024-05-17T22:00:26.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:26 smithi107 ceph-mon[93224]: fsmap cephfs:1/1 {0=cephfs.smithi107.idmbgj=up:replay} 3 up:standby 2024-05-17T22:00:26.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:26 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi138.zckrvs"}]: dispatch 2024-05-17T22:00:26.911 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:26 smithi138 ceph-mon[81376]: mds.? [v2:172.21.15.138:6826/1801131392,v1:172.21.15.138:6827/1801131392] up:boot 2024-05-17T22:00:26.911 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:26 smithi138 ceph-mon[81376]: fsmap cephfs:1/1 {0=cephfs.smithi107.idmbgj=up:replay} 3 up:standby 2024-05-17T22:00:26.912 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:26 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi138.zckrvs"}]: dispatch 2024-05-17T22:00:27.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:27 smithi107 ceph-mon[93224]: pgmap v214: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 23 MiB/s rd, 7 op/s 2024-05-17T22:00:27.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:27 smithi107 ceph-mon[93224]: reconnect by client.14522 192.168.0.1:0/3969818366 after 0.000999984 2024-05-17T22:00:27.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:27 smithi107 ceph-mon[93224]: reconnect by client.24295 192.168.0.1:0/262157930 after 0.00199997 2024-05-17T22:00:27.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:27 smithi107 ceph-mon[93224]: mds.? [v2:172.21.15.107:6826/2235359521,v1:172.21.15.107:6827/2235359521] up:reconnect 2024-05-17T22:00:27.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:27 smithi107 ceph-mon[93224]: fsmap cephfs:1/1 {0=cephfs.smithi107.idmbgj=up:reconnect} 3 up:standby 2024-05-17T22:00:28.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:27 smithi138 ceph-mon[81376]: pgmap v214: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 23 MiB/s rd, 7 op/s 2024-05-17T22:00:28.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:27 smithi138 ceph-mon[81376]: reconnect by client.14522 192.168.0.1:0/3969818366 after 0.000999984 2024-05-17T22:00:28.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:27 smithi138 ceph-mon[81376]: reconnect by client.24295 192.168.0.1:0/262157930 after 0.00199997 2024-05-17T22:00:28.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:27 smithi138 ceph-mon[81376]: mds.? [v2:172.21.15.107:6826/2235359521,v1:172.21.15.107:6827/2235359521] up:reconnect 2024-05-17T22:00:28.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:27 smithi138 ceph-mon[81376]: fsmap cephfs:1/1 {0=cephfs.smithi107.idmbgj=up:reconnect} 3 up:standby 2024-05-17T22:00:29.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:28 smithi138 ceph-mon[81376]: mds.? [v2:172.21.15.107:6826/2235359521,v1:172.21.15.107:6827/2235359521] up:rejoin 2024-05-17T22:00:29.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:28 smithi138 ceph-mon[81376]: fsmap cephfs:1/1 {0=cephfs.smithi107.idmbgj=up:rejoin} 3 up:standby 2024-05-17T22:00:29.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:28 smithi138 ceph-mon[81376]: daemon mds.cephfs.smithi107.idmbgj is now active in filesystem cephfs as rank 0 2024-05-17T22:00:29.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:28 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:00:29.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:28 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:00:29.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:28 smithi107 ceph-mon[93224]: mds.? [v2:172.21.15.107:6826/2235359521,v1:172.21.15.107:6827/2235359521] up:rejoin 2024-05-17T22:00:29.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:28 smithi107 ceph-mon[93224]: fsmap cephfs:1/1 {0=cephfs.smithi107.idmbgj=up:rejoin} 3 up:standby 2024-05-17T22:00:29.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:28 smithi107 ceph-mon[93224]: daemon mds.cephfs.smithi107.idmbgj is now active in filesystem cephfs as rank 0 2024-05-17T22:00:29.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:28 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:00:29.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:28 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:00:29.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:29 smithi107 ceph-mon[93224]: pgmap v215: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 6 op/s 2024-05-17T22:00:29.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:29 smithi107 ceph-mon[93224]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-17T22:00:29.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:29 smithi107 ceph-mon[93224]: mds.? [v2:172.21.15.107:6826/2235359521,v1:172.21.15.107:6827/2235359521] up:active 2024-05-17T22:00:29.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:29 smithi107 ceph-mon[93224]: fsmap cephfs:1 {0=cephfs.smithi107.idmbgj=up:active} 3 up:standby 2024-05-17T22:00:29.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:29 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:00:29.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:29 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:00:29.860 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:29 smithi138 ceph-mon[81376]: pgmap v215: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 6 op/s 2024-05-17T22:00:29.860 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:29 smithi138 ceph-mon[81376]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-17T22:00:29.860 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:29 smithi138 ceph-mon[81376]: mds.? [v2:172.21.15.107:6826/2235359521,v1:172.21.15.107:6827/2235359521] up:active 2024-05-17T22:00:29.861 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:29 smithi138 ceph-mon[81376]: fsmap cephfs:1 {0=cephfs.smithi107.idmbgj=up:active} 3 up:standby 2024-05-17T22:00:29.861 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:29 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:00:29.861 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:29 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:00:32.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:31 smithi138 ceph-mon[81376]: pgmap v216: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 19 MiB/s rd, 5 op/s 2024-05-17T22:00:32.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:31 smithi107 ceph-mon[93224]: pgmap v216: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 19 MiB/s rd, 5 op/s 2024-05-17T22:00:34.011 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:33 smithi138 ceph-mon[81376]: pgmap v217: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 18 MiB/s rd, 4.3 KiB/s wr, 10 op/s 2024-05-17T22:00:34.011 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:33 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:00:34.012 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:33 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:00:34.012 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:33 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:00:34.012 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:33 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:00:34.012 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:33 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:00:34.012 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:33 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:00:34.012 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:33 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:00:34.012 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:33 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:00:34.012 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:33 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:00:34.012 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:33 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:00:34.012 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:33 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:00:34.012 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:33 smithi138 ceph-mon[81376]: Upgrade: Setting container_image for all mds 2024-05-17T22:00:34.012 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:33 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:00:34.013 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:33 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi107.drqody"}]: dispatch 2024-05-17T22:00:34.013 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:33 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi107.drqody"}]': finished 2024-05-17T22:00:34.013 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:33 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi107.idmbgj"}]: dispatch 2024-05-17T22:00:34.013 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:33 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi107.idmbgj"}]': finished 2024-05-17T22:00:34.013 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:33 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi138.gwbtyd"}]: dispatch 2024-05-17T22:00:34.013 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:33 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi138.gwbtyd"}]': finished 2024-05-17T22:00:34.013 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:33 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi138.zckrvs"}]: dispatch 2024-05-17T22:00:34.013 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:33 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi138.zckrvs"}]': finished 2024-05-17T22:00:34.013 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:33 smithi138 ceph-mon[81376]: Upgrade: Enabling allow_standby_replay on filesystem cephfs 2024-05-17T22:00:34.013 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:33 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]: dispatch 2024-05-17T22:00:34.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:33 smithi107 ceph-mon[93224]: pgmap v217: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 18 MiB/s rd, 4.3 KiB/s wr, 10 op/s 2024-05-17T22:00:34.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:33 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:00:34.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:33 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:00:34.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:33 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:00:34.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:33 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:00:34.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:33 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:00:34.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:33 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:00:34.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:33 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:00:34.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:33 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:00:34.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:33 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:00:34.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:33 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:00:34.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:33 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:00:34.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:33 smithi107 ceph-mon[93224]: Upgrade: Setting container_image for all mds 2024-05-17T22:00:34.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:33 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:00:34.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:33 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi107.drqody"}]: dispatch 2024-05-17T22:00:34.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:33 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi107.drqody"}]': finished 2024-05-17T22:00:34.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:33 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi107.idmbgj"}]: dispatch 2024-05-17T22:00:34.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:33 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi107.idmbgj"}]': finished 2024-05-17T22:00:34.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:33 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi138.gwbtyd"}]: dispatch 2024-05-17T22:00:34.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:33 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi138.gwbtyd"}]': finished 2024-05-17T22:00:34.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:33 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi138.zckrvs"}]: dispatch 2024-05-17T22:00:34.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:33 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi138.zckrvs"}]': finished 2024-05-17T22:00:34.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:33 smithi107 ceph-mon[93224]: Upgrade: Enabling allow_standby_replay on filesystem cephfs 2024-05-17T22:00:34.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:33 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]: dispatch 2024-05-17T22:00:35.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:34 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]': finished 2024-05-17T22:00:35.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:34 smithi107 ceph-mon[93224]: fsmap cephfs:1 {0=cephfs.smithi107.idmbgj=up:active} 3 up:standby 2024-05-17T22:00:35.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:34 smithi107 ceph-mon[93224]: fsmap cephfs:1 {0=cephfs.smithi107.idmbgj=up:active} 1 up:standby-replay 2 up:standby 2024-05-17T22:00:35.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:34 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:00:35.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:34 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:00:35.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:34 smithi107 ceph-mon[93224]: Upgrade: Setting container_image for all rgw 2024-05-17T22:00:35.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:34 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:00:35.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:34 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:00:35.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:34 smithi107 ceph-mon[93224]: Upgrade: Setting container_image for all rbd-mirror 2024-05-17T22:00:35.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:34 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:00:35.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:34 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:00:35.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:34 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:00:35.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:34 smithi107 ceph-mon[93224]: Upgrade: Setting container_image for all ceph-exporter 2024-05-17T22:00:35.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:34 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:00:35.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:34 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:00:35.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:34 smithi107 ceph-mon[93224]: Upgrade: Setting container_image for all iscsi 2024-05-17T22:00:35.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:34 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:00:35.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:34 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:00:35.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:34 smithi107 ceph-mon[93224]: Upgrade: Setting container_image for all nfs 2024-05-17T22:00:35.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:34 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:00:35.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:34 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:00:35.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:34 smithi107 ceph-mon[93224]: Upgrade: Setting container_image for all nvmeof 2024-05-17T22:00:35.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:34 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:00:35.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:34 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:00:35.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:34 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T22:00:35.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:34 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]': finished 2024-05-17T22:00:35.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:34 smithi138 ceph-mon[81376]: fsmap cephfs:1 {0=cephfs.smithi107.idmbgj=up:active} 3 up:standby 2024-05-17T22:00:35.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:34 smithi138 ceph-mon[81376]: fsmap cephfs:1 {0=cephfs.smithi107.idmbgj=up:active} 1 up:standby-replay 2 up:standby 2024-05-17T22:00:35.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:34 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:00:35.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:34 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:00:35.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:34 smithi138 ceph-mon[81376]: Upgrade: Setting container_image for all rgw 2024-05-17T22:00:35.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:34 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:00:35.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:34 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:00:35.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:34 smithi138 ceph-mon[81376]: Upgrade: Setting container_image for all rbd-mirror 2024-05-17T22:00:35.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:34 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:00:35.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:34 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:00:35.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:34 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:00:35.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:34 smithi138 ceph-mon[81376]: Upgrade: Setting container_image for all ceph-exporter 2024-05-17T22:00:35.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:34 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:00:35.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:34 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:00:35.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:34 smithi138 ceph-mon[81376]: Upgrade: Setting container_image for all iscsi 2024-05-17T22:00:35.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:34 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:00:35.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:34 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:00:35.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:34 smithi138 ceph-mon[81376]: Upgrade: Setting container_image for all nfs 2024-05-17T22:00:35.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:34 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:00:35.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:34 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:00:35.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:34 smithi138 ceph-mon[81376]: Upgrade: Setting container_image for all nvmeof 2024-05-17T22:00:35.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:34 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:00:35.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:34 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:00:35.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:34 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T22:00:36.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:35 smithi107 ceph-mon[93224]: pgmap v218: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 6.2 MiB/s rd, 4.3 KiB/s wr, 6 op/s 2024-05-17T22:00:36.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:35 smithi138 ceph-mon[81376]: pgmap v218: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 6.2 MiB/s rd, 4.3 KiB/s wr, 6 op/s 2024-05-17T22:00:37.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:37.940+0000 7f05ccb2a700 1 -- 172.21.15.107:0/9859133 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05c80fcf60 msgr2=0x7f05c80fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:37.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:37.940+0000 7f05ccb2a700 1 --2- 172.21.15.107:0/9859133 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05c80fcf60 0x7f05c80fd3d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f05bc004940 tx=0x7f05bc009e30 comp rx=0 tx=0).stop 2024-05-17T22:00:37.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:37.940+0000 7f05ccb2a700 1 -- 172.21.15.107:0/9859133 shutdown_connections 2024-05-17T22:00:37.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:37.940+0000 7f05ccb2a700 1 --2- 172.21.15.107:0/9859133 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05c80fcf60 0x7f05c80fd3d0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:37.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:37.940+0000 7f05ccb2a700 1 --2- 172.21.15.107:0/9859133 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05c80fbcb0 0x7f05c80fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:37.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:37.940+0000 7f05ccb2a700 1 -- 172.21.15.107:0/9859133 >> 172.21.15.107:0/9859133 conn(0x7f05c80f7430 msgr2=0x7f05c80f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:37.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:37.940+0000 7f05ccb2a700 1 -- 172.21.15.107:0/9859133 shutdown_connections 2024-05-17T22:00:37.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:37.940+0000 7f05ccb2a700 1 -- 172.21.15.107:0/9859133 wait complete. 2024-05-17T22:00:37.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:37.941+0000 7f05ccb2a700 1 Processor -- start 2024-05-17T22:00:37.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:37.941+0000 7f05ccb2a700 1 -- start start 2024-05-17T22:00:37.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:37.941+0000 7f05ccb2a700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05c80fbcb0 0x7f05c8110720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:37.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:37.941+0000 7f05ccb2a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05c80fcf60 0x7f05c8112c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:37.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:37.941+0000 7f05ccb2a700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05c8066330 con 0x7f05c80fbcb0 2024-05-17T22:00:37.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:37.941+0000 7f05ccb2a700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05c8066470 con 0x7f05c80fcf60 2024-05-17T22:00:37.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:37.941+0000 7f05c659c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05c80fbcb0 0x7f05c8110720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:37.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:37.941+0000 7f05c659c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05c80fbcb0 0x7f05c8110720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42306/0 (socket says 172.21.15.107:42306) 2024-05-17T22:00:37.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:37.941+0000 7f05c659c700 1 -- 172.21.15.107:0/3316057280 learned_addr learned my addr 172.21.15.107:0/3316057280 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T22:00:37.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:37.942+0000 7f05c5d9b700 1 --2- 172.21.15.107:0/3316057280 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05c80fcf60 0x7f05c8112c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:37.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:37.942+0000 7f05c659c700 1 -- 172.21.15.107:0/3316057280 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05c80fcf60 msgr2=0x7f05c8112c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:37.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:37.942+0000 7f05c659c700 1 --2- 172.21.15.107:0/3316057280 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05c80fcf60 0x7f05c8112c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:37.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:37.942+0000 7f05c659c700 1 -- 172.21.15.107:0/3316057280 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05bc005040 con 0x7f05c80fbcb0 2024-05-17T22:00:37.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:37.942+0000 7f05c659c700 1 --2- 172.21.15.107:0/3316057280 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05c80fbcb0 0x7f05c8110720 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f05b8002a50 tx=0x7f05b80046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:37.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:37.942+0000 7f05b6ffd700 1 -- 172.21.15.107:0/3316057280 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f05b8007cc0 con 0x7f05c80fbcb0 2024-05-17T22:00:37.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:37.942+0000 7f05ccb2a700 1 -- 172.21.15.107:0/3316057280 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05c8113210 con 0x7f05c80fbcb0 2024-05-17T22:00:37.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:37.942+0000 7f05b6ffd700 1 -- 172.21.15.107:0/3316057280 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f05b800f040 con 0x7f05c80fbcb0 2024-05-17T22:00:37.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:37.942+0000 7f05b6ffd700 1 -- 172.21.15.107:0/3316057280 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f05b8009640 con 0x7f05c80fbcb0 2024-05-17T22:00:37.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:37.942+0000 7f05ccb2a700 1 -- 172.21.15.107:0/3316057280 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05c81137d0 con 0x7f05c80fbcb0 2024-05-17T22:00:37.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:37.943+0000 7f05ccb2a700 1 -- 172.21.15.107:0/3316057280 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05c805f5c0 con 0x7f05c80fbcb0 2024-05-17T22:00:37.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:37.944+0000 7f05b6ffd700 1 -- 172.21.15.107:0/3316057280 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f05b80097a0 con 0x7f05c80fbcb0 2024-05-17T22:00:37.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:37.945+0000 7f05b6ffd700 1 --2- 172.21.15.107:0/3316057280 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f05b0078990 0x7f05b007ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:37.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:37.945+0000 7f05b6ffd700 1 -- 172.21.15.107:0/3316057280 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f05b809a5b0 con 0x7f05c80fbcb0 2024-05-17T22:00:37.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:37.945+0000 7f05c5d9b700 1 --2- 172.21.15.107:0/3316057280 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f05b0078990 0x7f05b007ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:37.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:37.945+0000 7f05c5d9b700 1 --2- 172.21.15.107:0/3316057280 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f05b0078990 0x7f05b007ae40 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f05bc00b3b0 tx=0x7f05bc009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:37.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:37.953+0000 7f05b6ffd700 1 -- 172.21.15.107:0/3316057280 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f05b8062170 con 0x7f05c80fbcb0 2024-05-17T22:00:38.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:37 smithi107 ceph-mon[93224]: pgmap v219: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 10 MiB/s rd, 4.3 KiB/s wr, 8 op/s 2024-05-17T22:00:38.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:38.187+0000 7f05ccb2a700 1 -- 172.21.15.107:0/3316057280 --> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f05c81012c0 con 0x7f05b0078990 2024-05-17T22:00:38.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:38.188+0000 7f05b6ffd700 1 -- 172.21.15.107:0/3316057280 <== mgr.34100 v2:172.21.15.107:6800/3205268226 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+441 (secure 0 0 0) 0x7f05c81012c0 con 0x7f05b0078990 2024-05-17T22:00:38.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:38.190+0000 7f05ccb2a700 1 -- 172.21.15.107:0/3316057280 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f05b0078990 msgr2=0x7f05b007ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:38.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:38.190+0000 7f05ccb2a700 1 --2- 172.21.15.107:0/3316057280 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f05b0078990 0x7f05b007ae40 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f05bc00b3b0 tx=0x7f05bc009e30 comp rx=0 tx=0).stop 2024-05-17T22:00:38.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:38.190+0000 7f05ccb2a700 1 -- 172.21.15.107:0/3316057280 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05c80fbcb0 msgr2=0x7f05c8110720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:38.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:38.190+0000 7f05ccb2a700 1 --2- 172.21.15.107:0/3316057280 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05c80fbcb0 0x7f05c8110720 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f05b8002a50 tx=0x7f05b80046b0 comp rx=0 tx=0).stop 2024-05-17T22:00:38.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:38.190+0000 7f05ccb2a700 1 -- 172.21.15.107:0/3316057280 shutdown_connections 2024-05-17T22:00:38.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:38.190+0000 7f05ccb2a700 1 --2- 172.21.15.107:0/3316057280 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f05b0078990 0x7f05b007ae40 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:38.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:38.190+0000 7f05ccb2a700 1 --2- 172.21.15.107:0/3316057280 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05c80fbcb0 0x7f05c8110720 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:38.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:38.190+0000 7f05ccb2a700 1 --2- 172.21.15.107:0/3316057280 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05c80fcf60 0x7f05c8112c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:38.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:38.191+0000 7f05ccb2a700 1 -- 172.21.15.107:0/3316057280 >> 172.21.15.107:0/3316057280 conn(0x7f05c80f7430 msgr2=0x7f05c80f8b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:38.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:38.191+0000 7f05ccb2a700 1 -- 172.21.15.107:0/3316057280 shutdown_connections 2024-05-17T22:00:38.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:38.191+0000 7f05ccb2a700 1 -- 172.21.15.107:0/3316057280 wait complete. 2024-05-17T22:00:38.195 INFO:teuthology.orchestra.run.smithi107.stdout:true 2024-05-17T22:00:38.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:38.300+0000 7f122f712700 1 -- 172.21.15.107:0/2052821603 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12280662c0 msgr2=0x7f12280ff9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:38.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:38.300+0000 7f122f712700 1 --2- 172.21.15.107:0/2052821603 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12280662c0 0x7f12280ff9a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1224004940 tx=0x7f1224009e30 comp rx=0 tx=0).stop 2024-05-17T22:00:38.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:38.300+0000 7f122f712700 1 -- 172.21.15.107:0/2052821603 shutdown_connections 2024-05-17T22:00:38.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:38.300+0000 7f122f712700 1 --2- 172.21.15.107:0/2052821603 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f12280ffee0 0x7f1228102300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:38.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:38.300+0000 7f122f712700 1 --2- 172.21.15.107:0/2052821603 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12280662c0 0x7f12280ff9a0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:38.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:38.300+0000 7f122f712700 1 -- 172.21.15.107:0/2052821603 >> 172.21.15.107:0/2052821603 conn(0x7f12280f7430 msgr2=0x7f12280f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:38.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:38.300+0000 7f122f712700 1 -- 172.21.15.107:0/2052821603 shutdown_connections 2024-05-17T22:00:38.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:38.300+0000 7f122f712700 1 -- 172.21.15.107:0/2052821603 wait complete. 2024-05-17T22:00:38.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:38.301+0000 7f122f712700 1 Processor -- start 2024-05-17T22:00:38.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:38.301+0000 7f122f712700 1 -- start start 2024-05-17T22:00:38.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:38.301+0000 7f122f712700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f12280662c0 0x7f122810da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:38.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:38.301+0000 7f122f712700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12280ffee0 0x7f122810df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:38.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:38.301+0000 7f122f712700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f122810e570 con 0x7f12280ffee0 2024-05-17T22:00:38.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:38.301+0000 7f122f712700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f122810edc0 con 0x7f12280662c0 2024-05-17T22:00:38.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:38.301+0000 7f122ccad700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12280ffee0 0x7f122810df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:38.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:38.301+0000 7f122ccad700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12280ffee0 0x7f122810df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42326/0 (socket says 172.21.15.107:42326) 2024-05-17T22:00:38.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:38.301+0000 7f122ccad700 1 -- 172.21.15.107:0/3171669332 learned_addr learned my addr 172.21.15.107:0/3171669332 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T22:00:38.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:38.301+0000 7f122d4ae700 1 --2- 172.21.15.107:0/3171669332 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f12280662c0 0x7f122810da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:38.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:38.302+0000 7f122ccad700 1 -- 172.21.15.107:0/3171669332 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f12280662c0 msgr2=0x7f122810da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:38.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:38.302+0000 7f122ccad700 1 --2- 172.21.15.107:0/3171669332 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f12280662c0 0x7f122810da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:38.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:38.302+0000 7f122ccad700 1 -- 172.21.15.107:0/3171669332 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1224005040 con 0x7f12280ffee0 2024-05-17T22:00:38.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:38.302+0000 7f122d4ae700 1 --2- 172.21.15.107:0/3171669332 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f12280662c0 0x7f122810da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:00:38.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:38.302+0000 7f122ccad700 1 --2- 172.21.15.107:0/3171669332 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12280ffee0 0x7f122810df70 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f121800c830 tx=0x7f1218004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:38.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:38.302+0000 7f121dffb700 1 -- 172.21.15.107:0/3171669332 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f121800a760 con 0x7f12280ffee0 2024-05-17T22:00:38.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:38.302+0000 7f121dffb700 1 -- 172.21.15.107:0/3171669332 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f121800ad90 con 0x7f12280ffee0 2024-05-17T22:00:38.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:38.302+0000 7f122f712700 1 -- 172.21.15.107:0/3171669332 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f122810f0a0 con 0x7f12280ffee0 2024-05-17T22:00:38.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:38.303+0000 7f121dffb700 1 -- 172.21.15.107:0/3171669332 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f12180113a0 con 0x7f12280ffee0 2024-05-17T22:00:38.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:38.303+0000 7f122f712700 1 -- 172.21.15.107:0/3171669332 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f122810f5e0 con 0x7f12280ffee0 2024-05-17T22:00:38.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:38.304+0000 7f122f712700 1 -- 172.21.15.107:0/3171669332 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f122805f5c0 con 0x7f12280ffee0 2024-05-17T22:00:38.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:38.305+0000 7f121dffb700 1 -- 172.21.15.107:0/3171669332 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f121800a8c0 con 0x7f12280ffee0 2024-05-17T22:00:38.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:38.305+0000 7f121dffb700 1 --2- 172.21.15.107:0/3171669332 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f1214078770 0x7f121407ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:38.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:38.306+0000 7f121dffb700 1 -- 172.21.15.107:0/3171669332 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f121809a850 con 0x7f12280ffee0 2024-05-17T22:00:38.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:38.306+0000 7f122d4ae700 1 --2- 172.21.15.107:0/3171669332 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f1214078770 0x7f121407ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:38.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:38.306+0000 7f122d4ae700 1 --2- 172.21.15.107:0/3171669332 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f1214078770 0x7f121407ac20 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1224004890 tx=0x7f1224009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:38.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:38.310+0000 7f121dffb700 1 -- 172.21.15.107:0/3171669332 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f1218062490 con 0x7f12280ffee0 2024-05-17T22:00:38.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:37 smithi138 ceph-mon[81376]: pgmap v219: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 10 MiB/s rd, 4.3 KiB/s wr, 8 op/s 2024-05-17T22:00:38.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:38.582+0000 7f122f712700 1 -- 172.21.15.107:0/3171669332 --> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f122810f860 con 0x7f1214078770 2024-05-17T22:00:38.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:38.583+0000 7f121dffb700 1 -- 172.21.15.107:0/3171669332 <== mgr.34100 v2:172.21.15.107:6800/3205268226 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+441 (secure 0 0 0) 0x7f122810f860 con 0x7f1214078770 2024-05-17T22:00:38.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:38.585+0000 7f122f712700 1 -- 172.21.15.107:0/3171669332 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f1214078770 msgr2=0x7f121407ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:38.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:38.585+0000 7f122f712700 1 --2- 172.21.15.107:0/3171669332 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f1214078770 0x7f121407ac20 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1224004890 tx=0x7f1224009e30 comp rx=0 tx=0).stop 2024-05-17T22:00:38.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:38.585+0000 7f122f712700 1 -- 172.21.15.107:0/3171669332 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12280ffee0 msgr2=0x7f122810df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:38.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:38.585+0000 7f122f712700 1 --2- 172.21.15.107:0/3171669332 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12280ffee0 0x7f122810df70 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f121800c830 tx=0x7f1218004470 comp rx=0 tx=0).stop 2024-05-17T22:00:38.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:38.586+0000 7f122f712700 1 -- 172.21.15.107:0/3171669332 shutdown_connections 2024-05-17T22:00:38.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:38.586+0000 7f122f712700 1 --2- 172.21.15.107:0/3171669332 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f1214078770 0x7f121407ac20 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:38.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:38.586+0000 7f122f712700 1 --2- 172.21.15.107:0/3171669332 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f12280662c0 0x7f122810da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:38.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:38.586+0000 7f122f712700 1 --2- 172.21.15.107:0/3171669332 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12280ffee0 0x7f122810df70 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:38.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:38.586+0000 7f122f712700 1 -- 172.21.15.107:0/3171669332 >> 172.21.15.107:0/3171669332 conn(0x7f12280f7430 msgr2=0x7f12280f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:38.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:38.586+0000 7f122f712700 1 -- 172.21.15.107:0/3171669332 shutdown_connections 2024-05-17T22:00:38.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:38.586+0000 7f122f712700 1 -- 172.21.15.107:0/3171669332 wait complete. 2024-05-17T22:00:38.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:38.692+0000 7f5132e3c700 1 -- 172.21.15.107:0/2542894568 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f512c0fe080 msgr2=0x7f512c0fe4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:38.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:38.692+0000 7f5132e3c700 1 --2- 172.21.15.107:0/2542894568 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f512c0fe080 0x7f512c0fe4f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5118004940 tx=0x7f5118009e30 comp rx=0 tx=0).stop 2024-05-17T22:00:38.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:38.692+0000 7f5132e3c700 1 -- 172.21.15.107:0/2542894568 shutdown_connections 2024-05-17T22:00:38.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:38.692+0000 7f5132e3c700 1 --2- 172.21.15.107:0/2542894568 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f512c0fe080 0x7f512c0fe4f0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:38.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:38.692+0000 7f5132e3c700 1 --2- 172.21.15.107:0/2542894568 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f512c0fce80 0x7f512c0fd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:38.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:38.692+0000 7f5132e3c700 1 -- 172.21.15.107:0/2542894568 >> 172.21.15.107:0/2542894568 conn(0x7f512c0f8590 msgr2=0x7f512c0fa9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:38.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:38.692+0000 7f5132e3c700 1 -- 172.21.15.107:0/2542894568 shutdown_connections 2024-05-17T22:00:38.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:38.692+0000 7f5132e3c700 1 -- 172.21.15.107:0/2542894568 wait complete. 2024-05-17T22:00:38.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:38.693+0000 7f5132e3c700 1 Processor -- start 2024-05-17T22:00:38.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:38.693+0000 7f5132e3c700 1 -- start start 2024-05-17T22:00:38.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:38.693+0000 7f5132e3c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f512c0fce80 0x7f512c10da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:38.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:38.693+0000 7f5132e3c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f512c0fe080 0x7f512c10df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:38.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:38.693+0000 7f5132e3c700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f512c10e570 con 0x7f512c0fce80 2024-05-17T22:00:38.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:38.693+0000 7f5132e3c700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f512c10edc0 con 0x7f512c0fe080 2024-05-17T22:00:38.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:38.693+0000 7f5130bd8700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f512c0fce80 0x7f512c10da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:38.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:38.693+0000 7f5130bd8700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f512c0fce80 0x7f512c10da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42332/0 (socket says 172.21.15.107:42332) 2024-05-17T22:00:38.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:38.693+0000 7f5130bd8700 1 -- 172.21.15.107:0/1778133424 learned_addr learned my addr 172.21.15.107:0/1778133424 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T22:00:38.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:38.693+0000 7f512bfff700 1 --2- 172.21.15.107:0/1778133424 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f512c0fe080 0x7f512c10df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:38.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:38.694+0000 7f5130bd8700 1 -- 172.21.15.107:0/1778133424 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f512c0fe080 msgr2=0x7f512c10df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:38.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:38.694+0000 7f5130bd8700 1 --2- 172.21.15.107:0/1778133424 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f512c0fe080 0x7f512c10df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:38.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:38.694+0000 7f5130bd8700 1 -- 172.21.15.107:0/1778133424 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5118005040 con 0x7f512c0fce80 2024-05-17T22:00:38.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:38.694+0000 7f5130bd8700 1 --2- 172.21.15.107:0/1778133424 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f512c0fce80 0x7f512c10da30 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5120002a80 tx=0x7f5120004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:38.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:38.694+0000 7f51297fa700 1 -- 172.21.15.107:0/1778133424 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5120007cc0 con 0x7f512c0fce80 2024-05-17T22:00:38.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:38.694+0000 7f5132e3c700 1 -- 172.21.15.107:0/1778133424 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f512c10f0a0 con 0x7f512c0fce80 2024-05-17T22:00:38.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:38.694+0000 7f51297fa700 1 -- 172.21.15.107:0/1778133424 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f512000f040 con 0x7f512c0fce80 2024-05-17T22:00:38.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:38.694+0000 7f51297fa700 1 -- 172.21.15.107:0/1778133424 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f512000da30 con 0x7f512c0fce80 2024-05-17T22:00:38.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:38.694+0000 7f5132e3c700 1 -- 172.21.15.107:0/1778133424 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f512c10f5e0 con 0x7f512c0fce80 2024-05-17T22:00:38.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:38.695+0000 7f5132e3c700 1 -- 172.21.15.107:0/1778133424 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f512c05f5c0 con 0x7f512c0fce80 2024-05-17T22:00:38.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:38.696+0000 7f51297fa700 1 -- 172.21.15.107:0/1778133424 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f512000db90 con 0x7f512c0fce80 2024-05-17T22:00:38.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:38.696+0000 7f51297fa700 1 --2- 172.21.15.107:0/1778133424 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f511c078990 0x7f511c07ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:38.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:38.697+0000 7f51297fa700 1 -- 172.21.15.107:0/1778133424 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f512009a530 con 0x7f512c0fce80 2024-05-17T22:00:38.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:38.697+0000 7f512bfff700 1 --2- 172.21.15.107:0/1778133424 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f511c078990 0x7f511c07ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:38.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:38.697+0000 7f512bfff700 1 --2- 172.21.15.107:0/1778133424 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f511c078990 0x7f511c07ae40 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f511800b970 tx=0x7f5118009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:38.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:38.701+0000 7f51297fa700 1 -- 172.21.15.107:0/1778133424 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f51200620f0 con 0x7f512c0fce80 2024-05-17T22:00:38.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:38.926+0000 7f5132e3c700 1 -- 172.21.15.107:0/1778133424 --> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f512c10f9d0 con 0x7f511c078990 2024-05-17T22:00:38.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:38.937+0000 7f51297fa700 1 -- 172.21.15.107:0/1778133424 <== mgr.34100 v2:172.21.15.107:6800/3205268226 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f512c10f9d0 con 0x7f511c078990 2024-05-17T22:00:38.927 INFO:teuthology.orchestra.run.smithi107.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-17T22:00:38.927 INFO:teuthology.orchestra.run.smithi107.stdout:alertmanager.smithi107 smithi107 *:9093,9094 running (8m) 55s ago 16m 24.3M - 0.25.0 c8568f914cd2 bac2a944ae42 2024-05-17T22:00:38.927 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi107 smithi107 running (4m) 55s ago 17m 6669k - 19.0.0-3794-g931809a9 8880f7613d02 face35de205b 2024-05-17T22:00:38.927 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi138 smithi138 running (4m) 11s ago 15m 6665k - 19.0.0-3794-g931809a9 8880f7613d02 ad59e33e6d43 2024-05-17T22:00:38.927 INFO:teuthology.orchestra.run.smithi107.stdout:grafana.smithi107 smithi107 *:3000 running (8m) 55s ago 16m 86.4M - 9.4.7 954c08fa6188 8c747e777bc5 2024-05-17T22:00:38.928 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.drqody smithi107 running (59s) 55s ago 11m 81.2M - 19.0.0-3794-g931809a9 8880f7613d02 4f02280f8312 2024-05-17T22:00:38.928 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.idmbgj smithi107 running (77s) 55s ago 11m 82.6M - 19.0.0-3794-g931809a9 8880f7613d02 d541bff38965 2024-05-17T22:00:38.928 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi138.gwbtyd smithi138 running (34s) 11s ago 11m 80.1M - 19.0.0-3794-g931809a9 8880f7613d02 7beb9bd77fc8 2024-05-17T22:00:38.928 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi138.zckrvs smithi138 running (13s) 11s ago 11m 79.1M - 19.0.0-3794-g931809a9 8880f7613d02 17edb3a75f7b 2024-05-17T22:00:38.928 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi107.xhhyiw smithi107 *:8443,9283,8765 running (7m) 55s ago 18m 592M - 19.0.0-3794-g931809a9 8880f7613d02 5acf3e511342 2024-05-17T22:00:38.928 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi138.napnll smithi138 *:8443,9283,8765 running (6m) 11s ago 15m 534M - 19.0.0-3794-g931809a9 8880f7613d02 d627761dbd0d 2024-05-17T22:00:38.928 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi107 smithi107 running (6m) 55s ago 18m 118M 2048M 19.0.0-3794-g931809a9 8880f7613d02 9717f3169f2f 2024-05-17T22:00:38.928 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi138 smithi138 running (5m) 11s ago 15m 107M 2048M 19.0.0-3794-g931809a9 8880f7613d02 55aeed62bf4d 2024-05-17T22:00:38.928 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi107 smithi107 *:9100 running (8m) 55s ago 16m 17.7M - 1.5.0 0da6a335fe13 49bec71f5487 2024-05-17T22:00:38.928 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi138 smithi138 *:9100 running (7m) 11s ago 15m 15.7M - 1.5.0 0da6a335fe13 e52824376658 2024-05-17T22:00:38.928 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0 smithi107 running (4m) 55s ago 14m 161M 1557M 19.0.0-3794-g931809a9 8880f7613d02 b319bc93500d 2024-05-17T22:00:38.928 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1 smithi107 running (3m) 55s ago 14m 154M 1557M 19.0.0-3794-g931809a9 8880f7613d02 f94b206be6eb 2024-05-17T22:00:38.928 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2 smithi107 running (3m) 55s ago 14m 144M 1557M 19.0.0-3794-g931809a9 8880f7613d02 add7015647af 2024-05-17T22:00:38.928 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3 smithi138 running (2m) 11s ago 13m 173M 2581M 19.0.0-3794-g931809a9 8880f7613d02 416273d327c7 2024-05-17T22:00:38.928 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4 smithi138 running (2m) 11s ago 13m 158M 2581M 19.0.0-3794-g931809a9 8880f7613d02 e33f411da8e6 2024-05-17T22:00:38.929 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5 smithi138 running (94s) 11s ago 12m 158M 2581M 19.0.0-3794-g931809a9 8880f7613d02 63810d431961 2024-05-17T22:00:38.929 INFO:teuthology.orchestra.run.smithi107.stdout:prometheus.smithi107 smithi107 *:9095 running (6m) 55s ago 16m 67.2M - 2.43.0 a07b618ecd1d bf5f609bbd6e 2024-05-17T22:00:38.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:38.939+0000 7f5132e3c700 1 -- 172.21.15.107:0/1778133424 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f511c078990 msgr2=0x7f511c07ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:38.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:38.939+0000 7f5132e3c700 1 --2- 172.21.15.107:0/1778133424 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f511c078990 0x7f511c07ae40 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f511800b970 tx=0x7f5118009e30 comp rx=0 tx=0).stop 2024-05-17T22:00:38.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:38.939+0000 7f5132e3c700 1 -- 172.21.15.107:0/1778133424 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f512c0fce80 msgr2=0x7f512c10da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:38.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:38.939+0000 7f5132e3c700 1 --2- 172.21.15.107:0/1778133424 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f512c0fce80 0x7f512c10da30 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5120002a80 tx=0x7f5120004600 comp rx=0 tx=0).stop 2024-05-17T22:00:38.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:38.939+0000 7f5132e3c700 1 -- 172.21.15.107:0/1778133424 shutdown_connections 2024-05-17T22:00:38.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:38.939+0000 7f5132e3c700 1 --2- 172.21.15.107:0/1778133424 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f511c078990 0x7f511c07ae40 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:38.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:38.939+0000 7f5132e3c700 1 --2- 172.21.15.107:0/1778133424 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f512c0fce80 0x7f512c10da30 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:38.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:38.939+0000 7f5132e3c700 1 --2- 172.21.15.107:0/1778133424 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f512c0fe080 0x7f512c10df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:38.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:38.939+0000 7f5132e3c700 1 -- 172.21.15.107:0/1778133424 >> 172.21.15.107:0/1778133424 conn(0x7f512c0f8590 msgr2=0x7f512c0f9ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:38.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:38.939+0000 7f5132e3c700 1 -- 172.21.15.107:0/1778133424 shutdown_connections 2024-05-17T22:00:38.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:38.939+0000 7f5132e3c700 1 -- 172.21.15.107:0/1778133424 wait complete. 2024-05-17T22:00:39.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.049+0000 7fb89f280700 1 -- 172.21.15.107:0/3155538539 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8981022e0 msgr2=0x7fb8981046c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:39.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.049+0000 7fb89f280700 1 --2- 172.21.15.107:0/3155538539 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8981022e0 0x7fb8981046c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb888004940 tx=0x7fb888009e30 comp rx=0 tx=0).stop 2024-05-17T22:00:39.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.049+0000 7fb89f280700 1 -- 172.21.15.107:0/3155538539 shutdown_connections 2024-05-17T22:00:39.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.049+0000 7fb89f280700 1 --2- 172.21.15.107:0/3155538539 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8981022e0 0x7fb8981046c0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:39.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.049+0000 7fb89f280700 1 --2- 172.21.15.107:0/3155538539 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb8980662c0 0x7fb898101da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:39.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.049+0000 7fb89f280700 1 -- 172.21.15.107:0/3155538539 >> 172.21.15.107:0/3155538539 conn(0x7fb8980f7430 msgr2=0x7fb8980f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:39.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.049+0000 7fb89f280700 1 -- 172.21.15.107:0/3155538539 shutdown_connections 2024-05-17T22:00:39.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.049+0000 7fb89f280700 1 -- 172.21.15.107:0/3155538539 wait complete. 2024-05-17T22:00:39.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.049+0000 7fb89f280700 1 Processor -- start 2024-05-17T22:00:39.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.050+0000 7fb89f280700 1 -- start start 2024-05-17T22:00:39.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.050+0000 7fb89f280700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb8980662c0 0x7fb89810fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:39.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.050+0000 7fb89f280700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8981022e0 0x7fb898110120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:39.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.050+0000 7fb89f280700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb898110720 con 0x7fb8981022e0 2024-05-17T22:00:39.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.050+0000 7fb89f280700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb898110f20 con 0x7fb8980662c0 2024-05-17T22:00:39.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.050+0000 7fb89c81b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8981022e0 0x7fb898110120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:39.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.050+0000 7fb89c81b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8981022e0 0x7fb898110120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42350/0 (socket says 172.21.15.107:42350) 2024-05-17T22:00:39.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.050+0000 7fb89c81b700 1 -- 172.21.15.107:0/3087046693 learned_addr learned my addr 172.21.15.107:0/3087046693 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T22:00:39.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.050+0000 7fb89c81b700 1 -- 172.21.15.107:0/3087046693 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb8980662c0 msgr2=0x7fb89810fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:39.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.050+0000 7fb89c81b700 1 --2- 172.21.15.107:0/3087046693 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb8980662c0 0x7fb89810fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:39.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.050+0000 7fb89c81b700 1 -- 172.21.15.107:0/3087046693 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb888005040 con 0x7fb8981022e0 2024-05-17T22:00:39.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.050+0000 7fb89c81b700 1 --2- 172.21.15.107:0/3087046693 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8981022e0 0x7fb898110120 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb888013870 tx=0x7fb888009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:39.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.051+0000 7fb88dffb700 1 -- 172.21.15.107:0/3087046693 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb88800e070 con 0x7fb8981022e0 2024-05-17T22:00:39.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.051+0000 7fb89f280700 1 -- 172.21.15.107:0/3087046693 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8981111a0 con 0x7fb8981022e0 2024-05-17T22:00:39.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.051+0000 7fb88dffb700 1 -- 172.21.15.107:0/3087046693 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb88801c400 con 0x7fb8981022e0 2024-05-17T22:00:39.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.051+0000 7fb88dffb700 1 -- 172.21.15.107:0/3087046693 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb88800b630 con 0x7fb8981022e0 2024-05-17T22:00:39.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.051+0000 7fb89f280700 1 -- 172.21.15.107:0/3087046693 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb898111680 con 0x7fb8981022e0 2024-05-17T22:00:39.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.053+0000 7fb89f280700 1 -- 172.21.15.107:0/3087046693 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb89805f5c0 con 0x7fb8981022e0 2024-05-17T22:00:39.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.054+0000 7fb88dffb700 1 -- 172.21.15.107:0/3087046693 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7fb88801c570 con 0x7fb8981022e0 2024-05-17T22:00:39.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.054+0000 7fb88dffb700 1 --2- 172.21.15.107:0/3087046693 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fb884078980 0x7fb88407ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:39.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.054+0000 7fb88dffb700 1 -- 172.21.15.107:0/3087046693 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7fb88809c9f0 con 0x7fb8981022e0 2024-05-17T22:00:39.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.054+0000 7fb89d01c700 1 --2- 172.21.15.107:0/3087046693 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fb884078980 0x7fb88407ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:39.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.055+0000 7fb89d01c700 1 --2- 172.21.15.107:0/3087046693 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fb884078980 0x7fb88407ae30 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb898066000 tx=0x7fb89400b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:39.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.062+0000 7fb88dffb700 1 -- 172.21.15.107:0/3087046693 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7fb8880646d0 con 0x7fb8981022e0 2024-05-17T22:00:39.100 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:38 smithi107 ceph-mon[93224]: from='client.34478 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:00:39.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:38 smithi138 ceph-mon[81376]: from='client.34478 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:00:39.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.396+0000 7fb89f280700 1 -- 172.21.15.107:0/3087046693 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb898107ab0 con 0x7fb8981022e0 2024-05-17T22:00:39.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.397+0000 7fb88dffb700 1 -- 172.21.15.107:0/3087046693 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fb888064070 con 0x7fb8981022e0 2024-05-17T22:00:39.387 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-05-17T22:00:39.387 INFO:teuthology.orchestra.run.smithi107.stdout: "mon": { 2024-05-17T22:00:39.387 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev)": 2 2024-05-17T22:00:39.387 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-05-17T22:00:39.387 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr": { 2024-05-17T22:00:39.388 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev)": 2 2024-05-17T22:00:39.388 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-05-17T22:00:39.388 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": { 2024-05-17T22:00:39.388 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev)": 6 2024-05-17T22:00:39.388 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-05-17T22:00:39.388 INFO:teuthology.orchestra.run.smithi107.stdout: "mds": { 2024-05-17T22:00:39.388 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev)": 4 2024-05-17T22:00:39.388 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-05-17T22:00:39.388 INFO:teuthology.orchestra.run.smithi107.stdout: "overall": { 2024-05-17T22:00:39.388 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev)": 14 2024-05-17T22:00:39.388 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-05-17T22:00:39.388 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-05-17T22:00:39.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.399+0000 7fb89f280700 1 -- 172.21.15.107:0/3087046693 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fb884078980 msgr2=0x7fb88407ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:39.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.399+0000 7fb89f280700 1 --2- 172.21.15.107:0/3087046693 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fb884078980 0x7fb88407ae30 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb898066000 tx=0x7fb89400b040 comp rx=0 tx=0).stop 2024-05-17T22:00:39.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.399+0000 7fb89f280700 1 -- 172.21.15.107:0/3087046693 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8981022e0 msgr2=0x7fb898110120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:39.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.399+0000 7fb89f280700 1 --2- 172.21.15.107:0/3087046693 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8981022e0 0x7fb898110120 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb888013870 tx=0x7fb888009e30 comp rx=0 tx=0).stop 2024-05-17T22:00:39.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.399+0000 7fb89f280700 1 -- 172.21.15.107:0/3087046693 shutdown_connections 2024-05-17T22:00:39.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.399+0000 7fb89f280700 1 --2- 172.21.15.107:0/3087046693 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fb884078980 0x7fb88407ae30 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:39.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.399+0000 7fb89f280700 1 --2- 172.21.15.107:0/3087046693 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb8980662c0 0x7fb89810fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:39.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.399+0000 7fb89f280700 1 --2- 172.21.15.107:0/3087046693 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8981022e0 0x7fb898110120 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:39.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.399+0000 7fb89f280700 1 -- 172.21.15.107:0/3087046693 >> 172.21.15.107:0/3087046693 conn(0x7fb8980f7430 msgr2=0x7fb8980f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:39.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.399+0000 7fb89f280700 1 -- 172.21.15.107:0/3087046693 shutdown_connections 2024-05-17T22:00:39.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.399+0000 7fb89f280700 1 -- 172.21.15.107:0/3087046693 wait complete. 2024-05-17T22:00:39.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.508+0000 7f5dba311700 1 -- 172.21.15.107:0/345430919 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5db41022e0 msgr2=0x7f5db41046c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:39.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.508+0000 7f5dba311700 1 --2- 172.21.15.107:0/345430919 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5db41022e0 0x7f5db41046c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5da8004940 tx=0x7f5da8009e30 comp rx=0 tx=0).stop 2024-05-17T22:00:39.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.508+0000 7f5dba311700 1 -- 172.21.15.107:0/345430919 shutdown_connections 2024-05-17T22:00:39.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.508+0000 7f5dba311700 1 --2- 172.21.15.107:0/345430919 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5db41022e0 0x7f5db41046c0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:39.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.508+0000 7f5dba311700 1 --2- 172.21.15.107:0/345430919 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5db40662c0 0x7f5db4101da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:39.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.508+0000 7f5dba311700 1 -- 172.21.15.107:0/345430919 >> 172.21.15.107:0/345430919 conn(0x7f5db40f7430 msgr2=0x7f5db40f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:39.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.508+0000 7f5dba311700 1 -- 172.21.15.107:0/345430919 shutdown_connections 2024-05-17T22:00:39.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.508+0000 7f5dba311700 1 -- 172.21.15.107:0/345430919 wait complete. 2024-05-17T22:00:39.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.508+0000 7f5dba311700 1 Processor -- start 2024-05-17T22:00:39.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.509+0000 7f5dba311700 1 -- start start 2024-05-17T22:00:39.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.509+0000 7f5dba311700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5db40662c0 0x7f5db410fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:39.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.509+0000 7f5dba311700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5db41022e0 0x7f5db4110140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:39.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.509+0000 7f5dba311700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5db4110740 con 0x7f5db41022e0 2024-05-17T22:00:39.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.509+0000 7f5dba311700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5db4110f40 con 0x7f5db40662c0 2024-05-17T22:00:39.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.509+0000 7f5db37fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5db41022e0 0x7f5db4110140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:39.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.509+0000 7f5db37fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5db41022e0 0x7f5db4110140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42370/0 (socket says 172.21.15.107:42370) 2024-05-17T22:00:39.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.509+0000 7f5db37fe700 1 -- 172.21.15.107:0/2935271538 learned_addr learned my addr 172.21.15.107:0/2935271538 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T22:00:39.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.509+0000 7f5db3fff700 1 --2- 172.21.15.107:0/2935271538 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5db40662c0 0x7f5db410fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:39.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.509+0000 7f5db37fe700 1 -- 172.21.15.107:0/2935271538 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5db40662c0 msgr2=0x7f5db410fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:39.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.509+0000 7f5db37fe700 1 --2- 172.21.15.107:0/2935271538 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5db40662c0 0x7f5db410fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:39.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.509+0000 7f5db37fe700 1 -- 172.21.15.107:0/2935271538 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5da8005040 con 0x7f5db41022e0 2024-05-17T22:00:39.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.510+0000 7f5db37fe700 1 --2- 172.21.15.107:0/2935271538 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5db41022e0 0x7f5db4110140 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5da8013870 tx=0x7f5da8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:39.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.510+0000 7f5db0ff9700 1 -- 172.21.15.107:0/2935271538 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5da8021070 con 0x7f5db41022e0 2024-05-17T22:00:39.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.510+0000 7f5db0ff9700 1 -- 172.21.15.107:0/2935271538 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5da8009430 con 0x7f5db41022e0 2024-05-17T22:00:39.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.510+0000 7f5dba311700 1 -- 172.21.15.107:0/2935271538 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5db41111c0 con 0x7f5db41022e0 2024-05-17T22:00:39.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.510+0000 7f5db0ff9700 1 -- 172.21.15.107:0/2935271538 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5da800e070 con 0x7f5db41022e0 2024-05-17T22:00:39.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.510+0000 7f5dba311700 1 -- 172.21.15.107:0/2935271538 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5db4111700 con 0x7f5db41022e0 2024-05-17T22:00:39.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.511+0000 7f5dba311700 1 -- 172.21.15.107:0/2935271538 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5db405f5c0 con 0x7f5db41022e0 2024-05-17T22:00:39.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.511+0000 7f5db0ff9700 1 -- 172.21.15.107:0/2935271538 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f5da801c750 con 0x7f5db41022e0 2024-05-17T22:00:39.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.512+0000 7f5db0ff9700 1 --2- 172.21.15.107:0/2935271538 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f5d9c078910 0x7f5d9c07adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:39.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.512+0000 7f5db3fff700 1 --2- 172.21.15.107:0/2935271538 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f5d9c078910 0x7f5d9c07adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:39.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.512+0000 7f5db0ff9700 1 -- 172.21.15.107:0/2935271538 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f5da809b930 con 0x7f5db41022e0 2024-05-17T22:00:39.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.513+0000 7f5db3fff700 1 --2- 172.21.15.107:0/2935271538 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f5d9c078910 0x7f5d9c07adc0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5da4009b80 tx=0x7f5da400b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:39.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.517+0000 7f5db0ff9700 1 -- 172.21.15.107:0/2935271538 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f5da8063570 con 0x7f5db41022e0 2024-05-17T22:00:39.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.777+0000 7f5dba311700 1 -- 172.21.15.107:0/2935271538 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f5db404bed0 con 0x7f5db41022e0 2024-05-17T22:00:39.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.777+0000 7f5db0ff9700 1 -- 172.21.15.107:0/2935271538 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 35 v35) v1 ==== 76+0+1968 (secure 0 0 0) 0x7f5da8004490 con 0x7f5db41022e0 2024-05-17T22:00:39.768 INFO:teuthology.orchestra.run.smithi107.stdout:e35 2024-05-17T22:00:39.768 INFO:teuthology.orchestra.run.smithi107.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-17T22:00:39.768 INFO:teuthology.orchestra.run.smithi107.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-17T22:00:39.768 INFO:teuthology.orchestra.run.smithi107.stdout:legacy client fscid: 1 2024-05-17T22:00:39.768 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T22:00:39.769 INFO:teuthology.orchestra.run.smithi107.stdout:Filesystem 'cephfs' (1) 2024-05-17T22:00:39.769 INFO:teuthology.orchestra.run.smithi107.stdout:fs_name cephfs 2024-05-17T22:00:39.769 INFO:teuthology.orchestra.run.smithi107.stdout:epoch 35 2024-05-17T22:00:39.769 INFO:teuthology.orchestra.run.smithi107.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-17T22:00:39.769 INFO:teuthology.orchestra.run.smithi107.stdout:created 2024-05-17T21:49:16.836333+0000 2024-05-17T22:00:39.769 INFO:teuthology.orchestra.run.smithi107.stdout:modified 2024-05-17T22:00:33.850863+0000 2024-05-17T22:00:39.769 INFO:teuthology.orchestra.run.smithi107.stdout:tableserver 0 2024-05-17T22:00:39.769 INFO:teuthology.orchestra.run.smithi107.stdout:root 0 2024-05-17T22:00:39.769 INFO:teuthology.orchestra.run.smithi107.stdout:session_timeout 60 2024-05-17T22:00:39.769 INFO:teuthology.orchestra.run.smithi107.stdout:session_autoclose 300 2024-05-17T22:00:39.769 INFO:teuthology.orchestra.run.smithi107.stdout:max_file_size 1099511627776 2024-05-17T22:00:39.769 INFO:teuthology.orchestra.run.smithi107.stdout:max_xattr_size 65536 2024-05-17T22:00:39.769 INFO:teuthology.orchestra.run.smithi107.stdout:required_client_features {} 2024-05-17T22:00:39.769 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure 0 2024-05-17T22:00:39.770 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure_osd_epoch 84 2024-05-17T22:00:39.770 INFO:teuthology.orchestra.run.smithi107.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-17T22:00:39.770 INFO:teuthology.orchestra.run.smithi107.stdout:max_mds 1 2024-05-17T22:00:39.770 INFO:teuthology.orchestra.run.smithi107.stdout:in 0 2024-05-17T22:00:39.770 INFO:teuthology.orchestra.run.smithi107.stdout:up {0=34408} 2024-05-17T22:00:39.770 INFO:teuthology.orchestra.run.smithi107.stdout:failed 2024-05-17T22:00:39.770 INFO:teuthology.orchestra.run.smithi107.stdout:damaged 2024-05-17T22:00:39.770 INFO:teuthology.orchestra.run.smithi107.stdout:stopped 2024-05-17T22:00:39.770 INFO:teuthology.orchestra.run.smithi107.stdout:data_pools [3] 2024-05-17T22:00:39.770 INFO:teuthology.orchestra.run.smithi107.stdout:metadata_pool 2 2024-05-17T22:00:39.770 INFO:teuthology.orchestra.run.smithi107.stdout:inline_data enabled 2024-05-17T22:00:39.770 INFO:teuthology.orchestra.run.smithi107.stdout:balancer 2024-05-17T22:00:39.770 INFO:teuthology.orchestra.run.smithi107.stdout:bal_rank_mask -1 2024-05-17T22:00:39.771 INFO:teuthology.orchestra.run.smithi107.stdout:standby_count_wanted 1 2024-05-17T22:00:39.771 INFO:teuthology.orchestra.run.smithi107.stdout:qdb_cluster leader: 34408 members: 34408 2024-05-17T22:00:39.771 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.idmbgj{0:34408} state up:active seq 20 join_fscid=1 addr [v2:172.21.15.107:6826/2235359521,v1:172.21.15.107:6827/2235359521] compat {c=[1],r=[1],i=[1fff]}] 2024-05-17T22:00:39.771 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.drqody{0:34440} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.107:6828/369507752,v1:172.21.15.107:6829/369507752] compat {c=[1],r=[1],i=[1fff]}] 2024-05-17T22:00:39.771 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T22:00:39.771 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T22:00:39.771 INFO:teuthology.orchestra.run.smithi107.stdout:Standby daemons: 2024-05-17T22:00:39.771 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T22:00:39.771 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi138.gwbtyd{-1:44335} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6824/1700112281,v1:172.21.15.138:6825/1700112281] compat {c=[1],r=[1],i=[1fff]}] 2024-05-17T22:00:39.771 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi138.zckrvs{-1:44365} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6826/1801131392,v1:172.21.15.138:6827/1801131392] compat {c=[1],r=[1],i=[1fff]}] 2024-05-17T22:00:39.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.780+0000 7f5dba311700 1 -- 172.21.15.107:0/2935271538 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f5d9c078910 msgr2=0x7f5d9c07adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:39.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.780+0000 7f5dba311700 1 --2- 172.21.15.107:0/2935271538 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f5d9c078910 0x7f5d9c07adc0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5da4009b80 tx=0x7f5da400b040 comp rx=0 tx=0).stop 2024-05-17T22:00:39.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.780+0000 7f5dba311700 1 -- 172.21.15.107:0/2935271538 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5db41022e0 msgr2=0x7f5db4110140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:39.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.780+0000 7f5dba311700 1 --2- 172.21.15.107:0/2935271538 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5db41022e0 0x7f5db4110140 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5da8013870 tx=0x7f5da8009e30 comp rx=0 tx=0).stop 2024-05-17T22:00:39.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.780+0000 7f5dba311700 1 -- 172.21.15.107:0/2935271538 shutdown_connections 2024-05-17T22:00:39.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.780+0000 7f5dba311700 1 --2- 172.21.15.107:0/2935271538 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f5d9c078910 0x7f5d9c07adc0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:39.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.781+0000 7f5dba311700 1 --2- 172.21.15.107:0/2935271538 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5db40662c0 0x7f5db410fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:39.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.781+0000 7f5dba311700 1 --2- 172.21.15.107:0/2935271538 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5db41022e0 0x7f5db4110140 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:39.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.781+0000 7f5dba311700 1 -- 172.21.15.107:0/2935271538 >> 172.21.15.107:0/2935271538 conn(0x7f5db40f7430 msgr2=0x7f5db40f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:39.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.781+0000 7f5dba311700 1 -- 172.21.15.107:0/2935271538 shutdown_connections 2024-05-17T22:00:39.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.781+0000 7f5dba311700 1 -- 172.21.15.107:0/2935271538 wait complete. 2024-05-17T22:00:39.773 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 35 2024-05-17T22:00:39.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.888+0000 7f0509324700 1 -- 172.21.15.107:0/2797477158 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05040fbcb0 msgr2=0x7f05040fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:39.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.888+0000 7f0509324700 1 --2- 172.21.15.107:0/2797477158 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05040fbcb0 0x7f05040fc0c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f04f4004940 tx=0x7f04f4009e30 comp rx=0 tx=0).stop 2024-05-17T22:00:39.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.889+0000 7f0509324700 1 -- 172.21.15.107:0/2797477158 shutdown_connections 2024-05-17T22:00:39.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.889+0000 7f0509324700 1 --2- 172.21.15.107:0/2797477158 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05040fcf60 0x7f05040fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:39.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.889+0000 7f0509324700 1 --2- 172.21.15.107:0/2797477158 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05040fbcb0 0x7f05040fc0c0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:39.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.889+0000 7f0509324700 1 -- 172.21.15.107:0/2797477158 >> 172.21.15.107:0/2797477158 conn(0x7f05040f7430 msgr2=0x7f05040f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:39.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.889+0000 7f0509324700 1 -- 172.21.15.107:0/2797477158 shutdown_connections 2024-05-17T22:00:39.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.889+0000 7f0509324700 1 -- 172.21.15.107:0/2797477158 wait complete. 2024-05-17T22:00:39.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.889+0000 7f0509324700 1 Processor -- start 2024-05-17T22:00:39.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.889+0000 7f0509324700 1 -- start start 2024-05-17T22:00:39.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.890+0000 7f0509324700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05040fbcb0 0x7f05041106e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:39.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.890+0000 7f0509324700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05040fcf60 0x7f0504112c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:39.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.890+0000 7f0509324700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05040662e0 con 0x7f05040fbcb0 2024-05-17T22:00:39.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.890+0000 7f0509324700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0504066420 con 0x7f05040fcf60 2024-05-17T22:00:39.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.890+0000 7f0502ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05040fbcb0 0x7f05041106e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:39.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.890+0000 7f0502ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05040fbcb0 0x7f05041106e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42376/0 (socket says 172.21.15.107:42376) 2024-05-17T22:00:39.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.890+0000 7f0502ffd700 1 -- 172.21.15.107:0/79499655 learned_addr learned my addr 172.21.15.107:0/79499655 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T22:00:39.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.890+0000 7f05027fc700 1 --2- 172.21.15.107:0/79499655 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05040fcf60 0x7f0504112c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:39.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.890+0000 7f0502ffd700 1 -- 172.21.15.107:0/79499655 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05040fcf60 msgr2=0x7f0504112c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:39.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.890+0000 7f0502ffd700 1 --2- 172.21.15.107:0/79499655 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05040fcf60 0x7f0504112c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:39.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.890+0000 7f0502ffd700 1 -- 172.21.15.107:0/79499655 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04f4005040 con 0x7f05040fbcb0 2024-05-17T22:00:39.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.890+0000 7f0502ffd700 1 --2- 172.21.15.107:0/79499655 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05040fbcb0 0x7f05041106e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f04f4013870 tx=0x7f04f4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:39.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.891+0000 7f04eb7fe700 1 -- 172.21.15.107:0/79499655 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f04f4021070 con 0x7f05040fbcb0 2024-05-17T22:00:39.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.891+0000 7f0509324700 1 -- 172.21.15.107:0/79499655 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0504113170 con 0x7f05040fbcb0 2024-05-17T22:00:39.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.891+0000 7f04eb7fe700 1 -- 172.21.15.107:0/79499655 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f04f4009430 con 0x7f05040fbcb0 2024-05-17T22:00:39.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.891+0000 7f0509324700 1 -- 172.21.15.107:0/79499655 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0504113660 con 0x7f05040fbcb0 2024-05-17T22:00:39.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.892+0000 7f04eb7fe700 1 -- 172.21.15.107:0/79499655 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f04f400e070 con 0x7f05040fbcb0 2024-05-17T22:00:39.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.892+0000 7f0509324700 1 -- 172.21.15.107:0/79499655 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04f00052e0 con 0x7f05040fbcb0 2024-05-17T22:00:39.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.894+0000 7f04eb7fe700 1 -- 172.21.15.107:0/79499655 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f04f4013af0 con 0x7f05040fbcb0 2024-05-17T22:00:39.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.894+0000 7f04eb7fe700 1 --2- 172.21.15.107:0/79499655 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f04ec078990 0x7f04ec07ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:39.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.894+0000 7f04eb7fe700 1 -- 172.21.15.107:0/79499655 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f04f409bf90 con 0x7f05040fbcb0 2024-05-17T22:00:39.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.894+0000 7f05027fc700 1 --2- 172.21.15.107:0/79499655 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f04ec078990 0x7f04ec07ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:39.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.895+0000 7f05027fc700 1 --2- 172.21.15.107:0/79499655 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f04ec078990 0x7f04ec07ae40 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f04f8009b80 tx=0x7f04f800b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:39.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:39.898+0000 7f04eb7fe700 1 -- 172.21.15.107:0/79499655 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f04f4063c70 con 0x7f05040fbcb0 2024-05-17T22:00:40.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:39 smithi107 ceph-mon[93224]: pgmap v220: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 11 MiB/s rd, 4.3 KiB/s wr, 8 op/s 2024-05-17T22:00:40.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:39 smithi107 ceph-mon[93224]: from='client.34482 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:00:40.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:39 smithi107 ceph-mon[93224]: from='client.34486 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:00:40.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:39 smithi107 ceph-mon[93224]: from='client.? 172.21.15.107:0/3087046693' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:00:40.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:39 smithi107 ceph-mon[93224]: from='client.? 172.21.15.107:0/2935271538' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-17T22:00:40.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:40.138+0000 7f0509324700 1 -- 172.21.15.107:0/79499655 --> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f04f0000bf0 con 0x7f04ec078990 2024-05-17T22:00:40.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:40.140+0000 7f04eb7fe700 1 -- 172.21.15.107:0/79499655 <== mgr.34100 v2:172.21.15.107:6800/3205268226 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+441 (secure 0 0 0) 0x7f04f0000bf0 con 0x7f04ec078990 2024-05-17T22:00:40.130 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-05-17T22:00:40.130 INFO:teuthology.orchestra.run.smithi107.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b", 2024-05-17T22:00:40.130 INFO:teuthology.orchestra.run.smithi107.stdout: "in_progress": true, 2024-05-17T22:00:40.130 INFO:teuthology.orchestra.run.smithi107.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-17T22:00:40.130 INFO:teuthology.orchestra.run.smithi107.stdout: "services_complete": [ 2024-05-17T22:00:40.130 INFO:teuthology.orchestra.run.smithi107.stdout: "mon", 2024-05-17T22:00:40.130 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr", 2024-05-17T22:00:40.130 INFO:teuthology.orchestra.run.smithi107.stdout: "mds", 2024-05-17T22:00:40.131 INFO:teuthology.orchestra.run.smithi107.stdout: "osd", 2024-05-17T22:00:40.131 INFO:teuthology.orchestra.run.smithi107.stdout: "crash" 2024-05-17T22:00:40.131 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-05-17T22:00:40.131 INFO:teuthology.orchestra.run.smithi107.stdout: "progress": "16/21 daemons upgraded", 2024-05-17T22:00:40.131 INFO:teuthology.orchestra.run.smithi107.stdout: "message": "Currently upgrading node-exporter daemons", 2024-05-17T22:00:40.131 INFO:teuthology.orchestra.run.smithi107.stdout: "is_paused": false 2024-05-17T22:00:40.131 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-05-17T22:00:40.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:40.143+0000 7f0509324700 1 -- 172.21.15.107:0/79499655 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f04ec078990 msgr2=0x7f04ec07ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:40.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:40.143+0000 7f0509324700 1 --2- 172.21.15.107:0/79499655 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f04ec078990 0x7f04ec07ae40 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f04f8009b80 tx=0x7f04f800b040 comp rx=0 tx=0).stop 2024-05-17T22:00:40.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:40.143+0000 7f0509324700 1 -- 172.21.15.107:0/79499655 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05040fbcb0 msgr2=0x7f05041106e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:40.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:40.143+0000 7f0509324700 1 --2- 172.21.15.107:0/79499655 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05040fbcb0 0x7f05041106e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f04f4013870 tx=0x7f04f4009e30 comp rx=0 tx=0).stop 2024-05-17T22:00:40.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:40.143+0000 7f0509324700 1 -- 172.21.15.107:0/79499655 shutdown_connections 2024-05-17T22:00:40.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:40.143+0000 7f0509324700 1 --2- 172.21.15.107:0/79499655 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f04ec078990 0x7f04ec07ae40 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:40.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:40.143+0000 7f0509324700 1 --2- 172.21.15.107:0/79499655 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05040fbcb0 0x7f05041106e0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:40.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:40.143+0000 7f0509324700 1 --2- 172.21.15.107:0/79499655 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05040fcf60 0x7f0504112c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:40.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:40.143+0000 7f0509324700 1 -- 172.21.15.107:0/79499655 >> 172.21.15.107:0/79499655 conn(0x7f05040f7430 msgr2=0x7f05040f8ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:40.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:40.143+0000 7f0509324700 1 -- 172.21.15.107:0/79499655 shutdown_connections 2024-05-17T22:00:40.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:40.143+0000 7f0509324700 1 -- 172.21.15.107:0/79499655 wait complete. 2024-05-17T22:00:40.247 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:40.256+0000 7f5f8b7d3700 1 -- 172.21.15.107:0/201711493 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5f840fdfd0 msgr2=0x7f5f840fe440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:40.247 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:40.256+0000 7f5f8b7d3700 1 --2- 172.21.15.107:0/201711493 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5f840fdfd0 0x7f5f840fe440 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5f80004940 tx=0x7f5f80009e30 comp rx=0 tx=0).stop 2024-05-17T22:00:40.247 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:40.256+0000 7f5f8b7d3700 1 -- 172.21.15.107:0/201711493 shutdown_connections 2024-05-17T22:00:40.248 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:40.256+0000 7f5f8b7d3700 1 --2- 172.21.15.107:0/201711493 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5f840fdfd0 0x7f5f840fe440 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:40.248 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:40.256+0000 7f5f8b7d3700 1 --2- 172.21.15.107:0/201711493 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f840fce60 0x7f5f840fd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:40.248 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:40.256+0000 7f5f8b7d3700 1 -- 172.21.15.107:0/201711493 >> 172.21.15.107:0/201711493 conn(0x7f5f840f8540 msgr2=0x7f5f840fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:40.248 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:40.256+0000 7f5f8b7d3700 1 -- 172.21.15.107:0/201711493 shutdown_connections 2024-05-17T22:00:40.248 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:40.256+0000 7f5f8b7d3700 1 -- 172.21.15.107:0/201711493 wait complete. 2024-05-17T22:00:40.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:40.257+0000 7f5f8b7d3700 1 Processor -- start 2024-05-17T22:00:40.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:40.257+0000 7f5f8b7d3700 1 -- start start 2024-05-17T22:00:40.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:40.257+0000 7f5f8b7d3700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5f840fce60 0x7f5f8410d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:40.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:40.257+0000 7f5f8b7d3700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f840fdfd0 0x7f5f8410de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:40.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:40.257+0000 7f5f8b7d3700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f8410e400 con 0x7f5f840fce60 2024-05-17T22:00:40.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:40.257+0000 7f5f8b7d3700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f8410e570 con 0x7f5f840fdfd0 2024-05-17T22:00:40.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:40.257+0000 7f5f8956f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5f840fce60 0x7f5f8410d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:40.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:40.257+0000 7f5f8956f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5f840fce60 0x7f5f8410d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42402/0 (socket says 172.21.15.107:42402) 2024-05-17T22:00:40.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:40.257+0000 7f5f8956f700 1 -- 172.21.15.107:0/1939394354 learned_addr learned my addr 172.21.15.107:0/1939394354 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T22:00:40.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:40.257+0000 7f5f88d6e700 1 --2- 172.21.15.107:0/1939394354 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f840fdfd0 0x7f5f8410de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:40.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:40.257+0000 7f5f8956f700 1 -- 172.21.15.107:0/1939394354 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f840fdfd0 msgr2=0x7f5f8410de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:40.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:40.257+0000 7f5f8956f700 1 --2- 172.21.15.107:0/1939394354 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f840fdfd0 0x7f5f8410de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:40.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:40.257+0000 7f5f8956f700 1 -- 172.21.15.107:0/1939394354 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f80005040 con 0x7f5f840fce60 2024-05-17T22:00:40.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:40.257+0000 7f5f8956f700 1 --2- 172.21.15.107:0/1939394354 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5f840fce60 0x7f5f8410d8c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5f7800eb40 tx=0x7f5f7800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:40.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:40.258+0000 7f5f75ffb700 1 -- 172.21.15.107:0/1939394354 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5f78007cc0 con 0x7f5f840fce60 2024-05-17T22:00:40.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:40.258+0000 7f5f75ffb700 1 -- 172.21.15.107:0/1939394354 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5f7800f040 con 0x7f5f840fce60 2024-05-17T22:00:40.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:40.258+0000 7f5f8b7d3700 1 -- 172.21.15.107:0/1939394354 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f84112f70 con 0x7f5f840fce60 2024-05-17T22:00:40.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:40.258+0000 7f5f75ffb700 1 -- 172.21.15.107:0/1939394354 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5f7800ca10 con 0x7f5f840fce60 2024-05-17T22:00:40.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:40.258+0000 7f5f8b7d3700 1 -- 172.21.15.107:0/1939394354 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f84113420 con 0x7f5f840fce60 2024-05-17T22:00:40.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:40.259+0000 7f5f8b7d3700 1 -- 172.21.15.107:0/1939394354 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f8404c710 con 0x7f5f840fce60 2024-05-17T22:00:40.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:40.260+0000 7f5f75ffb700 1 -- 172.21.15.107:0/1939394354 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f5f7800cb70 con 0x7f5f840fce60 2024-05-17T22:00:40.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:40.260+0000 7f5f75ffb700 1 --2- 172.21.15.107:0/1939394354 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f5f70078980 0x7f5f7007ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:40.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:40.261+0000 7f5f75ffb700 1 -- 172.21.15.107:0/1939394354 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f5f7809a2a0 con 0x7f5f840fce60 2024-05-17T22:00:40.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:40.261+0000 7f5f88d6e700 1 --2- 172.21.15.107:0/1939394354 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f5f70078980 0x7f5f7007ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:40.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:40.261+0000 7f5f88d6e700 1 --2- 172.21.15.107:0/1939394354 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f5f70078980 0x7f5f7007ae30 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5f8000b0d0 tx=0x7f5f80009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:40.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:40.265+0000 7f5f75ffb700 1 -- 172.21.15.107:0/1939394354 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f5f78061f90 con 0x7f5f840fce60 2024-05-17T22:00:40.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:39 smithi138 ceph-mon[81376]: pgmap v220: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 11 MiB/s rd, 4.3 KiB/s wr, 8 op/s 2024-05-17T22:00:40.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:39 smithi138 ceph-mon[81376]: from='client.34482 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:00:40.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:39 smithi138 ceph-mon[81376]: from='client.34486 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:00:40.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:39 smithi138 ceph-mon[81376]: from='client.? 172.21.15.107:0/3087046693' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:00:40.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:39 smithi138 ceph-mon[81376]: from='client.? 172.21.15.107:0/2935271538' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-17T22:00:40.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:40.579+0000 7f5f8b7d3700 1 -- 172.21.15.107:0/1939394354 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f5f84113660 con 0x7f5f840fce60 2024-05-17T22:00:40.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:40.579+0000 7f5f75ffb700 1 -- 172.21.15.107:0/1939394354 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f5f78061930 con 0x7f5f840fce60 2024-05-17T22:00:40.571 INFO:teuthology.orchestra.run.smithi107.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-17T22:00:40.571 INFO:teuthology.orchestra.run.smithi107.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-17T22:00:40.571 INFO:teuthology.orchestra.run.smithi107.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-17T22:00:40.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:40.581+0000 7f5f8b7d3700 1 -- 172.21.15.107:0/1939394354 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f5f70078980 msgr2=0x7f5f7007ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:40.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:40.581+0000 7f5f8b7d3700 1 --2- 172.21.15.107:0/1939394354 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f5f70078980 0x7f5f7007ae30 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5f8000b0d0 tx=0x7f5f80009e30 comp rx=0 tx=0).stop 2024-05-17T22:00:40.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:40.581+0000 7f5f8b7d3700 1 -- 172.21.15.107:0/1939394354 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5f840fce60 msgr2=0x7f5f8410d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:40.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:40.581+0000 7f5f8b7d3700 1 --2- 172.21.15.107:0/1939394354 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5f840fce60 0x7f5f8410d8c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5f7800eb40 tx=0x7f5f7800b040 comp rx=0 tx=0).stop 2024-05-17T22:00:40.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:40.581+0000 7f5f8b7d3700 1 -- 172.21.15.107:0/1939394354 shutdown_connections 2024-05-17T22:00:40.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:40.581+0000 7f5f8b7d3700 1 --2- 172.21.15.107:0/1939394354 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f5f70078980 0x7f5f7007ae30 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:40.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:40.581+0000 7f5f8b7d3700 1 --2- 172.21.15.107:0/1939394354 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5f840fce60 0x7f5f8410d8c0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:40.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:40.581+0000 7f5f8b7d3700 1 --2- 172.21.15.107:0/1939394354 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f840fdfd0 0x7f5f8410de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:40.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:40.581+0000 7f5f8b7d3700 1 -- 172.21.15.107:0/1939394354 >> 172.21.15.107:0/1939394354 conn(0x7f5f840f8540 msgr2=0x7f5f840f9b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:40.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:40.581+0000 7f5f8b7d3700 1 -- 172.21.15.107:0/1939394354 shutdown_connections 2024-05-17T22:00:40.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:00:40.581+0000 7f5f8b7d3700 1 -- 172.21.15.107:0/1939394354 wait complete. 2024-05-17T22:00:41.159 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:40 smithi107 ceph-mon[93224]: from='client.34498 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:00:41.160 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:40 smithi107 ceph-mon[93224]: from='client.? 172.21.15.107:0/1939394354' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-17T22:00:41.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:40 smithi138 ceph-mon[81376]: from='client.34498 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:00:41.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:40 smithi138 ceph-mon[81376]: from='client.? 172.21.15.107:0/1939394354' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-17T22:00:42.215 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:41 smithi107 ceph-mon[93224]: pgmap v221: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 9.5 MiB/s rd, 4.3 KiB/s wr, 8 op/s 2024-05-17T22:00:42.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:41 smithi138 ceph-mon[81376]: pgmap v221: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 9.5 MiB/s rd, 4.3 KiB/s wr, 8 op/s 2024-05-17T22:00:43.278 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:42 smithi107 ceph-mon[93224]: Upgrade: Updating node-exporter.smithi107 (1/2) 2024-05-17T22:00:43.278 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:42 smithi107 ceph-mon[93224]: Deploying daemon node-exporter.smithi107 on smithi107 2024-05-17T22:00:43.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:42 smithi138 ceph-mon[81376]: Upgrade: Updating node-exporter.smithi107 (1/2) 2024-05-17T22:00:43.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:42 smithi138 ceph-mon[81376]: Deploying daemon node-exporter.smithi107 on smithi107 2024-05-17T22:00:44.230 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:43 smithi107 ceph-mon[93224]: pgmap v222: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 19 MiB/s rd, 4.3 KiB/s wr, 10 op/s 2024-05-17T22:00:44.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:43 smithi138 ceph-mon[81376]: pgmap v222: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 19 MiB/s rd, 4.3 KiB/s wr, 10 op/s 2024-05-17T22:00:45.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:45 smithi138 ceph-mon[81376]: pgmap v223: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 19 MiB/s rd, 5 op/s 2024-05-17T22:00:45.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:45 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:00:45.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:45 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:00:45.745 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:45 smithi107 ceph-mon[93224]: pgmap v223: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 19 MiB/s rd, 5 op/s 2024-05-17T22:00:45.745 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:45 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:00:45.746 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:45 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:00:47.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:47 smithi138 ceph-mon[81376]: Upgrade: Updating node-exporter.smithi138 (2/2) 2024-05-17T22:00:47.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:47 smithi138 ceph-mon[81376]: Deploying daemon node-exporter.smithi138 on smithi138 2024-05-17T22:00:47.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:47 smithi138 ceph-mon[81376]: pgmap v224: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 6 op/s 2024-05-17T22:00:47.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:47 smithi107 ceph-mon[93224]: Upgrade: Updating node-exporter.smithi138 (2/2) 2024-05-17T22:00:47.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:47 smithi107 ceph-mon[93224]: Deploying daemon node-exporter.smithi138 on smithi138 2024-05-17T22:00:47.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:47 smithi107 ceph-mon[93224]: pgmap v224: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 6 op/s 2024-05-17T22:00:50.109 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:50 smithi138 ceph-mon[81376]: pgmap v225: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 17 MiB/s rd, 5 op/s 2024-05-17T22:00:50.109 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:50 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:00:50.109 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:50 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:00:50.109 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:50 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:00:50.110 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:50 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T22:00:50.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:50 smithi107 ceph-mon[93224]: pgmap v225: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 17 MiB/s rd, 5 op/s 2024-05-17T22:00:50.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:50 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:00:50.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:50 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:00:50.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:50 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:00:50.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:50 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T22:00:52.276 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:52 smithi107 ceph-mon[93224]: pgmap v226: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 12 MiB/s rd, 3 op/s 2024-05-17T22:00:52.276 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:52 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:00:52.276 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:52 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:00:52.324 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:52 smithi138 ceph-mon[81376]: pgmap v226: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 12 MiB/s rd, 3 op/s 2024-05-17T22:00:52.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:52 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:00:52.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:52 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:00:53.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:53 smithi138 ceph-mon[81376]: pgmap v227: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 12 MiB/s rd, 4 op/s 2024-05-17T22:00:53.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:53 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:00:53.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:53 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:00:53.850 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:53 smithi107 ceph-mon[93224]: pgmap v227: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 12 MiB/s rd, 4 op/s 2024-05-17T22:00:53.850 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:53 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:00:53.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:53 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:00:54.710 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:54 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:00:54.710 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:54 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:00:54.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:54 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:00:54.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:54 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:00:55.990 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:55 smithi138 ceph-mon[81376]: pgmap v228: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 2.3 MiB/s rd, 1 op/s 2024-05-17T22:00:55.990 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:55 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:00:55.990 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:55 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:00:56.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:55 smithi107 ceph-mon[93224]: pgmap v228: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 2.3 MiB/s rd, 1 op/s 2024-05-17T22:00:56.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:55 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:00:56.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:55 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:00:57.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:57 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:00:57.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:57 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:00:57.350 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:57 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:00:57.350 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:57 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:00:58.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:58 smithi138 ceph-mon[81376]: pgmap v229: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 2.3 MiB/s rd, 1 op/s 2024-05-17T22:00:58.350 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:58 smithi107 ceph-mon[93224]: pgmap v229: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 2.3 MiB/s rd, 1 op/s 2024-05-17T22:00:59.178 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:59 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:00:59.178 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:59 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:00:59.178 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:59 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:00:59.178 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:59 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:00:59.178 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:59 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:00:59.178 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:59 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:00:59.179 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:59 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:00:59.179 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:59 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:00:59.179 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:59 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:00:59.179 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:59 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:00:59.179 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:59 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:00:59.179 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:59 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:00:59.179 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:59 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:00:59.179 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:59 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:00:59.179 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:59 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:00:59.179 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:59 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:00:59.179 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:59 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:00:59.179 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:59 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:00:59.179 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:00:59 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:00:59.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:59 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:00:59.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:59 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:00:59.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:59 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:00:59.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:59 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:00:59.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:59 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:00:59.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:59 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:00:59.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:59 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:00:59.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:59 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:00:59.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:59 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:00:59.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:59 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:00:59.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:59 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:00:59.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:59 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:00:59.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:59 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:00:59.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:59 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:00:59.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:59 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:00:59.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:59 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:00:59.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:59 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:00:59.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:59 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:00:59.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:00:59 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:01:00.461 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:00 smithi107 ceph-mon[93224]: pgmap v230: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 2 op/s 2024-05-17T22:01:00.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:00 smithi138 ceph-mon[81376]: pgmap v230: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 2 op/s 2024-05-17T22:01:01.471 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:01 smithi107 ceph-mon[93224]: Upgrade: Updating prometheus.smithi107 2024-05-17T22:01:01.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:01 smithi138 ceph-mon[81376]: Upgrade: Updating prometheus.smithi107 2024-05-17T22:01:02.179 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:02 smithi138 ceph-mon[81376]: Deploying daemon prometheus.smithi107 on smithi107 2024-05-17T22:01:02.180 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:02 smithi138 ceph-mon[81376]: pgmap v231: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:01:02.436 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:02 smithi107 ceph-mon[93224]: Deploying daemon prometheus.smithi107 on smithi107 2024-05-17T22:01:02.436 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:02 smithi107 ceph-mon[93224]: pgmap v231: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:01:04.498 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:04 smithi107 ceph-mon[93224]: pgmap v232: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:01:04.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:04 smithi138 ceph-mon[81376]: pgmap v232: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:01:05.483 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:05 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T22:01:05.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:05 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T22:01:06.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:06 smithi138 ceph-mon[81376]: pgmap v233: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:01:06.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:06 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:01:06.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:06 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:01:06.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:06 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:01:06.600 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:06 smithi107 ceph-mon[93224]: pgmap v233: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:01:06.600 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:06 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:01:06.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:06 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:01:06.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:06 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:01:08.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:08 smithi107 ceph-mon[93224]: pgmap v234: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:01:08.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:08 smithi138 ceph-mon[81376]: pgmap v234: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:01:09.484 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:09 smithi107 ceph-mon[93224]: pgmap v235: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:01:09.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:09 smithi138 ceph-mon[81376]: pgmap v235: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:01:10.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:10.690+0000 7fe531c0a700 1 -- 172.21.15.107:0/900487526 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe52c0fce80 msgr2=0x7fe52c0fd290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:10.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:10.690+0000 7fe531c0a700 1 --2- 172.21.15.107:0/900487526 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe52c0fce80 0x7fe52c0fd290 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe51c004940 tx=0x7fe51c009e30 comp rx=0 tx=0).stop 2024-05-17T22:01:10.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:10.690+0000 7fe531c0a700 1 -- 172.21.15.107:0/900487526 shutdown_connections 2024-05-17T22:01:10.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:10.690+0000 7fe531c0a700 1 --2- 172.21.15.107:0/900487526 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe52c0fdff0 0x7fe52c0fe460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:10.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:10.690+0000 7fe531c0a700 1 --2- 172.21.15.107:0/900487526 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe52c0fce80 0x7fe52c0fd290 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:10.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:10.690+0000 7fe531c0a700 1 -- 172.21.15.107:0/900487526 >> 172.21.15.107:0/900487526 conn(0x7fe52c0f8540 msgr2=0x7fe52c0fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:01:10.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:10.690+0000 7fe531c0a700 1 -- 172.21.15.107:0/900487526 shutdown_connections 2024-05-17T22:01:10.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:10.690+0000 7fe531c0a700 1 -- 172.21.15.107:0/900487526 wait complete. 2024-05-17T22:01:10.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:10.690+0000 7fe531c0a700 1 Processor -- start 2024-05-17T22:01:10.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:10.691+0000 7fe531c0a700 1 -- start start 2024-05-17T22:01:10.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:10.691+0000 7fe531c0a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe52c0fce80 0x7fe52c10da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:10.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:10.691+0000 7fe531c0a700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe52c0fdff0 0x7fe52c10df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:10.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:10.691+0000 7fe531c0a700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe52c10e550 con 0x7fe52c0fdff0 2024-05-17T22:01:10.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:10.691+0000 7fe531c0a700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe52c10eda0 con 0x7fe52c0fce80 2024-05-17T22:01:10.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:10.691+0000 7fe52affd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe52c0fdff0 0x7fe52c10df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:10.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:10.691+0000 7fe52affd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe52c0fdff0 0x7fe52c10df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56334/0 (socket says 172.21.15.107:56334) 2024-05-17T22:01:10.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:10.691+0000 7fe52affd700 1 -- 172.21.15.107:0/3660660160 learned_addr learned my addr 172.21.15.107:0/3660660160 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T22:01:10.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:10.691+0000 7fe52b7fe700 1 --2- 172.21.15.107:0/3660660160 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe52c0fce80 0x7fe52c10da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:10.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:10.691+0000 7fe52affd700 1 -- 172.21.15.107:0/3660660160 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe52c0fce80 msgr2=0x7fe52c10da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:10.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:10.691+0000 7fe52affd700 1 --2- 172.21.15.107:0/3660660160 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe52c0fce80 0x7fe52c10da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:10.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:10.691+0000 7fe52affd700 1 -- 172.21.15.107:0/3660660160 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe51c005040 con 0x7fe52c0fdff0 2024-05-17T22:01:10.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:10.691+0000 7fe52b7fe700 1 --2- 172.21.15.107:0/3660660160 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe52c0fce80 0x7fe52c10da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:01:10.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:10.692+0000 7fe52affd700 1 --2- 172.21.15.107:0/3660660160 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe52c0fdff0 0x7fe52c10df50 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe52000c830 tx=0x7fe520004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:10.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:10.692+0000 7fe513fff700 1 -- 172.21.15.107:0/3660660160 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe52000a760 con 0x7fe52c0fdff0 2024-05-17T22:01:10.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:10.692+0000 7fe513fff700 1 -- 172.21.15.107:0/3660660160 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe52000ad90 con 0x7fe52c0fdff0 2024-05-17T22:01:10.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:10.692+0000 7fe531c0a700 1 -- 172.21.15.107:0/3660660160 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe52c10f080 con 0x7fe52c0fdff0 2024-05-17T22:01:10.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:10.692+0000 7fe513fff700 1 -- 172.21.15.107:0/3660660160 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe5200113a0 con 0x7fe52c0fdff0 2024-05-17T22:01:10.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:10.692+0000 7fe531c0a700 1 -- 172.21.15.107:0/3660660160 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe52c10f5c0 con 0x7fe52c0fdff0 2024-05-17T22:01:10.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:10.693+0000 7fe531c0a700 1 -- 172.21.15.107:0/3660660160 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe52c05f5c0 con 0x7fe52c0fdff0 2024-05-17T22:01:10.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:10.694+0000 7fe513fff700 1 -- 172.21.15.107:0/3660660160 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7fe520012400 con 0x7fe52c0fdff0 2024-05-17T22:01:10.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:10.694+0000 7fe513fff700 1 --2- 172.21.15.107:0/3660660160 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fe514078990 0x7fe51407ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:10.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:10.694+0000 7fe513fff700 1 -- 172.21.15.107:0/3660660160 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7fe52009a7e0 con 0x7fe52c0fdff0 2024-05-17T22:01:10.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:10.694+0000 7fe52b7fe700 1 --2- 172.21.15.107:0/3660660160 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fe514078990 0x7fe51407ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:10.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:10.695+0000 7fe52b7fe700 1 --2- 172.21.15.107:0/3660660160 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fe514078990 0x7fe51407ae40 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe51c00b3b0 tx=0x7fe51c009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:10.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:10.699+0000 7fe513fff700 1 -- 172.21.15.107:0/3660660160 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7fe520062420 con 0x7fe52c0fdff0 2024-05-17T22:01:10.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:10.930+0000 7fe531c0a700 1 -- 172.21.15.107:0/3660660160 --> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe52c10fa00 con 0x7fe514078990 2024-05-17T22:01:10.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:10.931+0000 7fe513fff700 1 -- 172.21.15.107:0/3660660160 <== mgr.34100 v2:172.21.15.107:6800/3205268226 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+438 (secure 0 0 0) 0x7fe52c10fa00 con 0x7fe514078990 2024-05-17T22:01:10.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:10.933+0000 7fe531c0a700 1 -- 172.21.15.107:0/3660660160 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fe514078990 msgr2=0x7fe51407ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:10.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:10.933+0000 7fe531c0a700 1 --2- 172.21.15.107:0/3660660160 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fe514078990 0x7fe51407ae40 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe51c00b3b0 tx=0x7fe51c009e30 comp rx=0 tx=0).stop 2024-05-17T22:01:10.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:10.933+0000 7fe531c0a700 1 -- 172.21.15.107:0/3660660160 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe52c0fdff0 msgr2=0x7fe52c10df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:10.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:10.933+0000 7fe531c0a700 1 --2- 172.21.15.107:0/3660660160 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe52c0fdff0 0x7fe52c10df50 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe52000c830 tx=0x7fe520004470 comp rx=0 tx=0).stop 2024-05-17T22:01:10.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:10.933+0000 7fe531c0a700 1 -- 172.21.15.107:0/3660660160 shutdown_connections 2024-05-17T22:01:10.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:10.933+0000 7fe531c0a700 1 --2- 172.21.15.107:0/3660660160 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fe514078990 0x7fe51407ae40 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:10.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:10.933+0000 7fe531c0a700 1 --2- 172.21.15.107:0/3660660160 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe52c0fce80 0x7fe52c10da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:10.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:10.933+0000 7fe531c0a700 1 --2- 172.21.15.107:0/3660660160 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe52c0fdff0 0x7fe52c10df50 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:10.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:10.934+0000 7fe531c0a700 1 -- 172.21.15.107:0/3660660160 >> 172.21.15.107:0/3660660160 conn(0x7fe52c0f8540 msgr2=0x7fe52c0f9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:01:10.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:10.934+0000 7fe531c0a700 1 -- 172.21.15.107:0/3660660160 shutdown_connections 2024-05-17T22:01:10.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:10.934+0000 7fe531c0a700 1 -- 172.21.15.107:0/3660660160 wait complete. 2024-05-17T22:01:10.937 INFO:teuthology.orchestra.run.smithi107.stdout:true 2024-05-17T22:01:11.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.045+0000 7fc6b0a5f700 1 -- 172.21.15.107:0/2005280649 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6ac0fbe30 msgr2=0x7fc6ac0fc240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:11.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.045+0000 7fc6b0a5f700 1 --2- 172.21.15.107:0/2005280649 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6ac0fbe30 0x7fc6ac0fc240 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc69c004940 tx=0x7fc69c009e30 comp rx=0 tx=0).stop 2024-05-17T22:01:11.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.045+0000 7fc6b0a5f700 1 -- 172.21.15.107:0/2005280649 shutdown_connections 2024-05-17T22:01:11.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.045+0000 7fc6b0a5f700 1 --2- 172.21.15.107:0/2005280649 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc6ac0fcfa0 0x7fc6ac0fd410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:11.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.045+0000 7fc6b0a5f700 1 --2- 172.21.15.107:0/2005280649 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6ac0fbe30 0x7fc6ac0fc240 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:11.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.045+0000 7fc6b0a5f700 1 -- 172.21.15.107:0/2005280649 >> 172.21.15.107:0/2005280649 conn(0x7fc6ac0f7440 msgr2=0x7fc6ac0f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:01:11.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.045+0000 7fc6b0a5f700 1 -- 172.21.15.107:0/2005280649 shutdown_connections 2024-05-17T22:01:11.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.045+0000 7fc6b0a5f700 1 -- 172.21.15.107:0/2005280649 wait complete. 2024-05-17T22:01:11.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.045+0000 7fc6b0a5f700 1 Processor -- start 2024-05-17T22:01:11.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.045+0000 7fc6b0a5f700 1 -- start start 2024-05-17T22:01:11.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.045+0000 7fc6b0a5f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6ac0fbe30 0x7fc6ac110720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:11.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.045+0000 7fc6b0a5f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc6ac0fcfa0 0x7fc6ac112c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:11.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.045+0000 7fc6b0a5f700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6ac066c50 con 0x7fc6ac0fbe30 2024-05-17T22:01:11.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.045+0000 7fc6b0a5f700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6ac066d90 con 0x7fc6ac0fcfa0 2024-05-17T22:01:11.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.045+0000 7fc6aa59c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6ac0fbe30 0x7fc6ac110720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:11.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.046+0000 7fc6aa59c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6ac0fbe30 0x7fc6ac110720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56352/0 (socket says 172.21.15.107:56352) 2024-05-17T22:01:11.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.046+0000 7fc6aa59c700 1 -- 172.21.15.107:0/1314892153 learned_addr learned my addr 172.21.15.107:0/1314892153 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T22:01:11.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.046+0000 7fc6a3fff700 1 --2- 172.21.15.107:0/1314892153 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc6ac0fcfa0 0x7fc6ac112c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:11.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.046+0000 7fc6aa59c700 1 -- 172.21.15.107:0/1314892153 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc6ac0fcfa0 msgr2=0x7fc6ac112c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:11.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.046+0000 7fc6aa59c700 1 --2- 172.21.15.107:0/1314892153 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc6ac0fcfa0 0x7fc6ac112c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:11.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.046+0000 7fc6aa59c700 1 -- 172.21.15.107:0/1314892153 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc69c005040 con 0x7fc6ac0fbe30 2024-05-17T22:01:11.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.046+0000 7fc6aa59c700 1 --2- 172.21.15.107:0/1314892153 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6ac0fbe30 0x7fc6ac110720 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc69c009150 tx=0x7fc69c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:11.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.046+0000 7fc6a2ffd700 1 -- 172.21.15.107:0/1314892153 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc69c00e040 con 0x7fc6ac0fbe30 2024-05-17T22:01:11.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.046+0000 7fc6b0a5f700 1 -- 172.21.15.107:0/1314892153 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6ac113270 con 0x7fc6ac0fbe30 2024-05-17T22:01:11.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.046+0000 7fc6a2ffd700 1 -- 172.21.15.107:0/1314892153 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc69c004290 con 0x7fc6ac0fbe30 2024-05-17T22:01:11.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.046+0000 7fc6a2ffd700 1 -- 172.21.15.107:0/1314892153 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc69c01c630 con 0x7fc6ac0fbe30 2024-05-17T22:01:11.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.046+0000 7fc6b0a5f700 1 -- 172.21.15.107:0/1314892153 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6ac113800 con 0x7fc6ac0fbe30 2024-05-17T22:01:11.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.047+0000 7fc6b0a5f700 1 -- 172.21.15.107:0/1314892153 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6ac05f5c0 con 0x7fc6ac0fbe30 2024-05-17T22:01:11.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.048+0000 7fc6a2ffd700 1 -- 172.21.15.107:0/1314892153 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7fc69c00bbf0 con 0x7fc6ac0fbe30 2024-05-17T22:01:11.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.048+0000 7fc6a2ffd700 1 --2- 172.21.15.107:0/1314892153 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fc698078990 0x7fc69807ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:11.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.049+0000 7fc6a2ffd700 1 -- 172.21.15.107:0/1314892153 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7fc69c09b3a0 con 0x7fc6ac0fbe30 2024-05-17T22:01:11.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.049+0000 7fc6a3fff700 1 --2- 172.21.15.107:0/1314892153 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fc698078990 0x7fc69807ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:11.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.049+0000 7fc6a3fff700 1 --2- 172.21.15.107:0/1314892153 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fc698078990 0x7fc69807ae40 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc694009bf0 tx=0x7fc6940091a0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:11.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.053+0000 7fc6a2ffd700 1 -- 172.21.15.107:0/1314892153 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7fc69c062fe0 con 0x7fc6ac0fbe30 2024-05-17T22:01:11.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.297+0000 7fc6b0a5f700 1 -- 172.21.15.107:0/1314892153 --> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc6ac101270 con 0x7fc698078990 2024-05-17T22:01:11.289 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:10 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:01:11.289 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:10 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:01:11.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.299+0000 7fc6a2ffd700 1 -- 172.21.15.107:0/1314892153 <== mgr.34100 v2:172.21.15.107:6800/3205268226 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+438 (secure 0 0 0) 0x7fc6ac101270 con 0x7fc698078990 2024-05-17T22:01:11.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.302+0000 7fc6b0a5f700 1 -- 172.21.15.107:0/1314892153 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fc698078990 msgr2=0x7fc69807ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:11.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.302+0000 7fc6b0a5f700 1 --2- 172.21.15.107:0/1314892153 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fc698078990 0x7fc69807ae40 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc694009bf0 tx=0x7fc6940091a0 comp rx=0 tx=0).stop 2024-05-17T22:01:11.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.302+0000 7fc6b0a5f700 1 -- 172.21.15.107:0/1314892153 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6ac0fbe30 msgr2=0x7fc6ac110720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:11.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.302+0000 7fc6b0a5f700 1 --2- 172.21.15.107:0/1314892153 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6ac0fbe30 0x7fc6ac110720 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc69c009150 tx=0x7fc69c009e30 comp rx=0 tx=0).stop 2024-05-17T22:01:11.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.302+0000 7fc6b0a5f700 1 -- 172.21.15.107:0/1314892153 shutdown_connections 2024-05-17T22:01:11.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.302+0000 7fc6b0a5f700 1 --2- 172.21.15.107:0/1314892153 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fc698078990 0x7fc69807ae40 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:11.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.302+0000 7fc6b0a5f700 1 --2- 172.21.15.107:0/1314892153 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6ac0fbe30 0x7fc6ac110720 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:11.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.302+0000 7fc6b0a5f700 1 --2- 172.21.15.107:0/1314892153 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc6ac0fcfa0 0x7fc6ac112c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:11.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.302+0000 7fc6b0a5f700 1 -- 172.21.15.107:0/1314892153 >> 172.21.15.107:0/1314892153 conn(0x7fc6ac0f7440 msgr2=0x7fc6ac0f8a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:01:11.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.302+0000 7fc6b0a5f700 1 -- 172.21.15.107:0/1314892153 shutdown_connections 2024-05-17T22:01:11.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.302+0000 7fc6b0a5f700 1 -- 172.21.15.107:0/1314892153 wait complete. 2024-05-17T22:01:11.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:10 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:01:11.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:10 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:01:11.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.408+0000 7f4182d36700 1 -- 172.21.15.107:0/224987704 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f417c0fce60 msgr2=0x7f417c0fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:11.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.408+0000 7f4182d36700 1 --2- 172.21.15.107:0/224987704 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f417c0fce60 0x7f417c0fd270 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f4170004940 tx=0x7f4170009e30 comp rx=0 tx=0).stop 2024-05-17T22:01:11.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.408+0000 7f4182d36700 1 -- 172.21.15.107:0/224987704 shutdown_connections 2024-05-17T22:01:11.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.408+0000 7f4182d36700 1 --2- 172.21.15.107:0/224987704 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f417c0fdfd0 0x7f417c0fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:11.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.408+0000 7f4182d36700 1 --2- 172.21.15.107:0/224987704 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f417c0fce60 0x7f417c0fd270 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:11.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.408+0000 7f4182d36700 1 -- 172.21.15.107:0/224987704 >> 172.21.15.107:0/224987704 conn(0x7f417c0f8540 msgr2=0x7f417c0fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:01:11.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.408+0000 7f4182d36700 1 -- 172.21.15.107:0/224987704 shutdown_connections 2024-05-17T22:01:11.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.408+0000 7f4182d36700 1 -- 172.21.15.107:0/224987704 wait complete. 2024-05-17T22:01:11.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.409+0000 7f4182d36700 1 Processor -- start 2024-05-17T22:01:11.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.409+0000 7f4182d36700 1 -- start start 2024-05-17T22:01:11.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.409+0000 7f4182d36700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f417c0fce60 0x7f417c10da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:11.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.409+0000 7f4182d36700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f417c0fdfd0 0x7f417c10df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:11.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.409+0000 7f4182d36700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f417c10e5e0 con 0x7f417c0fdfd0 2024-05-17T22:01:11.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.409+0000 7f4182d36700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f417c10edd0 con 0x7f417c0fce60 2024-05-17T22:01:11.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.409+0000 7f417bfff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f417c0fdfd0 0x7f417c10df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:11.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.409+0000 7f417bfff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f417c0fdfd0 0x7f417c10df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56360/0 (socket says 172.21.15.107:56360) 2024-05-17T22:01:11.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.409+0000 7f417bfff700 1 -- 172.21.15.107:0/1702555440 learned_addr learned my addr 172.21.15.107:0/1702555440 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T22:01:11.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.410+0000 7f4180ad2700 1 --2- 172.21.15.107:0/1702555440 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f417c0fce60 0x7f417c10da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:11.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.410+0000 7f417bfff700 1 -- 172.21.15.107:0/1702555440 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f417c0fce60 msgr2=0x7f417c10da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:11.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.410+0000 7f417bfff700 1 --2- 172.21.15.107:0/1702555440 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f417c0fce60 0x7f417c10da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:11.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.410+0000 7f417bfff700 1 -- 172.21.15.107:0/1702555440 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4170005040 con 0x7f417c0fdfd0 2024-05-17T22:01:11.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.410+0000 7f417bfff700 1 --2- 172.21.15.107:0/1702555440 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f417c0fdfd0 0x7f417c10df50 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f416800b790 tx=0x7f4168009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:11.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.410+0000 7f41797fa700 1 -- 172.21.15.107:0/1702555440 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f416800f6f0 con 0x7f417c0fdfd0 2024-05-17T22:01:11.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.410+0000 7f41797fa700 1 -- 172.21.15.107:0/1702555440 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f416800fd20 con 0x7f417c0fdfd0 2024-05-17T22:01:11.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.410+0000 7f4182d36700 1 -- 172.21.15.107:0/1702555440 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f417c10f0b0 con 0x7f417c0fdfd0 2024-05-17T22:01:11.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.410+0000 7f41797fa700 1 -- 172.21.15.107:0/1702555440 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f41680103a0 con 0x7f417c0fdfd0 2024-05-17T22:01:11.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.410+0000 7f4182d36700 1 -- 172.21.15.107:0/1702555440 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f417c10f670 con 0x7f417c0fdfd0 2024-05-17T22:01:11.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.411+0000 7f4182d36700 1 -- 172.21.15.107:0/1702555440 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f417c05f5c0 con 0x7f417c0fdfd0 2024-05-17T22:01:11.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.412+0000 7f41797fa700 1 -- 172.21.15.107:0/1702555440 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f4168011400 con 0x7f417c0fdfd0 2024-05-17T22:01:11.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.412+0000 7f41797fa700 1 --2- 172.21.15.107:0/1702555440 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f416c07cda0 0x7f416c07f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:11.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.413+0000 7f41797fa700 1 -- 172.21.15.107:0/1702555440 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f416809aff0 con 0x7f417c0fdfd0 2024-05-17T22:01:11.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.413+0000 7f4180ad2700 1 --2- 172.21.15.107:0/1702555440 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f416c07cda0 0x7f416c07f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:11.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.413+0000 7f4180ad2700 1 --2- 172.21.15.107:0/1702555440 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f416c07cda0 0x7f416c07f250 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f417000b970 tx=0x7f4170009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:11.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.416+0000 7f41797fa700 1 -- 172.21.15.107:0/1702555440 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f4168062cd0 con 0x7f417c0fdfd0 2024-05-17T22:01:11.655 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.663+0000 7f4182d36700 1 -- 172.21.15.107:0/1702555440 --> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f417c10f990 con 0x7f416c07cda0 2024-05-17T22:01:11.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.673+0000 7f41797fa700 1 -- 172.21.15.107:0/1702555440 <== mgr.34100 v2:172.21.15.107:6800/3205268226 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f417c10f990 con 0x7f416c07cda0 2024-05-17T22:01:11.665 INFO:teuthology.orchestra.run.smithi107.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-17T22:01:11.665 INFO:teuthology.orchestra.run.smithi107.stdout:alertmanager.smithi107 smithi107 *:9093,9094 running (8m) 1s ago 17m 24.6M - 0.25.0 c8568f914cd2 bac2a944ae42 2024-05-17T22:01:11.665 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi107 smithi107 running (5m) 1s ago 17m 6669k - 19.0.0-3794-g931809a9 8880f7613d02 face35de205b 2024-05-17T22:01:11.665 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi138 smithi138 running (5m) 20s ago 16m 6665k - 19.0.0-3794-g931809a9 8880f7613d02 ad59e33e6d43 2024-05-17T22:01:11.665 INFO:teuthology.orchestra.run.smithi107.stdout:grafana.smithi107 smithi107 *:3000 running (8m) 1s ago 17m 85.9M - 9.4.7 954c08fa6188 8c747e777bc5 2024-05-17T22:01:11.665 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.drqody smithi107 running (92s) 1s ago 11m 155M - 19.0.0-3794-g931809a9 8880f7613d02 4f02280f8312 2024-05-17T22:01:11.665 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.idmbgj smithi107 running (110s) 1s ago 11m 153M - 19.0.0-3794-g931809a9 8880f7613d02 d541bff38965 2024-05-17T22:01:11.666 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi138.gwbtyd smithi138 running (67s) 20s ago 11m 80.2M - 19.0.0-3794-g931809a9 8880f7613d02 7beb9bd77fc8 2024-05-17T22:01:11.666 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi138.zckrvs smithi138 running (46s) 20s ago 11m 79.2M - 19.0.0-3794-g931809a9 8880f7613d02 17edb3a75f7b 2024-05-17T22:01:11.666 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi107.xhhyiw smithi107 *:8443,9283,8765 running (8m) 1s ago 18m 596M - 19.0.0-3794-g931809a9 8880f7613d02 5acf3e511342 2024-05-17T22:01:11.666 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi138.napnll smithi138 *:8443,9283,8765 running (7m) 20s ago 16m 534M - 19.0.0-3794-g931809a9 8880f7613d02 d627761dbd0d 2024-05-17T22:01:11.666 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi107 smithi107 running (6m) 1s ago 18m 121M 2048M 19.0.0-3794-g931809a9 8880f7613d02 9717f3169f2f 2024-05-17T22:01:11.666 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi138 smithi138 running (6m) 20s ago 16m 107M 2048M 19.0.0-3794-g931809a9 8880f7613d02 55aeed62bf4d 2024-05-17T22:01:11.666 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi107 smithi107 *:9100 running (27s) 1s ago 17m 8036k - 1.5.0 0da6a335fe13 73f1896972ff 2024-05-17T22:01:11.666 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi138 smithi138 *:9100 running (23s) 20s ago 16m 3565k - 1.5.0 0da6a335fe13 05bb3093f9aa 2024-05-17T22:01:11.666 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0 smithi107 running (5m) 1s ago 15m 165M 1557M 19.0.0-3794-g931809a9 8880f7613d02 b319bc93500d 2024-05-17T22:01:11.666 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1 smithi107 running (4m) 1s ago 15m 166M 1557M 19.0.0-3794-g931809a9 8880f7613d02 f94b206be6eb 2024-05-17T22:01:11.667 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2 smithi107 running (3m) 1s ago 14m 154M 1557M 19.0.0-3794-g931809a9 8880f7613d02 add7015647af 2024-05-17T22:01:11.667 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3 smithi138 running (3m) 20s ago 14m 174M 2581M 19.0.0-3794-g931809a9 8880f7613d02 416273d327c7 2024-05-17T22:01:11.667 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4 smithi138 running (2m) 20s ago 13m 159M 2581M 19.0.0-3794-g931809a9 8880f7613d02 e33f411da8e6 2024-05-17T22:01:11.667 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5 smithi138 running (2m) 20s ago 13m 159M 2581M 19.0.0-3794-g931809a9 8880f7613d02 63810d431961 2024-05-17T22:01:11.667 INFO:teuthology.orchestra.run.smithi107.stdout:prometheus.smithi107 smithi107 *:9095 running (6s) 1s ago 16m 43.6M - 2.43.0 a07b618ecd1d aee4bd577d3d 2024-05-17T22:01:11.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.676+0000 7f4182d36700 1 -- 172.21.15.107:0/1702555440 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f416c07cda0 msgr2=0x7f416c07f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:11.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.676+0000 7f4182d36700 1 --2- 172.21.15.107:0/1702555440 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f416c07cda0 0x7f416c07f250 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f417000b970 tx=0x7f4170009e30 comp rx=0 tx=0).stop 2024-05-17T22:01:11.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.676+0000 7f4182d36700 1 -- 172.21.15.107:0/1702555440 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f417c0fdfd0 msgr2=0x7f417c10df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:11.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.676+0000 7f4182d36700 1 --2- 172.21.15.107:0/1702555440 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f417c0fdfd0 0x7f417c10df50 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f416800b790 tx=0x7f4168009250 comp rx=0 tx=0).stop 2024-05-17T22:01:11.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.676+0000 7f4182d36700 1 -- 172.21.15.107:0/1702555440 shutdown_connections 2024-05-17T22:01:11.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.676+0000 7f4182d36700 1 --2- 172.21.15.107:0/1702555440 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f416c07cda0 0x7f416c07f250 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:11.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.676+0000 7f4182d36700 1 --2- 172.21.15.107:0/1702555440 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f417c0fce60 0x7f417c10da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:11.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.676+0000 7f4182d36700 1 --2- 172.21.15.107:0/1702555440 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f417c0fdfd0 0x7f417c10df50 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:11.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.676+0000 7f4182d36700 1 -- 172.21.15.107:0/1702555440 >> 172.21.15.107:0/1702555440 conn(0x7f417c0f8540 msgr2=0x7f417c0f9b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:01:11.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.676+0000 7f4182d36700 1 -- 172.21.15.107:0/1702555440 shutdown_connections 2024-05-17T22:01:11.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.676+0000 7f4182d36700 1 -- 172.21.15.107:0/1702555440 wait complete. 2024-05-17T22:01:11.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.781+0000 7f9eb6726700 1 -- 172.21.15.107:0/178712821 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9eb0066f30 msgr2=0x7f9eb00ff9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:11.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.781+0000 7f9eb6726700 1 --2- 172.21.15.107:0/178712821 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9eb0066f30 0x7f9eb00ff9b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9ea0004940 tx=0x7f9ea0009e30 comp rx=0 tx=0).stop 2024-05-17T22:01:11.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.781+0000 7f9eb6726700 1 -- 172.21.15.107:0/178712821 shutdown_connections 2024-05-17T22:01:11.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.781+0000 7f9eb6726700 1 --2- 172.21.15.107:0/178712821 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9eb00ffef0 0x7f9eb0102310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:11.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.781+0000 7f9eb6726700 1 --2- 172.21.15.107:0/178712821 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9eb0066f30 0x7f9eb00ff9b0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:11.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.781+0000 7f9eb6726700 1 -- 172.21.15.107:0/178712821 >> 172.21.15.107:0/178712821 conn(0x7f9eb00f7440 msgr2=0x7f9eb00f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:01:11.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.781+0000 7f9eb6726700 1 -- 172.21.15.107:0/178712821 shutdown_connections 2024-05-17T22:01:11.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.781+0000 7f9eb6726700 1 -- 172.21.15.107:0/178712821 wait complete. 2024-05-17T22:01:11.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.781+0000 7f9eb6726700 1 Processor -- start 2024-05-17T22:01:11.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.781+0000 7f9eb6726700 1 -- start start 2024-05-17T22:01:11.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.782+0000 7f9eb6726700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9eb0066f30 0x7f9eb00fd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:11.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.782+0000 7f9eb6726700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9eb00ffef0 0x7f9eb00fb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:11.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.782+0000 7f9eb6726700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb00fd840 con 0x7f9eb00ffef0 2024-05-17T22:01:11.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.782+0000 7f9eb6726700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb00fbe20 con 0x7f9eb0066f30 2024-05-17T22:01:11.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.782+0000 7f9ea7fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9eb00ffef0 0x7f9eb00fb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:11.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.782+0000 7f9ea7fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9eb00ffef0 0x7f9eb00fb8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56370/0 (socket says 172.21.15.107:56370) 2024-05-17T22:01:11.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.782+0000 7f9ea7fff700 1 -- 172.21.15.107:0/243983761 learned_addr learned my addr 172.21.15.107:0/243983761 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T22:01:11.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.782+0000 7f9eaffff700 1 --2- 172.21.15.107:0/243983761 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9eb0066f30 0x7f9eb00fd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:11.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.782+0000 7f9ea7fff700 1 -- 172.21.15.107:0/243983761 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9eb0066f30 msgr2=0x7f9eb00fd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:11.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.782+0000 7f9ea7fff700 1 --2- 172.21.15.107:0/243983761 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9eb0066f30 0x7f9eb00fd1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:11.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.782+0000 7f9ea7fff700 1 -- 172.21.15.107:0/243983761 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ea0005040 con 0x7f9eb00ffef0 2024-05-17T22:01:11.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.782+0000 7f9ea7fff700 1 --2- 172.21.15.107:0/243983761 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9eb00ffef0 0x7f9eb00fb8b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9e9800b790 tx=0x7f9e98009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:11.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.782+0000 7f9ead7fa700 1 -- 172.21.15.107:0/243983761 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9e9800f6f0 con 0x7f9eb00ffef0 2024-05-17T22:01:11.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.782+0000 7f9ead7fa700 1 -- 172.21.15.107:0/243983761 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9e9800fd20 con 0x7f9eb00ffef0 2024-05-17T22:01:11.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.782+0000 7f9eb6726700 1 -- 172.21.15.107:0/243983761 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9eb00fc100 con 0x7f9eb00ffef0 2024-05-17T22:01:11.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.782+0000 7f9ead7fa700 1 -- 172.21.15.107:0/243983761 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9e980103a0 con 0x7f9eb00ffef0 2024-05-17T22:01:11.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.782+0000 7f9eb6726700 1 -- 172.21.15.107:0/243983761 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9eb00fc6c0 con 0x7f9eb00ffef0 2024-05-17T22:01:11.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.783+0000 7f9eb6726700 1 -- 172.21.15.107:0/243983761 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9eb004c710 con 0x7f9eb00ffef0 2024-05-17T22:01:11.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.784+0000 7f9ead7fa700 1 -- 172.21.15.107:0/243983761 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f9e98011400 con 0x7f9eb00ffef0 2024-05-17T22:01:11.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.785+0000 7f9ead7fa700 1 --2- 172.21.15.107:0/243983761 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f9e9c078990 0x7f9e9c07ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:11.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.785+0000 7f9ead7fa700 1 -- 172.21.15.107:0/243983761 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f9e9809a5d0 con 0x7f9eb00ffef0 2024-05-17T22:01:11.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.785+0000 7f9eaffff700 1 --2- 172.21.15.107:0/243983761 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f9e9c078990 0x7f9e9c07ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:11.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.785+0000 7f9eaffff700 1 --2- 172.21.15.107:0/243983761 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f9e9c078990 0x7f9e9c07ae40 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9ea0004890 tx=0x7f9ea0009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:11.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:11.789+0000 7f9ead7fa700 1 -- 172.21.15.107:0/243983761 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f9e98062210 con 0x7f9eb00ffef0 2024-05-17T22:01:12.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.087+0000 7f9eb6726700 1 -- 172.21.15.107:0/243983761 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f9eb0105920 con 0x7f9eb00ffef0 2024-05-17T22:01:12.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.088+0000 7f9ead7fa700 1 -- 172.21.15.107:0/243983761 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f9e98061bb0 con 0x7f9eb00ffef0 2024-05-17T22:01:12.080 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-05-17T22:01:12.080 INFO:teuthology.orchestra.run.smithi107.stdout: "mon": { 2024-05-17T22:01:12.080 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev)": 2 2024-05-17T22:01:12.080 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-05-17T22:01:12.080 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr": { 2024-05-17T22:01:12.080 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev)": 2 2024-05-17T22:01:12.080 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-05-17T22:01:12.080 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": { 2024-05-17T22:01:12.080 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev)": 6 2024-05-17T22:01:12.080 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-05-17T22:01:12.081 INFO:teuthology.orchestra.run.smithi107.stdout: "mds": { 2024-05-17T22:01:12.081 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev)": 4 2024-05-17T22:01:12.081 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-05-17T22:01:12.081 INFO:teuthology.orchestra.run.smithi107.stdout: "overall": { 2024-05-17T22:01:12.081 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev)": 14 2024-05-17T22:01:12.081 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-05-17T22:01:12.081 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-05-17T22:01:12.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.090+0000 7f9eb6726700 1 -- 172.21.15.107:0/243983761 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f9e9c078990 msgr2=0x7f9e9c07ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:12.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.090+0000 7f9eb6726700 1 --2- 172.21.15.107:0/243983761 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f9e9c078990 0x7f9e9c07ae40 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9ea0004890 tx=0x7f9ea0009e30 comp rx=0 tx=0).stop 2024-05-17T22:01:12.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.090+0000 7f9eb6726700 1 -- 172.21.15.107:0/243983761 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9eb00ffef0 msgr2=0x7f9eb00fb8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:12.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.090+0000 7f9eb6726700 1 --2- 172.21.15.107:0/243983761 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9eb00ffef0 0x7f9eb00fb8b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9e9800b790 tx=0x7f9e98009250 comp rx=0 tx=0).stop 2024-05-17T22:01:12.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.090+0000 7f9eb6726700 1 -- 172.21.15.107:0/243983761 shutdown_connections 2024-05-17T22:01:12.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.090+0000 7f9eb6726700 1 --2- 172.21.15.107:0/243983761 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f9e9c078990 0x7f9e9c07ae40 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:12.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.090+0000 7f9eb6726700 1 --2- 172.21.15.107:0/243983761 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9eb0066f30 0x7f9eb00fd1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:12.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.090+0000 7f9eb6726700 1 --2- 172.21.15.107:0/243983761 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9eb00ffef0 0x7f9eb00fb8b0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:12.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.090+0000 7f9eb6726700 1 -- 172.21.15.107:0/243983761 >> 172.21.15.107:0/243983761 conn(0x7f9eb00f7440 msgr2=0x7f9eb00f7810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:01:12.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.090+0000 7f9eb6726700 1 -- 172.21.15.107:0/243983761 shutdown_connections 2024-05-17T22:01:12.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.091+0000 7f9eb6726700 1 -- 172.21.15.107:0/243983761 wait complete. 2024-05-17T22:01:12.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.194+0000 7f5b8ce6d700 1 -- 172.21.15.107:0/2693531540 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5b880fcf90 msgr2=0x7f5b880fd400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:12.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.194+0000 7f5b8ce6d700 1 --2- 172.21.15.107:0/2693531540 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5b880fcf90 0x7f5b880fd400 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c004940 tx=0x7f5b7c009e30 comp rx=0 tx=0).stop 2024-05-17T22:01:12.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.194+0000 7f5b8ce6d700 1 -- 172.21.15.107:0/2693531540 shutdown_connections 2024-05-17T22:01:12.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.194+0000 7f5b8ce6d700 1 --2- 172.21.15.107:0/2693531540 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5b880fcf90 0x7f5b880fd400 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:12.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.194+0000 7f5b8ce6d700 1 --2- 172.21.15.107:0/2693531540 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b880fbce0 0x7f5b880fc0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:12.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.194+0000 7f5b8ce6d700 1 -- 172.21.15.107:0/2693531540 >> 172.21.15.107:0/2693531540 conn(0x7f5b880f7480 msgr2=0x7f5b880f9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:01:12.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.194+0000 7f5b8ce6d700 1 -- 172.21.15.107:0/2693531540 shutdown_connections 2024-05-17T22:01:12.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.194+0000 7f5b8ce6d700 1 -- 172.21.15.107:0/2693531540 wait complete. 2024-05-17T22:01:12.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.194+0000 7f5b8ce6d700 1 Processor -- start 2024-05-17T22:01:12.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.194+0000 7f5b8ce6d700 1 -- start start 2024-05-17T22:01:12.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.194+0000 7f5b8ce6d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5b880fbce0 0x7f5b8810d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:12.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.194+0000 7f5b8ce6d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b880fcf90 0x7f5b8810de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:12.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.194+0000 7f5b8ce6d700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b8810e440 con 0x7f5b880fbce0 2024-05-17T22:01:12.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.194+0000 7f5b8ce6d700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b8810e5b0 con 0x7f5b880fcf90 2024-05-17T22:01:12.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.195+0000 7f5b8659c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5b880fbce0 0x7f5b8810d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:12.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.195+0000 7f5b8659c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5b880fbce0 0x7f5b8810d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56384/0 (socket says 172.21.15.107:56384) 2024-05-17T22:01:12.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.195+0000 7f5b8659c700 1 -- 172.21.15.107:0/1268200970 learned_addr learned my addr 172.21.15.107:0/1268200970 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T22:01:12.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.195+0000 7f5b85d9b700 1 --2- 172.21.15.107:0/1268200970 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b880fcf90 0x7f5b8810de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:12.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.195+0000 7f5b8659c700 1 -- 172.21.15.107:0/1268200970 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b880fcf90 msgr2=0x7f5b8810de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:12.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.195+0000 7f5b8659c700 1 --2- 172.21.15.107:0/1268200970 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b880fcf90 0x7f5b8810de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:12.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.195+0000 7f5b8659c700 1 -- 172.21.15.107:0/1268200970 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b7c005040 con 0x7f5b880fbce0 2024-05-17T22:01:12.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.195+0000 7f5b85d9b700 1 --2- 172.21.15.107:0/1268200970 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b880fcf90 0x7f5b8810de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:01:12.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.195+0000 7f5b8659c700 1 --2- 172.21.15.107:0/1268200970 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5b880fbce0 0x7f5b8810d900 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5b78002a50 tx=0x7f5b780046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:12.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.195+0000 7f5b76ffd700 1 -- 172.21.15.107:0/1268200970 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b78007cc0 con 0x7f5b880fbce0 2024-05-17T22:01:12.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.195+0000 7f5b8ce6d700 1 -- 172.21.15.107:0/1268200970 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b88112fb0 con 0x7f5b880fbce0 2024-05-17T22:01:12.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.195+0000 7f5b76ffd700 1 -- 172.21.15.107:0/1268200970 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5b7800f040 con 0x7f5b880fbce0 2024-05-17T22:01:12.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.195+0000 7f5b76ffd700 1 -- 172.21.15.107:0/1268200970 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b78009640 con 0x7f5b880fbce0 2024-05-17T22:01:12.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.195+0000 7f5b8ce6d700 1 -- 172.21.15.107:0/1268200970 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b88113540 con 0x7f5b880fbce0 2024-05-17T22:01:12.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.196+0000 7f5b8ce6d700 1 -- 172.21.15.107:0/1268200970 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b8804c710 con 0x7f5b880fbce0 2024-05-17T22:01:12.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.197+0000 7f5b76ffd700 1 -- 172.21.15.107:0/1268200970 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f5b780097a0 con 0x7f5b880fbce0 2024-05-17T22:01:12.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.197+0000 7f5b76ffd700 1 --2- 172.21.15.107:0/1268200970 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f5b70078990 0x7f5b7007ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:12.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.197+0000 7f5b85d9b700 1 --2- 172.21.15.107:0/1268200970 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f5b70078990 0x7f5b7007ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:12.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.197+0000 7f5b76ffd700 1 -- 172.21.15.107:0/1268200970 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f5b7809a590 con 0x7f5b880fbce0 2024-05-17T22:01:12.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.198+0000 7f5b85d9b700 1 --2- 172.21.15.107:0/1268200970 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f5b70078990 0x7f5b7007ae40 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c00b3b0 tx=0x7f5b7c009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:12.193 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.202+0000 7f5b76ffd700 1 -- 172.21.15.107:0/1268200970 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f5b78062150 con 0x7f5b880fbce0 2024-05-17T22:01:12.304 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:12 smithi107 ceph-mon[93224]: pgmap v236: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-05-17T22:01:12.304 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:12 smithi107 ceph-mon[93224]: from='client.34506 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:01:12.304 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:12 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:01:12.304 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:12 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:01:12.324 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:12 smithi138 ceph-mon[81376]: pgmap v236: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-05-17T22:01:12.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:12 smithi138 ceph-mon[81376]: from='client.34506 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:01:12.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:12 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:01:12.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:12 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:01:12.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.453+0000 7f5b8ce6d700 1 -- 172.21.15.107:0/1268200970 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f5b88113760 con 0x7f5b880fbce0 2024-05-17T22:01:12.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.454+0000 7f5b76ffd700 1 -- 172.21.15.107:0/1268200970 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 35 v35) v1 ==== 76+0+1968 (secure 0 0 0) 0x7f5b78061af0 con 0x7f5b880fbce0 2024-05-17T22:01:12.446 INFO:teuthology.orchestra.run.smithi107.stdout:e35 2024-05-17T22:01:12.446 INFO:teuthology.orchestra.run.smithi107.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-17T22:01:12.446 INFO:teuthology.orchestra.run.smithi107.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-17T22:01:12.446 INFO:teuthology.orchestra.run.smithi107.stdout:legacy client fscid: 1 2024-05-17T22:01:12.446 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T22:01:12.446 INFO:teuthology.orchestra.run.smithi107.stdout:Filesystem 'cephfs' (1) 2024-05-17T22:01:12.446 INFO:teuthology.orchestra.run.smithi107.stdout:fs_name cephfs 2024-05-17T22:01:12.446 INFO:teuthology.orchestra.run.smithi107.stdout:epoch 35 2024-05-17T22:01:12.446 INFO:teuthology.orchestra.run.smithi107.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-17T22:01:12.446 INFO:teuthology.orchestra.run.smithi107.stdout:created 2024-05-17T21:49:16.836333+0000 2024-05-17T22:01:12.447 INFO:teuthology.orchestra.run.smithi107.stdout:modified 2024-05-17T22:00:33.850863+0000 2024-05-17T22:01:12.447 INFO:teuthology.orchestra.run.smithi107.stdout:tableserver 0 2024-05-17T22:01:12.447 INFO:teuthology.orchestra.run.smithi107.stdout:root 0 2024-05-17T22:01:12.447 INFO:teuthology.orchestra.run.smithi107.stdout:session_timeout 60 2024-05-17T22:01:12.447 INFO:teuthology.orchestra.run.smithi107.stdout:session_autoclose 300 2024-05-17T22:01:12.447 INFO:teuthology.orchestra.run.smithi107.stdout:max_file_size 1099511627776 2024-05-17T22:01:12.447 INFO:teuthology.orchestra.run.smithi107.stdout:max_xattr_size 65536 2024-05-17T22:01:12.447 INFO:teuthology.orchestra.run.smithi107.stdout:required_client_features {} 2024-05-17T22:01:12.447 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure 0 2024-05-17T22:01:12.447 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure_osd_epoch 84 2024-05-17T22:01:12.447 INFO:teuthology.orchestra.run.smithi107.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-17T22:01:12.447 INFO:teuthology.orchestra.run.smithi107.stdout:max_mds 1 2024-05-17T22:01:12.447 INFO:teuthology.orchestra.run.smithi107.stdout:in 0 2024-05-17T22:01:12.447 INFO:teuthology.orchestra.run.smithi107.stdout:up {0=34408} 2024-05-17T22:01:12.448 INFO:teuthology.orchestra.run.smithi107.stdout:failed 2024-05-17T22:01:12.448 INFO:teuthology.orchestra.run.smithi107.stdout:damaged 2024-05-17T22:01:12.448 INFO:teuthology.orchestra.run.smithi107.stdout:stopped 2024-05-17T22:01:12.448 INFO:teuthology.orchestra.run.smithi107.stdout:data_pools [3] 2024-05-17T22:01:12.448 INFO:teuthology.orchestra.run.smithi107.stdout:metadata_pool 2 2024-05-17T22:01:12.448 INFO:teuthology.orchestra.run.smithi107.stdout:inline_data enabled 2024-05-17T22:01:12.448 INFO:teuthology.orchestra.run.smithi107.stdout:balancer 2024-05-17T22:01:12.448 INFO:teuthology.orchestra.run.smithi107.stdout:bal_rank_mask -1 2024-05-17T22:01:12.448 INFO:teuthology.orchestra.run.smithi107.stdout:standby_count_wanted 1 2024-05-17T22:01:12.448 INFO:teuthology.orchestra.run.smithi107.stdout:qdb_cluster leader: 34408 members: 34408 2024-05-17T22:01:12.448 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.idmbgj{0:34408} state up:active seq 20 join_fscid=1 addr [v2:172.21.15.107:6826/2235359521,v1:172.21.15.107:6827/2235359521] compat {c=[1],r=[1],i=[1fff]}] 2024-05-17T22:01:12.448 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.drqody{0:34440} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.107:6828/369507752,v1:172.21.15.107:6829/369507752] compat {c=[1],r=[1],i=[1fff]}] 2024-05-17T22:01:12.448 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T22:01:12.448 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T22:01:12.449 INFO:teuthology.orchestra.run.smithi107.stdout:Standby daemons: 2024-05-17T22:01:12.449 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T22:01:12.449 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi138.gwbtyd{-1:44335} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6824/1700112281,v1:172.21.15.138:6825/1700112281] compat {c=[1],r=[1],i=[1fff]}] 2024-05-17T22:01:12.449 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi138.zckrvs{-1:44365} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6826/1801131392,v1:172.21.15.138:6827/1801131392] compat {c=[1],r=[1],i=[1fff]}] 2024-05-17T22:01:12.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.457+0000 7f5b8ce6d700 1 -- 172.21.15.107:0/1268200970 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f5b70078990 msgr2=0x7f5b7007ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:12.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.457+0000 7f5b8ce6d700 1 --2- 172.21.15.107:0/1268200970 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f5b70078990 0x7f5b7007ae40 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c00b3b0 tx=0x7f5b7c009e30 comp rx=0 tx=0).stop 2024-05-17T22:01:12.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.457+0000 7f5b8ce6d700 1 -- 172.21.15.107:0/1268200970 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5b880fbce0 msgr2=0x7f5b8810d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:12.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.457+0000 7f5b8ce6d700 1 --2- 172.21.15.107:0/1268200970 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5b880fbce0 0x7f5b8810d900 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5b78002a50 tx=0x7f5b780046b0 comp rx=0 tx=0).stop 2024-05-17T22:01:12.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.457+0000 7f5b8ce6d700 1 -- 172.21.15.107:0/1268200970 shutdown_connections 2024-05-17T22:01:12.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.457+0000 7f5b8ce6d700 1 --2- 172.21.15.107:0/1268200970 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f5b70078990 0x7f5b7007ae40 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:12.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.457+0000 7f5b8ce6d700 1 --2- 172.21.15.107:0/1268200970 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5b880fbce0 0x7f5b8810d900 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:12.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.457+0000 7f5b8ce6d700 1 --2- 172.21.15.107:0/1268200970 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b880fcf90 0x7f5b8810de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:12.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.457+0000 7f5b8ce6d700 1 -- 172.21.15.107:0/1268200970 >> 172.21.15.107:0/1268200970 conn(0x7f5b880f7480 msgr2=0x7f5b880f8b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:01:12.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.457+0000 7f5b8ce6d700 1 -- 172.21.15.107:0/1268200970 shutdown_connections 2024-05-17T22:01:12.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.457+0000 7f5b8ce6d700 1 -- 172.21.15.107:0/1268200970 wait complete. 2024-05-17T22:01:12.450 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 35 2024-05-17T22:01:12.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.561+0000 7fdb0852f700 1 -- 172.21.15.107:0/3956994581 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb000fcf60 msgr2=0x7fdb000fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:12.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.561+0000 7fdb0852f700 1 --2- 172.21.15.107:0/3956994581 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb000fcf60 0x7fdb000fd3d0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fdafc004940 tx=0x7fdafc009e30 comp rx=0 tx=0).stop 2024-05-17T22:01:12.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.561+0000 7fdb0852f700 1 -- 172.21.15.107:0/3956994581 shutdown_connections 2024-05-17T22:01:12.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.561+0000 7fdb0852f700 1 --2- 172.21.15.107:0/3956994581 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb000fcf60 0x7fdb000fd3d0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:12.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.561+0000 7fdb0852f700 1 --2- 172.21.15.107:0/3956994581 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb000fbcb0 0x7fdb000fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:12.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.561+0000 7fdb0852f700 1 -- 172.21.15.107:0/3956994581 >> 172.21.15.107:0/3956994581 conn(0x7fdb000f7430 msgr2=0x7fdb000f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:01:12.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.561+0000 7fdb0852f700 1 -- 172.21.15.107:0/3956994581 shutdown_connections 2024-05-17T22:01:12.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.561+0000 7fdb0852f700 1 -- 172.21.15.107:0/3956994581 wait complete. 2024-05-17T22:01:12.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.562+0000 7fdb0852f700 1 Processor -- start 2024-05-17T22:01:12.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.562+0000 7fdb0852f700 1 -- start start 2024-05-17T22:01:12.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.562+0000 7fdb0852f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb000fbcb0 0x7fdb00110720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:12.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.562+0000 7fdb0852f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb000fcf60 0x7fdb00112c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:12.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.562+0000 7fdb0852f700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb00066330 con 0x7fdb000fbcb0 2024-05-17T22:01:12.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.562+0000 7fdb0852f700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb00066470 con 0x7fdb000fcf60 2024-05-17T22:01:12.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.562+0000 7fdb062cb700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb000fbcb0 0x7fdb00110720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:12.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.562+0000 7fdb062cb700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb000fbcb0 0x7fdb00110720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56404/0 (socket says 172.21.15.107:56404) 2024-05-17T22:01:12.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.562+0000 7fdb062cb700 1 -- 172.21.15.107:0/2529071407 learned_addr learned my addr 172.21.15.107:0/2529071407 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T22:01:12.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.562+0000 7fdb05aca700 1 --2- 172.21.15.107:0/2529071407 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb000fcf60 0x7fdb00112c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:12.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.562+0000 7fdb062cb700 1 -- 172.21.15.107:0/2529071407 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb000fcf60 msgr2=0x7fdb00112c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:12.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.562+0000 7fdb062cb700 1 --2- 172.21.15.107:0/2529071407 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb000fcf60 0x7fdb00112c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:12.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.562+0000 7fdb062cb700 1 -- 172.21.15.107:0/2529071407 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdafc005040 con 0x7fdb000fbcb0 2024-05-17T22:01:12.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.562+0000 7fdb062cb700 1 --2- 172.21.15.107:0/2529071407 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb000fbcb0 0x7fdb00110720 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fdaf4002a50 tx=0x7fdaf40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:12.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.563+0000 7fdaf2ffd700 1 -- 172.21.15.107:0/2529071407 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdaf4007cc0 con 0x7fdb000fbcb0 2024-05-17T22:01:12.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.563+0000 7fdaf2ffd700 1 -- 172.21.15.107:0/2529071407 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdaf400f040 con 0x7fdb000fbcb0 2024-05-17T22:01:12.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.563+0000 7fdb0852f700 1 -- 172.21.15.107:0/2529071407 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb00113210 con 0x7fdb000fbcb0 2024-05-17T22:01:12.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.563+0000 7fdaf2ffd700 1 -- 172.21.15.107:0/2529071407 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdaf4009640 con 0x7fdb000fbcb0 2024-05-17T22:01:12.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.563+0000 7fdb0852f700 1 -- 172.21.15.107:0/2529071407 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb001137d0 con 0x7fdb000fbcb0 2024-05-17T22:01:12.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.564+0000 7fdb0852f700 1 -- 172.21.15.107:0/2529071407 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb0005f5c0 con 0x7fdb000fbcb0 2024-05-17T22:01:12.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.564+0000 7fdaf2ffd700 1 -- 172.21.15.107:0/2529071407 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7fdaf40097c0 con 0x7fdb000fbcb0 2024-05-17T22:01:12.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.565+0000 7fdaf2ffd700 1 --2- 172.21.15.107:0/2529071407 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fdaec078990 0x7fdaec07ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:12.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.565+0000 7fdaf2ffd700 1 -- 172.21.15.107:0/2529071407 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7fdaf409a4a0 con 0x7fdb000fbcb0 2024-05-17T22:01:12.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.565+0000 7fdb05aca700 1 --2- 172.21.15.107:0/2529071407 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fdaec078990 0x7fdaec07ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:12.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.565+0000 7fdb05aca700 1 --2- 172.21.15.107:0/2529071407 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fdaec078990 0x7fdaec07ae40 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fdafc00b3b0 tx=0x7fdafc009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:12.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.570+0000 7fdaf2ffd700 1 -- 172.21.15.107:0/2529071407 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7fdaf40620e0 con 0x7fdb000fbcb0 2024-05-17T22:01:12.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.795+0000 7fdb0852f700 1 -- 172.21.15.107:0/2529071407 --> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdb001012c0 con 0x7fdaec078990 2024-05-17T22:01:12.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.797+0000 7fdaf2ffd700 1 -- 172.21.15.107:0/2529071407 <== mgr.34100 v2:172.21.15.107:6800/3205268226 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+438 (secure 0 0 0) 0x7fdb001012c0 con 0x7fdaec078990 2024-05-17T22:01:12.788 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-05-17T22:01:12.788 INFO:teuthology.orchestra.run.smithi107.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b", 2024-05-17T22:01:12.788 INFO:teuthology.orchestra.run.smithi107.stdout: "in_progress": true, 2024-05-17T22:01:12.788 INFO:teuthology.orchestra.run.smithi107.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-17T22:01:12.788 INFO:teuthology.orchestra.run.smithi107.stdout: "services_complete": [ 2024-05-17T22:01:12.788 INFO:teuthology.orchestra.run.smithi107.stdout: "mon", 2024-05-17T22:01:12.788 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr", 2024-05-17T22:01:12.788 INFO:teuthology.orchestra.run.smithi107.stdout: "mds", 2024-05-17T22:01:12.788 INFO:teuthology.orchestra.run.smithi107.stdout: "osd", 2024-05-17T22:01:12.788 INFO:teuthology.orchestra.run.smithi107.stdout: "crash" 2024-05-17T22:01:12.789 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-05-17T22:01:12.789 INFO:teuthology.orchestra.run.smithi107.stdout: "progress": "16/21 daemons upgraded", 2024-05-17T22:01:12.789 INFO:teuthology.orchestra.run.smithi107.stdout: "message": "Currently upgrading prometheus daemons", 2024-05-17T22:01:12.789 INFO:teuthology.orchestra.run.smithi107.stdout: "is_paused": false 2024-05-17T22:01:12.789 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-05-17T22:01:12.790 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.799+0000 7fdb0852f700 1 -- 172.21.15.107:0/2529071407 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fdaec078990 msgr2=0x7fdaec07ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:12.790 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.799+0000 7fdb0852f700 1 --2- 172.21.15.107:0/2529071407 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fdaec078990 0x7fdaec07ae40 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fdafc00b3b0 tx=0x7fdafc009e30 comp rx=0 tx=0).stop 2024-05-17T22:01:12.790 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.799+0000 7fdb0852f700 1 -- 172.21.15.107:0/2529071407 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb000fbcb0 msgr2=0x7fdb00110720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:12.790 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.799+0000 7fdb0852f700 1 --2- 172.21.15.107:0/2529071407 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb000fbcb0 0x7fdb00110720 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fdaf4002a50 tx=0x7fdaf40046b0 comp rx=0 tx=0).stop 2024-05-17T22:01:12.790 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.799+0000 7fdb0852f700 1 -- 172.21.15.107:0/2529071407 shutdown_connections 2024-05-17T22:01:12.791 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.799+0000 7fdb0852f700 1 --2- 172.21.15.107:0/2529071407 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fdaec078990 0x7fdaec07ae40 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:12.791 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.799+0000 7fdb0852f700 1 --2- 172.21.15.107:0/2529071407 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb000fbcb0 0x7fdb00110720 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:12.791 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.799+0000 7fdb0852f700 1 --2- 172.21.15.107:0/2529071407 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb000fcf60 0x7fdb00112c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:12.791 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.799+0000 7fdb0852f700 1 -- 172.21.15.107:0/2529071407 >> 172.21.15.107:0/2529071407 conn(0x7fdb000f7430 msgr2=0x7fdb000f8b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:01:12.791 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.799+0000 7fdb0852f700 1 -- 172.21.15.107:0/2529071407 shutdown_connections 2024-05-17T22:01:12.791 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.799+0000 7fdb0852f700 1 -- 172.21.15.107:0/2529071407 wait complete. 2024-05-17T22:01:12.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.904+0000 7f1cbbd39700 1 -- 172.21.15.107:0/1437481980 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1cb40fbcb0 msgr2=0x7f1cb40fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:12.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.904+0000 7f1cbbd39700 1 --2- 172.21.15.107:0/1437481980 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1cb40fbcb0 0x7f1cb40fc0c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f1cb0004940 tx=0x7f1cb0009e30 comp rx=0 tx=0).stop 2024-05-17T22:01:12.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.904+0000 7f1cbbd39700 1 -- 172.21.15.107:0/1437481980 shutdown_connections 2024-05-17T22:01:12.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.904+0000 7f1cbbd39700 1 --2- 172.21.15.107:0/1437481980 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1cb40fcf60 0x7f1cb40fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:12.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.904+0000 7f1cbbd39700 1 --2- 172.21.15.107:0/1437481980 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1cb40fbcb0 0x7f1cb40fc0c0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:12.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.904+0000 7f1cbbd39700 1 -- 172.21.15.107:0/1437481980 >> 172.21.15.107:0/1437481980 conn(0x7f1cb40f7430 msgr2=0x7f1cb40f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:01:12.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.904+0000 7f1cbbd39700 1 -- 172.21.15.107:0/1437481980 shutdown_connections 2024-05-17T22:01:12.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.904+0000 7f1cbbd39700 1 -- 172.21.15.107:0/1437481980 wait complete. 2024-05-17T22:01:12.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.904+0000 7f1cbbd39700 1 Processor -- start 2024-05-17T22:01:12.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.904+0000 7f1cbbd39700 1 -- start start 2024-05-17T22:01:12.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.905+0000 7f1cbbd39700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1cb40fbcb0 0x7f1cb410c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:12.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.905+0000 7f1cbbd39700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1cb40fcf60 0x7f1cb410ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:12.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.905+0000 7f1cbbd39700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb410d440 con 0x7f1cb40fcf60 2024-05-17T22:01:12.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.905+0000 7f1cbbd39700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb410dc90 con 0x7f1cb40fbcb0 2024-05-17T22:01:12.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.905+0000 7f1cb92d4700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1cb40fcf60 0x7f1cb410ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:12.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.905+0000 7f1cb92d4700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1cb40fcf60 0x7f1cb410ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56426/0 (socket says 172.21.15.107:56426) 2024-05-17T22:01:12.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.905+0000 7f1cb92d4700 1 -- 172.21.15.107:0/4062714770 learned_addr learned my addr 172.21.15.107:0/4062714770 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T22:01:12.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.905+0000 7f1cb9ad5700 1 --2- 172.21.15.107:0/4062714770 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1cb40fbcb0 0x7f1cb410c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:12.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.905+0000 7f1cb92d4700 1 -- 172.21.15.107:0/4062714770 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1cb40fbcb0 msgr2=0x7f1cb410c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:12.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.905+0000 7f1cb92d4700 1 --2- 172.21.15.107:0/4062714770 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1cb40fbcb0 0x7f1cb410c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:12.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.905+0000 7f1cb92d4700 1 -- 172.21.15.107:0/4062714770 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cb0005040 con 0x7f1cb40fcf60 2024-05-17T22:01:12.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.905+0000 7f1cb92d4700 1 --2- 172.21.15.107:0/4062714770 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1cb40fcf60 0x7f1cb410ce40 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1ca4002a50 tx=0x7f1ca40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:12.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.905+0000 7f1caa7fc700 1 -- 172.21.15.107:0/4062714770 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1ca4007cc0 con 0x7f1cb40fcf60 2024-05-17T22:01:12.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.906+0000 7f1caa7fc700 1 -- 172.21.15.107:0/4062714770 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1ca400f040 con 0x7f1cb40fcf60 2024-05-17T22:01:12.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.906+0000 7f1cbbd39700 1 -- 172.21.15.107:0/4062714770 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cb410df70 con 0x7f1cb40fcf60 2024-05-17T22:01:12.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.906+0000 7f1caa7fc700 1 -- 172.21.15.107:0/4062714770 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1ca4009640 con 0x7f1cb40fcf60 2024-05-17T22:01:12.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.906+0000 7f1cbbd39700 1 -- 172.21.15.107:0/4062714770 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cb410e4b0 con 0x7f1cb40fcf60 2024-05-17T22:01:12.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.907+0000 7f1cbbd39700 1 -- 172.21.15.107:0/4062714770 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cb405f5c0 con 0x7f1cb40fcf60 2024-05-17T22:01:12.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.907+0000 7f1caa7fc700 1 -- 172.21.15.107:0/4062714770 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f1ca40097a0 con 0x7f1cb40fcf60 2024-05-17T22:01:12.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.908+0000 7f1caa7fc700 1 --2- 172.21.15.107:0/4062714770 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f1ca007ab90 0x7f1ca007d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:12.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.908+0000 7f1caa7fc700 1 -- 172.21.15.107:0/4062714770 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f1ca409a5b0 con 0x7f1cb40fcf60 2024-05-17T22:01:12.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.908+0000 7f1cb9ad5700 1 --2- 172.21.15.107:0/4062714770 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f1ca007ab90 0x7f1ca007d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:12.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.908+0000 7f1cb9ad5700 1 --2- 172.21.15.107:0/4062714770 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f1ca007ab90 0x7f1ca007d040 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1cb000b970 tx=0x7f1cb0009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:12.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:12.913+0000 7f1caa7fc700 1 -- 172.21.15.107:0/4062714770 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f1ca4062170 con 0x7f1cb40fcf60 2024-05-17T22:01:13.173 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:13 smithi107 ceph-mon[93224]: from='client.34510 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:01:13.173 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:13 smithi107 ceph-mon[93224]: from='client.34514 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:01:13.173 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:13 smithi107 ceph-mon[93224]: from='client.? 172.21.15.107:0/243983761' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:01:13.173 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:13 smithi107 ceph-mon[93224]: from='client.? 172.21.15.107:0/1268200970' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-17T22:01:13.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:13.223+0000 7f1cbbd39700 1 -- 172.21.15.107:0/4062714770 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f1cb404bed0 con 0x7f1cb40fcf60 2024-05-17T22:01:13.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:13.223+0000 7f1caa7fc700 1 -- 172.21.15.107:0/4062714770 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f1ca4061b10 con 0x7f1cb40fcf60 2024-05-17T22:01:13.218 INFO:teuthology.orchestra.run.smithi107.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-17T22:01:13.218 INFO:teuthology.orchestra.run.smithi107.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-17T22:01:13.218 INFO:teuthology.orchestra.run.smithi107.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-17T22:01:13.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:13.226+0000 7f1cbbd39700 1 -- 172.21.15.107:0/4062714770 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f1ca007ab90 msgr2=0x7f1ca007d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:13.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:13.226+0000 7f1cbbd39700 1 --2- 172.21.15.107:0/4062714770 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f1ca007ab90 0x7f1ca007d040 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1cb000b970 tx=0x7f1cb0009e30 comp rx=0 tx=0).stop 2024-05-17T22:01:13.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:13.226+0000 7f1cbbd39700 1 -- 172.21.15.107:0/4062714770 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1cb40fcf60 msgr2=0x7f1cb410ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:13.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:13.226+0000 7f1cbbd39700 1 --2- 172.21.15.107:0/4062714770 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1cb40fcf60 0x7f1cb410ce40 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1ca4002a50 tx=0x7f1ca40046b0 comp rx=0 tx=0).stop 2024-05-17T22:01:13.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:13.226+0000 7f1cbbd39700 1 -- 172.21.15.107:0/4062714770 shutdown_connections 2024-05-17T22:01:13.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:13.226+0000 7f1cbbd39700 1 --2- 172.21.15.107:0/4062714770 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f1ca007ab90 0x7f1ca007d040 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:13.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:13.226+0000 7f1cbbd39700 1 --2- 172.21.15.107:0/4062714770 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1cb40fbcb0 0x7f1cb410c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:13.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:13.226+0000 7f1cbbd39700 1 --2- 172.21.15.107:0/4062714770 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1cb40fcf60 0x7f1cb410ce40 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:13.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:13.226+0000 7f1cbbd39700 1 -- 172.21.15.107:0/4062714770 >> 172.21.15.107:0/4062714770 conn(0x7f1cb40f7430 msgr2=0x7f1cb40f8b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:01:13.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:13.226+0000 7f1cbbd39700 1 -- 172.21.15.107:0/4062714770 shutdown_connections 2024-05-17T22:01:13.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:13.226+0000 7f1cbbd39700 1 -- 172.21.15.107:0/4062714770 wait complete. 2024-05-17T22:01:13.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:13 smithi138 ceph-mon[81376]: from='client.34510 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:01:13.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:13 smithi138 ceph-mon[81376]: from='client.34514 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:01:13.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:13 smithi138 ceph-mon[81376]: from='client.? 172.21.15.107:0/243983761' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:01:13.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:13 smithi138 ceph-mon[81376]: from='client.? 172.21.15.107:0/1268200970' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-17T22:01:14.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:14 smithi107 ceph-mon[93224]: pgmap v237: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:01:14.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:14 smithi107 ceph-mon[93224]: from='client.34526 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:01:14.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:14 smithi107 ceph-mon[93224]: from='client.? 172.21.15.107:0/4062714770' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-17T22:01:14.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:14 smithi138 ceph-mon[81376]: pgmap v237: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:01:14.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:14 smithi138 ceph-mon[81376]: from='client.34526 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:01:14.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:14 smithi138 ceph-mon[81376]: from='client.? 172.21.15.107:0/4062714770' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-17T22:01:16.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:15 smithi138 ceph-mon[81376]: pgmap v238: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:01:16.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:15 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:01:16.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:15 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:01:16.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:15 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:01:16.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:15 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:01:16.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:15 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:01:16.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:15 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-17T22:01:16.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:15 smithi138 ceph-mon[81376]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-17T22:01:16.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:15 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:01:16.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:15 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:01:16.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:15 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:01:16.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:15 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:01:16.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:15 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:01:16.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:15 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:01:16.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:15 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:01:16.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:15 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:01:16.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:15 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:01:16.076 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:15 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:01:16.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:15 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:01:16.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:15 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:01:16.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:15 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:01:16.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:15 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:01:16.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:15 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:01:16.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:15 smithi107 ceph-mon[93224]: pgmap v238: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:01:16.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:15 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:01:16.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:15 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:01:16.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:15 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:01:16.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:15 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:01:16.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:15 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:01:16.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:15 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-17T22:01:16.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:15 smithi107 ceph-mon[93224]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-17T22:01:16.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:15 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:01:16.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:15 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:01:16.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:15 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:01:16.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:15 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:01:16.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:15 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:01:16.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:15 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:01:16.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:15 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:01:16.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:15 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:01:16.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:15 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:01:16.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:15 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:01:16.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:15 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:01:16.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:15 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:01:16.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:15 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:01:16.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:15 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:01:16.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:15 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:01:18.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:17 smithi138 ceph-mon[81376]: Upgrade: Updating alertmanager.smithi107 2024-05-17T22:01:18.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:17 smithi138 ceph-mon[81376]: Deploying daemon alertmanager.smithi107 on smithi107 2024-05-17T22:01:18.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:17 smithi138 ceph-mon[81376]: pgmap v239: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:01:18.100 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:17 smithi107 ceph-mon[93224]: Upgrade: Updating alertmanager.smithi107 2024-05-17T22:01:18.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:17 smithi107 ceph-mon[93224]: Deploying daemon alertmanager.smithi107 on smithi107 2024-05-17T22:01:18.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:17 smithi107 ceph-mon[93224]: pgmap v239: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:01:19.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:19 smithi107 ceph-mon[93224]: pgmap v240: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:01:19.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:19 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T22:01:20.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:19 smithi138 ceph-mon[81376]: pgmap v240: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:01:20.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:19 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T22:01:21.820 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:21 smithi107 ceph-mon[93224]: pgmap v241: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:01:22.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:21 smithi138 ceph-mon[81376]: pgmap v241: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:01:23.481 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:23 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:01:23.481 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:23 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:01:23.481 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:23 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:01:23.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:23 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:01:23.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:23 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:01:23.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:23 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:01:24.436 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:24 smithi107 ceph-mon[93224]: pgmap v242: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:01:24.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:24 smithi138 ceph-mon[81376]: pgmap v242: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:01:26.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:26 smithi107 ceph-mon[93224]: pgmap v243: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-05-17T22:01:26.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:26 smithi138 ceph-mon[81376]: pgmap v243: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-05-17T22:01:27.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:27 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:01:27.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:27 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:01:27.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:27 smithi138 ceph-mon[81376]: pgmap v244: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-05-17T22:01:27.850 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:27 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:01:27.881 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:27 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:01:27.881 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:27 smithi107 ceph-mon[93224]: pgmap v244: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-05-17T22:01:28.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:28 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:01:28.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:28 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:01:28.827 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:28 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:01:28.827 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:28 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:01:29.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:29 smithi138 ceph-mon[81376]: pgmap v245: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:01:29.850 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:29 smithi107 ceph-mon[93224]: pgmap v245: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:01:31.930 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:31 smithi107 ceph-mon[93224]: pgmap v246: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-05-17T22:01:31.930 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:31 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:01:31.931 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:31 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:01:31.931 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:31 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:01:31.931 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:31 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:01:31.931 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:31 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:01:31.931 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:31 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-05-17T22:01:31.931 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:31 smithi107 ceph-mon[93224]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-05-17T22:01:31.931 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:31 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:01:31.931 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:31 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:01:31.931 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:31 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:01:31.931 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:31 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:01:31.931 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:31 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:01:31.931 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:31 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:01:31.931 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:31 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:01:31.931 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:31 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:01:31.932 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:31 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:01:31.932 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:31 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:01:31.932 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:31 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:01:31.932 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:31 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:01:31.932 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:31 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:01:31.932 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:31 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:01:31.932 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:31 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:01:31.932 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:31 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:01:32.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:31 smithi138 ceph-mon[81376]: pgmap v246: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-05-17T22:01:32.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:31 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:01:32.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:31 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:01:32.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:31 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:01:32.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:31 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:01:32.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:31 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:01:32.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:31 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-05-17T22:01:32.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:31 smithi138 ceph-mon[81376]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-05-17T22:01:32.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:31 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:01:32.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:31 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:01:32.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:31 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:01:32.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:31 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:01:32.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:31 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:01:32.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:31 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:01:32.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:31 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:01:32.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:31 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:01:32.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:31 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:01:32.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:31 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:01:32.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:31 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:01:32.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:31 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:01:32.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:31 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:01:32.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:31 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:01:32.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:31 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:01:32.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:31 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:01:34.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:33 smithi138 ceph-mon[81376]: pgmap v247: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:01:34.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:33 smithi138 ceph-mon[81376]: Upgrade: Updating grafana.smithi107 2024-05-17T22:01:34.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:33 smithi138 ceph-mon[81376]: Deploying daemon grafana.smithi107 on smithi107 2024-05-17T22:01:34.350 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:33 smithi107 ceph-mon[93224]: pgmap v247: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:01:34.350 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:33 smithi107 ceph-mon[93224]: Upgrade: Updating grafana.smithi107 2024-05-17T22:01:34.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:33 smithi107 ceph-mon[93224]: Deploying daemon grafana.smithi107 on smithi107 2024-05-17T22:01:35.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:34 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T22:01:35.350 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:34 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T22:01:36.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:35 smithi138 ceph-mon[81376]: pgmap v248: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:01:36.350 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:35 smithi107 ceph-mon[93224]: pgmap v248: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:01:38.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:37 smithi138 ceph-mon[81376]: pgmap v249: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:01:38.350 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:37 smithi107 ceph-mon[93224]: pgmap v249: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:01:40.089 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:39 smithi107 ceph-mon[93224]: pgmap v250: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:01:40.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:39 smithi138 ceph-mon[81376]: pgmap v250: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:01:42.100 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:41 smithi107 ceph-mon[93224]: pgmap v251: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:01:42.324 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:41 smithi138 ceph-mon[81376]: pgmap v251: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:01:43.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:43.342+0000 7f421dedc700 1 -- 172.21.15.107:0/3247417916 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f421806ecc0 msgr2=0x7f421806f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:43.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:43.342+0000 7f421dedc700 1 --2- 172.21.15.107:0/3247417916 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f421806ecc0 0x7f421806f130 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f420c004940 tx=0x7f420c009e30 comp rx=0 tx=0).stop 2024-05-17T22:01:43.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:43.342+0000 7f421dedc700 1 -- 172.21.15.107:0/3247417916 shutdown_connections 2024-05-17T22:01:43.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:43.342+0000 7f421dedc700 1 --2- 172.21.15.107:0/3247417916 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f421806ecc0 0x7f421806f130 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:43.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:43.342+0000 7f421dedc700 1 --2- 172.21.15.107:0/3247417916 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f421806fd50 0x7f421806e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:43.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:43.342+0000 7f421dedc700 1 -- 172.21.15.107:0/3247417916 >> 172.21.15.107:0/3247417916 conn(0x7f421806a360 msgr2=0x7f421806c790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:01:43.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:43.342+0000 7f421dedc700 1 -- 172.21.15.107:0/3247417916 shutdown_connections 2024-05-17T22:01:43.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:43.343+0000 7f421dedc700 1 -- 172.21.15.107:0/3247417916 wait complete. 2024-05-17T22:01:43.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:43.343+0000 7f421dedc700 1 Processor -- start 2024-05-17T22:01:43.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:43.343+0000 7f421dedc700 1 -- start start 2024-05-17T22:01:43.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:43.343+0000 7f421dedc700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f421806ecc0 0x7f4218116160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:43.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:43.343+0000 7f421dedc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f421806fd50 0x7f42181166a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:43.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:43.343+0000 7f421dedc700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4218116ca0 con 0x7f421806fd50 2024-05-17T22:01:43.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:43.343+0000 7f421dedc700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4218116e10 con 0x7f421806ecc0 2024-05-17T22:01:43.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:43.344+0000 7f4216ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f421806fd50 0x7f42181166a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:43.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:43.344+0000 7f4216ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f421806fd50 0x7f42181166a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46638/0 (socket says 172.21.15.107:46638) 2024-05-17T22:01:43.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:43.344+0000 7f4216ffd700 1 -- 172.21.15.107:0/3503443903 learned_addr learned my addr 172.21.15.107:0/3503443903 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T22:01:43.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:43.344+0000 7f42177fe700 1 --2- 172.21.15.107:0/3503443903 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f421806ecc0 0x7f4218116160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:43.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:43.344+0000 7f4216ffd700 1 -- 172.21.15.107:0/3503443903 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f421806ecc0 msgr2=0x7f4218116160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:43.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:43.344+0000 7f4216ffd700 1 --2- 172.21.15.107:0/3503443903 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f421806ecc0 0x7f4218116160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:43.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:43.344+0000 7f4216ffd700 1 -- 172.21.15.107:0/3503443903 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f420c005040 con 0x7f421806fd50 2024-05-17T22:01:43.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:43.344+0000 7f4216ffd700 1 --2- 172.21.15.107:0/3503443903 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f421806fd50 0x7f42181166a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f420c009150 tx=0x7f420c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:43.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:43.344+0000 7f41fffff700 1 -- 172.21.15.107:0/3503443903 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f420c00e040 con 0x7f421806fd50 2024-05-17T22:01:43.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:43.344+0000 7f421dedc700 1 -- 172.21.15.107:0/3503443903 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f421811b7c0 con 0x7f421806fd50 2024-05-17T22:01:43.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:43.344+0000 7f41fffff700 1 -- 172.21.15.107:0/3503443903 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f420c00bbf0 con 0x7f421806fd50 2024-05-17T22:01:43.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:43.344+0000 7f41fffff700 1 -- 172.21.15.107:0/3503443903 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f420c01c630 con 0x7f421806fd50 2024-05-17T22:01:43.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:43.344+0000 7f421dedc700 1 -- 172.21.15.107:0/3503443903 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f421811bd80 con 0x7f421806fd50 2024-05-17T22:01:43.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:43.345+0000 7f421dedc700 1 -- 172.21.15.107:0/3503443903 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f421804c710 con 0x7f421806fd50 2024-05-17T22:01:43.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:43.346+0000 7f41fffff700 1 -- 172.21.15.107:0/3503443903 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f420c00bd60 con 0x7f421806fd50 2024-05-17T22:01:43.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:43.346+0000 7f41fffff700 1 --2- 172.21.15.107:0/3503443903 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f4200078980 0x7f420007ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:43.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:43.347+0000 7f42177fe700 1 --2- 172.21.15.107:0/3503443903 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f4200078980 0x7f420007ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:43.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:43.347+0000 7f41fffff700 1 -- 172.21.15.107:0/3503443903 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f420c09bb10 con 0x7f421806fd50 2024-05-17T22:01:43.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:43.347+0000 7f42177fe700 1 --2- 172.21.15.107:0/3503443903 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f4200078980 0x7f420007ae30 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4218066ec0 tx=0x7f420800b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:43.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:43.351+0000 7f41fffff700 1 -- 172.21.15.107:0/3503443903 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f420c063750 con 0x7f421806fd50 2024-05-17T22:01:43.585 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:43.594+0000 7f421dedc700 1 -- 172.21.15.107:0/3503443903 --> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f421810a2c0 con 0x7f4200078980 2024-05-17T22:01:43.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:43.595+0000 7f41fffff700 1 -- 172.21.15.107:0/3503443903 <== mgr.34100 v2:172.21.15.107:6800/3205268226 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+435 (secure 0 0 0) 0x7f421810a2c0 con 0x7f4200078980 2024-05-17T22:01:43.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:43.597+0000 7f421dedc700 1 -- 172.21.15.107:0/3503443903 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f4200078980 msgr2=0x7f420007ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:43.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:43.597+0000 7f421dedc700 1 --2- 172.21.15.107:0/3503443903 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f4200078980 0x7f420007ae30 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4218066ec0 tx=0x7f420800b040 comp rx=0 tx=0).stop 2024-05-17T22:01:43.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:43.597+0000 7f421dedc700 1 -- 172.21.15.107:0/3503443903 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f421806fd50 msgr2=0x7f42181166a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:43.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:43.597+0000 7f421dedc700 1 --2- 172.21.15.107:0/3503443903 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f421806fd50 0x7f42181166a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f420c009150 tx=0x7f420c009e30 comp rx=0 tx=0).stop 2024-05-17T22:01:43.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:43.597+0000 7f421dedc700 1 -- 172.21.15.107:0/3503443903 shutdown_connections 2024-05-17T22:01:43.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:43.597+0000 7f421dedc700 1 --2- 172.21.15.107:0/3503443903 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f4200078980 0x7f420007ae30 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:43.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:43.597+0000 7f421dedc700 1 --2- 172.21.15.107:0/3503443903 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f421806ecc0 0x7f4218116160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:43.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:43.597+0000 7f421dedc700 1 --2- 172.21.15.107:0/3503443903 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f421806fd50 0x7f42181166a0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:43.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:43.597+0000 7f421dedc700 1 -- 172.21.15.107:0/3503443903 >> 172.21.15.107:0/3503443903 conn(0x7f421806a360 msgr2=0x7f421806ca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:01:43.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:43.598+0000 7f421dedc700 1 -- 172.21.15.107:0/3503443903 shutdown_connections 2024-05-17T22:01:43.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:43.598+0000 7f421dedc700 1 -- 172.21.15.107:0/3503443903 wait complete. 2024-05-17T22:01:43.599 INFO:teuthology.orchestra.run.smithi107.stdout:true 2024-05-17T22:01:43.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:43.713+0000 7fa3f2046700 1 -- 172.21.15.107:0/3687640183 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa3ec0fed30 msgr2=0x7fa3ec0fae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:43.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:43.713+0000 7fa3f2046700 1 --2- 172.21.15.107:0/3687640183 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa3ec0fed30 0x7fa3ec0fae70 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fa3d4004940 tx=0x7fa3d4009e30 comp rx=0 tx=0).stop 2024-05-17T22:01:43.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:43.713+0000 7fa3f2046700 1 -- 172.21.15.107:0/3687640183 shutdown_connections 2024-05-17T22:01:43.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:43.713+0000 7fa3f2046700 1 --2- 172.21.15.107:0/3687640183 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa3ec0fed30 0x7fa3ec0fae70 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:43.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:43.713+0000 7fa3f2046700 1 --2- 172.21.15.107:0/3687640183 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3ec0fe3e0 0x7fa3ec0fe7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:43.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:43.713+0000 7fa3f2046700 1 -- 172.21.15.107:0/3687640183 >> 172.21.15.107:0/3687640183 conn(0x7fa3ec0f6b80 msgr2=0x7fa3ec0f8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:01:43.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:43.714+0000 7fa3f2046700 1 -- 172.21.15.107:0/3687640183 shutdown_connections 2024-05-17T22:01:43.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:43.714+0000 7fa3f2046700 1 -- 172.21.15.107:0/3687640183 wait complete. 2024-05-17T22:01:43.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:43.714+0000 7fa3f2046700 1 Processor -- start 2024-05-17T22:01:43.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:43.714+0000 7fa3f2046700 1 -- start start 2024-05-17T22:01:43.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:43.715+0000 7fa3f2046700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa3ec0fe3e0 0x7fa3ec10d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:43.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:43.715+0000 7fa3f2046700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3ec0fed30 0x7fa3ec10df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:43.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:43.715+0000 7fa3f2046700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3ec10e500 con 0x7fa3ec0fe3e0 2024-05-17T22:01:43.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:43.715+0000 7fa3f2046700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3ec10ed50 con 0x7fa3ec0fed30 2024-05-17T22:01:43.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:43.715+0000 7fa3eb7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa3ec0fe3e0 0x7fa3ec10d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:43.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:43.715+0000 7fa3eb7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa3ec0fe3e0 0x7fa3ec10d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46654/0 (socket says 172.21.15.107:46654) 2024-05-17T22:01:43.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:43.715+0000 7fa3eb7fe700 1 -- 172.21.15.107:0/2895345536 learned_addr learned my addr 172.21.15.107:0/2895345536 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T22:01:43.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:43.715+0000 7fa3e2ffd700 1 --2- 172.21.15.107:0/2895345536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3ec0fed30 0x7fa3ec10df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:43.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:43.715+0000 7fa3eb7fe700 1 -- 172.21.15.107:0/2895345536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3ec0fed30 msgr2=0x7fa3ec10df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:43.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:43.715+0000 7fa3eb7fe700 1 --2- 172.21.15.107:0/2895345536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3ec0fed30 0x7fa3ec10df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:43.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:43.715+0000 7fa3eb7fe700 1 -- 172.21.15.107:0/2895345536 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3d4005040 con 0x7fa3ec0fe3e0 2024-05-17T22:01:43.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:43.715+0000 7fa3eb7fe700 1 --2- 172.21.15.107:0/2895345536 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa3ec0fe3e0 0x7fa3ec10d9c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa3dc002a50 tx=0x7fa3dc0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:43.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:43.715+0000 7fa3e8ff9700 1 -- 172.21.15.107:0/2895345536 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa3dc007cc0 con 0x7fa3ec0fe3e0 2024-05-17T22:01:43.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:43.715+0000 7fa3f2046700 1 -- 172.21.15.107:0/2895345536 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3ec10f030 con 0x7fa3ec0fe3e0 2024-05-17T22:01:43.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:43.715+0000 7fa3e8ff9700 1 -- 172.21.15.107:0/2895345536 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa3dc00f040 con 0x7fa3ec0fe3e0 2024-05-17T22:01:43.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:43.715+0000 7fa3e8ff9700 1 -- 172.21.15.107:0/2895345536 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa3dc009640 con 0x7fa3ec0fe3e0 2024-05-17T22:01:43.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:43.715+0000 7fa3f2046700 1 -- 172.21.15.107:0/2895345536 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3ec10f570 con 0x7fa3ec0fe3e0 2024-05-17T22:01:43.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:43.716+0000 7fa3f2046700 1 -- 172.21.15.107:0/2895345536 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3ec05f5c0 con 0x7fa3ec0fe3e0 2024-05-17T22:01:43.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:43.717+0000 7fa3e8ff9700 1 -- 172.21.15.107:0/2895345536 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7fa3dc0097a0 con 0x7fa3ec0fe3e0 2024-05-17T22:01:43.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:43.718+0000 7fa3e8ff9700 1 --2- 172.21.15.107:0/2895345536 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fa3d8078990 0x7fa3d807ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:43.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:43.718+0000 7fa3e8ff9700 1 -- 172.21.15.107:0/2895345536 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7fa3dc09abe0 con 0x7fa3ec0fe3e0 2024-05-17T22:01:43.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:43.718+0000 7fa3e2ffd700 1 --2- 172.21.15.107:0/2895345536 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fa3d8078990 0x7fa3d807ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:43.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:43.718+0000 7fa3e2ffd700 1 --2- 172.21.15.107:0/2895345536 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fa3d8078990 0x7fa3d807ae40 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa3d4000c00 tx=0x7fa3d4009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:43.712 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:43.722+0000 7fa3e8ff9700 1 -- 172.21.15.107:0/2895345536 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7fa3dc062820 con 0x7fa3ec0fe3e0 2024-05-17T22:01:43.968 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:43.977+0000 7fa3f2046700 1 -- 172.21.15.107:0/2895345536 --> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa3ec10f950 con 0x7fa3d8078990 2024-05-17T22:01:43.969 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:43.978+0000 7fa3e8ff9700 1 -- 172.21.15.107:0/2895345536 <== mgr.34100 v2:172.21.15.107:6800/3205268226 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+435 (secure 0 0 0) 0x7fa3ec10f950 con 0x7fa3d8078990 2024-05-17T22:01:43.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:43.980+0000 7fa3f2046700 1 -- 172.21.15.107:0/2895345536 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fa3d8078990 msgr2=0x7fa3d807ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:43.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:43.980+0000 7fa3f2046700 1 --2- 172.21.15.107:0/2895345536 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fa3d8078990 0x7fa3d807ae40 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa3d4000c00 tx=0x7fa3d4009e30 comp rx=0 tx=0).stop 2024-05-17T22:01:43.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:43.980+0000 7fa3f2046700 1 -- 172.21.15.107:0/2895345536 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa3ec0fe3e0 msgr2=0x7fa3ec10d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:43.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:43.980+0000 7fa3f2046700 1 --2- 172.21.15.107:0/2895345536 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa3ec0fe3e0 0x7fa3ec10d9c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa3dc002a50 tx=0x7fa3dc0046b0 comp rx=0 tx=0).stop 2024-05-17T22:01:43.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:43.980+0000 7fa3f2046700 1 -- 172.21.15.107:0/2895345536 shutdown_connections 2024-05-17T22:01:43.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:43.980+0000 7fa3f2046700 1 --2- 172.21.15.107:0/2895345536 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fa3d8078990 0x7fa3d807ae40 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:43.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:43.980+0000 7fa3f2046700 1 --2- 172.21.15.107:0/2895345536 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa3ec0fe3e0 0x7fa3ec10d9c0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:43.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:43.980+0000 7fa3f2046700 1 --2- 172.21.15.107:0/2895345536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3ec0fed30 0x7fa3ec10df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:43.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:43.980+0000 7fa3f2046700 1 -- 172.21.15.107:0/2895345536 >> 172.21.15.107:0/2895345536 conn(0x7fa3ec0f6b80 msgr2=0x7fa3ec0f8240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:01:43.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:43.981+0000 7fa3f2046700 1 -- 172.21.15.107:0/2895345536 shutdown_connections 2024-05-17T22:01:43.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:43.981+0000 7fa3f2046700 1 -- 172.21.15.107:0/2895345536 wait complete. 2024-05-17T22:01:44.077 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.084+0000 7f0cd4608700 1 -- 172.21.15.107:0/2721904930 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0ccc0fe430 msgr2=0x7f0ccc0fe840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:44.077 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.084+0000 7f0cd4608700 1 --2- 172.21.15.107:0/2721904930 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0ccc0fe430 0x7f0ccc0fe840 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f0cc8004940 tx=0x7f0cc8009e30 comp rx=0 tx=0).stop 2024-05-17T22:01:44.077 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.084+0000 7f0cd4608700 1 -- 172.21.15.107:0/2721904930 shutdown_connections 2024-05-17T22:01:44.077 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.084+0000 7f0cd4608700 1 --2- 172.21.15.107:0/2721904930 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ccc0fed80 0x7f0ccc0faec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:44.077 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.084+0000 7f0cd4608700 1 --2- 172.21.15.107:0/2721904930 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0ccc0fe430 0x7f0ccc0fe840 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:44.077 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.084+0000 7f0cd4608700 1 -- 172.21.15.107:0/2721904930 >> 172.21.15.107:0/2721904930 conn(0x7f0ccc0f6bd0 msgr2=0x7f0ccc0f9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:01:44.077 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.084+0000 7f0cd4608700 1 -- 172.21.15.107:0/2721904930 shutdown_connections 2024-05-17T22:01:44.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.084+0000 7f0cd4608700 1 -- 172.21.15.107:0/2721904930 wait complete. 2024-05-17T22:01:44.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.085+0000 7f0cd4608700 1 Processor -- start 2024-05-17T22:01:44.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.085+0000 7f0cd4608700 1 -- start start 2024-05-17T22:01:44.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.085+0000 7f0cd4608700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0ccc0fe430 0x7f0ccc10d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:44.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.085+0000 7f0cd4608700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ccc0fed80 0x7f0ccc10de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:44.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.085+0000 7f0cd4608700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ccc10e460 con 0x7f0ccc0fe430 2024-05-17T22:01:44.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.085+0000 7f0cd4608700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ccc10e5d0 con 0x7f0ccc0fed80 2024-05-17T22:01:44.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.085+0000 7f0cd23a4700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0ccc0fe430 0x7f0ccc10d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:44.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.085+0000 7f0cd23a4700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0ccc0fe430 0x7f0ccc10d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46660/0 (socket says 172.21.15.107:46660) 2024-05-17T22:01:44.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.085+0000 7f0cd23a4700 1 -- 172.21.15.107:0/1138999656 learned_addr learned my addr 172.21.15.107:0/1138999656 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T22:01:44.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.085+0000 7f0cd1ba3700 1 --2- 172.21.15.107:0/1138999656 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ccc0fed80 0x7f0ccc10de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:44.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.085+0000 7f0cd23a4700 1 -- 172.21.15.107:0/1138999656 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ccc0fed80 msgr2=0x7f0ccc10de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:44.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.085+0000 7f0cd23a4700 1 --2- 172.21.15.107:0/1138999656 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ccc0fed80 0x7f0ccc10de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:44.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.085+0000 7f0cd23a4700 1 -- 172.21.15.107:0/1138999656 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cc8005040 con 0x7f0ccc0fe430 2024-05-17T22:01:44.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.086+0000 7f0cd23a4700 1 --2- 172.21.15.107:0/1138999656 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0ccc0fe430 0x7f0ccc10d920 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0cc800b3b0 tx=0x7f0cc8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:44.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.086+0000 7f0cc2ffd700 1 -- 172.21.15.107:0/1138999656 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0cc800e040 con 0x7f0ccc0fe430 2024-05-17T22:01:44.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.086+0000 7f0cc2ffd700 1 -- 172.21.15.107:0/1138999656 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0cc800bcc0 con 0x7f0ccc0fe430 2024-05-17T22:01:44.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.086+0000 7f0cd4608700 1 -- 172.21.15.107:0/1138999656 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ccc112fd0 con 0x7f0ccc0fe430 2024-05-17T22:01:44.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.086+0000 7f0cc2ffd700 1 -- 172.21.15.107:0/1138999656 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0cc801c630 con 0x7f0ccc0fe430 2024-05-17T22:01:44.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.086+0000 7f0cd4608700 1 -- 172.21.15.107:0/1138999656 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ccc113500 con 0x7f0ccc0fe430 2024-05-17T22:01:44.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.087+0000 7f0cd4608700 1 -- 172.21.15.107:0/1138999656 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ccc04c710 con 0x7f0ccc0fe430 2024-05-17T22:01:44.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.088+0000 7f0cc2ffd700 1 -- 172.21.15.107:0/1138999656 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f0cc8013980 con 0x7f0ccc0fe430 2024-05-17T22:01:44.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.088+0000 7f0cc2ffd700 1 --2- 172.21.15.107:0/1138999656 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f0cb8078990 0x7f0cb807ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:44.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.088+0000 7f0cc2ffd700 1 -- 172.21.15.107:0/1138999656 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f0cc809bc00 con 0x7f0ccc0fe430 2024-05-17T22:01:44.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.088+0000 7f0cd1ba3700 1 --2- 172.21.15.107:0/1138999656 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f0cb8078990 0x7f0cb807ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:44.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.088+0000 7f0cd1ba3700 1 --2- 172.21.15.107:0/1138999656 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f0cb8078990 0x7f0cb807ae40 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc004d60 tx=0x7f0cbc00b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:44.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.092+0000 7f0cc2ffd700 1 -- 172.21.15.107:0/1138999656 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f0cc8063840 con 0x7f0ccc0fe430 2024-05-17T22:01:44.164 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:43 smithi107 ceph-mon[93224]: pgmap v252: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:01:44.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.313+0000 7f0cd4608700 1 -- 172.21.15.107:0/1138999656 --> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f0ccc102640 con 0x7f0cb8078990 2024-05-17T22:01:44.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.323+0000 7f0cc2ffd700 1 -- 172.21.15.107:0/1138999656 <== mgr.34100 v2:172.21.15.107:6800/3205268226 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f0ccc102640 con 0x7f0cb8078990 2024-05-17T22:01:44.315 INFO:teuthology.orchestra.run.smithi107.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-17T22:01:44.315 INFO:teuthology.orchestra.run.smithi107.stdout:alertmanager.smithi107 smithi107 *:9093,9094 running (22s) 17s ago 17m 17.2M - 0.25.0 c8568f914cd2 8b363231ac61 2024-05-17T22:01:44.315 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi107 smithi107 running (6m) 17s ago 18m 6669k - 19.0.0-3794-g931809a9 8880f7613d02 face35de205b 2024-05-17T22:01:44.315 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi138 smithi138 running (5m) 52s ago 17m 6665k - 19.0.0-3794-g931809a9 8880f7613d02 ad59e33e6d43 2024-05-17T22:01:44.315 INFO:teuthology.orchestra.run.smithi107.stdout:grafana.smithi107 smithi107 *:3000 running (9m) 17s ago 17m 85.9M - 9.4.7 954c08fa6188 8c747e777bc5 2024-05-17T22:01:44.315 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.drqody smithi107 running (2m) 17s ago 12m 155M - 19.0.0-3794-g931809a9 8880f7613d02 4f02280f8312 2024-05-17T22:01:44.315 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.idmbgj smithi107 running (2m) 17s ago 12m 153M - 19.0.0-3794-g931809a9 8880f7613d02 d541bff38965 2024-05-17T22:01:44.315 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi138.gwbtyd smithi138 running (99s) 52s ago 12m 80.2M - 19.0.0-3794-g931809a9 8880f7613d02 7beb9bd77fc8 2024-05-17T22:01:44.315 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi138.zckrvs smithi138 running (79s) 52s ago 12m 79.2M - 19.0.0-3794-g931809a9 8880f7613d02 17edb3a75f7b 2024-05-17T22:01:44.316 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi107.xhhyiw smithi107 *:8443,9283,8765 running (8m) 17s ago 19m 596M - 19.0.0-3794-g931809a9 8880f7613d02 5acf3e511342 2024-05-17T22:01:44.316 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi138.napnll smithi138 *:8443,9283,8765 running (7m) 52s ago 16m 534M - 19.0.0-3794-g931809a9 8880f7613d02 d627761dbd0d 2024-05-17T22:01:44.316 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi107 smithi107 running (7m) 17s ago 19m 123M 2048M 19.0.0-3794-g931809a9 8880f7613d02 9717f3169f2f 2024-05-17T22:01:44.316 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi138 smithi138 running (6m) 52s ago 16m 107M 2048M 19.0.0-3794-g931809a9 8880f7613d02 55aeed62bf4d 2024-05-17T22:01:44.316 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi107 smithi107 *:9100 running (60s) 17s ago 18m 8476k - 1.5.0 0da6a335fe13 73f1896972ff 2024-05-17T22:01:44.316 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi138 smithi138 *:9100 running (55s) 52s ago 16m 3565k - 1.5.0 0da6a335fe13 05bb3093f9aa 2024-05-17T22:01:44.316 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0 smithi107 running (5m) 17s ago 16m 165M 1557M 19.0.0-3794-g931809a9 8880f7613d02 b319bc93500d 2024-05-17T22:01:44.316 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1 smithi107 running (5m) 17s ago 15m 166M 1557M 19.0.0-3794-g931809a9 8880f7613d02 f94b206be6eb 2024-05-17T22:01:44.316 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2 smithi107 running (4m) 17s ago 15m 154M 1557M 19.0.0-3794-g931809a9 8880f7613d02 add7015647af 2024-05-17T22:01:44.316 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3 smithi138 running (3m) 52s ago 14m 174M 2581M 19.0.0-3794-g931809a9 8880f7613d02 416273d327c7 2024-05-17T22:01:44.316 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4 smithi138 running (3m) 52s ago 14m 159M 2581M 19.0.0-3794-g931809a9 8880f7613d02 e33f411da8e6 2024-05-17T22:01:44.316 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5 smithi138 running (2m) 52s ago 13m 159M 2581M 19.0.0-3794-g931809a9 8880f7613d02 63810d431961 2024-05-17T22:01:44.316 INFO:teuthology.orchestra.run.smithi107.stdout:prometheus.smithi107 smithi107 *:9095 running (39s) 17s ago 17m 48.8M - 2.43.0 a07b618ecd1d aee4bd577d3d 2024-05-17T22:01:44.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.326+0000 7f0cd4608700 1 -- 172.21.15.107:0/1138999656 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f0cb8078990 msgr2=0x7f0cb807ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:44.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.326+0000 7f0cd4608700 1 --2- 172.21.15.107:0/1138999656 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f0cb8078990 0x7f0cb807ae40 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc004d60 tx=0x7f0cbc00b040 comp rx=0 tx=0).stop 2024-05-17T22:01:44.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.326+0000 7f0cd4608700 1 -- 172.21.15.107:0/1138999656 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0ccc0fe430 msgr2=0x7f0ccc10d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:44.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.326+0000 7f0cd4608700 1 --2- 172.21.15.107:0/1138999656 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0ccc0fe430 0x7f0ccc10d920 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0cc800b3b0 tx=0x7f0cc8009e30 comp rx=0 tx=0).stop 2024-05-17T22:01:44.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.326+0000 7f0cd4608700 1 -- 172.21.15.107:0/1138999656 shutdown_connections 2024-05-17T22:01:44.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.326+0000 7f0cd4608700 1 --2- 172.21.15.107:0/1138999656 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f0cb8078990 0x7f0cb807ae40 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:44.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.326+0000 7f0cd4608700 1 --2- 172.21.15.107:0/1138999656 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0ccc0fe430 0x7f0ccc10d920 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:44.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.326+0000 7f0cd4608700 1 --2- 172.21.15.107:0/1138999656 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ccc0fed80 0x7f0ccc10de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:44.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.326+0000 7f0cd4608700 1 -- 172.21.15.107:0/1138999656 >> 172.21.15.107:0/1138999656 conn(0x7f0ccc0f6bd0 msgr2=0x7f0ccc0f82d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:01:44.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.326+0000 7f0cd4608700 1 -- 172.21.15.107:0/1138999656 shutdown_connections 2024-05-17T22:01:44.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.326+0000 7f0cd4608700 1 -- 172.21.15.107:0/1138999656 wait complete. 2024-05-17T22:01:44.324 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:43 smithi138 ceph-mon[81376]: pgmap v252: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:01:44.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.434+0000 7f6cbb741700 1 -- 172.21.15.107:0/691244142 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6cb40fdfd0 msgr2=0x7f6cb40fe440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:44.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.434+0000 7f6cbb741700 1 --2- 172.21.15.107:0/691244142 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6cb40fdfd0 0x7f6cb40fe440 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6ca4004940 tx=0x7f6ca4009e30 comp rx=0 tx=0).stop 2024-05-17T22:01:44.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.434+0000 7f6cbb741700 1 -- 172.21.15.107:0/691244142 shutdown_connections 2024-05-17T22:01:44.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.434+0000 7f6cbb741700 1 --2- 172.21.15.107:0/691244142 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6cb40fdfd0 0x7f6cb40fe440 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:44.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.434+0000 7f6cbb741700 1 --2- 172.21.15.107:0/691244142 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6cb40fce60 0x7f6cb40fd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:44.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.434+0000 7f6cbb741700 1 -- 172.21.15.107:0/691244142 >> 172.21.15.107:0/691244142 conn(0x7f6cb40f8540 msgr2=0x7f6cb40fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:01:44.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.434+0000 7f6cbb741700 1 -- 172.21.15.107:0/691244142 shutdown_connections 2024-05-17T22:01:44.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.434+0000 7f6cbb741700 1 -- 172.21.15.107:0/691244142 wait complete. 2024-05-17T22:01:44.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.434+0000 7f6cbb741700 1 Processor -- start 2024-05-17T22:01:44.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.435+0000 7f6cbb741700 1 -- start start 2024-05-17T22:01:44.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.435+0000 7f6cbb741700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6cb40fce60 0x7f6cb410b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:44.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.435+0000 7f6cbb741700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6cb40fdfd0 0x7f6cb410bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:44.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.435+0000 7f6cbb741700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb410c3d0 con 0x7f6cb40fce60 2024-05-17T22:01:44.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.435+0000 7f6cbb741700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb410cb90 con 0x7f6cb40fdfd0 2024-05-17T22:01:44.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.435+0000 7f6cb94dd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6cb40fce60 0x7f6cb410b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:44.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.435+0000 7f6cb94dd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6cb40fce60 0x7f6cb410b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46676/0 (socket says 172.21.15.107:46676) 2024-05-17T22:01:44.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.435+0000 7f6cb94dd700 1 -- 172.21.15.107:0/686509196 learned_addr learned my addr 172.21.15.107:0/686509196 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T22:01:44.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.435+0000 7f6cb8cdc700 1 --2- 172.21.15.107:0/686509196 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6cb40fdfd0 0x7f6cb410bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:44.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.435+0000 7f6cb94dd700 1 -- 172.21.15.107:0/686509196 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6cb40fdfd0 msgr2=0x7f6cb410bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:44.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.435+0000 7f6cb94dd700 1 --2- 172.21.15.107:0/686509196 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6cb40fdfd0 0x7f6cb410bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:44.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.435+0000 7f6cb94dd700 1 -- 172.21.15.107:0/686509196 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ca4005040 con 0x7f6cb40fce60 2024-05-17T22:01:44.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.436+0000 7f6cb94dd700 1 --2- 172.21.15.107:0/686509196 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6cb40fce60 0x7f6cb410b7b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6cb000b790 tx=0x7f6cb0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:44.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.436+0000 7f6ca9ffb700 1 -- 172.21.15.107:0/686509196 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6cb000f770 con 0x7f6cb40fce60 2024-05-17T22:01:44.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.436+0000 7f6ca9ffb700 1 -- 172.21.15.107:0/686509196 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6cb000fda0 con 0x7f6cb40fce60 2024-05-17T22:01:44.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.436+0000 7f6ca9ffb700 1 -- 172.21.15.107:0/686509196 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6cb000d310 con 0x7f6cb40fce60 2024-05-17T22:01:44.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.436+0000 7f6cbb741700 1 -- 172.21.15.107:0/686509196 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cb410ce70 con 0x7f6cb40fce60 2024-05-17T22:01:44.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.436+0000 7f6cbb741700 1 -- 172.21.15.107:0/686509196 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cb410d3d0 con 0x7f6cb40fce60 2024-05-17T22:01:44.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.437+0000 7f6cbb741700 1 -- 172.21.15.107:0/686509196 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cb4101960 con 0x7f6cb40fce60 2024-05-17T22:01:44.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.438+0000 7f6ca9ffb700 1 -- 172.21.15.107:0/686509196 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f6cb0010420 con 0x7f6cb40fce60 2024-05-17T22:01:44.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.438+0000 7f6ca9ffb700 1 --2- 172.21.15.107:0/686509196 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f6ca0078990 0x7f6ca007ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:44.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.439+0000 7f6ca9ffb700 1 -- 172.21.15.107:0/686509196 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f6cb009a750 con 0x7f6cb40fce60 2024-05-17T22:01:44.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.439+0000 7f6cb8cdc700 1 --2- 172.21.15.107:0/686509196 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f6ca0078990 0x7f6ca007ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:44.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.439+0000 7f6cb8cdc700 1 --2- 172.21.15.107:0/686509196 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f6ca0078990 0x7f6ca007ae40 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6ca400b3b0 tx=0x7f6ca4009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:44.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.444+0000 7f6ca9ffb700 1 -- 172.21.15.107:0/686509196 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f6cb0062430 con 0x7f6cb40fce60 2024-05-17T22:01:44.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.792+0000 7f6cbb741700 1 -- 172.21.15.107:0/686509196 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f6cb405f5c0 con 0x7f6cb40fce60 2024-05-17T22:01:44.784 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.793+0000 7f6ca9ffb700 1 -- 172.21.15.107:0/686509196 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f6cb0061dd0 con 0x7f6cb40fce60 2024-05-17T22:01:44.784 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-05-17T22:01:44.784 INFO:teuthology.orchestra.run.smithi107.stdout: "mon": { 2024-05-17T22:01:44.784 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev)": 2 2024-05-17T22:01:44.784 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-05-17T22:01:44.784 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr": { 2024-05-17T22:01:44.784 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev)": 2 2024-05-17T22:01:44.785 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-05-17T22:01:44.785 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": { 2024-05-17T22:01:44.785 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev)": 6 2024-05-17T22:01:44.785 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-05-17T22:01:44.785 INFO:teuthology.orchestra.run.smithi107.stdout: "mds": { 2024-05-17T22:01:44.785 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev)": 4 2024-05-17T22:01:44.785 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-05-17T22:01:44.785 INFO:teuthology.orchestra.run.smithi107.stdout: "overall": { 2024-05-17T22:01:44.785 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev)": 14 2024-05-17T22:01:44.785 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-05-17T22:01:44.785 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-05-17T22:01:44.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.795+0000 7f6cbb741700 1 -- 172.21.15.107:0/686509196 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f6ca0078990 msgr2=0x7f6ca007ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:44.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.795+0000 7f6cbb741700 1 --2- 172.21.15.107:0/686509196 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f6ca0078990 0x7f6ca007ae40 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6ca400b3b0 tx=0x7f6ca4009e30 comp rx=0 tx=0).stop 2024-05-17T22:01:44.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.795+0000 7f6cbb741700 1 -- 172.21.15.107:0/686509196 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6cb40fce60 msgr2=0x7f6cb410b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:44.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.795+0000 7f6cbb741700 1 --2- 172.21.15.107:0/686509196 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6cb40fce60 0x7f6cb410b7b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6cb000b790 tx=0x7f6cb0009250 comp rx=0 tx=0).stop 2024-05-17T22:01:44.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.795+0000 7f6cbb741700 1 -- 172.21.15.107:0/686509196 shutdown_connections 2024-05-17T22:01:44.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.795+0000 7f6cbb741700 1 --2- 172.21.15.107:0/686509196 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f6ca0078990 0x7f6ca007ae40 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:44.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.795+0000 7f6cbb741700 1 --2- 172.21.15.107:0/686509196 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6cb40fce60 0x7f6cb410b7b0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:44.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.795+0000 7f6cbb741700 1 --2- 172.21.15.107:0/686509196 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6cb40fdfd0 0x7f6cb410bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:44.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.795+0000 7f6cbb741700 1 -- 172.21.15.107:0/686509196 >> 172.21.15.107:0/686509196 conn(0x7f6cb40f8540 msgr2=0x7f6cb40f9ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:01:44.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.795+0000 7f6cbb741700 1 -- 172.21.15.107:0/686509196 shutdown_connections 2024-05-17T22:01:44.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.795+0000 7f6cbb741700 1 -- 172.21.15.107:0/686509196 wait complete. 2024-05-17T22:01:44.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.907+0000 7fd4b4684700 1 -- 172.21.15.107:0/1983340193 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd4ac0fcf60 msgr2=0x7fd4ac0fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:44.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.907+0000 7fd4b4684700 1 --2- 172.21.15.107:0/1983340193 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd4ac0fcf60 0x7fd4ac0fd3d0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd49c004940 tx=0x7fd49c009e30 comp rx=0 tx=0).stop 2024-05-17T22:01:44.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.907+0000 7fd4b4684700 1 -- 172.21.15.107:0/1983340193 shutdown_connections 2024-05-17T22:01:44.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.907+0000 7fd4b4684700 1 --2- 172.21.15.107:0/1983340193 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd4ac0fcf60 0x7fd4ac0fd3d0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:44.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.907+0000 7fd4b4684700 1 --2- 172.21.15.107:0/1983340193 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4ac0fbcb0 0x7fd4ac0fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:44.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.907+0000 7fd4b4684700 1 -- 172.21.15.107:0/1983340193 >> 172.21.15.107:0/1983340193 conn(0x7fd4ac0f7430 msgr2=0x7fd4ac0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:01:44.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.907+0000 7fd4b4684700 1 -- 172.21.15.107:0/1983340193 shutdown_connections 2024-05-17T22:01:44.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.907+0000 7fd4b4684700 1 -- 172.21.15.107:0/1983340193 wait complete. 2024-05-17T22:01:44.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.907+0000 7fd4b4684700 1 Processor -- start 2024-05-17T22:01:44.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.907+0000 7fd4b4684700 1 -- start start 2024-05-17T22:01:44.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.908+0000 7fd4b4684700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4ac0fbcb0 0x7fd4ac1106a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:44.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.908+0000 7fd4b4684700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd4ac0fcf60 0x7fd4ac112bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:44.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.908+0000 7fd4b4684700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ac0662e0 con 0x7fd4ac0fcf60 2024-05-17T22:01:44.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.908+0000 7fd4b4684700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ac066420 con 0x7fd4ac0fbcb0 2024-05-17T22:01:44.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.908+0000 7fd4b1c1f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd4ac0fcf60 0x7fd4ac112bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:44.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.908+0000 7fd4b1c1f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd4ac0fcf60 0x7fd4ac112bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46688/0 (socket says 172.21.15.107:46688) 2024-05-17T22:01:44.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.908+0000 7fd4b1c1f700 1 -- 172.21.15.107:0/616990132 learned_addr learned my addr 172.21.15.107:0/616990132 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T22:01:44.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.908+0000 7fd4b2420700 1 --2- 172.21.15.107:0/616990132 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4ac0fbcb0 0x7fd4ac1106a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:44.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.908+0000 7fd4b1c1f700 1 -- 172.21.15.107:0/616990132 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4ac0fbcb0 msgr2=0x7fd4ac1106a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:44.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.908+0000 7fd4b1c1f700 1 --2- 172.21.15.107:0/616990132 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4ac0fbcb0 0x7fd4ac1106a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:44.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.908+0000 7fd4b1c1f700 1 -- 172.21.15.107:0/616990132 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd49c005040 con 0x7fd4ac0fcf60 2024-05-17T22:01:44.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.908+0000 7fd4b1c1f700 1 --2- 172.21.15.107:0/616990132 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd4ac0fcf60 0x7fd4ac112bf0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd49c009150 tx=0x7fd49c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:44.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.908+0000 7fd4a2ffd700 1 -- 172.21.15.107:0/616990132 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd49c00e040 con 0x7fd4ac0fcf60 2024-05-17T22:01:44.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.908+0000 7fd4b4684700 1 -- 172.21.15.107:0/616990132 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4ac113130 con 0x7fd4ac0fcf60 2024-05-17T22:01:44.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.909+0000 7fd4a2ffd700 1 -- 172.21.15.107:0/616990132 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd49c00bcc0 con 0x7fd4ac0fcf60 2024-05-17T22:01:44.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.909+0000 7fd4b4684700 1 -- 172.21.15.107:0/616990132 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4ac1136a0 con 0x7fd4ac0fcf60 2024-05-17T22:01:44.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.909+0000 7fd4a2ffd700 1 -- 172.21.15.107:0/616990132 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd49c01c630 con 0x7fd4ac0fcf60 2024-05-17T22:01:44.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.909+0000 7fd4b4684700 1 -- 172.21.15.107:0/616990132 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4900052e0 con 0x7fd4ac0fcf60 2024-05-17T22:01:44.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.911+0000 7fd4a2ffd700 1 -- 172.21.15.107:0/616990132 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7fd49c013420 con 0x7fd4ac0fcf60 2024-05-17T22:01:44.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.912+0000 7fd4a2ffd700 1 --2- 172.21.15.107:0/616990132 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fd498078990 0x7fd49807ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:44.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.912+0000 7fd4a2ffd700 1 -- 172.21.15.107:0/616990132 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7fd49c09ba60 con 0x7fd4ac0fcf60 2024-05-17T22:01:44.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.912+0000 7fd4b2420700 1 --2- 172.21.15.107:0/616990132 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fd498078990 0x7fd49807ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:44.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.913+0000 7fd4b2420700 1 --2- 172.21.15.107:0/616990132 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fd498078990 0x7fd49807ae40 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd4a8009fd0 tx=0x7fd4a8009f60 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:44.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:44.916+0000 7fd4a2ffd700 1 -- 172.21.15.107:0/616990132 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7fd49c063740 con 0x7fd4ac0fcf60 2024-05-17T22:01:45.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:44 smithi107 ceph-mon[93224]: from='client.34534 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:01:45.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:44 smithi107 ceph-mon[93224]: from='client.34538 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:01:45.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:44 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:01:45.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:44 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:01:45.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:44 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:01:45.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:44 smithi107 ceph-mon[93224]: from='client.? 172.21.15.107:0/686509196' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:01:45.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:45.197+0000 7fd4b4684700 1 -- 172.21.15.107:0/616990132 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fd490005860 con 0x7fd4ac0fcf60 2024-05-17T22:01:45.193 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:45.202+0000 7fd4a2ffd700 1 -- 172.21.15.107:0/616990132 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 35 v35) v1 ==== 76+0+1968 (secure 0 0 0) 0x7fd49c0630e0 con 0x7fd4ac0fcf60 2024-05-17T22:01:45.194 INFO:teuthology.orchestra.run.smithi107.stdout:e35 2024-05-17T22:01:45.194 INFO:teuthology.orchestra.run.smithi107.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-17T22:01:45.194 INFO:teuthology.orchestra.run.smithi107.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-17T22:01:45.194 INFO:teuthology.orchestra.run.smithi107.stdout:legacy client fscid: 1 2024-05-17T22:01:45.194 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T22:01:45.194 INFO:teuthology.orchestra.run.smithi107.stdout:Filesystem 'cephfs' (1) 2024-05-17T22:01:45.194 INFO:teuthology.orchestra.run.smithi107.stdout:fs_name cephfs 2024-05-17T22:01:45.194 INFO:teuthology.orchestra.run.smithi107.stdout:epoch 35 2024-05-17T22:01:45.194 INFO:teuthology.orchestra.run.smithi107.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-17T22:01:45.194 INFO:teuthology.orchestra.run.smithi107.stdout:created 2024-05-17T21:49:16.836333+0000 2024-05-17T22:01:45.195 INFO:teuthology.orchestra.run.smithi107.stdout:modified 2024-05-17T22:00:33.850863+0000 2024-05-17T22:01:45.195 INFO:teuthology.orchestra.run.smithi107.stdout:tableserver 0 2024-05-17T22:01:45.195 INFO:teuthology.orchestra.run.smithi107.stdout:root 0 2024-05-17T22:01:45.195 INFO:teuthology.orchestra.run.smithi107.stdout:session_timeout 60 2024-05-17T22:01:45.195 INFO:teuthology.orchestra.run.smithi107.stdout:session_autoclose 300 2024-05-17T22:01:45.195 INFO:teuthology.orchestra.run.smithi107.stdout:max_file_size 1099511627776 2024-05-17T22:01:45.195 INFO:teuthology.orchestra.run.smithi107.stdout:max_xattr_size 65536 2024-05-17T22:01:45.195 INFO:teuthology.orchestra.run.smithi107.stdout:required_client_features {} 2024-05-17T22:01:45.195 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure 0 2024-05-17T22:01:45.195 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure_osd_epoch 84 2024-05-17T22:01:45.195 INFO:teuthology.orchestra.run.smithi107.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-17T22:01:45.195 INFO:teuthology.orchestra.run.smithi107.stdout:max_mds 1 2024-05-17T22:01:45.195 INFO:teuthology.orchestra.run.smithi107.stdout:in 0 2024-05-17T22:01:45.195 INFO:teuthology.orchestra.run.smithi107.stdout:up {0=34408} 2024-05-17T22:01:45.195 INFO:teuthology.orchestra.run.smithi107.stdout:failed 2024-05-17T22:01:45.196 INFO:teuthology.orchestra.run.smithi107.stdout:damaged 2024-05-17T22:01:45.196 INFO:teuthology.orchestra.run.smithi107.stdout:stopped 2024-05-17T22:01:45.196 INFO:teuthology.orchestra.run.smithi107.stdout:data_pools [3] 2024-05-17T22:01:45.196 INFO:teuthology.orchestra.run.smithi107.stdout:metadata_pool 2 2024-05-17T22:01:45.196 INFO:teuthology.orchestra.run.smithi107.stdout:inline_data enabled 2024-05-17T22:01:45.196 INFO:teuthology.orchestra.run.smithi107.stdout:balancer 2024-05-17T22:01:45.196 INFO:teuthology.orchestra.run.smithi107.stdout:bal_rank_mask -1 2024-05-17T22:01:45.196 INFO:teuthology.orchestra.run.smithi107.stdout:standby_count_wanted 1 2024-05-17T22:01:45.196 INFO:teuthology.orchestra.run.smithi107.stdout:qdb_cluster leader: 34408 members: 34408 2024-05-17T22:01:45.196 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.idmbgj{0:34408} state up:active seq 20 join_fscid=1 addr [v2:172.21.15.107:6826/2235359521,v1:172.21.15.107:6827/2235359521] compat {c=[1],r=[1],i=[1fff]}] 2024-05-17T22:01:45.196 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.drqody{0:34440} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.107:6828/369507752,v1:172.21.15.107:6829/369507752] compat {c=[1],r=[1],i=[1fff]}] 2024-05-17T22:01:45.196 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T22:01:45.196 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T22:01:45.196 INFO:teuthology.orchestra.run.smithi107.stdout:Standby daemons: 2024-05-17T22:01:45.196 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T22:01:45.197 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi138.gwbtyd{-1:44335} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6824/1700112281,v1:172.21.15.138:6825/1700112281] compat {c=[1],r=[1],i=[1fff]}] 2024-05-17T22:01:45.197 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi138.zckrvs{-1:44365} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6826/1801131392,v1:172.21.15.138:6827/1801131392] compat {c=[1],r=[1],i=[1fff]}] 2024-05-17T22:01:45.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:45.205+0000 7fd4b4684700 1 -- 172.21.15.107:0/616990132 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fd498078990 msgr2=0x7fd49807ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:45.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:45.205+0000 7fd4b4684700 1 --2- 172.21.15.107:0/616990132 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fd498078990 0x7fd49807ae40 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd4a8009fd0 tx=0x7fd4a8009f60 comp rx=0 tx=0).stop 2024-05-17T22:01:45.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:45.205+0000 7fd4b4684700 1 -- 172.21.15.107:0/616990132 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd4ac0fcf60 msgr2=0x7fd4ac112bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:45.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:45.205+0000 7fd4b4684700 1 --2- 172.21.15.107:0/616990132 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd4ac0fcf60 0x7fd4ac112bf0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd49c009150 tx=0x7fd49c009e30 comp rx=0 tx=0).stop 2024-05-17T22:01:45.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:45.205+0000 7fd4b4684700 1 -- 172.21.15.107:0/616990132 shutdown_connections 2024-05-17T22:01:45.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:45.205+0000 7fd4b4684700 1 --2- 172.21.15.107:0/616990132 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fd498078990 0x7fd49807ae40 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:45.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:45.205+0000 7fd4b4684700 1 --2- 172.21.15.107:0/616990132 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4ac0fbcb0 0x7fd4ac1106a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:45.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:45.205+0000 7fd4b4684700 1 --2- 172.21.15.107:0/616990132 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd4ac0fcf60 0x7fd4ac112bf0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:45.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:45.205+0000 7fd4b4684700 1 -- 172.21.15.107:0/616990132 >> 172.21.15.107:0/616990132 conn(0x7fd4ac0f7430 msgr2=0x7fd4ac0f8a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:01:45.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:45.205+0000 7fd4b4684700 1 -- 172.21.15.107:0/616990132 shutdown_connections 2024-05-17T22:01:45.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:45.205+0000 7fd4b4684700 1 -- 172.21.15.107:0/616990132 wait complete. 2024-05-17T22:01:45.198 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 35 2024-05-17T22:01:45.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:45.320+0000 7fdfbd660700 1 -- 172.21.15.107:0/367672062 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdfb80fdff0 msgr2=0x7fdfb80fe460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:45.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:45.320+0000 7fdfbd660700 1 --2- 172.21.15.107:0/367672062 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdfb80fdff0 0x7fdfb80fe460 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fdfac004940 tx=0x7fdfac009e30 comp rx=0 tx=0).stop 2024-05-17T22:01:45.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:45.320+0000 7fdfbd660700 1 -- 172.21.15.107:0/367672062 shutdown_connections 2024-05-17T22:01:45.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:45.320+0000 7fdfbd660700 1 --2- 172.21.15.107:0/367672062 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdfb80fdff0 0x7fdfb80fe460 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:45.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:45.320+0000 7fdfbd660700 1 --2- 172.21.15.107:0/367672062 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfb80fce80 0x7fdfb80fd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:45.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:45.320+0000 7fdfbd660700 1 -- 172.21.15.107:0/367672062 >> 172.21.15.107:0/367672062 conn(0x7fdfb80f8520 msgr2=0x7fdfb80fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:01:45.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:45.320+0000 7fdfbd660700 1 -- 172.21.15.107:0/367672062 shutdown_connections 2024-05-17T22:01:45.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:45.320+0000 7fdfbd660700 1 -- 172.21.15.107:0/367672062 wait complete. 2024-05-17T22:01:45.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:45.321+0000 7fdfbd660700 1 Processor -- start 2024-05-17T22:01:45.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:45.321+0000 7fdfbd660700 1 -- start start 2024-05-17T22:01:45.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:45.321+0000 7fdfbd660700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfb80fce80 0x7fdfb810da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:45.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:45.321+0000 7fdfbd660700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdfb80fdff0 0x7fdfb810df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:45.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:45.321+0000 7fdfbd660700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb810e540 con 0x7fdfb80fdff0 2024-05-17T22:01:45.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:45.321+0000 7fdfbd660700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb810ed90 con 0x7fdfb80fce80 2024-05-17T22:01:45.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:45.321+0000 7fdfb67fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdfb80fdff0 0x7fdfb810df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:45.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:45.321+0000 7fdfb67fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdfb80fdff0 0x7fdfb810df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46700/0 (socket says 172.21.15.107:46700) 2024-05-17T22:01:45.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:45.321+0000 7fdfb67fc700 1 -- 172.21.15.107:0/196700106 learned_addr learned my addr 172.21.15.107:0/196700106 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T22:01:45.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:45.321+0000 7fdfb6ffd700 1 --2- 172.21.15.107:0/196700106 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfb80fce80 0x7fdfb810da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:45.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:45.321+0000 7fdfb67fc700 1 -- 172.21.15.107:0/196700106 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfb80fce80 msgr2=0x7fdfb810da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:45.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:45.321+0000 7fdfb67fc700 1 --2- 172.21.15.107:0/196700106 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfb80fce80 0x7fdfb810da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:45.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:45.322+0000 7fdfb67fc700 1 -- 172.21.15.107:0/196700106 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfac005040 con 0x7fdfb80fdff0 2024-05-17T22:01:45.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:45.322+0000 7fdfb67fc700 1 --2- 172.21.15.107:0/196700106 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdfb80fdff0 0x7fdfb810df40 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fdfac009150 tx=0x7fdfac009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:45.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:45.322+0000 7fdf9f7fe700 1 -- 172.21.15.107:0/196700106 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdfac00e040 con 0x7fdfb80fdff0 2024-05-17T22:01:45.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:45.322+0000 7fdfbd660700 1 -- 172.21.15.107:0/196700106 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfb810f010 con 0x7fdfb80fdff0 2024-05-17T22:01:45.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:45.322+0000 7fdf9f7fe700 1 -- 172.21.15.107:0/196700106 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdfac00bbf0 con 0x7fdfb80fdff0 2024-05-17T22:01:45.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:45.322+0000 7fdf9f7fe700 1 -- 172.21.15.107:0/196700106 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdfac01c630 con 0x7fdfb80fdff0 2024-05-17T22:01:45.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:45.322+0000 7fdfbd660700 1 -- 172.21.15.107:0/196700106 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfb810f550 con 0x7fdfb80fdff0 2024-05-17T22:01:45.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:45.323+0000 7fdfbd660700 1 -- 172.21.15.107:0/196700106 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfb805f5c0 con 0x7fdfb80fdff0 2024-05-17T22:01:45.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:45.324+0000 7fdf9f7fe700 1 -- 172.21.15.107:0/196700106 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7fdfac00bd60 con 0x7fdfb80fdff0 2024-05-17T22:01:45.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:45.324+0000 7fdf9f7fe700 1 --2- 172.21.15.107:0/196700106 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fdfa0078980 0x7fdfa007ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:45.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:45.324+0000 7fdf9f7fe700 1 -- 172.21.15.107:0/196700106 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7fdfac09bb10 con 0x7fdfb80fdff0 2024-05-17T22:01:45.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:45.324+0000 7fdfb6ffd700 1 --2- 172.21.15.107:0/196700106 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fdfa0078980 0x7fdfa007ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:45.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:45.325+0000 7fdfb6ffd700 1 --2- 172.21.15.107:0/196700106 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fdfa0078980 0x7fdfa007ae30 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fdfa8004f40 tx=0x7fdfa800b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:45.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:45.329+0000 7fdf9f7fe700 1 -- 172.21.15.107:0/196700106 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7fdfac063750 con 0x7fdfb80fdff0 2024-05-17T22:01:45.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:44 smithi138 ceph-mon[81376]: from='client.34534 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:01:45.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:44 smithi138 ceph-mon[81376]: from='client.34538 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:01:45.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:44 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:01:45.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:44 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:01:45.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:44 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:01:45.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:44 smithi138 ceph-mon[81376]: from='client.? 172.21.15.107:0/686509196' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:01:45.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:45.568+0000 7fdfbd660700 1 -- 172.21.15.107:0/196700106 --> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdfb810f8d0 con 0x7fdfa0078980 2024-05-17T22:01:45.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:45.569+0000 7fdf9f7fe700 1 -- 172.21.15.107:0/196700106 <== mgr.34100 v2:172.21.15.107:6800/3205268226 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+435 (secure 0 0 0) 0x7fdfb810f8d0 con 0x7fdfa0078980 2024-05-17T22:01:45.562 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-05-17T22:01:45.562 INFO:teuthology.orchestra.run.smithi107.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b", 2024-05-17T22:01:45.562 INFO:teuthology.orchestra.run.smithi107.stdout: "in_progress": true, 2024-05-17T22:01:45.562 INFO:teuthology.orchestra.run.smithi107.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-17T22:01:45.562 INFO:teuthology.orchestra.run.smithi107.stdout: "services_complete": [ 2024-05-17T22:01:45.562 INFO:teuthology.orchestra.run.smithi107.stdout: "mon", 2024-05-17T22:01:45.562 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr", 2024-05-17T22:01:45.562 INFO:teuthology.orchestra.run.smithi107.stdout: "mds", 2024-05-17T22:01:45.562 INFO:teuthology.orchestra.run.smithi107.stdout: "osd", 2024-05-17T22:01:45.562 INFO:teuthology.orchestra.run.smithi107.stdout: "crash" 2024-05-17T22:01:45.562 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-05-17T22:01:45.563 INFO:teuthology.orchestra.run.smithi107.stdout: "progress": "16/21 daemons upgraded", 2024-05-17T22:01:45.563 INFO:teuthology.orchestra.run.smithi107.stdout: "message": "Currently upgrading grafana daemons", 2024-05-17T22:01:45.563 INFO:teuthology.orchestra.run.smithi107.stdout: "is_paused": false 2024-05-17T22:01:45.563 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-05-17T22:01:45.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:45.571+0000 7fdfbd660700 1 -- 172.21.15.107:0/196700106 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fdfa0078980 msgr2=0x7fdfa007ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:45.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:45.572+0000 7fdfbd660700 1 --2- 172.21.15.107:0/196700106 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fdfa0078980 0x7fdfa007ae30 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fdfa8004f40 tx=0x7fdfa800b040 comp rx=0 tx=0).stop 2024-05-17T22:01:45.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:45.572+0000 7fdfbd660700 1 -- 172.21.15.107:0/196700106 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdfb80fdff0 msgr2=0x7fdfb810df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:45.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:45.572+0000 7fdfbd660700 1 --2- 172.21.15.107:0/196700106 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdfb80fdff0 0x7fdfb810df40 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fdfac009150 tx=0x7fdfac009e30 comp rx=0 tx=0).stop 2024-05-17T22:01:45.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:45.572+0000 7fdfbd660700 1 -- 172.21.15.107:0/196700106 shutdown_connections 2024-05-17T22:01:45.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:45.572+0000 7fdfbd660700 1 --2- 172.21.15.107:0/196700106 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fdfa0078980 0x7fdfa007ae30 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:45.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:45.572+0000 7fdfbd660700 1 --2- 172.21.15.107:0/196700106 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfb80fce80 0x7fdfb810da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:45.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:45.572+0000 7fdfbd660700 1 --2- 172.21.15.107:0/196700106 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdfb80fdff0 0x7fdfb810df40 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:45.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:45.572+0000 7fdfbd660700 1 -- 172.21.15.107:0/196700106 >> 172.21.15.107:0/196700106 conn(0x7fdfb80f8520 msgr2=0x7fdfb80f9c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:01:45.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:45.572+0000 7fdfbd660700 1 -- 172.21.15.107:0/196700106 shutdown_connections 2024-05-17T22:01:45.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:45.572+0000 7fdfbd660700 1 -- 172.21.15.107:0/196700106 wait complete. 2024-05-17T22:01:45.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:45.676+0000 7fb2ba174700 1 -- 172.21.15.107:0/321674255 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2b40fcf70 msgr2=0x7fb2b40fd3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:45.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:45.676+0000 7fb2ba174700 1 --2- 172.21.15.107:0/321674255 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2b40fcf70 0x7fb2b40fd3e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb2a8004940 tx=0x7fb2a8009e30 comp rx=0 tx=0).stop 2024-05-17T22:01:45.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:45.676+0000 7fb2ba174700 1 -- 172.21.15.107:0/321674255 shutdown_connections 2024-05-17T22:01:45.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:45.676+0000 7fb2ba174700 1 --2- 172.21.15.107:0/321674255 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2b40fcf70 0x7fb2b40fd3e0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:45.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:45.676+0000 7fb2ba174700 1 --2- 172.21.15.107:0/321674255 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2b40fbcc0 0x7fb2b40fc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:45.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:45.676+0000 7fb2ba174700 1 -- 172.21.15.107:0/321674255 >> 172.21.15.107:0/321674255 conn(0x7fb2b40f7440 msgr2=0x7fb2b40f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:01:45.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:45.676+0000 7fb2ba174700 1 -- 172.21.15.107:0/321674255 shutdown_connections 2024-05-17T22:01:45.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:45.676+0000 7fb2ba174700 1 -- 172.21.15.107:0/321674255 wait complete. 2024-05-17T22:01:45.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:45.677+0000 7fb2ba174700 1 Processor -- start 2024-05-17T22:01:45.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:45.677+0000 7fb2ba174700 1 -- start start 2024-05-17T22:01:45.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:45.677+0000 7fb2ba174700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2b40fbcc0 0x7fb2b4066cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:45.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:45.677+0000 7fb2ba174700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2b40fcf70 0x7fb2b41126f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:45.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:45.677+0000 7fb2ba174700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b4112c30 con 0x7fb2b40fbcc0 2024-05-17T22:01:45.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:45.677+0000 7fb2ba174700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b4112d70 con 0x7fb2b40fcf70 2024-05-17T22:01:45.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:45.677+0000 7fb2b37fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2b40fbcc0 0x7fb2b4066cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:45.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:45.677+0000 7fb2b37fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2b40fbcc0 0x7fb2b4066cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40158/0 (socket says 172.21.15.107:40158) 2024-05-17T22:01:45.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:45.677+0000 7fb2b37fe700 1 -- 172.21.15.107:0/624218479 learned_addr learned my addr 172.21.15.107:0/624218479 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T22:01:45.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:45.677+0000 7fb2b2ffd700 1 --2- 172.21.15.107:0/624218479 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2b40fcf70 0x7fb2b41126f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:45.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:45.678+0000 7fb2b37fe700 1 -- 172.21.15.107:0/624218479 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2b40fcf70 msgr2=0x7fb2b41126f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:45.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:45.678+0000 7fb2b37fe700 1 --2- 172.21.15.107:0/624218479 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2b40fcf70 0x7fb2b41126f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:45.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:45.678+0000 7fb2b37fe700 1 -- 172.21.15.107:0/624218479 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2a8005040 con 0x7fb2b40fbcc0 2024-05-17T22:01:45.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:45.678+0000 7fb2b37fe700 1 --2- 172.21.15.107:0/624218479 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2b40fbcc0 0x7fb2b4066cf0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb2a4002a80 tx=0x7fb2a40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:45.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:45.678+0000 7fb29bfff700 1 -- 172.21.15.107:0/624218479 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb2a4007cc0 con 0x7fb2b40fbcc0 2024-05-17T22:01:45.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:45.678+0000 7fb29bfff700 1 -- 172.21.15.107:0/624218479 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb2a400f040 con 0x7fb2b40fbcc0 2024-05-17T22:01:45.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:45.678+0000 7fb2ba174700 1 -- 172.21.15.107:0/624218479 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2b4112fa0 con 0x7fb2b40fbcc0 2024-05-17T22:01:45.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:45.678+0000 7fb29bfff700 1 -- 172.21.15.107:0/624218479 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb2a400da60 con 0x7fb2b40fbcc0 2024-05-17T22:01:45.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:45.678+0000 7fb2ba174700 1 -- 172.21.15.107:0/624218479 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2b4113530 con 0x7fb2b40fbcc0 2024-05-17T22:01:45.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:45.679+0000 7fb2ba174700 1 -- 172.21.15.107:0/624218479 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2b404c710 con 0x7fb2b40fbcc0 2024-05-17T22:01:45.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:45.680+0000 7fb29bfff700 1 -- 172.21.15.107:0/624218479 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7fb2a400dc90 con 0x7fb2b40fbcc0 2024-05-17T22:01:45.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:45.680+0000 7fb29bfff700 1 --2- 172.21.15.107:0/624218479 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fb29c078990 0x7fb29c07ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:45.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:45.680+0000 7fb29bfff700 1 -- 172.21.15.107:0/624218479 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7fb2a409a520 con 0x7fb2b40fbcc0 2024-05-17T22:01:45.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:45.680+0000 7fb2b2ffd700 1 --2- 172.21.15.107:0/624218479 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fb29c078990 0x7fb29c07ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:45.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:45.681+0000 7fb2b2ffd700 1 --2- 172.21.15.107:0/624218479 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fb29c078990 0x7fb29c07ae40 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb2a8004f90 tx=0x7fb2a8009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:45.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:45.685+0000 7fb29bfff700 1 -- 172.21.15.107:0/624218479 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7fb2a40620e0 con 0x7fb2b40fbcc0 2024-05-17T22:01:45.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:45.999+0000 7fb2ba174700 1 -- 172.21.15.107:0/624218479 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fb2b4113be0 con 0x7fb2b40fbcc0 2024-05-17T22:01:45.991 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:45 smithi107 ceph-mon[93224]: from='client.34542 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:01:45.991 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:45 smithi107 ceph-mon[93224]: pgmap v253: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:01:45.991 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:45 smithi107 ceph-mon[93224]: from='client.? 172.21.15.107:0/616990132' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-17T22:01:45.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:46.000+0000 7fb29bfff700 1 -- 172.21.15.107:0/624218479 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7fb2a4061a80 con 0x7fb2b40fbcc0 2024-05-17T22:01:45.991 INFO:teuthology.orchestra.run.smithi107.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-17T22:01:45.992 INFO:teuthology.orchestra.run.smithi107.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-17T22:01:45.992 INFO:teuthology.orchestra.run.smithi107.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-17T22:01:45.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:46.002+0000 7fb2ba174700 1 -- 172.21.15.107:0/624218479 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fb29c078990 msgr2=0x7fb29c07ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:45.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:46.002+0000 7fb2ba174700 1 --2- 172.21.15.107:0/624218479 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fb29c078990 0x7fb29c07ae40 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb2a8004f90 tx=0x7fb2a8009e30 comp rx=0 tx=0).stop 2024-05-17T22:01:45.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:46.002+0000 7fb2ba174700 1 -- 172.21.15.107:0/624218479 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2b40fbcc0 msgr2=0x7fb2b4066cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:45.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:46.002+0000 7fb2ba174700 1 --2- 172.21.15.107:0/624218479 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2b40fbcc0 0x7fb2b4066cf0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb2a4002a80 tx=0x7fb2a40046b0 comp rx=0 tx=0).stop 2024-05-17T22:01:45.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:46.002+0000 7fb2ba174700 1 -- 172.21.15.107:0/624218479 shutdown_connections 2024-05-17T22:01:45.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:46.002+0000 7fb2ba174700 1 --2- 172.21.15.107:0/624218479 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fb29c078990 0x7fb29c07ae40 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:45.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:46.002+0000 7fb2ba174700 1 --2- 172.21.15.107:0/624218479 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2b40fbcc0 0x7fb2b4066cf0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:45.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:46.002+0000 7fb2ba174700 1 --2- 172.21.15.107:0/624218479 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2b40fcf70 0x7fb2b41126f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:45.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:46.002+0000 7fb2ba174700 1 -- 172.21.15.107:0/624218479 >> 172.21.15.107:0/624218479 conn(0x7fb2b40f7440 msgr2=0x7fb2b40f8ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:01:45.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:46.002+0000 7fb2ba174700 1 -- 172.21.15.107:0/624218479 shutdown_connections 2024-05-17T22:01:45.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:01:46.002+0000 7fb2ba174700 1 -- 172.21.15.107:0/624218479 wait complete. 2024-05-17T22:01:46.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:45 smithi138 ceph-mon[81376]: from='client.34542 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:01:46.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:45 smithi138 ceph-mon[81376]: pgmap v253: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:01:46.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:45 smithi138 ceph-mon[81376]: from='client.? 172.21.15.107:0/616990132' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-17T22:01:47.100 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:46 smithi107 ceph-mon[93224]: from='client.34554 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:01:47.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:46 smithi107 ceph-mon[93224]: from='client.? 172.21.15.107:0/624218479' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-17T22:01:47.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:46 smithi138 ceph-mon[81376]: from='client.34554 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:01:47.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:46 smithi138 ceph-mon[81376]: from='client.? 172.21.15.107:0/624218479' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-17T22:01:48.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:47 smithi138 ceph-mon[81376]: pgmap v254: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:01:48.350 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:47 smithi107 ceph-mon[93224]: pgmap v254: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:01:50.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:49 smithi138 ceph-mon[81376]: pgmap v255: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:01:50.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:49 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:01:50.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:49 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:01:50.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:49 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T22:01:50.350 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:49 smithi107 ceph-mon[93224]: pgmap v255: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:01:50.350 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:49 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:01:50.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:49 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:01:50.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:49 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T22:01:51.600 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:51 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:01:51.600 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:51 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:01:51.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:51 smithi107 ceph-mon[93224]: pgmap v256: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:01:51.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:51 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:01:51.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:51 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:01:51.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:51 smithi138 ceph-mon[81376]: pgmap v256: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:01:53.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:53 smithi138 ceph-mon[81376]: pgmap v257: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:01:53.850 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:53 smithi107 ceph-mon[93224]: pgmap v257: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:01:55.825 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:55 smithi138 ceph-mon[81376]: pgmap v258: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:01:55.850 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:55 smithi107 ceph-mon[93224]: pgmap v258: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:01:58.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:57 smithi138 ceph-mon[81376]: pgmap v259: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:01:58.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:57 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:01:58.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:57 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:01:58.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:57 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:01:58.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:57 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:01:58.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:57 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:01:58.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:57 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-05-17T22:01:58.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:57 smithi138 ceph-mon[81376]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-05-17T22:01:58.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:57 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:01:58.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:57 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:01:58.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:57 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:01:58.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:57 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:01:58.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:57 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:01:58.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:57 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:01:58.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:57 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:01:58.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:57 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:01:58.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:57 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:01:58.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:57 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:01:58.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:57 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:01:58.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:57 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:01:58.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:57 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:01:58.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:57 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:01:58.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:57 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:01:58.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:57 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:01:58.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:57 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:01:58.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:57 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:01:58.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:57 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:01:58.328 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:57 smithi138 ceph-mon[81376]: Upgrade: Finalizing container_image settings 2024-05-17T22:01:58.328 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:57 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:01:58.328 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:57 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-05-17T22:01:58.328 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:57 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-05-17T22:01:58.328 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:57 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-17T22:01:58.328 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:57 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-05-17T22:01:58.328 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:57 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-05-17T22:01:58.328 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:57 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-05-17T22:01:58.328 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:57 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-05-17T22:01:58.328 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:57 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-05-17T22:01:58.328 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:57 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-05-17T22:01:58.328 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:57 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-05-17T22:01:58.328 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:57 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-05-17T22:01:58.328 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:57 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-05-17T22:01:58.328 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:57 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-05-17T22:01:58.328 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:57 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-05-17T22:01:58.329 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:57 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-17T22:01:58.329 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:57 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-05-17T22:01:58.329 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:57 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-05-17T22:01:58.329 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:57 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-05-17T22:01:58.329 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:57 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-05-17T22:01:58.329 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:57 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-05-17T22:01:58.329 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:57 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-05-17T22:01:58.329 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:57 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-05-17T22:01:58.329 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:57 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-05-17T22:01:58.329 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:57 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-17T22:01:58.329 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:57 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-17T22:01:58.329 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:57 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-17T22:01:58.329 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:57 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-17T22:01:58.329 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:57 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-17T22:01:58.329 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:57 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-17T22:01:58.330 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:57 smithi138 ceph-mon[81376]: Upgrade: Complete! 2024-05-17T22:01:58.330 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:57 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-05-17T22:01:58.330 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:57 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-05-17T22:01:58.330 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:57 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:01:58.330 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:57 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:01:58.330 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:57 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:01:58.330 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:57 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:01:58.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:57 smithi107 ceph-mon[93224]: pgmap v259: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:01:58.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:57 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:01:58.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:57 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:01:58.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:57 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:01:58.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:57 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:01:58.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:57 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:01:58.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:57 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-05-17T22:01:58.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:57 smithi107 ceph-mon[93224]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-05-17T22:01:58.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:57 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:01:58.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:57 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:01:58.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:57 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:01:58.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:57 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:01:58.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:57 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:01:58.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:57 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:01:58.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:57 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:01:58.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:57 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:01:58.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:57 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:01:58.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:57 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:01:58.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:57 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:01:58.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:57 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:01:58.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:57 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:01:58.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:57 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:01:58.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:57 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:01:58.353 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:57 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:01:58.353 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:57 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:01:58.353 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:57 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:01:58.353 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:57 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:01:58.353 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:57 smithi107 ceph-mon[93224]: Upgrade: Finalizing container_image settings 2024-05-17T22:01:58.353 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:57 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:01:58.353 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:57 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-05-17T22:01:58.353 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:57 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-05-17T22:01:58.353 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:57 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-17T22:01:58.353 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:57 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-05-17T22:01:58.353 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:57 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-05-17T22:01:58.353 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:57 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-05-17T22:01:58.353 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:57 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-05-17T22:01:58.353 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:57 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-05-17T22:01:58.353 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:57 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-05-17T22:01:58.354 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:57 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-05-17T22:01:58.354 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:57 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-05-17T22:01:58.354 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:57 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-05-17T22:01:58.354 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:57 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-05-17T22:01:58.354 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:57 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-05-17T22:01:58.354 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:57 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-17T22:01:58.354 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:57 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-05-17T22:01:58.354 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:57 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-05-17T22:01:58.354 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:57 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-05-17T22:01:58.354 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:57 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-05-17T22:01:58.355 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:57 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-05-17T22:01:58.355 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:57 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-05-17T22:01:58.355 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:57 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-05-17T22:01:58.355 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:57 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-05-17T22:01:58.355 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:57 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-17T22:01:58.355 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:57 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-17T22:01:58.355 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:57 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-17T22:01:58.355 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:57 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-17T22:01:58.355 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:57 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-17T22:01:58.355 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:57 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-17T22:01:58.355 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:57 smithi107 ceph-mon[93224]: Upgrade: Complete! 2024-05-17T22:01:58.355 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:57 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-05-17T22:01:58.355 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:57 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-05-17T22:01:58.356 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:57 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:01:58.356 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:57 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:01:58.356 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:57 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:01:58.356 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:57 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:01:59.255 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:58 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:01:59.256 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:58 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:01:59.256 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:58 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:01:59.256 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:58 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:01:59.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:58 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:01:59.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:58 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:01:59.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:58 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:01:59.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:58 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:02:00.267 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:59 smithi107 ceph-mon[93224]: pgmap v260: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:02:00.267 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:01:59 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:02:00.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:59 smithi138 ceph-mon[81376]: pgmap v260: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:02:00.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:01:59 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:02:02.303 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:02:01 smithi107 ceph-mon[93224]: pgmap v261: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:02:02.324 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:02:01 smithi138 ceph-mon[81376]: pgmap v261: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:02:04.324 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:02:03 smithi138 ceph-mon[81376]: pgmap v262: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:02:04.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:02:03 smithi107 ceph-mon[93224]: pgmap v262: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:02:05.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:02:04 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T22:02:05.350 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:02:04 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T22:02:06.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:02:05 smithi138 ceph-mon[81376]: pgmap v263: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:02:06.350 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:02:05 smithi107 ceph-mon[93224]: pgmap v263: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:02:08.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:02:07 smithi138 ceph-mon[81376]: pgmap v264: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:02:08.350 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:02:07 smithi107 ceph-mon[93224]: pgmap v264: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:02:10.309 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:02:09 smithi107 ceph-mon[93224]: pgmap v265: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:02:10.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:02:09 smithi138 ceph-mon[81376]: pgmap v265: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:02:12.303 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:02:11 smithi107 ceph-mon[93224]: pgmap v266: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:02:12.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:02:11 smithi138 ceph-mon[81376]: pgmap v266: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:02:14.324 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:02:13 smithi138 ceph-mon[81376]: pgmap v267: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:02:14.350 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:02:13 smithi107 ceph-mon[93224]: pgmap v267: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:02:16.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:16.121+0000 7f483d52c700 1 -- 172.21.15.107:0/1187817720 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f48380fcf70 msgr2=0x7f48380fd3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:16.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:16.121+0000 7f483d52c700 1 --2- 172.21.15.107:0/1187817720 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f48380fcf70 0x7f48380fd3e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f482c004940 tx=0x7f482c009e30 comp rx=0 tx=0).stop 2024-05-17T22:02:16.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:16.121+0000 7f483d52c700 1 -- 172.21.15.107:0/1187817720 shutdown_connections 2024-05-17T22:02:16.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:16.121+0000 7f483d52c700 1 --2- 172.21.15.107:0/1187817720 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f48380fcf70 0x7f48380fd3e0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:16.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:16.121+0000 7f483d52c700 1 --2- 172.21.15.107:0/1187817720 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f48380fbcc0 0x7f48380fc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:16.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:16.121+0000 7f483d52c700 1 -- 172.21.15.107:0/1187817720 >> 172.21.15.107:0/1187817720 conn(0x7f48380f7440 msgr2=0x7f48380f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:16.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:16.121+0000 7f483d52c700 1 -- 172.21.15.107:0/1187817720 shutdown_connections 2024-05-17T22:02:16.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:16.121+0000 7f483d52c700 1 -- 172.21.15.107:0/1187817720 wait complete. 2024-05-17T22:02:16.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:16.122+0000 7f483d52c700 1 Processor -- start 2024-05-17T22:02:16.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:16.122+0000 7f483d52c700 1 -- start start 2024-05-17T22:02:16.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:16.122+0000 7f483d52c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f48380fbcc0 0x7f483810c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:16.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:16.122+0000 7f483d52c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f48380fcf70 0x7f483810cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:16.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:16.122+0000 7f483d52c700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f483810d3b0 con 0x7f48380fcf70 2024-05-17T22:02:16.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:16.122+0000 7f483d52c700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f483810d520 con 0x7f48380fbcc0 2024-05-17T22:02:16.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:16.122+0000 7f48367fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f48380fcf70 0x7f483810cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:16.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:16.122+0000 7f48367fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f48380fcf70 0x7f483810cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51422/0 (socket says 172.21.15.107:51422) 2024-05-17T22:02:16.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:16.122+0000 7f48367fc700 1 -- 172.21.15.107:0/1065018385 learned_addr learned my addr 172.21.15.107:0/1065018385 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T22:02:16.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:16.122+0000 7f4836ffd700 1 --2- 172.21.15.107:0/1065018385 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f48380fbcc0 0x7f483810c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:16.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:16.122+0000 7f48367fc700 1 -- 172.21.15.107:0/1065018385 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f48380fbcc0 msgr2=0x7f483810c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:16.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:16.122+0000 7f48367fc700 1 --2- 172.21.15.107:0/1065018385 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f48380fbcc0 0x7f483810c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:16.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:16.122+0000 7f48367fc700 1 -- 172.21.15.107:0/1065018385 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f482c005040 con 0x7f48380fcf70 2024-05-17T22:02:16.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:16.123+0000 7f48367fc700 1 --2- 172.21.15.107:0/1065018385 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f48380fcf70 0x7f483810cdb0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f482c00b970 tx=0x7f482c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:16.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:16.123+0000 7f481f7fe700 1 -- 172.21.15.107:0/1065018385 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f482c00e040 con 0x7f48380fcf70 2024-05-17T22:02:16.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:16.123+0000 7f481f7fe700 1 -- 172.21.15.107:0/1065018385 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f482c00bbf0 con 0x7f48380fcf70 2024-05-17T22:02:16.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:16.123+0000 7f483d52c700 1 -- 172.21.15.107:0/1065018385 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4838111ed0 con 0x7f48380fcf70 2024-05-17T22:02:16.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:16.123+0000 7f481f7fe700 1 -- 172.21.15.107:0/1065018385 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f482c01c630 con 0x7f48380fcf70 2024-05-17T22:02:16.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:16.123+0000 7f483d52c700 1 -- 172.21.15.107:0/1065018385 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4838112460 con 0x7f48380fcf70 2024-05-17T22:02:16.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:16.124+0000 7f483d52c700 1 -- 172.21.15.107:0/1065018385 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f483804c710 con 0x7f48380fcf70 2024-05-17T22:02:16.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:16.125+0000 7f481f7fe700 1 -- 172.21.15.107:0/1065018385 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f482c00bd60 con 0x7f48380fcf70 2024-05-17T22:02:16.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:16.125+0000 7f481f7fe700 1 --2- 172.21.15.107:0/1065018385 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f4820078990 0x7f482007ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:16.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:16.125+0000 7f481f7fe700 1 -- 172.21.15.107:0/1065018385 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f482c09bb00 con 0x7f48380fcf70 2024-05-17T22:02:16.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:16.125+0000 7f4836ffd700 1 --2- 172.21.15.107:0/1065018385 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f4820078990 0x7f482007ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:16.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:16.126+0000 7f4836ffd700 1 --2- 172.21.15.107:0/1065018385 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f4820078990 0x7f482007ae40 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4828004f40 tx=0x7f482800b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:16.121 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:16.130+0000 7f481f7fe700 1 -- 172.21.15.107:0/1065018385 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f482c063740 con 0x7f48380fcf70 2024-05-17T22:02:16.324 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:02:15 smithi138 ceph-mon[81376]: pgmap v268: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:02:16.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:16.350+0000 7f483d52c700 1 -- 172.21.15.107:0/1065018385 --> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f48381012e0 con 0x7f4820078990 2024-05-17T22:02:16.341 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:02:15 smithi107 ceph-mon[93224]: pgmap v268: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:02:16.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:16.351+0000 7f481f7fe700 1 -- 172.21.15.107:0/1065018385 <== mgr.34100 v2:172.21.15.107:6800/3205268226 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7f48381012e0 con 0x7f4820078990 2024-05-17T22:02:16.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:16.353+0000 7f483d52c700 1 -- 172.21.15.107:0/1065018385 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f4820078990 msgr2=0x7f482007ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:16.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:16.353+0000 7f483d52c700 1 --2- 172.21.15.107:0/1065018385 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f4820078990 0x7f482007ae40 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4828004f40 tx=0x7f482800b040 comp rx=0 tx=0).stop 2024-05-17T22:02:16.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:16.353+0000 7f483d52c700 1 -- 172.21.15.107:0/1065018385 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f48380fcf70 msgr2=0x7f483810cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:16.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:16.353+0000 7f483d52c700 1 --2- 172.21.15.107:0/1065018385 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f48380fcf70 0x7f483810cdb0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f482c00b970 tx=0x7f482c009e30 comp rx=0 tx=0).stop 2024-05-17T22:02:16.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:16.353+0000 7f483d52c700 1 -- 172.21.15.107:0/1065018385 shutdown_connections 2024-05-17T22:02:16.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:16.353+0000 7f483d52c700 1 --2- 172.21.15.107:0/1065018385 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f4820078990 0x7f482007ae40 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:16.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:16.353+0000 7f483d52c700 1 --2- 172.21.15.107:0/1065018385 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f48380fbcc0 0x7f483810c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:16.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:16.353+0000 7f483d52c700 1 --2- 172.21.15.107:0/1065018385 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f48380fcf70 0x7f483810cdb0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:16.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:16.353+0000 7f483d52c700 1 -- 172.21.15.107:0/1065018385 >> 172.21.15.107:0/1065018385 conn(0x7f48380f7440 msgr2=0x7f48380f8b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:16.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:16.353+0000 7f483d52c700 1 -- 172.21.15.107:0/1065018385 shutdown_connections 2024-05-17T22:02:16.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:16.353+0000 7f483d52c700 1 -- 172.21.15.107:0/1065018385 wait complete. 2024-05-17T22:02:17.059 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -e sha1=931809a9e5d295bd61984693cac44fe131ded4bc -- bash -c 'ceph orch ps' 2024-05-17T22:02:17.358 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/mon.smithi107/config 2024-05-17T22:02:18.100 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:02:17 smithi107 ceph-mon[93224]: from='client.34562 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:02:18.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:02:17 smithi107 ceph-mon[93224]: pgmap v269: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:02:18.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:02:17 smithi138 ceph-mon[81376]: from='client.34562 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:02:18.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:02:17 smithi138 ceph-mon[81376]: pgmap v269: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:02:19.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:19.086+0000 7fcd632d6700 1 -- 172.21.15.107:0/581014973 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd5c1037a0 msgr2=0x7fcd5c0fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:19.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:19.086+0000 7fcd632d6700 1 --2- 172.21.15.107:0/581014973 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd5c1037a0 0x7fcd5c0fce10 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fcd58004940 tx=0x7fcd58009e30 comp rx=0 tx=0).stop 2024-05-17T22:02:19.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:19.086+0000 7fcd632d6700 1 -- 172.21.15.107:0/581014973 shutdown_connections 2024-05-17T22:02:19.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:19.086+0000 7fcd632d6700 1 --2- 172.21.15.107:0/581014973 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd5c1037a0 0x7fcd5c0fce10 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:19.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:19.086+0000 7fcd632d6700 1 --2- 172.21.15.107:0/581014973 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd5c102e00 0x7fcd5c1031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:19.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:19.086+0000 7fcd632d6700 1 -- 172.21.15.107:0/581014973 >> 172.21.15.107:0/581014973 conn(0x7fcd5c0f89b0 msgr2=0x7fcd5c0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:19.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:19.087+0000 7fcd632d6700 1 -- 172.21.15.107:0/581014973 shutdown_connections 2024-05-17T22:02:19.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:19.087+0000 7fcd632d6700 1 -- 172.21.15.107:0/581014973 wait complete. 2024-05-17T22:02:19.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:19.087+0000 7fcd632d6700 1 Processor -- start 2024-05-17T22:02:19.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:19.087+0000 7fcd632d6700 1 -- start start 2024-05-17T22:02:19.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:19.088+0000 7fcd632d6700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd5c102e00 0x7fcd5c113980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:19.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:19.088+0000 7fcd632d6700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd5c1037a0 0x7fcd5c113ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:19.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:19.088+0000 7fcd632d6700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd5c114510 con 0x7fcd5c1037a0 2024-05-17T22:02:19.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:19.088+0000 7fcd632d6700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd5c10db30 con 0x7fcd5c102e00 2024-05-17T22:02:19.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:19.088+0000 7fcd60871700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd5c1037a0 0x7fcd5c113ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:19.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:19.088+0000 7fcd60871700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd5c1037a0 0x7fcd5c113ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51430/0 (socket says 172.21.15.107:51430) 2024-05-17T22:02:19.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:19.088+0000 7fcd60871700 1 -- 172.21.15.107:0/109380670 learned_addr learned my addr 172.21.15.107:0/109380670 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T22:02:19.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:19.088+0000 7fcd61072700 1 --2- 172.21.15.107:0/109380670 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd5c102e00 0x7fcd5c113980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:19.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:19.088+0000 7fcd60871700 1 -- 172.21.15.107:0/109380670 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd5c102e00 msgr2=0x7fcd5c113980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:19.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:19.088+0000 7fcd60871700 1 --2- 172.21.15.107:0/109380670 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd5c102e00 0x7fcd5c113980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:19.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:19.088+0000 7fcd60871700 1 -- 172.21.15.107:0/109380670 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd58005040 con 0x7fcd5c1037a0 2024-05-17T22:02:19.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:19.088+0000 7fcd60871700 1 --2- 172.21.15.107:0/109380670 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd5c1037a0 0x7fcd5c113ec0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fcd58009180 tx=0x7fcd58009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:19.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:19.088+0000 7fcd51ffb700 1 -- 172.21.15.107:0/109380670 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcd5800e040 con 0x7fcd5c1037a0 2024-05-17T22:02:19.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:19.088+0000 7fcd632d6700 1 -- 172.21.15.107:0/109380670 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd5c10ddb0 con 0x7fcd5c1037a0 2024-05-17T22:02:19.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:19.088+0000 7fcd51ffb700 1 -- 172.21.15.107:0/109380670 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcd58013420 con 0x7fcd5c1037a0 2024-05-17T22:02:19.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:19.088+0000 7fcd51ffb700 1 -- 172.21.15.107:0/109380670 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcd5801c610 con 0x7fcd5c1037a0 2024-05-17T22:02:19.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:19.089+0000 7fcd632d6700 1 -- 172.21.15.107:0/109380670 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd5c10e370 con 0x7fcd5c1037a0 2024-05-17T22:02:19.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:19.090+0000 7fcd632d6700 1 -- 172.21.15.107:0/109380670 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd5c1052d0 con 0x7fcd5c1037a0 2024-05-17T22:02:19.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:19.090+0000 7fcd51ffb700 1 -- 172.21.15.107:0/109380670 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7fcd58013a00 con 0x7fcd5c1037a0 2024-05-17T22:02:19.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:19.091+0000 7fcd51ffb700 1 --2- 172.21.15.107:0/109380670 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fcd48078990 0x7fcd4807ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:19.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:19.091+0000 7fcd51ffb700 1 -- 172.21.15.107:0/109380670 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7fcd5809b7d0 con 0x7fcd5c1037a0 2024-05-17T22:02:19.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:19.091+0000 7fcd61072700 1 --2- 172.21.15.107:0/109380670 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fcd48078990 0x7fcd4807ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:19.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:19.091+0000 7fcd61072700 1 --2- 172.21.15.107:0/109380670 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fcd48078990 0x7fcd4807ae40 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c007fd0 tx=0x7fcd4c009040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:19.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:19.096+0000 7fcd51ffb700 1 -- 172.21.15.107:0/109380670 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7fcd580634b0 con 0x7fcd5c1037a0 2024-05-17T22:02:19.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:19.307+0000 7fcd632d6700 1 -- 172.21.15.107:0/109380670 --> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fcd5c10efd0 con 0x7fcd48078990 2024-05-17T22:02:19.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:19.317+0000 7fcd51ffb700 1 -- 172.21.15.107:0/109380670 <== mgr.34100 v2:172.21.15.107:6800/3205268226 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7fcd5c10efd0 con 0x7fcd48078990 2024-05-17T22:02:19.308 INFO:teuthology.orchestra.run.smithi107.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-17T22:02:19.308 INFO:teuthology.orchestra.run.smithi107.stdout:alertmanager.smithi107 smithi107 *:9093,9094 running (57s) 29s ago 18m 17.2M - 0.25.0 c8568f914cd2 8b363231ac61 2024-05-17T22:02:19.308 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi107 smithi107 running (6m) 29s ago 18m 6669k - 19.0.0-3794-g931809a9 8880f7613d02 face35de205b 2024-05-17T22:02:19.308 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi138 smithi138 running (6m) 87s ago 17m 6665k - 19.0.0-3794-g931809a9 8880f7613d02 ad59e33e6d43 2024-05-17T22:02:19.308 INFO:teuthology.orchestra.run.smithi107.stdout:grafana.smithi107 smithi107 *:3000 running (35s) 29s ago 18m 52.2M - 9.4.12 2bacad6d85d8 1e7c2af290dc 2024-05-17T22:02:19.308 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.drqody smithi107 running (2m) 29s ago 12m 155M - 19.0.0-3794-g931809a9 8880f7613d02 4f02280f8312 2024-05-17T22:02:19.308 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.idmbgj smithi107 running (2m) 29s ago 12m 153M - 19.0.0-3794-g931809a9 8880f7613d02 d541bff38965 2024-05-17T22:02:19.308 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi138.gwbtyd smithi138 running (2m) 87s ago 12m 80.2M - 19.0.0-3794-g931809a9 8880f7613d02 7beb9bd77fc8 2024-05-17T22:02:19.309 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi138.zckrvs smithi138 running (114s) 87s ago 12m 79.2M - 19.0.0-3794-g931809a9 8880f7613d02 17edb3a75f7b 2024-05-17T22:02:19.309 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi107.xhhyiw smithi107 *:8443,9283,8765 running (9m) 29s ago 19m 604M - 19.0.0-3794-g931809a9 8880f7613d02 5acf3e511342 2024-05-17T22:02:19.309 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi138.napnll smithi138 *:8443,9283,8765 running (8m) 87s ago 17m 534M - 19.0.0-3794-g931809a9 8880f7613d02 d627761dbd0d 2024-05-17T22:02:19.309 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi107 smithi107 running (7m) 29s ago 20m 124M 2048M 19.0.0-3794-g931809a9 8880f7613d02 9717f3169f2f 2024-05-17T22:02:19.309 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi138 smithi138 running (7m) 87s ago 17m 107M 2048M 19.0.0-3794-g931809a9 8880f7613d02 55aeed62bf4d 2024-05-17T22:02:19.309 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi107 smithi107 *:9100 running (95s) 29s ago 18m 8841k - 1.5.0 0da6a335fe13 73f1896972ff 2024-05-17T22:02:19.309 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi138 smithi138 *:9100 running (90s) 87s ago 17m 3565k - 1.5.0 0da6a335fe13 05bb3093f9aa 2024-05-17T22:02:19.309 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0 smithi107 running (6m) 29s ago 16m 166M 1557M 19.0.0-3794-g931809a9 8880f7613d02 b319bc93500d 2024-05-17T22:02:19.309 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1 smithi107 running (5m) 29s ago 16m 166M 1557M 19.0.0-3794-g931809a9 8880f7613d02 f94b206be6eb 2024-05-17T22:02:19.309 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2 smithi107 running (4m) 29s ago 15m 154M 1557M 19.0.0-3794-g931809a9 8880f7613d02 add7015647af 2024-05-17T22:02:19.309 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3 smithi138 running (4m) 87s ago 15m 174M 2581M 19.0.0-3794-g931809a9 8880f7613d02 416273d327c7 2024-05-17T22:02:19.309 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4 smithi138 running (3m) 87s ago 14m 159M 2581M 19.0.0-3794-g931809a9 8880f7613d02 e33f411da8e6 2024-05-17T22:02:19.309 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5 smithi138 running (3m) 87s ago 14m 159M 2581M 19.0.0-3794-g931809a9 8880f7613d02 63810d431961 2024-05-17T22:02:19.309 INFO:teuthology.orchestra.run.smithi107.stdout:prometheus.smithi107 smithi107 *:9095 running (74s) 29s ago 17m 51.3M - 2.43.0 a07b618ecd1d aee4bd577d3d 2024-05-17T22:02:19.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:19.319+0000 7fcd632d6700 1 -- 172.21.15.107:0/109380670 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fcd48078990 msgr2=0x7fcd4807ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:19.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:19.319+0000 7fcd632d6700 1 --2- 172.21.15.107:0/109380670 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fcd48078990 0x7fcd4807ae40 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c007fd0 tx=0x7fcd4c009040 comp rx=0 tx=0).stop 2024-05-17T22:02:19.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:19.319+0000 7fcd632d6700 1 -- 172.21.15.107:0/109380670 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd5c1037a0 msgr2=0x7fcd5c113ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:19.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:19.319+0000 7fcd632d6700 1 --2- 172.21.15.107:0/109380670 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd5c1037a0 0x7fcd5c113ec0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fcd58009180 tx=0x7fcd58009e30 comp rx=0 tx=0).stop 2024-05-17T22:02:19.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:19.319+0000 7fcd632d6700 1 -- 172.21.15.107:0/109380670 shutdown_connections 2024-05-17T22:02:19.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:19.319+0000 7fcd632d6700 1 --2- 172.21.15.107:0/109380670 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fcd48078990 0x7fcd4807ae40 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:19.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:19.319+0000 7fcd632d6700 1 --2- 172.21.15.107:0/109380670 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd5c102e00 0x7fcd5c113980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:19.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:19.319+0000 7fcd632d6700 1 --2- 172.21.15.107:0/109380670 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd5c1037a0 0x7fcd5c113ec0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:19.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:19.319+0000 7fcd632d6700 1 -- 172.21.15.107:0/109380670 >> 172.21.15.107:0/109380670 conn(0x7fcd5c0f89b0 msgr2=0x7fcd5c1021c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:19.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:19.319+0000 7fcd632d6700 1 -- 172.21.15.107:0/109380670 shutdown_connections 2024-05-17T22:02:19.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:19.319+0000 7fcd632d6700 1 -- 172.21.15.107:0/109380670 wait complete. 2024-05-17T22:02:20.001 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -e sha1=931809a9e5d295bd61984693cac44fe131ded4bc -- bash -c 'ceph orch upgrade status' 2024-05-17T22:02:20.257 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/mon.smithi107/config 2024-05-17T22:02:20.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:02:19 smithi138 ceph-mon[81376]: pgmap v270: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:02:20.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:02:19 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T22:02:20.350 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:02:19 smithi107 ceph-mon[93224]: pgmap v270: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:02:20.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:02:19 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T22:02:21.058 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:02:20 smithi107 ceph-mon[93224]: from='client.34566 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:02:21.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:02:20 smithi138 ceph-mon[81376]: from='client.34566 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:02:21.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:21.822+0000 7f299b544700 1 -- 172.21.15.107:0/1165386427 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2994103760 msgr2=0x7f29940fcdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:21.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:21.822+0000 7f299b544700 1 --2- 172.21.15.107:0/1165386427 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2994103760 0x7f29940fcdd0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2988009ce0 tx=0x7f2988004d10 comp rx=0 tx=0).stop 2024-05-17T22:02:21.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:21.822+0000 7f299b544700 1 -- 172.21.15.107:0/1165386427 shutdown_connections 2024-05-17T22:02:21.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:21.822+0000 7f299b544700 1 --2- 172.21.15.107:0/1165386427 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2994103760 0x7f29940fcdd0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:21.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:21.822+0000 7f299b544700 1 --2- 172.21.15.107:0/1165386427 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2994102dc0 0x7f2994103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:21.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:21.822+0000 7f299b544700 1 -- 172.21.15.107:0/1165386427 >> 172.21.15.107:0/1165386427 conn(0x7f29940f8930 msgr2=0x7f29940fad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:21.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:21.822+0000 7f299b544700 1 -- 172.21.15.107:0/1165386427 shutdown_connections 2024-05-17T22:02:21.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:21.822+0000 7f299b544700 1 -- 172.21.15.107:0/1165386427 wait complete. 2024-05-17T22:02:21.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:21.823+0000 7f299b544700 1 Processor -- start 2024-05-17T22:02:21.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:21.823+0000 7f299b544700 1 -- start start 2024-05-17T22:02:21.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:21.823+0000 7f299b544700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2994102dc0 0x7f2994113ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:21.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:21.823+0000 7f299b544700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2994103760 0x7f2994114000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:21.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:21.823+0000 7f299b544700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2994114650 con 0x7f2994102dc0 2024-05-17T22:02:21.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:21.823+0000 7f299b544700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f299410dc30 con 0x7f2994103760 2024-05-17T22:02:21.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:21.823+0000 7f29992e0700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2994102dc0 0x7f2994113ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:21.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:21.823+0000 7f29992e0700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2994102dc0 0x7f2994113ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51448/0 (socket says 172.21.15.107:51448) 2024-05-17T22:02:21.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:21.823+0000 7f29992e0700 1 -- 172.21.15.107:0/212601633 learned_addr learned my addr 172.21.15.107:0/212601633 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T22:02:21.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:21.823+0000 7f2998adf700 1 --2- 172.21.15.107:0/212601633 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2994103760 0x7f2994114000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:21.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:21.824+0000 7f29992e0700 1 -- 172.21.15.107:0/212601633 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2994103760 msgr2=0x7f2994114000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:21.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:21.824+0000 7f29992e0700 1 --2- 172.21.15.107:0/212601633 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2994103760 0x7f2994114000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:21.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:21.824+0000 7f29992e0700 1 -- 172.21.15.107:0/212601633 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2988009940 con 0x7f2994102dc0 2024-05-17T22:02:21.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:21.824+0000 7f29992e0700 1 --2- 172.21.15.107:0/212601633 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2994102dc0 0x7f2994113ac0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f2984002a50 tx=0x7f29840046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:21.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:21.824+0000 7f2991ffb700 1 -- 172.21.15.107:0/212601633 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2984007cc0 con 0x7f2994102dc0 2024-05-17T22:02:21.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:21.824+0000 7f2991ffb700 1 -- 172.21.15.107:0/212601633 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f298400f040 con 0x7f2994102dc0 2024-05-17T22:02:21.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:21.824+0000 7f2991ffb700 1 -- 172.21.15.107:0/212601633 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f298400da30 con 0x7f2994102dc0 2024-05-17T22:02:21.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:21.824+0000 7f299b544700 1 -- 172.21.15.107:0/212601633 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f299410df10 con 0x7f2994102dc0 2024-05-17T22:02:21.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:21.824+0000 7f299b544700 1 -- 172.21.15.107:0/212601633 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f299410e4a0 con 0x7f2994102dc0 2024-05-17T22:02:21.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:21.825+0000 7f299b544700 1 -- 172.21.15.107:0/212601633 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29940fe510 con 0x7f2994102dc0 2024-05-17T22:02:21.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:21.826+0000 7f2991ffb700 1 -- 172.21.15.107:0/212601633 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f298400db90 con 0x7f2994102dc0 2024-05-17T22:02:21.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:21.826+0000 7f2991ffb700 1 --2- 172.21.15.107:0/212601633 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f297c078990 0x7f297c07ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:21.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:21.826+0000 7f2991ffb700 1 -- 172.21.15.107:0/212601633 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f298409a530 con 0x7f2994102dc0 2024-05-17T22:02:21.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:21.826+0000 7f2998adf700 1 --2- 172.21.15.107:0/212601633 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f297c078990 0x7f297c07ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:21.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:21.827+0000 7f2998adf700 1 --2- 172.21.15.107:0/212601633 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f297c078990 0x7f297c07ae40 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f298800b740 tx=0x7f2988004d10 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:21.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:21.831+0000 7f2991ffb700 1 -- 172.21.15.107:0/212601633 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f29840620f0 con 0x7f2994102dc0 2024-05-17T22:02:22.048 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:22.057+0000 7f299b544700 1 -- 172.21.15.107:0/212601633 --> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f299410f180 con 0x7f297c078990 2024-05-17T22:02:22.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:22.058+0000 7f2991ffb700 1 -- 172.21.15.107:0/212601633 <== mgr.34100 v2:172.21.15.107:6800/3205268226 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7f299410f180 con 0x7f297c078990 2024-05-17T22:02:22.049 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-05-17T22:02:22.049 INFO:teuthology.orchestra.run.smithi107.stdout: "target_image": null, 2024-05-17T22:02:22.049 INFO:teuthology.orchestra.run.smithi107.stdout: "in_progress": false, 2024-05-17T22:02:22.050 INFO:teuthology.orchestra.run.smithi107.stdout: "which": "", 2024-05-17T22:02:22.050 INFO:teuthology.orchestra.run.smithi107.stdout: "services_complete": [], 2024-05-17T22:02:22.050 INFO:teuthology.orchestra.run.smithi107.stdout: "progress": null, 2024-05-17T22:02:22.050 INFO:teuthology.orchestra.run.smithi107.stdout: "message": "", 2024-05-17T22:02:22.050 INFO:teuthology.orchestra.run.smithi107.stdout: "is_paused": false 2024-05-17T22:02:22.050 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-05-17T22:02:22.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:22.060+0000 7f299b544700 1 -- 172.21.15.107:0/212601633 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f297c078990 msgr2=0x7f297c07ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:22.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:22.060+0000 7f299b544700 1 --2- 172.21.15.107:0/212601633 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f297c078990 0x7f297c07ae40 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f298800b740 tx=0x7f2988004d10 comp rx=0 tx=0).stop 2024-05-17T22:02:22.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:22.060+0000 7f299b544700 1 -- 172.21.15.107:0/212601633 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2994102dc0 msgr2=0x7f2994113ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:22.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:22.060+0000 7f299b544700 1 --2- 172.21.15.107:0/212601633 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2994102dc0 0x7f2994113ac0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f2984002a50 tx=0x7f29840046b0 comp rx=0 tx=0).stop 2024-05-17T22:02:22.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:22.061+0000 7f299b544700 1 -- 172.21.15.107:0/212601633 shutdown_connections 2024-05-17T22:02:22.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:22.061+0000 7f299b544700 1 --2- 172.21.15.107:0/212601633 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f297c078990 0x7f297c07ae40 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:22.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:22.061+0000 7f299b544700 1 --2- 172.21.15.107:0/212601633 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2994102dc0 0x7f2994113ac0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:22.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:22.061+0000 7f299b544700 1 --2- 172.21.15.107:0/212601633 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2994103760 0x7f2994114000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:22.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:22.061+0000 7f299b544700 1 -- 172.21.15.107:0/212601633 >> 172.21.15.107:0/212601633 conn(0x7f29940f8930 msgr2=0x7f29940f98f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:22.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:22.061+0000 7f299b544700 1 -- 172.21.15.107:0/212601633 shutdown_connections 2024-05-17T22:02:22.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:22.061+0000 7f299b544700 1 -- 172.21.15.107:0/212601633 wait complete. 2024-05-17T22:02:22.303 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:02:21 smithi107 ceph-mon[93224]: pgmap v271: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:02:22.324 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:02:21 smithi138 ceph-mon[81376]: pgmap v271: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:02:22.650 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -e sha1=931809a9e5d295bd61984693cac44fe131ded4bc -- bash -c 'ceph health detail' 2024-05-17T22:02:22.907 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/mon.smithi107/config 2024-05-17T22:02:23.286 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:02:22 smithi107 ceph-mon[93224]: from='client.34570 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:02:23.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:02:22 smithi138 ceph-mon[81376]: from='client.34570 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:02:24.032 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:02:23 smithi107 ceph-mon[93224]: pgmap v272: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:02:24.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:02:23 smithi138 ceph-mon[81376]: pgmap v272: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:02:24.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:24.547+0000 7fd5b3732700 1 -- 172.21.15.107:0/1742454596 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd5ac101cd0 msgr2=0x7fd5ac1020a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:24.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:24.547+0000 7fd5b3732700 1 --2- 172.21.15.107:0/1742454596 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd5ac101cd0 0x7fd5ac1020a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd5a4004940 tx=0x7fd5a4009e30 comp rx=0 tx=0).stop 2024-05-17T22:02:24.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:24.547+0000 7fd5b3732700 1 -- 172.21.15.107:0/1742454596 shutdown_connections 2024-05-17T22:02:24.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:24.547+0000 7fd5b3732700 1 --2- 172.21.15.107:0/1742454596 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd5ac102670 0x7fd5ac0fbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:24.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:24.547+0000 7fd5b3732700 1 --2- 172.21.15.107:0/1742454596 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd5ac101cd0 0x7fd5ac1020a0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:24.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:24.547+0000 7fd5b3732700 1 -- 172.21.15.107:0/1742454596 >> 172.21.15.107:0/1742454596 conn(0x7fd5ac0f7860 msgr2=0x7fd5ac0f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:24.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:24.548+0000 7fd5b3732700 1 -- 172.21.15.107:0/1742454596 shutdown_connections 2024-05-17T22:02:24.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:24.548+0000 7fd5b3732700 1 -- 172.21.15.107:0/1742454596 wait complete. 2024-05-17T22:02:24.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:24.548+0000 7fd5b3732700 1 Processor -- start 2024-05-17T22:02:24.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:24.548+0000 7fd5b3732700 1 -- start start 2024-05-17T22:02:24.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:24.548+0000 7fd5b3732700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd5ac101cd0 0x7fd5ac10eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:24.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:24.548+0000 7fd5b3732700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd5ac102670 0x7fd5ac10f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:24.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:24.548+0000 7fd5b3732700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5ac10f570 con 0x7fd5ac101cd0 2024-05-17T22:02:24.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:24.548+0000 7fd5b3732700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5ac10f6e0 con 0x7fd5ac102670 2024-05-17T22:02:24.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:24.548+0000 7fd5b14ce700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd5ac101cd0 0x7fd5ac10eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:24.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:24.549+0000 7fd5b14ce700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd5ac101cd0 0x7fd5ac10eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51456/0 (socket says 172.21.15.107:51456) 2024-05-17T22:02:24.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:24.549+0000 7fd5b14ce700 1 -- 172.21.15.107:0/2360911378 learned_addr learned my addr 172.21.15.107:0/2360911378 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T22:02:24.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:24.549+0000 7fd5b0ccd700 1 --2- 172.21.15.107:0/2360911378 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd5ac102670 0x7fd5ac10f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:24.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:24.549+0000 7fd5b14ce700 1 -- 172.21.15.107:0/2360911378 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd5ac102670 msgr2=0x7fd5ac10f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:24.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:24.549+0000 7fd5b14ce700 1 --2- 172.21.15.107:0/2360911378 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd5ac102670 0x7fd5ac10f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:24.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:24.549+0000 7fd5b14ce700 1 -- 172.21.15.107:0/2360911378 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5a4005040 con 0x7fd5ac101cd0 2024-05-17T22:02:24.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:24.549+0000 7fd5b0ccd700 1 --2- 172.21.15.107:0/2360911378 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd5ac102670 0x7fd5ac10f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:02:24.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:24.549+0000 7fd5b14ce700 1 --2- 172.21.15.107:0/2360911378 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd5ac101cd0 0x7fd5ac10eaf0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd5a400b3b0 tx=0x7fd5a4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:24.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:24.549+0000 7fd5a9ffb700 1 -- 172.21.15.107:0/2360911378 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd5a400e040 con 0x7fd5ac101cd0 2024-05-17T22:02:24.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:24.549+0000 7fd5b3732700 1 -- 172.21.15.107:0/2360911378 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5ac10f960 con 0x7fd5ac101cd0 2024-05-17T22:02:24.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:24.549+0000 7fd5a9ffb700 1 -- 172.21.15.107:0/2360911378 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd5a400bbc0 con 0x7fd5ac101cd0 2024-05-17T22:02:24.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:24.549+0000 7fd5a9ffb700 1 -- 172.21.15.107:0/2360911378 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd5a401c3f0 con 0x7fd5ac101cd0 2024-05-17T22:02:24.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:24.549+0000 7fd5b3732700 1 -- 172.21.15.107:0/2360911378 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5ac117e60 con 0x7fd5ac101cd0 2024-05-17T22:02:24.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:24.550+0000 7fd5b3732700 1 -- 172.21.15.107:0/2360911378 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5ac0fd420 con 0x7fd5ac101cd0 2024-05-17T22:02:24.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:24.551+0000 7fd5a9ffb700 1 -- 172.21.15.107:0/2360911378 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7fd5a400bd30 con 0x7fd5ac101cd0 2024-05-17T22:02:24.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:24.551+0000 7fd5a9ffb700 1 --2- 172.21.15.107:0/2360911378 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fd594078990 0x7fd59407ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:24.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:24.552+0000 7fd5a9ffb700 1 -- 172.21.15.107:0/2360911378 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7fd5a409b900 con 0x7fd5ac101cd0 2024-05-17T22:02:24.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:24.552+0000 7fd5b0ccd700 1 --2- 172.21.15.107:0/2360911378 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fd594078990 0x7fd59407ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:24.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:24.552+0000 7fd5b0ccd700 1 --2- 172.21.15.107:0/2360911378 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fd594078990 0x7fd59407ae40 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd5a0009fd0 tx=0x7fd5a0009f20 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:24.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:24.556+0000 7fd5a9ffb700 1 -- 172.21.15.107:0/2360911378 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7fd5a40634c0 con 0x7fd5ac101cd0 2024-05-17T22:02:24.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:24.848+0000 7fd5b3732700 1 -- 172.21.15.107:0/2360911378 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fd5ac10f900 con 0x7fd5ac101cd0 2024-05-17T22:02:24.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:24.848+0000 7fd5a9ffb700 1 -- 172.21.15.107:0/2360911378 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7fd5a4062e60 con 0x7fd5ac101cd0 2024-05-17T22:02:24.839 INFO:teuthology.orchestra.run.smithi107.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-17T22:02:24.839 INFO:teuthology.orchestra.run.smithi107.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-17T22:02:24.839 INFO:teuthology.orchestra.run.smithi107.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-17T22:02:24.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:24.851+0000 7fd5b3732700 1 -- 172.21.15.107:0/2360911378 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fd594078990 msgr2=0x7fd59407ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:24.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:24.851+0000 7fd5b3732700 1 --2- 172.21.15.107:0/2360911378 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fd594078990 0x7fd59407ae40 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd5a0009fd0 tx=0x7fd5a0009f20 comp rx=0 tx=0).stop 2024-05-17T22:02:24.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:24.851+0000 7fd5b3732700 1 -- 172.21.15.107:0/2360911378 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd5ac101cd0 msgr2=0x7fd5ac10eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:24.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:24.851+0000 7fd5b3732700 1 --2- 172.21.15.107:0/2360911378 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd5ac101cd0 0x7fd5ac10eaf0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd5a400b3b0 tx=0x7fd5a4009e30 comp rx=0 tx=0).stop 2024-05-17T22:02:24.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:24.851+0000 7fd5b3732700 1 -- 172.21.15.107:0/2360911378 shutdown_connections 2024-05-17T22:02:24.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:24.851+0000 7fd5b3732700 1 --2- 172.21.15.107:0/2360911378 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fd594078990 0x7fd59407ae40 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:24.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:24.851+0000 7fd5b3732700 1 --2- 172.21.15.107:0/2360911378 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd5ac101cd0 0x7fd5ac10eaf0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:24.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:24.851+0000 7fd5b3732700 1 --2- 172.21.15.107:0/2360911378 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd5ac102670 0x7fd5ac10f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:24.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:24.851+0000 7fd5b3732700 1 -- 172.21.15.107:0/2360911378 >> 172.21.15.107:0/2360911378 conn(0x7fd5ac0f7860 msgr2=0x7fd5ac0f87f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:24.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:24.851+0000 7fd5b3732700 1 -- 172.21.15.107:0/2360911378 shutdown_connections 2024-05-17T22:02:24.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:24.851+0000 7fd5b3732700 1 -- 172.21.15.107:0/2360911378 wait complete. 2024-05-17T22:02:25.457 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -e sha1=931809a9e5d295bd61984693cac44fe131ded4bc -- bash -c 'ceph versions' 2024-05-17T22:02:25.714 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/mon.smithi107/config 2024-05-17T22:02:25.989 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:02:25 smithi107 ceph-mon[93224]: pgmap v273: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:02:25.989 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:02:25 smithi107 ceph-mon[93224]: from='client.? 172.21.15.107:0/2360911378' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-17T22:02:26.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:02:25 smithi138 ceph-mon[81376]: pgmap v273: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:02:26.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:02:25 smithi138 ceph-mon[81376]: from='client.? 172.21.15.107:0/2360911378' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-17T22:02:27.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:27.461+0000 7f928de0a700 1 -- 172.21.15.107:0/2797473308 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9288101d00 msgr2=0x7f92881020d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:27.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:27.461+0000 7f928de0a700 1 --2- 172.21.15.107:0/2797473308 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9288101d00 0x7f92881020d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9278004940 tx=0x7f9278009e30 comp rx=0 tx=0).stop 2024-05-17T22:02:27.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:27.461+0000 7f928de0a700 1 -- 172.21.15.107:0/2797473308 shutdown_connections 2024-05-17T22:02:27.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:27.461+0000 7f928de0a700 1 --2- 172.21.15.107:0/2797473308 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f92881026a0 0x7f92880fbd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:27.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:27.461+0000 7f928de0a700 1 --2- 172.21.15.107:0/2797473308 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9288101d00 0x7f92881020d0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:27.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:27.461+0000 7f928de0a700 1 -- 172.21.15.107:0/2797473308 >> 172.21.15.107:0/2797473308 conn(0x7f92880f7870 msgr2=0x7f92880f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:27.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:27.461+0000 7f928de0a700 1 -- 172.21.15.107:0/2797473308 shutdown_connections 2024-05-17T22:02:27.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:27.461+0000 7f928de0a700 1 -- 172.21.15.107:0/2797473308 wait complete. 2024-05-17T22:02:27.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:27.461+0000 7f928de0a700 1 Processor -- start 2024-05-17T22:02:27.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:27.462+0000 7f928de0a700 1 -- start start 2024-05-17T22:02:27.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:27.462+0000 7f928de0a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9288101d00 0x7f92881139e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:27.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:27.462+0000 7f928de0a700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92881026a0 0x7f9288113f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:27.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:27.462+0000 7f928de0a700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92881144d0 con 0x7f92881026a0 2024-05-17T22:02:27.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:27.462+0000 7f928de0a700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f928810db20 con 0x7f9288101d00 2024-05-17T22:02:27.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:27.462+0000 7f9286ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92881026a0 0x7f9288113f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:27.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:27.462+0000 7f9286ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92881026a0 0x7f9288113f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48638/0 (socket says 172.21.15.107:48638) 2024-05-17T22:02:27.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:27.462+0000 7f9286ffd700 1 -- 172.21.15.107:0/4234640006 learned_addr learned my addr 172.21.15.107:0/4234640006 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T22:02:27.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:27.462+0000 7f92877fe700 1 --2- 172.21.15.107:0/4234640006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9288101d00 0x7f92881139e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:27.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:27.462+0000 7f9286ffd700 1 -- 172.21.15.107:0/4234640006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9288101d00 msgr2=0x7f92881139e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:27.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:27.462+0000 7f9286ffd700 1 --2- 172.21.15.107:0/4234640006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9288101d00 0x7f92881139e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:27.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:27.462+0000 7f9286ffd700 1 -- 172.21.15.107:0/4234640006 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9278005040 con 0x7f92881026a0 2024-05-17T22:02:27.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:27.462+0000 7f9286ffd700 1 --2- 172.21.15.107:0/4234640006 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92881026a0 0x7f9288113f20 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f927c002a50 tx=0x7f927c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:27.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:27.463+0000 7f926ffff700 1 -- 172.21.15.107:0/4234640006 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f927c007cc0 con 0x7f92881026a0 2024-05-17T22:02:27.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:27.463+0000 7f928de0a700 1 -- 172.21.15.107:0/4234640006 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f928810de00 con 0x7f92881026a0 2024-05-17T22:02:27.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:27.463+0000 7f926ffff700 1 -- 172.21.15.107:0/4234640006 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f927c00f040 con 0x7f92881026a0 2024-05-17T22:02:27.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:27.463+0000 7f926ffff700 1 -- 172.21.15.107:0/4234640006 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f927c00d9c0 con 0x7f92881026a0 2024-05-17T22:02:27.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:27.463+0000 7f928de0a700 1 -- 172.21.15.107:0/4234640006 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f928810e3c0 con 0x7f92881026a0 2024-05-17T22:02:27.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:27.464+0000 7f928de0a700 1 -- 172.21.15.107:0/4234640006 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92880fd450 con 0x7f92881026a0 2024-05-17T22:02:27.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:27.464+0000 7f926ffff700 1 -- 172.21.15.107:0/4234640006 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f927c009730 con 0x7f92881026a0 2024-05-17T22:02:27.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:27.465+0000 7f926ffff700 1 --2- 172.21.15.107:0/4234640006 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f9270078990 0x7f927007ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:27.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:27.465+0000 7f926ffff700 1 -- 172.21.15.107:0/4234640006 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f927c09a530 con 0x7f92881026a0 2024-05-17T22:02:27.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:27.465+0000 7f92877fe700 1 --2- 172.21.15.107:0/4234640006 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f9270078990 0x7f927007ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:27.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:27.465+0000 7f92877fe700 1 --2- 172.21.15.107:0/4234640006 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f9270078990 0x7f927007ae40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9278004f90 tx=0x7f9278009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:27.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:27.469+0000 7f926ffff700 1 -- 172.21.15.107:0/4234640006 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f927c0620f0 con 0x7f92881026a0 2024-05-17T22:02:27.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:27.768+0000 7f928de0a700 1 -- 172.21.15.107:0/4234640006 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f9288108e00 con 0x7f92881026a0 2024-05-17T22:02:27.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:27.768+0000 7f926ffff700 1 -- 172.21.15.107:0/4234640006 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f927c061a90 con 0x7f92881026a0 2024-05-17T22:02:27.759 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-05-17T22:02:27.759 INFO:teuthology.orchestra.run.smithi107.stdout: "mon": { 2024-05-17T22:02:27.759 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev)": 2 2024-05-17T22:02:27.759 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-05-17T22:02:27.759 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr": { 2024-05-17T22:02:27.759 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev)": 2 2024-05-17T22:02:27.759 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-05-17T22:02:27.759 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": { 2024-05-17T22:02:27.759 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev)": 6 2024-05-17T22:02:27.759 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-05-17T22:02:27.759 INFO:teuthology.orchestra.run.smithi107.stdout: "mds": { 2024-05-17T22:02:27.759 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev)": 4 2024-05-17T22:02:27.760 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-05-17T22:02:27.760 INFO:teuthology.orchestra.run.smithi107.stdout: "overall": { 2024-05-17T22:02:27.760 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev)": 14 2024-05-17T22:02:27.760 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-05-17T22:02:27.760 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-05-17T22:02:27.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:27.771+0000 7f928de0a700 1 -- 172.21.15.107:0/4234640006 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f9270078990 msgr2=0x7f927007ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:27.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:27.771+0000 7f928de0a700 1 --2- 172.21.15.107:0/4234640006 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f9270078990 0x7f927007ae40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9278004f90 tx=0x7f9278009e30 comp rx=0 tx=0).stop 2024-05-17T22:02:27.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:27.771+0000 7f928de0a700 1 -- 172.21.15.107:0/4234640006 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92881026a0 msgr2=0x7f9288113f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:27.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:27.771+0000 7f928de0a700 1 --2- 172.21.15.107:0/4234640006 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92881026a0 0x7f9288113f20 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f927c002a50 tx=0x7f927c004600 comp rx=0 tx=0).stop 2024-05-17T22:02:27.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:27.771+0000 7f928de0a700 1 -- 172.21.15.107:0/4234640006 shutdown_connections 2024-05-17T22:02:27.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:27.771+0000 7f928de0a700 1 --2- 172.21.15.107:0/4234640006 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f9270078990 0x7f927007ae40 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:27.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:27.771+0000 7f928de0a700 1 --2- 172.21.15.107:0/4234640006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9288101d00 0x7f92881139e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:27.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:27.771+0000 7f928de0a700 1 --2- 172.21.15.107:0/4234640006 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92881026a0 0x7f9288113f20 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:27.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:27.771+0000 7f928de0a700 1 -- 172.21.15.107:0/4234640006 >> 172.21.15.107:0/4234640006 conn(0x7f92880f7870 msgr2=0x7f9288066250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:27.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:27.771+0000 7f928de0a700 1 -- 172.21.15.107:0/4234640006 shutdown_connections 2024-05-17T22:02:27.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:02:27.771+0000 7f928de0a700 1 -- 172.21.15.107:0/4234640006 wait complete. 2024-05-17T22:02:28.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:02:27 smithi138 ceph-mon[81376]: pgmap v274: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:02:28.100 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:02:27 smithi107 ceph-mon[93224]: pgmap v274: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:02:28.469 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -e sha1=931809a9e5d295bd61984693cac44fe131ded4bc -- bash -c 'echo "wait for servicemap items w/ changing names to refresh"' 2024-05-17T22:02:28.748 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/mon.smithi107/config 2024-05-17T22:02:29.069 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:02:28 smithi107 ceph-mon[93224]: from='client.? 172.21.15.107:0/4234640006' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:02:29.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:02:28 smithi138 ceph-mon[81376]: from='client.? 172.21.15.107:0/4234640006' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:02:30.042 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:02:29 smithi107 ceph-mon[93224]: pgmap v275: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:02:30.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:02:29 smithi138 ceph-mon[81376]: pgmap v275: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:02:30.387 INFO:teuthology.orchestra.run.smithi107.stdout:wait for servicemap items w/ changing names to refresh 2024-05-17T22:02:31.193 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -e sha1=931809a9e5d295bd61984693cac44fe131ded4bc -- bash -c 'sleep 60' 2024-05-17T22:02:31.452 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/mon.smithi107/config 2024-05-17T22:02:31.850 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:02:31 smithi107 ceph-mon[93224]: pgmap v276: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:02:32.074 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:02:31 smithi138 ceph-mon[81376]: pgmap v276: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:02:34.074 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:02:33 smithi138 ceph-mon[81376]: pgmap v277: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:02:34.100 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:02:33 smithi107 ceph-mon[93224]: pgmap v277: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:02:35.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:02:34 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T22:02:35.100 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:02:34 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T22:02:36.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:02:35 smithi138 ceph-mon[81376]: pgmap v278: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:02:36.100 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:02:35 smithi107 ceph-mon[93224]: pgmap v278: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:02:38.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:02:37 smithi138 ceph-mon[81376]: pgmap v279: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:02:38.100 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:02:37 smithi107 ceph-mon[93224]: pgmap v279: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:02:40.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:02:39 smithi138 ceph-mon[81376]: pgmap v280: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:02:40.100 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:02:39 smithi107 ceph-mon[93224]: pgmap v280: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:02:42.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:02:41 smithi138 ceph-mon[81376]: pgmap v281: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:02:42.100 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:02:41 smithi107 ceph-mon[93224]: pgmap v281: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:02:44.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:02:43 smithi138 ceph-mon[81376]: pgmap v282: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:02:44.100 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:02:43 smithi107 ceph-mon[93224]: pgmap v282: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:02:46.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:02:45 smithi138 ceph-mon[81376]: pgmap v283: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:02:46.100 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:02:45 smithi107 ceph-mon[93224]: pgmap v283: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:02:48.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:02:47 smithi138 ceph-mon[81376]: pgmap v284: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:02:48.100 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:02:47 smithi107 ceph-mon[93224]: pgmap v284: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:02:50.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:02:49 smithi138 ceph-mon[81376]: pgmap v285: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:02:50.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:02:49 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T22:02:50.100 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:02:49 smithi107 ceph-mon[93224]: pgmap v285: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:02:50.100 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:02:49 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T22:02:52.074 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:02:51 smithi138 ceph-mon[81376]: pgmap v286: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:02:52.100 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:02:51 smithi107 ceph-mon[93224]: pgmap v286: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:02:54.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:02:53 smithi138 ceph-mon[81376]: pgmap v287: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:02:54.100 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:02:53 smithi107 ceph-mon[93224]: pgmap v287: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:02:56.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:02:55 smithi138 ceph-mon[81376]: pgmap v288: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:02:56.100 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:02:55 smithi107 ceph-mon[93224]: pgmap v288: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:02:58.074 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:02:57 smithi138 ceph-mon[81376]: pgmap v289: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:02:58.100 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:02:57 smithi107 ceph-mon[93224]: pgmap v289: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:02:59.074 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:02:58 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:02:59.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:02:58 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:02:59.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:02:58 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:02:59.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:02:58 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:02:59.100 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:02:58 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:02:59.100 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:02:58 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:02:59.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:02:58 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:02:59.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:02:58 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:03:00.074 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:02:59 smithi138 ceph-mon[81376]: pgmap v290: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:03:00.100 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:02:59 smithi107 ceph-mon[93224]: pgmap v290: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:03:02.074 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:03:01 smithi138 ceph-mon[81376]: pgmap v291: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:03:02.100 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:03:01 smithi107 ceph-mon[93224]: pgmap v291: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:03:04.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:03:03 smithi138 ceph-mon[81376]: pgmap v292: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:03:04.100 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:03:03 smithi107 ceph-mon[93224]: pgmap v292: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:03:05.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:03:04 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T22:03:05.100 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:03:04 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T22:03:06.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:03:05 smithi138 ceph-mon[81376]: pgmap v293: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:03:06.100 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:03:05 smithi107 ceph-mon[93224]: pgmap v293: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:03:08.074 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:03:07 smithi138 ceph-mon[81376]: pgmap v294: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:03:08.100 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:03:07 smithi107 ceph-mon[93224]: pgmap v294: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:03:10.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:03:09 smithi138 ceph-mon[81376]: pgmap v295: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:03:10.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:03:09 smithi107 ceph-mon[93224]: pgmap v295: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:03:12.075 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:03:11 smithi138 ceph-mon[81376]: pgmap v296: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:03:12.100 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:03:11 smithi107 ceph-mon[93224]: pgmap v296: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:03:14.074 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:03:13 smithi138 ceph-mon[81376]: pgmap v297: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:03:14.100 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:03:13 smithi107 ceph-mon[93224]: pgmap v297: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:03:16.100 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:03:15 smithi107 ceph-mon[93224]: pgmap v298: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:03:16.324 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:03:15 smithi138 ceph-mon[81376]: pgmap v298: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:03:18.100 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:03:17 smithi107 ceph-mon[93224]: pgmap v299: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:03:18.324 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:03:17 smithi138 ceph-mon[81376]: pgmap v299: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:03:20.100 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:03:19 smithi107 ceph-mon[93224]: pgmap v300: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:03:20.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:03:19 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T22:03:20.324 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:03:19 smithi138 ceph-mon[81376]: pgmap v300: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:03:20.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:03:19 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T22:03:22.100 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:03:21 smithi107 ceph-mon[93224]: pgmap v301: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:03:22.324 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:03:21 smithi138 ceph-mon[81376]: pgmap v301: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:03:24.100 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:03:23 smithi107 ceph-mon[93224]: pgmap v302: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:03:24.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:03:23 smithi138 ceph-mon[81376]: pgmap v302: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:03:26.100 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:03:25 smithi107 ceph-mon[93224]: pgmap v303: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:03:26.324 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:03:25 smithi138 ceph-mon[81376]: pgmap v303: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:03:28.100 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:03:27 smithi107 ceph-mon[93224]: pgmap v304: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:03:28.324 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:03:27 smithi138 ceph-mon[81376]: pgmap v304: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:03:30.100 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:03:29 smithi107 ceph-mon[93224]: pgmap v305: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:03:30.324 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:03:29 smithi138 ceph-mon[81376]: pgmap v305: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:03:32.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:03:31 smithi107 ceph-mon[93224]: pgmap v306: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:03:32.324 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:03:31 smithi138 ceph-mon[81376]: pgmap v306: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:03:33.631 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -e sha1=931809a9e5d295bd61984693cac44fe131ded4bc -- bash -c 'ceph orch ps' 2024-05-17T22:03:33.896 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/mon.smithi107/config 2024-05-17T22:03:34.019 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:03:33 smithi107 ceph-mon[93224]: pgmap v307: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:03:34.127 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:03:33 smithi138 ceph-mon[81376]: pgmap v307: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:03:34.990 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:03:34 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T22:03:35.324 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:03:34 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T22:03:35.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:35.545+0000 7fc79ed71700 1 -- 172.21.15.107:0/2636135642 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc798101e10 msgr2=0x7fc7981021e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:35.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:35.545+0000 7fc79ed71700 1 --2- 172.21.15.107:0/2636135642 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc798101e10 0x7fc7981021e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc780004940 tx=0x7fc780009e30 comp rx=0 tx=0).stop 2024-05-17T22:03:35.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:35.545+0000 7fc79ed71700 1 -- 172.21.15.107:0/2636135642 shutdown_connections 2024-05-17T22:03:35.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:35.545+0000 7fc79ed71700 1 --2- 172.21.15.107:0/2636135642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7980fc630 0x7fc7980fcaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:35.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:35.545+0000 7fc79ed71700 1 --2- 172.21.15.107:0/2636135642 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc798101e10 0x7fc7981021e0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:35.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:35.545+0000 7fc79ed71700 1 -- 172.21.15.107:0/2636135642 >> 172.21.15.107:0/2636135642 conn(0x7fc7980714d0 msgr2=0x7fc7980718d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:03:35.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:35.545+0000 7fc79ed71700 1 -- 172.21.15.107:0/2636135642 shutdown_connections 2024-05-17T22:03:35.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:35.545+0000 7fc79ed71700 1 -- 172.21.15.107:0/2636135642 wait complete. 2024-05-17T22:03:35.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:35.546+0000 7fc79ed71700 1 Processor -- start 2024-05-17T22:03:35.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:35.546+0000 7fc79ed71700 1 -- start start 2024-05-17T22:03:35.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:35.546+0000 7fc79ed71700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc7980fc630 0x7fc79810eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:35.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:35.546+0000 7fc79ed71700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc798101e10 0x7fc79810f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:35.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:35.546+0000 7fc79ed71700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc79810f5d0 con 0x7fc7980fc630 2024-05-17T22:03:35.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:35.546+0000 7fc79ed71700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc79810f710 con 0x7fc798101e10 2024-05-17T22:03:35.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:35.546+0000 7fc79cb0d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc7980fc630 0x7fc79810eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:35.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:35.546+0000 7fc79cb0d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc7980fc630 0x7fc79810eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48172/0 (socket says 172.21.15.107:48172) 2024-05-17T22:03:35.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:35.546+0000 7fc79cb0d700 1 -- 172.21.15.107:0/3163519928 learned_addr learned my addr 172.21.15.107:0/3163519928 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T22:03:35.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:35.547+0000 7fc797fff700 1 --2- 172.21.15.107:0/3163519928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc798101e10 0x7fc79810f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:35.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:35.547+0000 7fc79cb0d700 1 -- 172.21.15.107:0/3163519928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc798101e10 msgr2=0x7fc79810f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:35.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:35.547+0000 7fc79cb0d700 1 --2- 172.21.15.107:0/3163519928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc798101e10 0x7fc79810f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:35.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:35.547+0000 7fc79cb0d700 1 -- 172.21.15.107:0/3163519928 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc780005040 con 0x7fc7980fc630 2024-05-17T22:03:35.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:35.547+0000 7fc79cb0d700 1 --2- 172.21.15.107:0/3163519928 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc7980fc630 0x7fc79810eb50 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc78000b8b0 tx=0x7fc780009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:35.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:35.547+0000 7fc7957fa700 1 -- 172.21.15.107:0/3163519928 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc78000e040 con 0x7fc7980fc630 2024-05-17T22:03:35.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:35.547+0000 7fc7957fa700 1 -- 172.21.15.107:0/3163519928 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc780004290 con 0x7fc7980fc630 2024-05-17T22:03:35.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:35.547+0000 7fc7957fa700 1 -- 172.21.15.107:0/3163519928 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc78001c3f0 con 0x7fc7980fc630 2024-05-17T22:03:35.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:35.547+0000 7fc79ed71700 1 -- 172.21.15.107:0/3163519928 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc79810f990 con 0x7fc7980fc630 2024-05-17T22:03:35.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:35.547+0000 7fc79ed71700 1 -- 172.21.15.107:0/3163519928 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc798117e40 con 0x7fc7980fc630 2024-05-17T22:03:35.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:35.548+0000 7fc79ed71700 1 -- 172.21.15.107:0/3163519928 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7980ff130 con 0x7fc7980fc630 2024-05-17T22:03:35.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:35.549+0000 7fc7957fa700 1 -- 172.21.15.107:0/3163519928 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7fc7800137b0 con 0x7fc7980fc630 2024-05-17T22:03:35.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:35.549+0000 7fc7957fa700 1 --2- 172.21.15.107:0/3163519928 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fc784078990 0x7fc78407ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:35.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:35.549+0000 7fc7957fa700 1 -- 172.21.15.107:0/3163519928 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7fc78009ba20 con 0x7fc7980fc630 2024-05-17T22:03:35.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:35.549+0000 7fc797fff700 1 --2- 172.21.15.107:0/3163519928 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fc784078990 0x7fc78407ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:35.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:35.550+0000 7fc797fff700 1 --2- 172.21.15.107:0/3163519928 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fc784078990 0x7fc78407ae40 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc7980fbd10 tx=0x7fc788009040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:35.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:35.554+0000 7fc7957fa700 1 -- 172.21.15.107:0/3163519928 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7fc780063700 con 0x7fc7980fc630 2024-05-17T22:03:35.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:35.770+0000 7fc79ed71700 1 -- 172.21.15.107:0/3163519928 --> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fc7980fbac0 con 0x7fc784078990 2024-05-17T22:03:35.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:35.780+0000 7fc7957fa700 1 -- 172.21.15.107:0/3163519928 <== mgr.34100 v2:172.21.15.107:6800/3205268226 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7fc7980fbac0 con 0x7fc784078990 2024-05-17T22:03:35.770 INFO:teuthology.orchestra.run.smithi107.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-17T22:03:35.770 INFO:teuthology.orchestra.run.smithi107.stdout:alertmanager.smithi107 smithi107 *:9093,9094 running (2m) 106s ago 19m 17.2M - 0.25.0 c8568f914cd2 8b363231ac61 2024-05-17T22:03:35.770 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi107 smithi107 running (7m) 106s ago 19m 6669k - 19.0.0-3794-g931809a9 8880f7613d02 face35de205b 2024-05-17T22:03:35.770 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi138 smithi138 running (7m) 2m ago 18m 6665k - 19.0.0-3794-g931809a9 8880f7613d02 ad59e33e6d43 2024-05-17T22:03:35.770 INFO:teuthology.orchestra.run.smithi107.stdout:grafana.smithi107 smithi107 *:3000 running (111s) 106s ago 19m 52.2M - 9.4.12 2bacad6d85d8 1e7c2af290dc 2024-05-17T22:03:35.770 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.drqody smithi107 running (3m) 106s ago 14m 155M - 19.0.0-3794-g931809a9 8880f7613d02 4f02280f8312 2024-05-17T22:03:35.770 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.idmbgj smithi107 running (4m) 106s ago 14m 153M - 19.0.0-3794-g931809a9 8880f7613d02 d541bff38965 2024-05-17T22:03:35.770 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi138.gwbtyd smithi138 running (3m) 2m ago 14m 80.2M - 19.0.0-3794-g931809a9 8880f7613d02 7beb9bd77fc8 2024-05-17T22:03:35.770 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi138.zckrvs smithi138 running (3m) 2m ago 14m 79.2M - 19.0.0-3794-g931809a9 8880f7613d02 17edb3a75f7b 2024-05-17T22:03:35.771 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi107.xhhyiw smithi107 *:8443,9283,8765 running (10m) 106s ago 21m 604M - 19.0.0-3794-g931809a9 8880f7613d02 5acf3e511342 2024-05-17T22:03:35.771 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi138.napnll smithi138 *:8443,9283,8765 running (9m) 2m ago 18m 534M - 19.0.0-3794-g931809a9 8880f7613d02 d627761dbd0d 2024-05-17T22:03:35.771 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi107 smithi107 running (9m) 106s ago 21m 124M 2048M 19.0.0-3794-g931809a9 8880f7613d02 9717f3169f2f 2024-05-17T22:03:35.771 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi138 smithi138 running (8m) 2m ago 18m 107M 2048M 19.0.0-3794-g931809a9 8880f7613d02 55aeed62bf4d 2024-05-17T22:03:35.771 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi107 smithi107 *:9100 running (2m) 106s ago 19m 8841k - 1.5.0 0da6a335fe13 73f1896972ff 2024-05-17T22:03:35.771 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi138 smithi138 *:9100 running (2m) 2m ago 18m 3565k - 1.5.0 0da6a335fe13 05bb3093f9aa 2024-05-17T22:03:35.771 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0 smithi107 running (7m) 106s ago 17m 166M 1557M 19.0.0-3794-g931809a9 8880f7613d02 b319bc93500d 2024-05-17T22:03:35.771 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1 smithi107 running (6m) 106s ago 17m 166M 1557M 19.0.0-3794-g931809a9 8880f7613d02 f94b206be6eb 2024-05-17T22:03:35.771 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2 smithi107 running (6m) 106s ago 17m 154M 1557M 19.0.0-3794-g931809a9 8880f7613d02 add7015647af 2024-05-17T22:03:35.771 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3 smithi138 running (5m) 2m ago 16m 174M 2581M 19.0.0-3794-g931809a9 8880f7613d02 416273d327c7 2024-05-17T22:03:35.771 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4 smithi138 running (5m) 2m ago 16m 159M 2581M 19.0.0-3794-g931809a9 8880f7613d02 e33f411da8e6 2024-05-17T22:03:35.771 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5 smithi138 running (4m) 2m ago 15m 159M 2581M 19.0.0-3794-g931809a9 8880f7613d02 63810d431961 2024-05-17T22:03:35.771 INFO:teuthology.orchestra.run.smithi107.stdout:prometheus.smithi107 smithi107 *:9095 running (2m) 106s ago 19m 51.3M - 2.43.0 a07b618ecd1d aee4bd577d3d 2024-05-17T22:03:35.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:35.782+0000 7fc79ed71700 1 -- 172.21.15.107:0/3163519928 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fc784078990 msgr2=0x7fc78407ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:35.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:35.782+0000 7fc79ed71700 1 --2- 172.21.15.107:0/3163519928 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fc784078990 0x7fc78407ae40 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc7980fbd10 tx=0x7fc788009040 comp rx=0 tx=0).stop 2024-05-17T22:03:35.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:35.782+0000 7fc79ed71700 1 -- 172.21.15.107:0/3163519928 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc7980fc630 msgr2=0x7fc79810eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:35.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:35.782+0000 7fc79ed71700 1 --2- 172.21.15.107:0/3163519928 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc7980fc630 0x7fc79810eb50 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc78000b8b0 tx=0x7fc780009e30 comp rx=0 tx=0).stop 2024-05-17T22:03:35.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:35.782+0000 7fc79ed71700 1 -- 172.21.15.107:0/3163519928 shutdown_connections 2024-05-17T22:03:35.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:35.782+0000 7fc79ed71700 1 --2- 172.21.15.107:0/3163519928 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fc784078990 0x7fc78407ae40 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:35.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:35.782+0000 7fc79ed71700 1 --2- 172.21.15.107:0/3163519928 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc7980fc630 0x7fc79810eb50 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:35.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:35.782+0000 7fc79ed71700 1 --2- 172.21.15.107:0/3163519928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc798101e10 0x7fc79810f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:35.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:35.782+0000 7fc79ed71700 1 -- 172.21.15.107:0/3163519928 >> 172.21.15.107:0/3163519928 conn(0x7fc7980714d0 msgr2=0x7fc7980fa7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:03:35.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:35.782+0000 7fc79ed71700 1 -- 172.21.15.107:0/3163519928 shutdown_connections 2024-05-17T22:03:35.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:35.782+0000 7fc79ed71700 1 -- 172.21.15.107:0/3163519928 wait complete. 2024-05-17T22:03:36.324 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:03:35 smithi138 ceph-mon[81376]: pgmap v308: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:03:36.350 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:03:35 smithi107 ceph-mon[93224]: pgmap v308: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:03:36.407 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -e sha1=931809a9e5d295bd61984693cac44fe131ded4bc -- bash -c 'ceph versions' 2024-05-17T22:03:36.664 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/mon.smithi107/config 2024-05-17T22:03:37.100 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:03:36 smithi107 ceph-mon[93224]: from='client.34582 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:03:37.324 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:03:36 smithi138 ceph-mon[81376]: from='client.34582 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:03:38.033 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:03:37 smithi107 ceph-mon[93224]: pgmap v309: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:03:38.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:38.277+0000 7f4909bd7700 1 -- 172.21.15.107:0/1101270469 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49040fe790 msgr2=0x7f4904106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:38.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:38.277+0000 7f4909bd7700 1 --2- 172.21.15.107:0/1101270469 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49040fe790 0x7f4904106e60 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f48ec004940 tx=0x7f48ec009e30 comp rx=0 tx=0).stop 2024-05-17T22:03:38.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:38.278+0000 7f4909bd7700 1 -- 172.21.15.107:0/1101270469 shutdown_connections 2024-05-17T22:03:38.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:38.278+0000 7f4909bd7700 1 --2- 172.21.15.107:0/1101270469 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49040fe790 0x7f4904106e60 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:38.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:38.278+0000 7f4909bd7700 1 --2- 172.21.15.107:0/1101270469 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f49040fddf0 0x7f49040fe1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:38.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:38.278+0000 7f4909bd7700 1 -- 172.21.15.107:0/1101270469 >> 172.21.15.107:0/1101270469 conn(0x7f49040f7870 msgr2=0x7f49040f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:03:38.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:38.278+0000 7f4909bd7700 1 -- 172.21.15.107:0/1101270469 shutdown_connections 2024-05-17T22:03:38.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:38.278+0000 7f4909bd7700 1 -- 172.21.15.107:0/1101270469 wait complete. 2024-05-17T22:03:38.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:38.278+0000 7f4909bd7700 1 Processor -- start 2024-05-17T22:03:38.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:38.278+0000 7f4909bd7700 1 -- start start 2024-05-17T22:03:38.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:38.278+0000 7f4909bd7700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49040fddf0 0x7f4904113970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:38.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:38.278+0000 7f4909bd7700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f49040fe790 0x7f4904113eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:38.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:38.278+0000 7f4909bd7700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4904114500 con 0x7f49040fddf0 2024-05-17T22:03:38.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:38.278+0000 7f4909bd7700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f490410db20 con 0x7f49040fe790 2024-05-17T22:03:38.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:38.279+0000 7f49037fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49040fddf0 0x7f4904113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:38.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:38.279+0000 7f49037fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49040fddf0 0x7f4904113970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48200/0 (socket says 172.21.15.107:48200) 2024-05-17T22:03:38.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:38.279+0000 7f49037fe700 1 -- 172.21.15.107:0/40777331 learned_addr learned my addr 172.21.15.107:0/40777331 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T22:03:38.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:38.279+0000 7f48fbfff700 1 --2- 172.21.15.107:0/40777331 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f49040fe790 0x7f4904113eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:38.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:38.279+0000 7f49037fe700 1 -- 172.21.15.107:0/40777331 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f49040fe790 msgr2=0x7f4904113eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:38.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:38.279+0000 7f49037fe700 1 --2- 172.21.15.107:0/40777331 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f49040fe790 0x7f4904113eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:38.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:38.279+0000 7f49037fe700 1 -- 172.21.15.107:0/40777331 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48ec005040 con 0x7f49040fddf0 2024-05-17T22:03:38.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:38.279+0000 7f48fbfff700 1 --2- 172.21.15.107:0/40777331 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f49040fe790 0x7f4904113eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:03:38.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:38.279+0000 7f49037fe700 1 --2- 172.21.15.107:0/40777331 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49040fddf0 0x7f4904113970 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f48f4002a50 tx=0x7f48f40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:38.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:38.279+0000 7f4900ff9700 1 -- 172.21.15.107:0/40777331 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f48f4007cc0 con 0x7f49040fddf0 2024-05-17T22:03:38.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:38.279+0000 7f4909bd7700 1 -- 172.21.15.107:0/40777331 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f490410de00 con 0x7f49040fddf0 2024-05-17T22:03:38.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:38.279+0000 7f4900ff9700 1 -- 172.21.15.107:0/40777331 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f48f400f040 con 0x7f49040fddf0 2024-05-17T22:03:38.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:38.279+0000 7f4900ff9700 1 -- 172.21.15.107:0/40777331 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f48f4009640 con 0x7f49040fddf0 2024-05-17T22:03:38.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:38.279+0000 7f4909bd7700 1 -- 172.21.15.107:0/40777331 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f490410e340 con 0x7f49040fddf0 2024-05-17T22:03:38.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:38.280+0000 7f4909bd7700 1 -- 172.21.15.107:0/40777331 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4904066ab0 con 0x7f49040fddf0 2024-05-17T22:03:38.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:38.281+0000 7f4900ff9700 1 -- 172.21.15.107:0/40777331 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f48f40097a0 con 0x7f49040fddf0 2024-05-17T22:03:38.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:38.281+0000 7f4900ff9700 1 --2- 172.21.15.107:0/40777331 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f48f0078990 0x7f48f007ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:38.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:38.281+0000 7f4900ff9700 1 -- 172.21.15.107:0/40777331 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f48f409a4b0 con 0x7f49040fddf0 2024-05-17T22:03:38.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:38.281+0000 7f48fbfff700 1 --2- 172.21.15.107:0/40777331 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f48f0078990 0x7f48f007ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:38.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:38.282+0000 7f48fbfff700 1 --2- 172.21.15.107:0/40777331 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f48f0078990 0x7f48f007ae40 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f48ec00b3b0 tx=0x7f48ec009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:38.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:38.286+0000 7f4900ff9700 1 -- 172.21.15.107:0/40777331 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f48f40620f0 con 0x7f49040fddf0 2024-05-17T22:03:38.324 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:03:37 smithi138 ceph-mon[81376]: pgmap v309: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:03:38.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:38.574+0000 7f4909bd7700 1 -- 172.21.15.107:0/40777331 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f490410efc0 con 0x7f49040fddf0 2024-05-17T22:03:38.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:38.575+0000 7f4900ff9700 1 -- 172.21.15.107:0/40777331 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f48f4061a90 con 0x7f49040fddf0 2024-05-17T22:03:38.565 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-05-17T22:03:38.565 INFO:teuthology.orchestra.run.smithi107.stdout: "mon": { 2024-05-17T22:03:38.565 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev)": 2 2024-05-17T22:03:38.565 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-05-17T22:03:38.565 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr": { 2024-05-17T22:03:38.565 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev)": 2 2024-05-17T22:03:38.565 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-05-17T22:03:38.566 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": { 2024-05-17T22:03:38.566 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev)": 6 2024-05-17T22:03:38.566 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-05-17T22:03:38.566 INFO:teuthology.orchestra.run.smithi107.stdout: "mds": { 2024-05-17T22:03:38.566 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev)": 4 2024-05-17T22:03:38.566 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-05-17T22:03:38.566 INFO:teuthology.orchestra.run.smithi107.stdout: "overall": { 2024-05-17T22:03:38.566 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev)": 14 2024-05-17T22:03:38.566 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-05-17T22:03:38.566 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-05-17T22:03:38.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:38.577+0000 7f4909bd7700 1 -- 172.21.15.107:0/40777331 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f48f0078990 msgr2=0x7f48f007ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:38.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:38.577+0000 7f4909bd7700 1 --2- 172.21.15.107:0/40777331 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f48f0078990 0x7f48f007ae40 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f48ec00b3b0 tx=0x7f48ec009e30 comp rx=0 tx=0).stop 2024-05-17T22:03:38.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:38.577+0000 7f4909bd7700 1 -- 172.21.15.107:0/40777331 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49040fddf0 msgr2=0x7f4904113970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:38.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:38.577+0000 7f4909bd7700 1 --2- 172.21.15.107:0/40777331 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49040fddf0 0x7f4904113970 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f48f4002a50 tx=0x7f48f40046b0 comp rx=0 tx=0).stop 2024-05-17T22:03:38.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:38.577+0000 7f4909bd7700 1 -- 172.21.15.107:0/40777331 shutdown_connections 2024-05-17T22:03:38.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:38.577+0000 7f4909bd7700 1 --2- 172.21.15.107:0/40777331 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f48f0078990 0x7f48f007ae40 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:38.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:38.577+0000 7f4909bd7700 1 --2- 172.21.15.107:0/40777331 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49040fddf0 0x7f4904113970 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:38.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:38.577+0000 7f4909bd7700 1 --2- 172.21.15.107:0/40777331 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f49040fe790 0x7f4904113eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:38.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:38.577+0000 7f4909bd7700 1 -- 172.21.15.107:0/40777331 >> 172.21.15.107:0/40777331 conn(0x7f49040f7870 msgr2=0x7f49040f8d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:03:38.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:38.577+0000 7f4909bd7700 1 -- 172.21.15.107:0/40777331 shutdown_connections 2024-05-17T22:03:38.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:38.577+0000 7f4909bd7700 1 -- 172.21.15.107:0/40777331 wait complete. 2024-05-17T22:03:39.214 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:03:38 smithi107 ceph-mon[93224]: from='client.? 172.21.15.107:0/40777331' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:03:39.239 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -e sha1=931809a9e5d295bd61984693cac44fe131ded4bc -- bash -c 'ceph versions | jq -e '"'"'.overall | length == 1'"'"'' 2024-05-17T22:03:39.324 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:03:38 smithi138 ceph-mon[81376]: from='client.? 172.21.15.107:0/40777331' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:03:39.501 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/mon.smithi107/config 2024-05-17T22:03:40.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:03:39 smithi107 ceph-mon[93224]: pgmap v310: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:03:40.324 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:03:39 smithi138 ceph-mon[81376]: pgmap v310: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:03:41.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:41.229+0000 7f0567631700 1 -- 172.21.15.107:0/1117253049 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0560101d00 msgr2=0x7f05601020d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:41.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:41.229+0000 7f0567631700 1 --2- 172.21.15.107:0/1117253049 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0560101d00 0x7f05601020d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f055c004940 tx=0x7f055c009e30 comp rx=0 tx=0).stop 2024-05-17T22:03:41.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:41.230+0000 7f0567631700 1 -- 172.21.15.107:0/1117253049 shutdown_connections 2024-05-17T22:03:41.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:41.230+0000 7f0567631700 1 --2- 172.21.15.107:0/1117253049 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05601026a0 0x7f05600fbd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:41.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:41.230+0000 7f0567631700 1 --2- 172.21.15.107:0/1117253049 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0560101d00 0x7f05601020d0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:41.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:41.230+0000 7f0567631700 1 -- 172.21.15.107:0/1117253049 >> 172.21.15.107:0/1117253049 conn(0x7f05600f7870 msgr2=0x7f05600f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:03:41.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:41.230+0000 7f0567631700 1 -- 172.21.15.107:0/1117253049 shutdown_connections 2024-05-17T22:03:41.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:41.230+0000 7f0567631700 1 -- 172.21.15.107:0/1117253049 wait complete. 2024-05-17T22:03:41.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:41.230+0000 7f0567631700 1 Processor -- start 2024-05-17T22:03:41.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:41.230+0000 7f0567631700 1 -- start start 2024-05-17T22:03:41.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:41.231+0000 7f0567631700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0560101d00 0x7f056010c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:41.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:41.231+0000 7f0567631700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05601026a0 0x7f056010ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:41.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:41.231+0000 7f0567631700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0560067190 con 0x7f05601026a0 2024-05-17T22:03:41.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:41.231+0000 7f0567631700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f056010d390 con 0x7f0560101d00 2024-05-17T22:03:41.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:41.231+0000 7f0564bcc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05601026a0 0x7f056010ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:41.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:41.231+0000 7f0564bcc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05601026a0 0x7f056010ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48216/0 (socket says 172.21.15.107:48216) 2024-05-17T22:03:41.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:41.231+0000 7f0564bcc700 1 -- 172.21.15.107:0/2711370649 learned_addr learned my addr 172.21.15.107:0/2711370649 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T22:03:41.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:41.231+0000 7f05653cd700 1 --2- 172.21.15.107:0/2711370649 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0560101d00 0x7f056010c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:41.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:41.231+0000 7f0564bcc700 1 -- 172.21.15.107:0/2711370649 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0560101d00 msgr2=0x7f056010c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:41.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:41.231+0000 7f0564bcc700 1 --2- 172.21.15.107:0/2711370649 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0560101d00 0x7f056010c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:41.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:41.231+0000 7f0564bcc700 1 -- 172.21.15.107:0/2711370649 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f055c005040 con 0x7f05601026a0 2024-05-17T22:03:41.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:41.231+0000 7f0564bcc700 1 --2- 172.21.15.107:0/2711370649 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05601026a0 0x7f056010ce50 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f055000b790 tx=0x7f0550009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:41.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:41.231+0000 7f0555ffb700 1 -- 172.21.15.107:0/2711370649 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f055000f6f0 con 0x7f05601026a0 2024-05-17T22:03:41.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:41.231+0000 7f0555ffb700 1 -- 172.21.15.107:0/2711370649 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f055000fd20 con 0x7f05601026a0 2024-05-17T22:03:41.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:41.231+0000 7f0567631700 1 -- 172.21.15.107:0/2711370649 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f056010d610 con 0x7f05601026a0 2024-05-17T22:03:41.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:41.231+0000 7f0555ffb700 1 -- 172.21.15.107:0/2711370649 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f05500103a0 con 0x7f05601026a0 2024-05-17T22:03:41.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:41.232+0000 7f0567631700 1 -- 172.21.15.107:0/2711370649 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f056011a210 con 0x7f05601026a0 2024-05-17T22:03:41.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:41.233+0000 7f0567631700 1 -- 172.21.15.107:0/2711370649 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05600fd450 con 0x7f05601026a0 2024-05-17T22:03:41.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:41.233+0000 7f0555ffb700 1 -- 172.21.15.107:0/2711370649 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f0550011400 con 0x7f05601026a0 2024-05-17T22:03:41.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:41.234+0000 7f0555ffb700 1 --2- 172.21.15.107:0/2711370649 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f054c078990 0x7f054c07ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:41.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:41.234+0000 7f0555ffb700 1 -- 172.21.15.107:0/2711370649 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f055009a550 con 0x7f05601026a0 2024-05-17T22:03:41.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:41.234+0000 7f05653cd700 1 --2- 172.21.15.107:0/2711370649 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f054c078990 0x7f054c07ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:41.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:41.234+0000 7f05653cd700 1 --2- 172.21.15.107:0/2711370649 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f054c078990 0x7f054c07ae40 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f055c004f90 tx=0x7f055c009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:41.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:41.239+0000 7f0555ffb700 1 -- 172.21.15.107:0/2711370649 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f0550062190 con 0x7f05601026a0 2024-05-17T22:03:41.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:41.533+0000 7f0567631700 1 -- 172.21.15.107:0/2711370649 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f056004c710 con 0x7f05601026a0 2024-05-17T22:03:41.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:41.534+0000 7f0555ffb700 1 -- 172.21.15.107:0/2711370649 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f0550061b30 con 0x7f05601026a0 2024-05-17T22:03:41.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:41.536+0000 7f0567631700 1 -- 172.21.15.107:0/2711370649 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f054c078990 msgr2=0x7f054c07ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:41.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:41.536+0000 7f0567631700 1 --2- 172.21.15.107:0/2711370649 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f054c078990 0x7f054c07ae40 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f055c004f90 tx=0x7f055c009e30 comp rx=0 tx=0).stop 2024-05-17T22:03:41.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:41.536+0000 7f0567631700 1 -- 172.21.15.107:0/2711370649 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05601026a0 msgr2=0x7f056010ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:41.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:41.536+0000 7f0567631700 1 --2- 172.21.15.107:0/2711370649 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05601026a0 0x7f056010ce50 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f055000b790 tx=0x7f0550009220 comp rx=0 tx=0).stop 2024-05-17T22:03:41.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:41.536+0000 7f0567631700 1 -- 172.21.15.107:0/2711370649 shutdown_connections 2024-05-17T22:03:41.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:41.536+0000 7f0567631700 1 --2- 172.21.15.107:0/2711370649 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f054c078990 0x7f054c07ae40 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:41.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:41.536+0000 7f0567631700 1 --2- 172.21.15.107:0/2711370649 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0560101d00 0x7f056010c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:41.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:41.536+0000 7f0567631700 1 --2- 172.21.15.107:0/2711370649 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05601026a0 0x7f056010ce50 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:41.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:41.536+0000 7f0567631700 1 -- 172.21.15.107:0/2711370649 >> 172.21.15.107:0/2711370649 conn(0x7f05600f7870 msgr2=0x7f05600f8800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:03:41.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:41.536+0000 7f0567631700 1 -- 172.21.15.107:0/2711370649 shutdown_connections 2024-05-17T22:03:41.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:41.536+0000 7f0567631700 1 -- 172.21.15.107:0/2711370649 wait complete. 2024-05-17T22:03:41.536 INFO:teuthology.orchestra.run.smithi107.stdout:true 2024-05-17T22:03:42.100 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:03:41 smithi107 ceph-mon[93224]: pgmap v311: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:03:42.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:03:41 smithi107 ceph-mon[93224]: from='client.? 172.21.15.107:0/2711370649' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:03:42.131 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -e sha1=931809a9e5d295bd61984693cac44fe131ded4bc -- bash -c 'ceph versions | jq -e '"'"'.overall | keys'"'"' | grep $sha1' 2024-05-17T22:03:42.324 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:03:41 smithi138 ceph-mon[81376]: pgmap v311: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:03:42.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:03:41 smithi138 ceph-mon[81376]: from='client.? 172.21.15.107:0/2711370649' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:03:42.387 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/mon.smithi107/config 2024-05-17T22:03:44.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:44.050+0000 7fa277996700 1 -- 172.21.15.107:0/4014861853 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa270102de0 msgr2=0x7fa2701031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:44.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:44.050+0000 7fa277996700 1 --2- 172.21.15.107:0/4014861853 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa270102de0 0x7fa2701031b0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa26c004940 tx=0x7fa26c009e30 comp rx=0 tx=0).stop 2024-05-17T22:03:44.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:44.050+0000 7fa277996700 1 -- 172.21.15.107:0/4014861853 shutdown_connections 2024-05-17T22:03:44.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:44.050+0000 7fa277996700 1 --2- 172.21.15.107:0/4014861853 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa270103780 0x7fa2700fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:44.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:44.050+0000 7fa277996700 1 --2- 172.21.15.107:0/4014861853 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa270102de0 0x7fa2701031b0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:44.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:44.050+0000 7fa277996700 1 -- 172.21.15.107:0/4014861853 >> 172.21.15.107:0/4014861853 conn(0x7fa2700f8990 msgr2=0x7fa2700fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:03:44.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:44.051+0000 7fa277996700 1 -- 172.21.15.107:0/4014861853 shutdown_connections 2024-05-17T22:03:44.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:44.051+0000 7fa277996700 1 -- 172.21.15.107:0/4014861853 wait complete. 2024-05-17T22:03:44.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:44.051+0000 7fa277996700 1 Processor -- start 2024-05-17T22:03:44.043 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:03:43 smithi107 ceph-mon[93224]: pgmap v312: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:03:44.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:44.051+0000 7fa277996700 1 -- start start 2024-05-17T22:03:44.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:44.052+0000 7fa277996700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa270102de0 0x7fa270113970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:44.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:44.052+0000 7fa277996700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa270103780 0x7fa270113eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:44.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:44.052+0000 7fa277996700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa270114500 con 0x7fa270102de0 2024-05-17T22:03:44.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:44.052+0000 7fa277996700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa27010db50 con 0x7fa270103780 2024-05-17T22:03:44.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:44.052+0000 7fa275732700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa270102de0 0x7fa270113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:44.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:44.052+0000 7fa275732700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa270102de0 0x7fa270113970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48226/0 (socket says 172.21.15.107:48226) 2024-05-17T22:03:44.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:44.052+0000 7fa275732700 1 -- 172.21.15.107:0/3587442952 learned_addr learned my addr 172.21.15.107:0/3587442952 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T22:03:44.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:44.052+0000 7fa274f31700 1 --2- 172.21.15.107:0/3587442952 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa270103780 0x7fa270113eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:44.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:44.052+0000 7fa275732700 1 -- 172.21.15.107:0/3587442952 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa270103780 msgr2=0x7fa270113eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:44.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:44.052+0000 7fa275732700 1 --2- 172.21.15.107:0/3587442952 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa270103780 0x7fa270113eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:44.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:44.052+0000 7fa275732700 1 -- 172.21.15.107:0/3587442952 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa26c005040 con 0x7fa270102de0 2024-05-17T22:03:44.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:44.052+0000 7fa274f31700 1 --2- 172.21.15.107:0/3587442952 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa270103780 0x7fa270113eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:03:44.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:44.052+0000 7fa275732700 1 --2- 172.21.15.107:0/3587442952 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa270102de0 0x7fa270113970 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fa26c00b3b0 tx=0x7fa26c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:44.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:44.053+0000 7fa265ffb700 1 -- 172.21.15.107:0/3587442952 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa26c00e040 con 0x7fa270102de0 2024-05-17T22:03:44.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:44.053+0000 7fa277996700 1 -- 172.21.15.107:0/3587442952 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa27010ddd0 con 0x7fa270102de0 2024-05-17T22:03:44.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:44.053+0000 7fa265ffb700 1 -- 172.21.15.107:0/3587442952 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa26c00bbc0 con 0x7fa270102de0 2024-05-17T22:03:44.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:44.053+0000 7fa265ffb700 1 -- 172.21.15.107:0/3587442952 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa26c01c3f0 con 0x7fa270102de0 2024-05-17T22:03:44.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:44.053+0000 7fa277996700 1 -- 172.21.15.107:0/3587442952 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa27010e390 con 0x7fa270102de0 2024-05-17T22:03:44.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:44.054+0000 7fa277996700 1 -- 172.21.15.107:0/3587442952 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2701000f0 con 0x7fa270102de0 2024-05-17T22:03:44.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:44.055+0000 7fa265ffb700 1 -- 172.21.15.107:0/3587442952 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7fa26c00bd30 con 0x7fa270102de0 2024-05-17T22:03:44.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:44.055+0000 7fa265ffb700 1 --2- 172.21.15.107:0/3587442952 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fa25c078990 0x7fa25c07ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:44.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:44.055+0000 7fa265ffb700 1 -- 172.21.15.107:0/3587442952 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7fa26c09c400 con 0x7fa270102de0 2024-05-17T22:03:44.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:44.055+0000 7fa274f31700 1 --2- 172.21.15.107:0/3587442952 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fa25c078990 0x7fa25c07ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:44.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:44.056+0000 7fa274f31700 1 --2- 172.21.15.107:0/3587442952 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fa25c078990 0x7fa25c07ae40 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa27010ecc0 tx=0x7fa260009040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:44.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:44.059+0000 7fa265ffb700 1 -- 172.21.15.107:0/3587442952 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7fa26c0640e0 con 0x7fa270102de0 2024-05-17T22:03:44.324 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:03:43 smithi138 ceph-mon[81376]: pgmap v312: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:03:44.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:44.349+0000 7fa277996700 1 -- 172.21.15.107:0/3587442952 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fa27010efc0 con 0x7fa270102de0 2024-05-17T22:03:44.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:44.349+0000 7fa265ffb700 1 -- 172.21.15.107:0/3587442952 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fa26c063a80 con 0x7fa270102de0 2024-05-17T22:03:44.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:44.351+0000 7fa277996700 1 -- 172.21.15.107:0/3587442952 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fa25c078990 msgr2=0x7fa25c07ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:44.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:44.351+0000 7fa277996700 1 --2- 172.21.15.107:0/3587442952 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fa25c078990 0x7fa25c07ae40 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa27010ecc0 tx=0x7fa260009040 comp rx=0 tx=0).stop 2024-05-17T22:03:44.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:44.352+0000 7fa277996700 1 -- 172.21.15.107:0/3587442952 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa270102de0 msgr2=0x7fa270113970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:44.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:44.352+0000 7fa277996700 1 --2- 172.21.15.107:0/3587442952 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa270102de0 0x7fa270113970 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fa26c00b3b0 tx=0x7fa26c009e30 comp rx=0 tx=0).stop 2024-05-17T22:03:44.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:44.352+0000 7fa277996700 1 -- 172.21.15.107:0/3587442952 shutdown_connections 2024-05-17T22:03:44.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:44.352+0000 7fa277996700 1 --2- 172.21.15.107:0/3587442952 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fa25c078990 0x7fa25c07ae40 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:44.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:44.352+0000 7fa277996700 1 --2- 172.21.15.107:0/3587442952 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa270102de0 0x7fa270113970 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:44.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:44.352+0000 7fa277996700 1 --2- 172.21.15.107:0/3587442952 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa270103780 0x7fa270113eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:44.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:44.352+0000 7fa277996700 1 -- 172.21.15.107:0/3587442952 >> 172.21.15.107:0/3587442952 conn(0x7fa2700f8990 msgr2=0x7fa270107380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:03:44.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:44.352+0000 7fa277996700 1 -- 172.21.15.107:0/3587442952 shutdown_connections 2024-05-17T22:03:44.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:44.352+0000 7fa277996700 1 -- 172.21.15.107:0/3587442952 wait complete. 2024-05-17T22:03:44.352 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev)" 2024-05-17T22:03:44.979 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:03:44 smithi107 ceph-mon[93224]: from='client.? 172.21.15.107:0/3587442952' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:03:45.006 DEBUG:teuthology.parallel:result is None 2024-05-17T22:03:45.006 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-17T22:03:45.017 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi107.front.sepia.ceph.com 2024-05-17T22:03:45.017 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -- bash -c 'ceph fs dump' 2024-05-17T22:03:45.274 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/mon.smithi107/config 2024-05-17T22:03:45.324 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:03:44 smithi138 ceph-mon[81376]: from='client.? 172.21.15.107:0/3587442952' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-17T22:03:46.100 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:03:45 smithi107 ceph-mon[93224]: pgmap v313: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:03:46.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:03:45 smithi138 ceph-mon[81376]: pgmap v313: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:03:46.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:46.892+0000 7f576efed700 1 -- 172.21.15.107:0/586394412 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5768103780 msgr2=0x7f57680fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:46.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:46.892+0000 7f576efed700 1 --2- 172.21.15.107:0/586394412 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5768103780 0x7f57680fcdf0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5754004940 tx=0x7f5754009e30 comp rx=0 tx=0).stop 2024-05-17T22:03:46.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:46.892+0000 7f576efed700 1 -- 172.21.15.107:0/586394412 shutdown_connections 2024-05-17T22:03:46.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:46.892+0000 7f576efed700 1 --2- 172.21.15.107:0/586394412 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5768103780 0x7f57680fcdf0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:46.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:46.892+0000 7f576efed700 1 --2- 172.21.15.107:0/586394412 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5768102de0 0x7f57681031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:46.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:46.892+0000 7f576efed700 1 -- 172.21.15.107:0/586394412 >> 172.21.15.107:0/586394412 conn(0x7f57680f8990 msgr2=0x7f57680fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:03:46.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:46.892+0000 7f576efed700 1 -- 172.21.15.107:0/586394412 shutdown_connections 2024-05-17T22:03:46.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:46.892+0000 7f576efed700 1 -- 172.21.15.107:0/586394412 wait complete. 2024-05-17T22:03:46.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:46.892+0000 7f576efed700 1 Processor -- start 2024-05-17T22:03:46.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:46.892+0000 7f576efed700 1 -- start start 2024-05-17T22:03:46.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:46.893+0000 7f576efed700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5768102de0 0x7f5768113990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:46.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:46.893+0000 7f576efed700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5768103780 0x7f5768113ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:46.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:46.893+0000 7f576efed700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5768114520 con 0x7f5768102de0 2024-05-17T22:03:46.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:46.893+0000 7f576efed700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f576810db20 con 0x7f5768103780 2024-05-17T22:03:46.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:46.893+0000 7f576cd89700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5768102de0 0x7f5768113990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:46.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:46.893+0000 7f576cd89700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5768102de0 0x7f5768113990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48426/0 (socket says 172.21.15.107:48426) 2024-05-17T22:03:46.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:46.893+0000 7f576cd89700 1 -- 172.21.15.107:0/73342048 learned_addr learned my addr 172.21.15.107:0/73342048 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T22:03:46.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:46.893+0000 7f5767fff700 1 --2- 172.21.15.107:0/73342048 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5768103780 0x7f5768113ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:46.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:46.893+0000 7f576cd89700 1 -- 172.21.15.107:0/73342048 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5768103780 msgr2=0x7f5768113ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:46.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:46.893+0000 7f576cd89700 1 --2- 172.21.15.107:0/73342048 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5768103780 0x7f5768113ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:46.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:46.893+0000 7f576cd89700 1 -- 172.21.15.107:0/73342048 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5754005040 con 0x7f5768102de0 2024-05-17T22:03:46.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:46.893+0000 7f576cd89700 1 --2- 172.21.15.107:0/73342048 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5768102de0 0x7f5768113990 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f575c00c830 tx=0x7f575c004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:46.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:46.894+0000 7f57657fa700 1 -- 172.21.15.107:0/73342048 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f575c00a760 con 0x7f5768102de0 2024-05-17T22:03:46.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:46.894+0000 7f57657fa700 1 -- 172.21.15.107:0/73342048 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f575c00ad90 con 0x7f5768102de0 2024-05-17T22:03:46.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:46.894+0000 7f576efed700 1 -- 172.21.15.107:0/73342048 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f576810de00 con 0x7f5768102de0 2024-05-17T22:03:46.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:46.894+0000 7f57657fa700 1 -- 172.21.15.107:0/73342048 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f575c0113a0 con 0x7f5768102de0 2024-05-17T22:03:46.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:46.894+0000 7f576efed700 1 -- 172.21.15.107:0/73342048 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f576810e390 con 0x7f5768102de0 2024-05-17T22:03:46.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:46.895+0000 7f576efed700 1 -- 172.21.15.107:0/73342048 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57680fe530 con 0x7f5768102de0 2024-05-17T22:03:46.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:46.896+0000 7f57657fa700 1 -- 172.21.15.107:0/73342048 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f575c012400 con 0x7f5768102de0 2024-05-17T22:03:46.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:46.896+0000 7f57657fa700 1 --2- 172.21.15.107:0/73342048 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f5758078990 0x7f575807ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:46.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:46.896+0000 7f57657fa700 1 -- 172.21.15.107:0/73342048 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f575c09b250 con 0x7f5768102de0 2024-05-17T22:03:46.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:46.896+0000 7f5767fff700 1 --2- 172.21.15.107:0/73342048 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f5758078990 0x7f575807ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:46.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:46.897+0000 7f5767fff700 1 --2- 172.21.15.107:0/73342048 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f5758078990 0x7f575807ae40 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f576810ee00 tx=0x7f5754009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:46.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:46.900+0000 7f57657fa700 1 -- 172.21.15.107:0/73342048 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f575c062f30 con 0x7f5768102de0 2024-05-17T22:03:47.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:47.148+0000 7f576efed700 1 -- 172.21.15.107:0/73342048 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f5768109ef0 con 0x7f5768102de0 2024-05-17T22:03:47.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:47.148+0000 7f57657fa700 1 -- 172.21.15.107:0/73342048 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 35 v35) v1 ==== 76+0+1968 (secure 0 0 0) 0x7f575c0628d0 con 0x7f5768102de0 2024-05-17T22:03:47.140 INFO:teuthology.orchestra.run.smithi107.stdout:e35 2024-05-17T22:03:47.140 INFO:teuthology.orchestra.run.smithi107.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-17T22:03:47.140 INFO:teuthology.orchestra.run.smithi107.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-17T22:03:47.140 INFO:teuthology.orchestra.run.smithi107.stdout:legacy client fscid: 1 2024-05-17T22:03:47.140 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T22:03:47.140 INFO:teuthology.orchestra.run.smithi107.stdout:Filesystem 'cephfs' (1) 2024-05-17T22:03:47.140 INFO:teuthology.orchestra.run.smithi107.stdout:fs_name cephfs 2024-05-17T22:03:47.140 INFO:teuthology.orchestra.run.smithi107.stdout:epoch 35 2024-05-17T22:03:47.140 INFO:teuthology.orchestra.run.smithi107.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-17T22:03:47.140 INFO:teuthology.orchestra.run.smithi107.stdout:created 2024-05-17T21:49:16.836333+0000 2024-05-17T22:03:47.140 INFO:teuthology.orchestra.run.smithi107.stdout:modified 2024-05-17T22:00:33.850863+0000 2024-05-17T22:03:47.141 INFO:teuthology.orchestra.run.smithi107.stdout:tableserver 0 2024-05-17T22:03:47.141 INFO:teuthology.orchestra.run.smithi107.stdout:root 0 2024-05-17T22:03:47.141 INFO:teuthology.orchestra.run.smithi107.stdout:session_timeout 60 2024-05-17T22:03:47.141 INFO:teuthology.orchestra.run.smithi107.stdout:session_autoclose 300 2024-05-17T22:03:47.141 INFO:teuthology.orchestra.run.smithi107.stdout:max_file_size 1099511627776 2024-05-17T22:03:47.141 INFO:teuthology.orchestra.run.smithi107.stdout:max_xattr_size 65536 2024-05-17T22:03:47.141 INFO:teuthology.orchestra.run.smithi107.stdout:required_client_features {} 2024-05-17T22:03:47.141 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure 0 2024-05-17T22:03:47.141 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure_osd_epoch 84 2024-05-17T22:03:47.141 INFO:teuthology.orchestra.run.smithi107.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-17T22:03:47.141 INFO:teuthology.orchestra.run.smithi107.stdout:max_mds 1 2024-05-17T22:03:47.141 INFO:teuthology.orchestra.run.smithi107.stdout:in 0 2024-05-17T22:03:47.142 INFO:teuthology.orchestra.run.smithi107.stdout:up {0=34408} 2024-05-17T22:03:47.142 INFO:teuthology.orchestra.run.smithi107.stdout:failed 2024-05-17T22:03:47.142 INFO:teuthology.orchestra.run.smithi107.stdout:damaged 2024-05-17T22:03:47.142 INFO:teuthology.orchestra.run.smithi107.stdout:stopped 2024-05-17T22:03:47.142 INFO:teuthology.orchestra.run.smithi107.stdout:data_pools [3] 2024-05-17T22:03:47.142 INFO:teuthology.orchestra.run.smithi107.stdout:metadata_pool 2 2024-05-17T22:03:47.142 INFO:teuthology.orchestra.run.smithi107.stdout:inline_data enabled 2024-05-17T22:03:47.142 INFO:teuthology.orchestra.run.smithi107.stdout:balancer 2024-05-17T22:03:47.142 INFO:teuthology.orchestra.run.smithi107.stdout:bal_rank_mask -1 2024-05-17T22:03:47.142 INFO:teuthology.orchestra.run.smithi107.stdout:standby_count_wanted 1 2024-05-17T22:03:47.142 INFO:teuthology.orchestra.run.smithi107.stdout:qdb_cluster leader: 34408 members: 34408 2024-05-17T22:03:47.142 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.idmbgj{0:34408} state up:active seq 20 join_fscid=1 addr [v2:172.21.15.107:6826/2235359521,v1:172.21.15.107:6827/2235359521] compat {c=[1],r=[1],i=[1fff]}] 2024-05-17T22:03:47.143 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.drqody{0:34440} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.107:6828/369507752,v1:172.21.15.107:6829/369507752] compat {c=[1],r=[1],i=[1fff]}] 2024-05-17T22:03:47.143 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T22:03:47.143 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T22:03:47.143 INFO:teuthology.orchestra.run.smithi107.stdout:Standby daemons: 2024-05-17T22:03:47.143 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T22:03:47.143 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi138.gwbtyd{-1:44335} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6824/1700112281,v1:172.21.15.138:6825/1700112281] compat {c=[1],r=[1],i=[1fff]}] 2024-05-17T22:03:47.143 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi138.zckrvs{-1:44365} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6826/1801131392,v1:172.21.15.138:6827/1801131392] compat {c=[1],r=[1],i=[1fff]}] 2024-05-17T22:03:47.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:47.151+0000 7f576efed700 1 -- 172.21.15.107:0/73342048 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f5758078990 msgr2=0x7f575807ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:47.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:47.151+0000 7f576efed700 1 --2- 172.21.15.107:0/73342048 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f5758078990 0x7f575807ae40 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f576810ee00 tx=0x7f5754009e30 comp rx=0 tx=0).stop 2024-05-17T22:03:47.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:47.151+0000 7f576efed700 1 -- 172.21.15.107:0/73342048 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5768102de0 msgr2=0x7f5768113990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:47.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:47.151+0000 7f576efed700 1 --2- 172.21.15.107:0/73342048 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5768102de0 0x7f5768113990 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f575c00c830 tx=0x7f575c004470 comp rx=0 tx=0).stop 2024-05-17T22:03:47.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:47.151+0000 7f576efed700 1 -- 172.21.15.107:0/73342048 shutdown_connections 2024-05-17T22:03:47.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:47.151+0000 7f576efed700 1 --2- 172.21.15.107:0/73342048 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f5758078990 0x7f575807ae40 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:47.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:47.151+0000 7f576efed700 1 --2- 172.21.15.107:0/73342048 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5768102de0 0x7f5768113990 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:47.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:47.151+0000 7f576efed700 1 --2- 172.21.15.107:0/73342048 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5768103780 0x7f5768113ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:47.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:47.151+0000 7f576efed700 1 -- 172.21.15.107:0/73342048 >> 172.21.15.107:0/73342048 conn(0x7f57680f8990 msgr2=0x7f57680f9970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:03:47.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:47.151+0000 7f576efed700 1 -- 172.21.15.107:0/73342048 shutdown_connections 2024-05-17T22:03:47.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:47.151+0000 7f576efed700 1 -- 172.21.15.107:0/73342048 wait complete. 2024-05-17T22:03:47.144 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 35 2024-05-17T22:03:47.730 INFO:teuthology.run_tasks:Running task fs.post_upgrade_checks... 2024-05-17T22:03:47.741 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T22:03:47.999 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/mon.smithi107/config 2024-05-17T22:03:48.220 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:03:47 smithi107 ceph-mon[93224]: pgmap v314: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:03:48.220 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:03:47 smithi107 ceph-mon[93224]: from='client.? 172.21.15.107:0/73342048' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-17T22:03:48.324 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:03:47 smithi138 ceph-mon[81376]: pgmap v314: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:03:48.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:03:47 smithi138 ceph-mon[81376]: from='client.? 172.21.15.107:0/73342048' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-17T22:03:49.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:49.490+0000 7fe57dd32700 1 -- 172.21.15.107:0/1698524646 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe578102de0 msgr2=0x7fe5781031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:49.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:49.490+0000 7fe57dd32700 1 --2- 172.21.15.107:0/1698524646 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe578102de0 0x7fe5781031b0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe568004940 tx=0x7fe568009e30 comp rx=0 tx=0).stop 2024-05-17T22:03:49.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:49.490+0000 7fe57dd32700 1 -- 172.21.15.107:0/1698524646 shutdown_connections 2024-05-17T22:03:49.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:49.490+0000 7fe57dd32700 1 --2- 172.21.15.107:0/1698524646 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe578103780 0x7fe5780fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:49.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:49.490+0000 7fe57dd32700 1 --2- 172.21.15.107:0/1698524646 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe578102de0 0x7fe5781031b0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:49.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:49.490+0000 7fe57dd32700 1 -- 172.21.15.107:0/1698524646 >> 172.21.15.107:0/1698524646 conn(0x7fe5780f8990 msgr2=0x7fe5780fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:03:49.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:49.490+0000 7fe57dd32700 1 -- 172.21.15.107:0/1698524646 shutdown_connections 2024-05-17T22:03:49.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:49.490+0000 7fe57dd32700 1 -- 172.21.15.107:0/1698524646 wait complete. 2024-05-17T22:03:49.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:49.491+0000 7fe57dd32700 1 Processor -- start 2024-05-17T22:03:49.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:49.491+0000 7fe57dd32700 1 -- start start 2024-05-17T22:03:49.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:49.491+0000 7fe57dd32700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe578102de0 0x7fe578113930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:49.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:49.491+0000 7fe57dd32700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe578103780 0x7fe578113e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:49.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:49.491+0000 7fe57dd32700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe578114550 con 0x7fe578102de0 2024-05-17T22:03:49.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:49.491+0000 7fe57dd32700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe57810dac0 con 0x7fe578103780 2024-05-17T22:03:49.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:49.491+0000 7fe5777fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe578102de0 0x7fe578113930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:49.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:49.491+0000 7fe5777fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe578102de0 0x7fe578113930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48440/0 (socket says 172.21.15.107:48440) 2024-05-17T22:03:49.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:49.491+0000 7fe5777fe700 1 -- 172.21.15.107:0/2427126369 learned_addr learned my addr 172.21.15.107:0/2427126369 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T22:03:49.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:49.492+0000 7fe576ffd700 1 --2- 172.21.15.107:0/2427126369 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe578103780 0x7fe578113e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:49.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:49.492+0000 7fe5777fe700 1 -- 172.21.15.107:0/2427126369 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe578103780 msgr2=0x7fe578113e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:49.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:49.492+0000 7fe5777fe700 1 --2- 172.21.15.107:0/2427126369 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe578103780 0x7fe578113e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:49.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:49.492+0000 7fe5777fe700 1 -- 172.21.15.107:0/2427126369 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe568005040 con 0x7fe578102de0 2024-05-17T22:03:49.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:49.492+0000 7fe5777fe700 1 --2- 172.21.15.107:0/2427126369 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe578102de0 0x7fe578113930 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe56800b8b0 tx=0x7fe568009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:49.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:49.492+0000 7fe55ffff700 1 -- 172.21.15.107:0/2427126369 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe56800e040 con 0x7fe578102de0 2024-05-17T22:03:49.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:49.492+0000 7fe55ffff700 1 -- 172.21.15.107:0/2427126369 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe568004290 con 0x7fe578102de0 2024-05-17T22:03:49.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:49.492+0000 7fe57dd32700 1 -- 172.21.15.107:0/2427126369 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe57810dd40 con 0x7fe578102de0 2024-05-17T22:03:49.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:49.492+0000 7fe55ffff700 1 -- 172.21.15.107:0/2427126369 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe56801c3f0 con 0x7fe578102de0 2024-05-17T22:03:49.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:49.492+0000 7fe57dd32700 1 -- 172.21.15.107:0/2427126369 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe57810e300 con 0x7fe578102de0 2024-05-17T22:03:49.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:49.493+0000 7fe57dd32700 1 -- 172.21.15.107:0/2427126369 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5781000f0 con 0x7fe578102de0 2024-05-17T22:03:49.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:49.494+0000 7fe55ffff700 1 -- 172.21.15.107:0/2427126369 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7fe5680137b0 con 0x7fe578102de0 2024-05-17T22:03:49.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:49.494+0000 7fe55ffff700 1 --2- 172.21.15.107:0/2427126369 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fe560078990 0x7fe56007ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:49.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:49.494+0000 7fe55ffff700 1 -- 172.21.15.107:0/2427126369 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7fe56809bad0 con 0x7fe578102de0 2024-05-17T22:03:49.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:49.494+0000 7fe576ffd700 1 --2- 172.21.15.107:0/2427126369 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fe560078990 0x7fe56007ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:49.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:49.495+0000 7fe576ffd700 1 --2- 172.21.15.107:0/2427126369 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fe560078990 0x7fe56007ae40 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe56c006fd0 tx=0x7fe56c008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:49.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:49.499+0000 7fe55ffff700 1 -- 172.21.15.107:0/2427126369 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7fe568063710 con 0x7fe578102de0 2024-05-17T22:03:49.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:49.749+0000 7fe57dd32700 1 -- 172.21.15.107:0/2427126369 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe57804c710 con 0x7fe578102de0 2024-05-17T22:03:49.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:49.750+0000 7fe55ffff700 1 -- 172.21.15.107:0/2427126369 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+5224 (secure 0 0 0) 0x7fe5680630b0 con 0x7fe578102de0 2024-05-17T22:03:49.741 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T22:03:49.741 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44335,"name":"cephfs.smithi138.gwbtyd","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6825/1700112281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1700112281},{"type":"v1","addr":"172.21.15.138:6825","nonce":1700112281}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":44365,"name":"cephfs.smithi138.zckrvs","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6827/1801131392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1801131392},{"type":"v1","addr":"172.21.15.138:6827","nonce":1801131392}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":35,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T21:49:16.836333+0000","modified":"2024-05-17T22:00:33.850863+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34408},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34408":{"gid":34408,"name":"cephfs.smithi107.idmbgj","rank":0,"incarnation":29,"state":"up:active","state_seq":20,"addr":"172.21.15.107:6827/2235359521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":2235359521},{"type":"v1","addr":"172.21.15.107:6827","nonce":2235359521}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34440":{"gid":34440,"name":"cephfs.smithi107.drqody","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6829/369507752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":369507752},{"type":"v1","addr":"172.21.15.107:6829","nonce":369507752}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34408,"qdb_cluster":[34408]},"id":1}]} 2024-05-17T22:03:49.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:49.752+0000 7fe57dd32700 1 -- 172.21.15.107:0/2427126369 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fe560078990 msgr2=0x7fe56007ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:49.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:49.752+0000 7fe57dd32700 1 --2- 172.21.15.107:0/2427126369 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fe560078990 0x7fe56007ae40 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe56c006fd0 tx=0x7fe56c008040 comp rx=0 tx=0).stop 2024-05-17T22:03:49.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:49.752+0000 7fe57dd32700 1 -- 172.21.15.107:0/2427126369 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe578102de0 msgr2=0x7fe578113930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:49.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:49.752+0000 7fe57dd32700 1 --2- 172.21.15.107:0/2427126369 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe578102de0 0x7fe578113930 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe56800b8b0 tx=0x7fe568009e30 comp rx=0 tx=0).stop 2024-05-17T22:03:49.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:49.752+0000 7fe57dd32700 1 -- 172.21.15.107:0/2427126369 shutdown_connections 2024-05-17T22:03:49.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:49.752+0000 7fe57dd32700 1 --2- 172.21.15.107:0/2427126369 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fe560078990 0x7fe56007ae40 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:49.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:49.752+0000 7fe57dd32700 1 --2- 172.21.15.107:0/2427126369 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe578102de0 0x7fe578113930 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:49.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:49.752+0000 7fe57dd32700 1 --2- 172.21.15.107:0/2427126369 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe578103780 0x7fe578113e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:49.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:49.752+0000 7fe57dd32700 1 -- 172.21.15.107:0/2427126369 >> 172.21.15.107:0/2427126369 conn(0x7fe5780f8990 msgr2=0x7fe5780fa6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:03:49.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:49.752+0000 7fe57dd32700 1 -- 172.21.15.107:0/2427126369 shutdown_connections 2024-05-17T22:03:49.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:49.753+0000 7fe57dd32700 1 -- 172.21.15.107:0/2427126369 wait complete. 2024-05-17T22:03:49.744 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 35 2024-05-17T22:03:50.076 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:03:49 smithi107 ceph-mon[93224]: pgmap v315: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:03:50.077 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:03:49 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T22:03:50.077 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:03:49 smithi107 ceph-mon[93224]: from='client.? 172.21.15.107:0/2427126369' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T22:03:50.324 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:03:49 smithi138 ceph-mon[81376]: pgmap v315: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:03:50.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:03:49 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T22:03:50.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:03:49 smithi138 ceph-mon[81376]: from='client.? 172.21.15.107:0/2427126369' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T22:03:50.369 DEBUG:tasks.fs:checking fs fscid=1,name=cephfs state = {'epoch': 15, 'max_mds': 1, 'flags': 50} 2024-05-17T22:03:50.369 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 16 2024-05-17T22:03:50.626 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/mon.smithi107/config 2024-05-17T22:03:52.005 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:03:51 smithi107 ceph-mon[93224]: pgmap v316: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:03:52.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:52.264+0000 7f23d2ba8700 1 -- 172.21.15.107:0/3089307551 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23cc102de0 msgr2=0x7f23cc1031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:52.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:52.264+0000 7f23d2ba8700 1 --2- 172.21.15.107:0/3089307551 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23cc102de0 0x7f23cc1031b0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f23c0004940 tx=0x7f23c0009e30 comp rx=0 tx=0).stop 2024-05-17T22:03:52.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:52.264+0000 7f23d2ba8700 1 -- 172.21.15.107:0/3089307551 shutdown_connections 2024-05-17T22:03:52.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:52.264+0000 7f23d2ba8700 1 --2- 172.21.15.107:0/3089307551 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23cc103780 0x7f23cc0fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:52.264+0000 7f23d2ba8700 1 --2- 172.21.15.107:0/3089307551 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23cc102de0 0x7f23cc1031b0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:52.264+0000 7f23d2ba8700 1 -- 172.21.15.107:0/3089307551 >> 172.21.15.107:0/3089307551 conn(0x7f23cc0f8950 msgr2=0x7f23cc0fad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:03:52.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:52.264+0000 7f23d2ba8700 1 -- 172.21.15.107:0/3089307551 shutdown_connections 2024-05-17T22:03:52.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:52.264+0000 7f23d2ba8700 1 -- 172.21.15.107:0/3089307551 wait complete. 2024-05-17T22:03:52.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:52.265+0000 7f23d2ba8700 1 Processor -- start 2024-05-17T22:03:52.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:52.265+0000 7f23d2ba8700 1 -- start start 2024-05-17T22:03:52.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:52.265+0000 7f23d2ba8700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23cc102de0 0x7f23cc113c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:52.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:52.265+0000 7f23d2ba8700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23cc103780 0x7f23cc114140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:52.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:52.265+0000 7f23d2ba8700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23cc114820 con 0x7f23cc102de0 2024-05-17T22:03:52.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:52.265+0000 7f23d2ba8700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23cc10db80 con 0x7f23cc103780 2024-05-17T22:03:52.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:52.265+0000 7f23d0944700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23cc102de0 0x7f23cc113c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:52.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:52.265+0000 7f23d0944700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23cc102de0 0x7f23cc113c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48454/0 (socket says 172.21.15.107:48454) 2024-05-17T22:03:52.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:52.265+0000 7f23d0944700 1 -- 172.21.15.107:0/356671442 learned_addr learned my addr 172.21.15.107:0/356671442 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T22:03:52.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:52.266+0000 7f23cbfff700 1 --2- 172.21.15.107:0/356671442 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23cc103780 0x7f23cc114140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:52.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:52.266+0000 7f23d0944700 1 -- 172.21.15.107:0/356671442 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23cc103780 msgr2=0x7f23cc114140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:52.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:52.266+0000 7f23d0944700 1 --2- 172.21.15.107:0/356671442 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23cc103780 0x7f23cc114140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:52.266+0000 7f23d0944700 1 -- 172.21.15.107:0/356671442 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23c0005040 con 0x7f23cc102de0 2024-05-17T22:03:52.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:52.266+0000 7f23d0944700 1 --2- 172.21.15.107:0/356671442 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23cc102de0 0x7f23cc113c00 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f23c0009180 tx=0x7f23c0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:52.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:52.266+0000 7f23c97fa700 1 -- 172.21.15.107:0/356671442 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f23c000e040 con 0x7f23cc102de0 2024-05-17T22:03:52.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:52.266+0000 7f23d2ba8700 1 -- 172.21.15.107:0/356671442 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23cc10de00 con 0x7f23cc102de0 2024-05-17T22:03:52.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:52.266+0000 7f23c97fa700 1 -- 172.21.15.107:0/356671442 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f23c0013420 con 0x7f23cc102de0 2024-05-17T22:03:52.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:52.266+0000 7f23c97fa700 1 -- 172.21.15.107:0/356671442 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f23c001c630 con 0x7f23cc102de0 2024-05-17T22:03:52.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:52.266+0000 7f23d2ba8700 1 -- 172.21.15.107:0/356671442 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23cc10e340 con 0x7f23cc102de0 2024-05-17T22:03:52.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:52.267+0000 7f23d2ba8700 1 -- 172.21.15.107:0/356671442 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23cc1000f0 con 0x7f23cc102de0 2024-05-17T22:03:52.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:52.268+0000 7f23c97fa700 1 -- 172.21.15.107:0/356671442 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f23c0013590 con 0x7f23cc102de0 2024-05-17T22:03:52.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:52.269+0000 7f23c97fa700 1 --2- 172.21.15.107:0/356671442 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f23bc078980 0x7f23bc07ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:52.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:52.269+0000 7f23c97fa700 1 -- 172.21.15.107:0/356671442 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f23c009c3d0 con 0x7f23cc102de0 2024-05-17T22:03:52.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:52.269+0000 7f23cbfff700 1 --2- 172.21.15.107:0/356671442 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f23bc078980 0x7f23bc07ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:52.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:52.269+0000 7f23cbfff700 1 --2- 172.21.15.107:0/356671442 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f23bc078980 0x7f23bc07ae30 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f23cc10f430 tx=0x7f23b80091a0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:52.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:52.273+0000 7f23c97fa700 1 -- 172.21.15.107:0/356671442 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f23c00640b0 con 0x7f23cc102de0 2024-05-17T22:03:52.324 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:03:51 smithi138 ceph-mon[81376]: pgmap v316: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:03:52.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:52.518+0000 7f23d2ba8700 1 -- 172.21.15.107:0/356671442 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 16, "format": "json"} v 0) v1 -- 0x7f23cc109f80 con 0x7f23cc102de0 2024-05-17T22:03:52.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:52.518+0000 7f23c97fa700 1 -- 172.21.15.107:0/356671442 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 16, "format": "json"}]=0 dumped fsmap epoch 16 v35) v1 ==== 107+0+4094 (secure 0 0 0) 0x7f23c0063a50 con 0x7f23cc102de0 2024-05-17T22:03:52.510 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T22:03:52.510 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14498,"name":"cephfs.smithi107.idmbgj","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6827/1460186079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1460186079},{"type":"v1","addr":"172.21.15.107:6827","nonce":1460186079}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":24267,"name":"cephfs.smithi138.zckrvs","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.138:6827/170628661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":170628661},{"type":"v1","addr":"172.21.15.138:6827","nonce":170628661}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T21:49:16.836333+0000","modified":"2024-05-17T21:59:13.802503+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24259},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24259":{"gid":24259,"name":"cephfs.smithi138.gwbtyd","rank":0,"incarnation":12,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6825/2779634919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":2779634919},{"type":"v1","addr":"172.21.15.138:6825","nonce":2779634919}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24259,"qdb_cluster":[24259]},"id":1}]} 2024-05-17T22:03:52.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:52.521+0000 7f23d2ba8700 1 -- 172.21.15.107:0/356671442 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f23bc078980 msgr2=0x7f23bc07ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:52.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:52.521+0000 7f23d2ba8700 1 --2- 172.21.15.107:0/356671442 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f23bc078980 0x7f23bc07ae30 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f23cc10f430 tx=0x7f23b80091a0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:52.521+0000 7f23d2ba8700 1 -- 172.21.15.107:0/356671442 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23cc102de0 msgr2=0x7f23cc113c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:52.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:52.521+0000 7f23d2ba8700 1 --2- 172.21.15.107:0/356671442 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23cc102de0 0x7f23cc113c00 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f23c0009180 tx=0x7f23c0009e30 comp rx=0 tx=0).stop 2024-05-17T22:03:52.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:52.521+0000 7f23d2ba8700 1 -- 172.21.15.107:0/356671442 shutdown_connections 2024-05-17T22:03:52.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:52.521+0000 7f23d2ba8700 1 --2- 172.21.15.107:0/356671442 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f23bc078980 0x7f23bc07ae30 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:52.521+0000 7f23d2ba8700 1 --2- 172.21.15.107:0/356671442 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23cc102de0 0x7f23cc113c00 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:52.521+0000 7f23d2ba8700 1 --2- 172.21.15.107:0/356671442 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23cc103780 0x7f23cc114140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:52.521+0000 7f23d2ba8700 1 -- 172.21.15.107:0/356671442 >> 172.21.15.107:0/356671442 conn(0x7f23cc0f8950 msgr2=0x7f23cc1073d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:03:52.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:52.521+0000 7f23d2ba8700 1 -- 172.21.15.107:0/356671442 shutdown_connections 2024-05-17T22:03:52.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:52.521+0000 7f23d2ba8700 1 -- 172.21.15.107:0/356671442 wait complete. 2024-05-17T22:03:52.514 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 16 2024-05-17T22:03:53.167 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:03:52 smithi107 ceph-mon[93224]: from='client.? 172.21.15.107:0/356671442' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 16, "format": "json"}]: dispatch 2024-05-17T22:03:53.168 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 16 2024-05-17T22:03:53.169 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 17 2024-05-17T22:03:53.324 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:03:52 smithi138 ceph-mon[81376]: from='client.? 172.21.15.107:0/356671442' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 16, "format": "json"}]: dispatch 2024-05-17T22:03:53.424 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/mon.smithi107/config 2024-05-17T22:03:54.225 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:03:53 smithi107 ceph-mon[93224]: pgmap v317: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:03:54.324 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:03:53 smithi138 ceph-mon[81376]: pgmap v317: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:03:55.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:55.063+0000 7ff1ecbc5700 1 -- 172.21.15.107:0/2429357882 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff1e8102eb0 msgr2=0x7ff1e8103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:55.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:55.063+0000 7ff1ecbc5700 1 --2- 172.21.15.107:0/2429357882 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff1e8102eb0 0x7ff1e8103280 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ff1d8004940 tx=0x7ff1d8009e30 comp rx=0 tx=0).stop 2024-05-17T22:03:55.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:55.063+0000 7ff1ecbc5700 1 -- 172.21.15.107:0/2429357882 shutdown_connections 2024-05-17T22:03:55.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:55.063+0000 7ff1ecbc5700 1 --2- 172.21.15.107:0/2429357882 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1e80fb3c0 0x7ff1e80fb830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:55.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:55.063+0000 7ff1ecbc5700 1 --2- 172.21.15.107:0/2429357882 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff1e8102eb0 0x7ff1e8103280 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:55.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:55.063+0000 7ff1ecbc5700 1 -- 172.21.15.107:0/2429357882 >> 172.21.15.107:0/2429357882 conn(0x7ff1e8072380 msgr2=0x7ff1e8072780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:03:55.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:55.063+0000 7ff1ecbc5700 1 -- 172.21.15.107:0/2429357882 shutdown_connections 2024-05-17T22:03:55.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:55.063+0000 7ff1ecbc5700 1 -- 172.21.15.107:0/2429357882 wait complete. 2024-05-17T22:03:55.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:55.064+0000 7ff1ecbc5700 1 Processor -- start 2024-05-17T22:03:55.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:55.064+0000 7ff1ecbc5700 1 -- start start 2024-05-17T22:03:55.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:55.064+0000 7ff1ecbc5700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff1e80fb3c0 0x7ff1e80fe030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:55.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:55.064+0000 7ff1ecbc5700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1e8102eb0 0x7ff1e80fc5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:55.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:55.064+0000 7ff1ecbc5700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e80fcae0 con 0x7ff1e80fb3c0 2024-05-17T22:03:55.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:55.064+0000 7ff1ecbc5700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e80fcc50 con 0x7ff1e8102eb0 2024-05-17T22:03:55.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:55.064+0000 7ff1e659c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff1e80fb3c0 0x7ff1e80fe030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:55.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:55.064+0000 7ff1e659c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff1e80fb3c0 0x7ff1e80fe030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48466/0 (socket says 172.21.15.107:48466) 2024-05-17T22:03:55.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:55.064+0000 7ff1e659c700 1 -- 172.21.15.107:0/1239482468 learned_addr learned my addr 172.21.15.107:0/1239482468 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T22:03:55.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:55.064+0000 7ff1e5d9b700 1 --2- 172.21.15.107:0/1239482468 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1e8102eb0 0x7ff1e80fc5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:55.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:55.065+0000 7ff1e659c700 1 -- 172.21.15.107:0/1239482468 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1e8102eb0 msgr2=0x7ff1e80fc5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:55.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:55.065+0000 7ff1e659c700 1 --2- 172.21.15.107:0/1239482468 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1e8102eb0 0x7ff1e80fc5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:55.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:55.065+0000 7ff1e659c700 1 -- 172.21.15.107:0/1239482468 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1d8005040 con 0x7ff1e80fb3c0 2024-05-17T22:03:55.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:55.065+0000 7ff1e659c700 1 --2- 172.21.15.107:0/1239482468 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff1e80fb3c0 0x7ff1e80fe030 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff1d8009100 tx=0x7ff1d8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:55.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:55.065+0000 7ff1deffd700 1 -- 172.21.15.107:0/1239482468 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff1d800e040 con 0x7ff1e80fb3c0 2024-05-17T22:03:55.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:55.065+0000 7ff1ecbc5700 1 -- 172.21.15.107:0/1239482468 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1e80fced0 con 0x7ff1e80fb3c0 2024-05-17T22:03:55.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:55.065+0000 7ff1deffd700 1 -- 172.21.15.107:0/1239482468 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff1d8009af0 con 0x7ff1e80fb3c0 2024-05-17T22:03:55.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:55.065+0000 7ff1deffd700 1 -- 172.21.15.107:0/1239482468 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff1d801c3f0 con 0x7ff1e80fb3c0 2024-05-17T22:03:55.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:55.065+0000 7ff1ecbc5700 1 -- 172.21.15.107:0/1239482468 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1e80fd410 con 0x7ff1e80fb3c0 2024-05-17T22:03:55.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:55.066+0000 7ff1ecbc5700 1 -- 172.21.15.107:0/1239482468 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1e8100160 con 0x7ff1e80fb3c0 2024-05-17T22:03:55.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:55.067+0000 7ff1deffd700 1 -- 172.21.15.107:0/1239482468 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7ff1d800baf0 con 0x7ff1e80fb3c0 2024-05-17T22:03:55.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:55.067+0000 7ff1deffd700 1 --2- 172.21.15.107:0/1239482468 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7ff1d4078990 0x7ff1d407ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:55.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:55.067+0000 7ff1deffd700 1 -- 172.21.15.107:0/1239482468 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7ff1d809b780 con 0x7ff1e80fb3c0 2024-05-17T22:03:55.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:55.067+0000 7ff1e5d9b700 1 --2- 172.21.15.107:0/1239482468 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7ff1d4078990 0x7ff1d407ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:55.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:55.068+0000 7ff1e5d9b700 1 --2- 172.21.15.107:0/1239482468 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7ff1d4078990 0x7ff1d407ae40 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff1e80fdcd0 tx=0x7ff1d000b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:55.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:55.072+0000 7ff1deffd700 1 -- 172.21.15.107:0/1239482468 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7ff1d8063460 con 0x7ff1e80fb3c0 2024-05-17T22:03:55.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:55.320+0000 7ff1ecbc5700 1 -- 172.21.15.107:0/1239482468 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 17, "format": "json"} v 0) v1 -- 0x7ff1e80fd410 con 0x7ff1e80fb3c0 2024-05-17T22:03:55.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:55.321+0000 7ff1deffd700 1 -- 172.21.15.107:0/1239482468 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 17, "format": "json"}]=0 dumped fsmap epoch 17 v35) v1 ==== 107+0+4873 (secure 0 0 0) 0x7ff1e80fd410 con 0x7ff1e80fb3c0 2024-05-17T22:03:55.312 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T22:03:55.312 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14498,"name":"cephfs.smithi107.idmbgj","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6827/1460186079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1460186079},{"type":"v1","addr":"172.21.15.107:6827","nonce":1460186079}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":24267,"name":"cephfs.smithi138.zckrvs","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.138:6827/170628661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":170628661},{"type":"v1","addr":"172.21.15.138:6827","nonce":170628661}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":34404,"name":"cephfs.smithi107.drqody","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6829/878488679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":878488679},{"type":"v1","addr":"172.21.15.107:6829","nonce":878488679}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T21:49:16.836333+0000","modified":"2024-05-17T21:59:13.802503+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24259},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24259":{"gid":24259,"name":"cephfs.smithi138.gwbtyd","rank":0,"incarnation":12,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6825/2779634919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":2779634919},{"type":"v1","addr":"172.21.15.138:6825","nonce":2779634919}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24259,"qdb_cluster":[24259]},"id":1}]} 2024-05-17T22:03:55.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:55.323+0000 7ff1ecbc5700 1 -- 172.21.15.107:0/1239482468 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7ff1d4078990 msgr2=0x7ff1d407ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:55.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:55.323+0000 7ff1ecbc5700 1 --2- 172.21.15.107:0/1239482468 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7ff1d4078990 0x7ff1d407ae40 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff1e80fdcd0 tx=0x7ff1d000b040 comp rx=0 tx=0).stop 2024-05-17T22:03:55.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:55.323+0000 7ff1ecbc5700 1 -- 172.21.15.107:0/1239482468 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff1e80fb3c0 msgr2=0x7ff1e80fe030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:55.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:55.323+0000 7ff1ecbc5700 1 --2- 172.21.15.107:0/1239482468 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff1e80fb3c0 0x7ff1e80fe030 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff1d8009100 tx=0x7ff1d8009e30 comp rx=0 tx=0).stop 2024-05-17T22:03:55.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:55.323+0000 7ff1ecbc5700 1 -- 172.21.15.107:0/1239482468 shutdown_connections 2024-05-17T22:03:55.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:55.323+0000 7ff1ecbc5700 1 --2- 172.21.15.107:0/1239482468 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7ff1d4078990 0x7ff1d407ae40 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:55.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:55.323+0000 7ff1ecbc5700 1 --2- 172.21.15.107:0/1239482468 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff1e80fb3c0 0x7ff1e80fe030 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:55.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:55.323+0000 7ff1ecbc5700 1 --2- 172.21.15.107:0/1239482468 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1e8102eb0 0x7ff1e80fc5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:55.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:55.323+0000 7ff1ecbc5700 1 -- 172.21.15.107:0/1239482468 >> 172.21.15.107:0/1239482468 conn(0x7ff1e8072380 msgr2=0x7ff1e80fa4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:03:55.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:55.323+0000 7ff1ecbc5700 1 -- 172.21.15.107:0/1239482468 shutdown_connections 2024-05-17T22:03:55.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:55.323+0000 7ff1ecbc5700 1 -- 172.21.15.107:0/1239482468 wait complete. 2024-05-17T22:03:55.315 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 17 2024-05-17T22:03:55.976 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:03:55 smithi107 ceph-mon[93224]: pgmap v318: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:03:55.976 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:03:55 smithi107 ceph-mon[93224]: from='client.? 172.21.15.107:0/1239482468' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 17, "format": "json"}]: dispatch 2024-05-17T22:03:56.000 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 17 2024-05-17T22:03:56.000 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 18 2024-05-17T22:03:56.255 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/mon.smithi107/config 2024-05-17T22:03:56.324 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:03:55 smithi138 ceph-mon[81376]: pgmap v318: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:03:56.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:03:55 smithi138 ceph-mon[81376]: from='client.? 172.21.15.107:0/1239482468' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 17, "format": "json"}]: dispatch 2024-05-17T22:03:57.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:57.887+0000 7f78b9713700 1 -- 172.21.15.107:0/4014523994 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f78b4102e00 msgr2=0x7f78b41031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:57.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:57.887+0000 7f78b9713700 1 --2- 172.21.15.107:0/4014523994 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f78b4102e00 0x7f78b41031d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f78a4004940 tx=0x7f78a4009e30 comp rx=0 tx=0).stop 2024-05-17T22:03:57.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:57.887+0000 7f78b9713700 1 -- 172.21.15.107:0/4014523994 shutdown_connections 2024-05-17T22:03:57.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:57.887+0000 7f78b9713700 1 --2- 172.21.15.107:0/4014523994 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f78b41037a0 0x7f78b40fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:57.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:57.887+0000 7f78b9713700 1 --2- 172.21.15.107:0/4014523994 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f78b4102e00 0x7f78b41031d0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:57.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:57.887+0000 7f78b9713700 1 -- 172.21.15.107:0/4014523994 >> 172.21.15.107:0/4014523994 conn(0x7f78b40f89b0 msgr2=0x7f78b40fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:03:57.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:57.887+0000 7f78b9713700 1 -- 172.21.15.107:0/4014523994 shutdown_connections 2024-05-17T22:03:57.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:57.887+0000 7f78b9713700 1 -- 172.21.15.107:0/4014523994 wait complete. 2024-05-17T22:03:57.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:57.888+0000 7f78b9713700 1 Processor -- start 2024-05-17T22:03:57.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:57.888+0000 7f78b9713700 1 -- start start 2024-05-17T22:03:57.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:57.888+0000 7f78b9713700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f78b4102e00 0x7f78b4070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:57.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:57.888+0000 7f78b9713700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f78b41037a0 0x7f78b40707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:57.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:57.888+0000 7f78b9713700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78b4070ce0 con 0x7f78b4102e00 2024-05-17T22:03:57.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:57.888+0000 7f78b9713700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78b4070e20 con 0x7f78b41037a0 2024-05-17T22:03:57.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:57.888+0000 7f78b2ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f78b4102e00 0x7f78b4070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:57.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:57.888+0000 7f78b2ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f78b4102e00 0x7f78b4070260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47556/0 (socket says 172.21.15.107:47556) 2024-05-17T22:03:57.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:57.888+0000 7f78b2ffd700 1 -- 172.21.15.107:0/3558003272 learned_addr learned my addr 172.21.15.107:0/3558003272 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T22:03:57.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:57.889+0000 7f78b27fc700 1 --2- 172.21.15.107:0/3558003272 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f78b41037a0 0x7f78b40707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:57.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:57.889+0000 7f78b2ffd700 1 -- 172.21.15.107:0/3558003272 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f78b41037a0 msgr2=0x7f78b40707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:57.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:57.889+0000 7f78b2ffd700 1 --2- 172.21.15.107:0/3558003272 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f78b41037a0 0x7f78b40707a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:57.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:57.889+0000 7f78b2ffd700 1 -- 172.21.15.107:0/3558003272 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78a4005040 con 0x7f78b4102e00 2024-05-17T22:03:57.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:57.889+0000 7f78b2ffd700 1 --2- 172.21.15.107:0/3558003272 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f78b4102e00 0x7f78b4070260 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f78a400b3b0 tx=0x7f78a4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:57.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:57.889+0000 7f789b7fe700 1 -- 172.21.15.107:0/3558003272 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f78a400e040 con 0x7f78b4102e00 2024-05-17T22:03:57.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:57.889+0000 7f78b9713700 1 -- 172.21.15.107:0/3558003272 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78b4071070 con 0x7f78b4102e00 2024-05-17T22:03:57.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:57.889+0000 7f789b7fe700 1 -- 172.21.15.107:0/3558003272 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f78a4013420 con 0x7f78b4102e00 2024-05-17T22:03:57.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:57.889+0000 7f789b7fe700 1 -- 172.21.15.107:0/3558003272 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f78a401c630 con 0x7f78b4102e00 2024-05-17T22:03:57.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:57.889+0000 7f78b9713700 1 -- 172.21.15.107:0/3558003272 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78b411a2c0 con 0x7f78b4102e00 2024-05-17T22:03:57.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:57.890+0000 7f78b9713700 1 -- 172.21.15.107:0/3558003272 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78b4100110 con 0x7f78b4102e00 2024-05-17T22:03:57.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:57.891+0000 7f789b7fe700 1 -- 172.21.15.107:0/3558003272 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f78a4013590 con 0x7f78b4102e00 2024-05-17T22:03:57.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:57.891+0000 7f789b7fe700 1 --2- 172.21.15.107:0/3558003272 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f789c078990 0x7f789c07ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:57.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:57.892+0000 7f78b27fc700 1 --2- 172.21.15.107:0/3558003272 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f789c078990 0x7f789c07ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:57.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:57.892+0000 7f789b7fe700 1 -- 172.21.15.107:0/3558003272 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f78a409c380 con 0x7f78b4102e00 2024-05-17T22:03:57.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:57.892+0000 7f78b27fc700 1 --2- 172.21.15.107:0/3558003272 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f789c078990 0x7f789c07ae40 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f78b4071b50 tx=0x7f78a800b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:57.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:57.896+0000 7f789b7fe700 1 -- 172.21.15.107:0/3558003272 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f78a4064060 con 0x7f78b4102e00 2024-05-17T22:03:58.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:58.147+0000 7f78b9713700 1 -- 172.21.15.107:0/3558003272 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 18, "format": "json"} v 0) v1 -- 0x7f78b40fe550 con 0x7f78b4102e00 2024-05-17T22:03:58.139 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:03:57 smithi107 ceph-mon[93224]: pgmap v319: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:03:58.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:58.148+0000 7f789b7fe700 1 -- 172.21.15.107:0/3558003272 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 18, "format": "json"}]=0 dumped fsmap epoch 18 v35) v1 ==== 107+0+4091 (secure 0 0 0) 0x7f78a4063a00 con 0x7f78b4102e00 2024-05-17T22:03:58.139 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T22:03:58.139 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24267,"name":"cephfs.smithi138.zckrvs","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.138:6827/170628661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":170628661},{"type":"v1","addr":"172.21.15.138:6827","nonce":170628661}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":34404,"name":"cephfs.smithi107.drqody","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6829/878488679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":878488679},{"type":"v1","addr":"172.21.15.107:6829","nonce":878488679}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T21:49:16.836333+0000","modified":"2024-05-17T21:59:13.802503+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24259},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24259":{"gid":24259,"name":"cephfs.smithi138.gwbtyd","rank":0,"incarnation":12,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6825/2779634919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":2779634919},{"type":"v1","addr":"172.21.15.138:6825","nonce":2779634919}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24259,"qdb_cluster":[24259]},"id":1}]} 2024-05-17T22:03:58.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:58.150+0000 7f78b9713700 1 -- 172.21.15.107:0/3558003272 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f789c078990 msgr2=0x7f789c07ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:58.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:58.150+0000 7f78b9713700 1 --2- 172.21.15.107:0/3558003272 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f789c078990 0x7f789c07ae40 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f78b4071b50 tx=0x7f78a800b040 comp rx=0 tx=0).stop 2024-05-17T22:03:58.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:58.150+0000 7f78b9713700 1 -- 172.21.15.107:0/3558003272 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f78b4102e00 msgr2=0x7f78b4070260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:58.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:58.150+0000 7f78b9713700 1 --2- 172.21.15.107:0/3558003272 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f78b4102e00 0x7f78b4070260 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f78a400b3b0 tx=0x7f78a4009e30 comp rx=0 tx=0).stop 2024-05-17T22:03:58.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:58.151+0000 7f78b9713700 1 -- 172.21.15.107:0/3558003272 shutdown_connections 2024-05-17T22:03:58.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:58.151+0000 7f78b9713700 1 --2- 172.21.15.107:0/3558003272 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f789c078990 0x7f789c07ae40 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:58.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:58.151+0000 7f78b9713700 1 --2- 172.21.15.107:0/3558003272 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f78b4102e00 0x7f78b4070260 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:58.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:58.151+0000 7f78b9713700 1 --2- 172.21.15.107:0/3558003272 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f78b41037a0 0x7f78b40707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:58.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:58.151+0000 7f78b9713700 1 -- 172.21.15.107:0/3558003272 >> 172.21.15.107:0/3558003272 conn(0x7f78b40f89b0 msgr2=0x7f78b41073a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:03:58.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:58.151+0000 7f78b9713700 1 -- 172.21.15.107:0/3558003272 shutdown_connections 2024-05-17T22:03:58.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:03:58.151+0000 7f78b9713700 1 -- 172.21.15.107:0/3558003272 wait complete. 2024-05-17T22:03:58.143 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 18 2024-05-17T22:03:58.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:03:57 smithi138 ceph-mon[81376]: pgmap v319: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:03:58.766 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 18 2024-05-17T22:03:58.767 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 19 2024-05-17T22:03:59.038 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/mon.smithi107/config 2024-05-17T22:03:59.100 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:03:58 smithi107 ceph-mon[93224]: from='client.? 172.21.15.107:0/3558003272' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 18, "format": "json"}]: dispatch 2024-05-17T22:03:59.100 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:03:58 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:03:59.324 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:03:58 smithi138 ceph-mon[81376]: from='client.? 172.21.15.107:0/3558003272' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 18, "format": "json"}]: dispatch 2024-05-17T22:03:59.324 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:03:58 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:04:00.185 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:03:59 smithi107 ceph-mon[93224]: pgmap v320: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:04:00.185 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:03:59 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:04:00.185 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:03:59 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:04:00.324 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:03:59 smithi138 ceph-mon[81376]: pgmap v320: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:04:00.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:03:59 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:04:00.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:03:59 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:04:00.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:00.743+0000 7fea6e808700 1 -- 172.21.15.107:0/1215746707 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea681026a0 msgr2=0x7fea680fbd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:00.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:00.743+0000 7fea6e808700 1 --2- 172.21.15.107:0/1215746707 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea681026a0 0x7fea680fbd10 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fea50004940 tx=0x7fea50009e30 comp rx=0 tx=0).stop 2024-05-17T22:04:00.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:00.743+0000 7fea6e808700 1 -- 172.21.15.107:0/1215746707 shutdown_connections 2024-05-17T22:04:00.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:00.743+0000 7fea6e808700 1 --2- 172.21.15.107:0/1215746707 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea681026a0 0x7fea680fbd10 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:00.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:00.743+0000 7fea6e808700 1 --2- 172.21.15.107:0/1215746707 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fea68101d00 0x7fea681020d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:00.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:00.743+0000 7fea6e808700 1 -- 172.21.15.107:0/1215746707 >> 172.21.15.107:0/1215746707 conn(0x7fea680f78d0 msgr2=0x7fea680f9ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:04:00.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:00.743+0000 7fea6e808700 1 -- 172.21.15.107:0/1215746707 shutdown_connections 2024-05-17T22:04:00.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:00.743+0000 7fea6e808700 1 -- 172.21.15.107:0/1215746707 wait complete. 2024-05-17T22:04:00.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:00.744+0000 7fea6e808700 1 Processor -- start 2024-05-17T22:04:00.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:00.744+0000 7fea6e808700 1 -- start start 2024-05-17T22:04:00.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:00.745+0000 7fea6e808700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fea68101d00 0x7fea6810ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:00.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:00.745+0000 7fea6e808700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea681026a0 0x7fea6810f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:00.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:00.745+0000 7fea6e808700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea6810f7c0 con 0x7fea681026a0 2024-05-17T22:04:00.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:00.745+0000 7fea6e808700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea6810f900 con 0x7fea68101d00 2024-05-17T22:04:00.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:00.745+0000 7fea677fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea681026a0 0x7fea6810f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:00.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:00.745+0000 7fea677fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea681026a0 0x7fea6810f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47570/0 (socket says 172.21.15.107:47570) 2024-05-17T22:04:00.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:00.745+0000 7fea677fe700 1 -- 172.21.15.107:0/3590187114 learned_addr learned my addr 172.21.15.107:0/3590187114 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T22:04:00.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:00.745+0000 7fea67fff700 1 --2- 172.21.15.107:0/3590187114 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fea68101d00 0x7fea6810ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:00.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:00.745+0000 7fea677fe700 1 -- 172.21.15.107:0/3590187114 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fea68101d00 msgr2=0x7fea6810ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:00.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:00.745+0000 7fea677fe700 1 --2- 172.21.15.107:0/3590187114 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fea68101d00 0x7fea6810ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:00.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:00.745+0000 7fea677fe700 1 -- 172.21.15.107:0/3590187114 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea50005040 con 0x7fea681026a0 2024-05-17T22:04:00.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:00.745+0000 7fea677fe700 1 --2- 172.21.15.107:0/3590187114 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea681026a0 0x7fea6810f280 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fea5000b3b0 tx=0x7fea50009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:00.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:00.746+0000 7fea64ff9700 1 -- 172.21.15.107:0/3590187114 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fea5000e040 con 0x7fea681026a0 2024-05-17T22:04:00.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:00.746+0000 7fea64ff9700 1 -- 172.21.15.107:0/3590187114 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fea50009b10 con 0x7fea681026a0 2024-05-17T22:04:00.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:00.746+0000 7fea64ff9700 1 -- 172.21.15.107:0/3590187114 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fea50013420 con 0x7fea681026a0 2024-05-17T22:04:00.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:00.746+0000 7fea6e808700 1 -- 172.21.15.107:0/3590187114 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea68117bf0 con 0x7fea681026a0 2024-05-17T22:04:00.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:00.746+0000 7fea6e808700 1 -- 172.21.15.107:0/3590187114 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea681180b0 con 0x7fea681026a0 2024-05-17T22:04:00.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:00.747+0000 7fea6e808700 1 -- 172.21.15.107:0/3590187114 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea680fd450 con 0x7fea681026a0 2024-05-17T22:04:00.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:00.748+0000 7fea64ff9700 1 -- 172.21.15.107:0/3590187114 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7fea5000baf0 con 0x7fea681026a0 2024-05-17T22:04:00.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:00.749+0000 7fea64ff9700 1 --2- 172.21.15.107:0/3590187114 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fea54078770 0x7fea5407ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:00.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:00.749+0000 7fea64ff9700 1 -- 172.21.15.107:0/3590187114 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7fea5009b2f0 con 0x7fea681026a0 2024-05-17T22:04:00.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:00.749+0000 7fea67fff700 1 --2- 172.21.15.107:0/3590187114 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fea54078770 0x7fea5407ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:00.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:00.749+0000 7fea67fff700 1 --2- 172.21.15.107:0/3590187114 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fea54078770 0x7fea5407ac20 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fea58004f40 tx=0x7fea5800b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:00.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:00.753+0000 7fea64ff9700 1 -- 172.21.15.107:0/3590187114 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7fea50062fe0 con 0x7fea681026a0 2024-05-17T22:04:01.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:01.011+0000 7fea6e808700 1 -- 172.21.15.107:0/3590187114 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 19, "format": "json"} v 0) v1 -- 0x7fea6804bed0 con 0x7fea681026a0 2024-05-17T22:04:01.002 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:04:00 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:04:01.002 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:04:00 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:04:01.002 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:04:00 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:04:01.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:01.011+0000 7fea64ff9700 1 -- 172.21.15.107:0/3590187114 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 19, "format": "json"}]=0 dumped fsmap epoch 19 v35) v1 ==== 107+0+4941 (secure 0 0 0) 0x7fea50062980 con 0x7fea681026a0 2024-05-17T22:04:01.003 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T22:04:01.003 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24267,"name":"cephfs.smithi138.zckrvs","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.138:6827/170628661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":170628661},{"type":"v1","addr":"172.21.15.138:6827","nonce":170628661}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":34404,"name":"cephfs.smithi107.drqody","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6829/878488679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":878488679},{"type":"v1","addr":"172.21.15.107:6829","nonce":878488679}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17},{"gid":34408,"name":"cephfs.smithi107.idmbgj","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6827/2235359521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":2235359521},{"type":"v1","addr":"172.21.15.107:6827","nonce":2235359521}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T21:49:16.836333+0000","modified":"2024-05-17T21:59:13.802503+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24259},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24259":{"gid":24259,"name":"cephfs.smithi138.gwbtyd","rank":0,"incarnation":12,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6825/2779634919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":2779634919},{"type":"v1","addr":"172.21.15.138:6825","nonce":2779634919}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24259,"qdb_cluster":[24259]},"id":1}]} 2024-05-17T22:04:01.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:01.014+0000 7fea6e808700 1 -- 172.21.15.107:0/3590187114 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fea54078770 msgr2=0x7fea5407ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:01.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:01.014+0000 7fea6e808700 1 --2- 172.21.15.107:0/3590187114 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fea54078770 0x7fea5407ac20 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fea58004f40 tx=0x7fea5800b040 comp rx=0 tx=0).stop 2024-05-17T22:04:01.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:01.014+0000 7fea6e808700 1 -- 172.21.15.107:0/3590187114 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea681026a0 msgr2=0x7fea6810f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:01.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:01.014+0000 7fea6e808700 1 --2- 172.21.15.107:0/3590187114 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea681026a0 0x7fea6810f280 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fea5000b3b0 tx=0x7fea50009e30 comp rx=0 tx=0).stop 2024-05-17T22:04:01.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:01.014+0000 7fea6e808700 1 -- 172.21.15.107:0/3590187114 shutdown_connections 2024-05-17T22:04:01.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:01.014+0000 7fea6e808700 1 --2- 172.21.15.107:0/3590187114 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fea54078770 0x7fea5407ac20 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:01.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:01.014+0000 7fea6e808700 1 --2- 172.21.15.107:0/3590187114 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fea68101d00 0x7fea6810ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:01.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:01.014+0000 7fea6e808700 1 --2- 172.21.15.107:0/3590187114 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea681026a0 0x7fea6810f280 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:01.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:01.014+0000 7fea6e808700 1 -- 172.21.15.107:0/3590187114 >> 172.21.15.107:0/3590187114 conn(0x7fea680f78d0 msgr2=0x7fea680f88d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:04:01.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:01.014+0000 7fea6e808700 1 -- 172.21.15.107:0/3590187114 shutdown_connections 2024-05-17T22:04:01.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:01.014+0000 7fea6e808700 1 -- 172.21.15.107:0/3590187114 wait complete. 2024-05-17T22:04:01.006 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 19 2024-05-17T22:04:01.324 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:04:00 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:04:01.324 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:04:00 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:04:01.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:04:00 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:04:01.619 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 19 2024-05-17T22:04:01.619 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 20 2024-05-17T22:04:01.878 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/mon.smithi107/config 2024-05-17T22:04:02.280 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:04:01 smithi107 ceph-mon[93224]: pgmap v321: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:04:02.280 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:04:01 smithi107 ceph-mon[93224]: from='client.? 172.21.15.107:0/3590187114' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 19, "format": "json"}]: dispatch 2024-05-17T22:04:02.324 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:04:01 smithi138 ceph-mon[81376]: pgmap v321: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:04:02.324 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:04:01 smithi138 ceph-mon[81376]: from='client.? 172.21.15.107:0/3590187114' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 19, "format": "json"}]: dispatch 2024-05-17T22:04:03.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:03.558+0000 7f5735c43700 1 -- 172.21.15.107:0/2933029996 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57301026b0 msgr2=0x7f57300fbd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:03.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:03.558+0000 7f5735c43700 1 --2- 172.21.15.107:0/2933029996 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57301026b0 0x7f57300fbd20 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f5724004940 tx=0x7f5724009e30 comp rx=0 tx=0).stop 2024-05-17T22:04:03.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:03.558+0000 7f5735c43700 1 -- 172.21.15.107:0/2933029996 shutdown_connections 2024-05-17T22:04:03.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:03.558+0000 7f5735c43700 1 --2- 172.21.15.107:0/2933029996 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57301026b0 0x7f57300fbd20 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:03.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:03.558+0000 7f5735c43700 1 --2- 172.21.15.107:0/2933029996 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5730101d10 0x7f57301020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:03.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:03.558+0000 7f5735c43700 1 -- 172.21.15.107:0/2933029996 >> 172.21.15.107:0/2933029996 conn(0x7f57300f7860 msgr2=0x7f57300f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:04:03.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:03.558+0000 7f5735c43700 1 -- 172.21.15.107:0/2933029996 shutdown_connections 2024-05-17T22:04:03.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:03.558+0000 7f5735c43700 1 -- 172.21.15.107:0/2933029996 wait complete. 2024-05-17T22:04:03.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:03.559+0000 7f5735c43700 1 Processor -- start 2024-05-17T22:04:03.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:03.559+0000 7f5735c43700 1 -- start start 2024-05-17T22:04:03.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:03.559+0000 7f5735c43700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5730101d10 0x7f5730102b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:03.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:03.559+0000 7f5735c43700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57301026b0 0x7f57301030b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:03.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:03.559+0000 7f5735c43700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5730104b50 con 0x7f57301026b0 2024-05-17T22:04:03.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:03.559+0000 7f5735c43700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57301035f0 con 0x7f5730101d10 2024-05-17T22:04:03.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:03.559+0000 7f572effd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57301026b0 0x7f57301030b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:03.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:03.559+0000 7f572effd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57301026b0 0x7f57301030b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47586/0 (socket says 172.21.15.107:47586) 2024-05-17T22:04:03.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:03.559+0000 7f572effd700 1 -- 172.21.15.107:0/242355842 learned_addr learned my addr 172.21.15.107:0/242355842 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T22:04:03.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:03.559+0000 7f572effd700 1 -- 172.21.15.107:0/242355842 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5730101d10 msgr2=0x7f5730102b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:03.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:03.559+0000 7f572f7fe700 1 --2- 172.21.15.107:0/242355842 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5730101d10 0x7f5730102b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:03.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:03.559+0000 7f572effd700 1 --2- 172.21.15.107:0/242355842 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5730101d10 0x7f5730102b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:03.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:03.559+0000 7f572effd700 1 -- 172.21.15.107:0/242355842 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5724005040 con 0x7f57301026b0 2024-05-17T22:04:03.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:03.559+0000 7f572f7fe700 1 --2- 172.21.15.107:0/242355842 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5730101d10 0x7f5730102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:04:03.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:03.560+0000 7f572effd700 1 --2- 172.21.15.107:0/242355842 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57301026b0 0x7f57301030b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f5724009180 tx=0x7f5724009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:03.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:03.560+0000 7f5717fff700 1 -- 172.21.15.107:0/242355842 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f572400e040 con 0x7f57301026b0 2024-05-17T22:04:03.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:03.560+0000 7f5735c43700 1 -- 172.21.15.107:0/242355842 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5730103870 con 0x7f57301026b0 2024-05-17T22:04:03.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:03.560+0000 7f5735c43700 1 -- 172.21.15.107:0/242355842 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f573011a1a0 con 0x7f57301026b0 2024-05-17T22:04:03.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:03.560+0000 7f5717fff700 1 -- 172.21.15.107:0/242355842 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f572400bbc0 con 0x7f57301026b0 2024-05-17T22:04:03.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:03.560+0000 7f5717fff700 1 -- 172.21.15.107:0/242355842 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5724025400 con 0x7f57301026b0 2024-05-17T22:04:03.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:03.561+0000 7f5735c43700 1 -- 172.21.15.107:0/242355842 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f571c0052b0 con 0x7f57301026b0 2024-05-17T22:04:03.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:03.562+0000 7f5717fff700 1 -- 172.21.15.107:0/242355842 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f572400bd30 con 0x7f57301026b0 2024-05-17T22:04:03.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:03.562+0000 7f5717fff700 1 --2- 172.21.15.107:0/242355842 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f5718078990 0x7f571807ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:03.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:03.562+0000 7f572f7fe700 1 --2- 172.21.15.107:0/242355842 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f5718078990 0x7f571807ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:03.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:03.562+0000 7f5717fff700 1 -- 172.21.15.107:0/242355842 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f572409ba40 con 0x7f57301026b0 2024-05-17T22:04:03.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:03.563+0000 7f572f7fe700 1 --2- 172.21.15.107:0/242355842 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f5718078990 0x7f571807ae40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f5720007fd0 tx=0x7f57200074f0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:03.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:03.567+0000 7f5717fff700 1 -- 172.21.15.107:0/242355842 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f5724063680 con 0x7f57301026b0 2024-05-17T22:04:03.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:03.814+0000 7f5735c43700 1 -- 172.21.15.107:0/242355842 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 20, "format": "json"} v 0) v1 -- 0x7f571c005560 con 0x7f57301026b0 2024-05-17T22:04:03.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:03.815+0000 7f5717fff700 1 -- 172.21.15.107:0/242355842 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 20, "format": "json"}]=0 dumped fsmap epoch 20 v35) v1 ==== 107+0+4162 (secure 0 0 0) 0x7f5724063020 con 0x7f57301026b0 2024-05-17T22:04:03.807 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T22:04:03.807 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24267,"name":"cephfs.smithi138.zckrvs","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.138:6827/170628661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":170628661},{"type":"v1","addr":"172.21.15.138:6827","nonce":170628661}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":34408,"name":"cephfs.smithi107.idmbgj","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6827/2235359521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":2235359521},{"type":"v1","addr":"172.21.15.107:6827","nonce":2235359521}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T21:49:16.836333+0000","modified":"2024-05-17T21:59:13.802503+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24259},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24259":{"gid":24259,"name":"cephfs.smithi138.gwbtyd","rank":0,"incarnation":12,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6825/2779634919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":2779634919},{"type":"v1","addr":"172.21.15.138:6825","nonce":2779634919}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24259,"qdb_cluster":[24259]},"id":1}]} 2024-05-17T22:04:03.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:03.817+0000 7f5735c43700 1 -- 172.21.15.107:0/242355842 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f5718078990 msgr2=0x7f571807ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:03.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:03.817+0000 7f5735c43700 1 --2- 172.21.15.107:0/242355842 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f5718078990 0x7f571807ae40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f5720007fd0 tx=0x7f57200074f0 comp rx=0 tx=0).stop 2024-05-17T22:04:03.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:03.817+0000 7f5735c43700 1 -- 172.21.15.107:0/242355842 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57301026b0 msgr2=0x7f57301030b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:03.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:03.817+0000 7f5735c43700 1 --2- 172.21.15.107:0/242355842 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57301026b0 0x7f57301030b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f5724009180 tx=0x7f5724009e30 comp rx=0 tx=0).stop 2024-05-17T22:04:03.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:03.817+0000 7f5735c43700 1 -- 172.21.15.107:0/242355842 shutdown_connections 2024-05-17T22:04:03.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:03.817+0000 7f5735c43700 1 --2- 172.21.15.107:0/242355842 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f5718078990 0x7f571807ae40 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:03.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:03.817+0000 7f5735c43700 1 --2- 172.21.15.107:0/242355842 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5730101d10 0x7f5730102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:03.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:03.817+0000 7f5735c43700 1 --2- 172.21.15.107:0/242355842 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57301026b0 0x7f57301030b0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:03.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:03.817+0000 7f5735c43700 1 -- 172.21.15.107:0/242355842 >> 172.21.15.107:0/242355842 conn(0x7f57300f7860 msgr2=0x7f57300f8770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:04:03.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:03.818+0000 7f5735c43700 1 -- 172.21.15.107:0/242355842 shutdown_connections 2024-05-17T22:04:03.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:03.818+0000 7f5735c43700 1 -- 172.21.15.107:0/242355842 wait complete. 2024-05-17T22:04:03.810 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 20 2024-05-17T22:04:04.100 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:04:03 smithi107 ceph-mon[93224]: pgmap v322: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:04:04.100 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:04:03 smithi107 ceph-mon[93224]: from='client.? 172.21.15.107:0/242355842' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 20, "format": "json"}]: dispatch 2024-05-17T22:04:04.324 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:04:03 smithi138 ceph-mon[81376]: pgmap v322: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:04:04.324 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:04:03 smithi138 ceph-mon[81376]: from='client.? 172.21.15.107:0/242355842' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 20, "format": "json"}]: dispatch 2024-05-17T22:04:04.494 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 20 2024-05-17T22:04:04.494 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 21 2024-05-17T22:04:04.753 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/mon.smithi107/config 2024-05-17T22:04:05.171 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:04:04 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T22:04:05.324 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:04:04 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T22:04:06.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:04:05 smithi107 ceph-mon[93224]: pgmap v323: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:04:06.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:06.304+0000 7f09f10c4700 1 -- 172.21.15.107:0/3491577588 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09ec102de0 msgr2=0x7f09ec1031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:06.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:06.304+0000 7f09f10c4700 1 --2- 172.21.15.107:0/3491577588 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09ec102de0 0x7f09ec1031b0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f09dc004940 tx=0x7f09dc009e30 comp rx=0 tx=0).stop 2024-05-17T22:04:06.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:06.304+0000 7f09f10c4700 1 -- 172.21.15.107:0/3491577588 shutdown_connections 2024-05-17T22:04:06.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:06.304+0000 7f09f10c4700 1 --2- 172.21.15.107:0/3491577588 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f09ec103780 0x7f09ec0fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:06.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:06.304+0000 7f09f10c4700 1 --2- 172.21.15.107:0/3491577588 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09ec102de0 0x7f09ec1031b0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:06.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:06.304+0000 7f09f10c4700 1 -- 172.21.15.107:0/3491577588 >> 172.21.15.107:0/3491577588 conn(0x7f09ec0f8990 msgr2=0x7f09ec0fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:04:06.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:06.304+0000 7f09f10c4700 1 -- 172.21.15.107:0/3491577588 shutdown_connections 2024-05-17T22:04:06.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:06.304+0000 7f09f10c4700 1 -- 172.21.15.107:0/3491577588 wait complete. 2024-05-17T22:04:06.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:06.305+0000 7f09f10c4700 1 Processor -- start 2024-05-17T22:04:06.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:06.305+0000 7f09f10c4700 1 -- start start 2024-05-17T22:04:06.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:06.305+0000 7f09f10c4700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09ec102de0 0x7f09ec113970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:06.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:06.305+0000 7f09f10c4700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f09ec103780 0x7f09ec113eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:06.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:06.305+0000 7f09f10c4700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09ec114500 con 0x7f09ec102de0 2024-05-17T22:04:06.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:06.305+0000 7f09f10c4700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09ec10db50 con 0x7f09ec103780 2024-05-17T22:04:06.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:06.305+0000 7f09ead9d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09ec102de0 0x7f09ec113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:06.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:06.305+0000 7f09ead9d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09ec102de0 0x7f09ec113970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42762/0 (socket says 172.21.15.107:42762) 2024-05-17T22:04:06.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:06.305+0000 7f09ead9d700 1 -- 172.21.15.107:0/33537583 learned_addr learned my addr 172.21.15.107:0/33537583 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T22:04:06.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:06.306+0000 7f09ea59c700 1 --2- 172.21.15.107:0/33537583 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f09ec103780 0x7f09ec113eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:06.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:06.306+0000 7f09ead9d700 1 -- 172.21.15.107:0/33537583 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f09ec103780 msgr2=0x7f09ec113eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:06.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:06.306+0000 7f09ead9d700 1 --2- 172.21.15.107:0/33537583 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f09ec103780 0x7f09ec113eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:06.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:06.306+0000 7f09ead9d700 1 -- 172.21.15.107:0/33537583 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09dc005040 con 0x7f09ec102de0 2024-05-17T22:04:06.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:06.306+0000 7f09ead9d700 1 --2- 172.21.15.107:0/33537583 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09ec102de0 0x7f09ec113970 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f09dc00b3b0 tx=0x7f09dc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:06.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:06.306+0000 7f09d37fe700 1 -- 172.21.15.107:0/33537583 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f09dc00e040 con 0x7f09ec102de0 2024-05-17T22:04:06.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:06.306+0000 7f09f10c4700 1 -- 172.21.15.107:0/33537583 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09ec10ddd0 con 0x7f09ec102de0 2024-05-17T22:04:06.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:06.306+0000 7f09d37fe700 1 -- 172.21.15.107:0/33537583 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f09dc013420 con 0x7f09ec102de0 2024-05-17T22:04:06.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:06.306+0000 7f09f10c4700 1 -- 172.21.15.107:0/33537583 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09ec10e390 con 0x7f09ec102de0 2024-05-17T22:04:06.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:06.306+0000 7f09d37fe700 1 -- 172.21.15.107:0/33537583 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f09dc01c630 con 0x7f09ec102de0 2024-05-17T22:04:06.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:06.307+0000 7f09f10c4700 1 -- 172.21.15.107:0/33537583 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09ec1000f0 con 0x7f09ec102de0 2024-05-17T22:04:06.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:06.308+0000 7f09d37fe700 1 -- 172.21.15.107:0/33537583 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f09dc013590 con 0x7f09ec102de0 2024-05-17T22:04:06.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:06.308+0000 7f09d37fe700 1 --2- 172.21.15.107:0/33537583 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f09d4078990 0x7f09d407ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:06.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:06.308+0000 7f09d37fe700 1 -- 172.21.15.107:0/33537583 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f09dc09b990 con 0x7f09ec102de0 2024-05-17T22:04:06.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:06.308+0000 7f09ea59c700 1 --2- 172.21.15.107:0/33537583 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f09d4078990 0x7f09d407ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:06.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:06.309+0000 7f09ea59c700 1 --2- 172.21.15.107:0/33537583 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f09d4078990 0x7f09d407ae40 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f09ec10ecc0 tx=0x7f09e000b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:06.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:06.313+0000 7f09d37fe700 1 -- 172.21.15.107:0/33537583 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f09dc0635d0 con 0x7f09ec102de0 2024-05-17T22:04:06.324 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:04:05 smithi138 ceph-mon[81376]: pgmap v323: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:04:06.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:06.565+0000 7f09f10c4700 1 -- 172.21.15.107:0/33537583 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 21, "format": "json"} v 0) v1 -- 0x7f09ec0fe530 con 0x7f09ec102de0 2024-05-17T22:04:06.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:06.565+0000 7f09d37fe700 1 -- 172.21.15.107:0/33537583 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 21, "format": "json"}]=0 dumped fsmap epoch 21 v35) v1 ==== 107+0+5009 (secure 0 0 0) 0x7f09dc062f70 con 0x7f09ec102de0 2024-05-17T22:04:06.557 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T22:04:06.557 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24267,"name":"cephfs.smithi138.zckrvs","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.138:6827/170628661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":170628661},{"type":"v1","addr":"172.21.15.138:6827","nonce":170628661}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":34408,"name":"cephfs.smithi107.idmbgj","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6827/2235359521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":2235359521},{"type":"v1","addr":"172.21.15.107:6827","nonce":2235359521}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":34440,"name":"cephfs.smithi107.drqody","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6829/369507752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":369507752},{"type":"v1","addr":"172.21.15.107:6829","nonce":369507752}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T21:49:16.836333+0000","modified":"2024-05-17T21:59:13.802503+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24259},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24259":{"gid":24259,"name":"cephfs.smithi138.gwbtyd","rank":0,"incarnation":12,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6825/2779634919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":2779634919},{"type":"v1","addr":"172.21.15.138:6825","nonce":2779634919}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24259,"qdb_cluster":[24259]},"id":1}]} 2024-05-17T22:04:06.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:06.567+0000 7f09f10c4700 1 -- 172.21.15.107:0/33537583 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f09d4078990 msgr2=0x7f09d407ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:06.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:06.567+0000 7f09f10c4700 1 --2- 172.21.15.107:0/33537583 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f09d4078990 0x7f09d407ae40 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f09ec10ecc0 tx=0x7f09e000b040 comp rx=0 tx=0).stop 2024-05-17T22:04:06.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:06.568+0000 7f09f10c4700 1 -- 172.21.15.107:0/33537583 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09ec102de0 msgr2=0x7f09ec113970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:06.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:06.568+0000 7f09f10c4700 1 --2- 172.21.15.107:0/33537583 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09ec102de0 0x7f09ec113970 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f09dc00b3b0 tx=0x7f09dc009e30 comp rx=0 tx=0).stop 2024-05-17T22:04:06.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:06.568+0000 7f09f10c4700 1 -- 172.21.15.107:0/33537583 shutdown_connections 2024-05-17T22:04:06.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:06.568+0000 7f09f10c4700 1 --2- 172.21.15.107:0/33537583 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f09d4078990 0x7f09d407ae40 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:06.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:06.568+0000 7f09f10c4700 1 --2- 172.21.15.107:0/33537583 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09ec102de0 0x7f09ec113970 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:06.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:06.568+0000 7f09f10c4700 1 --2- 172.21.15.107:0/33537583 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f09ec103780 0x7f09ec113eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:06.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:06.568+0000 7f09f10c4700 1 -- 172.21.15.107:0/33537583 >> 172.21.15.107:0/33537583 conn(0x7f09ec0f8990 msgr2=0x7f09ec107380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:04:06.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:06.568+0000 7f09f10c4700 1 -- 172.21.15.107:0/33537583 shutdown_connections 2024-05-17T22:04:06.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:06.568+0000 7f09f10c4700 1 -- 172.21.15.107:0/33537583 wait complete. 2024-05-17T22:04:06.560 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 21 2024-05-17T22:04:07.133 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 21 2024-05-17T22:04:07.133 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 22 2024-05-17T22:04:07.159 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:04:06 smithi107 ceph-mon[93224]: from='client.? 172.21.15.107:0/33537583' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 21, "format": "json"}]: dispatch 2024-05-17T22:04:07.324 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:04:06 smithi138 ceph-mon[81376]: from='client.? 172.21.15.107:0/33537583' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 21, "format": "json"}]: dispatch 2024-05-17T22:04:07.392 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/mon.smithi107/config 2024-05-17T22:04:08.100 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:04:07 smithi107 ceph-mon[93224]: pgmap v324: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:04:08.324 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:04:07 smithi138 ceph-mon[81376]: pgmap v324: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:04:08.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:08.988+0000 7f7c070e9700 1 -- 172.21.15.107:0/1035597128 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c00101d10 msgr2=0x7f7c001020e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:08.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:08.988+0000 7f7c070e9700 1 --2- 172.21.15.107:0/1035597128 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c00101d10 0x7f7c001020e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7bf4009ce0 tx=0x7f7bf4004d10 comp rx=0 tx=0).stop 2024-05-17T22:04:08.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:08.988+0000 7f7c070e9700 1 -- 172.21.15.107:0/1035597128 shutdown_connections 2024-05-17T22:04:08.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:08.988+0000 7f7c070e9700 1 --2- 172.21.15.107:0/1035597128 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c001026b0 0x7f7c000fbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:08.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:08.988+0000 7f7c070e9700 1 --2- 172.21.15.107:0/1035597128 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c00101d10 0x7f7c001020e0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:08.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:08.988+0000 7f7c070e9700 1 -- 172.21.15.107:0/1035597128 >> 172.21.15.107:0/1035597128 conn(0x7f7c000f7860 msgr2=0x7f7c000f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:04:08.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:08.988+0000 7f7c070e9700 1 -- 172.21.15.107:0/1035597128 shutdown_connections 2024-05-17T22:04:08.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:08.988+0000 7f7c070e9700 1 -- 172.21.15.107:0/1035597128 wait complete. 2024-05-17T22:04:08.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:08.988+0000 7f7c070e9700 1 Processor -- start 2024-05-17T22:04:08.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:08.988+0000 7f7c070e9700 1 -- start start 2024-05-17T22:04:08.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:08.989+0000 7f7c070e9700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c00101d10 0x7f7c00113b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:08.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:08.989+0000 7f7c070e9700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c001026b0 0x7f7c00114060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:08.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:08.989+0000 7f7c070e9700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c001146b0 con 0x7f7c001026b0 2024-05-17T22:04:08.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:08.989+0000 7f7c070e9700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c0010dc90 con 0x7f7c00101d10 2024-05-17T22:04:08.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:08.989+0000 7f7bfffff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c001026b0 0x7f7c00114060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:08.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:08.989+0000 7f7bfffff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c001026b0 0x7f7c00114060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42784/0 (socket says 172.21.15.107:42784) 2024-05-17T22:04:08.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:08.989+0000 7f7bfffff700 1 -- 172.21.15.107:0/2695161827 learned_addr learned my addr 172.21.15.107:0/2695161827 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T22:04:08.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:08.989+0000 7f7c04e85700 1 --2- 172.21.15.107:0/2695161827 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c00101d10 0x7f7c00113b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:08.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:08.989+0000 7f7bfffff700 1 -- 172.21.15.107:0/2695161827 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c00101d10 msgr2=0x7f7c00113b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:08.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:08.989+0000 7f7bfffff700 1 --2- 172.21.15.107:0/2695161827 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c00101d10 0x7f7c00113b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:08.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:08.989+0000 7f7bfffff700 1 -- 172.21.15.107:0/2695161827 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bf4009940 con 0x7f7c001026b0 2024-05-17T22:04:08.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:08.989+0000 7f7bfffff700 1 --2- 172.21.15.107:0/2695161827 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c001026b0 0x7f7c00114060 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f7bec00c800 tx=0x7f7bec004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:08.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:08.990+0000 7f7bfd7fa700 1 -- 172.21.15.107:0/2695161827 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7bec0106f0 con 0x7f7c001026b0 2024-05-17T22:04:08.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:08.990+0000 7f7bfd7fa700 1 -- 172.21.15.107:0/2695161827 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7bec010d20 con 0x7f7c001026b0 2024-05-17T22:04:08.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:08.990+0000 7f7c070e9700 1 -- 172.21.15.107:0/2695161827 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c0010df70 con 0x7f7c001026b0 2024-05-17T22:04:08.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:08.990+0000 7f7bfd7fa700 1 -- 172.21.15.107:0/2695161827 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7bec0113a0 con 0x7f7c001026b0 2024-05-17T22:04:08.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:08.990+0000 7f7c070e9700 1 -- 172.21.15.107:0/2695161827 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c0010e530 con 0x7f7c001026b0 2024-05-17T22:04:08.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:08.991+0000 7f7c070e9700 1 -- 172.21.15.107:0/2695161827 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c000fd460 con 0x7f7c001026b0 2024-05-17T22:04:08.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:08.991+0000 7f7bfd7fa700 1 -- 172.21.15.107:0/2695161827 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f7bec012400 con 0x7f7c001026b0 2024-05-17T22:04:08.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:08.992+0000 7f7bfd7fa700 1 --2- 172.21.15.107:0/2695161827 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f7bf0078990 0x7f7bf007ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:08.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:08.992+0000 7f7bfd7fa700 1 -- 172.21.15.107:0/2695161827 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f7bec09a730 con 0x7f7c001026b0 2024-05-17T22:04:08.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:08.992+0000 7f7c04e85700 1 --2- 172.21.15.107:0/2695161827 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f7bf0078990 0x7f7bf007ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:08.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:08.992+0000 7f7c04e85700 1 --2- 172.21.15.107:0/2695161827 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f7bf0078990 0x7f7bf007ae40 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7bf400b740 tx=0x7f7bf4004d10 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:08.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:08.996+0000 7f7bfd7fa700 1 -- 172.21.15.107:0/2695161827 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f7bec062370 con 0x7f7c001026b0 2024-05-17T22:04:09.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:09.245+0000 7f7c070e9700 1 -- 172.21.15.107:0/2695161827 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 22, "format": "json"} v 0) v1 -- 0x7f7c0004c710 con 0x7f7c001026b0 2024-05-17T22:04:09.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:09.245+0000 7f7bfd7fa700 1 -- 172.21.15.107:0/2695161827 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 22, "format": "json"}]=0 dumped fsmap epoch 22 v35) v1 ==== 107+0+4208 (secure 0 0 0) 0x7f7bec061d10 con 0x7f7c001026b0 2024-05-17T22:04:09.237 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T22:04:09.237 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24267,"name":"cephfs.smithi138.zckrvs","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.138:6827/170628661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":170628661},{"type":"v1","addr":"172.21.15.138:6827","nonce":170628661}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":34408,"name":"cephfs.smithi107.idmbgj","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6827/2235359521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":2235359521},{"type":"v1","addr":"172.21.15.107:6827","nonce":2235359521}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":34440,"name":"cephfs.smithi107.drqody","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6829/369507752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":369507752},{"type":"v1","addr":"172.21.15.107:6829","nonce":369507752}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T21:49:16.836333+0000","modified":"2024-05-17T21:59:52.171619+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-05-17T22:04:09.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:09.248+0000 7f7c070e9700 1 -- 172.21.15.107:0/2695161827 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f7bf0078990 msgr2=0x7f7bf007ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:09.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:09.248+0000 7f7c070e9700 1 --2- 172.21.15.107:0/2695161827 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f7bf0078990 0x7f7bf007ae40 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7bf400b740 tx=0x7f7bf4004d10 comp rx=0 tx=0).stop 2024-05-17T22:04:09.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:09.248+0000 7f7c070e9700 1 -- 172.21.15.107:0/2695161827 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c001026b0 msgr2=0x7f7c00114060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:09.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:09.248+0000 7f7c070e9700 1 --2- 172.21.15.107:0/2695161827 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c001026b0 0x7f7c00114060 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f7bec00c800 tx=0x7f7bec004470 comp rx=0 tx=0).stop 2024-05-17T22:04:09.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:09.248+0000 7f7c070e9700 1 -- 172.21.15.107:0/2695161827 shutdown_connections 2024-05-17T22:04:09.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:09.248+0000 7f7c070e9700 1 --2- 172.21.15.107:0/2695161827 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f7bf0078990 0x7f7bf007ae40 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:09.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:09.248+0000 7f7c070e9700 1 --2- 172.21.15.107:0/2695161827 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c00101d10 0x7f7c00113b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:09.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:09.248+0000 7f7c070e9700 1 --2- 172.21.15.107:0/2695161827 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c001026b0 0x7f7c00114060 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:09.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:09.248+0000 7f7c070e9700 1 -- 172.21.15.107:0/2695161827 >> 172.21.15.107:0/2695161827 conn(0x7f7c000f7860 msgr2=0x7f7c000f8f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:04:09.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:09.248+0000 7f7c070e9700 1 -- 172.21.15.107:0/2695161827 shutdown_connections 2024-05-17T22:04:09.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:09.248+0000 7f7c070e9700 1 -- 172.21.15.107:0/2695161827 wait complete. 2024-05-17T22:04:09.240 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 22 2024-05-17T22:04:09.858 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 22 2024-05-17T22:04:09.858 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 23 2024-05-17T22:04:10.100 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:04:09 smithi107 ceph-mon[93224]: pgmap v325: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:04:10.100 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:04:09 smithi107 ceph-mon[93224]: from='client.? 172.21.15.107:0/2695161827' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-05-17T22:04:10.156 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/mon.smithi107/config 2024-05-17T22:04:10.324 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:04:09 smithi138 ceph-mon[81376]: pgmap v325: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:04:10.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:04:09 smithi138 ceph-mon[81376]: from='client.? 172.21.15.107:0/2695161827' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-05-17T22:04:11.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:11.651+0000 7fc935b43700 1 -- 172.21.15.107:0/3092016509 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc930102670 msgr2=0x7fc9300fbce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:11.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:11.651+0000 7fc935b43700 1 --2- 172.21.15.107:0/3092016509 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc930102670 0x7fc9300fbce0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc918004940 tx=0x7fc918009e30 comp rx=0 tx=0).stop 2024-05-17T22:04:11.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:11.652+0000 7fc935b43700 1 -- 172.21.15.107:0/3092016509 shutdown_connections 2024-05-17T22:04:11.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:11.652+0000 7fc935b43700 1 --2- 172.21.15.107:0/3092016509 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc930102670 0x7fc9300fbce0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:11.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:11.652+0000 7fc935b43700 1 --2- 172.21.15.107:0/3092016509 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc930101cd0 0x7fc9301020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:11.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:11.652+0000 7fc935b43700 1 -- 172.21.15.107:0/3092016509 >> 172.21.15.107:0/3092016509 conn(0x7fc9300f7860 msgr2=0x7fc9300f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:04:11.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:11.652+0000 7fc935b43700 1 -- 172.21.15.107:0/3092016509 shutdown_connections 2024-05-17T22:04:11.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:11.652+0000 7fc935b43700 1 -- 172.21.15.107:0/3092016509 wait complete. 2024-05-17T22:04:11.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:11.652+0000 7fc935b43700 1 Processor -- start 2024-05-17T22:04:11.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:11.652+0000 7fc935b43700 1 -- start start 2024-05-17T22:04:11.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:11.652+0000 7fc935b43700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc930101cd0 0x7fc9300fd950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:11.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:11.652+0000 7fc935b43700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc930102670 0x7fc9300fde90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:11.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:11.652+0000 7fc935b43700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9300ff8a0 con 0x7fc930101cd0 2024-05-17T22:04:11.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:11.652+0000 7fc935b43700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9300fe3d0 con 0x7fc930102670 2024-05-17T22:04:11.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:11.652+0000 7fc92f7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc930101cd0 0x7fc9300fd950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:11.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:11.653+0000 7fc92f7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc930101cd0 0x7fc9300fd950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42804/0 (socket says 172.21.15.107:42804) 2024-05-17T22:04:11.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:11.653+0000 7fc92f7fe700 1 -- 172.21.15.107:0/8796562 learned_addr learned my addr 172.21.15.107:0/8796562 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T22:04:11.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:11.653+0000 7fc927fff700 1 --2- 172.21.15.107:0/8796562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc930102670 0x7fc9300fde90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:11.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:11.653+0000 7fc92f7fe700 1 -- 172.21.15.107:0/8796562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc930102670 msgr2=0x7fc9300fde90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:11.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:11.653+0000 7fc92f7fe700 1 --2- 172.21.15.107:0/8796562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc930102670 0x7fc9300fde90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:11.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:11.653+0000 7fc92f7fe700 1 -- 172.21.15.107:0/8796562 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc918005040 con 0x7fc930101cd0 2024-05-17T22:04:11.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:11.653+0000 7fc927fff700 1 --2- 172.21.15.107:0/8796562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc930102670 0x7fc9300fde90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:04:11.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:11.653+0000 7fc92f7fe700 1 --2- 172.21.15.107:0/8796562 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc930101cd0 0x7fc9300fd950 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc920002a50 tx=0x7fc9200046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:11.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:11.653+0000 7fc92cff9700 1 -- 172.21.15.107:0/8796562 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc920007cc0 con 0x7fc930101cd0 2024-05-17T22:04:11.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:11.653+0000 7fc92cff9700 1 -- 172.21.15.107:0/8796562 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc92000f040 con 0x7fc930101cd0 2024-05-17T22:04:11.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:11.653+0000 7fc935b43700 1 -- 172.21.15.107:0/8796562 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9300fe6b0 con 0x7fc930101cd0 2024-05-17T22:04:11.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:11.653+0000 7fc92cff9700 1 -- 172.21.15.107:0/8796562 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc920009640 con 0x7fc930101cd0 2024-05-17T22:04:11.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:11.653+0000 7fc935b43700 1 -- 172.21.15.107:0/8796562 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc93011a130 con 0x7fc930101cd0 2024-05-17T22:04:11.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:11.654+0000 7fc935b43700 1 -- 172.21.15.107:0/8796562 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9300fd420 con 0x7fc930101cd0 2024-05-17T22:04:11.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:11.655+0000 7fc92cff9700 1 -- 172.21.15.107:0/8796562 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7fc9200097c0 con 0x7fc930101cd0 2024-05-17T22:04:11.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:11.655+0000 7fc92cff9700 1 --2- 172.21.15.107:0/8796562 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fc91c078990 0x7fc91c07ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:11.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:11.655+0000 7fc92cff9700 1 -- 172.21.15.107:0/8796562 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7fc92009a560 con 0x7fc930101cd0 2024-05-17T22:04:11.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:11.655+0000 7fc927fff700 1 --2- 172.21.15.107:0/8796562 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fc91c078990 0x7fc91c07ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:11.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:11.656+0000 7fc927fff700 1 --2- 172.21.15.107:0/8796562 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fc91c078990 0x7fc91c07ae40 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc91800b3b0 tx=0x7fc918009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:11.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:11.660+0000 7fc92cff9700 1 -- 172.21.15.107:0/8796562 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7fc920062120 con 0x7fc930101cd0 2024-05-17T22:04:11.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:11.907+0000 7fc935b43700 1 -- 172.21.15.107:0/8796562 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 23, "format": "json"} v 0) v1 -- 0x7fc93011a3e0 con 0x7fc930101cd0 2024-05-17T22:04:11.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:11.908+0000 7fc92cff9700 1 -- 172.21.15.107:0/8796562 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 23, "format": "json"}]=0 dumped fsmap epoch 23 v35) v1 ==== 107+0+4219 (secure 0 0 0) 0x7fc920061ac0 con 0x7fc930101cd0 2024-05-17T22:04:11.899 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T22:04:11.899 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34408,"name":"cephfs.smithi107.idmbgj","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6827/2235359521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":2235359521},{"type":"v1","addr":"172.21.15.107:6827","nonce":2235359521}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":34440,"name":"cephfs.smithi107.drqody","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6829/369507752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":369507752},{"type":"v1","addr":"172.21.15.107:6829","nonce":369507752}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T21:49:16.836333+0000","modified":"2024-05-17T21:59:52.177625+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24267},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24267":{"gid":24267,"name":"cephfs.smithi138.zckrvs","rank":0,"incarnation":23,"state":"up:replay","state_seq":2,"addr":"172.21.15.138:6827/170628661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":170628661},{"type":"v1","addr":"172.21.15.138:6827","nonce":170628661}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-05-17T22:04:11.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:11.910+0000 7fc935b43700 1 -- 172.21.15.107:0/8796562 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fc91c078990 msgr2=0x7fc91c07ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:11.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:11.910+0000 7fc935b43700 1 --2- 172.21.15.107:0/8796562 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fc91c078990 0x7fc91c07ae40 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc91800b3b0 tx=0x7fc918009e30 comp rx=0 tx=0).stop 2024-05-17T22:04:11.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:11.910+0000 7fc935b43700 1 -- 172.21.15.107:0/8796562 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc930101cd0 msgr2=0x7fc9300fd950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:11.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:11.910+0000 7fc935b43700 1 --2- 172.21.15.107:0/8796562 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc930101cd0 0x7fc9300fd950 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc920002a50 tx=0x7fc9200046b0 comp rx=0 tx=0).stop 2024-05-17T22:04:11.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:11.910+0000 7fc935b43700 1 -- 172.21.15.107:0/8796562 shutdown_connections 2024-05-17T22:04:11.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:11.910+0000 7fc935b43700 1 --2- 172.21.15.107:0/8796562 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fc91c078990 0x7fc91c07ae40 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:11.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:11.910+0000 7fc935b43700 1 --2- 172.21.15.107:0/8796562 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc930101cd0 0x7fc9300fd950 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:11.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:11.910+0000 7fc935b43700 1 --2- 172.21.15.107:0/8796562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc930102670 0x7fc9300fde90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:11.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:11.910+0000 7fc935b43700 1 -- 172.21.15.107:0/8796562 >> 172.21.15.107:0/8796562 conn(0x7fc9300f7860 msgr2=0x7fc9300f8f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:04:11.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:11.910+0000 7fc935b43700 1 -- 172.21.15.107:0/8796562 shutdown_connections 2024-05-17T22:04:11.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:11.910+0000 7fc935b43700 1 -- 172.21.15.107:0/8796562 wait complete. 2024-05-17T22:04:11.903 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 23 2024-05-17T22:04:12.214 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:04:11 smithi107 ceph-mon[93224]: pgmap v326: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:04:12.215 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:04:11 smithi107 ceph-mon[93224]: from='client.? 172.21.15.107:0/8796562' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-05-17T22:04:12.324 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:04:11 smithi138 ceph-mon[81376]: pgmap v326: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:04:12.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:04:11 smithi138 ceph-mon[81376]: from='client.? 172.21.15.107:0/8796562' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-05-17T22:04:12.552 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 23 2024-05-17T22:04:12.552 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 24 2024-05-17T22:04:12.810 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/mon.smithi107/config 2024-05-17T22:04:14.175 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:04:13 smithi107 ceph-mon[93224]: pgmap v327: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:04:14.324 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:04:13 smithi138 ceph-mon[81376]: pgmap v327: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:04:14.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:14.385+0000 7f79a1428700 1 -- 172.21.15.107:0/1755209073 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f799c103780 msgr2=0x7f799c0fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:14.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:14.385+0000 7f79a1428700 1 --2- 172.21.15.107:0/1755209073 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f799c103780 0x7f799c0fcdf0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f7990004940 tx=0x7f7990009e30 comp rx=0 tx=0).stop 2024-05-17T22:04:14.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:14.385+0000 7f79a1428700 1 -- 172.21.15.107:0/1755209073 shutdown_connections 2024-05-17T22:04:14.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:14.385+0000 7f79a1428700 1 --2- 172.21.15.107:0/1755209073 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f799c103780 0x7f799c0fcdf0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:14.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:14.385+0000 7f79a1428700 1 --2- 172.21.15.107:0/1755209073 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f799c102de0 0x7f799c1031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:14.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:14.385+0000 7f79a1428700 1 -- 172.21.15.107:0/1755209073 >> 172.21.15.107:0/1755209073 conn(0x7f799c0f8990 msgr2=0x7f799c0fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:04:14.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:14.385+0000 7f79a1428700 1 -- 172.21.15.107:0/1755209073 shutdown_connections 2024-05-17T22:04:14.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:14.385+0000 7f79a1428700 1 -- 172.21.15.107:0/1755209073 wait complete. 2024-05-17T22:04:14.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:14.386+0000 7f79a1428700 1 Processor -- start 2024-05-17T22:04:14.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:14.386+0000 7f79a1428700 1 -- start start 2024-05-17T22:04:14.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:14.386+0000 7f79a1428700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f799c102de0 0x7f799c113990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:14.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:14.386+0000 7f79a1428700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f799c103780 0x7f799c113ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:14.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:14.386+0000 7f79a1428700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f799c114520 con 0x7f799c102de0 2024-05-17T22:04:14.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:14.386+0000 7f79a1428700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f799c10db20 con 0x7f799c103780 2024-05-17T22:04:14.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:14.386+0000 7f799affd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f799c102de0 0x7f799c113990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:14.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:14.386+0000 7f799affd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f799c102de0 0x7f799c113990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42834/0 (socket says 172.21.15.107:42834) 2024-05-17T22:04:14.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:14.386+0000 7f799affd700 1 -- 172.21.15.107:0/3318583983 learned_addr learned my addr 172.21.15.107:0/3318583983 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T22:04:14.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:14.387+0000 7f799a7fc700 1 --2- 172.21.15.107:0/3318583983 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f799c103780 0x7f799c113ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:14.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:14.387+0000 7f799affd700 1 -- 172.21.15.107:0/3318583983 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f799c103780 msgr2=0x7f799c113ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:14.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:14.387+0000 7f799affd700 1 --2- 172.21.15.107:0/3318583983 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f799c103780 0x7f799c113ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:14.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:14.387+0000 7f799affd700 1 -- 172.21.15.107:0/3318583983 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7990005040 con 0x7f799c102de0 2024-05-17T22:04:14.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:14.387+0000 7f799affd700 1 --2- 172.21.15.107:0/3318583983 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f799c102de0 0x7f799c113990 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f798c00c830 tx=0x7f798c004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:14.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:14.387+0000 7f79837fe700 1 -- 172.21.15.107:0/3318583983 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f798c0106f0 con 0x7f799c102de0 2024-05-17T22:04:14.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:14.387+0000 7f79a1428700 1 -- 172.21.15.107:0/3318583983 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f799c10dda0 con 0x7f799c102de0 2024-05-17T22:04:14.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:14.387+0000 7f79837fe700 1 -- 172.21.15.107:0/3318583983 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f798c010d20 con 0x7f799c102de0 2024-05-17T22:04:14.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:14.387+0000 7f79837fe700 1 -- 172.21.15.107:0/3318583983 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f798c0113a0 con 0x7f799c102de0 2024-05-17T22:04:14.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:14.387+0000 7f79a1428700 1 -- 172.21.15.107:0/3318583983 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f799c10e330 con 0x7f799c102de0 2024-05-17T22:04:14.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:14.388+0000 7f79a1428700 1 -- 172.21.15.107:0/3318583983 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f799c0fe530 con 0x7f799c102de0 2024-05-17T22:04:14.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:14.389+0000 7f79837fe700 1 -- 172.21.15.107:0/3318583983 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f798c010850 con 0x7f799c102de0 2024-05-17T22:04:14.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:14.389+0000 7f79837fe700 1 --2- 172.21.15.107:0/3318583983 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f7984078990 0x7f798407ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:14.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:14.389+0000 7f79837fe700 1 -- 172.21.15.107:0/3318583983 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f798c09a780 con 0x7f799c102de0 2024-05-17T22:04:14.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:14.389+0000 7f799a7fc700 1 --2- 172.21.15.107:0/3318583983 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f7984078990 0x7f798407ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:14.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:14.390+0000 7f799a7fc700 1 --2- 172.21.15.107:0/3318583983 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f7984078990 0x7f798407ae40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7990004890 tx=0x7f7990009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:14.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:14.394+0000 7f79837fe700 1 -- 172.21.15.107:0/3318583983 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f798c0623c0 con 0x7f799c102de0 2024-05-17T22:04:14.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:14.646+0000 7f79a1428700 1 -- 172.21.15.107:0/3318583983 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 24, "format": "json"} v 0) v1 -- 0x7f799c109f10 con 0x7f799c102de0 2024-05-17T22:04:14.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:14.647+0000 7f79837fe700 1 -- 172.21.15.107:0/3318583983 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 24, "format": "json"}]=0 dumped fsmap epoch 24 v35) v1 ==== 107+0+4224 (secure 0 0 0) 0x7f798c061d60 con 0x7f799c102de0 2024-05-17T22:04:14.639 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T22:04:14.639 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34408,"name":"cephfs.smithi107.idmbgj","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6827/2235359521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":2235359521},{"type":"v1","addr":"172.21.15.107:6827","nonce":2235359521}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":34440,"name":"cephfs.smithi107.drqody","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6829/369507752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":369507752},{"type":"v1","addr":"172.21.15.107:6829","nonce":369507752}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T21:49:16.836333+0000","modified":"2024-05-17T22:00:01.503815+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24267},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24267":{"gid":24267,"name":"cephfs.smithi138.zckrvs","rank":0,"incarnation":23,"state":"up:reconnect","state_seq":159,"addr":"172.21.15.138:6827/170628661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":170628661},{"type":"v1","addr":"172.21.15.138:6827","nonce":170628661}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-05-17T22:04:14.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:14.649+0000 7f79a1428700 1 -- 172.21.15.107:0/3318583983 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f7984078990 msgr2=0x7f798407ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:14.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:14.649+0000 7f79a1428700 1 --2- 172.21.15.107:0/3318583983 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f7984078990 0x7f798407ae40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7990004890 tx=0x7f7990009e30 comp rx=0 tx=0).stop 2024-05-17T22:04:14.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:14.649+0000 7f79a1428700 1 -- 172.21.15.107:0/3318583983 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f799c102de0 msgr2=0x7f799c113990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:14.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:14.649+0000 7f79a1428700 1 --2- 172.21.15.107:0/3318583983 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f799c102de0 0x7f799c113990 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f798c00c830 tx=0x7f798c004470 comp rx=0 tx=0).stop 2024-05-17T22:04:14.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:14.649+0000 7f79a1428700 1 -- 172.21.15.107:0/3318583983 shutdown_connections 2024-05-17T22:04:14.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:14.649+0000 7f79a1428700 1 --2- 172.21.15.107:0/3318583983 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f7984078990 0x7f798407ae40 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:14.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:14.649+0000 7f79a1428700 1 --2- 172.21.15.107:0/3318583983 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f799c102de0 0x7f799c113990 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:14.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:14.649+0000 7f79a1428700 1 --2- 172.21.15.107:0/3318583983 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f799c103780 0x7f799c113ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:14.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:14.649+0000 7f79a1428700 1 -- 172.21.15.107:0/3318583983 >> 172.21.15.107:0/3318583983 conn(0x7f799c0f8990 msgr2=0x7f799c0f9970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:04:14.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:14.649+0000 7f79a1428700 1 -- 172.21.15.107:0/3318583983 shutdown_connections 2024-05-17T22:04:14.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:14.649+0000 7f79a1428700 1 -- 172.21.15.107:0/3318583983 wait complete. 2024-05-17T22:04:14.641 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 24 2024-05-17T22:04:15.315 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:04:14 smithi107 ceph-mon[93224]: from='client.? 172.21.15.107:0/3318583983' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-05-17T22:04:15.324 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:04:14 smithi138 ceph-mon[81376]: from='client.? 172.21.15.107:0/3318583983' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-05-17T22:04:15.340 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 24 2024-05-17T22:04:15.341 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 25 2024-05-17T22:04:15.599 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/mon.smithi107/config 2024-05-17T22:04:16.026 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:04:15 smithi107 ceph-mon[93224]: pgmap v328: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:04:16.324 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:04:15 smithi138 ceph-mon[81376]: pgmap v328: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:04:17.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:17.304+0000 7f5dc7095700 1 -- 172.21.15.107:0/344196453 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5dc0101ce0 msgr2=0x7f5dc01020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:17.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:17.304+0000 7f5dc7095700 1 --2- 172.21.15.107:0/344196453 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5dc0101ce0 0x7f5dc01020b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f5db4004940 tx=0x7f5db4009e30 comp rx=0 tx=0).stop 2024-05-17T22:04:17.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:17.304+0000 7f5dc7095700 1 -- 172.21.15.107:0/344196453 shutdown_connections 2024-05-17T22:04:17.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:17.304+0000 7f5dc7095700 1 --2- 172.21.15.107:0/344196453 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5dc0102680 0x7f5dc00fbcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:17.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:17.304+0000 7f5dc7095700 1 --2- 172.21.15.107:0/344196453 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5dc0101ce0 0x7f5dc01020b0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:17.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:17.304+0000 7f5dc7095700 1 -- 172.21.15.107:0/344196453 >> 172.21.15.107:0/344196453 conn(0x7f5dc00f7890 msgr2=0x7f5dc00f9ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:04:17.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:17.304+0000 7f5dc7095700 1 -- 172.21.15.107:0/344196453 shutdown_connections 2024-05-17T22:04:17.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:17.304+0000 7f5dc7095700 1 -- 172.21.15.107:0/344196453 wait complete. 2024-05-17T22:04:17.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:17.305+0000 7f5dc7095700 1 Processor -- start 2024-05-17T22:04:17.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:17.305+0000 7f5dc7095700 1 -- start start 2024-05-17T22:04:17.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:17.305+0000 7f5dc7095700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5dc0101ce0 0x7f5dc0112a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:17.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:17.305+0000 7f5dc7095700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5dc0102680 0x7f5dc0112f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:17.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:17.305+0000 7f5dc7095700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc01135a0 con 0x7f5dc0102680 2024-05-17T22:04:17.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:17.305+0000 7f5dc7095700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc010c990 con 0x7f5dc0101ce0 2024-05-17T22:04:17.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:17.305+0000 7f5dbffff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5dc0102680 0x7f5dc0112f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:17.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:17.305+0000 7f5dbffff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5dc0102680 0x7f5dc0112f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48396/0 (socket says 172.21.15.107:48396) 2024-05-17T22:04:17.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:17.305+0000 7f5dbffff700 1 -- 172.21.15.107:0/2028477464 learned_addr learned my addr 172.21.15.107:0/2028477464 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T22:04:17.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:17.305+0000 7f5dc4e31700 1 --2- 172.21.15.107:0/2028477464 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5dc0101ce0 0x7f5dc0112a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:17.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:17.305+0000 7f5dbffff700 1 -- 172.21.15.107:0/2028477464 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5dc0101ce0 msgr2=0x7f5dc0112a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:17.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:17.305+0000 7f5dbffff700 1 --2- 172.21.15.107:0/2028477464 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5dc0101ce0 0x7f5dc0112a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:17.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:17.305+0000 7f5dbffff700 1 -- 172.21.15.107:0/2028477464 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5db4005040 con 0x7f5dc0102680 2024-05-17T22:04:17.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:17.306+0000 7f5dbffff700 1 --2- 172.21.15.107:0/2028477464 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5dc0102680 0x7f5dc0112f50 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f5dac00eb40 tx=0x7f5dac00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:17.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:17.306+0000 7f5dbd7fa700 1 -- 172.21.15.107:0/2028477464 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5dac007cc0 con 0x7f5dc0102680 2024-05-17T22:04:17.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:17.306+0000 7f5dc7095700 1 -- 172.21.15.107:0/2028477464 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dc010cc10 con 0x7f5dc0102680 2024-05-17T22:04:17.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:17.306+0000 7f5dbd7fa700 1 -- 172.21.15.107:0/2028477464 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5dac00f040 con 0x7f5dc0102680 2024-05-17T22:04:17.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:17.306+0000 7f5dbd7fa700 1 -- 172.21.15.107:0/2028477464 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5dac00c9d0 con 0x7f5dc0102680 2024-05-17T22:04:17.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:17.306+0000 7f5dc7095700 1 -- 172.21.15.107:0/2028477464 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dc010d150 con 0x7f5dc0102680 2024-05-17T22:04:17.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:17.307+0000 7f5dc7095700 1 -- 172.21.15.107:0/2028477464 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dc00fd430 con 0x7f5dc0102680 2024-05-17T22:04:17.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:17.308+0000 7f5dbd7fa700 1 -- 172.21.15.107:0/2028477464 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f5dac00cc40 con 0x7f5dc0102680 2024-05-17T22:04:17.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:17.308+0000 7f5dbd7fa700 1 --2- 172.21.15.107:0/2028477464 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f5db0078990 0x7f5db007ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:17.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:17.308+0000 7f5dc4e31700 1 --2- 172.21.15.107:0/2028477464 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f5db0078990 0x7f5db007ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:17.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:17.308+0000 7f5dbd7fa700 1 -- 172.21.15.107:0/2028477464 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f5dac09a1d0 con 0x7f5dc0102680 2024-05-17T22:04:17.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:17.309+0000 7f5dc4e31700 1 --2- 172.21.15.107:0/2028477464 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f5db0078990 0x7f5db007ae40 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5db4009b30 tx=0x7f5db4009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:17.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:17.313+0000 7f5dbd7fa700 1 -- 172.21.15.107:0/2028477464 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f5dac061e40 con 0x7f5dc0102680 2024-05-17T22:04:17.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:17.561+0000 7f5dc7095700 1 -- 172.21.15.107:0/2028477464 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 25, "format": "json"} v 0) v1 -- 0x7f5dc0108de0 con 0x7f5dc0102680 2024-05-17T22:04:17.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:17.561+0000 7f5dbd7fa700 1 -- 172.21.15.107:0/2028477464 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 25, "format": "json"}]=0 dumped fsmap epoch 25 v35) v1 ==== 107+0+4221 (secure 0 0 0) 0x7f5dac0617e0 con 0x7f5dc0102680 2024-05-17T22:04:17.553 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T22:04:17.553 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34408,"name":"cephfs.smithi107.idmbgj","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6827/2235359521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":2235359521},{"type":"v1","addr":"172.21.15.107:6827","nonce":2235359521}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":34440,"name":"cephfs.smithi107.drqody","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6829/369507752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":369507752},{"type":"v1","addr":"172.21.15.107:6829","nonce":369507752}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T21:49:16.836333+0000","modified":"2024-05-17T22:00:01.517533+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24267},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24267":{"gid":24267,"name":"cephfs.smithi138.zckrvs","rank":0,"incarnation":23,"state":"up:rejoin","state_seq":160,"addr":"172.21.15.138:6827/170628661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":170628661},{"type":"v1","addr":"172.21.15.138:6827","nonce":170628661}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-05-17T22:04:17.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:17.563+0000 7f5dc7095700 1 -- 172.21.15.107:0/2028477464 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f5db0078990 msgr2=0x7f5db007ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:17.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:17.563+0000 7f5dc7095700 1 --2- 172.21.15.107:0/2028477464 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f5db0078990 0x7f5db007ae40 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5db4009b30 tx=0x7f5db4009e30 comp rx=0 tx=0).stop 2024-05-17T22:04:17.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:17.563+0000 7f5dc7095700 1 -- 172.21.15.107:0/2028477464 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5dc0102680 msgr2=0x7f5dc0112f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:17.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:17.563+0000 7f5dc7095700 1 --2- 172.21.15.107:0/2028477464 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5dc0102680 0x7f5dc0112f50 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f5dac00eb40 tx=0x7f5dac00b040 comp rx=0 tx=0).stop 2024-05-17T22:04:17.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:17.564+0000 7f5dc7095700 1 -- 172.21.15.107:0/2028477464 shutdown_connections 2024-05-17T22:04:17.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:17.564+0000 7f5dc7095700 1 --2- 172.21.15.107:0/2028477464 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f5db0078990 0x7f5db007ae40 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:17.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:17.564+0000 7f5dc7095700 1 --2- 172.21.15.107:0/2028477464 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5dc0101ce0 0x7f5dc0112a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:17.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:17.564+0000 7f5dc7095700 1 --2- 172.21.15.107:0/2028477464 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5dc0102680 0x7f5dc0112f50 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:17.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:17.564+0000 7f5dc7095700 1 -- 172.21.15.107:0/2028477464 >> 172.21.15.107:0/2028477464 conn(0x7f5dc00f7890 msgr2=0x7f5dc00f8790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:04:17.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:17.564+0000 7f5dc7095700 1 -- 172.21.15.107:0/2028477464 shutdown_connections 2024-05-17T22:04:17.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:17.564+0000 7f5dc7095700 1 -- 172.21.15.107:0/2028477464 wait complete. 2024-05-17T22:04:17.556 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 25 2024-05-17T22:04:18.164 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:04:17 smithi107 ceph-mon[93224]: pgmap v329: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:04:18.165 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:04:17 smithi107 ceph-mon[93224]: from='client.? 172.21.15.107:0/2028477464' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-05-17T22:04:18.190 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 25 2024-05-17T22:04:18.190 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 26 2024-05-17T22:04:18.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:04:17 smithi138 ceph-mon[81376]: pgmap v329: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:04:18.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:04:17 smithi138 ceph-mon[81376]: from='client.? 172.21.15.107:0/2028477464' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-05-17T22:04:18.446 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/mon.smithi107/config 2024-05-17T22:04:20.066 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:04:19 smithi107 ceph-mon[93224]: pgmap v330: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:04:20.066 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:04:19 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T22:04:20.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:20.074+0000 7f5e7c111700 1 -- 172.21.15.107:0/2656438957 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e741026a0 msgr2=0x7f5e740fbd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:20.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:20.074+0000 7f5e7c111700 1 --2- 172.21.15.107:0/2656438957 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e741026a0 0x7f5e740fbd10 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f5e68009ce0 tx=0x7f5e68004d10 comp rx=0 tx=0).stop 2024-05-17T22:04:20.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:20.074+0000 7f5e7c111700 1 -- 172.21.15.107:0/2656438957 shutdown_connections 2024-05-17T22:04:20.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:20.074+0000 7f5e7c111700 1 --2- 172.21.15.107:0/2656438957 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e741026a0 0x7f5e740fbd10 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:20.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:20.074+0000 7f5e7c111700 1 --2- 172.21.15.107:0/2656438957 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e74101d00 0x7f5e741020d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:20.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:20.074+0000 7f5e7c111700 1 -- 172.21.15.107:0/2656438957 >> 172.21.15.107:0/2656438957 conn(0x7f5e740f7870 msgr2=0x7f5e740f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:04:20.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:20.074+0000 7f5e7c111700 1 -- 172.21.15.107:0/2656438957 shutdown_connections 2024-05-17T22:04:20.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:20.074+0000 7f5e7c111700 1 -- 172.21.15.107:0/2656438957 wait complete. 2024-05-17T22:04:20.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:20.075+0000 7f5e7c111700 1 Processor -- start 2024-05-17T22:04:20.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:20.075+0000 7f5e7c111700 1 -- start start 2024-05-17T22:04:20.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:20.075+0000 7f5e7c111700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e74101d00 0x7f5e74113b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:20.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:20.075+0000 7f5e7c111700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e741026a0 0x7f5e74114060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:20.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:20.075+0000 7f5e7c111700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e741146b0 con 0x7f5e74101d00 2024-05-17T22:04:20.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:20.075+0000 7f5e7c111700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e7410dc90 con 0x7f5e741026a0 2024-05-17T22:04:20.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:20.075+0000 7f5e79ead700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e74101d00 0x7f5e74113b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:20.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:20.075+0000 7f5e79ead700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e74101d00 0x7f5e74113b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48412/0 (socket says 172.21.15.107:48412) 2024-05-17T22:04:20.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:20.075+0000 7f5e79ead700 1 -- 172.21.15.107:0/2182300501 learned_addr learned my addr 172.21.15.107:0/2182300501 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T22:04:20.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:20.075+0000 7f5e796ac700 1 --2- 172.21.15.107:0/2182300501 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e741026a0 0x7f5e74114060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:20.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:20.076+0000 7f5e79ead700 1 -- 172.21.15.107:0/2182300501 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e741026a0 msgr2=0x7f5e74114060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:20.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:20.076+0000 7f5e79ead700 1 --2- 172.21.15.107:0/2182300501 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e741026a0 0x7f5e74114060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:20.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:20.076+0000 7f5e79ead700 1 -- 172.21.15.107:0/2182300501 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e68009940 con 0x7f5e74101d00 2024-05-17T22:04:20.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:20.076+0000 7f5e79ead700 1 --2- 172.21.15.107:0/2182300501 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e74101d00 0x7f5e74113b20 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c002a50 tx=0x7f5e6c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:20.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:20.076+0000 7f5e727fc700 1 -- 172.21.15.107:0/2182300501 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e6c007cc0 con 0x7f5e74101d00 2024-05-17T22:04:20.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:20.076+0000 7f5e727fc700 1 -- 172.21.15.107:0/2182300501 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5e6c00f040 con 0x7f5e74101d00 2024-05-17T22:04:20.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:20.076+0000 7f5e7c111700 1 -- 172.21.15.107:0/2182300501 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e7410df70 con 0x7f5e74101d00 2024-05-17T22:04:20.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:20.076+0000 7f5e727fc700 1 -- 172.21.15.107:0/2182300501 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e6c00db70 con 0x7f5e74101d00 2024-05-17T22:04:20.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:20.076+0000 7f5e7c111700 1 -- 172.21.15.107:0/2182300501 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e7410e530 con 0x7f5e74101d00 2024-05-17T22:04:20.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:20.077+0000 7f5e7c111700 1 -- 172.21.15.107:0/2182300501 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e740fd450 con 0x7f5e74101d00 2024-05-17T22:04:20.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:20.078+0000 7f5e727fc700 1 -- 172.21.15.107:0/2182300501 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f5e6c00dd10 con 0x7f5e74101d00 2024-05-17T22:04:20.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:20.078+0000 7f5e727fc700 1 --2- 172.21.15.107:0/2182300501 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f5e5c078990 0x7f5e5c07ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:20.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:20.078+0000 7f5e727fc700 1 -- 172.21.15.107:0/2182300501 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f5e6c099ed0 con 0x7f5e74101d00 2024-05-17T22:04:20.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:20.078+0000 7f5e796ac700 1 --2- 172.21.15.107:0/2182300501 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f5e5c078990 0x7f5e5c07ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:20.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:20.079+0000 7f5e796ac700 1 --2- 172.21.15.107:0/2182300501 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f5e5c078990 0x7f5e5c07ae40 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5e7410f3c0 tx=0x7f5e68004d10 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:20.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:20.083+0000 7f5e727fc700 1 -- 172.21.15.107:0/2182300501 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f5e6c061a90 con 0x7f5e74101d00 2024-05-17T22:04:20.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:20.332+0000 7f5e7c111700 1 -- 172.21.15.107:0/2182300501 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 26, "format": "json"} v 0) v1 -- 0x7f5e7410f170 con 0x7f5e74101d00 2024-05-17T22:04:20.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:20.333+0000 7f5e727fc700 1 -- 172.21.15.107:0/2182300501 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 26, "format": "json"}]=0 dumped fsmap epoch 26 v35) v1 ==== 107+0+4230 (secure 0 0 0) 0x7f5e6c061430 con 0x7f5e74101d00 2024-05-17T22:04:20.324 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T22:04:20.325 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34408,"name":"cephfs.smithi107.idmbgj","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6827/2235359521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":2235359521},{"type":"v1","addr":"172.21.15.107:6827","nonce":2235359521}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":34440,"name":"cephfs.smithi107.drqody","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6829/369507752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":369507752},{"type":"v1","addr":"172.21.15.107:6829","nonce":369507752}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T21:49:16.836333+0000","modified":"2024-05-17T22:00:03.521665+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24267},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24267":{"gid":24267,"name":"cephfs.smithi138.zckrvs","rank":0,"incarnation":23,"state":"up:active","state_seq":161,"addr":"172.21.15.138:6827/170628661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":170628661},{"type":"v1","addr":"172.21.15.138:6827","nonce":170628661}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24267,"qdb_cluster":[24267]},"id":1}]} 2024-05-17T22:04:20.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:04:19 smithi138 ceph-mon[81376]: pgmap v330: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:04:20.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:04:19 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T22:04:20.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:20.335+0000 7f5e7c111700 1 -- 172.21.15.107:0/2182300501 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f5e5c078990 msgr2=0x7f5e5c07ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:20.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:20.335+0000 7f5e7c111700 1 --2- 172.21.15.107:0/2182300501 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f5e5c078990 0x7f5e5c07ae40 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5e7410f3c0 tx=0x7f5e68004d10 comp rx=0 tx=0).stop 2024-05-17T22:04:20.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:20.336+0000 7f5e7c111700 1 -- 172.21.15.107:0/2182300501 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e74101d00 msgr2=0x7f5e74113b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:20.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:20.336+0000 7f5e7c111700 1 --2- 172.21.15.107:0/2182300501 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e74101d00 0x7f5e74113b20 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c002a50 tx=0x7f5e6c0046b0 comp rx=0 tx=0).stop 2024-05-17T22:04:20.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:20.336+0000 7f5e7c111700 1 -- 172.21.15.107:0/2182300501 shutdown_connections 2024-05-17T22:04:20.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:20.336+0000 7f5e7c111700 1 --2- 172.21.15.107:0/2182300501 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f5e5c078990 0x7f5e5c07ae40 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:20.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:20.336+0000 7f5e7c111700 1 --2- 172.21.15.107:0/2182300501 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e74101d00 0x7f5e74113b20 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:20.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:20.336+0000 7f5e7c111700 1 --2- 172.21.15.107:0/2182300501 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e741026a0 0x7f5e74114060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:20.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:20.336+0000 7f5e7c111700 1 -- 172.21.15.107:0/2182300501 >> 172.21.15.107:0/2182300501 conn(0x7f5e740f7870 msgr2=0x7f5e740f8f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:04:20.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:20.336+0000 7f5e7c111700 1 -- 172.21.15.107:0/2182300501 shutdown_connections 2024-05-17T22:04:20.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:20.336+0000 7f5e7c111700 1 -- 172.21.15.107:0/2182300501 wait complete. 2024-05-17T22:04:20.328 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 26 2024-05-17T22:04:21.024 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 26 2024-05-17T22:04:21.025 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 27 2024-05-17T22:04:21.281 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/mon.smithi107/config 2024-05-17T22:04:21.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:04:20 smithi138 ceph-mon[81376]: from='client.? 172.21.15.107:0/2182300501' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-05-17T22:04:21.350 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:04:20 smithi107 ceph-mon[93224]: from='client.? 172.21.15.107:0/2182300501' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-05-17T22:04:22.297 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:04:21 smithi107 ceph-mon[93224]: pgmap v331: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:04:22.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:04:21 smithi138 ceph-mon[81376]: pgmap v331: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:04:22.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:22.851+0000 7f9478bd8700 1 -- 172.21.15.107:0/4037804038 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9474103780 msgr2=0x7f94740fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:22.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:22.851+0000 7f9478bd8700 1 --2- 172.21.15.107:0/4037804038 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9474103780 0x7f94740fcdf0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f945c004940 tx=0x7f945c009e30 comp rx=0 tx=0).stop 2024-05-17T22:04:22.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:22.852+0000 7f9478bd8700 1 -- 172.21.15.107:0/4037804038 shutdown_connections 2024-05-17T22:04:22.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:22.852+0000 7f9478bd8700 1 --2- 172.21.15.107:0/4037804038 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9474103780 0x7f94740fcdf0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:22.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:22.852+0000 7f9478bd8700 1 --2- 172.21.15.107:0/4037804038 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9474102de0 0x7f94741031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:22.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:22.852+0000 7f9478bd8700 1 -- 172.21.15.107:0/4037804038 >> 172.21.15.107:0/4037804038 conn(0x7f94740f8990 msgr2=0x7f94740fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:04:22.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:22.852+0000 7f9478bd8700 1 -- 172.21.15.107:0/4037804038 shutdown_connections 2024-05-17T22:04:22.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:22.852+0000 7f9478bd8700 1 -- 172.21.15.107:0/4037804038 wait complete. 2024-05-17T22:04:22.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:22.852+0000 7f9478bd8700 1 Processor -- start 2024-05-17T22:04:22.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:22.852+0000 7f9478bd8700 1 -- start start 2024-05-17T22:04:22.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:22.852+0000 7f9478bd8700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9474102de0 0x7f9474070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:22.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:22.852+0000 7f9478bd8700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9474103780 0x7f94740707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:22.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:22.852+0000 7f9478bd8700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9474072240 con 0x7f9474102de0 2024-05-17T22:04:22.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:22.852+0000 7f9478bd8700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9474070ce0 con 0x7f9474103780 2024-05-17T22:04:22.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:22.853+0000 7f947259c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9474102de0 0x7f9474070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:22.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:22.853+0000 7f947259c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9474102de0 0x7f9474070260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48430/0 (socket says 172.21.15.107:48430) 2024-05-17T22:04:22.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:22.853+0000 7f947259c700 1 -- 172.21.15.107:0/1630619470 learned_addr learned my addr 172.21.15.107:0/1630619470 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T22:04:22.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:22.853+0000 7f9471d9b700 1 --2- 172.21.15.107:0/1630619470 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9474103780 0x7f94740707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:22.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:22.853+0000 7f947259c700 1 -- 172.21.15.107:0/1630619470 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9474103780 msgr2=0x7f94740707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:22.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:22.853+0000 7f947259c700 1 --2- 172.21.15.107:0/1630619470 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9474103780 0x7f94740707a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:22.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:22.853+0000 7f947259c700 1 -- 172.21.15.107:0/1630619470 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f945c005040 con 0x7f9474102de0 2024-05-17T22:04:22.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:22.853+0000 7f9471d9b700 1 --2- 172.21.15.107:0/1630619470 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9474103780 0x7f94740707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:04:22.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:22.853+0000 7f947259c700 1 --2- 172.21.15.107:0/1630619470 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9474102de0 0x7f9474070260 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f946400b790 tx=0x7f9464009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:22.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:22.853+0000 7f946affd700 1 -- 172.21.15.107:0/1630619470 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f946400f770 con 0x7f9474102de0 2024-05-17T22:04:22.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:22.853+0000 7f9478bd8700 1 -- 172.21.15.107:0/1630619470 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9474070fc0 con 0x7f9474102de0 2024-05-17T22:04:22.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:22.853+0000 7f946affd700 1 -- 172.21.15.107:0/1630619470 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f946400fda0 con 0x7f9474102de0 2024-05-17T22:04:22.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:22.853+0000 7f946affd700 1 -- 172.21.15.107:0/1630619470 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f946400d310 con 0x7f9474102de0 2024-05-17T22:04:22.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:22.853+0000 7f9478bd8700 1 -- 172.21.15.107:0/1630619470 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f947411a210 con 0x7f9474102de0 2024-05-17T22:04:22.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:22.854+0000 7f9478bd8700 1 -- 172.21.15.107:0/1630619470 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94741000f0 con 0x7f9474102de0 2024-05-17T22:04:22.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:22.855+0000 7f946affd700 1 -- 172.21.15.107:0/1630619470 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f9464010420 con 0x7f9474102de0 2024-05-17T22:04:22.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:22.855+0000 7f946affd700 1 --2- 172.21.15.107:0/1630619470 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f9460078990 0x7f946007ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:22.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:22.855+0000 7f946affd700 1 -- 172.21.15.107:0/1630619470 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f946409a7f0 con 0x7f9474102de0 2024-05-17T22:04:22.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:22.855+0000 7f9471d9b700 1 --2- 172.21.15.107:0/1630619470 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f9460078990 0x7f946007ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:22.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:22.856+0000 7f9471d9b700 1 --2- 172.21.15.107:0/1630619470 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f9460078990 0x7f946007ae40 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f945c00b3b0 tx=0x7f945c009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:22.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:22.860+0000 7f946affd700 1 -- 172.21.15.107:0/1630619470 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f9464062430 con 0x7f9474102de0 2024-05-17T22:04:23.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:23.106+0000 7f9478bd8700 1 -- 172.21.15.107:0/1630619470 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 27, "format": "json"} v 0) v1 -- 0x7f94740fe530 con 0x7f9474102de0 2024-05-17T22:04:23.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:23.107+0000 7f946affd700 1 -- 172.21.15.107:0/1630619470 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 27, "format": "json"}]=0 dumped fsmap epoch 27 v35) v1 ==== 107+0+5080 (secure 0 0 0) 0x7f9464061dd0 con 0x7f9474102de0 2024-05-17T22:04:23.098 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T22:04:23.098 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34408,"name":"cephfs.smithi107.idmbgj","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6827/2235359521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":2235359521},{"type":"v1","addr":"172.21.15.107:6827","nonce":2235359521}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":34440,"name":"cephfs.smithi107.drqody","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6829/369507752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":369507752},{"type":"v1","addr":"172.21.15.107:6829","nonce":369507752}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":44335,"name":"cephfs.smithi138.gwbtyd","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6825/1700112281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1700112281},{"type":"v1","addr":"172.21.15.138:6825","nonce":1700112281}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T21:49:16.836333+0000","modified":"2024-05-17T22:00:03.521665+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24267},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24267":{"gid":24267,"name":"cephfs.smithi138.zckrvs","rank":0,"incarnation":23,"state":"up:active","state_seq":161,"addr":"172.21.15.138:6827/170628661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":170628661},{"type":"v1","addr":"172.21.15.138:6827","nonce":170628661}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24267,"qdb_cluster":[24267]},"id":1}]} 2024-05-17T22:04:23.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:23.109+0000 7f9478bd8700 1 -- 172.21.15.107:0/1630619470 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f9460078990 msgr2=0x7f946007ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:23.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:23.109+0000 7f9478bd8700 1 --2- 172.21.15.107:0/1630619470 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f9460078990 0x7f946007ae40 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f945c00b3b0 tx=0x7f945c009e30 comp rx=0 tx=0).stop 2024-05-17T22:04:23.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:23.109+0000 7f9478bd8700 1 -- 172.21.15.107:0/1630619470 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9474102de0 msgr2=0x7f9474070260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:23.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:23.109+0000 7f9478bd8700 1 --2- 172.21.15.107:0/1630619470 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9474102de0 0x7f9474070260 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f946400b790 tx=0x7f9464009250 comp rx=0 tx=0).stop 2024-05-17T22:04:23.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:23.109+0000 7f9478bd8700 1 -- 172.21.15.107:0/1630619470 shutdown_connections 2024-05-17T22:04:23.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:23.109+0000 7f9478bd8700 1 --2- 172.21.15.107:0/1630619470 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f9460078990 0x7f946007ae40 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:23.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:23.109+0000 7f9478bd8700 1 --2- 172.21.15.107:0/1630619470 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9474102de0 0x7f9474070260 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:23.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:23.109+0000 7f9478bd8700 1 --2- 172.21.15.107:0/1630619470 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9474103780 0x7f94740707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:23.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:23.109+0000 7f9478bd8700 1 -- 172.21.15.107:0/1630619470 >> 172.21.15.107:0/1630619470 conn(0x7f94740f8990 msgr2=0x7f94740fa750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:04:23.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:23.109+0000 7f9478bd8700 1 -- 172.21.15.107:0/1630619470 shutdown_connections 2024-05-17T22:04:23.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:23.109+0000 7f9478bd8700 1 -- 172.21.15.107:0/1630619470 wait complete. 2024-05-17T22:04:23.101 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 27 2024-05-17T22:04:23.773 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 27 2024-05-17T22:04:23.774 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 28 2024-05-17T22:04:24.033 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/mon.smithi107/config 2024-05-17T22:04:24.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:04:23 smithi107 ceph-mon[93224]: pgmap v332: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:04:24.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:04:23 smithi107 ceph-mon[93224]: from='client.? 172.21.15.107:0/1630619470' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-05-17T22:04:24.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:04:23 smithi138 ceph-mon[81376]: pgmap v332: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:04:24.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:04:23 smithi138 ceph-mon[81376]: from='client.? 172.21.15.107:0/1630619470' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-05-17T22:04:25.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:25.700+0000 7f6bb9581700 1 -- 172.21.15.107:0/473989623 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6bb40fddd0 msgr2=0x7f6bb40fe1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:25.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:25.700+0000 7f6bb9581700 1 --2- 172.21.15.107:0/473989623 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6bb40fddd0 0x7f6bb40fe1a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f6ba4004940 tx=0x7f6ba4009e30 comp rx=0 tx=0).stop 2024-05-17T22:04:25.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:25.700+0000 7f6bb9581700 1 -- 172.21.15.107:0/473989623 shutdown_connections 2024-05-17T22:04:25.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:25.700+0000 7f6bb9581700 1 --2- 172.21.15.107:0/473989623 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6bb40fe770 0x7f6bb4106db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:25.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:25.700+0000 7f6bb9581700 1 --2- 172.21.15.107:0/473989623 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6bb40fddd0 0x7f6bb40fe1a0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:25.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:25.700+0000 7f6bb9581700 1 -- 172.21.15.107:0/473989623 >> 172.21.15.107:0/473989623 conn(0x7f6bb40f7860 msgr2=0x7f6bb40f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:04:25.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:25.700+0000 7f6bb9581700 1 -- 172.21.15.107:0/473989623 shutdown_connections 2024-05-17T22:04:25.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:25.700+0000 7f6bb9581700 1 -- 172.21.15.107:0/473989623 wait complete. 2024-05-17T22:04:25.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:25.701+0000 7f6bb9581700 1 Processor -- start 2024-05-17T22:04:25.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:25.701+0000 7f6bb9581700 1 -- start start 2024-05-17T22:04:25.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:25.701+0000 7f6bb9581700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6bb40fddd0 0x7f6bb40fbc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:25.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:25.701+0000 7f6bb9581700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6bb40fe770 0x7f6bb40fc140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:25.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:25.701+0000 7f6bb9581700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb40fdb50 con 0x7f6bb40fe770 2024-05-17T22:04:25.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:25.701+0000 7f6bb9581700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb40fc680 con 0x7f6bb40fddd0 2024-05-17T22:04:25.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:25.702+0000 7f6bb27fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6bb40fe770 0x7f6bb40fc140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:25.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:25.702+0000 7f6bb27fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6bb40fe770 0x7f6bb40fc140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45232/0 (socket says 172.21.15.107:45232) 2024-05-17T22:04:25.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:25.702+0000 7f6bb27fc700 1 -- 172.21.15.107:0/3326461326 learned_addr learned my addr 172.21.15.107:0/3326461326 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T22:04:25.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:25.702+0000 7f6bb2ffd700 1 --2- 172.21.15.107:0/3326461326 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6bb40fddd0 0x7f6bb40fbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:25.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:25.702+0000 7f6bb27fc700 1 -- 172.21.15.107:0/3326461326 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6bb40fddd0 msgr2=0x7f6bb40fbc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:25.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:25.702+0000 7f6bb27fc700 1 --2- 172.21.15.107:0/3326461326 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6bb40fddd0 0x7f6bb40fbc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:25.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:25.702+0000 7f6bb27fc700 1 -- 172.21.15.107:0/3326461326 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ba4005040 con 0x7f6bb40fe770 2024-05-17T22:04:25.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:25.702+0000 7f6bb27fc700 1 --2- 172.21.15.107:0/3326461326 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6bb40fe770 0x7f6bb40fc140 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f6ba8002a80 tx=0x7f6ba80046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:25.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:25.702+0000 7f6b9b7fe700 1 -- 172.21.15.107:0/3326461326 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ba8007cc0 con 0x7f6bb40fe770 2024-05-17T22:04:25.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:25.702+0000 7f6b9b7fe700 1 -- 172.21.15.107:0/3326461326 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6ba800f040 con 0x7f6bb40fe770 2024-05-17T22:04:25.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:25.702+0000 7f6bb9581700 1 -- 172.21.15.107:0/3326461326 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bb40fc960 con 0x7f6bb40fe770 2024-05-17T22:04:25.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:25.702+0000 7f6b9b7fe700 1 -- 172.21.15.107:0/3326461326 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ba80096f0 con 0x7f6bb40fe770 2024-05-17T22:04:25.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:25.702+0000 7f6bb9581700 1 -- 172.21.15.107:0/3326461326 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bb4119fe0 con 0x7f6bb40fe770 2024-05-17T22:04:25.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:25.703+0000 7f6bb9581700 1 -- 172.21.15.107:0/3326461326 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bb4066b70 con 0x7f6bb40fe770 2024-05-17T22:04:25.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:25.704+0000 7f6b9b7fe700 1 -- 172.21.15.107:0/3326461326 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f6ba800d690 con 0x7f6bb40fe770 2024-05-17T22:04:25.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:25.704+0000 7f6b9b7fe700 1 --2- 172.21.15.107:0/3326461326 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f6b9c078990 0x7f6b9c07ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:25.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:25.705+0000 7f6b9b7fe700 1 -- 172.21.15.107:0/3326461326 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f6ba8099e80 con 0x7f6bb40fe770 2024-05-17T22:04:25.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:25.705+0000 7f6bb2ffd700 1 --2- 172.21.15.107:0/3326461326 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f6b9c078990 0x7f6b9c07ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:25.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:25.705+0000 7f6bb2ffd700 1 --2- 172.21.15.107:0/3326461326 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f6b9c078990 0x7f6b9c07ae40 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f6ba4004560 tx=0x7f6ba4009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:25.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:25.709+0000 7f6b9b7fe700 1 -- 172.21.15.107:0/3326461326 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f6ba8061ac0 con 0x7f6bb40fe770 2024-05-17T22:04:25.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:25.961+0000 7f6bb9581700 1 -- 172.21.15.107:0/3326461326 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 28, "format": "json"} v 0) v1 -- 0x7f6bb411a220 con 0x7f6bb40fe770 2024-05-17T22:04:25.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:25.962+0000 7f6b9b7fe700 1 -- 172.21.15.107:0/3326461326 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 28, "format": "json"}]=0 dumped fsmap epoch 28 v35) v1 ==== 107+0+4279 (secure 0 0 0) 0x7f6ba8061460 con 0x7f6bb40fe770 2024-05-17T22:04:25.953 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T22:04:25.954 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34408,"name":"cephfs.smithi107.idmbgj","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6827/2235359521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":2235359521},{"type":"v1","addr":"172.21.15.107:6827","nonce":2235359521}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":34440,"name":"cephfs.smithi107.drqody","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6829/369507752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":369507752},{"type":"v1","addr":"172.21.15.107:6829","nonce":369507752}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":44335,"name":"cephfs.smithi138.gwbtyd","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6825/1700112281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1700112281},{"type":"v1","addr":"172.21.15.138:6825","nonce":1700112281}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T21:49:16.836333+0000","modified":"2024-05-17T22:00:16.636492+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-05-17T22:04:25.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:25.964+0000 7f6bb9581700 1 -- 172.21.15.107:0/3326461326 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f6b9c078990 msgr2=0x7f6b9c07ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:25.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:25.964+0000 7f6bb9581700 1 --2- 172.21.15.107:0/3326461326 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f6b9c078990 0x7f6b9c07ae40 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f6ba4004560 tx=0x7f6ba4009e30 comp rx=0 tx=0).stop 2024-05-17T22:04:25.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:25.964+0000 7f6bb9581700 1 -- 172.21.15.107:0/3326461326 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6bb40fe770 msgr2=0x7f6bb40fc140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:25.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:25.964+0000 7f6bb9581700 1 --2- 172.21.15.107:0/3326461326 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6bb40fe770 0x7f6bb40fc140 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f6ba8002a80 tx=0x7f6ba80046b0 comp rx=0 tx=0).stop 2024-05-17T22:04:25.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:25.964+0000 7f6bb9581700 1 -- 172.21.15.107:0/3326461326 shutdown_connections 2024-05-17T22:04:25.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:25.964+0000 7f6bb9581700 1 --2- 172.21.15.107:0/3326461326 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f6b9c078990 0x7f6b9c07ae40 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:25.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:25.964+0000 7f6bb9581700 1 --2- 172.21.15.107:0/3326461326 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6bb40fddd0 0x7f6bb40fbc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:25.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:25.964+0000 7f6bb9581700 1 --2- 172.21.15.107:0/3326461326 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6bb40fe770 0x7f6bb40fc140 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:25.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:25.964+0000 7f6bb9581700 1 -- 172.21.15.107:0/3326461326 >> 172.21.15.107:0/3326461326 conn(0x7f6bb40f7860 msgr2=0x7f6bb40f8de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:04:25.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:25.964+0000 7f6bb9581700 1 -- 172.21.15.107:0/3326461326 shutdown_connections 2024-05-17T22:04:25.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:25.964+0000 7f6bb9581700 1 -- 172.21.15.107:0/3326461326 wait complete. 2024-05-17T22:04:25.957 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 28 2024-05-17T22:04:26.261 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:04:25 smithi107 ceph-mon[93224]: pgmap v333: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:04:26.261 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:04:25 smithi107 ceph-mon[93224]: from='client.? 172.21.15.107:0/3326461326' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-05-17T22:04:26.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:04:25 smithi138 ceph-mon[81376]: pgmap v333: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:04:26.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:04:25 smithi138 ceph-mon[81376]: from='client.? 172.21.15.107:0/3326461326' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-05-17T22:04:26.580 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 28 2024-05-17T22:04:26.580 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 29 2024-05-17T22:04:26.836 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/mon.smithi107/config 2024-05-17T22:04:28.246 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:04:27 smithi107 ceph-mon[93224]: pgmap v334: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:04:28.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:04:27 smithi138 ceph-mon[81376]: pgmap v334: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:04:28.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:28.491+0000 7f745676c700 1 -- 172.21.15.107:0/3941989910 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74501026b0 msgr2=0x7f74500fbd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:28.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:28.491+0000 7f745676c700 1 --2- 172.21.15.107:0/3941989910 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74501026b0 0x7f74500fbd20 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f7438004940 tx=0x7f7438009e30 comp rx=0 tx=0).stop 2024-05-17T22:04:28.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:28.491+0000 7f745676c700 1 -- 172.21.15.107:0/3941989910 shutdown_connections 2024-05-17T22:04:28.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:28.491+0000 7f745676c700 1 --2- 172.21.15.107:0/3941989910 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74501026b0 0x7f74500fbd20 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:28.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:28.491+0000 7f745676c700 1 --2- 172.21.15.107:0/3941989910 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7450101d10 0x7f74501020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:28.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:28.491+0000 7f745676c700 1 -- 172.21.15.107:0/3941989910 >> 172.21.15.107:0/3941989910 conn(0x7f74500f7860 msgr2=0x7f74500f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:04:28.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:28.491+0000 7f745676c700 1 -- 172.21.15.107:0/3941989910 shutdown_connections 2024-05-17T22:04:28.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:28.491+0000 7f745676c700 1 -- 172.21.15.107:0/3941989910 wait complete. 2024-05-17T22:04:28.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:28.492+0000 7f745676c700 1 Processor -- start 2024-05-17T22:04:28.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:28.492+0000 7f745676c700 1 -- start start 2024-05-17T22:04:28.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:28.492+0000 7f745676c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7450101d10 0x7f74500fd990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:28.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:28.492+0000 7f745676c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74501026b0 0x7f74500fded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:28.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:28.492+0000 7f745676c700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74500ff8e0 con 0x7f74501026b0 2024-05-17T22:04:28.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:28.492+0000 7f745676c700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74500fe410 con 0x7f7450101d10 2024-05-17T22:04:28.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:28.492+0000 7f7447fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74501026b0 0x7f74500fded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:28.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:28.492+0000 7f7447fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74501026b0 0x7f74500fded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45236/0 (socket says 172.21.15.107:45236) 2024-05-17T22:04:28.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:28.492+0000 7f7447fff700 1 -- 172.21.15.107:0/1649920422 learned_addr learned my addr 172.21.15.107:0/1649920422 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T22:04:28.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:28.492+0000 7f744ffff700 1 --2- 172.21.15.107:0/1649920422 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7450101d10 0x7f74500fd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:28.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:28.492+0000 7f7447fff700 1 -- 172.21.15.107:0/1649920422 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7450101d10 msgr2=0x7f74500fd990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:28.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:28.492+0000 7f7447fff700 1 --2- 172.21.15.107:0/1649920422 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7450101d10 0x7f74500fd990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:28.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:28.492+0000 7f7447fff700 1 -- 172.21.15.107:0/1649920422 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7438005040 con 0x7f74501026b0 2024-05-17T22:04:28.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:28.492+0000 7f7447fff700 1 --2- 172.21.15.107:0/1649920422 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74501026b0 0x7f74500fded0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f7438009100 tx=0x7f7438009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:28.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:28.492+0000 7f744d7fa700 1 -- 172.21.15.107:0/1649920422 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f743800e040 con 0x7f74501026b0 2024-05-17T22:04:28.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:28.493+0000 7f744d7fa700 1 -- 172.21.15.107:0/1649920422 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7438009af0 con 0x7f74501026b0 2024-05-17T22:04:28.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:28.493+0000 7f745676c700 1 -- 172.21.15.107:0/1649920422 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74500fe690 con 0x7f74501026b0 2024-05-17T22:04:28.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:28.493+0000 7f744d7fa700 1 -- 172.21.15.107:0/1649920422 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f743801c3f0 con 0x7f74501026b0 2024-05-17T22:04:28.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:28.493+0000 7f745676c700 1 -- 172.21.15.107:0/1649920422 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f745011a110 con 0x7f74501026b0 2024-05-17T22:04:28.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:28.493+0000 7f745676c700 1 -- 172.21.15.107:0/1649920422 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74500fd460 con 0x7f74501026b0 2024-05-17T22:04:28.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:28.494+0000 7f744d7fa700 1 -- 172.21.15.107:0/1649920422 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f743800baf0 con 0x7f74501026b0 2024-05-17T22:04:28.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:28.495+0000 7f744d7fa700 1 --2- 172.21.15.107:0/1649920422 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f743c078990 0x7f743c07ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:28.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:28.495+0000 7f744d7fa700 1 -- 172.21.15.107:0/1649920422 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f743809b9f0 con 0x7f74501026b0 2024-05-17T22:04:28.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:28.495+0000 7f744ffff700 1 --2- 172.21.15.107:0/1649920422 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f743c078990 0x7f743c07ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:28.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:28.495+0000 7f744ffff700 1 --2- 172.21.15.107:0/1649920422 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f743c078990 0x7f743c07ae40 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f74400041d0 tx=0x7f74400043c0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:28.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:28.499+0000 7f744d7fa700 1 -- 172.21.15.107:0/1649920422 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f743809de60 con 0x7f74501026b0 2024-05-17T22:04:28.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:28.746+0000 7f745676c700 1 -- 172.21.15.107:0/1649920422 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 29, "format": "json"} v 0) v1 -- 0x7f745011a300 con 0x7f74501026b0 2024-05-17T22:04:28.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:28.746+0000 7f744d7fa700 1 -- 172.21.15.107:0/1649920422 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 29, "format": "json"}]=0 dumped fsmap epoch 29 v35) v1 ==== 107+0+4358 (secure 0 0 0) 0x7f7438063630 con 0x7f74501026b0 2024-05-17T22:04:28.738 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T22:04:28.738 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34440,"name":"cephfs.smithi107.drqody","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6829/369507752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":369507752},{"type":"v1","addr":"172.21.15.107:6829","nonce":369507752}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":44335,"name":"cephfs.smithi138.gwbtyd","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6825/1700112281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1700112281},{"type":"v1","addr":"172.21.15.138:6825","nonce":1700112281}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T21:49:16.836333+0000","modified":"2024-05-17T22:00:16.642128+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34408},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34408":{"gid":34408,"name":"cephfs.smithi107.idmbgj","rank":0,"incarnation":29,"state":"up:replay","state_seq":1,"addr":"172.21.15.107:6827/2235359521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":2235359521},{"type":"v1","addr":"172.21.15.107:6827","nonce":2235359521}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-05-17T22:04:28.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:28.750+0000 7f745676c700 1 -- 172.21.15.107:0/1649920422 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f743c078990 msgr2=0x7f743c07ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:28.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:28.750+0000 7f745676c700 1 --2- 172.21.15.107:0/1649920422 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f743c078990 0x7f743c07ae40 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f74400041d0 tx=0x7f74400043c0 comp rx=0 tx=0).stop 2024-05-17T22:04:28.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:28.750+0000 7f745676c700 1 -- 172.21.15.107:0/1649920422 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74501026b0 msgr2=0x7f74500fded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:28.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:28.750+0000 7f745676c700 1 --2- 172.21.15.107:0/1649920422 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74501026b0 0x7f74500fded0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f7438009100 tx=0x7f7438009e30 comp rx=0 tx=0).stop 2024-05-17T22:04:28.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:28.750+0000 7f745676c700 1 -- 172.21.15.107:0/1649920422 shutdown_connections 2024-05-17T22:04:28.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:28.750+0000 7f745676c700 1 --2- 172.21.15.107:0/1649920422 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f743c078990 0x7f743c07ae40 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:28.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:28.750+0000 7f745676c700 1 --2- 172.21.15.107:0/1649920422 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7450101d10 0x7f74500fd990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:28.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:28.750+0000 7f745676c700 1 --2- 172.21.15.107:0/1649920422 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74501026b0 0x7f74500fded0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:28.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:28.750+0000 7f745676c700 1 -- 172.21.15.107:0/1649920422 >> 172.21.15.107:0/1649920422 conn(0x7f74500f7860 msgr2=0x7f74500f8f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:04:28.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:28.750+0000 7f745676c700 1 -- 172.21.15.107:0/1649920422 shutdown_connections 2024-05-17T22:04:28.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:28.750+0000 7f745676c700 1 -- 172.21.15.107:0/1649920422 wait complete. 2024-05-17T22:04:28.741 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 29 2024-05-17T22:04:29.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:04:29 smithi138 ceph-mon[81376]: from='client.? 172.21.15.107:0/1649920422' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 29, "format": "json"}]: dispatch 2024-05-17T22:04:29.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:04:29 smithi107 ceph-mon[93224]: from='client.? 172.21.15.107:0/1649920422' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 29, "format": "json"}]: dispatch 2024-05-17T22:04:29.504 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 29 2024-05-17T22:04:29.505 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 30 2024-05-17T22:04:29.765 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/mon.smithi107/config 2024-05-17T22:04:30.136 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:04:30 smithi107 ceph-mon[93224]: pgmap v335: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:04:30.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:04:30 smithi138 ceph-mon[81376]: pgmap v335: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:04:31.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:31.410+0000 7f13d75a0700 1 -- 172.21.15.107:0/3116297518 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13d00702a0 msgr2=0x7f13d0070670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:31.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:31.410+0000 7f13d75a0700 1 --2- 172.21.15.107:0/3116297518 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13d00702a0 0x7f13d0070670 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f13cc009d10 tx=0x7f13cc004d10 comp rx=0 tx=0).stop 2024-05-17T22:04:31.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:31.410+0000 7f13d75a0700 1 -- 172.21.15.107:0/3116297518 shutdown_connections 2024-05-17T22:04:31.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:31.410+0000 7f13d75a0700 1 --2- 172.21.15.107:0/3116297518 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13d0070bb0 0x7f13d0109160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:31.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:31.410+0000 7f13d75a0700 1 --2- 172.21.15.107:0/3116297518 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13d00702a0 0x7f13d0070670 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:31.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:31.410+0000 7f13d75a0700 1 -- 172.21.15.107:0/3116297518 >> 172.21.15.107:0/3116297518 conn(0x7f13d00f8980 msgr2=0x7f13d00fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:04:31.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:31.410+0000 7f13d75a0700 1 -- 172.21.15.107:0/3116297518 shutdown_connections 2024-05-17T22:04:31.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:31.410+0000 7f13d75a0700 1 -- 172.21.15.107:0/3116297518 wait complete. 2024-05-17T22:04:31.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:31.411+0000 7f13d75a0700 1 Processor -- start 2024-05-17T22:04:31.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:31.411+0000 7f13d75a0700 1 -- start start 2024-05-17T22:04:31.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:31.411+0000 7f13d75a0700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13d00702a0 0x7f13d0115f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:31.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:31.411+0000 7f13d75a0700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13d0070bb0 0x7f13d0116490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:31.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:31.411+0000 7f13d75a0700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13d0116ae0 con 0x7f13d00702a0 2024-05-17T22:04:31.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:31.411+0000 7f13d75a0700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13d010feb0 con 0x7f13d0070bb0 2024-05-17T22:04:31.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:31.411+0000 7f13d533c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13d00702a0 0x7f13d0115f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:31.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:31.411+0000 7f13d533c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13d00702a0 0x7f13d0115f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45250/0 (socket says 172.21.15.107:45250) 2024-05-17T22:04:31.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:31.411+0000 7f13d533c700 1 -- 172.21.15.107:0/1048381627 learned_addr learned my addr 172.21.15.107:0/1048381627 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T22:04:31.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:31.412+0000 7f13d4b3b700 1 --2- 172.21.15.107:0/1048381627 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13d0070bb0 0x7f13d0116490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:31.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:31.412+0000 7f13d533c700 1 -- 172.21.15.107:0/1048381627 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13d0070bb0 msgr2=0x7f13d0116490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:31.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:31.412+0000 7f13d533c700 1 --2- 172.21.15.107:0/1048381627 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13d0070bb0 0x7f13d0116490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:31.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:31.412+0000 7f13d533c700 1 -- 172.21.15.107:0/1048381627 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13cc009940 con 0x7f13d00702a0 2024-05-17T22:04:31.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:31.412+0000 7f13d4b3b700 1 --2- 172.21.15.107:0/1048381627 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13d0070bb0 0x7f13d0116490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:04:31.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:31.412+0000 7f13d533c700 1 --2- 172.21.15.107:0/1048381627 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13d00702a0 0x7f13d0115f50 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f13cc008df0 tx=0x7f13cc004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:31.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:31.412+0000 7f13c5ffb700 1 -- 172.21.15.107:0/1048381627 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f13cc00e040 con 0x7f13d00702a0 2024-05-17T22:04:31.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:31.412+0000 7f13d75a0700 1 -- 172.21.15.107:0/1048381627 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13d0110130 con 0x7f13d00702a0 2024-05-17T22:04:31.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:31.412+0000 7f13c5ffb700 1 -- 172.21.15.107:0/1048381627 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f13cc004610 con 0x7f13d00702a0 2024-05-17T22:04:31.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:31.412+0000 7f13c5ffb700 1 -- 172.21.15.107:0/1048381627 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f13cc01c7e0 con 0x7f13d00702a0 2024-05-17T22:04:31.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:31.412+0000 7f13d75a0700 1 -- 172.21.15.107:0/1048381627 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13d0110670 con 0x7f13d00702a0 2024-05-17T22:04:31.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:31.413+0000 7f13d75a0700 1 -- 172.21.15.107:0/1048381627 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13d00719e0 con 0x7f13d00702a0 2024-05-17T22:04:31.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:31.414+0000 7f13c5ffb700 1 -- 172.21.15.107:0/1048381627 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f13cc012410 con 0x7f13d00702a0 2024-05-17T22:04:31.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:31.414+0000 7f13c5ffb700 1 --2- 172.21.15.107:0/1048381627 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f13bc078990 0x7f13bc07ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:31.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:31.415+0000 7f13c5ffb700 1 -- 172.21.15.107:0/1048381627 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f13cc09c4d0 con 0x7f13d00702a0 2024-05-17T22:04:31.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:31.415+0000 7f13d4b3b700 1 --2- 172.21.15.107:0/1048381627 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f13bc078990 0x7f13bc07ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:31.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:31.415+0000 7f13d4b3b700 1 --2- 172.21.15.107:0/1048381627 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f13bc078990 0x7f13bc07ae40 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f13c0009fd0 tx=0x7f13c0009ee0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:31.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:31.419+0000 7f13c5ffb700 1 -- 172.21.15.107:0/1048381627 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f13cc064090 con 0x7f13d00702a0 2024-05-17T22:04:31.655 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:31.665+0000 7f13d75a0700 1 -- 172.21.15.107:0/1048381627 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 30, "format": "json"} v 0) v1 -- 0x7f13d004bed0 con 0x7f13d00702a0 2024-05-17T22:04:31.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:31.666+0000 7f13c5ffb700 1 -- 172.21.15.107:0/1048381627 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 30, "format": "json"}]=0 dumped fsmap epoch 30 v35) v1 ==== 107+0+5208 (secure 0 0 0) 0x7f13cc063a30 con 0x7f13d00702a0 2024-05-17T22:04:31.656 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T22:04:31.656 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34440,"name":"cephfs.smithi107.drqody","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6829/369507752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":369507752},{"type":"v1","addr":"172.21.15.107:6829","nonce":369507752}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":44335,"name":"cephfs.smithi138.gwbtyd","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6825/1700112281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1700112281},{"type":"v1","addr":"172.21.15.138:6825","nonce":1700112281}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":44365,"name":"cephfs.smithi138.zckrvs","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6827/1801131392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1801131392},{"type":"v1","addr":"172.21.15.138:6827","nonce":1801131392}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T21:49:16.836333+0000","modified":"2024-05-17T22:00:16.642128+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34408},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34408":{"gid":34408,"name":"cephfs.smithi107.idmbgj","rank":0,"incarnation":29,"state":"up:replay","state_seq":1,"addr":"172.21.15.107:6827/2235359521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":2235359521},{"type":"v1","addr":"172.21.15.107:6827","nonce":2235359521}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-05-17T22:04:31.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:31.668+0000 7f13d75a0700 1 -- 172.21.15.107:0/1048381627 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f13bc078990 msgr2=0x7f13bc07ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:31.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:31.668+0000 7f13d75a0700 1 --2- 172.21.15.107:0/1048381627 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f13bc078990 0x7f13bc07ae40 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f13c0009fd0 tx=0x7f13c0009ee0 comp rx=0 tx=0).stop 2024-05-17T22:04:31.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:31.668+0000 7f13d75a0700 1 -- 172.21.15.107:0/1048381627 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13d00702a0 msgr2=0x7f13d0115f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:31.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:31.668+0000 7f13d75a0700 1 --2- 172.21.15.107:0/1048381627 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13d00702a0 0x7f13d0115f50 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f13cc008df0 tx=0x7f13cc004d10 comp rx=0 tx=0).stop 2024-05-17T22:04:31.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:31.668+0000 7f13d75a0700 1 -- 172.21.15.107:0/1048381627 shutdown_connections 2024-05-17T22:04:31.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:31.668+0000 7f13d75a0700 1 --2- 172.21.15.107:0/1048381627 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f13bc078990 0x7f13bc07ae40 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:31.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:31.668+0000 7f13d75a0700 1 --2- 172.21.15.107:0/1048381627 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13d00702a0 0x7f13d0115f50 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:31.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:31.668+0000 7f13d75a0700 1 --2- 172.21.15.107:0/1048381627 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13d0070bb0 0x7f13d0116490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:31.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:31.668+0000 7f13d75a0700 1 -- 172.21.15.107:0/1048381627 >> 172.21.15.107:0/1048381627 conn(0x7f13d00f8980 msgr2=0x7f13d00f9f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:04:31.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:31.668+0000 7f13d75a0700 1 -- 172.21.15.107:0/1048381627 shutdown_connections 2024-05-17T22:04:31.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:31.668+0000 7f13d75a0700 1 -- 172.21.15.107:0/1048381627 wait complete. 2024-05-17T22:04:31.660 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 30 2024-05-17T22:04:32.304 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:04:32 smithi107 ceph-mon[93224]: pgmap v336: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:04:32.305 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:04:32 smithi107 ceph-mon[93224]: from='client.? 172.21.15.107:0/1048381627' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 30, "format": "json"}]: dispatch 2024-05-17T22:04:32.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:04:32 smithi138 ceph-mon[81376]: pgmap v336: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:04:32.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:04:32 smithi138 ceph-mon[81376]: from='client.? 172.21.15.107:0/1048381627' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 30, "format": "json"}]: dispatch 2024-05-17T22:04:32.380 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 30 2024-05-17T22:04:32.380 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 31 2024-05-17T22:04:32.636 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/mon.smithi107/config 2024-05-17T22:04:34.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:04:34 smithi107 ceph-mon[93224]: pgmap v337: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:04:34.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:04:34 smithi138 ceph-mon[81376]: pgmap v337: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:04:34.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:34.359+0000 7f01bc27e700 1 -- 172.21.15.107:0/152337369 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01b40fde20 msgr2=0x7f01b40fe1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:34.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:34.359+0000 7f01bc27e700 1 --2- 172.21.15.107:0/152337369 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01b40fde20 0x7f01b40fe1f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f01b0004970 tx=0x7f01b0009e30 comp rx=0 tx=0).stop 2024-05-17T22:04:34.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:34.359+0000 7f01bc27e700 1 -- 172.21.15.107:0/152337369 shutdown_connections 2024-05-17T22:04:34.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:34.359+0000 7f01bc27e700 1 --2- 172.21.15.107:0/152337369 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f01b40fe7c0 0x7f01b4106e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:34.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:34.359+0000 7f01bc27e700 1 --2- 172.21.15.107:0/152337369 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01b40fde20 0x7f01b40fe1f0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:34.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:34.359+0000 7f01bc27e700 1 -- 172.21.15.107:0/152337369 >> 172.21.15.107:0/152337369 conn(0x7f01b40f7860 msgr2=0x7f01b40f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:04:34.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:34.359+0000 7f01bc27e700 1 -- 172.21.15.107:0/152337369 shutdown_connections 2024-05-17T22:04:34.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:34.359+0000 7f01bc27e700 1 -- 172.21.15.107:0/152337369 wait complete. 2024-05-17T22:04:34.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:34.359+0000 7f01bc27e700 1 Processor -- start 2024-05-17T22:04:34.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:34.359+0000 7f01bc27e700 1 -- start start 2024-05-17T22:04:34.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:34.360+0000 7f01bc27e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f01b40fde20 0x7f01b41139a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:34.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:34.360+0000 7f01bc27e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01b40fe7c0 0x7f01b4113ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:34.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:34.360+0000 7f01bc27e700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01b41145c0 con 0x7f01b40fe7c0 2024-05-17T22:04:34.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:34.360+0000 7f01bc27e700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01b410db30 con 0x7f01b40fde20 2024-05-17T22:04:34.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:34.360+0000 7f01b9819700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01b40fe7c0 0x7f01b4113ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:34.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:34.360+0000 7f01b9819700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01b40fe7c0 0x7f01b4113ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45278/0 (socket says 172.21.15.107:45278) 2024-05-17T22:04:34.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:34.360+0000 7f01b9819700 1 -- 172.21.15.107:0/3659314273 learned_addr learned my addr 172.21.15.107:0/3659314273 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T22:04:34.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:34.360+0000 7f01ba01a700 1 --2- 172.21.15.107:0/3659314273 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f01b40fde20 0x7f01b41139a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:34.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:34.360+0000 7f01b9819700 1 -- 172.21.15.107:0/3659314273 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f01b40fde20 msgr2=0x7f01b41139a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:34.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:34.360+0000 7f01b9819700 1 --2- 172.21.15.107:0/3659314273 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f01b40fde20 0x7f01b41139a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:34.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:34.360+0000 7f01b9819700 1 -- 172.21.15.107:0/3659314273 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01b0005040 con 0x7f01b40fe7c0 2024-05-17T22:04:34.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:34.360+0000 7f01b9819700 1 --2- 172.21.15.107:0/3659314273 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01b40fe7c0 0x7f01b4113ee0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f01a400b790 tx=0x7f01a4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:34.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:34.360+0000 7f01aaffd700 1 -- 172.21.15.107:0/3659314273 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f01a400f770 con 0x7f01b40fe7c0 2024-05-17T22:04:34.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:34.360+0000 7f01aaffd700 1 -- 172.21.15.107:0/3659314273 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f01a400fda0 con 0x7f01b40fe7c0 2024-05-17T22:04:34.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:34.360+0000 7f01aaffd700 1 -- 172.21.15.107:0/3659314273 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f01a400d310 con 0x7f01b40fe7c0 2024-05-17T22:04:34.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:34.360+0000 7f01bc27e700 1 -- 172.21.15.107:0/3659314273 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01b410de10 con 0x7f01b40fe7c0 2024-05-17T22:04:34.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:34.360+0000 7f01bc27e700 1 -- 172.21.15.107:0/3659314273 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01b410e350 con 0x7f01b40fe7c0 2024-05-17T22:04:34.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:34.361+0000 7f01bc27e700 1 -- 172.21.15.107:0/3659314273 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01b4066ab0 con 0x7f01b40fe7c0 2024-05-17T22:04:34.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:34.362+0000 7f01aaffd700 1 -- 172.21.15.107:0/3659314273 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f01a4010420 con 0x7f01b40fe7c0 2024-05-17T22:04:34.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:34.362+0000 7f01aaffd700 1 --2- 172.21.15.107:0/3659314273 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f01a0078990 0x7f01a007ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:34.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:34.363+0000 7f01aaffd700 1 -- 172.21.15.107:0/3659314273 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f01a409a750 con 0x7f01b40fe7c0 2024-05-17T22:04:34.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:34.363+0000 7f01ba01a700 1 --2- 172.21.15.107:0/3659314273 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f01a0078990 0x7f01a007ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:34.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:34.363+0000 7f01ba01a700 1 --2- 172.21.15.107:0/3659314273 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f01a0078990 0x7f01a007ae40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f01b0004f50 tx=0x7f01b0009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:34.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:34.367+0000 7f01aaffd700 1 -- 172.21.15.107:0/3659314273 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f01a4062430 con 0x7f01b40fe7c0 2024-05-17T22:04:34.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:34.618+0000 7f01bc27e700 1 -- 172.21.15.107:0/3659314273 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 31, "format": "json"} v 0) v1 -- 0x7f01b404c710 con 0x7f01b40fe7c0 2024-05-17T22:04:34.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:34.619+0000 7f01aaffd700 1 -- 172.21.15.107:0/3659314273 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 31, "format": "json"}]=0 dumped fsmap epoch 31 v35) v1 ==== 107+0+5212 (secure 0 0 0) 0x7f01a4061dd0 con 0x7f01b40fe7c0 2024-05-17T22:04:34.610 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T22:04:34.610 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34440,"name":"cephfs.smithi107.drqody","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6829/369507752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":369507752},{"type":"v1","addr":"172.21.15.107:6829","nonce":369507752}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":44335,"name":"cephfs.smithi138.gwbtyd","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6825/1700112281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1700112281},{"type":"v1","addr":"172.21.15.138:6825","nonce":1700112281}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":44365,"name":"cephfs.smithi138.zckrvs","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6827/1801131392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1801131392},{"type":"v1","addr":"172.21.15.138:6827","nonce":1801131392}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T21:49:16.836333+0000","modified":"2024-05-17T22:00:26.355966+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34408},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34408":{"gid":34408,"name":"cephfs.smithi107.idmbgj","rank":0,"incarnation":29,"state":"up:reconnect","state_seq":18,"addr":"172.21.15.107:6827/2235359521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":2235359521},{"type":"v1","addr":"172.21.15.107:6827","nonce":2235359521}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-05-17T22:04:34.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:34.621+0000 7f01bc27e700 1 -- 172.21.15.107:0/3659314273 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f01a0078990 msgr2=0x7f01a007ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:34.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:34.621+0000 7f01bc27e700 1 --2- 172.21.15.107:0/3659314273 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f01a0078990 0x7f01a007ae40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f01b0004f50 tx=0x7f01b0009e30 comp rx=0 tx=0).stop 2024-05-17T22:04:34.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:34.621+0000 7f01bc27e700 1 -- 172.21.15.107:0/3659314273 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01b40fe7c0 msgr2=0x7f01b4113ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:34.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:34.621+0000 7f01bc27e700 1 --2- 172.21.15.107:0/3659314273 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01b40fe7c0 0x7f01b4113ee0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f01a400b790 tx=0x7f01a4009250 comp rx=0 tx=0).stop 2024-05-17T22:04:34.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:34.621+0000 7f01bc27e700 1 -- 172.21.15.107:0/3659314273 shutdown_connections 2024-05-17T22:04:34.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:34.621+0000 7f01bc27e700 1 --2- 172.21.15.107:0/3659314273 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f01a0078990 0x7f01a007ae40 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:34.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:34.621+0000 7f01bc27e700 1 --2- 172.21.15.107:0/3659314273 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f01b40fde20 0x7f01b41139a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:34.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:34.621+0000 7f01bc27e700 1 --2- 172.21.15.107:0/3659314273 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01b40fe7c0 0x7f01b4113ee0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:34.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:34.621+0000 7f01bc27e700 1 -- 172.21.15.107:0/3659314273 >> 172.21.15.107:0/3659314273 conn(0x7f01b40f7860 msgr2=0x7f01b40f8c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:04:34.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:34.622+0000 7f01bc27e700 1 -- 172.21.15.107:0/3659314273 shutdown_connections 2024-05-17T22:04:34.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:34.622+0000 7f01bc27e700 1 -- 172.21.15.107:0/3659314273 wait complete. 2024-05-17T22:04:34.613 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 31 2024-05-17T22:04:35.245 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 31 2024-05-17T22:04:35.246 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 32 2024-05-17T22:04:35.268 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:04:35 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T22:04:35.268 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:04:35 smithi107 ceph-mon[93224]: from='client.? 172.21.15.107:0/3659314273' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 31, "format": "json"}]: dispatch 2024-05-17T22:04:35.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:04:35 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T22:04:35.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:04:35 smithi138 ceph-mon[81376]: from='client.? 172.21.15.107:0/3659314273' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 31, "format": "json"}]: dispatch 2024-05-17T22:04:35.502 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/mon.smithi107/config 2024-05-17T22:04:36.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:04:36 smithi138 ceph-mon[81376]: pgmap v338: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:04:36.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:04:36 smithi107 ceph-mon[93224]: pgmap v338: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:04:37.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:37.193+0000 7fbdf1ba3700 1 -- 172.21.15.107:0/3650749 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbdec0fe790 msgr2=0x7fbdec106dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:37.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:37.193+0000 7fbdf1ba3700 1 --2- 172.21.15.107:0/3650749 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbdec0fe790 0x7fbdec106dd0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fbde0004940 tx=0x7fbde0009e30 comp rx=0 tx=0).stop 2024-05-17T22:04:37.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:37.193+0000 7fbdf1ba3700 1 -- 172.21.15.107:0/3650749 shutdown_connections 2024-05-17T22:04:37.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:37.193+0000 7fbdf1ba3700 1 --2- 172.21.15.107:0/3650749 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbdec0fe790 0x7fbdec106dd0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:37.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:37.193+0000 7fbdf1ba3700 1 --2- 172.21.15.107:0/3650749 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbdec0fddf0 0x7fbdec0fe1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:37.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:37.193+0000 7fbdf1ba3700 1 -- 172.21.15.107:0/3650749 >> 172.21.15.107:0/3650749 conn(0x7fbdec0f78c0 msgr2=0x7fbdec0f9cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:04:37.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:37.193+0000 7fbdf1ba3700 1 -- 172.21.15.107:0/3650749 shutdown_connections 2024-05-17T22:04:37.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:37.193+0000 7fbdf1ba3700 1 -- 172.21.15.107:0/3650749 wait complete. 2024-05-17T22:04:37.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:37.193+0000 7fbdf1ba3700 1 Processor -- start 2024-05-17T22:04:37.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:37.193+0000 7fbdf1ba3700 1 -- start start 2024-05-17T22:04:37.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:37.194+0000 7fbdf1ba3700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbdec0fddf0 0x7fbdec113820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:37.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:37.194+0000 7fbdf1ba3700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbdec0fe790 0x7fbdec113d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:37.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:37.194+0000 7fbdf1ba3700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdec1143b0 con 0x7fbdec0fddf0 2024-05-17T22:04:37.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:37.194+0000 7fbdf1ba3700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdec10d9f0 con 0x7fbdec0fe790 2024-05-17T22:04:37.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:37.194+0000 7fbdeb7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbdec0fddf0 0x7fbdec113820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:37.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:37.194+0000 7fbdeb7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbdec0fddf0 0x7fbdec113820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44874/0 (socket says 172.21.15.107:44874) 2024-05-17T22:04:37.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:37.194+0000 7fbdeb7fe700 1 -- 172.21.15.107:0/1184399533 learned_addr learned my addr 172.21.15.107:0/1184399533 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T22:04:37.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:37.194+0000 7fbdeb7fe700 1 -- 172.21.15.107:0/1184399533 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbdec0fe790 msgr2=0x7fbdec113d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:37.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:37.194+0000 7fbdeaffd700 1 --2- 172.21.15.107:0/1184399533 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbdec0fe790 0x7fbdec113d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:37.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:37.194+0000 7fbdeb7fe700 1 --2- 172.21.15.107:0/1184399533 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbdec0fe790 0x7fbdec113d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:37.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:37.194+0000 7fbdeb7fe700 1 -- 172.21.15.107:0/1184399533 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbde0005040 con 0x7fbdec0fddf0 2024-05-17T22:04:37.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:37.194+0000 7fbdeb7fe700 1 --2- 172.21.15.107:0/1184399533 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbdec0fddf0 0x7fbdec113820 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fbddc00eb40 tx=0x7fbddc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:37.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:37.194+0000 7fbdd3fff700 1 -- 172.21.15.107:0/1184399533 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbddc007cc0 con 0x7fbdec0fddf0 2024-05-17T22:04:37.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:37.194+0000 7fbdf1ba3700 1 -- 172.21.15.107:0/1184399533 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdec10dc70 con 0x7fbdec0fddf0 2024-05-17T22:04:37.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:37.194+0000 7fbdd3fff700 1 -- 172.21.15.107:0/1184399533 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbddc00f040 con 0x7fbdec0fddf0 2024-05-17T22:04:37.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:37.194+0000 7fbdd3fff700 1 -- 172.21.15.107:0/1184399533 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbddc009620 con 0x7fbdec0fddf0 2024-05-17T22:04:37.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:37.194+0000 7fbdf1ba3700 1 -- 172.21.15.107:0/1184399533 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdec10e1b0 con 0x7fbdec0fddf0 2024-05-17T22:04:37.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:37.195+0000 7fbdf1ba3700 1 -- 172.21.15.107:0/1184399533 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdec066b90 con 0x7fbdec0fddf0 2024-05-17T22:04:37.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:37.196+0000 7fbdd3fff700 1 -- 172.21.15.107:0/1184399533 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7fbddc009780 con 0x7fbdec0fddf0 2024-05-17T22:04:37.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:37.196+0000 7fbdd3fff700 1 --2- 172.21.15.107:0/1184399533 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fbdd4078980 0x7fbdd407ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:37.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:37.197+0000 7fbdd3fff700 1 -- 172.21.15.107:0/1184399533 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7fbddc09a410 con 0x7fbdec0fddf0 2024-05-17T22:04:37.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:37.197+0000 7fbdeaffd700 1 --2- 172.21.15.107:0/1184399533 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fbdd4078980 0x7fbdd407ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:37.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:37.197+0000 7fbdeaffd700 1 --2- 172.21.15.107:0/1184399533 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fbdd4078980 0x7fbdd407ae30 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fbde0004560 tx=0x7fbde0009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:37.192 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:37.201+0000 7fbdd3fff700 1 -- 172.21.15.107:0/1184399533 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7fbddc062080 con 0x7fbdec0fddf0 2024-05-17T22:04:37.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:37.455+0000 7fbdf1ba3700 1 -- 172.21.15.107:0/1184399533 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 32, "format": "json"} v 0) v1 -- 0x7fbdec10ef00 con 0x7fbdec0fddf0 2024-05-17T22:04:37.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:37.456+0000 7fbdd3fff700 1 -- 172.21.15.107:0/1184399533 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 32, "format": "json"}]=0 dumped fsmap epoch 32 v35) v1 ==== 107+0+5209 (secure 0 0 0) 0x7fbddc061a20 con 0x7fbdec0fddf0 2024-05-17T22:04:37.447 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T22:04:37.447 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34440,"name":"cephfs.smithi107.drqody","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6829/369507752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":369507752},{"type":"v1","addr":"172.21.15.107:6829","nonce":369507752}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":44335,"name":"cephfs.smithi138.gwbtyd","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6825/1700112281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1700112281},{"type":"v1","addr":"172.21.15.138:6825","nonce":1700112281}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":44365,"name":"cephfs.smithi138.zckrvs","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6827/1801131392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1801131392},{"type":"v1","addr":"172.21.15.138:6827","nonce":1801131392}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T21:49:16.836333+0000","modified":"2024-05-17T22:00:26.600439+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34408},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34408":{"gid":34408,"name":"cephfs.smithi107.idmbgj","rank":0,"incarnation":29,"state":"up:rejoin","state_seq":19,"addr":"172.21.15.107:6827/2235359521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":2235359521},{"type":"v1","addr":"172.21.15.107:6827","nonce":2235359521}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-05-17T22:04:37.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:37.458+0000 7fbdf1ba3700 1 -- 172.21.15.107:0/1184399533 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fbdd4078980 msgr2=0x7fbdd407ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:37.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:37.458+0000 7fbdf1ba3700 1 --2- 172.21.15.107:0/1184399533 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fbdd4078980 0x7fbdd407ae30 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fbde0004560 tx=0x7fbde0009e30 comp rx=0 tx=0).stop 2024-05-17T22:04:37.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:37.458+0000 7fbdf1ba3700 1 -- 172.21.15.107:0/1184399533 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbdec0fddf0 msgr2=0x7fbdec113820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:37.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:37.458+0000 7fbdf1ba3700 1 --2- 172.21.15.107:0/1184399533 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbdec0fddf0 0x7fbdec113820 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fbddc00eb40 tx=0x7fbddc00b040 comp rx=0 tx=0).stop 2024-05-17T22:04:37.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:37.458+0000 7fbdf1ba3700 1 -- 172.21.15.107:0/1184399533 shutdown_connections 2024-05-17T22:04:37.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:37.458+0000 7fbdf1ba3700 1 --2- 172.21.15.107:0/1184399533 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fbdd4078980 0x7fbdd407ae30 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:37.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:37.458+0000 7fbdf1ba3700 1 --2- 172.21.15.107:0/1184399533 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbdec0fddf0 0x7fbdec113820 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:37.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:37.458+0000 7fbdf1ba3700 1 --2- 172.21.15.107:0/1184399533 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbdec0fe790 0x7fbdec113d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:37.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:37.458+0000 7fbdf1ba3700 1 -- 172.21.15.107:0/1184399533 >> 172.21.15.107:0/1184399533 conn(0x7fbdec0f78c0 msgr2=0x7fbdec0f8d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:04:37.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:37.458+0000 7fbdf1ba3700 1 -- 172.21.15.107:0/1184399533 shutdown_connections 2024-05-17T22:04:37.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:37.458+0000 7fbdf1ba3700 1 -- 172.21.15.107:0/1184399533 wait complete. 2024-05-17T22:04:37.449 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 32 2024-05-17T22:04:38.173 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 32 2024-05-17T22:04:38.174 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 33 2024-05-17T22:04:38.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:04:38 smithi138 ceph-mon[81376]: pgmap v339: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:04:38.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:04:38 smithi138 ceph-mon[81376]: from='client.? 172.21.15.107:0/1184399533' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 32, "format": "json"}]: dispatch 2024-05-17T22:04:38.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:04:38 smithi107 ceph-mon[93224]: pgmap v339: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:04:38.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:04:38 smithi107 ceph-mon[93224]: from='client.? 172.21.15.107:0/1184399533' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 32, "format": "json"}]: dispatch 2024-05-17T22:04:38.476 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/mon.smithi107/config 2024-05-17T22:04:40.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:04:40 smithi107 ceph-mon[93224]: pgmap v340: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:04:40.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:40.168+0000 7fdacc6bd700 1 -- 172.21.15.107:0/1051044569 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdac4102e50 msgr2=0x7fdac4103220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:40.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:40.168+0000 7fdacc6bd700 1 --2- 172.21.15.107:0/1051044569 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdac4102e50 0x7fdac4103220 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fdab4009ce0 tx=0x7fdab4004d10 comp rx=0 tx=0).stop 2024-05-17T22:04:40.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:40.168+0000 7fdacc6bd700 1 -- 172.21.15.107:0/1051044569 shutdown_connections 2024-05-17T22:04:40.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:40.168+0000 7fdacc6bd700 1 --2- 172.21.15.107:0/1051044569 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdac4065640 0x7fdac4065ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:40.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:40.168+0000 7fdacc6bd700 1 --2- 172.21.15.107:0/1051044569 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdac4102e50 0x7fdac4103220 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:40.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:40.168+0000 7fdacc6bd700 1 -- 172.21.15.107:0/1051044569 >> 172.21.15.107:0/1051044569 conn(0x7fdac4071d00 msgr2=0x7fdac4072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:04:40.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:40.168+0000 7fdacc6bd700 1 -- 172.21.15.107:0/1051044569 shutdown_connections 2024-05-17T22:04:40.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:40.168+0000 7fdacc6bd700 1 -- 172.21.15.107:0/1051044569 wait complete. 2024-05-17T22:04:40.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:40.169+0000 7fdacc6bd700 1 Processor -- start 2024-05-17T22:04:40.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:40.169+0000 7fdacc6bd700 1 -- start start 2024-05-17T22:04:40.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:40.169+0000 7fdacc6bd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdac4065640 0x7fdac4113cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:40.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:40.169+0000 7fdacc6bd700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdac4102e50 0x7fdac4114230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:40.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:40.169+0000 7fdacc6bd700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdac4114880 con 0x7fdac4065640 2024-05-17T22:04:40.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:40.169+0000 7fdacc6bd700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdac410dc50 con 0x7fdac4102e50 2024-05-17T22:04:40.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:40.169+0000 7fdaca459700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdac4065640 0x7fdac4113cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:40.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:40.169+0000 7fdaca459700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdac4065640 0x7fdac4113cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44884/0 (socket says 172.21.15.107:44884) 2024-05-17T22:04:40.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:40.169+0000 7fdaca459700 1 -- 172.21.15.107:0/2958621817 learned_addr learned my addr 172.21.15.107:0/2958621817 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T22:04:40.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:40.169+0000 7fdaca459700 1 -- 172.21.15.107:0/2958621817 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdac4102e50 msgr2=0x7fdac4114230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:40.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:40.169+0000 7fdaca459700 1 --2- 172.21.15.107:0/2958621817 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdac4102e50 0x7fdac4114230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:40.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:40.169+0000 7fdaca459700 1 -- 172.21.15.107:0/2958621817 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdab4009940 con 0x7fdac4065640 2024-05-17T22:04:40.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:40.169+0000 7fdaca459700 1 --2- 172.21.15.107:0/2958621817 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdac4065640 0x7fdac4113cf0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fdab4003400 tx=0x7fdab4004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:40.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:40.170+0000 7fdac2ffd700 1 -- 172.21.15.107:0/2958621817 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdab400e040 con 0x7fdac4065640 2024-05-17T22:04:40.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:40.170+0000 7fdacc6bd700 1 -- 172.21.15.107:0/2958621817 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdac410ded0 con 0x7fdac4065640 2024-05-17T22:04:40.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:40.170+0000 7fdac2ffd700 1 -- 172.21.15.107:0/2958621817 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdab4004610 con 0x7fdac4065640 2024-05-17T22:04:40.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:40.170+0000 7fdac2ffd700 1 -- 172.21.15.107:0/2958621817 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdab401c6d0 con 0x7fdac4065640 2024-05-17T22:04:40.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:40.170+0000 7fdacc6bd700 1 -- 172.21.15.107:0/2958621817 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdac410e490 con 0x7fdac4065640 2024-05-17T22:04:40.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:40.170+0000 7fdacc6bd700 1 -- 172.21.15.107:0/2958621817 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdac4100100 con 0x7fdac4065640 2024-05-17T22:04:40.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:40.171+0000 7fdac2ffd700 1 -- 172.21.15.107:0/2958621817 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7fdab4012b00 con 0x7fdac4065640 2024-05-17T22:04:40.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:40.172+0000 7fdac2ffd700 1 --2- 172.21.15.107:0/2958621817 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fdaac078990 0x7fdaac07ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:40.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:40.172+0000 7fdac9c58700 1 --2- 172.21.15.107:0/2958621817 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fdaac078990 0x7fdaac07ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:40.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:40.172+0000 7fdac2ffd700 1 -- 172.21.15.107:0/2958621817 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7fdab409c200 con 0x7fdac4065640 2024-05-17T22:04:40.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:40.172+0000 7fdac9c58700 1 --2- 172.21.15.107:0/2958621817 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fdaac078990 0x7fdaac07ae40 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fdac410f590 tx=0x7fdab8009ce0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:40.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:40.176+0000 7fdac2ffd700 1 -- 172.21.15.107:0/2958621817 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7fdab4063e40 con 0x7fdac4065640 2024-05-17T22:04:40.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:04:40 smithi138 ceph-mon[81376]: pgmap v340: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:04:40.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:40.423+0000 7fdacc6bd700 1 -- 172.21.15.107:0/2958621817 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 33, "format": "json"} v 0) v1 -- 0x7fdac410f370 con 0x7fdac4065640 2024-05-17T22:04:40.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:40.424+0000 7fdac2ffd700 1 -- 172.21.15.107:0/2958621817 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 33, "format": "json"}]=0 dumped fsmap epoch 33 v35) v1 ==== 107+0+5218 (secure 0 0 0) 0x7fdab4004e10 con 0x7fdac4065640 2024-05-17T22:04:40.415 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T22:04:40.415 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34440,"name":"cephfs.smithi107.drqody","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6829/369507752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":369507752},{"type":"v1","addr":"172.21.15.107:6829","nonce":369507752}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":44335,"name":"cephfs.smithi138.gwbtyd","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6825/1700112281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1700112281},{"type":"v1","addr":"172.21.15.138:6825","nonce":1700112281}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":44365,"name":"cephfs.smithi138.zckrvs","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6827/1801131392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1801131392},{"type":"v1","addr":"172.21.15.138:6827","nonce":1801131392}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T21:49:16.836333+0000","modified":"2024-05-17T22:00:28.604647+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34408},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34408":{"gid":34408,"name":"cephfs.smithi107.idmbgj","rank":0,"incarnation":29,"state":"up:active","state_seq":20,"addr":"172.21.15.107:6827/2235359521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":2235359521},{"type":"v1","addr":"172.21.15.107:6827","nonce":2235359521}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34408,"qdb_cluster":[34408]},"id":1}]} 2024-05-17T22:04:40.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:40.427+0000 7fdacc6bd700 1 -- 172.21.15.107:0/2958621817 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fdaac078990 msgr2=0x7fdaac07ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:40.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:40.427+0000 7fdacc6bd700 1 --2- 172.21.15.107:0/2958621817 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fdaac078990 0x7fdaac07ae40 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fdac410f590 tx=0x7fdab8009ce0 comp rx=0 tx=0).stop 2024-05-17T22:04:40.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:40.427+0000 7fdacc6bd700 1 -- 172.21.15.107:0/2958621817 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdac4065640 msgr2=0x7fdac4113cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:40.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:40.427+0000 7fdacc6bd700 1 --2- 172.21.15.107:0/2958621817 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdac4065640 0x7fdac4113cf0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fdab4003400 tx=0x7fdab4004d10 comp rx=0 tx=0).stop 2024-05-17T22:04:40.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:40.427+0000 7fdacc6bd700 1 -- 172.21.15.107:0/2958621817 shutdown_connections 2024-05-17T22:04:40.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:40.427+0000 7fdacc6bd700 1 --2- 172.21.15.107:0/2958621817 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fdaac078990 0x7fdaac07ae40 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:40.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:40.427+0000 7fdacc6bd700 1 --2- 172.21.15.107:0/2958621817 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdac4065640 0x7fdac4113cf0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:40.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:40.427+0000 7fdacc6bd700 1 --2- 172.21.15.107:0/2958621817 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdac4102e50 0x7fdac4114230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:40.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:40.427+0000 7fdacc6bd700 1 -- 172.21.15.107:0/2958621817 >> 172.21.15.107:0/2958621817 conn(0x7fdac4071d00 msgr2=0x7fdac40fb040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:04:40.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:40.427+0000 7fdacc6bd700 1 -- 172.21.15.107:0/2958621817 shutdown_connections 2024-05-17T22:04:40.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:40.427+0000 7fdacc6bd700 1 -- 172.21.15.107:0/2958621817 wait complete. 2024-05-17T22:04:40.420 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 33 2024-05-17T22:04:41.080 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 33 2024-05-17T22:04:41.080 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 34 2024-05-17T22:04:41.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:04:41 smithi138 ceph-mon[81376]: from='client.? 172.21.15.107:0/2958621817' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 33, "format": "json"}]: dispatch 2024-05-17T22:04:41.338 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/mon.smithi107/config 2024-05-17T22:04:41.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:04:41 smithi107 ceph-mon[93224]: from='client.? 172.21.15.107:0/2958621817' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 33, "format": "json"}]: dispatch 2024-05-17T22:04:42.305 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:04:42 smithi107 ceph-mon[93224]: pgmap v341: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:04:42.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:04:42 smithi138 ceph-mon[81376]: pgmap v341: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:04:43.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:43.034+0000 7fbf522a1700 1 -- 172.21.15.107:0/526703594 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf4c102e50 msgr2=0x7fbf4c103220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:43.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:43.034+0000 7fbf522a1700 1 --2- 172.21.15.107:0/526703594 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf4c102e50 0x7fbf4c103220 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c004940 tx=0x7fbf3c009e30 comp rx=0 tx=0).stop 2024-05-17T22:04:43.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:43.034+0000 7fbf522a1700 1 -- 172.21.15.107:0/526703594 shutdown_connections 2024-05-17T22:04:43.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:43.034+0000 7fbf522a1700 1 --2- 172.21.15.107:0/526703594 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf4c065640 0x7fbf4c065ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:43.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:43.034+0000 7fbf522a1700 1 --2- 172.21.15.107:0/526703594 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf4c102e50 0x7fbf4c103220 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:43.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:43.034+0000 7fbf522a1700 1 -- 172.21.15.107:0/526703594 >> 172.21.15.107:0/526703594 conn(0x7fbf4c071d00 msgr2=0x7fbf4c072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:04:43.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:43.034+0000 7fbf522a1700 1 -- 172.21.15.107:0/526703594 shutdown_connections 2024-05-17T22:04:43.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:43.034+0000 7fbf522a1700 1 -- 172.21.15.107:0/526703594 wait complete. 2024-05-17T22:04:43.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:43.035+0000 7fbf522a1700 1 Processor -- start 2024-05-17T22:04:43.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:43.035+0000 7fbf522a1700 1 -- start start 2024-05-17T22:04:43.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:43.035+0000 7fbf522a1700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf4c065640 0x7fbf4c0fbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:43.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:43.035+0000 7fbf522a1700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf4c102e50 0x7fbf4c0fc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:43.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:43.035+0000 7fbf522a1700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf4c0fc880 con 0x7fbf4c102e50 2024-05-17T22:04:43.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:43.035+0000 7fbf522a1700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf4c0fc9c0 con 0x7fbf4c065640 2024-05-17T22:04:43.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:43.035+0000 7fbf43fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf4c102e50 0x7fbf4c0fc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:43.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:43.035+0000 7fbf43fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf4c102e50 0x7fbf4c0fc2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44908/0 (socket says 172.21.15.107:44908) 2024-05-17T22:04:43.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:43.035+0000 7fbf43fff700 1 -- 172.21.15.107:0/97075011 learned_addr learned my addr 172.21.15.107:0/97075011 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T22:04:43.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:43.035+0000 7fbf4bfff700 1 --2- 172.21.15.107:0/97075011 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf4c065640 0x7fbf4c0fbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:43.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:43.036+0000 7fbf43fff700 1 -- 172.21.15.107:0/97075011 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf4c065640 msgr2=0x7fbf4c0fbdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:43.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:43.036+0000 7fbf43fff700 1 --2- 172.21.15.107:0/97075011 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf4c065640 0x7fbf4c0fbdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:43.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:43.036+0000 7fbf43fff700 1 -- 172.21.15.107:0/97075011 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf3c005040 con 0x7fbf4c102e50 2024-05-17T22:04:43.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:43.036+0000 7fbf43fff700 1 --2- 172.21.15.107:0/97075011 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf4c102e50 0x7fbf4c0fc2f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fbf34002a50 tx=0x7fbf340046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:43.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:43.036+0000 7fbf497fa700 1 -- 172.21.15.107:0/97075011 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbf34007cc0 con 0x7fbf4c102e50 2024-05-17T22:04:43.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:43.036+0000 7fbf497fa700 1 -- 172.21.15.107:0/97075011 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbf3400f040 con 0x7fbf4c102e50 2024-05-17T22:04:43.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:43.036+0000 7fbf522a1700 1 -- 172.21.15.107:0/97075011 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf4c119e90 con 0x7fbf4c102e50 2024-05-17T22:04:43.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:43.036+0000 7fbf497fa700 1 -- 172.21.15.107:0/97075011 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbf3400da70 con 0x7fbf4c102e50 2024-05-17T22:04:43.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:43.036+0000 7fbf522a1700 1 -- 172.21.15.107:0/97075011 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf4c11a310 con 0x7fbf4c102e50 2024-05-17T22:04:43.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:43.037+0000 7fbf522a1700 1 -- 172.21.15.107:0/97075011 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf4c100100 con 0x7fbf4c102e50 2024-05-17T22:04:43.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:43.038+0000 7fbf497fa700 1 -- 172.21.15.107:0/97075011 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7fbf3400dc10 con 0x7fbf4c102e50 2024-05-17T22:04:43.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:43.038+0000 7fbf497fa700 1 --2- 172.21.15.107:0/97075011 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fbf38078980 0x7fbf3807ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:43.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:43.038+0000 7fbf497fa700 1 -- 172.21.15.107:0/97075011 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7fbf34099e20 con 0x7fbf4c102e50 2024-05-17T22:04:43.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:43.038+0000 7fbf4bfff700 1 --2- 172.21.15.107:0/97075011 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fbf38078980 0x7fbf3807ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:43.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:43.039+0000 7fbf4bfff700 1 --2- 172.21.15.107:0/97075011 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fbf38078980 0x7fbf3807ae30 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c00b3b0 tx=0x7fbf3c009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:43.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:43.043+0000 7fbf497fa700 1 -- 172.21.15.107:0/97075011 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7fbf340619e0 con 0x7fbf4c102e50 2024-05-17T22:04:43.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:43.294+0000 7fbf522a1700 1 -- 172.21.15.107:0/97075011 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 34, "format": "json"} v 0) v1 -- 0x7fbf4c0fe540 con 0x7fbf4c102e50 2024-05-17T22:04:43.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:43.295+0000 7fbf497fa700 1 -- 172.21.15.107:0/97075011 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 34, "format": "json"}]=0 dumped fsmap epoch 34 v35) v1 ==== 107+0+5217 (secure 0 0 0) 0x7fbf34061380 con 0x7fbf4c102e50 2024-05-17T22:04:43.285 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-05-17T22:04:43.286 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34440,"name":"cephfs.smithi107.drqody","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6829/369507752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":369507752},{"type":"v1","addr":"172.21.15.107:6829","nonce":369507752}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":44335,"name":"cephfs.smithi138.gwbtyd","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6825/1700112281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1700112281},{"type":"v1","addr":"172.21.15.138:6825","nonce":1700112281}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":44365,"name":"cephfs.smithi138.zckrvs","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6827/1801131392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1801131392},{"type":"v1","addr":"172.21.15.138:6827","nonce":1801131392}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T21:49:16.836333+0000","modified":"2024-05-17T22:00:32.850178+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34408},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34408":{"gid":34408,"name":"cephfs.smithi107.idmbgj","rank":0,"incarnation":29,"state":"up:active","state_seq":20,"addr":"172.21.15.107:6827/2235359521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":2235359521},{"type":"v1","addr":"172.21.15.107:6827","nonce":2235359521}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34408,"qdb_cluster":[34408]},"id":1}]} 2024-05-17T22:04:43.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:43.297+0000 7fbf522a1700 1 -- 172.21.15.107:0/97075011 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fbf38078980 msgr2=0x7fbf3807ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:43.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:43.297+0000 7fbf522a1700 1 --2- 172.21.15.107:0/97075011 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fbf38078980 0x7fbf3807ae30 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c00b3b0 tx=0x7fbf3c009e30 comp rx=0 tx=0).stop 2024-05-17T22:04:43.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:43.297+0000 7fbf522a1700 1 -- 172.21.15.107:0/97075011 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf4c102e50 msgr2=0x7fbf4c0fc2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:43.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:43.297+0000 7fbf522a1700 1 --2- 172.21.15.107:0/97075011 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf4c102e50 0x7fbf4c0fc2f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fbf34002a50 tx=0x7fbf340046b0 comp rx=0 tx=0).stop 2024-05-17T22:04:43.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:43.297+0000 7fbf522a1700 1 -- 172.21.15.107:0/97075011 shutdown_connections 2024-05-17T22:04:43.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:43.297+0000 7fbf522a1700 1 --2- 172.21.15.107:0/97075011 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7fbf38078980 0x7fbf3807ae30 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:43.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:43.297+0000 7fbf522a1700 1 --2- 172.21.15.107:0/97075011 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf4c065640 0x7fbf4c0fbdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:43.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:43.297+0000 7fbf522a1700 1 --2- 172.21.15.107:0/97075011 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf4c102e50 0x7fbf4c0fc2f0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:43.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:43.297+0000 7fbf522a1700 1 -- 172.21.15.107:0/97075011 >> 172.21.15.107:0/97075011 conn(0x7fbf4c071d00 msgr2=0x7fbf4c0fb010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:04:43.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:43.297+0000 7fbf522a1700 1 -- 172.21.15.107:0/97075011 shutdown_connections 2024-05-17T22:04:43.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:43.297+0000 7fbf522a1700 1 -- 172.21.15.107:0/97075011 wait complete. 2024-05-17T22:04:43.289 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 34 2024-05-17T22:04:43.803 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-05-17T22:04:43.813 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-05-17T22:04:43.813 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-05-17T22:04:43.813 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T22:04:43.831 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-05-17T22:04:43.831 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T22:04:43.889 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -- ceph osd blocklist ls 2024-05-17T22:04:44.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:04:44 smithi107 ceph-mon[93224]: pgmap v342: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:04:44.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:04:44 smithi107 ceph-mon[93224]: from='client.? 172.21.15.107:0/97075011' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 34, "format": "json"}]: dispatch 2024-05-17T22:04:44.184 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/mon.smithi107/config 2024-05-17T22:04:44.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:04:44 smithi138 ceph-mon[81376]: pgmap v342: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:04:44.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:04:44 smithi138 ceph-mon[81376]: from='client.? 172.21.15.107:0/97075011' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 34, "format": "json"}]: dispatch 2024-05-17T22:04:45.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:45.916+0000 7f32a128d700 1 -- 172.21.15.107:0/3517721154 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f329c102de0 msgr2=0x7f329c1031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:45.913 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:45.916+0000 7f32a128d700 1 --2- 172.21.15.107:0/3517721154 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f329c102de0 0x7f329c1031b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f328c004940 tx=0x7f328c009e30 comp rx=0 tx=0).stop 2024-05-17T22:04:45.913 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:45.916+0000 7f32a128d700 1 -- 172.21.15.107:0/3517721154 shutdown_connections 2024-05-17T22:04:45.913 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:45.916+0000 7f32a128d700 1 --2- 172.21.15.107:0/3517721154 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f329c103780 0x7f329c0fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:45.913 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:45.916+0000 7f32a128d700 1 --2- 172.21.15.107:0/3517721154 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f329c102de0 0x7f329c1031b0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:45.913 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:45.916+0000 7f32a128d700 1 -- 172.21.15.107:0/3517721154 >> 172.21.15.107:0/3517721154 conn(0x7f329c0f8990 msgr2=0x7f329c0fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:04:45.913 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:45.916+0000 7f32a128d700 1 -- 172.21.15.107:0/3517721154 shutdown_connections 2024-05-17T22:04:45.913 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:45.916+0000 7f32a128d700 1 -- 172.21.15.107:0/3517721154 wait complete. 2024-05-17T22:04:45.913 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:45.916+0000 7f32a128d700 1 Processor -- start 2024-05-17T22:04:45.913 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:45.917+0000 7f32a128d700 1 -- start start 2024-05-17T22:04:45.913 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:45.917+0000 7f32a128d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f329c102de0 0x7f329c1138b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:45.913 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:45.917+0000 7f32a128d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f329c103780 0x7f329c113df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:45.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:45.917+0000 7f32a128d700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f329c114440 con 0x7f329c103780 2024-05-17T22:04:45.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:45.917+0000 7f32a128d700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f329c10da90 con 0x7f329c102de0 2024-05-17T22:04:45.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:45.917+0000 7f32927fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f329c103780 0x7f329c113df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:45.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:45.917+0000 7f32927fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f329c103780 0x7f329c113df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46846/0 (socket says 172.21.15.107:46846) 2024-05-17T22:04:45.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:45.917+0000 7f32927fc700 1 -- 172.21.15.107:0/964742981 learned_addr learned my addr 172.21.15.107:0/964742981 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T22:04:45.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:45.917+0000 7f329affd700 1 --2- 172.21.15.107:0/964742981 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f329c102de0 0x7f329c1138b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:45.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:45.917+0000 7f32927fc700 1 -- 172.21.15.107:0/964742981 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f329c102de0 msgr2=0x7f329c1138b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:45.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:45.917+0000 7f32927fc700 1 --2- 172.21.15.107:0/964742981 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f329c102de0 0x7f329c1138b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:45.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:45.917+0000 7f32927fc700 1 -- 172.21.15.107:0/964742981 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f328c005040 con 0x7f329c103780 2024-05-17T22:04:45.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:45.918+0000 7f32927fc700 1 --2- 172.21.15.107:0/964742981 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f329c103780 0x7f329c113df0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f3284002a50 tx=0x7f32840046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:45.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:45.918+0000 7f3293fff700 1 -- 172.21.15.107:0/964742981 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3284007cc0 con 0x7f329c103780 2024-05-17T22:04:45.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:45.918+0000 7f32a128d700 1 -- 172.21.15.107:0/964742981 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f329c10dd70 con 0x7f329c103780 2024-05-17T22:04:45.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:45.918+0000 7f3293fff700 1 -- 172.21.15.107:0/964742981 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f328400f040 con 0x7f329c103780 2024-05-17T22:04:45.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:45.918+0000 7f3293fff700 1 -- 172.21.15.107:0/964742981 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f328400db30 con 0x7f329c103780 2024-05-17T22:04:45.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:45.918+0000 7f32a128d700 1 -- 172.21.15.107:0/964742981 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f329c10e330 con 0x7f329c103780 2024-05-17T22:04:45.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:45.919+0000 7f32a128d700 1 -- 172.21.15.107:0/964742981 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f329c1000f0 con 0x7f329c103780 2024-05-17T22:04:45.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:45.920+0000 7f3293fff700 1 -- 172.21.15.107:0/964742981 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f328400dc90 con 0x7f329c103780 2024-05-17T22:04:45.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:45.920+0000 7f3293fff700 1 --2- 172.21.15.107:0/964742981 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f3288078990 0x7f328807ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:45.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:45.920+0000 7f3293fff700 1 -- 172.21.15.107:0/964742981 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f328409a510 con 0x7f329c103780 2024-05-17T22:04:45.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:45.920+0000 7f329affd700 1 --2- 172.21.15.107:0/964742981 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f3288078990 0x7f328807ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:45.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:45.921+0000 7f329affd700 1 --2- 172.21.15.107:0/964742981 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f3288078990 0x7f328807ae40 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f328c004f90 tx=0x7f328c009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:45.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:45.925+0000 7f3293fff700 1 -- 172.21.15.107:0/964742981 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f32840620d0 con 0x7f329c103780 2024-05-17T22:04:46.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:46.145+0000 7f32a128d700 1 -- 172.21.15.107:0/964742981 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f329c10ef50 con 0x7f329c103780 2024-05-17T22:04:46.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:46.145+0000 7f3293fff700 1 -- 172.21.15.107:0/964742981 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 41 entries v84) v1 ==== 81+0+2464 (secure 0 0 0) 0x7f3284061a70 con 0x7f329c103780 2024-05-17T22:04:46.136 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.138:6827/170628661 2024-05-18T22:00:16.636211+0000 2024-05-17T22:04:46.136 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.138:6825/2779634919 2024-05-18T21:59:52.171289+0000 2024-05-17T22:04:46.136 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.138:6824/2779634919 2024-05-18T21:59:52.171289+0000 2024-05-17T22:04:46.136 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/1436894442 2024-05-18T21:54:34.350728+0000 2024-05-17T22:04:46.137 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/3048013464 2024-05-18T21:42:52.099188+0000 2024-05-17T22:04:46.137 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.138:0/1997364823 2024-05-18T21:53:23.772077+0000 2024-05-17T22:04:46.137 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/2414403835 2024-05-18T21:44:32.293094+0000 2024-05-17T22:04:46.137 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6801/3405524625 2024-05-18T21:44:32.293094+0000 2024-05-17T22:04:46.137 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6801/1294361014 2024-05-18T21:51:49.290057+0000 2024-05-17T22:04:46.137 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6800/1718632476 2024-05-18T21:43:22.097723+0000 2024-05-17T22:04:46.137 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/3170894163 2024-05-18T21:43:22.097723+0000 2024-05-17T22:04:46.137 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.138:6826/170628661 2024-05-18T22:00:16.636211+0000 2024-05-17T22:04:46.137 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6800/3405524625 2024-05-18T21:44:32.293094+0000 2024-05-17T22:04:46.137 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6800/1677232931 2024-05-18T21:42:52.099188+0000 2024-05-17T22:04:46.137 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6827/2751824191 2024-05-18T21:49:32.033319+0000 2024-05-17T22:04:46.137 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.138:0/1723941910 2024-05-18T21:53:23.772077+0000 2024-05-17T22:04:46.138 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/1077444017 2024-05-18T21:44:32.293094+0000 2024-05-17T22:04:46.138 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/239507720 2024-05-18T21:51:49.290057+0000 2024-05-17T22:04:46.138 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6801/1718632476 2024-05-18T21:43:22.097723+0000 2024-05-17T22:04:46.138 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6800/416442229 2024-05-18T21:54:34.350728+0000 2024-05-17T22:04:46.138 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/2269613822 2024-05-18T21:51:49.290057+0000 2024-05-17T22:04:46.138 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/3836557655 2024-05-18T21:54:34.350728+0000 2024-05-17T22:04:46.138 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6800/1294361014 2024-05-18T21:51:49.290057+0000 2024-05-17T22:04:46.138 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/3105018805 2024-05-18T21:42:52.099188+0000 2024-05-17T22:04:46.138 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/1170468325 2024-05-18T21:51:49.290057+0000 2024-05-17T22:04:46.138 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/2872531133 2024-05-18T21:42:52.099188+0000 2024-05-17T22:04:46.138 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/2823347896 2024-05-18T21:51:49.290057+0000 2024-05-17T22:04:46.138 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/771372867 2024-05-18T21:44:32.293094+0000 2024-05-17T22:04:46.138 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/1692398111 2024-05-18T21:54:34.350728+0000 2024-05-17T22:04:46.139 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/2856704707 2024-05-18T21:43:22.097723+0000 2024-05-17T22:04:46.139 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6826/2751824191 2024-05-18T21:49:32.033319+0000 2024-05-17T22:04:46.139 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.138:0/4106387402 2024-05-18T21:53:23.772077+0000 2024-05-17T22:04:46.139 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/498460124 2024-05-18T21:43:22.097723+0000 2024-05-17T22:04:46.139 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.138:6828/1369055740 2024-05-18T21:53:23.772077+0000 2024-05-17T22:04:46.139 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.138:0/768325965 2024-05-18T21:53:23.772077+0000 2024-05-17T22:04:46.139 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/2503945499 2024-05-18T21:54:34.350728+0000 2024-05-17T22:04:46.139 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6801/416442229 2024-05-18T21:54:34.350728+0000 2024-05-17T22:04:46.139 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6801/1677232931 2024-05-18T21:42:52.099188+0000 2024-05-17T22:04:46.139 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.138:6829/1369055740 2024-05-18T21:53:23.772077+0000 2024-05-17T22:04:46.139 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/1116047992 2024-05-18T21:54:34.350728+0000 2024-05-17T22:04:46.139 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/792958116 2024-05-18T21:54:34.350728+0000 2024-05-17T22:04:46.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:46.147+0000 7f32a128d700 1 -- 172.21.15.107:0/964742981 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f3288078990 msgr2=0x7f328807ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:46.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:46.147+0000 7f32a128d700 1 --2- 172.21.15.107:0/964742981 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f3288078990 0x7f328807ae40 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f328c004f90 tx=0x7f328c009e30 comp rx=0 tx=0).stop 2024-05-17T22:04:46.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:46.147+0000 7f32a128d700 1 -- 172.21.15.107:0/964742981 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f329c103780 msgr2=0x7f329c113df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:46.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:46.147+0000 7f32a128d700 1 --2- 172.21.15.107:0/964742981 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f329c103780 0x7f329c113df0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f3284002a50 tx=0x7f32840046b0 comp rx=0 tx=0).stop 2024-05-17T22:04:46.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:46.147+0000 7f32a128d700 1 -- 172.21.15.107:0/964742981 shutdown_connections 2024-05-17T22:04:46.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:46.147+0000 7f32a128d700 1 --2- 172.21.15.107:0/964742981 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f3288078990 0x7f328807ae40 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:46.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:46.147+0000 7f32a128d700 1 --2- 172.21.15.107:0/964742981 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f329c102de0 0x7f329c1138b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:46.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:46.147+0000 7f32a128d700 1 --2- 172.21.15.107:0/964742981 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f329c103780 0x7f329c113df0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:46.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:46.147+0000 7f32a128d700 1 -- 172.21.15.107:0/964742981 >> 172.21.15.107:0/964742981 conn(0x7f329c0f8990 msgr2=0x7f329c0fa6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:04:46.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:46.147+0000 7f32a128d700 1 -- 172.21.15.107:0/964742981 shutdown_connections 2024-05-17T22:04:46.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:46.147+0000 7f32a128d700 1 -- 172.21.15.107:0/964742981 wait complete. 2024-05-17T22:04:46.141 INFO:teuthology.orchestra.run.smithi107.stderr:listed 41 entries 2024-05-17T22:04:46.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:04:46 smithi138 ceph-mon[81376]: pgmap v343: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:04:46.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:04:46 smithi107 ceph-mon[93224]: pgmap v343: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:04:46.778 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-05-17T22:04:46.778 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T22:04:46.797 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 3722731a-1496-11ef-bc9a-c7b262605968 -- ceph osd blocklist ls 2024-05-17T22:04:47.091 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/mon.smithi107/config 2024-05-17T22:04:47.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:04:47 smithi138 ceph-mon[81376]: from='client.? 172.21.15.107:0/964742981' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-17T22:04:47.348 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:04:47 smithi107 ceph-mon[93224]: from='client.? 172.21.15.107:0/964742981' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-17T22:04:48.294 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:04:48 smithi107 ceph-mon[93224]: pgmap v344: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:04:48.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:04:48 smithi138 ceph-mon[81376]: pgmap v344: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:04:48.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:48.821+0000 7f3e272f5700 1 -- 172.21.15.107:0/3349804179 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e20102680 msgr2=0x7f3e200fbcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:48.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:48.821+0000 7f3e272f5700 1 --2- 172.21.15.107:0/3349804179 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e20102680 0x7f3e200fbcf0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f3e1c004940 tx=0x7f3e1c009e30 comp rx=0 tx=0).stop 2024-05-17T22:04:48.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:48.821+0000 7f3e272f5700 1 -- 172.21.15.107:0/3349804179 shutdown_connections 2024-05-17T22:04:48.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:48.821+0000 7f3e272f5700 1 --2- 172.21.15.107:0/3349804179 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e20102680 0x7f3e200fbcf0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:48.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:48.821+0000 7f3e272f5700 1 --2- 172.21.15.107:0/3349804179 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e20101ce0 0x7f3e201020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:48.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:48.821+0000 7f3e272f5700 1 -- 172.21.15.107:0/3349804179 >> 172.21.15.107:0/3349804179 conn(0x7f3e200f7850 msgr2=0x7f3e200f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:04:48.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:48.821+0000 7f3e272f5700 1 -- 172.21.15.107:0/3349804179 shutdown_connections 2024-05-17T22:04:48.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:48.821+0000 7f3e272f5700 1 -- 172.21.15.107:0/3349804179 wait complete. 2024-05-17T22:04:48.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:48.822+0000 7f3e272f5700 1 Processor -- start 2024-05-17T22:04:48.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:48.822+0000 7f3e272f5700 1 -- start start 2024-05-17T22:04:48.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:48.822+0000 7f3e272f5700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e20101ce0 0x7f3e20113b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:48.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:48.822+0000 7f3e272f5700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e20102680 0x7f3e201140a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:48.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:48.822+0000 7f3e272f5700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e201146f0 con 0x7f3e20102680 2024-05-17T22:04:48.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:48.822+0000 7f3e272f5700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e2010dae0 con 0x7f3e20101ce0 2024-05-17T22:04:48.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:48.822+0000 7f3e24890700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e20102680 0x7f3e201140a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:48.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:48.822+0000 7f3e24890700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e20102680 0x7f3e201140a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46852/0 (socket says 172.21.15.107:46852) 2024-05-17T22:04:48.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:48.822+0000 7f3e24890700 1 -- 172.21.15.107:0/1448105139 learned_addr learned my addr 172.21.15.107:0/1448105139 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-05-17T22:04:48.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:48.822+0000 7f3e25091700 1 --2- 172.21.15.107:0/1448105139 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e20101ce0 0x7f3e20113b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:48.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:48.822+0000 7f3e24890700 1 -- 172.21.15.107:0/1448105139 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e20101ce0 msgr2=0x7f3e20113b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:48.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:48.822+0000 7f3e24890700 1 --2- 172.21.15.107:0/1448105139 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e20101ce0 0x7f3e20113b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:48.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:48.822+0000 7f3e24890700 1 -- 172.21.15.107:0/1448105139 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e1c005040 con 0x7f3e20102680 2024-05-17T22:04:48.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:48.823+0000 7f3e24890700 1 --2- 172.21.15.107:0/1448105139 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e20102680 0x7f3e201140a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f3e1c009100 tx=0x7f3e1c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:48.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:48.823+0000 7f3e15ffb700 1 -- 172.21.15.107:0/1448105139 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3e1c00e040 con 0x7f3e20102680 2024-05-17T22:04:48.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:48.823+0000 7f3e15ffb700 1 -- 172.21.15.107:0/1448105139 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3e1c009af0 con 0x7f3e20102680 2024-05-17T22:04:48.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:48.823+0000 7f3e272f5700 1 -- 172.21.15.107:0/1448105139 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e2010dd60 con 0x7f3e20102680 2024-05-17T22:04:48.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:48.823+0000 7f3e15ffb700 1 -- 172.21.15.107:0/1448105139 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3e1c01c3f0 con 0x7f3e20102680 2024-05-17T22:04:48.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:48.823+0000 7f3e272f5700 1 -- 172.21.15.107:0/1448105139 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e2010e320 con 0x7f3e20102680 2024-05-17T22:04:48.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:48.824+0000 7f3e272f5700 1 -- 172.21.15.107:0/1448105139 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e200fd430 con 0x7f3e20102680 2024-05-17T22:04:48.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:48.825+0000 7f3e15ffb700 1 -- 172.21.15.107:0/1448105139 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 38) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f3e1c00baf0 con 0x7f3e20102680 2024-05-17T22:04:48.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:48.825+0000 7f3e15ffb700 1 --2- 172.21.15.107:0/1448105139 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f3e0c078990 0x7f3e0c07ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:48.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:48.826+0000 7f3e25091700 1 --2- 172.21.15.107:0/1448105139 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f3e0c078990 0x7f3e0c07ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:48.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:48.826+0000 7f3e15ffb700 1 -- 172.21.15.107:0/1448105139 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f3e1c09b9e0 con 0x7f3e20102680 2024-05-17T22:04:48.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:48.826+0000 7f3e25091700 1 --2- 172.21.15.107:0/1448105139 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f3e0c078990 0x7f3e0c07ae40 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3e100041d0 tx=0x7f3e100043c0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:48.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:48.830+0000 7f3e15ffb700 1 -- 172.21.15.107:0/1448105139 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f3e1c063620 con 0x7f3e20102680 2024-05-17T22:04:49.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:49.047+0000 7f3e272f5700 1 -- 172.21.15.107:0/1448105139 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f3e2005f5c0 con 0x7f3e20102680 2024-05-17T22:04:49.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:49.048+0000 7f3e15ffb700 1 -- 172.21.15.107:0/1448105139 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 41 entries v84) v1 ==== 81+0+2464 (secure 0 0 0) 0x7f3e1c062fc0 con 0x7f3e20102680 2024-05-17T22:04:49.040 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.138:6827/170628661 2024-05-18T22:00:16.636211+0000 2024-05-17T22:04:49.040 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.138:6825/2779634919 2024-05-18T21:59:52.171289+0000 2024-05-17T22:04:49.040 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.138:6824/2779634919 2024-05-18T21:59:52.171289+0000 2024-05-17T22:04:49.040 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/1436894442 2024-05-18T21:54:34.350728+0000 2024-05-17T22:04:49.040 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/3048013464 2024-05-18T21:42:52.099188+0000 2024-05-17T22:04:49.040 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.138:0/1997364823 2024-05-18T21:53:23.772077+0000 2024-05-17T22:04:49.041 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/2414403835 2024-05-18T21:44:32.293094+0000 2024-05-17T22:04:49.041 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6801/3405524625 2024-05-18T21:44:32.293094+0000 2024-05-17T22:04:49.041 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6801/1294361014 2024-05-18T21:51:49.290057+0000 2024-05-17T22:04:49.041 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6800/1718632476 2024-05-18T21:43:22.097723+0000 2024-05-17T22:04:49.041 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/3170894163 2024-05-18T21:43:22.097723+0000 2024-05-17T22:04:49.041 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.138:6826/170628661 2024-05-18T22:00:16.636211+0000 2024-05-17T22:04:49.041 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6800/3405524625 2024-05-18T21:44:32.293094+0000 2024-05-17T22:04:49.041 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6800/1677232931 2024-05-18T21:42:52.099188+0000 2024-05-17T22:04:49.041 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6827/2751824191 2024-05-18T21:49:32.033319+0000 2024-05-17T22:04:49.041 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.138:0/1723941910 2024-05-18T21:53:23.772077+0000 2024-05-17T22:04:49.042 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/1077444017 2024-05-18T21:44:32.293094+0000 2024-05-17T22:04:49.042 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/239507720 2024-05-18T21:51:49.290057+0000 2024-05-17T22:04:49.042 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6801/1718632476 2024-05-18T21:43:22.097723+0000 2024-05-17T22:04:49.042 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6800/416442229 2024-05-18T21:54:34.350728+0000 2024-05-17T22:04:49.042 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/2269613822 2024-05-18T21:51:49.290057+0000 2024-05-17T22:04:49.042 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/3836557655 2024-05-18T21:54:34.350728+0000 2024-05-17T22:04:49.042 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6800/1294361014 2024-05-18T21:51:49.290057+0000 2024-05-17T22:04:49.042 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/3105018805 2024-05-18T21:42:52.099188+0000 2024-05-17T22:04:49.042 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/1170468325 2024-05-18T21:51:49.290057+0000 2024-05-17T22:04:49.042 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/2872531133 2024-05-18T21:42:52.099188+0000 2024-05-17T22:04:49.042 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/2823347896 2024-05-18T21:51:49.290057+0000 2024-05-17T22:04:49.043 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/771372867 2024-05-18T21:44:32.293094+0000 2024-05-17T22:04:49.043 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/1692398111 2024-05-18T21:54:34.350728+0000 2024-05-17T22:04:49.043 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/2856704707 2024-05-18T21:43:22.097723+0000 2024-05-17T22:04:49.043 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6826/2751824191 2024-05-18T21:49:32.033319+0000 2024-05-17T22:04:49.043 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.138:0/4106387402 2024-05-18T21:53:23.772077+0000 2024-05-17T22:04:49.043 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/498460124 2024-05-18T21:43:22.097723+0000 2024-05-17T22:04:49.043 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.138:6828/1369055740 2024-05-18T21:53:23.772077+0000 2024-05-17T22:04:49.043 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.138:0/768325965 2024-05-18T21:53:23.772077+0000 2024-05-17T22:04:49.043 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/2503945499 2024-05-18T21:54:34.350728+0000 2024-05-17T22:04:49.043 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6801/416442229 2024-05-18T21:54:34.350728+0000 2024-05-17T22:04:49.043 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6801/1677232931 2024-05-18T21:42:52.099188+0000 2024-05-17T22:04:49.044 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.138:6829/1369055740 2024-05-18T21:53:23.772077+0000 2024-05-17T22:04:49.044 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/1116047992 2024-05-18T21:54:34.350728+0000 2024-05-17T22:04:49.044 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/792958116 2024-05-18T21:54:34.350728+0000 2024-05-17T22:04:49.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:49.050+0000 7f3e272f5700 1 -- 172.21.15.107:0/1448105139 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f3e0c078990 msgr2=0x7f3e0c07ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:49.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:49.050+0000 7f3e272f5700 1 --2- 172.21.15.107:0/1448105139 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f3e0c078990 0x7f3e0c07ae40 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3e100041d0 tx=0x7f3e100043c0 comp rx=0 tx=0).stop 2024-05-17T22:04:49.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:49.050+0000 7f3e272f5700 1 -- 172.21.15.107:0/1448105139 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e20102680 msgr2=0x7f3e201140a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:49.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:49.050+0000 7f3e272f5700 1 --2- 172.21.15.107:0/1448105139 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e20102680 0x7f3e201140a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f3e1c009100 tx=0x7f3e1c009e30 comp rx=0 tx=0).stop 2024-05-17T22:04:49.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:49.050+0000 7f3e272f5700 1 -- 172.21.15.107:0/1448105139 shutdown_connections 2024-05-17T22:04:49.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:49.050+0000 7f3e272f5700 1 --2- 172.21.15.107:0/1448105139 >> [v2:172.21.15.107:6800/3205268226,v1:172.21.15.107:6801/3205268226] conn(0x7f3e0c078990 0x7f3e0c07ae40 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:49.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:49.050+0000 7f3e272f5700 1 --2- 172.21.15.107:0/1448105139 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e20101ce0 0x7f3e20113b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:49.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:49.050+0000 7f3e272f5700 1 --2- 172.21.15.107:0/1448105139 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e20102680 0x7f3e201140a0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:49.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:49.050+0000 7f3e272f5700 1 -- 172.21.15.107:0/1448105139 >> 172.21.15.107:0/1448105139 conn(0x7f3e200f7850 msgr2=0x7f3e200f8f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:04:49.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:49.050+0000 7f3e272f5700 1 -- 172.21.15.107:0/1448105139 shutdown_connections 2024-05-17T22:04:49.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-05-17T22:04:49.050+0000 7f3e272f5700 1 -- 172.21.15.107:0/1448105139 wait complete. 2024-05-17T22:04:49.045 INFO:teuthology.orchestra.run.smithi107.stderr:listed 41 entries 2024-05-17T22:04:49.828 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi107.front.sepia.ceph.com... 2024-05-17T22:04:49.829 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:04:49.829 DEBUG:teuthology.orchestra.run.smithi107:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-17T22:04:49.858 INFO:teuthology.orchestra.run:waiting for 300 2024-05-17T22:04:50.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:04:50 smithi107 ceph-mon[93224]: pgmap v345: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:04:50.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:04:50 smithi107 ceph-mon[93224]: from='client.? 172.21.15.107:0/1448105139' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-17T22:04:50.102 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:04:50 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T22:04:50.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:04:50 smithi138 ceph-mon[81376]: pgmap v345: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:04:50.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:04:50 smithi138 ceph-mon[81376]: from='client.? 172.21.15.107:0/1448105139' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-17T22:04:50.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:04:50 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T22:04:52.305 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:04:52 smithi107 ceph-mon[93224]: pgmap v346: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:04:52.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:04:52 smithi138 ceph-mon[81376]: pgmap v346: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:04:54.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:04:54 smithi138 ceph-mon[81376]: pgmap v347: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:04:54.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:04:54 smithi107 ceph-mon[93224]: pgmap v347: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:04:56.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:04:56 smithi138 ceph-mon[81376]: pgmap v348: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:04:56.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:04:56 smithi107 ceph-mon[93224]: pgmap v348: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:04:58.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:04:58 smithi138 ceph-mon[81376]: pgmap v349: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:04:58.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:04:58 smithi107 ceph-mon[93224]: pgmap v349: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:05:00.126 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:05:00 smithi107 ceph-mon[93224]: pgmap v350: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:05:00.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:05:00 smithi138 ceph-mon[81376]: pgmap v350: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:05:01.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:05:01 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:05:01.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:05:01 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:05:01.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:05:01 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:05:01.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:05:01 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:05:01.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:05:01 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:05:01.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:05:01 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:05:02.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:05:02 smithi138 ceph-mon[81376]: pgmap v351: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:05:02.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:05:02 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:05:02.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:05:02 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:05:02.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:05:02 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:05:02.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:05:02 smithi107 ceph-mon[93224]: pgmap v351: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:05:02.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:05:02 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:05:02.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:05:02 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:05:02.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:05:02 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:05:04.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:05:04 smithi138 ceph-mon[81376]: pgmap v352: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:05:04.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:05:04 smithi107 ceph-mon[93224]: pgmap v352: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:05:05.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:05:05 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T22:05:05.351 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:05:05 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T22:05:06.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:05:06 smithi138 ceph-mon[81376]: pgmap v353: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:05:06.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:05:06 smithi107 ceph-mon[93224]: pgmap v353: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:05:08.326 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:05:08 smithi138 ceph-mon[81376]: pgmap v354: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:05:08.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:05:08 smithi107 ceph-mon[93224]: pgmap v354: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:05:10.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:05:10 smithi107 ceph-mon[93224]: pgmap v355: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:05:10.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:05:10 smithi138 ceph-mon[81376]: pgmap v355: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:05:12.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:05:12 smithi107 ceph-mon[93224]: pgmap v356: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:05:12.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:05:12 smithi138 ceph-mon[81376]: pgmap v356: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:05:14.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:05:14 smithi107 ceph-mon[93224]: pgmap v357: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:05:14.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:05:14 smithi138 ceph-mon[81376]: pgmap v357: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:05:16.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:05:16 smithi107 ceph-mon[93224]: pgmap v358: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:05:16.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:05:16 smithi138 ceph-mon[81376]: pgmap v358: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:05:18.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:05:18 smithi107 ceph-mon[93224]: pgmap v359: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:05:18.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:05:18 smithi138 ceph-mon[81376]: pgmap v359: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:05:20.354 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:05:20 smithi107 ceph-mon[93224]: pgmap v360: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:05:20.355 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:05:20 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T22:05:20.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:05:20 smithi138 ceph-mon[81376]: pgmap v360: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:05:20.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:05:20 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T22:05:22.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:05:22 smithi107 ceph-mon[93224]: pgmap v361: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:05:22.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:05:22 smithi138 ceph-mon[81376]: pgmap v361: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:05:24.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:05:24 smithi107 ceph-mon[93224]: pgmap v362: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:05:24.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:05:24 smithi138 ceph-mon[81376]: pgmap v362: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:05:26.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:05:26 smithi138 ceph-mon[81376]: pgmap v363: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:05:26.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:05:26 smithi107 ceph-mon[93224]: pgmap v363: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:05:28.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:05:28 smithi138 ceph-mon[81376]: pgmap v364: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:05:28.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:05:28 smithi107 ceph-mon[93224]: pgmap v364: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:05:30.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:05:30 smithi138 ceph-mon[81376]: pgmap v365: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:05:30.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:05:30 smithi107 ceph-mon[93224]: pgmap v365: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:05:32.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:05:32 smithi138 ceph-mon[81376]: pgmap v366: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:05:32.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:05:32 smithi107 ceph-mon[93224]: pgmap v366: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:05:34.379 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:05:34 smithi138 ceph-mon[81376]: pgmap v367: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:05:34.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:05:34 smithi107 ceph-mon[93224]: pgmap v367: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:05:35.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:05:35 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T22:05:35.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:05:35 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T22:05:36.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:05:36 smithi138 ceph-mon[81376]: pgmap v368: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:05:36.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:05:36 smithi107 ceph-mon[93224]: pgmap v368: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:05:38.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:05:38 smithi138 ceph-mon[81376]: pgmap v369: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:05:38.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:05:38 smithi107 ceph-mon[93224]: pgmap v369: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:05:40.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:05:40 smithi138 ceph-mon[81376]: pgmap v370: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:05:40.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:05:40 smithi107 ceph-mon[93224]: pgmap v370: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:05:42.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:05:42 smithi138 ceph-mon[81376]: pgmap v371: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:05:42.577 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:05:42 smithi107 ceph-mon[93224]: pgmap v371: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:05:44.576 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:05:44 smithi138 ceph-mon[81376]: pgmap v372: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:05:44.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:05:44 smithi107 ceph-mon[93224]: pgmap v372: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:05:46.577 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:05:46 smithi138 ceph-mon[81376]: pgmap v373: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:05:46.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:05:46 smithi107 ceph-mon[93224]: pgmap v373: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:05:48.327 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:05:48 smithi138 ceph-mon[81376]: pgmap v374: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:05:48.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:05:48 smithi107 ceph-mon[93224]: pgmap v374: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:05:50.577 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:05:50 smithi138 ceph-mon[81376]: pgmap v375: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:05:50.577 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:05:50 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T22:05:50.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:05:50 smithi107 ceph-mon[93224]: pgmap v375: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:05:50.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:05:50 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T22:05:52.577 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:05:52 smithi138 ceph-mon[81376]: pgmap v376: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:05:52.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:05:52 smithi107 ceph-mon[93224]: pgmap v376: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:05:54.577 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:05:54 smithi138 ceph-mon[81376]: pgmap v377: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:05:54.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:05:54 smithi107 ceph-mon[93224]: pgmap v377: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:05:56.577 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:05:56 smithi138 ceph-mon[81376]: pgmap v378: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:05:56.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:05:56 smithi107 ceph-mon[93224]: pgmap v378: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:05:58.577 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:05:58 smithi138 ceph-mon[81376]: pgmap v379: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:05:58.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:05:58 smithi107 ceph-mon[93224]: pgmap v379: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:06:00.577 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:06:00 smithi138 ceph-mon[81376]: pgmap v380: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:06:00.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:06:00 smithi107 ceph-mon[93224]: pgmap v380: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:06:02.577 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:06:02 smithi138 ceph-mon[81376]: pgmap v381: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:06:02.577 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:06:02 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:06:02.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:06:02 smithi107 ceph-mon[93224]: pgmap v381: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:06:02.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:06:02 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:06:03.577 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:06:03 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:06:03.577 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:06:03 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:06:03.577 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:06:03 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:06:03.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:06:03 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:06:03.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:06:03 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:06:03.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:06:03 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:06:04.478 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:06:04 smithi138 ceph-mon[81376]: pgmap v382: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:06:04.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:06:04 smithi107 ceph-mon[93224]: pgmap v382: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:06:05.577 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:06:05 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T22:06:05.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:06:05 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T22:06:06.577 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:06:06 smithi138 ceph-mon[81376]: pgmap v383: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:06:06.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:06:06 smithi107 ceph-mon[93224]: pgmap v383: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:06:08.577 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:06:08 smithi138 ceph-mon[81376]: pgmap v384: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:06:08.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:06:08 smithi107 ceph-mon[93224]: pgmap v384: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:06:10.577 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:06:10 smithi138 ceph-mon[81376]: pgmap v385: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:06:10.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:06:10 smithi107 ceph-mon[93224]: pgmap v385: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:06:12.577 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:06:12 smithi138 ceph-mon[81376]: pgmap v386: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:06:12.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:06:12 smithi107 ceph-mon[93224]: pgmap v386: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:06:14.577 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:06:14 smithi138 ceph-mon[81376]: pgmap v387: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:06:14.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:06:14 smithi107 ceph-mon[93224]: pgmap v387: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:06:16.577 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:06:16 smithi138 ceph-mon[81376]: pgmap v388: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:06:16.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:06:16 smithi107 ceph-mon[93224]: pgmap v388: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:06:18.577 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:06:18 smithi138 ceph-mon[81376]: pgmap v389: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:06:18.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:06:18 smithi107 ceph-mon[93224]: pgmap v389: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:06:20.577 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:06:20 smithi138 ceph-mon[81376]: pgmap v390: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:06:20.577 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:06:20 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T22:06:20.605 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:06:20 smithi107 ceph-mon[93224]: pgmap v390: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:06:20.606 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:06:20 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T22:06:22.577 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:06:22 smithi138 ceph-mon[81376]: pgmap v391: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:06:22.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:06:22 smithi107 ceph-mon[93224]: pgmap v391: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:06:24.577 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:06:24 smithi138 ceph-mon[81376]: pgmap v392: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:06:24.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:06:24 smithi107 ceph-mon[93224]: pgmap v392: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:06:26.577 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:06:26 smithi138 ceph-mon[81376]: pgmap v393: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:06:26.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:06:26 smithi107 ceph-mon[93224]: pgmap v393: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:06:28.577 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:06:28 smithi138 ceph-mon[81376]: pgmap v394: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:06:28.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:06:28 smithi107 ceph-mon[93224]: pgmap v394: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:06:29.577 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:06:29 smithi138 ceph-mon[81376]: pgmap v395: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:06:29.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:06:29 smithi107 ceph-mon[93224]: pgmap v395: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:06:31.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:06:31 smithi107 ceph-mon[93224]: pgmap v396: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:06:32.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:06:31 smithi138 ceph-mon[81376]: pgmap v396: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:06:33.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:06:33 smithi107 ceph-mon[93224]: pgmap v397: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:06:34.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:06:33 smithi138 ceph-mon[81376]: pgmap v397: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:06:34.979 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:06:34 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T22:06:35.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:06:34 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T22:06:36.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:06:35 smithi138 ceph-mon[81376]: pgmap v398: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:06:36.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:06:35 smithi107 ceph-mon[93224]: pgmap v398: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:06:38.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:06:37 smithi138 ceph-mon[81376]: pgmap v399: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:06:38.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:06:37 smithi107 ceph-mon[93224]: pgmap v399: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:06:39.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:06:39 smithi107 ceph-mon[93224]: pgmap v400: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:06:40.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:06:39 smithi138 ceph-mon[81376]: pgmap v400: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:06:42.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:06:41 smithi138 ceph-mon[81376]: pgmap v401: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:06:42.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:06:41 smithi107 ceph-mon[93224]: pgmap v401: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:06:44.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:06:43 smithi138 ceph-mon[81376]: pgmap v402: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:06:44.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:06:43 smithi107 ceph-mon[93224]: pgmap v402: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:06:46.077 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:06:45 smithi138 ceph-mon[81376]: pgmap v403: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:06:46.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:06:45 smithi107 ceph-mon[93224]: pgmap v403: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:06:48.078 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:06:47 smithi138 ceph-mon[81376]: pgmap v404: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:06:48.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:06:47 smithi107 ceph-mon[93224]: pgmap v404: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:06:49.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:06:49 smithi107 ceph-mon[93224]: pgmap v405: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:06:49.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:06:49 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T22:06:50.078 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:06:49 smithi138 ceph-mon[81376]: pgmap v405: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:06:50.078 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:06:49 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T22:06:52.078 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:06:51 smithi138 ceph-mon[81376]: pgmap v406: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:06:52.104 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:06:51 smithi107 ceph-mon[93224]: pgmap v406: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:06:54.078 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:06:53 smithi138 ceph-mon[81376]: pgmap v407: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:06:54.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:06:53 smithi107 ceph-mon[93224]: pgmap v407: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:06:56.078 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:06:55 smithi138 ceph-mon[81376]: pgmap v408: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:06:56.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:06:55 smithi107 ceph-mon[93224]: pgmap v408: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:06:58.078 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:06:57 smithi138 ceph-mon[81376]: pgmap v409: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:06:58.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:06:57 smithi107 ceph-mon[93224]: pgmap v409: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:06:59.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:06:59 smithi107 ceph-mon[93224]: pgmap v410: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:07:00.078 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:06:59 smithi138 ceph-mon[81376]: pgmap v410: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:07:02.078 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:07:01 smithi138 ceph-mon[81376]: pgmap v411: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:07:02.104 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:07:01 smithi107 ceph-mon[93224]: pgmap v411: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:07:03.078 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:07:02 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:07:03.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:07:02 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:07:04.078 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:07:03 smithi138 ceph-mon[81376]: pgmap v412: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:07:04.078 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:07:03 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:07:04.078 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:07:03 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:07:04.078 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:07:03 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:07:04.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:07:03 smithi107 ceph-mon[93224]: pgmap v412: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:07:04.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:07:03 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:07:04.104 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:07:03 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:07:04.104 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:07:03 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:07:04.979 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:07:04 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T22:07:05.104 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:07:04 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T22:07:06.078 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:07:05 smithi138 ceph-mon[81376]: pgmap v413: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:07:06.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:07:05 smithi107 ceph-mon[93224]: pgmap v413: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:07:08.078 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:07:07 smithi138 ceph-mon[81376]: pgmap v414: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:07:08.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:07:07 smithi107 ceph-mon[93224]: pgmap v414: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:07:09.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:07:09 smithi107 ceph-mon[93224]: pgmap v415: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:07:10.078 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:07:09 smithi138 ceph-mon[81376]: pgmap v415: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:07:12.078 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:07:11 smithi138 ceph-mon[81376]: pgmap v416: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:07:12.104 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:07:11 smithi107 ceph-mon[93224]: pgmap v416: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:07:14.078 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:07:13 smithi138 ceph-mon[81376]: pgmap v417: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:07:14.104 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:07:13 smithi107 ceph-mon[93224]: pgmap v417: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:07:16.078 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:07:15 smithi138 ceph-mon[81376]: pgmap v418: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:07:16.104 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:07:15 smithi107 ceph-mon[93224]: pgmap v418: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:07:18.078 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:07:17 smithi138 ceph-mon[81376]: pgmap v419: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:07:18.104 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:07:17 smithi107 ceph-mon[93224]: pgmap v419: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:07:19.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:07:19 smithi107 ceph-mon[93224]: pgmap v420: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:07:19.854 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:07:19 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T22:07:20.078 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:07:19 smithi138 ceph-mon[81376]: pgmap v420: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:07:20.078 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:07:19 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T22:07:22.078 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:07:21 smithi138 ceph-mon[81376]: pgmap v421: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:07:22.104 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:07:21 smithi107 ceph-mon[93224]: pgmap v421: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:07:24.078 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:07:23 smithi138 ceph-mon[81376]: pgmap v422: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:07:24.104 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:07:23 smithi107 ceph-mon[93224]: pgmap v422: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:07:26.078 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:07:25 smithi138 ceph-mon[81376]: pgmap v423: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:07:26.104 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:07:25 smithi107 ceph-mon[93224]: pgmap v423: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:07:28.078 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:07:27 smithi138 ceph-mon[81376]: pgmap v424: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:07:28.104 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:07:27 smithi107 ceph-mon[93224]: pgmap v424: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:07:30.078 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:07:29 smithi138 ceph-mon[81376]: pgmap v425: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:07:30.104 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:07:29 smithi107 ceph-mon[93224]: pgmap v425: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:07:32.078 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:07:31 smithi138 ceph-mon[81376]: pgmap v426: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:07:32.104 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:07:31 smithi107 ceph-mon[93224]: pgmap v426: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:07:34.078 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:07:33 smithi138 ceph-mon[81376]: pgmap v427: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:07:34.104 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:07:33 smithi107 ceph-mon[93224]: pgmap v427: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:07:35.078 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:07:34 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T22:07:35.104 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:07:34 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T22:07:36.078 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:07:35 smithi138 ceph-mon[81376]: pgmap v428: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:07:36.104 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:07:35 smithi107 ceph-mon[93224]: pgmap v428: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:07:38.078 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:07:37 smithi138 ceph-mon[81376]: pgmap v429: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:07:38.104 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:07:37 smithi107 ceph-mon[93224]: pgmap v429: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:07:40.078 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:07:39 smithi138 ceph-mon[81376]: pgmap v430: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:07:40.104 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:07:39 smithi107 ceph-mon[93224]: pgmap v430: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:07:42.079 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:07:41 smithi138 ceph-mon[81376]: pgmap v431: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:07:42.104 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:07:41 smithi107 ceph-mon[93224]: pgmap v431: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:07:44.105 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:07:43 smithi107 ceph-mon[93224]: pgmap v432: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:07:44.328 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:07:43 smithi138 ceph-mon[81376]: pgmap v432: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:07:46.104 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:07:45 smithi107 ceph-mon[93224]: pgmap v433: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:07:46.328 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:07:45 smithi138 ceph-mon[81376]: pgmap v433: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:07:48.104 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:07:47 smithi107 ceph-mon[93224]: pgmap v434: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:07:48.328 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:07:47 smithi138 ceph-mon[81376]: pgmap v434: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:07:50.104 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:07:49 smithi107 ceph-mon[93224]: pgmap v435: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:07:50.104 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:07:49 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T22:07:50.328 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:07:49 smithi138 ceph-mon[81376]: pgmap v435: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:07:50.329 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:07:49 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T22:07:52.105 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:07:51 smithi107 ceph-mon[93224]: pgmap v436: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:07:52.329 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:07:51 smithi138 ceph-mon[81376]: pgmap v436: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:07:54.104 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:07:53 smithi107 ceph-mon[93224]: pgmap v437: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:07:54.328 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:07:53 smithi138 ceph-mon[81376]: pgmap v437: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:07:56.329 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:07:55 smithi138 ceph-mon[81376]: pgmap v438: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:07:56.354 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:07:55 smithi107 ceph-mon[93224]: pgmap v438: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:07:58.329 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:07:57 smithi138 ceph-mon[81376]: pgmap v439: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:07:58.354 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:07:57 smithi107 ceph-mon[93224]: pgmap v439: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:08:00.329 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:07:59 smithi138 ceph-mon[81376]: pgmap v440: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:08:00.355 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:07:59 smithi107 ceph-mon[93224]: pgmap v440: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:08:02.307 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:08:01 smithi107 ceph-mon[93224]: pgmap v441: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:08:02.329 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:08:01 smithi138 ceph-mon[81376]: pgmap v441: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:08:04.131 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:08:03 smithi138 ceph-mon[81376]: pgmap v442: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:08:04.131 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:08:03 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:08:04.131 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:08:03 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:08:04.131 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:08:03 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:08:04.131 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:08:03 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:08:04.355 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:08:03 smithi107 ceph-mon[93224]: pgmap v442: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:08:04.355 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:08:03 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:08:04.355 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:08:03 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:08:04.355 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:08:03 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:08:04.355 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:08:03 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:08:05.329 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:08:04 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T22:08:05.354 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:08:04 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T22:08:06.329 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:08:05 smithi138 ceph-mon[81376]: pgmap v443: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:08:06.354 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:08:05 smithi107 ceph-mon[93224]: pgmap v443: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:08:08.329 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:08:07 smithi138 ceph-mon[81376]: pgmap v444: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:08:08.355 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:08:07 smithi107 ceph-mon[93224]: pgmap v444: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:08:10.329 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:08:09 smithi138 ceph-mon[81376]: pgmap v445: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:08:10.355 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:08:09 smithi107 ceph-mon[93224]: pgmap v445: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:08:12.308 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:08:11 smithi107 ceph-mon[93224]: pgmap v446: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:08:12.329 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:08:11 smithi138 ceph-mon[81376]: pgmap v446: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:08:14.329 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:08:13 smithi138 ceph-mon[81376]: pgmap v447: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:08:14.355 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:08:13 smithi107 ceph-mon[93224]: pgmap v447: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:08:16.329 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:08:15 smithi138 ceph-mon[81376]: pgmap v448: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:08:16.355 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:08:15 smithi107 ceph-mon[93224]: pgmap v448: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:08:18.329 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:08:17 smithi138 ceph-mon[81376]: pgmap v449: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:08:18.354 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:08:17 smithi107 ceph-mon[93224]: pgmap v449: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:08:20.329 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:08:19 smithi138 ceph-mon[81376]: pgmap v450: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:08:20.329 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:08:19 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T22:08:20.355 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:08:19 smithi107 ceph-mon[93224]: pgmap v450: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:08:20.355 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:08:19 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T22:08:22.308 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:08:21 smithi107 ceph-mon[93224]: pgmap v451: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:08:22.329 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:08:21 smithi138 ceph-mon[81376]: pgmap v451: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:08:24.329 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:08:23 smithi138 ceph-mon[81376]: pgmap v452: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:08:24.355 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:08:23 smithi107 ceph-mon[93224]: pgmap v452: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:08:26.329 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:08:25 smithi138 ceph-mon[81376]: pgmap v453: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:08:26.355 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:08:25 smithi107 ceph-mon[93224]: pgmap v453: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:08:28.329 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:08:27 smithi138 ceph-mon[81376]: pgmap v454: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:08:28.355 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:08:27 smithi107 ceph-mon[93224]: pgmap v454: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:08:30.329 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:08:29 smithi138 ceph-mon[81376]: pgmap v455: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:08:30.355 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:08:29 smithi107 ceph-mon[93224]: pgmap v455: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:08:32.308 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:08:31 smithi107 ceph-mon[93224]: pgmap v456: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:08:32.329 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:08:31 smithi138 ceph-mon[81376]: pgmap v456: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:08:34.329 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:08:33 smithi138 ceph-mon[81376]: pgmap v457: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:08:34.355 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:08:33 smithi107 ceph-mon[93224]: pgmap v457: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:08:35.329 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:08:34 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T22:08:35.355 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:08:34 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T22:08:36.329 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:08:35 smithi138 ceph-mon[81376]: pgmap v458: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:08:36.355 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:08:35 smithi107 ceph-mon[93224]: pgmap v458: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:08:38.329 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:08:37 smithi138 ceph-mon[81376]: pgmap v459: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:08:38.355 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:08:37 smithi107 ceph-mon[93224]: pgmap v459: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:08:40.329 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:08:39 smithi138 ceph-mon[81376]: pgmap v460: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:08:40.355 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:08:39 smithi107 ceph-mon[93224]: pgmap v460: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:08:42.309 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:08:41 smithi107 ceph-mon[93224]: pgmap v461: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:08:42.329 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:08:41 smithi138 ceph-mon[81376]: pgmap v461: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:08:44.329 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:08:43 smithi138 ceph-mon[81376]: pgmap v462: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:08:44.355 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:08:43 smithi107 ceph-mon[93224]: pgmap v462: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:08:46.329 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:08:45 smithi138 ceph-mon[81376]: pgmap v463: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:08:46.355 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:08:45 smithi107 ceph-mon[93224]: pgmap v463: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:08:48.329 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:08:47 smithi138 ceph-mon[81376]: pgmap v464: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:08:48.355 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:08:47 smithi107 ceph-mon[93224]: pgmap v464: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:08:50.330 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:08:49 smithi138 ceph-mon[81376]: pgmap v465: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:08:50.330 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:08:49 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T22:08:50.355 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:08:49 smithi107 ceph-mon[93224]: pgmap v465: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:08:50.355 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:08:49 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T22:08:52.308 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:08:51 smithi107 ceph-mon[93224]: pgmap v466: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:08:52.329 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:08:51 smithi138 ceph-mon[81376]: pgmap v466: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:08:54.329 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:08:53 smithi138 ceph-mon[81376]: pgmap v467: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:08:54.355 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:08:53 smithi107 ceph-mon[93224]: pgmap v467: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:08:56.329 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:08:55 smithi138 ceph-mon[81376]: pgmap v468: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:08:56.355 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:08:55 smithi107 ceph-mon[93224]: pgmap v468: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:08:58.329 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:08:57 smithi138 ceph-mon[81376]: pgmap v469: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:08:58.355 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:08:57 smithi107 ceph-mon[93224]: pgmap v469: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:09:00.329 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:08:59 smithi138 ceph-mon[81376]: pgmap v470: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:09:00.355 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:08:59 smithi107 ceph-mon[93224]: pgmap v470: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:09:02.309 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:09:01 smithi107 ceph-mon[93224]: pgmap v471: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:09:02.329 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:09:01 smithi138 ceph-mon[81376]: pgmap v471: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:09:04.329 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:09:03 smithi138 ceph-mon[81376]: pgmap v472: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:09:04.355 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:09:03 smithi107 ceph-mon[93224]: pgmap v472: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:09:05.329 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:09:04 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:09:05.330 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:09:04 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T22:09:05.330 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:09:04 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:09:05.330 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:09:04 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:09:05.330 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:09:04 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:09:05.355 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:09:04 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:09:05.356 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:09:04 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T22:09:05.356 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:09:04 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:09:05.356 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:09:04 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:09:05.356 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:09:04 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' 2024-05-17T22:09:06.330 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:09:05 smithi138 ceph-mon[81376]: pgmap v473: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:09:06.355 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:09:05 smithi107 ceph-mon[93224]: pgmap v473: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:09:08.329 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:09:07 smithi138 ceph-mon[81376]: pgmap v474: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:09:08.355 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:09:07 smithi107 ceph-mon[93224]: pgmap v474: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:09:10.330 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:09:09 smithi138 ceph-mon[81376]: pgmap v475: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:09:10.355 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:09:09 smithi107 ceph-mon[93224]: pgmap v475: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:09:12.308 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:09:11 smithi107 ceph-mon[93224]: pgmap v476: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:09:12.329 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:09:11 smithi138 ceph-mon[81376]: pgmap v476: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:09:14.330 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:09:13 smithi138 ceph-mon[81376]: pgmap v477: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:09:14.355 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:09:13 smithi107 ceph-mon[93224]: pgmap v477: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:09:16.330 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:09:16 smithi138 ceph-mon[81376]: pgmap v478: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:09:16.355 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:09:16 smithi107 ceph-mon[93224]: pgmap v478: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:09:18.330 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:09:18 smithi138 ceph-mon[81376]: pgmap v479: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:09:18.355 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:09:18 smithi107 ceph-mon[93224]: pgmap v479: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:09:20.330 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:09:20 smithi138 ceph-mon[81376]: pgmap v480: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:09:20.330 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:09:20 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T22:09:20.355 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:09:20 smithi107 ceph-mon[93224]: pgmap v480: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:09:20.356 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:09:20 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T22:09:22.308 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:09:22 smithi107 ceph-mon[93224]: pgmap v481: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:09:22.330 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:09:22 smithi138 ceph-mon[81376]: pgmap v481: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:09:24.330 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:09:24 smithi138 ceph-mon[81376]: pgmap v482: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:09:24.356 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:09:24 smithi107 ceph-mon[93224]: pgmap v482: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:09:26.330 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:09:26 smithi138 ceph-mon[81376]: pgmap v483: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:09:26.355 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:09:26 smithi107 ceph-mon[93224]: pgmap v483: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:09:28.330 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:09:28 smithi138 ceph-mon[81376]: pgmap v484: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:09:28.356 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:09:28 smithi107 ceph-mon[93224]: pgmap v484: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:09:30.330 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:09:30 smithi138 ceph-mon[81376]: pgmap v485: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:09:30.356 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:09:30 smithi107 ceph-mon[93224]: pgmap v485: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:09:32.309 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:09:32 smithi107 ceph-mon[93224]: pgmap v486: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:09:32.330 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:09:32 smithi138 ceph-mon[81376]: pgmap v486: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:09:34.330 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:09:34 smithi138 ceph-mon[81376]: pgmap v487: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:09:34.355 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:09:34 smithi107 ceph-mon[93224]: pgmap v487: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:09:35.330 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:09:35 smithi138 ceph-mon[81376]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T22:09:35.356 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:09:35 smithi107 ceph-mon[93224]: from='mgr.34100 172.21.15.107:0/1923068332' entity='mgr.smithi107.xhhyiw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-17T22:09:36.330 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:09:36 smithi138 ceph-mon[81376]: pgmap v488: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:09:36.355 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:09:36 smithi107 ceph-mon[93224]: pgmap v488: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:09:38.330 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:09:38 smithi138 ceph-mon[81376]: pgmap v489: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:09:38.356 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:09:38 smithi107 ceph-mon[93224]: pgmap v489: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:09:40.330 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:09:40 smithi138 ceph-mon[81376]: pgmap v490: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:09:40.356 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:09:40 smithi107 ceph-mon[93224]: pgmap v490: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:09:42.309 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:09:42 smithi107 ceph-mon[93224]: pgmap v491: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:09:42.330 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:09:42 smithi138 ceph-mon[81376]: pgmap v491: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:09:44.330 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:09:44 smithi138 ceph-mon[81376]: pgmap v492: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:09:44.356 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:09:44 smithi107 ceph-mon[93224]: pgmap v492: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-17T22:09:46.330 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:09:46 smithi138 ceph-mon[81376]: pgmap v493: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:09:46.356 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:09:46 smithi107 ceph-mon[93224]: pgmap v493: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:09:48.334 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:09:48 smithi138 ceph-mon[81376]: pgmap v494: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:09:48.356 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:09:48 smithi107 ceph-mon[93224]: pgmap v494: 65 pgs: 65 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-17T22:09:48.929 ERROR:tasks.cephfs.fuse_mount:process failed to terminate after unmount. This probably indicates a bug within ceph-fuse. 2024-05-17T22:09:48.929 ERROR:teuthology.run_tasks:Manager failed: ceph-fuse Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_2127e87e3419439de550d2abbd7ed2aee2cdfb0e/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.10/contextlib.py", line 142, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_931809a9e5d295bd61984693cac44fe131ded4bc/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph-c_931809a9e5d295bd61984693cac44fe131ded4bc/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_2127e87e3419439de550d2abbd7ed2aee2cdfb0e/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_2127e87e3419439de550d2abbd7ed2aee2cdfb0e/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-05-17T22:09:48.931 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-05-17T22:09:48.944 INFO:tasks.cephadm:Teardown begin 2024-05-17T22:09:48.944 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_2127e87e3419439de550d2abbd7ed2aee2cdfb0e/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_931809a9e5d295bd61984693cac44fe131ded4bc/qa/tasks/cephadm.py", line 2270, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_2127e87e3419439de550d2abbd7ed2aee2cdfb0e/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.10/contextlib.py", line 142, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_931809a9e5d295bd61984693cac44fe131ded4bc/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph-c_931809a9e5d295bd61984693cac44fe131ded4bc/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_2127e87e3419439de550d2abbd7ed2aee2cdfb0e/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_2127e87e3419439de550d2abbd7ed2aee2cdfb0e/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-05-17T22:09:48.945 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-17T22:09:48.979 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-17T22:09:49.013 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-05-17T22:09:49.013 DEBUG:teuthology.orchestra.run.smithi107:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-17T22:09:49.037 DEBUG:teuthology.orchestra.run.smithi138:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-17T22:09:49.071 INFO:tasks.cephadm:Stopping all daemons... 2024-05-17T22:09:49.071 INFO:tasks.cephadm.mon.smithi107:Stopping mon.smithi107... 2024-05-17T22:09:49.071 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-3722731a-1496-11ef-bc9a-c7b262605968@mon.smithi107 2024-05-17T22:09:49.356 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:09:49 smithi107 systemd[1]: Stopping Ceph mon.smithi107 for 3722731a-1496-11ef-bc9a-c7b262605968... 2024-05-17T22:09:49.756 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:09:49 smithi107 ceph-3722731a-1496-11ef-bc9a-c7b262605968-mon-smithi107[93220]: 2024-05-17T22:09:49.393+0000 7f7d71593700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi107 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-17T22:09:49.757 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:09:49 smithi107 ceph-3722731a-1496-11ef-bc9a-c7b262605968-mon-smithi107[93220]: 2024-05-17T22:09:49.393+0000 7f7d71593700 -1 mon.smithi107@0(leader) e3 *** Got Signal Terminated *** 2024-05-17T22:09:49.757 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:09:49 smithi107 podman[150714]: 2024-05-17 22:09:49.455322198 +0000 UTC m=+0.177805959 container died 9717f3169f2f33e9afca7a12e2206b211d65c2fdda90950f2c75eb685a3f1bd8 (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-mon-smithi107, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-05-17T22:09:50.106 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:09:49 smithi107 podman[150714]: 2024-05-17 22:09:49.952424524 +0000 UTC m=+0.674908264 container cleanup 9717f3169f2f33e9afca7a12e2206b211d65c2fdda90950f2c75eb685a3f1bd8 (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-mon-smithi107, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-05-17T22:09:50.106 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:May 17 22:09:49 smithi107 bash[150714]: ceph-3722731a-1496-11ef-bc9a-c7b262605968-mon-smithi107 2024-05-17T22:09:50.321 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3722731a-1496-11ef-bc9a-c7b262605968@mon.smithi107.service' 2024-05-17T22:09:50.368 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-17T22:09:50.368 INFO:tasks.cephadm.mon.smithi107:Stopped mon.smithi107 2024-05-17T22:09:50.368 INFO:tasks.cephadm.mon.smithi138:Stopping mon.smithi138... 2024-05-17T22:09:50.368 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ceph-3722731a-1496-11ef-bc9a-c7b262605968@mon.smithi138 2024-05-17T22:09:50.705 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:09:50 smithi138 systemd[1]: Stopping Ceph mon.smithi138 for 3722731a-1496-11ef-bc9a-c7b262605968... 2024-05-17T22:09:51.005 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:09:50 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-mon-smithi138[81372]: 2024-05-17T22:09:50.708+0000 7fa547d52700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi138 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-17T22:09:51.005 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:09:50 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-mon-smithi138[81372]: 2024-05-17T22:09:50.708+0000 7fa547d52700 -1 mon.smithi138@1(peon) e3 *** Got Signal Terminated *** 2024-05-17T22:09:51.005 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:09:50 smithi138 podman[118267]: 2024-05-17 22:09:50.730025266 +0000 UTC m=+0.152805656 container died 55aeed62bf4d04dc3383ddd42054643c62cfda1d91c3cb4983c52feb5b0ce1b9 (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-mon-smithi138, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-05-17T22:09:51.330 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:09:51 smithi138 podman[118267]: 2024-05-17 22:09:51.192258735 +0000 UTC m=+0.615039111 container cleanup 55aeed62bf4d04dc3383ddd42054643c62cfda1d91c3cb4983c52feb5b0ce1b9 (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-mon-smithi138, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-17T22:09:51.330 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 17 22:09:51 smithi138 bash[118267]: ceph-3722731a-1496-11ef-bc9a-c7b262605968-mon-smithi138 2024-05-17T22:09:51.663 DEBUG:teuthology.orchestra.run.smithi138:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3722731a-1496-11ef-bc9a-c7b262605968@mon.smithi138.service' 2024-05-17T22:09:51.716 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-17T22:09:51.716 INFO:tasks.cephadm.mon.smithi138:Stopped mon.smithi138 2024-05-17T22:09:51.717 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-05-17T22:09:51.717 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-3722731a-1496-11ef-bc9a-c7b262605968@osd.0 2024-05-17T22:09:52.106 INFO:journalctl@ceph.osd.0.smithi107.stdout:May 17 22:09:51 smithi107 systemd[1]: Stopping Ceph osd.0 for 3722731a-1496-11ef-bc9a-c7b262605968... 2024-05-17T22:09:52.606 INFO:journalctl@ceph.osd.0.smithi107.stdout:May 17 22:09:52 smithi107 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-0[101648]: 2024-05-17T22:09:52.115+0000 7fcc75bfc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-17T22:09:52.606 INFO:journalctl@ceph.osd.0.smithi107.stdout:May 17 22:09:52 smithi107 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-0[101648]: 2024-05-17T22:09:52.115+0000 7fcc75bfc700 -1 osd.0 84 *** Got signal Terminated *** 2024-05-17T22:09:52.606 INFO:journalctl@ceph.osd.0.smithi107.stdout:May 17 22:09:52 smithi107 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-0[101648]: 2024-05-17T22:09:52.115+0000 7fcc75bfc700 -1 osd.0 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-17T22:09:57.853 INFO:journalctl@ceph.osd.0.smithi107.stdout:May 17 22:09:57 smithi107 podman[150847]: 2024-05-17 22:09:57.578850018 +0000 UTC m=+5.659329995 container died b319bc93500dc6145afd31b133efe5891067329fb4d6df2ee5208b2921697b56 (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-0, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514) 2024-05-17T22:09:58.107 INFO:journalctl@ceph.osd.0.smithi107.stdout:May 17 22:09:57 smithi107 podman[150847]: 2024-05-17 22:09:57.956054765 +0000 UTC m=+6.036534732 container cleanup b319bc93500dc6145afd31b133efe5891067329fb4d6df2ee5208b2921697b56 (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-17T22:09:58.107 INFO:journalctl@ceph.osd.0.smithi107.stdout:May 17 22:09:57 smithi107 bash[150847]: ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-0 2024-05-17T22:09:58.590 INFO:journalctl@ceph.osd.0.smithi107.stdout:May 17 22:09:58 smithi107 podman[150861]: 2024-05-17 22:09:58.257729317 +0000 UTC m=+0.675172847 container remove b319bc93500dc6145afd31b133efe5891067329fb4d6df2ee5208b2921697b56 (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-17T22:09:58.856 INFO:journalctl@ceph.osd.0.smithi107.stdout:May 17 22:09:58 smithi107 podman[150954]: 2024-05-17 22:09:58.497342245 +0000 UTC m=+0.017467467 image pull 8880f7613d0283efa7002505fdccdbc3e37d3b5da31c57fa22d62262c5dab66c quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b 2024-05-17T22:09:58.856 INFO:journalctl@ceph.osd.0.smithi107.stdout:May 17 22:09:58 smithi107 podman[150954]: 2024-05-17 22:09:58.608784779 +0000 UTC m=+0.128910023 container create 80c6152c6b16a662f9cfc605513cbcff2ee4cbe69b0d1ea145aa6a41bdff789b (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.schema-version=1.0) 2024-05-17T22:09:59.185 INFO:journalctl@ceph.osd.0.smithi107.stdout:May 17 22:09:58 smithi107 podman[150954]: 2024-05-17 22:09:58.894859217 +0000 UTC m=+0.414984425 container init 80c6152c6b16a662f9cfc605513cbcff2ee4cbe69b0d1ea145aa6a41bdff789b (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-05-17T22:09:59.185 INFO:journalctl@ceph.osd.0.smithi107.stdout:May 17 22:09:58 smithi107 podman[150954]: 2024-05-17 22:09:58.898665682 +0000 UTC m=+0.418790898 container start 80c6152c6b16a662f9cfc605513cbcff2ee4cbe69b0d1ea145aa6a41bdff789b (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514) 2024-05-17T22:09:59.185 INFO:journalctl@ceph.osd.0.smithi107.stdout:May 17 22:09:58 smithi107 podman[150954]: 2024-05-17 22:09:58.977970413 +0000 UTC m=+0.498095625 container attach 80c6152c6b16a662f9cfc605513cbcff2ee4cbe69b0d1ea145aa6a41bdff789b (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True) 2024-05-17T22:09:59.467 INFO:journalctl@ceph.osd.0.smithi107.stdout:May 17 22:09:59 smithi107 podman[150954]: 2024-05-17 22:09:59.189949259 +0000 UTC m=+0.710074469 container died 80c6152c6b16a662f9cfc605513cbcff2ee4cbe69b0d1ea145aa6a41bdff789b (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a) 2024-05-17T22:09:59.869 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3722731a-1496-11ef-bc9a-c7b262605968@osd.0.service' 2024-05-17T22:09:59.908 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-17T22:09:59.908 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-05-17T22:09:59.908 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-05-17T22:09:59.908 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-3722731a-1496-11ef-bc9a-c7b262605968@osd.1 2024-05-17T22:10:00.106 INFO:journalctl@ceph.osd.1.smithi107.stdout:May 17 22:09:59 smithi107 systemd[1]: Stopping Ceph osd.1 for 3722731a-1496-11ef-bc9a-c7b262605968... 2024-05-17T22:10:00.606 INFO:journalctl@ceph.osd.1.smithi107.stdout:May 17 22:10:00 smithi107 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-1[106965]: 2024-05-17T22:10:00.260+0000 7fa25f119700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-17T22:10:00.606 INFO:journalctl@ceph.osd.1.smithi107.stdout:May 17 22:10:00 smithi107 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-1[106965]: 2024-05-17T22:10:00.260+0000 7fa25f119700 -1 osd.1 84 *** Got signal Terminated *** 2024-05-17T22:10:00.606 INFO:journalctl@ceph.osd.1.smithi107.stdout:May 17 22:10:00 smithi107 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-1[106965]: 2024-05-17T22:10:00.260+0000 7fa25f119700 -1 osd.1 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-17T22:10:05.990 INFO:journalctl@ceph.osd.1.smithi107.stdout:May 17 22:10:05 smithi107 podman[151071]: 2024-05-17 22:10:05.715058093 +0000 UTC m=+5.570882204 container died f94b206be6eb344f1eb9d2119362212ebc214e71d5d379083314941a174b708c (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-1, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-17T22:10:06.348 INFO:journalctl@ceph.osd.1.smithi107.stdout:May 17 22:10:06 smithi107 podman[151071]: 2024-05-17 22:10:06.093349596 +0000 UTC m=+5.949173702 container cleanup f94b206be6eb344f1eb9d2119362212ebc214e71d5d379083314941a174b708c (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-1, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-17T22:10:06.348 INFO:journalctl@ceph.osd.1.smithi107.stdout:May 17 22:10:06 smithi107 bash[151071]: ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-1 2024-05-17T22:10:06.606 INFO:journalctl@ceph.osd.1.smithi107.stdout:May 17 22:10:06 smithi107 podman[151089]: 2024-05-17 22:10:06.352921823 +0000 UTC m=+0.633348140 container remove f94b206be6eb344f1eb9d2119362212ebc214e71d5d379083314941a174b708c (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-1, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a) 2024-05-17T22:10:06.936 INFO:journalctl@ceph.osd.1.smithi107.stdout:May 17 22:10:06 smithi107 podman[151180]: 2024-05-17 22:10:06.584985545 +0000 UTC m=+0.017401471 image pull 8880f7613d0283efa7002505fdccdbc3e37d3b5da31c57fa22d62262c5dab66c quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b 2024-05-17T22:10:06.936 INFO:journalctl@ceph.osd.1.smithi107.stdout:May 17 22:10:06 smithi107 podman[151180]: 2024-05-17 22:10:06.704023734 +0000 UTC m=+0.136439642 container create 384c52328b84a46137fde1434ace1907346368f00cda533b4df4f46ffbfa5fc3 (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-05-17T22:10:06.936 INFO:journalctl@ceph.osd.1.smithi107.stdout:May 17 22:10:06 smithi107 podman[151180]: 2024-05-17 22:10:06.940444586 +0000 UTC m=+0.372860511 container init 384c52328b84a46137fde1434ace1907346368f00cda533b4df4f46ffbfa5fc3 (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-1-deactivate, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-05-17T22:10:07.231 INFO:journalctl@ceph.osd.1.smithi107.stdout:May 17 22:10:06 smithi107 podman[151180]: 2024-05-17 22:10:06.944025469 +0000 UTC m=+0.376441383 container start 384c52328b84a46137fde1434ace1907346368f00cda533b4df4f46ffbfa5fc3 (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-1-deactivate, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-17T22:10:07.232 INFO:journalctl@ceph.osd.1.smithi107.stdout:May 17 22:10:07 smithi107 podman[151180]: 2024-05-17 22:10:07.039906119 +0000 UTC m=+0.472322044 container attach 384c52328b84a46137fde1434ace1907346368f00cda533b4df4f46ffbfa5fc3 (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS) 2024-05-17T22:10:07.565 INFO:journalctl@ceph.osd.1.smithi107.stdout:May 17 22:10:07 smithi107 podman[151180]: 2024-05-17 22:10:07.23635673 +0000 UTC m=+0.668772649 container died 384c52328b84a46137fde1434ace1907346368f00cda533b4df4f46ffbfa5fc3 (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-17T22:10:07.899 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3722731a-1496-11ef-bc9a-c7b262605968@osd.1.service' 2024-05-17T22:10:07.940 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-17T22:10:07.940 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-05-17T22:10:07.940 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-05-17T22:10:07.940 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-3722731a-1496-11ef-bc9a-c7b262605968@osd.2 2024-05-17T22:10:08.292 INFO:journalctl@ceph.osd.2.smithi107.stdout:May 17 22:10:08 smithi107 systemd[1]: Stopping Ceph osd.2 for 3722731a-1496-11ef-bc9a-c7b262605968... 2024-05-17T22:10:08.606 INFO:journalctl@ceph.osd.2.smithi107.stdout:May 17 22:10:08 smithi107 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-2[112698]: 2024-05-17T22:10:08.295+0000 7fc709966700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-17T22:10:08.606 INFO:journalctl@ceph.osd.2.smithi107.stdout:May 17 22:10:08 smithi107 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-2[112698]: 2024-05-17T22:10:08.295+0000 7fc709966700 -1 osd.2 84 *** Got signal Terminated *** 2024-05-17T22:10:08.606 INFO:journalctl@ceph.osd.2.smithi107.stdout:May 17 22:10:08 smithi107 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-2[112698]: 2024-05-17T22:10:08.295+0000 7fc709966700 -1 osd.2 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-17T22:10:13.926 INFO:journalctl@ceph.osd.2.smithi107.stdout:May 17 22:10:13 smithi107 podman[151287]: 2024-05-17 22:10:13.620199313 +0000 UTC m=+5.450026126 container died add7015647af5cff02edc5e90d98acaa4c2cc25e1fe4adbda60f6af5e6d9de76 (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-05-17T22:10:14.234 INFO:journalctl@ceph.osd.2.smithi107.stdout:May 17 22:10:14 smithi107 podman[151287]: 2024-05-17 22:10:14.028892389 +0000 UTC m=+5.858719191 container cleanup add7015647af5cff02edc5e90d98acaa4c2cc25e1fe4adbda60f6af5e6d9de76 (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-2, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-05-17T22:10:14.234 INFO:journalctl@ceph.osd.2.smithi107.stdout:May 17 22:10:14 smithi107 bash[151287]: ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-2 2024-05-17T22:10:14.578 INFO:journalctl@ceph.osd.2.smithi107.stdout:May 17 22:10:14 smithi107 podman[151303]: 2024-05-17 22:10:14.238669313 +0000 UTC m=+0.614532396 container remove add7015647af5cff02edc5e90d98acaa4c2cc25e1fe4adbda60f6af5e6d9de76 (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux ) 2024-05-17T22:10:14.854 INFO:journalctl@ceph.osd.2.smithi107.stdout:May 17 22:10:14 smithi107 podman[151396]: 2024-05-17 22:10:14.485294265 +0000 UTC m=+0.017577556 image pull 8880f7613d0283efa7002505fdccdbc3e37d3b5da31c57fa22d62262c5dab66c quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b 2024-05-17T22:10:14.855 INFO:journalctl@ceph.osd.2.smithi107.stdout:May 17 22:10:14 smithi107 podman[151396]: 2024-05-17 22:10:14.58952472 +0000 UTC m=+0.121808049 container create d6d5cc1034ab12966bd1c86df646a441dbc31bde6490ed11eb0f539c5ba58ba9 (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-2-deactivate, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-17T22:10:14.855 INFO:journalctl@ceph.osd.2.smithi107.stdout:May 17 22:10:14 smithi107 podman[151396]: 2024-05-17 22:10:14.858882771 +0000 UTC m=+0.391166074 container init d6d5cc1034ab12966bd1c86df646a441dbc31bde6490ed11eb0f539c5ba58ba9 (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-05-17T22:10:15.106 INFO:journalctl@ceph.osd.2.smithi107.stdout:May 17 22:10:14 smithi107 podman[151396]: 2024-05-17 22:10:14.862369266 +0000 UTC m=+0.394652556 container start d6d5cc1034ab12966bd1c86df646a441dbc31bde6490ed11eb0f539c5ba58ba9 (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-05-17T22:10:15.107 INFO:journalctl@ceph.osd.2.smithi107.stdout:May 17 22:10:14 smithi107 podman[151396]: 2024-05-17 22:10:14.941834144 +0000 UTC m=+0.474117441 container attach d6d5cc1034ab12966bd1c86df646a441dbc31bde6490ed11eb0f539c5ba58ba9 (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-2-deactivate, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-17T22:10:15.433 INFO:journalctl@ceph.osd.2.smithi107.stdout:May 17 22:10:15 smithi107 podman[151396]: 2024-05-17 22:10:15.154041698 +0000 UTC m=+0.686324986 container died d6d5cc1034ab12966bd1c86df646a441dbc31bde6490ed11eb0f539c5ba58ba9 (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-05-17T22:10:15.734 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3722731a-1496-11ef-bc9a-c7b262605968@osd.2.service' 2024-05-17T22:10:15.772 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-17T22:10:15.773 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-05-17T22:10:15.773 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-05-17T22:10:15.773 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ceph-3722731a-1496-11ef-bc9a-c7b262605968@osd.3 2024-05-17T22:10:16.074 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 17 22:10:15 smithi138 systemd[1]: Stopping Ceph osd.3 for 3722731a-1496-11ef-bc9a-c7b262605968... 2024-05-17T22:10:16.328 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 17 22:10:16 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-3[90913]: 2024-05-17T22:10:16.077+0000 7f497cbae700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-17T22:10:16.328 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 17 22:10:16 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-3[90913]: 2024-05-17T22:10:16.077+0000 7f497cbae700 -1 osd.3 84 *** Got signal Terminated *** 2024-05-17T22:10:16.328 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 17 22:10:16 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-3[90913]: 2024-05-17T22:10:16.077+0000 7f497cbae700 -1 osd.3 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-17T22:10:18.580 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 17 22:10:18 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-4[96037]: 2024-05-17T22:10:18.303+0000 7f4fe6761700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-05-17T22:09:53.910341+0000 front 2024-05-17T22:09:53.910426+0000 (oldest deadline 2024-05-17T22:10:18.010092+0000) 2024-05-17T22:10:19.081 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 17 22:10:18 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-5[101149]: 2024-05-17T22:10:18.587+0000 7f0e01377700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-05-17T22:09:54.959944+0000 front 2024-05-17T22:09:54.960044+0000 (oldest deadline 2024-05-17T22:10:17.859908+0000) 2024-05-17T22:10:19.580 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 17 22:10:19 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-3[90913]: 2024-05-17T22:10:19.161+0000 7f497515c700 -1 osd.3 84 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-05-17T22:09:56.054655+0000 front 2024-05-17T22:09:56.054570+0000 (oldest deadline 2024-05-17T22:10:18.854848+0000) 2024-05-17T22:10:19.581 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 17 22:10:19 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-4[96037]: 2024-05-17T22:10:19.306+0000 7f4fe6761700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-05-17T22:09:53.910341+0000 front 2024-05-17T22:09:53.910426+0000 (oldest deadline 2024-05-17T22:10:18.010092+0000) 2024-05-17T22:10:20.081 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 17 22:10:19 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-5[101149]: 2024-05-17T22:10:19.589+0000 7f0e01377700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-05-17T22:09:54.959944+0000 front 2024-05-17T22:09:54.960044+0000 (oldest deadline 2024-05-17T22:10:17.859908+0000) 2024-05-17T22:10:20.580 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 17 22:10:20 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-3[90913]: 2024-05-17T22:10:20.167+0000 7f497515c700 -1 osd.3 84 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-05-17T22:09:56.054655+0000 front 2024-05-17T22:09:56.054570+0000 (oldest deadline 2024-05-17T22:10:18.854848+0000) 2024-05-17T22:10:20.581 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 17 22:10:20 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-4[96037]: 2024-05-17T22:10:20.257+0000 7f4fe6761700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-05-17T22:09:53.910341+0000 front 2024-05-17T22:09:53.910426+0000 (oldest deadline 2024-05-17T22:10:18.010092+0000) 2024-05-17T22:10:21.080 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 17 22:10:20 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-5[101149]: 2024-05-17T22:10:20.637+0000 7f0e01377700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-05-17T22:09:54.959944+0000 front 2024-05-17T22:09:54.960044+0000 (oldest deadline 2024-05-17T22:10:17.859908+0000) 2024-05-17T22:10:21.551 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 17 22:10:21 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-4[96037]: 2024-05-17T22:10:21.234+0000 7f4fe6761700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-05-17T22:09:53.910341+0000 front 2024-05-17T22:09:53.910426+0000 (oldest deadline 2024-05-17T22:10:18.010092+0000) 2024-05-17T22:10:21.817 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 17 22:10:21 smithi138 podman[118391]: 2024-05-17 22:10:21.55554784 +0000 UTC m=+5.580277268 container died 416273d327c766b25b5e166104435102863eb6853763c87d19268c8211466a59 (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-3, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-17T22:10:21.818 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 17 22:10:21 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-5[101149]: 2024-05-17T22:10:21.659+0000 7f0e01377700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-05-17T22:09:54.959944+0000 front 2024-05-17T22:09:54.960044+0000 (oldest deadline 2024-05-17T22:10:17.859908+0000) 2024-05-17T22:10:22.080 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 17 22:10:21 smithi138 podman[118391]: 2024-05-17 22:10:21.920142907 +0000 UTC m=+5.944872328 container cleanup 416273d327c766b25b5e166104435102863eb6853763c87d19268c8211466a59 (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-3, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-17T22:10:22.081 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 17 22:10:21 smithi138 bash[118391]: ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-3 2024-05-17T22:10:22.517 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 17 22:10:22 smithi138 podman[118417]: 2024-05-17 22:10:22.196331764 +0000 UTC m=+0.641996072 container remove 416273d327c766b25b5e166104435102863eb6853763c87d19268c8211466a59 (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-3, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5) 2024-05-17T22:10:22.517 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 17 22:10:22 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-4[96037]: 2024-05-17T22:10:22.224+0000 7f4fe6761700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-05-17T22:09:53.910341+0000 front 2024-05-17T22:09:53.910426+0000 (oldest deadline 2024-05-17T22:10:18.010092+0000) 2024-05-17T22:10:22.780 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 17 22:10:22 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-5[101149]: 2024-05-17T22:10:22.623+0000 7f0e01377700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-05-17T22:09:54.959944+0000 front 2024-05-17T22:09:54.960044+0000 (oldest deadline 2024-05-17T22:10:17.859908+0000) 2024-05-17T22:10:22.780 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 17 22:10:22 smithi138 podman[118505]: 2024-05-17 22:10:22.42415583 +0000 UTC m=+0.017396592 image pull 8880f7613d0283efa7002505fdccdbc3e37d3b5da31c57fa22d62262c5dab66c quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b 2024-05-17T22:10:22.780 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 17 22:10:22 smithi138 podman[118505]: 2024-05-17 22:10:22.522103843 +0000 UTC m=+0.115344590 container create c981fb301cdf1144bcb449ee8d563075d731b6909993b1dd4bb4060c730736f6 (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-3-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-05-17T22:10:22.780 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 17 22:10:22 smithi138 podman[118505]: 2024-05-17 22:10:22.784120002 +0000 UTC m=+0.377360767 container init c981fb301cdf1144bcb449ee8d563075d731b6909993b1dd4bb4060c730736f6 (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-17T22:10:23.080 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 17 22:10:22 smithi138 podman[118505]: 2024-05-17 22:10:22.787893671 +0000 UTC m=+0.381134421 container start c981fb301cdf1144bcb449ee8d563075d731b6909993b1dd4bb4060c730736f6 (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a) 2024-05-17T22:10:23.081 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 17 22:10:22 smithi138 podman[118505]: 2024-05-17 22:10:22.866792731 +0000 UTC m=+0.460033491 container attach c981fb301cdf1144bcb449ee8d563075d731b6909993b1dd4bb4060c730736f6 (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-17T22:10:23.383 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 17 22:10:23 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-4[96037]: 2024-05-17T22:10:23.251+0000 7f4fe6761700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-05-17T22:09:53.910341+0000 front 2024-05-17T22:09:53.910426+0000 (oldest deadline 2024-05-17T22:10:18.010092+0000) 2024-05-17T22:10:23.383 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 17 22:10:23 smithi138 podman[118505]: 2024-05-17 22:10:23.085426213 +0000 UTC m=+0.678666965 container died c981fb301cdf1144bcb449ee8d563075d731b6909993b1dd4bb4060c730736f6 (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-3-deactivate, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-05-17T22:10:23.699 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 17 22:10:23 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-5[101149]: 2024-05-17T22:10:23.574+0000 7f0e01377700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-05-17T22:09:54.959944+0000 front 2024-05-17T22:09:54.960044+0000 (oldest deadline 2024-05-17T22:10:17.859908+0000) 2024-05-17T22:10:23.722 DEBUG:teuthology.orchestra.run.smithi138:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3722731a-1496-11ef-bc9a-c7b262605968@osd.3.service' 2024-05-17T22:10:23.763 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-17T22:10:23.763 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-05-17T22:10:23.763 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-05-17T22:10:23.764 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ceph-3722731a-1496-11ef-bc9a-c7b262605968@osd.4 2024-05-17T22:10:24.080 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 17 22:10:23 smithi138 systemd[1]: Stopping Ceph osd.4 for 3722731a-1496-11ef-bc9a-c7b262605968... 2024-05-17T22:10:24.540 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 17 22:10:24 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-4[96037]: 2024-05-17T22:10:24.115+0000 7f4fea1ab700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-17T22:10:24.540 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 17 22:10:24 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-4[96037]: 2024-05-17T22:10:24.115+0000 7f4fea1ab700 -1 osd.4 84 *** Got signal Terminated *** 2024-05-17T22:10:24.540 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 17 22:10:24 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-4[96037]: 2024-05-17T22:10:24.115+0000 7f4fea1ab700 -1 osd.4 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-17T22:10:24.540 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 17 22:10:24 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-4[96037]: 2024-05-17T22:10:24.218+0000 7f4fe6761700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-05-17T22:09:53.910341+0000 front 2024-05-17T22:09:53.910426+0000 (oldest deadline 2024-05-17T22:10:18.010092+0000) 2024-05-17T22:10:24.830 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 17 22:10:24 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-5[101149]: 2024-05-17T22:10:24.543+0000 7f0e01377700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-05-17T22:09:54.959944+0000 front 2024-05-17T22:09:54.960044+0000 (oldest deadline 2024-05-17T22:10:17.859908+0000) 2024-05-17T22:10:25.509 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 17 22:10:25 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-4[96037]: 2024-05-17T22:10:25.222+0000 7f4fe6761700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-05-17T22:09:53.910341+0000 front 2024-05-17T22:09:53.910426+0000 (oldest deadline 2024-05-17T22:10:18.010092+0000) 2024-05-17T22:10:25.830 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 17 22:10:25 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-5[101149]: 2024-05-17T22:10:25.513+0000 7f0e01377700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-05-17T22:09:54.959944+0000 front 2024-05-17T22:09:54.960044+0000 (oldest deadline 2024-05-17T22:10:17.859908+0000) 2024-05-17T22:10:26.580 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 17 22:10:26 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-5[101149]: 2024-05-17T22:10:26.473+0000 7f0e01377700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-05-17T22:09:54.959944+0000 front 2024-05-17T22:09:54.960044+0000 (oldest deadline 2024-05-17T22:10:17.859908+0000) 2024-05-17T22:10:26.581 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 17 22:10:26 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-5[101149]: 2024-05-17T22:10:26.473+0000 7f0e01377700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-05-17T22:10:01.360969+0000 front 2024-05-17T22:10:01.360994+0000 (oldest deadline 2024-05-17T22:10:26.061026+0000) 2024-05-17T22:10:26.581 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 17 22:10:26 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-4[96037]: 2024-05-17T22:10:26.235+0000 7f4fe6761700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-05-17T22:09:53.910341+0000 front 2024-05-17T22:09:53.910426+0000 (oldest deadline 2024-05-17T22:10:18.010092+0000) 2024-05-17T22:10:26.581 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 17 22:10:26 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-4[96037]: 2024-05-17T22:10:26.235+0000 7f4fe6761700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-05-17T22:10:01.511022+0000 front 2024-05-17T22:10:01.511110+0000 (oldest deadline 2024-05-17T22:10:26.210837+0000) 2024-05-17T22:10:27.580 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 17 22:10:27 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-4[96037]: 2024-05-17T22:10:27.223+0000 7f4fe6761700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-05-17T22:09:53.910341+0000 front 2024-05-17T22:09:53.910426+0000 (oldest deadline 2024-05-17T22:10:18.010092+0000) 2024-05-17T22:10:27.581 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 17 22:10:27 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-4[96037]: 2024-05-17T22:10:27.223+0000 7f4fe6761700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-05-17T22:10:01.511022+0000 front 2024-05-17T22:10:01.511110+0000 (oldest deadline 2024-05-17T22:10:26.210837+0000) 2024-05-17T22:10:27.581 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 17 22:10:27 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-5[101149]: 2024-05-17T22:10:27.459+0000 7f0e01377700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-05-17T22:09:54.959944+0000 front 2024-05-17T22:09:54.960044+0000 (oldest deadline 2024-05-17T22:10:17.859908+0000) 2024-05-17T22:10:27.581 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 17 22:10:27 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-5[101149]: 2024-05-17T22:10:27.459+0000 7f0e01377700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-05-17T22:10:01.360969+0000 front 2024-05-17T22:10:01.360994+0000 (oldest deadline 2024-05-17T22:10:26.061026+0000) 2024-05-17T22:10:28.452 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 17 22:10:28 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-4[96037]: 2024-05-17T22:10:28.182+0000 7f4fe6761700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-05-17T22:09:53.910341+0000 front 2024-05-17T22:09:53.910426+0000 (oldest deadline 2024-05-17T22:10:18.010092+0000) 2024-05-17T22:10:28.452 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 17 22:10:28 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-4[96037]: 2024-05-17T22:10:28.182+0000 7f4fe6761700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-05-17T22:10:01.511022+0000 front 2024-05-17T22:10:01.511110+0000 (oldest deadline 2024-05-17T22:10:26.210837+0000) 2024-05-17T22:10:28.830 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 17 22:10:28 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-5[101149]: 2024-05-17T22:10:28.455+0000 7f0e01377700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-05-17T22:09:54.959944+0000 front 2024-05-17T22:09:54.960044+0000 (oldest deadline 2024-05-17T22:10:17.859908+0000) 2024-05-17T22:10:28.831 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 17 22:10:28 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-5[101149]: 2024-05-17T22:10:28.455+0000 7f0e01377700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-05-17T22:10:01.360969+0000 front 2024-05-17T22:10:01.360994+0000 (oldest deadline 2024-05-17T22:10:26.061026+0000) 2024-05-17T22:10:29.771 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 17 22:10:29 smithi138 podman[118609]: 2024-05-17 22:10:29.497960332 +0000 UTC m=+5.495759074 container died e33f411da8e680c9efc6047708a801968d51f176a8ebed10ed5961165a3f046d (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-4, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514) 2024-05-17T22:10:29.771 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 17 22:10:29 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-5[101149]: 2024-05-17T22:10:29.464+0000 7f0e01377700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-05-17T22:09:54.959944+0000 front 2024-05-17T22:09:54.960044+0000 (oldest deadline 2024-05-17T22:10:17.859908+0000) 2024-05-17T22:10:29.771 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 17 22:10:29 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-5[101149]: 2024-05-17T22:10:29.464+0000 7f0e01377700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-05-17T22:10:01.360969+0000 front 2024-05-17T22:10:01.360994+0000 (oldest deadline 2024-05-17T22:10:26.061026+0000) 2024-05-17T22:10:30.080 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 17 22:10:29 smithi138 podman[118609]: 2024-05-17 22:10:29.856968727 +0000 UTC m=+5.854767460 container cleanup e33f411da8e680c9efc6047708a801968d51f176a8ebed10ed5961165a3f046d (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-4, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, GIT_BRANCH=HEAD) 2024-05-17T22:10:30.081 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 17 22:10:29 smithi138 bash[118609]: ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-4 2024-05-17T22:10:30.455 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 17 22:10:30 smithi138 podman[118623]: 2024-05-17 22:10:30.158309667 +0000 UTC m=+0.660001567 container remove e33f411da8e680c9efc6047708a801968d51f176a8ebed10ed5961165a3f046d (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-4, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-17T22:10:30.709 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 17 22:10:30 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-5[101149]: 2024-05-17T22:10:30.459+0000 7f0e01377700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-05-17T22:09:54.959944+0000 front 2024-05-17T22:09:54.960044+0000 (oldest deadline 2024-05-17T22:10:17.859908+0000) 2024-05-17T22:10:30.709 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 17 22:10:30 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-5[101149]: 2024-05-17T22:10:30.459+0000 7f0e01377700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-05-17T22:10:01.360969+0000 front 2024-05-17T22:10:01.360994+0000 (oldest deadline 2024-05-17T22:10:26.061026+0000) 2024-05-17T22:10:30.709 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 17 22:10:30 smithi138 podman[118715]: 2024-05-17 22:10:30.427357718 +0000 UTC m=+0.017284538 image pull 8880f7613d0283efa7002505fdccdbc3e37d3b5da31c57fa22d62262c5dab66c quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b 2024-05-17T22:10:30.709 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 17 22:10:30 smithi138 podman[118715]: 2024-05-17 22:10:30.559064843 +0000 UTC m=+0.148991637 container create 7fdb6b796f2b14f272cd77794525425fd15cd8ed5a75e3d72bad1bb39db296dd (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240514, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-17T22:10:31.080 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 17 22:10:30 smithi138 podman[118715]: 2024-05-17 22:10:30.803256386 +0000 UTC m=+0.393183187 container init 7fdb6b796f2b14f272cd77794525425fd15cd8ed5a75e3d72bad1bb39db296dd (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-05-17T22:10:31.081 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 17 22:10:30 smithi138 podman[118715]: 2024-05-17 22:10:30.806841749 +0000 UTC m=+0.396768556 container start 7fdb6b796f2b14f272cd77794525425fd15cd8ed5a75e3d72bad1bb39db296dd (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-05-17T22:10:31.081 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 17 22:10:30 smithi138 podman[118715]: 2024-05-17 22:10:30.886519189 +0000 UTC m=+0.476446000 container attach 7fdb6b796f2b14f272cd77794525425fd15cd8ed5a75e3d72bad1bb39db296dd (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-05-17T22:10:31.385 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 17 22:10:31 smithi138 podman[118715]: 2024-05-17 22:10:31.097403834 +0000 UTC m=+0.687330638 container died 7fdb6b796f2b14f272cd77794525425fd15cd8ed5a75e3d72bad1bb39db296dd (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS) 2024-05-17T22:10:31.644 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 17 22:10:31 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-5[101149]: 2024-05-17T22:10:31.415+0000 7f0e01377700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-05-17T22:09:54.959944+0000 front 2024-05-17T22:09:54.960044+0000 (oldest deadline 2024-05-17T22:10:17.859908+0000) 2024-05-17T22:10:31.644 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 17 22:10:31 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-5[101149]: 2024-05-17T22:10:31.415+0000 7f0e01377700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-05-17T22:10:01.360969+0000 front 2024-05-17T22:10:01.360994+0000 (oldest deadline 2024-05-17T22:10:26.061026+0000) 2024-05-17T22:10:31.667 DEBUG:teuthology.orchestra.run.smithi138:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3722731a-1496-11ef-bc9a-c7b262605968@osd.4.service' 2024-05-17T22:10:31.706 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-17T22:10:31.707 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-05-17T22:10:31.707 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-05-17T22:10:31.707 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ceph-3722731a-1496-11ef-bc9a-c7b262605968@osd.5 2024-05-17T22:10:32.039 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 17 22:10:31 smithi138 systemd[1]: Stopping Ceph osd.5 for 3722731a-1496-11ef-bc9a-c7b262605968... 2024-05-17T22:10:32.040 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 17 22:10:32 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-5[101149]: 2024-05-17T22:10:32.042+0000 7f0e08dc9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-17T22:10:32.040 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 17 22:10:32 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-5[101149]: 2024-05-17T22:10:32.042+0000 7f0e08dc9700 -1 osd.5 84 *** Got signal Terminated *** 2024-05-17T22:10:32.040 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 17 22:10:32 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-5[101149]: 2024-05-17T22:10:32.042+0000 7f0e08dc9700 -1 osd.5 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-17T22:10:32.830 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 17 22:10:32 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-5[101149]: 2024-05-17T22:10:32.457+0000 7f0e01377700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-05-17T22:09:54.959944+0000 front 2024-05-17T22:09:54.960044+0000 (oldest deadline 2024-05-17T22:10:17.859908+0000) 2024-05-17T22:10:32.831 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 17 22:10:32 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-5[101149]: 2024-05-17T22:10:32.457+0000 7f0e01377700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-05-17T22:10:01.360969+0000 front 2024-05-17T22:10:01.360994+0000 (oldest deadline 2024-05-17T22:10:26.061026+0000) 2024-05-17T22:10:33.830 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 17 22:10:33 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-5[101149]: 2024-05-17T22:10:33.429+0000 7f0e01377700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-05-17T22:09:54.959944+0000 front 2024-05-17T22:09:54.960044+0000 (oldest deadline 2024-05-17T22:10:17.859908+0000) 2024-05-17T22:10:33.831 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 17 22:10:33 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-5[101149]: 2024-05-17T22:10:33.429+0000 7f0e01377700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-05-17T22:10:01.360969+0000 front 2024-05-17T22:10:01.360994+0000 (oldest deadline 2024-05-17T22:10:26.061026+0000) 2024-05-17T22:10:34.830 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 17 22:10:34 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-5[101149]: 2024-05-17T22:10:34.416+0000 7f0e01377700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-05-17T22:09:54.959944+0000 front 2024-05-17T22:09:54.960044+0000 (oldest deadline 2024-05-17T22:10:17.859908+0000) 2024-05-17T22:10:34.830 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 17 22:10:34 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-5[101149]: 2024-05-17T22:10:34.416+0000 7f0e01377700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-05-17T22:10:01.360969+0000 front 2024-05-17T22:10:01.360994+0000 (oldest deadline 2024-05-17T22:10:26.061026+0000) 2024-05-17T22:10:35.830 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 17 22:10:35 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-5[101149]: 2024-05-17T22:10:35.381+0000 7f0e01377700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-05-17T22:09:54.959944+0000 front 2024-05-17T22:09:54.960044+0000 (oldest deadline 2024-05-17T22:10:17.859908+0000) 2024-05-17T22:10:35.831 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 17 22:10:35 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-5[101149]: 2024-05-17T22:10:35.381+0000 7f0e01377700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-05-17T22:10:01.360969+0000 front 2024-05-17T22:10:01.360994+0000 (oldest deadline 2024-05-17T22:10:26.061026+0000) 2024-05-17T22:10:36.831 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 17 22:10:36 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-5[101149]: 2024-05-17T22:10:36.363+0000 7f0e01377700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-05-17T22:09:54.959944+0000 front 2024-05-17T22:09:54.960044+0000 (oldest deadline 2024-05-17T22:10:17.859908+0000) 2024-05-17T22:10:36.831 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 17 22:10:36 smithi138 ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-5[101149]: 2024-05-17T22:10:36.363+0000 7f0e01377700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-05-17T22:10:01.360969+0000 front 2024-05-17T22:10:01.360994+0000 (oldest deadline 2024-05-17T22:10:26.061026+0000) 2024-05-17T22:10:37.606 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 17 22:10:37 smithi138 podman[118821]: 2024-05-17 22:10:37.352942797 +0000 UTC m=+5.418755830 container died 63810d431961f18ecc215176a3d29574ca47af1decbb07b62508b551df395075 (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-5, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux ) 2024-05-17T22:10:37.973 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 17 22:10:37 smithi138 podman[118821]: 2024-05-17 22:10:37.742487291 +0000 UTC m=+5.808300314 container cleanup 63810d431961f18ecc215176a3d29574ca47af1decbb07b62508b551df395075 (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS) 2024-05-17T22:10:37.973 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 17 22:10:37 smithi138 bash[118821]: ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-5 2024-05-17T22:10:38.294 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 17 22:10:37 smithi138 podman[118840]: 2024-05-17 22:10:37.976986102 +0000 UTC m=+0.622568583 container remove 63810d431961f18ecc215176a3d29574ca47af1decbb07b62508b551df395075 (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-5, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-17T22:10:38.581 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 17 22:10:38 smithi138 podman[118925]: 2024-05-17 22:10:38.201443875 +0000 UTC m=+0.016978418 image pull 8880f7613d0283efa7002505fdccdbc3e37d3b5da31c57fa22d62262c5dab66c quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b 2024-05-17T22:10:38.581 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 17 22:10:38 smithi138 podman[118925]: 2024-05-17 22:10:38.302866882 +0000 UTC m=+0.118401410 container create 3f8bd84dbc57f3e8e87120d5ac658e1792204a06f13151baf0c67a2f8292766f (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-5-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-05-17T22:10:38.581 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 17 22:10:38 smithi138 podman[118925]: 2024-05-17 22:10:38.522561627 +0000 UTC m=+0.338096164 container init 3f8bd84dbc57f3e8e87120d5ac658e1792204a06f13151baf0c67a2f8292766f (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-5-deactivate, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-17T22:10:38.581 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 17 22:10:38 smithi138 podman[118925]: 2024-05-17 22:10:38.526529943 +0000 UTC m=+0.342064473 container start 3f8bd84dbc57f3e8e87120d5ac658e1792204a06f13151baf0c67a2f8292766f (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, ceph=True, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-17T22:10:39.081 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 17 22:10:38 smithi138 podman[118925]: 2024-05-17 22:10:38.614385992 +0000 UTC m=+0.429920537 container attach 3f8bd84dbc57f3e8e87120d5ac658e1792204a06f13151baf0c67a2f8292766f (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2) 2024-05-17T22:10:39.081 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 17 22:10:38 smithi138 podman[118925]: 2024-05-17 22:10:38.820295701 +0000 UTC m=+0.635830235 container died 3f8bd84dbc57f3e8e87120d5ac658e1792204a06f13151baf0c67a2f8292766f (image=quay.ceph.io/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-3722731a-1496-11ef-bc9a-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux ) 2024-05-17T22:10:39.402 DEBUG:teuthology.orchestra.run.smithi138:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3722731a-1496-11ef-bc9a-c7b262605968@osd.5.service' 2024-05-17T22:10:39.441 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-17T22:10:39.441 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-05-17T22:10:39.441 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3722731a-1496-11ef-bc9a-c7b262605968 --force --keep-logs 2024-05-17T22:10:42.684 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi107.stderr:ceph-fuse[76055]: fuse finished with error 0 and tester_r 0 2024-05-17T22:11:00.222 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3722731a-1496-11ef-bc9a-c7b262605968 --force --keep-logs 2024-05-17T22:11:14.788 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-17T22:11:14.822 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-17T22:11:14.853 INFO:tasks.cephadm:Archiving crash dumps... 2024-05-17T22:11:14.855 DEBUG:teuthology.misc:Transferring archived files from smithi107:/var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-05-17_18:30:44-fs-wip-pdonnell-testing-20240517.012458-debug-distro-default-smithi/7711745/remote/smithi107/crash 2024-05-17T22:11:14.856 DEBUG:teuthology.orchestra.run.smithi107:> sudo tar c -f - -C /var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/crash -- . 2024-05-17T22:11:14.889 INFO:teuthology.orchestra.run.smithi107.stderr:tar: /var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/crash: Cannot open: No such file or directory 2024-05-17T22:11:14.890 INFO:teuthology.orchestra.run.smithi107.stderr:tar: Error is not recoverable: exiting now 2024-05-17T22:11:14.892 DEBUG:teuthology.misc:Transferring archived files from smithi138:/var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-05-17_18:30:44-fs-wip-pdonnell-testing-20240517.012458-debug-distro-default-smithi/7711745/remote/smithi138/crash 2024-05-17T22:11:14.892 DEBUG:teuthology.orchestra.run.smithi138:> sudo tar c -f - -C /var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/crash -- . 2024-05-17T22:11:14.916 INFO:teuthology.orchestra.run.smithi138.stderr:tar: /var/lib/ceph/3722731a-1496-11ef-bc9a-c7b262605968/crash: Cannot open: No such file or directory 2024-05-17T22:11:14.916 INFO:teuthology.orchestra.run.smithi138.stderr:tar: Error is not recoverable: exiting now 2024-05-17T22:11:14.917 INFO:tasks.cephadm:Checking cluster log for badness... 2024-05-17T22:11:14.918 DEBUG:teuthology.orchestra.run.smithi107:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/3722731a-1496-11ef-bc9a-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v OSD_DOWN | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | head -n 1 2024-05-17T22:11:14.970 INFO:teuthology.orchestra.run.smithi107.stdout:2024-05-17T22:00:00.000147+0000 mon.smithi107 (mon.0) 637 : cluster [WRN] fs cephfs is degraded 2024-05-17T22:11:14.971 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-05-17T22:11:14.971 INFO:tasks.cephadm:Compressing logs... 2024-05-17T22:11:14.971 DEBUG:teuthology.orchestra.run.smithi107:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-17T22:11:15.013 DEBUG:teuthology.orchestra.run.smithi138:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-17T22:11:15.035 INFO:teuthology.orchestra.run.smithi107.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose: No such file or directory 2024-05-17T22:11:15.036 INFO:teuthology.orchestra.run.smithi107.stderr: -- /var/log/ceph/3722731a-1496-11ef-bc9a-c7b262605968/ceph-mds.cephfs.smithi107.drqody.log 2024-05-17T22:11:15.036 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/3722731a-1496-11ef-bc9a-c7b262605968/ceph-mgr.smithi107.xhhyiw.log 2024-05-17T22:11:15.036 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/3722731a-1496-11ef-bc9a-c7b262605968/ceph.log 2024-05-17T22:11:15.038 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/3722731a-1496-11ef-bc9a-c7b262605968/ceph-mds.cephfs.smithi107.drqody.log: /var/log/ceph/3722731a-1496-11ef-bc9a-c7b262605968/ceph-mgr.smithi107.xhhyiw.log: gzip -5 --verbose -- /var/log/ceph/3722731a-1496-11ef-bc9a-c7b262605968/ceph-osd.2.log 2024-05-17T22:11:15.038 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/3722731a-1496-11ef-bc9a-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/3722731a-1496-11ef-bc9a-c7b262605968/ceph.audit.log 2024-05-17T22:11:15.038 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/3722731a-1496-11ef-bc9a-c7b262605968/ceph-osd.1.log 2024-05-17T22:11:15.038 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/3722731a-1496-11ef-bc9a-c7b262605968/ceph-osd.2.log: /var/log/ceph/3722731a-1496-11ef-bc9a-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/3722731a-1496-11ef-bc9a-c7b262605968/ceph-osd.0.log 2024-05-17T22:11:15.038 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/3722731a-1496-11ef-bc9a-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/3722731a-1496-11ef-bc9a-c7b262605968/ceph-mon.smithi107.log 2024-05-17T22:11:15.038 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/3722731a-1496-11ef-bc9a-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/3722731a-1496-11ef-bc9a-c7b262605968/ceph.cephadm.log 2024-05-17T22:11:15.040 INFO:teuthology.orchestra.run.smithi138.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/ceph-client.1.log 2024-05-17T22:11:15.040 INFO:teuthology.orchestra.run.smithi138.stderr:: No such file or directory 2024-05-17T22:11:15.040 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/3722731a-1496-11ef-bc9a-c7b262605968/ceph-mgr.smithi138.napnll.log 2024-05-17T22:11:15.040 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/3722731a-1496-11ef-bc9a-c7b262605968/ceph.log 2024-05-17T22:11:15.040 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.1.log: gzip -5 --verbose -- /var/log/ceph/3722731a-1496-11ef-bc9a-c7b262605968/ceph.audit.log 2024-05-17T22:11:15.040 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/3722731a-1496-11ef-bc9a-c7b262605968/ceph-mgr.smithi138.napnll.log: /var/log/ceph/3722731a-1496-11ef-bc9a-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/3722731a-1496-11ef-bc9a-c7b262605968/ceph-osd.5.log 2024-05-17T22:11:15.040 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/3722731a-1496-11ef-bc9a-c7b262605968/ceph-osd.3.log 2024-05-17T22:11:15.044 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/3722731a-1496-11ef-bc9a-c7b262605968/ceph-mon.smithi107.log: 88.5% -- replaced with /var/log/ceph/3722731a-1496-11ef-bc9a-c7b262605968/ceph.log.gz 2024-05-17T22:11:15.044 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/3722731a-1496-11ef-bc9a-c7b262605968/ceph-mds.cephfs.smithi107.idmbgj.log 2024-05-17T22:11:15.044 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/3722731a-1496-11ef-bc9a-c7b262605968/ceph.cephadm.log: 85.4% -- replaced with /var/log/ceph/3722731a-1496-11ef-bc9a-c7b262605968/ceph.cephadm.log.gz 2024-05-17T22:11:15.044 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/3722731a-1496-11ef-bc9a-c7b262605968/ceph-volume.log 2024-05-17T22:11:15.045 INFO:teuthology.orchestra.run.smithi107.stderr: 91.2% -- replaced with /var/log/ceph/3722731a-1496-11ef-bc9a-c7b262605968/ceph.audit.log.gz/var/log/ceph/3722731a-1496-11ef-bc9a-c7b262605968/ceph-mds.cephfs.smithi107.idmbgj.log: 2024-05-17T22:11:15.045 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/3722731a-1496-11ef-bc9a-c7b262605968/ceph.audit.log: /var/log/ceph/3722731a-1496-11ef-bc9a-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/3722731a-1496-11ef-bc9a-c7b262605968/ceph-osd.4.log 2024-05-17T22:11:15.045 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/3722731a-1496-11ef-bc9a-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/3722731a-1496-11ef-bc9a-c7b262605968/ceph-mds.cephfs.smithi138.gwbtyd.log 2024-05-17T22:11:15.045 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/3722731a-1496-11ef-bc9a-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/3722731a-1496-11ef-bc9a-c7b262605968/ceph-mds.cephfs.smithi138.zckrvs.log 2024-05-17T22:11:15.045 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/3722731a-1496-11ef-bc9a-c7b262605968/ceph-mds.cephfs.smithi138.gwbtyd.log: 88.5% -- replaced with /var/log/ceph/3722731a-1496-11ef-bc9a-c7b262605968/ceph.log.gz 2024-05-17T22:11:15.045 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/3722731a-1496-11ef-bc9a-c7b262605968/ceph.cephadm.log 2024-05-17T22:11:15.045 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/3722731a-1496-11ef-bc9a-c7b262605968/ceph-mds.cephfs.smithi138.zckrvs.log: 91.4% -- replaced with /var/log/ceph/3722731a-1496-11ef-bc9a-c7b262605968/ceph.audit.log.gz 2024-05-17T22:11:15.053 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/3722731a-1496-11ef-bc9a-c7b262605968/ceph-volume.log 2024-05-17T22:11:15.054 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/3722731a-1496-11ef-bc9a-c7b262605968/ceph.cephadm.log: 85.2% -- replaced with /var/log/ceph/3722731a-1496-11ef-bc9a-c7b262605968/ceph.cephadm.log.gz 2024-05-17T22:11:15.055 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.log 2024-05-17T22:11:15.066 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/3722731a-1496-11ef-bc9a-c7b262605968/ceph-mon.smithi138.log 2024-05-17T22:11:15.067 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/3722731a-1496-11ef-bc9a-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-17T22:11:15.079 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/3722731a-1496-11ef-bc9a-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-17T22:11:15.088 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/3722731a-1496-11ef-bc9a-c7b262605968/ceph-mon.smithi138.log: 91.1%/var/log/ceph/cephadm.log: -- replaced with /var/log/ceph/3722731a-1496-11ef-bc9a-c7b262605968/ceph-volume.log.gz 2024-05-17T22:11:15.091 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.0.log: /var/log/ceph/cephadm.log: 91.1% -- replaced with /var/log/ceph/3722731a-1496-11ef-bc9a-c7b262605968/ceph-volume.log.gz 2024-05-17T22:11:15.095 INFO:teuthology.orchestra.run.smithi138.stderr: 92.4% 89.7% -- replaced with /var/log/ceph/3722731a-1496-11ef-bc9a-c7b262605968/ceph-mgr.smithi138.napnll.log.gz 2024-05-17T22:11:15.103 INFO:teuthology.orchestra.run.smithi138.stderr: -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-17T22:11:15.114 INFO:teuthology.orchestra.run.smithi107.stderr: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-17T22:11:15.379 INFO:teuthology.orchestra.run.smithi138.stderr: 92.5% -- replaced with /var/log/ceph/3722731a-1496-11ef-bc9a-c7b262605968/ceph-mon.smithi138.log.gz 2024-05-17T22:11:15.441 INFO:teuthology.orchestra.run.smithi107.stderr: 88.6% -- replaced with /var/log/ceph/3722731a-1496-11ef-bc9a-c7b262605968/ceph-mgr.smithi107.xhhyiw.log.gz 2024-05-17T22:11:16.275 INFO:teuthology.orchestra.run.smithi107.stderr: 90.4% -- replaced with /var/log/ceph/3722731a-1496-11ef-bc9a-c7b262605968/ceph-mon.smithi107.log.gz 2024-05-17T22:11:18.603 INFO:teuthology.orchestra.run.smithi107.stderr: 93.6% -- replaced with /var/log/ceph/3722731a-1496-11ef-bc9a-c7b262605968/ceph-osd.2.log.gz 2024-05-17T22:11:19.964 INFO:teuthology.orchestra.run.smithi138.stderr: 93.5% -- replaced with /var/log/ceph/3722731a-1496-11ef-bc9a-c7b262605968/ceph-osd.4.log.gz 2024-05-17T22:11:20.157 INFO:teuthology.orchestra.run.smithi107.stderr: 93.5% -- replaced with /var/log/ceph/3722731a-1496-11ef-bc9a-c7b262605968/ceph-osd.0.log.gz 2024-05-17T22:11:20.271 INFO:teuthology.orchestra.run.smithi138.stderr: 93.6% -- replaced with /var/log/ceph/3722731a-1496-11ef-bc9a-c7b262605968/ceph-osd.3.log.gz 2024-05-17T22:11:21.044 INFO:teuthology.orchestra.run.smithi138.stderr: 93.7% -- replaced with /var/log/ceph/3722731a-1496-11ef-bc9a-c7b262605968/ceph-osd.5.log.gz 2024-05-17T22:11:21.233 INFO:teuthology.orchestra.run.smithi107.stderr: 93.7% -- replaced with /var/log/ceph/3722731a-1496-11ef-bc9a-c7b262605968/ceph-osd.1.log.gz 2024-05-17T22:11:22.366 INFO:teuthology.orchestra.run.smithi107.stderr: 94.8% -- replaced with /var/log/ceph/3722731a-1496-11ef-bc9a-c7b262605968/ceph-mds.cephfs.smithi107.idmbgj.log.gz 2024-05-17T22:11:22.872 INFO:teuthology.orchestra.run.smithi138.stderr: 94.8% -- replaced with /var/log/ceph/3722731a-1496-11ef-bc9a-c7b262605968/ceph-mds.cephfs.smithi138.zckrvs.log.gz 2024-05-17T22:11:28.752 INFO:teuthology.orchestra.run.smithi138.stderr:gzip: /var/log/ceph/ceph-client.1.log: file size changed while zipping 2024-05-17T22:11:28.753 INFO:teuthology.orchestra.run.smithi138.stderr: 93.1% -- replaced with /var/log/ceph/ceph-client.1.log.gz 2024-05-17T22:11:29.675 INFO:teuthology.orchestra.run.smithi107.stderr: 93.1% -- replaced with /var/log/ceph/ceph-client.0.log.gz 2024-05-17T22:11:29.884 INFO:teuthology.orchestra.run.smithi107.stderr: 94.8% -- replaced with /var/log/ceph/3722731a-1496-11ef-bc9a-c7b262605968/ceph-mds.cephfs.smithi107.drqody.log.gz 2024-05-17T22:11:29.886 INFO:teuthology.orchestra.run.smithi107.stderr: 2024-05-17T22:11:29.886 INFO:teuthology.orchestra.run.smithi107.stderr:real 0m14.860s 2024-05-17T22:11:29.886 INFO:teuthology.orchestra.run.smithi107.stderr:user 0m51.450s 2024-05-17T22:11:29.886 INFO:teuthology.orchestra.run.smithi107.stderr:sys 0m1.787s 2024-05-17T22:12:41.259 INFO:teuthology.orchestra.run.smithi138.stderr: 92.6% -- replaced with /var/log/ceph/3722731a-1496-11ef-bc9a-c7b262605968/ceph-mds.cephfs.smithi138.gwbtyd.log.gz 2024-05-17T22:12:41.261 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-05-17T22:12:41.261 INFO:teuthology.orchestra.run.smithi138.stderr:real 1m26.233s 2024-05-17T22:12:41.261 INFO:teuthology.orchestra.run.smithi138.stderr:user 2m0.319s 2024-05-17T22:12:41.261 INFO:teuthology.orchestra.run.smithi138.stderr:sys 0m3.724s 2024-05-17T22:12:41.261 INFO:tasks.cephadm:Archiving logs... 2024-05-17T22:12:41.261 DEBUG:teuthology.misc:Transferring archived files from smithi107:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-05-17_18:30:44-fs-wip-pdonnell-testing-20240517.012458-debug-distro-default-smithi/7711745/remote/smithi107/log 2024-05-17T22:12:41.262 DEBUG:teuthology.orchestra.run.smithi107:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-17T22:12:47.229 DEBUG:teuthology.misc:Transferring archived files from smithi138:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-05-17_18:30:44-fs-wip-pdonnell-testing-20240517.012458-debug-distro-default-smithi/7711745/remote/smithi138/log 2024-05-17T22:12:47.230 DEBUG:teuthology.orchestra.run.smithi138:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-17T22:13:04.633 INFO:tasks.cephadm:Removing cluster... 2024-05-17T22:13:04.633 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3722731a-1496-11ef-bc9a-c7b262605968 --force 2024-05-17T22:13:05.174 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3722731a-1496-11ef-bc9a-c7b262605968 --force 2024-05-17T22:13:05.773 INFO:tasks.cephadm:Removing cephadm ... 2024-05-17T22:13:05.773 DEBUG:teuthology.orchestra.run.smithi107:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-17T22:13:05.790 DEBUG:teuthology.orchestra.run.smithi138:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-17T22:13:05.806 INFO:tasks.cephadm:Teardown complete 2024-05-17T22:13:05.806 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-05-17T22:13:05.817 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_2127e87e3419439de550d2abbd7ed2aee2cdfb0e/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_2127e87e3419439de550d2abbd7ed2aee2cdfb0e/teuthology/task/install/__init__.py", line 632, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_2127e87e3419439de550d2abbd7ed2aee2cdfb0e/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.10/contextlib.py", line 142, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_931809a9e5d295bd61984693cac44fe131ded4bc/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph-c_931809a9e5d295bd61984693cac44fe131ded4bc/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_2127e87e3419439de550d2abbd7ed2aee2cdfb0e/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_2127e87e3419439de550d2abbd7ed2aee2cdfb0e/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-05-17T22:13:05.818 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-05-17T22:13:05.818 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-17T22:13:05.833 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-17T22:13:05.921 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-05-17T22:13:05.931 INFO:teuthology.task.clock:Checking final clock skew... 2024-05-17T22:13:05.931 DEBUG:teuthology.orchestra.run.smithi107:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-17T22:13:05.933 DEBUG:teuthology.orchestra.run.smithi138:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-17T22:13:05.946 INFO:teuthology.orchestra.run.smithi107.stderr:bash: line 1: ntpq: command not found 2024-05-17T22:13:05.947 INFO:teuthology.orchestra.run.smithi138.stderr:bash: line 1: ntpq: command not found 2024-05-17T22:13:05.951 INFO:teuthology.orchestra.run.smithi107.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-17T22:13:05.951 INFO:teuthology.orchestra.run.smithi107.stdout:=============================================================================== 2024-05-17T22:13:05.951 INFO:teuthology.orchestra.run.smithi107.stdout:^* hv01.front.sepia.ceph.com 3 6 377 18 +2620us[+2644us] +/- 48ms 2024-05-17T22:13:05.951 INFO:teuthology.orchestra.run.smithi107.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 19 -1708us[-1684us] +/- 72ms 2024-05-17T22:13:05.951 INFO:teuthology.orchestra.run.smithi107.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 19 -2281us[-2257us] +/- 72ms 2024-05-17T22:13:05.951 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-17T22:13:05.952 INFO:teuthology.orchestra.run.smithi138.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-17T22:13:05.952 INFO:teuthology.orchestra.run.smithi138.stdout:=============================================================================== 2024-05-17T22:13:05.953 INFO:teuthology.orchestra.run.smithi138.stdout:^* hv01.front.sepia.ceph.com 3 6 377 17 +2610us[+2607us] +/- 48ms 2024-05-17T22:13:05.953 INFO:teuthology.orchestra.run.smithi138.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 18 -1705us[-1708us] +/- 72ms 2024-05-17T22:13:05.953 INFO:teuthology.orchestra.run.smithi138.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 18 -2297us[-2300us] +/- 72ms 2024-05-17T22:13:05.953 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-17T22:13:05.953 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-05-17T22:13:05.993 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-05-17T22:13:05.994 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-05-17T22:13:06.044 DEBUG:teuthology.orchestra.run.smithi107:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-17T22:13:06.127 DEBUG:teuthology.orchestra.run.smithi138:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-17T22:13:06.203 DEBUG:teuthology.orchestra.run.smithi107:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-17T22:13:06.231 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-17T22:13:06.231 DEBUG:teuthology.orchestra.run.smithi138:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-17T22:13:06.260 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-17T22:13:06.260 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-05-17T22:13:06.273 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-05-17T22:13:06.349 INFO:teuthology.task.internal:Duration was 2286.556906 seconds 2024-05-17T22:13:06.350 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-05-17T22:13:06.374 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-05-17T22:13:06.374 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-17T22:13:06.376 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-17T22:13:06.413 INFO:teuthology.orchestra.run.smithi107.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-17T22:13:06.415 INFO:teuthology.orchestra.run.smithi138.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-17T22:13:06.870 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-05-17T22:13:06.870 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi107.front.sepia.ceph.com 2024-05-17T22:13:06.871 DEBUG:teuthology.orchestra.run.smithi107:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-17T22:13:06.891 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi138.front.sepia.ceph.com 2024-05-17T22:13:06.891 DEBUG:teuthology.orchestra.run.smithi138:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-17T22:13:06.936 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-05-17T22:13:06.936 DEBUG:teuthology.orchestra.run.smithi107:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-17T22:13:06.939 DEBUG:teuthology.orchestra.run.smithi138:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-17T22:13:07.003 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-05-17T22:13:07.003 DEBUG:teuthology.orchestra.run.smithi107:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-17T22:13:07.007 DEBUG:teuthology.orchestra.run.smithi138:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-17T22:13:07.686 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-05-17T22:13:07.698 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-05-17T22:13:07.698 DEBUG:teuthology.orchestra.run.smithi107:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-17T22:13:07.723 DEBUG:teuthology.orchestra.run.smithi138:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-17T22:13:07.750 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-05-17T22:13:07.761 DEBUG:teuthology.orchestra.run.smithi107:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-17T22:13:07.766 DEBUG:teuthology.orchestra.run.smithi138:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-17T22:13:07.787 INFO:teuthology.orchestra.run.smithi107.stdout:kernel.core_pattern = core 2024-05-17T22:13:07.814 INFO:teuthology.orchestra.run.smithi138.stdout:kernel.core_pattern = core 2024-05-17T22:13:07.845 DEBUG:teuthology.orchestra.run.smithi107:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-17T22:13:07.873 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-17T22:13:07.873 DEBUG:teuthology.orchestra.run.smithi138:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-17T22:13:07.900 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-17T22:13:07.900 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-05-17T22:13:07.912 INFO:teuthology.task.internal:Transferring archived files... 2024-05-17T22:13:07.914 DEBUG:teuthology.misc:Transferring archived files from smithi107:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-05-17_18:30:44-fs-wip-pdonnell-testing-20240517.012458-debug-distro-default-smithi/7711745/remote/smithi107 2024-05-17T22:13:07.914 DEBUG:teuthology.orchestra.run.smithi107:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-17T22:13:07.962 DEBUG:teuthology.misc:Transferring archived files from smithi138:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-05-17_18:30:44-fs-wip-pdonnell-testing-20240517.012458-debug-distro-default-smithi/7711745/remote/smithi138 2024-05-17T22:13:07.962 DEBUG:teuthology.orchestra.run.smithi138:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-17T22:13:08.000 INFO:teuthology.task.internal:Removing archive directory... 2024-05-17T22:13:08.001 DEBUG:teuthology.orchestra.run.smithi107:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-17T22:13:08.003 DEBUG:teuthology.orchestra.run.smithi138:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-17T22:13:08.050 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-05-17T22:13:08.062 INFO:teuthology.task.internal:Not uploading archives. 2024-05-17T22:13:08.063 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-05-17T22:13:08.097 INFO:teuthology.task.internal:Tidying up after the test... 2024-05-17T22:13:08.098 DEBUG:teuthology.orchestra.run.smithi107:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-17T22:13:08.100 DEBUG:teuthology.orchestra.run.smithi138:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-17T22:13:08.112 INFO:teuthology.orchestra.run.smithi107.stdout: 265834 4 drwxr-xr-x 3 ubuntu ubuntu 4096 May 17 22:13 /home/ubuntu/cephtest 2024-05-17T22:13:08.112 INFO:teuthology.orchestra.run.smithi107.stdout: 655363 4 d--------- 2 ubuntu ubuntu 4096 May 17 21:49 /home/ubuntu/cephtest/mnt.0 2024-05-17T22:13:08.113 INFO:teuthology.orchestra.run.smithi107.stderr:find: ‘/home/ubuntu/cephtest/mnt.0’: Permission denied 2024-05-17T22:13:08.113 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-17T22:13:08.114 INFO:teuthology.orchestra.run.smithi107.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2024-05-17T22:13:08.114 ERROR:teuthology.run_tasks:Manager failed: internal.base Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_2127e87e3419439de550d2abbd7ed2aee2cdfb0e/teuthology/task/internal/__init__.py", line 48, in base yield File "/home/teuthworker/src/git.ceph.com_teuthology_2127e87e3419439de550d2abbd7ed2aee2cdfb0e/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.10/contextlib.py", line 142, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_931809a9e5d295bd61984693cac44fe131ded4bc/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph-c_931809a9e5d295bd61984693cac44fe131ded4bc/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_2127e87e3419439de550d2abbd7ed2aee2cdfb0e/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_2127e87e3419439de550d2abbd7ed2aee2cdfb0e/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_2127e87e3419439de550d2abbd7ed2aee2cdfb0e/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.10/contextlib.py", line 153, in __exit__ self.gen.throw(typ, value, traceback) File "/home/teuthworker/src/git.ceph.com_teuthology_2127e87e3419439de550d2abbd7ed2aee2cdfb0e/teuthology/task/internal/__init__.py", line 53, in base run.wait( File "/home/teuthworker/src/git.ceph.com_teuthology_2127e87e3419439de550d2abbd7ed2aee2cdfb0e/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_2127e87e3419439de550d2abbd7ed2aee2cdfb0e/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_2127e87e3419439de550d2abbd7ed2aee2cdfb0e/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi107 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2024-05-17T22:13:08.114 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-05-17T22:13:08.126 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-05-17T22:13:08.146 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-05-17T22:13:08.147 INFO:teuthology.nuke:Checking targets against current locks 2024-05-17T22:13:08.167 DEBUG:teuthology.nuke:shortname: smithi107 2024-05-17T22:13:08.167 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-17T22:13:08.185 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi107.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-17_18:30:44-fs-wip-pdonnell-testing-20240517.012458-debug-distro-default-smithi/7711745', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-17 21:26:56.860779', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOVhqHUkeu3/sHQtyZnwHZdpUVdm2yZQUf5nrfLplOhc/6KlznKMYju/zfiin0dn087PMxpU5UNDYGi53eltz7o='} 2024-05-17T22:13:08.188 DEBUG:teuthology.nuke:shortname: smithi138 2024-05-17T22:13:08.188 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-17T22:13:08.208 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi138.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-17_18:30:44-fs-wip-pdonnell-testing-20240517.012458-debug-distro-default-smithi/7711745', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-17 21:26:56.858719', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDQDUV+Fh0J59rIGzIc3/ViaBBlSxATYaJ0MwefATjYN1bmlfC7K7qh5ZbWfWxUYH70EtBKj1oWup8Z2BabnPMs='} 2024-05-17T22:13:08.225 INFO:teuthology.orchestra.console.smithi107:Power off 2024-05-17T22:13:08.225 DEBUG:teuthology.orchestra.console.smithi107:pexpect command: ipmitool -H smithi107.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-05-17T22:13:08.252 DEBUG:teuthology.orchestra.console.smithi107:power off output: Chassis Power Control: Down/Off 2024-05-17T22:13:08.253 DEBUG:teuthology.orchestra.console.smithi107:pexpect command: ipmitool -H smithi107.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-17T22:13:08.266 INFO:teuthology.orchestra.console.smithi138:Power off 2024-05-17T22:13:08.266 DEBUG:teuthology.orchestra.console.smithi138:pexpect command: ipmitool -H smithi138.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-05-17T22:13:08.275 DEBUG:teuthology.orchestra.console.smithi107:check power output: Chassis Power is on 2024-05-17T22:13:08.288 DEBUG:teuthology.orchestra.console.smithi138:power off output: Chassis Power Control: Down/Off 2024-05-17T22:13:08.288 DEBUG:teuthology.orchestra.console.smithi138:pexpect command: ipmitool -H smithi138.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-17T22:13:08.310 DEBUG:teuthology.orchestra.console.smithi138:check power output: Chassis Power is on 2024-05-17T22:13:12.275 DEBUG:teuthology.orchestra.console.smithi107:pexpect command: ipmitool -H smithi107.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-17T22:13:12.402 DEBUG:teuthology.orchestra.console.smithi138:pexpect command: ipmitool -H smithi138.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-17T22:13:12.504 DEBUG:teuthology.orchestra.console.smithi107:check power output: Chassis Power is on 2024-05-17T22:13:12.512 DEBUG:teuthology.orchestra.console.smithi138:check power output: Chassis Power is on 2024-05-17T22:13:16.509 DEBUG:teuthology.orchestra.console.smithi107:pexpect command: ipmitool -H smithi107.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-17T22:13:16.516 DEBUG:teuthology.orchestra.console.smithi138:pexpect command: ipmitool -H smithi138.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-17T22:13:16.618 DEBUG:teuthology.orchestra.console.smithi107:check power output: Chassis Power is on 2024-05-17T22:13:16.625 DEBUG:teuthology.orchestra.console.smithi138:check power output: Chassis Power is on 2024-05-17T22:13:20.621 DEBUG:teuthology.orchestra.console.smithi107:pexpect command: ipmitool -H smithi107.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-17T22:13:20.629 DEBUG:teuthology.orchestra.console.smithi138:pexpect command: ipmitool -H smithi138.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-17T22:13:20.731 DEBUG:teuthology.orchestra.console.smithi107:check power output: Chassis Power is off 2024-05-17T22:13:20.739 DEBUG:teuthology.orchestra.console.smithi138:check power output: Chassis Power is off 2024-05-17T22:13:20.832 INFO:teuthology.orchestra.console.smithi107:Power off completed 2024-05-17T22:13:20.840 INFO:teuthology.orchestra.console.smithi138:Power off completed 2024-05-17T22:13:20.944 INFO:teuthology.run:Summary data: description: fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} duration: 2286.5569059848785 failure_reason: reached maximum tries (51) after waiting for 300 seconds flavor: default owner: scheduled_pdonnell@teuthology status: fail success: false 2024-05-17T22:13:20.945 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-17T22:13:21.031 INFO:teuthology.run:FAIL